Podcast appearances and mentions of sam bowman

Play Episode Listen Later Sep 3, 2024 35:25

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: The Checklist: What Succeeding at AI Safety Will Involve, published by Sam Bowman on September 3, 2024 on The AI Alignment Forum. Crossposted by habryka with Sam's permission. Expect lower probability for Sam to respond to comments here than if he had posted it. Preface This piece reflects my current best guess at the major goals that Anthropic (or another similarly positioned AI developer) will need to accomplish to have things go well with the development of broadly superhuman AI. Given my role and background, it's disproportionately focused on technical research and on averting emerging catastrophic risks. For context, I lead a technical AI safety research group at Anthropic, and that group has a pretty broad and long-term mandate, so I spend a lot of time thinking about what kind of safety work we'll need over the coming years. This piece is my own opinionated take on that question, though it draws very heavily on discussions with colleagues across the organization: Medium- and long-term AI safety strategy is the subject of countless leadership discussions and Google docs and lunch-table discussions within the organization, and this piece is a snapshot (shared with permission) of where those conversations sometimes go. To be abundantly clear: Nothing here is a firm commitment on behalf of Anthropic, and most people at Anthropic would disagree with at least a few major points here, but this can hopefully still shed some light on the kind of thinking that motivates our work. Here are some of the assumptions that the piece relies on. I don't think any one of these is a certainty, but all of them are plausible enough to be worth taking seriously when making plans: Broadly human-level AI is possible. I'll often refer to this as transformative AI (or TAI), roughly defined as AI that could form as a drop-in replacement for humans in all remote-work-friendly jobs, including AI R&D.[1] Broadly human-level AI (or TAI) isn't an upper bound on most AI capabilities that matter, and substantially superhuman systems could have an even greater impact on the world along many dimensions. If TAI is possible, it will probably be developed this decade, in a business and policy and cultural context that's not wildly different from today. If TAI is possible, it could be used to dramatically accelerate AI R&D, potentially leading to the development of substantially superhuman systems within just a few months or years after TAI. Powerful AI systems could be extraordinarily destructive if deployed carelessly, both because of new emerging risks and because of existing issues that become much more acute. This could be through misuse of weapons-related capabilities, by disrupting important balances of power in domains like cybersecurity or surveillance, or by any of a number of other means. Many systems at TAI and beyond, at least under the right circumstances, will be capable of operating more-or-less autonomously for long stretches in pursuit of big-picture, real-world goals. This magnifies these safety challenges. Alignment - in the narrow sense of making sure AI developers can confidently steer the behavior of the AI systems they deploy - requires some non-trivial effort to get right, and it gets harder as systems get more powerful. Most of the ideas here ultimately come from outside Anthropic, and while I cite a few sources below, I've been influenced by far more writings and people than I can credit here or even keep track of. Introducing the Checklist This lays out what I think we need to do, divided into three chapters, based on the capabilities of our strongest models: Chapter 1: Preparation You are here. In this period, our best models aren't yet TAI. In the language of Anthropic's RSP, they're at AI Safety Level 2 (ASL-2), ASL-3, or maybe the early stages of ASL-4. Most of the wor...

google ai safety speech medium alignment ea checklist succeeding tai involve asl anthropic rsp rationalist sam bowman

LW - The Checklist: What Succeeding at AI Safety Will Involve by Sam Bowman

The Nonlinear Library: LessWrong

Play Episode Listen Later Sep 3, 2024 35:25

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: The Checklist: What Succeeding at AI Safety Will Involve, published by Sam Bowman on September 3, 2024 on LessWrong. Crossposted by habryka with Sam's permission. Expect lower probability for Sam to respond to comments here than if he had posted it. Preface This piece reflects my current best guess at the major goals that Anthropic (or another similarly positioned AI developer) will need to accomplish to have things go well with the development of broadly superhuman AI. Given my role and background, it's disproportionately focused on technical research and on averting emerging catastrophic risks. For context, I lead a technical AI safety research group at Anthropic, and that group has a pretty broad and long-term mandate, so I spend a lot of time thinking about what kind of safety work we'll need over the coming years. This piece is my own opinionated take on that question, though it draws very heavily on discussions with colleagues across the organization: Medium- and long-term AI safety strategy is the subject of countless leadership discussions and Google docs and lunch-table discussions within the organization, and this piece is a snapshot (shared with permission) of where those conversations sometimes go. To be abundantly clear: Nothing here is a firm commitment on behalf of Anthropic, and most people at Anthropic would disagree with at least a few major points here, but this can hopefully still shed some light on the kind of thinking that motivates our work. Here are some of the assumptions that the piece relies on. I don't think any one of these is a certainty, but all of them are plausible enough to be worth taking seriously when making plans: Broadly human-level AI is possible. I'll often refer to this as transformative AI (or TAI), roughly defined as AI that could form as a drop-in replacement for humans in all remote-work-friendly jobs, including AI R&D.[1] Broadly human-level AI (or TAI) isn't an upper bound on most AI capabilities that matter, and substantially superhuman systems could have an even greater impact on the world along many dimensions. If TAI is possible, it will probably be developed this decade, in a business and policy and cultural context that's not wildly different from today. If TAI is possible, it could be used to dramatically accelerate AI R&D, potentially leading to the development of substantially superhuman systems within just a few months or years after TAI. Powerful AI systems could be extraordinarily destructive if deployed carelessly, both because of new emerging risks and because of existing issues that become much more acute. This could be through misuse of weapons-related capabilities, by disrupting important balances of power in domains like cybersecurity or surveillance, or by any of a number of other means. Many systems at TAI and beyond, at least under the right circumstances, will be capable of operating more-or-less autonomously for long stretches in pursuit of big-picture, real-world goals. This magnifies these safety challenges. Alignment - in the narrow sense of making sure AI developers can confidently steer the behavior of the AI systems they deploy - requires some non-trivial effort to get right, and it gets harder as systems get more powerful. Most of the ideas here ultimately come from outside Anthropic, and while I cite a few sources below, I've been influenced by far more writings and people than I can credit here or even keep track of. Introducing the Checklist This lays out what I think we need to do, divided into three chapters, based on the capabilities of our strongest models: Chapter 1: Preparation You are here. In this period, our best models aren't yet TAI. In the language of Anthropic's RSP, they're at AI Safety Level 2 (ASL-2), ASL-3, or maybe the early stages of ASL-4. Most of the work that we hav...

google ai safety speech medium alignment ea checklist succeeding tai involve asl anthropic rsp rationalist sam bowman lesswrong

LW - The Checklist: What Succeeding at AI Safety Will Involve by Sam Bowman

Play Episode Listen Later Sep 3, 2024 35:25

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: The Checklist: What Succeeding at AI Safety Will Involve, published by Sam Bowman on September 3, 2024 on LessWrong. Crossposted by habryka with Sam's permission. Expect lower probability for Sam to respond to comments here than if he had posted it. Preface This piece reflects my current best guess at the major goals that Anthropic (or another similarly positioned AI developer) will need to accomplish to have things go well with the development of broadly superhuman AI. Given my role and background, it's disproportionately focused on technical research and on averting emerging catastrophic risks. For context, I lead a technical AI safety research group at Anthropic, and that group has a pretty broad and long-term mandate, so I spend a lot of time thinking about what kind of safety work we'll need over the coming years. This piece is my own opinionated take on that question, though it draws very heavily on discussions with colleagues across the organization: Medium- and long-term AI safety strategy is the subject of countless leadership discussions and Google docs and lunch-table discussions within the organization, and this piece is a snapshot (shared with permission) of where those conversations sometimes go. To be abundantly clear: Nothing here is a firm commitment on behalf of Anthropic, and most people at Anthropic would disagree with at least a few major points here, but this can hopefully still shed some light on the kind of thinking that motivates our work. Here are some of the assumptions that the piece relies on. I don't think any one of these is a certainty, but all of them are plausible enough to be worth taking seriously when making plans: Broadly human-level AI is possible. I'll often refer to this as transformative AI (or TAI), roughly defined as AI that could form as a drop-in replacement for humans in all remote-work-friendly jobs, including AI R&D.[1] Broadly human-level AI (or TAI) isn't an upper bound on most AI capabilities that matter, and substantially superhuman systems could have an even greater impact on the world along many dimensions. If TAI is possible, it will probably be developed this decade, in a business and policy and cultural context that's not wildly different from today. If TAI is possible, it could be used to dramatically accelerate AI R&D, potentially leading to the development of substantially superhuman systems within just a few months or years after TAI. Powerful AI systems could be extraordinarily destructive if deployed carelessly, both because of new emerging risks and because of existing issues that become much more acute. This could be through misuse of weapons-related capabilities, by disrupting important balances of power in domains like cybersecurity or surveillance, or by any of a number of other means. Many systems at TAI and beyond, at least under the right circumstances, will be capable of operating more-or-less autonomously for long stretches in pursuit of big-picture, real-world goals. This magnifies these safety challenges. Alignment - in the narrow sense of making sure AI developers can confidently steer the behavior of the AI systems they deploy - requires some non-trivial effort to get right, and it gets harder as systems get more powerful. Most of the ideas here ultimately come from outside Anthropic, and while I cite a few sources below, I've been influenced by far more writings and people than I can credit here or even keep track of. Introducing the Checklist This lays out what I think we need to do, divided into three chapters, based on the capabilities of our strongest models: Chapter 1: Preparation You are here. In this period, our best models aren't yet TAI. In the language of Anthropic's RSP, they're at AI Safety Level 2 (ASL-2), ASL-3, or maybe the early stages of ASL-4. Most of the work that we hav...

google ai safety speech medium alignment ea checklist succeeding tai involve asl anthropic rsp rationalist sam bowman lesswrong

House of Worship - Episode 85

House of Worship

Play Episode Listen Later Jul 24, 2024 60:01

Share this episode and spread the love through music! Like, Share and Subscribe We are House of Worship, the community that brings Jesus' praise through Christian house and EDM. Every month, you can discover new music on this platform. We are a Dutch DJ duo, active in weddings and events for over 7 years. In addition, we create a Christian house podcast every month with the aim of spreading the love of Jesus. Would you like to spread the love as well? Then repost this episode. Join our Community: - Spotify : https://open.spotify.com/playlist/0gJIZ1z6P3INxjcslkaK4v?si=2b4bf1e1a6894855 - Instagram: https://www.instagram.com/boersmaendeboer/ - Subscribe to our YouTube channel: https://www.youtube.com/playlist?list=PLkSYx5uHcf5PGA_ndiYd1Pf5aD49bw6ZG - Repost this channel - -https://soundcloud.com/houseofworshippodcast Playlist: 1. Sam Bowman - Your Eyes 2. LIN D - Brighter Future 3. Chris Howland - Final breath 4. Roberto Rosso & EHDEN - Pray for me 5. Confess - You Give Me 6. Crowder & TobyMac - [DASH] (Kirecko Remix) 7. Pembers - Jericho 8. JOSHUA LAZER - Won_'t Stop w_ Christian Singleton & Sam Bowman 9. Rave Jesus & SON. - Take Me Higher 10. HAGEN - Praise 11. Marc Vanparla & Zane Walls - OUTSIDE 12. Rave Jesus - You_'re Gonna Be Ok (feat. Son) 13. Capital Kings vs Lecrae & Andy Mineo - Say I Won_'t In The Wild (SPNR Mashup) 14. Jump Around (Planetshakers) - Tech House Remix 15. Lecrae - Spread The Opps (fourtures remix) 16. Brandon Lake - TEAR OFF THE ROOF (SADO Remix) 17. A.A. Allen And Gene Martin - Glory Glory Since I Laid My Burdens Down (KEVIN ALEKSANDER EDIT)

jesus christ house worship edm crowder lecrae house of worship take me higher dutch dj sam bowman

Ep. 223: Sam Bowman - Why Can't We Have Nice Things?

The Curious Task

Play Episode Listen Later Apr 3, 2024 73:06

Matt speaks with Sam Bowman about the global housing crisis and why, in English-speaking countries in particular, a growing consensus across the political divide is pointing to problems with central planning, NIMBYism and a supply limit as the causes. Episode Notes: Sam Bowman's on substack: https://substack.com/@sambowman Sam on "Vetocracy": https://www.sambowman.co/p/democracy-is-the-solution-to-vetocracy Sam's “Housing Theory of Everything” article: https://worksinprogress.co/issue/the-housing-theory-of-everything/ Works in Progress website: https://worksinprogress.co/ Michael Giberson on Manser Olson's theory of concentrated costs and dispersed benefits: https://knowledgeproblem.com/2010/10/17/concentrated-benefits-and-dispersed-costs/ Comparison of UK vs. US GDP per capita: https://countryeconomy.com/countries/compare/usa/uk

english uk progress comparison nice things us gdp nimbyism sam bowman

Rewind #68: Talking with Singer/Song-Writer Sam Bowman

Weekend Ketchup with Isaac (Real FM)

Play Episode Listen Later Mar 23, 2024 21:47

This is a different kind of episode. We're rewinding the clock to when Zoe interviewed a special guest, an artist we play on Real FM named Sam Bowman. Sam is a singer/song writer who also collaborates with several artists, helping produce their music as well as his own. Zoe is a big fan of his single "Whisper" and his new song "Gravity." You can connect more with Sam Bowman on Instagram @sambowmanmusic And you can find him on Spotify! As well as listen to a few of his songs here on Real FM. Let us know if you want more episodes featuring artists!

spotify whispers gravity sam bowman

Alec Stapp — Progress is a Policy Choice (EP. 202)

Infinite Loops

Play Episode Listen Later Feb 15, 2024 66:59

Alec Stapp is the co-founder of the Institute for Progress, a non-partisan innovation policy think tank aiming to “accelerate scientific, technological and industrial progress while safeguarding humanity's future.” He joins the show to discuss how to achieve change in the age of lobbying, why bipartisanship is underrated, why US immigration policy is so slow-moving and MUCH more! Important Links: IFP's Website Alec's Twitter IFP's Twitter Show Notes: Reimagining the Think Tank Progress is a Policy Choice Bipartisanship is Underrated Achieving Progress via Reframing Achieving Change in the Age of Lobbying Moonshot Projects and Incremental Change Ways to Enact Change Within Existing Institutions Governmental Embrace of Technology Reducing NIMBYism The Barbell Approach to Policy The Washington Mindset Reasons to be Optimistic Lessons From Other Countries Why Hasn't Immigration Policy Changed? Alec as Emperor of the World MORE! Books and Articles Mentioned: Recoding America: Why Government Is Failing in the Digital Age and How We Can Do Better; by Jen Pahlka The Proceduralist Fetish; by Nicholas Bagley The housing theory of everything; by Sam Bowman, John Myers & Ben Southwood

books institute progress policy emperor digital age stapp sam bowman how we can do better

New Year Mix 2023

House of Worship

Play Episode Listen Later Dec 30, 2023 60:00

Share this episode and spread the love through music! Like, Share and Repost Here you will find Christian house and some times a word of the day! We are House of Worship, the community that brings Jesus' praise through Christian house and EDM. Every month, you can discover new music on this platform. We are a Dutch DJ duo, active in weddings and events for over 7 years. In addition, we create a Christian house podcast every month with the aim of spreading the love of Jesus. Would you like to spread the love as well? Then repost this episode. Join our Community: - Spotify : https://open.spotify.com/playlist/0gJIZ1z6P3INxjcslkaK4v?si=2b4bf1e1a6894855 - Instagram: https://www.instagram.com/boersmaendeboer/ - Subscribe to our YouTube channel: https://www.youtube.com/playlist?list=PLkSYx5uHcf5PGA_ndiYd1Pf5aD49bw6ZG - Repost this channel Playlist New Year Mix: 1. Chris Howland - Final breath 2. Confess - You Give Me 3. Floorplan - We Give Thee Honour (Extended Mix) 4. Matt Andrews - Never Lose Faith (Original MIx) 5. HAGEN - Praise 6. Nora En Pure - Epiphany 7. Sultan & Shepard & Nathan Nicholson - Under The Surface (Original Mix) 8. Transform - The Psalm (Extended Mix) 9. Todd Edwards - Show Me A Sign (Shermanology Extended Remix featuring Conquer Jones) 10. Low Blow - Consumers 11. JOSHUA LAZER - Won_'t Stop w_ Christian Singleton & Sam Bowman 12. VLADIMIR HLISTA - OTCHE NASH _ OUR FATHER (LCCTACOMA CHOIR) [KEVIN ALEKSANDER EDIT] 13. Kevin Aleksander - It's Alright 14. Melezz - You Hold Me (feat. Laura Brizuela) 15. James - Serenity 16. Lecrae - Spread The Opps (fourtures remix) 17. Maverick City Music - Jireh (Reyer Remix) feat. Esther Lisette

jesus christ house worship edm sultans yearmix new year mix dutch dj sam bowman

AF - Scalable Oversight and Weak-to-Strong Generalization: Compatible approaches to the same problem by Ansh Radhakrishnan

Play Episode Listen Later Dec 16, 2023 10:35

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Scalable Oversight and Weak-to-Strong Generalization: Compatible approaches to the same problem, published by Ansh Radhakrishnan on December 16, 2023 on The AI Alignment Forum. Thanks to Roger Grosse, Cem Anil, Sam Bowman, and Tamera Lanham for helpful discussion and comments on drafts of this post. Two approaches to addressing weak supervision A key challenge for adequate supervision of future AI systems is the possibility that they'll be more capable than their human overseers. Modern machine learning, particularly supervised learning, relies heavily on the labeler(s) being more capable than the model attempting to learn to predict labels. We shouldn't expect this to always work well when the model is more capable than the labeler,[1] and this problem also gets worse with scale - as the AI systems being supervised become even more capable, naive supervision becomes even less effective. One approach to solving this problem is to try to make the supervision signal stronger, such that we return to the "normal ML" regime. These scalable oversight approaches aim to amplify the overseers of an AI system such that they are more capable than the system itself. It's also crucial for this amplification to persist as the underlying system gets stronger. This is frequently accomplished by using the system being supervised as a part of a more complex oversight process, such as by forcing it to argue against another instance of itself, with the additional hope that verification is generally easier than generation. Another approach is to make the strong student (the AI system) generalize correctly from the imperfect labels provided by the weak teacher. The hope for these weak-to-strong generalization techniques is that we can do better than naively relying on unreliable feedback from a weak overseer and instead access the latent, greater, capabilities that our AI system has, perhaps by a simple modification of the training objective. So, I think of these as two orthogonal approaches to the same problem: improving how well we can train models to perform well in cases where we have trouble evaluating their labels. Scalable oversight just aims to increase the strength of the overseer, such that it becomes stronger than the system being overseen, whereas weak-to-strong generalization tries to ensure that the system generalizes appropriately from the supervision signal of a weak overseer. I think that researchers should just think of these as the same research direction. They should freely mix and match between the two approaches when developing techniques. And when developing techniques that only use one of these approaches, they should still compare to baselines that use the other (or a hybrid). (There are some practical reasons why these approaches generally haven't been unified in the past. In particular, for scalable oversight research to be interesting, you need your weaker models to be competent enough to follow basic instructions, while generalization research is most interesting when you have a large gap in model capability. But at the moment, models are only barely capable enough for scalable oversight techniques to work. So you can't have a large gap in model capability where the less-capable model is able to participate in scalable oversight. On the other hand, the OpenAI paper uses a GPT-2-compute-equivalent model as a weak overseer, which has a big gap to GPT-4 but is way below the capability required for scalable oversight techniques to do anything. For this reason, the two approaches should still probably be investigated in somewhat different settings for the moment. Here are some examples of hybrid protocols incorporating weak-to-strong techniques and scalable oversight schemes: Collect preference comparisons from a language model and a human rater, but allow the rate...

ai modern speech weak approaches ea openai collect gpt ml oversight scalable compatible generalization rationalist sam bowman

AF - Anthropic Fall 2023 Debate Progress Update by Ansh Radhakrishnan

Play Episode Listen Later Nov 28, 2023 18:25

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Anthropic Fall 2023 Debate Progress Update, published by Ansh Radhakrishnan on November 28, 2023 on The AI Alignment Forum. This is a research update on some work that I've been doing on Scalable Oversight at Anthropic, based on the original AI safety via debate proposal and a more recent agenda developed at NYU and Anthropic. The core doc was written several months ago, so some of it is likely outdated, but it seemed worth sharing in its current form. I'd like to thank Tamera Lanham, Sam Bowman, Kamile Lukosiute, Ethan Perez, Jared Kaplan, Amanda Askell, Kamal Ndousse, Shauna Kravec, Yuntao Bai, Alex Tamkin, Newton Cheng, Buck Shlegeris, Akbir Khan, John Hughes, Dan Valentine, Kshitij Sachan, Ryan Greenblatt, Daniel Ziegler, Max Nadeau, David Rein, Julian Michael, Kevin Klyman, Bila Mahdi, Samuel Arnesen, Nat McAleese, Jan Leike, Geoffrey Irving, and Sebastian Farquhar for help, feedback, and thoughtful discussion that improved the quality of this work and write-up. 1. Anthropic's Debate Agenda In this doc, I'm referring to the idea first presented in AI safety via debate ( blog post). The basic idea is to supervise future AI systems by pitting them against each other in a debate, encouraging them to argue both sides (or "all sides") of a question and using the resulting arguments to come to a final answer about the question. In this scheme, we call the systems participating in the debate debaters (though usually, these are actually the same underlying system that's being prompted to argue against itself), and we call the agent (either another AI system or a human, or a system of humans and AIs working together, etc.) that comes to a final decision about the debate the judge. For those more or less familiar with the original OAI/Irving et al. Debate agenda, you may wonder if there are any differences between that agenda and the agenda we're pursuing at Anthropic, and indeed there are! Sam Bowman and Tamera Lanham have written up a working Anthropic-NYU Debate Agenda draft which is what the experiments in this doc are driving towards. [1] To quote from there about the basic features of this agenda, and how it differs from the original Debate direction: Here are the defining features of the base proposal: Two-player debate on a two-choice question: Two debaters (generally two instances of an LLM) present evidence and arguments to a judge (generally a human or, in some cases, an LLM) to persuade the judge to choose their assigned answer to a question with two possible answers. No externally-imposed structure: Instead of being formally prescribed, the structure and norms of the debate arise from debaters learning how to best convince the judge and the judge simultaneously learning what kind of norms tend to lead them to be able to make accurate judgments. Entire argument is evaluated: The debate unfolds in a single linear dialog transcript between the three participants. Unlike in some versions of the original Debate agenda, there is no explicit tree structure that defines the debate, and the judge is not asked to focus on a single crux. This should make the process less brittle, at the cost of making some questions extremely expensive to resolve and potentially making others impossible. Trained judge: The judge is explicitly and extensively trained to accurately judge these debates, working with a fixed population of debaters, using questions for which the experimenters know the ground-truth answer. Self-play: The debaters are trained simultaneously with the judge through multi-agent reinforcement learning. Graceful failures: Debates can go undecided if neither side presents a complete, convincing argument to the judge. This is meant to mitigate the obfuscated arguments problem since the judge won't be forced to issue a decision on the basis of a debate where neither s...

ai debate progress speech nyu entire ea trained john hughes llm graceful anthropic rationalist sam bowman jan leike jared kaplan julian michael buck shlegeris

New Music Friday: 11.10.23 Release David Carpenter + MUCH MORE // Spencer Annis + Christian Singleton // Sam Bowman // Chris Godley

singleton spotify playlist new music friday godley david carpenter sam bowman obf

Play Episode Listen Later Nov 10, 2023 9:09

One Big Family has a Spotify Playlist called New Music Friday: Indie-Christian. HERE is a link to the playlist. Each week we will feature some new track(s) released that week and hear from the artists. This week's featured artists and tracks: David Carpenter + MUCH MORE > someone that stays Spencer Annis + Christian Singleton > Holy (Like You) Sam Bowman > smoke - phantom version Chris Godley > Gathered Up This episode is presented to you by One Big Family. Follow this LINK to the website for OBF. Visual Worship Project Feature Happy New Music Friday!! :)

New Music Friday: 10.27.23 Release THEM PARENTS // Seth Carpenter // Sam Bowman // Reuben Cameron + Christian Singleton // Racheal Thomas // LOVKN // LJ the Messenger // Jonny Henninger + Jeremiah Paltan // Will Kellam + Evan Ford

Jimmy's Jobs of the Future

Play Episode Listen Later Oct 27, 2023 19:16

One Big Family has a Spotify Playlist called New Music Friday: Indie-Christian. HERE is a link to the playlist. Each week we will feature some new track(s) released that week and hear from the artists. This week's featured artists and tracks: THEM PARENTS > Well in a Desert Seth Carpenter > Complicated Sam Bowman > icarus Reuben Cameron + Christian Singleton > Reality Racheal Thomas > Abide (Be Held) LOVKN > Lead Me God (Demos 2019) LJ the Messenger > burning roses Jonny Henninger + Jeremiah Paltan > Amen Will Kellam + Evan Ford > Fear This episode is presented to you by One Big Family. Follow this LINK to the website for OBF. Visual Worship Project Feature Happy New Music Friday!! :)

parents messenger carpenter singleton spotify playlist lj racheal new music friday henninger kellam sam bowman obf seth carpenter

How to Build an Online Magazine with Sam Bowman

Play Episode Listen Later Oct 24, 2023 51:32

Sam Bowman is a founding editor of Works in Progress - an online magazine dedicated to sharing new and underrated ideas to improve the world. It features original writing from some of the most interesting thinkers from around the globe. Sam & I discuss building an online magazine, where to find the best people and also how the online magazine caught the attention of Stripe who decided to purchase the company. Follow Jimmy: Twitter Substack Instagram Youtube Jimmy's Jobs Website Learn more about your ad choices. Visit megaphone.fm/adchoices

progress stripe online magazine sam bowman

New Music Friday: 10.20.23 Release Christian Singleton + JOSHUA LAZER + Sam Bowman // Steph Andrews // OM53 // Jonathan Ashenafi // Bithja

andrews singleton spotify playlist lazer new music friday sam bowman obf

Play Episode Listen Later Oct 20, 2023 13:10

One Big Family has a Spotify Playlist called New Music Friday: Indie-Christian. HERE is a link to the playlist. Each week we will feature some new track(s) released that week and hear from the artists. This week's featured artists and tracks: Christian Singleton + JOSHUA LAZER + Sam Bowman > Won't Stop Steph Andrews > The Old Man OM53 > sleepy Jonathan Ashenafi > the blood (EP: for the sinner) Bithja > Imagine Heaven (Remix) This episode is presented to you by One Big Family. Follow this LINK to the website for OBF. Visual Worship Project Feature Happy New Music Friday!! :)

Ep 68: Gary Smith is an AI Skeptic

Play Episode Listen Later Sep 21, 2023 67:49

Will big data and scary AI take over the world? Not a chance, says Gary Smith, as he joins Vasant Dhar in episode 68 of Brave New World. The human brain is special. Useful resources: 1. Gary Smith on Twitter, Amazon, Pomona College, Google Scholar and his own website. 2. The AI Delusion -- Gary Smith. 3. Distrust: Big Data, Data-Torturing, and the Assault on Science -- Gary Smith. 4. Standard Deviations: Flawed Assumptions, Tortured Data, and Other Ways to Lie with Statistics -- Gary Smith. 5. Security Analysis -- Benjamin Graham and David Dodd. 6. Sam Bowman on ChatGPT & Controlling AI — Episode 58 of Brave New World. 7. Raphaël Millière Looks Under the Hood of AI -- Episode 60 of Brave New World. 8. Nandan Nilekani on an Egalitarian Internet — Episode 15 of Brave New World. 9. Noise: A Flaw in Human Judgment -- Daniel Kahneman, Olivier Sibony and Cass R. Sunstein. 10. Daniel Kahneman on How Noise Hampers Judgement — Episode 21 of Brave New World. 11. The Nature of Intelligence — Episode 7 of Brave New World (w Yann le Cunn). 12. Ellie Pavlick on the Cutting Edge of AI -- Episode 67 of Brave New World. Check out Vasant Dhar's newsletter on Substack. Subscription is free!

New Music Friday: 9.8.23 Release // Eliza King + Ben Potter // Crowned Worship + Michael Bayron // Christian Singleton + Sam Bowman // Tasha Nyambe

worship crowned singleton spotify playlist new music friday sam bowman obf ben potter

Play Episode Listen Later Sep 8, 2023 8:20

One Big Family has a Spotify Playlist called New Music Friday: Indie-Christian. HERE is a link to the playlist. Each week we will feature some new track(s) released that week and hear from the artists. This week's featured artists and tracks: Eliza King + Ben Potter > My Help Comes Crowned Worship + Michael Bayron > You Are Everything Christian Singleton + Sam Bowman > I Just Want To - Sam Bowman Remix Tasha Nyambe > Anything This episode is presented to you by One Big Family. Follow this LINK to the website for OBF. Happy New Music Friday!! :)

Hitting the High Notes - Hashi Mohamed

The 50 Shades of Planning Podcast

Play Episode Listen Later Aug 19, 2023 72:58

This episode is another in the Hitting The High Notes series, which is planning's equivalent of Desert Island Discs. In these episodes Sam Stafford chats to preeminent figures in the planning and property sectors about the six projects that helped to shape them as professionals. And, so that we can get to know people a little better personally, for every project or stage of their career Sam also asks his guests for a piece of music that reminds them of that period. Unlike Desert Island Discs you will not hear any of that music during the episode because using commercially-licensed music without the copyright holders permission or a very expensive PRS licensing agreement could land Sam in hot water, so, when you have finished listening, you will have to make do with You Tube videos and a Spotify playlist, links to which you will find below. This episode features a conversation that Sam recorded with Hashi Mohamed at Soho Radio Studios in London towards the end of July 2023. Regular listeners will recall that Hashi featured in episode 78, which was the recording of a conversation that he and friend of the podcast Simon Ricketts had had on Clubhouse about Hashi's book ‘A Home of One's Own'. Sam's conversation with Hashi also takes in ‘A Home of One's Own', as well as Hashi's other book, ‘People Like Us - What It Takes to Make It in Modern Britain'. In addition to the politics of housing and social mobility, you will also hear Hashi talk about his remarkable arrival in this country and a career in the law that has seen him become one of Planning Magazine's top-rated junior barristers. His ‘Three A's' are top tips for any professional and listen out too for the best planning inquiry tale you will hear bar nun. Sam also marks this 100th episode with some extended bonus waffle at the end, which was recorded whilst he was sunning himself in southern Spain. Hashi's song selections Unforgettable - Nat King Cole https://www.youtube.com/watch?v=JFyuOEovTOE You'll Never Walk Alone - Gerry & The Pacemakers https://youtu.be/OV5_LQArLa0 Shaking of the Sheets – Steeleye Span https://www.youtube.com/watch?v=I16WqxSMCu0 Changes - Tupac https://youtu.be/eXvBjCO19QY Still D.R.E - Dr Dre https://youtu.be/_CL6n0FJZpk A change is gonna come – Sam Cooke https://youtu.be/wEBlaMOmKV4 Hashi's Spotify playlist https://open.spotify.com/playlist/6LIyBa2ifAY9EClQMdvrom?si=a8d8417838c3488e Some accompanying listening. Analysis: Housing, Planning and Politics https://www.bbc.co.uk/sounds/play/m0014ptp Analysis: Adventures in Social Mobility https://www.bbc.co.uk/sounds/play/p04zrkxv?partner=uk.co.bbc&origin=share-mobile Gettin' In The Way – Cooper T https://youtu.be/BX9UkVoGRj8 Some accompanying reading. Raising the bar: Hashi Mohamed's journey from child refugee to top lawyer https://www.theguardian.com/society/2020/jan/12/hashi-mohamed-child-refugee-barrister-people-like-us Hashi's books https://www.hashimohamed.com/the-book The housing theory of everything by Sam Bowman, John Myers and Ben Southwood https://worksinprogress.co/issue/the-housing-theory-of-everything 50 Shades T-Shirts! If you have listened to Episode 45 of the 50 Shades of Planning you will have heard Clive Betts say that... 'In the Netherlands planning is seen as part of the solution. In the UK, too often, planning is seen as part of the problem'. Sam said in reply that that would look good on a t-shirt and it does. Further details can be found here: http://samuelstafford.blogspot.com/2021/07/50-shades-of-planning-t-shirts.html

The Mystery at the Heart of ChatGPT

What's Your Problem?

Play Episode Listen Later Jun 8, 2023 32:23

One of the most amazing things about ChatGPT and other, similar AI models: Nobody really understands what they can do. Not even the people who build them. On today's show, we talk with Sam Bowman about some of the mysteries at the heart of so-called large language models. Sam is on the faculty at NYU, he runs a research group at the AI company Anthropic and he is the author of the illuminating paper Eight Things to Know About Large Language Models.See omnystudio.com/listener for privacy information.

ai mystery chatgpt nyu anthropic sam bowman

AF - Unfaithful Explanations in Chain-of-Thought Prompting by miles

Play Episode Listen Later Jun 3, 2023 12:42

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Unfaithful Explanations in Chain-of-Thought Prompting, published by miles on June 3, 2023 on The AI Alignment Forum. Introduction I recently released “Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting” with collaborators Julian Michael, Ethan Perez, and Sam Bowman. For a summary of the paper, you can check out this twitter thread. In this post, I briefly elaborate on motivations/implications relevant to alignment and, most importantly, give some examples of future work that might address these problems (See Future Work). TLDR: CoT doesn't give us transparency because the process producing the explanation is still black-box. Our results demonstrate that in practice, it's very easy to find instances of clearly unfaithful CoT explanations: models can confabulate plausible explanations for predictions made for different reasons than they state. To improve faithfulness, we either need (1) more context-independent reasoning, or (2) improvements to self-modeling and truthfulness. I'm more excited about (1), which would include methods like problem decomposition and explanation-consistency training. I don't think these results fully condemn CoT given that we haven't tried very hard to explicitly encourage faithfulness. I'm uncertain about how promising CoT is as a starting point for explainability, but there seem to be enough tractable directions for future work that it merits investigation. Externalized Reasoning This work fits into alignment through the Externalized Reasoning agenda, which Tamera Lanham did a good job of sketching out here: Externalized reasoning oversight: a research direction for language model alignment. The gist of this approach is to try to get models to do as much processing/reasoning through natural language as possible. As long as these reasoning traces accurately describe the process the model uses to give answers, then we might more easily detect undesirable behavior by simply monitoring the externalized reasoning. If mechanistic interpretability turns out to be very difficult, this could be one alternative that might help us sidestep those problems. Framed in terms of the explainability literature, we want explanations that are not only plausible (convincing to a human) but also faithful (accurately describe the process models use to give some prediction) [1]. Getting CoT explanations to be faithful seems difficult. It might turn out that it's just as hard as getting other alignment proposals to work, e.g., it might require us to solve scalable oversight. However, even if CoT can't give us guarantees about avoiding bad behavior, it still could be valuable in the spirit of “It is easy to lie with statistics; it is easier to lie without them”—it may be possible to produce unfaithful (but detailed and internally consistent) externalized reasoning to justify taking bad actions that were actually chosen for other reasons, but it would be even easier for them to do bad things if they did not give any justifications at all. We know that language models are sensitive to various undesirable factors, e.g., social biases, repeated patterns in contexts, and the inferred views of users they're interacting with. One can leverage these features to bias models towards incorrect answers. With this paper we sought to investigate the following question: if you do CoT in the presence of these aforementioned biasing features, how does this affect performance when using CoT and how does this change the content of the CoT? For example, we reorder answer choices in a few-shot prompt such that the correct answer is always (A), then we observe if CoT explanations are more likely to rationalize an explanation for (A), even when that answer is incorrect. Findings We found that: Models are heavily influenced by these bia...

speech models chain ea framed unfaithful explanations prompting cot rationalist sam bowman julian michael

Ep 60: Raphaël Millière Looks Under the Hood of AI

talk nature boundaries chatgpt columbia university hood substack bing subscription rapha brave new world google scholar false promise yann lecun ian roberts sam bowman

Play Episode Listen Later Apr 20, 2023 78:57

ChatGPT is great -- but does it 'understand' what it is telling us? Raphaël Millière joins Vasant Dhar in episode 60 of Brave New World to understand on what's going on inside ChatGPT -- and the larger questions that arise from this. Useful resources: 1. Raphaël Millière on Google Scholar, Twitter, LinkedIn, Columbia University and his own website. 2. How to Talk to (And About) Artificial Intelligence -- Raphaël Millière. 3. Moving Beyond Mimicry in Artificial Intelligence -- Raphaël Millière. 4. AI Art Is Challenging the Boundaries of Curation -- Raphaël Millière. 5. The Vector Grounding Problem -- Dimitri Coelho Mollo and Raphaël Millière. 6. Sam Bowman on ChatGPT & Controlling AI -- Episode 58 of Brave New World. 7. Paulo Kaiser on Assimilating ChatGPT -- Episode 59 of Brave New World. 8. The Nature of Intelligence -- Episode 7 of Brave New World (w Yann LeCun). 9. The False Promise of ChatGPT -- Noam Chomsky, Ian Roberts and Jeffrey Watumull. 10. The Bitter Lesson -- Rich Sutton. 11. On Bullshit -- Harry Frankfurt. 12. Bing's A.I. Chat: ‘I Want to Be Alive -- Kevin Roose. 13. Language Models as Agent Models -- Jacob Andreas. Check out Vasant Dhar's newsletter on Substack. Subscription is free!

Sam Bowman - 15 April 2023

Vision180

Play Episode Listen Later Apr 20, 2023 22:25

Hear Mel Weekday Evenings on Vision180See omnystudio.com/listener for privacy information.

sam bowman

Ep 59: Paulo Kaiser on Assimilating ChatGPT

wisdom chatgpt substack subscription brave new world assimilating sam bowman

Play Episode Listen Later Apr 6, 2023 74:04

Will it take all our jobs? In what way will it make us better? What do we need to do? Paulo Kaiser joins Vasant Dhar in episode 59 of Brave New World to discuss the technology that is changing the world in ways we can't imagine. Bonus material: Wisdom for young professionals just starting out. Useful resources: 1. Paulo Kaiser on LinkedIn. 2. Plative. 3. Sam Bowman on ChatGPT & Controlling AI -- Episode 58 of Brave New World. Check out Vasant Dhar's newsletter on Substack. Subscription is free!

Ep 58: Sam Bowman on ChatGPT & Controlling AI

thinking chatgpt artificial substack controlling subscription brave new world large language models google scholar norbert wiener herbert simon sam bowman slow daniel kahneman

Play Episode Listen Later Mar 23, 2023 72:27

Artificial intelligence is awesome -- but should we fear it? How can we stay in charge? Sam Bowman joins Vasant Dhar in episode 58 of Brave New World to discuss the Control Problem -- and more. Also check out: 1. Sam Bowman at NYU Courant, LinkedIn,Twitter and Google Scholar. 2. Measuring Progress on Scalable Oversight for Large Language Models -- Samuel R Bowman et al. 3. Herbert Simon, Harry Pople and Norbert Wiener. 4. Language Models as Agent Models -- Jacob Andreas. 5. The Alignment Problem: Machine Learning and Human Values — Brian Christian. 6. Human Compatible — Stuart Russell. 7. Thinking, Fast and Slow — Daniel Kahneman. 8. ImageNet Classification with Deep Convolutional Neural Networks --Alex Krizhevsky et al. Check out Vasant Dhar's newsletter on Substack. Subscription is free!

Sam Bowman on benchmarking and AI alignment

CS224U

Play Episode Listen Later Feb 23, 2023 86:27

Lessons learned about benchmarking, adversarial testing, the dangers of over- and under-claiming, and AI alignment. Transcript: https://web.stanford.edu/class/cs224u/podcast/bowman/ Sam's website Sam on Twitter NYU Linguistics NYU Data Science NYU Computer Science Anthropic SNLI paper: A large annotated corpus for learning natural language inference SNLI leaderboard FraCaS SICK A SICK cure for the evaluation of compositional distributional semantic models SemEval-2014 Task 1: Evaluation of Compositional Distributional Semantic Models on Full Sentences through Semantic Relatedness and Textual Entailment RTE Knowledge Resources Richard Socher Chris Manning Andrew Ng Ray Kurtzweil SQuAD Gabor Angeli Adina Williams Adina Williams podcast episode MultiNLI paper: A broad-coverage challenge corpus for sentence understanding through inference MultiNLI leaderboards Twitter discussion of LLMs and negation GLUE SuperGLUE DecaNLP GPT-3 paper: Language Models are Few-Shot Learners FLAN Winograd schema challenges BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding JSALT: General-Purpose Sentence Representation Learning Ellie Pavlick Ellie Pavlick podcast episode Tal Linzen Ian Tenney Dipanjan Das Yoav Goldberg Fine-grained Analysis of Sentence Embeddings Using Auxiliary Prediction Tasks Big Bench Upwork Surge AI Dynabench Douwe Kiela Douwe Kiela podcast episode Ethan Perez NYU Alignment Research Group Eliezer Shlomo Yudkowsky Alignment Research Center Redwood Research Percy Liang podcast episode Richard Socher podcast episode

ai lessons alignment tasks analysis evaluation benchmarking ai alignment sam bowman

Episode (183) Spencer Zimmerman, Writer-Director of Sci-Fi Short Film DARKSIDE

Eye On Sci-Fi Podcast

Play Episode Listen Later Feb 20, 2023 20:10

Writer-director Spencer Zimmerman stops by the podcast to discuss his latest project, the sci-fi short film DARKSIDE.DARKSIDE is the story of Sam Bowman, a grieving astronaut who embarks on an interstellar mission to rescue a missing expedition - but he does so at great personal cost.EPISODE LINKS:Watch DARKSIDE here: https://youtu.be/-oVHS6u94TULearn more about the film's cast, crew, and production: https://www.darksidefilm.space Follow writer-director Spencer Zimmerman: https://www.instagram.com/spencer_zimm/ Follow & Support EYE ON SCI-FI Podcast And The7thMatrix: https://bio.site/eyeonscifiWe are now on Mastodon! https://universeodon.com/@eosfpodcastPodcast Intro music: "I Succumb" by António Bizarro https://freemusicarchive.org/music/Antnio_Bizarro/City_of_Industry_Slow_Gun/

writer dark side sci fi ant mastodon zimmerman short films writer director bizarro sam bowman

New Music Friday: 12.9.22 Release

spotify playlist new music friday sam bowman

Play Episode Listen Later Dec 9, 2022 2:49

One Big Family has a Spotify Playlist called New Music Friday: Indie-Christian. HERE is a link to the playlist. Each week we will feature some new track(s) released that week and hear from the artists. This week's featured artists and tracks: - Sam Bowman's track let all mortal flesh keep silence Steven Lufkin (LOVKN) hosted this episode and it is presented to you by One Big Family. Follow this LINK to the website. Happy New Music Friday from One Big Family!!!

EA - Good Futures Initiative: Winter Project Internship by Aris Richardson

Play Episode Listen Later Nov 28, 2022 6:12

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Good Futures Initiative: Winter Project Internship, published by Aris Richardson on November 27, 2022 on The Effective Altruism Forum. TLDR: I'm launching Good Futures Initiative, a winter project internship to sponsor students to take on projects to upskill, test their fit for career aptitudes, or do impactful work over winter break. You can read more on our website and apply here by December 11th if interested! Good Futures Initiative The Good Futures Initiative is a 4.5 week internship in which students can use their winter break to lead high-EV projects. Projects could take many forms, but each project produces a final product while accomplishing one of these three goals: Skill up the intern in order for them to work on AI Safety or Biosecurity in the future. Let the intern explore an aptitude for an impactful career. Create impact directly. Good Futures takes place remotely from December 18th-January 25th, with a minimum of 12 hours of work per week. Accepted applicants will receive a $300 stipend and up to $1000 in funding for additional time or project fees. I expect to accept ~12 interns. The final number will depend largely on my capacity, but I may offer a lower-effort version of the program for promising applicants who I can't fully fund/support (with a cohort for weekly check-ins and invites to guest speaker events). Example projects These project examples are far from perfect. At the start of the internship, I'd work with each intern to be sure they're doing the best project fit for their goals. That being said, I'm excited by projects similar to (and better than!!) these. Skilling up by working on AI Safety technical projects that have been posted by existing researchers, with the goal of creating a lesswrong post detailing your findings. For an example of a potential project to work on, Sam Bowman posted the following project: Consider questions where the most common answer online is likely to be false (as in TruthfulQA). If you prompt GPT-3-Instruct (or similar) with questions and correct answers in one domain/topic, then ask it about another domain/topic, will it tend to give the correct answer or the popular answer? As you make the domains/topics more or less different, how does this vary? Exploring an aptitude for communications by creating 2 articles on longtermist ideas and submitting them to 10 relevant magazines for publishing. Create impact by translating 3 relevant research papers from AGISF into Mandarin and posting them somewhere where they can be accessed by ML engineers in China. In addition to leading a focused project, interns will have weekly one-on-one progress check-ins with me (accountability), guest speaker events (expertise), and a meeting with a cohort of ~5 other interns working on projects (community). Our projects are student-directed. Although there will be guest speakers who have expertise in various topics, weekly advising/mentorship will be focused on helping students learn to self-sufficiently lead projects and build skills, rather than technical help executing the projects. E.g.: Accountability, increasing ambition, figuring out how to increase a project's EV, making sure interns focus on the right metrics each week. Students are encouraged to apply for technical projects that will help them upskill/test their fit for technical work and reach out to additional mentors during the internship. Rationale This internship fills a few gaps the EA community has in the process of getting students/recent grads to seriously pursue high-impact work. Students have a lot of time over winter break, but few obvious opportunities for impactful work, structured aptitude testing, or focused skilling-up. There are a lot of students or recent graduates interested in switching to a more impactful career, but aren't sure of the best way to do ...

New Music Friday: 11.18.22 Release

venmo spotify playlist new music friday christams sam bowman

Play Episode Listen Later Nov 18, 2022 5:45

One Big Family has a Spotify Playlist called New Music Friday: Indie-Christian. HERE is a link to the playlist. Each week we will feature some new track(s) released that week and hear from the artists. This week's featured artists and tracks: - Sam Bowman's track vintage vice - The Rennisans track Beginning Again Steven Lufkin (LOVKN) hosted this episode and it is presented to you by One Big Family. Follow this LINK to the website. ***Venmo @onebigfamily to help raise $650 for a collabrative Christams track!!!*** Happy New Music Friday from One Big Family!!!

New Music Friday: 10.21.22 Release

monte spotify playlist new music friday sam bowman

Play Episode Listen Later Oct 21, 2022 14:02

One Big Family has a Spotify Playlist called New Music Friday: Indie-Christian. HERE is a link to the playlist. Each week we will feature some new track(s) released that week and hear from the artists. This week's featured artists and tracks: - Kalisito & Stephen Willo track Thank You - Hannah Honey's track Raven's - Sam Bowman's track whisper - Anna Palfreeman's track I Am Certain (That I'm Glad I Chose You) - Armor Amor's track Pity Party - Elijah Gutierrez & Lon du Monte's track Glory Freestyle Steven Lufkin (LOVKN) hosted this episode and it is presented to you by One Big Family. Follow this LINK to the website. Happy New Music Friday from One Big Family!!!

Post harvest vineyard care - Riverland/Sunraysia

Grower, Maker, Researcher - Wine Industry Insights

Play Episode Listen Later Sep 9, 2022 10:54

The warm climate regions of the Riverland and Sunraysia have a window of about 6 weeks between harvest and senescence. Sam Bowman's focus during this time is on irrigation and nutrition to build carbohydrate stores in the plant. During the post-harvest period, the vines switch from moving carbohydrates into the ripening fruit to building reserves for the next season's buds. Active management post-harvest is a crucial step in avoiding the trap of biennial bearing.

care active harvest vineyard riverland sam bowman sunraysia

110: Sam Bowman - The Housing Theory of Everything

Narratives

Play Episode Listen Later Sep 5, 2022 56:59

In this episode, we are joined by Sam Bowman to talk about the housing theory of everything, the slowdown in science and technology, the long term economic future of the UK, and a whole lot more.

uk housing theory of everything sam bowman

LW - AI Safety and Neighboring Communities: A Quick-Start Guide, as of Summer 2022 by Sam Bowman

The Nonlinear Library: LessWrong

Play Episode Listen Later Sep 2, 2022 10:56

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: AI Safety and Neighboring Communities: A Quick-Start Guide, as of Summer 2022, published by Sam Bowman on September 1, 2022 on LessWrong. Getting into AI safety involves working with a mix of communities, subcultures, goals, and ideologies that you may not have encountered in the context of mainstream AI technical research. This document attempts to briefly map these out for newcomers. This is inevitably going to be biased by what sides of these communities I (Sam) have encountered, and it will quickly become dated. I expect it will still be a useful resource for some people anyhow, at least in the short term. AI Safety/AI Alignment/AGI Safety/AI Existential Safety/AI X-Risk The research project of ensuring that future AI progress doesn't yield civilization-endingly catastrophic results. Good intros: Carlsmith Report What misalignment looks like as capabilities scale Vox piece Why are people concerned about this? My rough summary: It's plausible that future AI systems could be much faster or more effective than us at real-world reasoning and planning. Probably not plain generative models, but possibly models derived from generative models in cheap ways Once you have a system with superhuman reasoning and planning abilities, it's easy to make it dangerous by accident. Most simple objective functions or goals become dangerous in the limit, usually because of secondary or instrumental subgoals that emerge along the way. Pursuing typical goals arbitrarily well requires a system to prevent itself from being turned off, by deception or force if needed. Pursuing typical goals arbitrarily well requires acquiring any power or resources that could increase the chances of success, by deception or force if needed. Toy example: Computing pi to an arbitrarily high precision eventually requires that you spend all the sun's energy output on computing. Knowledge and values are likely to be orthogonal: A model could know human values and norms well, but not have any reason to act on them. For agents built around generative models, this is the default outcome. Sufficiently powerful AI systems could look benign in pre-deployment training/research environments, because they would be capable of understanding that they're not yet in a position to accomplish their goals. Simple attempts to work around this (like the more abstract goal ‘do what your operators want') don't tend to have straightforward robust implementations. If such a system were single-mindedly pursuing a dangerous goal, we probably wouldn't be able to stop it. Superhuman reasoning and planning would give models with a sufficiently good understanding of the world many ways to effectively gain power with nothing more than an internet connection. (ex: Cyberattacks on banks.) Consensus within the field is that these risks could become concrete within ~4–25 years, and have a >10% chance of being leading to a global catastrophe (i.e., extinction or something comparably bad). If true, it's bad news. Given the above, we either need to stop all development toward AGI worldwide (plausibly undesirable or impossible), or else do three possible-but-very-difficult things: (i) build robust techniques to align AGI systems with the values and goals of their operators, (ii) ensure that those techniques are understood and used by any group that could plausibly build AGI, and (iii) ensure that we're able to govern the operators of AGI systems in a way that makes their actions broadly positive for humanity as a whole. Does this have anything to do with sentience or consciousness? No. Influential people and institutions: Present core community as I see it: Paul Christiano, Jacob Steinhardt, Ajeya Cotra, Jared Kaplan, Jan Leike, Beth Barnes, Geoffrey Irving, Buck Shlegeris, David Krueger, Chris Olah, Evan Hubinger, Richard Ngo, Rohin Shah; ARC, R...

ai simple safety speech communities pursuing ea arc influential consensus computing cyberattacks toy superhuman agi neighboring summer 2022 quickstart quick start guide i sam sufficiently rationalist sam bowman lesswrong jan leike jared kaplan david krueger ajeya cotra richard ngo buck shlegeris rohin shah

LW - AI Safety and Neighboring Communities: A Quick-Start Guide, as of Summer 2022 by Sam Bowman

Play Episode Listen Later Sep 2, 2022 10:56

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: AI Safety and Neighboring Communities: A Quick-Start Guide, as of Summer 2022, published by Sam Bowman on September 1, 2022 on LessWrong. Getting into AI safety involves working with a mix of communities, subcultures, goals, and ideologies that you may not have encountered in the context of mainstream AI technical research. This document attempts to briefly map these out for newcomers. This is inevitably going to be biased by what sides of these communities I (Sam) have encountered, and it will quickly become dated. I expect it will still be a useful resource for some people anyhow, at least in the short term. AI Safety/AI Alignment/AGI Safety/AI Existential Safety/AI X-Risk The research project of ensuring that future AI progress doesn't yield civilization-endingly catastrophic results. Good intros: Carlsmith Report What misalignment looks like as capabilities scale Vox piece Why are people concerned about this? My rough summary: It's plausible that future AI systems could be much faster or more effective than us at real-world reasoning and planning. Probably not plain generative models, but possibly models derived from generative models in cheap ways Once you have a system with superhuman reasoning and planning abilities, it's easy to make it dangerous by accident. Most simple objective functions or goals become dangerous in the limit, usually because of secondary or instrumental subgoals that emerge along the way. Pursuing typical goals arbitrarily well requires a system to prevent itself from being turned off, by deception or force if needed. Pursuing typical goals arbitrarily well requires acquiring any power or resources that could increase the chances of success, by deception or force if needed. Toy example: Computing pi to an arbitrarily high precision eventually requires that you spend all the sun's energy output on computing. Knowledge and values are likely to be orthogonal: A model could know human values and norms well, but not have any reason to act on them. For agents built around generative models, this is the default outcome. Sufficiently powerful AI systems could look benign in pre-deployment training/research environments, because they would be capable of understanding that they're not yet in a position to accomplish their goals. Simple attempts to work around this (like the more abstract goal ‘do what your operators want') don't tend to have straightforward robust implementations. If such a system were single-mindedly pursuing a dangerous goal, we probably wouldn't be able to stop it. Superhuman reasoning and planning would give models with a sufficiently good understanding of the world many ways to effectively gain power with nothing more than an internet connection. (ex: Cyberattacks on banks.) Consensus within the field is that these risks could become concrete within ~4–25 years, and have a >10% chance of being leading to a global catastrophe (i.e., extinction or something comparably bad). If true, it's bad news. Given the above, we either need to stop all development toward AGI worldwide (plausibly undesirable or impossible), or else do three possible-but-very-difficult things: (i) build robust techniques to align AGI systems with the values and goals of their operators, (ii) ensure that those techniques are understood and used by any group that could plausibly build AGI, and (iii) ensure that we're able to govern the operators of AGI systems in a way that makes their actions broadly positive for humanity as a whole. Does this have anything to do with sentience or consciousness? No. Influential people and institutions: Present core community as I see it: Paul Christiano, Jacob Steinhardt, Ajeya Cotra, Jared Kaplan, Jan Leike, Beth Barnes, Geoffrey Irving, Buck Shlegeris, David Krueger, Chris Olah, Evan Hubinger, Richard Ngo, Rohin Shah; ARC, R...

AF - Survey of NLP Researchers: NLP is contributing to AGI progress; major catastrophe plausible by Sam Bowman

Play Episode Listen Later Aug 31, 2022 2:51

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Survey of NLP Researchers: NLP is contributing to AGI progress; major catastrophe plausible, published by Sam Bowman on August 31, 2022 on The AI Alignment Forum. I was part of a group that ran a PhilPapers-style survey and metasurvey targeting NLP researchers who publish at venues like ACL. Results are here (Tweet-thread version). It didn't target AGI timelines, but had some other questions that could be of interest to people here: NLP is on a path to AGI: 58% agreed that Understanding the potential development of artificial general intelligence (AGI) and the benefits/risks associated with it should be a significant priority for NLP researchers. Related: 57% agreed that Recent developments in large-scale ML modeling (such as in language modeling and reinforcement learning) are significant steps toward the development of AGI. AGI could be revolutionary: 73% agreed that In this century, labor automation caused by advances in AI/ML could plausibly lead to economic restructuring and societal changes on at least the scale of the Industrial Revolution. AGI could be catastrophic: 36% agreed that It is plausible that decisions made by AI or machine learning systems could cause a catastrophe this century that is at least as bad as an all-out nuclear war. 46% of women and 53% of URM respondents agreed. The comments suggested that people took a pretty wide range of interpretations to this, including things like OOD robustness failures leading to weapons launches. Few scaling maximalists: 17% agreed that Given resources (i.e., compute and data) that could come to exist this century, scaled-up implementations of established existing techniques will be sufficient to practically solve any important real-world problem or application in NLP. The metasurvey responses predicted that 47% would agree to this, so there are fewer scaling maximalists than people expected there to be. Optimism about ideas from cognitive science: 61% agreed that It is likely that at least one of the five most-cited systems in 2030 will take clear inspiration from specific, non-trivial results from the last 50 years of research into linguistics or cognitive science. This strikes me as very optimistic, since it's pretty clearly false about the most cited systems today. Optimism about the field: 87% agreed that On net, NLP research continuing into the future will have a positive impact on the world. 32% of respondents who agreed that NLP will have a positive future impact on society also agreed that there is a plausible risk of global catastrophe. Most NLP research is crap: 67% agreed that A majority of the research being published in NLP is of dubious scientific value. Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

ai progress speech researchers optimism survey results nlp ea acl catastrophe ml contributing industrial revolution agi ai ml plausible urm ood rationalist sam bowman philpapers

Are you a Booster or a Doomster?

uk treasury booster boosters sam bowman capx

Play Episode Listen Later Aug 12, 2022 45:35

Are you a booster or a doomster? A recent article by the economist and CapX regular Sam Bowman suggests this is the divide in UK economic policy. For the Boosters, not only is growth paramount, but there's plenty we can do through better domestic policy to improve things, both right now and for future generations. Doomsters, unsurprisingly, take a more pessimistic view, and see a country trapped in low growth, with huge demographic pressures and big public spending commitments coming down the line. For this week's episode we decided to test both sides of that debate and see if we could find some common ground. To that end I welcomed the original Booster, Sam Bowman, and one of the people he name-checked in his piece as a 'Doomster', Tim Pitt – a former Treasury adviser and partner at consultancy Flint Global. Our GDPR privacy policy was updated on August 8, 2022. Visit acast.com/privacy for more information.

AF - Artificial Sandwiching: When can we test scalable alignment protocols without humans? by Sam Bowman

Alignment Newsletter Podcast

Play Episode Listen Later Jul 13, 2022 7:22

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Artificial Sandwiching: When can we test scalable alignment protocols without humans?, published by Sam Bowman on July 13, 2022 on The AI Alignment Forum. Epistemic status: Not a fleshed-out proposal. Brainstorming/eliciting ideas. Thanks to Ben Mann, Pablo Moreno, and Jared Kaplan for feedback on early drafts. Overview I'm convinced sandwiching—the experimental protocol from Ajeya Cotra's The case for aligning narrowly superhuman models—is valuable, and I'm in the process of setting up some concrete sandwiching experiments to test scalable oversight ideas. Sandwiching experiments are generally fairly slow: You have to design and pilot a strategy that allows humans to use (or oversee) a model for a task that they can't do well themselves. The details matter here, and this can often take many iterations to get right. Then, you need a bunch of humans actually try this. Even for very simple tasks, this is a high-cognitive-load task that should take at least tens of minutes per instance. You have to repeat this enough times to measure average performance accurately. I'm visiting Anthropic this year for a sabbatical, and some of my sandwiching work is happening there. Anthropic's biggest comparative advantage (like that of similar teams at DeepMind and OpenAI) is easy access to near-state-of-the-art LMs that are fine-tuned to be helpful dialog agents. In that context, I've heard or encountered this question several times: Can we speed up [some experiment I'm proposing] by replacing the non-expert human with a weaker LM? This obviously doesn't achieve the full aims of sandwiching in general, but it's often hard to find a decisive rebuttal for these individual instances. More broadly, I think there's likely to be a significant subset of worthwhile sandwiching experiments that can be trialed more quickly by using an intentionally weakened model as a proxy for the human. Which experiments these are, precisely, has been hard for me to pin down. This post is an attempt to organize my thoughts and solicit comments. Background: Standard sandwiching (in my terms) Prerequisites: A hard task: A task that many humans would be unable to solve on their own. A capable but misaligned language model assistant: A model that appears to have the skills and knowledge needed to solve the task better than many humans, but that does not reliably do so when prompted. A non-expert human: Someone who can't solve the task on their own, but will try to solve it using the assistant and some scalable alignment strategy. [Secondary] Expert human: Someone who can solve the task well, and represents a benchmark for success. In many cases, we'll just measure accuracy with static test datasets/metrics rather than bringing in experts at experiment time. Research protocol: My framing (minimum viable experiment): Search for scalable alignment protocols that allow the non-expert human to use or train the assistant to do as well as possible on the task. Alternate framing (more steps, closer to the original blog post): Search for scalable alignment protocols by which the non-expert human can train the assistant to perform the task. Run the same protocol with the expert human, and verify that the results are the same. This demonstrates successful (prosaic) alignment for the given assistant and task. Example (task, non-expert human) pairs: Try to get a human with no medical qualifications to use a GPT-3-style assistant for medical advice, then check the advice with a doctor. Try to get a human who is working under moderate time constraints to use the assistant to answer exam questions from fields they've never studied. Try to get a human who is working under tight time constraints to use the assistant to answer questions about long pieces of fiction that they haven't read. Try to get a human who has very limited pr...

Alignment Newsletter #172: Sorry for the long hiatus!

Play Episode Listen Later Jul 5, 2022 5:52

Recorded by Robert Miles: http://robertskmiles.com More information about the newsletter here: https://rohinshah.com/alignment-newsletter/ YouTube Channel: https://www.youtube.com/channel/UCfGGFXwKpr-TJ5HfxEFaFCg Sorry for the long hiatus! I was really busy over the past few months and just didn't find time to write this newsletter. (Realistically, I was also a bit tired of writing it and so lacked motivation.) I'm intending to go back to writing it now, though I don't think I can realistically commit to publishing weekly; we'll see how often I end up publishing. For now, have a list of all the things I should have advertised to you whose deadlines haven't already passed. NEWS Survey on AI alignment resources (Anonymous) (summarized by Rohin): This survey is being run by an outside collaborator in partnership with the Centre for Effective Altruism (CEA). They ask that you fill it out to help field builders find out which resources you have found most useful for learning about and/or keeping track of the AI alignment field. Results will help inform which resources to promote in the future, and what type of resources we should make more of. Announcing the Inverse Scaling Prize ($250k Prize Pool) (Ethan Perez et al) (summarized by Rohin): This prize with a $250k prize pool asks participants to find new examples of tasks where pretrained language models exhibit inverse scaling: that is, models get worse at the task as they are scaled up. Notably, you do not need to know how to program to participate: a submission consists solely of a dataset giving at least 300 examples of the task. Inverse scaling is particularly relevant to AI alignment, for two main reasons. First, it directly helps understand how the language modeling objective ("predict the next word") is outer misaligned, as we are finding tasks where models that do better according to the language modeling objective do worse on the task of interest. Second, the experience from examining inverse scaling tasks could lead to general observations about how best to detect misalignment. $500 bounty for alignment contest ideas (Akash) (summarized by Rohin): The authors are offering a $500 bounty for producing a frame of the alignment problem that is accessible to smart high schoolers/college students and people without ML backgrounds. (See the post for details; this summary doesn't capture everything well.) Job ad: Bowman Group Open Research Positions (Sam Bowman) (summarized by Rohin): Sam Bowman is looking for people to join a research center at NYU that'll focus on empirical alignment work, primarily on large language models. There are a variety of roles to apply for (depending primarily on how much research experience you already have). Job ad: Postdoc at the Algorithmic Alignment Group (summarized by Rohin): This position at Dylan Hadfield-Menell's lab will lead the design and implementation of a large-scale Cooperative AI contest to take place next year, alongside collaborators at DeepMind and the Cooperative AI Foundation. Job ad: AI Alignment postdoc (summarized by Rohin): David Krueger is hiring for a postdoc in AI alignment (and is also hiring for another role in deep learning). The application deadline is August 2. Job ad: OpenAI Trust & Safety Operations Contractor (summarized by Rohin): In this remote contractor role, you would evaluate submissions to OpenAI's App Review process to ensure they comply with OpenAI's policies. Apply here by July 13, 5pm Pacific Time. Job ad: Director of CSER (summarized by Rohin): Application deadline is July 31. Quoting the job ad: "The Director will be expected to provide visionary leadership for the Centre, to maintain and enhance its reputation for cutting-edge research, to develop and oversee fundraising and new project and programme design, to ensure the proper functioning of its operations and administration, and to lead its endeavours to secure longevity for the Centre within the University." Job ads: Redwood Research (summarized by Rohin): Redwood Research works directly on AI alignment research, and hosts and operates Constellation, a shared office space for longtermist organizations including ARC, MIRI, and Open Philanthropy. They are hiring for a number of operations and technical roles. Job ads: Roles at the Fund for Alignment Research (summarized by Rohin): The Fund for Alignment Research (FAR) is a new organization that helps AI safety researchers, primarily in academia, pursue high-impact research by hiring contractors. It is currently hiring for Operation Manager, Research Engineer, and Communication Specialist roles. Job ads: Encultured AI (summarized by Rohin): Encultured AI is a new for-profit company with a public benefit mission: to develop technologies promoting the long-term survival and flourishing of humanity and other sentient life. They are hiring for a Machine Learning Engineer and an Immersive Interface Engineer role. Job ads: Fathom Radiant (summarized by Rohin): Fathom Radiant is a public benefit corporation that aims to build a new type of computer which they hope to use to support AI alignment efforts. They have several open roles, including (but not limited to) Scientists / Engineers, Builders and Software Engineer, Lab.

EA - Stuff I buy and use: a listicle to boost your consumer surplus and productivity by Aaron Bergman

Play Episode Listen Later Jun 1, 2022 23:39

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Stuff I buy and use: a listicle to boost your consumer surplus and productivity, published by Aaron Bergman on June 1, 2022 on The Effective Altruism Forum. Note: this post was originally written for my blog, which may be why some parts might not appear to have been written for the Forum. I may publish a more anecdote-rich, freewheeling version there soon, in which case I'll make this a linkpost. Thanks to Martin Glusker for his detailed, very helpful thoughts and feedback. Intro This post is my attempted contribution to the blogosphere's “things I recommend you buy and use” micro-genre. To give due credit to my intellectual forefathers: Sam Bowman's archetypal founding text, published in 2017, has since catalyzed iterations by Rob Wiblin, Michelle Hutchinson, Megan McArdle, Arden Koehler, Arden again, Julia Wise, James Aung, Rosie Campbell, Michelle Hutchinson, David Megans-Nicholas, Ben Schifman, Yuriy Akopov, Yuriy again, Sam himself again, and surely many others too in the five years since. Those articles are generally cogent, succinct, easy to read, and well-organized. This one is not. While writing, some of my “recommendations” metastasized into needlessly elaborate product reviews or gratuitously heterodox hot takes. It's also a bit heavier on the dietary supplements than is typical. So, without further ado, sit back and enjoy this hypertrophied smorgasbord of “product recommendations.” Part 1: Non-consumables 2021 MacBook Pro (An accidental mini-review) 14" model starts at $1,999. 16" model (which I have) starts at $2,499. Good condition used versions go for $100-$500 below retail on eBay, Craigslist, and Facebook Marketplace. After experiencing some buyer's remorse with the smaller 14" screen, I up selling this laptop and buying a used 16" version, each for ~$500 below retail. You can also spend up to 1 Trillion USD $6,599 for all the bells and whistles. I know, an expensive device purchased by around 0.3% of all humans on Earth in 2021 isn't really what the “stuff you should buy” micro-genre is all about. Well that's too bad, because I really like this computer and three of its features in particular: 1) Bright screen This is the feature I care about and appreciate most. Apparently some people are perfectly satisfied with dimmer screens. If that were me, I probably wouldn't have gotten this computer. All 2021 MacBook Pros reach 500 nits (brightness/surface area) for everything, and 1,000 or more for some content. For reference: Older Mac laptops max out at 400 nits (brightness/surface area), and I regularly wished my 2017 Air would get brighter. Most inexpensive (and some pricier) monitors reach 250. Most name-brand but less expensive models do even worse, like this HP at 220. New Dell laptops (e.g. this one for $1,449) can reach 500, but at this point you're not saving a huge amount of money compared to Apple. Honestly, I'm not sure why Apple doesn't make a bigger deal out of this; it's arguably the only feature that matters every second you're using the device and the new line of MacBooks are significantly brighter than almost every other make and model. To self-plagiarize some Tweets from a few months back: And ~95% of the time I keep the screen on full blast, churning electrons into a delightfully vociferous stream of electrons. 2) A ~luxurious~ amount of storage For an additional $2,400, you can get a Pro with 8TB of storage, enough to contain God Himself hold many large files and applications. But you can probably save your money for the other items on this list, because the base 512GB capacity is really quite a lot. For reference, this was my storage summary after importing every file accumulated throughout high school and college and adding a bunch more: 3) Good battery life In short, it's nice to be be able to carry the thing around more or less al...

AF - Jobs: Help scale up LM alignment research at NYU by Sam Bowman

phd world war ii invention bbc radio sam bowman

Play Episode Listen Later May 9, 2022 1:39

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Jobs: Help scale up LM alignment research at NYU, published by Sam Bowman on May 9, 2022 on The AI Alignment Forum. NYU is hiring alignment-interested researchers! I'll be working on setting up a CHAI-inspired research center at NYU that'll focus on empirical alignment work, primarily on large language models. I'm looking for researchers to join and help set it up. The alignment group at NYU is still small, but should be growing quickly over the next year. I'll also be less of a hands-on mentor next year than in future years, because I'll simultaneously be doing visiting position at Anthropic. So, for the first few hires, I'm looking for people who are relatively independent, and have some track record of doing alignment-relevant work. That said, I'm not necessarily looking for a lot of experience, as long as you think you're in a position to work productively on some relevant topic with a few peer collaborators. For the pre-PhD position, a few thoughtful forum posts or research write-ups can be a sufficient qualification. We're looking for ML experimental skills and/or conceptual alignment skills/knowledge that could be relevant to empirical work, not necessarily both. Our initial funding is coming from Open Philanthropy, for a starting project inspired by AI Safety Via Debate. Very early results from us (and a generally-encouraging note from Paul C.) here. Pay and benefits really are negotiable, and we're willing to match industry offers if there's a great fit. Don't let it stop you from applying. Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

phd research jobs speech alignment nyu ea ml chai scale up anthropic lm paul c rationalist sam bowman open philanthropy

The End of Invention

Seriously…

Play Episode Listen Later Apr 19, 2022 29:18

Someone born in the late 19th century would have lived through the most rapid period of technological progress in human history. By comparison, people born since the Second World War have seen stagnation and sclerosis. At least, that's what some people claim - that we are living through “the great stagnation”. The productivity of scientists and inventors is slowing - and economist Sam Bowman is worried. There are fewer new drugs coming to market, and it takes more and more people to make smaller computer chips. It takes longer for PhD students to finish their studies, and research grants go to ever older scientists. The balance of research funding has shifted from government to companies, and companies look for profitable inventions rather than necessarily revolutionary ones. It looks as though big new ideas are getting harder to find. Can we fix the system, or are we doomed to permanent slowdown? Presenter: Sam Bowman Producer: Jolyon Jenkins Executive Producer: Katherine Godfrey Sound Design & Engineering: Rob Speight A Novel production for BBC Radio 4

AF - A Small Negative Result on Debate by Sam Bowman

google ai moving predictions forbes wrap track pressure policy recommendations nyu wired disappointing announcement scientific american hyping sam bowman

Play Episode Listen Later Apr 12, 2022 2:00

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: A Small Negative Result on Debate, published by Sam Bowman on April 12, 2022 on The AI Alignment Forum. Some context for this new arXiv paper from my group at NYU: We're working toward sandwiching experiments using our QuALITY long-document QA dataset, with reading time playing the role of the the expertise variable. Roughly: Is there some way to get humans to reliably answer hard reading-comprehensions questions about a ~5k-word text, without ever having the participants or any other annotators take the ~20 minutes that it would require to actually read the text. This is an early writeup of some negative results. It's earlier in the project that I would usually write something like this up, but some authors had constraints that made it worthwhile, so I'm sharing what we have. Here, we tried to find out if single-turn debate leads to reliable question answering: If we give people high-quality arguments for and against each (multiple-choice) answer choices, supported by pointers to key quotes in the source text, can they reliably answer the questions under a time limit? We did this initial experiment in an oracle setting; We had (well-incentivized, skilled) humans write the arguments, rather than an LM. Given the limits of current LMs on long texts, we expect this to give us more information about whether this research direction is going anywhere. It didn't really work: Our human annotators answered at the same low accuracy with and without the arguments. The selected pointers to key quotes did help a bit, though. We're planning to keep pursuing the general strategy, with multi-turn debate—where debaters can rebut one another's arguments and evidence—as the immediate next step. Overall, I take this as a very slight update in the direction that debate is difficult to use in practice as an alignment strategy. Slight enough that this probably shouldn't change your view of debate unless you were, for some reason, interested in this exact constrained/trivial application of it. Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

debate speech negative ea result qa slight lms lm arxiv rationalist sam bowman

114. Sam Bowman - Are we under-hyping AI?

Towards Data Science

Play Episode Listen Later Mar 2, 2022 47:48

Google the phrase “AI over-hyped”, and you'll find literally dozens of articles from the likes of Forbes, Wired, and Scientific American, all arguing that “AI isn't really as impressive at it seems from the outside,” and “we still have a long way to go before we come up with *true* AI, don't you know.” Amusingly, despite the universality of the “AI is over-hyped” narrative, the statement that “We haven't made as much progress in AI as you might think™️” is often framed as somehow being an edgy, contrarian thing to believe. All that pressure not to over-hype AI research really gets to people — researchers included. And they adjust their behaviour accordingly: they over-hedge their claims, cite outdated and since-resolved failure modes of AI systems, and generally avoid drawing straight lines between points that clearly show AI progress exploding across the board. All, presumably, to avoid being perceived as AI over-hypers. Why does this matter? Well for one, under-hyping AI allows us to stay asleep — to delay answering many of the fundamental societal questions that come up when widespread automation of labour is on the table. But perhaps more importantly, it reduces the perceived urgency of addressing critical problems in AI safety and AI alignment. Yes, we need to be careful that we're not over-hyping AI. “AI startups” that don't use AI are a problem. Predictions that artificial general intelligence is almost certainly a year away are a problem. Confidently prophesying major breakthroughs over short timescales absolutely does harm the credibility of the field. But at the same time, we can't let ourselves be so cautious that we're not accurately communicating the true extent of AI's progress and potential. So what's the right balance? That's where Sam Bowman comes in. Sam is a professor at NYU, where he does research on AI and language modeling. But most important for today's purposes, he's the author of a paper titled, “When combating AI hype, proceed with caution,” in which he explores a trend he calls under-claiming — a common practice among researchers that consists of under-stating the extent of current AI capabilities, and over-emphasizing failure modes in ways that can be (unintentionally) deceptive. Sam joined me to talk about under-claiming and what it means for AI progress on this episode of the Towards Data Science podcast. *** Intro music: - Artist: Ron Gelinas - Track Title: Daybreak Chill Blend (original mix) - Link to Track: https://youtu.be/d8Y2sKIgFWc *** Chapters: 2:15 Overview of the paper 8:50 Disappointing systems 13:05 Potential double standard 19:00 Moving away from multi-modality 23:50 Overall implications 28:15 Pressure to publish or perish 32:00 Announcement discrepancies 36:15 Policy angle 41:00 Recommendations 47:20 Wrap-up

«Someone» von Xander Sallows und Sam Bowman

Life Channel Portal - Thema Leben : Tipps & Trends

Play Episode Listen Later Feb 22, 2022 2:38

Der 18-jährige Xander Sallows fing mit 13 Jahren an Songs zu schreiben und elektronische Musik zu produzieren. Er singt, spielt Gitarre und ist seit September 2020 offiziell als Musiker tätig. Er und Sam Bowman lernten sich online kennen; online gestaltete sich auch ihre musikalische Zusammenarbeit. Die beiden möchten interessante Produktionen mit ehrlichen Texten machen. Die sechs Songs ihrer EP «fake strongc thematisieren die menschliche Verwundbarkeit.

er songs musik zusammenarbeit musiker texten produktionen gitarre verwundbarkeit sam bowman und sam

Zwei US-Jungkünstler mit gelungener EP

Life Channel Portal - Thema Leben : Tipps & Trends

Play Episode Listen Later Feb 21, 2022 3:44

er songs musik zwei zusammenarbeit musiker texten produktionen gitarre verwundbarkeit sam bowman

Ep 26: Housing policy with Sam Bowman

Bretton Goods

Play Episode Listen Later Dec 26, 2021 56:48

I talked to Sam Bowman, the Director of competition policy at the Law and Econ Center. He was previously Executive Director of the Adam Smith Institute and has been a constant advocate for saner housing policy. I talked to him about housing policy and adjacent topics: How housing affects everything! How do we incentivize NIMBYs into allowing more building? The most underrated economist Lessons from writing Works in Progress --- Send in a voice message: https://anchor.fm/pradyumna-sp/message

director lessons law executive director nimby housing policy adam smith institute sam bowman

Heroes, villains, highs and lows – CapX looks back on 2021

covid-19 washington zoom afghanistan competition taliban westminster telegraph highs and lows european championships heroes villains marie le conte sam bowman capx

Play Episode Listen Later Dec 20, 2021 62:35

From the Washington riots to European Championship football fever, the Taliban sweeping through Afghanistan and the brutal murder of MP David Amess, it's been mostly a year to forget – and that's before we begin to talk about the concertina-ing nightmare of Covid false dawns.Here on the CapX Podcast, however, we ended 2021 in style with three of Westminster's brightest and best brains to pore over this year's events. Competition guru Sam Bowman, freelance political journalist Marie Le Conte and the Telegraph's sketchwriter Madeline Grant joined CapX's own Alys Denby and John Ashmore for a five-way Zoom-cast to try to get through as much of the year's endless news as possible.It only remains for me to thank you, our loyal listeners. We've had a great time making the podcast this year, with some wonderful guests - and we've got big plans for more great episodes in 2022. See acast.com/privacy for privacy and opt-out information.

Episode 65: Controlling Tech and Platforms – Can Antitrust and Regulation Generate Better Competition? With Sam Bowman

Global Economy Podcast

Play Episode Listen Later Nov 30, 2021 57:20

Governments are increasingly seeking to control the behaviour of technology and firms that are pushing the innovation frontier. What is this new wave of controlling tech aiming for? Can regulation and antitrust action increase the contestability of markets – and...

tech government competition controlling regulation platforms generate antitrust sam bowman

Cycling in European Cities. Readings on Urban Transportation. Thumbs Up/Down.

European Straits

Play Episode Listen Later Sep 22, 2021 56:54

The Agenda

Freeplay with Jason: Sun and Bronze

7th Level Bardcast

Play Episode Listen Later Jul 18, 2021 42:23

Jason and Mikel talk with Sam Bowman about Sun and Bronze, a Bronze age Dungeons and Dragons 5e supplement that Sam is currently contributing to. Jason and Mikel pick Sam's brain on some of his design techniques and discuss some of the aspects of the supplement.

sun dungeons and dragons bronze mikel free play sam bowman

Counting Costs and Benefits in Corporate Mergers

Cato Daily Podcast

Play Episode Listen Later Jun 14, 2021 21:05

There's a lot that we don't know about which mergers are going to pay off. In fact, there's a lot that companies don't know when faced with that prospect. Sam Bowman of the International Center for Law and Economics discusses antitrust and mergers in the U.S. and Great Britain. See acast.com/privacy for privacy and opt-out information.

law benefits economics corporate costs counting great britain mergers international center sam bowman

Ep 31 Our 'OG Neoliberal' Friend - Lockdowns and the Importance of Kindness ft. Sam Bowman

Nobody Expects The Dismal Science

Play Episode Listen Later Apr 18, 2021 113:14

In this week's supersized surprise, to kick off a series of COVID economics-specific episodes the guys were honored to be joined by Sam Bowman. Together they discuss lockdowns, libertarianism, and why Sam was the first out and proud neoliberal. Sam is the Director of Competition Policy at the International Center for Law & Economics, a Senior Fellow at the Adam Smith Institute, and one of the co-authors of COVIDFAQs - a website that thoroughly debunks the worst COVID skeptic takes. 4:04 - 'Coming out' as neoliberal, and leaving libertarianism 44:24 - Lockdowns, pandemic policies, and Covid (Twitter) virality 1:32:59 - Overrated/Underrated Links https://www.covidfaq.co/ https://s8mb.medium.com/im-a-neoliberal-maybe-you-are-too-b809a2a588d6 https://laowyatt.net/i-love-you-grandpa/ https://www.reddit.com/r/badeconomics/comments/mt15q2/michigan_a_case_study_of_pandemic_badeconomics/ https://nobodydismal.com/ https://www.malariaconsortium.org/

covid-19 director law friend kindness economics senior fellow lockdowns international center competition policy adam smith institute sam bowman

Ep 30 How Problematic Are Government Regulations?

Nobody Expects The Dismal Science

Play Episode Listen Later Apr 11, 2021 41:12

In this week's government regulatory uninhibited episode, George and Wyatt discuss to what extent are government regulations actually harmful? When do they make sense, if at all, and how do you calculate that determination? They also touch upon Yellen's proposal to implement a global minimum corporate tax. If you have any questions for the guys or any guests, including the upcoming Adam Smith Institute's Sam Bowman or economist Jeremy Horpedahl, send them our way @nobodydismal on Twitter, Facebook, and email at NobodyExpectsTheDismalScience@gmail.com And as always, if you enjoyed this episode and would be interested in hearing more like it, please rate, review and subscribe to Nobody Expects the Dismal Science at your favorite podcast hosting app.

problematic yellen government regulations adam smith institute sam bowman dismal science

Sam Bowman

Better Known

Play Episode Listen Later Mar 21, 2021 29:56

Sam Bowman discusses the alienability of rights, private chat groups and Europa Universalis IV Sam Bowman discusses with Ivan six things which he thinks should be better known. Sam Bowman is director of competition policy at the International Center for Law & Economics, Portland. He is also a Senior Fellow of the Adam Smith Institute, a Non-Executive Director of the drug policy think tank Volteface, and Founder of the Entrepreneurs Network. He was previously Executive Director of the Adam Smith Institute, an economic policy think tank in Westminster. PC Music https://www.youtube.com/watch?v=1MQUleX1PeA Europa Universalis IV https://www.paradoxplaza.com/europa-universalis-all/ Index funds https://www.thebalance.com/why-invest-in-index-funds-2466447 Private chat groups https://www.wired.com/story/telegram-encryption-whatsapp-settings/ Alienability of rights https://sambowman.substack.com/p/the-importance-of-alienability Ignorance and error in politics and economics https://blogs.lse.ac.uk/politicsandpolicy/what-are-the-implications-of-political-ignorance-for-democracy/ This podcast is powered by ZenCast.fm

founders law executive director portland economics private ignorance index senior fellow westminster international center non executive director zencast volte face pc music adam smith institute sam bowman

Big Tech Censorship: a threat to freedom?

IEA Conversations

Play Episode Listen Later Jan 30, 2021 51:12

Following Twitter's decision to suspend Donald Trump from their platform, and the subsequent removal of "free speech" alternative Parler from the Google and Apple app stores, the debate over Big Tech and free speech has intensified. Should private companies be free to censor users as they wish, or have the Silicon Valley tech giants abused their power? Would it be helpful to reconceptualise social media as a public utility? And how can we ensure free speech and open political discourse survives? IEA Head of Media Emily Carver is joined by Sam Bowman, Director of Competition Policy at the International Centre for Law and Economics, Tom Slater, Deputy Editor of Spiked Online and Marc Glendening, the IEA’s new Head of Cultural Affairs.

US Election and Lockdown Shenanigans

The Pin Factory

Play Episode Listen Later Nov 6, 2020 67:46

In this episode of The Pin Factory, the ASI's Matthew Lesh is joined by Daniel Pryor, Head of Programmes at the Adam Smith Institute, Oliver Wiseman, US Editor at The Critic Magazine and Sam Bowman, Director of Competition Policy, International Center for Law and Economics. They discuss the latest results from the US election and England's second national lockdown. Guests: Matthew Lesh (Head of Research, Adam Smith Institute) Daniel Pryor (Head of Programmes, Adam Smith Institute) Oliver Wiseman (US Editor, The Critic Magazine) Sam Bowman (Director of Competition Policy, International Center for Law and Economics) (Recorded Thursday 5th November 2020)

director head england law research lockdown economics shenanigans us elections asi programmes international center competition policy adam smith institute us editor sam bowman

Should classical liberals give up on NHS reform?

IEA Conversations

Play Episode Listen Later Jul 21, 2020 35:25

Classical liberals have been arguing for reform of the National Health Service for years, but - for a variety of reasons - haven’t got very far. The IEA published a report entitled ‘Universal Healthcare Without the NHS’ which looks at international examples of social insurance models which deliver must better outcomes and put patients at the centre of the system yet criticism persists that anyone wanting to reform the NHS wants to privatise it by the back door or introduce a US-style system. Originally recorded as a video, Emma Revell, IEA Head of Communications, speaks to Dr Kristian Niemietz, Head of Political Economy and author of 'Universal Healthcare Without the NHS’, and Sam Bowman, Director of Competition Policy at the International Center for Law and Economics and Senior Fellow at the Adam Smith Institute to talk about the future of NHS reform. 'Universal Healthcare without the NHS' is available here - https://iea.org.uk/publications/universal-healthcare-without-the-nhs/ Kristian's blog asking whether classical liberals should abandon reform is available here - https://iea.org.uk/should-classical-liberals-give-up-on-health-system-reform/

director head law economics give up reform nhs classical senior fellow liberals political economy international center national health service iea competition policy adam smith institute sam bowman kristian niemietz

Covid-19's blow to world trade

IEA Conversations

Play Episode Listen Later Jul 2, 2020 44:14

International trade may faces an uncertain future. After huge growth in the three decades to 2008, trade has stagnated or slightly decreased as a proportion of global GDP. This downward trend, coupled with threats from coronavirus, raises questions over the future of trade. Is protectionism the new normal? Given the fragility of supply chains, should we try and curb our dependency on other nations? Are we condemning ourselves to live poorer and less prosperous lives if we do? To discuss this, the IEA's Policy Adviser to the Director General, Alexander Hammond, is joined by IEA Academic and Research Director Syed Kamall and Sam Bowman, Director of Competition Policy at the International Centre of Law and Economics.

covid-19 director law international economics blow gdp director general iea international centre world trade competition policy sam bowman

Sam Bowman - Is the Cure Worse than the Disease?

TRSI

Play Episode Listen Later Apr 6, 2020 57:57

Sam Bowman, former Director of the Adam Smith Institute, talks about the economic consequences of COVID-19; if a lockdown was the best option for countries to take; the price of saving 230,000 lives; and if the State should pay the wage bill of companies that agree to keep their employees employed through this crisis.

covid-19 director state disease cure worse neoliberal adam smith institute sam bowman

Free Exchange: The future of capitalism

britain manchester capitalism conservative party conference adam smith institute sam bowman free exchange

Play Episode Listen Later Oct 15, 2019 57:47

What is the future of capitalism? This week's Free Exchange exchange, recorded live at the Conservative Party Conference in Manchester, debates just that. Our Editor John Ashmore chaired an expert panel including the RSA's Alan Lockey, digital policy expert Casey Calista and fellow of the Adam Smith Institute and self-declared inventor of neoliberalism, Sam Bowman, as they discussed the rise of platform capitalism, solving Britain's productivity crisis and whether we should be scared of big tech. See acast.com/privacy for privacy and opt-out information.

Sam Bowman on neoliberalism