Podcast appearances and mentions of shane legg

Ben Goertzel discusses AGI development, transhumanism, and the potential societal impacts of superintelligent AI. He predicts human-level AGI by 2029 and argues that the transition to superintelligence could happen within a few years after. Goertzel explores the challenges of AI regulation, the limitations of current language models, and the need for neuro-symbolic approaches in AGI research. He also addresses concerns about resource allocation and cultural perspectives on transhumanism. TOC: [00:00:00] AGI Timeline Predictions and Development Speed [00:00:45] Limitations of Language Models in AGI Development [00:02:18] Current State and Trends in AI Research and Development [00:09:02] Emergent Reasoning Capabilities and Limitations of LLMs [00:18:15] Neuro-Symbolic Approaches and the Future of AI Systems [00:20:00] Evolutionary Algorithms and LLMs in Creative Tasks [00:21:25] Symbolic vs. Sub-Symbolic Approaches in AI [00:28:05] Language as Internal Thought and External Communication [00:30:20] AGI Development and Goal-Directed Behavior [00:35:51] Consciousness and AI: Expanding States of Experience [00:48:50] AI Regulation: Challenges and Approaches [00:55:35] Challenges in AI Regulation [00:59:20] AI Alignment and Ethical Considerations [01:09:15] AGI Development Timeline Predictions [01:12:40] OpenCog Hyperon and AGI Progress [01:17:48] Transhumanism and Resource Allocation Debate [01:20:12] Cultural Perspectives on Transhumanism [01:23:54] AGI and Post-Scarcity Society [01:31:35] Challenges and Implications of AGI Development New! PDF Show notes: https://www.dropbox.com/scl/fi/fyetzwgoaf70gpovyfc4x/BenGoertzel.pdf?rlkey=pze5dt9vgf01tf2wip32p5hk5&st=svbcofm3&dl=0 Refs: 00:00:15 Ray Kurzweil's AGI timeline prediction, Ray Kurzweil, https://en.wikipedia.org/wiki/Technological_singularity 00:01:45 Ben Goertzel: SingularityNET founder, Ben Goertzel, https://singularitynet.io/ 00:02:35 AGI Conference series, AGI Conference Organizers, https://agi-conf.org/2024/ 00:03:55 Ben Goertzel's contributions to AGI, Wikipedia contributors, https://en.wikipedia.org/wiki/Ben_Goertzel 00:11:05 Chain-of-Thought prompting, Subbarao Kambhampati, https://arxiv.org/abs/2405.04776 00:11:35 Algorithmic information content, Pieter Adriaans, https://plato.stanford.edu/entries/information-entropy/ 00:12:10 Turing completeness in neural networks, Various contributors, https://plato.stanford.edu/entries/turing-machine/ 00:16:15 AlphaGeometry: AI for geometry problems, Trieu, Li, et al., https://www.nature.com/articles/s41586-023-06747-5 00:18:25 Shane Legg and Ben Goertzel's collaboration, Shane Legg, https://en.wikipedia.org/wiki/Shane_Legg 00:20:00 Evolutionary algorithms in music generation, Yanxu Chen, https://arxiv.org/html/2409.03715v1 00:22:00 Peirce's theory of semiotics, Charles Sanders Peirce, https://plato.stanford.edu/entries/peirce-semiotics/ 00:28:10 Chomsky's view on language, Noam Chomsky, https://chomsky.info/1983____/ 00:34:05 Greg Egan's 'Diaspora', Greg Egan, https://www.amazon.co.uk/Diaspora-post-apocalyptic-thriller-perfect-MIRROR/dp/0575082097 00:40:35 'The Consciousness Explosion', Ben Goertzel & Gabriel Axel Montes, https://www.amazon.com/Consciousness-Explosion-Technological-Experiential-Singularity/dp/B0D8C7QYZD 00:41:55 Ray Kurzweil's books on singularity, Ray Kurzweil, https://www.amazon.com/Singularity-Near-Humans-Transcend-Biology/dp/0143037889 00:50:50 California AI regulation bills, California State Senate, https://sd18.senate.ca.gov/news/senate-unanimously-approves-senator-padillas-artificial-intelligence-package 00:56:40 Limitations of Compute Thresholds, Sara Hooker, https://arxiv.org/abs/2407.05694 00:56:55 'Taming Silicon Valley', Gary F. Marcus, https://www.penguinrandomhouse.com/books/768076/taming-silicon-valley-by-gary-f-marcus/ 01:09:15 Kurzweil's AGI prediction update, Ray Kurzweil, https://www.theguardian.com/technology/article/2024/jun/29/ray-kurzweil-google-ai-the-singularity-is-nearer

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: AGI Safety and Alignment at Google DeepMind: A Summary of Recent Work, published by Rohin Shah on August 20, 2024 on The AI Alignment Forum. We wanted to share a recap of our recent outputs with the AF community. Below, we fill in some details about what we have been working on, what motivated us to do it, and how we thought about its importance. We hope that this will help people build off things we have done and see how their work fits with ours. Who are we? We're the main team at Google DeepMind working on technical approaches to existential risk from AI systems. Since our last post, we've evolved into the AGI Safety & Alignment team, which we think of as AGI Alignment (with subteams like mechanistic interpretability, scalable oversight, etc.), and Frontier Safety (working on the Frontier Safety Framework, including developing and running dangerous capability evaluations). We've also been growing since our last post: by 39% last year, and by 37% so far this year. The leadership team is Anca Dragan, Rohin Shah, Allan Dafoe, and Dave Orr, with Shane Legg as executive sponsor. We're part of the overall AI Safety and Alignment org led by Anca, which also includes Gemini Safety (focusing on safety training for the current Gemini models), and Voices of All in Alignment, which focuses on alignment techniques for value and viewpoint pluralism. What have we been up to? It's been a while since our last update, so below we list out some key work published in 2023 and the first part of 2024, grouped by topic / sub-team. Our big bets for the past 1.5 years have been 1) amplified oversight, to enable the right learning signal for aligning models so that they don't pose catastrophic risks, 2) frontier safety, to analyze whether models are capable of posing catastrophic risks in the first place, and 3) (mechanistic) interpretability, as a potential enabler for both frontier safety and alignment goals. Beyond these bets, we experimented with promising areas and ideas that help us identify new bets we should make. Frontier Safety The mission of the Frontier Safety team is to ensure safety from extreme harms by anticipating, evaluating, and helping Google prepare for powerful capabilities in frontier models. While the focus so far has been primarily around misuse threat models, we are also working on misalignment threat models. FSF We recently published our Frontier Safety Framework, which, in broad strokes, follows the approach of responsible capability scaling, similar to Anthropic's Responsible Scaling Policy and OpenAI's Preparedness Framework. The key difference is that the FSF applies to Google: there are many different frontier LLM deployments across Google, rather than just a single chatbot and API (this in turn affects stakeholder engagement, policy implementation, mitigation plans, etc). We're excited that our small team led the Google-wide strategy in this space, and demonstrated that responsible capability scaling can work for large tech companies in addition to small startups. A key area of the FSF we're focusing on as we pilot the Framework, is how to map between the critical capability levels (CCLs) and the mitigations we would take. This is high on our list of priorities as we iterate on future versions. Some commentary (e.g. here) also highlighted (accurately) that the FSF doesn't include commitments. This is because the science is in early stages and best practices will need to evolve. But ultimately, what we care about is whether the work is actually done. In practice, we did run and report dangerous capability evaluations for Gemini 1.5 that we think are sufficient to rule out extreme risk with high confidence. Dangerous Capability Evaluations Our paper on Evaluating Frontier Models for Dangerous Capabilities is the broadest suite of dangerous capability evaluati...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: AGI Safety and Alignment at Google DeepMind: A Summary of Recent Work, published by Rohin Shah on August 20, 2024 on LessWrong. We wanted to share a recap of our recent outputs with the AF community. Below, we fill in some details about what we have been working on, what motivated us to do it, and how we thought about its importance. We hope that this will help people build off things we have done and see how their work fits with ours. Who are we? We're the main team at Google DeepMind working on technical approaches to existential risk from AI systems. Since our last post, we've evolved into the AGI Safety & Alignment team, which we think of as AGI Alignment (with subteams like mechanistic interpretability, scalable oversight, etc.), and Frontier Safety (working on the Frontier Safety Framework, including developing and running dangerous capability evaluations). We've also been growing since our last post: by 39% last year, and by 37% so far this year. The leadership team is Anca Dragan, Rohin Shah, Allan Dafoe, and Dave Orr, with Shane Legg as executive sponsor. We're part of the overall AI Safety and Alignment org led by Anca, which also includes Gemini Safety (focusing on safety training for the current Gemini models), and Voices of All in Alignment, which focuses on alignment techniques for value and viewpoint pluralism. What have we been up to? It's been a while since our last update, so below we list out some key work published in 2023 and the first part of 2024, grouped by topic / sub-team. Our big bets for the past 1.5 years have been 1) amplified oversight, to enable the right learning signal for aligning models so that they don't pose catastrophic risks, 2) frontier safety, to analyze whether models are capable of posing catastrophic risks in the first place, and 3) (mechanistic) interpretability, as a potential enabler for both frontier safety and alignment goals. Beyond these bets, we experimented with promising areas and ideas that help us identify new bets we should make. Frontier Safety The mission of the Frontier Safety team is to ensure safety from extreme harms by anticipating, evaluating, and helping Google prepare for powerful capabilities in frontier models. While the focus so far has been primarily around misuse threat models, we are also working on misalignment threat models. FSF We recently published our Frontier Safety Framework, which, in broad strokes, follows the approach of responsible capability scaling, similar to Anthropic's Responsible Scaling Policy and OpenAI's Preparedness Framework. The key difference is that the FSF applies to Google: there are many different frontier LLM deployments across Google, rather than just a single chatbot and API (this in turn affects stakeholder engagement, policy implementation, mitigation plans, etc). We're excited that our small team led the Google-wide strategy in this space, and demonstrated that responsible capability scaling can work for large tech companies in addition to small startups. A key area of the FSF we're focusing on as we pilot the Framework, is how to map between the critical capability levels (CCLs) and the mitigations we would take. This is high on our list of priorities as we iterate on future versions. Some commentary (e.g. here) also highlighted (accurately) that the FSF doesn't include commitments. This is because the science is in early stages and best practices will need to evolve. But ultimately, what we care about is whether the work is actually done. In practice, we did run and report dangerous capability evaluations for Gemini 1.5 that we think are sufficient to rule out extreme risk with high confidence. Dangerous Capability Evaluations Our paper on Evaluating Frontier Models for Dangerous Capabilities is the broadest suite of dangerous capability evaluations published...

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: AGI Safety and Alignment at Google DeepMind: A Summary of Recent Work, published by Rohin Shah on August 20, 2024 on LessWrong. We wanted to share a recap of our recent outputs with the AF community. Below, we fill in some details about what we have been working on, what motivated us to do it, and how we thought about its importance. We hope that this will help people build off things we have done and see how their work fits with ours. Who are we? We're the main team at Google DeepMind working on technical approaches to existential risk from AI systems. Since our last post, we've evolved into the AGI Safety & Alignment team, which we think of as AGI Alignment (with subteams like mechanistic interpretability, scalable oversight, etc.), and Frontier Safety (working on the Frontier Safety Framework, including developing and running dangerous capability evaluations). We've also been growing since our last post: by 39% last year, and by 37% so far this year. The leadership team is Anca Dragan, Rohin Shah, Allan Dafoe, and Dave Orr, with Shane Legg as executive sponsor. We're part of the overall AI Safety and Alignment org led by Anca, which also includes Gemini Safety (focusing on safety training for the current Gemini models), and Voices of All in Alignment, which focuses on alignment techniques for value and viewpoint pluralism. What have we been up to? It's been a while since our last update, so below we list out some key work published in 2023 and the first part of 2024, grouped by topic / sub-team. Our big bets for the past 1.5 years have been 1) amplified oversight, to enable the right learning signal for aligning models so that they don't pose catastrophic risks, 2) frontier safety, to analyze whether models are capable of posing catastrophic risks in the first place, and 3) (mechanistic) interpretability, as a potential enabler for both frontier safety and alignment goals. Beyond these bets, we experimented with promising areas and ideas that help us identify new bets we should make. Frontier Safety The mission of the Frontier Safety team is to ensure safety from extreme harms by anticipating, evaluating, and helping Google prepare for powerful capabilities in frontier models. While the focus so far has been primarily around misuse threat models, we are also working on misalignment threat models. FSF We recently published our Frontier Safety Framework, which, in broad strokes, follows the approach of responsible capability scaling, similar to Anthropic's Responsible Scaling Policy and OpenAI's Preparedness Framework. The key difference is that the FSF applies to Google: there are many different frontier LLM deployments across Google, rather than just a single chatbot and API (this in turn affects stakeholder engagement, policy implementation, mitigation plans, etc). We're excited that our small team led the Google-wide strategy in this space, and demonstrated that responsible capability scaling can work for large tech companies in addition to small startups. A key area of the FSF we're focusing on as we pilot the Framework, is how to map between the critical capability levels (CCLs) and the mitigations we would take. This is high on our list of priorities as we iterate on future versions. Some commentary (e.g. here) also highlighted (accurately) that the FSF doesn't include commitments. This is because the science is in early stages and best practices will need to evolve. But ultimately, what we care about is whether the work is actually done. In practice, we did run and report dangerous capability evaluations for Gemini 1.5 that we think are sufficient to rule out extreme risk with high confidence. Dangerous Capability Evaluations Our paper on Evaluating Frontier Models for Dangerous Capabilities is the broadest suite of dangerous capability evaluations published...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Shane Legg's necessary properties for every AGI Safety plan, published by jacquesthibs on May 1, 2024 on LessWrong. I've been going through the FAR AI videos from the alignment workshop in December 2023. I'd like people to discuss their thoughts on Shane Legg's 'necessary properties' that every AGI safety plan needs to satisfy. The talk is only 5 minutes, give it a listen: Otherwise, here are some of the details: All AGI Safety plans must solve these problems (necessary properties to meet at the human level or beyond): Good world model Good reasoning Specification of the values and ethics to follow All of these require good capabilities, meaning capabilities and alignment are intertwined. Shane thinks future foundation models will solve conditions 1 and 2 at the human level. That leaves condition 3, which he sees as solvable if you want fairly normal human values and ethics. Shane basically thinks that if the above necessary properties are satisfied at a competent human level, then we can construct an agent that will consistently choose the most value-aligned actions. And you can do this via a cognitive loop that scaffolds the agent to do this. Shane says at the end of this talk: If you think this is a terrible idea, I want to hear from you. Come talk to me afterwards and tell me what's wrong with this idea. Since many of us weren't at the workshop, I figured I'd share the talk here to discuss it on LW. Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Shane Legg's necessary properties for every AGI Safety plan, published by jacquesthibs on May 1, 2024 on LessWrong. I've been going through the FAR AI videos from the alignment workshop in December 2023. I'd like people to discuss their thoughts on Shane Legg's 'necessary properties' that every AGI safety plan needs to satisfy. The talk is only 5 minutes, give it a listen: Otherwise, here are some of the details: All AGI Safety plans must solve these problems (necessary properties to meet at the human level or beyond): Good world model Good reasoning Specification of the values and ethics to follow All of these require good capabilities, meaning capabilities and alignment are intertwined. Shane thinks future foundation models will solve conditions 1 and 2 at the human level. That leaves condition 3, which he sees as solvable if you want fairly normal human values and ethics. Shane basically thinks that if the above necessary properties are satisfied at a competent human level, then we can construct an agent that will consistently choose the most value-aligned actions. And you can do this via a cognitive loop that scaffolds the agent to do this. Shane says at the end of this talk: If you think this is a terrible idea, I want to hear from you. Come talk to me afterwards and tell me what's wrong with this idea. Since many of us weren't at the workshop, I figured I'd share the talk here to discuss it on LW. Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org

For decades, Shane Legg has anticipated the arrival of “artificial general intelligence” or AGI. In other words: an artificial agent that can do all the kinds of cognitive tasks that people can typically do, and possibly more...Now as the Chief AGI Scientist and a co-founder of Google DeepMind, he stands by that prediction and is calling on the world to prepare. This week on How I Built This Lab, Shane's path to becoming an early AI expert and the work he and his team are doing to prepare for the technological revolution ahead. This episode was produced by Sam Paulson with music composed by Ramtin Arablouei. It was edited by John Isabella with research help from Carla Esteves. Our audio engineer was Cena Loffredo.You can follow HIBT on X & Instagram, and email us at hibt@id.wondery.com.See Privacy Policy at https://art19.com/privacy and California Privacy Notice at https://art19.com/privacy#do-not-sell-my-info.

In this episode, Nathan sits down with Dan O'Connell, Chief Strategy Officer at Dialpad. They discuss building their own language models using 5 billion minutes of business calls, custom speech recognition models for every customer, and the challenges of bringing AI into business. If you need an ecommerce platform, check out our sponsor Shopify: https://shopify.com/cognitive for a $1/month trial period. We're sharing a few of Nathan's favorite AI scouting episodes from other shows. Today, Shane Legg, Cofounder at Deepmind and its current Chief AGI Scientist, shares his insights with Dwarkesh Patel on AGI's timeline, the new architectures needed for AGI, and why multimodality will be the next big landmark. If you need an ecommerce platform, check out our sponsor Shopify: https://shopify.com/cognitive for a $1/month trial period. We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com. --- SPONSORS: Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off www.omneky.com NetSuite has 25 years of providing financial software for all your business needs. More than 36,000 businesses have already upgraded to NetSuite by Oracle, gaining visibility and control over their financials, inventory, HR, eCommerce, and more. If you're looking for an ERP platform ✅ head to NetSuite: http://netsuite.com/cognitive and download your own customized KPI checklist. X/SOCIAL: @labenz (Nathan) @dialdoc (Dan) @dialpad @CogRev_Podcast (Cognitive Revolution) TIMESTAMPS: (00:00) - Introduction and Welcome (06:50) - Interview with Dan O'Connell, Chief AI and Strategy Officer at Dialpad (07:13) - The Functionality and Utility of Dialpad (17:20) - The Development of Dialpad's Large Language Model Trained on 5Billion Minutes of Calls 19:56 The Future of AI in Business (22:21) - Sponsor Break: Shopify (23:56) - The Challenges and Opportunities of AI Development (31:17 ) - Prioritizing latency, capacity, and cost when evaluating AI (39:41) - Most Loved AI Features in Dialpad (42:01) - The Role of AI in Quality Assurance (43:10) - The Future of Transcription Accuracy (44:06) - The Importance of Speech Recognition in Business (46:59) - Personalizing AI for Better Business Interactions (47:01) - The Role of AI in Content Generation (52:47) - The Challenges and Opportunities of AI in Sales and Support

We're sharing a few of Nathan's favorite AI scouting episodes from other shows. Today, Shane Legg, Cofounder at Deepmind and its current Chief AGI Scientist, shares his insights with Dwarkesh Patel on AGI's timeline, the new architectures needed for AGI, and why multimodality will be the next big landmark. If you need an ecommerce platform, check out our sponsor Shopify: https://shopify.com/cognitive for a $1/month trial period. You can subscribe to The Dwarkesh Podcast here: https://www.youtube.com/@DwarkeshPatel We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com. --- SPONSORS: Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off www.omneky.com NetSuite has 25 years of providing financial software for all your business needs. More than 36,000 businesses have already upgraded to NetSuite by Oracle, gaining visibility and control over their financials, inventory, HR, eCommerce, and more. If you're looking for an ERP platform ✅ head to NetSuite: http://netsuite.com/cognitive and download your own customized KPI checklist. X/SOCIAL: @labenz (Nathan) @dwarkesh_sp (Dwarkesh) @shanelegg (Shane) @CogRev_Podcast (Cognitive Revolution) TIMESTAMPS: (00:00:00) - Episode Preview with Nathan's Intro (00:02:45) - Conversation with Dwarkesh and Shane begins (00:14:26) - Do we need new architectures? (00:17:31) - Sponsors: Shopify  (00:19:40) - Is search needed for creativity? (00:31:46) - Impact of Deepmind on safety vs capabilities (00:32:48) - Sponsors: Netsuite | Omneky (00:37:10) - Timelines (00:45:18) - Multimodality

Shane Legg, co-founder di Google DeepMind, è un ricercatore e imprenditore nel campo della Artificial General Intelligence. Oggi estrapoliamo alcuni spunti da una sua intervista che vale la pena appuntarsi, visto il momento storico che stiamo attraversando. P.S. Vuoi provare il tool di Edo che riassume video molto lunghi? Abbiamo il link anche di questo: www.heydrafter.com ▫️ Entra nella nostra community: https://bit.ly/3jUs8Ez ▫️ Vuoi ricevere gli Appunti (la nostra newsletter)? https://bit.ly/3RJkbA5CONTATTI▫️ Email: info@hacking-creativity.com▫️ Instagram: https://bit.ly/2HVP8D4CONVENZIONI SPECIALI▫️ Ricorda la convenzione con SERENIS: usa il codice HACKINGCREATIVITY7 per pagare le prime 3 sedute 42€ invece del prezzo standard di 49€ + colloquio gratuito: https://bit.ly/3RskS4z▫️ Se utilizzi il codice di CREATIVITYX3, avrai la possibilità di utilizzare Qonto gratuitamente per ben 3 mesi! Qonto ti aiuta ad aprire una partita IVA in meno di 24 ore, con la consulenza di un esperto che ti guida. Qui trovi tutti i dettagli: https://bit.ly/3Px3n1f▫️ Se vuoi acquistare carte creative dalla casa editrice Sefirot del nostro amico Matteo Di Pascale, ricordati di inserire il codice promozionale HACKINGCREATIVITY15 per avere il 15% di sconto!IL MERCATINO DELL'AFFILIAZIONE▫️ Usa il codice HACKING50 per ottenere 50 euro di sconto sull'abbonamento a Fiscozen: https://bit.ly/3v7lReZ

Men vad gör gänget bakom Deep Mind nu egentligen?  Demis Hassabis, Mustafa Suleyman och Shane Legg grundade Deep Mind 2010 för att utveckla och främja forskning inom AI. Men vad gör dom nu?

As the cofounder of Google DeepMind, Shane Legg is driving one of the greatest transformations in history: the development of artificial general intelligence (AGI). He envisions a system with human-like intelligence that would be exponentially smarter than today's AI, with limitless possibilities and applications. In conversation with head of TED Chris Anderson, Legg explores the evolution of AGI, what the world might look like when it arrives — and how to ensure it's built safely and ethically.

As the cofounder of Google DeepMind, Shane Legg is driving one of the greatest transformations in history: the development of artificial general intelligence (AGI). He envisions a system with human-like intelligence that would be exponentially smarter than today's AI, with limitless possibilities and applications. In conversation with head of TED Chris Anderson, Legg explores the evolution of AGI, what the world might look like when it arrives — and how to ensure it's built safely and ethically.

As the cofounder of Google DeepMind, Shane Legg is driving one of the greatest transformations in history: the development of artificial general intelligence (AGI). He envisions a system with human-like intelligence that would be exponentially smarter than today's AI, with limitless possibilities and applications. In conversation with head of TED Chris Anderson, Legg explores the evolution of AGI, what the world might look like when it arrives — and how to ensure it's built safely and ethically.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Possible OpenAI's Q* breakthrough and DeepMind's AlphaGo-type systems plus LLMs, published by Burny on November 23, 2023 on LessWrong. tl;dr: OpenAI leaked AI breakthrough called Q*, acing grade-school math. It is hypothesized combination of Q-learning and A*. It was then refuted. DeepMind is working on something similar with Gemini, AlphaGo-style Monte Carlo Tree Search. Scaling these might be crux of planning for increasingly abstract goals and agentic behavior. Academic community has been circling around these ideas for a while. https://www.reuters.com/technology/sam-altmans-ouster-openai-was-precipitated-by-letter-board-about-ai-breakthrough-2023-11-22/ https://twitter.com/MichaelTrazzi/status/1727473723597353386 "Ahead of OpenAI CEO Sam Altman's four days in exile, several staff researchers sent the board of directors a letter warning of a powerful artificial intelligence discovery that they said could threaten humanity Mira Murati told employees on Wednesday that a letter about the AI breakthrough called Q* (pronounced Q-Star), precipitated the board's actions. Given vast computing resources, the new model was able to solve certain mathematical problems. Though only performing math on the level of grade-school students, acing such tests made researchers very optimistic about Q*'s future success." https://twitter.com/SilasAlberti/status/1727486985336660347 "What could OpenAI's breakthrough Q* be about? It sounds like it's related to Q-learning. (For example, Q* denotes the optimal solution of the Bellman equation.) Alternatively, referring to a combination of the A* algorithm and Q learning. One natural guess is that it is AlphaGo-style Monte Carlo Tree Search of the token trajectory. It seems like a natural next step: Previously, papers like AlphaCode showed that even very naive brute force sampling in an LLM can get you huge improvements in competitive programming. The next logical step is to search the token tree in a more principled way. This particularly makes sense in settings like coding and math where there is an easy way to determine correctness. https://twitter.com/mark_riedl/status/1727476666329411975 "Anyone want to speculate on OpenAI's secret Q* project? Something similar to tree-of-thought with intermediate evaluation (like A*)? Monte-Carlo Tree Search like forward roll-outs with LLM decoder and q-learning (like AlphaGo)? Maybe they meant Q-Bert, which combines LLMs and deep Q-learning Before we get too excited, the academic community has been circling around these ideas for a while. There are a ton of papers in the last 6 months that could be said to combine some sort of tree-of-thought and graph search. Also some work on state-space RL and LLMs." https://www.theverge.com/2023/11/22/23973354/a-recent-openai-breakthrough-on-the-path-to-agi-has-caused-a-stir OpenAI spokesperson Lindsey Held Bolton refuted it: "refuted that notion in a statement shared with The Verge: "Mira told employees what the media reports were about but she did not comment on the accuracy of the information."" https://www.wired.com/story/google-deepmind-demis-hassabis-chatgpt/ Google DeepMind's Gemini, that is currently the biggest rival with GPT4, which was delayed to the start of 2024, is also trying similar things: AlphaZero-based MCTS through chains of thought, according to Hassabis. Demis Hassabis: "At a high level you can think of Gemini as combining some of the strengths of AlphaGo-type systems with the amazing language capabilities of the large models. We also have some new innovations that are going to be pretty interesting." https://twitter.com/abacaj/status/1727494917356703829 Aligns with DeepMind Chief AGI scientist Shane Legg saying: "To do really creative problem solving you need to start searching." https://twitter.com/iamgingertrash/status/1727482695356494132 "...

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Possible OpenAI's Q* breakthrough and DeepMind's AlphaGo-type systems plus LLMs, published by Burny on November 23, 2023 on LessWrong. tl;dr: OpenAI leaked AI breakthrough called Q*, acing grade-school math. It is hypothesized combination of Q-learning and A*. It was then refuted. DeepMind is working on something similar with Gemini, AlphaGo-style Monte Carlo Tree Search. Scaling these might be crux of planning for increasingly abstract goals and agentic behavior. Academic community has been circling around these ideas for a while. https://www.reuters.com/technology/sam-altmans-ouster-openai-was-precipitated-by-letter-board-about-ai-breakthrough-2023-11-22/ https://twitter.com/MichaelTrazzi/status/1727473723597353386 "Ahead of OpenAI CEO Sam Altman's four days in exile, several staff researchers sent the board of directors a letter warning of a powerful artificial intelligence discovery that they said could threaten humanity Mira Murati told employees on Wednesday that a letter about the AI breakthrough called Q* (pronounced Q-Star), precipitated the board's actions. Given vast computing resources, the new model was able to solve certain mathematical problems. Though only performing math on the level of grade-school students, acing such tests made researchers very optimistic about Q*'s future success." https://twitter.com/SilasAlberti/status/1727486985336660347 "What could OpenAI's breakthrough Q* be about? It sounds like it's related to Q-learning. (For example, Q* denotes the optimal solution of the Bellman equation.) Alternatively, referring to a combination of the A* algorithm and Q learning. One natural guess is that it is AlphaGo-style Monte Carlo Tree Search of the token trajectory. It seems like a natural next step: Previously, papers like AlphaCode showed that even very naive brute force sampling in an LLM can get you huge improvements in competitive programming. The next logical step is to search the token tree in a more principled way. This particularly makes sense in settings like coding and math where there is an easy way to determine correctness. https://twitter.com/mark_riedl/status/1727476666329411975 "Anyone want to speculate on OpenAI's secret Q* project? Something similar to tree-of-thought with intermediate evaluation (like A*)? Monte-Carlo Tree Search like forward roll-outs with LLM decoder and q-learning (like AlphaGo)? Maybe they meant Q-Bert, which combines LLMs and deep Q-learning Before we get too excited, the academic community has been circling around these ideas for a while. There are a ton of papers in the last 6 months that could be said to combine some sort of tree-of-thought and graph search. Also some work on state-space RL and LLMs." https://www.theverge.com/2023/11/22/23973354/a-recent-openai-breakthrough-on-the-path-to-agi-has-caused-a-stir OpenAI spokesperson Lindsey Held Bolton refuted it: "refuted that notion in a statement shared with The Verge: "Mira told employees what the media reports were about but she did not comment on the accuracy of the information."" https://www.wired.com/story/google-deepmind-demis-hassabis-chatgpt/ Google DeepMind's Gemini, that is currently the biggest rival with GPT4, which was delayed to the start of 2024, is also trying similar things: AlphaZero-based MCTS through chains of thought, according to Hassabis. Demis Hassabis: "At a high level you can think of Gemini as combining some of the strengths of AlphaGo-type systems with the amazing language capabilities of the large models. We also have some new innovations that are going to be pretty interesting." https://twitter.com/abacaj/status/1727494917356703829 Aligns with DeepMind Chief AGI scientist Shane Legg saying: "To do really creative problem solving you need to start searching." https://twitter.com/iamgingertrash/status/1727482695356494132 "...

I had a lot of fun chatting with Shane Legg - Founder and Chief AGI Scientist, Google DeepMind!We discuss:* Why he expects AGI around 2028* How to align superhuman models* What new architectures needed for AGI* Has Deepmind sped up capabilities or safety more?* Why multimodality will be next big landmark* and much moreWatch full episode on YouTube, Apple Podcasts, Spotify, or any other podcast platform. Read full transcript here.Timestamps(0:00:00) - Measuring AGI(0:11:41) - Do we need new architectures?(0:16:26) - Is search needed for creativity?(0:19:19) - Superhuman alignment(0:29:58) - Impact of Deepmind on safety vs capabilities(0:34:03) - Timelines(0:41:24) - Multimodality This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.dwarkeshpatel.com

We are thrilled to announce the third session of our new Incubator Program. If you have a business idea that involves a web or mobile app, we encourage you to apply to our eight-week program. We'll help you validate your market opportunity, experiment with messaging and product ideas, and move forward with confidence toward an MVP. Learn more and apply at tbot.io/incubator. We look forward to seeing your application in our inbox! Peter Voss is the CEO and Chief Scientist of Aigo.ai, a groundbreaking alternative to conventional chatbots and generative models like ChatGPT. Aigo's chatbot is powered by Artificial General Intelligence (AGI), enabling it to think, learn, and reason much like a human being. It boasts short-term and long-term memory, setting it apart in terms of personalized service and context-awareness. Along with host Chad Pytel, Peter talks about how most chatbots and AI systems today are basic. They can answer questions but can't understand or remember the context. Aigo.ai is different because it's built to think and learn more like humans. It can adapt and get better the more you use it. He also highlights the challenges Aigo.ai faces in securing venture capital, given that its innovative approach doesn't align with current investment models heavily focused on generative or deep learning AI. Peter and Chad agree that while generative AI serves certain functions well, the quest for a system that can think, learn, and reason like a human demands a fundamentally different approach. Aigo.ai (https://aigo.ai/) Follow Aigo.ai on LinkedIn (https://www.linkedin.com/company/aigo-ai/) or YouTube (https://www.youtube.com/channel/UCl3XKNOL5rEit0txjVA07Ew). Follow Peter Voss on LinkedIn (https://www.linkedin.com/in/vosspeter/). Visit his website: optimal.org/voss.html (http://optimal.org/voss.html) Follow thoughtbot on X (https://twitter.com/thoughtbot) or LinkedIn (https://www.linkedin.com/company/150727/). Become a Sponsor (https://thoughtbot.com/sponsorship) of Giant Robots! Transcript: CHAD: This is the Giant Robots Smashing Into Other Giant Robots Podcast, where we explore the design, development, and business of great products. I'm your host, Chad Pytel. And with me today is Peter Voss, CEO and Chief Scientist at Aigo.ai. Peter, thanks so much for joining me. PETER: Yes, thank you. CHAD: So, tell us a little bit about what Aigo.ai does. You've been working in AI for a long time. And it seems like Aigo is sort of the current culmination of a lot of your 15 years of work, so... PETER: Yes, exactly. So, the quick way to describe our current product is a chatbot with a brain, and the important part is the brain. That basically, for the last 15-plus years, I've been working on the core technology for what's called AGI, Artificial General Intelligence, a system that can think, learn, reason similar to the way humans do. Now, we're not yet at human level with this technology. But it's a lot smarter and a lot more usable than traditional chatbots that don't have a brain. CHAD: I want to dig into this idea a little bit. I think, like a lot of people, I've used just traditional chatbots, particularly like ChatGPT is the latest. I've built some things on top of it. What is the brain that makes it different? Especially if you've used one, what is using Aigo going to be different? PETER: Right. I can give a concrete example of one of our customers then I can talk about the technology. So, one of our big customers is the 1-800-Flowers group of companies, which is Harry & David Popcorn Factory and several others. And wanted to provide a hyper-personalized concierge service for their customers where, you know, the system learns who you buy gifts for, for what occasions, you know, what your relationship is to them, and to basically remember who you are and what you want for each of their 20 million customers. And they tried different technologies out there, you know, all the top brands and so on, and they just couldn't get it off the ground. And the reason is because they really don't learn. And we now have 89% self-service on the things that we've implemented, which is pretty much unheard of for complex conversations. So, why can we do that? The reason is that our system has deep understanding. So, we have deep pausing, deep understanding, but more importantly, that the system remembers. It has short-term memory. It has long-term memory. And it uses that as context. So, you know, when you call back a second time, it'll remember what your previous call was, you know, what your preferences are, and so on. And it can basically use that information, the short and long-term memory, and reason about it. And that is really a step forward. Now, until ChatGPT, which is really very different technology from chatbot technology, I mean, chatbot technology, you're assuming...the kind of thing we're talking about is really augmenting call center, you know, automatic call center calls. There, you need deep integration into the customers' back-end system. You obviously need to know what the latest product availability is, what the customers' outstanding orders are, you know, all sorts of things like, you know, delivery schedules. And we probably have, like, two dozen APIs that connect our system to their various corporate databases and so on. Now, traditional chatbots obviously can do that. You hook up the APIs and do things, and it's, you know, it's a lot of work. But traditional chatbot technology really hasn't really changed much in 30 years. You basically have a categorizer; how can I help you? Basically, try to...what is the intent, intent categorizer? And then once your intent has been identified, you basically have a flowchart-type program that, you know, forces you down a flowchart. And that's what makes them so horrible because it doesn't use context. It doesn't have short-term memory. CHAD: And I just wanted to clarify the product and where you mentioned call center. So, this isn't just...or only text-based chat. This is voice. PETER: Yes. We started off with chat, and we now also have voice, so omnichannel. And the beauty of the system having the brain as well is you can jump from text messaging to a chat on the website to Apple ABC to voice, you know. So, you can basically move from one channel to another seamlessly. You know, so that's against traditional chatbot technology, which is really what everybody is still using. Now, ChatGPT, of course, the fact that it's called ChatGPT sort of makes it a bit confusing. And, I mean, it's phenomenal. The technology is absolutely phenomenal in terms of what it can do, you know, write poems and give you ideas. And the amount of information it's amazing. However, it's really not suited for commercial-grade applications because it hallucinates and it doesn't have memory. CHAD: You can give it some context, but it's basically faking it. You're providing it information every time you start to use it. PETER: Correct. The next time you connect, that memory is gone, you know [crosstalk 05:58] CHAD: Unless you build an application that saves it and then feeds it in again. PETER: Right. Then you basically run out of context we know very quickly. In fact, I just published a white paper about how we can get to human-level AI. And one of the things we did and go over in the paper is we did a benchmark our technology where we fed the system about 300 or 400 facts, simple facts. You know, it might be my sister likes chocolate or, you know, it could be other things like I don't park my car in the garage or [chuckles], you know. It could be just simple facts, a few hundred of those. And then we asked questions about that. Now, ChatGPT scored less than 1% on that because, you know, with an 8K window, it basically just couldn't remember any of this stuff. So, we use -- CHAD: It also doesn't, in my experience...it's basically answering the way it thinks the answer should sound or look. And so, it doesn't actually understand the facts that you give it. PETER: Exactly. CHAD: And so, if you feed it a bunch of things which are similar, it gets really confused because it doesn't actually understand the things. It might answer correctly, but it will, in my experience, just as likely answer incorrectly. PETER: Yeah. So, it's extremely powerful technology for helping search as well if a company has all the documents and they...but the human always has to be in the loop. It just makes way too many mistakes. But it's very useful if it gives you information 8 out of 10 times and saves you a lot of time. And it's relatively easy to detect the other two times where it gives you wrong information. Now, I know in programming, sometimes, it's given me wrong information and ended up taking longer to debug the misinformation it gave me than it would have taken me. But overall, it's still a very, very powerful tool. But it really isn't suitable for, you know, serious chatbot applications that are integrated into back-end system because these need to be signed off by...legal department needs to be happy that it's not going to get the company into trouble. Marketing department needs to sign off on it and customer experience, you know. And a generative system like that, you really can't rely on what it's going to say, and that's apart from security concerns and, you know, the lack of memory and deep understanding. CHAD: Yeah. So, you mentioned generative AI, which is sort of one of the underlying pieces of ChatGPT. In your solutions, are you using any generative solutions? PETER: No, not at all. Well, I can give one example. You know, what 1-800-Flowers do is they have an option to write a poem for your mother's birthday or Mother's Day or something like it. And for that, we will use ChatGPT, or they use ChatGPT for that because that's what it's good at. But, you know, that's really just any other app that you might call up to do something for you, you know, like calling up FedEx to find out where your goods are. Apart from that, our technology...it's a good question you ask because, you know, statistical systems and generative AI now have really dominated the AI scene for the last about 12 years, really sort of since DeepMind started. Because it's been incredibly successful to take masses amounts of data and masses amounts of computing power and, you know, number crunch them and then be able to categorize and identify images and, you know, do all sorts of magical things. But, the approach we use is cognitive AI as opposed to generative. It's a relatively unknown approach, but that's what we've been working on for 15 years. And it starts with the question of what does intelligence require to build a system so that it doesn't use masses amounts of data? It's not the quantity of data that counts. It's the quality of data. And it's important that it can learn incrementally as you go along like humans do and that it can validate what it learns. It can reason about, you know, new information. Does this make sense? Do I need to ask a follow-up question? You know, that kind of thing. So, it's cognitive AI. That's the approach we're using. CHAD: And, obviously, you have a product, and you've productized it. But you said, you know, we've been working on this, or you've been working on this model for a long time. How has it progressed? PETER: Yes, we are now on, depending on how you count, but on the third major version of it that we've started. And really, the progress has been determined by resources really than any technology. You know, it's not that we sort of have a big R&D requirement. It's really more development. But we are a relatively small company. And because we're using such different technology, it's actually been pretty hard to raise VC money. You know, they look at it and, you know, ask you, "What's your training data? How big is your model?" You know, and that kind of thing. CHAD: Oh, so the questions investors or people know to ask aren't relevant. PETER: Correct. And, you know, they bring in the AI experts, and then they say, "Well, what kind of deep learning, machine learning, or generative, or what transformer model are using?" And we say, "Well, we don't." And typically, that's kind of, "Oh okay, well, then it can't possibly work, you know, we don't understand it." So, we just recently launched. You know, with all the excitement of generative AI now recently, with so much money flowing into it, we actually launched a major development effort. Now we want to hire an additional a hundred people to basically crank up the IQ. So, over the years, you know, we're working on two aspects of it: one is to continually crank up the IQ of the system, that it can understand more and more complex situations; it can reason better and be able to handle bigger amounts of data. So, that's sort of the technical part that we've been working on. But then the other side, of course, running a business, a lot of our effort over the last 15 years has gone into making it industrial strength, you know, security, scalability, robustness of the system. Our current technology, our first version, was actually a SaaS model that we deployed behind a customer's firewall. CHAD: Yeah, I noticed that you're targeting more enterprise deployments. PETER: Yeah, that's at the moment because, financially, it makes more sense for us to kind of get off the ground to work with, you know, larger companies where we supply the technology, and it's deployed usually in the cloud but in their own cloud behind their firewall. So, they're very happy with that. You know, they have complete control over their data and reliability, and so on. But we provide the technology and then just licensing it. CHAD: Now, a lot of people are familiar with generative AI, you know, it runs on GPUs and that kind of thing. Does the hardware profile for where you're hosting it look the same as that, or is it different? PETER: No, no, no, it requires much less horsepower. So, I mean, we can run an agent on a five-year-old laptop, you know, and it doesn't...instead of it costing $100 million to train the model, it's like pennies [laughter] to train the model. I mean, we train it during our regression testing, and that we train it several times a day. Mid-Roll Ad: When starting a new project, we understand that you want to make the right choices in technology, features, and investment but that you don't have all year to do extended research. In just a few weeks, thoughtbot's Discovery Sprints deliver a user-centered product journey, a clickable prototype or Proof of Concept, and key market insights from focused user research. We'll help you to identify the primary user flow, decide which framework should be used to bring it to life, and set a firm estimate on future development efforts. Maximize impact and minimize risk with a validated roadmap for your new product. Get started at: tbot.io/sprint. CHAD: So, you mentioned ramping up the IQ is a goal of yours. With a cognitive model, does that mean just teaching it more things? What does it entail? PETER: Yes, there's a little bit of tension between commercial requirements and what you ultimately want for intelligence because a truly intelligent system, you want it to be very autonomous and adaptive and have a wide range of knowledge. Now, for current commercial applications we're doing, you actually don't want the system to learn things by itself or to make up stuff, you know, you want it to be predictable. So, they develop and to ultimately get to full human-level or AGI capability requires a system to be more adaptive–be able to learn things more. So, the one big change we are making to the system right now is natural language understanding or English understanding. And our current commercial version was actually developed through our—we call them AI psychologists, our linguists, and cognitive psychologists—by basically teaching it the rules of English grammar. And we've always known that that's suboptimal. So, with the current version, we are now actually teaching it English from the ground up the way a child might learn a language, so the language itself. So, it can learn any language. So, for commercial applications, that wasn't really a need. But to ultimately get to human level, it needs to be more adaptive, more autonomous, and have a wider range of knowledge than the commercial version. That's basically where our focus is. And, you know, we know what needs to be done, but, you know, it's quite a bit of work. That's why we need to hire about 100 people to deal with all of the different training things. It's largely training the system, you know, but there are also some architectural improvements we need to make on performance and the way the system reasons. CHAD: Well, you used the term Artificial General Intelligence. I understand you're one of the people who coined that term [chuckles] or the person. PETER: Yes. In 2002, I got together with two other people who felt that the time was ripe to get back to the original dream of AI, you know, from 60 years ago, to build thinking machines basically. So, we decided to write a book on the topic to put our ideas out there. And we were looking for a title for the book, and three of us—myself, Ben Goertzel, and Shane Legg, who's actually one of the founders of DeepMind; he was working for me at the time. And we were brainstorming it, and that's what we came up with was AGI, Artificial General Intelligence. CHAD: So, for people who aren't familiar, it's what you were sort of alluding to. You're basically trying to replicate the human brain, the way humans learn, right? That's the basic idea is -- PETER: Yeah, the human cognition really, yeah, human mind, human cognition. That's exactly right. I mean, we want an AI that can think, learn, and reason the way humans do, you know, that it can hit the box and learn a new topic, you know, you can have any kind of conversation. And we really believe we have the technology to do that. We've built quite a number of different prototypes that already show this kind of capability where it can, you know, read Wikipedia, integrate that with existing knowledge, and then have a conversation about it. And if it's not sure about something, it'll ask for clarification and things like that. We really just need to scale it up. And, of course, it's a huge deal for us to eventually get to human-level AI. CHAD: Yeah. How much sort of studying of the brain or cognition do you do in your work, where, you know, sort of going back and saying, "Okay, we want to tackle this thing"? Do you do research into cognition? PETER: Yeah, that's a very interesting question. It really gets to the heart of why I think we haven't made more progress in developing AGI. In fact, another white paper I published recently is "Why Don't We Have AGI Yet?" And, you know, one of the big problems is that statistical AI has been so incredibly successful over the last decade or so that it sucked all of the oxygen out of the air. But to your question, before I started on this project, I actually took off five years to study intelligence because, to me, that's really what cognitive AI approach is all about is you start off by saying, what is intelligence? What does it require? And I studied it from the perspective of philosophy, epistemology, theory of knowledge. You know, what's reality? How do we know anything? CHAD: [laughs] PETER: How can we be sure? You know, really those most fundamental questions. Then how do children learn? What do IQ tests measure? How does our intelligence differ to animal intelligence? What is that magic difference between, you know, evolution? Suddenly, we have this high-level cognition. And the short answer of that is being able to form abstract concepts or concept formation is sort of key, and to have metacognition, to be able to think about your own thinking. So, those are kind of the things I discovered during the five years of study. Obviously, I also looked at what had already been done in the field of AI, as in good old-fashioned AI, and neural networks, and so on. So, this is what brought me together. So, absolutely, as a starting point to say, what is intelligence? Or what are the aspects of intelligence that are really important and core? Now, as far as studying the brain is concerned, I certainly looked at that, but I pretty quickly decided that that wasn't that relevant. It's, you know, you certainly get some ideas. I mean, neural networks, ours is kind of a neural network or knowledge graph, so there's some similarity with that. But the analogy one often gives, which I think is not bad, is, you know, we've had flying machines for 100 years. We are still nowhere near reverse engineering a bird. CHAD: Right. PETER: So, you know, evolution and biology are just very different from designing things and using the materials that we need to use in computers. So, definitely, understanding intelligence, I think, is key to being able to build it. CHAD: Well, in some ways, that is part of the reason why statistical AI has gotten so much attention with that sort of airplane analogy because it's like, maybe we need to not try to replicate human cognition [chuckles]. Maybe we need to just embrace what computers are good at and try to find a different way. PETER: Right, right. But that argument really falls down when you say you are ignoring intelligence, you know, or you're ignoring the kind of intelligence. And we can see how ridiculous the sort of the current...well, I mean, first of all, let me say Sam Altman, and everybody says...well, they say two things: one, we have no idea how these things work, which is not a good thing if you're [chuckles] trying to build something and improve it. And the second thing they say...Demis Hassabis and, you know, everybody says it, "This is not going to get us to human-level AI, to human-level intelligence." They realize that this is the wrong approach. But they also haven't come up with what the right approach is because they are stuck within the statistical big data approach, you know, we need another 100 billion dollars to build even bigger computers with bigger models, you know, but that's really -- CHAD: Right. It might be creating a tool, which has some uses, but it is not actual; I mean, it's not really even actual artificial intelligence -- PETER: Correct. And, I mean, you can sort of see this very easily if...imagine you hired a personal assistant for yourself, a human. And, you know, they come to you, and they know how to use Excel and do QuickBooks or whatever, and a lot of things, so great. They start working with you. But, you know, every now and again, they say something that's completely wrong with full confidence, so that's a problem. Then the second thing is you tell them, "Well, we've just introduced a new product. We shut down this branch here. And, you know, I've got a new partner in the business and a new board member." And the next day, they come in, and they remember nothing of that, you know, [chuckles] that's not very intelligent. CHAD: Right. No, no, it's not. It's possible that there's a way for these two things to use each other, like generating intelligent-sounding, understanding what someone is saying and finding like things to it, and being able to generate meaningful, intelligent language might be useful in a cognitive model. PETER: We obviously thought long and hard about this, especially when, you know, generative AI became so powerful. I mean, it does some amazing things. So, can we combine the technology? And the answer is quite simply no. As I mentioned earlier, we can use generative AI kind of as an API or as a tool or something. You know, so if our system needs to write a poem or something, then yes, you know, these systems can do a good job of it. But the reason you can't really just combine them and kind of build a Frankensteinian kind of [laughs] thing is you really need to have context that you currently have fully integrated. So you can't have two brains, you know, the one brain, which is a read-only brain, and then our brain, our cognitive brain, which basically constantly adapts and uses the context of what it's heard using short-term memory, long-term memory, reasoning, and so on. So, all of those mental mechanisms of deep understanding of context, short-term and long-term memory, reasoning, language generation–they all have to be tightly integrated and work together. And that's basically the approach that we have. Now, like a human to...if you write, you know, "Generate an essay," and you want to have it come up with maybe some ideas, changing the style, for example, you know, it would make sense for our system to use a generative AI system like a tool because humans are good tool users. You know, I wouldn't expect our system to be the world chess champion or Go champion. It can use a chess-playing AI or a Go-playing AI to do that job. CHAD: That's really cool. You mentioned the short-term, long-term memory. If I am using or working on a deployment for Aigo, is that something that I specify, like, oh, this thing where we've collected goes in short term versus long term, or does the system actually do that automatically? PETER: That's the beauty of the system that: it automatically has short and long-term memory. So, really, the only thing that needs to be sort of externally specified is things you don't want to keep in long-term memory, you know, that for some reason, security reasons, or a company gives you a password or whatever. So, then, they need to be tagged. So, we have, like, an ontology that describes all of the different kinds of knowledge that you have. And in the ontology, you can tag certain branches of the ontology or certain nodes in the ontology to say, this should not be remembered, or this should be encrypted or, you know, whatever. But by default, everything that comes into short-term memory is remembered. So, you know, a computer can have photographic memory. CHAD: You know, that is part of why...someone critical of what they've heard might say, "Well, you're just replicating a human brain. How is this going to be better?" And I think that that's where you're just...what you said, like, when we do artificial general intelligence with computers, they all have photographic memory. PETER: Right. Well, in my presentations, when I give talks on this, I have the one slide that actually talks about how AI is superior to humans in as far as getting work done in cognition, and there's actually quite a number of things. So, let me first kind of give one example here. So, imagine you train up one AI to be a PhD-level cancer researcher, you know, it goes through whatever training, and reading, and coaching, and so on. So, you now have this PhD-level cancer researcher. You now make a million copies of that, and you have a million PhD-level cancer researchers chipping away at the problem. Now, I'm sure we would make a lot more progress, and you can now replicate that idea, that same thinking, you know, in energy, pollution, poverty, whatever, I mean, any disease, that kind of approach. So, I mean, that already is one major difference that you make copies of an AI, which you can't of humans. But there are other things. First of all, they are significantly less expensive than humans. Humans are very expensive. So much lower cost. They work 24/7 without breaks, without getting tired. I don't know the best human on how many hours they can concentrate without needing a break, maybe a few hours a day, or six, maybe four hours a day. So, 24/7. Then, they can communicate with each other much better than humans do because they could share information sort of by transferring blocks of data across from one to the other without the ego getting in the way. I mean, you take humans, not very good at sharing information and discoveries. Then they don't have certain distractions that we have like romantic things and kids in schools and, you know. CHAD: Although if you actually do get a full [laughs] AGI, then it might start to have those things [laughs]. PETER: Well, yeah, that's a whole nother topic. But our AIs, we basically build them not to want to have children [laughs] so, you know. And then, of course, things we spoke about, photographic memory. It has instantaneous access to all the information in the world, all the databases, you know, much better than we have, like, if we had a direct connection to the internet and brain, you know, but at a much higher bandwidth than we could ever achieve with our wetware. And then, lastly, they are much better at reasoning than humans are. I mean, our ability to reason is what I call an evolutionary afterthought. We are not actually that good at logical thinking, and AIs can be, you know. CHAD: We like to think we are, though. PETER: [chuckles] Well, you know, compared to animals, yes, definitely. We are significantly better. But realistically, humans are not that good at rational, logical thinking. CHAD: You know, I read something that a lot of decisions are made at a different level than the logical part. And then, the logical part justifies the decision. PETER: Yeah, absolutely. And, in fact, this is why smart people are actually worse at that because they're really good at rationalizations. You know, they can rationalize their weird beliefs and/or their weird behavior or something. That's true. CHAD: You mentioned that your primary customers are enterprises. Who makes up your ideal customer? And if someone was listening who matched that profile and wanted to get in touch with you, what would they look like? PETER: The simplest and most obvious way is if they have call centers of 100 people or more—hundreds, or thousands, tens of thousands even. But the economics from about 100 people in the call center, where we might be able to save them 50% of that, you know, depending on the kind of business. CHAD: And are your solutions typically employed before the actual people, and then they fall back to people in certain circumstances? PETER: Correct. That's exactly right. And, you know, the advantage there is, whatever Aigo already gathers, we then summarize it and pop that to the human operator so that, you know, that the customer -- CHAD: That's great because that's super annoying. PETER: It is. CHAD: [laughs] PETER: It is super annoying and -- CHAD: When you finally get to a person, and it's like, I just spent five minutes providing all this information that you apparently don't have. PETER: Right. Yeah, no, absolutely, that's kind of one of the key things that the AI has that information. It can summarize it and provide it to the live operator. So that would be, you know, the sort of the most obvious use case. But we also have use cases on the go with student assistant, for example, where it's sort of more on an individual basis. You know, imagine your kid just starts at university. It's just overwhelming. It can have a personal personal assistant, you know, that knows all about you in particular. But then also knows about the university, knows its way around, where you get your books, your meals, and, you know, different societies and curriculum and so on. Or diabetes coach, you know, where it can help people with diabetes manage their meals and activities, where it can learn whether you love broccoli, or you're vegetarian, or whatever, and help guide you through that. Internal help desks are another application, of course. CHAD: Yeah. I was going to say even the same thing as at a university when people join a big company, you know, there's an onboarding process. PETER: Exactly. Yeah. CHAD: And there could be things that you're not aware of or don't know where to find. PETER: Internal HR and IT, absolutely, as you say, on onboarding. Those are other applications where our technology is well-suited. And one other category is what we call a co-pilot. So, think of it as Clippy on steroids, you know, where basically you have complex software like, you know, SAP, or Salesforce, or something like that. And you can basically just have Aigo as a front end to it, and you can just talk to it. And it will know where to navigate, what to get, and basically do things, complex things in the software. And software vendors like that idea because people utilize more features of the software than they would otherwise, you know. It can accelerate your learning curve and make it much easier to use the product. So, you know, really, the technology that we have is industry and application-agnostic to a large extent. We're just currently not yet at human level. CHAD: Right. I hope you get there eventually. It'll be certainly exciting when you do. PETER: Yes. Well, we do expect to get there. We just, you know, as I said, we've just launched a project now to raise the additional money we need to hire the people that we need. And we actually believe we are only a few years away from full human-level intelligence or AGI. CHAD: Wow, that's exciting. So, if the solution that you currently have and people want to go along for the journey with you, how can they get in touch with Aigo? PETER: They could contact me directly: peter@aigo.ai. I'm also active on Twitter, LinkedIn. CHAD: Cool. We'll include all of those links in the show notes, which people can find at giantrobots.fm. If you have questions for me, email me at hosts@giantrobots.fm. Find me on Mastodon @cpytel@thoughtbot.social. You can find a complete transcript for this episode as well at giantrobots.fm. Peter, thank you so much for joining me. I really appreciate it and all of the wisdom that you've shared with us today. PETER: Well, thank you. They were good questions. Thank you. CHAD: This podcast is brought to you by thoughtbot and produced and edited by Mandy Moore. Thanks for listening, and see you next time. ANNOUNCER: This podcast is brought to you by thoughtbot, your expert strategy, design, development, and product management partner. We bring digital products from idea to success and teach you how because we care. Learn more at thoughtbot.com. Special Guest: Peter Voss.

Martin Riedmiller of Google DeepMind on controlling nuclear fusion plasma in a tokamak with RL, the original Deep Q-Network, Neural Fitted Q-Iteration, Collect and Infer, AGI for control systems, and tons more!  Martin Riedmiller is a research scientist and team lead at DeepMind.   Featured References   Magnetic control of tokamak plasmas through deep reinforcement learning  Jonas Degrave, Federico Felici, Jonas Buchli, Michael Neunert, Brendan Tracey, Francesco Carpanese, Timo Ewalds, Roland Hafner, Abbas Abdolmaleki, Diego de las Casas, Craig Donner, Leslie Fritz, Cristian Galperti, Andrea Huber, James Keeling, Maria Tsimpoukelli, Jackie Kay, Antoine Merle, Jean-Marc Moret, Seb Noury, Federico Pesamosca, David Pfau, Olivier Sauter, Cristian Sommariva, Stefano Coda, Basil Duval, Ambrogio Fasoli, Pushmeet Kohli, Koray Kavukcuoglu, Demis Hassabis & Martin Riedmiller Human-level control through deep reinforcement learning Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A Rusu, Joel Veness, Marc G Bellemare, Alex Graves, Martin Riedmiller, Andreas K Fidjeland, Georg Ostrovski, Stig Petersen, Charles Beattie, Amir Sadik, Ioannis Antonoglou, Helen King, Dharshan Kumaran, Daan Wierstra, Shane Legg, Demis Hassabis  Neural fitted Q iteration–first experiences with a data efficient neural reinforcement learning method Martin Riedmiller  

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: [AISN #5]: Geoffrey Hinton speaks out on AI risk, the White House meets with AI labs, and Trojan attacks on language models, published by Center for AI Safety on May 9, 2023 on The Effective Altruism Forum. Welcome to the AI Safety Newsletter by the Center for AI Safety. We discuss developments in AI and AI safety. No technical background required. Subscribe here to receive future versions. Geoffrey Hinton is concerned about existential risks from AI Geoffrey Hinton won the Turing Award for his work on AI. Now he says that part of him regrets his life's work, as he believes that AI poses an existential threat to humanity. As Hinton puts it, “it's quite conceivable that humanity is just a passing phase in the evolution of intelligence.” AI is developing more rapidly than Hinton expected. In 2015, Andrew Ng argued that worrying about AI risk is like worrying about overpopulation on Mars. Geoffrey Hinton also used to believe that advanced AI was decades away, but recent progress has changed his views. Now he says that AI will become “smarter than a human” in “5 to 20 years, but without much confidence. We live in very uncertain times.” The AI race is heating up, but Hinton sees a way out. In an interview with MIT Technology Review, Hinton argues that building AI is “inevitable” given competition between companies and countries. But he argues that “we're all in the same boat with respect to existential risk,” so potentially “we could get the US and China to agree like we could with nuclear weapons.” Similar to climate change, AI risk will require coordination to solve. Hinton compared the two risks by saying, "I wouldn't like to devalue climate change. I wouldn't like to say, 'You shouldn't worry about climate change.' That's a huge risk too. But I think this might end up being more urgent." When AIs create their own subgoals, they will seek power. Hinton argues that AI agents like AutoGPT and BabyAGI demonstrate that people will build AIs that choose their own goals and pursue them. Hinton and others have argued that this is dangerous because “getting more control is a very good subgoal because it helps you achieve other goals.” Other experts are speaking up on AI risk. Demis Hassabis, CEO of DeepMind, recently said that he believes some form of AGI is “a few years, maybe within a decade away” and recommended “developing these types of AGI technologies in a cautious manner.” Shane Legg, co-founder of DeepMind, thinks AGI is likely to arrive around 2026. Warren Buffet compared AI to the nuclear bomb, and many others are concerned about advanced AI. White House meets with AI labs Vice President Kamala Harris met at the White House on Thursday with leaders of Microsoft, Google, Anthropic, and OpenAI to discuss risks from artificial intelligence. This is an important step towards AI governance, though it's a bit like inviting oil companies to a discussion on climate change—they have the power to solve the problem, but incentives to ignore it. New executive action on AI. After the meeting, the White House outlined three steps they plan to take to continue responding to the challenges posed by AI: To evaluate the risks of generative AI models, the White House will facilitate a public red-teaming competition. The event will take place at the DEF CON 31 conference and will feature cutting-edge models provided by leading AI labs. The White House continues to support investments in AI research, such as committing $140M over 5 years to National AI Research Institutes. Unfortunately, it's plausible that most of this investment will be used to accelerate AI development without being directed at making these systems more safe. The Office of Management and Budget will release guidelines for federal use of AI. Federal agencies promise enforcement action on AI. Four federal agencies iss...

In questa puntata Roberto e Filippo assieme a Lucio Bragagnolo parlano di Intelligenza Artificiale o quello che viene passato per essere e del futuro dell'assistente vocale di Apple. Note dell'episodio L'elefante nella stanza: Chat GPT Che cos'è? Un motore semantico Una versione agli steroidi del correttore automatico Che riesce a tener traccia delle sue precedenti interazioni È poliglotta: Può parlare e capire più lingue Come funziona sotto la scocca? Sistema statistico Calcola la probabilità che una parola abbia senso vicino ad un'altra Per avere un modello statistico Versione avanzata del correttore automatico di iOS Pensiero umano Conosce attraverso i 5 sensi Informazioni filtrate attraverso Attenzione: Riceviamo ogni secondo un'infinità di informazioni Solo una minima parte arriva alla mente cosciente Quella razionale Percorso di rafforzamento Più un certo collegamento sinaptico viene usato Più la risposta è veloce Più è fissato nella memoria Pensiero della macchina (attualmente) “Conosce” in base ad un mare di informazioni Informazioni che non vengono però filtrate O se vengono Controllo minimo Perché costa tempo e denaro Link utili / interessanti Ehi siri fammi parlare con ChatGPT (https://apple.quora.com/Ehi-Siri-fammi-parlare-con-ChatGPT) Versione originale in inglese (https://github.com/Yue-Yang/ChatGPT-Siri) Letture Riassumere video con ChatGPT (https://www.youtube.com/watch?v=RYZo9pi1Yzo) Stephen Wolfram Risponde Alle Domande In Diretta Su ChatGPT (https://youtu.be/zLnhg9kir3Q) Come viene addestrato ChatGPT (https://youtu.be/VPRSBzXzavo) Ma come Funziona Effettivamente ChatGPT? (https://youtu.be/aQguO9IeQWE) Il problema più grande con l'IA! (https://youtu.be/7emz4zZ226E) Ho provato a usare l'intelligenza artificiale. Mi ha spaventato. (https://youtu.be/jPhJbKBuNnA) Schemi ed analogie (https://macintelligence.org/posts/2023-03-23-schemi-e-analogie/) Machine learning ricerche ad Apple (https://machinelearning.apple.com/) ML di Apple Live Text CoreML Live Captions o sottotitoli live Idee di Bill Gates (https://www.gatesnotes.com/The-Age-of-AI-Has-Begun) Espunti interessanti estratti dall'articolo e tradotti (con il traduttore di Apple): Alla fine il tuo modo principale di controllare un computer non sarà più puntare e fare clic o toccare menu e finestre di dialogo. Invece, sarai in grado di scrivere una richiesta in inglese semplice. (E non solo l'inglese: le AI capiranno le lingue di tutto il mondo. In India all'inizio di quest'anno, ho incontrato sviluppatori che stanno lavorando su IA che capiranno molte delle lingue parlate lì.) Inoltre, i progressi nell'IA consentiranno la creazione di un agente personale. Pensalo come un assistente personale digitale: vedrà le tue ultime e-mail, saprà delle riunioni a cui partecipi, leggerà ciò che leggi e leggerà le cose di cui non vuoi preoccuparti. Questo migliorerà il tuo lavoro sui compiti che vuoi fare e ti libererà da quelli che non vuoi fare. Problemi con IA Quando chiedi a un'IA di inventare qualcosa di fittizio, può farlo bene. Ma quando chiedi consigli su un viaggio che vuoi fare, potrebbe suggerire hotel che non esistono. Questo perché l'IA non capisce abbastanza bene il contesto della tua richiesta da sapere se dovrebbe inventare hotel falsi o parlarti solo di quelli reali che hanno camere disponibili. Le IA superintelligenti sono nel nostro futuro. Rispetto a un computer, il nostro cervello opera a ritmo di lumaca: un segnale elettrico nel cervello si muove a 1/100.000 la velocità del segnale in un chip di silicio! Una volta che gli sviluppatori possono generalizzare un algoritmo di apprendimento ed eseguirlo alla velocità di un computer - un risultato che potrebbe essere a un decennio o un secolo di distanza - avremo un AGI incredibilmente potente. Sarà in grado di fare tutto ciò che un cervello umano può, ma senza alcun limite pratico alla dimensione della sua memoria o alla velocità con cui opera. Questo sarà un cambiamento profondo. L'intelligenza artificiale non controlla ancora il mondo fisico e non può stabilire i propri obiettivi [NdR FS non del tutto vero] Libri consigli da gates: - Superintelligence, by Nick Bostrom; - Life 3.0 by Max Tegmark; and - A Thousand Brains, by Jeff Hawkins (https://www.gatesnotes.com/A-Thousand-Brains). Modi di essere. Animali, piante e computer: al di là dell'intelligenza umana (https://amzn.to/3nzdfMB) di James Bridle Alcuni spunti interessanti estratti dal libro: Voglio suggerire tre principi che dovrebbero guidare quella conversazione. In primo luogo, dovremmo cercare di bilanciare le paure sugli aspetti negativi dell'IA, che sono comprensibili e validi, con la sua capacità di migliorare la vita delle persone. Per sfruttare al meglio questa straordinaria nuova tecnologia, dovremo proteggerci dai rischi e diffondere i benefici al maggior numero possibile di persone. In secondo luogo, le forze di mercato non produrranno naturalmente prodotti e servizi di intelligenza artificiale che aiutano i più poveri. È più probabile il contrario. Con finanziamenti affidabili e le giuste politiche, i governi e la filantropia possono garantire che le IA vengano utilizzate per ridurre l'iniquità. Proprio come il mondo ha bisogno delle sue persone più brillanti focalizzate sui suoi più grandi problemi, dovremo concentrare le migliori IA del mondo sui suoi più grandi problemi. Anche se non dovremmo aspettare che questo accada, è interessante pensare se l'intelligenza artificiale identificherebbe mai l'iniquità e cercherebbe di ridurla. Hai bisogno di avere un senso della moralità per vedere l'iniquità, o lo vedrebbe anche un'IA puramente razionale? Se riconoscesse l'iniquità, cosa suggerirebbe di fare al riguardo? Infine, dovremmo tenere a mente che siamo solo all'inizio di ciò che l'IA può realizzare. Qualunque limite abbia oggi sarà sparito prima che ce ne accorgiamo. IA competitiva L'attuale forma dominante di intelligenza artificiale, quella di cui tutti parlano, non è né creativa né collaborativa né fantasiosa. O è totalmente asservita - stupida, francamente - o è oppositiva, aggressiva e pericolosa (e forse sempre stupida). E analisi dei modelli, descrizione di immagini, riconoscimento facciale e gestione del traffico; è prospezione petrolifera, arbitraggio finanziario, sistemi di armi autonome e programmi scacchistici che distruggono completamente l'avversario umano. Compiti competitivi, profitti competitivi, intelligenza competitiva. In tutto questo, l'IA competitiva ha una caratteristica in comune con il mondo naturale, o meglio con l'errata concezione storica che abbiamo di quest'ultimo. Essa immagina un ambiente sanguinario in cui l'umanità nuda e fragile deve combattere con forze devastanti e soggiogarle, piegandole alla sua volontà (di solito, maschile) sotto forma di agricoltura, architettura, allevamento e addomesticamento. Questo modo di vedere il mondo ha prodotto un sistema di classificazione a tre livelli in base ai tipi di animali in cui ci imbattiamo: animali domestici, bestiame e fiere selvatiche, ciascuno con i suoi attributi e atteggiamenti. Trasferendo questa analogia al mondo dell'IA, sembra evidente che finora abbiamo creato perlopiù macchine addomesticate del primo tipo, iniziando a recintare un allevamento del secondo e vivendo nel timore di scatenare il terzo. Ecologia della tecnologia Dobbiamo imparare a convivere con il mondo, anziché cercare di dominarlo. In breve, dobbiamo scoprire un'ecologia della tecnologia. Il termine «ecologia» fu coniato alla metà del XIX secolo dal naturalista tedesco [[Ernst Haeckel]] nel libro Generelle Morphologie der Organismen («Morfologia generale degli organismi»). «Per ecologia,» scrive «intendiamo la totalità delle scienze delle relazioni dell'organismo con l'ambiente, incluse tutte le condizioni dell'esistenza nella loro accezione più ampia.» Il termine deriva dal greco oikos (oikos), che significa casa o ambiente; in una nota, Haeckel fa riferimento anche al greco xwpa (chora), cioè «luogo di residenza». L'ecologia non è semplicemente lo studio del posto in cui ci troviamo, ma di tutto ciò che ci circonda e che ci permette di vivere. John Muir, amante della vita all'aria aperta e padre del sistema dei parchi nazionali negli Stati Uniti. Riflettendo sull'abbondanza di vita complessa in cui si imbatté mentre scriveva il libro La mia prima estate sulla Sierra, afferma semplicemente: «Se cerchiamo di isolare un oggetto qualsiasi, scopriamo che ogni cosa è ancorata a tutto il resto dell'universo». La tecnologia è l'ultimo campo del sapere a scoprire la propria ecologia. Quest'ultima è lo studio del luogo in cui ci troviamo e delle relazioni tra i suoi abitanti, mentre la tecnologia è lo studio di ciò che facciamo in quel posto: Téxv (techne), ossia arte o mestiere. Se la mettiamo così, le due sembrano alleate per natura, ma la storia della tecnologia è perlopiù un racconto di cecità intenzionale nei confronti del contesto e delle conseguenze della sua attuazione. AI user friendly Molti di coloro che si occupano direttamente di IA presso Facebook, Google e altre aziende della Silicon Valley sono più che consapevoli delle potenziali minacce esistenziali della superintelligenza. Come abbiamo visto, alcuni dei protagonisti del settore tecnologico - da Bill Gates ed Elon Musk a Shane Legg, il fondatore della DeepMind di Google - hanno espresso preoccupazione per la sua comparsa. Ma la loro risposta è di tipo tecnologico: dobbiamo progettare l'IA in modo che sia friendly, incorporando nella sua programmazione le tutele e le procedure necessarie per garantire che non diventi mai una minaccia per la vita e per il benessere dell'uomo. Questo approccio sembra insieme ottimistico ai limiti dell'assurdo e ingenuo in misura preoccupante. È anche in contrasto con l'esperienza che abbiamo accumulato finora con i sistemi intelligenti. Nella storia dell'IA, i modelli di intelligenza che cercano di descrivere una mente completa attraverso un insieme di regole prestabilite non sono mai riusciti a raggiungere i loro obiettivi. L'azione giusta, in altre parole, non dipende dalla preesistenza della conoscenza giusta - una mappa delle strade o una gerarchia delle virtù - ma dal contesto, dalla sollecitudine e dalla cura. Una macchina preprogrammata per essere friendly non ha meno probabilità di investirvi, o di trasformarvi in graffette, di un'altra predisposta al commercio, se i suoi calcoli la considerano l'azione più etica in quelle circostanze. AI News Roundup: Alpaca, BritGPT, AI di Stanford in Gdocs & Sheets - le IA SaaS sono obsolete? (https://www.reddit.com/r/EntrepreneurRideAlong/comments/11yfl3i/ai_news_roundup_stanfords_alpaca_britgpt_ai_in/) Estratto della sintesi tradotto in italiano (traduzione servizio di Apple) L'alpaca di Stanford I ricercatori di Stanford hanno svelato un modello di intelligenza artificiale (AI) che si comporta quasi alla pari con ChatGPT, ma è costato loro solo 600 dollari per allenarsi. Alpaca è una variante di sette miliardi di parametri dell'LLaMA di Meta. È stato messo a punto utilizzando 52.000 istruzioni generate da GPT-3.5 (ChatGPT). (Proprio come il modo in cui i tester umani sono stati utilizzati per mettere a punto ChatGPT, Stanford ha usato il modello dietro ChatGPT per addestrare la loro Alpaca AI.) Nei test, Alpaca ha funzionato in modo paragonabile al modello di OpenAI, ma ha prodotto più allucinazioni. L'alpaca è significativo perché ha dimostrato che costruire e addestrare nuovi modelli di intelligenza artificiale può essere follemente economico. Questo potrebbe potenzialmente consentire a più persone, compresi i cattivi attori, di creare nuovi modelli economici. Mostra anche che una volta reso pubblico il tuo modello, anche senza rivelare il suo codice, può essere usato per costruire modelli migliori dai concorrenti (come usare ChatGPT per istruire Alpaca durante l'allenamento). Questo potrebbe rendere aziende come OpenAI, Google e Microsoft ancora più aggressive nel proteggere la loro tecnologia proprietaria? Sul lato positivo, il futuro in cui sarai in grado di allenare la tua IA ChatGPT-like usando il tuo computer si è appena avvicinato. AI in Google Workspace e Microsoft 365 Entrambe le società hanno indicato che l'IA sarà fortemente incorporata nelle loro app. Google ha mostrato immagini della loro intelligenza artificiale utilizzate in Gdocs per scrivere articoli completi, in fogli per scrivere formule e in diapositive per generare presentazioni complete con testo e immagini generate dall'IA. La demo di Microsoft era migliore. Hanno mostrato i loro strumenti di intelligenza artificiale dal vivo in azione. La loro Copilot AI sarà disponibile in app come Word, Presentation, Excel, ecc. e sarà in grado di aggregare i dati su qualsiasi argomento in queste app per rispondere a domande, pianificare riunioni, generare risposte, ecc. Copilot sarà anche in grado di prendere appunti dal vivo nelle riunioni e ricapitolare la discussione fatta finora. Anche se questo significa un enorme aumento della nostra produttività, significa anche la morte di molte aziende SaaS costruite attorno alla fornitura di funzionalità basate su GPT in queste app. Anche strumenti come Jasper potrebbero essere resi obsoleti. Rilascio limitato di Bard Google ha iniziato a lanciare il suo chatbot AI Bard, ma è disponibile solo per alcuni utenti negli Stati Uniti e nel Regno Unito e devono avere più di 18 anni. La risposta iniziale a Bard è stata tiepida con gli utenti che si lamentano che è molto inferiore a Bing Chat. Dimostra meno creatività ed è incline a più errori matematici. In effetti, un pulsante per "Google It" appare dopo ogni risposta del bot, come un indicatore della propria insicurezza. È apparentemente più veloce di Bing, ma questo è probabilmente dovuto a un modello più piccolo. Un modello più piccolo spiegherebbe anche le sue scarse prestazioni. OpenAI svela GPT-4 GPT-4 è multimodale, il che significa che accetta sia input di testo che di immagini. È meglio e più sicuro di ChatGPT. Alcune delle sue abilità: - GPT-4 può comprendere i mockup disegnati a mano e convertirli in codice del sito web funzionante. - Può analizzare documenti complessi come i codici fiscali, ma anche eseguire una matematica accurata oltre a citare leggi e principi appropriati per calcolare le tasse. - GPT-4 supera anche GPT 3.5 negli esami umani come Bar, SAT e GRE (punteggi nel 90° percentile rispetto al 10° percentile di ChatGPT) e ha una memoria contestuale molto più lunga. - È disponibile solo nell'abbonamento a pagamento ChatGPT Plus e tramite una lista d'attesa API. Bing chat ha anche utilizzato una prima versione di GPT-4 nelle ultime 5 settimane. Sono curioso dei risultati che potremmo ottenere se Alpaca fosse addestrato usando un modello LLama più grande e messo a punto con GPT-4. Inizia la corsa globale all'IA Il governo del Regno Unito sta investendo 900 milioni di sterline nel supercomputer come parte di una strategia di intelligenza artificiale che include la garanzia che il paese possa costruire il proprio "BritGPT". L'obiettivo è contrastare l'influenza dell'IA della Cina e garantire che il Regno Unito rimanga competitivo nel campo dell'IA. PaLM e Makersuite di Google Google Cloud ha annunciato che le sue applicazioni basate sull'intelligenza artificiale (AI), come l'API Pathways Language Model (PaLM) per i modelli linguistici e lo strumento di prototipazione Makersuite, sono ora disponibili per gli sviluppatori. Google Makersuite è un ambiente di prototipazione per testare e migliorare le idee per le applicazioni di intelligenza artificiale generativa. È un'app che aumenta l'API PaLM con modi per progettare prompt, produrre dati sintetici e personalizzare la messa a punto di un modello. Con MakerSuite, gli sviluppatori possono iterare sui prompt, aumentare il loro set di dati con dati sintetici e sintonizzare facilmente modelli personalizzati. Ernie di Baidu Ernie è un modello di deep-learning di elaborazione del linguaggio naturale (NLP) sviluppato da Baidu, una multinazionale tecnologica cinese. Il modello contiene parametri 10B e ha raggiunto un nuovo punteggio all'avanguardia sul benchmark SuperGLUE, superando il punteggio di base umano. È probabile che Ernie goda di un significativo vantaggio di mercato sul suo territorio d'origine rispetto ai prodotti fabbricati negli Stati Uniti, a causa sia della spinta della Cina per l'autosufficienza tecnologica che della rigorosa censura di Internet del paese. Tuttavia, la sua demo è stata una presentazione poco brillante con risposte pre-registrate. L'incursione di Apple nei LLM Si dice che Apple stia sviluppando un'IA per rivaleggiare con ChatGPT. Il rapporto del New York Times afferma che Apple ha recentemente condotto un evento interno incentrato sui contenuti di intelligenza artificiale generativa e sui grandi modelli linguistici (LLM), che sono le reti neurali che alimentano i chatbot come ChatGPT. Apple ha un'enorme riserva di cassa e con il loro track record AlexaTM di Amazon Il modello AlexaTM 20B di Amazon è stato recentemente nelle notizie. È un modello di linguaggio sequence-to-sequence (seq2seq) da 20 miliardi di parametri che presenta prestazioni all'avanguardia. Il modello è ora disponibile per uso non commerciale per aiutare lo sviluppo e la valutazione di modelli linguistici di grandi dimensioni multilingue (LLM). Il modello è disponibile anche in Amazon SageMaker JumpStart, l'hub di apprendimento automatico di SageMaker. E ha mostrato prestazioni competitive su compiti e benchmark NLP comuni (SuperGLUE e XNLI). Midjourney rilascia la V5 Midjourney v5 porta con sé "efficienza, coerenza e qualità" migliorate, ha detto Midjourney sul suo sito web. La V5 ora risponde con una "range stilistica molto più ampia" rispetto alla versione 4, pur essendo anche più sensibile ai suggerimenti, generando meno testo indesiderato e offrendo un aumento di 2 volte della risoluzione dell'immagine. Midjourney v5 può generare abbastanza bene mani umane realistiche, il che era un problema con le versioni precedenti. Ha anche generato ritratti credibili di esseri umani e persone in pose naturali. Allontanerà opportunità ai piccoli modelli e ai grafici? Bing AI Image Creator di Microsoft Microsoft ha dato al suo generatore di immagini AI il proprio sito Bing Create dedicato. Bing Image Creator è alimentato da una versione avanzata del modello DALL-E di OpenAI e funziona sorprendentemente bene anche con input di linguaggio naturale. Le immagini sono libere di creare e più sei descrittivo, migliore è l'output che ottieni. Sono particolarmente entusiasta dell'uso della generazione di immagini AI nella narrazione. C'è il potenziale per nuove startup di genere in questo spazio che utilizzano storie generate dall'IA e le combinano con immagini di intelligenza artificiale e voci fuori campo di intelligenza artificiale per esperienze di narrazione realistiche e personalizzate. Dove ci potete trovare? Lucio: Lucio (https://macintelligence.org/) Roberto: Mac e architettura: mach - dot - net.wordpress.com (https://marchdotnet.wordpress.com/) Podcast settimanale Snap - architettura imperfetta (https://www.spreaker.com/show/snap-archiettura-imperfetta) Filippo: Avvocati e Mac punto it (https://www.avvocati-e-mac.it/) Ci sentiamo tra 2 settimane.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: A newcomer's guide to the technical AI safety field, published by Chin Ze Shen on November 4, 2022 on The AI Alignment Forum. This post was written during Refine. Thanks to Jonathan Low, Linda Linsefors, Koen Holtman, Aaron Scher, and Nicholas Kees Dupuis for helpful discussion and feedback. Disclaimer: This post reflects my current understanding of the field and may not be an accurate representation of it. Feel free to comment if you feel that there are misrepresentations. Motivations I remember being fairly confused when I first started reading AI safety related posts, especially when it pertains to specific ideas or proposals, as there may be implicit assumptions behind those posts that relies on some background understanding about the research agenda. I have since had the opportunity to clear up many of those confusions by talking to many people especially while I was participating in Refine. Looking back, there were many background assumptions about the field I wish I had known earlier, so here's the post I never had. This post does not intend to cover topics like why AI safety is important, how to get into the field, or an overview of research agendas, as there are plenty of materials covering these topics already. Some terminology Artificial intelligence (AI) refers to intelligences that are created artificially, where intelligence measures an agent's ability to achieve goals in a wide range of environments as per Shane Legg and Marcus Hutter's definition. Artificial general intelligence (AGI) refers to a machine capable of behaving intelligently over many domains, unlike narrow AIs which only perform one task. Artificial superintelligence (ASI), or “superintelligent AI”, refers to an intellect that is much smarter than the best human brains in practically every field, including scientific creativity, general wisdom and social skills. On the other hand, transformative AI (TAI) is an AI that precipitates a transition at least comparable to the agricultural or industrial revolution, where the concept refers to its effects instead of its capabilities. In practice however, the terms AGI, ASI, and TAI are sometimes used interchangeably to loosely mean “an AI that is much more powerful than humans”, where “power” (related to “optimization”) is often used to loosely mean “intelligence”. AI safety is about making the development of AI go safely. It is often used to refer to AGI safety or AI alignment (or just “alignment” because “AI alignment” is too long), which roughly refers to aligning a hypothetical future AI to what humans want in a way that is not catastrophic to humans in the long term. There is of course the question of “which humans should we align the AI to” that is often raised, though the question of “how do we even properly align an intelligent system to anything at all” would be much a more central problem to most researchers in the field. In other communities however, AI safety is also used in the context of ensuring the safety of autonomous control systems such as self-driving cars and unmanned aircrafts, which is typically outside of the scope of AI alignment. In this post however, I will mostly use the term AI safety to mean AI alignment as how it is often used in introductory materials like this one, although it may be generally better to have clearer distinctions of these terms. In Steve Byrnes' diagram from this post, the red box serves as a good representation of the AI alignment field, and separates “alignment” from “existential risk (x-risk) mitigation” in a nice way: The diagram below from a talk by Paul Christiano also describes alignment (“make AI aligned”) as a specific subset of “making AI go well”. Brief history of AI and AI safety AI development can roughly be divided into the following era: 1952 - 1956: The birth of AI by a handful of ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: $20K in Bounties for AI Safety Public Materials, published by ThomasWoodside on August 5, 2022 on The Effective Altruism Forum. TLDR We are announcing a $20k bounty for publicly-understandable explainers of AI safety concepts. We are also releasing the results of the AI Safety Arguments competition. Background Of the technologists, ML researchers, and policymakers thinking about AI, very few are seriously thinking about AI existential safety. This results in less high-quality research and could also pose difficulties for deployment of safety solutions in the future. There is no single solution to this problem. However, an increase in the number of publicly accessible discussions of AI risk can help to shift the Overton window towards more serious consideration of AI safety. Capability advancements have surprised many in the broader ML community: as they have made discussion of AGI more possible, they can also contribute to making discussion of existential safety more possible. Still, there are not many good introductory resources to the topic or various subtopics. If somebody has no background, they might need to read books or very long sequences of posts to get an idea about why people are worried about AI x-risk. There are a few strong, short, introductions to AI x-risk, but some of them are out of date and they aren't suited for all audiences. Shane Legg, a co-founder of DeepMind, recently said the following about AGI: If you go back 10-12 years ago the whole notion of Artificial General Intelligence was lunatic fringe. People [in the field] would literally just roll their eyes and just walk away. [I had that happen] multiple times. [...] [But] every year [the number of people who roll their eyes] becomes less. We hope that the number of people rolling their eyes at AI safety can be reduced, too. In the case of AGI, increased AI capabilities and public relations efforts by major AI labs have fed more discussion. Similarly, conscious efforts to increase public understanding and knowledge of safety could have a similar effect. Bounty details The Center for AI Safety is announcing a $20,000 bounty for the best publicly-understandable explainers of topics in AI safety. Winners of the bounty will win $2,000 each, for a total of up to ten possible bounty recipients. The bounty is subject to the Terms and Conditions below. By publicly understandable, we mean understandable to somebody who has never read a book or technical paper on AI safety and who has never read LessWrong or the EA Forum. Work may or may not assume technical knowledge of deep learning and related math, but should make minimal assumptions beyond that. By explainer, we mean that it digests existing research and ideas into a coherent and comprehensible piece of writing. This means that the work should draw from multiple sources. This is not a bounty for original research, and is intended for work that covers more ground at a higher level than the distillation contest. Below are some examples of public materials that we value. This should not be taken as an exhaustive list of all existing valuable public contributions. AI risk executive summary (2014) Concrete Problems in AI Safety (2016) Robert Miles' YouTube channel (2017-present) AGI Safety From First Principles (2020) The case for taking AI risk seriously as a threat to humanity (2020) Unsolved Problems in ML Safety (2021) X-risk Analysis for AI Research (2022) Note that many of the works above are quite different and do not always agree with each other. Listing them isn't to say that we agree with everything in them, and we don't expect to necessarily agree with all claims in the pieces we award bounties to. However, we will not award bounties to work we believe is false or misleading. Here are some categories of work we believe could be valuable: Exe...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: $20K In Bounties for AI Safety Public Materials, published by Dan Hendrycks on August 5, 2022 on The AI Alignment Forum. TLDR We are announcing a $20k bounty for publicly-understandable explainers of AI safety concepts. We are also releasing the results of the AI Safety Arguments competition. Background Of the technologists, ML researchers, and policymakers thinking about AI, very few are seriously thinking about AI existential safety. This results in less high-quality research and could also pose difficulties for deployment of safety solutions in the future. There is no single solution to this problem. However, an increase in the number of publicly accessible discussions of AI risk can help to shift the Overton window towards more serious consideration of AI safety. Capability advancements have surprised many in the broader ML community: as they have made discussion of AGI more possible, they can also contribute to making discussion of existential safety more possible. Still, there are not many good introductory resources to the topic or various subtopics. If somebody has no background, they might need to read books or very long sequences of posts to get an idea about why people are worried about AI x-risk. There are a few strong, short, introductions to AI x-risk, but some of them are out of date and they aren't suited for all audiences. Shane Legg, a co-founder of DeepMind, recently said the following about AGI: If you go back 10-12 years ago the whole notion of Artificial General Intelligence was lunatic fringe. People [in the field] would literally just roll their eyes and just walk away. [I had that happen] multiple times. [...] [But] every year [the number of people who roll their eyes] becomes less. We hope that the number of people rolling their eyes at AI safety can be reduced, too. In the case of AGI, increased AI capabilities and public relations efforts by major AI labs have fed more discussion. Similarly, conscious efforts to increase public understanding and knowledge of safety could have a similar effect. Bounty details The Center for AI Safety is announcing a $20,000 bounty for the best publicly-understandable explainers of topics in AI safety. Winners of the bounty will win $2,000 each, for a total of up to ten possible bounty recipients. The bounty is subject to the Terms and Conditions below. By publicly understandable, we mean understandable to somebody who has never read a book or technical paper on AI safety and who has never read LessWrong or the EA Forum. Work may or may not assume technical knowledge of deep learning and related math, but should make minimal assumptions beyond that. By explainer, we mean that it digests existing research and ideas into a coherent and comprehensible piece of writing. This means that the work should draw from multiple sources. This is not a bounty for original research, and is intended for work that covers more ground at a higher level than the distillation contest. Below are some examples of public materials that we value. This should not be taken as an exhaustive list of all existing valuable public contributions. AI risk executive summary (2014) Concrete Problems in AI Safety (2016) Robert Miles' YouTube channel (2017-present) AGI Safety From First Principles (2020) The case for taking AI risk seriously as a threat to humanity (2020) Unsolved Problems in ML Safety (2021) X-risk Analysis for AI Research (2022) Note that many of the works above are quite different and do not always agree with each other. Listing them isn't to say that we agree with everything in them, and we don't expect to necessarily agree with all claims in the pieces we award bounties to. However, we will not award bounties to work we believe is false or misleading. Here are some categories of work we believe could be valuable: Executive ...

Hello Mr. Molyneux, Tl; dr: I just donated to you, because this was my first chance in thirteen years to do so. Your book Real Time Relationships saved my life. Now I'm thriving and can finally give back. I'm afraid I lost touch with your show after I defoo'd, and only recently started to hear your thoughts again and follow you on Telegram. I just wanted to thank you, because I read Real Time Relationships and it made me realize that I did not have to put up with the abuse I was going through. RTR and UPB accompanied me through the hardest moments of my life, and dissuaded me from committing suicide. I eventually found a good community of friends and we became family. A healthy, happy, non-coercive family. If you have time to read my story, and I hope you'll find this all — especially my research — interesting enough for a call-in show, here it goes:I grew up in [X], where my group of friends is. My father was an angel during my early life: He got me into reading and science through sci-fi, showed me how wonderful mathematics is, taught me more than I learned in college about computers, a ton about cars, and was generally a very kind, peaceful man. My mother, however, was the complete opposite: She was physically abusive in my youth and once I grew taller than her she became emotionally abusive. She is the archetype of the devouring mother, wanting my sisters and I to always be under her control, and she got her way with one of my sisters, and to this day I find any of her physical affection to stir deep, bodily disgust in me. Right around the time I hit puberty, my father went into a major depression, so bad that it went on after I left, and only five years back or so did he get on his feet. That time, when I was spiritually alone and hanging out with bad crowds, is when I started listening to your show. Since then, my dad and I have rebuilt some aspects of our relationship. I have two older sisters, the eldest helped me defoo, and I helped raise her son, whereas it took me seven years to rebuild my relationship with my other sister due to my mother's lies —strange, considering my grandmother did the same to her and her brother and she is both conscious and resentful of that. I've tried to rebuild with my mother, after understanding that she's mentally a child, and that her mother was much worse with her than she was with me, but she refuses to respect my boundaries even to this day, so we don't really have contact anymore. For about three years I was a tuba player in an opera company, which was barely enough to live along with other side gigs both with the tuba and, for example, as a bartender. Eventually I became a translator (mostly English to Spanish), and that's still my profession. I focus on technical/scientific articles, especially ones with a lot of statistics, so during the pandemic I was busier than I've ever been, but good academic music has served to lift my spirits. Three years ago, I decided I wanted to pursue philosophy as well and moved to America. I'm currently researching computable measures of ethics for my Master's thesis. That being said, the ideas I have are more related to AI; I have an MSCS and bachelors in CS, math, and philosophy. My thesis is for Computational Mathematics and Statistics, my second MS. I'm thriving through my research, started publishing papers on forensic linguistics, and getting educated in Quantum Computing, and won't stop until I have a PhD in logic. Now I have started making good friends in the US, surrounding myself with people I respect, but I'm dating a daddy's money leftist. Why do I do that to myself? Well, you could say we're sharing a drink that's called loneliness but it's better than drinking alone! So, about my research:My original idea was to survey objective ethical theories and derive mathematical formalisms for them, in order to find commonalities and propose a theory that included and expanded on the best aspects of all of them. That changed when I came across Shane Legg's doctoral thesis on Machine Super Intelligence, where he describes an objective, mathematical measurement for intelligence, and I said: We can make an ethical theory out of this! This was, ironically, inspired by my mother's abuse, as I figured the ethical responsibility of an actor is limited to what it can infer and how intelligent it is; I cannot have the same standards for all the adult children around me and for the people I actually respect. This led to a 100 or so pages of work where I review common human moral intuitions, God, Kant, and others while doing my best to bring down ideas from people like Habermas and Rawls (I still remember your video on Rawls, it certainly inspired me to do this work). I figure this is a big enough wall of text already, I'm sorry but I just wanted to tell you about some of the myriad things I owe to you. Let me apologize for my English as well; Spanish is my first language and even after three and a half years of living in the US I still have not mastered the grammar.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: DeepMind: The Podcast - Excerpts on AGI, published by WilliamKiely on April 7, 2022 on LessWrong. DeepMind: The Podcast - Season 2 was released over the last ~1-2 months. The two episodes most relevant to AGI are: The road to AGI - DeepMind: The Podcast (S2, Ep5) and The promise of AI with Demis Hassabis - DeepMind: The Podcast (S2, Ep9) I found a few quotes noteworthy and thought I'd share them here for anyone who didn't want to listen to the full episodes: The road to AGI (S2, Ep5) (Published February 15, 2022) Shane Legg's AI Timeline Shane Legg (4:03): If you go back 10-12 years ago the whole notion of AGI was lunatic fringe. People [in the field] would literally just roll their eyes and just walk away. [...] [I had that happen] multiple times. I have met quite a few of them since. There have even been cases where some of these people have applied for jobs at DeepMind years later. But yeah, it was a field where you know there were little bits of progress happening here and there, but powerful AGI and rapid progress seemed like it was very, very far away. [...] Every year [the number of people who roll their eyes at the notion of AGI] becomes less. Hannah Fry (5:02): For over 20 years, Shane has been quietly making predictions of when he expects to see AGI. Shane Legg (5:09): I always felt that somewhere around 2030-ish it was about a 50-50 chance. I still feel that seems reasonable. If you look at the amazing progress in the last 10 years and you imagine in the next 10 years we have something comparable, maybe there's some chance that we will have an AGI in a decade. And if not in a decade, well I don't know, say three decades or so. Hannah Fry (5:33): And what do you think [AGI] will look like? [Shane answers at length.] David Silver on it being okay to have AGIs with different goals (??) Hannah Fry (16:45): Last year David co-authored a provocatively titled paper called Reward is Enough. He believes reinforcement learning alone could lead all the way to artificial general intelligence. [...] (21:37) But not everyone at DeepMind is convinced that reinforcement learning on its own will be enough for AGI. Here's Raia Hadsell, Director of Robotics. Raia Hadsell (21:44): The question I usually have is where do we get that reward from. It's hard to design rewards and it's hard to imagine a single reward that's so all-consuming that it would drive learning everything else. Hannah Fry (21:59): I put this question about the difficulty of designing an all-powerful reward to David Silver. David Silver (22:05): I actually think this is just slightly off the mark–this question–in the sense that maybe we can put almost any reward into the system and if the environment's complex enough amazing things will happen just in maximizing that reward. Maybe we don't have to solve this "What's the right thing for intelligence to really emerge at the end of it?" kind of question and instead embrace the fact that there are many forms of intelligence, each of which is optimizing for its own target. And it's okay if we have AIs in the future some of which are trying to control satellites and some of which are trying to sail boats and some of which are trying to win games of chess and they may all come up with their own abilities in order to allow that intelligence to achieve its end as effectively as possible. [...] (26:14) But of course this is a hypothesis. I cannot offer any guarantee that reinforcement learning algorithms do exist which are powerful enough to just get all the way there. And yet the fact that if we can do it it would provide a path all the way to AGI should be enough for us to try really really hard. Promise of AI with Demis Hassabis (Ep9) (Published March 15, 2022) Demis Hassabis' AI Timeline Dennis Hassabis (6:23): From what we've seen so far [the development of AGI]...

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: DeepMind: The Podcast - Excerpts on AGI, published by WilliamKiely on April 7, 2022 on LessWrong. DeepMind: The Podcast - Season 2 was released over the last ~1-2 months. The two episodes most relevant to AGI are: The road to AGI - DeepMind: The Podcast (S2, Ep5) and The promise of AI with Demis Hassabis - DeepMind: The Podcast (S2, Ep9) I found a few quotes noteworthy and thought I'd share them here for anyone who didn't want to listen to the full episodes: The road to AGI (S2, Ep5) (Published February 15, 2022) Shane Legg's AI Timeline Shane Legg (4:03): If you go back 10-12 years ago the whole notion of AGI was lunatic fringe. People [in the field] would literally just roll their eyes and just walk away. [...] [I had that happen] multiple times. I have met quite a few of them since. There have even been cases where some of these people have applied for jobs at DeepMind years later. But yeah, it was a field where you know there were little bits of progress happening here and there, but powerful AGI and rapid progress seemed like it was very, very far away. [...] Every year [the number of people who roll their eyes at the notion of AGI] becomes less. Hannah Fry (5:02): For over 20 years, Shane has been quietly making predictions of when he expects to see AGI. Shane Legg (5:09): I always felt that somewhere around 2030-ish it was about a 50-50 chance. I still feel that seems reasonable. If you look at the amazing progress in the last 10 years and you imagine in the next 10 years we have something comparable, maybe there's some chance that we will have an AGI in a decade. And if not in a decade, well I don't know, say three decades or so. Hannah Fry (5:33): And what do you think [AGI] will look like? [Shane answers at length.] David Silver on it being okay to have AGIs with different goals (??) Hannah Fry (16:45): Last year David co-authored a provocatively titled paper called Reward is Enough. He believes reinforcement learning alone could lead all the way to artificial general intelligence. [...] (21:37) But not everyone at DeepMind is convinced that reinforcement learning on its own will be enough for AGI. Here's Raia Hadsell, Director of Robotics. Raia Hadsell (21:44): The question I usually have is where do we get that reward from. It's hard to design rewards and it's hard to imagine a single reward that's so all-consuming that it would drive learning everything else. Hannah Fry (21:59): I put this question about the difficulty of designing an all-powerful reward to David Silver. David Silver (22:05): I actually think this is just slightly off the mark–this question–in the sense that maybe we can put almost any reward into the system and if the environment's complex enough amazing things will happen just in maximizing that reward. Maybe we don't have to solve this "What's the right thing for intelligence to really emerge at the end of it?" kind of question and instead embrace the fact that there are many forms of intelligence, each of which is optimizing for its own target. And it's okay if we have AIs in the future some of which are trying to control satellites and some of which are trying to sail boats and some of which are trying to win games of chess and they may all come up with their own abilities in order to allow that intelligence to achieve its end as effectively as possible. [...] (26:14) But of course this is a hypothesis. I cannot offer any guarantee that reinforcement learning algorithms do exist which are powerful enough to just get all the way there. And yet the fact that if we can do it it would provide a path all the way to AGI should be enough for us to try really really hard. Promise of AI with Demis Hassabis (Ep9) (Published March 15, 2022) Demis Hassabis' AI Timeline Dennis Hassabis (6:23): From what we've seen so far [the development of AGI]...

Hannah meets DeepMind co-founder and chief scientist Shane Legg, the man who coined the phrase ‘artificial general intelligence', and explores how it might be built. Why does Shane think AGI is possible? When will it be realised? And what could it look like? Hannah also explores a simple theory of using trial and error to reach AGI and takes a deep dive into MuZero, an AI system which mastered complex board games from chess to Go, and is now generalising to solve a range of important tasks in the real world. For questions or feedback on the series, message us on Twitter @DeepMind or email podcast@deepmind.com.  Interviewees: DeepMind's Shane Legg, Doina Precup, Dave Silver & Jackson Broshear CreditsPresenter: Hannah FrySeries Producer: Dan HardoonProduction support: Jill AchinekuSounds design: Emma BarnabyMusic composition: Eleni ShawSound Engineer: Nigel AppletonEditor: David PrestCommissioned by DeepMind Thank you to everyone who made this season possible!  Further reading: Real-world challenges for AGI, DeepMind: https://deepmind.com/blog/article/real-world-challenges-for-agiAn executive primer on artificial general intelligence, McKinsey: https://www.mckinsey.com/business-functions/operations/our-insights/an-executive-primer-on-artificial-general-intelligenceMastering Go, chess, shogi and Atari without rules, DeepMind: https://deepmind.com/blog/article/muzero-mastering-go-chess-shogi-and-atari-without-rulesWhat is AGI?, Medium: https://medium.com/intuitionmachine/what-is-agi-99cdb671c88eA Definition of Machine Intelligence by Shane Legg, arXiv: https://arxiv.org/abs/0712.3329Reward is enough by David Silver, ScienceDirect: https://www.sciencedirect.com/science/article/pii/S0004370221000862

Hannah explores the potential of language models, the questions they raise, and if teaching a computer about language is enough to create artificial general intelligence (AGI). Beyond helping us communicate ideas, language plays a crucial role in memory, cooperation, and thinking – which is why AI researchers have long aimed to communicate with computers using natural language. Recently, there has been extraordinary progress using large-language models (LLM), which learn how to speak by processing huge amounts of data from the internet. The results can be very convincing, but pose significant ethical challenges.  For questions or feedback on the series, message us on Twitter @DeepMind or email podcast@deepmind.com.  Interviewees: DeepMind's Geoffrey Irving, Chris Dyer, Angeliki Lazaridou, Lisa-Anne Hendriks & Laura Weidinger  CreditsPresenter: Hannah FrySeries Producer: Dan HardoonProduction support: Jill AchinekuSounds design: Emma BarnabyMusic composition: Eleni ShawSound Engineer: Nigel AppletonEditor: David PrestCommissioned by DeepMind Thank you to everyone who made this season possible!  Further reading: GPT-3 Powers the Next Generation of Apps, OpenAI: https://openai.com/blog/gpt-3-apps/https://web.stanford.edu/class/linguist238/p36-weizenabaum.pdfNever Mind the Computer 1983 about the ELIZA program, BBC: https://www.bbc.co.uk/programmes/p023kpf8How Large Language Models Will Transform Science, Society, and AI, Stanford University: https://hai.stanford.edu/news/how-large-language-models-will-transform-science-society-and-aiChallenges in Detoxifying Language Models, DeepMind: https://deepmind.com/research/publications/2021/Challenges-in-Detoxifying-Language-ModelsExtending Machine Language Models toward Human-Level Language Understanding, DeepMind: https://deepmind.com/research/publications/2020/Extending-Machine-Language-Models-toward-Human-Level-Language-UnderstandingLanguage modelling at scale, DeepMind: https://deepmind.com/blog/article/language-modelling-at-scaleArtificial general intelligence, Technology Review: https://www.technologyreview.com/2020/10/15/1010461/artificial-general-intelligence-robots-ai-agi-deepmind-google-openai/A Definition of Machine Intelligence by Shane Legg, arXiv: https://arxiv.org/abs/0712.3329Stuart Russell - Living With Artificial Intelligence, BBC: https://www.bbc.co.uk/programmes/m001216k/episodes/player

The chart-topping podcast which uncovers the extraordinary ways artificial intelligence (AI) is transforming our world is back for a second season. Join mathematician and broadcaster Professor Hannah Fry behind the scenes of world-leading AI research lab DeepMind to get the inside story of how AI is being created – and how it can benefit our lives and the society we live in.Recorded over six months and featuring over 30 original interviews, including DeepMind co-founders Demis Hassabis and Shane Legg, the podcast gives listeners exclusive access to the brilliant people building the technology of the future. Throughout nine original episodes, Hannah discovers how DeepMind is using AI to advance science in critical areas, like solving a 50-year-old grand challenge in biology and developing nuclear fusion.Listeners hear stories of teaching robots to walk at home during lockdown, as well as using AI to forecast weather, help people regain their voices, and enhance game strategies with Liverpool Football Club. Hannah also takes an in-depth look at the challenges and potential of building artificial general intelligence (AGI) and explores what it takes to ensure AI is built to benefit society.“I hope this series gives people a better understanding of AI and a feeling for just how exhilarating an endeavour it is.” – Demis Hassabis, CEO and Co-Founder of DeepMindFor questions or feedback on the series, message us on Twitter @DeepMind or email podcast@deepmind.com.CreditsPresenter: Hannah FrySeries Producer: Dan HardoonProduction support: Jill AchinekuSounds design: Emma BarnabyMusic composition: Eleni ShawSound Engineer: Nigel AppletonEditor: David PrestCommissioned by DeepMind

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is The Blue-Minimizing Robot, Part 6: Preface, published by Scott Alexander. Related to: Shane Legg on Prospect Theory and Computational Finance This post is on prospect theory partly because it fits the theme of replacing simple utility functions with complicated reward functions, but mostly because somehow Less Wrong doesn't have any posts on prospect theory yet and that needs to change. Kahneman and Tversky, the first researchers to identify and rigorously study cognitive biases, proved that a simple version of expected utility theory did not accurately describe human behavior. Their response was to develop prospect theory, a model of how people really make decisions. Although the math is less elegant than that of expected utility, and the shapes of the curves have to be experimentally derived, it is worth a look because it successfully predicts many of the standard biases. (source: Wikipedia) A prospect theory agent tasked with a decision first sets it within a frame with a convenient zero point, allowing em to classify the results of the decision as either losses or gains. Ey then computes a subjective expected utility, where the subjective expected utility equals the subjective value times the subjective probability. The subjective value is calculated from the real value using a value function similar to the one on the left-hand graph, and the subjective probability is calculated from the real probability using a weighting function similar to the one on the right-hand graph. Clear as mud? Let's fill some numbers into the functions - the exact assignments don't really matter as long as we capture the spirit of where things change steeply versus slowly - and run through an example. Imagine a prospect theory agent - let's call him Prospero - trying to decide whether or not to buy an hurricane insurance policy costing $5000/year. Prospero owns assets worth $10,000, and estimates a 50%/year chance of a hurricane destroying his assets; to make things simple, he will be moving in one year and so need not consider the future. Under expected utility theory, he should feel neutral about the policy. Under prospect theory, he first sets a frame in which to consider the decision; his current state is a natural frame, so we'll go with that. We see on the left-hand graph that an objective $10,000 loss feels like a $5,000 loss, and an objective $5000 loss feels like a $4000 loss. And we see on the right-hand graph that a 50% probability feels like a 40% probability. Now Prospero's choice is a certain $4000 loss if he buys the insurance, versus a 40% chance of a $5000 loss if he doesn't. Buying has a subjective expected utility of -$4000; not buying has a subjective expected utility of -$2000. So Prospero decisively rejects the insurance. But suppose Prospero is fatalistic; he views his assets as already having been blown away. Here he might choose a different frame: the frame in which he starts with zero assets, and anything beyond that is viewed as a gain. Since the gain half of the value function levels off more quickly than the loss half, $5000 is now subjectively worth $3000, and $10000 is now subjectively worth $3500. Here he must choose between a certain gain of $5000 and a 50% chance of gaining $10000. Expected utility gives the same result as before, obviously. In prospect theory, he chooses between a certain subjective gain of $3000 and a 40% chance of gaining $3500. The insurance gives him subjective expected utility of $3000, and rejecting it gives him subjective expected utility of $1400. All of a sudden Prospero wants the insurance. We notice the opposite effect if there is only a a 1% chance of a hurricane. The insurance salesman lowers his price to $100 to preserve the neutrality of the insurance option when using utility. But subjective probability rises very quick...

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is The Blue-Minimizing Robot, Part 6: Preface, published by Scott Alexander. Related to: Shane Legg on Prospect Theory and Computational Finance This post is on prospect theory partly because it fits the theme of replacing simple utility functions with complicated reward functions, but mostly because somehow Less Wrong doesn't have any posts on prospect theory yet and that needs to change. Kahneman and Tversky, the first researchers to identify and rigorously study cognitive biases, proved that a simple version of expected utility theory did not accurately describe human behavior. Their response was to develop prospect theory, a model of how people really make decisions. Although the math is less elegant than that of expected utility, and the shapes of the curves have to be experimentally derived, it is worth a look because it successfully predicts many of the standard biases. (source: Wikipedia) A prospect theory agent tasked with a decision first sets it within a frame with a convenient zero point, allowing em to classify the results of the decision as either losses or gains. Ey then computes a subjective expected utility, where the subjective expected utility equals the subjective value times the subjective probability. The subjective value is calculated from the real value using a value function similar to the one on the left-hand graph, and the subjective probability is calculated from the real probability using a weighting function similar to the one on the right-hand graph. Clear as mud? Let's fill some numbers into the functions - the exact assignments don't really matter as long as we capture the spirit of where things change steeply versus slowly - and run through an example. Imagine a prospect theory agent - let's call him Prospero - trying to decide whether or not to buy an hurricane insurance policy costing $5000/year. Prospero owns assets worth $10,000, and estimates a 50%/year chance of a hurricane destroying his assets; to make things simple, he will be moving in one year and so need not consider the future. Under expected utility theory, he should feel neutral about the policy. Under prospect theory, he first sets a frame in which to consider the decision; his current state is a natural frame, so we'll go with that. We see on the left-hand graph that an objective $10,000 loss feels like a $5,000 loss, and an objective $5000 loss feels like a $4000 loss. And we see on the right-hand graph that a 50% probability feels like a 40% probability. Now Prospero's choice is a certain $4000 loss if he buys the insurance, versus a 40% chance of a $5000 loss if he doesn't. Buying has a subjective expected utility of -$4000; not buying has a subjective expected utility of -$2000. So Prospero decisively rejects the insurance. But suppose Prospero is fatalistic; he views his assets as already having been blown away. Here he might choose a different frame: the frame in which he starts with zero assets, and anything beyond that is viewed as a gain. Since the gain half of the value function levels off more quickly than the loss half, $5000 is now subjectively worth $3000, and $10000 is now subjectively worth $3500. Here he must choose between a certain gain of $5000 and a 50% chance of gaining $10000. Expected utility gives the same result as before, obviously. In prospect theory, he chooses between a certain subjective gain of $3000 and a 40% chance of gaining $3500. The insurance gives him subjective expected utility of $3000, and rejecting it gives him subjective expected utility of $1400. All of a sudden Prospero wants the insurance. We notice the opposite effect if there is only a a 1% chance of a hurricane. The insurance salesman lowers his price to $100 to preserve the neutrality of the insurance option when using utility. But subjective probability rises very quick...

As artificial intelligence gets more and more powerful, the need becomes greater to ensure that machines do the right thing. But what does that even mean? Brian Christian joins Vasant Dhar in episode 13 of Brave New World to discuss, as the title of his new book goes, the alignment problem. Useful resources: 1. Brian Christian's homepage. 2. The Alignment Problem: Machine Learning and Human Values -- Brian Christian. 3. Algorithms to Live By: The Computer Science of Human Decisions -- Brian Christian and Tom Griffiths. 4. The Most Human Human -- Brian Christian. 5. How Social Media Threatens Society -- Episode 8 of Brave New World (w Jonathan Haidt). 6. Are We Becoming a New Species? -- Episode 12 of Brave New World (w Molly Crockett). 7. The Nature of Intelligence -- Episode 7 of Brave New World (w Yann le Cunn) 8. Some Moral and Technical Consequences of Automation -- Norbert Wiener. 9.Superintelligence: Paths, Dangers, Strategies -- Nick Bostrom. 10. Human Compatible: AI and the Problem of Control -- Stuart Russell. 11. OpenAI. 12. Center for Human-Compatible AI. 13. Concrete Problems in AI Safety -- Dario Amodei, Chris Olah, Jacob Steinhardt, Paul Christiano, John Schulman, Dan Mané. 14. Machine Bias -- Julia Angwin, Jeff Larson, Surya Mattu and Lauren Kirchner. 15. Inherent Trade-Offs in the Fair Determination of Risk Scores -- Jon Kleinberg, Sendhil Mullainathan, Manish Raghavan. 16. Algorithmic Decision Making and the Cost of Fairness -- Sam Corbett-Davies, Emma Pierson, Avi Feller, Sharad Goel, Aziz Huq.. 17. Predictions Put Into Practice -- Jessica Saunders, Priscillia Hunt, John S. Hollywood 18. An Engine, Not a Camera: How Financial Models Shape Markets -- Donald MacKenzie. 19. An Anthropologist on Mars -- Oliver Sacks. 20. Deep Reinforcement Learning from Human Preferences -- Paul F Christiano, Jan Leike, Tom B Brown, Miljan Martic, Shane Legg, Dario Amadei for OpenAI & Deep Mind.

Professor Marc G. Bellemare is a Research Scientist at Google Research (Brain team), An Adjunct Professor at McGill University, and a Canada CIFAR AI Chair.Featured ReferencesThe Arcade Learning Environment: An Evaluation Platform for General AgentsMarc G. Bellemare, Yavar Naddaf, Joel Veness, Michael BowlingHuman-level control through deep reinforcement learningVolodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A. Rusu, Joel Veness, Marc G. Bellemare, Alex Graves, Martin Riedmiller, Andreas K. Fidjeland, Georg Ostrovski, Stig Petersen, Charles Beattie, Amir Sadik, Ioannis Antonoglou, Helen King, Dharshan Kumaran, Daan Wierstra, Shane Legg & Demis HassabisAutonomous navigation of stratospheric balloons using reinforcement learningMarc G. Bellemare, Salvatore Candido, Pablo Samuel Castro, Jun Gong, Marlos C. Machado, Subhodeep Moitra, Sameera S. Ponda & Ziyu WangAdditional References CAIDA Talk: A tour of distributional reinforcement learning November 18, 2020 - Marc G. Bellemare Amii AI Seminar Series:  Autonomous nav of stratospheric balloons using RL, Marlos C. Machado UMD RLSS | Marc Bellemare | A History of Reinforcement Learning: Atari to Stratospheric Balloons TalkRL: Marlos C. Machado, Dr. Machado also spoke to us about various aspects of ALE and Project Loon in depth Hyperbolic discounting and learning over multiple horizons, Fedus et al 2019 Marc G. Bellemare on Twitter