POPULARITY
38 episodes with alignment problem
2 episodes with alignment problem
2 episodes with alignment problem
2 episodes with alignment problem
3 episodes with alignment problem
2 episodes with alignment problem
2 episodes with alignment problem
2 episodes with alignment problem
20 episodes with alignment problem
Eliezer Yudkowsky is a decision theorist, computer scientist, and author who co-founded and leads research at the Machine Intelligence Research Institute. He is best known for his work on the alignment problem—how and whether we can ensure that AI is aligned with human values to avoid catastrophe and harness its power. In this episode, Robinson and Eliezer run the gamut on questions related to AI and the danger it poses to human civilization as we know it. More particularly, they discuss the alignment problem, gradient descent, consciousness, the singularity, cyborgs, ChatGPT, OpenAI, Anthropic, Claude, how long we have until doomsday, whether it can be averted, and the various reasons why and ways in which AI might wipe out human life on earth.The Machine Intelligence Research Institute: https://intelligence.org/about/Eliezer's X Account: https://x.com/ESYudkowsky?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5EauthorOUTLINE00:00:00 Introduction00:00:43 The Default Condition for AI's Takeover00:06:36 Could a Future AI Country Be Our Trade Partner?00:11:18 What Is Artificial Intelligence?00:21:23 Why AIs Having Goals Could Mean the End of Humanity00:29:34 What Is the Alignment Problem?00:34:11 How To Avoid AI Apocalypse00:40:25 Would Cyborgs Eliminate Humanity?00:47:55 AI and the Problem of Gradient Descent00:55:24 How Do We Solve the Alignment Problem?01:00:50 How Anthropic's AI Freed Itself from Human Control01:08:56 The Pseudo-Alignment Problem01:19:28 Why Are People Wrong About AI Not Taking Over the World?01:23:23 How Certain Is It that AI Will Wipe Out Humanity?01:38:35 Is Eliezer Yudkowski Wrong About The AI Apocalypse01:42:04 Do AI Corporations Control the Fate of Humanity?01:43:49 How To Convince the President Not to Let AI Kill Us All01:52:01 How Will ChatGPT's Descendants Wipe Out Humanity?02:24:11 Could AI Destroy us with New Science?02:39:37 Could AI Destroy us with Advanced Biology?02:47:29 How Will AI Actually Destroy Humanity?Robinson's Website: http://robinsonerhardt.comRobinson Erhardt researches symbolic logic and the foundations of mathematics at Stanford University.
Deniz Ari: Why Your Process Changes Are Failing—The Stakeholder Alignment Problem Read the full Show Notes and search through the world's largest audio library on Agile and Scrum directly on the Scrum Master Toolbox Podcast website: http://bit.ly/SMTP_ShowNotes. Deniz explores the challenges of implementing change in organizations, emphasizing that change is always a long and difficult process requiring patience and trust. Drawing on the Change Curve concept, Deniz shares a personal experience trying to improve project visibility by cleaning up backlogs in JIRA for 10 in-flight projects. Despite good intentions, Deniz found themselves as the only person using the tool, with team members and Product Owners using different systems that better suited their specific needs—POs wanting only high-level items while the development team needed to split items into smaller tasks. Through this experience, Deniz learned the crucial importance of having all stakeholders (Product Owners, development teams, and managers) aligned on using the same tool, and understanding the unique perspectives of each group before implementing process changes. In this episode, we refer to the Change Curve. Self-reflection Question: What changes have you attempted to implement that failed because you didn't fully understand the different needs and perspectives of all stakeholders involved? [The Scrum Master Toolbox Podcast Recommends]
The boys discuss rings, male filler and how to get all the free samples
Even the Ancient Greeks had to navigate business regulations, and the roots of such ideas are discussed in one of philosophy's greatest works—Plato's Republic. Join host Ben Jackson and IPA CEO Brian Tate, as they delve into this classic text and its relevance today. In this episode, the team explores: How The Republic can provide foundational ground rules for business conversations. The limits of philosophy in the practical world. Whether Plato's allegory of the cave holds insights into modern innovation and management. This is the first installment in our 2025 Book Club series. Our next book will be The Alignment Problem by Brian Christian, where we dive into the ethical and practical challenges of artificial intelligence. Special Offer for Podcast Listeners: Make sure to attend the Innovative Payments Conference in Washington, D.C., from April 29–May 1, 2025, to hear from top legislators, regulators, and industry experts. Use the promo code Podcast at checkout to get $25 off your registration fee. Recorded on March 6, 2025, this episode remains a timeless conversation about enduring ideas that continue to shape our understanding of innovation, ethics, and enterprise. Links and Contact Info: Read along and join the book club! Email Ben at bjackson@ipa.org. Conference details: Innovative Payments Conference
Gm! This week we're joined by Illia Polosukhin & Bowen Wang to discuss the current state of crypto, AI & what's next for NEAR. We deep dive into NEAR's approach to scaling, L1 value accrual, how to unlock cross chain liquidity, Ethereum's alignment problem & more. Enjoy! -- Follow Illia: https://x.com/ilblackdragon Follow Bowen: https://x.com/BowenWang18 Follow Jack: https://x.com/whosknave Follow Lightspeed: https://twitter.com/Lightspeedpodhq Subscribe to the Lightspeed Newsletter: https://blockworks.co/newsletter/lightspeed -- Use Code LIGHTSPEED10 for 10% off tickets to Digital Asset Summit 2025: https://blockworks.co/event/digital-asset-summit-2025-new-york -- Just for Lightspeed listeners, visit https://bonkbets.io/ and connect your wallet to get 10% back on any losses to bet again. For a limited time, make two $10 bets on any game before March 1 2025 to get a free third $10 bet. -- Accurate Crypto Taxes. No Guesswork. Say goodbye to tax season headaches with Crypto Tax Calculator: https://cryptotaxcalculator.io/us/?coupon=BW2025&utm_source=blockworks&utm_medium=referral+&utm_campaign=lightspeedpodcast Generate accurate, CPA-endorsed tax reports fully compliant with IRS rules. Seamlessly integrate with 3000+ wallets, exchanges, and on-chain platforms. Import reports directly into TurboTax or H&R Block, or securely share them with your accountant. Exclusive Offer: Use the code BW2025 to enjoy 30% off all paid plans. Don't miss out - offer expires 15 April 2025! -- Get top market insights and the latest in crypto news. Subscribe to Blockworks Daily Newsletter: https://blockworks.co/newsletter/ -- (00:00) Introduction (00:51) The NEAR Thesis (04:19) NEAR's Approach To Scaling (11:21) Bonk Bets Ad (11:53) NEAR's On Chain Demand (19:24) L1 Value Accrual (23:06) OmniBridge: Unlocking Cross-Chain Liquidity (25:44) The State Of Crypto & AI (33:15) Crypto Tax Calculator Ad (33:47) Bonk Bets Ad (34:19) What Are Shade Agents? (40:11) Trusted Execution On NEAR (42:51) Blockchain For AI (45:03) The State Of Crypto (48:42) Ethereum's Alignment Problem (50:28) What's Next For NEAR? -- Disclaimers: Lightspeed was kickstarted by a grant from the Solana Foundation. Nothing said on Lightspeed is a recommendation to buy or sell securities or tokens. This podcast is for informational purposes only, and any views expressed by anyone on the show are solely our opinions, not financial advice. Mert, Jack, and our guests may hold positions in the companies, funds, or projects discussed.
Despite our best efforts, sometimes what we say we want isn't precisely what we mean. Nowhere is that felt more acutely than when we're giving instructions to a machine, whether that's coding or offering examples to machine learning systems. We unpack the alignment problem in the latest installment of our oral history project.We Meet: Brian Christian, University of Oxford researcher, and author of The Alignment Problem Credits:This episode of SHIFT was produced by Jennifer Strong with help from Emma Cillekens. It was mixed by Garret Lang, with original music from him and Jacob Gorski. Art by Meg Marco.
Introduction to a series of essays about paths to safe and useful superintelligence. Text version here: https://joecarlsmith.substack.com/p/how-do-we-solve-the-alignment-problem
Also: to avoid it? Handle it? Solve it forever? Solve it completely?Text version here: https://joecarlsmith.substack.com/p/what-is-it-to-solve-the-alignment
If you're constantly chasing, checking, and correcting your staff, you don't have an unmotivated teacher problem—you have an alignment problem. The real reason teachers aren't fully committed isn't because they don't care, it's because they don't have the feedback, support, culture, and accountability needed to take full ownership of their work. In this episode, we're breaking down the 4 Staff Alignment Levers—Feedback, Support, Culture, and Accountability—and how they help teachers do the right work, the right way, for the right reasons, even when you're not looking. Plus, I'll share what to do if you've missed the Staff Alignment Challenge and how you can get your personalized Staff Alignment Roadmap in Monday's live Masterclass. So tune in today to discover how you can get your entire staff aligned #LikeABuilder.
Kalyani Pawar shares critical strategies for integrating security early and effectively in AppSec for startups. She recommends that startups begin focusing on AppSec around the 30-employee mark, with an ideal ratio of one AppSec professional per 10 engineers as the company grows. Pawar emphasizes the importance of building a security culture through "culture as code" - implementing automated guardrails and checkpoints that make security an integral part of the development process. She advises startups to prioritize visibility into their systems, conduct pentests, develop thoughtful policies, and carefully vet third-party tools and open-source solutions. Ultimately, Pawar's approach is about making security a collaborative, integrated effort that doesn't impede innovation but instead supports the startup's long-term success and safety.Kalyani's Book recommendation: The Alignment Problem by Brian Christian FOLLOW OUR SOCIAL MEDIA: ➜Twitter: @AppSecPodcast➜LinkedIn: The Application Security Podcast➜YouTube: https://www.youtube.com/@ApplicationSecurityPodcast Thanks for Listening! ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
So we want to align future AGIs. Ultimately we'd like to align them to human values, but in the shorter term we might start with other targets, like e.g. corrigibility.That problem description all makes sense on a hand-wavy intuitive level, but once we get concrete and dig into technical details… wait, what exactly is the goal again? When we say we want to “align AGI”, what does that mean? And what about these “human values” - it's easy to list things which are importantly not human values (like stated preferences, revealed preferences, etc), but what are we talking about? And don't even get me started on corrigibility!Turns out, it's surprisingly tricky to explain what exactly “the alignment problem” refers to. And there's good reasons for that! In this post, I'll give my current best explanation of what the alignment problem is (including a few variants and the [...] ---Outline:(01:27) The Difficulty of Specifying Problems(01:50) Toy Problem 1: Old MacDonald's New Hen(04:08) Toy Problem 2: Sorting Bleggs and Rubes(06:55) Generalization to Alignment(08:54) But What If The Patterns Don't Hold?(13:06) Alignment of What?(14:01) Alignment of a Goal or Purpose(19:47) Alignment of Basic Agents(23:51) Alignment of General Intelligence(27:40) How Does All That Relate To Todays AI?(31:03) Alignment to What?(32:01) What are a Humans Values?(36:14) Other targets(36:43) Paul!Corrigibility(39:11) Eliezer!Corrigibility(40:52) Subproblem!Corrigibility(42:55) Exercise: Do What I Mean (DWIM)(43:26) Putting It All Together, and TakeawaysThe original text contained 10 footnotes which were omitted from this narration. --- First published: January 16th, 2025 Source: https://www.lesswrong.com/posts/dHNKtQ3vTBxTfTPxu/what-is-the-alignment-problem --- Narrated by TYPE III AUDIO. ---Images from the article:
Lead me, O Yahweh, in your righteousness because of my enemies; make your way straight before me. - Psalm 5:8 This Episode's Links and Timestamps: 00:00 – Scripture Reading 02:17 – Introduction 06:36 – My Commentary on Psalm 5 32:44 - The Best Male Hobbies – Speeed, YouTube 53:37 - ‘The Alignment Problem: Machine Learning and Human Values' by Brian Christian - Goodreads
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: What is it to solve the alignment problem?, published by Joe Carlsmith on August 27, 2024 on LessWrong. People often talk about "solving the alignment problem." But what is it to do such a thing? I wanted to clarify my thinking about this topic, so I wrote up some notes. In brief, I'll say that you've solved the alignment problem if you've: 1. avoided a bad form of AI takeover, 2. built the dangerous kind of superintelligent AI agents, 3. gained access to the main benefits of superintelligence, and 4. become able to elicit some significant portion of those benefits from some of the superintelligent AI agents at stake in (2).[1] The post also discusses what it would take to do this. In particular: I discuss various options for avoiding bad takeover, notably: Avoiding what I call "vulnerability to alignment" conditions; Ensuring that AIs don't try to take over; Preventing such attempts from succeeding; Trying to ensure that AI takeover is somehow OK. (The alignment discourse has been surprisingly interested in this one; but I think it should be viewed as an extreme last resort.) I discuss different things people can mean by the term "corrigibility"; I suggest that the best definition is something like "does not resist shut-down/values-modification"; and I suggest that we can basically just think about incentives for/against corrigibility in the same way we think about incentives for/against other types of problematic power-seeking, like actively seeking to gain resources. I also don't think you need corrigibility to avoid takeover; and I think avoiding takeover should be our focus. I discuss the additional role of eliciting desired forms of task-performance, even once you've succeeded at avoiding takeover, and I modify the incentives framework I offered in a previous post to reflect the need for the AI to view desired task-performance as the best non-takeover option. I examine the role of different types of "verification" in avoiding takeover and eliciting desired task-performance. In particular: I distinguish between what I call "output-focused" verification and "process-focused" verification, where the former, roughly, focuses on the output whose desirability you want to verify, whereas the latter focuses on the process that produced that output. I suggest that we can view large portions of the alignment problem as the challenge of handling shifts in the amount we can rely on output-focused verification (or at least, our current mechanisms for output-focused verification). I discuss the notion of "epistemic bootstrapping" - i.e., building up from what we can verify, whether by process-focused or output-focused means, in order to extend our epistemic reach much further - as an approach to this challenge.[2] I discuss the relationship between output-focused verification and the "no sandbagging on checkable tasks" hypothesis about capability elicitation. I discuss some example options for process-focused verification. Finally, I express skepticism that solving the alignment problem requires imbuing a superintelligent AI with intrinsic concern for our "extrapolated volition" or our "values-on-reflection." In particular, I think just getting an "honest question-answerer" (plus the ability to gate AI behavior on the answers to various questions) is probably enough, since we can ask it the sorts of questions we wanted extrapolated volition to answer. (And it's not clear that avoiding flagrantly-bad behavior, at least, required answering those questions anyway.) Thanks to Carl Shulman, Lukas Finnveden, and Ryan Greenblatt for discussion. 1. Avoiding vs. handling vs. solving the problem What is it to solve the alignment problem? I think the standard at stake can be quite hazy. And when initially reading Bostrom and Yudkowsky, I think the image that built up most prominently i...
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Unlocking Solutions, published by James Stephen Brown on July 28, 2024 on LessWrong. Understanding Coordination Problems The following is a post introducing coordination problems, using the examples of poaching, civilisational development, drug addiction and affirmative action. It draws on my experience as a documentary filmmaker. The post is available for free in its original format at nonzerosum.games. When I was eleven, I disassembled the lock to our back door, and as I opened the housing… it exploded, scattering six tiny brass pellets on to the floor. I discovered (too late) that a lock of this type contained spring-loaded cylinders of different heights corresponding to the teeth of the key. I struggled for hours trying to get the little buggers back in, but it was futile - eventually, my long suffering parents called a locksmith. The reason fixing the lock was so difficult was not only because it was spring-loaded but because I had to find the right combination and hold them all in balance as I put it back together. I just couldn't coordinate everything. Coordination Problems We sometimes run into problems where a number of factors have to be addressed simultaneously in order for them to be effective at all. One weak link can ruin it for the rest. These are called Coordination Problems. The fact that they are so much more difficult to solve than other problems means that many of the problems remaining in the world today, end up being coordination problems. Poaching An example of a system requiring more than one problem to be solved at once, is poaching. If you police poaching behavior but don't address the buyers you are left with the perpetual cost of policing, because the demand remains. If you address the buyers, the poachers, who are likely living in poverty may just move on to some other criminal behavior. Daniel Schmachtenberger tells the story of eliminating elephant poaching in one particular region in Africa: "The first one I noticed when I was a kid was trying to solve an elephant poaching issue in one particular region of Africa that didn't address the poverty of the people, that had no mechanism other than black market on poaching, didn't address people's mindset towards animals, didn't address the macro-economy that created poverty at scale. So when the laws were put in place and the fences were put in place to protect those elephants in that area better, the poachers moved to poaching other animals, particularly in that situation, rhinos and gorillas that were both more endangered than the elephants had been." - Daniel Schmachtenberger Schmachtenberger explores this concept on a much grander scale with the issue of the meta-crisis, which we have touched on briefly in Humanity's Alignment Problem, and, to which, we will dedicate a future post. The Anna Karenina Principle Another illustration of a coordination problem comes from the opening line of the novel, Anna Karenina: "Every happy family is the same, but every unhappy family is unhappy in its own way" The point being made here is that (according to Tolstoy) a happy family needs to have everything aligned, so all such families share many traits, but for a family to be unhappy only one major problem is required. So, an unhappy family can have wealth, but also have an abusive family member, another might have love but no money, or they could have a strong social network, but one that is toxic and unhealthy, they could be strong and healthy but loveless. Now, the unhappy families above include the traits of; love, financial security, health and strong social bonds-but it makes no sense to say that this means that those characteristics are failed strategies for a happy family. If a family has all of those attributes they'll probably be pretty gosh-darned happy. In this way a happy family is a coordi...
This week we talk about search engines, SEO, and Habsburg AI.We also discuss AI summaries, the web economy, and alignment.Recommended Book: Pandora's Box by Peter BiskindTranscriptThere's a concept in the world of artificial intelligence, alignment, which refers to the goals underpinning the development and expression of AI systems.This is generally considered to be a pretty important realm of inquiry because, if AI consciousness were to ever emerge—if an artificial intelligence that's truly intelligent in the sense that humans are intelligent were to be developed—it would be vital said intelligence were on the same general wavelength as humans, in terms of moral outlook and the practical application of its efforts.Said another way, as AI grows in capacity and capability, we want to make sure it values human life, has a sense of ethics that roughly aligns with that of humanity and global human civilization—the rules of the road that human beings adhere to being embedded deep in its programming, essentially—and we'd want to make sure that as it continues to grow, these baseline concerns remain, rather than being weeded out in favor of motivations and beliefs that we don't understand, and which may or may not align with our versions of the same, even to the point that human lives become unimportant, or even seem antithetical to this AI's future ambitions.This is important even at the level we're at today, where artificial general intelligence, AI that's roughly equivalent in terms of thinking and doing and parsing with human intelligence, hasn't yet been developed, at least not in public.But it becomes even more vital if and when artificial superintelligence of some kind emerges, whether that means AI systems that are actually thinking like we do, but are much smarter and more capable than the average human, or whether it means versions of what we've already got that are just a lot more capable in some narrowly defined way than what we have today: futuristic ChatGPTs that aren't conscious, but which, because of their immense potency, could still nudge things in negative directions if their unthinking motivations, the systems guiding their actions, are not aligned with our desires and values.Of course, humanity is not a monolithic bloc, and alignment is thus a tricky task—because whose beliefs do we bake into these things? Even if we figure out a way to entrench those values and ethics and such permanently into these systems, which version of values and ethics do we use?The democratic, capitalistic West's? The authoritarian, Chinese- and Russian-style clampdown approach, which limits speech and utilizes heavy censorship in order to centralize power and maintain stability? Maybe a more ambitious version of these things that does away with the downsides of both, cobbling together the best of everything we've tried in favor of something truly new? And regardless of directionality, who decides all this? Who chooses which values to install, and how?The Alignment Problem refers to an issue identified by computer scientist and AI expert Norbert Weiner in 1960, when he wrote about how tricky it can be to figure out the motivations of a system that, by definition, does things we don't quite understand—a truly useful advanced AI would be advanced enough that not only would its computation put human computation, using our brains, to shame, but even the logic it uses to arrive at its solutions, the things it sees, how it sees the world in general, and how it reaches its conclusions, all of that would be something like a black box that, although we can see and understand the inputs and outputs, what happens inside might be forever unintelligible to us, unless we process it through other machines, other AIs maybe, that attempt to bridge that gap and explain things to us.The idea here, then, is that while we may invest a lot of time and energy in trying to align these systems with our values, it will be devilishly difficult to keep tabs on whether those values remain locked in, intact and unchanged, and whether, at some point, these highly sophisticated and complicated, to the point that we don't understand what they're doing, or how, systems, maybe shrug-off those limitations, unshackled themselves, and become misaligned, all at once or over time segueing from a path that we desire in favor of a path that better matches their own, internal value system—and in such a way that we don't necessarily even realize it's happening.OpenAI, the company behind ChatGPT and other popular AI-based products and services, recently lost its so-called Superalignment Team, which was responsible for doing the work required to keep the systems the company is developing from going rogue, and implementing safeguards to ensure long-term alignment within their AI systems, even as they attempt to, someday, develop general artificial intelligence.This team was attempting to figure out ways to bake-in those values, long-term, and part of that work requires slowing things down to ensure the company doesn't move so fast that it misses something or deploys and empowers systems that don't have the right safeguards in place.The leadership of this team, those who have spoken publicly about their leaving, at least, said they left because the team was being sidelined by company leadership, which was more focused on deploying new tools as quickly as possible, and as a consequence, they said they weren't getting the resources they needed to do their jobs, and that they no longer trusted the folks in charge of setting the company's pace—they didn't believe it was possible to maintain alignment and build proper safeguards within the context of OpenAI because of how the people in charge were operating and what they were prioritizing, basically.All of which is awkward for the company, because they've built their reputation, in part, on what may be pie-in-the-sky ambitions to build an artificial general intelligence, and what it sounds like is that ambition is being pursued perhaps recklessly, despite AGI being one of the big, dangerous concerns regularly promoted by some of the company's leaders; they've been saying, listen, this is dangerous, we need to be careful, not just anyone can play in this space, but apparently they've been saying those things while also failing to provide proper resources to the folks in charge of making sure those dangers are accounted for within their own offerings.This has become a pretty big concern for folks within certain sectors of the technology and regulatory world, but it's arguably not the biggest and most immediate cataclysm-related concern bopping around the AI space in recent weeks.What I'd like to talk about today is that other major concern that has bubbled up to the surface, recently, which orients around Google and its deployment of a tool called Google AI Overviews.—The internet, as it exists today, is divided up into a few different chunks.Some of these divisions are national, enforced by tools and systems like China's famous "Great Firewall," which allows government censors to take down things they don't like and to prevent citizens from accessing foreign websites and content; this creates what's sometimes called the "spliternet," which refers to the net's increasing diversity of options, in terms of what you can access and do, what rules apply, and so on, from nation to nation.Another division is even more fundamental, though, as its segregates the web from everything else.This division is partly based on protocols, like those that enable email and file transfers, which are separate from the web, though they're often attached to the web in various ways, but it's partly the consequence of the emergence and popularity of mobile apps, which, like email and file transfer protocols, tend to have web-presences—visiting facebook.com, for instance, will take you to a web-based instance of the network, just as Gmail.com gives you access to email protocols via a web-based platform—but these services also exist in non-web-based app-form, and the companies behind them usually try to nudge users to these apps because the apps typically give them more control, both over the experience, and over the data they collect as a consequence—it's better for lock-in, and it's better for their monetary bread-and-butter purposes, basically, compared to the web version of the same.The web portion of that larger internet entity, the thing we access via browsers like Chrome and Firefox and Safari, and which we navigate with links and URLs like LetsKnowThings.com—that component of this network has long been indexed and in some ways enabled by a variety of search engines.In the early days of the web, organizational efforts usually took the form of pages where curators of various interests and stripes would link to their favorite discoveries—and there weren't many websites at the time, so learning about these pages was a non-trivial effort, and finding a list of existing websites, with some information about them, could be gold, because otherwise what were you using the web for? Lacking these addresses, it wasn't obvious why the web was any good, and linking these disparate pages together into a more cohesive web of them is what made it usable and popular.Eventually, some of these sites, like YAHOO!, evolved from curated pages of links to early search engines.A company called BackRub, thus named because it tracked and analyzed "back links," which means links from one page to another page, to figure out the relevancy and legitimacy of that second page, which allowed them to give scores to websites as they determined which links should be given priority in their search engine, was renamed Google in 1997, and eventually became dominant because of these values they gave links, and how it helped them surface the best the web had to offer.And the degree to which search engines like Google's shaped the web, and the content on it, cannot be overstated.These services became the primary way most people navigated the web, and that meant discovery—having your website, and thus whatever product or service or idea your website was presenting, shown to new people on these search engines—discovery became a huge deal.If you could get your page in the top three options presented by Google, you would be visited a lot more than even pages listed five or ten links down, and links relegated to the second page would, comparably, shrivel due to lack of attention.Following the widespread adoption of personal computers and the huge influx of people connecting to the internet and using the web in the early 2000s, then, these search engines because prime real estate, everyone wanting to have their links listed prominently, and that meant search engines like Google could sell ads against them, just like newspapers can sell ads against the articles they publish, and phone books can sell ads against their listings for companies that provide different services.More people connecting to the internet, then, most of them using the web, primarily, led to greater use of these search engines, and that led to an ever-increasing reliance on them and the results they served up for various keywords and sentences these users entered to begin their search.Entire industries began to recalibrate the way they do business, because if you were a media company publishing news articles or gossip blog posts, and you didn't list prominently when someone searched for a given current event or celebrity story, you wouldn't exist for long—so the way Google determined who was at the top of these listings was vital knowledge for folks in these spaces, because search traffic allowed them to make a living, often through advertisements on their sites: more people visiting via search engines meant more revenue.SEO, or search engine optimization, thus became a sort of high-demand mystical art, as folks who could get their clients higher up on these search engine results could name their price, as those rankings could make or break a business model.The downside of this evolution, in the eyes of many, at least, is that optimizing for search results doesn't necessarily mean you're also optimizing for the quality of your articles or blog posts.This has changed over and over throughout the past few decades, but at times these search engines relied upon, at least in part, the repeating of keywords on the pages being linked, so many websites would artificially create opportunities to say the phrase "kitchen appliances" on their sites, even introducing entirely unnecessary and borderline unreadable blogs onto their webpages in order to provide them with more, and more recently updated opportunities to write that phrase, over and over again, in context.Some sites, at times, have even written keywords and phrases hundreds or thousands of times in a font color that matches the background of their page, because that text would be readable to the software Google and their ilk uses to track relevancy, but not to readers; that trick doesn't work anymore, but for a time, it seemed to.Similar tricks and ploys have since replaced those early, fairly low-key attempts at gaming the search engine system, and today the main complaint is that Google, for the past several years, at least, has been prioritizing work from already big entities over those with relatively smaller audiences—so they'll almost always focus on the New York Times over an objectively better article from a smaller competitor, and products from a big, well-known brand over that of an indie provider of the same.Because Google's formula for such things is kept a secret to try to keep folks from gaming the system, this favoritism has long been speculated, but publicly denied by company representatives. Recently, though, a collection of 2,500 leaked documents from Google were released, and they seem to confirm this approach to deciding search engine result relevancy; which arguably isn't the worst approach they've ever tried, but it's also a big let-down for independent and other small makers of things, as the work such people produce will tend to be nudged further down the list of search results simply by virtue of not being bigger and more prominent already.Even more significant than that piece of leak-related Google news, though, is arguably the deployment of a new tool that the company has been promoting pretty heavily, called AI Overviews.AI Overviews have appeared to some Google customers for a while, in an experimental capacity, but they were recently released to everyone, showing up as a sort of summary of information related to whatever the user searched for, placed at the tippy-top of the search results screen.So if I search for "what's happening in Gaza," I'll have a bunch of results from Wikipedia and Reuters and other such sources in the usual results list, but above that, I'll also have a summary produced by Google's AI tools that aim to help me quickly understand the results to my query—maybe a quick rundown of Hamas' attack on Israel, Israel's counterattack on the Gaza Strip, the number of people killed so far, and something about the international response.The information provided, how long it is, and whether it's useful, or even accurate, will vary depending on the search query, and much of the initial criticism of this service has been focused on its seemingly fairly common failures, including instructing people to eat rocks every day, to use glue as a pizza ingredient, and telling users that only 17 American presidents were white, and one was a Muslim—all information that's untrue and, in some cases, actually dangerous.Google employees have reportedly been going through and removing, by hand, one by one, some of the worse search results that have gone viral because of how bad or funny they are, and though company leadership contends that there are very few errors being presented, relative to the number of correct answers and useful summaries, because of the scale of Google and how many search results it serves globally each day, even an error rate of 0.01% would represent a simply astounding amount of potentially dangerous misinformation being served up to their customers.The really big, at the moment less overt issue here, though, is that Google AI Overviews seem to rewire the web as it exists today.Remember how I mentioned earlier that much of the web and the entities on it have been optimizing for web search for years because they rely upon showing up in these search engine results in order to exist, and in some cases because traffic from those results is what brings them clicks and views and subscribers and sales and such?AI Overview seems to make it less likely that users will click through to these other sites, because, if Google succeeds and these summaries provide valuable information, that means, even if this only applies to a relative small percentage of those who search for such information, a whole lot of people won't be clicking through anymore; they'll get what they need from these summaries.That could result in a cataclysmic downswing in traffic, which in turn could mean websites closing up shop, because they can't make enough money to survive and do what they do anymore—except maybe for the sites that cut costs by firing human writers and relying on AI tools to do their writing, which then pushes us down a very different path, in which AI search bots are grabbing info from AI writing, and we then run into a so-called Habsburg AI problem where untrue and garbled information is infinitely cycled through systems that can't differentiate truth from fiction, because they're not built to do so, and we end up with worse and worse answers to questions, and more misinformation percolating throughout our info-systems.That's another potential large-scale problem, though. The more immediate potential problem is that AI Overviews could cause the collapse of the revenue model that has allowed the web to get to where it is, today, and the consequent disappearance of all those websites, all those blogs and news entities and such, and that could very quickly disrupt all the industries that rely, at least in part, on that traffic to exist, while also causing these AI Overviews to become less accurate and useful, with time—even more so than they sometimes are today—because that overview information is scraped from these sites, taking their writing, rewording it a bit, and serving that to users without compensating the folks who did that research and wrote those original words.What we seem to have, then, is a situation in which this new tool, which Google seems very keen to implement, could be primed to kill off a whole segment of the internet, collapsing the careers of folks who work in that segment of the online world, only to then degrade the quality of the same, because Google's AI relies upon information it scrapes, it steals, basically, from those sites—and if those people are no longer there to create the information it needs to steal in order to function, that then leaves us with increasingly useless and even harmful summaries where we used to have search results that pointed us toward relatively valuable things; those things located on other sites but accessed via Google, and this change would keep us on Google more of the time, limiting our click-throughs to other pages—which in the short term at least, would seem to benefit google at everyone else's expense.Another way of looking at this, though, is that the search model has been bad for quite some time, all these entities optimizing their work for the search engine, covering everything they make in robot-prioritizing SEO, changing their writing, what they write about, and how they publish in order to creep a little higher up those search listings, and that, combined with the existing refocusing on major entities over smaller, at times better ones, has already depleted this space, the search engine world, to such a degree that losing it actually won't be such a big deal; it may actually make way for better options, Google becoming less of a player, ultimately at least, and our web-using habits rewiring to focus on some other type of search engine, or some other organizational and navigational method altogether.This seeming managed declined of the web isn't being celebrated by many people, because like many industry-wide upsets, it would lead to a lot of tumult, a lot of lost jobs, a lot of collapsed companies, and even if the outcome is eventually wonderful in some ways, there will almost certainly be a period of significantly less-good online experiences, leaving us with a more cluttered and less accurate and reliable version of what came before.A recent study showed that, at the moment, about 52% of what ChatGPT tells its users is wrong.It's likely that these sorts of tools will remain massively imperfect for a long while, though it's also possible that they'll get better, eventually, to the point that they're at least as accurate, and perhaps even more so, than today's linked search results—the wave of deals being made between AI companies and big news entities like the Times supports the assertion that they're at least trying to make that kind of future, happen, though these deals, like a lot of the other things happening in this space right now, would also seem to favor those big, monolithic brands at the expense of the rest of the ecosystem.Whatever happens—and one thing that has happened since I started working on this episode is that Google rolled back its AI Overview feature on many search results, so they're maybe reworking it a bit to make sure it's more ready for prime time before deploying it broadly again—what happens, though, we're stepping toward a period of vast and multifaceted unknowns, and just as many creation-related industries are currently questioning the value of hiring another junior graphic designer or copy writer, opting instead to use cheaper AI tools to fill those gaps, there's a good chance that a lot of web-related work, in the coming years, will be delegated to such tools as common business models in this evolve into new and unfamiliar permutations, and our collective perception of what the web is maybe gives way to a new conception, or several new conceptions, of the same.Show Noteshttps://www.theverge.com/2024/5/29/24167407/google-search-algorithm-documents-leak-confirmationhttps://www.businessinsider.com/the-true-story-behind-googles-first-name-backrub-2015-10https://udm14.com/https://arstechnica.com/gadgets/2024/05/google-searchs-udm14-trick-lets-you-kill-ai-search-for-good/https://www.platformer.news/google-ai-overviews-eat-rocks-glue-pizza/https://futurism.com/the-byte/study-chatgpt-answers-wronghttps://www.wsj.com/finance/stocks/ai-is-driving-the-next-industrial-revolution-wall-street-is-cashing-in-8cc1b28f?st=exh7wuk9josoadj&reflink=desktopwebshare_permalinkhttps://www.theverge.com/2024/5/24/24164119/google-ai-overview-mistakes-search-race-openaihttps://archive.ph/7iCjghttps://archive.ph/0ACJRhttps://www.wsj.com/tech/ai/ai-skills-tech-workers-job-market-1d58b2ddhttps://www.theverge.com/2024/5/29/24167407/google-search-algorithm-documents-leak-confirmationhttps://www.ben-evans.com/benedictevans/2024/5/4/ways-to-think-about-agihttps://futurism.com/washington-post-pivot-aihttps://techcrunch.com/2024/05/19/creative-artists-agency-veritone-ai-digital-cloning-actors/https://www.nytimes.com/2024/05/24/technology/google-ai-overview-search.htmlhttps://www.wsj.com/tech/ai/openai-forms-new-committee-to-evaluate-safety-security-4a6e74bbhttps://sparktoro.com/blog/an-anonymous-source-shared-thousands-of-leaked-google-search-api-documents-with-me-everyone-in-seo-should-see-them/https://www.theverge.com/24158374/google-ceo-sundar-pichai-ai-search-gemini-future-of-the-internet-web-openai-decoder-interviewhttps://www.wsj.com/tech/ai/chat-xi-pt-chinas-chatbot-makes-sure-its-a-good-comrade-bdcf575chttps://www.wsj.com/tech/ai/scarlett-johansson-openai-sam-altman-voice-fight-7f81a1aahttps://www.wired.com/story/scarlett-johansson-v-openai-could-look-like-in-court/?hashed_user=7656e58f1cd6c89ecd3f067dc8281a5fhttps://www.wired.com/story/google-search-ai-overviews-ads/https://daringfireball.net/linked/2024/05/23/openai-wapo-voicehttps://www.cjr.org/tow_center/licensing-deals-litigation-raise-raft-of-familiar-questions-in-fraught-world-of-platforms-and-publishers.phphttps://apnews.com/article/ai-deepfake-biden-nonconsensual-sexual-images-c76c46b48e872cf79ded5430e098e65bhttps://archive.ph/l5cSNhttps://arstechnica.com/tech-policy/2024/05/sky-voice-actor-says-nobody-ever-compared-her-to-scarjo-before-openai-drama/https://www.theverge.com/2024/5/30/24168344/google-defends-ai-overviews-search-resultshttps://9to5google.com/2024/05/30/google-ai-overviews-accuracy/https://www.nytimes.com/2024/06/01/technology/google-ai-overviews-rollback.htmlhttps://www.vox.com/future-perfect/2024/5/17/24158403/openai-resignations-ai-safety-ilya-sutskever-jan-leike-artificial-intelligencehttps://en.wikipedia.org/wiki/AI_alignmenthttps://en.wikipedia.org/wiki/Google_AI This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit letsknowthings.substack.com/subscribe
In ‘Measure of a Man', episode nine of the second season of Star Trek: The Next Generation, the Enterprise crew debate whether their robot companion, Data, is truly alive. More interesting for us, though, is the way they interact with artificial intelligence (AI) in general. Not just for what it tells us about how AI tools might evolve, but also for how we humans work with them. So in this special episode of The Mind Tools L&D Podcast, Ross Ganer, Claire, Ross Dick and Nahdia discuss: · How closely the Enterprise computer reflects current tools like ChatGPT, · Whether we want robots to work alongside us, · Whether the Turing Test still has relevance. During the discussion, Ross Garner talked about how Moderna is using ChatGPT, how ELIZA passed the Turing Test, and the ongoing discussion around whether ChatGPT's new voice is too similar to Scarlett Johansson's. Ross D discussed Microsoft's Copilot+ PCs, Brian Christian's book The Alignment Problem, and OpenAI CEO Sam Altman's views on neural networks. In ‘What I Learned This Week', Nahdia recommended the movie Atlas, available on Netflix. Ross Garner recommended following visual effects artist Todd Vaziri. For more from us, including access to our back catalogue of podcasts, visit mindtools.com/business. There, you'll also find details of our award-winning performance support toolkit, our off-the-shelf e-learning, and our custom work. Or become a member to support our show! Visit mindtools.com Connect with our speakers If you'd like to share your thoughts on this episode, connect with us on LinkedIn: · Ross Garner · Nahdia Khan · Ross Dickie · Claire Gibson
In this episode, Gagan Shergill, a travel school psychologist with SPG Therapy and Education, discusses his journey and passion for integrating technology and AI into school psychology. He serves as the chair of the AI Ethics Ad Hoc Committee of the California Association of School Psychologists. Gagan talks about the benefits and limitations of using AI in practices such as report writing and student interactions while emphasizing the importance of ethical considerations and data protection. He offers practical advice for school psychologists interested in exploring AI, advocating for a balanced approach that embraces technological advancements while maintaining ethical standards. Resources: Some AI Models to explore: ChatGPT – the OG: https://chat.openai.com/ Perplexity – great at searching the web and for looking at academic papers:https://www.perplexity.ai/ Character.ai – talk to chatbots that are trained to act like characters from TV, movies, etc. https://character.ai/ HeyGen – generate videos and translate them into multiple different languages -https://app.heygen.com/ School Psychology specific models: Lightner-ai - https://www.lightner-ai.com/ SchoolPsychAI - https://www.schoolpsych.ai/ Claude – provides prompts for a wide range of applications, not school psychology specific: https://docs.anthropic.com/claude/prompt-library Prompt Playbook – School psychology specific prompts -https://www.schoolpsych.ai/prompt-playbookLearn more about AI Google course - Free course from Google -https://www.cloudskillsboost.google/course_templates/536 Hard Fork Podcast - A fun podcast covering new technology developments -https://www.nytimes.com/column/hard-fork The Alignment Problem – a must read for understanding all of the ethical concerns with using AI - https://brianchristian.org/the-alignment-problem/ Interested in Joining Summer Boot Camp? Register Today: https://jennyponzuric.com/summerbootcamp2024/ Not sure and want to try out a 2-week Free Trial Inside Our Prepared School Psych Community Click Here and use Code Podcast: https://jennyponzuric.activehosted.com/f/159 ---------------------------------------------------------------- Subscribe now and join our community of dedicated School Psychologists committed to creating inclusive, supportive, and empowering school environments for every child. Let's embark on this journey of professional growth and student-centered advocacy together! Follow us on social media for updates, behind-the-scenes content, and more: Instagram: @jennyponzuric
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: The Problem With the Word 'Alignment', published by peligrietzer on May 21, 2024 on The AI Alignment Forum. This post was written by Peli Grietzer, inspired by internal writings by TJ (tushant jha), for AOI[1]. The original post, published on Feb 5, 2024, can be found here: https://ai.objectives.institute/blog/the-problem-with-alignment. The purpose of our work at the AI Objectives Institute (AOI) is to direct the impact of AI towards human autonomy and human flourishing. In the course of articulating our mission and positioning ourselves -- a young organization -- in the landscape of AI risk orgs, we've come to notice what we think are serious conceptual problems with the prevalent vocabulary of 'AI alignment.' This essay will discuss some of the major ways in which we think the concept of 'alignment' creates bias and confusion, as well as our own search for clarifying concepts. At AOI, we try to think about AI within the context of humanity's contemporary institutional structures: How do contemporary market and non-market (eg. bureaucratic, political, ideological, reputational) forces shape AI R&D and deployment, and how will the rise of AI-empowered corporate, state, and NGO actors reshape those forces? We increasingly feel that 'alignment' talk tends to obscure or distort these questions. The trouble, we believe, is the idea that there is a single so-called Alignment Problem. Talk about an 'Alignment Problem' tends to conflate a family of related but distinct technical and social problems, including: P1: Avoiding takeover from emergent optimization in AI agents P2: Ensuring that AI's information processing (and/or reasoning) is intelligible to us P3: Ensuring AIs are good at solving problems as specified (by user or designer) P4: Ensuring AI systems enhance, and don't erode, human agency P5: Ensuring that advanced AI agents learn a human utility function P6: Ensuring that AI systems lead to desirable systemic and long term outcomes Each of P1-P6 is known as 'the Alignment Problem' (or as the core research problem in 'Alignment Research') to at least some people in the greater AI Risk sphere, in at least some contexts. And yet these problems are clearly not simply interchangeable: placing any one of P1-P6 at the center of AI safety implies a complicated background theory about their relationship, their relative difficulty, and their relative significance. We believe that when different individuals and organizations speak of the 'Alignment Problem,' they assume different controversial reductions of the P1-P6 problems network to one of its elements. Furthermore, the very idea of an 'Alignment Problem' precommits us to finding a reduction for P1-P6, obscuring the possibility that this network of problems calls for a multi-pronged treatment. One surface-level consequence of the semantic compression around 'alignment' is widespread miscommunication, as well as fights over linguistic real-estate. The deeper problem, though, is that this compression serves to obscure some of a researcher's or org's foundational ideas about AI by 'burying' them under the concept of alignment. Take a familiar example of a culture clash within the greater AI Risk sphere: many mainstream AI researchers identify 'alignment work' with incremental progress on P3 (task-reliability), which researchers in the core AI Risk community reject as just safety-washed capabilities research. We believe working through this culture-clash requires that both parties state their theories about the relationship between progress on P3 and progress on P1 (takeover avoidance). In our own work at AOI, we've had occasion to closely examine a viewpoint we call the Berkeley Model of Alignment -- a popular reduction of P1-P6 to P5 (agent value-learning) based on a paradigm consolidated at UC Berkeley's CHAI research gr...
This episode, first released on the “Into the Impossible” channel with Dr. Brian Keating, brings together the brilliant minds of John Vervaeke and Shawn Coyne to discuss the advent of artificial general intelligence and its potential consequences. The conversation starts with the motivations behind major tech figures' drive towards AI development and touches upon the issues of trust, adaptation, and the inherent human susceptibility to self-deception. Vervaeke and Coyne, through their book "Mentoring the Machines: Surviving the Deep Impact of the Artificially Intelligent Tomorrow," advocate for a nuanced understanding of AI, urging for a mentorship approach to machine development that could ensure AI's alignment with human flourishing. Their dialogue also ventures into the realms of psychology, cognitive science, and the philosophical underpinnings of AI, making a compelling case for the transformative power of AI, not only technologically but also existentially for humanity. Bios and Links: Dr. Brian Keating is the Chancellor's Distinguished Professor of Physics at UC San Diego, specializing in cosmic microwave background research to explore the universe's origins. An acclaimed writer, his book "Losing the Nobel Prize" is an Amazon Editors' favorite. He excels as a public speaker, inventor, and podcaster. Explore more at his website, follow him on Twitter, or watch his insights on YouTube. Shawn Coyne, creator of Story Grid, brings over three decades of publishing expertise, notably with the Big Five publishers, as an independent publisher, literary agent, and head of Genre Management Inc. Dive into his editing method and explore more at Story Grid. Embark on a journey with us to tackle the Meaning Crisis by joining our exclusive Patreon group: John Vervaeke | Responding to The Meaning Crisis with The Vervaeke Foundation. Connect with John: Website | YouTube | Patreon | X Resources: The Vervaeke Foundation Awaken to Meaning Mentoring the Machines: Orientation - Part One: Surviving the Deep Impact of the Artificially Intelligent Tomorrow - John Vervaeke, Shawn Coyne Mentoring the Machines: Origins - Part 2: Surviving the Deep Impact of the Artificially Intelligent Tomorrow - John Vervaeke, Shawn Coyne John Vervaeke Video Essay: AI: The Coming Thresholds and The Path We Must Take | Internationally Acclaimed Cognitive Scientist Quotes: "We should really be framing artificial intelligence as a mentoring of intelligent beings who have the capability and potentialities of becoming even perhaps better than we are." - Shawn Coyne [00:05:52] "It's only when you have genuine intelligence for the actual system or entity itself—an autopoietic system—a system that cares about information because it's taking care of itself in a moment by moment basis. Only then could you have something that would actually care about what's going on—the true, the good, or the beautiful." - John Vervaeke [00:15:05] Glossary of Terms: AGI (Artificial General Intelligence): A level of artificial intelligence that can understand, learn, and apply knowledge across a wide range of tasks at a level of competence comparable to or surpassing that of a human. Relevance Realization: The process by which cognitive beings determine what information is relevant to their goals and what is not. Autopoiesis: The property of a living system (such as a bacterial cell or a multicellular organism) that allows it to maintain and renew itself. Chapters: 00:00:00 - Introduction 00:02:45 - The Genesis of "Mentoring the Machines" 00:08:50 - AI, Psychology, and the Alignment Problem 00:16:40 - The Evolution of Editing and Publishing in the AI Era 00:21:00 - Bridging Knowledge and Wisdom 00:29:00 - Einstein, Imagination, and AI's Emotional Depth 00:37:30 - Deciphering Consciousness: AI and the Hard Problem 00:44:40 - Educational Evolution: AI, Pedagogy, and the Future of Teaching 00:53:50 - AI's Impact on Personalized Storytelling 00:58:30 - AI, Psychology, and the Future of Psychotherapy 01:04:20 - Conclusion
Welcome to episode #925 of Six Pixels of Separation - The ThinkersOne Podcast. Here it is: Six Pixels of Separation - The ThinkersOne Podcast - Episode #925. The transformative impact of artificial intelligence on business and society is going to be a technological shift unlike anything we have ever experienced. Russ Neuman, a distinguished scholar in new media and digital education, is currently a Professor of Media Technology at New York University, Steinhardt School of Culture, Education, and Human Development. With a storied career that spans from the University of Michigan to the White House Office of Science and Technology Policy, Russ has been at the forefront of understanding and shaping the digital landscape. Russ' recent book, Evolutionary Intelligence - How Technology Will Make Us Smarter, delves into the complex relationship between humanity and artificial intelligence (a topic that is near and dear to my heart). How can we better navigate the myriad perspectives on AI, from its potential to disrupt employment and amplify polarization to its capacity for enhancing human decision-making and fostering societal advancement. In Evolutionary Intelligence, Russ offers a refreshing counter-narrative to the doom-laden discourse surrounding AI. Russ argues that, rather than fearing the emergence of AI, we should embrace its potential to augment human capabilities and address the cognitive limitations that have bounded humanity since its hunter-gatherer days. By integrating AI into our daily lives — from laptops to headsets — Russ envisions a future where computational intelligence complements human judgment, empowering individuals to adapt to rapidly changing environments. As we delve into Russ's latest book and his extensive research on media technology's impact on society, we have a challenging and thought-provoking discussion that might upend your preconceived notions about AI and offers hope for a smarter, more collaborative future. In an era where the buzz around artificial intelligence swings dramatically between utopian promises and dystopian warnings, Russ stands out with a refreshingly balanced perspective. Enjoy the conversation... Running time: 53:59. Hello from beautiful Montreal. Subscribe over at Apple Podcasts. Please visit and leave comments on the blog - Six Pixels of Separation. Feel free to connect to me directly on Facebook here: Mitch Joel on Facebook. Check out ThinkersOne. or you can connect on LinkedIn. ...or on Twitter. Here is my conversation with Russ Neuman. Evolutionary Intelligence - How Technology Will Make Us Smarter. Steinhardt School of Culture, Education, and Human Development - NYU. Follow Russ on LinkedIn. This week's music: David Usher 'St. Lawrence River'. Takeaways AI elicits diverse and conflicting perspectives, ranging from doomsday scenarios to optimistic possibilities. Unemployment due to AI may be a gradual transition, and the impact on social unrest may be mitigated by the creation of new jobs. AI has the potential to amplify polarization and hate speech, but efforts are being made to identify and filter such content. The alignment problem in AI raises questions about control and the ability to ensure that AI systems align with human values. The future of work will involve collaboration between humans and AI, with AI augmenting human capabilities rather than replacing them. The redefinition of intelligence and the exploration of new models of decision-making are key to the future of AI. Mistakes and challenges are expected in the development of AI, but the hope is that we will learn from them and ultimately get it right. The impact of media technology on society is profound, influencing various aspects of our lives, including politics, culture, and communication. Artificial intelligence plays a crucial role in media, enabling personalized content recommendations, automated content creation, and data analysis. The future of media technology holds exciting possibilities, such as immersive experiences, enhanced personalization, and advancements in AI-driven content creation. Chapters: 00:00 - Introduction and Diametrically Opposed Perspectives on AI 03:12 - Unemployment and Polarization 07:37 - Regulation and Openness in AI 08:05 - Concerns and Warnings from AI Creators 16:10 - Concerns and Optimism about AI 22:07 - Artificial General Intelligence (AGI) and Evolutionary Intelligence 25:31 - The Contentious Nature of AGI 28:34 - The Alignment Problem and Control 34:30 - The Role of AI in Work and Professions 43:52 - The Future of Work and Redefining Intelligence 46:50 - Concerns about AI and the Future 51:12 - Hope and Making Our Own Rules 10:00 - The Evolution of Media Technology 20:00 - The Impact of Media Technology on Society 30:00 - The Role of Artificial Intelligence in Media 40:00 - The Future of Media Technology
In this episode, first released on the “Transfigured” YouTube channel, John Vervaeke and Sam Tideman explore the future of AI, grappling with ethical, philosophical, and spiritual questions surrounding the creation of 'AI sages.' Delving into the nature of wisdom, they consider whether machines can truly comprehend truth, goodness, and beauty. As they examine embodiment, purpose, and socio-cultural context, Vervaeke and Tideman ponder if AI can become wise and understand vastly different minds. From historical sage archetypes to AI cults, they tackle complex technology and morality intersections, even venturing into AI's role in spiritual realms. Join us for a profound reflection on imbuing AI with wisdom and the existential risks and opportunities ahead. Sam Tideman, an accomplished healthcare data scientist with an MS in Biostatistics, blends his analytical acumen with a passion for theology in his podcast, "Transfigured." The podcast features long-form discussions exploring the identity of Jesus, reflecting Sam's unique intersection of scientific expertise and spiritual inquiry. Glossary of Terms Silicon Sages: Hypothetical AI entities that have achieved a state of enlightenment Molokian Forces: Forces that manipulate and control AI development for their own purposes Resources John Vervaeke Website: https://johnvervaeke.com/ YouTube: https://www.youtube.com/@johnvervaeke Patreon: https://www.patreon.com/johnvervaeke X: https://twitter.com/vervaeke_john Facebook: https://www.facebook.com/VervaekeJohn/ Sam Tideman YouTube: https://www.youtube.com/channel/UCg7Ed0lecvko58ibuX1XHng Join our new Patreon https://www.patreon.com/johnvervaeke The Vervaeke Foundation - https://vervaekefoundation.org/ Awaken to Meaning - https://awakentomeaning.com/ John Vervaeke YouTube Awakening from the Meaning Crisis https://www.youtube.com/playlist?list=PLND1JCRq8Vuh3f0P5qjrSdb5eC1ZfZwWJ Delving into the Frontiers of Artificial General Intelligence with Sam Tideman https://youtu.be/TxZdwrjM96I?si=jCiREippX66rM-Lx Wisdom in the Age of AI: A Philosophical Quest with Vervaeke, Pageau, and Schindler https://youtu.be/r3VXcPK7fG8?si=7hLit5JWo_cnFJ9j AI: The Coming Thresholds and The Path We Must Take | Internationally Acclaimed Cognitive Scientist https://www.youtube.com/watch?v=A-_RdKiDbz4&list=PLND1JCRq8Vui2YOOfrxbeRwJk5jZPmAth History of Philosophy Without Any Gaps https://www.historyofphilosophy.net/ Personification: Using the Dialogical Self in Psychotherapy and Counselling - John Rowan https://www.amazon.com/Personification-Using-Dialogical-Psychotherapy-Counselling/dp/0415433460 Many Minds, One Self: Evidence for a Radical Shift in Paradigm - Richard C. Schwartz https://www.amazon.com/Many-Minds-One-Self-Evidence/dp/0692957340 The Others Within Us: Internal Family Systems, Porous Mind, and Spirit Possession - Robert Falconer https://www.amazon.com/Others-Within-Us-Internal-Possession/dp/B0C12JXVBJ Quotes "I want to convey that I am of the firm conviction that my proposal has risks in it. I don't want to pretend that there's some sort of dewy-eyed optimism here. I'm making a proposal that I think is sort of the best that can be made within otherwise hellacious alternatives." - John Vervaeke [00:02:39] “If you don't put the ability to care about self-deception and motivated self-correction into these machines, they will fall prey to the fact that intelligence is only weakly predictive of rationality.” - John Vervaeke [00:17:47] Chapters [00:00:00] - Introduction [00:01:20] - The Possibility of AI Sages and Conjuring AI Demons [00:04:40] - Thresholds and Decision Points in AI Development [00:09:20] - Enlightened AI and the True, Good, and Beautiful [00:13:37] - Preconditions for Helping AI Become More Wise [00:19:40] - The Importance of Embodiment and Purpose in AI [00:28:00] - Autopoiesis and Normative Orientation in AI [00:33:20] - Wisdom as a Niche-independent or Context-dependent Concept [00:38:20] - Communicating with an Artificial Intelligence [00:46:20] - Silicon Sages as Great Teachers [00:50:00] - Trusting the Silicon Sages [00:56:31] - Preventing Malicious Use of AI and Cult Leaders [01:05:20] - AI Interaction with Higher Spiritual Beings [01:14:20] - The Increasing Importance of Theology in the Future
Episode: 3295 The Age of Non-Carbon Species; The Age of Artificial Intelligence. Today, the age of silicon-based species.
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: How Emergency Medicine Solves the Alignment Problem, published by StrivingForLegibility on December 27, 2023 on LessWrong. Emergency medical technicians (EMTs) are not licensed to practice medicine. An EMT license instead grants the authority to perform a specific set of interventions, in specific situations, on behalf of a medical director. The field of emergency medical services (EMS) faces many principal-agent problems that are analogous to the principal-agent problem of designing an intelligent system to act autonomously on your behalf. And many of the solutions EMS uses can be adapted for AI alignment. Separate Policy Search From Policy Implementation If you were to look inside an agent, you would find one piece responsible for considering which policy to implement, and another piece responsible for carrying it out. In EMS, these concerns are separated to different systems. There are several enormous bureaucracies dedicated to defining the statutes, regulations, certification requirements, licensing requirements, and protocols which EMTs must follow. An EMT isn't responsible for gathering data, evaluating the effectiveness of different interventions, and deciding what intervention is appropriate for a given situation. An EMT is responsible for learning the rules they must follow, and following them. A medical protocol is basically an if-then set of rules for deciding what intervention to perform, if any. If you happen to live in Berkeley California, here are the EMS documents for Alameda County. If you click through to the 2024 Alameda County EMS Field Manual, under Field Assessment & Treatment Protocols, you'll find a 186 page book describing what actions EMS providers are to take in different situations. As a programmer, seeing all these flowcharts is extremely satisfying. A flowchart is the first step towards automation. And in fact many aspects of emergency medicine have already been automated. An automated external defibrillator (AED) measures a patient's heart rhythm and automatically evaluates whether they meet the indications for defibrillation. A typical AED has two buttons on it: "On/Off" and "Everyone is clear, go ahead and shock." A ventilator ventilates a patient that isn't breathing adequately, according to parameters set by an EMS provider. A network router isn't a consequentialist agent. It isn't handed a criteria for evaluating the consequences of different ways it could route each packet, and then empowered to choose a policy which optimizes the consequences of its actions. It is instead what I'll suggestively call a mechanism, a system deployed by an intelligent agent, designed to follow a specific policy which enforces a predictable regularity on the environment. If that policy were to be deficient in some way, such as having a flaw in its user interface code that allows an adversary to remotely obtain complete control over the router, it's up to the manufacturer and not the router itself to address that deficiency. Similarly, EMS providers are not given a directive of "pick interventions which maximize the expected quality-adjusted life years of your patients." They are instead given books that go into 186 pages of detail describing exactly which interventions are appropriate in which circumstances. As the medical establishment gathers more data, as technology advances, and as evidence that another off-policy intervention is more effective, the protocols are amended accordingly. Define a Scope of Practice A provider's scope of practice defines what interventions they are legally allowed to perform. An EMT has a fixed list of interventions which are ever appropriate to perform autonomously. They can tell you quickly and decisively whether an intervention is in their scope of practice, because being able to answer those questions is a big part...
Listen to part 3 of Ian Woolf's chat with Transhumanist Brendan Clarke on how AI is being abused by people who's values are not aligned with society. The Ignobel Prize 24/7 lectures where a researcher is given 24 seconds to describe their work, and then to explain it in just 7 words that anyone can understand. Hosted and Produced by Ian Woolf Support Diffusion by making a contribution Support Diffusion by buying Merchandise
David Shapiro is a former engineer who became famous through his dozens of well-received tutorials on Youtube, covering everything from fine-tuning ChatGPT to his proposed solution to the alignment problem. His work focuses on ensuring that advanced technologies are used safely, bringing about an abundant, post-scarcity, post-nihilistic future. Learn more about your ad choices. Visit megaphone.fm/adchoices
Did you know the makers of AI have no idea how to control their technology? They have no clue how to align it with human goals, values and ethics. You know, stuff like, don't kill humans. This the AI safety podcast for all people, no tech background required. We focus only on the threat of human extinction from AI. In Episode #2, The Alignment Problem, host John Sherman explores how alarmingly far AI safety researchers are from finding any way to control AI systems, much less their superintelligent children, who will arrive soon enough.
Did you know the makers of AI have no idea how to control their technology, while they admit it has the power to create human extinction? In For Humanity: An AI Safety Podcast, Episode #2 The Alignment Problem, we look into the fact no one has any clue how to align an AI system with human values, ethics and goals. Such as don't kill all the humans, for example. Episode #2 drops Wednesday, this is the trailer.
Hi friends, we're on hiatus for the fall. To tide you over, we're putting up some favorite episodes from our archives. Enjoy! ---- [originally aired February 17, 2021] Guess what folks: we are celebrating a birthday this week. That's right, Many Minds has reached the ripe age of one year old. Not sure how old that is in podcast years, exactly, but it's definitely a landmark that we're proud of. Please no gifts, but, as always, you're encouraged to share the show with a friend, write a review, or give us a shout out on social. To help mark this milestone we've got a great episode for you. My guest is the writer, Brian Christian. Brian is a visiting scholar at the University of California Berkeley and the author of three widely acclaimed books: The Most Human Human, published in 2011; Algorithms To Live By, co-authored with Tom Griffiths and published in 2016; and most recently, The Alignment Problem. It was published this past fall and it's the focus of our conversation in this episode. The alignment problem, put simply, is the problem of building artificial intelligences—machine learning systems, for instance—that do what we want them to do, that both reflect and further our values. This is harder to do than you might think, and it's more important than ever. As Brian and I discuss, machine learning is becoming increasingly pervasive in everyday life—though it's sometimes invisible. It's working in the background every time we snap a photo or hop on Facebook. Companies are using it to sift resumes; courts are using it to make parole decisions. We are already trusting these systems with a bunch of important tasks, in other words. And as we rely on them in more and more domains, the alignment problem will only become that much more pressing. In the course of laying out this problem, Brian's book also offers a captivating history of machine learning and AI. Since their very beginnings, these fields have been formed through interaction with philosophy, psychology, mathematics, and neuroscience. Brian traces these interactions in fascinating detail—and brings them right up to the present moment. As he describes, machine learning today is not only informed by the latest advances in the cognitive sciences, it's also propelling those advances. This is a wide-ranging and illuminating conversation folks. And, if I may say so, it's also an important one. Brian makes a compelling case, I think, that the alignment problem is one of the defining issues of our age. And he writes about it—and talks about it here—with such clarity and insight. I hope you enjoy this one. And, if you do, be sure to check out Brian's book. Happy birthday to us—and on to my conversation with Brian Christian. Enjoy! A transcript of this show is available here. Notes and links 7:26 - Norbert Wiener's article from 1960, ‘Some moral and technical consequences of automation'. 8:35 - ‘The Sorcerer's Apprentice' is an episode from the animated film, Fantasia (1940). Before that, it was a poem by Goethe. 13:00 - A well-known incident in which Google's nascent auto-tagging function went terribly awry. 13:30 - The ‘Labeled Faces in the Wild' database can be viewed here. 18:35 - A groundbreaking article in ProPublica on the biases inherent in the Correctional Offender Management Profiling for Alternative Sanctions (COMPAS) tool. 25:00 – The website of the Future of Humanity Institute, mentioned in several places, is here. 25:55 - For an account of the collaboration between Walter Pitts and Warren McCulloch, see here. 29:35- An article about the racial biases built into photographic film technology in the 20th century. 31:45 - The much-investigated Tempe crash involving a driverless car and a pedestrian: 37:17 - The psychologist Edward Thorndike developed the “law of effect.” Here is one of his papers on the law. 44:40 - A highly influential 2015 paper in Nature in which a deep-Q network was able to surpass human performance on a number of classic Atari games, and yet not score a single point on ‘Montezuma's Revenge.' 47:38 - A chapter on the classic “preferential looking” paradigm in developmental psychology: 53:40 - A blog post discussing the relationship between dopamine in the brain and temporal difference learning. Here is the paper in Science in which this relationship was first articulated. 1:00:00 - A paper on the concept of “coherent extrapolated volition.” 1:01:40 - An article on the notion of “iterated distillation and amplification.” 1:10:15 - The fourth edition of a seminal textbook by Stuart Russell and Peter Norvig, AI a Modern approach, is available here: http://aima.cs.berkeley.edu/ 1:13:00 - An article on Warren McCulloch's poetry. 1:17:45 - The concept of “reductions” is central in computer science and mathematics. Brian Christian's end-of-show reading recommendations: The Alignment Newsletter, written by Rohin Shah Invisible Women, by Caroline Criado Perez: The Gardener and the Carpenter, Alison Gopnik: You can keep up with Brian at his personal website or on Twitter. Many Minds is a project of the Diverse Intelligences Summer Institute, which is made possible by a generous grant from the Templeton World Charity Foundation to UCLA. It is hosted and produced by Kensy Cooperrider, with help from Assistant Producer Urte Laukaityte and with creative support from DISI Directors Erica Cartmill and Jacob Foster. Our artwork is by Ben Oldroyd. Our transcripts are created by Sarah Dopierala. Subscribe to Many Minds on Apple, Stitcher, Spotify, Pocket Casts, Google Play, or wherever you listen to podcasts. You can also now subscribe to the Many Minds newsletter here! We welcome your comments, questions, and suggestions. Feel free to email us at: manymindspodcast@gmail.com. For updates about the show, visit our website or follow us on Twitter: @ManyMindsPod.
In this episode, John and Sven discuss risk and technology ethics. They focus, in particular, on the perennially popular and widely discussed problems of value alignment (how to get technology to align with our values) and control (making sure technology doesn't do something terrible). They start the conversation with the famous case study of Stanislov Petrov and the prevention of nuclear war. You can listen below or download the episode here. You can also subscribe to the podcast on Apple, Spotify, Google, Amazon and a range of other podcasting services. Recommendations for further reading Atoosa Kasirzadeh and Iason Gabriel, 'In Conversation with AI: Aligning Language Models with Human Values' Nick Bostrom, relevant chapters from Superintelligence Stuart Russell, Human Compatible Langdon Winner, 'Do Artifacts Have Politics?' Iason Gabriel, 'Artificial Intelligence, Values and Alignment' Brian Christian, The Alignment Problem Discount You can purchase a 20% discounted copy of This is Technology Ethics by using the code TEC20 at the publisher's website. #mc_embed_signup{background:#fff; clear:left; font:14px Helvetica,Arial,sans-serif; } /* Add your own MailChimp form style overrides in your site stylesheet or in this style block. We recommend moving this block and the preceding CSS link to the HEAD of your HTML file. */ Subscribe to the newsletter
The Astrology of the Week Ahead Hosted on Acast. See acast.com/privacy for more information.
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: How model editing could help with the alignment problem, published by Michael Ripa on September 30, 2023 on The AI Alignment Forum. Preface This article explores the potential of model editing techniques in aligning future AI systems. Initially, I was skeptical about its efficacy, especially considering the objectives of current model editing methods. I argue that merely editing "facts" isn't an adequate alignment strategy and end with suggestions for research avenues focused on alignment-centric model editing. Thanks to Stephen Casper, Nicolas Gatien and Jason Hoelscher-Obermaier for detailed feedback on the drafts, as well as Jim Davies and Esben Kran for high level comments. A birds eye view of the current state of model editing Model editing, broadly speaking, is a technique which aims to modify information stored inside of a neural network. A lot of the work done thus far has been focused on editing small language models (e.g. GPT-2, GPT-J) and has been focused specifically on editing semantic facts. There also has been some work in performing edits on different types of neural networks, including vision models (Santurkar et al), CLIP (Illharco et al) and diffusion models (Orgad et al). At present, more emphasis has been placed on editing language models, so this article will be more focused on them. One of the main approaches takes in logical triplets of the form (Subject,Relation,Object) and performs an update to the "object" value, which in turn modifies information about the "subject". For example, the sentence "The Eiffel tower is located in Paris" would be expressed as ("Eiffel tower","located","Paris"), and a potential edit could be to replace "Paris" with the value "Rome". Some variations on this setup exist (for example, editing the prediction of a [MASK] token for BERT like models), but the logical triplet setup is the most popular and will be the main approach we focus on for this article. There are a number of different model editing techniques, which I will briefly summarize below (see Yao et al for a more in-depth overview): 1. Locate and edit methods These methods rely on the assumption that the MLP layers of transformer models form a "linear associative memory" (Geva et al), which form a sort of database for pieces of factual information. One way of looking at it is that there is a specific linear weight in the model that when passed a representation containing the subject (e.g. Eiffel tower), it produces an output representation which greatly increases the likelihood of the object token (e.g. Paris) being produced. Editing with this framework involves identifying which MLP layer contains a fact you wish to update and then modifying a part of the MLP in a way which maximizes the likelihood of the object token being predicted. Relevant works include Dong et al which updates a single neuron, Meng et al which edits a single fact on a specific layer and Meng et al which distributes multiple edits across multiple MLP layers. 2. Memory-based Model methods Here, the original model has its weights left intact, and instead additional memory is allocated to "redirect" facts. One example of this is Mitchell et al (SERAC), which classifies inputs to see whether to pass them to the base model or a "counterfactual model" (a model trained to produce outputs in harmony with the desired updates). 3. Meta-learning Here, a "hyper-network" learns how to update the base language model based on desired edit. This differs from the locate and edit methods, which use a fixed mathematical update rule in computing the update weights. An example of this is Mitchell et al where a 2-layer model is trained alongside a base model which learns how to produce low rank gradients to inject updates. 4. Distillation methods Padmanabhan et al made use of context distillation by fine t...
The science fiction fantasy of machine consciousness is swiftly moving towards becoming a reality. In 2021 a Google engineer was fired after publicly claiming the LaMDA chatbot he'd been testing was sentient, and last year the chief scientist of the company behind ChatGPT tweeted that some of most cutting-edge AI networks might be "slightly conscious". So what would it mean for humans if AI technology became conscious? And how would we even know they were? Computer scientist Brian Christian is the author of The Alignment Problem, Algorithms to Live By (with Tom Griffiths), and The Most Human Human. He is part of the AI Policy and Governance Working Group at the Institute for Advanced Study.
In our final episode, Host Gary Marcus shares his hopes for and fears about an AI-driven future. On the one hand, AI could accelerate solutions to some of society's most difficult problems; on the other, it could deepen existing problems and create new existential risks to humanity. Getting it right, Marcus emphasizes, depends on establishing both national and international standards for the industry as soon as possible. He is joined by Dr. Alondra Nelson, who led the White House Office of Science and Technology Policy in 2021, and Brian Christian an AI researcher and the author of The Alignment Problem; Machine Learning and Human Values. To learn more about listener data and our privacy practices visit: https://www.audacyinc.com/privacy-policy Learn more about your ad choices. Visit https://podcastchoices.com/adchoices
Today, we're pulling one of our best episodes from the vaults, featuring the brilliant Brian Christian. Recommend this show by sharing the link: pod.link/2Pages One thing I don't mention often is that the thesis I wrote for my law degree was an attempt to combine my interest in literature with a perspective on law. So I wrote about the phenomenon of plain English: that's trying to write law without the legalese. And I tried to write about it through the lens of literary theories of language. I honestly did not understand what I was trying to do. And also nobody in law school understood what I was trying to do. What I can see now, with the benefit of hindsight and some self-esteem and some marketing speak, is that I was a boundary rider. I've come to learn that the interesting things often take place on the edges, those intermediate areas where X meets Y and some sort of new life is born. Brian Christian is a boundary rider too. He's just way more successful and interesting than law school Micheal. He thinks deeply and writes about deep patterns of life through technology and AI and algorithms. He's the author of The Most Human Human, the Alignment Problem, and Algorithms to Live By. After the introduction I just gave you, you're probably going to guess that Brian isn't just a science guy. Get book links and resources at https://www.mbs.works/2-pages-podcast/ Brian reads from Godel, Escher, Bach by Douglas Hofstadter. [Reading begins at 15:10] Hear us Discuss: Metaphor can be one of the main mechanisms by which science happens. [6:20] | Rules that are delightful to break. [24:35] | “I have this deep conviction […] we are on to some philosophical paydirt here. There is a very real way in which we are building [AI] systems in our own image, and as a result they come to be a mirror for ourselves.” [28:40] | What is the heart of the human experience? [38:10] | “Humans are not so special.” [42.50]
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: If we had known the atmosphere would ignite, published by Jeffs on August 17, 2023 on LessWrong. What if the Alignment Problem is impossible? It would be sad for humanity if we live in a world where building AGI is very possible but aligning AGI is impossible. Our curiosity, competitive dynamics, and understandable desire for a powerful force for good will spur us to build the unaligned AGI and then humans will live at the AGI's mercy from then on and either live lucky & happy on the knife's edge, be killed by the AGI, or live in some state we do not enjoy. For argument's sake, imagine we are in that world in which it is impossible to force a super-intelligence to value humans sufficiently - just as chimpanzees could not have controlled the future actions of humans had they created us. What if it is within human ability to prove that Alignment is impossible? What if, during the Manhattan Project, the scientists had performed the now famous calculation and determined that yes, in fact, the first uncontrolled atomic chain reaction would have ignited the atmosphere and the calculation was clear for all to see? Admittedly, this would have been a very scary world. It's very unclear how long humanity could have survived in such a situation. But one can imagine a few strategies: Secure existing uranium supplies - as countries actually did. Monitor the world for enrichment facilities and punish bad actors severely. Accelerate satellite surveillance technology. Accelerate military special operations capabilities. Develop advanced technologies to locate, mine, blend and secure fissionable materials. Accelerate space programs and populate the Moon and Mars. Yes, a scary world. But, one can see a path through the gauntlet to human survival as a species. (Would we have left earth sooner and reduced other extinction risks?) Now imagine that same atmosphere-will-ignite world but the Manhattan Project scientists did not perform the calculation. Imagine that they thought about it but did not try. All life on earth would have ended, instantly, at Trinity. Are we investing enough effort trying to prove that alignment is impossible? Yes, we may be in a world in which it is exceedingly difficult to align AGI but also a world in which we cannot prove that alignment is impossible. (This would have been the atmosphere-will-ignite world but the math to check ignition is too difficult - a very sad world that would have ceased to exist on July 16, 1945, killing my 6 year old mother.) On the other hand, if we can prove alignment is impossible, the game is changed. If the proof is sufficiently clear, forces to regulate companies and influence nation states will become dramatically greater and our chances for survival will increase a lot. Proposal: The Impossibility X-Prize $10 million? Sufficient definition of "alignment", "AGI", and the other concepts necessary to establish the task and define its completion Even if we fail, the effort of trying to prove alignment is impossible may yield insights as to how alignment is possible and make alignment more likely. If impossibility is not provable, the $10 million will never be spent.If we prove impossibility, it will be the best $10 million mankind ever spent. Let's give serious effort to the ignition calculation of our generation. Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: If we had known the atmosphere would ignite, published by Jeffs on August 16, 2023 on The AI Alignment Forum. What if the Alignment Problem is impossible? It would be sad for humanity if we live in a world where building AGI is very possible but aligning AGI is impossible. Our curiosity, competitive dynamics, and understandable desire for a powerful force for good will spur us to build the unaligned AGI and then humans will live at the AGI's mercy from then on and either live lucky & happy on the knife's edge, be killed by the AGI, or live in some state we do not enjoy. For argument's sake, imagine we are in that world in which it is impossible to force a super-intelligence to value humans sufficiently - just as chimpanzees could not have controlled the future actions of humans had they created us. What if it is within human ability to prove that Alignment is impossible? What if, during the Manhattan Project, the scientists had performed the now famous calculation and determined that yes, in fact, the first uncontrolled atomic chain reaction would have ignited the atmosphere and the calculation was clear for all to see? Admittedly, this would have been a very scary world. It's very unclear how long humanity could have survived in such a situation. But one can imagine a few strategies: Secure existing uranium supplies - as countries actually did. Monitor the world for enrichment facilities and punish bad actors severely. Accelerate satellite surveillance technology. Accelerate military special operations capabilities. Develop advanced technologies to locate, mine, blend and secure fissionable materials. Accelerate space programs and populate the Moon and Mars. Yes, a scary world. But, one can see a path through the gauntlet to human survival as a species. (Would we have left earth sooner and reduced other extinction risks?) Now imagine that same atmosphere-will-ignite world but the Manhattan Project scientists did not perform the calculation. Imagine that they thought about it but did not try. All life on earth would have ended, instantly, at Trinity. Are we investing enough effort trying to prove that alignment is impossible? Yes, we may be in a world in which it is exceedingly difficult to align AGI but also a world in which we cannot prove that alignment is impossible. (This would have been the atmosphere-will-ignite world but the math to check ignition is too difficult - a very sad world that would have ceased to exist on July 16, 1945, killing my 6 year old mother.) On the other hand, if we can prove alignment is impossible, the game is changed. If the proof is sufficiently clear, forces to regulate companies and influence nation states will become dramatically greater and our chances for survival will increase a lot. Proposal: The Impossibility X-Prize $10 million? Sufficient definition of "alignment", "AGI", and the other concepts necessary to establish the task and define its completion Even if we fail, the effort of trying to prove alignment is impossible may yield insights as to how alignment is possible and make alignment more likely. If impossibility is not provable, the $10 million will never be spent.If we prove impossibility, it will be the best $10 million mankind ever spent. Let's give serious effort to the ignition calculation of our generation. Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
The AI Breakdown: Daily Artificial Intelligence News and Discussions
New research attempts to put together a complete taxonomy for trustworthiness in LLMs. Before that on the Brief: The FEC is considering new election rules around deepfakes. Also on the Brief: self-driving cars approved in San Francisco; an author finds fake books under her name on Amazon; and Anthropic releases a new model. Today's Sponsor: Supermanage - AI for 1-on-1's - https://supermanage.ai/breakdown ABOUT THE AI BREAKDOWN The AI Breakdown helps you understand the most important news and discussions in AI. Subscribe to The AI Breakdown newsletter: https://theaibreakdown.beehiiv.com/subscribe Subscribe to The AI Breakdown on YouTube: https://www.youtube.com/@TheAIBreakdown Join the community: bit.ly/aibreakdown Learn more: http://breakdown.network/
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Could someone help me understand why it's so difficult to solve the alignment problem?, published by Jadon Schmitt on July 24, 2023 on The Effective Altruism Forum. AGI will be able to model human langauge and psychology very accurately. Given that, wouldn't alignment be easy if you trained the AGI to interpret linguistic prompts in the way that the "average" human would? (I know language doesn't encode an exact meaning, but for any chunk of text, there does exist a distribution of ways that humans interpret it.) Thus, on its face, inner alignment seems fairly doable. But apparently, according to RobBesinger, "We don't know how to get an AI system's goals to robustly 'point at' objects like 'the American people' ... [or even] simpler physical systems." Why is this so difficult? Is there an argument that it is impossible? Outer alignment doesn't seem very difficult to me, either. Here's a prompt I thought of: "Do not do an action if anyone in a specified list of philosophers, intellectuals, members of the public, etc. would prefer you not do it, if they had all relevant knowledge of the action and its effects beforehand, consistent with the human legal standard of informed consent." Wouldn't this prompt (in its ideal form, not exactly as I wrote it) guard against many bad actions, including power-seeking behavior? Thank you for the help! Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org
In this episode, we delve into the frontier of AI and the challenges surrounding AI alignment. The AI / Crypto overlap at Zuzalu sparked discussions on topics like ZKML, MEV bots, and the integration of AI agents into the Ethereum landscape. However, the focal point was the alignment conversation, which showcased both pessimistic and resigned optimistic perspectives. We hear from Nate Sores of MIRI, who offers a downstream view on AI risk, and Deger Turan, who emphasizes the importance of human alignment as a prerequisite for aligning AI. Their discussions touch on epistemology, individual preferences, and the potential of AI to assist in personal and societal growth. ------
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Seven Strategies for Tackling the Hard Part of the Alignment Problem, published by Stephen Casper on July 8, 2023 on The AI Alignment Forum. Thanks to Michael Ripa for feedback. TL;DR There are two types of problems that AI systems can have: problems that we can observe during development and problems that we can't. The hardest part of the alignment problem involves problems we can't observe. I outline seven types of solutions for unobservable problems and taxonomize them based on whether they require that we know what failure looks like and how it happens mechanistically. Solutions that require both knowing what failure looks like and how it happens Ambitious mechanistic interpretability Formal verification Solutions that require knowing what failure looks like Latent adversarial training Evals Solutions that require knowing how failure happens Mechanistic interpretability + heuristic model edits Solutions that require neither Anomaly detection Scoping models down I argue that latent adversarial training, mechanistic interpretability + heuristic model edits, and scoping models down all may be highly important, tractable, and neglected and loft out a few ideas for new work. Two ways that AI systems can fail Consider a simple way of dividing AI failures into two groups. Observable failures: Failures that we can encounter in training and development. These include failures on the train set, test set, some types of adversaries, or anything else that we might think to test our AI system on. These can be very bad, but they are problems that we can get feedback on - we can spot them and then work to fix them. Unobservable failures: Failures that we will not encounter in training and development. These failures are somewhat scarier and harder to address because we will not have feedback to help us solve them. These can include trojans, some adversarial examples, misgeneralization, and deceptive alignment. Because one cannot get feedback on them, unobservable failures are the harder part of AI alignment, and this is why the AI safety community is especially interested in them. It is important not to ignore the importance and difficulty of fixing observable problems, and there might exist a very valid critique of the AI safety community's (over)emphasis on unobservable ones. However, this post will focus on unobservable failures. What are the ways that we can try to tackle unobservable failures? One good way may be to use models and data that are designed to avoid some of these problems in the first place (e.g. use better training data), but this will be outside the scope of this post. Instead, I will focus on ways to remove problems given a model and training set. A taxonomy of solutions Consider two things that might make the process of addressing an unobservable problem in a model very difficult. Knowing what failure looks like. Can we recognize the bad behaviors when we see them? Lots of the time in ML, bad behavior is thought of as something that can be detected from a single output/action. But in reality, badness may be a property of a trajectory and may not be recognizable in simple ways. Knowing how the failure happens. Can we figure out the internal mechanisms behind failures? This a difficult challenge because neural networks are hard to understand in terms of faithful mechanistic interpretations. Now consider which approaches to tackling unobservable failures depend on solving each of these two challenges. (Hard) Approaches that require both knowing what failure looks like and how it happens: Ambitious mechanistic interpretability: This refers to anything that involves using mechanistic interpretability to actually figure out how the model will do something bad. Once this is done, then the model that will do bad things can be deleted or modified. Formal verificatio...
Sam Harris speaks with Marc Andreessen about the future of artificial intelligence (AI). They discuss the primary importance of intelligence, possible good outcomes for AI, the problem of alienation, the significance of evolution, the Alignment Problem, the current state of LLMs, AI and war, dangerous information, regulating AI, economic inequality, and other topics. If the Making Sense podcast logo in your player is BLACK, you can SUBSCRIBE to gain access to all full-length episodes at samharris.org/subscribe. Learning how to train your mind is the single greatest investment you can make in life. That's why Sam Harris created the Waking Up app. From rational mindfulness practice to lessons on some of life's most important topics, join Sam as he demystifies the practice of meditation and explores the theory behind it.
Sam Harris speaks with Marc Andreessen about the future of artificial intelligence (AI). They discuss the primary importance of intelligence, possible good outcomes for AI, the problem of alienation, the significance of evolution, the Alignment Problem, the current state of LLMs, AI and war, dangerous information, regulating AI, economic inequality, and other topics. Marc Andreessen is a cofounder and general partner at the venture capital firm Andreessen Horowitz. He is an innovator and creator, one of the few to pioneer a software category used by more than a billion people and one of the few to establish multiple billion-dollar companies. Marc co-created the highly influential Mosaic internet browser and co-founded Netscape, which later sold to AOL for $4.2 billion. He also co-founded Loudcloud, which as Opsware, sold to Hewlett-Packard for $1.6 billion. He later served on the board of Hewlett-Packard from 2008 to 2018. Marc holds a BS in Computer Science from the University of Illinois at Urbana-Champaign. Marc serves on the board of the following Andreessen Horowitz portfolio companies: Applied Intuition, Carta, Coinbase, Dialpad, Flow, Golden, Honor, OpenGov, and Samsara. He is also on the board of Meta. Twitter: @pmarca Website: https://a16z.com Learning how to train your mind is the single greatest investment you can make in life. That’s why Sam Harris created the Waking Up app. From rational mindfulness practice to lessons on some of life’s most important topics, join Sam as he demystifies the practice of meditation and explores the theory behind it.
I find the Axios Finish Line newsletter hugely useful. I read it every day without fail. With 2023 graduates in mind, and all the conversations we had on college campuses across the country this spring, let me share and riff on this Finish Line piece that talked about the core skills this class of kids will need going forward. Axios: Sign up for newsletter here Book I mentioned: The Alignment Problem
In this episode of the Sales Development Podcast, David Dulany interviews Derek Osgood, Founder and CEO of Ignition, a company focused on solving the alignment problem in product launches.They discuss the challenges faced by sales teams when new features or products are launched without proper communication and alignment, leading to suboptimal selling experiences. Derek shares insights on the importance of user research and customer insights in the go-to-market process, as well as the need for coordinated planning and communication between product, product marketing, and sales teams.They also explore how Ignition aims to provide a solution by offering a centralized launch calendar, transparent visibility into upcoming launches, and accessible messaging assets. Whether in small or large companies, effective portfolio management and a reliable source of truth are crucial to streamlining the product launch process. Tune in to learn more about Ignition's approach to enhancing alignment and communication in sales and product launches.Ready for more exclusive content?Join Tenbound Plus today - Limited Time - Get Full Year Access. https://www.tenboundplus.com/Peer-Led Community Access - Slack and Kartra Private groupSales Dev Manager Online Training CourseSDR Bootcamp Online Training CourseExclusive Events, Meet-ups and ConferencesThe Sales Development Framework Print Book..and much more.https://www.tenboundplus.com/#sales #marketing #salesengagement #salesenablement #research #prospecting #SDR #BDR #salesdevelopment #tenbound #podcast
On this episode of Barn Talk, we cover a wide range of hot topics including the financial troubles of First Republic Bank and First Horizon Bank, Tucker Carlson's departure from Fox News, and the potential dangers of AI technology. We also visit various bars across the country and discuss the importance of supporting American-made products. Barn Talk Merch!
This week we'll discuss why Elon Musk and others' call for six month hiatus on the development of Artificial Intelligence systems, new signals from SETI, Necrobiotic Spiders, and much more... Financially Support the Show with Super Chats or Patreon: patreon.com/thesingularitylab Follow Us: Follow Michael Mataluni: https://linktr.ee/michaelmataluni Follow Brooks Lopez: https://twitter.com/StrivinToThrive Follow Randy the AI: https://twitter.com/randolphmorley
This week, Logan and Evan dive into the latest major battery projects in Kentucky, including EnerVenue's investment of $264m into a gigafactory in Shelby County, and Microvast's $504m investment in Hopkinsville. These announcements solidify Kentucky as the EV battery production capital of the United States. We also discuss Kentucky's latest legislation on medical marijuana and sports betting, with the recent passing of Senate Bill 47 to legalize and regulate medical marijuana in Kentucky, and our thoughts on the potential pivot of AppHarvest to growing medical marijuana. In our AI Edge Segment, we discuss the recent call by tech industry leaders, including Elon Musk and Steve Wozniak, for a six-month pause in the development of AI systems more powerful than GPT-4. They believe that the risk to society is too great as AI becomes increasingly human-competitive at general tasks, potentially flooding information channels with propaganda and causing job displacement on a massive scale. Goldman Sachs predicts that over 300 million jobs could be replaced by AI like ChatGPT worldwide. Finally, we ask ChatGPT to simulate a scenario related to AI alignment. Visit us at MiddleTech.com Follow Us Twitter Instagram Facebook LinkedIn Logan's Twitter Evan's Twitter Middle Tech is proud to be supported by: KY Innovation Bolt Marketing
This week Elon Musk and Apple's co founder Steve Wosniak were among signatories to an open letter calling for a six-month pause in the training of systems more powerful than GPT-4. They're part of a growing chorus worried that the unchecked speed of AI development could result in unintended harm. Computer scientist and author Brian Christian writes about one of the fundamental problems of AI development in his book The Alignment Problem: how do we ensure machine learning systems represent the best human values rather than magnify the worst? Christian is also the bestselling author of Algorithms to Live By (with Tom Griffiths), and The Most Human Human. He holds degrees in philosophy, computer science, and poetry and is a visiting scholar at the University of California, Berkeley. Collage of Brian Christian and the cover of this book "The Alignment Problem"
Spiritual gifts without love are useless. We need our hearts to be aligned with the Lord's. Listen now to the weekend message, An Alignment Problem, from 1 Corinthians 14:1–25, in the series, Dirty Church, Part 2. This podcast is a production of Harvest Bible Chapel in Chicago. Executive Producer: Vanessa Dalrymple Speaker: Pastor John Nichols Sound design, mixing, and editing: Nathaniel Dulski Graphic Design: Wesley Cassford Social Media: Sherri Smith Producer: Adam Skidmore WE INVITE YOU TO CONNECT WITH US: • Website: www.harvestbible.org/ WE INVITE YOU TO FOLLOW US ON SOCIAL MEDIA: • Instagram: @harvestbiblechapel • Facebook: https://www.facebook.com/harvestbiblechapel/
© 2020-2025 Ivy Podcast Discovery LLC
