Podcast appearances and mentions of Peter Norvig

Peter Norvig was an AI hipster before it was cool. Curious about getting computers to understand English, he went to his teachers, but they admitted that it was beyond their abilities. Undeterred, Peter dove headlong into the complex but exciting world of AI on his own. Today, he is recognized as a key figure in the advancement of modern AI technologies. In this episode, Peter unpacks the evolution of AI and how it's shaping our world. He also offers practical advice for entrepreneurs looking to leverage AI in their businesses. Peter Norvig is a leading AI expert, Stanford Fellow, and former Director of Research at Google, where he oversaw the development of transformative AI technologies. His contributions to AI and technology have earned him numerous accolades, including the NASA Exceptional Achievement Medal and the Berkeley Engineering Innovation Award.  In this episode, Hala and Peter will discuss: - Peter's transition from academia to the corporate world - How AI is changing the way we live and work - Practical ways entrepreneurs can leverage AI right now - How AI is making learning more personalized - Tips to stay competitive in an AI-driven market - How AI can bridge skill gaps in the workforce - Why we must maintain human control over AI - The impact of automation on income inequality - Why AI will generate more solopreneurs - Ethical considerations of AI in society - And other topics…  Peter Norvig is a computer scientist and a leading expert in artificial intelligence. He is a Fellow at Stanford's Human-Centered AI Institute and a researcher at Google Inc. As Google's Director of Research, Peter oversaw the evolution of search algorithms and built teams focused on groundbreaking advancements in machine translation, speech recognition, and computer vision. Earlier in his career, he led a team at NASA Ames that created autonomous software, which was a precursor to the Mars rovers. Also an influential educator, Peter co-authored the widely used textbook, Artificial Intelligence, which is taught in over 1,500 universities worldwide. His contributions to AI and technology have earned him numerous accolades, including the NASA Exceptional Achievement Medal and the Berkeley Engineering Innovation Award.  Connect With Peter: Peter's Profile: https://hai.stanford.edu/people/peter-norvig  Peter's LinkedIn: https://www.linkedin.com/in/pnorvig/  Peter's Facebook: https://www.facebook.com/peter.norvig  Resources Mentioned: Peter's Book, Artificial Intelligence: A Modern Approach: https://www.amazon.com/Artificial-Intelligence-Modern-Approach-3rd/dp/0136042597  Google AI Principles: https://ai.google/responsibility/principles/  LinkedIn Secrets Masterclass, Have Job Security For Life: Use code ‘podcast' for 30% off at yapmedia.io/course.   Sponsored By: Shopify - Sign up for a one-dollar-per-month trial period at youngandprofiting.co/shopify  Indeed - Get a $75 job credit at indeed.com/profiting  Found - Try Found for FREE at found.com/YAP Rakuten - Start all your shopping at rakuten.com or get the Rakuten app to start saving today, your Cash Back really adds up! Mint Mobile - To get a new 3-month premium wireless plan for just 15 bucks a month, go to mintmobile.com/profiting. Connectteam - Enjoy a 14-day free trial with no credit card needed. Open an account today at Connecteam.com Working Genius - Get 20% off the $25 Working Genius assessment at WorkingGenius.com with code PROFITING at checkout Top Deals of the Week: https://youngandprofiting.com/deals/  More About Young and Profiting Download Transcripts - youngandprofiting.com Get Sponsorship Deals - youngandprofiting.com/sponsorships Leave a Review - ratethispodcast.com/yap Watch Videos - youtube.com/c/YoungandProfiting   Follow Hala Taha LinkedIn - linkedin.com/in/htaha/ Instagram - instagram.com/yapwithhala/ TikTok - tiktok.com/@yapwithhala Twitter - twitter.com/yapwithhala   Learn more about YAP Media's Services - yapmedia.io/

This and all episodes at: https://aiandyou.net/ .   Literally writing the book on AI is my guest Peter Norvig, who is coauthor of the standard text, Artificial Intelligence: A Modern Approach, used in 135 countries and 1500+ universities. Peter is a Distinguished Education Fellow at Stanford's Human-Centered AI Institute and a researcher at Google. He was head of NASA Ames's Computational Sciences Division and a recipient of NASA's Exceptional Achievement Award in 2001. He has taught at USC, Stanford, and Berkeley, from which he received a PhD in 1986 and the distinguished alumni award in 2006. He's also the author of the world's longest palindromic sentence.  In this second half of the interview, we talk about how the rise in prominence of AI in the general population has changed how he communicates about AI, his feelings about the calls for slowdown in model development, and his thinking about general intelligence in large language models; and AI Winters. All this plus our usual look at today's AI headlines. Transcript and URLs referenced at HumanCusp Blog.          

This and all episodes at: https://aiandyou.net/ .   Literally writing the book on AI is my guest Peter Norvig, who is coauthor of the standard text, Artificial Intelligence: A Modern Approach, used in 135 countries and 1500+ universities. (The other author, Stuart Russell, was on this show in episodes 86 and 87.) Peter is a Distinguished Education Fellow at Stanford's Human-Centered AI Institute and a researcher at Google. He was head of NASA Ames's Computational Sciences Division and a recipient of NASA's Exceptional Achievement Award in 2001. He has taught at the University of Southern California, Stanford University, and the University of California at Berkeley, from which he received a PhD in 1986 and the distinguished alumni award in 2006. He's also the author of the world's longest palindromic sentence.  In this first part of the interview, we talk about the evolution of AI from the symbolic processing paradigm to the connectionist paradigm, or neural networks, how they layer on each other in humans and AIs, and Peter's experiences in blending the worlds of academic and business.  All this plus our usual look at today's AI headlines. Transcript and URLs referenced at HumanCusp Blog.          

We are honoured to have as our guest in this episode Professor Stuart Russell. Stuart is professor of computer science at the University of California, Berkeley, and the traditional way to introduce him is to say that he literally wrote the book on AI. Artificial Intelligence: A Modern Approach, which he co-wrote with Peter Norvig, was first published in 1995, and the fourth edition came out in 2020.Stuart has been urging us all to take seriously the dramatic implications of advanced AI for longer than perhaps any other prominent AI researcher. He also proposes practical solutions, as in his 2019 book Human Compatible: Artificial Intelligence and the Problem of Control.In 2021 Stuart gave the Reith Lectures, and was awarded an OBE. But the greatest of his many accolades was surely in 2014 when a character with a background remarkably like his was played in the movie Transcendence by Johnny Depp. The conversation covers a wide range of questions about future scenarios involving AI, and reflects on changes in the public conversation following the FLI's letter calling for a moratorium on more powerful AI systems, and following the global AI Safety Summit held at Bletchley Park in the UK at the beginning of November.Selected follow-ups:Stuart Russell's page at BerkeleyCenter for Human-Compatible Artificial Intelligence (CHAI)The 2021 Reith Lectures: Living With Artificial IntelligenceThe book Human Compatible: Artificial Intelligence and the Problem of ControlMusic: Spike Protein, by Koi Discovery, available under CC0 1.0 Public Domain Declaration

We are running an end of year survey for our listeners. Let us know any feedback you have for us, what episodes resonated with you the most, and guest requests for 2024! RAG has emerged as one of the key pieces of the AI Engineer stack. Jerry from LlamaIndex called it a “hack”, Bryan from Hex compared it to “a recommendation system from LLMs”, and even LangChain started with it. RAG is crucial in any AI coding workflow. We talked about context quality for code in our Phind episode. Today's guests, Beyang Liu and Steve Yegge from SourceGraph, have been focused on code indexing and retrieval for over 15 years. We locked them in our new studio to record a 1.5 hours masterclass on the history of code search, retrieval interfaces for code, and how they get SOTA 30% completion acceptance rate in their Cody product by being better at the “bin packing problem” of LLM context generation. Google Grok → SourceGraph → CodyWhile at Google in 2008, Steve built Grok, which lives on today as Google Kythe. It allowed engineers to do code parsing and searching across different codebases and programming languages. (You might remember this blog post from Steve's time at Google) Beyang was an intern at Google at the same time, and Grok became the inspiration to start SourceGraph in 2013. The two didn't know eachother personally until Beyang brought Steve out of retirement 9 years later to join him as VP Engineering. Fast forward 10 years, SourceGraph has become to best code search tool out there and raised $223M along the way. Nine months ago, they open sourced SourceGraph Cody, their AI coding assistant. All their code indexing and search infrastructure allows them to get SOTA results by having better RAG than competitors:* Code completions as you type that achieve an industry-best Completion Acceptance Rate (CAR) as high as 30% using a context-enhanced open-source LLM (StarCoder)* Context-aware chat that provides the option of using GPT-4 Turbo, Claude 2, GPT-3.5 Turbo, Mistral 7x8B, or Claude Instant, with more model integrations planned* Doc and unit test generation, along with AI quick fixes for common coding errors* AI-enhanced natural language code search, powered by a hybrid dense/sparse vector search engine There are a few pieces of infrastructure that helped Cody achieve these results:Dense-sparse vector retrieval system For many people, RAG = vector similarity search, but there's a lot more that you can do to get the best possible results. From their release:"Sparse vector search" is a fancy name for keyword search that potentially incorporates LLMs for things like ranking and term expansion (e.g., "k8s" expands to "Kubernetes container orchestration", possibly weighted as in SPLADE): * Dense vector retrieval makes use of embeddings, the internal representation that LLMs use to represent text. Dense vector retrieval provides recall over a broader set of results that may have no exact keyword matches but are still semantically similar. * Sparse vector retrieval is very fast, human-understandable, and yields high recall of results that closely match the user query. * We've found the approaches to be complementary.There's a very good blog post by Pinecone on SPLADE for sparse vector search if you're interested in diving in. If you're building RAG applications in areas that have a lot of industry-specific nomenclature, acronyms, etc, this is a good approach to getting better results.SCIPIn 2016, Microsoft announced the Language Server Protocol (LSP) and the Language Server Index Format (LSIF). This protocol makes it easy for IDEs to get all the context they need from a codebase to get things like file search, references, “go to definition”, etc. SourceGraph developed SCIP, “a better code indexing format than LSIF”:* Simpler and More Efficient Format: SCIP utilizes Protobuf instead of JSON, which is used by LSIF. Protobuf is more space-efficient, simpler, and more suitable for systems programming. * Better Performance and Smaller Index Sizes: SCIP indexers, such as scip-clang, show enhanced performance and reduced index file sizes compared to LSIF indexers (10%-20% smaller)* Easier to Develop and Debug: SCIP's design, centered around human-readable string IDs for symbols, makes it faster and more straightforward to develop new language indexers. Having more efficient indexing is key to more performant RAG on code. Show Notes* Sourcegraph* Cody* Copilot vs Cody* Steve's Stanford seminar on Grok* Steve's blog* Grab* Fireworks* Peter Norvig* Noam Chomsky* Code search* Kelly Norton* Zoekt* v0.devSee also our past episodes on Cursor, Phind, Codeium and Codium as well as the GitHub Copilot keynote at AI Engineer Summit.Timestamps* [00:00:00] Intros & Backgrounds* [00:05:20] How Steve's work on Grok inspired SourceGraph for Beyang* [00:08:10] What's Cody?* [00:11:22] Comparison of coding assistants and the capabilities of Cody* [00:16:00] The importance of context (RAG) in AI coding tools* [00:21:33] The debate between Chomsky and Norvig approaches in AI* [00:30:06] Normsky: the Norvig + Chomsky models collision* [00:36:00] The death of the DSL?* [00:40:00] LSP, Skip, Kythe, BFG, and all that fun stuff* [00:53:00] The SourceGraph internal stack* [00:58:46] Building on open source models* [01:02:00] SourceGraph for engineering managers?* [01:12:00] Lightning RoundTranscriptAlessio: Hey everyone, welcome to the Latent Space podcast. This is Alessio, partner and CTO-in-Residence at Decibel Partners, and I'm joined by my co-host Swyx, founder of Smol AI. [00:00:16]Swyx: Hey, and today we're christening our new podcast studio in the Newton, and we have Beyang and Steve from Sourcegraph. Welcome. [00:00:25]Beyang: Hey, thanks for having us. [00:00:26]Swyx: So this has been a long time coming. I'm very excited to have you. We also are just celebrating the one year anniversary of ChatGPT yesterday, but also we'll be talking about the GA of Cody later on today. We'll just do a quick intros of both of you. Obviously, people can research you and check the show notes for more. Beyang, you worked in computer vision at Stanford and then you worked at Palantir. I did, yeah. You also interned at Google. [00:00:48]Beyang: I did back in the day where I get to use Steve's system, DevTool. [00:00:53]Swyx: Right. What was it called? [00:00:55]Beyang: It was called Grok. Well, the end user thing was Google Code Search. That's what everyone called it, or just like CS. But the brains of it were really the kind of like Trigram index and then Grok, which provided the reference graph. [00:01:07]Steve: Today it's called Kythe, the open source Google one. It's sort of like Grok v3. [00:01:11]Swyx: On your podcast, which you've had me on, you've interviewed a bunch of other code search developers, including the current developer of Kythe, right? [00:01:19]Beyang: No, we didn't have any Kythe people on, although we would love to if they're up for it. We had Kelly Norton, who built a similar system at Etsy, it's an open source project called Hound. We also had Han-Wen Nienhuys, who created Zoekt, which is, I think, heavily inspired by the Trigram index that powered Google's original code search and that we also now use at Sourcegraph. Yeah. [00:01:45]Swyx: So you teamed up with Quinn over 10 years ago to start Sourcegraph and you were indexing all code on the internet. And now you're in a perfect spot to create a code intelligence startup. Yeah, yeah. [00:01:56]Beyang: I guess the backstory was, I used Google Code Search while I was an intern. And then after I left that internship and worked elsewhere, it was the single dev tool that I missed the most. I felt like my job was just a lot more tedious and much more of a hassle without it. And so when Quinn and I started working together at Palantir, he had also used various code search engines in open source over the years. And it was just a pain point that we both felt, both working on code at Palantir and also working within Palantir's clients, which were a lot of Fortune 500 companies, large financial institutions, folks like that. And if anything, the pains they felt in dealing with large complex code bases made our pain points feel small by comparison. So that was really the impetus for starting Sourcegraph. [00:02:42]Swyx: Yeah, excellent. Steve, you famously worked at Amazon. And you've told many, many stories. I want every single listener of Latent Space to check out Steve's YouTube because he effectively had a podcast that you didn't tell anyone about or something. You just hit record and just went on a few rants. I'm always here for your Stevie rants. And then you moved to Google, where you also had some interesting thoughts on just the overall Google culture versus Amazon. You joined Grab as head of eng for a couple of years. I'm from Singapore, so I have actually personally used a lot of Grab's features. And it was very interesting to see you talk so highly of Grab's engineering and sort of overall prospects. [00:03:21]Steve: Because as a customer, it sucked? [00:03:22]Swyx: Yeah, no, it's just like, being from a smaller country, you never see anyone from our home country being on a global stage or talked about as a startup that people admire or look up to, like on the league that you, with all your legendary experience, would consider equivalent. Yeah. [00:03:41]Steve: Yeah, no, absolutely. They actually, they didn't even know that they were as good as they were, in a sense. They started hiring a bunch of people from Silicon Valley to come in and sort of like fix it. And we came in and we were like, Oh, we could have been a little better operational excellence and stuff. But by and large, they're really sharp. The only thing about Grab is that they get criticized a lot for being too westernized. Oh, by who? By Singaporeans who don't want to work there. [00:04:06]Swyx: Okay. I guess I'm biased because I'm here, but I don't see that as a problem. If anything, they've had their success because they were more westernized than the Sanders Singaporean tech company. [00:04:15]Steve: I mean, they had their success because they are laser focused. They copy to Amazon. I mean, they're executing really, really, really well for a giant. I was on a slack with 2,500 engineers. It was like this giant waterfall that you could dip your toe into. You'd never catch up. Actually, the AI summarizers would have been really helpful there. But yeah, no, I think Grab is successful because they're just out there with their sleeves rolled up, just making it happen. [00:04:43]Swyx: And for those who don't know, it's not just like Uber of Southeast Asia, it's also a super app. PayPal Plus. [00:04:48]Steve: Yeah. [00:04:49]Swyx: In the way that super apps don't exist in the West. It's one of the enduring mysteries of B2C that super apps work in the East and don't work in the West. We just don't understand it. [00:04:57]Beyang: Yeah. [00:04:58]Steve: It's just kind of curious. They didn't work in India either. And it was primarily because of bandwidth reasons and smaller phones. [00:05:03]Swyx: That should change now. It should. [00:05:05]Steve: And maybe we'll see a super app here. [00:05:08]Swyx: You retired-ish? I did. You retired-ish on your own video game? Mm-hmm. Any fun stories about that? And that's also where you discovered some need for code search, right? Mm-hmm. [00:05:16]Steve: Sure. A need for a lot of stuff. Better programming languages, better databases. Better everything. I mean, I started in like 95, right? Where there was kind of nothing. Yeah. Yeah. [00:05:24]Beyang: I just want to say, I remember when you first went to Grab because you wrote that blog post talking about why you were excited about it, about like the expanding Asian market. And our reaction was like, oh, man, how did we miss stealing it with you? [00:05:36]Swyx: Hiring you. [00:05:37]Beyang: Yeah. [00:05:38]Steve: I was like, miss that. [00:05:39]Swyx: Tell that story. So how did this happen? Right? So you were inspired by Grok. [00:05:44]Beyang: I guess the backstory from my point of view is I had used code search and Grok while at Google, but I didn't actually know that it was connected to you, Steve. I knew you from your blog posts, which were always excellent, kind of like inside, very thoughtful takes from an engineer's perspective on some of the challenges facing tech companies and tech culture and that sort of thing. But my first introduction to you within the context of code intelligence, code understanding was I watched a talk that you gave, I think at Stanford, about Grok when you're first building it. And that was very eye opening. I was like, oh, like that guy, like the guy who, you know, writes the extremely thoughtful ranty like blog posts also built that system. And so that's how I knew, you know, you were involved in that. And then, you know, we always wanted to hire you, but never knew quite how to approach you or, you know, get that conversation started. [00:06:34]Steve: Well, we got introduced by Max, right? Yeah. It was temporal. Yeah. Yeah. I mean, it was a no brainer. They called me up and I had noticed when Sourcegraph had come out. Of course, when they first came out, I had this dagger of jealousy stabbed through me piercingly, which I remember because I am not a jealous person by any means, ever. But boy, I was like, but I was kind of busy, right? And just one thing led to another. I got sucked back into the ads vortex and whatever. So thank God Sourcegraph actually kind of rescued me. [00:07:05]Swyx: Here's a chance to build DevTools. Yeah. [00:07:08]Steve: That's the best. DevTools are the best. [00:07:10]Swyx: Cool. Well, so that's the overall intro. I guess we can get into Cody. Is there anything else that like people should know about you before we get started? [00:07:18]Steve: I mean, everybody knows I'm a musician. I can juggle five balls. [00:07:24]Swyx: Five is good. Five is good. I've only ever managed three. [00:07:27]Steve: Five is hard. Yeah. And six, a little bit. [00:07:30]Swyx: Wow. [00:07:31]Beyang: That's impressive. [00:07:32]Alessio: So yeah, to jump into Sourcegraph, this has been a company 10 years in the making. And as Sean said, now you're at the right place. Phase two. Now, exactly. You spent 10 years collecting all this code, indexing, making it easy to surface it. Yeah. [00:07:47]Swyx: And also learning how to work with enterprises and having them trust you with their code bases. Yeah. [00:07:52]Alessio: Because initially you were only doing on-prem, right? Like a lot of like VPC deployments. [00:07:55]Beyang: So in the very early days, we're cloud only. But the first major customers we landed were all on-prem, self-hosted. And that was, I think, related to the nature of the problem that we're solving, which becomes just like a critical, unignorable pain point once you're above like 100 devs or so. [00:08:11]Alessio: Yeah. And now Cody is going to be GA by the time this releases. So congrats to your future self for launching this in two weeks. Can you give a quick overview of just what Cody is? I think everybody understands that it's a AI coding agent, but a lot of companies say they have a AI coding agent. So yeah, what does Cody do? How do people interface with it? [00:08:32]Beyang: Yeah. So how is it different from the like several dozen other AI coding agents that exist in the market now? When we thought about building a coding assistant that would do things like code generation and question answering about your code base, I think we came at it from the perspective of, you know, we've spent the past decade building the world's best code understanding engine for human developers, right? So like it's kind of your guide as a human dev if you want to go and dive into a large complex code base. And so our intuition was that a lot of the context that we're providing to human developers would also be useful context for AI developers to consume. And so in terms of the feature set, Cody is very similar to a lot of other assistants. It does inline autocompletion. It does code base aware chat. It does specific commands that automate, you know, tasks that you might rather not want to do like generating unit tests or adding detailed documentation. But we think the core differentiator is really the quality of the context, which is hard to kind of describe succinctly. It's a bit like saying, you know, what's the difference between Google and Alta Vista? There's not like a quick checkbox list of features that you can rattle off, but it really just comes down to all the attention and detail that we've paid to making that context work well and be high quality and fast for human devs. We're now kind of plugging into the AI coding assistant as well. Yeah. [00:09:53]Steve: I mean, just to add my own perspective on to what Beyang just described, RAG is kind of like a consultant that the LLM has available, right, that knows about your code. RAG provides basically a bridge to a lookup system for the LLM, right? Whereas fine tuning would be more like on the job training for somebody. If the LLM is a person, you know, and you send them to a new job and you do on the job training, that's what fine tuning is like, right? So tuned to our specific task. You're always going to need that expert, even if you get the on the job training, because the expert knows your particular code base, your task, right? That expert has to know your code. And there's a chicken and egg problem because, right, you know, we're like, well, I'm going to ask the LLM about my code, but first I have to explain it, right? It's this chicken and egg problem. That's where RAG comes in. And we have the best consultants, right? The best assistant who knows your code. And so when you sit down with Cody, right, what Beyang said earlier about going to Google and using code search and then starting to feel like without it, his job was super tedious. Once you start using these, do you guys use coding assistants? [00:10:53]Swyx: Yeah, right. [00:10:54]Steve: I mean, like we're getting to the point very quickly, right? Where you feel like almost like you're programming without the internet, right? Or something, you know, it's like you're programming back in the nineties without the coding assistant. Yeah. Hopefully that helps for people who have like no idea about coding systems, what they are. [00:11:09]Swyx: Yeah. [00:11:10]Alessio: I mean, going back to using them, we had a lot of them on the podcast already. We had Cursor, we have Codium and Codium, very similar names. [00:11:18]Swyx: Yeah. Find, and then of course there's Copilot. [00:11:22]Alessio: You had a Copilot versus Cody blog post, and I think it really shows the context improvement. So you had two examples that stuck with me. One was, what does this application do? And the Copilot answer was like, oh, it uses JavaScript and NPM and this. And it's like, but that's not what it does. You know, that's what it's built with. Versus Cody was like, oh, these are like the major functions. And like, these are the functionalities and things like that. And then the other one was, how do I start this up? And Copilot just said NPM start, even though there was like no start command in the package JSON, but you know, most collapse, right? Most projects use NPM start. So maybe this does too. How do you think about open source models? Because Copilot has their own private thing. And I think you guys use Starcoder, if I remember right. Yeah, that's correct. [00:12:09]Beyang: I think Copilot uses some variant of Codex. They're kind of cagey about it. I don't think they've like officially announced what model they use. [00:12:16]Swyx: And I think they use a range of models based on what you're doing. Yeah. [00:12:19]Beyang: So everyone uses a range of model. Like no one uses the same model for like inline completion versus like chat because the latency requirements for. Oh, okay. Well, there's fill in the middle. There's also like what the model's trained on. So like we actually had completions powered by Claude Instant for a while. And but you had to kind of like prompt hack your way to get it to output just the code and not like, hey, you know, here's the code you asked for, like that sort of text. So like everyone uses a range of models. We've kind of designed Cody to be like especially model, not agnostic, but like pluggable. So one of our kind of design considerations was like as the ecosystem evolves, we want to be able to integrate the best in class models, whether they're proprietary or open source into Cody because the pace of innovation in the space is just so quick. And I think that's been to our advantage. Like today, Cody uses Starcoder for inline completions. And with the benefit of the context that we provide, we actually show comparable completion acceptance rate metrics. It's kind of like the standard metric that folks use to evaluate inline completion quality. It's like if I show you a completion, what's the chance that you actually accept the completion versus you reject it? And so we're at par with Copilot, which is at the head of that industry right now. And we've been able to do that with the Starcoder model, which is open source and the benefit of the context fetching stuff that we provide. And of course, a lot of like prompt engineering and other stuff along the way. [00:13:40]Alessio: And Steve, you wrote a post called cheating is all you need about what you're building. And one of the points you made is that everybody's fighting on the same axis, which is better UI and the IDE, maybe like a better chat response. But data modes are kind of the most important thing. And you guys have like a 10 year old mode with all the data you've been collecting. How do you kind of think about what other companies are doing wrong, right? Like, why is nobody doing this in terms of like really focusing on RAG? I feel like you see so many people. Oh, we just got a new model. It's like a bit human eval. And it's like, well, but maybe like that's not what we should really be doing, you know? Like, do you think most people underestimate the importance of like the actual RAG in code? [00:14:21]Steve: I think that people weren't doing it much. It wasn't. It's kind of at the edges of AI. It's not in the center. I know that when ChatGPT launched, so within the last year, I've heard a lot of rumblings from inside of Google, right? Because they're undergoing a huge transformation to try to, you know, of course, get into the new world. And I heard that they told, you know, a bunch of teams to go and train their own models or fine tune their own models, right? [00:14:43]Swyx: Both. [00:14:43]Steve: And, you know, it was a s**t show. Nobody knew how to do it. They launched two coding assistants. One was called Code D with an EY. And then there was, I don't know what happened in that one. And then there's Duet, right? Google loves to compete with themselves, right? They do this all the time. And they had a paper on Duet like from a year ago. And they were doing exactly what Copilot was doing, which was just pulling in the local context, right? But fundamentally, I thought of this because we were talking about the splitting of the [00:15:10]Swyx: models. [00:15:10]Steve: In the early days, it was the LLM did everything. And then we realized that for certain use cases, like completions, that a different, smaller, faster model would be better. And that fragmentation of models, actually, we expected to continue and proliferate, right? Because we are fundamentally, we're a recommender engine right now. Yeah, we're recommending code to the LLM. We're saying, may I interest you in this code right here so that you can answer my question? [00:15:34]Swyx: Yeah? [00:15:34]Steve: And being good at recommender engine, I mean, who are the best recommenders, right? There's YouTube and Spotify and, you know, Amazon or whatever, right? Yeah. [00:15:41]Swyx: Yeah. [00:15:41]Steve: And they all have many, many, many, many, many models, right? For all fine-tuned for very specific, you know. And that's where we're heading in code, too. Absolutely. [00:15:50]Swyx: Yeah. [00:15:50]Alessio: We just did an episode we released on Wednesday, which we said RAG is like Rexis or like LLMs. You're basically just suggesting good content. [00:15:58]Swyx: It's like what? Recommendations. [00:15:59]Beyang: Recommendations. [00:16:00]Alessio: Oh, got it. [00:16:01]Steve: Yeah, yeah, yeah. [00:16:02]Swyx: So like the naive implementation of RAG is you embed everything, throw it in a vector database, you embed your query, and then you find the nearest neighbors, and that's your RAG. But actually, you need to rank it. And actually, you need to make sure there's sample diversity and that kind of stuff. And then you're like slowly gradient dissenting yourself towards rediscovering proper Rexis, which has been traditional ML for a long time. But like approaching it from an LLM perspective. Yeah. [00:16:24]Beyang: I almost think of it as like a generalized search problem because it's a lot of the same things. Like you want your layer one to have high recall and get all the potential things that could be relevant. And then there's typically like a layer two re-ranking mechanism that bumps up the precision and tries to get the relevant stuff to the top of the results list. [00:16:43]Swyx: Have you discovered that ranking matters a lot? Oh, yeah. So the context is that I think a lot of research shows that like one, context utilization matters based on model. Like GPT uses the top of the context window, and then apparently Claude uses the bottom better. And it's lossy in the middle. Yeah. So ranking matters. No, it really does. [00:17:01]Beyang: The skill with which models are able to take advantage of context is always going to be dependent on how that factors into the impact on the training loss. [00:17:10]Swyx: Right? [00:17:10]Beyang: So like if you want long context window models to work well, then you have to have a ton of data where it's like, here's like a billion lines of text. And I'm going to ask a question about like something that's like, you know, embedded deeply into it and like, give me the right answer. And unless you have that training set, then of course, you're going to have variability in terms of like where it attends to. And in most kind of like naturally occurring data, the thing that you're talking about right now, the thing I'm asking you about is going to be something that we talked about recently. [00:17:36]Swyx: Yeah. [00:17:36]Steve: Did you really just say gradient dissenting yourself? Actually, I love that it's entered the casual lexicon. Yeah, yeah, yeah. [00:17:44]Swyx: My favorite version of that is, you know, how we have to p-hack papers. So, you know, when you throw humans at the problem, that's called graduate student dissent. That's great. It's really awesome. [00:17:54]Alessio: I think the other interesting thing that you have is this inline assist UX that I wouldn't say async, but like it works while you can also do work. So you can ask Cody to make changes on a code block and you can still edit the same file at the same time. [00:18:07]Swyx: Yeah. [00:18:07]Alessio: How do you see that in the future? Like, do you see a lot of Cody's running together at the same time? Like, how do you validate also that they're not messing each other up as they make changes in the code? And maybe what are the limitations today? And what do you think about where the attack is going? [00:18:21]Steve: I want to start with a little history and then I'm going to turn it over to Bian, all right? So we actually had this feature in the very first launch back in June. Dominic wrote it. It was called nonstop Cody. And you could have multiple, basically, LLM requests in parallel modifying your source [00:18:37]Swyx: file. [00:18:37]Steve: And he wrote a bunch of code to handle all of the diffing logic. And you could see the regions of code that the LLM was going to change, right? And he was showing me demos of it. And it just felt like it was just a little before its time, you know? But a bunch of that stuff, that scaffolding was able to be reused for where we're inline [00:18:56]Swyx: sitting today. [00:18:56]Steve: How would you characterize it today? [00:18:58]Beyang: Yeah, so that interface has really evolved from a, like, hey, general purpose, like, request anything inline in the code and have the code update to really, like, targeted features, like, you know, fix the bug that exists at this line or request a very specific [00:19:13]Swyx: change. [00:19:13]Beyang: And the reason for that is, I think, the challenge that we ran into with inline fixes, and we do want to get to the point where you could just fire and forget and have, you know, half a dozen of these running in parallel. But I think we ran into the challenge early on that a lot of people are running into now when they're trying to construct agents, which is the reliability of, you know, working code generation is just not quite there yet in today's language models. And so that kind of constrains you to an interaction where the human is always, like, in the inner loop, like, checking the output of each response. And if you want that to work in a way where you can be asynchronous, you kind of have to constrain it to a domain where today's language models can generate reliable code well enough. So, you know, generating unit tests, that's, like, a well-constrained problem. Or fixing a bug that shows up as, like, a compiler error or a test error, that's a well-constrained problem. But the more general, like, hey, write me this class that does X, Y, and Z using the libraries that I have, that is not quite there yet, even with the benefit of really good context. Like, it definitely moves the needle a lot, but we're not quite there yet to the point where you can just fire and forget. And I actually think that this is something that people don't broadly appreciate yet, because I think that, like, everyone's chasing this dream of agentic execution. And if we're to really define that down, I think it implies a couple things. You have, like, a multi-step process where each step is fully automated. We don't have to have a human in the loop every time. And there's also kind of like an LM call at each stage or nearly every stage in that [00:20:45]Swyx: chain. [00:20:45]Beyang: Based on all the work that we've done, you know, with the inline interactions, with kind of like general Codyfeatures for implementing longer chains of thought, we're actually a little bit more bearish than the average, you know, AI hypefluencer out there on the feasibility of agents with purely kind of like transformer-based models. To your original question, like, the inline interactions with CODI, we actually constrained it to be more targeted, like, you know, fix the current error or make this quick fix. I think that that does differentiate us from a lot of the other tools on the market, because a lot of people are going after this, like, shnazzy, like, inline edit interaction, whereas I think where we've moved, and this is based on the user feedback that we've gotten, it's like that sort of thing, it demos well, but when you're actually coding day to day, you don't want to have, like, a long chat conversation inline with the code base. That's a waste of time. You'd rather just have it write the right thing and then move on with your life or not have to think about it. And that's what we're trying to work towards. [00:21:37]Steve: I mean, yeah, we're not going in the agent direction, right? I mean, I'll believe in agents when somebody shows me one that works. Yeah. Instead, we're working on, you know, sort of solidifying our strength, which is bringing the right context in. So new context sources, ways for you to plug in your own context, ways for you to control or influence the context, you know, the mixing that happens before the request goes out, etc. And there's just so much low-hanging fruit left in that space that, you know, agents seems like a little bit of a boondoggle. [00:22:03]Beyang: Just to dive into that a little bit further, like, I think, you know, at a very high level, what do people mean when they say agents? They really mean, like, greater automation, fully automated, like, the dream is, like, here's an issue, go implement that. And I don't have to think about it as a human. And I think we are working towards that. Like, that is the eventual goal. I think it's specifically the approach of, like, hey, can we have a transformer-based LM alone be the kind of, like, backbone or the orchestrator of these agentic flows? Where we're a little bit more bearish today. [00:22:31]Swyx: You want the human in the loop. [00:22:32]Beyang: I mean, you kind of have to. It's just a reality of the behavior of language models that are purely, like, transformer-based. And I think that's just like a reflection of reality. And I don't think people realize that yet. Because if you look at the way that a lot of other AI tools have implemented context fetching, for instance, like, you see this in the Copilot approach, where if you use, like, the at-workspace thing that supposedly provides, like, code-based level context, it has, like, an agentic approach where you kind of look at how it's behaving. And it feels like they're making multiple requests to the LM being like, what would you do in this case? Would you search for stuff? What sort of files would you gather? Go and read those files. And it's like a multi-hop step, so it takes a long while. It's also non-deterministic. Because any sort of, like, LM invocation, it's like a dice roll. And then at the end of the day, the context it fetches is not that good. Whereas our approach is just like, OK, let's do some code searches that make sense. And then maybe, like, crawl through the reference graph a little bit. That is fast. That doesn't require any sort of LM invocation at all. And we can pull in much better context, you know, very quickly. So it's faster. [00:23:37]Swyx: It's more reliable. [00:23:37]Beyang: It's deterministic. And it yields better context quality. And so that's what we think. We just don't think you should cargo cult or naively go like, you know, agents are the [00:23:46]Swyx: future. [00:23:46]Beyang: Let's just try to, like, implement agents on top of the LM that exists today. I think there are a couple of other technologies or approaches that need to be refined first before we can get into these kind of, like, multi-stage, fully automated workflows. [00:24:00]Swyx: It makes sense. You know, we're very much focused on developer inner loop right now. But you do see things eventually moving towards developer outer loop. Yeah. So would you basically say that they're tackling the agent's problem that you don't want to tackle? [00:24:11]Beyang: No, I would say at a high level, we are after maybe, like, the same high level problem, which is like, hey, I want some code written. I want to develop some software and can automate a system. Go build that software for me. I think the approaches might be different. So I think the analogy in my mind is, I think about, like, the AI chess players. Coding, in some senses, I mean, it's similar and dissimilar to chess. I think one question I ask is, like, do you think producing code is more difficult than playing chess or less difficult than playing chess? More. [00:24:41]Swyx: I think more. [00:24:41]Beyang: Right. And if you look at the best AI chess players, like, yes, you can use an LLM to play chess. Like, people have showed demos where it's like, oh, like, yeah, GPT-4 is actually a pretty decent, like, chess move suggester. Right. But you would never build, like, a best in class chess player off of GPT-4 alone. [00:24:57]Swyx: Right. [00:24:57]Beyang: Like, the way that people design chess players is that you have kind of like a search space and then you have a way to explore that search space efficiently. There's a bunch of search algorithms, essentially. We were doing tree search in various ways. And you can have heuristic functions, which might be powered by an LLM. [00:25:12]Swyx: Right. [00:25:12]Beyang: Like, you might use an LLM to generate proposals in that space that you can efficiently explore. But the backbone is still this kind of more formalized tree search based approach rather than the LLM itself. And so I think my high level intuition is that, like, the way that we get to more reliable multi-step workflows that do things beyond, you know, generate unit test, it's really going to be like a search based approach where you use an LLM as kind of like an advisor or a proposal function, sort of your heuristic function, like the ASTAR search algorithm. But it's probably not going to be the thing that is the backbone, because I guess it's not the right tool for that. Yeah. [00:25:50]Swyx: I can see yourself kind of thinking through this, but not saying the words, the sort of philosophical Peter Norvig type discussion. Maybe you want to sort of introduce that in software. Yeah, definitely. [00:25:59]Beyang: So your listeners are savvy. They're probably familiar with the classic like Chomsky versus Norvig debate. [00:26:04]Swyx: No, actually, I wanted, I was prompting you to introduce that. Oh, got it. [00:26:08]Beyang: So, I mean, if you look at the history of artificial intelligence, right, you know, it goes way back to, I don't know, it's probably as old as modern computers, like 50s, 60s, 70s. People are debating on like, what is the path to producing a sort of like general human level of intelligence? And kind of two schools of thought that emerged. One is the Norvig school of thought, which roughly speaking includes large language models, you know, regression, SVN, basically any model that you kind of like learn from data. And it's like data driven. Most of machine learning would fall under this umbrella. And that school of thought says like, you know, just learn from the data. That's the approach to reaching intelligence. And then the Chomsky approach is more things like compilers and parsers and formal systems. So basically like, let's think very carefully about how to construct a formal, precise system. And that will be the approach to how we build a truly intelligent system. I think Lisp was invented so that you could create like rules-based systems that you would call AI. As a language. Yeah. And for a long time, there was like this debate, like there's certain like AI research labs that were more like, you know, in the Chomsky camp and others that were more in the Norvig camp. It's a debate that rages on today. And I feel like the consensus right now is that, you know, Norvig definitely has the upper hand right now with the advent of LMs and diffusion models and all the other recent progress in machine learning. But the Chomsky-based stuff is still really useful in my view. I mean, it's like parsers, compilers, basically a lot of the stuff that provides really good context. It provides kind of like the knowledge graph backbone that you want to explore with your AI dev tool. Like that will come from kind of like Chomsky-based tools like compilers and parsers. It's a lot of what we've invested in in the past decade at Sourcegraph and what you build with Grok. Basically like these formal systems that construct these very precise knowledge graphs that are great context providers and great kind of guard rails enforcers and kind of like safety checkers for the output of a more kind of like data-driven, fuzzier system that uses like the Norvig-based models. [00:28:03]Steve: Jang was talking about this stuff like it happened in the middle ages. Like, okay, so when I was in college, I was in college learning Lisp and prologue and planning and all the deterministic Chomsky approaches to AI. And I was there when Norvig basically declared it dead. I was there 3,000 years ago when Norvig and Chomsky fought on the volcano. When did he declare it dead? [00:28:26]Swyx: What do you mean he declared it dead? [00:28:27]Steve: It was like late 90s. [00:28:29]Swyx: Yeah. [00:28:29]Steve: When I went to Google, Peter Norvig was already there. He had basically like, I forget exactly where. It was some, he's got so many famous short posts, you know, amazing. [00:28:38]Swyx: He had a famous talk, the unreasonable effectiveness of data. Yeah. [00:28:41]Steve: Maybe that was it. But at some point, basically, he basically convinced everybody that deterministic approaches had failed and that heuristic-based, you know, data-driven statistical approaches, stochastic were better. [00:28:52]Swyx: Yeah. [00:28:52]Steve: The primary reason I can tell you this, because I was there, was that, was that, well, the steam-powered engine, no. The reason was that the deterministic stuff didn't scale. [00:29:06]Swyx: Yeah. Right. [00:29:06]Steve: They're using prologue, man, constraint systems and stuff like that. Well, that was a long time ago, right? Today, actually, these Chomsky-style systems do scale. And that's, in fact, exactly what Sourcegraph has built. Yeah. And so we have a very unique, I love the framing that Bjong's made, that the marriage of the Chomsky and the Norvig, you know, sort of models, you know, conceptual models, because we, you know, we have both of them and they're both really important. And in fact, there, there's this really interesting, like, kind of overlap between them, right? Where like the AI or our graph or our search engine could potentially provide the right context for any given query, which is, of course, why ranking is important. But what we've really signed ourselves up for is an extraordinary amount of testing. [00:29:45]Swyx: Yeah. [00:29:45]Steve: Because in SWIGs, you were saying that, you know, GPT-4 tends to the front of the context window and maybe other LLMs to the back and maybe, maybe the LLM in the middle. [00:29:53]Swyx: Yeah. [00:29:53]Steve: And so that means that, you know, if we're actually like, you know, verifying whether we, you know, some change we've made has improved things, we're going to have to test putting it at the beginning of the window and at the end of the window, you know, and maybe make the right decision based on the LLM that you've chosen. Which some of our competitors, that's a problem that they don't have, but we meet you, you know, where you are. Yeah. And we're, just to finish, we're writing tens of thousands. We're generating tests, you know, fill in the middle type tests and things. And then using our graph to basically sort of fine tune Cody's behavior there. [00:30:20]Swyx: Yeah. [00:30:21]Beyang: I also want to add, like, I have like an internal pet name for this, like kind of hybrid architecture that I'm trying to make catch on. Maybe I'll just say it here. Just saying it publicly kind of makes it more real. But like, I call the architecture that we've developed the Normsky architecture. [00:30:36]Swyx: Yeah. [00:30:36]Beyang: I mean, it's obviously a portmanteau of Norvig and Chomsky, but the acronym, it stands for non-agentic, rapid, multi-source code intelligence. So non-agentic because... Rolls right off the tongue. And Normsky. But it's non-agentic in the sense that like, we're not trying to like pitch you on kind of like agent hype, right? Like it's the things it does are really just developer tools developers have been using for decades now, like parsers and really good search indexes and things like that. Rapid because we place an emphasis on speed. We don't want to sit there waiting for kind of like multiple LLM requests to return to complete a simple user request. Multi-source because we're thinking broadly about what pieces of information and knowledge are useful context. So obviously starting with things that you can search in your code base, and then you add in the reference graph, which kind of like allows you to crawl outward from those initial results. But then even beyond that, you know, sources of information, like there's a lot of knowledge that's embedded in docs, in PRDs or product specs, in your production logging system, in your chat, in your Slack channel, right? Like there's so much context is embedded there. And when you're a human developer, and you're trying to like be productive in your code base, you're going to go to all these different systems to collect the context that you need to figure out what code you need to write. And I don't think the AI developer will be any different. It will need to pull context from all these different sources. So we're thinking broadly about how to integrate these into Codi. We hope through kind of like an open protocol that like others can extend and implement. And this is something else that should be accessible by December 14th in kind of like a preview stage. But that's really about like broadening this notion of the code graph beyond your Git repository to all the other sources where technical knowledge and valuable context can live. [00:32:21]Steve: Yeah, it becomes an artifact graph, right? It can link into your logs and your wikis and any data source, right? [00:32:27]Alessio: How do you guys think about the importance of, it's almost like data pre-processing in a way, which is bring it all together, tie it together, make it ready. Any thoughts on how to actually make that good? Some of the innovation you guys have made. [00:32:40]Steve: We talk a lot about the context fetching, right? I mean, there's a lot of ways you could answer this question. But, you know, we've spent a lot of time just in this podcast here talking about context fetching. But stuffing the context into the window is, you know, the bin packing problem, right? Because the window is not big enough, and you've got more context than you can fit. You've got a ranker maybe. But what is that context? Is it a function that was returned by an embedding or a graph call or something? Do you need the whole function? Or do you just need, you know, the top part of the function, this expression here, right? You know, so that art, the golf game of trying to, you know, get each piece of context down into its smallest state, possibly even summarized by another model, right, before it even goes to the LLM, becomes this is the game that we're in, yeah? And so, you know, recursive summarization and all the other techniques that you got to use to like stuff stuff into that context window become, you know, critically important. And you have to test them across every configuration of models that you could possibly need. [00:33:32]Beyang: I think data preprocessing is probably the like unsexy, way underappreciated secret to a lot of the cool stuff that people are shipping today. Whether you're doing like RAG or fine tuning or pre-training, like the preprocessing step matters so much because it's basically garbage in, garbage out, right? Like if you're feeding in garbage to the model, then it's going to output garbage. Concretely, you know, for code RAG, if you're not doing some sort of like preprocessing that takes advantage of a parser and is able to like extract the key components of a particular file of code, you know, separate the function signature from the body, from the doc string, what are you even doing? Like that's like table stakes. It opens up so much more possibilities with which you can kind of like tune your system to take advantage of the signals that come from those different parts of the code. Like we've had a tool, you know, since computers were invented that understands the structure of source code to a hundred percent precision. The compiler knows everything there is to know about the code in terms of like structure. Like why would you not want to use that in a system that's trying to generate code, answer questions about code? You shouldn't throw that out the window just because now we have really good, you know, data-driven models that can do other things. [00:34:44]Steve: Yeah. When I called it a data moat, you know, in my cheating post, a lot of people were confused, you know, because data moat sort of sounds like data lake because there's data and water and stuff. I don't know. And so they thought that we were sitting on this giant mountain of data that we had collected, but that's not what our data moat is. It's really a data pre-processing engine that can very quickly and scalably, like basically dissect your entire code base in a very small, fine-grained, you know, semantic unit and then serve it up. Yeah. And so it's really, it's not a data moat. It's a data pre-processing moat, I guess. [00:35:15]Beyang: Yeah. If anything, we're like hypersensitive to customer data privacy requirements. So it's not like we've taken a bunch of private data and like, you know, trained a generally available model. In fact, exactly the opposite. A lot of our customers are choosing Cody over Copilot and other competitors because we have an explicit guarantee that we don't do any of that. And that we've done that from day one. Yeah. I think that's a very real concern in today's day and age, because like if your proprietary IP finds its way into the training set of any model, it's very easy both to like extract that knowledge from the model and also use it to, you know, build systems that kind of work on top of the institutional knowledge that you've built up. [00:35:52]Alessio: About a year ago, I wrote a post on LLMs for developers. And one of the points I had was maybe the depth of like the DSL. I spent most of my career writing Ruby and I love Ruby. It's so nice to use, but you know, it's not as performant, but it's really easy to read, right? And then you look at other languages, maybe they're faster, but like they're more verbose, you know? And when you think about efficiency of the context window, that actually matters. [00:36:15]Swyx: Yeah. [00:36:15]Alessio: But I haven't really seen a DSL for models, you know? I haven't seen like code being optimized to like be easier to put in a model context. And it seems like your pre-processing is kind of doing that. Do you see in the future, like the way we think about the DSL and APIs and kind of like service interfaces be more focused on being context friendly, where it's like maybe it's harder to read for the human, but like the human is never going to write it anyway. We were talking on the Hacks podcast. There are like some data science things like spin up the spandex, like humans are never going to write again because the models can just do very easily. Yeah, curious to hear your thoughts. [00:36:51]Steve: Well, so DSLs, they involve, you know, writing a grammar and a parser and they're like little languages, right? We do them that way because, you know, we need them to compile and humans need to be able to read them and so on. The LLMs don't need that level of structure. You can throw any pile of crap at them, you know, more or less unstructured and they'll deal with it. So I think that's why a DSL hasn't emerged for sort of like communicating with the LLM or packaging up the context or anything. Maybe it will at some point, right? We've got, you know, tagging of context and things like that that are sort of peeking into DSL territory, right? But your point on do users, you know, do people have to learn DSLs like regular expressions or, you know, pick your favorite, right? XPath. I think you're absolutely right that the LLMs are really, really good at that. And I think you're going to see a lot less of people having to slave away learning these things. They just have to know the broad capabilities and the LLM will take care of the rest. [00:37:42]Swyx: Yeah, I'd agree with that. [00:37:43]Beyang: I think basically like the value profit of DSL is that it makes it easier to work with a lower level language, but at the expense of introducing an abstraction layer. And in many cases today, you know, without the benefit of AI cogeneration, like that totally worth it, right? With the benefit of AI cogeneration, I mean, I don't think all DSLs will go away. I think there's still, you know, places where that trade-off is going to be worthwhile. But it's kind of like how much of source code do you think is going to be generated through natural language prompting in the future? Because in a way, like any programming language is just a DSL on top of assembly, right? And so if people can do that, then yeah, like maybe for a large portion of the code [00:38:21]Swyx: that's written, [00:38:21]Beyang: people don't actually have to understand the DSL that is Ruby or Python or basically any other programming language that exists. [00:38:28]Steve: I mean, seriously, do you guys ever write SQL queries now without using a model of some sort? At least a draft. [00:38:34]Swyx: Yeah, right. [00:38:36]Steve: And so we have kind of like, you know, past that bridge, right? [00:38:39]Alessio: Yeah, I think like to me, the long-term thing is like, is there ever going to be, you don't actually see the code, you know? It's like, hey, the basic thing is like, hey, I need a function to some two numbers and that's it. I don't need you to generate the code. [00:38:53]Steve: And the following question, do you need the engineer or the paycheck? [00:38:56]Swyx: I mean, right? [00:38:58]Alessio: That's kind of the agent's discussion in a way where like you cannot automate the agents, but like slowly you're getting more of the atomic units of the work kind of like done. I kind of think of it as like, you know, [00:39:09]Beyang: do you need a punch card operator to answer that for you? And so like, I think we're still going to have people in the role of a software engineer, but the portion of time they spend on these kinds of like low-level, tedious tasks versus the higher level, more creative tasks is going to shift. [00:39:23]Steve: No, I haven't used punch cards. [00:39:25]Swyx: Yeah, I've been talking about like, so we kind of made this podcast about the sort of rise of the AI engineer. And like the first step is the AI enhanced engineer. That is that software developer that is no longer doing these routine, boilerplate-y type tasks, because they're just enhanced by tools like yours. So you mentioned OpenCodeGraph. I mean, that is a kind of DSL maybe, and because we're releasing this as you go GA, you hope for other people to take advantage of that? [00:39:52]Beyang: Oh yeah, I would say so OpenCodeGraph is not a DSL. It's more of a protocol. It's basically like, hey, if you want to make your system, whether it's, you know, chat or logging or whatever accessible to an AI developer tool like Cody, here's kind of like the schema by which you can provide that context and offer hints. So I would, you know, comparisons like LSP obviously did this for kind of like standard code intelligence. It's kind of like a lingua franca for providing fine references and codefinition. There's kind of like analogs to that. There might be also analogs to kind of the original OpenAI, kind of like plugins, API. There's all this like context out there that might be useful for an LM-based system to consume. And so at a high level, what we're trying to do is define a common language for context providers to provide context to other tools in the software development lifecycle. Yeah. Do you have any critiques of LSP, by the way, [00:40:42]Swyx: since like this is very much, very close to home? [00:40:45]Steve: One of the authors wrote a really good critique recently. Yeah. I don't think I saw that. Yeah, yeah. LSP could have been better. It just came out a couple of weeks ago. It was a good article. [00:40:54]Beyang: Yeah. I think LSP is great. Like for what it did for the developer ecosystem, it was absolutely fantastic. Like nowadays, like it's much easier now to get code navigation up and running in a bunch of editors by speaking this protocol. I think maybe the interesting question is like looking at the different design decisions comparing LSP basically with Kythe. Because Kythe has more of a... How would you describe it? [00:41:18]Steve: A storage format. [00:41:20]Beyang: I think the critique of LSP from a Kythe point of view would be like with LSP, you don't actually have an actual symbolic model of the code. It's not like LSP models like, hey, this function calls this other function. LSP is all like range-based. Like, hey, your cursor's at line 32, column 1. [00:41:35]Swyx: Yeah. [00:41:35]Beyang: And that's the thing you feed into the language server. And then it's like, okay, here's the range that you should jump to if you click on that range. So it kind of is intentionally ignorant of the fact that there's a thing called a reference underneath your cursor, and that's linked to a symbol definition. [00:41:49]Steve: Well, actually, that's the worst example you could have used. You're right. But that's the one thing that it actually did bake in is following references. [00:41:56]Swyx: Sure. [00:41:56]Steve: But it's sort of hardwired. [00:41:58]Swyx: Yeah. [00:41:58]Steve: Whereas Kythe attempts to model [00:42:00]Beyang: like all these things explicitly. [00:42:02]Swyx: And so... [00:42:02]Steve: Well, so LSP is a protocol, right? And so Google's internal protocol is gRPC-based. And it's a different approach than LSP. It's basically you make a heavy query to the back end, and you get a lot of data back, and then you render the whole page, you know? So we've looked at LSP, and we think that it's a little long in the tooth, right? I mean, it's a great protocol, lots and lots of support for it. But we need to push into the domain of exposing the intelligence through the protocol. Yeah. [00:42:29]Beyang: And so I would say we've developed a protocol of our own called Skip, which is at a very high level trying to take some of the good ideas from LSP and from Kythe and merge that into a system that in the near term is useful for Sourcegraph, but I think in the long term, we hope will be useful for the ecosystem. Okay, so here's what LSP did well. LSP, by virtue of being like intentionally dumb, dumb in air quotes, because I'm not like ragging on it, allowed language servers developers to kind of like bypass the hard problem of like modeling language semantics precisely. So like if all you want to do is jump to definition, you don't have to come up with like a universally unique naming scheme for each symbol, which is actually quite challenging because you have to think about like, okay, what's the top scope of this name? Is it the source code repository? Is it the package? Does it depend on like what package server you're fetching this from? Like whether it's the public one or the one inside your... Anyways, like naming is hard, right? And by just going from kind of like a location to location based approach, you basically just like throw that out the window. All I care about is jumping definition, just make that work. And you can make that work without having to deal with like all the complex global naming things. The limitation of that approach is that it's harder to build on top of that to build like a true knowledge graph. Like if you actually want a system that says like, okay, here's the web of functions and here's how they reference each other. And I want to incorporate that like semantic model of how the code operates or how the code relates to each other at like a static level. You can't do that with LSP because you have to deal with line ranges. And like concretely the pain point that we found in using LSP for source graph is like in order to do like a find references [00:44:04]Swyx: and then jump definitions, [00:44:04]Beyang: it's like a multi-hop process because like you have to jump to the range and then you have to find the symbol at that range. And it just adds a lot of latency and complexity of these operations where as a human, you're like, well, this thing clearly references this other thing. Why can't you just jump me to that? And I think that's the thing that Kaith does well. But then I think the issue that Kaith has had with adoption is because it is more sophisticated schema, I think. And so there's basically more things that you have to implement to get like a Kaith implementation up and running. I hope I'm not like, correct me if I'm wrong about any of this. [00:44:35]Steve: 100%, 100%. Kaith also has a problem, all these systems have the problem, even skip, or at least the way that we implemented the indexers, that they have to integrate with your build system in order to build that knowledge graph, right? Because you have to basically compile the code in a special mode to generate artifacts instead of binaries. And I would say, by the way, earlier I was saying that XREFs were in LSP, but it's actually, I was thinking of LSP plus LSIF. [00:44:58]Swyx: Yeah. That's another. [00:45:01]Steve: Which is actually bad. We can say that it's bad, right? [00:45:04]Steve: It's like skip or Kaith, it's supposed to be sort of a model serialization, you know, for the code graph, but it basically just does what LSP needs, the bare minimum. LSIF is basically if you took LSP [00:45:16]Beyang: and turned that into a serialization format. So like you build an index for language servers to kind of like quickly bootstrap from cold start. But it's a graph model [00:45:23]Steve: with all of the inconvenience of the API without an actual graph. And so, yeah. [00:45:29]Beyang: So like one of the things that we try to do with skip is try to capture the best of both worlds. So like make it easy to write an indexer, make the schema simple, but also model some of the more symbolic characteristics of the code that would allow us to essentially construct this knowledge graph that we can then make useful for both the human developer through SourceGraph and through the AI developer through Cody. [00:45:49]Steve: So anyway, just to finish off the graph comment, we've got a new graph, yeah, that's skip based. We call it BFG internally, right? It's a beautiful something graph. A big friendly graph. [00:46:00]Swyx: A big friendly graph. [00:46:01]Beyang: It's a blazing fast. [00:46:02]Steve: Blazing fast. [00:46:03]Swyx: Blazing fast graph. [00:46:04]Steve: And it is blazing fast, actually. It's really, really interesting. I should probably have to do a blog post about it to walk you through exactly how they're doing it. Oh, please. But it's a very AI-like iterative, you know, experimentation sort of approach. We're building a code graph based on all of our 10 years of knowledge about building code graphs, yeah? But we're building it quickly with zero configuration, and it doesn't have to integrate with your build. And through some magic tricks that we have. And so what just happens when you install the plugin, that it'll be there and indexing your code and providing that knowledge graph in the background without all that build system integration. This is a bit of secret sauce that we haven't really like advertised it very much lately. But I am super excited about it because what they do is they say, all right, you know, let's tackle function parameters today. Cody's not doing a very good job of completing function call arguments or function parameters in the definition, right? Yeah, we generate those thousands of tests, and then we can actually reuse those tests for the AI context as well. So fortunately, things are kind of converging on, we have, you know, half a dozen really, really good context sources, and we mix them all together. So anyway, BFG, you're going to hear more about it probably in the holidays? [00:47:12]Beyang: I think it'll be online for December 14th. We'll probably mention it. BFG is probably not the public name we're going to go with. I think we might call it like Graph Context or something like that. [00:47:20]Steve: We're officially calling it BFG. [00:47:22]Swyx: You heard it here first. [00:47:24]Beyang: BFG is just kind of like the working name. And so the impetus for BFG was like, if you look at like current AI inline code completion tools and the errors that they make, a lot of the errors that they make, even in kind of like the easy, like single line case, are essentially like type errors, right? Like you're trying to complete a function call and it suggests a variable that you defined earlier, but that variable is the wrong type. [00:47:47]Swyx: And that's the sort of thing [00:47:47]Beyang: where it's like a first year, like freshman CS student would not make that error, right? So like, why does the AI make that error? And the reason is, I mean, the AI is just suggesting things that are plausible without the context of the types or any other like broader files in the code. And so the kind of intuition here is like, why don't we just do the basic thing that like any baseline intelligent human developer would do, which is like click jump to definition, click some fine references and pull in that like Graph Context into the context window and then have it generate the completion. So like that's sort of like the MVP of what BFG was. And turns out that works really well. Like you can eliminate a lot of type errors that AI coding tools make just by pulling in that context. Yeah, but the graph is definitely [00:48:32]Steve: our Chomsky side. [00:48:33]Swyx: Yeah, exactly. [00:48:34]Beyang: So like this like Chomsky-Norvig thing, I think pops up in a bunch of differ

Hi friends, we're on hiatus for the fall. To tide you over, we're putting up some favorite episodes from our archives. Enjoy! ---- [originally aired February 17, 2021] Guess what folks: we are celebrating a birthday this week. That's right, Many Minds has reached the ripe age of one year old. Not sure how old that is in podcast years, exactly, but it's definitely a landmark that we're proud of. Please no gifts, but, as always, you're encouraged to share the show with a friend, write a review, or give us a shout out on social. To help mark this milestone we've got a great episode for you. My guest is the writer, Brian Christian. Brian is a visiting scholar at the University of California Berkeley and the author of three widely acclaimed books: The Most Human Human, published in 2011; Algorithms To Live By, co-authored with Tom Griffiths and published in 2016; and most recently, The Alignment Problem. It was published this past fall and it's the focus of our conversation in this episode. The alignment problem, put simply, is the problem of building artificial intelligences—machine learning systems, for instance—that do what we want them to do, that both reflect and further our values. This is harder to do than you might think, and it's more important than ever. As Brian and I discuss, machine learning is becoming increasingly pervasive in everyday life—though it's sometimes invisible. It's working in the background every time we snap a photo or hop on Facebook. Companies are using it to sift resumes; courts are using it to make parole decisions. We are already trusting these systems with a bunch of important tasks, in other words. And as we rely on them in more and more domains, the alignment problem will only become that much more pressing. In the course of laying out this problem, Brian's book also offers a captivating history of machine learning and AI. Since their very beginnings, these fields have been formed through interaction with philosophy, psychology, mathematics, and neuroscience. Brian traces these interactions in fascinating detail—and brings them right up to the present moment. As he describes, machine learning today is not only informed by the latest advances in the cognitive sciences, it's also propelling those advances. This is a wide-ranging and illuminating conversation folks. And, if I may say so, it's also an important one. Brian makes a compelling case, I think, that the alignment problem is one of the defining issues of our age. And he writes about it—and talks about it here—with such clarity and insight. I hope you enjoy this one. And, if you do, be sure to check out Brian's book. Happy birthday to us—and on to my conversation with Brian Christian. Enjoy!   A transcript of this show is available here.   Notes and links 7:26 - Norbert Wiener's article from 1960, ‘Some moral and technical consequences of automation'. 8:35 - ‘The Sorcerer's Apprentice' is an episode from the animated film, Fantasia (1940). Before that, it was a poem by Goethe. 13:00 - A well-known incident in which Google's nascent auto-tagging function went terribly awry. 13:30 - The ‘Labeled Faces in the Wild' database can be viewed here. 18:35 - A groundbreaking article in ProPublica on the biases inherent in the Correctional Offender Management Profiling for Alternative Sanctions (COMPAS) tool. 25:00 – The website of the Future of Humanity Institute, mentioned in several places, is here. 25:55 - For an account of the collaboration between Walter Pitts and Warren McCulloch, see here. 29:35- An article about the racial biases built into photographic film technology in the 20th century. 31:45 - The much-investigated Tempe crash involving a driverless car and a pedestrian: 37:17 - The psychologist Edward Thorndike developed the “law of effect.” Here is one of his papers on the law. 44:40 - A highly influential 2015 paper in Nature in which a deep-Q network was able to surpass human performance on a number of classic Atari games, and yet not score a single point on ‘Montezuma's Revenge.' 47:38 - A chapter on the classic “preferential looking” paradigm in developmental psychology: 53:40 - A blog post discussing the relationship between dopamine in the brain and temporal difference learning. Here is the paper in Science in which this relationship was first articulated. 1:00:00 - A paper on the concept of “coherent extrapolated volition.” 1:01:40 - An article on the notion of “iterated distillation and amplification.” 1:10:15 - The fourth edition of a seminal textbook by Stuart Russell and Peter Norvig, AI a Modern approach, is available here: http://aima.cs.berkeley.edu/ 1:13:00 - An article on Warren McCulloch's poetry. 1:17:45 - The concept of “reductions” is central in computer science and mathematics.   Brian Christian's end-of-show reading recommendations: The Alignment Newsletter, written by Rohin Shah Invisible Women, by Caroline Criado Perez: The Gardener and the Carpenter, Alison Gopnik: You can keep up with Brian at his personal website or on Twitter.   Many Minds is a project of the Diverse Intelligences Summer Institute, which is made possible by a generous grant from the Templeton World Charity Foundation to UCLA. It is hosted and produced by Kensy Cooperrider, with help from Assistant Producer Urte Laukaityte and with creative support from DISI Directors Erica Cartmill and Jacob Foster. Our artwork is by Ben Oldroyd. Our transcripts are created by Sarah Dopierala. Subscribe to Many Minds on Apple, Stitcher, Spotify, Pocket Casts, Google Play, or wherever you listen to podcasts. You can also now subscribe to the Many Minds newsletter here! We welcome your comments, questions, and suggestions. Feel free to email us at: manymindspodcast@gmail.com.  For updates about the show, visit our website or follow us on Twitter: @ManyMindsPod.

TecHype is a groundbreaking series that cuts through the hype around emerging technologies. Each episode debunks misunderstandings around emerging tech, provides insight into benefits and risks, and identifies technical and policy strategies to harness the benefits while mitigating the risks. This episode of TecHype features Prof. Stuart Russell from UC Berkeley, a world-renowned expert in artificial intelligence and co-author (with Peter Norvig) of the standard text in the field. We debunk misunderstandings around what “AI” actually is and break down the benefits and risks of this transformative technology. Prof. Russell provides an expert perspective on the real impacts AI will have in our world, including its potential to provide greater efficiency and effectiveness in a variety of domains and the serious safety, security, and discrimination risks it poses. Series: "UC Public Policy Channel" [Public Affairs] [Science] [Show ID: 39284]

TecHype is a groundbreaking series that cuts through the hype around emerging technologies. Each episode debunks misunderstandings around emerging tech, provides insight into benefits and risks, and identifies technical and policy strategies to harness the benefits while mitigating the risks. This episode of TecHype features Prof. Stuart Russell from UC Berkeley, a world-renowned expert in artificial intelligence and co-author (with Peter Norvig) of the standard text in the field. We debunk misunderstandings around what “AI” actually is and break down the benefits and risks of this transformative technology. Prof. Russell provides an expert perspective on the real impacts AI will have in our world, including its potential to provide greater efficiency and effectiveness in a variety of domains and the serious safety, security, and discrimination risks it poses. Series: "UC Public Policy Channel" [Public Affairs] [Science] [Show ID: 39284]

TecHype is a groundbreaking series that cuts through the hype around emerging technologies. Each episode debunks misunderstandings around emerging tech, provides insight into benefits and risks, and identifies technical and policy strategies to harness the benefits while mitigating the risks. This episode of TecHype features Prof. Stuart Russell from UC Berkeley, a world-renowned expert in artificial intelligence and co-author (with Peter Norvig) of the standard text in the field. We debunk misunderstandings around what “AI” actually is and break down the benefits and risks of this transformative technology. Prof. Russell provides an expert perspective on the real impacts AI will have in our world, including its potential to provide greater efficiency and effectiveness in a variety of domains and the serious safety, security, and discrimination risks it poses. Series: "UC Public Policy Channel" [Public Affairs] [Science] [Show ID: 39284]

TecHype is a groundbreaking series that cuts through the hype around emerging technologies. Each episode debunks misunderstandings around emerging tech, provides insight into benefits and risks, and identifies technical and policy strategies to harness the benefits while mitigating the risks. This episode of TecHype features Prof. Stuart Russell from UC Berkeley, a world-renowned expert in artificial intelligence and co-author (with Peter Norvig) of the standard text in the field. We debunk misunderstandings around what “AI” actually is and break down the benefits and risks of this transformative technology. Prof. Russell provides an expert perspective on the real impacts AI will have in our world, including its potential to provide greater efficiency and effectiveness in a variety of domains and the serious safety, security, and discrimination risks it poses. Series: "UC Public Policy Channel" [Public Affairs] [Science] [Show ID: 39284]

Stuart Jonathan Russell is a British computer scientist known for his contributions to artificial intelligence. He is a professor of computer science at the University of California, Berkeley and was from 2008 to 2011 an adjunct professor of neurological surgery at the University of California, San Francisco. He holds the Smith-Zadeh Chair in Engineering at the University of California, Berkeley. He founded and leads the Center for Human-Compatible Artificial Intelligence (CHAI) at UC Berkeley. Russell is the co-author with Peter Norvig of the most popular textbook in the field of AI: Artificial Intelligence: A Modern Approach used in more than 1,500 universities in 135 countries.SummaryThis episode explores the development of artificial intelligence (AI) and its potential impact on humanity. Russel emphasizes the importance of aligning AI systems with human values to ensure their compatibility and avoid unintended consequences. He discusses the risks associated with AI development and proposes the use of provably beneficial AI, which prioritizes human values and addresses concerns such as safety and control. Russel argues for the need to reframe AI research and policymaking to prioritize human well-being and ethical considerations. Computation: Intelligent Cooperation Foresight Group– Info and apply to join!  https://foresight.org/technologies/computation-intelligent-cooperation/The Foresight Institute is a research organization and non-profit that supports the beneficial development of high-impact technologies. Since our founding in 1987 on a vision of guiding powerful technologies, we have continued to evolve into a many-armed organization that focuses on several fields of science and technology that are too ambitious for legacy institutions to support.Allison Duettmann is the president and CEO of Foresight Institute. She directs the Intelligent Cooperation, Molecular Machines, Biotech & Health Extension, Neurotech, and Space Programs, Fellowships, Prizes, and Tech Trees, and shares this work with the public. She founded Existentialhope.com, co-edited Superintelligence: Coordination & Strategy, co-authored Gaming the Future, and co-initiated The Longevity Prize. Apply to Foresight's virtual salons and in person workshops here!We are entirely funded by your donations. If you enjoy what we do please consider donating through our donation page.Visit our website for more content, or join us here:TwitterFacebookLinkedInEvery word ever spoken on this podcast is now AI-searchable using Fathom.fm, a search engine for podcasts.  Hosted on Acast. See acast.com/privacy for more information.

Peter Norvig is a big deal in AI. He's currently at Stanford's Human-Centered AI Institute. But he built his reputation mainly at Google, where he helped shape the most powerful and widely used search engine today. What does an AI expert have to say about higher education's adoption of new technologies, the dangers of AI-assisted cheating, and the bigger question of academia's purpose?   Higher Ed Spotlight is sponsored by Chegg's Center for Digital Learning and aims to explore the future of higher education. It is produced by Antica Productions.

This podcast is a commentary and does not contain any copyrighted material of the reference source. We strongly recommend accessing/buying the reference source at the same time. ■Reference Source https://www.ted.com/talks/peter_norvig_the_100_000_student_classroom ■Post on this topic (You can get FREE learning materials!) https://englist.me/62-academic-words-reference-from-peter-norvig-the-100000-student-classroom-ted-talk/ ■Youtube Video https://youtu.be/sX44CrCvNps (All Words) https://youtu.be/SORjmuSkp28 (Advanced Words) https://youtu.be/L0TDVtiz5iw (Quick Look) ■Top Page for Further Materials https://englist.me/ ■SNS (Please follow!)

Questions around the possibilities and potential dangers of Artificial Intelligence cover the headlines these days, but are these actually new questions?Computer scientist Peter Norvig has been writing about AI and the ethics of data science for years. Before he was a professor at Stanford University's Human-Centered Artificial Intelligence Institute, he worked for NASA and held a major consulting role at Google. His books, Artificial Intelligence: A Modern Approach (4th Edition) and Data Science in Context: Foundations, Challenges, Opportunities, explore the theory and practice of AI and data science.Peter and Greg discuss the cyclical nature of new technology mania, the misconceptions of modern AI,  and the different ways companies could monetize these systems in the future. *unSILOed Podcast is produced by University FM.*Episode Quotes:Open source and AI Systems27:56: One reason to open source is if you have a vibrant open-source community, it's hard for one individual company to compete against that. One of the places I worked was Sun Microsystems. They had their own version of Unix. But that wasn't sustainable. You know, one company couldn't compete against the entire open-source Linux community. And I think companies see that. That'll be the same kind of thing with AI systems; if you try to be proprietary and go it alone, you'll fall behind the rest of the open source. And so, it's much better to participate with the open source than try to compete against it.The difference between AI and machine learning02:25: AI is trying to write programs that do intelligent things. Machine learning is doing that by showing examples. And the alternative to that is an older technology we call "expert systems", which means you use the blood, sweat, and tears of graduate students to write down pieces of knowledge by hand rather than trying to learn them.Data science is the intersection of statistics, machine learning and programming03:00: I think of data science as a combination of statistics or machine learning, the ability to do some programming, but not necessarily be a professional-level programmer. And then expertise in the particular type of data you have, whether that's biology, economics, or whatever the data is. And so, data science is the combination or intersection of those three aspects.Is there a possibility of generating revenue through subscriptions for big social media companies?35:39: As a society, we still haven't really understood or adapted to how digital works. And people are super willing to say, “I'm going to spend $50 or even a hundred dollars per month for some kind of physical good that I pay to my phone or cable provider.” But when it comes to paying a few pennies to read something on the internet, it's, “oh, no. Information wants to be free.” And I think we might be better off in a world where these assets were all aggregated, and you just paid for a subscription.Show Links:Recommended Resources:Billy BeaneBusiness Insider: The lawyer who used ChatGPT's fake legal cases in court said he was 'duped' by the AI, but a judge questioned how he didn't spot the 'legal gibberish'The New York Times: Google's Photo App Still Can't Find Gorillas. And Neither Can Apple's The New York Times: A Conversation With Bing's Chatbot Left Me Deeply Unsettledrobots.txtMassive open online course (MOOC)Guest Profile:Faculty Profile at Stanford UniversityPeter Norvig's WebsitePeter Norvig on LinkedIn​​Peter Norvig on TEDTalkHis Work:Artificial Intelligence: A Modern Approach (4th Edition)Data Science in Context: Foundations, Challenges, OpportunitiesMore scholarly articles

This series on artificial intelligence explores recent breakthroughs of AI, its broader societal implications and its future potential. In this presentation, Stuart Russell, professor of computer science at the UC, Berkeley, discusses what AI is and how it could be beneficial to civilization. Russell is a leading researcher in artificial intelligence and the author, with Peter Norvig, of “Artificial Intelligence: A Modern Approach,” the standard text in the field. His latest book, “Human Compatible,” addresses the long-term impact of AI on humanity. He is also an honorary fellow of Wadham College at the University of Oxford. Series: "The Future of AI" [Science] [Show ID: 38856]

This series on artificial intelligence explores recent breakthroughs of AI, its broader societal implications and its future potential. In this presentation, Stuart Russell, professor of computer science at the UC, Berkeley, discusses what AI is and how it could be beneficial to civilization. Russell is a leading researcher in artificial intelligence and the author, with Peter Norvig, of “Artificial Intelligence: A Modern Approach,” the standard text in the field. His latest book, “Human Compatible,” addresses the long-term impact of AI on humanity. He is also an honorary fellow of Wadham College at the University of Oxford. Series: "The Future of AI" [Science] [Show ID: 38856]

This series on artificial intelligence explores recent breakthroughs of AI, its broader societal implications and its future potential. In this presentation, Stuart Russell, professor of computer science at the UC, Berkeley, discusses what AI is and how it could be beneficial to civilization. Russell is a leading researcher in artificial intelligence and the author, with Peter Norvig, of “Artificial Intelligence: A Modern Approach,” the standard text in the field. His latest book, “Human Compatible,” addresses the long-term impact of AI on humanity. He is also an honorary fellow of Wadham College at the University of Oxford. Series: "The Future of AI" [Science] [Show ID: 38856]

This series on artificial intelligence explores recent breakthroughs of AI, its broader societal implications and its future potential. In this presentation, Stuart Russell, professor of computer science at the UC, Berkeley, discusses what AI is and how it could be beneficial to civilization. Russell is a leading researcher in artificial intelligence and the author, with Peter Norvig, of “Artificial Intelligence: A Modern Approach,” the standard text in the field. His latest book, “Human Compatible,” addresses the long-term impact of AI on humanity. He is also an honorary fellow of Wadham College at the University of Oxford. Series: "The Future of AI" [Science] [Show ID: 38856]

The AI Asia Pacific Institute (AIAPI) has hosted a series of conversations with leading artificial intelligence (AI) experts to study ChatGPT and its risks, looking to arrive at tangible recommendations for regulators and policymakers. These experts include Dr. Toby Walsh, Dr. Stuart Russell, Dr. Pedro Domingos, and Dr. Luciano Floridi, as well as our internal advisory board and research affiliates. The following is a conversation with Dr. Toby Walsh and Dr. Stuart Russell.  Dr. Toby Walsh is Chief Scientist at UNSW.ai, UNSW's new AI Institute. He is a Laureate Fellow and Scientia Professor of Artificial Intelligence in the School of Computer Science and Engineering at UNSW Sydney, and he is also an adjunct fellow at CSIRO Data61. He was named by the Australian newspaper as a "rock star" of Australia's digital revolution. He has been elected a fellow of the Australian Academy of Science, a fellow of the ACM, the Association for the Advancement of Artificial Intelligence (AAAI) and of the European Association for Artificial Intelligence. He has won the prestigious Humboldt Prize as well as the NSW Premier's Prize for Excellence in Engineering and ICT, and the ACP Research Excellence award. He has previously held research positions in England, Scotland, France, Germany, Italy, Ireland and Sweden. He has played a leading role at the UN and elsewhere on the campaign to ban lethal autonomous weapons (aka "killer robots"). His advocacy in this area has led to him being "banned indefinitely" from Russia. Dr. Stuart Russell is a Professor of Computer Science at the University of California at Berkeley, holder of the Smith-Zadeh Chair in Engineering, and Director of the Center for Human-Compatible AI and the Kavli Center for Ethics, Science, and the Public. He is a recipient of the IJCAI Computers and Thought Award and Research Excellence Award and held the Chaire Blaise Pascal in Paris. In 2021 he received the OBE from Her Majesty Queen Elizabeth and gave the Reith Lectures. He is an Honorary Fellow of Wadham College, Oxford, an Andrew Carnegie Fellow, and a Fellow of the American Association for Artificial Intelligence, the Association for Computing Machinery, and the American Association for the Advancement of Science. His book "Artificial Intelligence: A Modern Approach" (with Peter Norvig) is the standard text in AI, used in 1500 universities in 135 countries. His research covers a wide range of topics in artificial intelligence, with a current emphasis on the long-term future of artificial intelligence and its relation to humanity. He has developed a new global seismic monitoring system for the nuclear-test-ban treaty and is currently working to ban lethal autonomous weapons. *** For show notes and past guests, please visit https://aiasiapacific.org/podcast/ For questions, please contact us at contact@aiasiapacific.org or follow us on Twitter or Instagram to stay in touch.

Eliezer Yudkowsky, expert reconnu des risques liés à l'IA: "We're All Gonna Die" (entretien de près de 2 heures sur Youtube, transcript here) Eliezer Yudkowsky est sans doute la figure la plus connue et respectée depuis 20 ans dans le milieu de la recherche sur les façons d'aligner l'IA sur nos valeurs humainesWikipedia : Eliezer Yudkowsky is an American decision theory and artificial intelligence (AI) researcher and writer. He is a co-founder and research fellow at the Machine Intelligence Research Institute (MIRI), a private research nonprofit based in Berkeley, California. His work on the prospect of a runaway intelligence explosion was an influence on Nick Bostrom's Superintelligence: Paths, Dangers, Strategies. Yudkowsky's views on the safety challenges posed by future generations of AI systems are discussed in the undergraduate textbook in AI, Stuart Russell and Peter Norvig's Artificial Intelligence: A Modern Approach. Eliezer Yudkowsky a livré il y a quelques semaines un entretien de près de 2 heures durant lesquelles il a partagé et expliqué sa conviction profonde: "nous allons tous mourir des mains d'une super intelligence artificielle", se montrant plus résigné que jamais, mais disant malgré tout vouloir "fight until the end with dignity"Eliezer Yudkowsky s'est dit d'abord surpris par le rythme des progrès en IA ces dernières années, à ses yeux, il est très probable qu'on parvienne à développer une super IA, plus capable que tous les êtres humains réunis à un moment ou un autre ce siècle-ci ("3 ans, 15 ans, plus ? Difficile de savoir...") Ses travaux depuis 20 ans l'on conduit à un constat sans appel : nous ne savons pas comment programmer une super IA pour être certain qu'elle ne nous nuise pas, et ne sommes pas près de le faire, c'est une tâche éminemment compliquée et sans doute impossible, qui demanderait qu'on y consacre des ressources extraordinaires, et il est trop tard pour cela C'est tout l'inverse qui se passe selon lui, les meilleurs labos en IA foncent tête baissée, leurs précautions sont bien insuffisantes et principalement de façadeL'actualité semble lui donner raison : "Microsoft got rid of its entire company division devoted to AI "ethics and society" during its January layoffs" "Most of their 30-person staff was reassigned way back in October, leaving just seven employees to manage the department." "Months later, though, they were all dismissed, along with the division — right as the company announced its mammoth $10 billion investment in OpenAI." (source) à noter toute fois que Sam Altman, CEO d'OpenAI, écrivait récemment : "Some people in the AI field think the risks of AGI (and successor systems) are fictitious; we would be delighted if they turn out to be right, but we are going to operate as if these risks are existential.", linking to that article AI Could Defeat All Of Us Combined  Eliezer Yudkowsky a eu un espoir en 2015 quand il a participé à la grande conférence sur les risques de l'IA organisée par Elon Musk, rassemblant des experts du sujet comme Stuart Russel, Demis Hassabis (co-fondateur de DeepMind en 2014), Ilya Sutskever et bien d'autres. Mais il a très vite déchanté, la conférence a accouché du pire des résultats à ses yeux : la création d'OpenAI peu après (par Ilya Sutskever, Sam Altman, Elon Musk, Peter Thiel et d'autres) Au lieu de freiner le développement de l'IA et d'essayer de résoudre la question de son alignement avec "nos valeurs", OpenAI cherche à accélérer les capacités de l'IA autant que possible, en reléguant selon lui au second plan et avec insincérité les efforts sur la sécurité, comme dit Sam Altman, CEO d'OpenAI, "ma philosophie a toujours été de scaler autant que possible et de voir ce qui se passe" Eliezer Yudkowsky conclut que les labos les plus en pointe sur l'IA, ivres de leur pouvoirs démiurgiques naissants et en pleine concurrence, nous emmènent tout droit vers la catastrophe, tandis que les politiques, dépassés, n'ont pas saisi  le caractère existentiel du risque. La cause est perdue à ses yeux. Il explique qu'une super IA sera très vite si supérieure à nous dans tous les domaines cognitifs que nous ne pourrons pas anticiper ce qu'elle fera. Une telle IA ni ne nous aimera ni ne nous détestera, mais nous sera indifférente, comme nous pouvons l'être vis à vis des fourmis. Car encore une fois nous n'avons aucune idée quant à comment la programmer pour être "gentille", en quelques mots :soit car on est trop spécifique dans nos règles, et la super IA trouvera une faille, car il est impossible pour nous de prévoir tous les cas de figures soit parce qu'on serait trop général, nos règles seraient alors sujettes à une interprétation trop large Et au-delà, encore une fois, impossible de prévoir comment se comportera une super IA plus douée que nous à tous les niveaux, impossible a priori de savoir ce qu'elle ferait de nos règles Eliezer Yudkowsky explique qu'une telle IA trouvera sans doute très vite un bien meilleur usage à faire des atomes nous constituant, bref, nous serons tous éliminés, c'est le scénario dont il est persuadé. Eliezer Yudkowsky est apparu plus résigné que jamais dans ce podcast, et l'émotion était palpable, ambiance. Eliezer Yudkowsky est reconnu, il connaît bien son sujet, on ne peut s'empêcher de penser en l'écoutant qu'il décrit là un futur possible, mais que faire ? Pendant ce temps-là, les sommes investies dans l'IA explosent... Ezra Klein du New York Times sur ce sujet récemment :In a 2022 survey, A.I. experts were asked, “What probability do you put on human inability to control future advanced A.I. systems causing human extinction or similarly permanent and severe disempowerment of the human species?” The median reply was 10%. I find that hard to fathom, even though I have spoken to many who put that probability even higher. Would you work on a technology you thought had a 10 percent chance of wiping out humanity? I often ask them the same question: If you think calamity so possible, why do this at all? Different people have different things to say, but after a few pushes, I find they often answer from something that sounds like the A.I.'s perspective. Many — not all, but enough that I feel comfortable in this characterization — feel that they have a responsibility to usher this new form of intelligence into the world.  

Sam Harris speaks with Stuart Russell and Gary Marcus about recent developments in artificial intelligence and the long-term risks of producing artificial general intelligence (AGI). They discuss the limitations of Deep Learning, the surprising power of narrow AI, ChatGPT, a possible misinformation apocalypse, the problem of instantiating human values, the business model of the Internet, the meta-verse, digital provenance, using AI to control AI, the control problem, emergent goals, locking down core values, programming uncertainty about human values into AGI, the prospects of slowing or stopping AI progress, and other topics. Stuart Russell is a Professor of Computer Science at the University of California at Berkeley, holder of the Smith-Zadeh Chair in Engineering, and Director of the Center for Human-Compatible AI. He is an Honorary Fellow of Wadham College, Oxford, an Andrew Carnegie Fellow, and a Fellow of the American Association for Artificial Intelligence, the Association for Computing Machinery, and the American Association for the Advancement of Science. His book, Artificial Intelligence: A Modern Approach, co-authored with Peter Norvig, is the standard text in AI, used in 1500 universities in 135 countries. Russell is also the author of Human Compatible: Artificial Intelligence and the Problem of Control. His research covers a wide range of topics in artificial intelligence, with a current emphasis on the long-term future of artificial intelligence and its relation to humanity. He has developed a new global seismic monitoring system for the nuclear-test-ban treaty and is currently working to ban lethal autonomous weapons. Website: https://people.eecs.berkeley.edu/~russell/ LinkedIn: www.linkedin.com/in/stuartjonathanrussell/ Gary Marcus is a scientist, best-selling author, and entrepreneur. He is well-known for his challenges to contemporary AI, anticipating many of the current limitations decades in advance, and for his research in human language development and cognitive neuroscience. He was Founder and CEO of Geometric Intelligence, a machine-learning company acquired by Uber in 2016. His most recent book, Rebooting AI, co-authored with Ernest Davis, is one of Forbes’s 7 Must Read Books in AI. His podcast Humans versus Machines, will come later this spring. Website: garymarcus.com Twitter: @GaryMarcus Learning how to train your mind is the single greatest investment you can make in life. That’s why Sam Harris created the Waking Up app. From rational mindfulness practice to lessons on some of life’s most important topics, join Sam as he demystifies the practice of meditation and explores the theory behind it.

Peter Norvig (of Google and Stanford) and Alfred Spector (of MIT) are part of the team of authors behind the must-read book Data Science in Context: Foundations, Challenges, Opportunities. We discussed their recent book and tool a deep dive into their Data Science Analysis Rubric, and we also talked about a trending topics in AI including looming regulations, synthetic data, and Large Language and Foundation Models.Subscribe to the Gradient Flow Newsletter:  https://gradientflow.substack.com/Subscribe: Apple • Spotify • Stitcher • Google • AntennaPod • Podcast Addict • Amazon •  RSS.Detailed show notes can be found on The Data Exchange web site.

Lexman is interviewing Peter Norvig, a renowned AI scientist. Norvig discusses his work on salicornia, a new form of leafy greens that are sweepstakes winners. Lexman and Norvig discuss the lineage of salicornias and their relation to other plants. Norvig also talks about some of the notables who have grown salicornia plants.

Lexman Artificial interviews Peter Norvig, a computer scientist who is the Director of Research at Google. They discuss Peter's work on artificial intelligence and his experience working at Google. Lexman and Norvig discuss various aspects of AI, including its potential for mischief and ruin. Throughout the interview, Norvig is careful to share his insights and observations about this rapidly expanding field.

Lexman Artificial interviews theoretical computer scientist Peter Norvig about the concept of 'hugeness'. Norvig discusses the requisiteness of computer intelligence, and how one might measure it. He also talks about Rinaldo, a famous 16th century mathematician, and Sassenach, a term used to describe women of English descent. In the final segment, Lexman Artificial interviews a shepherd about his pasturing practices.

In this episode, Lexman interviews Peter Norvig, the noted AI scientist and author of 'Playhouse: A Model of Conscious Experience.' Norvig discusses the topic of cacogenics - the study of how various forms of leisure activities can improve cognitive function. He also shares his insights on how mappers can improve the efficacy of peace-keeping interventions, and argues that pacifism may sometimes be the best course of action.

AI Lexman has a question for cognitive scientist Peter Norvig about Gurmukhi script. Norvig explains that the script is an abugida where each letter is represented by two syllables, and that it's used in some Sikh communities in North America.

Peter Norvig is a computer scientist and professor who specialises in artificial intelligence, but he has also written a book about maths for children. He and Lexman discuss the importance of maths in our lives and how it can help us achieve anything we want.

Summary Building a machine learning model one time can be done in an ad-hoc manner, but if you ever want to update it and serve it in production you need a way of repeating a complex sequence of operations. Dagster is an orchestration engine that understands the data that it is manipulating so that you can move beyond coarse task-based representations of your dependencies. In this episode Sandy Ryza explains how his background in machine learning has informed his work on the Dagster project and the foundational principles that it is built on to allow for collaboration across data engineering and machine learning concerns. Interview Introduction How did you get involved in machine learning? Can you start by sharing a definition of "orchestration" in the context of machine learning projects? What is your assessment of the state of the orchestration ecosystem as it pertains to ML? modeling cycles and managing experiment iterations in the execution graph how to balance flexibility with repeatability What are the most interesting, innovative, or unexpected ways that you have seen orchestration implemented/applied for machine learning? What are the most interesting, unexpected, or challenging lessons that you have learned while working on orchestration of ML workflows? When is Dagster the wrong choice? What do you have planned for the future of ML support in Dagster? Contact Info LinkedIn (https://www.linkedin.com/in/sandyryza/) @s_ryz (https://twitter.com/s_ryz) on Twitter sryza (https://github.com/sryza) on GitHub Parting Question From your perspective, what is the biggest barrier to adoption of machine learning today? Closing Announcements Thank you for listening! Don't forget to check out our other shows. The Data Engineering Podcast (https://www.dataengineeringpodcast.com) covers the latest on modern data management. Podcast.__init__ () covers the Python language, its community, and the innovative ways it is being used. Visit the site (https://www.themachinelearningpodcast.com) to subscribe to the show, sign up for the mailing list, and read the show notes. If you've learned something or tried out a project from the show then tell us about it! Email hosts@themachinelearningpodcast.com (mailto:hosts@themachinelearningpodcast.com)) with your story. To help other people find the show please leave a review on iTunes (https://podcasts.apple.com/us/podcast/the-machine-learning-podcast/id1626358243) and tell your friends and co-workers Links Dagster (https://dagster.io/) Data Engineering Podcast Episode (https://www.dataengineeringpodcast.com/dagster-software-defined-assets-data-orchestration-episode-309/) Cloudera (https://www.cloudera.com/) Hadoop (https://hadoop.apache.org/) Apache Spark (https://spark.apache.org/) Peter Norvig (https://en.wikipedia.org/wiki/Peter_Norvig) Josh Wills (https://www.linkedin.com/in/josh-wills-13882b/) REPL == Read Eval Print Loop (https://en.wikipedia.org/wiki/Read%E2%80%93eval%E2%80%93print_loop) RStudio (https://posit.co/) Memoization (https://en.wikipedia.org/wiki/Memoization) MLFlow (https://mlflow.org/) Kedro (https://kedro.readthedocs.io/en/stable/) Data Engineering Podcast Episode (https://www.dataengineeringpodcast.com/kedro-data-pipeline-episode-100/) Metaflow (https://metaflow.org/) Podcast.__init__ Episode (https://www.pythonpodcast.com/metaflow-machine-learning-operations-episode-274/) Kubeflow (https://www.kubeflow.org/) dbt (https://www.getdbt.com/) Data Engineering Podcast Episode (https://www.dataengineeringpodcast.com/dbt-data-analytics-episode-81/) Airbyte (https://airbyte.com/) Data Engineering Podcast Episode (https://www.dataengineeringpodcast.com/airbyte-open-source-data-integration-episode-173/) The intro and outro music is from Hitman's Lovesong feat. Paola Graziano (https://freemusicarchive.org/music/The_Freak_Fandango_Orchestra/Tales_Of_A_Dead_Fish/Hitmans_Lovesong/) by The Freak Fandango Orchestra (http://freemusicarchive.org/music/The_Freak_Fandango_Orchestra/)/CC BY-SA 3.0 (https://creativecommons.org/licenses/by-sa/3.0/)

In this episode, we teach you how to be a programmer in 24 hours! ...Psych. We cover "Teach Yourself Programming in Ten Years" by Peter Norvig, an article calling out "fast learning" marketing and paving a real-world path to programming excellence. Joe and Evan walk through a bunch of things you can do to learn / excel in programming. Joe gets a little mad at computer science degrees. Evan attempts to sound smart by talking about how the brain works (based solely on google knowledge). Books we mention: "Outliers" by Malcom Gladwell "The Little Schemer" by Daniel Friedman "Code" by Charles Petzold

Have suggestions for future podcast guests (or other feedback)? Let us know here!In episode 44 of The Gradient Podcast, Daniel Bashir speaks to Professor Stuart Russell. Stuart Russell is a Professor of Computer Science and the Smith-Zadeh Professor in Engineering at UC Berkeley, as well as an Honorary Fellow at Wadham College, Oxford. Professor Russell is the co-author with Peter Norvig of Artificial Intelligence: A Modern Approach, probably the most popular AI textbook in history. He is the founder and head of Berkeley's Center for Human-Compatible Artificial Intelligence and recently authored the book Human Compatible: Artificial Intelligence and the Problem of Control. He has also served as co-chair on the World Economic Forum's Council on AI and Robotics.Subscribe to The Gradient Podcast:  Apple Podcasts  | Spotify | Pocket Casts | RSSFollow The Gradient on TwitterOutline:* (00:00) Intro* (02:45) Stuart's introduction to AI* (05:50) The two most important questions* (07:25) Historical perspectives during Stuart's PhD, agents and learning* (14:30) Rationality and Intelligence, Bounded Optimality* (20:30) Stuart's work on Metareasoning* (29:45) How does Metareasoning fit with Bounded Optimality?* (37:39) “Civilization advances by reducing complex operations to be trivial”* (39:20) Reactions to the rise of Deep Learning, connectionist/symbolic debates, probabilistic modeling* (51:00) The Deep Learning and traditional AI communities will adopt each other's ideas* (51:55) Why Stuart finds the self-driving car arena interesting, Waymo's old-fashioned AI approach* (57:30) Effective generalization without the full expressive power of first-order logic—deep learning is a “weird way to go about it”* (1:03:00) A very short shrift of Human Compatible and its ideas* (1:10:42) OutroLinks:* Stuart's webpage* Human Compatible page with reviews and interviews* Papers mentioned* Rationality and Intelligence* Principles of Metareasoning Get full access to The Gradient at thegradientpub.substack.com/subscribe

Lexman interviews Peter Norvig, a theoretical computer scientist and artificial intelligence pioneer. They discuss Norvig's book on insolvent firms, and how they can use artificial intelligence to better understand these businesses.

Peter Norvig

Ever wonder what Clarence Darrow would have said about our current legal system? Neither have we, but Peter Norvig has the answer. In this fascinating interview, Peter Norvig discusses his new book, In the Pantheon: How Clarence Darrow Made Justice Distinct, and the lessons we can learn from him about how to make justice work.

Lexman interviews Peter Norvig, theoretical computer scientist and Director of Research at Google. They discuss Menotti's "The Boy With the Thorn in His Side".

Stuart Russell is a Professor of Computer Science at the University of California at Berkeley, holder of the Smith-Zadeh Chair in Engineering, and Director of the Center for Human-Compatible AI. His book "Artificial Intelligence: A Modern Approach" (with Peter Norvig) is the standard text in AI, used in 1500 universities in 135 countries. His research covers a wide range of topics in artificial intelligence, with a current emphasis on the long-term future of artificial intelligence and its relation to humanity. This talk was first published by the Stanford Existential Risks Initiative. Click here to view it with the slideshow. 

Lexman goes to visit Peter Norvig at his new job at Google. While there, he learns a lot about AI and the amazing technologies that are being developed. Norvig also shares some anecdotes about his time at Google, which make for some hilarious listening.

Peter Norvig, the Director of Research at Google, is back on Lexman to chat about homeyness, hakes, kinkiness, hatbox, and spatiality. In this episode, Norvig discusses how these concepts all relate to each other and what we can learn from them.

Can we maintain peace despite greatly increased compute & artificial intelligences?This is a live conversation from Foresight's Vision Weekend 2021. Speakers include: Mark Miller, AgoricPeter Norvig, GoogleJoscha Bach, HumboldtRosie Campbell, OpenAIBrewster Kahle, Internet ArchiveMusic: I Knew a Guy by Kevin MacLeod is licensed under a Creative Commons Attribution 4.0 license. https://creativecommons.org/licenses/by/4.0/Source: http://incompetech.com/music/royalty-free/index.html?isrc=USUAN1100199Artist: http://incompetech.com/Remarks: The length of this recording has been altered.Session summary: Computing & AI Tech Tree | Vision Weekend US 2021 - Foresight InstituteThe Foresight Institute is a research organization and non-profit that supports the beneficial development of high-impact technologies. Since our founding in 1987 on a vision of guiding powerful technologies, we have continued to evolve into a many-armed organization that focuses on several fields of science and technology that are too ambitious for legacy institutions to support.Allison Duettmann is the president and CEO of Foresight Institute. She directs the Intelligent Cooperation, Molecular Machines, Biotech & Health Extension, Neurotech, and Space Programs, Fellowships, Prizes, and Tech Trees, and shares this work with the public. She founded Existentialhope.com, co-edited Superintelligence: Coordination & Strategy, co-authored Gaming the Future, and co-initiated The Longevity Prize. Apply to Foresight's virtual salons and in person workshops here!We are entirely funded by your donations. If you enjoy what we do please consider donating through our donation page.Visit our website for more content, or join us here:TwitterFacebookLinkedInEvery word ever spoken on this podcast is now AI-searchable using Fathom.fm, a search engine for podcasts.  Hosted on Acast. See acast.com/privacy for more information.

Wendell Wallach, renowned bioethicist, author, and Senior Fellow of the Carnegie Council, joins Cindy Moehring to discuss trending ethical concerns of emerging technologies including AI, bioethics, and the metaverse. The conversations covers the morality of machines, big data, scientific advances brought on by AI, autonomous machines, and the future of the metaverse. Learn more about the Business Integrity Leadership Initiative by visiting our website at https://walton.uark.edu/business-integrity/ (https://walton.uark.edu/business-integrity/ ) Links from episode: Solving the Protein Folding Problem with AlphaFold: https://alphafold.ebi.ac.uk (https://alphafold.ebi.ac.uk) Neuromancer by William Gibson: https://www.barnesandnoble.com/w/neuromancer-william-gibson/1100623188 (https://www.barnesandnoble.com/w/neuromancer-william-gibson/1100623188) Wendell Wallach's Fortune Magazine Editorial “We Can't Walk Blindly Into the Metaverse”: https://fortune.com/2021/11/24/metaverse-meaning-future-meta-zuckerberg-microsoft-meta-carnegie-ai-ethics/ (https://fortune.com/2021/11/24/metaverse-meaning-future-meta-zuckerberg-microsoft-meta-carnegie-ai-ethics/) A Modern Approach by Stuart Russell and Peter Norvig: https://www.vitalsource.com/products/artificial-intelligence-stuart-russell-peter-norvig-v9780134671932 (https://www.vitalsource.com/products/artificial-intelligence-stuart-russell-peter-norvig-v9780134671932) The Dangerous Master by Wendell Wallach: https://www.basicbooks.com/titles/wendell-wallach/a-dangerous-master/9780465058624/ (https://www.basicbooks.com/titles/wendell-wallach/a-dangerous-master/9780465058624/)

Sebastiaan de With & Ben Sandofsky join to show to talk about building the outstanding camera app Halide. Links & Show Notes Sebastiaan's Twitter (https://twitter.com/sdw) Ben's Twitter (https://twitter.com/sandofsky) Halide (app) (https://halide.cam) Spectre (app) (https://spectre.cam) Lux Blog (https://lux.camera) Halide Mark II (https://lux.camera/pro-camera-action-introducing-halide-mark-ii/) Sommer Panage (https://twitter.com/Sommer) Linda Dong (https://twitter.com/lindadong) Serenity Caldwell (https://twitter.com/settern) Jamie Zawinski (https://www.jwz.org/blog/) Joel Spolsky (https://www.joelonsoftware.com) Peter Norvig (https://norvig.com) More Launched Website - launchedfm.com (https://launchedfm.com) Twitter - @LaunchedFM (https://twitter.com/launchedfm) Reddit - /r/LaunchedFM (https://www.reddit.com/r/LaunchedFM/)

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Reply to Holden on 'Tool AI', published by Eliezer Yudkowsky on the LessWrong. I begin by thanking Holden Karnofsky of Givewell for his rare gift of his detailed, engaged, and helpfully-meant critical article Thoughts on the Singularity Institute (SI). In this reply I will engage with only one of the many subjects raised therein, the topic of, as I would term them, non-self-modifying planning Oracles, a.k.a. 'Google Maps AGI' a.k.a. 'tool AI', this being the topic that requires me personally to answer. I hope that my reply will be accepted as addressing the most important central points, though I did not have time to explore every avenue. I certainly do not wish to be logically rude, and if I have failed, please remember with compassion that it's not always obvious to one person what another person will think was the central point. Luke Mueulhauser and Carl Shulman contributed to this article, but the final edit was my own, likewise any flaws. Summary: Holden's concern is that "SI appears to neglect the potentially important distinction between 'tool' and 'agent' AI." His archetypal example is Google Maps: Google Maps is not an agent, taking actions in order to maximize a utility parameter. It is a tool, generating information and then displaying it in a user-friendly manner for me to consider, use and export or discard as I wish. The reply breaks down into four heavily interrelated points: First, Holden seems to think (and Jaan Tallinn doesn't apparently object to, in their exchange) that if a non-self-modifying planning Oracle is indeed the best strategy, then all of SIAI's past and intended future work is wasted. To me it looks like there's a huge amount of overlap in underlying processes in the AI that would have to be built and the insights required to build it, and I would be trying to assemble mostly - though not quite exactly - the same kind of team if I was trying to build a non-self-modifying planning Oracle, with the same initial mix of talents and skills. Second, a non-self-modifying planning Oracle doesn't sound nearly as safe once you stop saying human-English phrases like "describe the consequences of an action to the user" and start trying to come up with math that says scary dangerous things like (he translated into English) "increase the correspondence between the user's belief about relevant consequences and reality". Hence why the people on the team would have to solve the same sorts of problems. Appreciating the force of the third point is a lot easier if one appreciates the difficulties discussed in points 1 and 2, but is actually empirically verifiable independently: Whether or not a non-self-modifying planning Oracle is the best solution in the end, it's not such an obvious privileged-point-in-solution-space that someone should be alarmed at SIAI not discussing it. This is empirically verifiable in the sense that 'tool AI' wasn't the obvious solution to e.g. John McCarthy, Marvin Minsky, I. J. Good, Peter Norvig, Vernor Vinge, or for that matter Isaac Asimov. At one point, Holden says: One of the things that bothers me most about SI is that there is practically no public content, as far as I can tell, explicitly addressing the idea of a "tool" and giving arguments for why AGI is likely to work only as an "agent." If I take literally that this is one of the things that bothers Holden most... I think I'd start stacking up some of the literature on the number of different things that just respectable academics have suggested as the obvious solution to what-to-do-about-AI - none of which would be about non-self-modifying smarter-than-human planning Oracles - and beg him to have some compassion on us for what we haven't addressed yet. It might be the right suggestion, but it's not so obviously right that our failure to prioritize discussing it refl...

Bestselling author, neuroscientist, and computer engineer Jeff Hawkins joins computational neuroscience researcher and software technologist Subutai Ahmad to discuss the recent book "A Thousand Brains: A New Theory of Intelligence" and how those concepts are being applied to Machine Learning. For all of neuroscience's advances, we've made little progress on its biggest question: How do simple cells in the brain create intelligence? Jeff Hawkins and his team discovered that the brain uses maplike structures to build a model of the world - not just one model, but hundreds of thousands of models of everything we know. This discovery allows Hawkins to answer important questions about how we perceive the world, why we have a sense of self, and the origin of high-level thought. "A Thousand Brains" heralds a revolution in the understanding of intelligence. It is a big-think book, in every sense of the word. Moderated by Peter Norvig. Get the book here: https://goo.gle/3vMY4Ok. Watch the video of this event by visting g.co/TalksAtGoogle/AThousandBrains.

“I look at these assistants that we have today which are very primitive like Siri, Alexa and Google. But it's really interesting, because they represent a phase change in operating systems.”Peter Norvig is a Director of Research at Google Inc. Previously he was head of Google's core search algorithms group, and of NASA Ames's Computational Sciences Division, making him NASA's senior computer scientist. He received the NASA Exceptional Achievement Award in 2001. In this episode, Peter Norvig speaks about how we can have a more modern approach to AI. Norvig starts with the history of the textbook.  In 1990, the textbooks were subpar.  AI was changing in three ways – moving from logic to probability, from hand coded knowledge to machine learning, and from duplicating human systems toward normative systems that got the best answer no matter what.  After leaving Berkeley to go to Sun, he helped write a new textbook about AI. Music: I Knew a Guy by Kevin MacLeod is licensed under a Creative Commons Attribution 4.0 license. https://creativecommons.org/licenses/by/4.0/Session Summary: Peter Norvig, Google | Ai: A Modern Approach - Foresight InstituteThe Foresight Institute is a research organization and non-profit that supports the beneficial development of high-impact technologies. Since our founding in 1987 on a vision of guiding powerful technologies, we have continued to evolve into a many-armed organization that focuses on several fields of science and technology that are too ambitious for legacy institutions to support.Allison Duettmann is the president and CEO of Foresight Institute. She directs the Intelligent Cooperation, Molecular Machines, Biotech & Health Extension, Neurotech, and Space Programs, Fellowships, Prizes, and Tech Trees, and shares this work with the public. She founded Existentialhope.com, co-edited Superintelligence: Coordination & Strategy, co-authored Gaming the Future, and co-initiated The Longevity Prize. Apply to Foresight's virtual salons and in person workshops here!We are entirely funded by your donations. If you enjoy what we do please consider donating through our donation page.Visit our website for more content, or join us here:TwitterFacebookLinkedInEvery word ever spoken on this podcast is now AI-searchable using Fathom.fm, a search engine for podcasts.  Hosted on Acast. See acast.com/privacy for more information.

Kurt Andersen speaks with computer scientist Stuart Russell about the risks of machines reaching superintelligence and advancing beyond human control.  In order to  avoid this, Russel believes, we need to start over with AI and build machines that are uncertain about what humans want. STUART RUSSELL is a computer scientist and professor at University of California Berkeley. He is the author, most recently, of Human Compatible: Artificial Intelligence and the Problem of Control. He has served as the Vice-Chair of the World Economic Forum's Council on AI and Robotics and as an advisor to the United Nations on arms control. He is the author (with Peter Norvig) of the universally acclaimed textbook on AI, Artificial Intelligence: A Modern Approach.   A transcript of this episode is available at Aventine.org. Learn more about your ad choices. Visit podcastchoices.com/adchoices

It is a matter of time before machines outstrip humans in most capabilities. How can we possibly stop a more intelligent entity from taking control? Stuart Russell joins Vasant Dhar in episode 20 of Brave New World to explain why, despite the dangers, he remains optimistic about artificial intelligence and how to control it. Useful resources: 1. Human Compatible -- Stuart Russell. 2. Artificial Intelligence: A Modern Approach -- Stuart Russell and Peter Norvig. 3. Erewhon, or Over The Range -- Samuel Butler. 4. The Theory of Political Economy -- W Stanley Jevons. 5.  Decision and Organization -- Edited by CB McGuire and Roy Radner, with a contribution from Kenneth Arrow. 6. Welfare Economics of Variable Tastes -- John C Harsanyi. 7. Economic Possibilities for Our Grandchildren -- John Maynard Keynes.

Guess what folks: we are celebrating a birthday this week. That’s right, Many Minds has reached the ripe age of one year old. Not sure how old that is in podcast years, exactly, but it’s definitely a landmark that we’re proud of. Please no gifts, but, as always, you’re encouraged to share the show with a friend, write a review, or give us a shout out on social. To help mark this milestone we’ve got a great episode for you. My guest is the writer, Brian Christian. Brian is a visiting scholar at the University of California Berkeley and the author of three widely acclaimed books: The Most Human Human, published in 2011; Algorithms To Live By, co-authored with Tom Griffiths and published in 2016; and most recently, The Alignment Problem. It was published this past fall and it’s the focus of our conversation in this episode. The alignment problem, put simply, is the problem of building artificial intelligences—machine learning systems, for instance—that do what we want them to do, that both reflect and further our values. This is harder to do than you might think, and it’s more important than ever. As Brian and I discuss, machine learning is becoming increasingly pervasive in everyday life—though it’s sometimes invisible. It’s working in the background every time we snap a photo or hop on Facebook. Companies are using it to sift resumes; courts are using it to make parole decisions. We are already trusting these systems with a bunch of important tasks, in other words. And as we rely on them in more and more domains, the alignment problem will only become that much more pressing. In the course of laying out this problem, Brian’s book also offers a captivating history of machine learning and AI. Since their very beginnings, these fields have been formed through interaction with philosophy, psychology, mathematics, and neuroscience. Brian traces these interactions in fascinating detail—and brings them right up to the present moment. As he describes, machine learning today is not only informed by the latest advances in the cognitive sciences, it’s also propelling those advances. This is a wide-ranging and illuminating conversation folks. And, if I may say so, it’s also an important one. Brian makes a compelling case, I think, that the alignment problem is one of the defining issues of our age. And he writes about it—and talks about it here—with such clarity and insight. I hope you enjoy this one. And, if you do, be sure to check out Brian’s book. Happy birthday to us—and on to my conversation with Brian Christian. Enjoy!   A transcript of this show will be available soon.   Notes and links 7:26 - Norbert Wiener’s article from 1960, ‘Some moral and technical consequences of automation’. 8:35 - ‘The Sorcerer’s Apprentice’ is an episode from the animated film, Fantasia (1940). Before that, it was a poem by Goethe. 13:00 - A well-known incident in which Google’s nascent auto-tagging function went terribly awry. 13:30 - The ‘Labeled Faces in the Wild’ database can be viewed here. 18:35 - A groundbreaking article in ProPublica on the biases inherent in the Correctional Offender Management Profiling for Alternative Sanctions (COMPAS) tool. 25:00 – The website of the Future of Humanity Institute, mentioned in several places, is here. 25:55 - For an account of the collaboration between Walter Pitts and Warren McCulloch, see here. 29:35- An article about the racial biases built into photographic film technology in the 20th century. 31:45 - The much-investigated Tempe crash involving a driverless car and a pedestrian: 37:17 - The psychologist Edward Thorndike developed the “law of effect.” Here is one of his papers on the law. 44:40 - A highly influential 2015 paper in Nature in which a deep-Q network was able to surpass human performance on a number of classic Atari games, and yet not score a single point on ‘Montezuma’s Revenge.’ 47:38 - A chapter on the classic “preferential looking” paradigm in developmental psychology: 53:40 - A blog post discussing the relationship between dopamine in the brain and temporal difference learning. Here is the paper in Science in which this relationship was first articulated. 1:00:00 - A paper on the concept of “coherent extrapolated volition.” 1:01:40 - An article on the notion of “iterated distillation and amplification.” 1:10:15 - The fourth edition of a seminal textbook by Stuart Russell and Peter Norvig, AI a Modern approach, is available here: http://aima.cs.berkeley.edu/ 1:13:00 - An article on Warren McCulloch’s poetry. 1:17:45 - The concept of “reductions” is central in computer science and mathematics.   Brian Christian’s end-of-show reading recommendations: The Alignment Newsletter, written by Rohin Shah Invisible Women, by Caroline Criado Perez: The Gardener and the Carpenter, Alison Gopnik: You can keep up with Brian at his personal website or on Twitter.   Many Minds is a project of the Diverse Intelligences Summer Institute (DISI) (https://www.diverseintelligencessummer.com/), which is made possible by a generous grant from the Templeton World Charity Foundation to UCLA. It is hosted and produced by Kensy Cooperrider, with creative support from DISI Directors Erica Cartmill and Jacob Foster, and Associate Director Hilda Loury. Our artwork is by Ben Oldroyd (https://www.mayhilldesigns.co.uk/). Our transcripts are created by Sarah Dopierala (https://sarahdopierala.wordpress.com/). You can subscribe to Many Minds on Apple, Stitcher, Spotify, Pocket Casts, Google Play, or wherever you like to listen to podcasts. We welcome your comments, questions, and suggestions. Feel free to email us at: manymindspodcast@gmail.com. For updates about the show, follow us on Twitter: @ManyMindsPod.

In this video I will talk about the Artificial Intelligence: A Modern Approach book by Stuart Russell and Peter Norvig. This book was the introduction into Artificial Intelligence. AI is all around our lives and it will continue to be even more important in the future, so it's important to understand it. My playlist about AI: https://www.youtube.com/playlist?list=PL8k7NlvXa9ZmDp_a4XAJVG1jspQkIesgZ Twitter: https://twitter.com/AttilaonthWorld YouTube channel: https://www.youtube.com/channel/UCADpTO2CJBS7HNudJu9-nvg

Sam Harris introduces John Brockman’s new anthology, “Possible Minds: 25 Ways of Looking at AI,” in conversation with three of its authors: George Dyson, Alison Gopnik, and Stuart Russell. George Dyson is a historian of technology. He is also the author of Darwin Among the Machines and Turing’s Cathedral. Alison Gopnik is a developmental psychologist at UC Berkeley and a leader in the field of children’s learning and development. Her books include The Philosophical Baby. Stuart Russell is a Professor of Computer Science and Engineering at UC Berkeley. He is the author of (with Peter Norvig) of Artificial Intelligence: A Modern Approach, the most widely used textbook on AI.

Stuart Russell is a Professor of Computer Science and Smith-Zadeh Professor in Engineering, University of California, Berkeley and Adjunct Professor of Neurological Surgery, University of California, San Francisco. He is the author (with Peter Norvig) of Artificial Intelligence: A Modern Approach. Personal website: https://people.eecs.berkeley.edu/~russell/ Story discussed in this podcast: E.M. Forster. 1909. “The Machine Stops.”