Podcast appearances and mentions of steve yegge

Steve Yegge is an evangelist at Sourcegraph, a startups that is industrializing software development with AI agents. This episode explores the paradigm shift in software development with the rise of “vibe coding” and AI agents, moving beyond traditional code completion. It discusses how developers are transitioning from line-by-line coding to orchestrating AI, emphasizing the crucial need for trust, verification, and new skill sets like AI engineering and humanities.Subscribe to the Gradient Flow Newsletter

Steve Yegge's latest rant about the future of "coding", Ethan McCue shares some life altering Postgres patterns, Hillel Wayne makes the case for Verification-First Development, Gerd Zellweger experienced lots of pain setting up GitHub Actions & Cascii is a web-based ASCII diagram builder.

Steve Yegge's latest rant about the future of "coding", Ethan McCue shares some life altering Postgres patterns, Hillel Wayne makes the case for Verification-First Development, Gerd Zellweger experienced lots of pain setting up GitHub Actions & Cascii is a web-based ASCII diagram builder.

Steve Yegge's latest rant about the future of "coding", Ethan McCue shares some life altering Postgres patterns, Hillel Wayne makes the case for Verification-First Development, Gerd Zellweger experienced lots of pain setting up GitHub Actions & Cascii is a web-based ASCII diagram builder.

Episode Notes This latest episode of the JUXTCast features Gene Kim, a Wall Street Journal bestselling author, celebrated researcher, and multiple award-winning Chief Technology Officer. Gene is widely recognized for his contributions to the DevOps movement and for co-authoring influential works such as The Phoenix Project and The DevOps Handbook. In this engaging discussion, Gene reflects on his career journey, from his time as the founder and CTO of Tripwire to his rediscovery of the joy of programming through Clojure. The episode explores key themes including high-performing technology organizations, the transformative role of AI in programming, and the strategic importance of modularity in systems design. The conversation also offers unique insights into the evolving role of AI in augmenting developer productivity and creativity. Gene shares his hands-on experience with pair programming and discusses the intersection of REPL-based programming, economic principles in software design, and the future of junior developers in an AI-enhanced ecosystem. Thoughts on a “DORA for GenAI and developers” study: https://x.com/RealGeneKim/status/1856146004724330862  2 hour pair programming with Steve Yegge! https://twitter.com/RealGeneKim/status/1860507119096869363 Description of what I did while walking dog: https://twitter.com/RealGeneKim/status/1853860996689064211  “From Naptime to Big Sleep: Using Large Language Models To Catch Vulnerabilities In Real-World Code,” https://googleprojectzero.blogspot.com/2024/10/from-naptime-to-big-sleep.html?m=1 XTDB: https://docs.xtdb.com/quickstart/sql-overview.html

Junior coders aren't going to be purged, but trained, just like we taught ditch-diggers to drive excavators. Find out why Squirrel thinks Steve Yegge is wrong, on this episode of Troubleshooting Agile. Links: Enterprise Technology Leadership Summit: https://itrevolution.com/product/enterprise-technology-leadership-summit-las-vegas-2024/ - Steve Yegge's talk at ETLS: article https://sourcegraph.com/blog/the-death-of-the-junior-developer , video https://videos.itrevolution.com/watch/1002959965 - Gene Kim at ETLS: https://itrevolution.com/articles/observing-the-impact-of-ai-on-law-firms-software-and-writing-winners-and-losers/ - Patrick Debois at ETLS: https://videos.itrevolution.com/watch/1002959794 -------------------------------------------------- You'll find free videos and practice material, plus our book Agile Conversations, at agileconversations.com And we'd love to hear any thoughts, ideas, or feedback you have about the show: email us at info@agileconversations.com -------------------------------------------------- About Your Hosts Douglas Squirrel and Jeffrey Fredrick joined forces at TIM Group in 2013, where they studied and practised the art of management through difficult conversations. Over a decade later, they remain united in their passion for growing profitable organisations through better communication. Squirrel is an advisor, author, keynote speaker, coach, and consultant, and he's helped over 300 companies of all sizes make huge, profitable improvements in their culture, skills, and processes. You can find out more about his work here: douglassquirrel.com/index.html Jeffrey is Vice President of Engineering at ION Analytics, Organiser at CITCON, the Continuous Integration and Testing Conference, and is an accomplished author and speaker. You can connect with him here: www.linkedin.com/in/jfredrick/

A Silicon Valley veteran and known for his writings like "The Death of the Junior Developer", Steve Yegge joins the show to chat about his "AI Midlife Crisis", the unique writing process he employs, and building the future of coding assistants.   Segments: (00:00:00) The AI Midlife Crisis (00:04:53) The power of rants (00:09:55) “You gotta be able to make yourself laugh” (00:11:46) Steve's writing process (00:14:10) “I published them… and nothing happened for six months” (00:17:30) Key to perseverance in writing? Get pissed. (00:23:24) Writing in one sitting (00:29:05) The AI Midlife Crisis (00:35:04) Management to IC (00:38:35) The acceleration and evolution of programming (00:41:43) Picking up new skills in a new domain (00:43:40) The power of prompt engineering (00:47:27) Secondary hashing (00:50:47) The importance of context in coding assistants (00:53:56) “The future of coding assistants is chat” (00:57:15) The importance of platforms in coding assistants (01:02:30) The nefarious T-word in AI (01:06:32) The death of the junior developer and its consequences (01:09:35) The future of code understanding and semantic indexing (01:13:15) The power of context in AI platforms (01:16:21) Surprising capabilities of LLMs (01:21:04) Transferable skills in AI product development (01:23:53) Mental health and the innovator's dilemma   Show Notes The Death of the Junior Developer: https://sourcegraph.com/blog/the-death-of-the-junior-developer Steve's blog rants: https://steve-yegge.blogspot.com/ Steve's medium posts: https://steve-yegge.medium.com/ Sourcegraph's blog: https://sourcegraph.com/blog Steve on twitter: https://x.com/steve_yegge   Stay in touch:

We are running an end of year survey for our listeners. Let us know any feedback you have for us, what episodes resonated with you the most, and guest requests for 2024! RAG has emerged as one of the key pieces of the AI Engineer stack. Jerry from LlamaIndex called it a “hack”, Bryan from Hex compared it to “a recommendation system from LLMs”, and even LangChain started with it. RAG is crucial in any AI coding workflow. We talked about context quality for code in our Phind episode. Today's guests, Beyang Liu and Steve Yegge from SourceGraph, have been focused on code indexing and retrieval for over 15 years. We locked them in our new studio to record a 1.5 hours masterclass on the history of code search, retrieval interfaces for code, and how they get SOTA 30% completion acceptance rate in their Cody product by being better at the “bin packing problem” of LLM context generation. Google Grok → SourceGraph → CodyWhile at Google in 2008, Steve built Grok, which lives on today as Google Kythe. It allowed engineers to do code parsing and searching across different codebases and programming languages. (You might remember this blog post from Steve's time at Google) Beyang was an intern at Google at the same time, and Grok became the inspiration to start SourceGraph in 2013. The two didn't know eachother personally until Beyang brought Steve out of retirement 9 years later to join him as VP Engineering. Fast forward 10 years, SourceGraph has become to best code search tool out there and raised $223M along the way. Nine months ago, they open sourced SourceGraph Cody, their AI coding assistant. All their code indexing and search infrastructure allows them to get SOTA results by having better RAG than competitors:* Code completions as you type that achieve an industry-best Completion Acceptance Rate (CAR) as high as 30% using a context-enhanced open-source LLM (StarCoder)* Context-aware chat that provides the option of using GPT-4 Turbo, Claude 2, GPT-3.5 Turbo, Mistral 7x8B, or Claude Instant, with more model integrations planned* Doc and unit test generation, along with AI quick fixes for common coding errors* AI-enhanced natural language code search, powered by a hybrid dense/sparse vector search engine There are a few pieces of infrastructure that helped Cody achieve these results:Dense-sparse vector retrieval system For many people, RAG = vector similarity search, but there's a lot more that you can do to get the best possible results. From their release:"Sparse vector search" is a fancy name for keyword search that potentially incorporates LLMs for things like ranking and term expansion (e.g., "k8s" expands to "Kubernetes container orchestration", possibly weighted as in SPLADE): * Dense vector retrieval makes use of embeddings, the internal representation that LLMs use to represent text. Dense vector retrieval provides recall over a broader set of results that may have no exact keyword matches but are still semantically similar. * Sparse vector retrieval is very fast, human-understandable, and yields high recall of results that closely match the user query. * We've found the approaches to be complementary.There's a very good blog post by Pinecone on SPLADE for sparse vector search if you're interested in diving in. If you're building RAG applications in areas that have a lot of industry-specific nomenclature, acronyms, etc, this is a good approach to getting better results.SCIPIn 2016, Microsoft announced the Language Server Protocol (LSP) and the Language Server Index Format (LSIF). This protocol makes it easy for IDEs to get all the context they need from a codebase to get things like file search, references, “go to definition”, etc. SourceGraph developed SCIP, “a better code indexing format than LSIF”:* Simpler and More Efficient Format: SCIP utilizes Protobuf instead of JSON, which is used by LSIF. Protobuf is more space-efficient, simpler, and more suitable for systems programming. * Better Performance and Smaller Index Sizes: SCIP indexers, such as scip-clang, show enhanced performance and reduced index file sizes compared to LSIF indexers (10%-20% smaller)* Easier to Develop and Debug: SCIP's design, centered around human-readable string IDs for symbols, makes it faster and more straightforward to develop new language indexers. Having more efficient indexing is key to more performant RAG on code. Show Notes* Sourcegraph* Cody* Copilot vs Cody* Steve's Stanford seminar on Grok* Steve's blog* Grab* Fireworks* Peter Norvig* Noam Chomsky* Code search* Kelly Norton* Zoekt* v0.devSee also our past episodes on Cursor, Phind, Codeium and Codium as well as the GitHub Copilot keynote at AI Engineer Summit.Timestamps* [00:00:00] Intros & Backgrounds* [00:05:20] How Steve's work on Grok inspired SourceGraph for Beyang* [00:08:10] What's Cody?* [00:11:22] Comparison of coding assistants and the capabilities of Cody* [00:16:00] The importance of context (RAG) in AI coding tools* [00:21:33] The debate between Chomsky and Norvig approaches in AI* [00:30:06] Normsky: the Norvig + Chomsky models collision* [00:36:00] The death of the DSL?* [00:40:00] LSP, Skip, Kythe, BFG, and all that fun stuff* [00:53:00] The SourceGraph internal stack* [00:58:46] Building on open source models* [01:02:00] SourceGraph for engineering managers?* [01:12:00] Lightning RoundTranscriptAlessio: Hey everyone, welcome to the Latent Space podcast. This is Alessio, partner and CTO-in-Residence at Decibel Partners, and I'm joined by my co-host Swyx, founder of Smol AI. [00:00:16]Swyx: Hey, and today we're christening our new podcast studio in the Newton, and we have Beyang and Steve from Sourcegraph. Welcome. [00:00:25]Beyang: Hey, thanks for having us. [00:00:26]Swyx: So this has been a long time coming. I'm very excited to have you. We also are just celebrating the one year anniversary of ChatGPT yesterday, but also we'll be talking about the GA of Cody later on today. We'll just do a quick intros of both of you. Obviously, people can research you and check the show notes for more. Beyang, you worked in computer vision at Stanford and then you worked at Palantir. I did, yeah. You also interned at Google. [00:00:48]Beyang: I did back in the day where I get to use Steve's system, DevTool. [00:00:53]Swyx: Right. What was it called? [00:00:55]Beyang: It was called Grok. Well, the end user thing was Google Code Search. That's what everyone called it, or just like CS. But the brains of it were really the kind of like Trigram index and then Grok, which provided the reference graph. [00:01:07]Steve: Today it's called Kythe, the open source Google one. It's sort of like Grok v3. [00:01:11]Swyx: On your podcast, which you've had me on, you've interviewed a bunch of other code search developers, including the current developer of Kythe, right? [00:01:19]Beyang: No, we didn't have any Kythe people on, although we would love to if they're up for it. We had Kelly Norton, who built a similar system at Etsy, it's an open source project called Hound. We also had Han-Wen Nienhuys, who created Zoekt, which is, I think, heavily inspired by the Trigram index that powered Google's original code search and that we also now use at Sourcegraph. Yeah. [00:01:45]Swyx: So you teamed up with Quinn over 10 years ago to start Sourcegraph and you were indexing all code on the internet. And now you're in a perfect spot to create a code intelligence startup. Yeah, yeah. [00:01:56]Beyang: I guess the backstory was, I used Google Code Search while I was an intern. And then after I left that internship and worked elsewhere, it was the single dev tool that I missed the most. I felt like my job was just a lot more tedious and much more of a hassle without it. And so when Quinn and I started working together at Palantir, he had also used various code search engines in open source over the years. And it was just a pain point that we both felt, both working on code at Palantir and also working within Palantir's clients, which were a lot of Fortune 500 companies, large financial institutions, folks like that. And if anything, the pains they felt in dealing with large complex code bases made our pain points feel small by comparison. So that was really the impetus for starting Sourcegraph. [00:02:42]Swyx: Yeah, excellent. Steve, you famously worked at Amazon. And you've told many, many stories. I want every single listener of Latent Space to check out Steve's YouTube because he effectively had a podcast that you didn't tell anyone about or something. You just hit record and just went on a few rants. I'm always here for your Stevie rants. And then you moved to Google, where you also had some interesting thoughts on just the overall Google culture versus Amazon. You joined Grab as head of eng for a couple of years. I'm from Singapore, so I have actually personally used a lot of Grab's features. And it was very interesting to see you talk so highly of Grab's engineering and sort of overall prospects. [00:03:21]Steve: Because as a customer, it sucked? [00:03:22]Swyx: Yeah, no, it's just like, being from a smaller country, you never see anyone from our home country being on a global stage or talked about as a startup that people admire or look up to, like on the league that you, with all your legendary experience, would consider equivalent. Yeah. [00:03:41]Steve: Yeah, no, absolutely. They actually, they didn't even know that they were as good as they were, in a sense. They started hiring a bunch of people from Silicon Valley to come in and sort of like fix it. And we came in and we were like, Oh, we could have been a little better operational excellence and stuff. But by and large, they're really sharp. The only thing about Grab is that they get criticized a lot for being too westernized. Oh, by who? By Singaporeans who don't want to work there. [00:04:06]Swyx: Okay. I guess I'm biased because I'm here, but I don't see that as a problem. If anything, they've had their success because they were more westernized than the Sanders Singaporean tech company. [00:04:15]Steve: I mean, they had their success because they are laser focused. They copy to Amazon. I mean, they're executing really, really, really well for a giant. I was on a slack with 2,500 engineers. It was like this giant waterfall that you could dip your toe into. You'd never catch up. Actually, the AI summarizers would have been really helpful there. But yeah, no, I think Grab is successful because they're just out there with their sleeves rolled up, just making it happen. [00:04:43]Swyx: And for those who don't know, it's not just like Uber of Southeast Asia, it's also a super app. PayPal Plus. [00:04:48]Steve: Yeah. [00:04:49]Swyx: In the way that super apps don't exist in the West. It's one of the enduring mysteries of B2C that super apps work in the East and don't work in the West. We just don't understand it. [00:04:57]Beyang: Yeah. [00:04:58]Steve: It's just kind of curious. They didn't work in India either. And it was primarily because of bandwidth reasons and smaller phones. [00:05:03]Swyx: That should change now. It should. [00:05:05]Steve: And maybe we'll see a super app here. [00:05:08]Swyx: You retired-ish? I did. You retired-ish on your own video game? Mm-hmm. Any fun stories about that? And that's also where you discovered some need for code search, right? Mm-hmm. [00:05:16]Steve: Sure. A need for a lot of stuff. Better programming languages, better databases. Better everything. I mean, I started in like 95, right? Where there was kind of nothing. Yeah. Yeah. [00:05:24]Beyang: I just want to say, I remember when you first went to Grab because you wrote that blog post talking about why you were excited about it, about like the expanding Asian market. And our reaction was like, oh, man, how did we miss stealing it with you? [00:05:36]Swyx: Hiring you. [00:05:37]Beyang: Yeah. [00:05:38]Steve: I was like, miss that. [00:05:39]Swyx: Tell that story. So how did this happen? Right? So you were inspired by Grok. [00:05:44]Beyang: I guess the backstory from my point of view is I had used code search and Grok while at Google, but I didn't actually know that it was connected to you, Steve. I knew you from your blog posts, which were always excellent, kind of like inside, very thoughtful takes from an engineer's perspective on some of the challenges facing tech companies and tech culture and that sort of thing. But my first introduction to you within the context of code intelligence, code understanding was I watched a talk that you gave, I think at Stanford, about Grok when you're first building it. And that was very eye opening. I was like, oh, like that guy, like the guy who, you know, writes the extremely thoughtful ranty like blog posts also built that system. And so that's how I knew, you know, you were involved in that. And then, you know, we always wanted to hire you, but never knew quite how to approach you or, you know, get that conversation started. [00:06:34]Steve: Well, we got introduced by Max, right? Yeah. It was temporal. Yeah. Yeah. I mean, it was a no brainer. They called me up and I had noticed when Sourcegraph had come out. Of course, when they first came out, I had this dagger of jealousy stabbed through me piercingly, which I remember because I am not a jealous person by any means, ever. But boy, I was like, but I was kind of busy, right? And just one thing led to another. I got sucked back into the ads vortex and whatever. So thank God Sourcegraph actually kind of rescued me. [00:07:05]Swyx: Here's a chance to build DevTools. Yeah. [00:07:08]Steve: That's the best. DevTools are the best. [00:07:10]Swyx: Cool. Well, so that's the overall intro. I guess we can get into Cody. Is there anything else that like people should know about you before we get started? [00:07:18]Steve: I mean, everybody knows I'm a musician. I can juggle five balls. [00:07:24]Swyx: Five is good. Five is good. I've only ever managed three. [00:07:27]Steve: Five is hard. Yeah. And six, a little bit. [00:07:30]Swyx: Wow. [00:07:31]Beyang: That's impressive. [00:07:32]Alessio: So yeah, to jump into Sourcegraph, this has been a company 10 years in the making. And as Sean said, now you're at the right place. Phase two. Now, exactly. You spent 10 years collecting all this code, indexing, making it easy to surface it. Yeah. [00:07:47]Swyx: And also learning how to work with enterprises and having them trust you with their code bases. Yeah. [00:07:52]Alessio: Because initially you were only doing on-prem, right? Like a lot of like VPC deployments. [00:07:55]Beyang: So in the very early days, we're cloud only. But the first major customers we landed were all on-prem, self-hosted. And that was, I think, related to the nature of the problem that we're solving, which becomes just like a critical, unignorable pain point once you're above like 100 devs or so. [00:08:11]Alessio: Yeah. And now Cody is going to be GA by the time this releases. So congrats to your future self for launching this in two weeks. Can you give a quick overview of just what Cody is? I think everybody understands that it's a AI coding agent, but a lot of companies say they have a AI coding agent. So yeah, what does Cody do? How do people interface with it? [00:08:32]Beyang: Yeah. So how is it different from the like several dozen other AI coding agents that exist in the market now? When we thought about building a coding assistant that would do things like code generation and question answering about your code base, I think we came at it from the perspective of, you know, we've spent the past decade building the world's best code understanding engine for human developers, right? So like it's kind of your guide as a human dev if you want to go and dive into a large complex code base. And so our intuition was that a lot of the context that we're providing to human developers would also be useful context for AI developers to consume. And so in terms of the feature set, Cody is very similar to a lot of other assistants. It does inline autocompletion. It does code base aware chat. It does specific commands that automate, you know, tasks that you might rather not want to do like generating unit tests or adding detailed documentation. But we think the core differentiator is really the quality of the context, which is hard to kind of describe succinctly. It's a bit like saying, you know, what's the difference between Google and Alta Vista? There's not like a quick checkbox list of features that you can rattle off, but it really just comes down to all the attention and detail that we've paid to making that context work well and be high quality and fast for human devs. We're now kind of plugging into the AI coding assistant as well. Yeah. [00:09:53]Steve: I mean, just to add my own perspective on to what Beyang just described, RAG is kind of like a consultant that the LLM has available, right, that knows about your code. RAG provides basically a bridge to a lookup system for the LLM, right? Whereas fine tuning would be more like on the job training for somebody. If the LLM is a person, you know, and you send them to a new job and you do on the job training, that's what fine tuning is like, right? So tuned to our specific task. You're always going to need that expert, even if you get the on the job training, because the expert knows your particular code base, your task, right? That expert has to know your code. And there's a chicken and egg problem because, right, you know, we're like, well, I'm going to ask the LLM about my code, but first I have to explain it, right? It's this chicken and egg problem. That's where RAG comes in. And we have the best consultants, right? The best assistant who knows your code. And so when you sit down with Cody, right, what Beyang said earlier about going to Google and using code search and then starting to feel like without it, his job was super tedious. Once you start using these, do you guys use coding assistants? [00:10:53]Swyx: Yeah, right. [00:10:54]Steve: I mean, like we're getting to the point very quickly, right? Where you feel like almost like you're programming without the internet, right? Or something, you know, it's like you're programming back in the nineties without the coding assistant. Yeah. Hopefully that helps for people who have like no idea about coding systems, what they are. [00:11:09]Swyx: Yeah. [00:11:10]Alessio: I mean, going back to using them, we had a lot of them on the podcast already. We had Cursor, we have Codium and Codium, very similar names. [00:11:18]Swyx: Yeah. Find, and then of course there's Copilot. [00:11:22]Alessio: You had a Copilot versus Cody blog post, and I think it really shows the context improvement. So you had two examples that stuck with me. One was, what does this application do? And the Copilot answer was like, oh, it uses JavaScript and NPM and this. And it's like, but that's not what it does. You know, that's what it's built with. Versus Cody was like, oh, these are like the major functions. And like, these are the functionalities and things like that. And then the other one was, how do I start this up? And Copilot just said NPM start, even though there was like no start command in the package JSON, but you know, most collapse, right? Most projects use NPM start. So maybe this does too. How do you think about open source models? Because Copilot has their own private thing. And I think you guys use Starcoder, if I remember right. Yeah, that's correct. [00:12:09]Beyang: I think Copilot uses some variant of Codex. They're kind of cagey about it. I don't think they've like officially announced what model they use. [00:12:16]Swyx: And I think they use a range of models based on what you're doing. Yeah. [00:12:19]Beyang: So everyone uses a range of model. Like no one uses the same model for like inline completion versus like chat because the latency requirements for. Oh, okay. Well, there's fill in the middle. There's also like what the model's trained on. So like we actually had completions powered by Claude Instant for a while. And but you had to kind of like prompt hack your way to get it to output just the code and not like, hey, you know, here's the code you asked for, like that sort of text. So like everyone uses a range of models. We've kind of designed Cody to be like especially model, not agnostic, but like pluggable. So one of our kind of design considerations was like as the ecosystem evolves, we want to be able to integrate the best in class models, whether they're proprietary or open source into Cody because the pace of innovation in the space is just so quick. And I think that's been to our advantage. Like today, Cody uses Starcoder for inline completions. And with the benefit of the context that we provide, we actually show comparable completion acceptance rate metrics. It's kind of like the standard metric that folks use to evaluate inline completion quality. It's like if I show you a completion, what's the chance that you actually accept the completion versus you reject it? And so we're at par with Copilot, which is at the head of that industry right now. And we've been able to do that with the Starcoder model, which is open source and the benefit of the context fetching stuff that we provide. And of course, a lot of like prompt engineering and other stuff along the way. [00:13:40]Alessio: And Steve, you wrote a post called cheating is all you need about what you're building. And one of the points you made is that everybody's fighting on the same axis, which is better UI and the IDE, maybe like a better chat response. But data modes are kind of the most important thing. And you guys have like a 10 year old mode with all the data you've been collecting. How do you kind of think about what other companies are doing wrong, right? Like, why is nobody doing this in terms of like really focusing on RAG? I feel like you see so many people. Oh, we just got a new model. It's like a bit human eval. And it's like, well, but maybe like that's not what we should really be doing, you know? Like, do you think most people underestimate the importance of like the actual RAG in code? [00:14:21]Steve: I think that people weren't doing it much. It wasn't. It's kind of at the edges of AI. It's not in the center. I know that when ChatGPT launched, so within the last year, I've heard a lot of rumblings from inside of Google, right? Because they're undergoing a huge transformation to try to, you know, of course, get into the new world. And I heard that they told, you know, a bunch of teams to go and train their own models or fine tune their own models, right? [00:14:43]Swyx: Both. [00:14:43]Steve: And, you know, it was a s**t show. Nobody knew how to do it. They launched two coding assistants. One was called Code D with an EY. And then there was, I don't know what happened in that one. And then there's Duet, right? Google loves to compete with themselves, right? They do this all the time. And they had a paper on Duet like from a year ago. And they were doing exactly what Copilot was doing, which was just pulling in the local context, right? But fundamentally, I thought of this because we were talking about the splitting of the [00:15:10]Swyx: models. [00:15:10]Steve: In the early days, it was the LLM did everything. And then we realized that for certain use cases, like completions, that a different, smaller, faster model would be better. And that fragmentation of models, actually, we expected to continue and proliferate, right? Because we are fundamentally, we're a recommender engine right now. Yeah, we're recommending code to the LLM. We're saying, may I interest you in this code right here so that you can answer my question? [00:15:34]Swyx: Yeah? [00:15:34]Steve: And being good at recommender engine, I mean, who are the best recommenders, right? There's YouTube and Spotify and, you know, Amazon or whatever, right? Yeah. [00:15:41]Swyx: Yeah. [00:15:41]Steve: And they all have many, many, many, many, many models, right? For all fine-tuned for very specific, you know. And that's where we're heading in code, too. Absolutely. [00:15:50]Swyx: Yeah. [00:15:50]Alessio: We just did an episode we released on Wednesday, which we said RAG is like Rexis or like LLMs. You're basically just suggesting good content. [00:15:58]Swyx: It's like what? Recommendations. [00:15:59]Beyang: Recommendations. [00:16:00]Alessio: Oh, got it. [00:16:01]Steve: Yeah, yeah, yeah. [00:16:02]Swyx: So like the naive implementation of RAG is you embed everything, throw it in a vector database, you embed your query, and then you find the nearest neighbors, and that's your RAG. But actually, you need to rank it. And actually, you need to make sure there's sample diversity and that kind of stuff. And then you're like slowly gradient dissenting yourself towards rediscovering proper Rexis, which has been traditional ML for a long time. But like approaching it from an LLM perspective. Yeah. [00:16:24]Beyang: I almost think of it as like a generalized search problem because it's a lot of the same things. Like you want your layer one to have high recall and get all the potential things that could be relevant. And then there's typically like a layer two re-ranking mechanism that bumps up the precision and tries to get the relevant stuff to the top of the results list. [00:16:43]Swyx: Have you discovered that ranking matters a lot? Oh, yeah. So the context is that I think a lot of research shows that like one, context utilization matters based on model. Like GPT uses the top of the context window, and then apparently Claude uses the bottom better. And it's lossy in the middle. Yeah. So ranking matters. No, it really does. [00:17:01]Beyang: The skill with which models are able to take advantage of context is always going to be dependent on how that factors into the impact on the training loss. [00:17:10]Swyx: Right? [00:17:10]Beyang: So like if you want long context window models to work well, then you have to have a ton of data where it's like, here's like a billion lines of text. And I'm going to ask a question about like something that's like, you know, embedded deeply into it and like, give me the right answer. And unless you have that training set, then of course, you're going to have variability in terms of like where it attends to. And in most kind of like naturally occurring data, the thing that you're talking about right now, the thing I'm asking you about is going to be something that we talked about recently. [00:17:36]Swyx: Yeah. [00:17:36]Steve: Did you really just say gradient dissenting yourself? Actually, I love that it's entered the casual lexicon. Yeah, yeah, yeah. [00:17:44]Swyx: My favorite version of that is, you know, how we have to p-hack papers. So, you know, when you throw humans at the problem, that's called graduate student dissent. That's great. It's really awesome. [00:17:54]Alessio: I think the other interesting thing that you have is this inline assist UX that I wouldn't say async, but like it works while you can also do work. So you can ask Cody to make changes on a code block and you can still edit the same file at the same time. [00:18:07]Swyx: Yeah. [00:18:07]Alessio: How do you see that in the future? Like, do you see a lot of Cody's running together at the same time? Like, how do you validate also that they're not messing each other up as they make changes in the code? And maybe what are the limitations today? And what do you think about where the attack is going? [00:18:21]Steve: I want to start with a little history and then I'm going to turn it over to Bian, all right? So we actually had this feature in the very first launch back in June. Dominic wrote it. It was called nonstop Cody. And you could have multiple, basically, LLM requests in parallel modifying your source [00:18:37]Swyx: file. [00:18:37]Steve: And he wrote a bunch of code to handle all of the diffing logic. And you could see the regions of code that the LLM was going to change, right? And he was showing me demos of it. And it just felt like it was just a little before its time, you know? But a bunch of that stuff, that scaffolding was able to be reused for where we're inline [00:18:56]Swyx: sitting today. [00:18:56]Steve: How would you characterize it today? [00:18:58]Beyang: Yeah, so that interface has really evolved from a, like, hey, general purpose, like, request anything inline in the code and have the code update to really, like, targeted features, like, you know, fix the bug that exists at this line or request a very specific [00:19:13]Swyx: change. [00:19:13]Beyang: And the reason for that is, I think, the challenge that we ran into with inline fixes, and we do want to get to the point where you could just fire and forget and have, you know, half a dozen of these running in parallel. But I think we ran into the challenge early on that a lot of people are running into now when they're trying to construct agents, which is the reliability of, you know, working code generation is just not quite there yet in today's language models. And so that kind of constrains you to an interaction where the human is always, like, in the inner loop, like, checking the output of each response. And if you want that to work in a way where you can be asynchronous, you kind of have to constrain it to a domain where today's language models can generate reliable code well enough. So, you know, generating unit tests, that's, like, a well-constrained problem. Or fixing a bug that shows up as, like, a compiler error or a test error, that's a well-constrained problem. But the more general, like, hey, write me this class that does X, Y, and Z using the libraries that I have, that is not quite there yet, even with the benefit of really good context. Like, it definitely moves the needle a lot, but we're not quite there yet to the point where you can just fire and forget. And I actually think that this is something that people don't broadly appreciate yet, because I think that, like, everyone's chasing this dream of agentic execution. And if we're to really define that down, I think it implies a couple things. You have, like, a multi-step process where each step is fully automated. We don't have to have a human in the loop every time. And there's also kind of like an LM call at each stage or nearly every stage in that [00:20:45]Swyx: chain. [00:20:45]Beyang: Based on all the work that we've done, you know, with the inline interactions, with kind of like general Codyfeatures for implementing longer chains of thought, we're actually a little bit more bearish than the average, you know, AI hypefluencer out there on the feasibility of agents with purely kind of like transformer-based models. To your original question, like, the inline interactions with CODI, we actually constrained it to be more targeted, like, you know, fix the current error or make this quick fix. I think that that does differentiate us from a lot of the other tools on the market, because a lot of people are going after this, like, shnazzy, like, inline edit interaction, whereas I think where we've moved, and this is based on the user feedback that we've gotten, it's like that sort of thing, it demos well, but when you're actually coding day to day, you don't want to have, like, a long chat conversation inline with the code base. That's a waste of time. You'd rather just have it write the right thing and then move on with your life or not have to think about it. And that's what we're trying to work towards. [00:21:37]Steve: I mean, yeah, we're not going in the agent direction, right? I mean, I'll believe in agents when somebody shows me one that works. Yeah. Instead, we're working on, you know, sort of solidifying our strength, which is bringing the right context in. So new context sources, ways for you to plug in your own context, ways for you to control or influence the context, you know, the mixing that happens before the request goes out, etc. And there's just so much low-hanging fruit left in that space that, you know, agents seems like a little bit of a boondoggle. [00:22:03]Beyang: Just to dive into that a little bit further, like, I think, you know, at a very high level, what do people mean when they say agents? They really mean, like, greater automation, fully automated, like, the dream is, like, here's an issue, go implement that. And I don't have to think about it as a human. And I think we are working towards that. Like, that is the eventual goal. I think it's specifically the approach of, like, hey, can we have a transformer-based LM alone be the kind of, like, backbone or the orchestrator of these agentic flows? Where we're a little bit more bearish today. [00:22:31]Swyx: You want the human in the loop. [00:22:32]Beyang: I mean, you kind of have to. It's just a reality of the behavior of language models that are purely, like, transformer-based. And I think that's just like a reflection of reality. And I don't think people realize that yet. Because if you look at the way that a lot of other AI tools have implemented context fetching, for instance, like, you see this in the Copilot approach, where if you use, like, the at-workspace thing that supposedly provides, like, code-based level context, it has, like, an agentic approach where you kind of look at how it's behaving. And it feels like they're making multiple requests to the LM being like, what would you do in this case? Would you search for stuff? What sort of files would you gather? Go and read those files. And it's like a multi-hop step, so it takes a long while. It's also non-deterministic. Because any sort of, like, LM invocation, it's like a dice roll. And then at the end of the day, the context it fetches is not that good. Whereas our approach is just like, OK, let's do some code searches that make sense. And then maybe, like, crawl through the reference graph a little bit. That is fast. That doesn't require any sort of LM invocation at all. And we can pull in much better context, you know, very quickly. So it's faster. [00:23:37]Swyx: It's more reliable. [00:23:37]Beyang: It's deterministic. And it yields better context quality. And so that's what we think. We just don't think you should cargo cult or naively go like, you know, agents are the [00:23:46]Swyx: future. [00:23:46]Beyang: Let's just try to, like, implement agents on top of the LM that exists today. I think there are a couple of other technologies or approaches that need to be refined first before we can get into these kind of, like, multi-stage, fully automated workflows. [00:24:00]Swyx: It makes sense. You know, we're very much focused on developer inner loop right now. But you do see things eventually moving towards developer outer loop. Yeah. So would you basically say that they're tackling the agent's problem that you don't want to tackle? [00:24:11]Beyang: No, I would say at a high level, we are after maybe, like, the same high level problem, which is like, hey, I want some code written. I want to develop some software and can automate a system. Go build that software for me. I think the approaches might be different. So I think the analogy in my mind is, I think about, like, the AI chess players. Coding, in some senses, I mean, it's similar and dissimilar to chess. I think one question I ask is, like, do you think producing code is more difficult than playing chess or less difficult than playing chess? More. [00:24:41]Swyx: I think more. [00:24:41]Beyang: Right. And if you look at the best AI chess players, like, yes, you can use an LLM to play chess. Like, people have showed demos where it's like, oh, like, yeah, GPT-4 is actually a pretty decent, like, chess move suggester. Right. But you would never build, like, a best in class chess player off of GPT-4 alone. [00:24:57]Swyx: Right. [00:24:57]Beyang: Like, the way that people design chess players is that you have kind of like a search space and then you have a way to explore that search space efficiently. There's a bunch of search algorithms, essentially. We were doing tree search in various ways. And you can have heuristic functions, which might be powered by an LLM. [00:25:12]Swyx: Right. [00:25:12]Beyang: Like, you might use an LLM to generate proposals in that space that you can efficiently explore. But the backbone is still this kind of more formalized tree search based approach rather than the LLM itself. And so I think my high level intuition is that, like, the way that we get to more reliable multi-step workflows that do things beyond, you know, generate unit test, it's really going to be like a search based approach where you use an LLM as kind of like an advisor or a proposal function, sort of your heuristic function, like the ASTAR search algorithm. But it's probably not going to be the thing that is the backbone, because I guess it's not the right tool for that. Yeah. [00:25:50]Swyx: I can see yourself kind of thinking through this, but not saying the words, the sort of philosophical Peter Norvig type discussion. Maybe you want to sort of introduce that in software. Yeah, definitely. [00:25:59]Beyang: So your listeners are savvy. They're probably familiar with the classic like Chomsky versus Norvig debate. [00:26:04]Swyx: No, actually, I wanted, I was prompting you to introduce that. Oh, got it. [00:26:08]Beyang: So, I mean, if you look at the history of artificial intelligence, right, you know, it goes way back to, I don't know, it's probably as old as modern computers, like 50s, 60s, 70s. People are debating on like, what is the path to producing a sort of like general human level of intelligence? And kind of two schools of thought that emerged. One is the Norvig school of thought, which roughly speaking includes large language models, you know, regression, SVN, basically any model that you kind of like learn from data. And it's like data driven. Most of machine learning would fall under this umbrella. And that school of thought says like, you know, just learn from the data. That's the approach to reaching intelligence. And then the Chomsky approach is more things like compilers and parsers and formal systems. So basically like, let's think very carefully about how to construct a formal, precise system. And that will be the approach to how we build a truly intelligent system. I think Lisp was invented so that you could create like rules-based systems that you would call AI. As a language. Yeah. And for a long time, there was like this debate, like there's certain like AI research labs that were more like, you know, in the Chomsky camp and others that were more in the Norvig camp. It's a debate that rages on today. And I feel like the consensus right now is that, you know, Norvig definitely has the upper hand right now with the advent of LMs and diffusion models and all the other recent progress in machine learning. But the Chomsky-based stuff is still really useful in my view. I mean, it's like parsers, compilers, basically a lot of the stuff that provides really good context. It provides kind of like the knowledge graph backbone that you want to explore with your AI dev tool. Like that will come from kind of like Chomsky-based tools like compilers and parsers. It's a lot of what we've invested in in the past decade at Sourcegraph and what you build with Grok. Basically like these formal systems that construct these very precise knowledge graphs that are great context providers and great kind of guard rails enforcers and kind of like safety checkers for the output of a more kind of like data-driven, fuzzier system that uses like the Norvig-based models. [00:28:03]Steve: Jang was talking about this stuff like it happened in the middle ages. Like, okay, so when I was in college, I was in college learning Lisp and prologue and planning and all the deterministic Chomsky approaches to AI. And I was there when Norvig basically declared it dead. I was there 3,000 years ago when Norvig and Chomsky fought on the volcano. When did he declare it dead? [00:28:26]Swyx: What do you mean he declared it dead? [00:28:27]Steve: It was like late 90s. [00:28:29]Swyx: Yeah. [00:28:29]Steve: When I went to Google, Peter Norvig was already there. He had basically like, I forget exactly where. It was some, he's got so many famous short posts, you know, amazing. [00:28:38]Swyx: He had a famous talk, the unreasonable effectiveness of data. Yeah. [00:28:41]Steve: Maybe that was it. But at some point, basically, he basically convinced everybody that deterministic approaches had failed and that heuristic-based, you know, data-driven statistical approaches, stochastic were better. [00:28:52]Swyx: Yeah. [00:28:52]Steve: The primary reason I can tell you this, because I was there, was that, was that, well, the steam-powered engine, no. The reason was that the deterministic stuff didn't scale. [00:29:06]Swyx: Yeah. Right. [00:29:06]Steve: They're using prologue, man, constraint systems and stuff like that. Well, that was a long time ago, right? Today, actually, these Chomsky-style systems do scale. And that's, in fact, exactly what Sourcegraph has built. Yeah. And so we have a very unique, I love the framing that Bjong's made, that the marriage of the Chomsky and the Norvig, you know, sort of models, you know, conceptual models, because we, you know, we have both of them and they're both really important. And in fact, there, there's this really interesting, like, kind of overlap between them, right? Where like the AI or our graph or our search engine could potentially provide the right context for any given query, which is, of course, why ranking is important. But what we've really signed ourselves up for is an extraordinary amount of testing. [00:29:45]Swyx: Yeah. [00:29:45]Steve: Because in SWIGs, you were saying that, you know, GPT-4 tends to the front of the context window and maybe other LLMs to the back and maybe, maybe the LLM in the middle. [00:29:53]Swyx: Yeah. [00:29:53]Steve: And so that means that, you know, if we're actually like, you know, verifying whether we, you know, some change we've made has improved things, we're going to have to test putting it at the beginning of the window and at the end of the window, you know, and maybe make the right decision based on the LLM that you've chosen. Which some of our competitors, that's a problem that they don't have, but we meet you, you know, where you are. Yeah. And we're, just to finish, we're writing tens of thousands. We're generating tests, you know, fill in the middle type tests and things. And then using our graph to basically sort of fine tune Cody's behavior there. [00:30:20]Swyx: Yeah. [00:30:21]Beyang: I also want to add, like, I have like an internal pet name for this, like kind of hybrid architecture that I'm trying to make catch on. Maybe I'll just say it here. Just saying it publicly kind of makes it more real. But like, I call the architecture that we've developed the Normsky architecture. [00:30:36]Swyx: Yeah. [00:30:36]Beyang: I mean, it's obviously a portmanteau of Norvig and Chomsky, but the acronym, it stands for non-agentic, rapid, multi-source code intelligence. So non-agentic because... Rolls right off the tongue. And Normsky. But it's non-agentic in the sense that like, we're not trying to like pitch you on kind of like agent hype, right? Like it's the things it does are really just developer tools developers have been using for decades now, like parsers and really good search indexes and things like that. Rapid because we place an emphasis on speed. We don't want to sit there waiting for kind of like multiple LLM requests to return to complete a simple user request. Multi-source because we're thinking broadly about what pieces of information and knowledge are useful context. So obviously starting with things that you can search in your code base, and then you add in the reference graph, which kind of like allows you to crawl outward from those initial results. But then even beyond that, you know, sources of information, like there's a lot of knowledge that's embedded in docs, in PRDs or product specs, in your production logging system, in your chat, in your Slack channel, right? Like there's so much context is embedded there. And when you're a human developer, and you're trying to like be productive in your code base, you're going to go to all these different systems to collect the context that you need to figure out what code you need to write. And I don't think the AI developer will be any different. It will need to pull context from all these different sources. So we're thinking broadly about how to integrate these into Codi. We hope through kind of like an open protocol that like others can extend and implement. And this is something else that should be accessible by December 14th in kind of like a preview stage. But that's really about like broadening this notion of the code graph beyond your Git repository to all the other sources where technical knowledge and valuable context can live. [00:32:21]Steve: Yeah, it becomes an artifact graph, right? It can link into your logs and your wikis and any data source, right? [00:32:27]Alessio: How do you guys think about the importance of, it's almost like data pre-processing in a way, which is bring it all together, tie it together, make it ready. Any thoughts on how to actually make that good? Some of the innovation you guys have made. [00:32:40]Steve: We talk a lot about the context fetching, right? I mean, there's a lot of ways you could answer this question. But, you know, we've spent a lot of time just in this podcast here talking about context fetching. But stuffing the context into the window is, you know, the bin packing problem, right? Because the window is not big enough, and you've got more context than you can fit. You've got a ranker maybe. But what is that context? Is it a function that was returned by an embedding or a graph call or something? Do you need the whole function? Or do you just need, you know, the top part of the function, this expression here, right? You know, so that art, the golf game of trying to, you know, get each piece of context down into its smallest state, possibly even summarized by another model, right, before it even goes to the LLM, becomes this is the game that we're in, yeah? And so, you know, recursive summarization and all the other techniques that you got to use to like stuff stuff into that context window become, you know, critically important. And you have to test them across every configuration of models that you could possibly need. [00:33:32]Beyang: I think data preprocessing is probably the like unsexy, way underappreciated secret to a lot of the cool stuff that people are shipping today. Whether you're doing like RAG or fine tuning or pre-training, like the preprocessing step matters so much because it's basically garbage in, garbage out, right? Like if you're feeding in garbage to the model, then it's going to output garbage. Concretely, you know, for code RAG, if you're not doing some sort of like preprocessing that takes advantage of a parser and is able to like extract the key components of a particular file of code, you know, separate the function signature from the body, from the doc string, what are you even doing? Like that's like table stakes. It opens up so much more possibilities with which you can kind of like tune your system to take advantage of the signals that come from those different parts of the code. Like we've had a tool, you know, since computers were invented that understands the structure of source code to a hundred percent precision. The compiler knows everything there is to know about the code in terms of like structure. Like why would you not want to use that in a system that's trying to generate code, answer questions about code? You shouldn't throw that out the window just because now we have really good, you know, data-driven models that can do other things. [00:34:44]Steve: Yeah. When I called it a data moat, you know, in my cheating post, a lot of people were confused, you know, because data moat sort of sounds like data lake because there's data and water and stuff. I don't know. And so they thought that we were sitting on this giant mountain of data that we had collected, but that's not what our data moat is. It's really a data pre-processing engine that can very quickly and scalably, like basically dissect your entire code base in a very small, fine-grained, you know, semantic unit and then serve it up. Yeah. And so it's really, it's not a data moat. It's a data pre-processing moat, I guess. [00:35:15]Beyang: Yeah. If anything, we're like hypersensitive to customer data privacy requirements. So it's not like we've taken a bunch of private data and like, you know, trained a generally available model. In fact, exactly the opposite. A lot of our customers are choosing Cody over Copilot and other competitors because we have an explicit guarantee that we don't do any of that. And that we've done that from day one. Yeah. I think that's a very real concern in today's day and age, because like if your proprietary IP finds its way into the training set of any model, it's very easy both to like extract that knowledge from the model and also use it to, you know, build systems that kind of work on top of the institutional knowledge that you've built up. [00:35:52]Alessio: About a year ago, I wrote a post on LLMs for developers. And one of the points I had was maybe the depth of like the DSL. I spent most of my career writing Ruby and I love Ruby. It's so nice to use, but you know, it's not as performant, but it's really easy to read, right? And then you look at other languages, maybe they're faster, but like they're more verbose, you know? And when you think about efficiency of the context window, that actually matters. [00:36:15]Swyx: Yeah. [00:36:15]Alessio: But I haven't really seen a DSL for models, you know? I haven't seen like code being optimized to like be easier to put in a model context. And it seems like your pre-processing is kind of doing that. Do you see in the future, like the way we think about the DSL and APIs and kind of like service interfaces be more focused on being context friendly, where it's like maybe it's harder to read for the human, but like the human is never going to write it anyway. We were talking on the Hacks podcast. There are like some data science things like spin up the spandex, like humans are never going to write again because the models can just do very easily. Yeah, curious to hear your thoughts. [00:36:51]Steve: Well, so DSLs, they involve, you know, writing a grammar and a parser and they're like little languages, right? We do them that way because, you know, we need them to compile and humans need to be able to read them and so on. The LLMs don't need that level of structure. You can throw any pile of crap at them, you know, more or less unstructured and they'll deal with it. So I think that's why a DSL hasn't emerged for sort of like communicating with the LLM or packaging up the context or anything. Maybe it will at some point, right? We've got, you know, tagging of context and things like that that are sort of peeking into DSL territory, right? But your point on do users, you know, do people have to learn DSLs like regular expressions or, you know, pick your favorite, right? XPath. I think you're absolutely right that the LLMs are really, really good at that. And I think you're going to see a lot less of people having to slave away learning these things. They just have to know the broad capabilities and the LLM will take care of the rest. [00:37:42]Swyx: Yeah, I'd agree with that. [00:37:43]Beyang: I think basically like the value profit of DSL is that it makes it easier to work with a lower level language, but at the expense of introducing an abstraction layer. And in many cases today, you know, without the benefit of AI cogeneration, like that totally worth it, right? With the benefit of AI cogeneration, I mean, I don't think all DSLs will go away. I think there's still, you know, places where that trade-off is going to be worthwhile. But it's kind of like how much of source code do you think is going to be generated through natural language prompting in the future? Because in a way, like any programming language is just a DSL on top of assembly, right? And so if people can do that, then yeah, like maybe for a large portion of the code [00:38:21]Swyx: that's written, [00:38:21]Beyang: people don't actually have to understand the DSL that is Ruby or Python or basically any other programming language that exists. [00:38:28]Steve: I mean, seriously, do you guys ever write SQL queries now without using a model of some sort? At least a draft. [00:38:34]Swyx: Yeah, right. [00:38:36]Steve: And so we have kind of like, you know, past that bridge, right? [00:38:39]Alessio: Yeah, I think like to me, the long-term thing is like, is there ever going to be, you don't actually see the code, you know? It's like, hey, the basic thing is like, hey, I need a function to some two numbers and that's it. I don't need you to generate the code. [00:38:53]Steve: And the following question, do you need the engineer or the paycheck? [00:38:56]Swyx: I mean, right? [00:38:58]Alessio: That's kind of the agent's discussion in a way where like you cannot automate the agents, but like slowly you're getting more of the atomic units of the work kind of like done. I kind of think of it as like, you know, [00:39:09]Beyang: do you need a punch card operator to answer that for you? And so like, I think we're still going to have people in the role of a software engineer, but the portion of time they spend on these kinds of like low-level, tedious tasks versus the higher level, more creative tasks is going to shift. [00:39:23]Steve: No, I haven't used punch cards. [00:39:25]Swyx: Yeah, I've been talking about like, so we kind of made this podcast about the sort of rise of the AI engineer. And like the first step is the AI enhanced engineer. That is that software developer that is no longer doing these routine, boilerplate-y type tasks, because they're just enhanced by tools like yours. So you mentioned OpenCodeGraph. I mean, that is a kind of DSL maybe, and because we're releasing this as you go GA, you hope for other people to take advantage of that? [00:39:52]Beyang: Oh yeah, I would say so OpenCodeGraph is not a DSL. It's more of a protocol. It's basically like, hey, if you want to make your system, whether it's, you know, chat or logging or whatever accessible to an AI developer tool like Cody, here's kind of like the schema by which you can provide that context and offer hints. So I would, you know, comparisons like LSP obviously did this for kind of like standard code intelligence. It's kind of like a lingua franca for providing fine references and codefinition. There's kind of like analogs to that. There might be also analogs to kind of the original OpenAI, kind of like plugins, API. There's all this like context out there that might be useful for an LM-based system to consume. And so at a high level, what we're trying to do is define a common language for context providers to provide context to other tools in the software development lifecycle. Yeah. Do you have any critiques of LSP, by the way, [00:40:42]Swyx: since like this is very much, very close to home? [00:40:45]Steve: One of the authors wrote a really good critique recently. Yeah. I don't think I saw that. Yeah, yeah. LSP could have been better. It just came out a couple of weeks ago. It was a good article. [00:40:54]Beyang: Yeah. I think LSP is great. Like for what it did for the developer ecosystem, it was absolutely fantastic. Like nowadays, like it's much easier now to get code navigation up and running in a bunch of editors by speaking this protocol. I think maybe the interesting question is like looking at the different design decisions comparing LSP basically with Kythe. Because Kythe has more of a... How would you describe it? [00:41:18]Steve: A storage format. [00:41:20]Beyang: I think the critique of LSP from a Kythe point of view would be like with LSP, you don't actually have an actual symbolic model of the code. It's not like LSP models like, hey, this function calls this other function. LSP is all like range-based. Like, hey, your cursor's at line 32, column 1. [00:41:35]Swyx: Yeah. [00:41:35]Beyang: And that's the thing you feed into the language server. And then it's like, okay, here's the range that you should jump to if you click on that range. So it kind of is intentionally ignorant of the fact that there's a thing called a reference underneath your cursor, and that's linked to a symbol definition. [00:41:49]Steve: Well, actually, that's the worst example you could have used. You're right. But that's the one thing that it actually did bake in is following references. [00:41:56]Swyx: Sure. [00:41:56]Steve: But it's sort of hardwired. [00:41:58]Swyx: Yeah. [00:41:58]Steve: Whereas Kythe attempts to model [00:42:00]Beyang: like all these things explicitly. [00:42:02]Swyx: And so... [00:42:02]Steve: Well, so LSP is a protocol, right? And so Google's internal protocol is gRPC-based. And it's a different approach than LSP. It's basically you make a heavy query to the back end, and you get a lot of data back, and then you render the whole page, you know? So we've looked at LSP, and we think that it's a little long in the tooth, right? I mean, it's a great protocol, lots and lots of support for it. But we need to push into the domain of exposing the intelligence through the protocol. Yeah. [00:42:29]Beyang: And so I would say we've developed a protocol of our own called Skip, which is at a very high level trying to take some of the good ideas from LSP and from Kythe and merge that into a system that in the near term is useful for Sourcegraph, but I think in the long term, we hope will be useful for the ecosystem. Okay, so here's what LSP did well. LSP, by virtue of being like intentionally dumb, dumb in air quotes, because I'm not like ragging on it, allowed language servers developers to kind of like bypass the hard problem of like modeling language semantics precisely. So like if all you want to do is jump to definition, you don't have to come up with like a universally unique naming scheme for each symbol, which is actually quite challenging because you have to think about like, okay, what's the top scope of this name? Is it the source code repository? Is it the package? Does it depend on like what package server you're fetching this from? Like whether it's the public one or the one inside your... Anyways, like naming is hard, right? And by just going from kind of like a location to location based approach, you basically just like throw that out the window. All I care about is jumping definition, just make that work. And you can make that work without having to deal with like all the complex global naming things. The limitation of that approach is that it's harder to build on top of that to build like a true knowledge graph. Like if you actually want a system that says like, okay, here's the web of functions and here's how they reference each other. And I want to incorporate that like semantic model of how the code operates or how the code relates to each other at like a static level. You can't do that with LSP because you have to deal with line ranges. And like concretely the pain point that we found in using LSP for source graph is like in order to do like a find references [00:44:04]Swyx: and then jump definitions, [00:44:04]Beyang: it's like a multi-hop process because like you have to jump to the range and then you have to find the symbol at that range. And it just adds a lot of latency and complexity of these operations where as a human, you're like, well, this thing clearly references this other thing. Why can't you just jump me to that? And I think that's the thing that Kaith does well. But then I think the issue that Kaith has had with adoption is because it is more sophisticated schema, I think. And so there's basically more things that you have to implement to get like a Kaith implementation up and running. I hope I'm not like, correct me if I'm wrong about any of this. [00:44:35]Steve: 100%, 100%. Kaith also has a problem, all these systems have the problem, even skip, or at least the way that we implemented the indexers, that they have to integrate with your build system in order to build that knowledge graph, right? Because you have to basically compile the code in a special mode to generate artifacts instead of binaries. And I would say, by the way, earlier I was saying that XREFs were in LSP, but it's actually, I was thinking of LSP plus LSIF. [00:44:58]Swyx: Yeah. That's another. [00:45:01]Steve: Which is actually bad. We can say that it's bad, right? [00:45:04]Steve: It's like skip or Kaith, it's supposed to be sort of a model serialization, you know, for the code graph, but it basically just does what LSP needs, the bare minimum. LSIF is basically if you took LSP [00:45:16]Beyang: and turned that into a serialization format. So like you build an index for language servers to kind of like quickly bootstrap from cold start. But it's a graph model [00:45:23]Steve: with all of the inconvenience of the API without an actual graph. And so, yeah. [00:45:29]Beyang: So like one of the things that we try to do with skip is try to capture the best of both worlds. So like make it easy to write an indexer, make the schema simple, but also model some of the more symbolic characteristics of the code that would allow us to essentially construct this knowledge graph that we can then make useful for both the human developer through SourceGraph and through the AI developer through Cody. [00:45:49]Steve: So anyway, just to finish off the graph comment, we've got a new graph, yeah, that's skip based. We call it BFG internally, right? It's a beautiful something graph. A big friendly graph. [00:46:00]Swyx: A big friendly graph. [00:46:01]Beyang: It's a blazing fast. [00:46:02]Steve: Blazing fast. [00:46:03]Swyx: Blazing fast graph. [00:46:04]Steve: And it is blazing fast, actually. It's really, really interesting. I should probably have to do a blog post about it to walk you through exactly how they're doing it. Oh, please. But it's a very AI-like iterative, you know, experimentation sort of approach. We're building a code graph based on all of our 10 years of knowledge about building code graphs, yeah? But we're building it quickly with zero configuration, and it doesn't have to integrate with your build. And through some magic tricks that we have. And so what just happens when you install the plugin, that it'll be there and indexing your code and providing that knowledge graph in the background without all that build system integration. This is a bit of secret sauce that we haven't really like advertised it very much lately. But I am super excited about it because what they do is they say, all right, you know, let's tackle function parameters today. Cody's not doing a very good job of completing function call arguments or function parameters in the definition, right? Yeah, we generate those thousands of tests, and then we can actually reuse those tests for the AI context as well. So fortunately, things are kind of converging on, we have, you know, half a dozen really, really good context sources, and we mix them all together. So anyway, BFG, you're going to hear more about it probably in the holidays? [00:47:12]Beyang: I think it'll be online for December 14th. We'll probably mention it. BFG is probably not the public name we're going to go with. I think we might call it like Graph Context or something like that. [00:47:20]Steve: We're officially calling it BFG. [00:47:22]Swyx: You heard it here first. [00:47:24]Beyang: BFG is just kind of like the working name. And so the impetus for BFG was like, if you look at like current AI inline code completion tools and the errors that they make, a lot of the errors that they make, even in kind of like the easy, like single line case, are essentially like type errors, right? Like you're trying to complete a function call and it suggests a variable that you defined earlier, but that variable is the wrong type. [00:47:47]Swyx: And that's the sort of thing [00:47:47]Beyang: where it's like a first year, like freshman CS student would not make that error, right? So like, why does the AI make that error? And the reason is, I mean, the AI is just suggesting things that are plausible without the context of the types or any other like broader files in the code. And so the kind of intuition here is like, why don't we just do the basic thing that like any baseline intelligent human developer would do, which is like click jump to definition, click some fine references and pull in that like Graph Context into the context window and then have it generate the completion. So like that's sort of like the MVP of what BFG was. And turns out that works really well. Like you can eliminate a lot of type errors that AI coding tools make just by pulling in that context. Yeah, but the graph is definitely [00:48:32]Steve: our Chomsky side. [00:48:33]Swyx: Yeah, exactly. [00:48:34]Beyang: So like this like Chomsky-Norvig thing, I think pops up in a bunch of differ

This week, we discuss why everyone is envious of Google's Internal Dev Tools, examine the state of Git, speculate about how 37 Signals plans to reinvent software licensing with ONCE, and share a few thoughts on the Salesforce CEO's recent comments about work from home. Watch the YouTube Live Recording of Episode (https://www.youtube.com/watch?v=AaX-PgF86bY) 433 (https://www.youtube.com/watch?v=AaX-PgF86bY) Runner-up Titles Lost in an acquisition hole. Headless Robot Dog. It's not better enough. GoogHub Why are you on the sad path Once version 2 is a paid upgrade You win interesting bingo Rundown The Full Circle on Developer Productivity with Steve Yegge (https://newsletter.pragmaticengineer.com/p/steve-yegge) Git is awful. GitHub isn't good enough. It's killing us! (Steve Yegge) (https://www.youtube.com/watch?v=EReooAZoMO0) Introducing ONCE (https://once.com/) Salesforce CEO takes a bold stand on remote work (https://www.thestreet.com/investing/salesforce-ceo-bold-stand-on-remote-work) Salesforce to Hire 3,300 People After Layoffs Earlier This Year (https://www.bloomberg.com/news/articles/2023-09-14/salesforce-to-hire-3-300-in-sales-engineering-data-after-earlier-job-cuts#xj4y7vzkg) Relevant to your Interests David Sacks has a new SaaS startup for other SaaS startups (https://www.axios.com/2023/09/06/david-sacks-has-a-new-saas-startup-for-other-saas-startups) Results of Major Technical Investigations for Storm-0558 Key Acquisition (https://msrc.microsoft.com/blog/2023/09/results-of-major-technical-investigations-for-storm-0558-key-acquisition/) Now it's PostgreSQL's turn to have a bogus CVE (https://opensourcewatch.beehiiv.com/p/now-postgresqls-turn-bogus-cve) HashiCorp Retools Licenses And Software To Grow Its Business - The Next Platform (https://www.nextplatform.com/2023/09/05/hashicorp-retools-licenses-and-software-to-grow-its-business/) Clouded Judgement 9.8.23 (https://cloudedjudgement.substack.com/p/clouded-judgement-9823?utm_source=post-email-title&publication_id=56878&post_id=136822157&isFreemail=true&r=2l9&utm_medium=email) Inside Hollywood's SBF Mad Scramble (https://theankler.com/p/inside-hollywoods-sbf-mad-scramble-c04?utm_source=newsletter&utm_medium=email&utm_campaign=newsletter_axiosprorata&stream=top) Tubi The Free Streaming Service, Hits 74 Million Monthly Active Users & Almost 250 Free Live Channels As Cord Cutting Grows | Cord Cutters News (https://cordcuttersnews.com/tubi-the-free-streaming-service-hits-74-million-monthly-active-users-almost-250-free-live-channels-as-cord-cutting-grows/) IBM Software mandates return to office for those within 80km (https://www.theregister.com/2023/09/11/ibm_software_tells_workers_to/) Cloud is here to stay, but at what cost, ask customers (https://www.theregister.com/2023/09/11/cloud_costs_feature/) Disney and Charter reach deal to end cable blackout in time for 'Monday Night Football' (https://www.cnbc.com/2023/09/11/disney-charter-near-carriage-deal-that-would-end-cable-blackout-sources-say.html) Microsoft to kill off third-party printer drivers in Windows (https://www.theregister.com/2023/09/11/go_native_or_go_home/) Oracle revenue misses estimates as tough economy hurts cloud spending (https://www.reuters.com/technology/oracle-reports-quarterly-revenue-narrowly-below-estimates-2023-09-11/) No privacy in cars (https://foundation.mozilla.org/en/privacynotincluded/articles/its-official-cars-are-the-worst-product-category-we-have-ever-reviewed-for-privacy/) Former CEO of China's Alibaba quits cloud business in surprise move during its leadership reshuffle (https://abcnews.go.com/Business/wireStory/former-ceo-chinas-alibaba-quits-cloud-business-surprise-103078368) A Look Back at Q2 '23 Public Cloud Software Earnings (https://cloudedjudgement.substack.com/p/a-look-back-at-q2-23-public-cloud?utm_source=post-email-title&publication_id=56878&post_id=136950716&utm_campaign=email-post-title&isFreemail=true&r=2l9&utm_medium=email) 1 big thing: A long-term plan to secure open-source software (https://www.axios.com/newsletters/axios-codebook-8200e5c5-aed7-4f42-a40e-117a390b57e3.html?chunk=0&utm_term=emshare#story0) MGM takes systems offline after cyberattack (https://www.axios.com/newsletters/axios-codebook-8200e5c5-aed7-4f42-a40e-117a390b57e3.html?chunk=1&utm_term=emshare#story1) Disney-Charter deal represents new era for TV bundles (https://www.axios.com/newsletters/axios-media-trends-fe1295c8-9b83-4403-bae2-06de14fede11.html?chunk=2&utm_term=emshare#story2) Salesforce introduces Einstein Copilot Studio to help customers customize their AI | TechCrunch (https://techcrunch.com/2023/09/12/salesforce-introduces-einstein-copilot-studio-to-customers-customize-their-ai/) Arm prices IPO at $51 per share, valuing company at over $54 billion (https://www.cnbc.com/2023/09/13/arm-prices-ipo-at-51-per-share.html) Tim Gurner's spray about ‘arrogant' workers lays bare the economic sadism of our time (https://www.theguardian.com/commentisfree/2023/sep/14/tim-gurner-ceo-comments-more-unemployment-millionaire-property-developer-workers-neoliberals) Cisco discontinues Hyperflex hyperconverged infrastructure (https://www.theregister.com/2023/09/14/cisco_discontinues_hyperflex_hci/) CloudBees Announces New Cloud Native DevSecOps Platform (https://www.cloudbees.com/newsroom/cloudbees-announces-new-cloud-native-devsecops-platform) Jet: Prepare For Liftoff (https://www.jetporch.com/) Artifact's new Links feature makes it much more than a news app (https://www.theverge.com/2023/9/13/23871561/artifact-links-news-reading-app-tiktok) TriggerMesh, RIP (https://triggermesh-community.slack.com/archives/C02GHUAQDCH/p1695048539668859) Clorox says last month's cyberattack is still disrupting production (https://www.cnbc.com/2023/09/18/clorox-says-last-months-cyberattack-is-still-disrupting-production.html) Excel clone built for Uber China exposed Microsoft mistake (https://www.theregister.com/2023/09/19/matt_uber_china_excel_clone/) Seattle startup MotherDuck raises $52.5M at a $400M valuation to fuel DuckDB analytics platform (https://www.geekwire.com/2023/seattle-startup-motherduck-raises-52-5m-at-a-400m-valuation-to-fuel-duckdb-analytics-platform/) Google's Bard chatbot can now find answers in your Gmail, Docs, Drive (https://www.theverge.com/2023/9/19/23878999/google-bard-ai-chatbot-gmail-docs-drive-extensions) Elon Musk says X may go behind a paywall for everyone so he can 'combat vast armies of bots' (https://www.businessinsider.com/elon-musk-x-twitter-paywall-for-everyone-2023-9) Restricted Source Licensing Is Here (https://www.forrester.com/blogs/restricted-source-licensing-is-here/) OpenTofu (https://opentofu.org/) RoboFab is ready to build 10,000 humanoid robots per year | TechCrunch (https://techcrunch.com/2023/09/18/the-robots-are-coming/) Unified Acceleration Foundation Forms to Drive Open Accelerated Compute and Cross-Platform Performance (https://www.linuxfoundation.org/press/announcing-unified-acceleration-foundation-uxl) Google gets its way, bakes a user-tracking ad platform directly into Chrome (https://arstechnica.com/gadgets/2023/09/googles-widely-opposed-ad-platform-the-privacy-sandbox-launches-in-chrome/) What is a service mesh? Why do you need a service mesh? And which is the best service mesh? (https://newsletter.cote.io/p/what-is-a-service-mesh-why-do-you) Did I Make a Mistake Selling My Social-Media Darling to Yahoo? (https://nymag.com/intelligencer/2018/10/did-i-make-a-mistake-selling-del-icio-us-to-yahoo.html?utm_source=substack&utm_medium=email) A new way of thinking about open source sustainability (https://www.infoworld.com/article/3706508/a-new-way-of-thinking-about-open-source-sustainability.html) Elon Musk moving servers himself shows his 'maniacal sense of urgency' at X, formerly Twitter (https://www.cnbc.com/2023/09/11/elon-musk-moved-twitter-servers-himself-in-the-night-new-biography-details-his-maniacal-sense-of-urgency.html) Cable TV Is on Life Support, but a New Bundle Is Coming Alive (https://www.nytimes.com/2023/09/14/business/media/cable-tv-bundle-streaming.html) Nonsense McDonald's is getting rid of self-serve soda machines | CNN Business (https://www.cnn.com/2023/09/12/business/mcdonalds-self-serve-soda-machines/index.html) Delta SkyMiles changes: Delta overhauls how you earn Medallion status in biggest change yet (https://thepointsguy.com/news/delta-skymiles-changes/) Australian baby named Methamphetamine Rules (https://www.1news.co.nz/2023/09/20/australian-baby-named-methamphetamine-rules/) ‘Take the Money and Run' Artist Must Repay Danish Museum (https://www.nytimes.com/2023/09/19/arts/design/jens-haaning-take-the-money-and-run.html?smid=nytcore-ios-share&referringSource=articleShare) Listener Feedback Jan recommends this Rich Roll interview: Mindset SECRETS From The World's Best Ultrarunner: Courtney Dauwalter (https://www.youtube.com/watch?v=WOtSvYSnzNk) Conferences October 6, 2023, KCD Texas 2023 (https://community.cncf.io/events/details/cncf-kcd-texas-presents-kcd-texas-2023/), CFP Closes: August 30, 2023 November 6-9, 2023, KubeCon NA (https://events.linuxfoundation.org/kubecon-cloudnativecon-north-america/), SDT's a sponsor, Matt's there November 6-9, 2023 VMware Explore Barcelona (https://www.vmware.com/explore/eu.html), Coté's attending Jan 29, 2024 to Feb 1, 2024 That Conference Texas (https://that.us/events/tx/2024/schedule/) If you want your conference mentioned, let's talk media sponsorships. SDT news & hype Join us in Slack (http://www.softwaredefinedtalk.com/slack). Get a SDT Sticker! Send your postal address to stickers@softwaredefinedtalk.com (mailto:stickers@softwaredefinedtalk.com) and we will send you free laptop stickers! Follow us: Twitch (https://www.twitch.tv/sdtpodcast), Twitter (https://twitter.com/softwaredeftalk), Instagram (https://www.instagram.com/softwaredefinedtalk/), Mastodon (https://hachyderm.io/@softwaredefinedtalk), BlueSky (https://bsky.app/profile/softwaredefinedtalk.com), LinkedIn (https://www.linkedin.com/company/software-defined-talk/), TikTok (https://www.tiktok.com/@softwaredefinedtalk), Threads (https://www.threads.net/@softwaredefinedtalk) and YouTube (https://www.youtube.com/channel/UCi3OJPV6h9tp-hbsGBLGsDQ/featured). Use the code SDT to get $20 off Coté's book, Digital WTF (https://leanpub.com/digitalwtf/c/sdt), so $5 total. Become a sponsor of Software Defined Talk (https://www.softwaredefinedtalk.com/ads)! Recommendations Brandon: YouTube TV (https://tv.youtube.com/welcome/) and NFL Sunday Ticket (https://tv.youtube.com/learn/nflsundayticket/) An Endgame for YouTube TV, Big Disney Decisions (And Whether Bob Iger Should Make Them), The Era Beyond Peak TV (https://sharptech.fm/member/episode/an-endgame-for-you-tube-tv-big-disney-decisions-and-whether-bob-iger-should-make-them-the-era-beyond-peak-tv) Matt: Airline wifi chat with Support Coté: Do Interesting (https://thedobook.co/products/do-interesting-notice-collect-share) book by Russel Davis. Photo Credits Header (https://unsplash.com/photos/m-Yot4dUd6s) Artwork (https://unsplash.com/photos/I7iJOE4fsYo)

This week it's storytime with Steve Yegge! Steve came out of retirement to join Sourcegraph as Head of Engineering. Their next frontier is Cody, their AI coding assistant that answers code questions and writes code for you by reading your entire codebase and the code graph. But, we really spent a lot of time talking with Steve about his time at Amazon, Google, and Grab. Ok, it's storytime!

This week it's storytime with Steve Yegge! Steve came out of retirement to join Sourcegraph as Head of Engineering. Their next frontier is Cody, their AI coding assistant that answers code questions and writes code for you by reading your entire codebase and the code graph. But, we really spent a lot of time talking with Steve about his time at Amazon, Google, and Grab. Ok, it's storytime!

It's now almost 6 months since Google declared Code Red, and the results — Jeff Dean's recap of 2022 achievements and a mass exodus of the top research talent that contributed to it in January, Bard's rushed launch in Feb, a slick video showing Google Workspace AI features and confusing doubly linked blogposts about PaLM API in March, and merging Google Brain and DeepMind in April — have not been inspiring. Google's internal panic is in full display now with the surfacing of a well written memo, written by software engineer Luke Sernau written in early April, revealing internal distress not seen since Steve Yegge's infamous Google Platforms Rant. Similar to 2011, the company's response to an external challenge has been to mobilize the entire company to go all-in on a (from the outside) vague vision.Google's misfortunes are well understood by now, but the last paragraph of the memo: “We have no moat, and neither does OpenAI”, was a banger of a mic drop.Combine this with news this morning that OpenAI lost $540m last year and will need as much as $100b more funding (after the complex $10b Microsoft deal in Jan), and the memo's assertion that both Google and OpenAI have “no moat” against the mighty open source horde have gained some credibility in the past 24 hours.Many are criticising this memo privately:* A CEO commented to me yesterday that Luke Sernau does not seem to work in AI related parts of Google and “software engineers don't understand moats”. * Emad Mostaque, himself a perma-champion of open source and open models, has repeatedly stated that “Closed models will always outperform open models” because closed models can just wrap open ones.* Emad has also commented on the moats he does see: “Unique usage data, Unique content, Unique talent, Unique product, Unique business model”, most of which Google does have, and OpenAI less so (though it is winning on the talent front)* Sam Altman famously said that “very few to no one is Silicon Valley has a moat - not even Facebook” (implying that moats don't actually matter, and you should spend your time thinking about more important things)* It is not actually clear what race the memo thinks Google and OpenAI are in vs Open Source. Neither are particularly concerned about running models locally on phones, and they are perfectly happy to let “a crazy European alpha male” run the last mile for them while they build actually monetizable cloud infrastructure.However moats are of intense interest by everybody keen on productized AI, cropping up in every Harvey, Jasper, and general AI startup vs incumbent debate. It is also interesting to take the memo at face value and discuss the searing hot pace of AI progress in open source. We hosted this discussion yesterday with Simon Willison, who apart from being an incredible communicator also wrote a great recap of the No Moat memo. 2,800 have now tuned in on Twitter Spaces, but we have taken the audio and cleaned it up here. Enjoy!Timestamps* [00:00:00] Introducing the Google Memo* [00:02:48] Open Source > Closed?* [00:05:51] Running Models On Device* [00:07:52] LoRA part 1* [00:08:42] On Moats - Size, Data* [00:11:34] Open Source Models are Comparable on Data* [00:13:04] Stackable LoRA* [00:19:44] The Need for Special Purpose Optimized Models* [00:21:12] Modular - Mojo from Chris Lattner* [00:23:33] The Promise of Language Supersets* [00:28:44] Google AI Strategy* [00:29:58] Zuck Releasing LLaMA* [00:30:42] Google Origin Confirmed* [00:30:57] Google's existential threat* [00:32:24] Non-Fiction AI Safety ("y-risk")* [00:35:17] Prompt Injection* [00:36:00] Google vs OpenAI* [00:41:04] Personal plugs: Simon and TravisTranscripts[00:00:00] Introducing the Google Memo[00:00:00] Simon Willison: So, yeah, this is a document, which Kate, which I first saw at three o'clock this morning, I think. It claims to be leaked from Google. There's good reasons to believe it is leaked from Google, and to be honest, if it's not, it doesn't actually matter because the quality of the analysis, I think stands alone.[00:00:15] If this was just a document by some anonymous person, I'd still think it was interesting and worth discussing. And the title of the document is We Have No Moat and neither does Open ai. And the argument it makes is that while Google and OpenAI have been competing on training bigger and bigger language models, the open source community is already starting to outrun them, given only a couple of months of really like really, really serious activity.[00:00:41] You know, Facebook lama was the thing that really kicked us off. There were open source language models like Bloom before that some G P T J, and they weren't very impressive. Like nobody was really thinking that they were. Chat. G P T equivalent Facebook Lama came out in March, I think March 15th. And was the first one that really sort of showed signs of being as capable maybe as chat G P T.[00:01:04] My, I don't, I think all of these models, they've been, the analysis of them has tend to be a bit hyped. Like I don't think any of them are even quite up to GT 3.5 standards yet, but they're within spitting distance in some respects. So anyway, Lama came out and then, Two weeks later Stanford Alpaca came out, which was fine tuned on top of Lama and was a massive leap forward in terms of quality.[00:01:27] And then a week after that Vicuna came out, which is to this date, the the best model I've been able to run on my own hardware. I, on my mobile phone now, like, it's astonishing how little resources you need to run these things. But anyway, the the argument that this paper made, which I found very convincing is it only took open source two months to get this far.[00:01:47] It's now every researcher in the world is kicking it on new, new things, but it feels like they're being there. There are problems that Google has been trying to solve that the open source models are already addressing, and really how do you compete with that, like with your, it's closed ecosystem, how are you going to beat these open models with all of this innovation going on?[00:02:04] But then the most interesting argument in there is it talks about the size of models and says that maybe large isn't a competitive advantage, maybe actually a smaller model. With lots of like different people fine tuning it and having these sort of, these LoRA l o r a stackable fine tuning innovations on top of it, maybe those can move faster.[00:02:23] And actually having to retrain your giant model every few months from scratch is, is way less useful than having small models that you can tr you can fine tune in a couple of hours on laptop. So it's, it's fascinating. I basically, if you haven't read this thing, you should read every word of it. It's not very long.[00:02:40] It's beautifully written. Like it's, it's, I mean, If you try and find the quotable lines in it, almost every line of it's quotable. Yeah. So, yeah, that's that, that, that's the status of this[00:02:48] Open Source > Closed?[00:02:48] swyx: thing. That's a wonderful summary, Simon. Yeah, there, there's so many angles we can take to this. I, I'll just observe one, one thing which if you think about the open versus closed narrative, Ima Mok, who is the CEO of Stability, has always been that open will trail behind closed, because the closed alternatives can always take.[00:03:08] Learnings and lessons from open source. And this is the first highly credible statement that is basically saying the exact opposite, that open source is moving than, than, than closed source. And they are scared. They seem to be scared. Which is interesting,[00:03:22] Travis Fischer: Travis. Yeah, the, the, the, a few things that, that I'll, I'll, I'll say the only thing which can keep up with the pace of AI these days is open source.[00:03:32] I think we're, we're seeing that unfold in real time before our eyes. And. You know, I, I think the other interesting angle of this is to some degree LLMs are they, they don't really have switching costs. They are going to be, become commoditized. At least that's, that's what a lot of, a lot of people kind of think to, to what extent is it Is it a, a rate in terms of, of pricing of these things?[00:03:55] , and they all kind of become roughly the, the, the same in, in terms of their, their underlying abilities. And, and open source is gonna, gonna be actively pushing, pushing that forward. And, and then this is kind of coming from, if it is to be believed the kind of Google or an insider type type mentality around you know, where is the actual competitive advantage?[00:04:14] What should they be focusing on? How can they get back in into the game? When you know, when, when, when, when currently the, the, the external view of, of Google is that they're kind of spinning their wheels and they have this code red,, and it's like they're, they're playing catch up already.[00:04:28] Like how could they use the open source community and work with them, which is gonna be really, really hard you know, from a structural perspective given Google's place in the ecosystem. But a, a lot, lot, a lot of jumping off points there.[00:04:42] Alessio Fanelli: I was gonna say, I think the Post is really focused on how do we get the best model, but it's not focused on like, how do we build the best product around it.[00:04:50] A lot of these models are limited by how many GPUs you can get to run them and we've seen on traditional open source, like everybody can use some of these projects like Kafka and like Alaska for free. But the reality is that not everybody can afford to run the infrastructure needed for it.[00:05:05] So I, I think like the main takeaway that I have from this is like, A lot of the moats are probably around just getting the, the sand, so to speak, and having the GPUs to actually serve these models. Because even if the best model is open source, like running it at large scale for an end is not easy and like, it's not super convenient to get a lot, a lot of the infrastructure.[00:05:27] And we've seen that model work in open source where you have. The opensource project, and then you have a enterprise cloud hosted version for it. I think that's gonna look really different in opensource models because just hosting a model doesn't have a lot of value. So I'm curious to hear how people end up getting rewarded to do opensource.[00:05:46] You know, it's, we figured that out in infrastructure, but we haven't figured it out in in Alans[00:05:51] Running Models On Device[00:05:51] Simon Willison: yet. I mean, one thing I'll say is that the the models that you can run on your own devices are so far ahead of what I ever dreamed they would be at this point. Like Vicuna 13 b i i, I, I think is the current best available open mo model that I've played with.[00:06:08] It's derived from Facebook Lama, so you can't use it for commercial purposes yet. But the point about MCK 13 B is it runs in the browser directly on web gpu. There's this amazing web l l M project where you literally, your browser downloaded a two gigabyte file. And it fires up a chat g D style interface and it's quite good.[00:06:27] It can do rap battles between different animals and all of the kind of fun stuff that you'd expect to be able to do the language model running entirely in Chrome canary. It's shocking to me that that's even possible, but that kind of shows that once, once you get to inference, if you can shrink the model down and the techniques for shrinking these models, the, the first one was the the quantization.[00:06:48] Which the Lama CPP project really sort of popularized Matt can by using four bits instead of 16 bit floating point numbers, you can shrink it down quite a lot. And then there was a paper that came out days ago suggesting that you can prune the models and ditch half the model and maintain the same level of quality.[00:07:05] So with, with things like that, with all of these tricks coming together, it's really astonishing how much you can get done on hardware that people actually have in their pockets even.[00:07:15] swyx: Just for completion I've been following all of your posts. Oh, sorry. Yes. I just wanna follow up, Simon. You're, you said you're running a model on your phone. Which model is it? And I don't think you've written it up.[00:07:27] Simon Willison: Yeah, that one's vina. I did, did I write it up? I did. I've got a blog post about how it it, it, it knows who I am, sort of, but it said that I invented a, a, a pattern for living called bear or bunny pattern, which I definitely didn't, but I loved that my phone decided that I did.[00:07:44] swyx: I will hunt for that because I'm not yet running Vic on my phone and I feel like I should and, and as like a very base thing, but I'll, okay.[00:07:52] Stackable LoRA Modules[00:07:52] swyx: Also, I'll follow up two things, right? Like one I'm very interesting and let's, let's talk about that a little bit more because this concept of stackable improvements to models I think is extremely interesting.[00:08:00] Like, I would love to MPM install abilities onto my models, right? Which is really awesome. But the, the first thing thing is under-discussed is I don't get the panic. Like, honestly, like Google has the most moats. I I, I was arguing maybe like three months ago on my blog. Like Google has the most mote out of a lot of people because, hey, we have your calendar.[00:08:21] Hey, we have your email. Hey, we have your you know, Google Docs. Like, isn't that a, a sufficient mode? Like, why are these guys panicking so much? I don't, I still don't get it. Like, Sure open source is running ahead and like, it's, it's on device and whatev, what have you, but they have so much more mode.[00:08:36] Like, what are we talking about here? There's many dimensions to compete on.[00:08:42] On Moats - Size, Data[00:08:42] Travis Fischer: Yeah, there's like one of, one of the, the things that, that the author you know, mentions in, in here is when, when you start to, to, to have the feeling of what we're trailing behind, then you're, you're, you're, you're brightest researchers jump ship and go to OpenAI or go to work at, at, at academia or, or whatever.[00:09:00] And like the talent drain. At the, the level of the, the senior AI researchers that are pushing these things ahead within Google, I think is a serious, serious concern. And my, my take on it's a good point, right? Like, like, like, like what Google has modes. They, they, they're not running outta money anytime soon.[00:09:16] You know, I think they, they do see the level of the, the defensibility and, and the fact that they want to be, I'll chime in the, the leader around pretty much anything. Tech first. There's definitely ha ha have lost that, that, that feeling. Right? , and to what degree they can, they can with the, the open source community to, to get that back and, and help drive that.[00:09:38] You know all of the llama subset of models with, with alpaca and Vicuna, et cetera, that all came from, from meta. Right. Like that. Yeah. Like it's not licensed in an open way where you can build a company on top of it, but is now kind of driving this family of, of models, like there's a tree of models that, that they're, they're leading.[00:09:54] And where is Google in that, in that playbook? Like for a long time they were the one releasing those models being super open and, and now it's just they, they've seem to be trailing and there's, there's people jumping ship and to what degree can they, can they, can they. Close off those wounds and, and focus on, on where, where they, they have unique ability to, to gain momentum.[00:10:15] I think is a core part of my takeaway from this. Yeah.[00:10:19] Alessio Fanelli: And think another big thing in the post is, oh, as long as you have high quality data, like you don't need that much data, you can just use that. The first party data loops are probably gonna be the most important going forward if we do believe that this is true.[00:10:32] So, Databricks. We have Mike Conover from Databricks on the podcast, and they talked about how they came up with the training set for Dolly, which they basically had Databricks employees write down very good questions and very good answers for it. Not every company as the scale to do that. And I think products like Google, they have millions of people writing Google Docs.[00:10:54] They have millions of people using Google Sheets, then millions of people writing stuff, creating content on YouTube. The question is, if you wanna compete against these companies, maybe the model is not what you're gonna do it with because the open source kind of commoditizes it. But how do you build even better data?[00:11:12] First party loops. And that's kind of the hardest thing for startups, right? Like even if we open up the, the models to everybody and everybody can just go on GitHub and. Or hugging face and get the waste to the best model, but get enough people to generate data for me so that I can still make it good. That's, that's what I would be worried about if I was a, a new company.[00:11:31] How do I make that happen[00:11:32] Simon Willison: really quickly?[00:11:34] Open Source Models are Comparable on Data[00:11:34] Simon Willison: I'm not convinced that the data is that big a challenge. So there's this PO project. So the problem with Facebook LAMA is that it's not available for, for commercial use. So people are now trying to train a alternative to LAMA that's entirely on openly licensed data.[00:11:48] And that the biggest project around that is this red pajama project, which They released their training data a few weeks ago and it was 2.7 terabytes. Right? So actually tiny, right? You can buy a laptop that you can fit 2.7 terabytes on. Got it. But it was the same exact data that Facebook, the same thing that Facebook Lamb had been trained on.[00:12:06] Cuz for your base model. You're not really trying to teach it fact about the world. You're just trying to teach it how English and other languages work, how they fit together. And then the real magic is when you fine tune on top of that. That's what Alpaca did on top of Lama and so on. And the fine tuning sets, it looks like, like tens of thousands of examples to kick one of these role models into shape.[00:12:26] And tens of thousands of examples like Databricks spent a month and got the 2000 employees of their company to help kick in and it worked. You've got the open assistant project of crowdsourcing this stuff now as well. So it's achievable[00:12:40] swyx: sore throat. I agree. I think it's a fa fascinating point. Actually, so I've heard through the grapevine then red pajamas model.[00:12:47] Trained on the, the data that they release is gonna be releasing tomorrow. And it's, it's this very exciting time because the, the, there, there's a, there's a couple more models that are coming down the pike, which independently we produced. And so yeah, that we, everyone is challenging all these assumptions from, from first principles, which is fascinating.[00:13:04] Stackable LoRA[00:13:04] swyx: I, I did, I did wanted to, to like try to get a little bit more technical in terms of like the, the, the, the specific points race. Cuz this doc, this doc was just amazing. Can we talk about LoRA. I, I, I'll open up to Simon again if he's back.[00:13:16] Simon Willison: I'd rather someone else take on. LoRA, I've, I, I know as much as I've read in that paper, but not much more than that.[00:13:21] swyx: So I thought it was this kind of like an optimization technique. So LoRA stands for lower rank adaptation. But this is the first mention of LoRA as a form of stackable improvements. Where he I forget what, let, just, let me just kind of Google this. But obviously anyone's more knowledgeable please.[00:13:39] So come on in.[00:13:40] Alessio Fanelli: I, all of Lauren is through GTS Man, about 20 minutes on GT four, trying to figure out word. It was I study computer science, but this is not this is not my area of expertise. What I got from it is that basically instead of having to retrain the whole model you can just pick one of the ranks and you take.[00:13:58] One of like the, the weight matrix tests and like make two smaller matrixes from it and then just two to be retrained and training the whole model. So[00:14:08] swyx: it save a lot of Yeah. You freeze part of the thing and then you just train the smaller part like that. Exactly. That seems to be a area of a lot of fruitful research.[00:14:15] Yeah. I think Mini GT four recently did something similar as well. And then there's, there's, there's a, there's a Spark Model people out today that also did the same thing.[00:14:23] Simon Willison: So I've seen a lot of LoRA stable, the stable diffusion community has been using LoRA a lot. So they, in that case, they had a, I, the thing I've seen is people releasing LoRA's that are like you, you train a concept like a, a a particular person's face or something you release.[00:14:38] And the, the LoRA version of this end up being megabytes of data, like, which is, it's. You know, it's small enough that you can just trade those around and you can effectively load multiple of those into the model. But what I haven't realized is that you can use the same trick on, on language models. That was one of the big new things for me in reading the the leaks Google paper today.[00:14:56] Alessio Fanelli: Yeah, and I think the point to make around on the infrastructure, so what tragedy has told me is that when you're figuring out what rank you actually wanna do this fine tuning at you can have either go too low and like the model doesn't actually learn it. Or you can go too high and the model overfit those learnings.[00:15:14] So if you have a base model that everybody agrees on, then all the subsequent like LoRA work is done around the same rank, which gives you an advantage. And the point they made in the, that, since Lama has been the base for a lot of this LoRA work like they own. The, the mind share of the community.[00:15:32] So everything that they're building is compatible with their architecture. But if Google Opensources their own model the rank that they chose For LoRA on Lama might not work on the Google model. So all of the existing work is not portable. So[00:15:46] Simon Willison: the impression I got is that one of the challenges with LoRA is that you train all these LoRAs on top of your model, but then if you retrain that base model as LoRA's becoming invalid, right?[00:15:55] They're essentially, they're, they're, they're built for an exact model version. So this means that being the big company with all of the GPUs that can afford to retrain a model every three months. That's suddenly not nearly as valuable as it used to be because now maybe there's an open source model that's five years old at this point and has like multiple, multiple stacks of LoRA's trained all over the world on top of it, which can outperform your brand new model just because there's been so much more iteration on that base.[00:16:20] swyx: I, I think it's, I think it's fascinating. It's I think Jim Fan from Envidia was recently making this argument for transformers. Like even if we do come up with a better. Architecture, then transformers, they're the sheer hundreds and millions of dollars that have been invested on top of transformers.[00:16:34] Make it actually there is some switching costs and it's not exactly obvious that better architecture. Equals equals we should all switch immediately tomorrow. It's, it's, it's[00:16:44] Simon Willison: kinda like the, the difficulty of launching a new programming language today Yes. Is that pipeline and JavaScript have a million packages.[00:16:51] So no matter how good your new language is, if it can't tap into those existing package libraries, it's, it's not gonna be useful for, which is why Moji is so clever, because they did build on top of Pips. They get all of that existing infrastructure, all of that existing code working already.[00:17:05] swyx: I mean, what, what thought you, since you co-create JAO and all that do, do we wanna take a diversion into mojo?[00:17:10] No, no. I[00:17:11] Travis Fischer: would, I, I'd be happy to, to, to jump in, and get Simon's take on, on Mojo. 1, 1, 1 small, small point on LoRA is I, I, I just think. If you think about at a high level, what the, the major down downsides are of these, these large language models. It's the fact that they well they're, they're, they're difficult to, to train, right?[00:17:32] They, they tend to hallucinate and they are, have, have a static, like, like they were trained at a certain date, right? And with, with LoRA, I think it makes it a lot more amenable to Training new, new updates on top of that, that like base model on the fly where you can incorporate new, new data and in a way that is, is, is an interesting and potentially more optimal alternative than Doing the kind of in context generation cuz, cuz most of like who at perplexity AI or, or any of these, these approaches currently, it's like all based off of doing real-time searches and then injecting as much into the, the, the local context window as possible so that you, you try to ground your, your, your, your language model.[00:18:16] Both in terms of the, the information it has access to that, that, that helps to reduce hallucinations. It can't reduce it, but helps to reduce it and then also gives it access to up-to-date information that wasn't around for that, that massive like, like pre-training step. And I think LoRA in, in, in mine really makes it more, more amenable to having.[00:18:36] Having constantly shifting lightweight pre-training on top of it that scales better than than normal. Pre I'm sorry. Fine tune, fine tuning. Yeah, that, that was just kinda my one takeaway[00:18:45] Simon Willison: there. I mean, for me, I've never been, I want to run models on my own hard, I don't actually care about their factual content.[00:18:52] Like I don't need a model that's been, that's trained on the most upstate things. What I need is a model that can do the bing and bar trick, right? That can tell when it needs to run a search. And then go and run a search to get extra information and, and bring that context in. And similarly, I wanted to be able to operate tools where it can access my email or look at my notes or all of those kinds of things.[00:19:11] And I don't think you need a very powerful model for that. Like that's one of the things where I feel like, yeah, vicuna running on my, on my laptop is probably powerful enough to drive a sort of personal research assistant, which can look things up for me and it can summarize things for my notes and it can do all of that and I don't care.[00:19:26] But it doesn't know about the Ukraine war because the Ukraine war training cutoff, that doesn't matter. If it's got those additional capabilities, which are quite easy to build the reason everyone's going crazy building agents and tools right now is that it's a few lines of Python code, and a sort of couple of paragraphs to get it to.[00:19:44] The Need for Special Purpose Optimized Models[00:19:44] Simon Willison: Well, let's, let's,[00:19:45] Travis Fischer: let's maybe dig in on that a little bit. And this, this also is, is very related to mojo. Cuz I, I do think there are use cases and domains where having the, the hyper optimized, like a version of these models running on device is, is very relevant where you can't necessarily make API calls out on the fly.[00:20:03] and Aug do context, augmented generation. And I was, I was talking with, with a a researcher. At Lockheed Martin yesterday, literally about like, like the, the version of this that's running of, of language models running on, on fighter jets. Right? And you, you talk about like the, the, the amount of engineering, precision and optimization that has to go into, to those type of models.[00:20:25] And the fact that, that you spend so much money, like, like training a super distilled ver version where milliseconds matter it's a life or death situation there. You know, and you couldn't even, even remotely ha ha have a use case there where you could like call out and, and have, have API calls or something.[00:20:40] So I, I do think there's like keeping in mind the, the use cases where, where. There, there'll be use cases that I'm more excited about at, at the application level where, where, yeah, I want to to just have it be super flexible and be able to call out to APIs and have this agentic type type thing.[00:20:56] And then there's also industries and, and use cases where, where you really need everything baked into the model.[00:21:01] swyx: Yep. Agreed. My, my favorite piece take on this is I think DPC four as a reasoning engine, which I think came from the from Nathan at every two. Which I think, yeah, I see the hundred score over there.[00:21:12] Modular - Mojo from Chris Lattner[00:21:12] swyx: Simon, do you do you have a, a few seconds on[00:21:14] Simon Willison: mojo. Sure. So Mojo is a brand new program language you just announced a few days ago. It's not actually available yet. I think there's an online demo, but to zooming it becomes an open source language we can use. It's got really some very interesting characteristics.[00:21:29] It's a super set of Python, so anything written in Python, Python will just work, but it adds additional features on top that let you basically do very highly optimized code with written. In Python syntax, it compiles down the the main thing that's exciting about it is the pedigree that it comes from.[00:21:47] It's a team led by Chris Latner, built L L V M and Clang, and then he designed Swift at Apple. So he's got like three, three for three on, on extraordinarily impactful high performance computing products. And he put together this team and they've basically, they're trying to go after the problem of how do you build.[00:22:06] A language which you can do really high performance optimized work in, but where you don't have to do everything again from scratch. And that's where building on top of Python is so clever. So I wasn't like, if this thing came along, I, I didn't really pay attention to it until j Jeremy Howard, who built Fast ai put up a very detailed blog post about why he was excited about Mojo, which included a, there's a video demo in there, which everyone should watch because in that video he takes Matrix multiplication implemented in Python.[00:22:34] And then he uses the mojo extras to 2000 x. The performance of that matrix multiplication, like he adds a few static types functions sort of struck instead of the class. And he gets 2000 times the performance out of it, which is phenomenal. Like absolutely extraordinary. So yeah, that, that got me really excited.[00:22:52] Like the idea that we can still use Python and all of this stuff we've got in Python, but we can. Just very slightly tweak some things and get literally like thousands times upwards performance out of the things that matter. That's really exciting.[00:23:07] swyx: Yeah, I, I, I'm curious, like, how come this wasn't thought of before?[00:23:11] It's not like the, the, the concept of a language super set hasn't hasn't, has, has isn't, is completely new. But all, as far as I know, all the previous Python interpreter approaches, like the alternate runtime approaches are like they, they, they're more, they're more sort of, Fit conforming to standard Python, but never really tried this additional approach of augmenting the language.[00:23:33] The Promise of Language Supersets[00:23:33] swyx: I, I'm wondering if you have many insights there on, like, why, like why is this a, a, a breakthrough?[00:23:38] Simon Willison: Yeah, that's a really interesting question. So, Jeremy Howard's piece talks about this thing called M L I R, which I hadn't heard of before, but this was another Chris Latner project. You know, he built L L VM as a low level virtual machine.[00:23:53] That you could build compilers on top of. And then M L I R was this one that he initially kicked off at Google, and I think it's part of TensorFlow and things like that. But it was very much optimized for multiple cores and GPU access and all of that kind of thing. And so my reading of Jeremy Howard's article is that they've basically built Mojo on top of M L I R.[00:24:13] So they had a huge, huge like a starting point where they'd, they, they knew this technology better than anyone else. And because they had this very, very robust high performance basis that they could build things on. I think maybe they're just the first people to try and build a high, try and combine a high level language with M L A R, with some extra things.[00:24:34] So it feels like they're basically taking a whole bunch of ideas people have been sort of experimenting with over the last decade and bundled them all together with exactly the right team, the right level of expertise. And it looks like they've got the thing to work. But yeah, I mean, I've, I've, I'm. Very intrigued to see, especially once this is actually available and we can start using it.[00:24:52] It, Jeremy Howard is someone I respect very deeply and he's, he's hyping this thing like crazy, right? His headline, his, and he's not the kind of person who hypes things if they're not worth hyping. He said Mojo may be the biggest programming language advanced in decades. And from anyone else, I'd kind of ignore that headline.[00:25:09] But from him it really means something.[00:25:11] swyx: Yes, because he doesn't hype things up randomly. Yeah, and, and, and he's a noted skeptic of Julia which is, which is also another data science hot topic. But from the TypeScript and web, web development worlds there has been a dialect of TypeScript that was specifically optimized to compile, to web assembly which I thought was like promising and then, and, and eventually never really took off.[00:25:33] But I, I like this approach because I think more. Frameworks should, should essentially be languages and recognize that they're language superset and maybe working compilers that that work on them. And then that is the, by the way, that's the direction that React is going right now. So fun times[00:25:50] Simon Willison: type scripts An interesting comparison actually, cuz type script is effectively a superset of Java script, right?[00:25:54] swyx: It's, but there's no, it's purely[00:25:57] Simon Willison: types, right? Gotcha. Right. So, so I guess mojo is the soup set python, but the emphasis is absolutely on tapping into the performance stuff. Right.[00:26:05] swyx: Well, the just things people actually care about.[00:26:08] Travis Fischer: Yeah. The, the one thing I've found is, is very similar to the early days of type script.[00:26:12] There was the, the, the, the most important thing was that it's incrementally adoptable. You know, cuz people had a script code basis and, and they wanted to incrementally like add. The, the, the main value prop for TypeScript was reliability and the, the, the, the static typing. And with Mojo, Lucia being basically anyone who's a target a large enterprise user of, of Mojo or even researchers, like they're all going to be coming from a, a hardcore.[00:26:36] Background in, in Python and, and have large existing libraries. And the the question will be for what use cases will mojo be like a, a, a really good fit for that incremental adoption where you can still tap into your, your, your massive, like python exi existing infrastructure workflows, data tooling, et cetera.[00:26:55] And, and what does, what does that path to adoption look like?[00:26:59] swyx: Yeah, we, we, we don't know cuz it's a wait listed language which people were complaining about. They, they, the, the mojo creators were like saying something about they had to scale up their servers. And I'm like, what language requires essential server?[00:27:10] So it's a little bit suss, a little bit, like there's a, there's a cloud product already in place and they're waiting for it. But we'll see. We'll see. I mean, emojis should be promising in it. I, I actually want more. Programming language innovation this way. You know, I was complaining years ago that programming language innovation is all about stronger types, all fun, all about like more functional, more strong types everywhere.[00:27:29] And, and this is, the first one is actually much more practical which I, which I really enjoy. This is why I wrote about self provisioning run types.[00:27:36] Simon Willison: And[00:27:37] Alessio Fanelli: I mean, this is kind of related to the post, right? Like if you stop all of a sudden we're like, the models are all the same and we can improve them.[00:27:45] Like, where can we get the improvements? You know, it's like, Better run times, better languages, better tooling, better data collection. Yeah. So if I were a founder today, I wouldn't worry as much about the model, maybe, but I would say, okay, what can I build into my product and like, or what can I do at the engineering level that maybe it's not model optimization because everybody's working on it, but like you said, it's like, why haven't people thought of this before?[00:28:09] It's like, it's, it's definitely super hard, but I'm sure that if you're like Google or you're like open AI or you're like, Databricks, we got smart enough people that can think about these problems, so hopefully we see more of this.[00:28:21] swyx: You need, Alan? Okay. I promise to keep this relatively tight. I know Simon on a beautiful day.[00:28:27] It is a very nice day in California. I wanted to go through a few more points that you have pulled out Simon and, and just give you the opportunity to, to rant and riff and, and what have you. I, I, are there any other points from going back to the sort of Google OpenAI mode documents that, that you felt like we, we should dive in on?[00:28:44] Google AI Strategy[00:28:44] Simon Willison: I mean, the really interesting stuff there is the strategy component, right? The this idea that that Facebook accidentally stumbled into leading this because they put out this model that everyone else is innovating on top of. And there's a very open question for me as to would Facebook relic Lama to allow for commercial usage?[00:29:03] swyx: Is there some rumor? Is that, is that today?[00:29:06] Simon Willison: Is there a rumor about that?[00:29:07] swyx: That would be interesting? Yeah, I saw, I saw something about Zuck saying that he would release the, the Lama weights officially.[00:29:13] Simon Willison: Oh my goodness. No, that I missed. That is, that's huge.[00:29:17] swyx: Let me confirm the tweet. Let me find the tweet and then, yeah.[00:29:19] Okay.[00:29:20] Simon Willison: Because actually I met somebody from Facebook machine learning research a couple of weeks ago, and I, I pressed 'em on this and they said, basically they don't think it'll ever happen because if it happens, and then somebody does horrible fascist stuff with this model, all of the headlines will be Meg releases a monster into the world.[00:29:36] So, so hi. His, the, the, the, a couple of weeks ago, his feeling was that it's just too risky for them to, to allow it to be used like that. But a couple of weeks is, is, is a couple of months in AI world. So yeah, it wouldn't be, it feels to me like strategically Facebook should be jumping right on this because this puts them at the very.[00:29:54] The very lead of, of open source innovation around this stuff.[00:29:58] Zuck Releasing LLaMA[00:29:58] swyx: So I've pinned the tweet talking about Zuck and Zuck saying that meta will open up Lama. It's from the founder of Obsidian, which gives it a slight bit more credibility, but it is the only. Tweet that I can find about it. So completely unsourced,[00:30:13] we shall see. I, I, I mean I have friends within meta, I should just go ask them. But yeah, I, I mean one interesting angle on, on the memo actually is is that and, and they were linking to this in, in, in a doc, which is apparently like. Facebook got a bunch of people to do because they, they never released it for commercial use, but a lot of people went ahead anyway and, and optimized and, and built extensions and stuff.[00:30:34] They, they got a bunch of free work out of opensource, which is an interesting strategy.[00:30:39] There's okay. I don't know if I.[00:30:42] Google Origin Confirmed[00:30:42] Simon Willison: I've got exciting piece of news. I've just heard from somebody with contacts at Google that they've heard people in Google confirm the leak. That that document wasn't even legit Google document, which I don't find surprising at all, but I'm now up to 10, outta 10 on, on whether that's, that's, that's real.[00:30:57] Google's existential threat[00:30:57] swyx: Excellent. Excellent. Yeah, it is fascinating. Yeah, I mean the, the strategy is, is, is really interesting. I think Google has been. Definitely sleeping on monetizing. You know, I, I, I heard someone call when Google Brain and Devrel I merged that they would, it was like goodbye to the Xerox Park of our era and it definitely feels like Google X and Google Brain would definitely Xerox parks of our, of our era, and I guess we all benefit from that.[00:31:21] Simon Willison: So, one thing I'll say about the, the Google side of things, like the there was a question earlier, why are Google so worried about this stuff? And I think it's, it's just all about the money. You know, the, the, the engine of money at Google is Google searching Google search ads, and who uses Chachi PT on a daily basis, like me, will have noticed that their usage of Google has dropped like a stone.[00:31:41] Because there are many, many questions that, that chat, e p t, which shows you no ads at all. Is, is, is a better source of information for than Google now. And so, yeah, I'm not, it doesn't surprise me that Google would see this as an existential threat because whether or not they can be Bard, it's actually, it's not great, but it, it exists, but it hasn't it yet either.[00:32:00] And if I've got a Chatbook chatbot that's not showing me ads and chatbot that is showing me ads, I'm gonna pick the one that's not showing[00:32:06] swyx: me ads. Yeah. Yeah. I, I agree. I did see a prototype of Bing with ads. Bing chat with ads. I haven't[00:32:13] Simon Willison: seen the prototype yet. No.[00:32:15] swyx: Yeah, yeah. Anyway, I I, it, it will come obviously, and then we will choose, we'll, we'll go out of our ways to avoid ads just like we always do.[00:32:22] We'll need ad blockers and chat.[00:32:23] Excellent.[00:32:24] Non-Fiction AI Safety ("y-risk")[00:32:24] Simon Willison: So I feel like on the safety side, the, the safety side, there are basically two areas of safety that I, I, I sort of split it into. There's the science fiction scenarios, the AI breaking out and killing all humans and creating viruses and all of that kind of thing. The sort of the terminated stuff. And then there's the the.[00:32:40] People doing bad things with ai and that's latter one is the one that I think is much more interesting and that cuz you could u like things like romance scams, right? Romance scams already take billions of dollars from, from vulner people every year. Those are very easy to automate using existing tools.[00:32:56] I'm pretty sure for QNA 13 b running on my laptop could spin up a pretty decent romance scam if I was evil and wanted to use it for them. So that's the kind of thing where, I get really nervous about it, like the fact that these models are out there and bad people can use these bad, do bad things.[00:33:13] Most importantly at scale, like romance scamming, you don't need a language model to pull off one romance scam, but if you wanna pull off a thousand at once, the language model might be the, the thing that that helps you scale to that point. And yeah, in terms of the science fiction stuff and also like a model on my laptop that can.[00:33:28] Guess what comes next in a sentence. I'm not worried that that's going to break out of my laptop and destroy the world. There. There's, I'm get slightly nervous about the huge number of people who are trying to build agis on top of this models, the baby AGI stuff and so forth, but I don't think they're gonna get anywhere.[00:33:43] I feel like if you actually wanted a model that was, was a threat to human, a language model would be a tiny corner of what that thing. Was actually built on top of, you'd need goal setting and all sorts of other bits and pieces. So yeah, for the moment, the science fiction stuff doesn't really interest me, although it is a little bit alarming seeing more and more of the very senior figures in this industry sort of tip the hat, say we're getting a little bit nervous about this stuff now.[00:34:08] Yeah.[00:34:09] swyx: So that would be Jeff Iton and and I, I saw this me this morning that Jan Lacoon was like happily saying, this is fine. Being the third cheer award winner.[00:34:20] Simon Willison: But you'll see a lot of the AI safe, the people who've been talking about AI safety for the longest are getting really angry about science fiction scenarios cuz they're like, no, the, the thing that we need to be talking about is the harm that you can cause with these models right now today, which is actually happening and the science fiction stuff kind of ends up distracting from that.[00:34:36] swyx: I love it. You, you. Okay. So, so Uher, I don't know how to pronounce his name. Elier has a list of ways that AI will kill us post, and I think, Simon, you could write a list of ways that AI will harm us, but not kill us, right? Like the, the, the non-science fiction actual harm ways, I think, right? I haven't seen a, a actual list of like, hey, romance scams spam.[00:34:57] I, I don't, I don't know what else, but. That could be very interesting as a Hmm. Okay. Practical. Practical like, here are the situations we need to guard against because they are more real today than that we need to. Think about Warren, about obviously you've been a big advocate of prompt injection awareness even though you can't really solve them, and I, I worked through a scenario with you, but Yeah,[00:35:17] Prompt Injection[00:35:17] Simon Willison: yeah.[00:35:17] Prompt injection is a whole other side of this, which is, I mean, that if you want a risk from ai, the risk right now is everyone who's building puts a building systems that attackers can trivially subvert into stealing all of their private data, unlocking their house, all of that kind of thing. So that's another very real risk that we have today.[00:35:35] swyx: I think in all our personal bios we should edit in prompt injections already, like in on my website, I wanna edit in a personal prompt injections so that if I get scraped, like I all know if someone's like reading from a script, right? That that is generated by any iBot. I've[00:35:49] Simon Willison: seen people do that on LinkedIn already and they get, they get recruiter emails saying, Hey, I didn't read your bio properly and I'm just an AI script, but would you like a job?[00:35:57] Yeah. It's fascinating.[00:36:00] Google vs OpenAI[00:36:00] swyx: Okay. Alright, so topic. I, I, I think, I think this this, this mote is is a peak under the curtain of the, the internal panic within Google. I think it is very val, very validated. I'm not so sure they should care so much about small models or, or like on device models.[00:36:17] But the other stuff is interesting. There is a comment at the end that you had by about as for opening open is themselves, open air, doesn't matter. So this is a Google document talking about Google's position in the market and what Google should be doing. But they had a comment here about open eye.[00:36:31] They also say open eye had no mode, which is a interesting and brave comment given that open eye is the leader in, in a lot of these[00:36:38] Simon Willison: innovations. Well, one thing I will say is that I think we might have identified who within Google wrote this document. Now there's a version of it floating around with a name.[00:36:48] And I look them up on LinkedIn. They're heavily involved in the AI corner of Google. So my guess is that at Google done this one, I've worked for companies. I'll put out a memo, I'll write up a Google doc and I'll email, email it around, and it's nowhere near the official position of the company or of the executive team.[00:37:04] It's somebody's opinion. And so I think it's more likely that this particular document is somebody who works for Google and has an opinion and distributed it internally and then it, and then it got leaked. I dunno if it's necessarily. Represents Google's sort of institutional thinking about this? I think it probably should.[00:37:19] Again, this is such a well-written document. It's so well argued that if I was an executive at Google and I read that, I would, I would be thinking pretty hard about it. But yeah, I don't think we should see it as, as sort of the official secret internal position of the company. Yeah. First[00:37:34] swyx: of all, I might promote that person.[00:37:35] Cuz he's clearly more,[00:37:36] Simon Willison: oh, definitely. He's, he's, he's really, this is a, it's, I, I would hire this person about the strength of that document.[00:37:42] swyx: But second of all, this is more about open eye. Like I'm not interested in Google's official statements about open, but I was interested like his assertion, open eye.[00:37:50] Doesn't have a mote. That's a bold statement. I don't know. It's got the best people.[00:37:55] Travis Fischer: Well, I, I would, I would say two things here. One, it's really interesting just at a meta, meta point that, that they even approached it this way of having this public leak. It, it, it kind of, Talks a little bit to the fact that they, they, they felt that that doing do internally, like wasn't going to get anywhere or, or maybe this speaks to, to some of the like, middle management type stuff or, or within Google.[00:38:18] And then to the, the, the, the point about like opening and not having a moat. I think for, for large language models, it, it, it will be over, over time kind of a race to the bottom just because the switching costs are, are, are so low compared with traditional cloud and sas. And yeah, there will be differences in, in, in quality, but, but like over time, if you, you look at the limit of these things like the, I I think Sam Altman has been quoted a few times saying that the, the, the price of marginal price of intelligence will go to zero.[00:38:47] Time and the marginal price of energy powering that intelligence will, will also hit over time. And in that world, if you're, you're providing large language models, they become commoditized. Like, yeah. What, what is, what is your mode at that point? I don't know. I think they're e extremely well positioned as a team and as a company for leading this space.[00:39:03] I'm not that, that worried about that, but it is something from a strategic point of view to keep in mind about large language models becoming a commodity. So[00:39:11] Simon Willison: it's quite short, so I think it's worth just reading the, in fact, that entire section, it says epilogue. What about open ai? All of this talk of open source can feel unfair given open AI's current closed policy.[00:39:21] Why do we have to share if they won't? That's talking about Google sharing, but the fact of the matter is we are already sharing everything with them. In the form of the steady flow of poached senior researchers until we spent that tide. Secrecy is a moot point. I love that. That's so salty. And, and in the end, open eye doesn't matter.[00:39:38] They are making the same mistakes that we are in their posture relative to open source. And their ability to maintain an edge is necessarily in question. Open source alternatives. Canned will eventually eclipse them. Unless they change their stance in this respect, at least we can make the first move. So the argument this, this paper is making is that Google should go, go like meta and, and just lean right into open sourcing it and engaging with the wider open source community much more deeply, which OpenAI have very much signaled they are not willing to do.[00:40:06] But yeah, it's it's, it's read the whole thing. The whole thing is full of little snippets like that. It's just super fun. Yes,[00:40:12] swyx: yes. Read the whole thing. I, I, I also appreciate that the timeline, because it set a lot of really great context for people who are out of the loop. So Yeah.[00:40:20] Alessio Fanelli: Yeah. And the final conspiracy theory is that right before Sundar and Satya and Sam went to the White House this morning, so.[00:40:29] swyx: Yeah. Did it happen? I haven't caught up the White House statements.[00:40:34] Alessio Fanelli: No. That I, I just saw, I just saw the photos of them going into the, the White House. I've been, I haven't seen any post-meeting updates.[00:40:41] swyx: I think it's a big win for philanthropic to be at that table.[00:40:44] Alessio Fanelli: Oh yeah, for sure. And co here it's not there.[00:40:46] I was like, hmm. Interesting. Well, anyway,[00:40:50] swyx: yeah. They need, they need some help. Okay. Well, I, I promise to keep this relatively tight. Spaces do tend to have a, have a tendency of dragging on. But before we go, anything that you all want to plug, anything that you're working on currently maybe go around Simon are you still working on dataset?[00:41:04] Personal plugs: Simon and Travis[00:41:04] Simon Willison: I am, I am, I'm having a bit of a, so datasets my open source project that I've been working on. It's about helping people analyze and publish data. I'm having an existential crisis of it at the moment because I've got access to the chat g p T code, interpreter mode, and you can upload the sequel light database to that and it will do all of the things that I, on my roadmap for the next 12 months.[00:41:24] Oh my God. So that's frustrating. So I'm basically, I'm leaning data. My interest in data and AI are, are rapidly crossing over a lot harder about the AI features that I need to build on top of dataset. Make sure it stays relevant in a chat. G p t can do most of the stuff that it does already. But yeah the thing, I'll plug my blog simon willis.net.[00:41:43] I'm now updating it daily with stuff because AI move moved so quickly and I have a sub newsletter, which is effectively my blog, but in email form sent out a couple of times a week, which Please subscribe to that or RSS feed on my blog or, or whatever because I'm, I'm trying to keep track of all sorts of things and I'm publishing a lot at the moment.[00:42:02] swyx: Yes. You, you are, and we love you very much for it because you, you are a very good reporter and technical deep diver into things, into all the things. Thank you, Simon. Travis are you ready to announce the, I guess you've announced it some somewhat. Yeah. Yeah.[00:42:14] Travis Fischer: So I'm I, I just founded a company.[00:42:16] I'm working on a framework for building reliable agents that aren't toys and focused on more constrained use cases. And you know, I I, I look at kind of agi. And these, these audigy type type projects as like jumping all the way to str to, to self-driving. And, and we, we, we kind of wanna, wanna start with some more enter and really focus on, on reliable primitives to, to start that.[00:42:38] And that'll be an open source type script project. I'll be releasing the first version of that soon. And that's, that's it. Follow me you know, on here for, for this type of stuff, I, I, I, everything, AI[00:42:48] swyx: and, and spa, his chat PT bot,[00:42:50] Travis Fischer: while you still can. Oh yeah, the chat VT Twitter bot is about 125,000 followers now.[00:42:55] It's still running. I, I'm not sure if it's your credit. Yeah. Can you say how much you spent actually, No, no. Well, I think probably totally like, like a thousand bucks or something, but I, it's, it's sponsored by OpenAI, so I haven't, I haven't actually spent any real money.[00:43:08] swyx: What? That's[00:43:09] awesome.[00:43:10] Travis Fischer: Yeah. Yeah.[00:43:11] Well, once, once I changed, originally the logo was the Chachi VUI logo and it was the green one, and then they, they hit me up and asked me to change it. So it's now it's a purple logo. And they're, they're, they're cool with that. Yeah.[00:43:21] swyx: Yeah. Sending take down notices to people with G B T stuff apparently now.[00:43:26] So it's, yeah, it's a little bit of a gray area. I wanna write more on, on mos. I've been actually collecting and meaning to write a piece of mos and today I saw the memo, I was like, oh, okay. Like I guess today's the day we talk about mos. So thank you all. Thanks. Thanks, Simon. Thanks Travis for, for jumping on and thanks to all the audience for engaging on this with us.[00:43:42] We'll continue to engage on Twitter, but thanks to everyone. Cool. Thanks everyone. Bye. Alright, thanks everyone. Bye. Get full access to Latent Space at www.latent.space/subscribe

How do we keep up with this rapidly changing AI revolution? Steve Yegge, Head of Engineering at Sourcegraph joins Rob to discuss all things generative AI. Rob and Steve discuss the “innovator's dilemma” and the concerns around jobs and productivity.Have a guest you'd like to hear on the podcast, let us know on Twitter @circleci_sandbox!

https://www.youtube.com/watch?v=sKE1S7PK1fY 14.45mins inMy article on Google vs OpenAI: https://lspace.swyx.io/p/google-vs-openai

Listen to Stevey's podcast: https://youtubetranscript.com/?v=Wi8SL-Tot-8&t=1212Transcriptso let me tell you about service mesheskind of like the terminology just to geteverybody up to speed because i knowsome of you haven't looked at this spaceor haven't looked at it recentlyyou're going to hear two terms controlplane and data plane bandied about a lotand it's very confusing at first okaybecause first of all they are sort ofpoorly named and second of all there isactually a fair amount of overlapbetween the two in the in the serviceofferings that we have today all rightand in the tech stacks that we haveavailable so let me walk you throughthem all rightso starting at the uh at the servicelevel so you have a bunch of servicesmaybe they're on vms maybe they're inkubernetes maybe they're in nomad orfargate or whatever right but you've gotservices vms or containers and you wantto have them communicate with each otherall rightwell having rather than having them allcommunicate with each otherwhich obviously means you're going tohave to build like service discoverylogic into the service itselfso if i have a player servicelet's say i have a game server and itwants to go call the player service andsay is this player real okay if so giveme their give me their information giveme their credentials okay typicalservice to service uh you know functioncall rpcall right well you could have the gameserver say well i'm going to call theservice registry service to see uh wherethe player service lives and then i'llmake a call to the player service rightbut now you're building that i'm goingto call the service registry servicewhich is this other service right thatyou would have to build or whatever oruse ncd like grab did or whateverand then it has to call and get theaddress of the player service and thenand then it makes the call and it's likeyou've builtrouting logicand discovery logic into your actualapplication logic which you do not wantyou do not want that okaysoalmost immediately people started movingto proxiesyou have a proxy that's your local proxythey call it a sidecar proxy inkubernetes land because it actually runsin your little cluster as anotherservice along alongside all of yourother servicesand it handles allnetwork uh ingress and egress for youso you the idea is that your applicationonly knows about the sidecar proxy rightso to your application the proxy is theoutside world if if you you know itknows about the service locations and italso knows about circuit breakers andtraffic splitting and load balancing andscaling and everything else that we'lltalk about in a bitand that proxy becomes the thing thatother people use to talk to your serviceas well because your service may be acluster right and so people if peoplewant to send something to the playerservice and there's a bunch of instancesof it your proxy is the one to choosewhich one maybe maybe it interacts withan external load balancer or maybe itdoes the load balancing itself the proxydoes okay by doing the health checks onits local service instances yeahdoes this model make sense so as soon asyou get this basic model of the of thesidecar proxy you've got a helperservice that goes along with everyclusterand it knows about the services in thatcluster and it knows about the outsideworldand your cluster talks to the outsideworld through the proxy and the outsideworld talks to your cluster through theproxy okay you can use nginx for thatand that's what dropbox is doing rightbut these days people always almostalways use envoy or link or d there area couple of other options in addition tothose in nginx but i mean those are thereally popular ones okayenvoy is the the super industrialstrengthdoes everything swiss army knife amazingdata plane okay by the way those sidecarproxies i'm going to introduce you nowto the to the second term you hear dataplane the other one being control planedata planes is just all of your sidecarproxies in aggregate because if you ifyou've got a whole bunch of clustersright uh or even a whole bunch ofservices and you want proxies for eachof them thenthat mesh of proxiesthat are all talking to each otherto work out the service discovery andthe routing and everything on behalf ofthe application services now you'veextracted all of that you know who who'stalking to who what where and how muchand all that you've extracted it intoyoursidecar proxiesthat's your data planeit's because the network data is goingthrough that and i think it's a terriblename it should have been called thenetwork plane or the proxy point proxyplane would have been an absolute greatname for it rightproxy plane but no they call it dataplane so it's completely confusingbecause you'd think the data plane wouldbe either your application logic or itwould be the data layer behind yourapplication logic but noso stupid name really stupid shame onwhoever chose that name really you justyou did a huge disservice to theindustry so if you patent yourself onthe back because you came up with a namedata plane like seriously like punchyourself in the mouth okay it just itwas a bad namenaming you know naming stuff matters manyou don't want to confuse everybody forthe rest of their liveswhatever but the name is stuck and thename is the name now and in fact thereare well we've been ahead of ourselveshere but they're even becoming universalstandards now for data plane uhinterfacesso the data plane i mean like you'rejust going to have to learn what dataplane means it means it's the proxylayer okay the proxies that can uh couldload balance and they can they theyhandle the network for you it's softwareload balancing they actually in envoythey actually communicate through aprotocol called a gossip protocol whichis a family protocols where they're sortof like udp multicast whereeverybody just kind of like spits outthe state and consumes the state and itsort of floods the networkand it's eventually consistentso that's one thing to know about envoyis they chose an eventually consistentmodelif you'll recalli said that etcd and technologies likeit like google's chubby or uhzookeeper or uh even hashicorp consolethey're all they're all key value storesthat areum transactional highly available andstrongly consistent okayuh and that actually makes them uh sortof a pain to operateuh in practiceall of the ones that i just mentionedchubby is an interesting one google'schubby it was probably the first uh mikeburrows i think uh did chubby and if youhaven't heard the name mike burrowsuh you really should know his namebecause you know he's easily one of thethe people who had 10 people who've hadthe most impact of google rightuh he's you know i don't know he's a deor whateverand uh and he he came up with chubby asfar as you know among other things andchubby is umchubby is distinguished ashaving something like seven nines ofavailability it was down for 30 secondsin 10 years something like thatso umso yeah and it's because google has acore competency of operating chubby atscaleright because it's the it's the centralyou know key value service for serviceuh discovery and information exchangefor all of google right so chubby couldcause global outages so seven nines ofavailability there it's prettyamazing you're not gonna get seven ninesof availability out of your ncd clusteri'll tell you that i think that i mighthave been talking into the back of mymicrophone this whole time as part of myum my new setup uh so that's that's kindof a bummer i hope i hope that i wasn'tand it just rolled over my goodnessumall right so yeah this is still work inprogress apologies folks okay so we weretalking about data planes you guys ithink understand now why data planesexist data planes exist to abstract awaythe network and the service topology andsecurity groups and circuit breaking andall of the other things that are stackedup on top of communication the serviceproxies also handle a lot of heavylifting ofyou know managing tcp proxying or theycan do udp tcp http http 2 http 3 theycan do grpc they can do all sorts ofprotocols envoy has filter chains whereyou can implement a lot of these thingsit's very very very powerful envoy is onlook everybody agrees that envoy is likethe data plane to usewith one exception which is if you areusing kubernetes my understanding isthat linker d is custom fituh has more or less the same protocolmuch fewer features but it's also muchhigher performance and i think easier tooperate so some people use link or d andthe control planes which are basicallyjust the configuration stores so itshould really be called theconfiguration plane but whatever thecontrol planes for these service meshesusually can use envoy or link or dokay but if not if they only have onethat they support it's usually envoybecause it does everything all rightokay let's build on what we've learnedso far all right we've learned aboutdata planes we've learned that they do alot of stuffenvoy you know out of the box does l7load balancing and it does l3 l4and they also do so what else does envoydo for you so you can just use envoy andby the way i started by talking aboutdropbox dropbox's article remember isthey they moved their data plane fromnginx to envoy and you can build anentire service mesh of your own on topof envoy although you're pro probablygoing to need something like etd rightor zookeeper depending on how you've setit up but but you don't absolutely youabsolutely don't have to have it if youthink about it scd is a little confusingbecause i mean depending on your needsright likeit might be okay envoy is eventuallyconsistent right and etcd and all theseother paxos based key value stores arestrongly consistent right so which onedo you need right well envoy argues thathey it's service discovery it's okay forit to be eventually consistent meaninglook if we accidentally route somebodyto a service that's going down and theywind up getting an error and have toretry as long as it's it's a tail caseand it doesn't happen very often then uhit's probably okay because of retriesright and so envoy you know pushes someof that that retry logic that you youdon't need in a strongly consistentsystem where as long as that cd is upyou're gonna get an accurate up-to-dateservice instance right but i mean envoytakes the approach that it's like wellwhat if you call lcd or zookeeper andyou get yourself a service instance andthen it immediately dies right likestrongly consistent doesn't necessarilymean that the service is going to beavailable for the duration of your callto it so why not go with eventualconsistency which dramaticallysimplifies things and speed things upand it's an interesting i don't knowit's an interesting take everything thati've seen built on top of it winds upusing strong consistency so i don't idon't know who's right here but it'sit's an interesting thing to know rightis that envoy is generally attuned foreventual consistency in that gossipprotocolenvoy is written in c plus i believe itwas created by lyftenvoy is it's its own thing now you knowit's it's a it's a huge huge system witha massive community contributing to itand it really does everything it alsodoes things like so it does loadbalancing like like i said and it doestraffic splitting and it does uh youknow filter chains the filter chains areamazingly powerful and can do all sortsof um important stuff like um you knowcalling your cert authority to you knowvalidate ssl certs and umtransformations between protocols andall sorts of stuff well hell let me pullup the listyeah grpc proxyingit can do health checks it can do stagerollouts that's why i looked over here ithought it could do red blue or stagerollouts with traffic splittingpercentage-based traffic splittingwhat's that forlike are all these words do you guysknow what all these things are you knowwhat load balancing and dynamic servicediscovery and auto scaling are tls isthe new ssl and tls termination is youknow you have to do it somewhere toactually integrate with the certauthority and whatnot so you can do allthat in the proxy circuit breaking is arelatively new concept where you ratherthan overloading a service causing itsperformance to degrade and all sorts ofalarms going off andpotentially scarythings happening like data corruption orwhateverwith circuit breaking you basicallyconfigure the circuit breaker to say i'mnot going to take more thannqps and then i'm just going to likeopen the circuit and we're just going tostop stop sending stuff through rightand so you get an immediate alarm and ofcourse a cascade of circuit breakersupstream andand they can be a little tricky tomanage especially since in a lot ofsituations the client of the service theperson calling the service is expectedto configure the circuit breaker andthey don't really know have theinformation to configure it properly socircuit breaking is a bit of an art tosay the leastbut it does seem to be preferable tonot circuit breaking which just allowsservices is an arrangement whereservices will just fail arbitrarilyunder heavy enough load you definitelydon't want that okay so envoy can handlecircuit breaking it can handle faultinjection so that you can do things likechaos testing and what is netflix'schaos monkey right you can actually dothat in the proxy plane the proxy planeyou know if i start calling it the proxyplane will you and you guys startcalling it the proxy plane maybe we'llactually be able to like eventually killthe term data plane for the proxy planewhatevercan't have it all i guessso and it'll also do logging accesslogging you know all the things youexpect out of things like nginx umso it's you know it really is a prettyuh pretty robustmesh on its own envoy is envoy's reallycool and linker d does some of that butas you can as you can imagine like youyou know you sure load balancing youknow some of these things you knowprotocol transformations filter changethings like that those make sense in theproxyright maybe but if your proxy is onlyfor kubernetesand not for redis then obviously linkerd doesn't need all that stuff rightlinker d probably has you know accesslogging and observability and and maybemaybe uh tls maybe tls terminationbut uh uh it doesn't have a lot of thefeatures of envoy but it performs muchmuch better it's a much much smallerbinary and so it's it's fairly bespokefor kubernetes again it's also a verygood piece of technologyand there are some other ones out therebut honestly like if you're a cio or ctoor just a team lead even and you justwant to like um you want to decide thatyou're going to as a team lead youprobably shouldn't be making thisdecision you should you know yourcompany your organization should not uselike multiple service meshes andmultiple data planes and so on youshould really probably try tostandardize on one but if you're a teamlead who's responsible for maybe provingone out before you roll it out morebroadly to the rest of the company thensure you could make this decision too soi'm telling you unless you're like akubernetes only shop and you you knowyou're basically being backed into usinglinker d by the stack on top of it useenvoy like that's that's just it's ait's a no-brainer envoy has basicallyreplaced all of the other like proxytechnology out there there's no reasonto use anything other than envoy unlessagain you really really need a verylightweight high performance kubernetesonly installation and then you can uselinker d all right blinker d pluswhatever things that you're going toneed to use because they're not in linkor dso you with me so far so everybodyagrees that envoy is the cat's meoweverybody does envoy is itokay and the and the the the guy thatinvented envoy i'm sorry i forgot yourname man but amazing job but i i readhis his blog posts periodically and hishis updates and he he talks about youknow the decision process you know forhow envoy started and how it evolved andhe talks about other data planes and thefact that they really all ought to bepluggable and they shouldn't all justassume envoy and so he has been drivingalong with some others a universal dataplane interface or apiudp it's calledi think it's udp universal data planewhich has an unfortunate collision withudpbut whateverso again this is probably the guy thatcame up with another reason not to callit the data plane folks it could becalled upp if it was the universal proxyplane and believe me everybody wants uppinstead of udp becauseuh all the collisions and names here allright universal data planes data planesproxies we went through the list of allthings envoy does look if you're a ctocio or team lead trying to provesomething out absolutely you're going towant to use envoy and if you don'tbelieve me believeall the service meshes because most ofthem use envoy under the covers as theirdata planeconsole has envoy integration i don'tthink it requires envoy but since somany people use envoy console happilyintegrates with it okayistio uses envoy and it's not pluggableit just assumes envoy all right andthere were a couple of others maybe kongi think kong's service mesh may be builton envoy as well but don't quote me onthatyeah i double checked and kong in factis deeply committed to the success ofenvoy and so even kong which is also afantastic offering that we'll talk aboutin a little bit

Gergely Orosz writes the #1 technology newsletter at Substack, called The Pragmatic Engineer. He started his career as a software developer in the U.K., spent three years at Skype, and followed that role with four years as an engineering manager at Uber before deciding to leave big tech and work for himself. Gergely began pursuing his newsletter full-time in September 2021 and in just one year has amassed 200,000 subscribers. He now makes more money than he did at his salaried tech job, and with freedom and flexibility. In today's podcast, Gergely shares why he left his well-paying job at Uber, how he got his first 1,000 subscribers, why this kind of work can be stressful and lonely (but ultimately rewarding), and why it takes hard work to build authority and become a great writer. Working solo can be challenging, and in this episode, both Lenny and Gergely offer tips for structuring your unstructured time and finding your focus.—Find the full transcript here: https://www.lennyspodcast.com/leaving-big-tech-to-build-the-1-technology-newsletter-gergely-orosz-the-pragmatic-engineer/#transcript—Where to find Gergely Orosz:• Website: https://www.pragmaticengineer.com/• Newsletter: https://newsletter.pragmaticengineer.com• Twitter: https://twitter.com/GergelyOrosz• LinkedIn: https://www.linkedin.com/in/gergelyorosz/—Where to find Lenny:• Newsletter: https://www.lennysnewsletter.com• Twitter: https://twitter.com/lennysan• LinkedIn: https://www.linkedin.com/in/lennyrachitsky/—Thank you to our wonderful sponsors for making this episode possible:• Lemon.io: https://lemon.io/lenny• Eppo: https://www.geteppo.com/• Vanta: https://vanta.com/lenny—Referenced:• Gergely's books: https://blog.pragmaticengineer.com/books/• Centered: https://www.centered.app/• The Pomodoro technique: https://www.forbes.com/sites/bryancollinseurope/2020/03/03/the-pomodoro-technique/• Coding Horror: https://blog.codinghorror.com/• How to Achieve Ultimate Blog Success in One Easy Step: https://blog.codinghorror.com/how-to-achieve-ultimate-blog-success-in-one-easy-step/• A Comment Is an Invitation for Refactoring: https://blog.pragmaticengineer.com/a-comment-is-an-invitation-for-refactoring/• Kent Beck's website: https://www.kentbeck.com/• Steve Yegge's famous rant on Google vs. Amazon: https://www.alexanderjarvis.com/steve-yegges-famous-rant-on-google-vs-amazon/• Stevey's Tech Talk: https://www.youtube.com/playlist?list=PLZfuUWMTtMcC1DZF6HxJhqsGrBXu8Jzi7—In this episode, we cover:(04:32) Gergely's background(07:19) The Pragmatic Engineer, growth and current subscribers (08:59) Compensation with a subscription-based newsletter vs. his salaried position at Uber(10:55) How the onset of Covid and layoffs at Uber prompted Gergely to start his newsletter(23:10) What he did immediately after leaving Uber(25:41) The day-to-day of writing a newsletter(35:08) Tips for productivity(41:19) Gergely's favorite parts of entrepreneurship (43:15) The downsides of solo work(50:39) Why Gergely stopped making long-term plans(54:30) How to get started writing a newsletter(1:04:48) Key advice on building a successful newsletter—Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email podcast@lennyrachitsky.com. Get full access to Lenny's Newsletter at www.lennysnewsletter.com/subscribe

From the Chariot Blog We’ve always got great content on the Chariot blog, written by our developers: it’s got over 20 years of tech reviews, tutorials, and more. We’re hiring! If you’re a senior software engineer interested in working with bright, curious colleagues who love to share what they learn, consider joining our team. Open ... Read More The post Chariot TechChat #54: Virtual threads in Java, Steve Yegge, and the forgotten OS keeping NYC’s subway alive appeared first on Chariot Solutions.

Listen: https://twitter.com/sourcegraph/status/1577687896814911488Steve's post about Sourcegraph:  https://about.sourcegraph.com/blog/introducing-steve-yegge https://www.youtube.com/watch?v=viNB7v3bgx8

watch Steve Yegge's podcast https://www.youtube.com/watch?v=7GurMGEDHUYTranscript[00:00:00] So this week we've been going through Steve yogis podcasts and his greatest hits his updated perspectives on the big clouds and what they're doing right. And what they're doing wrong. But the other thing that Steve is really well known for is his views on tech interviewing. And he's done in big tech interviews and quite a lot of them.  And we all know they're broken in some way, but it's often in very stark reminder of how broken it is. I think there are two anecdotes here. I want you to look out for, which is the first, the one on Jeff Dean. Just look out for that name. And second, the one on them reviewing their own packets and applying too high of a bar saying too many nos. There's a lot of false negatives in the industry.  Both false negatives and false positives. R a problem. Of course. And he's just some ways to handle them. But overall, I just think we, we deserve some reminder of how flawed it is when we do our own interviewing. I thought I had a bad run of it doing two interviews a week. And he did multiple a day, sometimes three at once. And i just think this is a fantastic story to go over So the thing about interviewing is it's a terrible signal. It's, it's better than a phone screen. And a phone screen is better than a resume screen. If you just look at someone's resume, how sure are you that they're good. I mean, in any, in any discipline, right? You know, you wanna, you wanna, you want an airplane, airline, pilot, you look at the resume. Will you just hire them based on the risk? Not usually. So the resume is, is your first filter. It's the first thing where you basically take a stack of resumes and there's an art to reviewing resumes and looking for people that are kind of trying to cover up, uh, things that, that, that, uh, they may not know. And they don't want you to know that they don't know. So they try to cover it up in their resume. So you can look for. Weasel words, and it's all kinds of things you need, but basically you're taking the resumes and you're, you're sorting them into two piles, right. That the keeps in the don't keeps and there's of course, the old running joke in the industry about how you want to take some resumes and just throw them in the trash can because you don't want to hire unlucky people. And so if you throw in the trashcan, that person was unlucky, but they do sort of the resumes into the I'm gonna follow up. And the ones that you just say pass. So writing a resume is really important. And part of, um, a book. Passing technical interviews would be on how to write a great resume. And this comes up again when you're writing your resume, so-called resume for what you've accomplished your company. When at time it's time to get promoted. So the art of resume writing never, never gets old. It never leaves you and is always an important part of your career. Being able to represent yourself. But that's a, that's just step one and it's a bad filter. You don't want to just base your decision on a resume. Would you marry somebody based on their race? Maybe, but probably you'd want to meet them first. Right? So the next step is a phone screen and everybody hates doing phone screens. I actually love doing phone screens. I, for some reason have, um, never really had an issue with them unless there's a bad connection or something, but a lot of people just hate talking on the phone and they even more hate having to ask people technical questions on the phone. So I often got stuck with phone screen duty at every company that I ever worked. Because you can actually do a pretty good job, not a great job, but a pretty good job of predicting whether they're going to pass their interviews based on my phone screen. Cause my phone screens would go for two hours if necessary to sort of, you know, get a comprehensive look at what this PR this candidate is good at because the general rule is like the longer you spend evaluating somebody than the better. Idea. You're going to have of whether they're going to work out. Long-term just like the longer you have a relationship with somebody before you decide whether to marry them or not the better you're going to know how that marriage is going to go. Most likely there is a point of diminishing returns and we'll talk about that. But by and large, The amount of vetting that we do in the industry today is nowhere near enough. And I'm going to, I'm going to talk about the consequences of that and how we, how we arrived at that conclusion. And so on in this, in this talk, but at a high level, I don't believe in interviewing anymore. I, I ha I'm a strong skeptic. I think that interviewing is so flawed. It's it re any company that really wants to get ahead of their competitors and succeed needs to spend some time re-inventing their interview process. And probably having people spend more time with candidates than they're spending today. It's, it's just not a very good signal. And I said that at Google once, uh, Google, I said it in, in an email, uh, replied on some public thread somewhere, um, in the early days, maybe 2008. And. Some director got mad at me and said, oh, we didn't like that. We didn't the [00:05:00] records in life that you said that you had, that you're a skeptic of the interview process. We were talking about a company that hires scientists. We're talking about a company that, you know, one of their models is speak truth to authority, and this director was an ass and, uh, he got what was coming to him eventually. At the time, you know, he was just like, well, everybody's upset because you're, you're, you know, you're questioning the sacred interview process. You farted in church is what he told me. And so, uh, and so I haven't really been able to tell people this for my entire career because they feel that it's undermining their, um, ability to attract the best. I guess, but the reality is if you marry somebody after dating him for four hours, you're probably going to get a surprise. Maybe it's a good surprise. Uh, but most surprises are not so good in that department. And interviewing is the same way. So if you're going to keep your interview. Uh, panels the exact same way that they've been doing it since Silicon valley was invented by the arse hole shot shot key. Uh, then, um, then you're going to need a better process for getting rid of people who are no good. You're you're going to need a, you're going to need to double down on your process for managing people out. That's actually how Amazon gets by and gets such great. They aggressively manage out under performance because they know that underperformers are gonna sneak in. And, uh, it's because the interview process is fluid. So it's just a best effort. The problem with the interview process is that it takes a lot of time. It's really miserable for engineers to do more than two or three interviews per week. And most companies try to cap it so that you're not talking to more than maybe two people per week. Okay. Or three, if they're really busy, uh, because it takes you. Uh, an hour out of your day to, to interview the person. And you may have a interview pre briefs where everybody gets together and maybe divides up what people are going to talk about. It's not recommended at some companies, but some companies do it anyway. And then you may have a post brief where everyone gets together and discusses a candidate afterwards. Also not recommend recommended. I could do a whole segment on Google's interview process and how it gets away from a lot of biases and little kids, soccer team situations where one person says, well, I didn't like him. And everyone was like, well, I didn't either because they don't want to rock the boat and all kinds of bad things can happen. Uh, so we can talk about that, but that's not what today is about today. It's just about just interviewing in general because what they found, this is a bit surprising, uh, is that, you know, cause Google hires a lot of statisticians. We're talking about people who are. World-class experts in their field. And since Google has this, you know, surfeit of, of statisticians, they can apply statistical methods to a variety of problem spaces. Google gets like a million plus resumes per year. And somebody has to go through all those resumes. It took me five applications to Google before they finally noticed my daughter's resume over a course of a year. I sent in one resume and then a couple months later I sent in another resume and a component's me and this went on until my fifth resume came in to Google and somebody looked at it and said, oh, we gotta get you in here for an interview. And I was like, great. You know, he could have done that like a year ago before the IEP. But whatever. So, uh, so they get a million resumes a year. That's a lot of data. That's a lot of statistical data. They do a lot of interviews. They do a lot of fun screens and they, and they add a lot of followup data, like performance data on everybody who got accepted. And so they can basically run a bunch of statistics and maybe even do experiments. They can set up special interview loops, where they set up conditions and control control groups and kind of try and see what happens when they try this or that to influence the interview process. Okay. Sounds pretty interesting. Right. And you probably think that they came up with a really, uh, really useful insights on how to conduct better interviews and the number one insight that they came up with and they shared this broadly with everybody and the, and they were just like, we're sharing this with you. We don't know what to do about it either. Was the, the result was they found out that there was zero correlation between your interview performance and your actual perform. Your, your interviews did not predict your performance. You could get a 4 0 4 us from every interviewer, the highest score, uh, at Google and wind up being a very mediocre performer or even a bad performer. And you could barely scrape by you by the skinnier teeth on your fifth application. Uh, and because I applied 55 times, but some people interviewed five times. I know one guy who actually got into Google. Fifth interview. He failed the first four and they make you wait six months. They're like, well, six months is long enough for you to go learn some stuff. So come back in six months, six more months. And he did that four times on the fifth time. K after, you know, two and a [00:10:00] half years have gone by, he finally got accepted to Google and he was a superstar. So you look at that and you say, well, okay, I've got some good anecdotes, but the statistics actually supported that. Basically your interview is now you're, you're either in or you're you're either in or out. And after that, your performance is, is it's not predictable from the interview. Another thing that they found, which was equally damning, I think no individual interviewer is better than any other interviewers when it comes to predicting whether the person will get hired or. And they specifically called out Jeff Dean. Uh, who's the number one engineer, not only at Google, but probably arguably in the whole world. And they said even Jeff Dean's interview scores were no better predictor than, than random of whether the person was going to get an offer. Which was kind of weird because everybody thinks they're really good at interviewing and it turns out no nobody's especially good at interviewing the best predictor was if around four people. All decided that they want to hire this person. And that's, that's, that's that's that's when you get the best signal. And once you start piling more than four on they, they, they found that the curve tapered off and immediately you hit diminishing returns. So, uh, starting at five interviewers, you're not getting any better information. And there are companies that I've been at where that have put people through, you know, 8, 9, 10, 11 interviews, 12 interviews. Trying to figure out is this person good or not? And they're just not getting any better information after the first four. That's the stark reality. And all these statistics basically backed up something that I had been fretting over and worrying about ever since I joined Google. Cause when I joined Google, of course, a bunch of people wanted to follow me there because, uh, I had a reputation. I was the Canary. When, when I left people, people were like, well, Either either the ship is seeking and we need to get off the ship or Steve's found something that's so much better that even though our current ship is good, this new one is something we've got to get into. And I had a reputation for that amongst my circle of friends of, you know, a hundred, a hundred plus people. And so a bunch of them tried to get in and they were good. Some of them were better than I was. And some of them were very clearly better versed in computer science than I was. They had contributed more. They were just better than me and they didn't get into Google and. And I knew right then that there was something seriously, seriously wrong with the process. Now, of course, we run into situations where, you know, I've said it before, why CEOs fail, they failed because they put their faith in the wrong people. And I've certainly done that myself in my career in recent times. I mean, it's very easy to do. Sometimes you're putting your faith in a person who might have all the qualifications on paper, but people just don't like. And there's nothing you can do about it, because if everybody hates you at your work, you're not going to be able to get stuff done no matter how good you are. So, you know, it's a problem because I I'm biased. And when I say the people who got turned away, uh, and I say they should have gotten jobs at Google and they got turned down by the interview process. You know, I am biased, you know, they're my friends. And in some cases, maybe I was wrong about them, but the ones that I'm thinking of, one on to, in their careers to become senior principal engineers at Amazon of which there are probably only 20 in existence, you know, or 25. And, and so these people are not slouches and they, they basically, you know, helped build the Amazon that you use today and they didn't get jobs. So, I guess one takeaway is if you interview at Google and you interview three or four times at Google and you don't get the job that doesn't actually mean that you weren't qualified to work at Google, it means you were what's called potentially. It potentially means you were a false negative, a false negative is where you, you do a test and the results of the test is no. And it should have been yet. So, for example, a false negative for, uh, an illness. If you take a test for COVID and they said, no, you don't have it, but you actually do have it. And the test dismissed it. That's a false negative, false negatives are, uh, you know, a huge, huge problem in the industry and their. Especially bad problem at Google. Google has probably a higher, false negative rate than maybe anybody else in the industry. Maybe Facebook, uh, since they got a lot of ex Googlers and they, they brought in a lot of people and they kind of copied a lot of Google's interview process. And so I'm sure Facebook has false negatives too. Uh, but those two are going to be the top. And false negatives really hurt. They really hurt the company. Like more than companies are willing to acknowledge they hurt because first of all, they've lost out on a great. And they just, the candidate showed up wanting to come and help the company and the candidate was great. Uh, but their interview process said, no, not great. So we're not going to hire them.  So that's that's problem. Number one is you is opportunity cost, but problem, number two is now that person is bitter and resentful because they know they're good enough to work at Google and they got turned down. And so now they start to hate Google and they start to bad mouth Google or Microsoft or whoever. [00:15:00] Did this, every company has false negatives and every company goes through this problem where they, they create enemies, Microsoft actually in the nineties. And I interviewed there once for an internship back when I was very early in college, so I didn't get the internship. And, uh, but they were, they were real jerks during the interview. At least Google was kind of nice, um, stressful, but nice in Microsoft, they were very arrogant and they would just sit there and make you struggle for 40 minutes without giving you any, any hints or indicators or anything. And it was just the way they ran things. And at Microsoft, they, they basically realized after 10 years of. They did some studies and they did some interviews with people out in the industry and they realized they were creating an army I'm fast army of people who hated Microsoft because of their terrible interview process, their experience that they had because people would, would be jerked around at the Microsoft interview and they would leave and they go, well, screw that company. And, uh, and, and screw the horse they wrote in on, and they go tell all their friends about it. And now everybody in the industry is like, well, Microsoft, a bunch of jerks, and this is such a big problem that it, it, it escalated all the way up to. You know, the senior executive levels and it was part of their corporate consciousness, you know, where they were like, oh no, we have an existential problem, which is we, we, we we've been screwing people over for a decade and now they're poisoned against us. And so they turned it around. They made a huge effort, a huge initiative to be nice to candidates during their interview so that even if they don't get the job, they still feel like they got a fair shot. And that in turn started, started to fix the problem of, of poisoning the well of candidates. So, you know, all, all very interesting. Right? What have we learned so far in my, uh, you know, 20 minutes of talking so far for this goes by fast, we've learned that interviewing is not a very good signal. There's a lot of false negatives. There are, um, a lot of interviewers out there who think they're really good at it, but statistics show that they're actually not. We've learned that if you do more than four interviews on a candidate, that you're wasting your time, because four is enough to tell you the right, the right answer. Uh, and of course the right answer is if you're not sure than, than Beaumont. Right, right. That's what everyone says. Is it though? I don't know. I don't know. Right. Maybe, maybe if in doubt you do hire them and then you fire them three months later, if they're just really, really, uh, out of their league, if they're, if they're underperforming, because the really the way it should work is is if they're close, because interviewing is just so flawed and cause false negatives are such a huge problem. Because it's opportunity costs plus it's poisoning the well, you know, not to mention the third problem with false negatives, in addition to in bettering everybody and losing you a great candidate, they go to one of your competitors. So no triple, triple whammy right there, uh, losing, turning away somebody who's really going to be actually do a good job at your company is really, really bad for your company. But companies are really scared of false positives. They're really scared that they're going to hire somebody who wasn't qualified. They're terrified of it. They would much rather have false negatives than false positives. And. Most of them can get away with it. Um, because well, they could up until recently because there were plenty of engineers out there and they could just keep, keep sifting through the pile until they get one that's just unambiguously a great candidate. And then they, then they're mediocre performers. Right. Because the whole, the whole thing is just, it's a mess. I don't know. I folks, I don't know what to tell you other than you need to do more than just interviews with somebody to really understand what their contribution is at the very least somebody should be tasked with finding a dossier on the person. Let me tell you a little bit about one thing that Google does. That's kind of cool that most other companies don't do it. It's called hiring committee and I was on a hiring committee for a long time. Hiring committee is this group of people that reviews the feedback from interviewers. So at Amazon and most other companies, the interviewers, the people who actually interviewed the candidate, they get together afterwards. Maybe it's immediately afterwards. Maybe it's later that day. Maybe it's the next day after it started to fade, it can kind of get kind of bad when a week goes by and you're like, oh, what did they do again? So at least take good notes after you interview someone. So they get together and they decide together as a group based on their experience with a candidate, should they hire them or not? That's how most companies work and it's actually terribly biased because, uh, these people biases. Like I mentioned a little kid's soccer team earlier, right. Where the ball shoots out and then the whole team runs after the ball. Somebody will say one thing and then everybody will just be like, oh yeah, this candidate sucks. Forget it. Forget everything. I, you know, yeah. They'll, they'll say, oh, I had a really good interview of the person, but don't pay too much. Don't read too much into it because of what Joe said. And so they wind up exacerbating the false negative. And in some cases, you know, they can sneak false positives through that way too, if you really [00:20:00] liked the candidate, but usually it's easier to say no. And so you get this false negative problem, a Google, what they do is everybody writes up their notes. You write up very detailed notes of what you asked the candidate and what their answers were. And you include if possible, any source code that they wrote as part of their. Or any drawings, they did everything. And you present that you, you, you put it into a system. And then later a hiring committee meets like once a week and they go through all of the interview results and they say, okay, let's decide who of these people are. And it's a really cool process because, uh, now you, you have no idea who the interviewer is, maybe, uh, and although you can get statistics on their, their, their past, on their distribution of yeses and nos or their scores. Right. And, uh, you know, one thing that they found is, is if people give. 2.0, meaning they're not sure, you know, I'm just going to punt. I didn't get a good enough signal in my interview. That's a useless interview in there a week. Interviewer, you should come out with a yes or no that lets you know, kind of seems obvious. And so they can, they can tweak and discount somebody. If somebody always says. They can say, well, this person does always say no. And they said no to this person over here who got hired and they're doing great. So they can discount it that way or they can discount it if the person is too easy and always says yes. So they have a little bit of leeway there in, in basically wiggling around. Uh, if one of the, one of the interviewers is really strong. Yes. Are really strong. But basically they don't know what's going on. They're blind. They just, they're just looking at feedback and deciding whether to hire the person or not. And it seems like a really, really cool process in principle because it gets rid of that bias of the individual interviewers, biasing, each other. There is a problem though. Okay. The recruiters because you work very, very closely with recruiters during this whole process. Tech recruiters, recruiters are great. I always say, be kind to your recruiter every day is be nice to your recruiter day because they are your partner. When it comes to getting great people into this company, the recruiter is the one representing the company. They're going to be the ones sweet talking to the candidate. They're going to be the ones telling the candidate. Oh, I'm so sorry that you didn't. You didn't get it, but you know, they thought very highly of you giving the, you know, the, the candidate, good experience recruiters can get you good resumes. They can, they can, they can, um, help sort of try to direct people your direction. If your team is hurting many, many, many reasons to be good to your recruiters. And a lot of people just treat them as administrators and they're just like, whatever, get that recruiter over here. And it's, it's, they're dumb. Those people are dumb. And they probably kick puppies. So anyway, uh, you're working with the recruiters and the recruiters came into us one day in hiring committee and they said we today, uh, before we review the resumes, we're going to do a little exercise. Okay. What we're going to do is we're going to present, we're presenting you. We're going to give you some packets to. These packets have been carefully selected. And what we're going to do, this is sort of a calibration exercise. We're going to see, we're going to try to see if, if you, uh, you know, if your results match with other results across the company. They gave us some story and they gave us a bunch of packets to review. And so we reviewed the packets. What's a packet, a packet is the candidates resume. Uh, and, uh, but they wouldn't show us the resume. They would only show us the interviewer's feedback. So we had to make our decision not based on the resume, but based on only what the interview was. Okay, fair enough. I mean, the interviewers give a lot of details. The resume almost doesn't matter anymore at that point, because what really, you know, what really matters is how did they do in the interview? Right? So we went through and we were out allowed to ask questions and they would go in and check the notes and stuff. But basically we were, we were doing this totally blind and we had to go in as a group and do our regular thing to decide whether or not to hire each one of these packets. And we ended up projecting 30 or 35% of them. We passed like 60%. If I remember correctly, it was, it was a, it was a pretty hefty number that we turned down and then the recruiters shared the results. They shared the secret with us. They, they, they let us in on the surprise, which was at the packets we were reviewing were ours. We were reviewing our own interview feedback and we had decided not to hire one-third of ourselves. And again, it did nothing, but disillusioned me even further, just, just leading new leave that this, this whole process is just it's garbage. It's speed dating. It's me making marriage decisions basically based on speed dating. And, uh, and I, and to this day, I think that the, the process is flawed. I understand why they do it. I get it. They do it because. It's a compromise. It's a compromise between how much time and resources are they willing to devote to it versus how good of a signal they're going to get? Because [00:25:00] it's really, really hard. Like I said, to, to participate in lots and lots of things. Amazon went through to get big, fast phase back in the early two thousands where they were just like, oh my God, we got a girl that crazy. We've got a bunch of, you know, funding and we've, we've got a bunch of things that we want to get done and we need a bunch of people fast, get big, fast GBF, and we're gonna, um, we're going to interview. They're crazy. And so we went through a couple of years at Amazon. I don't know if it's still like this. I hope not where, uh, we had to interview many, many people. Every day, not just, not just like many per week, but like I would get many interviews per day and we had to stay on top. You had to interview the people that would come up with interview questions and you had to write up your feedback and you had to, you know, do that between juggling your regular job, which was already hard. And then it got to the point where I was starting to get like double. So they would have me interviewing two different people in different conference rooms at the same time. And I'd be running back and forth, apologizing to the candidate saying, I'm sorry, I've got to run. I'll be back in five minutes, work on this problem. And I run to the other candidate. And then one time I actually got triple booked and I was running between three different rooms interviewing three different candidates. And it was just, it was untenable. It was ridiculous. And it was not a great candidate experience. And it certainly wasn't a great experience for us. And when I, for a few years, because we were just trying so hard to vet people and that's, that's all we had. It's. So, so you can try to throw the, you can try to scale up, but just making everybody work really, really hard, but that, that can lead to some serious burnout and really bad outcomes. And at that point you might as well just throw darts to figure out who you're going to hire. So, uh, so is there a better way? Well, yeah, there, there is actually a better way. This is Swyx here. So he goes on to talk about pair interviewing and internships, which I didn't think were very controversial. So I cut it out.  Don't believe too much in your interviewing process. Don't believe in it so much that you require somebody to go through 10 interviews. Cause it won't help. Don't believe that some of your interviewers are better than others because actually they're not. And, and that's been shown again and again, statistically at Google, they tried year after year. Uh, and, and, and don't believe that, um, that the, the, the interview performance, how well somebody does an interview, all that shows is how good they are. That's what we learned. We learned that, that if you do really relevant really well in an interview, then you can conclude beyond a shadow of doubt that this person is really good at interviewing, but they may not be any good at actually working and getting stuff. So do whatever you can to try to improve the quality of that signal. If you're on the interviewing side, you know, try to try to get, um, uh, you know, try to get two people into the room and do pair interviews and see how it works for your company and for your candidates. Because you're going to find that it's going to co. It's going to fix a lot of problems with bad interviewers, uh, and do your best to try to get the person, to try to understand what their actual work is like. Go look at their GitHub, go look at their, any, any projects that they got, where you can actually see. This is the dossier I started talking about earlier. Where you can see what the contributions are that got me, like I said, I got that, got me, my job at Google, my contributions, you know, in the form of, you know, what I did for my computer game was enough to sort of make a tie breaker and get me another round of interviews, you know, do that, do try to get a complete picture of the candidate because you know, you're forced be dates is not going to be enough. And then finally, maybe try to turn the. A little bit towards, remember how I said we didn't hire ourselves 30% of us. We decided not to hire, uh, you know, try to turn the novel a little bit, to be a little bit more forgiving and then have a process. By which you tell the candidate, you're in an evaluation period. And if after three months or six months, you know, you're not performing up to snuff, we're going to down level you, or we're going to find another role for you. Or unfortunately, we're going to have to, you know, find a way to get you a job at another company. Basically give you an opportunity. However, HR and recruiters say this, they have magic ways of saying it and don't ever let an engineer tell a person that they have to go. Cause they'll do a terrible job. Like I just did. But have a process for managing out your false positives. And then, and then you're just going to have, you're going to have better outcomes. You're going to have better people. You're going to have happier. People are going to have more productive teams and you're going to have a better candidate experience.

Listen to Steve Yegge's podcast: https://www.youtube.com/watch?v=L0xmHrQJdAwthefirst in this little series was i talkedabout their ability to root out diseaseand dysfunction in theorganization and squash it immediatelythe second one was about their focus onretail customers and individual peopleand how they put that front and centerin first and foremost and there was noalso customer service mentality therein this episode what we're talking aboutisthat situation where grabs servers wouldrun on amazon's cloudso it's like a rental service it's likewe rent computersfrom from amazon and we have otheroptions we could have been on google'scloud we could have been on microsoft'scloud and there were some effortsactually to get onto microsoft's cloudat least part of the computing just toreally mostly i think for negotiatingleverage butbut the reality was grab was not reallythat importanti mean there are a lot of companies oncloudessentially someday all companies thathave any sort of computing in thebackground which is most companies willhave uh a cloud presence okayand so you know huge huge names you knownetflix runs on amazongo figure they don't have their own datacenters as far as i knoweverything you know i certainly knowtheir biggestthey're uh they're amazon's biggestcustomer or they have been and they goin and out of being amazon's biggestcustomeryou look at the top 50 customers foramazon and uh grabs not in that listyou look at the top 100 customers andgrabs probably not in that list just interms of how much they're spending okaycorporate customers uh you know prettypretty sizable chunk of money but notnot really a blip at amazon scaleand yetuhwhenever i had a question uh aboutamazon's cloud let me tell you what iwhat i didokayumi would sayhey bob can you come over here for a secyeahnotice i'm not touching a phone or acomputer uhi'm talking to bob over here who who isfrom amazon he's an amazon employee he'sa cloud specialist and uh knows how toanswer a lot of customer questions uhhe's an engineer uh and and sue you knowbob and sue she would do the same thingthey'd come in we had all these thesedifferent account reps in a rotationuh and they would uh they would comeover and say yeah what do you need whatdo you need what were they doing in myoffice in graham's office in downtownbellevue we're not a top 100 customerthey can't that how does that even scalethey can't have enough people to go andsit on site with every single customernow you could make the argument oh wellgrabs kind of important because you knowthey're going to be the gateway tosoutheast asia and so on and so they'remasasan's investment they're big and youknow there's a lot of you know smoke andmirrors and you know it's it's all trueand it's going to come true and and grabis going to be dominant but it's neverbeen a foregone conclusion i mean uberwas competing with him and then nowgojek's competing with him and gojek hasa bunch of really big investors and it'snot clear-cut right you know thatthey're that they're gonna be big andwhy would you bet on a customer that'sgonna be big when you've already gotcustomers that are already bigand yet amazon had people sitting in ouroffices you know uh they offered we saidyesuh you know microsoft got into that andthey sent some people too and that wasthat was fine you know you know us tooumbut it was never really the sameso so i'm gonna i'm gonna close with awith a story about uh i'll close thisoff with a story about umthe conferences okay the developerconferences because those are sort of acustomer interaction sort of a way thattheycan demonstrate customer obsessionand it's kind of um it's not a directthing it's more of an indirect thing youknow and how successful the conferenceisbut you know it's a it's a it's a signaluh so the story is i was at my grab inmy ummy first yearit was 2018uh i joined just late the previous yearand uh my boss mark porterhe said hey steve yeah let's uh let's goto re inventreinvent is amazon's cloud conferenceokay it's about awsand it's in las vegas and you know i'min seattle and so it's only like a twoand a half hour flightand so it made sense you know for forfor me to go and represent uh you knowas a head of engineering and ads and allthat stuff uh but i didn't want to go uhyou know i like i don't like conferencesi don't know why i don't like them ijust don't like them like they'rethey're a waste of time they justthey're just like umi could go on and on about how howshallow they are but they're they'renothing gets done at a conferenceand they'rei don't see the point a lot of people dolike them they like they got their badgeand their lanyard and their packagesswag and they're like i'm in aconference and they feel important orsomething and people speaking atconferences feel important i've donethat too and then it was ultimately itwas like why did i do that what was whatwas the goal here right just buildingbrand recognition with developers iguessyou know fine fineit's fine that they have them and it'sfine that some people like them but ididn't want to go okay because i wasbusy like my job was very stressful andi'll talk about my job at grad and howworking with asia from the united statesis just in general in in another episodeand mark's like oh come on man you got ayear to come you got to come it's likeyou have toand he was very insistent and i'm likeokay fine you know fine i'll take thehit for the team and i'll go to lasvegas and bring my wife along and we'llupgrade our hotel room at our own on ourown dime and we'll try to make it a funtrip because conferences suck and idon't want to gobut we'll do some gamblingso we go to re invent which i've neverbeen to before in 2018anduh well i went and i learned umthere were basically three components ofthe conference that i want to compare tothe microsoft conference that i went toa couple of months laterthe uh the first one was the keynotespeechi want you to remember these things wheni talk about the microsoft ones uh thekeynote speech is you know by andy jassyi don't know if he still does it but atthe time you know for many years andyjassy would give the keynote and youknow what a keynote is right a keynoteis some self-important person standingup there and going well i'm really superglad that you all came and boy i'm suremaking a lot of money off this and let'shead up to the strip club you knowwhatever i don't keynote speeches umthis was not thatso i don't know if they're recorded manit would be really uh really interestingif you could just watch maybeuhandy jassy stood up on stage for liketwo and a half hours and this was inlike a vegas hotel like their conferencearea like i think it was the venetianand or the palazzo and they they likehad these giant rooms with giant screensand he was projected on the screens iwasn't in the room with andy jassybecause i showed up late because 65 000people showed up to the conference awhole football stadium filled withpeople a bunch of tech geeks all likewith this narrow-minded focus singlenessof purpose they came marching in therewas this buzz this excitement and i wasjust like you know this is new anddifferent it was like invasion of thebody snatchers to be honest and uh andeverybody got under the seats and therewas this electric you know buzz and andystarts talking he talks for like twohours two and a half hours okay it justgoes on and on and on and on and onand it wasn't you know hello folks gladto have you here no he stopped there andwent well folks here's our report cardand he just went through and like fortwo hours he just didstatements of the formyou guys asked for thiswe delivered ityou guys felt our performance wasn't upto snuff in this particular area here'sthe new numbersyou felt like our competitor had anoffering here that wasn't doing justiceto you know we didn't do justice in ourcloudwe answered okay oh and you know and hecould go on and on and on about thingsthat people had asked for and you knowwhat i as a customer of aws of amazonweb services i used a bunch of theirservices and i had things that i wantedand i saw some of them in thepresentation and i'm like wow i sawother ones that i knew were going to beuseful and i saw some i was like i don'teven know what i'm looking at it was sobig and complicated it would be so itwould take so long to learn all ofamazon's cloud i mean like i didn't evenhave time to do it allbut it was extraordinaryuh somebody told me mark told me thatandy you know goes to this like you knowgoes to his private retreat you knowlike in a movie and he like disappearsfor 10 days or two weeks or whatever andyou know psyches himself out for thetalk because it was a really impressiveperformance that he gave up on stagekind of from memory just go on featurefeature feature fix fix fix performancepricing all this stuffit was it was uh remarkable and it wasalllike getting just eaten up by theaudience because they were like yeahfinally finally right i can use thesespecial indexes you know and my redshiftwill talk to my dynamodb and blah blahblah they finally fixed it right everythere was there's excitement because itwas making people's lives easier rightthey were amazon was takingload off of people's shoulders andshouldering it themselvesso that was the uh keynotethe second dimension was uh the thevendors which was just overwhelmingright they were they were spread overmultiple hotels and you can go anywhereand everybody who's anybody who offers aservice offers it on aws and soeverybody was thereand then finally there were the uh themeetings okay the meetings with with theteams which i didn't know we were gonnahave but somebody at amazon set up somemeetings for us i was like oh okay we'regonna have some uh some meetings umwho are we talking toand mark's like you know the teamsyou're scheduled to talk to and he namedoff a bunch of teams that were teamsthat uh owned the services they builtand launched and ran the services that iwas using on my teams and i'm like okayyeah so like the account reps she's likenono no i mean the teams and so i went inthere and like i was like oh you know iknew some of these people i mean likesome of them have been in amazon foryears and years i mean like since i wasthere in 2005and it was like the engineering leadsand it was the program managers andproduct managers and general managersthe gm's and that all the actual peoplenot some proxies not some you know likeyou know ambassadors and they and theyand they were there going yo hey steveyou know what do you need what can we dofor you and this is where the humilitythat i mentioned earlier came in becausei was overwhelmed that they were takingtheir time out of their day becausefirst of all running aws is no joke andsecond of all they're all richlike rich as crisis i mean theythey all held their shares and theystarted in 2005 and the stock ran upfrom like forty dollars to threethousand three hundred dollars per shareduring the time that i had leftand you know what what possible businesscould they have sitting there asking mewhat my team needed because i was withgrab not netflix right and yet they gaveeach one of them gave me half an hour or40 minutes or whatever and they were youknowthey were seriouslike you know i'd say well we couldi mean i'll go on a limb here we couldreally use this and i'd give them someyou know some request you know andthey'd look at each other and they wouldthey would not you know and sometimesthey'd tell me that it was on the roadmap or sometimes they'd take reallydetailed notes and say okay this is andhow would you rank that relativity theseother features and they would like talkabout itand i realized that i was dictatingtheir freaking road map for them okayit doesn't work like that at google imean let me tell you when i was ingoogle cloud for five years how howstuff worked in google cloud people ingoogle would go well we solved thisproblem at google because google'spretty much solved all computingproblems and we did it our way and ourway is the best way and i know the restof the industry does it differently butthey're doing it wrong and what we'regoing to do is we're going to get athink tank together of all of our bestand brightest engineers who builtgoogle's cloud google's coreinfrastructureand we're going to have them build thecloud versionthey're going to go into a room andthey're going to close the doors and getsome snacks and seal themselves offand they're going to from from scratchfrom first principles they're going toinvent what the right solution is forthese customers out here that are usingthe wrong thing on awsand uh and they're when we're we'regoing to roll it out to great fanfareand they're gonna use it you'll see ifwe build it they'll come and and they'regonna like itthat's that's how google cloud operatesit's just nuts you can't talk to acustomer likemy boss okay got promoted uh in part youknow for her innovation and it reallywas an innovation which is the saddestthing ever of talking to customers shewould she would go out and findcustomers of the services that ourorganization built and uh you know theywere always small companies like justlikethey were on google cloud just for costand maybe no other reason or maybe theywere ex-googlers right so they had alittle bit of loyalty and there's noreal reason to be on google's cloud overamazon's cloudunless you already know google's cloudand so and it's because of this problemwhere google designs things differentfrom the rest of the industry and thenyou have to like you have to look at itand go well okay i can learn how to doit your way and then i'll be completelylocked into your cloud and i can'tmigrate because amazon does things theopen way rightand so uh you know she got promotedbecause she would like once a year uhbring in fiveyou know volunteer corporate customersand put him up in director's chairs onthe stage and our whole org wouldshuffle in and we'd all sit there forlike an hour hour and a half howeverlong he can go without having to go tothe bathroom and we would get to askthese people in the director's chairsyou know so what do you think of ourstuff and one of them they'd look ateach other and one of them would saywell you know we kind of think it's okaybut boy it really could be better in thefollowing 57 dimensions and we'd alllook at each other just like the amazonfolks the amazon teams did we look ateach other except we were sayingthat's never going to get fixed we knewit was never going to get fixed becausewe couldn't do anything about it becauseof blockers i'm going to talk inspecific gory detail about the kinds ofblockers that exist in google cloud andexist in other places all overi've got buddies people have left googlefriends who are telling me the samehorror stories and other companies it'severywhere but we knew it wasn't goingto get fixed because you can't fix stuffbecause google cloud has an executionproblem because they don't put thecustomer first it was innovative that weeven brought them into the building inthe first place once a year compared tograb go to ground every day and aws havea person in their officean engineer sitting there in theiroffice taking notes begging please tellus what we can do better so night andday rightso now let's now let's talk aboutmicrosoft's build conference and then wewill uh we'll wrapuh i mean i really could talk about thisstuff in much more detail umbut you know time constraintswe can revisit if it's a popular topicum a couple of months after re inventwhich re invent was just mind blowingit's across the board i was just likei couldn't believe what i was seeing uhthe first useful conference ever so iwent to microsoft bill and i was likehey maybe all conferences are good thesedaysand microsoft build was in thewashington state convention center and ialso had to go to that because it's likefive minutes from my houseand so of course it made sense and wewere using some microsoft uh stuffoff of their cloud and by the way justyou know up front hey um you know allthe folks i worked with at microsoft inin azure the cloud they were reallysmart and they were cool and they'reprofessional and they were humble andthey were nice and they were umvery accommodating they were they weregreat hosts i hope that we were goodguests uh and uh you know don't want todisparage anybody in microsoft cloudbecause they're they're working you knowwellumthey're not amazonand they know it right and you know theythey know they're so far behind thatit's it's almost comically badum but but you know they're tryingreally hard uh so but i went to theconference uh because that was thosethose were meetings that i had like onyou know in redmond in on microsoftcampusuh but the conference the buildconference is this big vague fuzzydirectionless pointless kind ofparade sort of a fair ground wherebasically a bunch of random people showup and they give random demos they'retrying to i don't it's it's back to theold conferences that i hate kind ofmodel where they're giving just randomdemos that are not useful to you so it'slike uh you know it's like when you goto disneyland with your family and youdecide that maybe just like you're goingto spend a few minutes in you know inwalt disney's imaginarium where you cansee his vision of what the 1950s thoughtthe future was going to be like and whatyou know a a modern home might be likeyou know andyou know you go in there and it'sair-conditioned and you're like okayi'll hang out for a while but it's notfun and then you go back to the ridesrightthat's kind of what microsoft build waslike their their expo was just a bunchof random people most of them withmacbooks uh giving demos of random stuffthat they did it wasn't cloud focused itwas a hodgepodge a mishmash now satyanadella you know who's you know hereally is brilliant and he's you knowhe's an amazing leader he's kind of amagicianuh you knowuh he came from microsoft cloud and heknows microsoft cloud is their businessit's their future like nothing elsematters at this point windows in officeyou know and they'll they'll drag onmaybe for another 10 years you know andthey can eat some more revenue out of itbut it's kind of over and they know itand they're not innovating anywhere elsethey're trying and they're failing it'scloud that's their only shot at stayingalive oracle's actually in even worseposition but in the same position rightthey just they just have it worseoracle's dying quickly and they knowthey need to get in the cloud and theycan't because they have a terribleexecution problemso so we go to build and satya gives hiskeynote remember i told you about thethree components of the conference andhow they were all amazingand then let's talk about how theyhappened at microsoftso i went to the keynoteand i was excited because andy jassy'skeynote was just mind-blowing okay and iwas like i want to see what microsofthas launched in the past year let's seelet's compare their progressand satya nadella stood up there and hegave a keynoteuh and the keynote was pretty short andit was about howthey were proud to announcethat microsoft had partnered withstarbucksand that the starbucks was usingmicrosoft machine learning algorithmsto be able to tell you where yourespresso beans came fromand that was the keynoteand it was quite franklyembarrassing to be there it wasembarrassing to watch i was embarrassedfor them i was embarrassed for myself iwas embarrassed for everyone else in theaudienceyoui i meanwhat the helland uh so i was like okay well uh youknow that maybe the keynote isn't reallythe right way to measure the success ofa of a developer conference maybethey're maybe theythey fell a little short there unlessyou're a really really big espresso beanfan uh but you know uh maybe the otherareas and to be fair you know in onearea which is the team meetings where wegot to meet with the teams same thing wegot to meet with the teams they werereal nice to us they were competing forour business from awsthey wanted us i think as a umjust to show off just like they couldshow off starbucks right they didn'tcare how big we were we were justsoutheast asia but they were still verynice and we had good meetings with themand they you know they offered to go fixthings uh don't know how how muchprogress they made on those things so becuriousbut the the third pillar which i onlybriefly touched on right was the the thevendorbooth the floorwhich is kind of like you know when yougo to the state fair you go to a you goto a big event you know uhan art fair or a crafts fair orsomething where there's like there's abunch of space where people can can payto have a boothright like a little tent a pavilion orwhatever and you know like buy toseattle you can go get food and ofcourse the measure the yardstick of likehow successful your event is and andwhether your venue was worth the moneyis how many vendors showed upand then you know in turn that'll driveyou know people showing upso my buddy uh my buddy who's uh nowebay and i hope to get him on the showtoohe says uh he's with me there at themicrosoft build conference he goes yeahlet's go check out the vendor pavilionbecause you know what that's how youtell how successful a cloud is or anyplatform for that matter he was right hesays you know it's a you can see ifthere you know if there's a if there's alot of let's say there's a lot ofdatabase companies there okay andthey're all saying hey use our databaseon microsoft azurewhat that means is that azure doesn'thave good enough databasesbecause there's a bunch of third-partyvendors going hey pay us a bunch ofextra money on top of your regular cloudbill for our better database than theone that microsoft has and there's allkinds of different you knowuh parts of a cloud that couldpotentially have third-party offeringsthat compete with the core cloud andamazon of course had all of them and sowe went to microsoft to see what holesthey had what was missing from theirlineup and we went in there and it wasjust like it was like the old west thetumbleweed rolls by and we were a couplegunslingers and the town was completelyempty and we were likeyou know like we could hear thethe spaghetti western music playing itwas empty there was nobody there i'm nottalking about people yeah there were nopeople there just people random peoplewandering around it looked like acarnival but there were nocompanies there there were no offendersthere were like four like total like iknew some of them personally they'relike oh hey steve i'm like oh yeah heywhat are you doing here and they're likeyou know because there was there was nopoint because there's like there's amarketplace at work here andmarketplaces require buyers and sellersand you have to have a lot of buyers andyou have to have a lot of sellers youcan't have one or the other with thatimbalancehalf of them will evaporateand sure enough i don't know if it'sbecause microsoft cloud doesn't haveenough customers or because microsoftcloud's really bad because they're notenough vendors but for one reason oranother all of the booths at thepavilion were filled with microsoftteams it was like whoever planned it waslike oh my god we didn't get anybody tosign up for this thing so get your teamget in there and just pretend to be athird party sort of right and so they'rewaving like hi you know we're we're amicrosoft team that you know is here inthe vendor floor and it was 80microsoft and i we were floored we werelike there's not a hole in their cloudtheir cloud is a big gaping hole thewhole thingand you know so we left you know verydisappointed the whole thing i meanhonestly if they they did more branddamage you know certainly to me as justa developer who uses cloud i usegoogle's cloud but it doesn't matter umyou knowthe whole point of a dev conference asfar as i know is to build you know brandexcitement and enthusiasm for the fortheir for their brand for the microsoftbrand the azure brandthey did more damage to that by havingthat conference that that that i gavefeedback on and i'm sure they didn'tread it uhthen then if they hadn't had theconference at all i mean seriously theyshould they should just pass why wasn'teven why wasn't it about cloud if cloudis going to be their business goingforward their only hope right obi-wantheir only hope is cloud and yet theyhad all this other stuffright and i'm sure part of this isbecause of the disease that we're goingto talk about which is that a lot ofteams get big and they have a lot ofsway when they shouldn'tso anyway this has been our episodeabout corporate customers to recapi became a corporate customer of a cloudat afairly medium-sized company grab for thefirst time in 2018 and i continued thatrelationship with amazon through untilthe pandemic started uh i was blown awayby how umcustomer obsessedi mean it's such a cliched term and yeti was blown away that they were sittingthere asking me whati wanted their roadmap to be and and butyou know what i wasn't surprised becausethat's exactly how they treat theirretail customers right that you're theretail customers in aggregate are votingfor what's next for amazonand uhand once you reach that point i'll tellyou when you're done because amazon'slike done in a sense okaydone for the day saythe the the guy sitting you know or thegal sitting at the table near my deskfrom amazon who i could ask questions today in day out month in month out rightthe entire time we were aws customersthey were doneand they had done their job and theycould feel goodwhen we saidwe're goodthank you you've doneso much great stuff for us and we reallycan't think of anything else that weneed at this timenot even speculative we just we're goodthat's when you know you're done andthat's when you break throughinto this new you know green field youknow sort of innovation space where youcan say wow our customers literally haveeverything their heart could desire thatthey know about that they can think ofuh let's start thinking of things forthemwhich is what google started withremember google's like i'm telling youi'm telling you there were google teamsthat would go in and design somethingand then a product manager from anothercompany there's a really good pm frommicrosoft and she came in and she waslike she wasjust rocking in google cloud and she gotattached to this one projectand she came to me and she said theyjust went into a room and designedsomething that's completely unusable andnobody would want it and they didn't askanyone and she had to she had to go inand she had to slap him around and sayyou need to start over again and thistime make sure there's a customer in theroom with youand they were like huhyou know because google it's all aboutalso the customeralso we do customer we talk to customerssometimes tooit's night and day 

Watch Steve Yegge's podcast https://www.youtube.com/watch?v=9v4z46Ea35Qacompany is like a bodyit's not like a person like a humanbeing it's like a thing it's an entitythat has its own agenda and its owngoals and its own control of resourcesand its own value systemand uhthe individual members of the companykinda don't matter as long as they'redoing their joband the company cares about them rightthe way you care about your heart andyour lungs but if you had a chance toreplace them with a better heart andlungs you would and that's the waycompanies operate too a company you knowsort of maintains its own healthuh uh or asks for government handoutsthose are the sort of two optionsand um and so to understand you know andthe original people who started thecompany sure when it's small and it'sjust a small group of people it's just agroup of people but when it grows to acertain size everybody becomesreplaceableokayand this is important to understandingwhy amazon is so dominant across the theboard okay in everything that they doit's it's really crazy soso what happens is umgroups can get diseasesand sometimes we call it dysfunction butit's it's really a disease it's anailment right uh you know to give you areally simple example you might have onefamily member who's uh a real problemsomebody who's in and out of jail andalways you know uh getting in troubleyou know with the law or always stirringup trouble at family gatherings or justgenerally a problem rightyou can have those in companies tooright maybe not getting in and out ofjail they won't last long at the companymost likely unless they're the ceobut you have people that are creatingproblems okayuhand uhso that's not really a disease so muchas like a wound you know like apulled muscle you know or a sore that'shaving trouble healingssdsdbut it's still a problem an illness anailment with the companybecause it's preventing other peoplefrom getting stuff doneif you have a whole bunch of those allover your body then it's a diseaseif you have a whole bunch of people inyour company who are holding on tokeeping other people from beingproductivein any way there's lots of differentways they can do this then your companyis diseased a great example of this ismicrosoft and we'll go into great detailabout this uh down the road in anotheranother episodeumit's a really common pattern there arethere are there are companies have awhole host of diseases that they can getand they're common like many companieswill have the same diseaseand the diseases could potentiallythere's a taxonomy you could name themand you could uhyou know learn how to diagnose them andlearn what the symptoms are and learnhow to treat them and learn which onesare fatali mean like nobody's done this you knowi'm going to start talking about them inmy show you can call me dr steveuh you you know it's really kind ofadvanced to the state of maybe veteranyou know horse medicine at this pointlook at a company and just like shoot itbut um you know the the reality is thatuh companies you know they get their owndiseases just like populations getdiseases they can get real diseases orthey can get diseases like beinganti-facts now i'm not blaminganti-vaxxers if you're anti-vaxx uh youknow don't angrily turn off my show youknow i'm not blaming you for beinganti-vaxxed it's really a failure of theeducation system and of uh science uhmarketing and of the government and abunch of other reasons uh that thatbecause it's a very real phenomenon imeanthere you know some 30 40 of the entireworld's population maybe is isfirmly anti-vaxuh but it is a disease in in aggregatebecause it's killing people i meanthat's kind of the definition of adiseaseand so you know how does this happen imean diseases can be diseases of themind in a sense and companies they donot have the willto cure their diseases i mean if you ifyou're like you're talking about the oldwest and you know you you get you knowan arrow to your to your knee and youhave uh you know uh an infection and youknow you're looking at it and it startsto gangrene and the docdoc you know who's your buddy who youknow drinks you know as much whiskey asyou says man we're gonna have to takethat offokayand so saw you know sawing your leg offto save the body to save your lifei mean it happens today still right it'svery painful and traumaticand breaking up a companycan be very painful and traumatic orrooting out a systemic illness from acompany because companies are made ofpeopleand even if companies don't reallymatter people do you know and uh and andthere's also a lot of like legalobstacles to companies just snuffingthings out we do have at will employmentwhich means they can fire you anytimethey wantat least in the united states and thatisabsolutely huge for productivity i'm noti'm not uh trying to justify ituh and you know in europe they protectpeople's rights workers rights more thanthey do in the united states or in asiabut in the us and asia which are farmore productive than europein the tech sector uh you can firepeople at will and it's that constantthreat of being fired that keeps peoplesort of behaving keeps the the lungs andthe circulatory system and everythinglike workinguh because people know that they'rebeing held accountable right for beingyou know for not not diseaseduh but diseases do happen and you knowamazon i'm i'm gonna just say it rightnowthe number one reason that amazonexecutes so well is that they aremerciless about rooting out disease assoon as they find itand i told you i'm going to talk about alot of different specific diseases thati've seen in action at corporations oreven been a part of okay it's a learningexperience for all of usuh and uh you know we'll talk at greatlength about them but basically there'ssituations wheregroups of people within the company canhold the company hostagethis happens all the time a specificgroup of people becomes large enough tobecome sort of like a a political lobbyor a labor union or you know some sortof you knowa sub entity within the company that hasits own agenda and it's starting tofight the host rightand uh and it it's just to give you anexample so that i'm not you know you'renot guessingbeing territorialis a huge diseasein companies a very common one beingturfy that's mine you can't work on thati'm not working on it and you can't workon it eitherso the company's trying to get as muchdone as humanly possible like literallyas humanly possibleand there are humans that are holdingthem back from getting stuff done thatthe company wants to get done happensagain and again and again it happens atall companiesexcept for amazon okay where amazon hasa core value called bias for actionand what it really means is greasy spotson chairs okaythere was a team that i uh witnessed ihad to work with them i had the sadmisfortune of working with a small teamofself uhself-proclaimed gatekeepersof the website uh at amazon in like 1998and 1999. mostly 99.it was a group of people who uh decidedthat in order to protectthe websiteto keep the website from uh you knowpushing out some some garbage you knowthat's broken that hasn't beenadequately tested that hasn't beenadequately vetted that hasn't metwhatever criteria they felt like comingup with okaythat they weren't going to allow it tolaunch so they actually it turned outthey had some keys okay they had thekeys to actually flip the switch to uhto move a build and a new set offeatures onto the to the live websiteproduction servers they were they werethe gatekeepers right there of thelaunched production and if they didn'tif they didn't uh turn turn their keyand and allow the thing to happen youcouldn't launchyou didn't have that abilityand uhthis got like gradually more and moreannoying because gatekeepers will justsort of like uh root themselves in andsettle down and dig in like like likefunnel web spiders and then they'll justcome by and just eat random peopleuh and and stuff won't get done and andwhat happened was jeff basil it finallygot to where projects weren't launchingand jeff bezos was like so why didn'tthis launch on time and they're likewell you know issues with the therelease train bubble what release trainwell you know there's a group ofwhat group right and then the next dayinstead ofthe group that was the gatekeepers theirchairs just had greasy spots where abolt of lightning had destroyed themutterlyin a biblical justwhoa where's bob and sally my my godwhat happenedand then there was a partybecause everybody hated them becausethey were gatekeepers because they wereslowing everybody down and amazon didn'tcare actually it turns out if theylaunched garbage to the website becausethey could just make people mad and thenthey'd call customer service and they'dget a gift certificate and they get abig apology and they get free stuff andthey go i love amazon even though theyshipped me the wrong thing and like i'ma catholic and they sent me pornographyand etc etc every bad thing that couldever happen has happened at amazon i wasin customer service for a couple yearswith their toolsand i tell youuhamazonuhdoes not toleratebeing slowed downit's a disease all rightuhwhat does that mean that means they havethe willpowerto uh to root it outwhich means you know have a stern talkwith the people you know who areresponsible uh and if they say no whichthey often do they say no you can't thisis mine and i'm keeping the company safeit's always in the guise of safetysecurity whatever right it's see youknow umuhone of the diseases that we have is thatthe so-called devops team and we'll talka whole episode about this you know thegroup that holds the keys to launchinguh decides that they're gonna they'regonna be an obstacleand most companies tolerate it they'relike oh you know you're just gonna haveto negotiate with them well i don'treally have the authority to blah blahmeanwhile inin amazon it looks like florida duringlightning seasonwhole teams will just disappearbecause they didn'tthey they screwed around they didn'tlaunchuhamazon has a systemuh of what you might call ambassadors ordiplomats okay they're called technicalprogram managers at amazon tpmsuh but what they really are is theirambassadors and diplomatsbecause the the companies get so bigthat they become like countries andthere's these different you know verypowerful warlords in charge of thedifferent um you know like google hasyou know gmail and you knowyou can't mess with us we got how manyyou know gazillion customers and youknow and then there's you know youtubeand you can't mess with them and and soit's really amazing actually that googlemanages to get everything anything doneuh given that they're so you knowbalkanized uhand you know amazon's the same wayexcept amazon does get stuff doneand and the reason is that the localwarlords have to deal with theseambassadors that they're not allowed toshoot called technical program managerswho come in and basically the technicalprogram manager is an engineer who alsoknows how to speak warlord and they comein and they say we need to get xyzlaunchedand the conversation always goes likethis the manager goes[Laughter]noand the tpm says or you'll be firedand the manager goeshow can i helprightthat's how the conversation goes atamazonhow the conversation goes at uh you knowgoogle and microsoft and everywhere elseis they go ha ha no and that's actuallythe end of the conversation there'sgonna be a lot of follow-upconversations and a lot of begging andwhining and angry escalating and fistpounding and gnashing of teeth andpulling of hair and name calling andwell i've been through all of thatand the answer is still nothat company has a disease and thediseaseis bias for inactionthe opposite of amazon's bias for actionthey have a bias for inactionand i tell you inaction doesn't get acompany very far

Listen to Stevey's Tech Talk (10mins in) https://www.youtube.com/watch?v=jUtUAc_ew9Y Playstation ad talked about in the clip: https://www.youtube.com/watch?v=kWSIFh8ICaA https://www.forbes.com/sites/insertcoin/2013/06/11/playstation-4s-price-and-policies-humiliate-microsofts-xbox-one-at-e3/?sh=377dd8aa133f https://en.wikipedia.org/wiki/Don_Mattrick

Oxide and Friends Twitter Space: July 26, 2021Agile + 20We've been holding a Twitter Space weekly on Mondays at 5p for about an hour. Even though it's not (yet?) a feature of Twitter Spaces, we have been recording them all; here is the recording for our Twitter Space for July 26, 2021.In addition to Bryan Cantrill and Adam Leventhal, speakers on July 26 included Tom Lyon, Tom Killalea, Dan Cross, Aaron Goldman, and others. (Did we miss your name and/or get it wrong? Drop a PR!)Some of the topics we hit on, in the order that we hit them: Al Tenhundfeld's Agile at 20: The Failed Rebellion The Agile Manifesto [@0:55](https://youtu.be/3tp5EtPdPwY?t=55) Adam's experiences From the Agile Manifesto history > The only concern with the term agile came from Martin Fowler > (a Brit for those who don't know him) who allowed that > most Americans didn't know how to pronounce the word ‘agile'. [@6:25](https://youtu.be/3tp5EtPdPwY?t=385) > The problem with agile is when it became so prescriptive that it > lost a lot of its agility. [@8:06](https://youtu.be/3tp5EtPdPwY?t=486) > There's so much that is unstructured in the way we develop software, > that we are constantly seeking people to tell us how to do it. > The answer is it's complicated. Steve Yegge's Good Agile, Bad Agile > So the consultants, now having lost their primary customer, were at > a bar one day, and one of them (named L. Ron Hubbard) said: > “This nickel-a-line-of-code gig is lame. You know where > the real money is at? You start your own religion.” > And that's how both Extreme Programming and Scientology were born. [@9:15](https://youtu.be/3tp5EtPdPwY?t=555) Edward Yourdon“Decline and Fall of the American Programmer” book [@10:26](https://youtu.be/3tp5EtPdPwY?t=626) “The principles are not all wrong. Some today even feel obvious.” > There's also a lack of specificity, which gives one lots of opportunity > for faith healers to come in. [@14:43](https://youtu.be/3tp5EtPdPwY?t=883) “Something I found surprising about Agile was how rigid it became.”  Dan's perils of personal tracking methodology Sun's engineers connecting directly with customers The Agile Ceremonies. (an ultimate guide) Sprint Planning, Daily Stand-Up, Sprint Review, Sprint Retrospective [@20:48](https://youtu.be/3tp5EtPdPwY?t=1248) “I think we overly enshrine schedule estimation. If there are any unknowns it becomes really hard.” > I think there's a Heisenberg principle at work with software: > you can tell what's in a release or when it ships, but not both. [@23:25](https://youtu.be/3tp5EtPdPwY?t=1405) Tom Killalea talks to success stories he's seen with Agile Building S3 at AWS [@28:31](https://youtu.be/3tp5EtPdPwY?t=1711) Sprint planning and backlogs Big work chunks, responding to changing priorities [@33:39](https://youtu.be/3tp5EtPdPwY?t=2019) Success or failure of an Agile team?  “Do demos and retrospectives” Unknowns in software development make estimation hard [@39:11](https://youtu.be/3tp5EtPdPwY?t=2351) Dan's experiences  Personal Software Process, Team software process, Software Engineering Institute > Some people really benefit from the level of rigidity that is set out > by these processes. Prior to that, they just weren't having > these conversations with their sales team, product owners, etc. Construction analogies, repeatability. Self-anchored suspension bridge [@46:40](https://youtu.be/3tp5EtPdPwY?t=2800) Software as both information and machine.  Consultancies, repeatability, incremental results. “For each success story, there are many failures.” Manifesto as a compromise between different methodologies Silver Bullet solutions, cure-alls. See Fred Brooks' (1987) “No Silver Bullet” paper [@51:18](https://youtu.be/3tp5EtPdPwY?t=3078) Demos: “Working software is the primary measure of progress.”  Experimentation and iteration No true Scotsman fallacy What does Agile even mean anymore? “Letting people pretend to agree while actually disagreeing, but then going off and building working software anyway.” [@59:45](https://youtu.be/3tp5EtPdPwY?t=3585) Ed Yourdon and the Y2K problem Maybe there are too many Agile books already. Tom Killalea conversation with Werner Vogels AWS development Agile is more like a guideline than a target to hit. Consistent team composition over time “Soul of a New Machine”: trust is risk The answer can't be “you're doing it wrong.” How do you know if it's working for your team? (Did we miss anything? PRs always welcome!)If we got something wrong or missed something, please file a PR! Our next Twitter space will likely be on Monday at 5p Pacific Time; stay tuned to our Twitter feeds for details. We'd love to have you join us, as we always love to hear from new speakers!

We chat about an old 2008 blog post by Steve Yegge about 'prototypal inheritance' and the prototype design pattern. We also discussed Datasette (datasette.io), a project that makes it easy for people to share data. Also Denny is embarking on a new project to make Hawaiian amaro. This week Cindy spent 14 hours in Jelly, and Denny spent 4 hours.

The Agenda

This is The Founders' List - audio versions of essays from technology’s most important leaders, selected by the founder community. Longtime Googler Steve Yegge posted an insightful rant on his Google+ page about how Google is failing to make platforms for its products. He also shares some interesting little tidbits about his six-year stint at Amazon working for the 'Dread Pirate Bezos'. The rant was intended to be shared only with his Google coworkers, but was accidentally made public. Read the full memo here - https://gist.github.com/chitchcock/1281611

The one after platforms where Steve Yegge shares Amazon war stories - steve-yegge-platform-rant-follow-up.md https://gist.github.com/kislayverma/6681d4cce736cd7041e6c8214469d2fd All gistsBack to GitHub Sign in Sign up Sign in Sign up steve-yegge-platform-rant-follow-up.md Star 3 Fork Star Code 1 3 Learn more about clone URLs Download ZIPSign up for freeSign in to commentPrivacy StatementLearn moreLearn more

On The Cloud Pod this week, your hosts introduce the idea of plaques to commemorate a feature suggestion becoming a product. A big thanks to this week's sponsors:     Foghorn Consulting, which provides full-stack cloud solutions with a focus on strategy, planning and execution for enterprises seeking to take advantage of the transformative capabilities of AWS, Google Cloud and Azure. When the girls get coding!. Join us on your screens, Oct 13, for the live@Manning “Women in Tech” conference to celebrate the rising movement of women in technology. http://mng.bz/MolW  This week's highlights Active Directory just will not die. Someone is excited about Google's Data Fusion pipelines. We just don't know them. Azure gets features that AWS and Google already have. General: Did You Do Your Homework? Former Google engineer Steve Yegge resurrects his blog to explain why Google's deprecation policy is killing user adoption. We're still bitter about Google Reader. The Cloud Pod is sponsoring the Rust Conference and Women in Tech conference. We're super excited about both of these conferences and supporting more women in the technical world.  Amazon Web Services: So confused

Dominic's back from the beach, where luckily there were no clouds. The question of the week is, can enterprises also avoid the cloud — and should they? We talked about cloud repatriation, and mentioned Lydia Leong's piece on the topic (https://cloudpundit.com/2020/07/27/hunting-the-dread-gazebo-of-repatriation/). Dominic also shamelessly plugged a piece of his own on multi-cloud in 2020 (https://diginomica.com/three-reasons-why-multi-cloud-back-and-here-stay), as well as Charity Major's piece on the future of Ops jobs (https://acloudguru.com/blog/engineering/the-future-of-ops-jobs). We didn't get around to mentioning Steve Yegge's piece on Google Cloud and how its deprecation policy makes it hard to love (https://medium.com/@steve.yegge/dear-google-cloud-your-deprecation-policy-is-killing-you-ee7525dc05dc). Recommendations: Meeter app: https://apps.apple.com/app/meeter-for-zoom-teams-co/id1510445899 Decathlon Forclaz solar panel: https://www.decathlon.it/pannello-solare-trek100-10w-id_8485459.html Autonomous AI Zen Work Pod: https://www.autonomous.ai/zen-work-pod Follow us on Twitter (https://twitter.com/Roll4Enterprise) and on LinkedIn (https://www.linkedin.com/company/roll-for-enterprise/)!

Join Pete Cheslock and me as we continue the Whiteboard Confessional series with a look at a recent article written by Steve Yegge about Google’s deprecation policy. We discuss the method behind Google’s madness and how AWS is pretty much the exact opposite with respect to deprecation, how Google assumes all engineers write code like Google engineers, what might happen if AWS deprecated something like Lambda or DynamoDB, how breaking backwards compatibility is a great way to crush your user base, how Microsoft might have many flaws but support is not one of them, and more.

On this week's episode of the freeCodeCamp podcast, Quincy interviews Shawn Wang (@swyx). We talk about "learning in public" and his transition into tech from finance, where he left behind a job that paid him US $350,000 per year. Shawn grew up in Singapore and came to the US as a college student. He worked in finance, but at age 30, he burned out. So he decided to learn to code. He used freeCodeCamp and a ton of other resources, and since then he's worked as a freelance developer, and at several companies including Netlify. Follow Shawn on Twitter: https://twitter.com/swyx Follow Quincy on Twitter: https://twitter.com/ossia Here are some links we discuss in the interview. Shawn's Projects: The official React subreddit that Shawn moderates: https://reddit.com/r/reactjs Shawn's article on No Zero Days: https://www.freecodecamp.org/forum/t/no-zero-days-my-roadmap-from-javascript-noob-to-full-stack-developer-in-12-months/164514 Job Search / Salary Negotation articles: Cracking the Coding Interview: https://fcc.im/2UihbNm Hasseeb Qureshi's story of getting a $250K/y developer job at Airbnb: https://haseebq.com/farewell-app-academy-hello-airbnb-part-i Steve Yegge's "Get that job at Google" essay: http://steve-yegge.blogspot.com/2008/03/get-that-job-at-google.html Patrick McKenzie on Salary Negotiation https://www.kalzumeus.com/2012/01/23/salary-negotiation/ Quincy's recommended article: I spent 3 months applying to jobs after a coding bootcamp. Here's what I learned: https://medium.freecodecamp.org/9a07468d2331 Algorithm Expert: https://www.algoexpert.io Full Stack Academy https://www.fullstackacademy.com Shawn's Learn In Public movement: Shawn's Learn In Public essay https://gist.github.com/sw-yx/9720bd4a30606ca3ffb8d407113c0fe5‌‌ Kent C Dodds' Zero to 60 in Software Development: How to Jumpstart Your Career https://www.youtube.com/watch?v=-qPh6I2hfjw&app=desktop‌‌ Cory House on Becoming an Outlier: https://vimeo.com/97415346‌‌ Brad Frost on Creative Exhaust: http://bradfrost.com/blog/post/creative-exhaust/‌‌ Patrick McKenzie on the origin of the word "friendcatcher": https://news.ycombinator.com/item?id=511089‌‌ Chris Coyier on "Working In Public": https://chriscoyier.net/2012/09/23/working-in-public/ Links to other things we discuss: Shawn's Software Engineering Daily Interview with Sacha Greif: https://softwareengineeringdaily.com/2017/08/09/state-of-javascript-with-sacha-greif/‌‌ The origin of No Zero Days: https://www.reddit.com/r/getdisciplined/comments/1q96b5/i_just_dont_care_about_myself/cdah4af/‌‌ John Resig, creator of jQuery, telling his team to rip out jQuery: http://bikeshed.fm/180 ‌‌Jeff Bezos' Two Pizza Team rule: https://buffer.com/resources/small-teams-why-startups-often-win-against-google-and-facebook-the-science-behind-why-smaller-teams-get-more-done‌‌ Shawn's "You can learn so much on the internet for the low, low price of your ego" quote draws from Paul Graham's Keep Your Identity Small: http://paulgraham.com/identity.html‌‌ Shawn's Impostor Syndrome Bootcamp Podcast: https://player.fm/series/impostor-syndrome‌‌ TypeScript's growth via npm surveys: https://mobile.twitter.com/seldo/status/1088240877107965953

We re-examine the drama around Tesla, and discuss its similarities and differences with Amazon. We also look at ARK Invest’s Catherine Wood’s open letter to Elon Musk asking him to keep the company public. We cover why we think the Bitcoin ETF rejections are a good thing, and the additional activity we are seeing on our alerting platform. Topics: Some thoughts on planning software projects up front, deploying early, and handling third party integrations early Is "deploy early, deploy often" a thing? Tesla drama returns Describing AWS and how AWS benefits its users Catherine Wood's open letter of why Tesla should remain public Bitcoin ETFs rejected Going through SEC logic Interesting alerts - Seeing multiple chats calling for no price discussion - VeChain token swap - NANO launches charges, first prototype of IoT economy - Lisk Mainnet launch imminent - GitHub activity from Raiden Links: QuantLayer Crypto Podcast #10  Article by Steve Yegge at Amazon How the S.E.C May Pursue a Case Against Elon Musk and Tesla ARK Invest Weekly Newsletter  An Open Letter Against Taking Tesla Private  Elon Musk blog: Staying Public  ProShares GraniteShares Direxion Great thread by Jake Chervisnky Hester Pierce Caitlin Long Two Things That Don't Mix Well: Bitcoin Rehypothecation And Chain Forks Andreas Antonopolous - Bitcoin Q&A: Why I'm Against ETFs Andreas Antonopolous - YouTube Channel  Andreas Antonopolous - Twitter VeChain token swap NANO Launches charges, first prototype of IoT economy Nano Center - Nano initiated charger prototype Link Mainnet launch imminent GitHub activity from Raiden

Come Monday, we’ll see what full-on “digital transformation” looks like when Amazon fully owns Whole Foods. Also, Oracle is looking to move JEE to a foundation, closing out a long era of Java stewardship: how will “open source” like this work in a mature market? We also discuss the trend of private equity buying tech firms and GitHib’s write-up of building their own platform with kubernetes and series of small bash scripts. Traveling to China Coté is a terrible work-trip tourist. AA 263, DFW to PEK (https://www.seatguru.com/airlines/American_Airlines/American_Airlines_Boeing_787-8.php?flightno=263&date=), seat 19K. Exit row seat is good, but the front part of the airplane looked good too (rows 8 to 13?). Pack some breakfast tacos. This VPN situation is a mess, rather, I didn’t prepare correctly. Sometimes Cloak works, sometimes it doesn’t. LTE seems better than hotel wifi, but the speeds are high. Amazon Whole Foods update All done on Monday (http://phx.corporate-ir.net/phoenix.zhtml?c=176060&p=irol-newsArticle&ID=2295514), August 28th. See (https://mobile.nytimes.com/2017/08/24/technology/whole-foods-amazon-lower-prices-prime.html?partner=rss&emc=rss&_r=0&referer=)NY Times (https://mobile.nytimes.com/2017/08/24/technology/whole-foods-amazon-lower-prices-prime.html?partner=rss&emc=rss&_r=0&referer=) article (https://mobile.nytimes.com/2017/08/24/technology/whole-foods-amazon-lower-prices-prime.html?partner=rss&emc=rss&_r=0&referer=) as well. John Mackey (http://one.npr.org/?sharedMediaId=527979061:528000104) interview. Cheaper private label (I think they were top three or five sold in US). Return items in Amazon lockers. Cheaper groceries is cool, but for us, the interesting/instructive things to watch will be how Whole Foods goes full on digital transformation (or, even more eyebrow raising, does not!). Will they move everything to AWS? true Omni-channel and digital madness. Alexa: ”You look fat in that t-shirt, Michael, would you like me to order you some organic kale smoothies from Whole Foods?” Also, the potential for a culture clash seems high. As a side-effect, expect grocers to be trying out new computer stuff more, and observe their experience. How will the razor thin margin set cope with Amazon who’s been consistently rewarded for loosing money? Walmart and Google Hub thing (https://www.digitaltrends.com/home/google-walmart-voice-ordering/), Andrew on the AI winter (https://twitter.com/littleidea/status/900577868383637504). The Undying J(2)EE Oracle looking to open source it (https://adtmag.com/articles/2017/08/17/java-open-source.aspx), move it to a foundation. This worked out relativly OK for Java proper. It was hella weird, though, and I’m not sure the OSS version ever gained traction: maybe for, like, whatever Google, AWS, and Azure’s JRE is. Using this as a competitive ¯_(ツ)_/¯ is dicey, most people who compete here do open core themselves…so you can’t really say it’s bad; and if Oracle’s goal is to move it away from Oracle, you can’t say that Oracle is mismanaging it, etc. John Waters’ round-up of opinions (https://adtmag.com/articles/2017/08/23/java-open-source.aspx), pretty predictable. Steve Yegge’s Kotlin Writeup (https://steve-yegge.blogspot.com.au/2017/05/why-kotlin-is-better-than-whatever-dumb.html), The RedMonk Programming Language Rankings: June 2017 (http://redmonk.com/sogrady/2017/06/08/language-rankings-6-17/). Kubernetes at GitHub Just a few bash scripts (https://githubengineering.com/kubernetes-at-github/), eh? Here, hold my beer (https://thenewstack.io/github-goes-kubernetes-tells/). Real world discussion about moving one of their most popular services to Kubernetes. Sounds like the real deal, but there are a few bumps in the road. # PE to do 25% of tech M&A At least the analysis (https://blogs.the451group.com/techdeals/investment-banking/meet-the-new-buyer-of-your-tech-company/) confirms this notion. That said, the underlying numbers are weird: “Between direct acquisitions and deals done by portfolio companies, PE firms are on pace to purchase roughly 900 tech companies in 2017.” Who exactly are these 900 tech companies? Speaking of, a PE firm bought ThoughtWorks. ICO stuff (http://www.investopedia.com/terms/i/initial-coin-offering-ico.asp), Coté is confused. BONUS LINKS! Not covered in show. Alibaba Dwarfs Amazon That’s a lot of revenue growth (https://www.linkedin.com/pulse/alibabas-revenue-growth-dwarfing-amazon-michael-spencer). Not on the cloud computing side yet, but definitely on the retail side. Coté: what’s the deal with Alipay being so hard to setup for Yankees? They really, really want a bankcard. Also, I don’t speak Chinese. Rescuing Open Source from Failed Startups bet365 (http://lists.basho.com/pipermail/riak-users_lists.basho.com/2017-August/019500.html) buying (http://lists.basho.com/pipermail/riak-users_lists.basho.com/2017-August/019500.html) and open sourcing Basho stuff. “It is our intention to open source all of Basho's products and all of the source code that they have been working on."Hi See previously RethinkDB by the CNCF (https://thenewstack.io/cloud-native-computing-foundation-scoops-orphaned-rethinkdb-project/) Pivotal news - build pipelines Concourse is out (https://thenewstack.io/pivotal-cloud-foundry-now-can-offer-automated-patching-concourse/), see also CRN (http://www.crn.com/news/cloud/300091001/pivotal-releases-commercial-version-of-concourse-an-internal-continuous-integration-tool-capable-of-rapidly-closing-security-vulnerabilities.htm) coverage (http://www.crn.com/news/cloud/300091001/pivotal-releases-commercial-version-of-concourse-an-internal-continuous-integration-tool-capable-of-rapidly-closing-security-vulnerabilities.htm). Meta, follow-up, etc. Patreon (https://www.patreon.com/sdt) - like anyone who starts these things, I have no idea WTF it is, if it’s a good idea, or if I should be ashamed. Need some product/market fit. Check out the Software Defined Talk Members Only White-Paper Exiguous podcast over there. Join us all in the SDT Slack (http://www.softwaredefinedtalk.com/slack). Mid-roll Get $50 off Casper mattresses with the code: horraymattray NEW DISCOUNT! DevOpsDays Nashville (https://www.devopsdays.org/events/2017-nashville/), $25 off with the code 2017NashDevOpsDays - Coté will be keynoting (https://www.devopsdays.org/events/2017-nashville/speakers/michael-cote/) - October 17th and 18th, 2017. NEW DISCOUNT! DevOpsDays Kansas City (https://www.devopsdays.org/events/2017-kansascity/welcome/), September 21st and 22nd. Use the code SDT2017 when you register (https://www.eventbrite.com/e/devopsdays-kansas-city-2017-tickets-31754843592?aff=ado). PLUS we have one free ticket to give away. So, we need to figure out how to do that. Coté speaking at DevOps Riga (https://www.devopsdays.org/events/2017-riga/welcome/), also will be at DevOpsDays London and Devoxx Belgium. Coté will also be at Devoxx Belgium (https://devoxx.be/), Nov 6th and 10th, in Antwerp. The train station there is nutty-balls awesome (https://www.flickr.com/photos/cote/5201413370/in/photolist-8UyvGj-8UmSEs-8VCAYS-8VzxWp-8VzxUc-8VzyVr-8VCA2Q-8VCA1o-8VCAVC-8VCATj-8VCA4q-8UmSx1-8VCAsm-8VCABs-8VzyHR-8Vzyii-8VCAbh-8VzyvT-8Vzydr-8VCARE-8UiNDe-8VCApA-8VCA7C-8VCAFo-8VCzTs-8V8gFi-yn3eBQ-yoqTCW-y7JJiS-yq594D-y7QtEk-y7Koam-yq4JLX-yn2Hdo-y7KmUf-8UmRYY), y’all. The Register’s conference, Continuous Lifecycle (https://continuouslifecycle.london/), in London (May 2018) has it’s CFP open, closed October 20th - submit something (https://continuouslifecycle.london/call-for-papers/)! SpringOne Platform registration open (https://2017.springoneplatform.io/ehome/s1p/registration), Dec 4th to 5th. Use the code S1P200_Cote for $200 off registration (https://2017.springoneplatform.io/ehome/s1p/registration). Matt’s on the Road! August 30th - AWS Australian Public Sector Summit (https://aws.amazon.com/summits/canberra-public-sector/) September 15-16 - DevOpsDays Bangalore (https://www.devopsdays.org/events/2017-bangalore/) September 20 - Azure Sydney Meetup (https://www.meetup.com/Azure-Sydney-User-Group/events/242374004/) October 3-4 - DevOpsDays New Zealand (https://www.devopsdays.org/events/2017-auckland/) October 11th - Brisbane Azure User Group (https://www.meetup.com/Brisbane-Azure-User-Group/events/240477415/) November 6-7 - AgileNZ (http://www.agilenz.co.nz) Andrew will be at DevOpsDays Singapore (so will Matt) October 25-26, and a few other places. He doesn’t want to make platinum. Recommendations Brandon: TRUECar (https://www.truecar.com/). Matt Ray: Baby Driver. Coté: Taco Deli (http://www.tacodeli.com/). Michael Christmas (https://twitter.com/MickeyChristmas/status/900775508975263745), not too shabby (https://soundcloud.com/michaelchristmas).

Yazılım sektöründeki, Google, Microsoft, Amazon gibi büyük teknoloji şirketlerinin kullandıkları işe alım süreçlerinin ve mülakatlerin nasıl çalıştığını anlamak, kariyer yönetimi açısından size çok faydalı. Bu süreçler neden böyle şekillenmiş, mülakatlerde çıkan sorular neden böyle? Bu soruları cevaplayabilmek için en başından başlayıp şirketlerin gerekçelerini ve aradıklarını irdeleyerek duruma hazırlanmak gerekiyor. Video’da bahsedilen Steve Yegge’nin blog yazısı: … Okumaya devam et "YKP 001 : Teknoloji Şirketlerinde Yazılım Mülakat Sistemleri"

How do computer programs work? How do computers work? (Explain That Stuff!) What is a computer program? (Wikipedia) How does a computer program work? (Dummies) How do computer languages work? (The Linux Documentation Project) How does coding work? (code conquest) How Java works (How Stuff Works, Tech) Readings Bookshop: The beautifully programmed website Johnny was talking about (Readings) Icelab: The people who programmed the lovely website (Icelab) Dyson vaccuums: An example of putting thought into something that sucks & making it better (no pun intended) (Dyson) What is an algorithm? (Wikipedia) What is a computer algorithm? (How Stuff Works, Tech) An algorithm for making a cup of tea (Aristides S. Bouras) What are heuristics in computer science? (Wikipedia) Alan Turing: Creator of modern computing (BBC, iWonder) What is a Turing machine? (University of Cambridge) Why do computers crash? (Scientific American) The Mac spinning beach ball of death (The X Lab) The Windows hourglass wait cursor (Wikipedia) The history of Microsoft Word: Version 1.0 was written by two guys in 1981 (Wikipedia) Microsoft Word 1.0 for Macintosh screenshots (Knubbel Mac) Microsoft makes source code for MS-DOS and Word for Windows available to the public (Microsoft, TechNet) What are abstraction layers? (Wikipedia) Programming language (Wikipedia) High-level programming language (Wikipedia) Which programming languages does Google use internally? (Quora) Microsoft Word was apparently written in Visual C++ (MYCPLUS|COM) There are hundreds of computer languages, which one you pick depends on what you want to achieve (Wikipedia) A comparison of programming languages (Cprogramming.com) Why are there so few female prgrammers? 'When women stopped coding' (NPR, Planet Money) Computer programming used to be women's work (Smithsonian) What is an 'event' in computing? (Wikipedia) Impress your friends & colleagues: Translate anything into binary code! (binarytranslator.com) Computer language interpreters (Wikipedia) What is machine code? (Wikipedia) A picture of binary code (factfile) A picture of hex code (hexblog) What is an operating system? (Wikipedia) Unix & Unix-like operating systems (Wikipedia) The Unix philosophy: 'Create small modular utilities that do one thing & do them well' (How-To Geek) What is a Rube Goldberg machine? (Wikipedia) What is recursion? (Wikipedia) What is a 'library' in computing? (Webopedia) What is a 'library' in computing? (Wikipedia) What is a driver? It's another kind of interpreter that helps you 'drive' a particular piece of hardware (Webopedia) Corrections LARGE DISCLAIMER: At the end when we talk about Amazon, it might be another company that Johnny was thinking of. I can't find evidence that they've banned meetings, just PowerPoint (Moving People To Action). We'll post a link when we figure out where this information came from & if it was about a different company. For now just think of it as 'Company X' who have implemented a freakin' great idea This is potentially the source for the above, a long read but interesting (Stevey's Google Platforms Rant, Steve Yegge) Cheeky review? (If we may be so bold) It'd be amazing if you gave us a short review...it'll make us easier to find in iTunes: Click here for instructions. You're the best! We owe you a free hug and/or a glass of wine from our cellar Click to subscribe in iTunes

  Fredrik och Kristoffer snackar om utvecklingen av programmeringskonsten och undrar varför saker inte går snabbare framåt än de gör. Från webben där alla verkar återuppfinna elementarpartiklar om och om igen, via våra likformiga utvecklingsmiljöer till programmeringsspråk där vi återupptäcker Lisp med jämna mellanrum. Famlar vi fortfarande i blindo i väntan på att någon ska upptäcka elden? Vi hinner också med lästips kring Lisp och problemen med de som predikar entydiga och enkla Svar på alla problem.   Avsnittet sponsras av Malmö startup studio. Länkar Steve Yegge Den statiskt typade säkerhetsvakten på flygplatsen Execution in the kingdom of nouns Steve Yegges blogg Äldre texter Steve Yegge skrev på Amazon Joe Armstrong - skaparen av programmeringsspråket Erlang Rob Pike Emacs - familj av utbyggbara textredigerare js2-mode - javascriptläge för Emacs som Steve Yegge ligger bakom React - javascriptbibliotek för användargränssnitt Origami verktyg för att skapa gränssnittsprototyper HHVM - Facebooks virtuella maskin för PHP och Hack I'm done with the web Cappuccino 280 slides Objective-J Playgrounds - interaktivt och visuellt verktyg för att experimentera med kod skriven i Swift ECMAScript 4 - versionen som sköts i sank ECMAScript - det "officiella" namnet på språket vilket Javascript är en implementation av Javascript och moduler är ett invecklat kapitel Arguments-objektet i Javascript är "arraylikt" men faktiskt inte en array Swift - nyligen släppt språk från Apple Första klassens funktioner Allt är redan upptäckt - i sluten på 1800-talet. Tyvärr inte sant Memristorer Paradigm - tydligt koncept eller tankemönster Delat minnesutrymme System 6 Windows 3 Amiga Actormodellen för samtidig beräkning STM - software transactional memory Race conditions Läckande abstraktioner - abstraktioner som inte döljer underliggande detaljer väl nog Licensen för HHVM - PHP- och Zendlicenserna till största delen Fall med mjukvarupatent till allmänhetens fördel Tesla motors släpper patent … eller? Uber - taxi för rika Doug Hoyte Let over lambda - bok om Lisp On Lisp - gratisboken Kristoffer rekommenderar att man läser före Let over lambda Instapaper - läsa-senare-tjänst The little schemer - en ovanlig och underbar liten bok om programmering Presentationer av Friedman Ten great books - Steve Yegge CAR och CDR Guy Steele Common Lisp the language Practical common Lisp Rabbit - a compiler for Scheme Tidernas första paper om Scheme VAX - gammal instruktionsarkitektur Netscape - företaget bakom den en gång stora och populära webbläsaren med samma namn W3C - World wide web consortium, arbetar bland annat med standarder för webben Bret Victor REPL - read-eval-print loop Agile och Scrum - populära sätt att filosofera kring mjukvaruutveckling Creativity, inc - bok om Pixar och hur de försökt arbeta för att fortsätta utvecklas och frodas Pixar Objektorienterad design TDD - testdriven utveckling Richard Feynman Cargo cult science Robert Martin SOLID-principerna Tage Danielsson

Steve Yegge talks about Grok, a large scale code analysis tool that is being developed at Google and the challenges encountered by cross-language issues. (October 10, 2012)

Should Nintendo be scared? Since the launch of the iPhone -- which encouraged the rise of the Android platform -- smartphone gaming is set to become a majority of the handheld market, eclipsing Nintendo for the first time since the release of the Game Boy in 1989. Vinnk and SeanOrange take a look at the data, how the mobile gaming pie is growing, why smartphone gaming has been so successful, and what (if anything) Nintendo can do about it. See the complete graph of handheld dominance, and read Steve Yegge's rant about platforms versus products in our show notes: https://famicomdojo.tv/podcast/29 Take our survey, and be entered into a drawing for a fee Famicom Dojo Season 1 DVD!. https://famicomdojo.tv/survey