Podcasts about PageRank

Learn how Plex Research combines PageRank-style algorithms with AI to navigate the complexity of biomedical data, transforming untapped research potential into validated drug discovery opportunities.

Send us a textLink building and link buying are fundamentally different approaches with vastly different risks for your SEO strategy. Google's original PageRank algorithm used backlinks as genuine votes of confidence, which is why artificially manipulating this system through paid links violates their guidelines.• Link buying is transactional and violates Google's terms of service• Google considers backlinks as votes of confidence between websites• Penguin update specifically targeted websites buying backlinks• Legitimate link building involves PR and outreach without payment• Digital PR is acceptable as long as no money changes hands• Paying an agency for link building services is fine if they use white hat methods• Be wary of agencies guaranteeing exact numbers of backlinks• Even simple Twitter outreach can generate legitimate backlinks• Good content naturally attracts links once you start the ball rolling• Natural link building takes time but creates sustainable resultsIf you'd like a personal demo of our tools at Keywords People Use, you can book a free, no obligation one-on-one video call with me where I show you how we can help you level up your content by finding and answering the questions your audience actually have. You can also ask me any SEO questions you have. Just go to keywordspeopleuse.com/demo where you can pick a time and date that suits you.SEO Is Not That Hard is hosted by Edd Dawson and brought to you by KeywordsPeopleUse.com Help feed the algorithm and leave a review at ratethispodcast.com/seo You can get your free copy of my 101 Quick SEO Tips at: https://seotips.edddawson.com/101-quick-seo-tipsTo get a personal no-obligation demo of how KeywordsPeopleUse could help you boost your SEO and get a 7 day FREE trial of our Standard Plan book a demo with me nowSee Edd's personal site at edddawson.comAsk me a question and get on the show Click here to record a questionFind Edd on Linkedin, Bluesky & TwitterFind KeywordsPeopleUse on Twitter @kwds_ppl_use"Werq" Kevin MacLeod (incompetech.com)Licensed under Creative Commons: By Attribution 4.0 Licensehttp://creativecommons.org/licenses/by/4.0/

Episode: 2557 Linear algebra, the mathematics behind Google's ranking algorithm.  Today, let's talk about how Google ranks your search results.

In this episode we talk with Bavo DC Campo, a data scientist and statistician, who shares his expertise on the intersection of actuarial science, fraud detection, and social network analytics. Together we will learn how to use graphs to fight against insurance fraud by uncovering hidden connections between fraudulent claims and bad actors. Key insights include how social network analytics can detect fraud rings by mapping relationships between policyholders, claims, and service providers, and how the BiRank algorithm, inspired by Google's PageRank, helps rank suspicious claims based on network structure. Bavo will also present his iFraud simulator that can be used to model fraudulent networks for detection training purposes. Do you have a question about fraud detection? Bavo says he will gladly help. Feel free to contact him.   ------------------------------- Want to listen ad-free?  Try our Graphs Course?  Join Data Skeptic+ for $5 / month of $50 / year https://plus.dataskeptic.com

If you're in SF: Join us for the Claude Plays Pokemon hackathon this Sunday!If you're not: Fill out the 2025 State of AI Eng survey for $250 in Amazon cards!We are SO excited to share our conversation with Dharmesh Shah, co-founder of HubSpot and creator of Agent.ai.A particularly compelling concept we discussed is the idea of "hybrid teams" - the next evolution in workplace organization where human workers collaborate with AI agents as team members. Just as we previously saw hybrid teams emerge in terms of full-time vs. contract workers, or in-office vs. remote workers, Dharmesh predicts that the next frontier will be teams composed of both human and AI members. This raises interesting questions about team dynamics, trust, and how to effectively delegate tasks between human and AI team members.The discussion of business models in AI reveals an important distinction between Work as a Service (WaaS) and Results as a Service (RaaS), something Dharmesh has written extensively about. While RaaS has gained popularity, particularly in customer support applications where outcomes are easily measurable, Dharmesh argues that this model may be over-indexed. Not all AI applications have clearly definable outcomes or consistent economic value per transaction, making WaaS more appropriate in many cases. This insight is particularly relevant for businesses considering how to monetize AI capabilities.The technical challenges of implementing effective agent systems are also explored, particularly around memory and authentication. Shah emphasizes the importance of cross-agent memory sharing and the need for more granular control over data access. He envisions a future where users can selectively share parts of their data with different agents, similar to how OAuth works but with much finer control. This points to significant opportunities in developing infrastructure for secure and efficient agent-to-agent communication and data sharing.Other highlights from our conversation* The Evolution of AI-Powered Agents – Exploring how AI agents have evolved from simple chatbots to sophisticated multi-agent systems, and the role of MCPs in enabling that.* Hybrid Digital Teams and the Future of Work – How AI agents are becoming teammates rather than just tools, and what this means for business operations and knowledge work.* Memory in AI Agents – The importance of persistent memory in AI systems and how shared memory across agents could enhance collaboration and efficiency.* Business Models for AI Agents – Exploring the shift from software as a service (SaaS) to work as a service (WaaS) and results as a service (RaaS), and what this means for monetization.* The Role of Standards Like MCP – Why MCP has been widely adopted and how it enables agent collaboration, tool use, and discovery.* The Future of AI Code Generation and Software Engineering – How AI-assisted coding is changing the role of software engineers and what skills will matter most in the future.* Domain Investing and Efficient Markets – Dharmesh's approach to domain investing and how inefficiencies in digital asset markets create business opportunities.* The Philosophy of Saying No – Lessons from "Sorry, You Must Pass" and how prioritization leads to greater productivity and focus.Timestamps* 00:00 Introduction and Guest Welcome* 02:29 Dharmesh Shah's Journey into AI* 05:22 Defining AI Agents* 06:45 The Evolution and Future of AI Agents* 13:53 Graph Theory and Knowledge Representation* 20:02 Engineering Practices and Overengineering* 25:57 The Role of Junior Engineers in the AI Era* 28:20 Multi-Agent Systems and MCP Standards* 35:55 LinkedIn's Legal Battles and Data Scraping* 37:32 The Future of AI and Hybrid Teams* 39:19 Building Agent AI: A Professional Network for Agents* 40:43 Challenges and Innovations in Agent AI* 45:02 The Evolution of UI in AI Systems* 01:00:25 Business Models: Work as a Service vs. Results as a Service* 01:09:17 The Future Value of Engineers* 01:09:51 Exploring the Role of Agents* 01:10:28 The Importance of Memory in AI* 01:11:02 Challenges and Opportunities in AI Memory* 01:12:41 Selective Memory and Privacy Concerns* 01:13:27 The Evolution of AI Tools and Platforms* 01:18:23 Domain Names and AI Projects* 01:32:08 Balancing Work and Personal Life* 01:35:52 Final Thoughts and ReflectionsTranscriptAlessio [00:00:04]: Hey everyone, welcome back to the Latent Space podcast. This is Alessio, partner and CTO at Decibel Partners, and I'm joined by my co-host Swyx, founder of Small AI.swyx [00:00:12]: Hello, and today we're super excited to have Dharmesh Shah to join us. I guess your relevant title here is founder of Agent AI.Dharmesh [00:00:20]: Yeah, that's true for this. Yeah, creator of Agent.ai and co-founder of HubSpot.swyx [00:00:25]: Co-founder of HubSpot, which I followed for many years, I think 18 years now, gonna be 19 soon. And you caught, you know, people can catch up on your HubSpot story elsewhere. I should also thank Sean Puri, who I've chatted with back and forth, who's been, I guess, getting me in touch with your people. But also, I think like, just giving us a lot of context, because obviously, My First Million joined you guys, and they've been chatting with you guys a lot. So for the business side, we can talk about that, but I kind of wanted to engage your CTO, agent, engineer side of things. So how did you get agent religion?Dharmesh [00:01:00]: Let's see. So I've been working, I'll take like a half step back, a decade or so ago, even though actually more than that. So even before HubSpot, the company I was contemplating that I had named for was called Ingenisoft. And the idea behind Ingenisoft was a natural language interface to business software. Now realize this is 20 years ago, so that was a hard thing to do. But the actual use case that I had in mind was, you know, we had data sitting in business systems like a CRM or something like that. And my kind of what I thought clever at the time. Oh, what if we used email as the kind of interface to get to business software? And the motivation for using email is that it automatically works when you're offline. So imagine I'm getting on a plane or I'm on a plane. There was no internet on planes back then. It's like, oh, I'm going through business cards from an event I went to. I can just type things into an email just to have them all in the backlog. When it reconnects, it sends those emails to a processor that basically kind of parses effectively the commands and updates the software, sends you the file, whatever it is. And there was a handful of commands. I was a little bit ahead of the times in terms of what was actually possible. And I reattempted this natural language thing with a product called ChatSpot that I did back 20...swyx [00:02:12]: Yeah, this is your first post-ChatGPT project.Dharmesh [00:02:14]: I saw it come out. Yeah. And so I've always been kind of fascinated by this natural language interface to software. Because, you know, as software developers, myself included, we've always said, oh, we build intuitive, easy-to-use applications. And it's not intuitive at all, right? Because what we're doing is... We're taking the mental model that's in our head of what we're trying to accomplish with said piece of software and translating that into a series of touches and swipes and clicks and things like that. And there's nothing natural or intuitive about it. And so natural language interfaces, for the first time, you know, whatever the thought is you have in your head and expressed in whatever language that you normally use to talk to yourself in your head, you can just sort of emit that and have software do something. And I thought that was kind of a breakthrough, which it has been. And it's gone. So that's where I first started getting into the journey. I started because now it actually works, right? So once we got ChatGPT and you can take, even with a few-shot example, convert something into structured, even back in the ChatGP 3.5 days, it did a decent job in a few-shot example, convert something to structured text if you knew what kinds of intents you were going to have. And so that happened. And that ultimately became a HubSpot project. But then agents intrigued me because I'm like, okay, well, that's the next step here. So chat's great. Love Chat UX. But if we want to do something even more meaningful, it felt like the next kind of advancement is not this kind of, I'm chatting with some software in a kind of a synchronous back and forth model, is that software is going to do things for me in kind of a multi-step way to try and accomplish some goals. So, yeah, that's when I first got started. It's like, okay, what would that look like? Yeah. And I've been obsessed ever since, by the way.Alessio [00:03:55]: Which goes back to your first experience with it, which is like you're offline. Yeah. And you want to do a task. You don't need to do it right now. You just want to queue it up for somebody to do it for you. Yes. As you think about agents, like, let's start at the easy question, which is like, how do you define an agent? Maybe. You mean the hardest question in the universe? Is that what you mean?Dharmesh [00:04:12]: You said you have an irritating take. I do have an irritating take. I think, well, some number of people have been irritated, including within my own team. So I have a very broad definition for agents, which is it's AI-powered software that accomplishes a goal. Period. That's it. And what irritates people about it is like, well, that's so broad as to be completely non-useful. And I understand that. I understand the criticism. But in my mind, if you kind of fast forward months, I guess, in AI years, the implementation of it, and we're already starting to see this, and we'll talk about this, different kinds of agents, right? So I think in addition to having a usable definition, and I like yours, by the way, and we should talk more about that, that you just came out with, the classification of agents actually is also useful, which is, is it autonomous or non-autonomous? Does it have a deterministic workflow? Does it have a non-deterministic workflow? Is it working synchronously? Is it working asynchronously? Then you have the different kind of interaction modes. Is it a chat agent, kind of like a customer support agent would be? You're having this kind of back and forth. Is it a workflow agent that just does a discrete number of steps? So there's all these different flavors of agents. So if I were to draw it in a Venn diagram, I would draw a big circle that says, this is agents, and then I have a bunch of circles, some overlapping, because they're not mutually exclusive. And so I think that's what's interesting, and we're seeing development along a bunch of different paths, right? So if you look at the first implementation of agent frameworks, you look at Baby AGI and AutoGBT, I think it was, not Autogen, that's the Microsoft one. They were way ahead of their time because they assumed this level of reasoning and execution and planning capability that just did not exist, right? So it was an interesting thought experiment, which is what it was. Even the guy that, I'm an investor in Yohei's fund that did Baby AGI. It wasn't ready, but it was a sign of what was to come. And so the question then is, when is it ready? And so lots of people talk about the state of the art when it comes to agents. I'm a pragmatist, so I think of the state of the practical. It's like, okay, well, what can I actually build that has commercial value or solves actually some discrete problem with some baseline of repeatability or verifiability?swyx [00:06:22]: There was a lot, and very, very interesting. I'm not irritated by it at all. Okay. As you know, I take a... There's a lot of anthropological view or linguistics view. And in linguistics, you don't want to be prescriptive. You want to be descriptive. Yeah. So you're a goals guy. That's the key word in your thing. And other people have other definitions that might involve like delegated trust or non-deterministic work, LLM in the loop, all that stuff. The other thing I was thinking about, just the comment on Baby AGI, LGBT. Yeah. In that piece that you just read, I was able to go through our backlog and just kind of track the winter of agents and then the summer now. Yeah. And it's... We can tell the whole story as an oral history, just following that thread. And it's really just like, I think, I tried to explain the why now, right? Like I had, there's better models, of course. There's better tool use with like, they're just more reliable. Yep. Better tools with MCP and all that stuff. And I'm sure you have opinions on that too. Business model shift, which you like a lot. I just heard you talk about RAS with MFM guys. Yep. Cost is dropping a lot. Yep. Inference is getting faster. There's more model diversity. Yep. Yep. I think it's a subtle point. It means that like, you have different models with different perspectives. You don't get stuck in the basin of performance of a single model. Sure. You can just get out of it by just switching models. Yep. Multi-agent research and RL fine tuning. So I just wanted to let you respond to like any of that.Dharmesh [00:07:44]: Yeah. A couple of things. Connecting the dots on the kind of the definition side of it. So we'll get the irritation out of the way completely. I have one more, even more irritating leap on the agent definition thing. So here's the way I think about it. By the way, the kind of word agent, I looked it up, like the English dictionary definition. The old school agent, yeah. Is when you have someone or something that does something on your behalf, like a travel agent or a real estate agent acts on your behalf. It's like proxy, which is a nice kind of general definition. So the other direction I'm sort of headed, and it's going to tie back to tool calling and MCP and things like that, is if you, and I'm not a biologist by any stretch of the imagination, but we have these single-celled organisms, right? Like the simplest possible form of what one would call life. But it's still life. It just happens to be single-celled. And then you can combine cells and then cells become specialized over time. And you have much more sophisticated organisms, you know, kind of further down the spectrum. In my mind, at the most fundamental level, you can almost think of having atomic agents. What is the simplest possible thing that's an agent that can still be called an agent? What is the equivalent of a kind of single-celled organism? And the reason I think that's useful is right now we're headed down the road, which I think is very exciting around tool use, right? That says, okay, the LLMs now can be provided a set of tools that it calls to accomplish whatever it needs to accomplish in the kind of furtherance of whatever goal it's trying to get done. And I'm not overly bothered by it, but if you think about it, if you just squint a little bit and say, well, what if everything was an agent? And what if tools were actually just atomic agents? Because then it's turtles all the way down, right? Then it's like, oh, well, all that's really happening with tool use is that we have a network of agents that know about each other through something like an MMCP and can kind of decompose a particular problem and say, oh, I'm going to delegate this to this set of agents. And why do we need to draw this distinction between tools, which are functions most of the time? And an actual agent. And so I'm going to write this irritating LinkedIn post, you know, proposing this. It's like, okay. And I'm not suggesting we should call even functions, you know, call them agents. But there is a certain amount of elegance that happens when you say, oh, we can just reduce it down to one primitive, which is an agent that you can combine in complicated ways to kind of raise the level of abstraction and accomplish higher order goals. Anyway, that's my answer. I'd say that's a success. Thank you for coming to my TED Talk on agent definitions.Alessio [00:09:54]: How do you define the minimum viable agent? Do you already have a definition for, like, where you draw the line between a cell and an atom? Yeah.Dharmesh [00:10:02]: So in my mind, it has to, at some level, use AI in order for it to—otherwise, it's just software. It's like, you know, we don't need another word for that. And so that's probably where I draw the line. So then the question, you know, the counterargument would be, well, if that's true, then lots of tools themselves are actually not agents because they're just doing a database call or a REST API call or whatever it is they're doing. And that does not necessarily qualify them, which is a fair counterargument. And I accept that. It's like a good argument. I still like to think about—because we'll talk about multi-agent systems, because I think—so we've accepted, which I think is true, lots of people have said it, and you've hopefully combined some of those clips of really smart people saying this is the year of agents, and I completely agree, it is the year of agents. But then shortly after that, it's going to be the year of multi-agent systems or multi-agent networks. I think that's where it's going to be headed next year. Yeah.swyx [00:10:54]: Opening eyes already on that. Yeah. My quick philosophical engagement with you on this. I often think about kind of the other spectrum, the other end of the cell spectrum. So single cell is life, multi-cell is life, and you clump a bunch of cells together in a more complex organism, they become organs, like an eye and a liver or whatever. And then obviously we consider ourselves one life form. There's not like a lot of lives within me. I'm just one life. And now, obviously, I don't think people don't really like to anthropomorphize agents and AI. Yeah. But we are extending our consciousness and our brain and our functionality out into machines. I just saw you were a Bee. Yeah. Which is, you know, it's nice. I have a limitless pendant in my pocket.Dharmesh [00:11:37]: I got one of these boys. Yeah.swyx [00:11:39]: I'm testing it all out. You know, got to be early adopters. But like, we want to extend our personal memory into these things so that we can be good at the things that we're good at. And, you know, machines are good at it. Machines are there. So like, my definition of life is kind of like going outside of my own body now. I don't know if you've ever had like reflections on that. Like how yours. How our self is like actually being distributed outside of you. Yeah.Dharmesh [00:12:01]: I don't fancy myself a philosopher. But you went there. So yeah, I did go there. I'm fascinated by kind of graphs and graph theory and networks and have been for a long, long time. And to me, we're sort of all nodes in this kind of larger thing. It just so happens that we're looking at individual kind of life forms as they exist right now. But so the idea is when you put a podcast out there, there's these little kind of nodes you're putting out there of like, you know, conceptual ideas. Once again, you have varying kind of forms of those little nodes that are up there and are connected in varying and sundry ways. And so I just think of myself as being a node in a massive, massive network. And I'm producing more nodes as I put content or ideas. And, you know, you spend some portion of your life collecting dots, experiences, people, and some portion of your life then connecting dots from the ones that you've collected over time. And I found that really interesting things happen and you really can't know in advance how those dots are necessarily going to connect in the future. And that's, yeah. So that's my philosophical take. That's the, yes, exactly. Coming back.Alessio [00:13:04]: Yep. Do you like graph as an agent? Abstraction? That's been one of the hot topics with LandGraph and Pydantic and all that.Dharmesh [00:13:11]: I do. The thing I'm more interested in terms of use of graphs, and there's lots of work happening on that now, is graph data stores as an alternative in terms of knowledge stores and knowledge graphs. Yeah. Because, you know, so I've been in software now 30 plus years, right? So it's not 10,000 hours. It's like 100,000 hours that I've spent doing this stuff. And so I've grew up with, so back in the day, you know, I started on mainframes. There was a product called IMS from IBM, which is basically an index database, what we'd call like a key value store today. Then we've had relational databases, right? We have tables and columns and foreign key relationships. We all know that. We have document databases like MongoDB, which is sort of a nested structure keyed by a specific index. We have vector stores, vector embedding database. And graphs are interesting for a couple of reasons. One is, so it's not classically structured in a relational way. When you say structured database, to most people, they're thinking tables and columns and in relational database and set theory and all that. Graphs still have structure, but it's not the tables and columns structure. And you could wonder, and people have made this case, that they are a better representation of knowledge for LLMs and for AI generally than other things. So that's kind of thing number one conceptually, and that might be true, I think is possibly true. And the other thing that I really like about that in the context of, you know, I've been in the context of data stores for RAG is, you know, RAG, you say, oh, I have a million documents, I'm going to build the vector embeddings, I'm going to come back with the top X based on the semantic match, and that's fine. All that's very, very useful. But the reality is something gets lost in the chunking process and the, okay, well, those tend, you know, like, you don't really get the whole picture, so to speak, and maybe not even the right set of dimensions on the kind of broader picture. And it makes intuitive sense to me that if we did capture it properly in a graph form, that maybe that feeding into a RAG pipeline will actually yield better results for some use cases, I don't know, but yeah.Alessio [00:15:03]: And do you feel like at the core of it, there's this difference between imperative and declarative programs? Because if you think about HubSpot, it's like, you know, people and graph kind of goes hand in hand, you know, but I think maybe the software before was more like primary foreign key based relationship, versus now the models can traverse through the graph more easily.Dharmesh [00:15:22]: Yes. So I like that representation. There's something. It's just conceptually elegant about graphs and just from the representation of it, they're much more discoverable, you can kind of see it, there's observability to it, versus kind of embeddings, which you can't really do much with as a human. You know, once they're in there, you can't pull stuff back out. But yeah, I like that kind of idea of it. And the other thing that's kind of, because I love graphs, I've been long obsessed with PageRank from back in the early days. And, you know, one of the kind of simplest algorithms in terms of coming up, you know, with a phone, everyone's been exposed to PageRank. And the idea is that, and so I had this other idea for a project, not a company, and I have hundreds of these, called NodeRank, is to be able to take the idea of PageRank and apply it to an arbitrary graph that says, okay, I'm going to define what authority looks like and say, okay, well, that's interesting to me, because then if you say, I'm going to take my knowledge store, and maybe this person that contributed some number of chunks to the graph data store has more authority on this particular use case or prompt that's being submitted than this other one that may, or maybe this one was more. popular, or maybe this one has, whatever it is, there should be a way for us to kind of rank nodes in a graph and sort them in some, some useful way. Yeah.swyx [00:16:34]: So I think that's generally useful for, for anything. I think the, the problem, like, so even though at my conferences, GraphRag is super popular and people are getting knowledge, graph religion, and I will say like, it's getting space, getting traction in two areas, conversation memory, and then also just rag in general, like the, the, the document data. Yeah. It's like a source. Most ML practitioners would say that knowledge graph is kind of like a dirty word. The graph database, people get graph religion, everything's a graph, and then they, they go really hard into it and then they get a, they get a graph that is too complex to navigate. Yes. And so like the, the, the simple way to put it is like you at running HubSpot, you know, the power of graphs, the way that Google has pitched them for many years, but I don't suspect that HubSpot itself uses a knowledge graph. No. Yeah.Dharmesh [00:17:26]: So when is it over engineering? Basically? It's a great question. I don't know. So the question now, like in AI land, right, is the, do we necessarily need to understand? So right now, LLMs for, for the most part are somewhat black boxes, right? We sort of understand how the, you know, the algorithm itself works, but we really don't know what's going on in there and, and how things come out. So if a graph data store is able to produce the outcomes we want, it's like, here's a set of queries I want to be able to submit and then it comes out with useful content. Maybe the underlying data store is as opaque as a vector embeddings or something like that, but maybe it's fine. Maybe we don't necessarily need to understand it to get utility out of it. And so maybe if it's messy, that's okay. Um, that's, it's just another form of lossy compression. Uh, it's just lossy in a way that we just don't completely understand in terms of, because it's going to grow organically. Uh, and it's not structured. It's like, ah, we're just gonna throw a bunch of stuff in there. Let the, the equivalent of the embedding algorithm, whatever they called in graph land. Um, so the one with the best results wins. I think so. Yeah.swyx [00:18:26]: Or is this the practical side of me is like, yeah, it's, if it's useful, we don't necessarilyDharmesh [00:18:30]: need to understand it.swyx [00:18:30]: I have, I mean, I'm happy to push back as long as you want. Uh, it's not practical to evaluate like the 10 different options out there because it takes time. It takes people, it takes, you know, resources, right? Set. That's the first thing. Second thing is your evals are typically on small things and some things only work at scale. Yup. Like graphs. Yup.Dharmesh [00:18:46]: Yup. That's, yeah, no, that's fair. And I think this is one of the challenges in terms of implementation of graph databases is that the most common approach that I've seen developers do, I've done it myself, is that, oh, I've got a Postgres database or a MySQL or whatever. I can represent a graph with a very set of tables with a parent child thing or whatever. And that sort of gives me the ability, uh, why would I need anything more than that? And the answer is, well, if you don't need anything more than that, you don't need anything more than that. But there's a high chance that you're sort of missing out on the actual value that, uh, the graph representation gives you. Which is the ability to traverse the graph, uh, efficiently in ways that kind of going through the, uh, traversal in a relational database form, even though structurally you have the data, practically you're not gonna be able to pull it out in, in useful ways. Uh, so you wouldn't like represent a social graph, uh, in, in using that kind of relational table model. It just wouldn't scale. It wouldn't work.swyx [00:19:36]: Uh, yeah. Uh, I think we want to move on to MCP. Yeah. But I just want to, like, just engineering advice. Yeah. Uh, obviously you've, you've, you've run, uh, you've, you've had to do a lot of projects and run a lot of teams. Do you have a general rule for over-engineering or, you know, engineering ahead of time? You know, like, because people, we know premature engineering is the root of all evil. Yep. But also sometimes you just have to. Yep. When do you do it? Yes.Dharmesh [00:19:59]: It's a great question. This is, uh, a question as old as time almost, which is what's the right and wrong levels of abstraction. That's effectively what, uh, we're answering when we're trying to do engineering. I tend to be a pragmatist, right? So here's the thing. Um, lots of times doing something the right way. Yeah. It's like a marginal increased cost in those cases. Just do it the right way. And this is what makes a, uh, a great engineer or a good engineer better than, uh, a not so great one. It's like, okay, all things being equal. If it's going to take you, you know, roughly close to constant time anyway, might as well do it the right way. Like, so do things well, then the question is, okay, well, am I building a framework as the reusable library? To what degree, uh, what am I anticipating in terms of what's going to need to change in this thing? Uh, you know, along what dimension? And then I think like a business person in some ways, like what's the return on calories, right? So, uh, and you look at, um, energy, the expected value of it's like, okay, here are the five possible things that could happen, uh, try to assign probabilities like, okay, well, if there's a 50% chance that we're going to go down this particular path at some day, like, or one of these five things is going to happen and it costs you 10% more to engineer for that. It's basically, it's something that yields a kind of interest compounding value. Um, as you get closer to the time of, of needing that versus having to take on debt, which is when you under engineer it, you're taking on debt. You're going to have to pay off when you do get to that eventuality where something happens. One thing as a pragmatist, uh, so I would rather under engineer something than over engineer it. If I were going to err on the side of something, and here's the reason is that when you under engineer it, uh, yes, you take on tech debt, uh, but the interest rate is relatively known and payoff is very, very possible, right? Which is, oh, I took a shortcut here as a result of which now this thing that should have taken me a week is now going to take me four weeks. Fine. But if that particular thing that you thought might happen, never actually, you never have that use case transpire or just doesn't, it's like, well, you just save yourself time, right? And that has value because you were able to do other things instead of, uh, kind of slightly over-engineering it away, over-engineering it. But there's no perfect answers in art form in terms of, uh, and yeah, we'll, we'll bring kind of this layers of abstraction back on the code generation conversation, which we'll, uh, I think I have later on, butAlessio [00:22:05]: I was going to ask, we can just jump ahead quickly. Yeah. Like, as you think about vibe coding and all that, how does the. Yeah. Percentage of potential usefulness change when I feel like we over-engineering a lot of times it's like the investment in syntax, it's less about the investment in like arc exacting. Yep. Yeah. How does that change your calculus?Dharmesh [00:22:22]: A couple of things, right? One is, um, so, you know, going back to that kind of ROI or a return on calories, kind of calculus or heuristic you think through, it's like, okay, well, what is it going to cost me to put this layer of abstraction above the code that I'm writing now, uh, in anticipating kind of future needs. If the cost of fixing, uh, or doing under engineering right now. Uh, we'll trend towards zero that says, okay, well, I don't have to get it right right now because even if I get it wrong, I'll run the thing for six hours instead of 60 minutes or whatever. It doesn't really matter, right? Like, because that's going to trend towards zero to be able, the ability to refactor a code. Um, and because we're going to not that long from now, we're going to have, you know, large code bases be able to exist, uh, you know, as, as context, uh, for a code generation or a code refactoring, uh, model. So I think it's going to make it, uh, make the case for under engineering, uh, even stronger. Which is why I take on that cost. You just pay the interest when you get there, it's not, um, just go on with your life vibe coded and, uh, come back when you need to. Yeah.Alessio [00:23:18]: Sometimes I feel like there's no decision-making in some things like, uh, today I built a autosave for like our internal notes platform and I literally just ask them cursor. Can you add autosave? Yeah. I don't know if it's over under engineer. Yep. I just vibe coded it. Yep. And I feel like at some point we're going to get to the point where the models kindDharmesh [00:23:36]: of decide where the right line is, but this is where the, like the, in my mind, the danger is, right? So there's two sides to this. One is the cost of kind of development and coding and things like that stuff that, you know, we talk about. But then like in your example, you know, one of the risks that we have is that because adding a feature, uh, like a save or whatever the feature might be to a product as that price tends towards zero, are we going to be less discriminant about what features we add as a result of making more product products more complicated, which has a negative impact on the user and navigate negative impact on the business. Um, and so that's the thing I worry about if it starts to become too easy, are we going to be. Too promiscuous in our, uh, kind of extension, adding product extensions and things like that. It's like, ah, why not add X, Y, Z or whatever back then it was like, oh, we only have so many engineering hours or story points or however you measure things. Uh, that least kept us in check a little bit. Yeah.Alessio [00:24:22]: And then over engineering, you're like, yeah, it's kind of like you're putting that on yourself. Yeah. Like now it's like the models don't understand that if they add too much complexity, it's going to come back to bite them later. Yep. So they just do whatever they want to do. Yeah. And I'm curious where in the workflow that's going to be, where it's like, Hey, this is like the amount of complexity and over-engineering you can do before you got to ask me if we should actually do it versus like do something else.Dharmesh [00:24:45]: So you know, we've already, let's like, we're leaving this, uh, in the code generation world, this kind of compressed, um, cycle time. Right. It's like, okay, we went from auto-complete, uh, in the GitHub co-pilot to like, oh, finish this particular thing and hit tab to a, oh, I sort of know your file or whatever. I can write out a full function to you to now I can like hold a bunch of the context in my head. Uh, so we can do app generation, which we have now with lovable and bolt and repletage. Yeah. Association and other things. So then the question is, okay, well, where does it naturally go from here? So we're going to generate products. Make sense. We might be able to generate platforms as though I want a platform for ERP that does this, whatever. And that includes the API's includes the product and the UI, and all the things that make for a platform. There's no nothing that says we would stop like, okay, can you generate an entire software company someday? Right. Uh, with the platform and the monetization and the go-to-market and the whatever. And you know, that that's interesting to me in terms of, uh, you know, what, when you take it to almost ludicrous levels. of abstract.swyx [00:25:39]: It's like, okay, turn it to 11. You mentioned vibe coding, so I have to, this is a blog post I haven't written, but I'm kind of exploring it. Is the junior engineer dead?Dharmesh [00:25:49]: I don't think so. I think what will happen is that the junior engineer will be able to, if all they're bringing to the table is the fact that they are a junior engineer, then yes, they're likely dead. But hopefully if they can communicate with carbon-based life forms, they can interact with product, if they're willing to talk to customers, they can take their kind of basic understanding of engineering and how kind of software works. I think that has value. So I have a 14-year-old right now who's taking Python programming class, and some people ask me, it's like, why is he learning coding? And my answer is, is because it's not about the syntax, it's not about the coding. What he's learning is like the fundamental thing of like how things work. And there's value in that. I think there's going to be timeless value in systems thinking and abstractions and what that means. And whether functions manifested as math, which he's going to get exposed to regardless, or there are some core primitives to the universe, I think, that the more you understand them, those are what I would kind of think of as like really large dots in your life that will have a higher gravitational pull and value to them that you'll then be able to. So I want him to collect those dots, and he's not resisting. So it's like, okay, while he's still listening to me, I'm going to have him do things that I think will be useful.swyx [00:26:59]: You know, part of one of the pitches that I evaluated for AI engineer is a term. And the term is that maybe the traditional interview path or career path of software engineer goes away, which is because what's the point of lead code? Yeah. And, you know, it actually matters more that you know how to work with AI and to implement the things that you want. Yep.Dharmesh [00:27:16]: That's one of the like interesting things that's happened with generative AI. You know, you go from machine learning and the models and just that underlying form, which is like true engineering, right? Like the actual, what I call real engineering. I don't think of myself as a real engineer, actually. I'm a developer. But now with generative AI. We call it AI and it's obviously got its roots in machine learning, but it just feels like fundamentally different to me. Like you have the vibe. It's like, okay, well, this is just a whole different approach to software development to so many different things. And so I'm wondering now, it's like an AI engineer is like, if you were like to draw the Venn diagram, it's interesting because the cross between like AI things, generative AI and what the tools are capable of, what the models do, and this whole new kind of body of knowledge that we're still building out, it's still very young, intersected with kind of classic engineering, software engineering. Yeah.swyx [00:28:04]: I just described the overlap as it separates out eventually until it's its own thing, but it's starting out as a software. Yeah.Alessio [00:28:11]: That makes sense. So to close the vibe coding loop, the other big hype now is MCPs. Obviously, I would say Cloud Desktop and Cursor are like the two main drivers of MCP usage. I would say my favorite is the Sentry MCP. I can pull in errors and then you can just put the context in Cursor. How do you think about that abstraction layer? Does it feel... Does it feel almost too magical in a way? Do you think it's like you get enough? Because you don't really see how the server itself is then kind of like repackaging theDharmesh [00:28:41]: information for you? I think MCP as a standard is one of the better things that's happened in the world of AI because a standard needed to exist and absent a standard, there was a set of things that just weren't possible. Now, we can argue whether it's the best possible manifestation of a standard or not. Does it do too much? Does it do too little? I get that, but it's just simple enough to both be useful and unobtrusive. It's understandable and adoptable by mere mortals, right? It's not overly complicated. You know, a reasonable engineer can put a stand up an MCP server relatively easily. The thing that has me excited about it is like, so I'm a big believer in multi-agent systems. And so that's going back to our kind of this idea of an atomic agent. So imagine the MCP server, like obviously it calls tools, but the way I think about it, so I'm working on my current passion project is agent.ai. And we'll talk more about that in a little bit. More about the, I think we should, because I think it's interesting not to promote the project at all, but there's some interesting ideas in there. One of which is around, we're going to need a mechanism for, if agents are going to collaborate and be able to delegate, there's going to need to be some form of discovery and we're going to need some standard way. It's like, okay, well, I just need to know what this thing over here is capable of. We're going to need a registry, which Anthropic's working on. I'm sure others will and have been doing directories of, and there's going to be a standard around that too. How do you build out a directory of MCP servers? I think that's going to unlock so many things just because, and we're already starting to see it. So I think MCP or something like it is going to be the next major unlock because it allows systems that don't know about each other, don't need to, it's that kind of decoupling of like Sentry and whatever tools someone else was building. And it's not just about, you know, Cloud Desktop or things like, even on the client side, I think we're going to see very interesting consumers of MCP, MCP clients versus just the chat body kind of things. Like, you know, Cloud Desktop and Cursor and things like that. But yeah, I'm very excited about MCP in that general direction.swyx [00:30:39]: I think the typical cynical developer take, it's like, we have OpenAPI. Yeah. What's the new thing? I don't know if you have a, do you have a quick MCP versus everything else? Yeah.Dharmesh [00:30:49]: So it's, so I like OpenAPI, right? So just a descriptive thing. It's OpenAPI. OpenAPI. Yes, that's what I meant. So it's basically a self-documenting thing. We can do machine-generated, lots of things from that output. It's a structured definition of an API. I get that, love it. But MCPs sort of are kind of use case specific. They're perfect for exactly what we're trying to use them for around LLMs in terms of discovery. It's like, okay, I don't necessarily need to know kind of all this detail. And so right now we have, we'll talk more about like MCP server implementations, but We will? I think, I don't know. Maybe we won't. At least it's in my head. It's like a back processor. But I do think MCP adds value above OpenAPI. It's, yeah, just because it solves this particular thing. And if we had come to the world, which we have, like, it's like, hey, we already have OpenAPI. It's like, if that were good enough for the universe, the universe would have adopted it already. There's a reason why MCP is taking office because marginally adds something that was missing before and doesn't go too far. And so that's why the kind of rate of adoption, you folks have written about this and talked about it. Yeah, why MCP won. Yeah. And it won because the universe decided that this was useful and maybe it gets supplanted by something else. Yeah. And maybe we discover, oh, maybe OpenAPI was good enough the whole time. I doubt that.swyx [00:32:09]: The meta lesson, this is, I mean, he's an investor in DevTools companies. I work in developer experience at DevRel in DevTools companies. Yep. Everyone wants to own the standard. Yeah. I'm sure you guys have tried to launch your own standards. Actually, it's Houseplant known for a standard, you know, obviously inbound marketing. But is there a standard or protocol that you ever tried to push? No.Dharmesh [00:32:30]: And there's a reason for this. Yeah. Is that? And I don't mean, need to mean, speak for the people of HubSpot, but I personally. You kind of do. I'm not smart enough. That's not the, like, I think I have a. You're smart. Not enough for that. I'm much better off understanding the standards that are out there. And I'm more on the composability side. Let's, like, take the pieces of technology that exist out there, combine them in creative, unique ways. And I like to consume standards. I don't like to, and that's not that I don't like to create them. I just don't think I have the, both the raw wattage or the credibility. It's like, okay, well, who the heck is Dharmesh, and why should we adopt a standard he created?swyx [00:33:07]: Yeah, I mean, there are people who don't monetize standards, like OpenTelemetry is a big standard, and LightStep never capitalized on that.Dharmesh [00:33:15]: So, okay, so if I were to do a standard, there's two things that have been in my head in the past. I was one around, a very, very basic one around, I don't even have the domain, I have a domain for everything, for open marketing. Because the issue we had in HubSpot grew up in the marketing space. There we go. There was no standard around data formats and things like that. It doesn't go anywhere. But the other one, and I did not mean to go here, but I'm going to go here. It's called OpenGraph. I know the term was already taken, but it hasn't been used for like 15 years now for its original purpose. But what I think should exist in the world is right now, our information, all of us, nodes are in the social graph at Meta or the professional graph at LinkedIn. Both of which are actually relatively closed in actually very annoying ways. Like very, very closed, right? Especially LinkedIn. Especially LinkedIn. I personally believe that if it's my data, and if I would get utility out of it being open, I should be able to make my data open or publish it in whatever forms that I choose, as long as I have control over it as opt-in. So the idea is around OpenGraph that says, here's a standard, here's a way to publish it. I should be able to go to OpenGraph.org slash Dharmesh dot JSON and get it back. And it's like, here's your stuff, right? And I can choose along the way and people can write to it and I can prove. And there can be an entire system. And if I were to do that, I would do it as a... Like a public benefit, non-profit-y kind of thing, as this is a contribution to society. I wouldn't try to commercialize that. Have you looked at AdProto? What's that? AdProto.swyx [00:34:43]: It's the protocol behind Blue Sky. Okay. My good friend, Dan Abramov, who was the face of React for many, many years, now works there. And he actually did a talk that I can send you, which basically kind of tries to articulate what you just said. But he does, he loves doing these like really great analogies, which I think you'll like. Like, you know, a lot of our data is behind a handle, behind a domain. Yep. So he's like, all right, what if we flip that? What if it was like our handle and then the domain? Yep. So, and that's really like your data should belong to you. Yep. And I should not have to wait 30 days for my Twitter data to export. Yep.Dharmesh [00:35:19]: you should be able to at least be able to automate it or do like, yes, I should be able to plug it into an agentic thing. Yeah. Yes. I think we're... Because so much of our data is... Locked up. I think the trick here isn't that standard. It is getting the normies to care.swyx [00:35:37]: Yeah. Because normies don't care.Dharmesh [00:35:38]: That's true. But building on that, normies don't care. So, you know, privacy is a really hot topic and an easy word to use, but it's not a binary thing. Like there are use cases where, and we make these choices all the time, that I will trade, not all privacy, but I will trade some privacy for some productivity gain or some benefit to me that says, oh, I don't care about that particular data being online if it gives me this in return, or I don't mind sharing this information with this company.Alessio [00:36:02]: If I'm getting, you know, this in return, but that sort of should be my option. I think now with computer use, you can actually automate some of the exports. Yes. Like something we've been doing internally is like everybody exports their LinkedIn connections. Yep. And then internally, we kind of merge them together to see how we can connect our companies to customers or things like that.Dharmesh [00:36:21]: And not to pick on LinkedIn, but since we're talking about it, but they feel strongly enough on the, you know, do not take LinkedIn data that they will block even browser use kind of things or whatever. They go to great, great lengths, even to see patterns of usage. And it says, oh, there's no way you could have, you know, gotten that particular thing or whatever without, and it's, so it's, there's...swyx [00:36:42]: Wasn't there a Supreme Court case that they lost? Yeah.Dharmesh [00:36:45]: So the one they lost was around someone that was scraping public data that was on the public internet. And that particular company had not signed any terms of service or whatever. It's like, oh, I'm just taking data that's on, there was no, and so that's why they won. But now, you know, the question is around, can LinkedIn... I think they can. Like, when you use, as a user, you use LinkedIn, you are signing up for their terms of service. And if they say, well, this kind of use of your LinkedIn account that violates our terms of service, they can shut your account down, right? They can. And they, yeah, so, you know, we don't need to make this a discussion. By the way, I love the company, don't get me wrong. I'm an avid user of the product. You know, I've got... Yeah, I mean, you've got over a million followers on LinkedIn, I think. Yeah, I do. And I've known people there for a long, long time, right? And I have lots of respect. And I understand even where the mindset originally came from of this kind of members-first approach to, you know, a privacy-first. I sort of get that. But sometimes you sort of have to wonder, it's like, okay, well, that was 15, 20 years ago. There's likely some controlled ways to expose some data on some member's behalf and not just completely be a binary. It's like, no, thou shalt not have the data.swyx [00:37:54]: Well, just pay for sales navigator.Alessio [00:37:57]: Before we move to the next layer of instruction, anything else on MCP you mentioned? Let's move back and then I'll tie it back to MCPs.Dharmesh [00:38:05]: So I think the... Open this with agent. Okay, so I'll start with... Here's my kind of running thesis, is that as AI and agents evolve, which they're doing very, very quickly, we're going to look at them more and more. I don't like to anthropomorphize. We'll talk about why this is not that. Less as just like raw tools and more like teammates. They'll still be software. They should self-disclose as being software. I'm totally cool with that. But I think what's going to happen is that in the same way you might collaborate with a team member on Slack or Teams or whatever you use, you can imagine a series of agents that do specific things just like a team member might do, that you can delegate things to. You can collaborate. You can say, hey, can you take a look at this? Can you proofread that? Can you try this? You can... Whatever it happens to be. So I think it is... I will go so far as to say it's inevitable that we're going to have hybrid teams someday. And what I mean by hybrid teams... So back in the day, hybrid teams were, oh, well, you have some full-time employees and some contractors. Then it was like hybrid teams are some people that are in the office and some that are remote. That's the kind of form of hybrid. The next form of hybrid is like the carbon-based life forms and agents and AI and some form of software. So let's say we temporarily stipulate that I'm right about that over some time horizon that eventually we're going to have these kind of digitally hybrid teams. So if that's true, then the question you sort of ask yourself is that then what needs to exist in order for us to get the full value of that new model? It's like, okay, well... You sort of need to... It's like, okay, well, how do I... If I'm building a digital team, like, how do I... Just in the same way, if I'm interviewing for an engineer or a designer or a PM, whatever, it's like, well, that's why we have professional networks, right? It's like, oh, they have a presence on likely LinkedIn. I can go through that semi-structured, structured form, and I can see the experience of whatever, you know, self-disclosed. But, okay, well, agents are going to need that someday. And so I'm like, okay, well, this seems like a thread that's worth pulling on. That says, okay. So I... So agent.ai is out there. And it's LinkedIn for agents. It's LinkedIn for agents. It's a professional network for agents. And the more I pull on that thread, it's like, okay, well, if that's true, like, what happens, right? It's like, oh, well, they have a profile just like anyone else, just like a human would. It's going to be a graph underneath, just like a professional network would be. It's just that... And you can have its, you know, connections and follows, and agents should be able to post. That's maybe how they do release notes. Like, oh, I have this new version. Whatever they decide to post, it should just be able to... Behave as a node on the network of a professional network. As it turns out, the more I think about that and pull on that thread, the more and more things, like, start to make sense to me. So it may be more than just a pure professional network. So my original thought was, okay, well, it's a professional network and agents as they exist out there, which I think there's going to be more and more of, will kind of exist on this network and have the profile. But then, and this is always dangerous, I'm like, okay, I want to see a world where thousands of agents are out there in order for the... Because those digital employees, the digital workers don't exist yet in any meaningful way. And so then I'm like, oh, can I make that easier for, like... And so I have, as one does, it's like, oh, I'll build a low-code platform for building agents. How hard could that be, right? Like, very hard, as it turns out. But it's been fun. So now, agent.ai has 1.3 million users. 3,000 people have actually, you know, built some variation of an agent, sometimes just for their own personal productivity. About 1,000 of which have been published. And the reason this comes back to MCP for me, so imagine that and other networks, since I know agent.ai. So right now, we have an MCP server for agent.ai that exposes all the internally built agents that we have that do, like, super useful things. Like, you know, I have access to a Twitter API that I can subsidize the cost. And I can say, you know, if you're looking to build something for social media, these kinds of things, with a single API key, and it's all completely free right now, I'm funding it. That's a useful way for it to work. And then we have a developer to say, oh, I have this idea. I don't have to worry about open AI. I don't have to worry about, now, you know, this particular model is better. It has access to all the models with one key. And we proxy it kind of behind the scenes. And then expose it. So then we get this kind of community effect, right? That says, oh, well, someone else may have built an agent to do X. Like, I have an agent right now that I built for myself to do domain valuation for website domains because I'm obsessed with domains, right? And, like, there's no efficient market for domains. There's no Zillow for domains right now that tells you, oh, here are what houses in your neighborhood sold for. It's like, well, why doesn't that exist? We should be able to solve that problem. And, yes, you're still guessing. Fine. There should be some simple heuristic. So I built that. It's like, okay, well, let me go look for past transactions. You say, okay, I'm going to type in agent.ai, agent.com, whatever domain. What's it actually worth? I'm looking at buying it. It can go and say, oh, which is what it does. It's like, I'm going to go look at are there any published domain transactions recently that are similar, either use the same word, same top-level domain, whatever it is. And it comes back with an approximate value, and it comes back with its kind of rationale for why it picked the value and comparable transactions. Oh, by the way, this domain sold for published. Okay. So that agent now, let's say, existed on the web, on agent.ai. Then imagine someone else says, oh, you know, I want to build a brand-building agent for startups and entrepreneurs to come up with names for their startup. Like a common problem, every startup is like, ah, I don't know what to call it. And so they type in five random words that kind of define whatever their startup is. And you can do all manner of things, one of which is like, oh, well, I need to find the domain for it. What are possible choices? Now it's like, okay, well, it would be nice to know if there's an aftermarket price for it, if it's listed for sale. Awesome. Then imagine calling this valuation agent. It's like, okay, well, I want to find where the arbitrage is, where the agent valuation tool says this thing is worth $25,000. It's listed on GoDaddy for $5,000. It's close enough. Let's go do that. Right? And that's a kind of composition use case that in my future state. Thousands of agents on the network, all discoverable through something like MCP. And then you as a developer of agents have access to all these kind of Lego building blocks based on what you're trying to solve. Then you blend in orchestration, which is getting better and better with the reasoning models now. Just describe the problem that you have. Now, the next layer that we're all contending with is that how many tools can you actually give an LLM before the LLM breaks? That number used to be like 15 or 20 before you kind of started to vary dramatically. And so that's the thing I'm thinking about now. It's like, okay, if I want to... If I want to expose 1,000 of these agents to a given LLM, obviously I can't give it all 1,000. Is there some intermediate layer that says, based on your prompt, I'm going to make a best guess at which agents might be able to be helpful for this particular thing? Yeah.Alessio [00:44:37]: Yeah, like RAG for tools. Yep. I did build the Latent Space Researcher on agent.ai. Okay. Nice. Yeah, that seems like, you know, then there's going to be a Latent Space Scheduler. And then once I schedule a research, you know, and you build all of these things. By the way, my apologies for the user experience. You realize I'm an engineer. It's pretty good.swyx [00:44:56]: I think it's a normie-friendly thing. Yeah. That's your magic. HubSpot does the same thing.Alessio [00:45:01]: Yeah, just to like quickly run through it. You can basically create all these different steps. And these steps are like, you know, static versus like variable-driven things. How did you decide between this kind of like low-code-ish versus doing, you know, low-code with code backend versus like not exposing that at all? Any fun design decisions? Yeah. And this is, I think...Dharmesh [00:45:22]: I think lots of people are likely sitting in exactly my position right now, coming through the choosing between deterministic. Like if you're like in a business or building, you know, some sort of agentic thing, do you decide to do a deterministic thing? Or do you go non-deterministic and just let the alum handle it, right, with the reasoning models? The original idea and the reason I took the low-code stepwise, a very deterministic approach. A, the reasoning models did not exist at that time. That's thing number one. Thing number two is if you can get... If you know in your head... If you know in your head what the actual steps are to accomplish whatever goal, why would you leave that to chance? There's no upside. There's literally no upside. Just tell me, like, what steps do you need executed? So right now what I'm playing with... So one thing we haven't talked about yet, and people don't talk about UI and agents. Right now, the primary interaction model... Or they don't talk enough about it. I know some people have. But it's like, okay, so we're used to the chatbot back and forth. Fine. I get that. But I think we're going to move to a blend of... Some of those things are going to be synchronous as they are now. But some are going to be... Some are going to be async. It's just going to put it in a queue, just like... And this goes back to my... Man, I talk fast. But I have this... I only have one other speed. It's even faster. So imagine it's like if you're working... So back to my, oh, we're going to have these hybrid digital teams. Like, you would not go to a co-worker and say, I'm going to ask you to do this thing, and then sit there and wait for them to go do it. Like, that's not how the world works. So it's nice to be able to just, like, hand something off to someone. It's like, okay, well, maybe I expect a response in an hour or a day or something like that.Dharmesh [00:46:52]: In terms of when things need to happen. So the UI around agents. So if you look at the output of agent.ai agents right now, they are the simplest possible manifestation of a UI, right? That says, oh, we have inputs of, like, four different types. Like, we've got a dropdown, we've got multi-select, all the things. It's like back in HTML, the original HTML 1.0 days, right? Like, you're the smallest possible set of primitives for a UI. And it just says, okay, because we need to collect some information from the user, and then we go do steps and do things. And generate some output in HTML or markup are the two primary examples. So the thing I've been asking myself, if I keep going down that path. So people ask me, I get requests all the time. It's like, oh, can you make the UI sort of boring? I need to be able to do this, right? And if I keep pulling on that, it's like, okay, well, now I've built an entire UI builder thing. Where does this end? And so I think the right answer, and this is what I'm going to be backcoding once I get done here, is around injecting a code generation UI generation into, the agent.ai flow, right? As a builder, you're like, okay, I'm going to describe the thing that I want, much like you would do in a vibe coding world. But instead of generating the entire app, it's going to generate the UI that exists at some point in either that deterministic flow or something like that. It says, oh, here's the thing I'm trying to do. Go generate the UI for me. And I can go through some iterations. And what I think of it as a, so it's like, I'm going to generate the code, generate the code, tweak it, go through this kind of prompt style, like we do with vibe coding now. And at some point, I'm going to be happy with it. And I'm going to hit save. And that's going to become the action in that particular step. It's like a caching of the generated code that I can then, like incur any inference time costs. It's just the actual code at that point.Alessio [00:48:29]: Yeah, I invested in a company called E2B, which does code sandbox. And they powered the LM arena web arena. So it's basically the, just like you do LMS, like text to text, they do the same for like UI generation. So if you're asking a model, how do you do it? But yeah, I think that's kind of where.Dharmesh [00:48:45]: That's the thing I'm really fascinated by. So the early LLM, you know, we're understandably, but laughably bad at simple arithmetic, right? That's the thing like my wife, Normies would ask us, like, you call this AI, like it can't, my son would be like, it's just stupid. It can't even do like simple arithmetic. And then like we've discovered over time that, and there's a reason for this, right? It's like, it's a large, there's, you know, the word language is in there for a reason in terms of what it's been trained on. It's not meant to do math, but now it's like, okay, well, the fact that it has access to a Python interpreter that I can actually call at runtime, that solves an entire body of problems that it wasn't trained to do. And it's basically a form of delegation. And so the thought that's kind of rattling around in my head is that that's great. So it's, it's like took the arithmetic problem and took it first. Now, like anything that's solvable through a relatively concrete Python program, it's able to do a bunch of things that I couldn't do before. Can we get to the same place with UI? I don't know what the future of UI looks like in a agentic AI world, but maybe let the LLM handle it, but not in the classic sense. Maybe it generates it on the fly, or maybe we go through some iterations and hit cache or something like that. So it's a little bit more predictable. Uh, I don't know, but yeah.Alessio [00:49:48]: And especially when is the human supposed to intervene? So, especially if you're composing them, most of them should not have a UI because then they're just web hooking to somewhere else. I just want to touch back. I don't know if you have more comments on this.swyx [00:50:01]: I was just going to ask when you, you said you got, you're going to go back to code. What

Come faccio a rendere più visibili i miei contenuti su Web e Social Media? Come funziona l'algoritmo che regola la diffusione dei contenuti sui social? Esistono trucchi o strategie segrete per avere successo?Scopriamolo insieme ripercorrendo l'evoluzione delle logiche di visibilità dei contenuti digitali, dal Page Rank di Google fino alle strategie per TikTok e Instagram, come le ha descritte Fabrizio Politi ( https://www.youtube.com/watch?v=21fnbw0EctM ).

This week I welcome on the show two of the most important technologists ever, in any field.Jeff Dean is Google's Chief Scientist, and through 25 years at the company, has worked on basically the most transformative systems in modern computing: from MapReduce, BigTable, Tensorflow, AlphaChip, to Gemini.Noam Shazeer invented or co-invented all the main architectures and techniques that are used for modern LLMs: from the Transformer itself, to Mixture of Experts, to Mesh Tensorflow, to Gemini and many other things.We talk about their 25 years at Google, going from PageRank to MapReduce to the Transformer to MoEs to AlphaChip – and maybe soon to ASI.My favorite part was Jeff's vision for Pathways, Google's grand plan for a mutually-reinforcing loop of hardware and algorithmic design and for going past autoregression. That culminates in us imagining *all* of Google-the-company, going through one huge MoE model.And Noam just bites every bullet: 100x world GDP soon; let's get a million automated researchers running in the Google datacenter; living to see the year 3000.SponsorsScale partners with major AI labs like Meta, Google Deepmind, and OpenAI. Through Scale's Data Foundry, labs get access to high-quality data to fuel post-training, including advanced reasoning capabilities. If you're an AI researcher or engineer, learn about how Scale's Data Foundry and research lab, SEAL, can help you go beyond the current frontier at scale.com/dwarkesh.Curious how Jane Street teaches their new traders? They use Figgie, a rapid-fire card game that simulates the most exciting parts of markets and trading. It's become so popular that Jane Street hosts an inter-office Figgie championship every year. Download from the app store or play on your desktop at figgie.com.Meter wants to radically improve the digital world we take for granted. They're developing a foundation model that automates network management end-to-end. To do this, they just announced a long-term partnership with Microsoft for tens of thousands of GPUs, and they're recruiting a world class AI research team. To learn more, go to meter.com/dwarkesh.Advertisers:To sponsor a future episode, visit: dwarkeshpatel.com/p/advertise.Timestamps00:00:00 - Intro00:02:44 - Joining Google in 199900:05:36 - Future of Moore's Law00:10:21 - Future TPUs00:13:13 - Jeff's undergrad thesis: parallel backprop00:15:10 - LLMs in 200700:23:07 - “Holy s**t” moments00:29:46 - AI fulfills Google's original mission00:34:19 - Doing Search in-context00:38:32 - The internal coding model00:39:49 - What will 2027 models do?00:46:00 - A new architecture every day?00:49:21 - Automated chip design and intelligence explosion00:57:31 - Future of inference scaling01:03:56 - Already doing multi-datacenter runs01:22:33 - Debugging at scale01:26:05 - Fast takeoff and superalignment01:34:40 - A million evil Jeff Deans01:38:16 - Fun times at Google01:41:50 - World compute demand in 203001:48:21 - Getting back to modularity01:59:13 - Keeping a giga-MoE in-memory02:04:09 - All of Google in one model02:12:43 - What's missing from distillation02:18:03 - Open research, pros and cons02:24:54 - Going the distance Get full access to Dwarkesh Podcast at www.dwarkeshpatel.com/subscribe

Skalierung und verteilte Berechnungen: Sind mehr CPUs wirklich immer schneller?Stell dir vor, du bist Softwareentwickler*in und jeder spricht von Skalierung und verteilten Systemen. Doch wie effizient sind diese eigentlich wirklich? Heißt mehr Rechenpower gleich schnellere Ergebnisse?In dieser Episode werfen wir einen Blick auf ein wissenschaftliches Paper, das behauptet, die wahre Leistung von verteilten Systemen kritisch zu hinterfragen. Wir diskutieren, ab wann es sich lohnt, mehr Ressourcen einzusetzen, und was es mit der mysteriösen Metrik COST (ausgesprochen Configuration that Outperforms a Single Thread) auf sich hat. Hör rein, wenn du wissen willst, ob Single-Threaded Algorithmen in vielen Fällen die bessere Wahl sind.Bonus: Ggf. machen nicht alle Wissenschaftler so wissenschaftliche Arbeit.Unsere aktuellen Werbepartner findest du auf https://engineeringkiosk.dev/partnersDas schnelle Feedback zur Episode:

Applications close Monday for the NYC AI Engineer Summit focusing on AI Leadership and Agent Engineering! If you applied, invites should be rolling out shortly.The search landscape is experiencing a fundamental shift. Google built a >$2T company with the “10 blue links” experience, driven by PageRank as the core innovation for ranking. This was a big improvement from the previous directory-based experiences of AltaVista and Yahoo. Almost 4 decades later, Google is now stuck in this links-based experience, especially from a business model perspective. This legacy architecture creates fundamental constraints:* Must return results in ~400 milliseconds* Required to maintain comprehensive web coverage* Tied to keyword-based matching algorithms* Cost structures optimized for traditional indexingAs we move from the era of links to the era of answers, the way search works is changing. You're not showing a user links, but the goal is to provide context to an LLM. This means moving from keyword based search to more semantic understanding of the content:The link prediction objective can be seen as like a neural PageRank because what you're doing is you're predicting the links people share... but it's more powerful than PageRank. It's strictly more powerful because people might refer to that Paul Graham fundraising essay in like a thousand different ways. And so our model learns all the different ways.All of this is now powered by a $5M cluster with 144 H200s:This architectural choice enables entirely new search capabilities:* Comprehensive result sets instead of approximations* Deep semantic understanding of queries* Ability to process complex, natural language requestsAs search becomes more complex, time to results becomes a variable:People think of searches as like, oh, it takes 500 milliseconds because we've been conditioned... But what if searches can take like a minute or 10 minutes or a whole day, what can you then do?Unlike traditional search engines' fixed-cost indexing, Exa employs a hybrid approach:* Front-loaded compute for indexing and embeddings* Variable inference costs based on query complexity* Mix of owned infrastructure ($5M H200 cluster) and cloud resourcesExa sees a lot of competition from products like Perplexity and ChatGPT Search which layer AI on top of traditional search backends, but Exa is betting that true innovation requires rethinking search from the ground up. For example, the recently launched Websets, a way to turn searches into structured output in grid format, allowing you to create lists and databases out of web pages. The company raised a $17M Series A to build towards this mission, so keep an eye out for them in 2025. Chapters* 00:00:00 Introductions* 00:01:12 ExaAI's initial pitch and concept* 00:02:33 Will's background at SpaceX and Zoox* 00:03:45 Evolution of ExaAI (formerly Metaphor Systems)* 00:05:38 Exa's link prediction technology* 00:09:20 Meaning of the name "Exa"* 00:10:36 ExaAI's new product launch and capabilities* 00:13:33 Compute budgets and variable compute products* 00:14:43 Websets as a B2B offering* 00:19:28 How do you build a search engine?* 00:22:43 What is Neural PageRank?* 00:27:58 Exa use cases * 00:35:00 Auto-prompting* 00:38:42 Building agentic search* 00:44:19 Is o1 on the path to AGI?* 00:49:59 Company culture and nap pods* 00:54:52 Economics of AI search and the future of search technologyFull YouTube TranscriptPlease like and subscribe!Show Notes* ExaAI* Web Search Product* Websets* Series A Announcement* Exa Nap Pods* Perplexity AI* Character.AITranscriptAlessio [00:00:00]: Hey, everyone. Welcome to the Latent Space podcast. This is Alessio, partner and CTO at Decibel Partners, and I'm joined by my co-host Swyx, founder of Smol.ai.Swyx [00:00:10]: Hey, and today we're in the studio with my good friend and former landlord, Will Bryk. Roommate. How you doing? Will, you're now CEO co-founder of ExaAI, used to be Metaphor Systems. What's your background, your story?Will [00:00:30]: Yeah, sure. So, yeah, I'm CEO of Exa. I've been doing it for three years. I guess I've always been interested in search, whether I knew it or not. Like, since I was a kid, I've always been interested in, like, high-quality information. And, like, you know, even in high school, wanted to improve the way we get information from news. And then in college, built a mini search engine. And then with Exa, like, you know, it's kind of like fulfilling the dream of actually being able to solve all the information needs I wanted as a kid. Yeah, I guess. I would say my entire life has kind of been rotating around this problem, which is pretty cool. Yeah.Swyx [00:00:50]: What'd you enter YC with?Will [00:00:53]: We entered YC with, uh, we are better than Google. Like, Google 2.0.Swyx [00:01:12]: What makes you say that? Like, that's so audacious to come out of the box with.Will [00:01:16]: Yeah, okay, so you have to remember the time. This was summer 2021. And, uh, GPT-3 had come out. Like, here was this magical thing that you could talk to, you could enter a whole paragraph, and it understands what you mean, understands the subtlety of your language. And then there was Google. Uh, which felt like it hadn't changed in a decade, uh, because it really hadn't. And it, like, you would give it a simple query, like, I don't know, uh, shirts without stripes, and it would give you a bunch of results for the shirts with stripes. And so, like, Google could barely understand you, and GBD3 could. And the theory was, what if you could make a search engine that actually understood you? What if you could apply the insights from LLMs to a search engine? And it's really been the same idea ever since. And we're actually a lot closer now, uh, to doing that. Yeah.Alessio [00:01:55]: Did you have any trouble making people believe? Obviously, there's the same element. I mean, YC overlap, was YC pretty AI forward, even 2021, or?Will [00:02:03]: It's nothing like it is today. But, um, uh, there were a few AI companies, but, uh, we were definitely, like, bold. And I think people, VCs generally like boldness, and we definitely had some AI background, and we had a working demo. So there was evidence that we could build something that was going to work. But yeah, I think, like, the fundamentals were there. I think people at the time were talking about how, you know, Google was failing in a lot of ways. And so there was a bit of conversation about it, but AI was not a big, big thing at the time. Yeah. Yeah.Alessio [00:02:33]: Before we jump into Exa, any fun background stories? I know you interned at SpaceX, any Elon, uh, stories? I know you were at Zoox as well, you know, kind of like robotics at Harvard. Any stuff that you saw early that you thought was going to get solved that maybe it's not solved today?Will [00:02:48]: Oh yeah. I mean, lots of things like that. Like, uh, I never really learned how to drive because I believed Elon that self-driving cars would happen. It did happen. And I take them every night to get home. But it took like 10 more years than I thought. Do you still not know how to drive? I know how to drive now. I learned it like two years ago. That would have been great to like, just, you know, Yeah, yeah, yeah. You know? Um, I was obsessed with Elon. Yeah. I mean, I worked at SpaceX because I really just wanted to work at one of his companies. And I remember they had a rule, like interns cannot touch Elon. And, um, that rule actually influenced my actions.Swyx [00:03:18]: Is it, can Elon touch interns? Ooh, like physically?Will [00:03:22]: Or like talk? Physically, physically, yeah, yeah, yeah, yeah. Okay, interesting. He's changed a lot, but, um, I mean, his companies are amazing. Um,Swyx [00:03:28]: What if you beat him at Diablo 2, Diablo 4, you know, like, Ah, maybe.Alessio [00:03:34]: I want to jump into, I know there's a lot of backstory used to be called metaphor system. So, um, and it, you've always been kind of like a prominent company, maybe at least RAI circles in the NSF.Swyx [00:03:45]: I'm actually curious how Metaphor got its initial aura. You launched with like, very little. We launched very little. Like there was, there was this like big splash image of like, this is Aurora or something. Yeah. Right. And then I was like, okay, what this thing, like the vibes are good, but I don't know what it is. And I think, I think it was much more sort of maybe consumer facing than what you are today. Would you say that's true?Will [00:04:06]: No, it's always been about building a better search algorithm, like search, like, just like the vision has always been perfect search. And if you do that, uh, we will figure out the downstream use cases later. It started on this fundamental belief that you could have perfect search over the web and we could talk about what that means. And like the initial thing we released was really just like our first search engine, like trying to get it out there. Kind of like, you know, an open source. So when OpenAI released, uh, ChachBt, like they didn't, I don't know how, how much of a game plan they had. They kind of just wanted to get something out there.Swyx [00:04:33]: Spooky research preview.Will [00:04:34]: Yeah, exactly. And it kind of morphed from a research company to a product company at that point. And I think similarly for us, like we were research, we started as a research endeavor with a, you know, clear eyes that like, if we succeed, it will be a massive business to make out of it. And that's kind of basically what happened. I think there are actually a lot of parallels to, of w between Exa and OpenAI. I often say we're the OpenAI of search. Um, because. Because we're a research company, we're a research startup that does like fundamental research into, uh, making like AGI for search in a, in a way. Uh, and then we have all these like, uh, business products that come out of that.Swyx [00:05:08]: Interesting. I want to ask a little bit more about Metaforesight and then we can go full Exa. When I first met you, which was really funny, cause like literally I stayed in your house in a very historic, uh, Hayes, Hayes Valley place. You said you were building sort of like link prediction foundation model, and I think there's still a lot of foundation model work. I mean, within Exa today, but what does that even mean? I cannot be the only person confused by that because like there's a limited vocabulary or tokens you're telling me, like the tokens are the links or, you know, like it's not, it's not clear. Yeah.Will [00:05:38]: Uh, what we meant by link prediction is that you are literally predicting, like given some texts, you're predicting the links that follow. Yes. That refers to like, it's how we describe the training procedure, which is that we find links on the web. Uh, we take the text surrounding the link. And then we predict. Which link follows you, like, uh, you know, similar to transformers where, uh, you're trying to predict the next token here, you're trying to predict the next link. And so you kind of like hide the link from the transformer. So if someone writes, you know, imagine some article where someone says, Hey, check out this really cool aerospace startup. And they, they say spacex.com afterwards, uh, we hide the spacex.com and ask the model, like what link came next. And by doing that many, many times, you know, billions of times, you could actually build a search engine out of that because then, uh, at query time at search time. Uh, you type in, uh, a query that's like really cool aerospace startup and the model will then try to predict what are the most likely links. So there's a lot of analogs to transformers, but like to actually make this work, it does require like a different architecture than, but it's transformer inspired. Yeah.Alessio [00:06:41]: What's the design decision between doing that versus extracting the link and the description and then embedding the description and then using, um, yeah. What do you need to predict the URL versus like just describing, because you're kind of do a similar thing in a way. Right. It's kind of like based on this description, it was like the closest link for it. So one thing is like predicting the link. The other approach is like I extract the link and the description, and then based on the query, I searched the closest description to it more. Yeah.Will [00:07:09]: That, that, by the way, that is, that is the link refers here to a document. It's not, I think one confusing thing is it's not, you're not actually predicting the URL, the URL itself that would require like the, the system to have memorized URLs. You're actually like getting the actual document, a more accurate name could be document prediction. I see. This was the initial like base model that Exo was trained on, but we've moved beyond that similar to like how, you know, uh, to train a really good like language model, you might start with this like self-supervised objective of predicting the next token and then, uh, just from random stuff on the web. But then you, you want to, uh, add a bunch of like synthetic data and like supervised fine tuning, um, stuff like that to make it really like controllable and robust. Yeah.Alessio [00:07:48]: Yeah. We just have flow from Lindy and, uh, their Lindy started to like hallucinate recrolling YouTube links instead of like, uh, something. Yeah. Support guide. So. Oh, interesting. Yeah.Swyx [00:07:57]: So round about January, you announced your series A and renamed to Exo. I didn't like the name at the, at the initial, but it's grown on me. I liked metaphor, but apparently people can spell metaphor. What would you say are the major components of Exo today? Right? Like, I feel like it used to be very model heavy. Then at the AI engineer conference, Shreyas gave a really good talk on the vector database that you guys have. What are the other major moving parts of Exo? Okay.Will [00:08:23]: So Exo overall is a search engine. Yeah. We're trying to make it like a perfect search engine. And to do that, you have to build lots of, and we're doing it from scratch, right? So to do that, you have to build lots of different. The crawler. Yeah. You have to crawl a bunch of the web. First of all, you have to find the URLs to crawl. Uh, it's connected to the crawler, but yeah, you find URLs, you crawl those URLs. Then you have to process them with some, you know, it could be an embedding model. It could be something more complex, but you need to take, you know, or like, you know, in the past it was like a keyword inverted index. Like you would process all these documents you gather into some processed index, and then you have to serve that. Uh, you had high throughput at low latency. And so that, and that's like the vector database. And so it's like the crawling system, the AI processing system, and then the serving system. Those are all like, you know, teams of like hundreds, maybe thousands of people at Google. Um, but for us, it's like one or two people each typically, but yeah.Alessio [00:09:13]: Can you explain the meaning of, uh, Exo, just the story 10 to the 16th, uh, 18, 18.Will [00:09:20]: Yeah, yeah, yeah, sure. So. Exo means 10 to the 18th, which is in stark contrast to. To Google, which is 10 to the hundredth. Uh, we actually have these like awesome shirts that are like 10th to 18th is greater than 10th to the hundredth. Yeah, it's great. And it's great because it's provocative. It's like every engineer in Silicon Valley is like, what? No, it's not true. Um, like, yeah. And, uh, and then you, you ask them, okay, what does it actually mean? And like the creative ones will, will recognize it. But yeah, I mean, 10 to the 18th is better than 10 to the hundredth when it comes to search, because with search, you want like the actual list of, of things that match what you're asking for. You don't want like the whole web. You want to basically with search filter, the, like everything that humanity has ever created to exactly what you want. And so the idea is like smaller is better there. You want like the best 10th to the 18th and not the 10th to the hundredth. I'm like, one way to say this is like, you know how Google often says at the top, uh, like, you know, 30 million results found. And it's like crazy. Cause you're looking for like the first startups in San Francisco that work on hardware or something. And like, they're not 30 million results like that. What you want is like 325 results found. And those are all the results. That's what you really want with search. And that's, that's our vision. It's like, it just gives you. Perfectly what you asked for.Swyx [00:10:24]: We're recording this ahead of your launch. Uh, we haven't released, we haven't figured out the, the, the name of the launch yet, but what is the product that you're launching? I guess now that we're coinciding this podcast with. Yeah.Will [00:10:36]: So we've basically developed the next version of Exa, which is the ability to get a near perfect list of results of whatever you want. And what that means is you can make a complex query now to Exa, for example, startups working on hardware in SF, and then just get a huge list of all the things that match. And, you know, our goal is if there are 325 startups that match that we find you all of them. And this is just like, there's just like a new experience that's never existed before. It's really like, I don't know how you would go about that right now with current tools and you can apply this same type of like technology to anything. Like, let's say you want, uh, you want to find all the blog posts that talk about Alessio's podcast, um, that have come out in the past year. That is 30 million results. Yeah. Right.Will [00:11:24]: But that, I mean, that would, I'm sure that would be extremely useful to you guys. And like, I don't really know how you would get that full comprehensive list.Swyx [00:11:29]: I just like, how do you, well, there's so many questions with regards to how do you know it's complete, right? Cause you're saying there's only 30 million, 325, whatever. And then how do you do the semantic understanding that it might take, right? So working in hardware, like I might not use the words hardware. I might use the words robotics. I might use the words wearables. I might use like whatever. Yes. So yeah, just tell us more. Yeah. Yeah. Sure. Sure.Will [00:11:53]: So one aspect of this, it's a little subjective. So like certainly providing, you know, at some point we'll provide parameters to the user to like, you know, some sort of threshold to like, uh, gauge like, okay, like this is a cutoff. Like, this is actually not what I mean, because sometimes it's subjective and there needs to be a feedback loop. Like, oh, like it might give you like a few examples and you say, yeah, exactly. And so like, you're, you're kind of like creating a classifier on the fly, but like, that's ultimately how you solve the problem. So the subject, there's a subjectivity problem and then there's a comprehensiveness problem. Those are two different problems. So. Yeah. So you have the comprehensiveness problem. What you basically have to do is you have to put more compute into the query, into the search until you get the full comprehensiveness. Yeah. And I think there's an interesting point here, which is that not all queries are made equal. Some queries just like this blog post one might require scanning, like scavenging, like throughout the whole web in a way that just, just simply requires more compute. You know, at some point there's some amount of compute where you will just be comprehensive. You could imagine, for example, running GPT-4 over the internet. You could imagine running GPT-4 over the entire web and saying like, is this a blog post about Alessio's podcast, like, is this a blog post about Alessio's podcast? And then that would work, right? It would take, you know, a year, maybe cost like a million dollars, but, or many more, but, um, it would work. Uh, the point is that like, given sufficient compute, you can solve the query. And so it's really a question of like, how comprehensive do you want it given your compute budget? I think it's very similar to O1, by the way. And one way of thinking about what we built is like O1 for search, uh, because O1 is all about like, you know, some, some, some questions require more compute than others, and we'll put as much compute into the question as we need to solve it. So similarly with our search, we will put as much compute into the query in order to get comprehensiveness. Yeah.Swyx [00:13:33]: Does that mean you have like some kind of compute budget that I can specify? Yes. Yes. Okay. And like, what are the upper and lower bounds?Will [00:13:42]: Yeah, there's something we're still figuring out. I think like, like everyone is a new paradigm of like variable compute products. Yeah. How do you specify the amount of compute? Like what happens when you. Run out? Do you just like, ah, do you, can you like keep going with it? Like, do you just put in more credits to get more, um, for some, like this can get complex at like the really large compute queries. And like, one thing we do is we give you a preview of what you're going to get, and then you could then spin up like a much larger job, uh, to get like way more results. But yes, there is some compute limit, um, at, at least right now. Yeah. People think of searches as like, oh, it takes 500 milliseconds because we've been conditioned, uh, to have search that takes 500 milliseconds. But like search engines like Google, right. No matter how complex your query to Google, it will take like, you know, roughly 400 milliseconds. But what if searches can take like a minute or 10 minutes or a whole day, what can you then do? And you can do very powerful things. Um, you know, you can imagine, you know, writing a search, going and get a cup of coffee, coming back and you have a perfect list. Like that's okay for a lot of use cases. Yeah.Alessio [00:14:43]: Yeah. I mean, the use case closest to me is venture capital, right? So, uh, no, I mean, eight years ago, I built one of the first like data driven sourcing platforms. So we were. You look at GitHub, Twitter, Product Hunt, all these things, look at interesting things, evaluate them. If you think about some jobs that people have, it's like literally just make a list. If you're like an analyst at a venture firm, your job is to make a list of interesting companies. And then you reach out to them. How do you think about being infrastructure versus like a product you could say, Hey, this is like a product to find companies. This is a product to find things versus like offering more as a blank canvas that people can build on top of. Oh, right. Right.Will [00:15:20]: Uh, we are. We are a search infrastructure company. So we want people to build, uh, on top of us, uh, build amazing products on top of us. But with this one, we try to build something that makes it really easy for users to just log in, put a few, you know, put some credits in and just get like amazing results right away and not have to wait to build some API integration. So we're kind of doing both. Uh, we, we want, we want people to integrate this into all their applications at the same time. We want to just make it really easy to use very similar again to open AI. Like they'll have, they have an API, but they also have. Like a ChatGPT interface so that you could, it's really easy to use, but you could also build it in your applications. Yeah.Alessio [00:15:56]: I'm still trying to wrap my head around a lot of the implications. So, so many businesses run on like information arbitrage, you know, like I know this thing that you don't, especially in investment and financial services. So yeah, now all of a sudden you have these tools for like, oh, actually everybody can get the same information at the same time, the same quality level as an API call. You know, it just kind of changes a lot of things. Yeah.Will [00:16:19]: I think, I think what we're grappling with here. What, what you're just thinking about is like, what is the world like if knowledge is kind of solved, if like any knowledge request you want is just like right there on your computer, it's kind of different from when intelligence is solved. There's like a good, I've written before about like a different super intelligence, super knowledge. Yeah. Like I think that the, the distinction between intelligence and knowledge is actually a pretty good one. They're definitely connected and related in all sorts of ways, but there is a distinction. You could have a world and we are going to have this world where you have like GP five level systems and beyond that could like answer any complex request. Um, unless it requires some. Like, if you say like, uh, you know, give me a list of all the PhDs in New York city who, I don't know, have thought about search before. And even though this, this super intelligence is going to be like, I can't find it on Google, right. Which is kind of crazy. Like we're literally going to have like super intelligences that are using Google. And so if Google can't find them information, there's nothing they could do. They can't find it. So, but if you also have a super knowledge system where it's like, you know, I'm calling this term super knowledge where you just get whatever knowledge you want, then you can pair with a super intelligence system. And then the super intelligence can, we'll never. Be blocked by lack of knowledge.Alessio [00:17:23]: Yeah. You told me this, uh, when we had lunch, I forget how it came out, but we were talking about AGI and whatnot. And you were like, even AGI is going to need search. Yeah.Swyx [00:17:32]: Yeah. Right. Yeah. Um, so we're actually referencing a blog post that you wrote super intelligence and super knowledge. Uh, so I would refer people to that. And this is actually a discussion we've had on the podcast a couple of times. Um, there's so much of model weights that are just memorizing facts. Some of the, some of those might be outdated. Some of them are incomplete or not. Yeah. So like you just need search. So I do wonder, like, is there a maximum language model size that will be the intelligence layer and then the rest is just search, right? Like maybe we should just always use search. And then that sort of workhorse model is just like, and it like, like, like one B or three B parameter model that just drives everything. Yes.Will [00:18:13]: I believe this is a much more optimal system to have a smaller LM. That's really just like an intelligence module. And it makes a call to a search. Tool that's way more efficient because if, okay, I mean the, the opposite of that would be like the LM is so big that can memorize the whole web. That would be like way, but you know, it's not practical at all. I don't, it's not possible to train that at least right now. And Carpathy has actually written about this, how like he could, he could see models moving more and more towards like intelligence modules using various tools. Yeah.Swyx [00:18:39]: So for listeners, that's the, that was him on the no priors podcast. And for us, we talked about this and the, on the Shin Yu and Harrison chase podcasts. I'm doing search in my head. I told you 30 million results. I forgot about our neural link integration. Self-hosted exit.Will [00:18:54]: Yeah. Yeah. No, I do see that that is a much more, much more efficient world. Yeah. I mean, you could also have GB four level systems calling search, but it's just because of the cost of inference. It's just better to have a very efficient search tool and a very efficient LM and they're built for different things. Yeah.Swyx [00:19:09]: I'm just kind of curious. Like it is still something so audacious that I don't want to elide, which is you're, you're, you're building a search engine. Where do you start? How do you, like, are there any reference papers or implementation? That would really influence your thinking, anything like that? Because I don't even know where to start apart from just crawl a bunch of s**t, but there's gotta be more insight than that.Will [00:19:28]: I mean, yeah, there's more insight, but I'm always surprised by like, if you have a group of people who are really focused on solving a problem, um, with the tools today, like there's some in, in software, like there are all sorts of creative solutions that just haven't been thought of before, particularly in the information retrieval field. Yeah. I think a lot of the techniques are just very old, frankly. Like I know how Google and Bing work and. They're just not using new methods. There are all sorts of reasons for that. Like one, like Google has to be comprehensive over the web. So they're, and they have to return in 400 milliseconds. And those two things combined means they are kind of limit and it can't cost too much. They're kind of limited in, uh, what kinds of algorithms they could even deploy at scale. So they end up using like a limited keyword based algorithm. Also like Google was built in a time where like in, you know, in 1998, where we didn't have LMS, we didn't have embeddings. And so they never thought to build those things. And so now they have this like gigantic system that is built on old technology. Yeah. And so a lot of the information retrieval field we found just like thinks in terms of that framework. Yeah. Whereas we came in as like newcomers just thinking like, okay, there here's GB three. It's magical. Obviously we're going to build search that is using that technology. And we never even thought about using keywords really ever. Uh, like we were neural all the way we're building an end to end neural search engine. And just that whole framing just makes us ask different questions, like pursue different lines of work. And there's just a lot of low hanging fruit because no one else is thinking about it. We're just on the frontier of neural search. We just are, um, for, for at web scale, um, because there's just not a lot of people thinking that way about it.Swyx [00:20:57]: Yeah. Maybe let's spell this out since, uh, we're already on this topic, elephants in the room are Perplexity and SearchGPT. That's the, I think that it's all, it's no longer called SearchGPT. I think they call it ChatGPT Search. How would you contrast your approaches to them based on what we know of how they work and yeah, just any, anything in that, in that area? Yeah.Will [00:21:15]: So these systems, there are a few of them now, uh, they basically rely on like traditional search engines like Google or Bing, and then they combine them with like LLMs at the end to, you know, output some power graphics, uh, answering your question. So they like search GPT perplexity. I think they have their own crawlers. No. So there's this important distinction between like having your own search system and like having your own cache of the web. Like for example, so you could create, you could crawl a bunch of the web. Imagine you crawl a hundred billion URLs, and then you create a key value store of like mapping from URL to the document that is technically called an index, but it's not a search algorithm. So then to actually like, when you make a query to search GPT, for example, what is it actually doing it? Let's say it's, it's, it could, it's using the Bing API, uh, getting a list of results and then it could go, it has this cache of like all the contents of those results and then could like bring in the cache, like the index cache, but it's not actually like, it's not like they've built a search engine from scratch over, you know, hundreds of billions of pages. It's like, does that distinction clear? It's like, yeah, you could have like a mapping from URL to documents, but then rely on traditional search engines to actually get the list of results because it's a very hard problem to take. It's not hard. It's not hard to use DynamoDB and, and, and map URLs to documents. It's a very hard problem to take a hundred billion or more documents and given a query, like instantly get the list of results that match. That's a much harder problem that very few entities on, in, on the planet have done. Like there's Google, there's Bing, uh, you know, there's Yandex, but you know, there are not that many companies that are, that are crazy enough to actually build their search engine from scratch when you could just use traditional search APIs.Alessio [00:22:43]: So Google had PageRank as like the big thing. Is there a LLM equivalent or like any. Stuff that you're working on that you want to highlight?Will [00:22:51]: The link prediction objective can be seen as like a neural PageRank because what you're doing is you're predicting the links people share. And so if everyone is sharing some Paul Graham essay about fundraising, then like our model is more likely to predict it. So like inherent in our training objective is this, uh, a sense of like high canonicity and like high quality, but it's more powerful than PageRank. It's strictly more powerful because people might refer to that Paul Graham fundraising essay in like a thousand different ways. And so our model learns all the different ways. That someone refers that Paul Graham, I say, while also learning how important that Paul Graham essay is. Um, so it's like, it's like PageRank on steroids kind of thing. Yeah.Alessio [00:23:26]: I think to me, that's the most interesting thing about search today, like with Google and whatnot, it's like, it's mostly like domain authority. So like if you get back playing, like if you search any AI term, you get this like SEO slop websites with like a bunch of things in them. So this is interesting, but then how do you think about more timeless maybe content? So if you think about, yeah. You know, maybe the founder mode essay, right. It gets shared by like a lot of people, but then you might have a lot of other essays that are also good, but they just don't really get a lot of traction. Even though maybe the people that share them are high quality. How do you kind of solve that thing when you don't have the people authority, so to speak of who's sharing, whether or not they're worth kind of like bumping up? Yeah.Will [00:24:10]: I mean, you do have a lot of control over the training data, so you could like make sure that the training data contains like high quality sources so that, okay. Like if you, if you're. Training data, I mean, it's very similar to like language, language model training. Like if you train on like a bunch of crap, your prediction will be crap. Our model will match the training distribution is trained on. And so we could like, there are lots of ways to tweak the training data to refer to high quality content that we want. Yeah. I would say also this, like this slop that is returned by, by traditional search engines, like Google and Bing, you have the slop is then, uh, transferred into the, these LLMs in like a search GBT or, you know, our other systems like that. Like if slop comes in, slop will go out. And so, yeah, that's another answer to how we're different is like, we're not like traditional search engines. We want to give like the highest quality results and like have full control over whatever you want. If you don't want slop, you get that. And then if you put an LM on top of that, which our customers do, then you just get higher quality results or high quality output.Alessio [00:25:06]: And I use Excel search very often and it's very good. Especially.Swyx [00:25:09]: Wave uses it too.Alessio [00:25:10]: Yeah. Yeah. Yeah. Yeah. Yeah. Like the slop is everywhere, especially when it comes to AI, when it comes to investment. When it comes to all of these things for like, it's valuable to be at the top. And this problem is only going to get worse because. Yeah, no, it's totally. What else is in the toolkit? So you have search API, you have ExaSearch, kind of like the web version. Now you have the list builder. I think you also have web scraping. Maybe just touch on that. Like, I guess maybe people, they want to search and then they want to scrape. Right. So is that kind of the use case that people have? Yeah.Will [00:25:41]: A lot of our customers, they don't just want, because they're building AI applications on top of Exa, they don't just want a list of URLs. They actually want. Like the full content, like cleans, parsed. Markdown. Markdown, maybe chunked, whatever they want, we'll give it to them. And so that's been like huge for customers. Just like getting the URLs and instantly getting the content for each URL is like, and you can do this for 10 or 100 or 1,000 URLs, wherever you want. That's very powerful.Swyx [00:26:05]: Yeah. I think this is the first thing I asked you for when I tried using Exa.Will [00:26:09]: Funny story is like when I built the first version of Exa, it's like, we just happened to store the content. Yes. Like the first 1,024 tokens. Because I just kind of like kept it because I thought of, you know, I don't know why. Really for debugging purposes. And so then when people started asking for content, it was actually pretty easy to serve it. But then, and then we did that, like Exa took off. So the computer's content was so useful. So that was kind of cool.Swyx [00:26:30]: It is. I would say there are other players like Gina, I think is in this space. Firecrawl is in this space. There's a bunch of scraper companies. And obviously scraper is just one part of your stack, but you might as well offer it since you already do it.Will [00:26:43]: Yeah, it makes sense. It's just easy to have an all-in-one solution. And like. We are, you know, building the best scraper in the world. So scraping is a hard problem and it's easy to get like, you know, a good scraper. It's very hard to get a great scraper and it's super hard to get a perfect scraper. So like, and, and scraping really matters to people. Do you have a perfect scraper? Not yet. Okay.Swyx [00:27:05]: The web is increasingly closing to the bots and the scrapers, Twitter, Reddit, Quora, Stack Overflow. I don't know what else. How are you dealing with that? How are you navigating those things? Like, you know. You know, opening your eyes, like just paying them money.Will [00:27:19]: Yeah, no, I mean, I think it definitely makes it harder for search engines. One response is just that there's so much value in the long tail of sites that are open. Okay. Um, and just like, even just searching over those well gets you most of the value. But I mean, there, there is definitely a lot of content that is increasingly not unavailable. And so you could get through that through data partnerships. The bigger we get as a company, the more, the easier it is to just like, uh, make partnerships. But I, I mean, I do see the world as like the future where the. The data, the, the data producers, the content creators will make partnerships with the entities that find that data.Alessio [00:27:53]: Any other fun use case that maybe people are not thinking about? Yeah.Will [00:27:58]: Oh, I mean, uh, there are so many customers. Yeah. What are people doing on AXA? Well, I think dating is a really interesting, uh, application of search that is completely underserved because there's a lot of profiles on the web and a lot of people who want to find love and that I'll use it. They give me. Like, you know, age boundaries, you know, education level location. Yeah. I mean, you want to, what, what do you want to do with data? You want to find like a partner who matches this education level, who like, you know, maybe has written about these types of topics before. Like if you could get a list of all the people like that, like, I think you will unblock a lot of people. I mean, there, I mean, I think this is a very Silicon Valley view of dating for sure. And I'm, I'm well aware of that, but it's just an interesting application of like, you know, I would love to meet like an intellectual partner, um, who like shares a lot of ideas. Yeah. Like if you could do that through better search and yeah.Swyx [00:28:48]: But what is it with Jeff? Jeff has already set me up with a few people. So like Jeff, I think it's my personal exit.Will [00:28:55]: my mom's actually a matchmaker and has got a lot of married. Yeah. No kidding. Yeah. Yeah. Search is built into the book. It's in your jeans. Yeah. Yeah.Swyx [00:29:02]: Yeah. Other than dating, like I know you're having quite some success in colleges. I would just love to map out some more use cases so that our listeners can just use those examples to think about use cases for XR, right? Because it's such a general technology that it's hard to. Uh, really pin down, like, what should I use it for and what kind of products can I build with it?Will [00:29:20]: Yeah, sure. So, I mean, there are so many applications of XR and we have, you know, many, many companies using us for very diverse range of use cases, but I'll just highlight some interesting ones. Like one customer, a big customer is using us to, um, basically build like a, a writing assistant for students who want to write, uh, research papers. And basically like XR will search for, uh, like a list of research papers related to what the student is writing. And then this product has. Has like an LLM that like summarizes the papers to basically it's like a next word prediction, but in, uh, you know, prompted by like, you know, 20 research papers that X has returned. It's like literally just doing their homework for them. Yeah. Yeah. the key point is like, it's, it's, uh, you know, it's, it's, you know, research is, is a really hard thing to do and you need like high quality content as input.Swyx [00:30:08]: Oh, so we've had illicit on the podcast. I think it's pretty similar. Uh, they, they do focus pretty much on just, just research papers and, and that research. Basically, I think dating, uh, research, like I just wanted to like spell out more things, like just the big verticals.Will [00:30:23]: Yeah, yeah, no, I mean, there, there are so many use cases. So finance we talked about, yeah. I mean, one big vertical is just finding a list of companies, uh, so it's useful for VCs, like you said, who want to find like a list of competitors to a specific company they're investigating or just a list of companies in some field. Like, uh, there was one VC that told me that him and his team, like we're using XR for like eight hours straight. Like, like that. For many days on end, just like, like, uh, doing like lots of different queries of different types, like, oh, like all the companies in AI for law or, uh, all the companies for AI for, uh, construction and just like getting lists of things because you just can't find this information with, with traditional search engines. And then, you know, finding companies is also useful for, for selling. If you want to find, you know, like if we want to find a list of, uh, writing assistants to sell to, then we can just, we just use XR ourselves to find that is actually how we found a lot of our customers. Ooh, you can find your own customers using XR. Oh my God. I, in the spirit of. Uh, using XR to bolster XR, like recruiting is really helpful. It is really great use case of XR, um, because we can just get like a list of, you know, people who thought about search and just get like a long list and then, you know, reach out to those people.Swyx [00:31:29]: When you say thought about, are you, are you thinking LinkedIn, Twitter, or are you thinking just blogs?Will [00:31:33]: Or they've written, I mean, it's pretty general. So in that case, like ideally XR would return like the, the really blogs written by people who have just. So if I don't blog, I don't show up to XR, right? Like I have to blog. well, I mean, you could show up. That's like an incentive for people to blog.Swyx [00:31:47]: Well, if you've written about, uh, search in on Twitter and we, we do, we do index a bunch of tweets and then we, we should be able to service that. Yeah. Um, I mean, this is something I tell people, like you have to make yourself discoverable to the web, uh, you know, it's called learning in public, but like, it's even more imperative now because otherwise you don't exist at all.Will [00:32:07]: Yeah, no, no, this is a huge, uh, thing, which is like search engines completely influence. They have downstream effects. They influence the internet itself. They influence what people. Choose to create. And so Google, because they're a keyword based search engine, people like kind of like keyword stuff. Yeah. They're, they're, they're incentivized to create things that just match a lot of keywords, which is not very high quality. Uh, whereas XR is a search algorithm that, uh, optimizes for like high quality and actually like matching what you mean. And so people are incentivized to create content that is high quality, that like the create content that they know will be found by the right person. So like, you know, if I am a search researcher and I want to be found. By XR, I should blog about search and all the things I'm building because, because now we have a search engine like XR that's powerful enough to find them. And so the search engine will influence like the downstream internet in all sorts of amazing ways. Yeah. Uh, whatever the search engine optimizes for is what the internet looks like. Yeah.Swyx [00:33:01]: Are you familiar with the term? McLuhanism? No, it's not. Uh, it's this concept that, uh, like first we shape tools and then the tools shape us. Okay. Yeah. Uh, so there's like this reflexive connection between the things we search for and the things that get searched. Yes. So like once you change the tool. The tool that searches the, the, the things that get searched also change. Yes.Will [00:33:18]: I mean, there was a clear example of that with 30 years of Google. Yeah, exactly. Google has basically trained us to think of search and Google has Google is search like in people's heads. Right. It's one, uh, hard part about XR is like, uh, ripping people away from that notion of search and expanding their sense of what search could be. Because like when people think search, they think like a few keywords, or at least they used to, they think of a few keywords and that's it. They don't think to make these like really complex paragraph long requests for information and get a perfect list. ChatGPT was an interesting like thing that expanded people's understanding of search because you start using ChatGPT for a few hours and you go back to Google and you like paste in your code and Google just doesn't work and you're like, oh, wait, it, Google doesn't do work that way. So like ChatGPT expanded our understanding of what search can be. And I think XR is, uh, is part of that. We want to expand people's notion, like, Hey, you could actually get whatever you want. Yeah.Alessio [00:34:06]: I search on XR right now, people writing about learning in public. I was like, is it gonna come out with Alessio? Am I, am I there? You're not because. Bro. It's. So, no, it's, it's so about, because it thinks about learning, like in public, like public schools and like focuses more on that. You know, it's like how, when there are like these highly overlapping things, like this is like a good result based on the query, you know, but like, how do I get to Alessio? Right. So if you're like in these subcultures, I don't think this would work in Google well either, you know, but I, I don't know if you have any learnings.Swyx [00:34:40]: No, I'm the first result on Google.Alessio [00:34:42]: People writing about learning. In public, you're not first result anymore, I guess.Swyx [00:34:48]: Just type learning public in Google.Alessio [00:34:49]: Well, yeah, yeah, yeah, yeah. But this is also like, this is in Google, it doesn't work either. That's what I'm saying. It's like how, when you have like a movement.Will [00:34:56]: There's confusion about the, like what you mean, like your intention is a little, uh. Yeah.Alessio [00:35:00]: It's like, yeah, I'm using, I'm using a term that like I didn't invent, but I'm kind of taking over, but like, they're just so much about that term already that it's hard to overcome. If that makes sense, because public schools is like, well, it's, it's hard to overcome.Will [00:35:14]: Public schools, you know, so there's the right solution to this, which is to specify more clearly what you mean. And I'm not expecting you to do that, but so the, the right interface to search is actually an LLM.Swyx [00:35:25]: Like you should be talking to an LLM about what you want and the LLM translates its knowledge of you or knowledge of what people usually mean into a query that excellent uses, which you have called auto prompts, right?Will [00:35:35]: Or, yeah, but it's like a very light version of that. And really it's just basically the right answer is it's the wrong interface and like very soon interface to search and really to everything will be LLM. And the LLM just has a full knowledge of you, right? So we're kind of building for that world. We're skating to where the puck is going to be. And so since we're moving to a world where like LLMs are interfaced to everything, you should build a search engine that can handle complex LLM queries, queries that come from LLMs. Because you're probably too lazy, I'm too lazy too, to write like a whole paragraph explaining, okay, this is what I mean by this word. But an LLM is not lazy. And so like the LLM will spit out like a paragraph or more explaining exactly what it wants. You need a search engine that can handle that. Traditional search engines like Google or Bing, they're actually... Designed for humans typing keywords. If you give a paragraph to Google or Bing, they just completely fail. And so Exa can handle paragraphs and we want to be able to handle it more and more until it's like perfect.Alessio [00:36:24]: What about opinions? Do you have lists? When you think about the list product, do you think about just finding entries? Do you think about ranking entries? I'll give you a dumb example. So on Lindy, I've been building the spot that every week gives me like the top fantasy football waiver pickups. But every website is like different opinions. I'm like, you should pick up. These five players, these five players. When you're making lists, do you want to be kind of like also ranking and like telling people what's best? Or like, are you mostly focused on just surfacing information?Will [00:36:56]: There's a really good distinction between filtering to like things that match your query and then ranking based on like what is like your preferences. And ranking is like filtering is objective. It's like, does this document match what you asked for? Whereas ranking is more subjective. It's like, what is the best? Well, it depends what you mean by best, right? So first, first table stakes is let's get the filtering into a perfect place where you actually like every document matches what you asked for. No surgeon can do that today. And then ranking, you know, there are all sorts of interesting ways to do that where like you've maybe for, you know, have the user like specify more clearly what they mean by best. You could do it. And if the user doesn't specify, you do your best, you do your best based on what people typically mean by best. But ideally, like the user can specify, oh, when I mean best, I actually mean ranked by the, you know, the number of people who visited that site. Let's say is, is one example ranking or, oh, what I mean by best, let's say you're listing companies. What I mean by best is like the ones that have, uh, you know, have the most employees or something like that. Like there are all sorts of ways to rank a list of results that are not captured by something as subjective as best. Yeah. Yeah.Alessio [00:38:00]: I mean, it's like, who are the best NBA players in the history? It's like everybody has their own. Right.Will [00:38:06]: Right. But I mean, the, the, the search engine should definitely like, even if you don't specify it, it should do as good of a job as possible. Yeah. Yeah. No, no, totally. Yeah. Yeah. Yeah. Yeah. It's a new topic to people because we're not used to a search engine that can handle like a very complex ranking system. Like you think to type in best basketball players and not something more specific because you know, that's the only thing Google could handle. But if Google could handle like, oh, basketball players ranked by like number of shots scored on average per game, then you would do that. But you know, they can't do that. So.Swyx [00:38:32]: Yeah. That's fascinating. So you haven't used the word agents, but you're kind of building a search agent. Do you believe that that is agentic in feature? Do you think that term is distracting?Will [00:38:42]: I think it's a good term. I do think everything will eventually become agentic. And so then the term will lose power, but yes, like what we're building is agentic it in a sense that it takes actions. It decides when to go deeper into something, it has a loop, right? It feels different from traditional search, which is like an algorithm, not an agent. Ours is a combination of an algorithm and an agent.Swyx [00:39:05]: I think my reflection from seeing this in the coding space where there's basically sort of classic. Framework for thinking about this stuff is the self-driving levels of autonomy, right? Level one to five, typically the level five ones all failed because there's full autonomy and we're not, we're not there yet. And people like control. People like to be in the loop. So the, the, the level ones was co-pilot first and now it's like cursor and whatever. So I feel like if it's too agentic, it's too magical, like, like a, like a one shot, I stick a, stick a paragraph into the text box and then it spits it back to me. It might feel like I'm too disconnected from the process and I don't trust it. As opposed to something where I'm more intimately involved with the research product. I see. So like, uh, wait, so the earlier versions are, so if trying to stick to the example of the basketball thing, like best basketball player, but instead of best, you, you actually get to customize it with like, whatever the metric is that you, you guys care about. Yeah. I'm still not a basketballer, but, uh, but, but, you know, like, like B people like to be in my, my thesis is that agents level five agents failed because people like to. To kind of have drive assist rather than full self-driving.Will [00:40:15]: I mean, a lot of this has to do with how good agents are. Like at some point, if agents for coding are better than humans at all tests and then humans block, yeah, we're not there yet.Swyx [00:40:25]: So like in a world where we're not there yet, what you're pitching us is like, you're, you're kind of saying you're going all the way there. Like I kind of, I think all one is also very full, full self-driving. You don't get to see the plan. You don't get to affect the plan yet. You just fire off a query and then it goes away for a couple of minutes and comes back. Right. Which is effectively what you're saying you're going to do too. And you think there's.Will [00:40:42]: There's a, there's an in-between. I saw. Okay. So in building this product, we're exploring new interfaces because what does it mean to kick off a search that goes and takes 10 minutes? Like, is that a good interface? Because what if the search is actually wrong or it's not exactly, exactly specified to what you mean, which is why you get previews. Yeah. You get previews. So it is iterative, but ultimately once you've specified exactly what you mean, then you kind of do just want to kick off a batch job. Right. So perhaps what you're getting at is like, uh, there's this barrier with agents where you have to like explain the full context of what you mean, and a lot of failure modes happen when you have, when you don't. Yeah. There's failure modes from the agent, just not being smart enough. And then there's failure modes from the agent, not understanding exactly what you mean. And there's a lot of context that is shared between humans that is like lost between like humans and, and this like new creature.Alessio [00:41:32]: Yeah. Yeah. Because people don't know what's going on. I mean, to me, the best example of like system prompts is like, why are you writing? You're a helpful assistant. Like. Of course you should be an awful, but people don't yet know, like, can I assume that, you know, that, you know, it's like, why did the, and now people write, oh, you're a very smart software engineer, but like, you never made, you never make mistakes. Like, were you going to try and make mistakes before? So I think people don't yet have an understanding, like with, with driving people know what good driving is. It's like, don't crash, stay within kind of like a certain speed range. It's like, follow the directions. It's like, I don't really have to explain all of those things. I hope. But with. AI and like models and like search, people are like, okay, what do you actually know? What are like your assumptions about how search, how you're going to do search? And like, can I trust it? You know, can I influence it? So I think that's kind of the, the middle ground, like before you go ahead and like do all the search, it's like, can I see how you're doing it? And then maybe help show your work kind of like, yeah, steer you. Yeah. Yeah.Will [00:42:32]: No, I mean, yeah. Sure. Saying, even if you've crafted a great system prompt, you want to be part of the process itself. Uh, because the system prompt doesn't, it doesn't capture everything. Right. So yeah. A system prompt is like, you get to choose the person you work with. It's like, oh, like I want, I want a software engineer who thinks this way about code. But then even once you've chosen that person, you can't just give them a high level command and they go do it perfectly. You have to be part of that process. So yeah, I agree.Swyx [00:42:58]: Just a side note for my system, my favorite system, prompt programming anecdote now is the Apple intelligence system prompt that someone, someone's a prompt injected it and seen it. And like the Apple. Intelligence has the words, like, please don't, don't hallucinate. And it's like, of course we don't want you to hallucinate. Right. Like, so it's exactly that, that what you're talking about, like we should train this behavior into the model, but somehow we still feel the need to inject into the prompt. And I still don't even think that we are very scientific about it. Like it, I think it's almost like cargo culting. Like we have this like magical, like turn around three times, throw salt over your shoulder before you do something. And like, it worked the last time. So let's just do it the same time now. And like, we do, there's no science to this.Will [00:43:35]: I do think a lot of these problems might be ironed out in future versions. Right. So, and like, they might, they might hide the details from you. So it's like, they actually, all of them have a system prompt. That's like, you are a helpful assistant. You don't actually have to include it, even though it might actually be the way they've implemented in the backend. It should be done in RLE AF.Swyx [00:43:52]: Okay. Uh, one question I was just kind of curious about this episode is I'm going to try to frame this in terms of this, the general AI search wars, you know, you're, you're one player in that, um, there's perplexity, chat, GPT, search, and Google, but there's also like the B2B side, uh, we had. Drew Houston from Dropbox on, and he's competing with Glean, who've, uh, we've also had DD from, from Glean on, is there an appetite for Exa for my company's documents?Will [00:44:19]: There is appetite, but I think we have to be disciplined, focused, disciplined. I mean, we're already taking on like perfect web search, which is a lot. Um, but I mean, ultimately we want to build a perfect search engine, which definitely for a lot of queries involves your, your personal information, your company's information. And so, yeah, I mean, the grandest vision of Exa is perfect search really over everything, every domain, you know, we're going to have an Exa satellite, uh, because, because satellites can gather information that, uh, is not available publicly. Uh, gotcha. Yeah.Alessio [00:44:51]: Can we talk about AGI? We never, we never talk about AGI, but you had, uh, this whole tweet about, oh, one being the biggest kind of like AI step function towards it. Why does it feel so important to you? I know there's kind of like always criticism and saying, Hey, it's not the smartest son is better. It's like, blah, blah, blah. What? You choose C. So you say, this is what Ilias see or Sam see what they will see.Will [00:45:13]: I've just, I've just, you know, been connecting the dots. I mean, this was the key thing that a bunch of labs were working on, which is like, can you create a reward signal? Can you teach yourself based on a reward signal? Whether you're, if you're trying to learn coding or math, if you could have one model say, uh, be a grading system that says like you have successfully solved this programming assessment and then one model, like be the generative system. That's like, here are a bunch of programming assessments. You could train on that. It's basically whenever you could create a reward signal for some task, you could just generate a bunch of tasks for yourself. See that like, oh, on two of these thousand, you did well. And then you just train on that data. It's basically like, I mean, creating your own data for yourself and like, you know, all the labs working on that opening, I built the most impressive product doing that. And it's just very, it's very easy now to see how that could like scale to just solving, like, like solving programming or solving mathematics, which sounds crazy, but everything about our world right now is crazy.Alessio [00:46:07]: Um, and so I think if you remove that whole, like, oh, that's impossible, and you just think really clearly about like, what's now possible with like what, what they've done with O1, it's easy to see how that scales. How do you think about older GPT models then? Should people still work on them? You know, if like, obviously they just had the new Haiku, like, is it even worth spending time, like making these models better versus just, you know, Sam talked about O2 at that day. So obviously they're, they're spending a lot of time in it, but then you have maybe. The GPU poor, which are still working on making Lama good. Uh, and then you have the follower labs that do not have an O1 like model out yet. Yeah.Will [00:46:47]: This kind of gets into like, uh, what will the ecosystem of, of models be like in the future? And is there room is, is everything just gonna be O1 like models? I think, well, I mean, there's definitely a question of like inference speed and if certain things like O1 takes a long time, because that's the thing. Well, I mean, O1 is, is two things. It's like one it's it's use it's bootstrapping itself. It's teaching itself. And so the base model is smarter. But then it also has this like inference time compute where it could like spend like many minutes or many hours thinking. And so even the base model, which is also fast, it doesn't have to take minutes. It could take is, is better, smarter. I believe all models will be trained with this paradigm. Like you'll want to train on the best data, but there will be many different size models from different, very many different like companies, I believe. Yeah. Because like, I don't, yeah, I mean, it's hard, hard to predict, but I don't think opening eye is going to dominate like every possible LLM for every possible. Use case. I think for a lot of things, like you just want the fastest model and that might not involve O1 methods at all.Swyx [00:47:42]: I would say if you were to take the exit being O1 for search, literally, you really need to prioritize search trajectories, like almost maybe paying a bunch of grad students to go research things. And then you kind of track what they search and what the sequence of searching is, because it seems like that is the gold mine here, like the chain of thought or the thinking trajectory. Yeah.Will [00:48:05]: When it comes to search, I've always been skeptical. I've always been skeptical of human labeled data. Okay. Yeah, please. We tried something at our company at Exa recently where me and a bunch of engineers on the team like labeled a bunch of queries and it was really hard. Like, you know, you have all these niche queries and you're looking at a bunch of results and you're trying to identify which is matched to query. It's talking about, you know, the intricacies of like some biological experiment or something. I have no idea. Like, I don't know what matches and what, what labelers like me tend to do is just match by keyword. I'm like, oh, I don't know. Oh, like this document matches a bunch of keywords, so it must be good. But then you're actually completely missing the meaning of the document. Whereas an LLM like GB4 is really good at labeling. And so I actually think like you just we get by, which we are right now doing using like LLM

Recorded on-stage at Øredev 2024, Fredrik talks to Ingrid af Sandeberg about AI and people’s perception of it. While it’s very powerful to be able to interact with models through natural language, that interface in itself hides a lot of what’s actually going on. Many thanks to Øredev for inviting Kodsnack again, they paid for the trip and the editing time of these keynote recordings, but have no say about the content of these or any other episodes. Thank you Cloudnet for sponsoring our VPS! Comments, questions or tips? We a re @kodsnack, @tobiashieta, @oferlund and @bjoreman on Twitter, have a page on Facebook and can be emailed at info@kodsnack.se if you want to write longer. We read everything we receive. If you enjoy Kodsnack we would love a review in iTunes! You can also support the podcast by buying us a coffee (or two!) through Ko-fi. Links Øredev All the presentation videos from Øredev 2024 Ingrid AI, truth, and the new information environment - Ingrid’s keynote The five levels of vehicle autonomy Support us on Ko-fi! SLM - small language models Hugging face Googles pagerank Mayo clinic Titles AI is a lot wider A different type of error This chaos element

OpenAI DevDay is almost here! Per tradition, we are hosting a DevDay pregame event for everyone coming to town! Join us with demos and gossip!Also sign up for related events across San Francisco: the AI DevTools Night, the xAI open house, the Replicate art show, the DevDay Watch Party (for non-attendees), Hack Night with OpenAI at Cloudflare. For everyone else, join the Latent Space Discord for our online watch party and find fellow AI Engineers in your city.OpenAI's recent o1 release (and Reflection 70b debacle) has reignited broad interest in agentic general reasoning and tree search methods.While we have covered some of the self-taught reasoning literature on the Latent Space Paper Club, it is notable that the Eric Zelikman ended up at xAI, whereas OpenAI's hiring of Noam Brown and now Shunyu suggests more interest in tool-using chain of thought/tree of thought/generator-verifier architectures for Level 3 Agents.We were more than delighted to learn that Shunyu is a fellow Latent Space enjoyer, and invited him back (after his first appearance on our NeurIPS 2023 pod) for a look through his academic career with Harrison Chase (one year after his first LS show).ReAct: Synergizing Reasoning and Acting in Language Modelspaper linkFollowing seminal Chain of Thought papers from Wei et al and Kojima et al, and reflecting on lessons from building the WebShop human ecommerce trajectory benchmark, Shunyu's first big hit, the ReAct paper showed that using LLMs to “generate both reasoning traces and task-specific actions in an interleaved manner” achieved remarkably greater performance (less hallucination/error propagation, higher ALFWorld/WebShop benchmark success) than CoT alone. In even better news, ReAct scales fabulously with finetuning:As a member of the elite Princeton NLP group, Shunyu was also a coauthor of the Reflexion paper, which we discuss in this pod.Tree of Thoughtspaper link hereShunyu's next major improvement on the CoT literature was Tree of Thoughts:Language models are increasingly being deployed for general problem solving across a wide range of tasks, but are still confined to token-level, left-to-right decision-making processes during inference. This means they can fall short in tasks that require exploration, strategic lookahead, or where initial decisions play a pivotal role…ToT allows LMs to perform deliberate decision making by considering multiple different reasoning paths and self-evaluating choices to decide the next course of action, as well as looking ahead or backtracking when necessary to make global choices.The beauty of ToT is it doesnt require pretraining with exotic methods like backspace tokens or other MCTS architectures. You can listen to Shunyu explain ToT in his own words on our NeurIPS pod, but also the ineffable Yannic Kilcher:Other WorkWe don't have the space to summarize the rest of Shunyu's work, you can listen to our pod with him now, and recommend the CoALA paper and his initial hit webinar with Harrison, today's guest cohost:as well as Shunyu's PhD Defense Lecture:as well as Shunyu's latest lecture covering a Brief History of LLM Agents:As usual, we are live on YouTube! Show Notes* Harrison Chase* LangChain, LangSmith, LangGraph* Shunyu Yao* Alec Radford* ReAct Paper* Hotpot QA* Tau Bench* WebShop* SWE-Agent* SWE-Bench* Trees of Thought* CoALA Paper* Related Episodes* Our Thomas Scialom (Meta) episode* Shunyu on our NeurIPS 2023 Best Papers episode* Harrison on our LangChain episode* Mentions* Sierra* Voyager* Jason Wei* Tavily* SERP API* ExaTimestamps* [00:00:00] Opening Song by Suno* [00:03:00] Introductions* [00:06:16] The ReAct paper* [00:12:09] Early applications of ReAct in LangChain* [00:17:15] Discussion of the Reflection paper* [00:22:35] Tree of Thoughts paper and search algorithms in language models* [00:27:21] SWE-Agent and SWE-Bench for coding benchmarks* [00:39:21] CoALA: Cognitive Architectures for Language Agents* [00:45:24] Agent-Computer Interfaces (ACI) and tool design for agents* [00:49:24] Designing frameworks for agents vs humans* [00:53:52] UX design for AI applications and agents* [00:59:53] Data and model improvements for agent capabilities* [01:19:10] TauBench* [01:23:09] Promising areas for AITranscriptAlessio [00:00:01]: Hey, everyone, welcome to the Latent Space podcast. This is Alessio, partner and CTO of Residence at Decibel Partners, and I'm joined by my co-host Swyx, founder of Small AI.Swyx [00:00:12]: Hey, and today we have a super special episode. I actually always wanted to take like a selfie and go like, you know, POV, you're about to revolutionize the world of agents because we have two of the most awesome hiring agents in the house. So first, we're going to welcome back Harrison Chase. Welcome. Excited to be here. What's new with you recently in sort of like the 10, 20 second recap?Harrison [00:00:34]: Linkchain, Linksmith, Lingraph, pushing on all of them. Lots of cool stuff related to a lot of the stuff that we're going to talk about today, probably.Swyx [00:00:42]: Yeah.Alessio [00:00:43]: We'll mention it in there. And the Celtics won the title.Swyx [00:00:45]: And the Celtics won the title. You got that going on for you. I don't know. Is that like floorball? Handball? Baseball? Basketball.Alessio [00:00:52]: Basketball, basketball.Harrison [00:00:53]: Patriots aren't looking good though, so that's...Swyx [00:00:56]: And then Xun Yu, you've also been on the pod, but only in like a sort of oral paper presentation capacity. But welcome officially to the LinkedSpace pod.Shunyu [00:01:03]: Yeah, I've been a huge fan. So thanks for the invitation. Thanks.Swyx [00:01:07]: Well, it's an honor to have you on. You're one of like, you're maybe the first PhD thesis defense I've ever watched in like this AI world, because most people just publish single papers, but every paper of yours is a banger. So congrats.Shunyu [00:01:22]: Thanks.Swyx [00:01:24]: Yeah, maybe we'll just kick it off with, you know, what was your journey into using language models for agents? I like that your thesis advisor, I didn't catch his name, but he was like, you know... Karthik. Yeah. It's like, this guy just wanted to use language models and it was such a controversial pick at the time. Right.Shunyu [00:01:39]: The full story is that in undergrad, I did some computer vision research and that's how I got into AI. But at the time, I feel like, you know, you're just composing all the GAN or 3D perception or whatever together and it's not exciting anymore. And one day I just see this transformer paper and that's really cool. But I really got into language model only when I entered my PhD and met my advisor Karthik. So he was actually the second author of GPT-1 when he was like a visiting scientist at OpenAI. With Alec Redford?Swyx [00:02:10]: Yes.Shunyu [00:02:11]: Wow. That's what he told me. It's like back in OpenAI, they did this GPT-1 together and Ilya just said, Karthik, you should stay because we just solved the language. But apparently Karthik is not fully convinced. So he went to Princeton, started his professorship and I'm really grateful. So he accepted me as a student, even though I have no prior knowledge in NLP. And you know, we just met for the first time and he's like, you know, what do you want to do? And I'm like, you know, you have done those test game scenes. That's really cool. I wonder if we can just redo them with language models. And that's how the whole journey began. Awesome.Alessio [00:02:46]: So GPT-2 was out at the time? Yes, that was 2019.Shunyu [00:02:48]: Yeah.Alessio [00:02:49]: Way too dangerous to release. And then I guess the first work of yours that I came across was React, which was a big part of your defense. But also Harrison, when you came on The Pockets last year, you said that was one of the first papers that you saw when you were getting inspired for BlankChain. So maybe give a recap of why you thought it was cool, because you were already working in AI and machine learning. And then, yeah, you can kind of like intro the paper formally. What was that interesting to you specifically?Harrison [00:03:16]: Yeah, I mean, I think the interesting part was using these language models to interact with the outside world in some form. And I think in the paper, you mostly deal with Wikipedia. And I think there's some other data sets as well. But the outside world is the outside world. And so interacting with things that weren't present in the LLM and APIs and calling into them and thinking about the React reasoning and acting and kind of like combining those together and getting better results. I'd been playing around with LLMs, been talking with people who were playing around with LLMs. People were trying to get LLMs to call into APIs, do things, and it was always, how can they do it more reliably and better? And so this paper was basically a step in that direction. And I think really interesting and also really general as well. Like I think that's part of the appeal is just how general and simple in a good way, I think the idea was. So that it was really appealing for all those reasons.Shunyu [00:04:07]: Simple is always good. Yeah.Alessio [00:04:09]: Do you have a favorite part? Because I have one favorite part from your PhD defense, which I didn't understand when I read the paper, but you said something along the lines, React doesn't change the outside or the environment, but it does change the insight through the context, putting more things in the context. You're not actually changing any of the tools around you to work for you, but you're changing how the model thinks. And I think that was like a very profound thing when I, not that I've been using these tools for like 18 months. I'm like, I understand what you meant, but like to say that at the time you did the PhD defense was not trivial. Yeah.Shunyu [00:04:41]: Another way to put it is like thinking can be an extra tool that's useful.Alessio [00:04:47]: Makes sense. Checks out.Swyx [00:04:49]: Who would have thought? I think it's also more controversial within his world because everyone was trying to use RL for agents. And this is like the first kind of zero gradient type approach. Yeah.Shunyu [00:05:01]: I think the bigger kind of historical context is that we have this two big branches of AI. So if you think about RL, right, that's pretty much the equivalent of agent at a time. And it's like agent is equivalent to reinforcement learning and reinforcement learning is equivalent to whatever game environment they're using, right? Atari game or go or whatever. So you have like a pretty much, you know, you have a biased kind of like set of methodologies in terms of reinforcement learning and represents agents. On the other hand, I think NLP is like a historical kind of subject. It's not really into agents, right? It's more about reasoning. It's more about solving those concrete tasks. And if you look at SEL, right, like each task has its own track, right? Summarization has a track, question answering has a track. So I think really it's about rethinking agents in terms of what could be the new environments that we came to have is not just Atari games or whatever video games, but also those text games or language games. And also thinking about, could there be like a more general kind of methodology beyond just designing specific pipelines for each NLP task? That's like the bigger kind of context, I would say.Alessio [00:06:14]: Is there an inspiration spark moment that you remember or how did you come to this? We had Trida on the podcast and he mentioned he was really inspired working with like systems people to think about Flash Attention. What was your inspiration journey?Shunyu [00:06:27]: So actually before React, I spent the first two years of my PhD focusing on text-based games, or in other words, text adventure games. It's a very kind of small kind of research area and quite ad hoc, I would say. And there are like, I don't know, like 10 people working on that at the time. And have you guys heard of Zork 1, for example? So basically the idea is you have this game and you have text observations, like you see a monster, you see a dragon.Swyx [00:06:57]: You're eaten by a grue.Shunyu [00:06:58]: Yeah, you're eaten by a grue. And you have actions like kill the grue with a sword or whatever. And that's like a very typical setup of a text game. So I think one day after I've seen all the GPT-3 stuff, I just think about, you know, how can I solve the game? Like why those AI, you know, machine learning methods are pretty stupid, but we are pretty good at solving the game relatively, right? So for the context, the predominant method to solve this text game is obviously reinforcement learning. And the idea is you just try out an arrow in those games for like millions of steps and you kind of just overfit to the game. But there's no language understanding at all. And I'm like, why can't I solve the game better? And it's kind of like, because we think about the game, right? Like when we see this very complex text observation, like you see a grue and you might see a sword, you know, in the right of the room and you have to go through the wooden door to go to that room. You will think, you know, oh, I have to kill the monster and to kill that monster, I have to get the sword, I have to get the sword, I have to go, right? And this kind of thinking actually helps us kind of throw shots off the game. And it's like, why don't we also enable the text agents to think? And that's kind of the prototype of React. And I think that's actually very interesting because the prototype, I think, was around November of 2021. So that's even before like chain of thought or whatever came up. So we did a bunch of experiments in the text game, but it was not really working that well. Like those text games are just too hard. I think today it's still very hard. Like if you use GPD 4 to solve it, it's still very hard. So the change came when I started the internship in Google. And apparently Google care less about text game, they care more about what's more practical. So pretty much I just reapplied the idea, but to more practical kind of environments like Wikipedia or simpler text games like Alphard, and it just worked. It's kind of like you first have the idea and then you try to find the domains and the problems to demonstrate the idea, which is, I would say, different from most of the AI research, but it kind of worked out for me in that case.Swyx [00:09:09]: For Harrison, when you were implementing React, what were people applying React to in the early days?Harrison [00:09:14]: I think the first demo we did probably had like a calculator tool and a search tool. So like general things, we tried to make it pretty easy to write your own tools and plug in your own things. And so this is one of the things that we've seen in LangChain is people who build their own applications generally write their own tools. Like there are a few common ones. I'd say like the three common ones might be like a browser, a search tool, and a code interpreter. But then other than that-Swyx [00:09:37]: The LMS. Yep.Harrison [00:09:39]: Yeah, exactly. It matches up very nice with that. And we actually just redid like our integrations docs page, and if you go to the tool section, they like highlight those three, and then there's a bunch of like other ones. And there's such a long tail of other ones. But in practice, like when people go to production, they generally have their own tools or maybe one of those three, maybe some other ones, but like very, very few other ones. So yeah, I think the first demos was a search and a calculator one. And there's- What's the data set?Shunyu [00:10:04]: Hotpot QA.Harrison [00:10:05]: Yeah. Oh, so there's that one. And then there's like the celebrity one by the same author, I think.Swyx [00:10:09]: Olivier Wilde's boyfriend squared. Yeah. 0.23. Yeah. Right, right, right.Harrison [00:10:16]: I'm forgetting the name of the author, but there's-Swyx [00:10:17]: I was like, we're going to over-optimize for Olivier Wilde's boyfriend, and it's going to change next year or something.Harrison [00:10:21]: There's a few data sets kind of like in that vein that require multi-step kind of like reasoning and thinking. So one of the questions I actually had for you in this vein, like the React paper, there's a few things in there, or at least when I think of that, there's a few things that I think of. There's kind of like the specific prompting strategy. Then there's like this general idea of kind of like thinking and then taking an action. And then there's just even more general idea of just like taking actions in a loop. Today, like obviously language models have changed a lot. We have tool calling. The specific prompting strategy probably isn't used super heavily anymore. Would you say that like the concept of React is still used though? Or like do you think that tool calling and running tool calling in a loop, is that ReactSwyx [00:11:02]: in your mind?Shunyu [00:11:03]: I would say like it's like more implicitly used than explicitly used. To be fair, I think the contribution of React is actually twofold. So first is this idea of, you know, we should be able to use calls in a very general way. Like there should be a single kind of general method to handle interaction with various environments. I think React is the first paper to demonstrate the idea. But then I think later there are two form or whatever, and this becomes like a trivial idea. But I think at the time, that's like a pretty non-trivial thing. And I think the second contribution is this idea of what people call like inner monologue or thinking or reasoning or whatever, to be paired with tool use. I think that's still non-trivial because if you look at the default function calling or whatever, like there's no inner monologue. And in practice, that actually is important, especially if the tool that you use is pretty different from the training distribution of the language model. I think those are the two main things that are kind of inherited.Harrison [00:12:10]: On that note, I think OpenAI even recommended when you're doing tool calling, it's sometimes helpful to put a thought field in the tool, along with all the actual acquired arguments,Swyx [00:12:19]: and then have that one first.Harrison [00:12:20]: So it fills out that first, and they've shown that that's yielded better results. The reason I ask is just like this same concept is still alive, and I don't know whether to call it a React agent or not. I don't know what to call it. I think of it as React, like it's the same ideas that were in the paper, but it's obviously a very different implementation at this point in time. And so I just don't know what to call it.Shunyu [00:12:40]: I feel like people will sometimes think more in terms of different tools, right? Because if you think about a web agent versus, you know, like a function calling agent, calling a Python API, you would think of them as very different. But in some sense, the methodology is the same. It depends on how you view them, right? I think people will tend to think more in terms of the environment and the tools rather than the methodology. Or, in other words, I think the methodology is kind of trivial and simple, so people will try to focus more on the different tools. But I think it's good to have a single underlying principle of those things.Alessio [00:13:17]: How do you see the surface of React getting molded into the model? So a function calling is a good example of like, now the model does it. What about the thinking? Now most models that you use kind of do chain of thought on their own, they kind of produce steps. Do you think that more and more of this logic will be in the model? Or do you think the context window will still be the main driver of reasoning and thinking?Shunyu [00:13:39]: I think it's already default, right? You do some chain of thought and you do some tool call, the cost of adding the chain of thought is kind of relatively low compared to other things. So it's not hurting to do that. And I think it's already kind of common practice, I would say.Swyx [00:13:56]: This is a good place to bring in either Tree of Thought or Reflection, your pick.Shunyu [00:14:01]: Maybe Reflection, to respect the time order, I would say.Swyx [00:14:05]: Any backstory as well, like the people involved with NOAA and the Princeton group. We talked about this offline, but people don't understand how these research pieces come together and this ideation.Shunyu [00:14:15]: I think Reflection is mostly NOAA's work, I'm more like advising kind of role. The story is, I don't remember the time, but one day we just see this pre-print that's like Reflection and Autonomous Agent with memory or whatever. And it's kind of like an extension to React, which uses this self-reflection. I'm like, oh, somehow you've become very popular. And NOAA reached out to me, it's like, do you want to collaborate on this and make this from an archive pre-print to something more solid, like a conference submission? I'm like, sure. We started collaborating and we remain good friends today. And I think another interesting backstory is NOAA was contacted by OpenAI at the time. It's like, this is pretty cool, do you want to just work at OpenAI? And I think Sierra also reached out at the same time. It's like, this is pretty cool, do you want to work at Sierra? And I think NOAA chose Sierra, but it's pretty cool because he was still like a second year undergrad and he's a very smart kid.Swyx [00:15:16]: Based on one paper. Oh my god.Shunyu [00:15:19]: He's done some other research based on programming language or chemistry or whatever, but I think that's the paper that got the attention of OpenAI and Sierra.Swyx [00:15:28]: For those who haven't gone too deep on it, the way that you present the inside of React, can you do that also for reflection? Yeah.Shunyu [00:15:35]: I think one way to think of reflection is that the traditional idea of reinforcement learning is you have a scalar reward and then you somehow back-propagate the signal of the scalar reward to the rest of your neural network through whatever algorithm, like policy grading or A2C or whatever. And if you think about the real life, most of the reward signal is not scalar. It's like your boss told you, you should have done a better job in this, but you could jump on that or whatever. It's not like a scalar reward, like 29 or something. I think in general, humans deal more with long scalar reward, or you can say language feedback. And the way that they deal with language feedback also has this back-propagation process, right? Because you start from this, you did a good job on job B, and then you reflect what could have been done differently to change to make it better. And you kind of change your prompt, right? Basically, you change your prompt on how to do job A and how to do job B, and then you do the whole thing again. So it's really like a pipeline of language where in self-graded descent, you have something like text reasoning to replace those gradient descent algorithms. I think that's one way to think of reflection.Harrison [00:16:47]: One question I have about reflection is how general do you think the algorithm there is? And so for context, I think at LangChain and at other places as well, we found it pretty easy to implement React in a standard way. You plug in any tools and it kind of works off the shelf, can get it up and running. I don't think we have an off-the-shelf kind of implementation of reflection and kind of the general sense. I think the concepts, absolutely, we see used in different kind of specific cognitive architectures, but I don't think we have one that comes off the shelf. I don't think any of the other frameworks have one that comes off the shelf. And I'm curious whether that's because it's not general enough or it's complex as well, because it also requires running it more times.Swyx [00:17:28]: Maybe that's not feasible.Harrison [00:17:30]: I'm curious how you think about the generality, complexity. Should we have one that comes off the shelf?Shunyu [00:17:36]: I think the algorithm is general in the sense that it's just as general as other algorithms, if you think about policy grading or whatever, but it's not applicable to all tasks, just like other algorithms. So you can argue PPO is also general, but it works better for those set of tasks, but not on those set of tasks. I think it's the same situation for reflection. And I think a key bottleneck is the evaluator, right? Basically, you need to have a good sense of the signal. So for example, if you are trying to do a very hard reasoning task, say mathematics, for example, and you don't have any tools, you're operating in this chain of thought setup, then reflection will be pretty hard because in order to reflect upon your thoughts, you have to have a very good evaluator to judge whether your thought is good or not. But that might be as hard as solving the problem itself or even harder. The principle of self-reflection is probably more applicable if you have a good evaluator, for example, in the case of coding. If you have those arrows, then you can just reflect on that and how to solve the bug andSwyx [00:18:37]: stuff.Shunyu [00:18:38]: So I think another criteria is that it depends on the application, right? If you have this latency or whatever need for an actual application with an end-user, the end-user wouldn't let you do two hours of tree-of-thought or reflection, right? You need something as soon as possible. So in that case, maybe this is better to be used as a training time technique, right? You do those reflection or tree-of-thought or whatever, you get a lot of data, and then you try to use the data to train your model better. And then in test time, you still use something as simple as React, but that's already improved.Alessio [00:19:11]: And if you think of the Voyager paper as a way to store skills and then reuse them, how would you compare this reflective memory and at what point it's just ragging on the memory versus you want to start to fine-tune some of them or what's the next step once you get a very long reflective corpus? Yeah.Shunyu [00:19:30]: So I think there are two questions here. The first question is, what type of information or memory are you considering, right? Is it like semantic memory that stores knowledge about the word, or is it the episodic memory that stores trajectories or behaviors, or is it more of a procedural memory like in Voyager's case, like skills or code snippets that you can use to do actions, right?Swyx [00:19:54]: That's one dimension.Shunyu [00:19:55]: And the second dimension is obviously how you use the memory, either retrieving from it, using it in the context, or fine-tuning it. I think the Cognitive Architecture for Language Agents paper has a good categorization of all the different combinations. And of course, which way you use depends on the concrete application and the concrete need and the concrete task. But I think in general, it's good to think of those systematic dimensions and all the possible options there.Swyx [00:20:25]: Harrison also has in LangMEM, I think you did a presentation in my meetup, and I think you've done it at a couple other venues as well. User state, semantic memory, and append-only state, I think kind of maps to what you just said.Shunyu [00:20:38]: What is LangMEM? Can I give it like a quick...Harrison [00:20:40]: One of the modules of LangChain for a long time has been something around memory. And I think we're still obviously figuring out what that means, as is everyone kind of in the space. But one of the experiments that we did, and one of the proof of concepts that we did was, technically what it was is you would basically create threads, you'd push messages to those threads in the background, we process the data in a few ways. One, we put it into some semantic store, that's the semantic memory. And then two, we do some extraction and reasoning over the memories to extract. And we let the user define this, but extract key facts or anything that's of interest to the user. Those aren't exactly trajectories, they're maybe more closer to the procedural memory. Is that how you'd think about it or classify it?Shunyu [00:21:22]: Is it like about knowledge about the word, or is it more like how to do something?Swyx [00:21:27]: It's reflections, basically.Harrison [00:21:28]: So in generative worlds.Shunyu [00:21:30]: Generative agents.Swyx [00:21:31]: The Smallville. Yeah, the Smallville one.Harrison [00:21:33]: So the way that they had their memory there was they had the sequence of events, and that's kind of like the raw events that happened. But then every N events, they'd run some synthesis over those events for the LLM to insert its own memory, basically. It's that type of memory.Swyx [00:21:49]: I don't know how that would be classified.Shunyu [00:21:50]: I think of that as more of the semantic memory, but to be fair, I think it's just one way to think of that. But whether it's semantic memory or procedural memory or whatever memory, that's like an abstraction layer. But in terms of implementation, you can choose whatever implementation for whatever memory. So they're totally kind of orthogonal. I think it's more of a good way to think of the things, because from the history of cognitive science and cognitive architecture and how people study even neuroscience, that's the way people think of how the human brain organizes memory. And I think it's more useful as a way to think of things. But it's not like for semantic memory, you have to do this kind of way to retrieve or fine-tune, and for procedural memory, you have to do that. I think those are totally orthogonal kind of dimensions.Harrison [00:22:34]: How much background do you have in cognitive sciences, and how much do you model some of your thoughts on?Shunyu [00:22:40]: That's a great question, actually. I think one of the undergrad influences for my follow-up research is I was doing an internship at MIT's Computational Cognitive Science Lab with Josh Tannenbaum, and he's a very famous cognitive scientist. And I think a lot of his ideas still influence me today, like thinking of things in computational terms and getting interested in language and a lot of stuff, or even developing psychology kind of stuff. So I think it still influences me today.Swyx [00:23:14]: As a developer that tried out LangMEM, the way I view it is just it's a materialized view of a stream of logs. And if anything, that's just useful for context compression. I don't have to use the full context to run it over everything. But also it's kind of debuggable. If it's wrong, I can show it to the user, the user can manually fix it, and I can carry on. That's a really good analogy. I like that. I'm going to steal that. Sure. Please, please. You know I'm bullish on memory databases. I guess, Tree of Thoughts? Yeah, Tree of Thoughts.Shunyu [00:23:39]: I feel like I'm relieving the defense in like a podcast format. Yeah, no.Alessio [00:23:45]: I mean, you had a banger. Well, this is the one where you're already successful and we just highlight the glory. It was really good. You mentioned that since thinking is kind of like taking an action, you can use action searching algorithms to think of thinking. So just like you will use Tree Search to find the next thing. And the idea behind Tree of Thought is that you generate all these possible outcomes and then find the best tree to get to the end. Maybe back to the latency question, you can't really do that if you have to respond in real time. So what are maybe some of the most helpful use cases for things like this? Where have you seen people adopt it where the high latency is actually worth the wait?Shunyu [00:24:21]: For things that you don't care about latency, obviously. For example, if you're trying to do math, if you're just trying to come up with a proof. But I feel like one type of task is more about searching for a solution. You can try a hundred times, but if you find one solution, that's good. For example, if you're finding a math proof or if you're finding a good code to solve a problem or whatever, I think another type of task is more like reacting. For example, if you're doing customer service, you're like a web agent booking a ticket for an end user. Those are more reactive kind of tasks, or more real-time tasks. You have to do things fast. They might be easy, but you have to do it reliably. And you care more about can you solve 99% of the time out of a hundred. But for the type of search type of tasks, then you care more about can I find one solution out of a hundred. So it's kind of symmetric and different.Alessio [00:25:11]: Do you have any data or intuition from your user base? What's the split of these type of use cases? How many people are doing more reactive things and how many people are experimenting with deep, long search?Harrison [00:25:23]: I would say React's probably the most popular. I think there's aspects of reflection that get used. Tree of thought, probably the least so. There's a great tweet from Jason Wei, I think you're now a colleague, and he was talking about prompting strategies and how he thinks about them. And I think the four things that he had was, one, how easy is it to implement? How much compute does it take? How many tasks does it solve? And how much does it improve on those tasks? And I'd add a fifth, which is how likely is it to be relevant when the next generation of models come out? And I think if you look at those axes and then you look at React, reflection, tree of thought, it tracks that the ones that score better are used more. React is pretty easy to implement. Tree of thought's pretty hard to implement. The amount of compute, yeah, a lot more for tree of thought. The tasks and how much it improves, I don't have amazing visibility there. But I think if we're comparing React versus tree of thought, React just dominates the first two axes so much that my question around that was going to be like, how do you think about these prompting strategies, cognitive architectures, whatever you want to call them? When you're thinking of them, what are the axes that you're judging them on in your head when you're thinking whether it's a good one or a less good one?Swyx [00:26:38]: Right.Shunyu [00:26:39]: Right. I think there is a difference between a prompting method versus research, in the sense that for research, you don't really even care about does it actually work on practical tasks or does it help? Whatever. I think it's more about the idea or the principle, right? What is the direction that you're unblocking and whatever. And I think for an actual prompting method to solve a concrete problem, I would say simplicity is very important because the simpler it is, the less decision you have to make about it. And it's easier to design. It's easier to propagate. And it's easier to do stuff. So always try to be as simple as possible. And I think latency obviously is important. If you can do things fast and you don't want to do things slow. And I think in terms of the actual prompting method to use for a particular problem, I think we should all be in the minimalist kind of camp, right? You should try the minimum thing and see if it works. And if it doesn't work and there's absolute reason to add something, then you add something, right? If there's absolute reason that you need some tool, then you should add the tool thing. If there's absolute reason to add reflection or whatever, you should add that. Otherwise, if a chain of thought can already solve something, then you don't even need to use any of that.Harrison [00:27:57]: Yeah. Or if it's just better prompting can solve it. Like, you know, you could add a reflection step or you could make your instructions a little bit clearer.Swyx [00:28:03]: And it's a lot easier to do that.Shunyu [00:28:04]: I think another interesting thing is like, I personally have never done those kind of like weird tricks. I think all the prompts that I write are kind of like just talking to a human, right? It's like, I don't know. I never say something like, your grandma is dying and you have to solve it. I mean, those are cool, but I feel like we should all try to solve things in a very intuitive way. Just like talking to your co-worker. That should work 99% of the time. That's my personal take.Swyx [00:28:29]: The problem with how language models, at least in the GPC 3 era, was that they over-optimized to some sets of tokens in sequence. So like reading the Kojima et al. paper that was listing step-by-step, like he tried a bunch of them and they had wildly different results. It should not be the case, but it is the case. And hopefully we're getting better there.Shunyu [00:28:51]: Yeah. I think it's also like a timing thing in the sense that if you think about this whole line of language model, right? Like at the time it was just like a text generator. We don't have any idea how it's going to be used, right? And obviously at the time you will find all kinds of weird issues because it's not trained to do any of that, right? But then I think we have this loop where once we realize chain of thought is important or agent is important or tool using is important, what we see is today's language models are heavily optimized towards those things. So I think in some sense they become more reliable and robust over those use cases. And you don't need to do as much prompt engineering tricks anymore to solve those things. I feel like in some sense, I feel like prompt engineering even is like a slightly negative word at the time because it refers to all those kind of weird tricks that you have to apply. But I think we don't have to do that anymore. Like given today's progress, you should just be able to talk to like a coworker. And if you're clear and concrete and being reasonable, then it should do reasonable things for you.Swyx [00:29:51]: Yeah. The way I put this is you should not be a prompt engineer because it is the goal of the big labs to put you out of a job.Shunyu [00:29:58]: You should just be a good communicator. Like if you're a good communicator to humans, you should be a good communicator to languageSwyx [00:30:02]: models.Harrison [00:30:03]: That's the key though, because oftentimes people aren't good communicators to these language models and that is a very important skill and that's still messing around with the prompt. And so it depends what you're talking about when you're saying prompt engineer.Shunyu [00:30:14]: But do you think it's like very correlated with like, are they like a good communicator to humans? You know, it's like.Harrison [00:30:20]: It may be, but I also think I would say on average, people are probably worse at communicating with language models than to humans right now, at least, because I think we're still figuring out how to do it. You kind of expect it to be magical and there's probably some correlation, but I'd say there's also just like, people are worse at it right now than talking to humans.Shunyu [00:30:36]: We should make it like a, you know, like an elementary school class or whatever, how toSwyx [00:30:41]: talk to language models. Yeah. I don't know. Very pro that. Yeah. Before we leave the topic of trees and searching, not specific about QSTAR, but there's a lot of questions about MCTS and this combination of tree search and language models. And I just had to get in a question there about how seriously should people take this?Shunyu [00:30:59]: Again, I think it depends on the tasks, right? So MCTS was magical for Go, but it's probably not as magical for robotics, right? So I think right now the problem is not even that we don't have good methodologies, it's more about we don't have good tasks. It's also very interesting, right? Because if you look at my citation, it's like, obviously the most cited are React, Refraction and Tree of Thought. Those are methodologies. But I think like equally important, if not more important line of my work is like benchmarks and environments, right? Like WebShop or SuiteVenture or whatever. And I think in general, what people do in academia that I think is not good is they choose a very simple task, like Alford, and then they apply overly complex methods to show they improve 2%. I think you should probably match the level of complexity of your task and your method. I feel like where tasks are kind of far behind the method in some sense, right? Because we have some good test-time approaches, like whatever, React or Refraction or Tree of Thought, or like there are many, many more complicated test-time methods afterwards. But on the benchmark side, we have made a lot of good progress this year, last year. But I think we still need more progress towards that, like better coding benchmark, better web agent benchmark, better agent benchmark, not even for web or code. I think in general, we need to catch up with tasks.Harrison [00:32:27]: What are the biggest reasons in your mind why it lags behind?Shunyu [00:32:31]: I think incentive is one big reason. Like if you see, you know, all the master paper are cited like a hundred times more than the task paper. And also making a good benchmark is actually quite hard. It's almost like a different set of skills in some sense, right? I feel like if you want to build a good benchmark, you need to be like a good kind of product manager kind of mindset, right? You need to think about why people should use your benchmark, why it's challenging, why it's useful. If you think about like a PhD going into like a school, right? The prior skill that expected to have is more about, you know, can they code this method and can they just run experiments and can solve that? I think building a benchmark is not the typical prior skill that we have, but I think things are getting better. I think more and more people are starting to build benchmarks and people are saying that it's like a way to get more impact in some sense, right? Because like if you have a really good benchmark, a lot of people are going to use it. But if you have a super complicated test time method, like it's very hard for people to use it.Harrison [00:33:35]: Are evaluation metrics also part of the reason? Like for some of these tasks that we might want to ask these agents or language models to do, is it hard to evaluate them? And so it's hard to get an automated benchmark. Obviously with SweetBench you can, and with coding, it's easier, but.Shunyu [00:33:50]: I think that's part of the skillset thing that I mentioned, because I feel like it's like a product manager because there are many dimensions and you need to strike a balance and it's really hard, right? If you want to make sense, very easy to autogradable, like automatically gradable, like either to grade or either to evaluate, then you might lose some of the realness or practicality. Or like it might be practical, but it might not be as scalable, right? For example, if you think about text game, human have pre-annotated all the rewards and all the language are real. So it's pretty good on autogradable dimension and the practical dimension. If you think about, you know, practical, like actual English being practical, but it's not scalable, right? It takes like a year for experts to build that game. So it's not really that scalable. And I think part of the reason that SweetBench is so popular now is it kind of hits the balance between these three dimensions, right? Easy to evaluate and being actually practical and being scalable. Like if I were to criticize upon some of my prior work, I think webshop, like it's my initial attempt to get into benchmark world and I'm trying to do a good job striking the balance. But obviously we make it all gradable and it's really scalable, but then I think the practicality is not as high as actually just using GitHub issues, right? Because you're just creating those like synthetic tasks.Harrison [00:35:13]: Are there other areas besides coding that jump to mind as being really good for being autogradable?Shunyu [00:35:20]: Maybe mathematics.Swyx [00:35:21]: Classic. Yeah. Do you have thoughts on alpha proof, the new DeepMind paper? I think it's pretty cool.Shunyu [00:35:29]: I think it's more of a, you know, it's more of like a confidence boost or like sometimes, you know, the work is not even about, you know, the technical details or the methodology that it chooses or the concrete results. I think it's more about a signal, right?Swyx [00:35:47]: Yeah. Existence proof. Yeah.Shunyu [00:35:50]: Yeah. It can be done. This direction is exciting. It kind of encourages people to work more towards that direction. I think it's more like a boost of confidence, I would say.Swyx [00:35:59]: Yeah. So we're going to focus more on agents now and, you know, all of us have a special interest in coding agents. I would consider Devin to be the sort of biggest launch of the year as far as AI startups go. And you guys in the Princeton group worked on Suiagents alongside of Suibench. Tell us the story about Suiagent. Sure.Shunyu [00:36:21]: I think it's kind of like a triology, it's actually a series of three works now. So actually the first work is called Intercode, but it's not as famous, I know. And the second work is called Suibench and the third work is called Suiagent. And I'm just really confused why nobody is working on coding. You know, it's like a year ago, but I mean, not everybody's working on coding, obviously, but a year ago, like literally nobody was working on coding. I was really confused. And the people that were working on coding are, you know, trying to solve human evil in like a sick-to-sick way. There's no agent, there's no chain of thought, there's no anything, they're just, you know, fine tuning the model and improve some points and whatever, like, I was really confused because obviously coding is the best application for agents because it's autogradable, it's super important, you can make everything like API or code action, right? So I was confused and I collaborated with some of the students in Princeton and we have this work called Intercode and the idea is, first, if you care about coding, then you should solve coding in an interactive way, meaning more like a Jupyter Notebook kind of way than just writing a program and seeing if it fails or succeeds and stop, right? You should solve it in an interactive way because that's exactly how humans solve it, right? You don't have to, you know, write a program like next token, next token, next token and stop and never do any edits and you cannot really use any terminal or whatever tool. It doesn't make sense, right? And that's the way people are solving coding at the time, basically like sampling a program from a language model without chain of thought, without tool call, without refactoring, without anything. So the first point is we should solve coding in a very interactive way and that's a very general principle that applies for various coding benchmarks. And also, I think you can make a lot of the agent task kind of like interactive coding. If you have Python and you can call any package, then you can literally also browse internet or do whatever you want, like control a robot or whatever. So that seems to be a very general paradigm. But obviously I think a bottleneck is at the time we're still doing, you know, very simple tasks like human eval or whatever coding benchmark people proposed. They were super hard in 2021, like 20%, but they're like 95% already in 2023. So obviously the next step is we need a better benchmark. And Carlos and John, which are the first authors of Swaybench, I think they come up with this great idea that we should just script GitHub and solve whatever human engineers are solving. And I think it's actually pretty easy to come up with the idea. And I think in the first week, they already made a lot of progress. They script the GitHub and they make all the same, but then there's a lot of painful info work and whatever, you know. I think the idea is super easy, but the engineering is super hard. And I feel like that's a very typical signal of a good work in the AI era now.Swyx [00:39:17]: I think also, I think the filtering was challenging, because if you look at open source PRs, a lot of them are just like, you know, fixing typos. I think it's challenging.Shunyu [00:39:27]: And to be honest, we didn't do a perfect job at the time. So if you look at the recent blog post with OpenAI, we improved the filtering so that it's more solvable.Swyx [00:39:36]: I think OpenAI was just like, look, this is a thing now. We have to fix this. These students just rushed it.Shunyu [00:39:45]: It's a good convergence of interests for me.Alessio [00:39:48]: Was that tied to you joining OpenAI? Or was that just unrelated?Shunyu [00:39:52]: It's a coincidence for me, but it's a good coincidence.Swyx [00:39:55]: There is a history of anytime a big lab adopts a benchmark, they fix it. Otherwise, it's a broken benchmark.Shunyu [00:40:03]: So naturally, once we propose swimmage, the next step is to solve it. But I think the typical way you solve something now is you collect some training samples, or you design some complicated agent method, and then you try to solve it. Either super complicated prompt, or you build a better model with more training data. But I think at the time, we realized that even before those things, there's a fundamental problem with the interface or the tool that you're supposed to use. Because that's like an ignored problem in some sense. What your tool is, or how that matters for your task. So what we found concretely is that if you just use the text terminal off the shelf as a tool for those agents, there's a lot of problems. For example, if you edit something, there's no feedback. So you don't know whether your edit is good or not. That makes the agent very confused and makes a lot of mistakes. There are a lot of small problems, you would say. Well, you can try to do prompt engineering and improve that, but it turns out to be actually very hard. We realized that the interface design is actually a very omitted part of agent design. So we did this switch agent work. And the key idea is just, even before you talk about what the agent is, you should talk about what the environment is. You should make sure that the environment is actually friendly to whatever agent you're trying to apply. That's the same idea for humans. Text terminal is good for some tasks, like git, pool, or whatever. But it's not good if you want to look at browser and whatever. Also, browser is a good tool for some tasks, but it's not a good tool for other tasks. We need to talk about how design interface, in some sense, where we should treat agents as our customers. It's like when we treat humans as a customer, we design human computer interfaces. We design those beautiful desktops or browsers or whatever, so that it's very intuitive and easy for humans to use. And this whole great subject of HCI is all about that. I think now the research idea of switch agent is just, we should treat agents as our customers. And we should do like, you know… AICI.Swyx [00:42:16]: AICI, exactly.Harrison [00:42:18]: So what are the tools that a suite agent should have, or a coding agent in general should have?Shunyu [00:42:24]: For suite agent, it's like a modified text terminal, which kind of adapts to a lot of the patterns of language models to make it easier for language models to use. For example, now for edit, instead of having no feedback, it will actually have a feedback of, you know, actually here you introduced like a syntax error, and you should probably want to fix that, and there's an ended error there. And that makes it super easy for the model to actually do that. And there's other small things, like how exactly you write arguments, right? Like, do you want to write like a multi-line edit, or do you want to write a single line edit? I think it's more interesting to think about the way of the development process of an ACI rather than the actual ACI for like a concrete application. Because I think the general paradigm is very similar to HCI and psychology, right? Basically, for how people develop HCIs, they do behavior experiments on humans, right? I do every test, right? Like, which interface is actually better? And I do those behavior experiments, kind of like psychology experiments to humans, and I change things. And I think what's really interesting for me, for this three-agent paper, is we can probably do the same thing for agents, right? We can do every test for those agents and do behavior tests. And through the process, we not only invent better interfaces for those agents, that's the practical value, but we also better understand agents. Just like when we do those A-B tests, we do those HCI, we better understand humans. Doing those ACI experiments, we actually better understand agents. And that's pretty cool.Harrison [00:43:51]: Besides that A-B testing, what are other processes that people can use to think about this in a good way?Swyx [00:43:57]: That's a great question.Shunyu [00:43:58]: And I think three-agent is an initial work. And what we do is the kind of the naive approach, right? You just try some interface, and you see what's going wrong, and then you try to fix that. We do this kind of iterative fixing. But I think what's really interesting is there'll be a lot of future directions that's very promising if we can apply some of the HCI principles more systematically into the interface design. I think that would be a very cool interdisciplinary research opportunity.Harrison [00:44:26]: You talked a lot about agent-computer interfaces and interactions. What about human-to-agent UX patterns? Curious for any thoughts there that you might have.Swyx [00:44:38]: That's a great question.Shunyu [00:44:39]: And in some sense, I feel like prompt engineering is about human-to-agent interface. But I think there can be a lot of interesting research done about... So prompting is about how humans can better communicate with the agent. But I think there could be interesting research on how agents can better communicate with humans, right? When to ask questions, how to ask questions, what's the frequency of asking questions. And I think those kinds of stuff could be very cool research.Harrison [00:45:07]: Yeah, I think some of the most interesting stuff that I saw here was also related to coding with Devin from Cognition. And they had the three or four different panels where you had the chat, the browser, the terminal, and I guess the code editor as well.Swyx [00:45:19]: There's more now.Harrison [00:45:19]: There's more. Okay, I'm not up to date. Yeah, I think they also did a good job on ACI.Swyx [00:45:25]: I think that's the main learning I have from Devin. They cracked that. Actually, there was no foundational planning breakthrough. The planner is actually pretty simple, but ACI that they broke through on.Shunyu [00:45:35]: I think making the tool good and reliable is probably like 90% of the whole agent. Once the tool is actually good, then the agent design can be much, much simpler. On the other hand, if the tool is bad, then no matter how much you put into the agent design, planning or search or whatever, it's still going to be trash.Harrison [00:45:53]: Yeah, I'd argue the same. Same with like context and instructions. Like, yeah, go hand in hand.Alessio [00:46:00]: On the tool, how do you think about the tension of like, for both of you, I mean, you're building a library, so even more for you. The tension between making now a language or a library that is like easy for the agent to grasp and write versus one that is easy for like the human to grasp and write. Because, you know, the trend is like more and more code gets written by the agent. So why wouldn't you optimize the framework to be as easy as possible for the model versus for the person?Swyx [00:46:24]: I think it's possible to design an interfaceShunyu [00:46:25]: that's both friendly to humans and agents. But what do you think?Harrison [00:46:29]: We haven't thought about that from the perspective, like we're not trying to design LangChain or LangGraph to be friendly. But I mean, I think to be friendly for agents to write.Swyx [00:46:42]: But I mean, I think we see this with like,Harrison [00:46:43]: I saw some paper that used TypeScript notation instead of JSON notation for tool calling and it got a lot better performance. So it's definitely a thing. I haven't really heard of anyone designing like a syntax or a language explicitly for agents, but there's clearly syntaxes that are better.Shunyu [00:46:59]: I think function calling is a good example where it's like a good interface for both human programmers and for agents, right? Like for developers, it's actually a very friendly interface because it's very concrete and you don't have to do prompt engineering anymore. You can be very systematic. And for models, it's also pretty good, right? Like it can use all the existing coding content. So I think we need more of those kinds of designs.Swyx [00:47:21]: I will mostly agree and I'll slightly disagree in terms of this, which is like, whether designing for humans also overlaps with designing for AI. So Malte Ubo, who's the CTO of Vercel, who is creating basically JavaScript's competitor to LangChain, they're observing that basically, like if the API is easy to understand for humans, it's actually much easier to understand for LLMs, for example, because they're not overloaded functions. They don't behave differently under different contexts. They do one thing and they always work the same way. It's easy for humans, it's easy for LLMs. And like that makes a lot of sense. And obviously adding types is another one. Like type annotations only help give extra context, which is really great. So that's the agreement. And then a disagreement is that when I use structured output to do my chain of thought, I have found that I change my field names to hint to the LLM of what the field is supposed to do. So instead of saying topics, I'll say candidate topics. And that gives me a better result because the LLM was like, ah, this is just a draft thing I can use for chain of thought. And instead of like summaries, I'll say topic summaries to link the previous field to the current field. So like little stuff like that, I find myself optimizing for the LLM where I, as a human, would never do that. Interesting.Shunyu [00:48:32]: It's kind of like the way you optimize the prompt, it might be different for humans and for machines. You can have a common ground that's both clear for humans and agents, but to improve the human performance versus improving the agent performance, they might move to different directions.Swyx [00:48:48]: Might move different directions. There's a lot more use of metadata as well, like descriptions, comments, code comments, annotations and stuff like that. Yeah.Harrison [00:48:56]: I would argue that's just you communicatingSwyx [00:48:58]: to the agent what it should do.Harrison [00:49:00]: And maybe you need to communicate a little bit more than to humans because models aren't quite good enough yet.Swyx [00:49:06]: But like, I don't think that's crazy.Harrison [00:49:07]: I don't think that's like- It's not crazy.Swyx [00:49:09]: I will bring this in because it just happened to me yesterday. I was at the cursor office. They held their first user meetup and I was telling them about the LLM OS concept and why basically every interface, every tool was being redesigned for AIs to use rather than humans. And they're like, why? Like, can we just use Bing and Google for LLM search? Why must I use Exa? Or what's the other one that you guys work with?Harrison [00:49:32]: Tavilli.Swyx [00:49:33]: Tavilli. Web Search API dedicated for LLMs. What's the difference?Shunyu [00:49:36]: Exactly. To Bing API.Swyx [00:49:38]: Exactly.Harrison [00:49:38]: There weren't great APIs for search. Like the best one, like the one that we used initially in LangChain was SERP API, which is like maybe illegal. I'm not sure.Swyx [00:49:49]: And like, you know,Harrison [00:49:52]: and now there are like venture-backed companies.Swyx [00:49:53]: Shout out to DuckDuckGo, which is free.Harrison [00:49:55]: Yes, yes.Swyx [00:49:56]: Yeah.Harrison [00:49:56]: I do think there are some differences though. I think you want, like, I think generally these APIs try to return small amounts of text information, clear legible field. It's not a massive JSON blob. And I think that matters. I think like when you talk about designing tools, it's not only the, it's the interface in the entirety, not only the inputs, but also the outputs that really matter. And so I think they try to make the outputs.Shunyu [00:50:18]: They're doing ACI.Swyx [00:50:19]: Yeah, yeah, absolutely.Harrison [00:50:20]: Really?Swyx [00:50:21]: Like there's a whole set of industries that are just being redone for ACI. It's weird. And so my simple answer to them was like the error messages. When you give error messages, they should be basically prompts for the LLM to take and then self-correct. Then your error messages get more verbose, actually, than you normally would with a human. Stuff like that. Like a little, honestly, it's not that big. Again, like, is this worth a venture-backed industry? Unless you can tell us. But like, I think Code Interpreter, I think is a new thing. I hope so.Alessio [00:50:52]: We invested in it to be so.Shunyu [00:50:53]: I think that's a very interesting point. You're trying to optimize to the extreme, then obviously they're going to be different. For example, the error—Swyx [00:51:00]: Because we take it very seriously. Right.Shunyu [00:51:01]: The error for like language model, the longer the better. But for humans, that will make them very nervous and very tired, right? But I guess the point is more like, maybe we should try to find a co-optimized common ground as much as possible. And then if we have divergence, then we should try to diverge. But it's more philosophical now.Alessio [00:51:19]: But I think like part of it is like how you use it. So Google invented the PageRank because ideally you only click on one link, you know, like the top three should have the answer. But with models, it's like, well, you can get 20. So those searches are more like semantic grouping in a way. It's like for this query, I'll return you like 20, 30 things that are kind of good, you know? So it's less about ranking and it's more about grouping.Shunyu [00:51:42]: Another fundamental thing about HCI is the difference between human and machine's kind of memory limit, right? So I think what's really interesting about this concept HCI versus HCI is interfaces that's optimized for them. You can kind of understand some of the fundamental characteristics, differences of humans and machines, right? Why, you know, if you look at find or whatever terminal command, you know, you can only look at one thing at a time or that's because we have a very small working memory. You can only deal with one thing at a time. You can only look at one paragraph of text at the same time. So the interface for us is by design, you know, a small piece of information, but more temporal steps. But for machines, that should be the opposite, right? You should just give them a hundred different results and they should just decide in context what's the most relevant stuff and trade off the context for temporal steps. That's actually also better for language models because like the cost is smaller or whatever. So it's interesting to connect those interfaces to the fundamental kind of differences of those.Harrison [00:52:43]: When you said earlier, you know, we should try to design these to maybe be similar as possible and diverge if we need to.Swyx [00:52:49]: I actually don't have a problem with them diverging nowHarrison [00:52:51]: and seeing venture-backed startups emerging now because we are different from machines code AI. And it's just so early on, like they may still look kind of similar and they may still be small differences, but it's still just so early. And I think we'll only discover more ways that they differ. And so I'm totally fine with them kind of like diverging earlySwyx [00:53:10]: and optimizing for the...Harrison [00:53:11]: I agree. I think it's more like, you know,Shunyu [00:53:14]: we should obviously try to optimize human interface just for humans. We're already doing that for 50 years. We should optimize agent interface just for agents, but we might also try to co-optimize both and see how far we can get. There's enough people to try all three directions. Yeah.Swyx [00:53:31]: There's a thesis I sometimes push, which is the sour lesson as opposed to the bitter lesson, which we're always inspired by human development, but actually AI develops its own path.Shunyu [00:53:40]: Right. We need to understand better, you know, what are the fundamental differences between those creatures.Swyx [00:53:45]: It's funny when really early on this pod, you were like, how much grounding do you have in cognitive development and human brain stuff? And I'm like

Nofollow — это ссылки, закрытые от сканирования поисковыми системами. Они не передают PageRank, однако все равно важны для эффективного продвижения вашего сайта. Почему? Рассказал Николай Шмичков в новом подкасте. Кому будет полезен подкаст: SEO-специалистам, которые хотят улучшить стратегию работы со ссылками. Владельцам сайтов, стремящимся защитить свой сайт от спам-ссылок. Новичкам в SEO, желающим разобраться в […]

Le nuove metriche di Google per il page rank dei sitiSeo: dove siamo arrivati ? Un recap sulle novità degli ultimi anni di Google nel valutare i siti.Lighthouse e Chrome: ecco come viene misurata la qualità di un sito e come ottimizzarlo oggi.Una montagna di considerazioni che rischiano di distrarre dall'obiettivo: fare fare all'utente finale l'azione che gli suggeriamo.Ancora i vecchi criteri valgono (v. pagespeed e gtmetrix) ma la misurazione dei browser chrome (soprattutto mobile) influiscono anche essi.

Send us a Text Message.Ever wondered what Google truly perceives as a document? Join me, Ed Dawson, as we unlock the secrets to what actually influences your webpage's performance in the latest episode of "SEO is Not That Hard." Get ready to uncover how Google views PDFs, images, and more as documents and why page speed is not just a technical detail but a game-changer for your website's user experience and ranking. Learn how employing content delivery networks can make your site lightning fast without breaking a sweat.Then, dive deep into the foundational yet still crucial concept of PageRank. Understand why this algorithm, despite its evolution, remains pivotal in determining your webpage's rank and how the value of links still plays a critical role. Finally, we tackle the controversial topic of paid links and the pitfalls of not adhering to Google's guidelines on nofollow and sponsored links. Packed with actionable insights, this episode is a must-listen for anyone looking to stay ahead in the ever-changing world of SEO.SEO Is Not That Hard is hosted by Edd Dawson and brought to you by KeywordsPeopleUse.comYou can get your free copy of my 101 Quick SEO Tips at: https://seotips.edddawson.com/101-quick-seo-tipsTo get a personal no-obligation demo of how KeywordsPeopleUse could help you boost your SEO then book an appointment with me nowSee Edd's personal site at edddawson.comAsk me a question and get on the show Click here to record a questionFind Edd on Twitter @channel5Find KeywordsPeopleUse on Twitter @kwds_ppl_use"Werq" Kevin MacLeod (incompetech.com)Licensed under Creative Commons: By Attribution 4.0 Licensehttp://creativecommons.org/licenses/by/4.0/

Nathan interviews Bernal Jimenez Gutierrez, creator of HippoRAG, a novel approach to retrieval augmented generation inspired by the human hippocampus. In this episode of The Cognitive Revolution, we explore how HippoRAG improves on traditional RAG systems, its neuroanatomical inspiration, and its potential for handling complex queries requiring multi-hop reasoning. Join us for an insightful discussion on the future of AI memory systems. Apply to join over 400 founders and execs in the Turpentine Network: https://hmplogxqz0y.typeform.com/to/JCkphVqj RECOMMENDED PODCAST: Complex Systems Patrick McKenzie (@patio11) talks to experts who understand the complicated but not unknowable systems we rely on. You might be surprised at how quickly Patrick and his guests can put you in the top 1% of understanding for stock trading, tech hiring, and more. Spotify: https://open.spotify.com/show/3Mos4VE3figVXleHDqfXOH Apple: https://podcasts.apple.com/us/podcast/complex-systems-with-patrick-mckenzie-patio11/id1753399812 SPONSORS: Oracle Cloud Infrastructure (OCI) is a single platform for your infrastructure, database, application development, and AI needs. OCI has four to eight times the bandwidth of other clouds; offers one consistent price, and nobody does data better than Oracle. If you want to do more and spend less, take a free test drive of OCI at https://oracle.com/cognitiveThe Brave search API can be used to assemble a data set to train your AI models and help with retrieval augmentation at the time of inference. All while remaining affordable with developer first pricing, integrating the Brave search API into your workflow translates to more ethical data sourcing and more human representative data sets. Try the Brave search API for free for up to 2000 queries per month at https://bit.ly/BraveTCR Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off https://www.omneky.com/ Head to Squad to access global engineering without the headache and at a fraction of the cost: head to https://choosesquad.com/ and mention “Turpentine” to skip the waitlist. CHAPTERS: (00:00:00) About the Show (00:02:52) Intro (00:05:10) RAG (00:09:29) Hippocampal Memory Indexing Theory (00:14:29) Neocortex, Parahippocampal Regions, Hippocampus (00:18:35) Hipporag (Part 1) (00:21:12) Sponsors: Oracle | Brave (00:23:20) Hipporag (Part 2) (00:23:29) Understanding the Hippocampus (00:29:37) RAG (00:31:38) Runtime (Part 1) (00:36:30) Sponsors: Omneky | Squad (00:38:17) Runtime (Part 2) (00:38:17) PageRank (00:39:29) Headline Results (00:43:22) Gaia Benchmark (00:46:33) Pathfinding vs Path Following (00:51:24) Future work (00:53:26) Starting with a query (00:58:30) Long context LLMs (01:01:50) Hybrid approach (01:05:03) AI Town (01:06:58) Flair (01:09:05) Getting it right, not cheap (01:12:33) Technologies to highlight (01:16:57) AI capabilities that are still unsolved (01:19:47) Transformers meet neural algorithmic reasoners (01:21:02) Closing (01:21:31) Outro

In this episode of the Cognitive Revolution, join us as we dive into a compelling conversation with Josep M. Pujol, Chief of Search at Brave, about the complexities of developing a privacy-focused search engine. Explore how Brave maintains user data privacy while managing over 1 million searches per hour with an AI-powered system. Gain insights into the significance of human evaluation in AI and learn about the potential of the Brave Search API. Don't miss the shared Google collab notebook link in the show notes!

Send us a Text Message.Could the next major shift in SEO be hidden within a recent Google API documentation leak? Find out as we uncover what could be the "actual surfer model," a potential new approach to PageRank that promises to revolutionize how we understand link quality and web page ranking. Join host Ed Dawson in this episode of "SEO is Not That Hard," where we trace the evolution of Google's PageRank from the original "random surfer" model, designed to identify the most significant pages by simulating random clicks, to the more refined "reasonable surfer" model, which predicts the most likely user behavior. In an electrifying discussion, Ed dissects the leaked documentation, revealing how Google categorizes links into high, medium, and low-quality tiers. Learn about the implications of these tiers on your SEO strategy as Ed explains the nuances of high-quality base documents, medium-quality supplemental documents, and low-quality black hole documents. This episode is a treasure trove of insights, offering practical tips for both seasoned SEO professionals and newcomers eager to stay ahead in the ever-evolving landscape of search engine optimization. Don't miss out on the chance to understand these groundbreaking changes and what they mean for your future SEO endeavors.SEO Is Not That Hard is hosted by Edd Dawson and brought to you by KeywordsPeopleUse.comYou can get your free copy of my 101 Quick SEO Tips at: https://seotips.edddawson.com/101-quick-seo-tipsTo get a personal no-obligation demo of how KeywordsPeopleUse could help you boost your SEO then book an appointment with me nowAsk me a question and get on the show Click here to record a questionFind Edd on Twitter @channel5Find KeywordsPeopleUse on Twitter @kwds_ppl_use"Werq" Kevin MacLeod (incompetech.com)Licensed under Creative Commons: By Attribution 4.0 Licensehttp://creativecommons.org/licenses/by/4.0/

Imran and Qiao sat down with Kipit and Mikado to share an inside look at Fantasy Top.No BS crypto insights for founders.Timestamps(00:00) Intro(01:05) Welcome to Good Game(01:56) Innovation in Product Development(04:34) What They Built Before Fantasy(06:17) Mikado and Kipit Applied Twice to Alliance Before(07:54) How Qiao Noticed Fantasy(08:56) Mistakes From Their Previous Product(10:09) What They Do Differently with Fantasy(13:48) "The Market Could Be Wrong Too"(14:19) Embracing Speculation(18:25) Shift in Crypto Perception(20:32) "I feel Like nobody really cares about consumer either"(21:59) Explaining Fantasy to Normies(22:37) How Does Fantasy Expand From Twitter?(25:46) Is Fantasy Worried About Getting Deplatformed By Twitter?(26:57) Fantasy's Scoring System(30:41) PageRank(32:34) Fantasy Expanding to Different Ecosystems(37:10) Fantasy's Current Goal(38:26) "You have to make users rich early on if you want to have a sticky product"(40:34) What Kipit and Mikado Think of Friendtech(42:33) "Fantasy is what Friendtech should have been"(44:02) Thoughts on Friendtech's Current Form(48:48) The Best Aspect of Fantasy(50:22) The Inflation for Cards(54:34) "The new meta is gonna be Fantasy is now deflationary"(57:16) Qiao: Every consumer product that I loved can be described in one sentence(01:00:26) Questions From Crypto Twitter(01:00:39) "How do you transition well from serving segment A crypto customer to a more mass market non-crypto customer?"(01:02:09) Would Fantasy Ever Do Their Own Rollup?(01:03:04) "What does Fantasy look like post points/(potential) airdrop?"(01:04:33) "When $fan token?"(01:05:40) Final Thoughts(01:05:58) Who's Who?Kipit Twitter/X: https://twitter.com/0xkipitMikado Twitter/X: https://twitter.com/0xMikadoSpotify: https://spoti.fi/3N675w3Apple Podcast: https://apple.co/3snLsxUWebsite: https://goodgamepod.xyzTwitter: https://twitter.com/goodgamepodxyzWeb3 Founders:Apply to Alliance: https://alliance.xyzAlliance Twitter: https://twitter.com/alliancedaoDISCLAIMER: The views expressed herein are personal to the speaker(s) and do not necessarily reflect the views of any other person or entity. Discussions and answers to questions are intended as generalized, non-personalized information. Nothing herein should be construed or relied upon as investment, legal, tax, or other advice.

Georgia digs deep into the enshittification of dating apps, and pays dearly as a result. Mike talks to Aftermath co-founder Luke Plunkett about recent, massive changes to Google's Page Rank algorithm, and the risk of reconfiguring entire industries to pander to search traffic. And also: An AI Voice Clone of Mike is set to *maximum chaos*. Show Notes:Become a Never Post member at https://www.neverpo.st/–Call us at 651 615 5007 to leave a voice mailDrop us a voice memo via airtableOr email us at theneverpost at gmail dot comSee what interstitials we need submissions for–Intro LinksBelle Delphine earned over $90K selling jars of her bathwater in 2019. PayPal only released her money this week – Katie Notopulous, Business InsiderSlack has been using data from your chats to train its machine learning models – Will Shanklin, EngadgetIt's the End of Google Search As We Know It – Lauren Goode, WIREDUber and Lyft agree to deal with state lawmakers on minimum pay rates for drivers – Max Nesterak, Minnesota ReformerIGN Entertainment acquires Eurogamer, GI, VG247, Rock Paper Shotgun and more –  Christopher Dring, gameindustry.bizIGN buys Eurogamer, VG247, and Rock Paper Shotgun – layoffs have already started – Game Central, Metro.co.ukNever Post at XOXO 2024 12 Best Podcasts of 2024 (So Far) – Lauren Passell, Lifehacker Mike on Blocked Party–Dating Apps (derogatory)Dating Apps: The Uncertainty of Marketised Lovehttps://www.tiktok.com/@officialbrept/video/7028325781537393925?_t=8lrDLmaKbTa&_r=1https://www.tiktok.com/@hannahgraser/video/7117094997484309806?_r=1&_t=8ly4uYqQrPDhttps://www.tiktok.com/@harmonythread/video/7262952885158235435?_t=8lwOVY3O0LP&_r=1https://www.tiktok.com/@hannahgraser/video/7125991346489314603?_t=8lwOSDK1l3H&_r=1https://www.tiktok.com/@jordanzhang13/video/7036251495951535366?_t=8m88ojNTm3y&_r=1–Google ScruplesFind Luke at Aftermath.siteGoogle Kneecaps Loads Of Very Big Websites After SEO ChangeWorkers At The Gamurs Group Of Video Game Websites Describe It As ‘Hell'The Perfect Webpage: How the internet reshaped itself around Google's search algorithms –Never Post's producers are Audrey Evans, Georgia Hampton and The Mysterious Dr. Firstname Lastname. Our senior producer is Hans Buetow. Our executive producer is Jason Oberholtzer. The show's host is Mike Rugnetta. Foot, how you press me to keep that old contact alivethe repeated daily sentiment of pace so grim, always thatuntrusting silenceuntitled, by JH Prynne, from The White StonesNever Post is a production of Charts & Leisure ★ Support this podcast ★

In this episode of Content in the Kitchen Nik Ranger delves into the significance of internal linking within an SEO strategy. Nick emphasizes the value of internal links in enhancing user experience and outlines a robust internal linking optimization framework. She discusses the use of machine learning tools like Screaming Frog, SEMrush, Ahrefs, and specialized AI tools like InLinks and LinkBERT to automate and optimize internal link placement for maximizing SEO impact.This episode is packed with practical advice on developing intentional and purpose-driven internal links, from understanding PageRank and content relevance to avoiding link spam and distributing link equity effectively. Nik also highlights the broader strategies for building a solid internal link foundation, especially for e-commerce sites, and shares his secret sauce and favorite tools for SEO success.Subscribe now for your weekly dose of content wisdom, direct from the content marketing experts to your kitchen table.Website:https://contentyum.com/Socials:https://www.instagram.com/contentyumm/https://www.linkedin.com/company/contentyumhttps://www.tiktok.com/@contentyumhttps://www.facebook.com/contentyummhttps://www.youtube.com/@Content-Yum

Ever wondered why the once-mighty PageRank has slipped into the shadows of SEO chatter? Join me, Ed Dawson, as I unlock the secrets behind Google's initial search algorithm and its waning role in modern-day SEO strategies. PageRank's legacy began as the foundation of Google's search engine dominance, where the web's hierarchy was determined by the link love pages received. In this episode, I'll take you through an SEO history lesson, revealing how PageRank's transparency once spawned a whole economy of link buying—and how Google's countermeasures, like the 'nofollow' attribute, aimed to curtail this manipulation.Fasten your seatbelts for a nostalgia trip back to the toolbar era and discover why this system's visible metrics are no longer the talk of the town. As an SEO veteran with over two decades under my belt, I'll dissect Google's strategic moves to obscure PageRank details and the impact it had on the art of optimizing websites. Whether you're new to the SEO game or a seasoned pro looking to reminisce, you're in for a wealth of insights on the evolution of internet search and the algorithms that once commanded our every move.SEO Is Not That Hard is hosted by Edd Dawson and brought to you by KeywordsPeopleUse.comYou can get your free copy of my 101 Quick SEO Tips at: https://seotips.edddawson.com/101-quick-seo-tipsTo get a personal no-obligation demo of how KeywordsPeopleUse could help you boost your SEO then book an appointment with me nowAsk me a question and get on the show Click here to record a questionFind Edd on Twitter @channel5Find KeywordsPeopleUse on Twitter @kwds_ppl_use"Werq" Kevin MacLeod (incompetech.com)Licensed under Creative Commons: By Attribution 4.0 Licensehttp://creativecommons.org/licenses/by/4.0/

Discover all of the podcasts in our network, search for specific episodes, get the Optimal Living Daily workbook, and learn more at: OLDPodcast.com. Episode 1296: Margo Aaron demystifies SEO, transforming it from a daunting task into a manageable one, even for the non-tech savvy. Through her insightful conversation with Michael Tesalona, she outlines the simplicity behind SEO's perceived complexity, focusing on foundational elements and actionable steps that can significantly boost your website's visibility without delving deep into technicalities. Read along with the original article(s) here: https://www.thatseemsimportant.com/business/5-things-you-need-to-know-about-seo/ Quotes to ponder: "No one really knows what Google's search engine ranking criteria are, but after years of following trends and getting results for clients, we have a pretty good idea of what definitely matters." "If you get those basics down, you've satisfied the geek quota and the rest of the time you need to focus on backlinks." "Since Google doesn't publicly state Page Rank, the SEO community is split on what the ‘gold standard' is." Learn more about your ad choices. Visit megaphone.fm/adchoices

Discover all of the podcasts in our network, search for specific episodes, get the Optimal Living Daily workbook, and learn more at: OLDPodcast.com. Episode 1296: Margo Aaron demystifies SEO, transforming it from a daunting task into a manageable one, even for the non-tech savvy. Through her insightful conversation with Michael Tesalona, she outlines the simplicity behind SEO's perceived complexity, focusing on foundational elements and actionable steps that can significantly boost your website's visibility without delving deep into technicalities. Read along with the original article(s) here: https://www.thatseemsimportant.com/business/5-things-you-need-to-know-about-seo/ Quotes to ponder: "No one really knows what Google's search engine ranking criteria are, but after years of following trends and getting results for clients, we have a pretty good idea of what definitely matters." "If you get those basics down, you've satisfied the geek quota and the rest of the time you need to focus on backlinks." "Since Google doesn't publicly state Page Rank, the SEO community is split on what the ‘gold standard' is." Learn more about your ad choices. Visit megaphone.fm/adchoices

Evan Zimmerman is the Co-Founder and CEO of Edge, that helps patent attorneys, patent agents, and inventors make the patent process less painful and more effective. He earned a Juris Doctor (JD) degree from the UC Berkeley Law School, where he specialized in IP law. Seeing the potential of AI to transform the patent system, Zimmerman teamed up with Len Boyette, an early employee at Okta. Together, they went through the Y Combinator accelerator program in 2022 to build Edge. Edge's AI-powered patent assistant aims to automate and streamline every step of the patent process, from recreating patent claims from basic technology descriptions to suggesting improvements and identifying relevant prior art. This approach removes the grunt work for attorneys and patent agents while making high-quality patents more accessible for inventors and companies. The patent system is critical for protecting inventions and stimulating innovation, but is often cumbersome for all involved. For attorneys and patent agents, the system involves painstaking hours poring over documents, extracting key technical details, and translating those details into watertight patent claims. For inventors and companies, the costs and complexities of patents can be prohibitive. Edge aims to change all that by using AI to automate and streamline every step of the patent process. Evan describes Edge as "the ultimate patent assistant", able to recreate patent claims from basic technology descriptions, suggest areas for improvement, and more. This removes grunt work for practitioners while making high-quality patents more accessible for innovators. Edge's product offering is centered on a patent editor and assistant app. For example, Edge generated claims mimicking those from the famous PageRank patent solely from a description of the technology, capturing nuances an inventor might have missed. Its AI also suggests ways to strengthen claims, identifies relevant prior art, and more. Evan sees Edge as revolutionizing how patents are created and managed. For attorneys and agents, it removes the drudgery of claim drafting and prior art searching. For inventors and companies, it makes robust patents far more accessible. He sees patents as a rising tide that can lift all boats when done right. By using technology to democratize access to effective patents, Edge seeks to empower inventors. Just as the Kitty Hawk Flyer gave rise to modern aviation, robust yet accessible patents can spur the next wave of human ingenuity.

Journalist Ernie Smith got an odd DMCA takedown notice for a picture he included in his tech newsletter. It turns out that the notice was a ploy to try and trick Smith into including a backlink to another site in an effort to boost that site's search rankings. What the what?See omnystudio.com/listener for privacy information.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: How do LLMs give truthful answers? A discussion of LLM vs. human reasoning, ensembles & parrots, published by Owain Evans on March 28, 2024 on The AI Alignment Forum. Summary Large language models (LLMs) like ChatGPT and Claude 3 become increasingly truthful as they scale up in size and are finetuned for factual accuracy and calibration. However, the way LLMs arrive at truthful answers is nuanced. When an LLM answers a question immediately without chain-of-thought reasoning, the answer is typically not the result of the LLM reasoning about the question and weighing the evidence. Instead, the answer is based on human answers from pretraining documents that are (i) contextually relevant and (ii) resemble sources that led to truthful answers in finetuning. By contrast, when LLMs do explicit chain-of-thought reasoning before answering the question, the reasoning steps are more likely to causally determine the LLM's answer. This has parallels in human cognition. Many people can state Fermat's Theorem without having evaluated the proof themselves. Does this mean LLMs just parrot humans when answering without chain-of-thought reasoning? No. LLMs don't mimic a single human's answers. They aggregate over many human answers, weighted by relevance and whether the source is correlated with truthfulness. This is loosely analogous to mechanisms that aggregate many human judgments and outperform most individual humans, such as ensembling forecasts, markets, PageRank, and Bayesian Truth Serum. Moreover, LLMs have some conceptual understanding of their answers, even if they did not evaluate the answers before giving them. Epistemic Status: This essay is framed as a dialogue. There are no new experimental results but only my quick takes. Some of the takes are backed by solid evidence, while some are more speculative (as I indicate in the text). How do LLMs give truthful answers? Q: We'd like to have LLMs that are truthful, i.e. that systematically say true things and avoid saying false or inaccurate things wherever possible. Can we make LLMs like this? Owain: Current finetuned models like GPT-4 and Claude 3 still make mistakes on obscure long-tail questions and on controversial questions. However, they are substantially more truthful than earlier LLMs (e.g. GPT-2 or GPT-3). Moreover, they are more truthful than their own base models, after being finetuned specifically for truthfulness (or "honesty" or "factuality") via RLHF. In general, scaling up models and refining the RLHF finetuning leads to more truthful models, i.e. models that avoid falsehoods when answering questions. Q: But how does this work? Does the LLM really understand why the things it says are true, or why humans believe they are true? Owain: This is a complicated question and needs a longer answer. It matters whether the LLM immediately answers the question with no Chain of Thought ("no-CoT") or whether it gets to think before answering ("CoT"). Let's start with the no-CoT case, as in Figure 1 above. Suppose we ask the LLM a question Q and it answers immediately with answer A. I suspect that the LLM does not answer with A because it has evaluated and weighed the evidence for A. Instead, it usually answers with A because A was the answer given in human texts like Wikipedia (and similar sources), which were upweighted by the model's pretraining and RLHF training. Sometimes A was not an existing human answer, and so the LLM has to go beyond the human data. (Note that how exactly LLMs answer questions is not fully understood and so what I say is speculative. See "Addendum" below for more discussion.) Now, after the LLM has given answer A, we can ask the LLM to verify the claim. For example, it can verify mathematical assertions by a proof and scientific claims by citing empirical evidence. The LLM will also make some asse...

Like this? Subscribe to our newsletter at https://thinkfuture.com --- Get AIDAILY, delivered to your inbox, every weekday. Subscribe to our newsletter at https://aidaily.us --- In this eye-opening episode, we dive into the provocative idea that the best and most valuable content on the internet might not be the most popular. The host challenges the conventional wisdom that popularity equals quality, a belief deeply ingrained in our digital culture from search engine algorithms to social media feeds. Starting with a whimsical anecdote about ordering the least popular menu item at restaurants, the conversation expands to question the foundational principles of how content is surfaced and valued online. We explore how traditional metrics of popularity can often overshadow genuinely high-quality content that lacks widespread recognition. By examining the roots of Google's PageRank algorithm and its influence on content visibility, the episode reveals the systemic bias towards popularity over quality. The host advocates for a paradigm shift towards a more nuanced appreciation of digital content, suggesting that true gems often lie hidden beyond the first page of search results or the most followed social media accounts. The episode concludes with a call to action for both creators and consumers to seek out and elevate less popular but high-quality content, and for tech innovators to develop algorithms and platforms that prioritize relevance and quality over mere popularity. --- Send in a voice message: https://podcasters.spotify.com/pod/show/thinkfuture/message Support this podcast: https://podcasters.spotify.com/pod/show/thinkfuture/support

Podcasting 2.0 February 23d 2024 Episode 168: "Frog Giggin'" Adam & Dave are joined by Benjamin Bellamy from Paris to catch up on his Castopod project and the le latest cuisine ShowNotes We are LIT Benjamin Bellamy - Castopod Farmer protests Umbrel or Start9/OS Here are some talking points I thought we could talk about tomorrow: - Audience measurement in France (IAB, ACPM, Médiamétrie) and how OP3 may shake this - Is there a GDPR problem with IABv2 standard? - What is so different with podcasting in France? - Why is Youtube podcast a really bad thing? - Why Apple Podcasts adding the transcript tag is a game changer? - SEO for podcasts sucks. Will transcripts will change that? - Do we need a “Page Rank” for podcasting? - The podcast:recommendations is almost 3 years old (and almost forgotten). It's probably the wrong answer to a good question… We probably need something else… - We need a podcast:analytics tag so that we know where to find OP3 analytics - We need a podcast:source tag that says if a podcast was published someplace before - We need a podcast:ad tag to create an ad ecosystem that respects all actors (podcasters, hosts, apps…) - Castopod roadmap - Why affiliation is the only solution to save the Internet from online ads List of ISO 3166 country codes - Wikipedia V4V Roundtable Why is the industry always talking about 'growing your podcast'? Tennis or golf Not everything needs to be as a profession Took me years before I even thought about being paid Serve a community Looking to start podcasting? Do a music podcast - it's wide open territory! ETF approval for bitcoin – the naked emperor’s new clothes ------------------------------------- MKUltra chat Transcript Search What is Value4Value? - Read all about it at Value4Value.info V4V Stats Last Modified 02/23/2024 14:47:55 by Freedom Controller

Podcasting 2.0 February 23d 2024 Episode 168: "Frog Giggin'" Adam & Dave are joined by Benjamin Bellamy from Paris to catch up on his Castopod project and the le latest cuisine ShowNotes We are LIT Benjamin Bellamy - Castopod Farmer protests Umbrel or Start9/OS Here are some talking points I thought we could talk about tomorrow: - Audience measurement in France (IAB, ACPM, Médiamétrie) and how OP3 may shake this - Is there a GDPR problem with IABv2 standard? - What is so different with podcasting in France? - Why is Youtube podcast a really bad thing? - Why Apple Podcasts adding the transcript tag is a game changer? - SEO for podcasts sucks. Will transcripts will change that? - Do we need a “Page Rank” for podcasting? - The podcast:recommendations is almost 3 years old (and almost forgotten). It's probably the wrong answer to a good question… We probably need something else… - We need a podcast:analytics tag so that we know where to find OP3 analytics - We need a podcast:source tag that says if a podcast was published someplace before - We need a podcast:ad tag to create an ad ecosystem that respects all actors (podcasters, hosts, apps…) - Castopod roadmap - Why affiliation is the only solution to save the Internet from online ads List of ISO 3166 country codes - Wikipedia V4V Roundtable Why is the industry always talking about 'growing your podcast'? Tennis or golf Not everything needs to be as a profession Took me years before I even thought about being paid Serve a community Looking to start podcasting? Do a music podcast - it's wide open territory! ETF approval for bitcoin – the naked emperor’s new clothes ------------------------------------- MKUltra chat Transcript Search What is Value4Value? - Read all about it at Value4Value.info V4V Stats Last Modified 02/23/2024 14:47:55 by Freedom Controller

Welcome back, everybody, to another episode of the Niche Pursuits News Podcast! This is the place to hear the latest news in the SEO and content publishing industries and it's where Spencer and Jared break it all down. This week, as always, they also share progress on their side hustles as well as two very, very weird niche sites. The first topic up for discussion is Google's decision to rebrand Bard as Gemini. Google certainly is putting a lot of effort into building out AI tools, and this will be one of their flagship products.  What do Spencer and Jared think of the company's decision? And what new features will this tool have?  The next topic is how Google's updated its SEO Starter Guide. The guide originally dates back to 2008 And is a guide specifically for beginners. Although there's probably nothing meaty there for people that are familiar with SEO, Spencer and Jared did highlight a few interesting items, in particular, from the section on things you should not do. What kind of advice did they share? What does Google say about article length? And how about PageRank and EEAT? And what about business cards?  The last news item they discuss is how Google won't be able to proceed with third-party cookie deprecation. This means that the phasing out of third-party cookies may or may not be delayed as Google has been asked to comply with additional safety requirements in the UK. This is big news because it could potentially affect content creators' earnings, Although it's true that advertising is Google's bread and butter and it will likely do everything it can to maintain it, this is definitely a topic you'll want to keep an eye on! In the Shiny Object Shenanigans portion of the podcast, Spencer goes first with an update on his Facebook traffic side hustle. He was able to reduce the cost of his likes campaign to grow his page. He shares a lot of the data behind the campaigns, but the real question is whether those likes are driving traffic to his website. Is it working? What does his Google Analytics reveal? And what is he going to do moving forward? When it's Jared's turn, he provides an update on his Weekend Growth YouTube channel, which officially qualifies for monetization. He shares the stats behind his growth and reveals that the site is less than a year old.  He also offers an update on his public speaking side hustle with an exciting announcement. Check out the episode to hear what it is! When it comes to weird niche sites, Spencer takes the lead with a very weird site called Is It Friday Yet. According to Ahrefs, the site doesn't get any traffic, although there is a very entertaining video on YouTube that reviews the website, so tune in to hear more about that. When it was Jared's turn, he revealed a 1-page weird niche site: The Office Stare Machine, which lets you input an emotion and it will show stares from Office characters conveying that emotion. They test it out, and the results are pretty funny!  This DR30 site ranks for just 28 keywords, but it does have 49k likes from Facebook. It doesn't get a lot of organic traffic, and Spencer and Jared talk about the potential strategies the owner might be using on Facebook. Where does it have the potential to do well? And that brings us to the end of another great episode of The Niche Pursuits News Podcast. We hope this weekly dose of the latest news has brought you up to speed in the SEO and content creation space and you're feeling inspired by Spencer and Jared's side hustles. See you next Friday! Be sure to get more content like this in the Niche Pursuits Newsletter Right Here: https://www.nichepursuits.com/newsletter Want a Faster and Easier Way to Build Internal Links? Get $15 off Link Whisper with Discount Code "Podcast" on the Checkout Screen: https://www.nichepursuits.com/linkwhisper Get SEO Consulting from the Niche Pursuits Podcast Host, Jared Bauman: https://www.nichepursuits.com/201creative

Episode 192 contains the Digital Marketing News and Updates from the week of Dec 18-22, 2023.1. Site Structure Strategy - Understanding the basics of SEO (Search Engine Optimization) can significantly enhance your online presence. Gary Illyes from Google has recently shed light on the importance of using a hierarchical site structure for SEO, a strategy crucial for making your website more accessible and understandable to both users and search engines.Illyes explains two types of site structures: hierarchical and flat. A flat site structure links every page directly from the home page, making each page just one click away. This approach was popular when sites relied heavily on web directories and reciprocal linking. However, as Google reduced the influence of PageRank as a ranking factor, the flat structure became less relevant.In contrast, a hierarchical site structure organizes content from general to specific. The home page covers the most general topic, with links to categories, subcategories, and individual pages that delve into more specific topics. This structure not only makes it easier for users to navigate your site but also helps search engines understand and categorize your content effectively.A hierarchical structure offers several advantages: Improved User Experience: It makes it easier for visitors to find what they're looking for, enhancing their overall experience on your site. Better SEO: By clearly categorizing your content, search engines can more easily index and rank your pages. Flexibility: It allows you to create distinct sections on your site, like a news section, which can be crawled and indexed differently by search engines. The choice between a hierarchical and a flat structure depends on your site's size and complexity. For larger sites with diverse content, a hierarchical structure is more beneficial. It allows for better organization and easier management of different content sections. He explained, "hierarchical structure will allow you to do funky stuff on just one section and will also allow search engines to potentially treat different sections differently. Especially when it comes to crawling. For example, having news section for newsy content and archives for old content would allow search engines to crawl news faster than the other directory. If you put everything in one directory that's not really possible."For small business owners, adopting a hierarchical site structure, as suggested by Gary Illyes from Google, can significantly improve your website's SEO performance. It's not just about organizing content; it's about making your site more accessible and relevant to both your audience and search engines. By implementing this structure, you can enhance user experience, improve search rankings, and ultimately drive more traffic to your site.2. Decoding the Dec 21 Spam Attack: Key Lessons to Elevate Your SEO Strategy! - On December 21, 2023, Google's search results were overwhelmed by a massive spam attack. This event highlights the vulnerability of search engines to spam tactics and the potential impact on businesses relying on online visibility.The attack involved numerous domains ranking for hundreds of thousands of keywords, indicating a large-scale operation. The spam was first noticed when almost all top search results for specific queries, like "Craigslist used auto parts," turned out to be spam, except for a few legitimate listings.The spam sites exploited three main opportunities within Google's ranking system: Local Search Algorithm: This algorithm is more permissive, allowing local businesses to rank without many links. Spammers used this to their advantage, targeting local search queries. Longtail Keywords: These are low-volume, specific phrases. Due to their low competition, it's easier for spammers to rank in these areas. New Domain Advantage: Google gives new sites a short period of 'benefit of the doubt' to rank in search results. Many spam domains were newly registered, exploiting this window. The effectiveness of this technique lies in the different algorithms Google uses for local and non-local searches. Local search algorithms are more lenient, allowing these spam sites to rank with minimal effort. he December 21, 2023, spam attack on Google's search results offers valuable insights for business owners looking to enhance their SEO strategies. This incident, where numerous domains ranked for an unusually high number of keywords, sheds light on the vulnerabilities and opportunities within Google's ranking system.Key Learnings from the Spam Attack Exploiting Low-Competition Areas: The spam attack targeted low-competition keywords, particularly in local search and longtail queries. For legitimate businesses, this highlights the potential of focusing on niche, specific keywords where competition is lower, increasing the chances of ranking higher. Understanding Google's Algorithms: The spammers took advantage of the local search algorithm's leniency and the initial ranking boost given to new domains. This underscores the importance of understanding how different SEO factors work, including the impact of new content and the specific requirements of local SEO. The Power of Longtail Keywords: The attack successfully utilized longtail keywords, which are specific and often less targeted by major competitors. For businesses, incorporating longtail keywords into their SEO strategy can capture niche markets and attract highly targeted traffic. Applying These Insights to Your SEO Strategy Focus on Local SEO: If you're a local business, optimize for local search queries. Ensure your business is listed accurately on Google My Business, and use local keywords in your website's content. Leverage Long Tail Keywords: Conduct thorough keyword research to identify longtail keywords relevant to your business. These keywords can drive targeted traffic and are generally easier to rank for. Monitor New Trends and Updates: Stay informed about the latest SEO trends and Google algorithm updates. Understanding these changes can help you adapt your strategies effectively. Diversify Your Online Presence: Don't rely solely on organic search rankings. Utilize social media, email marketing, and other channels to build a robust online presence. 3. Is Your Company Blog Google News Worthy? - Google's John Mueller addressed a crucial question: Can company blogs be eligible for Google News? This is particularly relevant for small business owners seeking to expand their reach and visibility online.Mueller clarified that while he works on search, which is somewhat separate from Google News, there's nothing in Google News content policies specifically excluding company blogs. This opens up an opportunity for business blogs to be featured, provided they meet certain criteria.To be considered for Google News, your blog content must adhere to specific guidelines. These include: Clear Dates and Bylines: Each article should have a visible publication date and author byline. Author, Publication, and Publisher Information: Details about the authors, the publication, and the company or network behind the content are essential. Contact Information: Providing contact details adds credibility and transparency to your content. While Google can automatically discover news content, being proactive can increase your chances. You can submit your blog URL for consideration through Google's Publisher Center. This step is crucial for small business owners looking to leverage their company blog for greater visibility.FYI: Google News does feature content from company blogs. For instance, GridinSoft company's blog and Adobe's company webpage have been shown in Google News. This demonstrates that while dedicated news sites are more common, company blogs that publish news are also considered.For small business owners, this information is a game-changer. It means that your company blog has the potential to be featured in Google News, provided it meets Google's content policies. This can lead to increased exposure, traffic, and potentially, a steady stream of advertising income. It's an opportunity to elevate your content strategy and expand your digital footprint in a meaningful way.4. Perfect SEO Isn't a Reality for Your Business - Google's John Mueller in his last SEO office hours of December 2023, where he stated, "no SEO is perfect." This insight is particularly relevant for business owners who may feel overwhelmed by the constantly evolving landscape of SEO.SEO is an ever-changing field, influenced by the continuous evolution of the internet, search engines, and user behavior. This fluidity means that what works today in SEO might not be as effective tomorrow. The technical elements like structured data and quality considerations are always in flux, making the idea of achieving 'perfect' SEO unattainable.Despite the impossibility of perfect SEO, Mueller emphasizes the importance of engaging in SEO practices. The goal isn't to achieve perfection but to adapt and evolve with the changes. SEO remains a crucial element in enhancing online visibility, driving traffic, and improving user engagement.Key Takeaways for Business Owners Adaptability is Key: Stay informed about the latest SEO trends and algorithm updates. Being adaptable in your SEO strategy is more valuable than striving for perfection. Focus on Quality and Relevance: Instead of chasing perfection, concentrate on creating high-quality, relevant content that resonates with your audience and adheres to SEO best practices. Continuous Learning and Improvement: SEO is a journey, not a destination. Regularly review and update your SEO strategies to align with current best practices and user preferences. Don't Be Discouraged: The complexity of SEO can be daunting, but don't let the pursuit of perfection discourage you. Even small, consistent efforts in SEO can yield significant benefits over time. For small business owners, understanding that 'no SEO is perfect' can be liberating. It shifts the focus from chasing an unattainable goal to developing a flexible, quality-focused approach that grows with your business and the digital landscape. Embracing this mindset allows you to navigate the complexities of SEO with more confidence and less stress, ultimately leading to a more robust and effective online presence.5. Does a Double Slash in URLs Affect Your SEO? - Google's Gary Illyes addressed a common query: does a double forward slash in a URL affect a website's SEO? Double forward slashes in URLs often result from coding issues in the CMS (Content Management System) or the .htaccess file. This can lead to the creation of duplicate webpages that differ only in their URL structure. Resolving this issue isn't as simple as rewriting the URL to remove the extra slash; the root cause must be identified and corrected.Gary Illyes clarified that from a technical SEO perspective, having double slashes in a URL is not problematic. According to RFC 3986, section 3, a forward slash is a standard separator in URLs and can appear multiple times, even consecutively. However, from a usability standpoint, double slashes are not ideal. They could potentially confuse users and some web crawlers.The usability of a website is crucial because it can affect user satisfaction and, indirectly, the site's popularity and visibility. If a site is difficult to navigate or understand, it may deter users and reduce the likelihood of being recommended or linked to by other sites. Similarly, anything that causes confusion for web crawlers can directly impact SEO. It's essential for a site to be easily crawlable and understandable.To avoid potential issues: Regularly check your website for double slashes and other URL anomalies. Consult with an htaccess expert or a developer to identify and fix the source of the problem. Use tools like Screaming Frog to pinpoint where the double forward slash issue starts, providing clues to the underlying technical issue. For small business owners, understanding and addressing these seemingly minor details can make a significant difference in SEO performance. While Google may be able to navigate through such issues, relying on this is not a best practice. Proactively managing your site's technical health ensures a better user experience and optimizes your site for search engines.6. DBAs Now Accepted for Advertiser Verification! - Google Ads has made a change to its Advertiser Verification Program and now accepts DBAs (Doing Business As) or trade names for verification. This development is particularly important for small business owners who often operate under trade names or DBAs.Previously, the Google Ads Advertiser Verification Program required advertisers to use their legal business names for verification. This posed a challenge for many businesses that operate under a DBA or a trade name different from their legal name. With this update, Google Ads acknowledges the common practice of using DBAs and adapts its verification process accordingly.Implications for Business Owners Broader Accessibility: This change makes the verification process more accessible to a wider range of businesses, especially small and medium-sized enterprises that commonly use DBAs. Brand Consistency: Businesses can now maintain brand consistency across their advertising and legal documentation. This is crucial for brand recognition and trust among consumers. Simplified Verification Process: The inclusion of DBAs simplifies the verification process for many businesses, reducing the administrative burden and potential confusion. To be verified under a DBA or trade name, the legal document submitted for verification must include both the legal name and the DBA/trade name. This ensures that Google can accurately associate the trade name with the legal entity behind it.7. Reservation Campaigns in YouTube Ads - YouTube/Google Ads has simplified the process of setting up reservation video campaigns, a type of advertising that offers fixed-rate impressions, ideal for brand awareness and product promotions.Reservation campaigns are a form of advertising where ad placements are purchased in advance at a fixed rate, typically on a cost-per-thousand impressions (CPM) basis. Unlike auction-based ads, where placements are bid on in real-time, reservation campaigns guarantee ad placement, making them ideal for high-impact advertising and ensuring visibility for crucial campaigns.Key Features of the New System Self-Service Options: Advertisers can now easily set up reservation video campaigns through Google Ads, streamlining the process of buying high-visibility ad placements like YouTube Select lineups and Masthead. Enhanced Targeting Options: The update includes advanced targeting capabilities, such as YouTube Select topic and interest-based targeting, along with demographic targeting, allowing advertisers to reach their desired audience more precisely. Access to Premier Content: Advertisers gain access to prominent placements like the YouTube Masthead and premier content via YouTube Select, ensuring a broader audience reach. Diverse Ad Formats: The system offers various ad formats, including non-skippable in-stream ads and bumper ads, catering to different campaign needs and audience preferences. Benefits for Business Owners Greater Control and Visibility: With fixed-rate impressions and guaranteed placements, reservation campaigns offer more control over ad impressions and higher visibility for your brand. Targeted Reach: The expanded targeting options enable businesses to tailor their campaigns more effectively, reaching the right audience with relevant content. Efficiency and Flexibility: The streamlined process saves time and effort, allowing businesses to focus more on the creative aspects and strategy of their campaigns. For small business owners, Google's update to reservation video campaigns on YouTube simplifies the process of creating impactful brand awareness and product promotion campaigns, leveraging YouTube's vast audience. Familiarizing yourself with this new system and aligning your campaigns with Google's policies will be key to maximizing your brand's exposure on one of the world's most popular video platforms.

Zero to One - Peter Thiel Contrarian Thinker + Disruption AZ TRT S04 EP50 (213) 12-17-2023   What We Learned This Week Contrarian Thinking – think for yourself and differently than everyone else Innovation great companies have unique products that go from Zero to one, vertical Founders are important and challenge the Status Quo to change the world Competition is for losers, strive for a Monopoly Secrets – What Great Company is No One Building? Disruption in Business & Tech World - How to Handle The Innovator's Dilemma     Zero to One: Notes on Startups, or How to Build the Future (c- 2014)       #1 NEW YORK TIMES BESTSELLER • “This book delivers completely new and refreshing ideas on how to create value in the world.”—Mark Zuckerberg, CEO of Meta “Peter Thiel has built multiple breakthrough companies, and Zero to One shows how.”—Elon Musk, CEO of SpaceX and Tesla The great secret of our time is that there are still uncharted frontiers to explore and new inventions to create. In Zero to One, legendary entrepreneur and investor Peter Thiel shows how we can find singular ways to create those new things. Thiel begins with the contrarian premise that we live in an age of technological stagnation, even if we're too distracted by shiny mobile devices to notice. Information technology has improved rapidly, but there is no reason why progress should be limited to computers or Silicon Valley. Progress can be achieved in any industry or area of business. It comes from the most important skill that every leader must master: learning to think for yourself. Doing what someone else already knows how to do takes the world from 1 to n, adding more of something familiar. But when you do something new, you go from 0 to 1. The next Bill Gates will not build an operating system. The next Larry Page or Sergey Brin won't make a search engine. Tomorrow's champions will not win by competing ruthlessly in today's marketplace. They will escape competition altogether, because their businesses will be unique. Zero to One presents at once an optimistic view of the future of progress in America and a new way of thinking about innovation: it starts by learning to ask the questions that lead you to find value in unexpected places. Book on Amazon: HERE     Zero to One Book Summary: HERE   By XDEV 200 from 8/2020     Notes:   Seg. 1:   Zero to One - Rethinking the Future   Zero to One - 0 to 1 The idea that new innovation goes vertical or up, technological progress If you just make a car that goes a little faster, that is horizontal progress (1 to n), like globalization, copying existing ideas and then improve a little   Founders are Important, and challenge the Status Quo   Competition is over-rated, and you should strive to be a Monopoly.   Innovation is based on Secrets   Startups, Cults, & The PayPal Mafia   There Has been Little Progress…     Contrarian Thinking   Thiel believes contrarian thinking can change the future.  “What important truth do very few people agree with you on?”     Innovation Easier to copy a model than to make something new. Doing what we already know how to do takes the world from 1 to n, adding more of something familiar. But every time we create something new, we go from 0 to 1. The act of creation is singular, as is the moment of creation, and the result is something fresh and strange. Thiel's approach for this question stems from a phrase that he used, “Brilliant thinking is rare but courage is in even shorter supply than genius.” Mark Twain: “If you find yourself on the side of the majority its time to pause and reflect.”   Build a hyper niche company with a product 10x better than predecessors   Go from Zero to One and truly innovate to change the world.     Founders The next Bill Gates will not build an operating system. The next Larry Page or Sergey Brin won't make a search engine. And the next Mark Zuckerberg won't create a social network. If you are copying these guys, you aren't learning from them. There is no entrepreneur roadmap. It's all different and unique than before.   You have to think for yourself and create your own path.   Founders have vision and know how to build a startup team that believes in them so much – like a cult. Founders are not like everyone else. They challenge themselves, their team and the status quo.        Seg 2:   Competition Per Thiel - ”All happy companies are different: each one earns a monopoly by solving a unique problem. All failed companies are the same: they failed to escape competition.”   He asks the difficult question: “What valuable company is nobody building?” Your company must be unique and serve a niche to create value, and not be a commodity.   You are looking for Blue Oceans with little competition vs a Red Ocean with business' eating away at each other and no profits.   Thiel explains the differences between a Monopoly (inherently not evil) vs. a Perfect Competition (arguably dangerous for businesses vitality). Oddly, Monopolies try to act like they are not dominant, while competitive business act as if they are unique.   Examples: Firm A — disguised as a monopoly: Google has a monopoly on search but emphasizes the small share of global online advertising, and other miscellaneous business models.   Firm B — disguised as a perfect competition: A local restaurant tries to find fake differentiators by being the “only British restaurant in Palo Alto” yet they are using inaccurate metrics. The real marker would be “restaurants” not “Restaurant type”   Business and MBA students obsess over competition and use the Art of War for metaphors.   Thiel, asks a challenge question: “Why do people compete?”   1.    Marx model: Since we are inherently different and possess distinct goals, and 2.    Shakespeare model: All competitors are more-or-less similar (ex: Montague vs Capulet)   This distracts companies to focus on the competition and not their core goal of good products and customers.   For example, while Microsoft and Google were obsessively competing with each other Apple emerged and surpassed both.   What defines a monopoly? 1.    Proprietary technology (10 times better than any existing solution), 2.    Network effect (start with a hyper-niche market. If you think its too big it is), 3.    The economy of scale (SaaS vs employee labor-intensive), and 4.    Excellent Branding (Apple Branding to stay continual trend).   How can we build a monopoly? 1.    Actively attempt to seek a hyper-niche target market that has little to no competitors. Serve them, and do it well (all that matters == customer: “Anything You Want”), 2.    Once you have dominated the market expands to the nearest adjacent market. Similar to Amazon selling CDs, DVDs then everything else, 3.    Do not disrupt current giants. PayPal worked with Visa. Everyone won, and 4.    Attempt to make the last great development in a specific market and reap all the benefits of a mature ecosystem.       Secrets   Companies are based on secrets, and when the secret is revealed, the company could change the world. Thiel questions what secrets are left, and are companies even looking for secrets?   Q: What happens when a company stops believing in secrets?   Companies can lose their dominant position by not innovating, but resting on past success.   Hewlett-Packard Example: 1.    1990 company worth $9Bn 2.    2000 after a decade of inventions (first affordable color printer, first super-portable laptops) worth $135Bn 3.    2005 worth $70Bn (failed merge with Compaq, failed consulting/support shops) 4.    2012 worth $23Bn as a result of an abandoned search for technological secrets.   Every great business is built around a secret that's hidden from the outside. Inner workings of Google's PageRank algorithm, Apple iPhone in 2007, etc…       Seg. 3:   Replay Clip from Seg. 2 of 3/6/2022 Show –   BRT S03 EP10 (109) 3-6-2022 – Topic: Best of Host Matt on Business Topics – McDonalds, Apple, Disruption, 80/20   MB on Disruption in Business & The Innovator's Dilemma book by Clayton Christensen Clayton Christensen's book, “The Innovator's Dilemma” Tech Disruption – technology changes and a small company startup can up-end big tech companies. Hence,  disruption - the power of disruption, why market leaders are often set up to fail as technologies and industries change and what incumbents can do to secure their market leadership for a long time. Innovator's Dilemma – how can big companies stay up with tech changes and pivot without hurting core business? All businesses (including tech companies) have trouble with disruption.   Example: Blockbuster – rented movies, DVDs, lost market share to Red Box (vending movie rental), then both disrupted by streaming movies. Music industry went from records to cassettes to CDs to streaming (Napster). MySpace taken out by Facebook in social networks. Yahoo search taken out by Google (controls 75% of the search market) Kodak afraid to get out of film business and passed on digital film, lost market share.   To solve the Innovator's Dilemma, big companies acquire smaller tech companies; have in house R&D to be ready for next tech wave. Steve Jobs of Apple was very influenced by  Innovator's Dilemma  and took this idea seriously. If you do not try to put your company out of business (w/ disruption / new tech), someone else will. Jobs was not afraid to innovate, and cannibalize his own company and products to stay relevant. Apple created iPhone, and now computer is in your pocket Peter Thiel – “Zero to One” book - Great innovation is not A to B to C, it is vertical, jumps curves. Current smart phones have more computing power than a computer 20 years ago. Guy Kawasaaki (former Apple) Talk - “12 Lessons From Steve Jobs”   Full Show: HERE       Best of Biotech from AZ Bio & Life Sciences to Jellatech: HERE   Biotech Shows: HERE   AZ Tech Council Shows:  https://brt-show.libsyn.com/size/5/?search=az+tech+council *Includes Best of AZ Tech Council show from 2/12/2023     ‘Best Of' Topic: https://brt-show.libsyn.com/category/Best+of+BRT      Thanks for Listening. Please Subscribe to the BRT Podcast.     AZ Tech Roundtable 2.0 with Matt Battaglia The show where Entrepreneurs, Top Executives, Founders, and Investors come to share insights about the future of business.  AZ TRT 2.0 looks at the new trends in business, & how classic industries are evolving.  Common Topics Discussed: Startups, Founders, Funds & Venture Capital, Business, Entrepreneurship, Biotech, Blockchain / Crypto, Executive Comp, Investing, Stocks, Real Estate + Alternative Investments, and more…    AZ TRT Podcast Home Page: http://aztrtshow.com/ ‘Best Of' AZ TRT Podcast: Click Here Podcast on Google: Click Here Podcast on Spotify: Click Here                    More Info: https://www.economicknight.com/azpodcast/ KFNX Info: https://1100kfnx.com/weekend-featured-shows/   Disclaimer: The views and opinions expressed in this program are those of the Hosts, Guests and Speakers, and do not necessarily reflect the views or positions of any entities they represent (or affiliates, members, managers, employees or partners), or any Station, Podcast Platform, Website or Social Media that this show may air on. All information provided is for educational and entertainment purposes. Nothing said on this program should be considered advice or recommendations in: business, legal, real estate, crypto, tax accounting, investment, etc. Always seek the advice of a professional in all business ventures, including but not limited to: investments, tax, loans, legal, accounting, real estate, crypto, contracts, sales, marketing, other business arrangements, etc.  

Callum Scott talks with Jason Barnard about understanding and recovering from Google traffic drops. Callum Scott specializes in conducting data-driven and qualitative SEO analysis, focusing primarily on traffic drop analysis, technical SEO and content quality. Callum is expert on Google's Knowledge Graph and Google's use of Entity Understanding for information retrieval and the entire search ecosystem. With over 5 years of experience in complex SEO environments, Callum has conducted nearly 100 technical and content-focused SEO audits, helped many websites achieve consistent growth and integrated well with an organisation's existing team and framework. Imagine you're navigating through a busy city and suddenly your GPS goes off. That's the same kind of confusion and vexation you feel when Google traffic drops affect your website. It's unsettling, irritating, and can definitely impact your profits. But once you understand why it happens and how to recover from it, you'll be back on track in no time. This essential knowledge not only protects your website's performance, but also gives you the ability to navigate the ever-changing world of search engine optimization. In this incredibly awesome episode, Callum (Callie) Scott reveals great nuggets and some real-life examples about traffic drops, their causes and how a website can recover from them. There are also three categories of traffic drops due to core updates - Broad Site Quality Reassessments, Searcher Intent Shift and Relevance, which Callum insightfully explains and suggests alternative strategies to help websites with ranking issues. Callum also highlights how to deal with Google's changing understanding of intent and the shift in SERPs. As always, the show ends with passing the baton… Callum passes the virtual baton to next week's super groovy guest, Alex Sanfilippo. What you'll learn from Callum Scott 00:00 Callum Scott and Jason Barnard 01:08 Callum Scott's Generative AI Result on Google 01:30 Kalicube Support Group 01:38 Blue Orchid Digital Ltd Brand SERP 03:22 When Did Google Start Relying More on Quality Signals Than on Pagerank or Word Count? 04:51 How Has AI Affected Google's Categorization of Website Quality? 05:53 What Significant Algorithm Changes Did Google Make in 2017? 06:26 How Did the Shift Towards Machine Learning Unfold Within the Google Search Team Between 2014 and 2017? 07:34 Understanding the Role of Features in Machine Learning for E-E-A-T 08:25 How Does Google's Confidence in Displaying a Knowledge Panel Affect Users' Trust in Their Search Results? 09:51 What are Some Examples of Traffic Drops, Causes and Recoveries? 10:55 Three Categories of Traffic Drops Due to Core Updates 11:04 First Category: Broad Site Quality Reassessments 12:16 How Does the Persistence of Low-quality Content Affect Google's Focus and Resource Allocation for a Website? 12:48 Second Category: Searcher Intent Shift 14:29 How Feasible is it for a Single Website to be Ranked for Both Informational and Transactional Intents? 16:01 How to Deal with the Change in Google's Understanding of Intent and the Shift in SERPs 18:12 Third Category: Relevance 19:30 What is the Best Alternative Strategy for Websites with Ranking Issues? 21:25 What are the Challenges of Convincing Clients to Address Traffic Drops by Individually Prioritizing Pages and Queries? 23:18 Traffic Drop Following a Core Update: Wait or Act Immediately 24:28 How to Convince Clients Not to Panic When Traffic Drops? 27:36 How Can Branded Search Help to Mitigate a Traffic Drop 29:35 Padding the Baton: Callum (Callie) Scott to Alex Sanfilippo This episode was recorded live on video August 29th 2023

Callum Scott talks with Jason Barnard about understanding and recovering from Google traffic drops. Callum Scott specializes in conducting data-driven and qualitative SEO analysis, focusing primarily on traffic drop analysis, technical SEO and content quality. Callum is expert on Google's Knowledge Graph and Google's use of Entity Understanding for information retrieval and the entire search ecosystem. With over 5 years of experience in complex SEO environments, Callum has conducted nearly 100 technical and content-focused SEO audits, helped many websites achieve consistent growth and integrated well with an organisation's existing team and framework. Imagine you're navigating through a busy city and suddenly your GPS goes off. That's the same kind of confusion and vexation you feel when Google traffic drops affect your website. It's unsettling, irritating, and can definitely impact your profits. But once you understand why it happens and how to recover from it, you'll be back on track in no time. This essential knowledge not only protects your website's performance, but also gives you the ability to navigate the ever-changing world of search engine optimization. In this incredibly awesome episode, Callum (Callie) Scott reveals great nuggets and some real-life examples about traffic drops, their causes and how a website can recover from them. There are also three categories of traffic drops due to core updates - Broad Site Quality Reassessments, Searcher Intent Shift and Relevance, which Callum insightfully explains and suggests alternative strategies to help websites with ranking issues. Callum also highlights how to deal with Google's changing understanding of intent and the shift in SERPs. As always, the show ends with passing the baton… Callum passes the virtual baton to next week's super groovy guest, Alex Sanfilippo. What you'll learn from Callum Scott 00:00 Callum Scott and Jason Barnard 01:08 Callum Scott's Generative AI Result on Google 01:30 Kalicube Support Group 01:38 Blue Orchid Digital Ltd Brand SERP 03:22 When Did Google Start Relying More on Quality Signals Than on Pagerank or Word Count? 04:51 How Has AI Affected Google's Categorization of Website Quality? 05:53 What Significant Algorithm Changes Did Google Make in 2017? 06:26 How Did the Shift Towards Machine Learning Unfold Within the Google Search Team Between 2014 and 2017? 07:34 Understanding the Role of Features in Machine Learning for E-E-A-T 08:25 How Does Google's Confidence in Displaying a Knowledge Panel Affect Users' Trust in Their Search Results? 09:51 What are Some Examples of Traffic Drops, Causes and Recoveries? 10:55 Three Categories of Traffic Drops Due to Core Updates 11:04 First Category: Broad Site Quality Reassessments 12:16 How Does the Persistence of Low-quality Content Affect Google's Focus and Resource Allocation for a Website? 12:48 Second Category: Searcher Intent Shift 14:29 How Feasible is it for a Single Website to be Ranked for Both Informational and Transactional Intents? 16:01 How to Deal with the Change in Google's Understanding of Intent and the Shift in SERPs 18:12 Third Category: Relevance 19:30 What is the Best Alternative Strategy for Websites with Ranking Issues? 21:25 What are the Challenges of Convincing Clients to Address Traffic Drops by Individually Prioritizing Pages and Queries? 23:18 Traffic Drop Following a Core Update: Wait or Act Immediately 24:28 How to Convince Clients Not to Panic When Traffic Drops? 27:36 How Can Branded Search Help to Mitigate a Traffic Drop 29:35 Padding the Baton: Callum (Callie) Scott to Alex Sanfilippo This episode was recorded live on video August 29th 2023

Welcome to Episode 161 of the "Marketing Guides for Small Business Podcast"! Today's episode promises to be a treasure trove for any business striving to understand the complex interplay between content and SEO. Paul, our host, is joined by marketing mavens, Ian and Ken, to dissect the critical components of successful content. They delve into the age-old question - should we write for people or search engines? Ken offers an in-depth analysis, emphasizing the importance of striking a balance to achieve stellar SEO outcomes. Ever wondered about Google's mysterious PageRank? Ian breaks it down for you, explaining the factors determining a page's rank and providing actionable tips tailored for local SEO. And if you're curious about the myriad content types at your disposal, don't miss Ken's comprehensive list and their respective utilities. This episode is a must-listen for anyone seeking clarity in the world of content-driven SEO. Dive in to boost your marketing game and remember, quality content is king!

Episode 177 contains the notable Digital Marketing News and Updates from the week of Sep 4-8, 2023.1. Google August 2023 Core Update - Google has confirmed that the August 2023 core update, which began rolling out on August 22, 2023, has completed on Sept 7, 2023. This is the second core update of 2023, following the March 2023 core update.FYI: Core updates are major changes to Google's search algorithm that aim to improve the quality of search results. They are not focused on any particular kind of content or website, and they can affect rankings for a wide variety of websites. The early SEO industry chatter suggested this update was a fairly impactful update compared to previous core updates. The exact impact of the August 2023 core update is still unknown. Google stressed that pages impacted by core updates aren't necessarily flawed. According to Google, this update focused on improving Google's overall content assessment. As always, Google advised site owners to focus on quality content as a response to fluctuations in search rankings. Sites that are experiencing a dip should consider conducting an audit to understand which pages were most impacted and for what types of searches. Google also cautioned that improving your website content may not lead to an immediate change in rankings.2. Google Updates Helpful Content Guidelines: Self-Assess Your Content & Remove Unhelpful Content - Google recently updated its documentation on the helpful content system, designed to identify and demote low-quality content from search results. The updated documentation now states that website owners should self-assess their content to determine if it is helpful to visitors. If website owners find their content unhelpful, they are encouraged to remove it.Marie Haynes asked Danny Sullivan, the Google Search Liaison:"Google's documentation on the helpful content system talks about recovering by "removing unhelpful content" in order to get the unhelpful content classification removed. Any chance we could get more clarity on this? Do you mean: -remove parts of pages that contain large amounts of text readers will likely skip over? -remove entire pages that offer little original value? -perhaps both? Does a site need to remove unhelpful content published in the past in order to recover? Or could they focus on simply producing content that is helpful and original from this point onwards. Would that be enough?"Danny replied: "People should self-assess their content to understand if they believe it will be helpful to visitors. Keep content on pages or entire pages or whatever they believe is helpful. Get rid of things that aren't, if they're looking critically at them as a visitor."I know that Google does not provide specific guidance on what constitutes unhelpful content. However, the documentation does provide some general examples, such as content that is plagiarized, spammy, or irrelevant to the user's search intent.Website owners concerned about their content being labeled as unhelpful should carefully review their pages or enlist the help of a trusted third party. How do you all go about assessing if your content is helpful or not?3. Google Sites: Not Ideal for SEO - Google Sites is a free website builder that allows users to create and publish websites without coding knowledge. The Google site is a hosted website builder that's free and published under a sites.google.com domain, although one can opt to use an actual domain name. Now, Google's John Mueller shared additional details on Google Sites and SEO after someone asked him why Google did not index his Google site. Here is what John replied:"Taking a step back, websites created on Google Sites can and do get indexed in Google Search. However, the URLs used in Google Sites are hard to track since the public version can differ from the URL you see when logged in. To be blunt, while it's technically indexable, it's not ideal for SEO purposes and can be complex for tracking in Search Console. If SEO is your primary consideration, exploring other options and checking the pros and cons before committing might be worthwhile. For performance tracking in the Search Console, you could also use your domain name for the Google Sites content. Using your domain name makes it easier to migrate, should you choose, and allows you to verify ownership of the whole domain for Search Console. "P.S: The Google Sites service is popular with link spammers who create links on Google subdomains in a tactic called "Google Stacking." The idea behind Google Stacking is that spammers generate a page of links on Google Sheets, Google Docs, etc., and then interlink them all from a Google Sites. Google Stacking is based on the mistaken belief that there's "authority" and "trust" in Google subdomains that is transferred over to the spammer sites through links. Of course, that's wishful thinking. There's no such thing used by Google called "trust" or "authority" that gets transferred from one site to another through links.4. The Best Way to Do Paid Guest Posting (According to Google) - Whether paid or unpaid, guest posts are an old tactic. In 2014, Google's Matt Cutts wrote a blog post telling SEO practitioners to "put a fork in it," since guest blogging does not work anymore.The same year Google issued a series of penalties on guest blogging platforms. But these days, Google doesn't hand out penalties like it used to. Google stops the links from passing PageRank. That makes it hard to know whether the guest post is working. So people keep guest posting because the penalty feedback isn't there.During the September 2023 Google SEO Office Hours, an individual asked: "Most websites only offer the option to purchase a "guest post" (to gain a backlink) from them nowadays. Is this against Google's guidelines if I'm writing valuable content?"John Mueller answered: "It sounds like you're already on the right track.Yes, using guest posts for links is against our spam policies. In particular, it's important that these links are qualified in a way that signal that they don't affect search results.You can do this with the rel=nofollow or rel=sponsored attributes on links. It's fine to use advertising to promote your site, but the links should be blocked as mentioned."So, Paid Guest Posts with links are advertisements as far as Google is concerned. Failure to label advertisements is not only misleading to readers but may also violate laws that prohibit misleading advertisements. 

Episode 176 contains the notable Digital Marketing News and Updates from the week of August 28 - Sep 1, 2023.1. Google Announces Lighthouse 11 with New Accessibility Audits, and LCP Bug Fix - Google PageSpeed Insights (PSI) is a free tool to help you find and fix issues slowing down your web application. PageSpeed Insights (PSI) reports on the user experience of a page on both mobile and desktop devices, and provides suggestions on how that page may be improved. An open-source tool called Lighthouse collects and analyzes lab data that's combined with real-world data from the Chrome User Experience Report dataset. Google has released the latest version (v.11) of Lighthouse, an open-source tool that helps developers and webmasters measure the performance of their websites. Lighthouse 11 includes a number of new features and improvements, including: New accessibility audits: Website accessibility is not currently a ranking factor and quite likely not a quality signal. However it's a best practice for a website to function correctly for as many people as possible. Lighthouse 11 introduces thirteen new accessibility audits that help developers identify and fix accessibility issues on their websites. Changes to how best practices are scored: Lighthouse 11 has changed the way that best practices are scored. This makes it easier for developers to understand how their websites are performing and what they can do to improve their scores. Largest Contentful Paint scoring bug fixed: Lighthouse 11 has fixed a bug that was affecting the scoring of Largest Contentful Paint (LCP). LCP is a measure of how long it takes for the largest content element on a page to become visible. Updated Interaction to Next Paint (INP) to reflect it's no longer experimental: Lighthouse 11 has updated the Interaction to Next Paint (INP) metric to reflect that it is no longer experimental. INP measures the time it takes for a user to be able to interact with a page after it has loaded. In my opinion, INP is in line to become an official Core Web Vital in 2024. These changes make Lighthouse 11 a more powerful and useful tool for developers and webmasters who want to improve the performance of their websites.2. YouTube Creators Can Now Remove Community Guideline Strikes - YouTube has announced that creators will now be able to remove Community Guideline strikes from their channels by completing educational courses. This is a new policy that was introduced in June 2023, and it is designed to help creators learn about the Community Guidelines and how to avoid violating them in the future.To remove a strike, creators will need to complete a course that covers the Community Guidelines and how to create compliant content. The course is available in several languages, and it takes about an hour to complete. Once the course is completed, the strike will be removed from the channel.This new policy is a positive step for YouTube creators. It provides them with a way to learn from their mistakes and avoid getting strikes in the future. It also shows that YouTube is committed to creating a safe and positive environment for its users.3. Your Site's Language Doesn't Protect It From Google's Penalty - Google can issue manual actions to any site, regardless of the language it is written in. This means that sites written in non-native English can still be penalized by Google if they violate the company's webmaster guidelines.Google Search Advocate, John Muller was asked if Google penalizes sites written by non-native English writers. Mueller responded that manual actions and algorithm changes are independent of the native language of the authors or the site language. He also said that Google does not have a list of "bad" languages, and that all sites are treated equally.This means that site owners who write in non-native English need to be just as careful as those who write in English as their first language. They should avoid any practices that could lead to a manual action, such as keyword stuffing, cloaking, and duplicate content.4. How Googlebot Handles AI-Generated Content

Larry Page Cofounder And Board Member, Alphabet $111.1B$1.4B (1.32%)Real Time Net Worth as of 8/10/23#7 in the world today About Larry Page Larry Page stepped down as CEO of Alphabet, the parent company of Google, in 2019 but remains a board member and a controlling shareholder. He cofounded Google in 1998 with fellow Stanford Ph.D. student Sergey Brin. With Brin, Page invented Google's PageRank algorithm, which powers the search engine. Page was CEO until 2001, when Eric Schmidt took over, and then from 2011 until 2015, when he became CEO of Google's new parent firm Alphabet. He is a founding investor in space exploration company Planetary Resources and is also funding "flying car" startups Kitty Hawk and Opener. --- Send in a voice message: https://podcasters.spotify.com/pod/show/michael-kaagee-mante/message

PageRank is a Google algorithm for ranking pages based on the flow of authority via links, created by Larry Page and Sergey Brin. Every SEO pro should have a good grasp of what PageRank was – and what it still is today.Author: Dixon JonesSource: https://www.searchenginejournal.com/google-pagerank/483521/---The Unknown Secrets of Internet Marketing podcast is a weekly podcast hosted by internet marketing experts Matt Bertram and Chris Burres. The show provides insights and advice on digital marketing, SEO, and online business. Topics covered include keyword research, content optimization, link building, local SEO, and more. The show also features interviews with industry leaders and experts who share their experiences and tips. Additionally, Matt and Chris share their own experiences and strategies, as well as their own successes and failures, to help listeners learn from their experiences and apply the same principles to their own businesses. The show is designed to help entrepreneurs and business owners become successful online and get the most out of their digital marketing efforts.Please leave us a review if you enjoyed this podcast: https://g.page/r/CccGEk37CLosEB0/reviewFind more great episodes here: bestseopodcast.com/Follow us on:Facebook: @bestseopodcastInstagram: @thebestseopodcastTiktok: @bestseopodcastLinkedIn: @bestseopodcastPowered by: ewrdigital.comHosts: Matt Bertram & Chris BurresDisclaimer: For Educational and Entertainment purposes Only.

Jack Roehrig, Technology Evangelist at Uptycs, joins Corey on Screaming in the Cloud for a conversation about security awareness, ChatGPT, and more. Jack describes some of the recent developments at Uptycs, which leads to fascinating insights about the paradox of scaling engineering teams large and small. Jack also shares how his prior experience working with AskJeeves.com has informed his perspective on ChatGPT and its potential threat to Google. Jack and Corey also discuss the evolution of Reddit, and the nuances of developing security awareness trainings that are approachable and effective.About JackJack has been passionate about (obsessed with) information security and privacy since he was a child. Attending 2600 meetings before reaching his teenage years, and DEF CON conferences shortly after, he quickly turned an obsession into a career. He began his first professional, full-time information-security role at the world's first internet privacy company; focusing on direct-to-consumer privacy. After working the startup scene in the 90's, Jack realized that true growth required a renaissance education. He enrolled in college, completing almost six years of coursework in a two-year period. Studying a variety of disciplines, before focusing on obtaining his two computer science degrees. University taught humility, and empathy. These were key to pursuing and achieving a career as a CSO lasting over ten years. Jack primarily focuses his efforts on mentoring his peers (as well as them mentoring him), advising young companies (especially in the information security and privacy space), and investing in businesses that he believes are both innovative, and ethical.Links Referenced: Uptycs: https://www.uptycs.com/ jack@jackroehrig.com: mailto:jack@jackroehrig.com jroehrig@uptycs.com: mailto:jroehrig@uptycs.com TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey:  LANs of the late 90's and early 2000's were a magical place to learn about computers, hang out with your friends, and do cool stuff like share files, run websites & game servers, and occasionally bring the whole thing down with some ill-conceived software or network configuration. That's not how things are done anymore, but what if we could have a 90's style LAN experience along with the best parts of the 21st century internet? (Most of which are very hard to find these days.) Tailscale thinks we can, and I'm inclined to agree. With Tailscale I can use trusted identity providers like Google, or Okta, or GitHub to authenticate users, and automatically generate & rotate keys to authenticate devices I've added to my network. I can also share access to those devices with friends and teammates, or tag devices to give my team broader access. And that's the magic of it, your data is protected by the simple yet powerful social dynamics of small groups that you trust. Try now - it's free forever for personal use. I've been using it for almost two years personally, and am moderately annoyed that they haven't attempted to charge me for what's become an absolutely-essential-to-my-workflow service.Corey: Kentik provides Cloud and NetOps teams with complete visibility into hybrid and multi-cloud networks. Ensure an amazing customer experience, reduce cloud and network costs, and optimize performance at scale — from internet to data center to container to cloud. Learn how you can get control of complex cloud networks at www.kentik.com, and see why companies like Zoom, Twitch, New Relic, Box, Ebay, Viasat, GoDaddy, booking.com, and many, many more choose Kentik as their network observability platform. Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. This promoted episode is brought to us by our friends at Uptycs and they have once again subjected Jack Roehrig, Technology Evangelist, to the slings, arrows, and other various implements of misfortune that I like to hurl at people. Jack, thanks for coming back. Brave of you.Jack: I am brave [laugh]. Thanks for having me. Honestly, it was a blast last time and I'm looking forward to having fun this time, too.Corey: It's been a month or two, ish. Basically, the passing of time is one of those things that is challenging for me to wrap my head around in this era. What have you folks been up to? What's changed since the last time we've spoken? What's coming out of Uptycs? What's new? What's exciting? Or what's old with a new and exciting description?Jack: Well, we've GA'ed our agentless architecture scanning system. So, this is one of the reasons why I joined Uptycs that was so fascinating to me is they had kind of nailed XDR. And I love the acronyms: XDR and CNAPP is what we're going with right now. You know, and we have to use these acronyms so that people can understand what we do without me speaking for hours about it. But in short, our agentless system looks at the current resting risk state of production environment without the need to deploy agents, you know, as we talked about last time.And then the XDR piece, that's the thing that you get to justify the extra money on once you go to your CTO or whoever your boss is and show them all that risk that you've uncovered with our agentless piece. It's something I've done in the past with technologies that were similar, but Uptycs is continuously improving, our anomaly detection is getting better, our threat intel team is getting better. I looked at our engineering team the other day. I think we have over 300 engineers or over 250 at least. That's a lot.Corey: It's always wild for folks who work in small shops to imagine what that number of engineers could possibly be working on. Then you go and look at some of the bigger shops and you talk to them and you hear about all the different ways their stuff is built and how they all integrate together and you come away, on some level, surprised that they're able to work with that few engineers. So, it feels like there's a different perspective on scale. And no one has it right, but it is easy, I think, in the layperson's mindset to hear that a company like Twitter, for example, before it got destroyed, had 5000 engineers. And, “What are they all doing?” And, “Well, I can see where that question comes from and the answer is complicated and nuanced, which means that no one is going to want to hear it if it doesn't fit into a tweet itself.” But once you get into the space, you start realizing that everything is way more complicated than it looks.Jack: It is. Yeah. You know, it's interesting that you mention that about Twitter. I used to work for a company called Interactive Corporation. And Interactive Corporation is an internet conglomerate that owns a lot of those things that are at the corners of the internet that not many people know about. And also, like, the entire online dating space. So, I mean, it was a blast working there, but at one point in my career, I got heavily involved in M&A. And I was given the nickname Jack the RIFer. RIF standing for Reduction In Force.Corey: Oof.Jack: So, Jack the RIFer was—yeah [laugh] I know, right?Corey: It's like Buzzsaw Ted. Like, when you bring in the CEO with the nickname of Buzzsaw in there, it's like, “Hmm, I wonder who's going to hire a lot of extra people?” Not so much.Jack: [laugh]. Right? It's like, hey, they said they were sending, “Jack out to hang out with us,” you know, in whatever country we're based out of. And I go out there and I would drink them under the table. And I'd find out the dirty secrets, you know.We would be buying these companies because they would need optimized. But it would be amazing to me to see some of these companies that were massive and they produced what I thought was so little, and then to go on to analyze everybody's job and see that they were also intimately necessary.Corey: Yeah. And the question then becomes, if you were to redesign what that company did from scratch. Which again, is sort of an architectural canard; it was the easiest thing in the world to do is to design an architecture from scratch on a whiteboard with almost an arbitrary number of constraints. The problem is that most companies grow organically and in order to get to that idealized architecture, you've got to turn everything off and rebuild it from scratch. The problem is getting to something that's better without taking 18 months of downtime while you rebuild everything. Most companies cannot and will not sustain that.Jack: Right. And there's another way of looking at it, too, which is something that's been kind of a thought experiment for me for a long time. One of the companies that I worked with back at IC was Ask Jeeves. Remember Ask Jeeves?Corey: Oh, yes. That was sort of the closest thing we had at the time to natural language search.Jack: Right. That was the whole selling point. But I don't believe we actually did any natural language processing back then [laugh]. So, back in those days, it was just a search index. And if you wanted to redefine search right now and you wanted to find something that was like truly a great search engine, what would you do differently?If you look at the space right now with ChatGPT and with Google, and there's all this talk about, well, ChatGPT is the next Google killer. And then people, like, “Well, Google has Lambda.” What are they worried about ChatGPT for? And then you've got the folks at Google who are saying, “ChatGPT is going to destroy us,” and the folks in Google who are saying, “ChatGPT's got nothing on us.” So, if I had to go and do it all over from scratch for search, it wouldn't have anything to do with ChatGPT. I would go back and make a directed, cyclical graph and I would use node weight assignments based on outbound links. Which is exactly what Google was with the original PageRank algorithm, right [laugh]?Corey: I've heard this described as almost a vector database in various terms depending upon what it is that—how it is you're structuring this and what it looks like. It's beyond my ken personally, but I do see that there's an awful lot of hype around ChatGPT these days, and I am finding myself getting professionally—how do I put it—annoyed by most of it. I think that's probably the best way to frame it.Jack: Isn't it annoying?Corey: It is because it's—people ask, “Oh, are you worried that it's going to take over what you do?” And my answer is, “No. I'm worried it's going to make my job harder more than anything else.” Because back when I was a terrible student, great, write an essay on this thing, or write a paper on this. It needs to be five pages long.And I would write what I thought was a decent coverage of it and it turned out to be a page-and-a-half. And oh, great. What I need now is a whole bunch of filler fluff that winds up taking up space and word count but doesn't actually get us to anywhere—Jack: [laugh].Corey: —that is meaningful or useful. And it feels like that is what GPT excels at. If I worked in corporate PR for a lot of these companies, I would worry because it takes an announcement that fits in a tweet—again, another reference to that ailing social network—and then it turns it into an arbitrary length number of pages. And it's frustrating for me just because that's a lot more nonsense I have to sift through in order to get the actual, viable answer to whatever it is I'm going for here.Jack: Well, look at that viable answer. That's a really interesting point you're making. That fluff, right, when you're writing that essay. Yeah, that one-and-a-half pages out. That's gold. That one-and-a-half pages, that's the shit. That's the stuff you want, right? That's the good shit [laugh]. Excuse my French. But ChatGPT is what's going to give you that filler, right? The GPT-3 dataset, I believe, was [laugh] I think it was—there's a lot of Reddit question-and-answers that were used to train it. And it was trained, I believe—the data that it was trained with ceased to be recent in 2021, right? It's already over a year old. So, if your teacher asked you to write a very contemporary essay, ChatGPT might not be able to help you out much. But I don't think that that kind of gets the whole thing because you just said filler, right? You can get it to write that extra three-and-a-half pages from that five pages you're required to write. Well, hey, teachers shouldn't be demanding that you write five pages anyways. I once heard, a friend of mine arguing about one presidential candidate saying, “This presidential candidate speaks at a third-grade level.” And the other person said, “Well, your presidential candidate speaks at a fourth-grade level.” And I said, “I wish I could convey presidential ideas at a level that a third or a fourth grader could understand” You know? Right?Corey: On some level, it's actually not a terrible thing because if you can only convey a concept at an extremely advanced reading level, then how well do you understand—it felt for a long time like that was the problem with AI itself and machine-learning and the rest. The only value I saw was when certain large companies would trot out someone who was themselves deep into the space and their first language was obviously math and they spoke with a heavy math accent through everything that they had to say. And at the end of it, I didn't feel like I understood what they were talking about any better than I had at the start. And in time, it took things like ChatGPT to say, “Oh, this is awesome.” People made fun of the Hot Dog/Not A Hot Dog App, but that made it understandable and accessible to people. And I really think that step is not given nearly enough credit.Jack: Yeah. That's a good point. And it's funny, you mentioned that because I started off talking about search and redefining search, and I think I use the word digraph for—you know, directed gra—that's like a stupid math concept; nobody understands what that is. I learned that in discrete mathematics a million years ago in college, right? I mean, I'm one of the few people that remembers it because I worked in search for so long.Corey: Is that the same thing is a directed acyclic graph, or am I thinking of something else?Jack: Ah you're—that's, you know, close. A directed acyclic graph has no cycles. So, that means you'll never go around in a loop. But of course, if you're just mapping links from one website to another website, A can link from B, which can then link back to A, so that creates a cycle, right? So, an acyclic graph is something that doesn't have that cycle capability in it.Corey: Got it. Yeah. Obviously, my higher math is somewhat limited. It turns out that cloud economics doesn't generally tend to go too far past basic arithmetic. But don't tell them. That's the secret of cloud economics.Jack: I think that's most everything, I mean, even in search nowadays. People aren't familiar with graph theory. I'll tell you what people are familiar with. They're familiar with Google. And they're familiar with going to Google and Googling for something, and when you Google for something, you typically want results that are recent.And if you're going to write an essay, you typically don't care because only the best teachers out there who might not be tricked by ChatGPT—honestly, they probably would be, but the best teachers are the ones that are going to be writing the syllabi that require the recency. Almost nobody's going to be writing syllabi that requires essay recency. They're going to reuse the same syllabus they've been using for ten years.Corey: And even that is an interesting question there because if we talk about the results people want from search, you're right, I have to imagine the majority of cases absolutely care about recency. But I can think of a tremendous number of counterexamples where I have been looking for things explicitly and I do not want recent results, sometimes explicitly. Other times because no, I'm looking for something that was talked about heavily in the 1960s and not a lot since. I don't want to basically turn up a bunch of SEO garbage that trawled it from who knows where. I want to turn up some of the stuff that was digitized and then put forward. And that can be a deceptively challenging problem in its own right.Jack: Well, if you're looking for stuff has been digitized, you could use archive.org or one of the web archive projects. But if you look into the web archive community, you will notice that they're very secretive about their data set. I think one of the best archive internet search indices that I know of is in Portugal. It's a Portuguese project.I can't recall the name of it. But yeah, there's a Portuguese project that is probably like the axiomatic standard or like the ultimate prototype of how internet archiving should be done. Search nowadays, though, when you say things like, “I want explicitly to get this result,” search does not want to show you explicitly what you want. Search wants to show you whatever is going to generate them the most advertising revenue. And I remember back in the early search engine marketing days, back in the algorithmic trading days of search engine marketing keywords, you could spend $4 on an ad for flowers and if you typed the word flowers into Google, you just—I mean, it was just ad city.You typed the word rehabilitation clinic into Google, advertisements everywhere, right? And then you could type certain other things into Google and you would receive a curated list. These things are obvious things that are identified as flaws in the secrecy of the PageRank algorithm, but I always thought it was interesting because ChatGPT takes care of a lot of the stuff that you don't want to be recent, right? It provides this whole other end to this idea that we've been trained not to use search for, right?So, I was reviewing a contract the other day. I had this virtual assistant and English is not her first language. And she and I red-lined this contract for four hours. It was brutal because I kept on having to Google—for lack of a better word—I had to Google all these different terms to try and make sense of it. Two days later, I'm playing around with ChatGPT and I start typing some very abstract commands to it and I swear to you, it generated that same contract I was red-lining. Verbatim. I was able to get into generating multiple [laugh] clauses in the contract. And by changing the wording in ChatGPT to save, “Create it, you know, more plaintiff-friendly,” [laugh] that contract all of a sudden, was red-lined in a way that I wanted it to be [laugh].Corey: This is a fascinating example of this because I'm married to a corporate attorney who does this for a living, and talking to her and other folks in her orbit, the problem they have with it is that it works to a point, on a limited basis, but it then veers very quickly into terms that are nonsensical, terms that would absolutely not pass muster, but sound like something a lawyer would write. And realistically, it feels like what we've built is basically the distillation of a loud, overconfident white guy in tech because—Jack: Yes.Corey: —they don't know exactly what they're talking about, but by God is it confident when it says it.Jack: [laugh]. Yes. You hit the nail on that. Ah, thank you. Thank you.Corey: And there's as an easy way to prove this is pick any topic in the world in which you are either an expert or damn close to it or know more than the average bear about and ask ChatGPT to explain that to you. And then notice all the things that glosses over or what it gets subtly wrong or is outright wrong about, but it doesn't ever call that out. It just says it with the same confident air of a failing interview candidate who gets nine out of ten questions absolutely right, but the one they don't know they bluff on, and at that point, you realize you can't trust them because you never know if they're bluffing or they genuinely know the answer.Jack: Wow, that is a great analogy. I love that. You know, I mentioned earlier that the—I believe the part of the big portion of the GPT-3 training data was based on Reddit questions and answers. And now you can't categorize Reddit into a single community, of course; that would be just as bad as the way Reddit categories [laugh] our community, but Reddit did have a problem a wh—I remember, there was the Ellen Pao debacle for Reddit. And I don't know if it was so much of a debacle if it was more of a scapegoat situation, but—Corey: I'm very much left with a sense that it's the scapegoat. But still, continue.Jack: Yeah, we're adults. We know what happened here, right? Ellen Pao is somebody who is going through some very difficult times in her career. She's hired to be a martyr. They had a community called fatpeoplehate, right?I mean, like, Reddit had become a bizarre place. I used Reddit when I was younger and it didn't have subreddits. It was mostly about programming. It was more like Hacker News. And then I remember all these people went to Hacker News, and a bunch of them stayed at Reddit and there was this weird limbo of, like, the super pretentious people over at Hacker News.And then Reddit started to just get weirder and weirder. And then you just described ChatGPT in a way that just struck me as so Reddit, you know? It's like some guy mansplaining some answer. It starts off good and then it's overconfidently continues to state nonsensical things.Corey: Oh yeah, I was a moderator of the legal advice and personal finance subreddits for years, and—Jack: No way. Were you really?Corey: Oh, absolutely. Those corners were relatively reasonable. And like, “Well, wait a minute, you're not a lawyer. You're correct and I'm also not a financial advisor.” However, in both of those scenarios, what people were really asking for was, “How do I be a functional adult in society?”In high school curricula in the United States, we insist that people go through four years of English literature class, but we don't ever sit down and tell them how to file their taxes or how to navigate large transactions that are going to be the sort of thing that you encounter in adulthood: buying a car, signing a lease. And it's more or less yeah, at some point, you wind up seeing someone with a circumstance that yeah, talk to a lawyer. Don't take advice on the internet for this. But other times, it's no, “You cannot sue a dog. You have to learn to interact with people as a grown-up. Here's how to approach that.” And that manifests as legal questions or finance questions, but it all comes down to I have been left on prepared for the world I live in by the school system. How do I wind up addressing these things? And that is what I really enjoyed.Jack: That's just prolifically, prolifically sound. I'm almost speechless. You're a hundred percent correct. I remember those two subreddits. It always amazes me when I talk to my friends about finances.I'm not a financial person. I mean, I'm an investor, right, I'm a private equity investor. And I was on a call with a young CEO that I've been advising for while. He runs a security awareness training company, and he's like, you know, you've made 39% off of your investment three months. And I said, “I haven't made anything off of my investment.”I bought a safe and, you know—it's like, this is conversion equity. And I'm sitting here thinking, like, I don't know any of the stuff. And I'm like, I talk to my buddies in the—you know, that are financial planners and I ask them about finances, and it's—that's also interesting to me because financial planning is really just about when are you going to buy a car? When are you going to buy a house? When are you going to retire? And what are the things, the securities, the companies, what should you do with your money rather than store it under your mattress?And I didn't really think about money being stored under a mattress until the first time I went to Eastern Europe where I am now. I'm in Hungary right now. And first time I went to Eastern Europe, I think I was in Belgrade in Serbia. And my uncle at the time, he was talking about how he kept all of his money in cash in a bank account. In Serbian Dinar.And Serbian Dinar had already gone through hyperinflation, like, ten years prior. Or no, it went through hyperinflation in 1996. So, it was not—it hadn't been that long [laugh]. And he was asking me for financial advice. And here I am, I'm like, you know, in my early-20s.And I'm like, I don't know what you should do with your money, but don't put it under your mattress. And that's the kind of data that Reddit—that ChatGPT seems to have been trained on, this GPT-3 data, it seems like a lot of [laugh] Redditors, specifically Redditors sub-2001. I haven't used Reddit very much in the last half a decade or so.Corey: Yeah, I mean, I still use it in a variety of different ways, but I got out of both of those cases, primarily due to both time constraints, as well as my circumstances changed to a point where the things I spent my time thinking about in a personal finance sense, no longer applied to an awful lot of folk because the common wisdom is aimed at folks who are generally on a something that resembles a recurring salary where they can calculate in a certain percentage raises, in most cases, for the rest of their life, plan for other things. But when I started the company, a lot of the financial best practices changed significantly. And what makes sense for me to do becomes actively harmful for folks who are not in similar situations. And I just became further and further attenuated from the way that you generally want to give common case advice. So, it wasn't particularly useful at that point anymore.Jack: Very. Yeah, that's very well put. I went through a similar thing. I watched Reddit quite a bit through the Ellen Pao thing because I thought it was a very interesting lesson in business and in social engineering in general, right? And we saw this huge community, this huge community of people, and some of these people were ridiculously toxic.And you saw a lot of groupthink, you saw a lot of manipulation. There was a lot of heavy-handed moderation, there was a lot of too-late moderation. And then Ellen Pao comes in and I'm, like, who the heck is Ellen Pao? Oh, Ellen Pao is this person who has some corporate scandal going on. Oh, Ellen Pao is a scapegoat.And here we are, watching a community being socially engineered, right, into hating the CEO who's just going to be let go or step down anyways. And now they ha—their conversations have been used to train intelligence, which is being used to socially engineer people [laugh] into [crosstalk 00:22:13].Corey: I mean you just listed something else that's been top-of-mind for me lately, where it is time once again here at The Duckbill Group for us to go through our annual security awareness training. And our previous vendor has not been terrific, so I start looking to see what else is available in that space. And I see that the world basically divides into two factions when it comes to this. The first is something that is designed to check the compliance boxes at big companies. And some of the advice that those things give is actively harmful as in, when I've used things like that in the past, I would have an addenda that I would send out to the team. “Yeah, ignore this part and this part and this part because it does not work for us.”And there are other things that start trying to surface it all the time as it becomes a constant awareness thing, which makes sense, but it also doesn't necessarily check any contractual boxes. So it's, isn't there something in between that makes sense? I found one company that offered a Slackbot that did this, which sounded interesting. The problem is it was the most condescendingly rude and infuriatingly slow experience that I've had. It demanded itself a whole bunch of permissions to the Slack workspace just to try it out, so I had to spin up a false Slack workspace for testing just to see what happens, and it was, start to finish, the sort of thing that I would not inflict upon my team. So, the hell with it and I moved over to other stuff now. And I'm still looking, but it's the sort of thing where I almost feel like, this is something ChatGPT could have built and cool, give me something that sounds confident, but it's often wrong. Go.Jack: [laugh]. Yeah, Uptycs actually is—we have something called a Otto M8—spelled O-T-T-O space M and then the number eight—and I personally think that's the cutest name ever for Slackbot. I don't have a picture of him to show you, but I would personally give him a bit of a makeover. He's a little nerdy for my likes. But he's got—it's one of those Slackbots.And I'm a huge compliance geek. I was a CISO for over a decade and I know exactly what you mean with that security awareness training and ticking those boxes because I was the guy who wrote the boxes that needed to be ticked because I wrote those control frameworks. And I'm not a CISO anymore because I've already subjected myself to an absolute living hell for long enough, at least for now [laugh]. So, I quit the CISO world.Corey: Oh yeah.Jack: Yeah.Corey: And so, much of it also assumes certain things like I've had people reach out to me trying to shill whatever it is they've built in this space. And okay, great. The problem is that they've built something that is aligned at engineers and developers. Go, here you go. And that's awesome, but we are really an engineering-first company.Yes, most people here have an engineering background and we build some internal tooling, but we don't need an entire curriculum on how to secure the tools that we're building as web interfaces and public-facing SaaS because that's not what we do. Not to mention, what am I supposed to do with the accountants in the sales folks and the marketing staff that wind up working on a lot of these things that need to also go through training? Do I want to sit here and teach them about SQL injection attacks? No, Jack. I do not want to teach them that.Jack: No you don't.Corey: I want them to not plug random USB things into the work laptop and to use a password manager. I'm not here trying to turn them into security engineers.Jack: I used to give a presentation and I onboarded every single employee personally for security. And in the presentation, I would talk about password security. And I would have all these complex passwords up. But, like, “You know what? Let me just show you what a hacker does.”And I'd go and load up dhash and I'd type in my old email address. And oh, there's my password, right? And then I would—I copied the cryptographic hash from dhash and I'd paste that into Google. And I'd be like, “And that's how you crack passwords.” Is you Google the cryptographic hash, the insecure cryptographic hash and hope somebody else has already cracked it.But yeah, it's interesting. The security awareness training is absolutely something that's supposed to be guided for the very fundamental everyman employee. It should not be something entirely technical. I worked at a company where—and I love this, by the way; this is one of the best things I've ever read on Slack—and it was not a message that I was privy to. I had to have the IT team pull the Slack logs so that I could read these direct communications. But it was from one—I think it was the controller to the Vice President of accounting, and the VP of accounting says how could I have done this after all of those phishing emails that Jack sent [laugh]?Corey: Oh God, the phishing emails drives me up a wall, too. It's you're basically training your staff not to trust you and waste their time and playing gotcha. It really creates an adversarial culture. I refuse to do that stuff, too.Jack: My phishing emails are fun, all right? I did one where I pretended that I installed a camera in the break room refrigerator, and I said, we've had a problem with food theft out of the Oakland refrigerator and so I've we've installed this webcam. Log into the sketchy website with your username and password. And I got, like, a 14% phish rate. I've used this campaign at multinational companies.I used to travel around the world and I'd grab a mic at the offices that wanted me to speak there and I'd put the mic real close to my head and I say, “Why did you guys click on the link to the Oakland refrigerator?” [laugh]. I said, “You're in Stockholm for God's sake.” Like, it works. Phishing campaigns work.They just don't work if they're dumb, honestly. There's a lot of things that do work in the security awareness space. One of the biggest problems with security awareness is that people seem to think that there's some minimum amount of time an employee should have to spend on security awareness training, which is just—Corey: Right. Like, for example, here in California, we're required to spend two hours on harassment training every so often—I think it's every two years—and—Jack: Every two years. Yes.Corey: —at least for managerial staff. And it's great, but that leads to things such as, “Oh, we're not going to give you a transcript if you can read the video more effectively. You have to listen to it and make sure it takes enough time.” And it's maddening to me just because that is how the law is written. And yes, it's important to obey the law, don't get me wrong, but at the same time, it just feels like it's an intentional time suck.Jack: It is. It is an intentional time suck. I think what happens is a lot of people find ways to game the system. Look, when I did security awareness training, my controls, the way I worded them, didn't require people to take any training whatsoever. The phishing emails themselves satisfied it completely.I worded that into my control framework. I still held the trainings, they still made people take them seriously. And then if we have a—you know, if somebody got phished horrifically, and let's say wired $2 million to Hong Kong—you know who I'm talking about, all right, person who might is probably not listening to this, thankfully—but [laugh] she did. And I know she didn't complete my awareness training. I know she never took any of it.She also wired $2 million to Hong Kong. Well, we never got that money back. But we sure did spend a lot of executive time trying to. I spent a lot of time on the phone, getting passed around from department to department at the FBI. Obviously, the FBI couldn't help us.It was wired from Mexico to Hong Kong. Like the FBI doesn't have anything to do with it. You know, bless them for taking their time to humor me because I needed to humor my CEO. But, you know, I use those awareness training things as a way to enforce the Code of Conduct. The Code of Conduct requiring disciplinary action for people who didn't follow the security awareness training.If you had taken the 15 minutes of awareness training that I had asked people to do—I mean, I told them to do it; it was the Code of Conduct; they had to—then there would be no disciplinary action for accidentally wiring that money. But people are pretty darn diligent on not doing things like that. It's just a select few that seems to be the ones that get repeatedly—Corey: And then you have the group conversations. One person screws something up and then you wind up with the emails to everyone. And then you have the people who are basically doing the right thing thinking they're being singled out. And—ugh, management is hard, people is hard, but it feels like a lot of these things could be a lot less hard.Jack: You know, I don't think management is hard. I think management is about empathy. And management is really about just positive reinforce—you know what management is? This is going to sound real pretentious. Management's kind of like raising a kid, you know? You want to have a really well-adjusted kid? Every time that kid says, “Hey, Dad,” answer. [crosstalk 00:30:28]—Corey: Yeah, that's a good—that's a good approach.Jack: I mean, just be there. Be clear, consistent, let them know what to expect. People loved my security program at the places that I've implemented it because it was very clear, it was concise, it was easy to understand, and I was very approachable. If anybody had a security concern and they came to me about it, they would [laugh] not get any shame. They certainly wouldn't get ignored.I don't care if they were reporting the same email I had had reported to me 50 times that day. I would personally thank them. And, you know what I learned? I learned that from raising a kid, you know? It was interesting because it was like, the kid I was raising, when he would ask me a question, I would give him the same answer every time in the same tone. He'd be like, “Hey, Jack, can I have a piece of candy?” Like, “No, your mom says you can't have any candy today.” They'd be like, “Oh, okay.” “Can I have a piece of candy?” And I would be like, “No, your mom says you can't have any candy today.” “Can I have a piece of candy, Jack?” I said, “No. Your mom says he can't have any candy.” And I'd just be like a broken record.And he immediately wouldn't ask me for a piece of candy six different times. And I realized the reason why he was asking me for a piece of candy six different times is because he would get a different response the sixth time or the third time or the second time. It was the inconsistency. Providing consistency and predictability in the workforce is key to management and it's key to keeping things safe and secure.Corey: I think there's a lot of truth to that. I really want to thank you for taking so much time out of your day to talk to me about think topics ranging from GPT and ethics to parenting. If people want to learn more, where's the best place to find you?Jack: I'm jack@jackroehrig.com, and I'm also jroehrig@uptycs.com. My last name is spelled—heh, no, I'm kidding. It's a J-A-C-K-R-O-E-H-R-I-G dot com. So yeah, hit me up. You will get a response from me.Corey: Excellent. And I will of course include links to that in the show notes. Thank you so much for your time. I appreciate it.Jack: Likewise.Corey: This promoted guest episode has been brought to us by our friends at Uptycs, featuring Jack Roehrig, Technology Evangelist at same. I'm Cloud Economist Corey Quinn and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice along with an angry comment ghostwritten for you by ChatGPT so it has absolutely no content worth reading.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.

Get up to speed with the Digital Marketing News and Updates from the week of Feb 27-Mar 3, 2023.1. PSA: US TikTok Ban Moves a Step Closer - More bad news for TikTok, with the US House Foreign Affairs Committee voted to give President Joe Biden the power to ban the Chinese-owned app, if he deems such a move necessary, amid ongoing security discussions around its potential connection to the Chinese Communist Part (CCP).TikTok responded to the vote by tweeting that “A U.S ban on TikTok is a ban on the export of American culture and values to the billion-plus people who use our  service worldwide…”While Today's announcement doesn't give Biden the full green light to ban the app, with the US Senate still required to give sign-off before a ban could be implemented. But it's another step towards that next stage, which increasingly feels like it will lead to a TikTok ban, or at the least, a significant change in direction for the app.Remember that TikTok, along with 58 other Chinese-created apps, was banned completely in India by the Ministry of Electronics and Information Technology on 29 June 2020. So if you are relying on traffic from TikTok then it is high time you diversify your traffic sources.2. Google Shares How Its Keyword-Matching System For Search Ads Work - Google has released a 28 page comprehensive guide during Google Search Ads Week 2023, providing a unique behind-the-scenes glimpse into its keyword-matching system for search ads.To achieve better results, advertisers can optimise their campaigns by gaining an understanding of Google Ads keyword-matching process.Google's guide provides a comprehensive breakdown of the system, which includes how the company utilises machine learning and natural language understanding technologies to determine keyword eligibility, and how the responsive search ads creative system selects the best-performing creative for users.It is essential to note that grouping keywords is critical to campaign optimisation. By eliminating the need to add the same keyword in multiple match types, advertisers can avoid segmenting and reducing the available data that Smart Bidding can use for optimisation, which can result in fewer conversions and higher costs.The guide is an invaluable resource for anyone seeking to enhance their Google Ads campaigns. Incorporating the insights and best practices outlined in the guide can boost the chances of success and drive more conversions.  This is why I always tell my listeners to work with a reputable learning and growing agency who is in the know. Afterall, you can not make moves or leverage opportunities if you are not in the know.3. Google Ads Is Changing Location Targeting Settings In March 2023 - Starting March 2023, “Search Interest” targeting will no longer be available in Google Ads. Campaigns that use “Search Interest” targeting will be migrated to “Presence or Interest” targeting. These changes will be consistent in Search, Display, Performance Max, and Shopping campaigns. The Presence option lets you show your ads to people who are likely to be located, or regularly located in the locations you've targeted.The Search Interest option lets you show your ads to anyone searching on Google for your targeted location. If a person doesn't specify a location in their search, then the system uses the location where a user is likely to be located for targeting. This option is only available for Search campaigns.So after this change is in effect, a person who lives in Northern VA but often travels to Maryland for shopping or work. While home in VA, the person searches for "plumber near me." Now Google is going to show some Maryland plumbers who are not licensed in VA.  Am I the only one who thinks that the real winner of this change is Google!!4. Google Ads Introduces AI-Powered Search Ads - During the Google's Search Ads Week, a new customer acquisition goal for Search campaigns has been launched globally. This goal utilizes Smart Bidding and first-party data to optimize campaigns and attract new customers during peak periods. According to Google, by combining the new customer acquisition goal with bidding strategies like Maximize conversion value with a target ROAS, advertisers can prioritize and target high-value customers. The new customer acquisition goal has two modes that help you to reach your campaign goals: Value New Customer: Bid higher for new customers than for existing customers New Customers Only: Bid for new customers only. 5. Microsoft Bing's Fabrice Canel : SEO Will Never Be "dead" - Fabrice Canel, the Principal Product Manager for Microsoft Bing, gave a keynote presentation at the Pubcon convention in Austin, Texas. His presentation offered valuable information on optimizing websites for the new Bing search experience as well as shared the benefits of using Bing Webmaster Tools to monitor traffic data and make necessary adjustments to improve visibility in search results.First, Canel suggested to stay with the same SEO playbooks for optimizing content for Bing's AI experience because it's still the early days for AI search. Throughout his keynote at Pubcon, Canel stressed the importance of SEO professionals in guiding Bing's search crawlers to high-quality content.Then Canel emphasized the importance of setting the lastmod tag to the date a page was last modified, not when the sitemap was generated. Remember lastmod was covered in previous episodes in details. ICYMI, the lastmod tag is an HTML attribute indicating when a particular webpage or URL received significant changes. This tag is used in sitemaps to help search engines like Bing understand when a page was last updated. Lastmod also helps searchers identify and access the most up-to-date content available. When a lastmod tag is present, Bing will display the updated date in search results. This signals to searchers that the webpage may have new or updated information they haven't seen yet. According to Canel, 18% of sitemaps have lastmod values not correctly set, typically set to the date and time the sitemap is generated.Thirdly, Canel recommended website to  adopting IndexNow to inform search engines of recent modifications to website content instantly. FYI: IndexNow was covered in episode# 90 (Jan 10-15, 2022). According to Canel, 20 million websites have already adopted IndexNow, and he expects more top websites, search engines, and content management systems to follow suit. Canel adds that manually crawling a webpage to see if its content has changed wastes resources and energy and creates CO2. He also suggests having sitemaps to provide search engines with all relevant URLs and corresponding modification dates.Most importantly, he wanted website owner focus on writing quality content and use semantic markup to convey information about the pages.Lastly, we learned Bing Webmaster Tools will soon include traffic data from Bing's AI chat.6. Google On ‘lastmod' Tag In XML Sitemap - I covered “lastmod” in episode#146. It is back again. Google's John Mueller said on Twitter if you are "providing something new for search engines that you'd like reflected in search," then update the date, if not, then don't. John added, "The issue is more that some CMS's / servers set the lastmod to the current date/time for all pages. This makes that data useless. Good CMS's setting it thoughtfully, even if not always perfect, is much more useful."The current Google documentation says, "Google uses the lastmod value if it's consistently and verifiably (for example by comparing to the last modification of the page) accurate." And according to a recent study at Bing (also covered in episode#146) revealed that among websites with at least one URL indexed by Bing: 58% of hosts have at least one XML sitemap (sitemap known by Bing).84% of these sitemaps have a lastmod attribute set 79% have lastmod values correct.  18% have lastmod values not correctly set.  3% has lastmod values for only some of the URLs. 42% of hosts don't have one XML sitemap (Bing does not know it) P.S: Don't be the business that is skipping the basics and easy to do stuff and looking to do advanced stuff. #DoTheBasics first.7. Google: Don't Combine Site Moves With Other Big Changes - Sometimes businesses make changes to their top-level domain as well as update their website. So Google Search Advocate John Mueller during a recent Search Of The Record Podcast with Gary Illyes, and Senior Technical Writer Lizzi Sassman asked “What happens if I do a domain change, and move from a “.ch”, which is a Swiss top level domain, to “.com”? Is that a problem? Like if I combine a domain change with other stuff?”In response, Illyes, shared that these changes should be done in smaller pieces over months. Making too many changes at once could result in lower rankings and lost traffic. For example, if a website is moving from “example.ch” and “example.fr” to “example.com,” Illyes recommended moving “example.fr” first and waiting before moving “example.ch.”Mueller and Sassman questioned Illyes on why he's so concerned about spreading out site moves. Illyes admitted that many site moves he's been involved with have resulted in lost traffic. Illyes also mentioned that misconfigurations, such as incorrect redirects, are common mistakes that can cause traffic loss. However, traffic shouldn't be lost during a domain change if everything is done correctly.If all you're doing is redirecting URLs from one site to another, there's a low risk for adverse effects. On the other hand, if you do lose rankings and traffic, there's no specific timeframe for a full recovery.8. Google's Gary Illyes: Google Does Not Care Who Authors or Links To The Content - Gary Illyes from Google gave a keynote and a Q&A session at PubCon and while the keynote was pretty vanilla stuff, the Q&A did reconfirm a lot of what has been said in the past around authorship, links and disavowing links. In short, Google does not give too much weight to who writes your content. So if you get a Walt Mossberg to write a piece of content on your site, just because it is Walt, doesn't make it rank well. If the content is written well, it will rank well, but by default, just because Walt wrote it, doesn't make it rank well. Gary also said that links are not as important as SEOs think they are.  And disavowing links is just a waste of time.P.S: All these topics have been covered in the past shows. 9. Google: PageRank Sculpting Is A Myth - Every website is assigned a unique value by the Google PageRank algorithm. This value, also called PageRank, has long been an important factor in link building and link exchange. PageRank sculpting is a technique in which an attempt is made to distribute the PageRank of a website to other subpages. Assuming that the home page receives the highest PageRank because it is the most important within the sites hierarchy, the PageRank will decrease as you go further down into the structure. Before 2009, it was common practice to control the PageRank through sculpting so that only certain pages would benefit. For example, function pages such as the imprint or contact page were linked internally with the attribute “nofollow.” Thus, the link power increased (as measured by PageRank) for the remaining internal links. Unfortunately, some SEO Experts still feel that they can control how Google passes your link equity throughout your site by using the nofollow link attribute. So Google's John Muller said on Twitter that it is an SEO myth to say you can use the nofollow attribute on links sculpt PageRank. Remember, back in 2019 he tweeted that Internal PageRank Sculpting Is A Waste Of Time. Another #SEOMythBusted. I'll file this under #AvoidBadSEOAdvice.10. Check Domain Reputation Before You Buy A Domain - Google's John Mueller was asked about a domain name purchased several months ago but still does not rank well in Google Search. John explained that if a domain has a "long and complicated history." "It's going to be hard to convince search engines that it's something very different & unrelated to what was done in the past decades," John added.In short, he is saying that not only was this domain abusing search engines for a long, long time, but also that the new content on this old domain is not different enough or unrelated enough from what the topic was previously where the search engine would consider it a brand new site and wipe the site clean.Basically the issue here is “domain legacy penalty” - It's a penalty that's associated with a domain from when it was registered by someone else in the past. Apparently the penalty remains after the domain is registered by someone else years later. Which makes sense or else bad actors will keep on transferring domain ownership to bypass the penalty. The way to prevent is to check the past history of a domain name is to visit Archive.org. Archive.org downloads and creates an archive of websites throughout the Internet.A similar issue happened a few years ago to ZDNet. One of their domains was hyphenated (CXO-Talk.com). So they purchased the non-hyphenated variant (CXOTalk.com) from a third party domain auction. ZDNet was unaware that the domain had been used by spammers.  Soon after ZDNet migrated all their content from CXO-Talk.com to CXOTalk.com, their website was banned from Google. ZDNet wrote an article about what happened to them and had the following advice: Before purchasing any domain at auction, be sure to check its history using backlink tools If the domain has a bad history, use Google Webmaster Tools to do a clean-up before putting the domain into service Google's system of problem remediation lacks transparency and responsiveness. They can and should do better. I still don't really know what caused the problem or how to fix it. 11. Should You Rewrite Your Content With ChatGPT? - Google's John Mueller went back and forth on Twitter with some SEO practitioners on the topic of using ChatGPT to (re)write existing content. Basically Ujesh was wondering if he can rewrite his own content with the help of tools like #ChatGPT without losing its helpfulness and relevancy. He was curious to see if it will  reduce the quality of the article due to AI involvement or does it boost the article considering the quality revamp ?To that question, John asked “Why do you need to rewrite your own content? Is it bad?” IMO, this is a fair question.To John's question Paulo replied, “let's say that English is not my main language. Then, I write something in my mother tongue, translate it in my own limited vocabulary, and ask AI to enhance the vocabulary. The content is not bad, but limited by my knowledge of a language, not the topic I'm trying to cover.”And John responded by saying “Why do you want to just publish something for the sake of publishing something, rather than publishing something you know to be useful & good? (This is not unique to LLM/AI NLG, it's the same with unknown-quality human-written content.) What do you want your site known for?”John is saying that, if your content is bad, why are you writing it in the first place? If you know your content is bad, then it is not helpful, will ChatGPT make it helpful for you? How do you know if the ChatGPT version is helpful and quality if your content you originally wrote is not quality? Maybe instead of using ChatGPT to improve the quality of your content, maybe you should focus on topics that you can write quality content about?

Episode: 2557 Linear algebra, the mathematics behind Google's ranking algorithm.  Today, let's talk about how Google ranks your search results.

I interviewed Charles Floate, an SEO known for ranking websites in ways that go against Google's spam guidelines. We discussed PageRank and the importance of links in Google's algorithms. Charles revealed ways he's been able to manipulate Google's guidelines and succeed, though I don't condone such practices. Our chat helped me reform my thoughts on links and their power to move the needle for rankings. Links are most definitely important in Google's algorithms and will likely remain the core for quite some time. But as Google develops new ways to identify quality content, most of which comes with AI innovations, I believe PageRank is not as important to SEO as it used to be. In this video Charles shows me how he is able to get sites to rank. Some of what he is doing is actually improving E-A-T! I must reiterate that breaking Google's policies can lead to serious consequences While good links can be powerful, building your own links is often either ignored by Google, or can cause manual or algorithmic penalties which can be very difficult to identify and recover from. My hope with this video is not to encourage blackhat SEO tactics, but rather, to talk about the power of GOOD links and how that power is much more than PageRank. Thanks for doing this, Charles. I enjoyed our chat! Charles Floate on Twitter: https://twitter.com/charles_seo https://www.youtube.com/user/D4rkHacking?app=desktop Marie's info: https://twitter.com/Marie_Haynes https://mariehaynes.com/contact/ Contact Marie for a referral to companies that can help you get good links and mentions from authoritative sites in ways that comply with Google's guidelines. My book on auditing links (used by many agencies to train their link auditors). My book on using the Quality Raters' Guidelines as a checklist for site quality. Wix SERPs Up Podcast with Mordy Oberstein and Crystal Carter: https://www.wix.com/seo/learn/podcast

Cord & Einar discuss all things linking. Can JavaScript links pass PageRank? How to HTTP to HTTPS redirects work? Are old links irrelevant? Will links always be a ranking factor? What's a link graph?Sources Cited:Search Engine Roundtable: Google Updates Help Documentation To Say Using JavaScript For Links Can Be FineGoogle Search Central: Make your links crawlableSearch Engine Roundtable: Google Can Pass PageRank From HTTP To HTTPS URLs With RedirectsSearch Engine Roundtable: Google: Some Old Links May Be Irrelevant (AKA Link Rot)Search Engine Roundtable: Google: Links Will Be Less Important As A Ranking Factor In The FutureKevin Indig: https://twitter.com/kevin_indig/status/1590115045236965376SEMRush: Get a Bird's Eye View of Any Website's Link NetworkSearch Engine Roundtable: Google Search Console API Delay To Be Fixed In DaysSearch Engine Journal: Is Bounce Rate A Google Ranking Factor?

God has given each of us as believers a story to share with the world.   Today's guest, Janet Whisnant Harllee, talk show host of "Faith In An Ever Changing World," discusses the importance of sharing your faith story with others to bring about encouragement, Inspiration, and hope while sharing the love of Jesus.   Please join us to hear examples of why YOUR story might make a difference in another's life.   *****   Welcome to Empowering Lives With Purpose. And I'm your host, Kimberly Hobbs. I am the founder of Women World Leaders. And we are so happy that you have joined us today. Today. Let's welcome our guest, Janet Harllee. Welcome, Janet, we are so happy to have you.  Janet Harllee   Thank you, Kimberly, thank you for giving me this opportunity.  Kimberly Hobbs   Of course, of course and ladies today. This is really fun because Janet has a program where others are invited on to share their stories. And I'm going to introduce her in just a moment. But I just wanted to say a little bit about who we are women, world leaders, the name of our podcast today is empowering lives with purpose. And it is our desire ladies to inspire you encourage you in the Word of God, and also to walk out that beautiful purpose that God has just for you. God tells us in His word that we are a masterpiece Ephesians 210 says we are God's masterpiece, we are created anew in Christ Jesus to do that very good things that he has planned for us long ago. And ladies, we know that you have a beautiful purpose. And whether you're walking it out now, or you have those reservations of walking it out, we want to help you we want to propel you forward in your faith, to walk out that purpose and serve the Lord wholeheartedly. And so as we have some of our guests on that is our purpose that they can help inspire you through sharing their stories of how they came to walk with Jesus and share their purpose. And when they share with you. Hopefully that will just jog something inside of you to say, Okay, Lord, if they can do it, I can do it. Right. So, Janet, I just want to share a little bit about our guest, Janet Harley today. She is a storyteller, and she's a speaker, a broadcaster and an author. And she has a passion to share God's truth and deliver it to her audiences, where she encourages, inspires and entertains with messages of faith in an ever changing world. Her other experiences include theater, radio, television, and she's currently the host of her broadcast, which is faith in an ever changing world, which gives encouragement and hope, as she interviews with others to share their faith story. And also pastors who share various faith topics with faith focus. She enjoys encouraging people through coffee breaks, and she loves making new friends. And Janet Harley is available on YouTube. So Janet, again, we welcome you. And thank you for being here. We, you're so welcome. We each have a story to tell ladies. And the stories could be ranging from how we give birth and pains and trials that we've had, you know, we all have different stories that we share with people. But as Christ followers, ladies, those stories are of the utmost important, what has he done in your life? What are our Faith Journeys? What is your faith journey, and the goal of our faith journey is to deepen our relationship with God. Right. And by sharing our faith story, and telling our story, we encourage others to share their stories. And God says in Revelation 1211, And they overcame him by the blood of the Lamb. And by the word of their testimony, we overcome the enemy, and all that he's trying to do in our life. When we proclaim our testimony to the world. God says that in his word, and that is how important it is. Janet, you have a story. So how did you get started in encouraging others to share their story? Let's hear a little bit about your story. Janet Harllee   Well, I retired a couple of years ago at the age of 70 and And I've never really thought about retiring, Kimberly, I enjoy what I was doing. And the Lord has blessed me with good health. And so, but at 70, I thought, well, it's just time. But you know, you never retire because God's work never stops. So I wondered what was going to be next? My next chapter, what, what, Lord, what do you have for me? Now, what can I do? And one of my speaker topics was faith in an ever changing world. And I added encouragement and hope. Because for the broadcast, he put on my heart to, to do a Facebook page. And it's really his Facebook page, not mine. And he wanted me to help others or to get other people to share their stories, because in how faith, either how faith had gotten them through a certain situation in their life, or their testimony, that we each have a testimony a story. And when we tell our story, we tell HEDIS. Amen. So I started the PageRank. So just let it go around in my head for a while, you know, and in September last year, and this September will be a year that the broadcast has been going on Facebook. And so I've just pray every day that the Lord will use me will show me what he wants me to do. Because when you retire, when I retired, I repurposed. I retired to repurpose. And I just think that the Lord has says he has put this on my heart, that this is what he wants me to do at this particular time. Kimberly Hobbs   Oh, my goodness, Janet, you are so delightful. And I love that when you retired, you said you repurposed? Yeah. How awesome. Is that? Right? That? I said, I want to be like you when I. When I'm at that age to retire, I'm gonna say no, Lord, let me repurpose it, because serving Him is amazing, right? Janet Harllee   That's right. That's right. And we are, we are to do that. That certainly is our purpose is to encourage and empower each other, especially as women, Kimberly Hobbs   especially right as older women. And so ladies, your job on earth here is never done. Don't you think about retiring? Because God says no, he's gonna use you. You're here for a purpose. Remember that? If anything in this podcast today, you're here for a purpose, ladies. And by sharing your faith story, it touches it touches other people's hearts and helps them see their story through your story. Okay, so as they're seeing and listening to you, they identify with what you're saying. That's the importance of sharing the story. They can have compassion, they can be inspired. But they can also say, Wow, if she did it, I can do it. Right. Right. So it's through sharing these stories that also brings us closer to God, ladies, and I think God for Janet hardly that she saw a purpose in this. And she decided, You know what, she's going to go and gather those stories that God brings across her path to share with the world. So thank you, Janet, so much for doing that. So much. So I have another question for you. So tell us how you would encourage someone else to share their God's story. Janet Harllee   Well, stories connect us on so many levels, Kimberly, they are inspirational to to us. They encourage, they are empowering. And it's important for us to share those stories because we can certainly apply as you have said help other women. Women because we go through a lot of the same kinds of trials. And I'm a member of a group testimony Tuesdays on Facebook. And so we Every Tuesday we share testimony. Well, I shared a testimony Yesterday, it was a testimony of obedience because my sister and I became caregivers for my parents. And you know, when when the roles switch and you become the parent, it's, it's difficult, it's emotional. It's tiring, but it's also the most rewarding of anything I ever do. And my parents were such godly parents. And so it was just so in sharing that through the other comments you see, of other people, they can relate. Maybe they've been a caregiver too, and they knew what I was talking about. And so, in our stories in our storytelling, we certainly want to share this. And a lot of times, women think that they don't have a story. And they, but you do, yes. Um, now my story about how I came to know the Lord. I don't have a heart wrenching story to share as Sunday because I was blessed to grow up in a Christian home. But I do, it's my story. And I enjoy telling you, and as I've gotten older, and in this stage of life, where I am right now, in my last quarter of life, I am more closer to God than I ever have been. And each day, I just seem to be even closer to him. Kimberly Hobbs   So is that saying true that your later days are your glory days? Janet Harllee   Oh, yes, I tell you, he has just been another can look back and see how faithful and how he has worked in my life. From the pit pitfalls and consequences that could have happened. And Kimberly Hobbs   he prepared you for such a time as this, he prepared you, that you you know, you may have not had like you said that heart wrenching story. But yet he knew that you were going to be compassionate. And you are going to encourage others that we have those crazy, you know, stories of what they've been through those heart wrenching moments, you know, that are and you encourage them, to talk about it, to share it to express it to others, in hopes that it will draw them closer to our Lord and Savior. And that is one of the best ways to share your story as you are you're drawing others closer to God by letting them listen into your life being vulnerable being open, right? Be a friend. Yes, it'd be kind of godly examples. That's what we're called to do. And you talked about obedience to, you know, you were obedient to take care of your parents, like, you know, you can be doing a lot of other things in your retirement years. Right. Janet Harllee   Absolutely. Still working. This was while I was still working. Wow. So yeah, we had to move. And, of course, I needed a job and a place to live. Me and my husband. So Robert found an apartment that was brand new, nobody had lived in it. So we we got to got a place to live. I called a lifelong friend. And she was taking a new position and needed someone. So I had a job. I mean, it's just things like that how things fall together. When we obey. Kimberly Hobbs   Oh, that is so true. That did you hear that? Ladies? When you walk in obedience with the Lord, He's going to guide your steps, things are going to come together for you. If you think everything's in array right now, check your obedience to the Word of God. Put it up against the scriptures. Are you walking in obedience to Him? Are you doing your own thing? Yeah. So Janet, you said it right. Obedience is key to unlocking the blessings that God has in our lives. And he blesses. That's right. That's right. He sure does. So God gives us examples, ladies all over his word about how we should share our testimonies and why we should share our testimonies. And one of those is John 439, where it talks about the Samaritan woman, and she had an encounter with Jesus. And then she went out and she shared that encounter. She shared that story of how it transformed her life. She was an AHA of this wonderful Savior that forgave her for what she was doing knew everything about her. And so she ran out and told all of her fellow neighbors her story. And in that the town believed in Jesus. And it's all because of this woman testifying lives were transformed. So that's what we're called to do Janet Harllee   I love that story. Amen. It's just and that's what we should do. And with joy, Kimberly Hobbs   with joy. So one more verse I want to share is First Chronicles 16, eight, where it says, Oh, give thanks to the Lord call upon his name, make known his deeds among the people. Does he say, keep quiet about everything God did in his word? No, it says, Make known his deeds among the peoples. So ladies, we have a job to do. Janet, I know that through interviewing all of these women and men that you interviewed, because you do both you do men and women, you've heard some amazing stories. Yeah. So can you encourage the women by sharing maybe some of those that have had an impact on not only you, but others that have really made a difference and why they made a difference? Janet Harllee   Absolutely. There are many, but there are a couple of women that come to my mind right away. One woman had three boys, and they of course, her husband. And one night they were awakened with fire in their home. And so they were trying to get out and get the children out. And I she was badly burned. And still have scars today, but of the burn. But she passed out during a while she was in the ambulance, and didn't wake up for six weeks. And but God healed her it took a long time. But the the very, very another sad part of it was she did lose, they could not get her youngest son out. So he did perish in the fire. So she speaks to women today about losing those who have lost a child, and all you know the emotions, and she has a podcast and called Purple thoughts. And if any women listening wants to know about it, and to know more about it, please, we'll give you the information to get in touch with me a little while. But she is just such a wonderful, inspirational woman and speaks a testimony all the time, into the halls of how God has been so faithful and good and how he still is being faithful and good. The other day, I noticed she showed a picture of her getting in there taking her into the ambulance, and then a picture of her today. How alive she is and how beautiful she is. And Kimberly Hobbs   She had to take a step forward to faith right? Yeah, absolutely. And not be in that place. I mean, because we all know if we've gone through sickness, you know, we can wallow in our pity of woe is me for a long time. Yeah. And it took a lot to overcome what she did, but she stepped down and shared her faith story and what she's doing now and I know you have more, share some more of that story. Janet Harllee   Oh, one more, one more real quick, about a lady who was kidnapped at the age of five, and found at the age of 10. And during those five years, she was abused every day, by her kidnapper. She now has has turned her life around. Of course there's so much that can be said about her because of his She was so young. And to have this happen to her. At such a young age, that's all she knew was the behavior and the the language that she heard from this man. And so that's all she knew. And when she was found at the age of 10. She was so relieved she was Happy to be away from that environment. But social services took over and so forth. And at some point they they did it to her into school. And in the fifth in the fifth grade, she had never been to school at all. And so she didn't know how to act didn't know how to react, to interact with other children, and the teachers, but she, she got through. And then later a lady came in her life that introduced her to the Lord. And so her life, of course changed. And her faith every day now is spent helping other women who have been in similar situations, and how they can come to know the Lord and how she forgave her kidnapper, and how she brought him to the Lord. And didn't get to meet her mother. And I didn't say this, but she was abandoned at the hospital by her mother. So she was abandoned, kidnapped. It's just amazing how God got her through this situation. And now, how her how he's using her life to help other women. Wow, wow, is just amazing. So these are encouraging stories, amazing stories. And I can't imagine being in this situation, because it nothing like that has ever happened to me. But it just, it just inspires me to, to be more grateful, and more thankful to God about house so faithful and good in my life. And how I can help other women and men to share their stories to encourage and bring hope to others. Kimberly Hobbs   Right. Wow, Janet, and thank you, thank you for encouraging this woman to come forward and share that story. And even you sharing it today, how many lives are being touched, just listening to this. And knowing that when she came to know Jesus, she took her whole life story, and she put it into purpose in serving Him. And how powerful is it because others listening may have just come out of something tragic, just like that. They can identify with the pain and the hurt and the suffering. And then you get their ear because they'll listen to you because they relate to you. And you can share with them, you know, the overcoming that when we know Jesus, and we're following Him in His steps. So we have a story to tell of turn to Jesus and give him all your hurts and pains. They'll listen to you at that moment because they're identifying with you. Yes. So don't you doubt ladies for a moment that your story isn't important. Just like Janet said. She didn't grow up with a unbelievable story. She grew up and God was preparing her life for such a time as this to help others share their stories. Do you see this? Like, it just goes on and on and on. And we don't know what time in our lives God is going to use us or when he wants to use us every day of our life, by the way. So honestly, we are so appreciative of what you do Janet and encouraging others to share their story. Yes. And so tell me how I you know we have to close again wrapping up. It's just like our time with alive. It does it goes so fast. But I would like for you to share a couple of closing words to encourage the woman to share her story. And also tell us how ladies can get a hold of you Janet. Janet Harllee   Okay. First of all, I'd like to share my life scripture is found in Proverbs 356. Trust in the Lord with all your heart and lean not on your own understanding. In all your ways acknowledge him, and he shall direct your paths. No matter what age you are women, no matter what age we are, that he shall direct our paths and one other one. Which is it At this particular time in my life, it's how he is going to be with us, even when our hairs turn gray. And my now is turning gray. And I love it Kimberly Hobbs   With age comes wisdom, Janet. Janet Harllee   That's right. That's exactly right. Kimberly is wisdom. And but I just pray every day for his wisdom, and discernment. And just stay true to God, trust God, in all things. Kimberly Hobbs   Amen, that that Scripture is so special to me too, because that's my mom's verse. And you know, I always hear or, you know, I can hear her with her little finger up in the year, Trust the Lord with all your heart. You know, like that's always saying that, so it's a great one. So how can ladies get in touch with you, Janet? Janet Harllee   All right. Of course, I have a YouTube and you just type in my name, Janet Harley. And the email that I have is Janet storyteller@gmail.com. Kimberly Hobbs   Hey, man, I love that. Okay. And Janet is also a woman world leader, we are so grateful that she has come on board to serving and just being a part of just a unity of women that are going out into the world with their special purposes, doing what God has called them to do. And I thank you for that. Janet again, to serve with you is a pleasure. Janet Harllee   Oh, it's been a pleasure with us. Well, thank you so much. Kimberly Hobbs   Oh, you're so welcome. Ladies, I just in closing, want to share Luke 839, which says return to your home and declare how much God has done for you. Proclaiming throughout the whole city how much Jesus has done. Oh, my goodness, right. He tells us go out into the city ladies proclaim it. It doesn't mean just be quiet and just tell somebody here on the corner, you know, in secret? No, it says proclaim it that means be loud about it. Proclaim what he has done with for you ladies share your story. It's so important that is the purpose of empowering lives with purpose is to get women on here to share their stories. So others can be inspired. And we hope that we're doing that for you. And another way that women world leaders loves to inspire is we do some books each year. And one of the books that we put out is tears to triumph. And it is releasing pain to receive God's restoration. So by some of the women that have shared in this amazing book, their stories of pain of suffering, it has inspired others to find Jesus. And we are so this book went to number one best seller. And number one international best seller, tears to triumph. It's available on our website, women world leaders.com. It's also available on Amazon, but we'd prefer you get it through women, world leaders. Ladies, this is a book that you will just you will cry through this book, but you will be inspired. And also we have another amazing tool for you ladies. And this is voice of truth women were leaders puts out a publication by monthly magazine. I see Janet is holding up hers. For those watching on YouTube. Yes, yay. Well, voice of truth is your tool ladies to inspire you encourage you strengthen you in the Lord filled with the gospel message of Jesus and every addition. Also, we have many ways that you can get involved with women were a leader as you can find us through voice of truth, of course, and be inspired to share your story somehow, someway, maybe, right? Maybe do a podcast with us. Whatever it it is that God's putting on your heart. We give opportunities here in this ministry, for you to get involved and share your beautiful purpose with the world just as God has asked you to do as to all of us to do some ladies and close. Again, I want to say thank you to our guests, Janet. I love you Janet, you are just beautiful treasure. Thank you so much. God bless you ladies each and every one of you and again I just pray that something today touched your heart and that you will go out into the world and proclaim what God has done share your story flames there is a world out there that is hurting and needs to be inspired through Jesus Christ live The good side. So ladies, from our heart, from his heart to yours, we are women, world leaders and all content is copyrighted and cannot be used without expressed written. I bless you all and have a wonderful day.    

Steve Hsu is a Professor of Theoretical Physics at Michigan State University and cofounder of the company Genomic Prediction.We go deep into the weeds on how embryo selection can make babies healthier and smarter. Steve also explains the advice Richard Feynman gave him to pick up girls, the genetics of aging and intelligence, & the psychometric differences between shape rotators and wordcels.Watch on YouTube. Listen on Apple Podcasts, Spotify, or any other podcast platform.Subscribe to find out about future episodes!Read the full transcript here.Follow Steve on Twitter. Follow me on Twitter for updates on future episodes.Please share if you enjoyed this episode! Helps out a ton!Timestamps(0:00:14) - Feynman’s advice on picking up women(0:11:46) - Embryo selection(0:24:19) - Why hasn't natural selection already optimized humans?(0:34:13) - Aging(0:43:18) - First Mover Advantage(0:53:49) - Genomics in dating(1:00:31) - Ancestral populations(1:07:58) - Is this eugenics?(1:15:59) - Tradeoffs to intelligence(1:25:01) - Consumer preferences(1:30:14) - Gwern(1:34:35) - Will parents matter?(1:45:25) - Word cells and shape rotators(1:57:29) - Bezos and brilliant physicists(2:10:23) - Elite educationTranscriptDwarkesh Patel  0:00  Today I have the pleasure of speaking with Steve Hsu. Steve, thanks for coming on the podcast. I'm excited about this.Steve Hsu  0:04  Hey, it's my pleasure! I'm excited too and I just want to say I've listened to some of your earlier interviews and thought you were very insightful, which is why I was excited to have a conversation with you.Dwarkesh Patel 0:14That means a lot for me to hear you say because I'm a big fan of your podcast.Feynman’s advice on picking up womenDwarkesh Patel  0:17  So my first question is: “What advice did Richard Feynman give you about picking up girls?”Steve Hsu  0:24   Haha, wow! So one day in the spring of my senior year, I was walking across campus and saw Feynman coming toward me. We knew each other from various things—it's a small campus, I was a physics major and he was my hero–– so I'd known him since my first year. He sees me, and he's got this Long Island or New York borough accent and says, "Hey, Hsu!"  I'm like, "Hi, Professor Feynman." We start talking. And he says to me, "Wow, you're a big guy." Of course, I was much bigger back then because I was a linebacker on the Caltech football team. So I was about 200 pounds and slightly over 6 feet tall. I was a gym rat at the time and I was much bigger than him. He said, "Steve, I got to ask you something." Feynman was born in 1918, so he's not from the modern era. He was going through graduate school when the Second World War started. So, he couldn't understand the concept of a health club or a gym. This was the 80s and was when Gold's Gym was becoming a world national franchise. There were gyms all over the place like 24-Hour Fitness. But, Feynman didn't know what it was. He's a fascinating guy. He says to me, "What do you guys do there? Is it just a thing to meet girls? Or is it really for training? Do you guys go there to get buff?" So, I started explaining to him that people are there to get big, but people are also checking out the girls. A lot of stuff is happening at the health club or the weight room. Feynman grills me on this for a long time. And one of the famous things about Feynman is that he has a laser focus. So if there's something he doesn't understand and wants to get to the bottom of it, he will focus on you and start questioning you and get to the bottom of it. That's the way his brain worked. So he did that to me for a while because he didn't understand lifting weights and everything. In the end, he says to me, "Wow, Steve, I appreciate that. Let me give you some good advice."Then, he starts telling me how to pick up girls—which he's an expert on. He says to me, "I don't know how much girls like guys that are as big as you." He thought it might be a turn-off. "But you know what, you have a nice smile." So that was the one compliment he gave me. Then, he starts to tell me that it's a numbers game. You have to be rational about it. You're at an airport lounge, or you're at a bar. It's Saturday night in Pasadena or Westwood, and you're talking to some girl. He says, "You're never going to see her again. This is your five-minute interaction. Do what you have to do. If she doesn't like you, go to the next one." He also shares some colorful details. But, the point is that you should not care what they think of you. You're trying to do your thing. He did have a reputation at Caltech as a womanizer, and I could go into that too but I heard all this from the secretaries.Dwarkesh Patel  4:30  With the students or only the secretaries? Steve Hsu  4:35  Secretaries! Well mostly secretaries. They were almost all female at that time. He had thought about this a lot, and thought of it as a numbers game. The PUA guys (pick-up artists) will say, “Follow the algorithm, and whatever happens, it's not a reflection on your self-esteem. It's just what happened. And you go on to the next one.” That was the advice he was giving me, and he said other things that were pretty standard: Be funny, be confident—just basic stuff. Steve Hu: But the main thing I remember was the operationalization of it as an algorithm. You shouldn’t internalize whatever happens if you get rejected, because that hurts. When we had to go across the bar to talk to that girl (maybe it doesn’t happen in your generation), it was terrifying. We had to go across the bar and talk to some lady! It’s loud and you’ve got a few minutes to make your case. Nothing is scarier than walking up to the girl and her friends. Feynman was telling me to train yourself out of that. You're never going to see them again, the face space of humanity is so big that you'll probably never re-encounter them again. It doesn't matter. So, do your best. Dwarkesh Patel  6:06  Yeah, that's interesting because.. I wonder whether he was doing this in the 40’–– like when he was at that age, was he doing this? I don't know what the cultural conventions were at the time. Were there bars in the 40s where you could just go ahead and hit on girls or? Steve Hsu  6:19  Oh yeah absolutely. If you read literature from that time, or even a little bit earlier like Hemingway or John O'Hara, they talk about how men and women interacted in bars and stuff in New York City. So, that was much more of a thing back than when compared to your generation. That's what I can’t figure out with my kids! What is going on? How do boys and girls meet these days? Back in the day, the guy had to do all the work. It was the most terrifying thing you could do, and you had  to train yourself out of that.Dwarkesh Patel  6:57  By the way, for the context for the audience, when Feynman says you were a big guy, you were a football player at Caltech, right? There's a picture of you on your website, maybe after college or something, but you look pretty ripped. Today, it seems more common because of the gym culture. But I don’t know about back then. I don't know how common that body physique was.Steve Hsu  7:24  It’s amazing that you asked this question. I'll tell you a funny story. One of the reasons Feynman found this so weird was because of the way body-building entered the United States.  They  were regarded as freaks and homosexuals at first. I remember swimming and football in high school (swimming is different because it's international) and in swimming, I picked up a lot of advanced training techniques from the Russians and East Germans. But football was more American and not very international. So our football coach used to tell us not to lift weights when we were in junior high school because it made you slow. “You’re no good if you’re bulky.” “You gotta be fast in football.” Then, something changed around the time I was in high school–the coaches figured it out. I began lifting weights since I was an age group swimmer, like maybe age 12 or 14. Then, the football coaches got into it mainly because the University of Nebraska had a famous strength program that popularized it.At the time, there just weren't a lot of big guys. The people who knew how to train were using what would be considered “advanced knowledge” back in the 80s. For example, they’d know how to do a split routine or squat on one day and do upper body on the next day–– that was considered advanced knowledge at that time. I remember once.. I had an injury, and I was in the trainer's room at the Caltech athletic facility. The lady was looking at my quadriceps. I’d pulled a muscle, and she was looking at the quadriceps right above your kneecap. If you have well-developed quads, you'd have a bulge, a bump right above your cap. And she was looking at it from this angle where she was in front of me, and she was looking at my leg from the front. She's like, “Wow, it's swollen.” And I was like, “That's not the injury. That's my quadricep!” And she was a trainer! So, at that time, I could probably squat 400 pounds. So I was pretty strong and had big legs. The fact that the trainer didn't really understand what well-developed anatomy was supposed to look like blew my mind!So anyway, we've come a long way. This isn't one of these things where you have to be old to have any understanding of how this stuff evolved over the last 30-40 years.Dwarkesh Patel  10:13  But, I wonder if that was a phenomenon of that particular time or if people were not that muscular throughout human history. You hear stories of  Roman soldiers who are carrying 80 pounds for 10 or 20 miles a day. I mean, there's a lot of sculptures in the ancient world, or not that ancient, but the people look like they have a well-developed musculature.Steve Hsu  10:34  So the Greeks were very special because they were the first to think about the word gymnasium. It was a thing called the Palaestra, where they were trained in wrestling and boxing. They were the first people who were seriously into physical culture specific training for athletic competition.Even in the 70s, when I was a little kid, I look back at the guys from old photos and they were skinny. So skinny! The guys who went off and fought World War Two, whether they were on the German side, or the American side, were like 5’8-5’9 weighing around 130 pounds - 140 pounds. They were much different from what modern US Marines would look like. So yeah, physical culture was a new thing. Of course, the Romans and the Greeks had it to some degree, but it was lost for a long time. And, it was just coming back to the US when I was growing up. So if you were reasonably lean (around 200 pounds) and you could bench over 300.. that was pretty rare back in those days.Embryo selectionDwarkesh Patel  11:46  Okay, so let's talk about your company Genomic Prediction. Do you want to talk about this company and give an intro about what it is?Steve Hsu  11:55  Yeah. So there are two ways to introduce it. One is the scientific view. The other is the IVF view. I can do a little of both. So scientifically, the issue is that we have more and more genomic data. If you give me the genomes of a bunch of people and then give me some information about each person, ex. Do they have diabetes? How tall are they? What's their IQ score?  It’s a natural AI machine learning problem to figure out which features in the DNA variation between people are predictive of whatever variable you're trying to predict.This is the ancient scientific question of how you relate the genotype of the organism (the specific DNA pattern), to the phenotype (the expressed characteristics of the organism). If you think about it, this is what biology is! We had the molecular revolution and figured out that it’s people's DNA that stores the information which is passed along. Evolution selects on the basis of the variation in the DNA that’s expressed as phenotype, as that phenotype affects fitness/reproductive success. That's the whole ballgame for biology. As a physicist who's trained in mathematics and computation, I'm lucky that I arrived on the scene at a time when we're going to solve this basic fundamental problem of biology through brute force, AI, and machine learning. So that's how I got into this. Now you ask as an entrepreneur, “Okay, fine Steve, you're doing this in your office with your postdocs and collaborators on your computers. What use is it?” The most direct application of this is in the following setting: Every year around the world, millions of families go through IVF—typically because they're having some fertility issues, and also mainly because the mother is in her 30s or maybe 40s. In the process of IVF, they use hormone stimulation to produce more eggs. Instead of one per cycle, depending on the age of the woman, they might produce anywhere between five to twenty, or even sixty to a hundred eggs for young women who are hormonally stimulated (egg donors).From there, it’s trivial because men produce sperm all the time. You can fertilize eggs pretty easily in a little dish, and get a bunch of embryos that grow. They start growing once they're fertilized. The problem is that if you're a family and produce more embryos than you’re going to use, you have the embryo choice problem. You have to figure out which embryo to choose out of  say, 20 viable embryos. The most direct application of the science that I described is that we can now genotype those embryos from a small biopsy. I can tell you things about the embryos. I could tell you things like your fourth embryo being an outlier. For breast cancer risk, I would think carefully about using number four. Number ten is an outlier for cardiovascular disease risk. You might want to think about not using that one. The other ones are okay. So, that’s what genomic prediction does. We work with 200 or 300 different IVF clinics in six continents.Dwarkesh Patel  15:46  Yeah, so the super fascinating thing about this is that the diseases you talked about—or at least their risk profiles—are polygenic. You can have thousands of SNPs (single nucleotide polymorphisms) determining whether you will get a disease. So, I'm curious to learn how you were able to transition to this space and how your knowledge of mathematics and physics was able to help you figure out how to make sense of all this data.Steve Hsu  16:16  Yeah, that's a great question. So again, I was stressing the fundamental scientific importance of all this stuff. If you go into a slightly higher level of detail—which you were getting at with the individual SNPs, or polymorphisms—there are individual locations in the genome, where I might differ from you, and you might differ from another person. Typically, each pair of individuals will differ at a few million places in the genome—and that controls why I look a little different than youA lot of times, theoretical physicists have a little spare energy and they get tired of thinking about quarks or something. They want to maybe dabble in biology, or they want to dabble in computer science, or some other field. As theoretical physicists, we always feel, “Oh, I have a lot of horsepower, I can figure a lot out.” (For example, Feynman helped design the first parallel processors for thinking machines.) I have to figure out which problems I can make an impact on because I can waste a lot of time. Some people spend their whole lives studying one problem, one molecule or something, or one biological system. I don't have time for that, I'm just going to jump in and jump out. I'm a physicist. That's a typical attitude among theoretical physicists. So, I had to confront sequencing costs about ten years ago because I knew the rate at which they were going down. I could anticipate that we’d get to the day (today) when millions of genomes with good phenotype data became available for analysis. A typical training run might involve almost a million genomes, or half a million genomes. The mathematical question then was: What is the most effective algorithm given a set of genomes and phenotype information to build the best predictor?  This can be  boiled down to a very well-defined machine learning problem. It turns out, for some subset of algorithms, there are theorems— performance guarantees that give you a bound on how much data you need to capture almost all of the variation in the features. I spent a fair amount of time, probably a year or two, studying these very famous results, some of which were proved by a guy named Terence Tao, a Fields medalist. These are results on something called compressed sensing: a penalized form of high dimensional regression that tries to build sparse predictors. Machine learning people might notice L1-penalized optimization. The very first paper we wrote on this was to prove that using accurate genomic data and these very abstract theorems in combination could predict how much data you need to “solve” individual human traits. We showed that you would need at least a few hundred thousand individuals and their genomes and their heights to solve for height as a phenotype. We proved that in a paper using all this fancy math in 2012. Then around 2017, when we got a hold of half a million genomes, we were able to implement it in practical terms and show that our mathematical result from some years ago was correct. The transition from the low performance of the predictor to high performance (which is what we call a “phase transition boundary” between those two domains) occurred just where we said it was going to occur. Some of these technical details are not understood even by practitioners in computational genomics who are not quite mathematical. They don't understand these results in our earlier papers and don't know why we can do stuff that other people can't, or why we can predict how much data we'll need to do stuff. It's not well-appreciated, even in the field. But when the big AI in our future in the singularity looks back and says, “Hey, who gets the most credit for this genomics revolution that happened in the early 21st century?”, they're going to find these papers on the archive where we proved this was possible, and how five years later, we actually did it. Right now it's under-appreciated, but the future AI––that Roko's Basilisk AI–will look back and will give me a little credit for it. Dwarkesh Patel  21:03  Yeah, I was a little interested in this a few years ago. At that time, I looked into how these polygenic risk scores were calculated. Basically, you find the correlation between the phenotype and the alleles that correlate with it. You add up how many copies of these alleles you have, what the correlations are, and you do a weighted sum of that. So that seemed very simple, especially in an era where we have all this machine learning, but it seems like they're getting good predictive results out of this concept. So, what is the delta between how good you can go with all this fancy mathematics versus a simple sum of correlations?Steve Hsu  21:43  You're right that the ultimate models that are used when you've done all the training, and when the dust settles, are straightforward. They’re pretty simple and have an additive structure. Basically, I either assign a nonzero weight to this particular region in the genome, or I don't. Then, I need to know what the weighting is, but then the function is a linear function or additive function of the state of your genome at some subset of positions. The ultimate model that you get is straightforward. Now, if you go back ten years, when we were doing this, there were lots of claims that it was going to be super nonlinear—that it wasn't going to be additive the way I just described it. There were going to be lots of interaction terms between regions. Some biologists are still convinced that's true, even though we already know we have predictors that don't have interactions.The other question, which is more technical, is whether in any small region of your genome, the state of the individual variants is highly correlated because you inherit them in chunks. You need to figure out which one you want to use. You don't want to activate all of them because you might be overcounting. So that's where these L-1 penalization sparse methods force the predictor to be sparse. That is a key step. Otherwise, you might overcount. If you do some simple regression math, you might have 10-10 different variants close by that have roughly the same statistical significance.But, you don't know which one of those tends to be used, and you might be overcounting effects or undercounting effects. So, you end up doing a high-dimensional optimization, where you grudgingly activate a SNP when the signal is strong enough. Once you activate that one, the algorithm has to be smart enough to penalize the other ones nearby and not activate them because you're over counting effects if you do that. There's a little bit of subtlety in it. But, the main point you made is that the ultimate predictors, which are very simple and addictive—sum over effect sizes and time states—work well. That’s related to a deep statement about the additive structure of the genetic architecture of individual differences. In other words, it's weird that the ways that I differ from you are merely just because I have more of something or you have less of something. It’s not like these things are interacting in some incredibly understandable way. That's a deep thing—which is not appreciated that much by biologists yet. But over time, they'll figure out something interesting here.Why hasn’t natural selection already optimized humans?Dwarkesh Patel  24:19  Right. I thought that was super fascinating, and I commented on that on Twitter. What is interesting about that is two things. One is that you have this fascinating evolutionary argument about why that would be the case that you might want to explain. The second is that it makes you wonder if becoming more intelligent is just a matter of turning on certain SNPs. It's not a matter of all this incredible optimization being like solving a sudoku puzzle or anything. If that's the case, then why hasn't the human population already been selected to be maxed out on all these traits if it's just a matter of a bit flip?Steve Hsu  25:00  Okay, so the first issue is why is this genetic architecture so surprisingly simple? Again, we didn't know it would be simple ten years ago. So when I was checking to see whether this was a field that I should go into depending on our capabilities to make progress, we had to study the more general problem of the nonlinear possibilities. But eventually, we realized that most of the variance would probably be captured in an additive way. So, we could narrow down the problem quite a bit. There are evolutionary reasons for this. There’s a famous theorem by Fisher, the father of population genetics (aka. frequentist statistics). Fisher proved something called Fisher's Fundamental Theorem of Natural Selection, which says that if you impose some selection pressure on a population, the rate at which that population responds to the selection pressure (lets say it’s the bigger rats that out-compete the smaller rats) then at what rate does the rat population start getting bigger? He showed that it's the additive variants that dominate the rate of evolution. It's easy to understand why if it's a nonlinear mechanism, you need to make the rat bigger. When you sexually reproduce, and that gets chopped apart, you might break the mechanism. Whereas, if each short allele has its own independent effect, you can inherit them without worrying about breaking the mechanisms. It was well known among a tiny theoretical population of biologists that adding variants was the dominant way that populations would respond to selection. That was already known. The other thing is that humans have been through a pretty tight bottleneck, and we're not that different from each other. It's very plausible that if I wanted to edit a human embryo, and make it into a frog, then there are all kinds of subtle nonlinear things I’d have to do. But all those identical nonlinear complicated subsystems are fixed in humans. You have the same system as I do. You have the not human, not frog or ape, version of that region of DNA, and so do I. But the small ways we differ are mostly little additive switches. That's this deep scientific discovery from over the last 5-10 years of work in this area. Now, you were asking about why evolution hasn't completely “optimized” all traits in humans already. I don't know if you’ve ever done deep learning or high-dimensional optimization, but in that high-dimensional space, you're often moving on a slightly-tilted surface. So, you're getting gains, but it's also flat. Even though you scale up your compute or data size by order of magnitude, you don't move that much farther. You get some gains, but you're never really at the global max of anything in these high dimensional spaces. I don't know if that makes sense to you. But it's pretty plausible to me that two things are important here. One is that evolution has not had that much time to optimize humans. The environment that humans live in changed radically in the last 10,000 years. For a while, we didn't have agriculture, and now we have agriculture. Now, we have a swipe left if you want to have sex tonight. The environment didn't stay fixed. So, when you say fully optimized for the environment, what do you mean? The ability to diagonalize matrices might not have been very adaptive 10,000 years ago. It might not even be adaptive now. But anyway, it's a complicated question that one can't reason naively about. “If God wanted us to be 10 feet tall, we'd be 10 feet tall.” Or “if it's better to be smart, my brain would be *this* big or something.” You can't reason naively about stuff like that.Dwarkesh Patel  29:04  I see. Yeah.. Okay. So I guess it would make sense then that for example, with certain health risks, the thing that makes you more likely to get diabetes or heart disease today might be… I don't know what the pleiotropic effect of that could be. But maybe that's not that important one year from now.Steve Hsu  29:17  Let me point out that most of the diseases we care about now—not the rare ones, but the common ones—manifest when you're 50-60 years old. So there was never any evolutionary advantage of being super long-lived. There's even a debate about whether the grandparents being around to help raise the kids lifts the fitness of the family unit.But, most of the time in our evolutionary past, humans just died fairly early. So, many of these diseases would never have been optimized against evolution. But, we see them now because we live under such good conditions, we can regulate people over 80 or 90 years.Dwarkesh Patel  29:57  Regarding the linearity and additivity point, I was going to make the analogy that– and I'm curious if this is valid– but when you're programming, one thing that's good practice is to have all the implementation details in separate function calls or separate programs or something, and then have your main loop of operation just be called different functions like, “Do this, do that”, so that you can easily comment stuff away or change arguments. This seemed very similar to that where by turning these names on and off, you can change what the next offering will be. And, you don't have to worry about actually implementing whatever the underlying mechanism is. Steve Hsu  30:41  Well, what you said is related to what Fisher proved in his theorems. Which is that, if suddenly, it becomes advantageous to have X, (like white fur instead of black fur) or something, it would be best if there were little levers that you could move somebody from black fur to white fur continuously by modifying those switches in an additive way. It turns out that for sexually reproducing species where the DNA gets scrambled up in every generation, it's better to have switches of that kind. The other point related to your software analogy is that there seem to be modular, fairly modular things going on in the genome. When we looked at it, we were the first group to have, initially, 20 primary disease conditions we had decent predictors for. We started looking carefully at just something as trivial as the overlap of my sparsely trained predictor. It turns on and uses *these* features for diabetes, but it uses *these* features for schizophrenia. It’s the stupidest metric, it’s literally just how much overlap or variance accounted for overlap is there between pairs of disease conditions. It's very modest. It's the opposite of what naive biologists would say when they talk about pleiotropy.They're just disjoint! Disjoint regions of your genome that govern certain things. And why not? You have 3 billion base pairs—there's a lot you can do in there. There's a lot of information there. If you need 1000 to control diabetes risk, I estimated you could easily have 1000 roughly independent traits that are just disjoint in their genetic dependencies. So, if you think about D&D,  your strength, decks, wisdom, intelligence, and charisma—those are all disjoint. They're all just independent variables. So it's like a seven-dimensional space that your character lives in. Well, there's enough information in the few million differences between you and me. There's enough for 1000-dimensional space of variation.“Oh, how considerable is your spleen?” My spleen is a little bit smaller, yours is a little bit bigger - that can vary independently of your IQ. Oh, it's a big surprise. The size of your spleen can vary independently of the size of your big toe. If you do information theory, there are about 1000 different parameters, and I can vary independently with the number of variants I have between you and me. Because you understand some information theory, it’s trivial to explain, but try explaining to a biologist, you won't get very far.Dwarkesh Patel  33:27  Yeah, yeah, do the log two of the number of.. is that basically how you do it? Yeah.Steve Hsu  33:33  Okay. That's all it is. I mean, it's in our paper. We look at how many variants typically account for most of the variation for any of these major traits, and then imagine that they're mostly disjoint. Then it’s just all about: how many variants you need to independently vary 1000 traits? Well, a few million differences between you and me are enough. It's very trivial math. Once you understand the base and how to reason about information theory, then it's very trivial. But, it ain’t trivial for theoretical biologists, as far as I can tell.AgingDwarkesh Patel  34:13  But the result is so interesting because I remember reading in The Selfish Gene that, as he (Dawkins) hypothesizes that the reason we could be aging is an antagonistic clash. There's something that makes you healthier when you're young and fertile that makes you unhealthy when you're old. Evolution would have selected for such a trade-off because when you're young and fertile, evolution and your genes care about you. But, if there's enough space in the genome —where these trade-offs are not necessarily necessary—then this could be a bad explanation for aging, or do you think I'm straining the analogy?Steve Hsu  34:49  I love your interviews because the point you're making here is really good. So Dawkins, who is an evolutionary theorist from the old school when they had almost no data—you can imagine how much data they had compared to today—he would tell you a story about a particular gene that maybe has a positive effect when you're young, but it makes you age faster. So, there's a trade-off. We know about things like sickle cell anemia. We know stories about that. No doubt, some stories are true about specific variants in your genome. But that's not the general story. The general story you only discovered in the last five years is that thousands of variants control almost every trait and those variants tend to be disjoint from the ones that control the other trait. They weren't wrong, but they didn't have the big picture.Dwarkesh Patel  35:44  Yeah, I see. So, you had this paper, it had polygenic, health index, general health, and disease risk.. You showed that with ten embryos, you could increase disability-adjusted life years by four, which is a massive increase if you think about it. Like what if you could live four years longer and in a healthy state? Steve Hsu  36:05  Yeah, what's the value of that? What would you pay to buy that for your kid?Dwarkesh Patel  36:08  Yeah. But, going back to the earlier question about the trade-offs and why this hasn't already been selected for,  if you're right and there's no trade-off to do this, just living four years older (even if that's beyond your fertility) just being a grandpa or something seems like an unmitigated good. So why hasn’t this kind of assurance hasn't already been selected for? Steve Hsu  36:35  I’m glad you're asking about these questions because these are things that people are very confused about, even in the field. First of all, let me say that when you have a trait that's controlled by  10,000 variants (eg. height is controlled by order 10,000 variants and probably cognitive ability a little bit more), the square root of 10,000 is 100.  So, if I could come to this little embryo, and I want to give it one extra standard deviation of height, I only need to edit 100. I only need to flip 100 minus variance to plus variance. These are very rough numbers. But, one standard deviation is the square root of “n”. If I flip a coin “n” times, I want a better outcome in terms of the number of ratio heads to tails. I want to increase it by one standard deviation. I only need to flip the square root of “n” heads because if you flip a lot, you will get a narrow distribution that peaks around half, and the width of that distribution is the square root of “n”. Once I tell you, “Hey, your height is controlled by 10,000 variants, and I only need to flip 100 genetic variants to make you one standard deviation for a male,” (that would be three inches tall, two and a half or three inches taller), you suddenly realize, “Wait a minute, there are a lot of variants up for grabs there. If I could flip 500 variants in your genome, I would make you five standard deviations taller, you'd be seven feet tall.”  I didn't even have to do that much work, and there's a lot more variation where that came from. I could have flipped even more because I only flipped 500 out of 10,000, right? So, there's this  quasi-infinite well of variation that evolution or genetic engineers could act on. Again, the early population geneticists who bred corn and animals know this. This is something they explicitly know about because they've done calculations. Interestingly, the human geneticists who are mainly concerned with diseases and stuff, are often unfamiliar with the math that the animal breeders already know. You might be interested to know that the milk you drink comes from heavily genetically-optimized cows bred artificially using almost exactly the same technologies that we use at genomic prediction. But, they're doing it to optimize milk production and stuff like this. So there is a big well of variance. It's a consequence of the trait's poly genicity. On the longevity side of things, it does look like people could “be engineered” to live much longer by flipping the variants that make the risk for diseases that shorten your life. The question is then “Why didn't evolution give us life spans of thousands of years?” People in the Bible used to live for thousands of years. Why don't we? I mean, *chuckles* that probably didn’t happen. But the question is, you have this very high dimensional space, and you have a fitness function. How big is the slope in a particular direction of that fitness function? How much more successful reproductively would Joe caveman have been if he lived to be 150 instead of only, 100 or something? There just hasn't been enough time to explore this super high dimensional space. That's the actual answer. But now, we have the technology, and we're going to f*****g explore it fast. That's the point that the big lightbulb should go off. We’re mapping this space out now. Pretty confident in 10 years or so, with the CRISPR gene editing technologies will be ready for massively multiplexed edits. We'll start navigating in this high-dimensional space as much as we like. So that's the more long-term consequence of the scientific insights.Dwarkesh Patel  40:53  Yeah, that's super interesting. What do you think will be the plateau for a trait of how long you’ll live? With the current data and techniques, you think it could be significantly greater than that?Steve Hsu  41:05  We did a simple calculation—which amazingly gives the correct result. This polygenic predictor that we built (which isn't perfect yet but will improve as we gather more data) is used in selecting embryos today. If you asked, out of a billion people, “What's the best person typically, what would their score be on this index and then how long would they be predicted to live?”’ It's about 120 years. So it's spot on. One in a billion types of person lives to be 120 years old. How much better can you do? Probably a lot better. I don't want to speculate, but other nonlinear effects, things that we're not taking into account will start to play a role at some point. So, it's a little bit hard to estimate what the true limiting factors will be. But one super robust statement, and I'll stand by it, debate any Nobel Laureate in biology who wants to discuss it even,  is that there are many variants available to be selected or edited. There's no question about that. That's been established in animal breeding in plant breeding for a long time now. If you want a chicken that grows to be *this* big, instead of *this* big, you can do it. You can do it if you want a cow that produces 10 times or 100 times more milk than a regular cow. The egg you ate for breakfast this morning, those bio-engineered chickens that lay almost an egg a day… A chicken in the wild lays an egg a month. How the hell did we do that? By genetic engineering. That's how we did it. Dwarkesh Patel  42:51  Yeah. That was through brute artificial selection. No fancy machine learning there.Steve Hsu  42:58  Last ten years, it's gotten sophisticated machine learning genotyping of chickens. Artificial insemination, modeling of the traits using ML last ten years. For cow breeding, it's done by ML. First Mover AdvantageDwarkesh Patel  43:18  I had no idea. That's super interesting. So, you mentioned that you're accumulating data and improving your techniques over time, is there a first mover advantage to a genomic prediction company like this? Or is it whoever has the newest best algorithm for going through the biobank data? Steve Hsu  44:16  That's another super question. For the entrepreneurs in your audience, I would say in the short run, if you ask what the valuation of GPB should be? That's how the venture guys would want me to answer the question. There is a huge first mover advantage because they're important in the channel relationships between us and the clinics. Nobody will be able to get in there very easily when they come later because we're developing trust and an extensive track record with clinics worldwide—and we're well-known. So could 23andme or some company with a huge amount of data—if they were to get better AI/ML people working on this—blow us away a little bit and build better predictors because they have much more data than we do? Possibly, yes. Now, we have had core expertise in doing this work for years that we're just good at it. Even though we don't have as much data as 23andme, our predictors might still be better than theirs. I'm out there all the time, working with biobanks all around the world. I don't want to say all the names, but other countries are trying to get my hands on as much data as possible.But, there may not be a lasting advantage beyond the actual business channel connections to that particular market. It may not be a defensible, purely scientific moat around the company. We have patents on specific technologies about how to do the genotyping or error correction on the embryo, DNA, and stuff like this. We do have patents on stuff like that. But this general idea of who will best predict human traits from DNA? It's unclear who's going to be the winner in that race. Maybe it'll be the Chinese government in 50 years? Who knows?Dwarkesh Patel  46:13  Yeah, that's interesting. If you think about a company Google, theoretically, it's possible that you could come up with a better algorithm than PageRank and beat them. But it seems like the engineer at Google is going to come up with whatever edge case or whatever improvement is possible.Steve Hsu  46:28  That's exactly what I would say. PageRank is deprecated by now. But, even if somebody else comes up with a somewhat better algorithm if they have a little bit more data, if you have a team doing this for a long time and you're focused and good, it's still tough to beat you, especially if you have a lead in the market.Dwarkesh Patel  46:50  So, are you guys doing the actual biopsy? Or is it just that they upload the genome, and you're the one processing just giving recommendations? Is it an API call, basically?Steve Hsu  47:03  It's great, I love your question. It is totally standard. Every good IVF clinic in the world regularly takes embryo biopsies. So that's standard. There’s a lab tech doing that. Okay. Then, they take the little sample, put it on ice, and ship it. The DNA as a molecule is exceptionally robust and stable. My other startup solves crimes that are 100 years old from DNA that we get from some semen stain on some rape victim, serial killer victims bra strap, we've done stuff that.Dwarkesh Patel  47:41  Jack the Ripper, when are we going to solve that mystery?Steve Hsu  47:44  If they can give me samples, we can get into that. For example, we just learned that you could recover DNA pretty well if someone licks a stamp and puts on their correspondence. If you can do Neanderthals, you can do a lot to solve crimes. In the IVF workflow, our lab, which is in New Jersey, can service every clinic in the world because they take the biopsy, put it in a standard shipping container, and send it to us. We’re actually genotyping DNA in our lab, but we've trained a few of the bigger  clinics to do the genotyping on their site. At that point, they upload some data into the cloud and then they get back some stuff from our platform. And at that point it's going to be the whole world, every human who wants their kid to be healthy and get the best they can– that data is going to come up to us, and the report is going to come back down to their IVF physician. Dwarkesh Patel  48:46  Which is great if you think that there's a potential that this technology might get regulated in some way, you could go to Mexico or something, have them upload the genome (you don't care what they upload it from), and then get the recommendations there. Steve Hsu  49:05  I think we’re going to evolve to a point where we are going to be out of the wet part of this business, and only in the cloud and bit part of this business. No matter where it is, the clinics are going to have a sequencer, which is *this* big, and their tech is going to quickly upload and retrieve the report for the physician three seconds later. Then, the parents are going to look at it on their phones or whatever. We’re basically there with some clinics. It’s going to be tough to regulate because it’s just this. You have the bits and you’re in some repressive, terrible country that doesn’t allow you to select for some special traits that people are nervous about, but you can upload it to some vendor that’s in Singapore or some free country, and they give you the report back. Doesn’t have to be us, we don’t do the edgy stuff. We only do the health-related stuff right now. But, if you want to know how tall this embryo is going to be…I’ll tell you a mind-blower! When you do face recognition in AI, you're mapping someone's face into a parameter space on the order of hundreds of parameters, each of those parameters is super heritable. In other words, if I take two twins and photograph them, and the algorithm gives me the value of that parameter for twin one and two, they're very close. That's why I can't tell the two twins apart, and face recognition can ultimately tell them apart if it’s really good system. But you can conclude that almost all these parameters are identical for those twins. So it's highly heritable. We're going to get to a point soon where I can do the inverse problem where I have your DNA  and I predict each of those parameters in the face recognition algorithm and then reconstruct the face. If I say that when this embryo will be 16, that is what she will look like. When she's 32, this is what she's going to look like. I'll be able to do that, for sure. It's only an AI/ML problem right now. But basic biology is clearly going to work. So then you're going to be able to say, “Here's a report. Embryo four is so cute.” Before, we didn't know we wouldn't do that, but it will be possible. Dwarkesh Patel  51:37  Before we get married, you'll want to see what their genotype implies about their faces' longevity. It's interesting that you hear stories about these cartel leaders who will get plastic surgery or something to evade the law, you could have a check where you look at a lab and see if it matches the face you would have had five years ago when they caught you on tape.Steve Hsu  52:02  This is a little bit back to old-school Gattaca, but you don't even need the face! You can just take a few molecules of skin cells and phenotype them and know exactly who they are. I've had conversations with these spooky Intel folks. They're very interested in, “Oh, if some Russian diplomat comes in, and we think he's a spy, but he's with the embassy, and he has a coffee with me, and I save the cup and send it to my buddy at Langley, can we figure out who this guy is? And that he has a daughter who's going to Chote? Can do all that now.Dwarkesh Patel  52:49  If that's true, then in the future, world leaders will not want to eat anything or drink. They'll be wearing a hazmat suit to make sure they don't lose a hair follicle.Steve Hsu  53:04  The next time Pelosi goes, she will be in a spacesuit if she cares. Or the other thing is, they're going to give it. They're just going to be, “Yeah, my DNA is everywhere. If I'm a public figure, I can't track my DNA. It's all over.”Dwarkesh Patel  53:17  But the thing is, there's so much speculation that Putin might have cancer or something. If we have his DNA, we can see his probability of having cancer at age 70, or whatever he is, is 85%. So yeah, that’d be a very verified rumor. That would be interesting. Steve Hsu  53:33  I don't think that would be very definitive. I don't think we'll reach that point where you can say that Putin has cancer because of his DNA—which I could have known when he was an embryo. I don't think it's going to reach that level. But, we could say he is at high risk for a type of cancer. Genomics in datingDwarkesh Patel  53:49  In 50 or 100 years, if the majority of the population is doing this, and if the highly heritable diseases get pruned out of the population, does that mean we'll only be left with lifestyle diseases? So, you won't get breast cancer anymore, but you will still get fat or lung cancer from smoking?Steve Hsu  54:18  It's hard to discuss the asymptotic limit of what will happen here. I'm not very confident about making predictions like that. It could get to the point where everybody who's rich or has been through this stuff for a while, (especially if we get the editing working) is super low risk for all the top 20 killer diseases that have the most life expectancy impact. Maybe those people live to be 300 years old naturally. I don't think that's excluded at all. So, that's within the realm of possibility. But it's going to happen for a few lucky people like Elon Musk before it happens for shlubs like you and me. There are going to be very angry inequality protesters about the Trump grandchildren, who, models predict will live to be 200 years old. People are not going to be happy about that.Dwarkesh Patel  55:23  So interesting. So, one way to think about these different embryos is if you're producing multiple embryos, and you get to select from one of them, each of them has a call option, right? Therefore, you probably want to optimize for volatility as much, or if not more than just the expected value of the trait. So, I'm wondering if there are mechanisms where you can  increase the volatility in meiosis or some other process. You just got a higher variance, and you can select from the tail better.Steve Hsu  55:55  Well, I'll tell you something related, which is quite amusing. So I talked with some pretty senior people at the company that owns all the dating apps. So you can look up what company this is, but they own Tinder and Match. They’re kind of interested in perhaps including a special feature where you upload your genome instead of Tinder Gold / Premium.  And when you match- you can talk about how well you match the other person based on your genome. One person told me something shocking. Guys lie about their height on these apps. Dwarkesh Patel  56:41  I’m shocked, truly shocked hahaha. Steve Hsu  56:45  Suppose you could have a DNA-verified height. It would prevent gross distortions if someone claims they're 6’2 and they’re 5’9. The DNA could say that's unlikely. But no, the application to what you were discussing is more like, “Let's suppose that we're selecting on intelligence or something. Let's suppose that the regions where your girlfriend has all the plus stuff are complementary to the regions where you have your plus stuff. So, we could model that and say,  because of the complementarity structure of your genome in the regions that affect intelligence, you're very likely to have some super intelligent kids way above your, the mean of your you and your girlfriend's values. So, you could say things like it being better for you to marry that girl than another. As long as you go through embryo selection, we can throw out the bad outliers. That's all that's technically feasible. It's true that one of the earliest patent applications, they'll deny it now. What's her name? Gosh, the CEO of 23andme…Wojcicki, yeah. She'll deny it now. But, if you look in the patent database, one of the very earliest patents that 23andme filed when they were still a tiny startup was about precisely this: Advising parents about mating and how their kids would turn out and stuff like this. We don't even go that far in GP, we don't even talk about stuff like that, but they were thinking about it when they founded 23andme.Dwarkesh Patel  58:38  That is unbelievably interesting. By the way, this just occurred to me—it's supposed to be highly heritable, especially people in Asian countries, who have the experience of having grandparents that are much shorter than us, and then parents that are shorter than us, which suggests that  the environment has a big part to play in it malnutrition or something. So how do you square that our parents are often shorter than us with the idea that height is supposed to be super heritable.Steve Hsu  59:09  Another great observation. So the correct scientific statement is that we can predict height for people who will be born and raised in a favorable environment. In other words, if you live close to a McDonald's and you're able to afford all the food you want, then the height phenotype becomes super heritable because the environmental variation doesn't matter very much. But, you and I both know that people are much smaller if we return to where our ancestors came from, and also, if you look at how much food, calories, protein, and calcium they eat, it's different from what I ate and what you ate growing up. So we're never saying the environmental effects are zero. We're saying that for people raised in a particularly favorable environment, maybe the genes are capped on what can be achieved, and we can predict that. In fact, we have data from Asia, where you can see much bigger environmental effects. Age affects older people, for fixed polygenic scores on the trait are much shorter than younger people.Ancestral populationsDwarkesh Patel  1:00:31  Oh, okay. Interesting. That raises that next question I was about to ask: how applicable are these scores across different ancestral populations?Steve Hsu  1:00:44  Huge problem is that most of the data is from Europeans. What happens is that if you train a predictor in this ancestry group and go to a more distant ancestry group, there's a fall-off in the prediction quality. Again, this is a frontier question, so we don't know the answer for sure. But many people believe that there's a particular correlational structure in each population, where if I know the state of this SNP, I can predict the state of these neighboring SNPs. That is a product of that group's mating patterns and ancestry. Sometimes, the predictor, which is just using statistical power to figure things out, will grab one of these SNPs as a tag for the truly causal SNP in there. It doesn't know which one is genuinely causal, it is just grabbing a tag, but the tagging quality falls off if you go to another population (eg. This was a very good tag for the truly causal SNP in the British population. But it's not so good a tag in the South Asian population for the truly causal SNP, which we hypothesize is the same). It's the same underlying genetic architecture in these different ancestry groups. We don't know if that's a hypothesis. But even so, the tagging quality falls off. So my group spent a lot of our time looking at the performance of predictor training population A, and on distant population B, and modeling it trying to figure out trying to test hypotheses as to whether it's just the tagging decay that’s responsible for most of the faults. So all of this is an area of active investigation. It'll probably be solved in five years. The first big biobanks that are non-European are coming online. We're going to solve it in a number of years.Dwarkesh Patel  1:02:38  Oh, what does the solution look like?  Unless you can identify the causal mechanism by which each SNP is having an effect, how can you know that something is a tag or whether it's the actual underlying switch?Steve Hsu  1:02:54  The nature of reality will determine how this is going to go. So we don't truly  know if the  innate underlying biology is true. This is an amazing thing. People argue about human biodiversity and all this stuff, and we don't even know whether these specific mechanisms that predispose you to be tall or having heart disease are the same  in these different ancestry groups. We assume that it is, but we don't know that. As we get further away to Neanderthals or Homo Erectus, you might see that they have a slightly different architecture than we do. But let's assume that the causal structure is the same for South Asians and British people. Then it's a matter of improving the tags. How do I know if I don't know which one is causal? What do I mean by improving the tags? This is a machine learning problem. If there's a SNP, which is always coming up as very significant when I use it across multiple ancestry groups, maybe that one's casual. As I vary the tagging correlations in the neighborhood of that SNP, I always find that that one is the intersection of all these different sets, making me think that one's going to be causal. That's a process we're engaged in now—trying to do that. Again, it's just a machine learning problem. But we need data. That's the main issue.Dwarkesh Patel  1:04:32  I was hoping that wouldn't be possible, because one way we might go about this research is that it itself becomes taboo or causes other sorts of bad social consequences if you can definitively show that on certain traits, there are differences between ancestral populations, right? So, I was hoping that maybe there was an evasion button where we can't say because they're just tags and the tags might be different between different ancestral populations. But with machine learning, we’ll know.Steve Hsu  1:04:59  That's the situation we're in now, where you have to do some fancy analysis if you want to claim that Italians have lower height potential than Nordics—which is possible. There's been a ton of research about this because there are signals of selection. The alleles, which are activated in height predictors, look like they've been under some selection between North and South Europe over the last 5000 years for whatever reason. But, this is a thing debated by people who study molecular evolution. But suppose it's true, okay? That would mean that when we finally get to the bottom of it, we find all the causal loci for height, and the average value for the Italians is lower than that for those living in Stockholm. That might be true. People don't get that excited? They get a little bit excited about height. But they would get really excited if this were true for some other traits, right?Suppose the causal variants affecting your level of extraversion are systematic, that the average value of those weighed the weighted average of those states is different in Japan versus Sicily. People might freak out over that. I'm supposed to say that's obviously not true. How could it possibly be true? There hasn't been enough evolutionary time for those differences to arise. After all, it's not possible that despite what looks to be the case for height over the last 5000 years in Europe, no other traits could have been differentially selected for over the last 5000 years. That's the dangerous thing. Few people understand this field well enough to understand what you and I just discussed and are so alarmed by it that they're just trying to suppress everything. Most of them don't follow it at this technical level that you and I are just discussing. So, they're somewhat instinctively negative about it, but they don't understand it very well.Dwarkesh Patel  1:07:19  That's good to hear. You see this pattern that by the time that somebody might want to regulate or in some way interfere with some technology or some information, it already has achieved wide adoption. You could argue that that's the case with crypto today. But if it's true that a bunch of IVF clinics worldwide are using these scores to do selection and other things, by the time people realize the implications of this data for other kinds of social questions, this has already been an existing consumer technology.Is this eugenics?Steve Hsu  1:07:58  That's true, and the main outcry will be if it turns out that there are massive gains to be had, and only the billionaires are getting them. But that might have the consequence of causing countries to make this free part of their national health care system. So Denmark and Israel pay for IVF. For infertile couples, it's part of their national health care system. They're pretty aggressive about genetic testing. In Denmark, one in 10 babies are born through IVF. It's not clear how it will go. But we're in for some fun times. There's no doubt about that.Dwarkesh Patel  1:08:45  Well, one way you could go is that some countries decided to ban it altogether. And another way it could go is if countries decided to give everybody free access to it. If you had to choose between the two,  you would want to go for the second one. Which would be the hope. Maybe only those two are compatible with people's moral intuitions about this stuff. Steve Hsu  1:09:10  It’s very funny because most wokist people today hate this stuff. But, most progressives like Margaret Sanger, or anybody who was the progressive intellectual forebears of today's wokist, in the early 20th century, were all that we would call today in Genesis because they were like, “Thanks to Darwin, we now know how this all works. We should take steps to keep society healthy and (not in a negative way where we kill people we don't like, but we should help society do healthy things when they reproduce, and have healthy kids).” Now, this whole thing has just been flipped over among progressives. Dwarkesh Patel  1:09:52  Even in India, less than 50 years ago, Indira Gandhi, she's on the left side of India's political spectrum. She was infamous for putting on these forced sterilization programs. Somebody made an interesting comment about this where they were asked, “Oh, is it true that history always tilts towards progressives? And if so, isn't everybody else doomed? Aren't their views doomed?”The person made a fascinating point: whatever we consider left at the time tends to be winning. But what is left has changed a lot over time, right? In the early 20th century, prohibition was a left cause. It was a progressive cause, and that changed, and now the opposite is the left cause. But now, legalizing pot is progressive. Exactly. So, if Conquest’s second law is true, and everything tilts leftover time, just change what is left is, right? That's the solution. Steve Hsu  1:10:59  No one can demand that any of these woke guys be intellectually self-consistent, or even say the same things from one year to another? But one could wonder what they think about these literally Communist Chinese. They’re recycling huge parts of their GDP to help the poor and the southern stuff. Medicine is free, education is free, right? They're clearly socialists, and literally communists. But in Chinese, the Chinese characters for eugenics is a positive thing. It means healthy production. But more or less, the whole viewpoint on all this stuff is 180 degrees off in East Asia compared to here, and even among the literal communists—so go figure.Dwarkesh Patel  1:11:55  Yeah, very based. So let's talk about one of the traits that people might be interested in potentially selecting for: intelligence. What is the potential for us to acquire the data to correlate the genotype with intelligence?Steve Hsu  1:12:15  Well, that's the most personally frustrating aspect of all of this stuff. If you asked me ten years ago when I started doing this stuff what were we going to get, everything was gone. On the optimistic side of what I would have predicted, so everything's good. Didn't turn out to be interactively nonlinear, or it didn't turn out to be interactively pleiotropic. All these good things, —which nobody could have known a priori how they would work—turned out to be good for gene engineers of the 21st century. The one frustrating thing is because of crazy wokeism, and fear of crazy wokists, the most interesting phenotype of all is lagging b

In this episode Marie shares information on how Google's Hummingbird algorithm dramatically changed search to rely less on PageRank as their algorithms learned how to make use of entities. Understanding Google's use of entities in their algorithms can help us improve E-A-T. More importantly, understanding how Google uses AI to extract entities from content can help us write and optimize content. Marie also discusses the third release of the product reviews update, sharing which types of sites saw changes and how Google could be algorithmically evaluating product review sites. Links mentioned in this episode: Product reviews update https://blog.google/products/search/more-helpful-product-reviews/ Interview with Jason Barnard re entity use and John Lennon's confidence score in the knowledge graph   https://fajela.com/seobox/orm/ https://appft1.uspto.gov/netacgi/nph-Parser?Sect1=PTO2&Sect2=HITOFF&u=%2Fnetahtml%2FPTO%2Fsearch-adv.html&r=1&f=G&l=50&d=PG01&p=1&S1=20050055341&OS=20050055341&RS=20050055341 FAQ:All About the New Hummingbird Algorithm:  https://searchengineland.com/google-hummingbird-172816 Google's Hummingbird Takes Flight: SEOs Give Insight On Google's New Algorithm: https://searchengineland.com/hummingbird-has-the-industry-flapping-its-wings-in-excitement-reactions-from-seo-experts-on-googles-new-algorithm-173030 This episode corresponds with newsletter episode 228:
 https://mariehaynes.com/newsletter/episode-228-light-version/ Past episodes https://Mariehaynes.com/seo-newsletter/seo-podcast Contact MHC https://Mariehaynes.com/contact Book on Unnatural Links and Manual Action Removal https://Mariehaynes.com/product/unnatural-links-book Quality Raters Book https://Mariehaynes.com/product/quality-raters-guidelines Submit a question for the next Q&A https://Mariehaynes.com/qa-with-mhc Subscribe to the newsletter https://Mariehaynes.com/newsletter

Seldom do we think about self-reference, but it is a huge part of the world we live in. Every time that we say 'myself', for instance, we are engaging in self-reference. Long ago, the Liar Paradox and the Golden Ratio were among the first formal examples of self-reference. Freedom to refer to the self has given us fruitful results in mathematics and technology. Recursion, for example, is used in algorithms such as PageRank, which is one of the primary algorithms in Google's search engine. Elements of self-reference can also be found in foundational shifts in the way we understand mathematics, and has propelled our understanding of mathematics forward. Forming modern set theory was only possible due to a paradox called Russel's paradox, for example. Even humor uses self-reference. Realizing this, can we find harmony in self-reference? Even in a podcast intro, are there elements of self-reference? Nobody knows, but I'd check if I were you. Catch all of this, and more, on this episode of Breaking Math. Episode 70.1: Episode Seventy Point One of Breaking Math Podcast [Featuring: Sofía Baca, Gabriel Hesch; Millicent Oriana] --- This episode is sponsored by · Anchor: The easiest way to make a podcast. https://anchor.fm/app Support this podcast: https://anchor.fm/breakingmathpodcast/support

3 Ways To Optimize Internal LinkingPoor internal link structure can end up harming your site's rankings. At SMX Next, Jonathan Epstein showed how marketers can prevent PageRank loss with better internal linking.Presenter: Jonathan Epstein Source: https://searchengineland.com/3-ways-to-optimize-internal-linking-381289-Having 1st aired in 2009, with over 3.6 million downloads in 100+ counties, “SEO Podcast, Unknown Secrets of Internet Marketing” has become one of the longest-running and most authoritative podcasts for staying ahead of the perpetually changing digital marketing landscape.Great for internal marketers, business owners, and agencies from novice to experienced in using the internet to market and grow a brand!You can also watch his episode here: https://bestseopodcast.com/Follow us on:https://www.facebook.com/EWRDigitalhttps://www.instagram.com/thebestseopodcast/https://www.tiktok.com/@bestseopodcasthttps://www.linkedin.com/company/bestseopodcastPowered by: ewrdigital.comHost's: Matt Bertram & Chris Burres Disclaimer: For Educational and Entertainment  purposes Only.

Poor internal link structure can end up harming your site's rankings. At SMX Next, Jonathan Epstein showed how marketers can prevent PageRank loss with better internal linking.Presenter: Jonathan Epstein Source: https://searchengineland.com/3-ways-to-optimize-internal-linking-381289---Presented by BestSEOPodcast.com (The Unknown Secrets of Internet Marketing Podcast)Having 1st aired in 2009, with over 3.6 million downloads in 100+ counties, “SEO Podcast, Unknown Secrets of Internet Marketing” has become one of the longest-running and most authoritative podcasts for staying ahead of the perpetually changing digital marketing landscape.Great for marketers, business owners, and agencies from novice to experienced in using the internet to market and grow a brand!You can also watch his episode here: https://youtu.be/XXqRDfAmwdI