Podcasts about AltaVista

El misterioso caso de un delito que nunca fue cometido y un pueblo en alerta ECDQEMSD podcast episodio 6035 Secuestro Involuntario Conducen: El Pirata y El Sr. Lagartija https://canaltrans.com Noticias del Mundo: Marchas en el mundo por el Día del Trabajo - Reapareció Kamala Harris - La violencia en Haití - El león de Culiacán - No tenemos tobilleras - Fortuna negada - Cuestión de métodos - Pronóstico del Tiempo Historias Desintegradas: Una tarde de calor - Jugando en el patio - Viendo una película - Rumores al anochecer - Composta y lombrices - El México prehistórico - Taxis del DF - Chica fresa - Olor sintético - Geocities, Encarta, Altavista y muchos más - El liso santafesino - Clásico entre sabaleros y tatengues - Hay historias de éxito - El buen Atún - No al acoso escolar - Día mundial del asma y más... En Caso De Que El Mundo Se Desintegre - Podcast no tiene publicidad, sponsors ni organizaciones que aporten para mantenerlo al aire. Solo el sistema cooperativo de los que aportan a través de las suscripciones hacen posible que todo esto siga siendo una realidad. Gracias Dragones Dorados!! NO AI: ECDQEMSD Podcast no utiliza ninguna inteligencia artificial de manera directa para su realización. Diseño, guionado, música, edición y voces son de  nuestra completa intervención humana.

Along with the rest of the morning's headlines, we'll have a little fun remembering the early days of the internet—when the information superhighway was more like a gravel road by comparison. At the time, Alta Vista, Ask Jeeves, and CompuServe were superstars for early users. How many others do you remember? Mornings with Pat Kreitlow airs on several stations across the Civic Media radio network, Monday through Friday from 6-9 am. Subscribe to the podcast to be sure not to miss out on a single episode! To learn more about the show and all of the programming across the Civic Media network, head over to https://civicmedia.us/shows to see the entire broadcast line up.

Kenmore is Home Electricity Made Easy - Modernize the Smart Home from Appliances to the Electric Grid – Revisited w/ CEO Sri Solur   - AZ TRT S06 EP04 (265) 2-23-2025          What We Learned This Week ·         Kenmore is home electricity made easy.  Kenmore is on a mission to modernize the home. Live More & Live Better. Also need to make it Affordable. ·         Clean Tech goes w/ the smart home, smart appliances (that connect to the home) and the electrical power grid for better living Electrical Grid needs to be modernized – cannot handle the current & future power demands ·         Homes built Pre-1990 run on Electric Panels that are outdated – costs of $40K + to modernize to handle charging EVs at home ·         Design of the Future House would have a Battery in it that could recharge your appliances and electronics during down hours. ·         Solving problems in electricity and energy also have the same issues with working on better water and clean food. It is more than just an energy and electric issue.     Guest: Sri Solur, CEO, Kenmore / Brands  https://www.linkedin.com/in/solur https://www.kenmore.com/   Sri Solur is chief executive officer of brands for Kenmore at Transformco. An industry veteran with 25+ years of experience, Sri has a rich history of success leading high tech products and businesses. He previously served as CPO and GM at Berkshire Grey, a leader in industrial robotics, and was a member of the leadership team that took the company public. Sri also served as CPO at SharkNinja, and was instrumental in bringing the Shark IQ Robot vacuum and NinjaFoodi products to market, while also holding a leadership role to take the company public. Sri spent 20 years at Hewlett Packard, serving as founder and CPO of CloudPrint, the company's wearables and IOT business. In his career, Sri has created products for world-renowned brands including Hugo Boss, Movado, Ferrari, Juicy Couture, and more. Sri holds a bachelor's degree in Engineering from NIT and an MBA from Boston University.         As Earth Day approaches (April), Kenmore is empowering greener homes and people.    The trusted appliance maker recently unveiled a new “Home Electrification Made Easy” program that looks to simplify the electrification process and reduce overall costs in transitioning to electric appliances.    Kenmore has set an ambitious goal with the program to electrify one million homes that will ultimately save homeowners one billion dollars over the next decade.    Kenmore's innovation and energy programs are driving a new generation of electrification for today's home ecosystem. Some of the company's core innovations include:    Expansion of electrification and smart products for every room in the home.  Addition of electrification enablers, such as smart electrical panels and dynamic Level 2 EV chargers, that help eliminate roadblocks many homeowners have in wanting to electrify their entire home.  Simplifying rebate and savings programs, such as Congress' Inflation Reduction Act, to help customers cut costs by taking advantage of available local and national funding and discounts.  Building relationships with industry leaders in product, service and consumer education to supplement and amplify their mission to electrify American homes.    This electric push comes as a new generation of homeowners seek to invest in smarter, greener home solutions and previous generations are coming up against new government standards making accessibility to like-for-like replacement equipment for their home obsolete.    With Kenmore's electrification program delivering a quick onramp to affordable green energy homes, homeowners of all backgrounds and budgets have a more attainable path to smart, green home adoption.          Notes:   Kenmore CEO and Appliances   Seg. 1   Major appliances and clean tech and sustainability energy security is a big issue on the macro end. The effect on the electric grid and power lines.   There is lots of demand and potential blackouts. This is a fuel and demand issue. The government and utility companies are working on clean energy. Currently they use fossil fuels and working on using less.   Design of the future house would have a battery in it that could recharge your appliances and electronics storing down ours.   The electric layout of most homes, especially homes built pre-1990s has an 100 amp circuit. If you have modern tech like an EV charger in your house, an electrician cannot set it up because the EV charger will blow up your 100 amp circuit.   It would cost you between $20 and 60 K to upgrade a house for a modern electric set up. Kenmore will install electric panel with load balance for EV vehicles and in-home appliances.   Seg. 2   Electrical layout of a house as you install new appliances. There is a booster within the inflation reduction act. There are rebates for lower income people, where it pays you for getting new appliances. 10 K instant credit for new appliances.   The comparison of older appliances versus new appliances. Many older appliances may run on fossil fuels like a gas range oven or gas water heater. Older HVAC unit has more wear and tear.   On a hot days and really cold days appliances operate at peak and are putting demand on the electric red. Looking for new ways of sustainable clean energy and examples hydroelectric power.   You would have a back up in high demand times, where are you fire up a generator running on fossil fuels.   Do you want to protect the grid for maintenance but also things like cyber attacks. One way you could do this is make all homes standalone energy producers.   Peak rates for electricity or 6 to 10 PM at night. At these times electricity use taxes the grid and also taxes your wallet. Do you want to run your dishwasher post 10 PM.   Seg. 3   We are moving from a world of done by you to a world of done for you. The smart home of the future will help you.   The electrical panel would work with the grid and decide when to charge electronics in your house. Kenmore has electric appliances that works with the electric red. These appliances save you money and also save the grid.   On a bigger scale we need to modernize the electric road. Then in the future build better homes cars and appliances. Inflation reduction act has multiprong incentives for all of this.   When we saw the bull run of tech starting in 2010 it had three things working together. Social mobile and the cloud all came together to create this tech rise. Do you need electricity plus clean energy plus clean water.   A rising tide that can raise all. Do you want to solve problems, what are the pain killers?   Seg. 4   CEO was an engineer by trade. Worked in Boston went to business school and after that he built some products. Worked at Altavista on firewalls and search.   Cloud print on printing mobile with the HP e-print. Worked in wearables at Hugo boss and Ferrari.   Worked at Comcast on Xfinity digital security and high-speed Internet. Worked with shark and ninja on home robots. Worked at Bershire Gray, consumer robots which went public with an IPO.   Then at Brands / Kenmore (also Diehard batteries) - Building better and smarter appliances   Span that I/O build a smart electrical panel. Do you want your appliances to give you repair and maintenance updates.   Whole home electrification. A whole home dashboard controlling your smart home. An example would be your fridge would tell you when you need a new filter. Kenmore is a tech forward company.   Solving problems in electricity and energy also have the same issues with working on better water and clean food. It is more than just an energy and electric issue.   Live more and live better. Also need to make it affordable. Kenmore is home electricity made easy. rebates.kenmore.com they have the blue-collar work ethic with the idea of progress over perfection. Kenmore is a consumer centric team.         Biotech Shows: https://brt-show.libsyn.com/category/Biotech-Life+Sciences-Science   AZ Tech Council Shows:  https://brt-show.libsyn.com/size/5/?search=az+tech+council *Includes Best of AZ Tech Council show from 2/12/2023   Tech Topic: https://brt-show.libsyn.com/category/Tech-Startup-VC-Cybersecurity-Energy-Science  Best of Tech: https://brt-show.libsyn.com/size/5/?search=best+of+tech   ‘Best Of' Topic: https://brt-show.libsyn.com/category/Best+of+BRT      Thanks for Listening. Please Subscribe to the AZ TRT Podcast.     AZ Tech Roundtable 2.0 with Matt Battaglia The show where Entrepreneurs, Top Executives, Founders, and Investors come to share insights about the future of business.  AZ TRT 2.0 looks at the new trends in business, & how classic industries are evolving.  Common Topics Discussed: Startups, Founders, Funds & Venture Capital, Business, Entrepreneurship, Biotech, Blockchain / Crypto, Executive Comp, Investing, Stocks, Real Estate + Alternative Investments, and more…    AZ TRT Podcast Home Page: http://aztrtshow.com/ ‘Best Of' AZ TRT Podcast: Click Here Podcast on Google: Click Here Podcast on Spotify: Click Here                    More Info: https://www.economicknight.com/azpodcast/ KFNX Info: https://1100kfnx.com/weekend-featured-shows/     Disclaimer: The views and opinions expressed in this program are those of the Hosts, Guests and Speakers, and do not necessarily reflect the views or positions of any entities they represent (or affiliates, members, managers, employees or partners), or any Station, Podcast Platform, Website or Social Media that this show may air on. All information provided is for educational and entertainment purposes. Nothing said on this program should be considered advice or recommendations in: business, legal, real estate, crypto, tax accounting, investment, etc. Always seek the advice of a professional in all business ventures, including but not limited to: investments, tax, loans, legal, accounting, real estate, crypto, contracts, sales, marketing, other business arrangements, etc.  

Idag är första gången vi välkomnar in någon från ett börsbolag. Anledningen till att det dröjt såhär länge är för att tyvärr tenderar ju poddar med folk i börsbolag att bli rätt slätstrukna p.g.a regelverk samt oro för vad man bör och inte bör säga. Men Ulrika Klinkert är en frisk fläkt i börs-Sverige och hon bjuder på mängder av insikter, anekdoter och skratt. Rugvista säljer mattor online till hela Europa (samt några länder till) och det är ett inspirerande avsnitt om ett bolag som sålt online sedan Altavista var en grej...

Join Simtheory: https://simtheory.ai----Grok 3 Dis Track (cringe): https://simulationtheory.ai/aff9ba04-ca0e-4572-84f4-687739c7b84bGrok 3 Dis Track written by Sonnet: https://simulationtheory.ai/edaed525-b9b6-473b-a6d6-f9cca9673868----Community: https://thisdayinai.com----Chapters:00:00 - First Impressions of Grok 310:00 - Discussion about Deep Search, Deep Research24:28 - Market landscape: Is OpenAI Rattled by xAI's Grok 3? Rumors of GPT-4.5 and GPT-548:48 - Why does Grok and xAI Exist? Will anyone care about Grok 3 next week?54:45 - Diss track battle with Grok 3 (re-written by Sonnet) & Model Tuning for Use Cases1:07:50 - GPT-4.5 and Anthropic Claude Thinking Next Week? & Are we a podcast about Altavista?1:13:25 - Economically productive agents & freaky muscular robot1:22:00 - Final thoughts of the week1:27:26 - Grok 3 Dis Track in Full (Sonnet Version)Thanks for your support and listening!

Applications close Monday for the NYC AI Engineer Summit focusing on AI Leadership and Agent Engineering! If you applied, invites should be rolling out shortly.The search landscape is experiencing a fundamental shift. Google built a >$2T company with the “10 blue links” experience, driven by PageRank as the core innovation for ranking. This was a big improvement from the previous directory-based experiences of AltaVista and Yahoo. Almost 4 decades later, Google is now stuck in this links-based experience, especially from a business model perspective. This legacy architecture creates fundamental constraints:* Must return results in ~400 milliseconds* Required to maintain comprehensive web coverage* Tied to keyword-based matching algorithms* Cost structures optimized for traditional indexingAs we move from the era of links to the era of answers, the way search works is changing. You're not showing a user links, but the goal is to provide context to an LLM. This means moving from keyword based search to more semantic understanding of the content:The link prediction objective can be seen as like a neural PageRank because what you're doing is you're predicting the links people share... but it's more powerful than PageRank. It's strictly more powerful because people might refer to that Paul Graham fundraising essay in like a thousand different ways. And so our model learns all the different ways.All of this is now powered by a $5M cluster with 144 H200s:This architectural choice enables entirely new search capabilities:* Comprehensive result sets instead of approximations* Deep semantic understanding of queries* Ability to process complex, natural language requestsAs search becomes more complex, time to results becomes a variable:People think of searches as like, oh, it takes 500 milliseconds because we've been conditioned... But what if searches can take like a minute or 10 minutes or a whole day, what can you then do?Unlike traditional search engines' fixed-cost indexing, Exa employs a hybrid approach:* Front-loaded compute for indexing and embeddings* Variable inference costs based on query complexity* Mix of owned infrastructure ($5M H200 cluster) and cloud resourcesExa sees a lot of competition from products like Perplexity and ChatGPT Search which layer AI on top of traditional search backends, but Exa is betting that true innovation requires rethinking search from the ground up. For example, the recently launched Websets, a way to turn searches into structured output in grid format, allowing you to create lists and databases out of web pages. The company raised a $17M Series A to build towards this mission, so keep an eye out for them in 2025. Chapters* 00:00:00 Introductions* 00:01:12 ExaAI's initial pitch and concept* 00:02:33 Will's background at SpaceX and Zoox* 00:03:45 Evolution of ExaAI (formerly Metaphor Systems)* 00:05:38 Exa's link prediction technology* 00:09:20 Meaning of the name "Exa"* 00:10:36 ExaAI's new product launch and capabilities* 00:13:33 Compute budgets and variable compute products* 00:14:43 Websets as a B2B offering* 00:19:28 How do you build a search engine?* 00:22:43 What is Neural PageRank?* 00:27:58 Exa use cases * 00:35:00 Auto-prompting* 00:38:42 Building agentic search* 00:44:19 Is o1 on the path to AGI?* 00:49:59 Company culture and nap pods* 00:54:52 Economics of AI search and the future of search technologyFull YouTube TranscriptPlease like and subscribe!Show Notes* ExaAI* Web Search Product* Websets* Series A Announcement* Exa Nap Pods* Perplexity AI* Character.AITranscriptAlessio [00:00:00]: Hey, everyone. Welcome to the Latent Space podcast. This is Alessio, partner and CTO at Decibel Partners, and I'm joined by my co-host Swyx, founder of Smol.ai.Swyx [00:00:10]: Hey, and today we're in the studio with my good friend and former landlord, Will Bryk. Roommate. How you doing? Will, you're now CEO co-founder of ExaAI, used to be Metaphor Systems. What's your background, your story?Will [00:00:30]: Yeah, sure. So, yeah, I'm CEO of Exa. I've been doing it for three years. I guess I've always been interested in search, whether I knew it or not. Like, since I was a kid, I've always been interested in, like, high-quality information. And, like, you know, even in high school, wanted to improve the way we get information from news. And then in college, built a mini search engine. And then with Exa, like, you know, it's kind of like fulfilling the dream of actually being able to solve all the information needs I wanted as a kid. Yeah, I guess. I would say my entire life has kind of been rotating around this problem, which is pretty cool. Yeah.Swyx [00:00:50]: What'd you enter YC with?Will [00:00:53]: We entered YC with, uh, we are better than Google. Like, Google 2.0.Swyx [00:01:12]: What makes you say that? Like, that's so audacious to come out of the box with.Will [00:01:16]: Yeah, okay, so you have to remember the time. This was summer 2021. And, uh, GPT-3 had come out. Like, here was this magical thing that you could talk to, you could enter a whole paragraph, and it understands what you mean, understands the subtlety of your language. And then there was Google. Uh, which felt like it hadn't changed in a decade, uh, because it really hadn't. And it, like, you would give it a simple query, like, I don't know, uh, shirts without stripes, and it would give you a bunch of results for the shirts with stripes. And so, like, Google could barely understand you, and GBD3 could. And the theory was, what if you could make a search engine that actually understood you? What if you could apply the insights from LLMs to a search engine? And it's really been the same idea ever since. And we're actually a lot closer now, uh, to doing that. Yeah.Alessio [00:01:55]: Did you have any trouble making people believe? Obviously, there's the same element. I mean, YC overlap, was YC pretty AI forward, even 2021, or?Will [00:02:03]: It's nothing like it is today. But, um, uh, there were a few AI companies, but, uh, we were definitely, like, bold. And I think people, VCs generally like boldness, and we definitely had some AI background, and we had a working demo. So there was evidence that we could build something that was going to work. But yeah, I think, like, the fundamentals were there. I think people at the time were talking about how, you know, Google was failing in a lot of ways. And so there was a bit of conversation about it, but AI was not a big, big thing at the time. Yeah. Yeah.Alessio [00:02:33]: Before we jump into Exa, any fun background stories? I know you interned at SpaceX, any Elon, uh, stories? I know you were at Zoox as well, you know, kind of like robotics at Harvard. Any stuff that you saw early that you thought was going to get solved that maybe it's not solved today?Will [00:02:48]: Oh yeah. I mean, lots of things like that. Like, uh, I never really learned how to drive because I believed Elon that self-driving cars would happen. It did happen. And I take them every night to get home. But it took like 10 more years than I thought. Do you still not know how to drive? I know how to drive now. I learned it like two years ago. That would have been great to like, just, you know, Yeah, yeah, yeah. You know? Um, I was obsessed with Elon. Yeah. I mean, I worked at SpaceX because I really just wanted to work at one of his companies. And I remember they had a rule, like interns cannot touch Elon. And, um, that rule actually influenced my actions.Swyx [00:03:18]: Is it, can Elon touch interns? Ooh, like physically?Will [00:03:22]: Or like talk? Physically, physically, yeah, yeah, yeah, yeah. Okay, interesting. He's changed a lot, but, um, I mean, his companies are amazing. Um,Swyx [00:03:28]: What if you beat him at Diablo 2, Diablo 4, you know, like, Ah, maybe.Alessio [00:03:34]: I want to jump into, I know there's a lot of backstory used to be called metaphor system. So, um, and it, you've always been kind of like a prominent company, maybe at least RAI circles in the NSF.Swyx [00:03:45]: I'm actually curious how Metaphor got its initial aura. You launched with like, very little. We launched very little. Like there was, there was this like big splash image of like, this is Aurora or something. Yeah. Right. And then I was like, okay, what this thing, like the vibes are good, but I don't know what it is. And I think, I think it was much more sort of maybe consumer facing than what you are today. Would you say that's true?Will [00:04:06]: No, it's always been about building a better search algorithm, like search, like, just like the vision has always been perfect search. And if you do that, uh, we will figure out the downstream use cases later. It started on this fundamental belief that you could have perfect search over the web and we could talk about what that means. And like the initial thing we released was really just like our first search engine, like trying to get it out there. Kind of like, you know, an open source. So when OpenAI released, uh, ChachBt, like they didn't, I don't know how, how much of a game plan they had. They kind of just wanted to get something out there.Swyx [00:04:33]: Spooky research preview.Will [00:04:34]: Yeah, exactly. And it kind of morphed from a research company to a product company at that point. And I think similarly for us, like we were research, we started as a research endeavor with a, you know, clear eyes that like, if we succeed, it will be a massive business to make out of it. And that's kind of basically what happened. I think there are actually a lot of parallels to, of w between Exa and OpenAI. I often say we're the OpenAI of search. Um, because. Because we're a research company, we're a research startup that does like fundamental research into, uh, making like AGI for search in a, in a way. Uh, and then we have all these like, uh, business products that come out of that.Swyx [00:05:08]: Interesting. I want to ask a little bit more about Metaforesight and then we can go full Exa. When I first met you, which was really funny, cause like literally I stayed in your house in a very historic, uh, Hayes, Hayes Valley place. You said you were building sort of like link prediction foundation model, and I think there's still a lot of foundation model work. I mean, within Exa today, but what does that even mean? I cannot be the only person confused by that because like there's a limited vocabulary or tokens you're telling me, like the tokens are the links or, you know, like it's not, it's not clear. Yeah.Will [00:05:38]: Uh, what we meant by link prediction is that you are literally predicting, like given some texts, you're predicting the links that follow. Yes. That refers to like, it's how we describe the training procedure, which is that we find links on the web. Uh, we take the text surrounding the link. And then we predict. Which link follows you, like, uh, you know, similar to transformers where, uh, you're trying to predict the next token here, you're trying to predict the next link. And so you kind of like hide the link from the transformer. So if someone writes, you know, imagine some article where someone says, Hey, check out this really cool aerospace startup. And they, they say spacex.com afterwards, uh, we hide the spacex.com and ask the model, like what link came next. And by doing that many, many times, you know, billions of times, you could actually build a search engine out of that because then, uh, at query time at search time. Uh, you type in, uh, a query that's like really cool aerospace startup and the model will then try to predict what are the most likely links. So there's a lot of analogs to transformers, but like to actually make this work, it does require like a different architecture than, but it's transformer inspired. Yeah.Alessio [00:06:41]: What's the design decision between doing that versus extracting the link and the description and then embedding the description and then using, um, yeah. What do you need to predict the URL versus like just describing, because you're kind of do a similar thing in a way. Right. It's kind of like based on this description, it was like the closest link for it. So one thing is like predicting the link. The other approach is like I extract the link and the description, and then based on the query, I searched the closest description to it more. Yeah.Will [00:07:09]: That, that, by the way, that is, that is the link refers here to a document. It's not, I think one confusing thing is it's not, you're not actually predicting the URL, the URL itself that would require like the, the system to have memorized URLs. You're actually like getting the actual document, a more accurate name could be document prediction. I see. This was the initial like base model that Exo was trained on, but we've moved beyond that similar to like how, you know, uh, to train a really good like language model, you might start with this like self-supervised objective of predicting the next token and then, uh, just from random stuff on the web. But then you, you want to, uh, add a bunch of like synthetic data and like supervised fine tuning, um, stuff like that to make it really like controllable and robust. Yeah.Alessio [00:07:48]: Yeah. We just have flow from Lindy and, uh, their Lindy started to like hallucinate recrolling YouTube links instead of like, uh, something. Yeah. Support guide. So. Oh, interesting. Yeah.Swyx [00:07:57]: So round about January, you announced your series A and renamed to Exo. I didn't like the name at the, at the initial, but it's grown on me. I liked metaphor, but apparently people can spell metaphor. What would you say are the major components of Exo today? Right? Like, I feel like it used to be very model heavy. Then at the AI engineer conference, Shreyas gave a really good talk on the vector database that you guys have. What are the other major moving parts of Exo? Okay.Will [00:08:23]: So Exo overall is a search engine. Yeah. We're trying to make it like a perfect search engine. And to do that, you have to build lots of, and we're doing it from scratch, right? So to do that, you have to build lots of different. The crawler. Yeah. You have to crawl a bunch of the web. First of all, you have to find the URLs to crawl. Uh, it's connected to the crawler, but yeah, you find URLs, you crawl those URLs. Then you have to process them with some, you know, it could be an embedding model. It could be something more complex, but you need to take, you know, or like, you know, in the past it was like a keyword inverted index. Like you would process all these documents you gather into some processed index, and then you have to serve that. Uh, you had high throughput at low latency. And so that, and that's like the vector database. And so it's like the crawling system, the AI processing system, and then the serving system. Those are all like, you know, teams of like hundreds, maybe thousands of people at Google. Um, but for us, it's like one or two people each typically, but yeah.Alessio [00:09:13]: Can you explain the meaning of, uh, Exo, just the story 10 to the 16th, uh, 18, 18.Will [00:09:20]: Yeah, yeah, yeah, sure. So. Exo means 10 to the 18th, which is in stark contrast to. To Google, which is 10 to the hundredth. Uh, we actually have these like awesome shirts that are like 10th to 18th is greater than 10th to the hundredth. Yeah, it's great. And it's great because it's provocative. It's like every engineer in Silicon Valley is like, what? No, it's not true. Um, like, yeah. And, uh, and then you, you ask them, okay, what does it actually mean? And like the creative ones will, will recognize it. But yeah, I mean, 10 to the 18th is better than 10 to the hundredth when it comes to search, because with search, you want like the actual list of, of things that match what you're asking for. You don't want like the whole web. You want to basically with search filter, the, like everything that humanity has ever created to exactly what you want. And so the idea is like smaller is better there. You want like the best 10th to the 18th and not the 10th to the hundredth. I'm like, one way to say this is like, you know how Google often says at the top, uh, like, you know, 30 million results found. And it's like crazy. Cause you're looking for like the first startups in San Francisco that work on hardware or something. And like, they're not 30 million results like that. What you want is like 325 results found. And those are all the results. That's what you really want with search. And that's, that's our vision. It's like, it just gives you. Perfectly what you asked for.Swyx [00:10:24]: We're recording this ahead of your launch. Uh, we haven't released, we haven't figured out the, the, the name of the launch yet, but what is the product that you're launching? I guess now that we're coinciding this podcast with. Yeah.Will [00:10:36]: So we've basically developed the next version of Exa, which is the ability to get a near perfect list of results of whatever you want. And what that means is you can make a complex query now to Exa, for example, startups working on hardware in SF, and then just get a huge list of all the things that match. And, you know, our goal is if there are 325 startups that match that we find you all of them. And this is just like, there's just like a new experience that's never existed before. It's really like, I don't know how you would go about that right now with current tools and you can apply this same type of like technology to anything. Like, let's say you want, uh, you want to find all the blog posts that talk about Alessio's podcast, um, that have come out in the past year. That is 30 million results. Yeah. Right.Will [00:11:24]: But that, I mean, that would, I'm sure that would be extremely useful to you guys. And like, I don't really know how you would get that full comprehensive list.Swyx [00:11:29]: I just like, how do you, well, there's so many questions with regards to how do you know it's complete, right? Cause you're saying there's only 30 million, 325, whatever. And then how do you do the semantic understanding that it might take, right? So working in hardware, like I might not use the words hardware. I might use the words robotics. I might use the words wearables. I might use like whatever. Yes. So yeah, just tell us more. Yeah. Yeah. Sure. Sure.Will [00:11:53]: So one aspect of this, it's a little subjective. So like certainly providing, you know, at some point we'll provide parameters to the user to like, you know, some sort of threshold to like, uh, gauge like, okay, like this is a cutoff. Like, this is actually not what I mean, because sometimes it's subjective and there needs to be a feedback loop. Like, oh, like it might give you like a few examples and you say, yeah, exactly. And so like, you're, you're kind of like creating a classifier on the fly, but like, that's ultimately how you solve the problem. So the subject, there's a subjectivity problem and then there's a comprehensiveness problem. Those are two different problems. So. Yeah. So you have the comprehensiveness problem. What you basically have to do is you have to put more compute into the query, into the search until you get the full comprehensiveness. Yeah. And I think there's an interesting point here, which is that not all queries are made equal. Some queries just like this blog post one might require scanning, like scavenging, like throughout the whole web in a way that just, just simply requires more compute. You know, at some point there's some amount of compute where you will just be comprehensive. You could imagine, for example, running GPT-4 over the internet. You could imagine running GPT-4 over the entire web and saying like, is this a blog post about Alessio's podcast, like, is this a blog post about Alessio's podcast? And then that would work, right? It would take, you know, a year, maybe cost like a million dollars, but, or many more, but, um, it would work. Uh, the point is that like, given sufficient compute, you can solve the query. And so it's really a question of like, how comprehensive do you want it given your compute budget? I think it's very similar to O1, by the way. And one way of thinking about what we built is like O1 for search, uh, because O1 is all about like, you know, some, some, some questions require more compute than others, and we'll put as much compute into the question as we need to solve it. So similarly with our search, we will put as much compute into the query in order to get comprehensiveness. Yeah.Swyx [00:13:33]: Does that mean you have like some kind of compute budget that I can specify? Yes. Yes. Okay. And like, what are the upper and lower bounds?Will [00:13:42]: Yeah, there's something we're still figuring out. I think like, like everyone is a new paradigm of like variable compute products. Yeah. How do you specify the amount of compute? Like what happens when you. Run out? Do you just like, ah, do you, can you like keep going with it? Like, do you just put in more credits to get more, um, for some, like this can get complex at like the really large compute queries. And like, one thing we do is we give you a preview of what you're going to get, and then you could then spin up like a much larger job, uh, to get like way more results. But yes, there is some compute limit, um, at, at least right now. Yeah. People think of searches as like, oh, it takes 500 milliseconds because we've been conditioned, uh, to have search that takes 500 milliseconds. But like search engines like Google, right. No matter how complex your query to Google, it will take like, you know, roughly 400 milliseconds. But what if searches can take like a minute or 10 minutes or a whole day, what can you then do? And you can do very powerful things. Um, you know, you can imagine, you know, writing a search, going and get a cup of coffee, coming back and you have a perfect list. Like that's okay for a lot of use cases. Yeah.Alessio [00:14:43]: Yeah. I mean, the use case closest to me is venture capital, right? So, uh, no, I mean, eight years ago, I built one of the first like data driven sourcing platforms. So we were. You look at GitHub, Twitter, Product Hunt, all these things, look at interesting things, evaluate them. If you think about some jobs that people have, it's like literally just make a list. If you're like an analyst at a venture firm, your job is to make a list of interesting companies. And then you reach out to them. How do you think about being infrastructure versus like a product you could say, Hey, this is like a product to find companies. This is a product to find things versus like offering more as a blank canvas that people can build on top of. Oh, right. Right.Will [00:15:20]: Uh, we are. We are a search infrastructure company. So we want people to build, uh, on top of us, uh, build amazing products on top of us. But with this one, we try to build something that makes it really easy for users to just log in, put a few, you know, put some credits in and just get like amazing results right away and not have to wait to build some API integration. So we're kind of doing both. Uh, we, we want, we want people to integrate this into all their applications at the same time. We want to just make it really easy to use very similar again to open AI. Like they'll have, they have an API, but they also have. Like a ChatGPT interface so that you could, it's really easy to use, but you could also build it in your applications. Yeah.Alessio [00:15:56]: I'm still trying to wrap my head around a lot of the implications. So, so many businesses run on like information arbitrage, you know, like I know this thing that you don't, especially in investment and financial services. So yeah, now all of a sudden you have these tools for like, oh, actually everybody can get the same information at the same time, the same quality level as an API call. You know, it just kind of changes a lot of things. Yeah.Will [00:16:19]: I think, I think what we're grappling with here. What, what you're just thinking about is like, what is the world like if knowledge is kind of solved, if like any knowledge request you want is just like right there on your computer, it's kind of different from when intelligence is solved. There's like a good, I've written before about like a different super intelligence, super knowledge. Yeah. Like I think that the, the distinction between intelligence and knowledge is actually a pretty good one. They're definitely connected and related in all sorts of ways, but there is a distinction. You could have a world and we are going to have this world where you have like GP five level systems and beyond that could like answer any complex request. Um, unless it requires some. Like, if you say like, uh, you know, give me a list of all the PhDs in New York city who, I don't know, have thought about search before. And even though this, this super intelligence is going to be like, I can't find it on Google, right. Which is kind of crazy. Like we're literally going to have like super intelligences that are using Google. And so if Google can't find them information, there's nothing they could do. They can't find it. So, but if you also have a super knowledge system where it's like, you know, I'm calling this term super knowledge where you just get whatever knowledge you want, then you can pair with a super intelligence system. And then the super intelligence can, we'll never. Be blocked by lack of knowledge.Alessio [00:17:23]: Yeah. You told me this, uh, when we had lunch, I forget how it came out, but we were talking about AGI and whatnot. And you were like, even AGI is going to need search. Yeah.Swyx [00:17:32]: Yeah. Right. Yeah. Um, so we're actually referencing a blog post that you wrote super intelligence and super knowledge. Uh, so I would refer people to that. And this is actually a discussion we've had on the podcast a couple of times. Um, there's so much of model weights that are just memorizing facts. Some of the, some of those might be outdated. Some of them are incomplete or not. Yeah. So like you just need search. So I do wonder, like, is there a maximum language model size that will be the intelligence layer and then the rest is just search, right? Like maybe we should just always use search. And then that sort of workhorse model is just like, and it like, like, like one B or three B parameter model that just drives everything. Yes.Will [00:18:13]: I believe this is a much more optimal system to have a smaller LM. That's really just like an intelligence module. And it makes a call to a search. Tool that's way more efficient because if, okay, I mean the, the opposite of that would be like the LM is so big that can memorize the whole web. That would be like way, but you know, it's not practical at all. I don't, it's not possible to train that at least right now. And Carpathy has actually written about this, how like he could, he could see models moving more and more towards like intelligence modules using various tools. Yeah.Swyx [00:18:39]: So for listeners, that's the, that was him on the no priors podcast. And for us, we talked about this and the, on the Shin Yu and Harrison chase podcasts. I'm doing search in my head. I told you 30 million results. I forgot about our neural link integration. Self-hosted exit.Will [00:18:54]: Yeah. Yeah. No, I do see that that is a much more, much more efficient world. Yeah. I mean, you could also have GB four level systems calling search, but it's just because of the cost of inference. It's just better to have a very efficient search tool and a very efficient LM and they're built for different things. Yeah.Swyx [00:19:09]: I'm just kind of curious. Like it is still something so audacious that I don't want to elide, which is you're, you're, you're building a search engine. Where do you start? How do you, like, are there any reference papers or implementation? That would really influence your thinking, anything like that? Because I don't even know where to start apart from just crawl a bunch of s**t, but there's gotta be more insight than that.Will [00:19:28]: I mean, yeah, there's more insight, but I'm always surprised by like, if you have a group of people who are really focused on solving a problem, um, with the tools today, like there's some in, in software, like there are all sorts of creative solutions that just haven't been thought of before, particularly in the information retrieval field. Yeah. I think a lot of the techniques are just very old, frankly. Like I know how Google and Bing work and. They're just not using new methods. There are all sorts of reasons for that. Like one, like Google has to be comprehensive over the web. So they're, and they have to return in 400 milliseconds. And those two things combined means they are kind of limit and it can't cost too much. They're kind of limited in, uh, what kinds of algorithms they could even deploy at scale. So they end up using like a limited keyword based algorithm. Also like Google was built in a time where like in, you know, in 1998, where we didn't have LMS, we didn't have embeddings. And so they never thought to build those things. And so now they have this like gigantic system that is built on old technology. Yeah. And so a lot of the information retrieval field we found just like thinks in terms of that framework. Yeah. Whereas we came in as like newcomers just thinking like, okay, there here's GB three. It's magical. Obviously we're going to build search that is using that technology. And we never even thought about using keywords really ever. Uh, like we were neural all the way we're building an end to end neural search engine. And just that whole framing just makes us ask different questions, like pursue different lines of work. And there's just a lot of low hanging fruit because no one else is thinking about it. We're just on the frontier of neural search. We just are, um, for, for at web scale, um, because there's just not a lot of people thinking that way about it.Swyx [00:20:57]: Yeah. Maybe let's spell this out since, uh, we're already on this topic, elephants in the room are Perplexity and SearchGPT. That's the, I think that it's all, it's no longer called SearchGPT. I think they call it ChatGPT Search. How would you contrast your approaches to them based on what we know of how they work and yeah, just any, anything in that, in that area? Yeah.Will [00:21:15]: So these systems, there are a few of them now, uh, they basically rely on like traditional search engines like Google or Bing, and then they combine them with like LLMs at the end to, you know, output some power graphics, uh, answering your question. So they like search GPT perplexity. I think they have their own crawlers. No. So there's this important distinction between like having your own search system and like having your own cache of the web. Like for example, so you could create, you could crawl a bunch of the web. Imagine you crawl a hundred billion URLs, and then you create a key value store of like mapping from URL to the document that is technically called an index, but it's not a search algorithm. So then to actually like, when you make a query to search GPT, for example, what is it actually doing it? Let's say it's, it's, it could, it's using the Bing API, uh, getting a list of results and then it could go, it has this cache of like all the contents of those results and then could like bring in the cache, like the index cache, but it's not actually like, it's not like they've built a search engine from scratch over, you know, hundreds of billions of pages. It's like, does that distinction clear? It's like, yeah, you could have like a mapping from URL to documents, but then rely on traditional search engines to actually get the list of results because it's a very hard problem to take. It's not hard. It's not hard to use DynamoDB and, and, and map URLs to documents. It's a very hard problem to take a hundred billion or more documents and given a query, like instantly get the list of results that match. That's a much harder problem that very few entities on, in, on the planet have done. Like there's Google, there's Bing, uh, you know, there's Yandex, but you know, there are not that many companies that are, that are crazy enough to actually build their search engine from scratch when you could just use traditional search APIs.Alessio [00:22:43]: So Google had PageRank as like the big thing. Is there a LLM equivalent or like any. Stuff that you're working on that you want to highlight?Will [00:22:51]: The link prediction objective can be seen as like a neural PageRank because what you're doing is you're predicting the links people share. And so if everyone is sharing some Paul Graham essay about fundraising, then like our model is more likely to predict it. So like inherent in our training objective is this, uh, a sense of like high canonicity and like high quality, but it's more powerful than PageRank. It's strictly more powerful because people might refer to that Paul Graham fundraising essay in like a thousand different ways. And so our model learns all the different ways. That someone refers that Paul Graham, I say, while also learning how important that Paul Graham essay is. Um, so it's like, it's like PageRank on steroids kind of thing. Yeah.Alessio [00:23:26]: I think to me, that's the most interesting thing about search today, like with Google and whatnot, it's like, it's mostly like domain authority. So like if you get back playing, like if you search any AI term, you get this like SEO slop websites with like a bunch of things in them. So this is interesting, but then how do you think about more timeless maybe content? So if you think about, yeah. You know, maybe the founder mode essay, right. It gets shared by like a lot of people, but then you might have a lot of other essays that are also good, but they just don't really get a lot of traction. Even though maybe the people that share them are high quality. How do you kind of solve that thing when you don't have the people authority, so to speak of who's sharing, whether or not they're worth kind of like bumping up? Yeah.Will [00:24:10]: I mean, you do have a lot of control over the training data, so you could like make sure that the training data contains like high quality sources so that, okay. Like if you, if you're. Training data, I mean, it's very similar to like language, language model training. Like if you train on like a bunch of crap, your prediction will be crap. Our model will match the training distribution is trained on. And so we could like, there are lots of ways to tweak the training data to refer to high quality content that we want. Yeah. I would say also this, like this slop that is returned by, by traditional search engines, like Google and Bing, you have the slop is then, uh, transferred into the, these LLMs in like a search GBT or, you know, our other systems like that. Like if slop comes in, slop will go out. And so, yeah, that's another answer to how we're different is like, we're not like traditional search engines. We want to give like the highest quality results and like have full control over whatever you want. If you don't want slop, you get that. And then if you put an LM on top of that, which our customers do, then you just get higher quality results or high quality output.Alessio [00:25:06]: And I use Excel search very often and it's very good. Especially.Swyx [00:25:09]: Wave uses it too.Alessio [00:25:10]: Yeah. Yeah. Yeah. Yeah. Yeah. Like the slop is everywhere, especially when it comes to AI, when it comes to investment. When it comes to all of these things for like, it's valuable to be at the top. And this problem is only going to get worse because. Yeah, no, it's totally. What else is in the toolkit? So you have search API, you have ExaSearch, kind of like the web version. Now you have the list builder. I think you also have web scraping. Maybe just touch on that. Like, I guess maybe people, they want to search and then they want to scrape. Right. So is that kind of the use case that people have? Yeah.Will [00:25:41]: A lot of our customers, they don't just want, because they're building AI applications on top of Exa, they don't just want a list of URLs. They actually want. Like the full content, like cleans, parsed. Markdown. Markdown, maybe chunked, whatever they want, we'll give it to them. And so that's been like huge for customers. Just like getting the URLs and instantly getting the content for each URL is like, and you can do this for 10 or 100 or 1,000 URLs, wherever you want. That's very powerful.Swyx [00:26:05]: Yeah. I think this is the first thing I asked you for when I tried using Exa.Will [00:26:09]: Funny story is like when I built the first version of Exa, it's like, we just happened to store the content. Yes. Like the first 1,024 tokens. Because I just kind of like kept it because I thought of, you know, I don't know why. Really for debugging purposes. And so then when people started asking for content, it was actually pretty easy to serve it. But then, and then we did that, like Exa took off. So the computer's content was so useful. So that was kind of cool.Swyx [00:26:30]: It is. I would say there are other players like Gina, I think is in this space. Firecrawl is in this space. There's a bunch of scraper companies. And obviously scraper is just one part of your stack, but you might as well offer it since you already do it.Will [00:26:43]: Yeah, it makes sense. It's just easy to have an all-in-one solution. And like. We are, you know, building the best scraper in the world. So scraping is a hard problem and it's easy to get like, you know, a good scraper. It's very hard to get a great scraper and it's super hard to get a perfect scraper. So like, and, and scraping really matters to people. Do you have a perfect scraper? Not yet. Okay.Swyx [00:27:05]: The web is increasingly closing to the bots and the scrapers, Twitter, Reddit, Quora, Stack Overflow. I don't know what else. How are you dealing with that? How are you navigating those things? Like, you know. You know, opening your eyes, like just paying them money.Will [00:27:19]: Yeah, no, I mean, I think it definitely makes it harder for search engines. One response is just that there's so much value in the long tail of sites that are open. Okay. Um, and just like, even just searching over those well gets you most of the value. But I mean, there, there is definitely a lot of content that is increasingly not unavailable. And so you could get through that through data partnerships. The bigger we get as a company, the more, the easier it is to just like, uh, make partnerships. But I, I mean, I do see the world as like the future where the. The data, the, the data producers, the content creators will make partnerships with the entities that find that data.Alessio [00:27:53]: Any other fun use case that maybe people are not thinking about? Yeah.Will [00:27:58]: Oh, I mean, uh, there are so many customers. Yeah. What are people doing on AXA? Well, I think dating is a really interesting, uh, application of search that is completely underserved because there's a lot of profiles on the web and a lot of people who want to find love and that I'll use it. They give me. Like, you know, age boundaries, you know, education level location. Yeah. I mean, you want to, what, what do you want to do with data? You want to find like a partner who matches this education level, who like, you know, maybe has written about these types of topics before. Like if you could get a list of all the people like that, like, I think you will unblock a lot of people. I mean, there, I mean, I think this is a very Silicon Valley view of dating for sure. And I'm, I'm well aware of that, but it's just an interesting application of like, you know, I would love to meet like an intellectual partner, um, who like shares a lot of ideas. Yeah. Like if you could do that through better search and yeah.Swyx [00:28:48]: But what is it with Jeff? Jeff has already set me up with a few people. So like Jeff, I think it's my personal exit.Will [00:28:55]: my mom's actually a matchmaker and has got a lot of married. Yeah. No kidding. Yeah. Yeah. Search is built into the book. It's in your jeans. Yeah. Yeah.Swyx [00:29:02]: Yeah. Other than dating, like I know you're having quite some success in colleges. I would just love to map out some more use cases so that our listeners can just use those examples to think about use cases for XR, right? Because it's such a general technology that it's hard to. Uh, really pin down, like, what should I use it for and what kind of products can I build with it?Will [00:29:20]: Yeah, sure. So, I mean, there are so many applications of XR and we have, you know, many, many companies using us for very diverse range of use cases, but I'll just highlight some interesting ones. Like one customer, a big customer is using us to, um, basically build like a, a writing assistant for students who want to write, uh, research papers. And basically like XR will search for, uh, like a list of research papers related to what the student is writing. And then this product has. Has like an LLM that like summarizes the papers to basically it's like a next word prediction, but in, uh, you know, prompted by like, you know, 20 research papers that X has returned. It's like literally just doing their homework for them. Yeah. Yeah. the key point is like, it's, it's, uh, you know, it's, it's, you know, research is, is a really hard thing to do and you need like high quality content as input.Swyx [00:30:08]: Oh, so we've had illicit on the podcast. I think it's pretty similar. Uh, they, they do focus pretty much on just, just research papers and, and that research. Basically, I think dating, uh, research, like I just wanted to like spell out more things, like just the big verticals.Will [00:30:23]: Yeah, yeah, no, I mean, there, there are so many use cases. So finance we talked about, yeah. I mean, one big vertical is just finding a list of companies, uh, so it's useful for VCs, like you said, who want to find like a list of competitors to a specific company they're investigating or just a list of companies in some field. Like, uh, there was one VC that told me that him and his team, like we're using XR for like eight hours straight. Like, like that. For many days on end, just like, like, uh, doing like lots of different queries of different types, like, oh, like all the companies in AI for law or, uh, all the companies for AI for, uh, construction and just like getting lists of things because you just can't find this information with, with traditional search engines. And then, you know, finding companies is also useful for, for selling. If you want to find, you know, like if we want to find a list of, uh, writing assistants to sell to, then we can just, we just use XR ourselves to find that is actually how we found a lot of our customers. Ooh, you can find your own customers using XR. Oh my God. I, in the spirit of. Uh, using XR to bolster XR, like recruiting is really helpful. It is really great use case of XR, um, because we can just get like a list of, you know, people who thought about search and just get like a long list and then, you know, reach out to those people.Swyx [00:31:29]: When you say thought about, are you, are you thinking LinkedIn, Twitter, or are you thinking just blogs?Will [00:31:33]: Or they've written, I mean, it's pretty general. So in that case, like ideally XR would return like the, the really blogs written by people who have just. So if I don't blog, I don't show up to XR, right? Like I have to blog. well, I mean, you could show up. That's like an incentive for people to blog.Swyx [00:31:47]: Well, if you've written about, uh, search in on Twitter and we, we do, we do index a bunch of tweets and then we, we should be able to service that. Yeah. Um, I mean, this is something I tell people, like you have to make yourself discoverable to the web, uh, you know, it's called learning in public, but like, it's even more imperative now because otherwise you don't exist at all.Will [00:32:07]: Yeah, no, no, this is a huge, uh, thing, which is like search engines completely influence. They have downstream effects. They influence the internet itself. They influence what people. Choose to create. And so Google, because they're a keyword based search engine, people like kind of like keyword stuff. Yeah. They're, they're, they're incentivized to create things that just match a lot of keywords, which is not very high quality. Uh, whereas XR is a search algorithm that, uh, optimizes for like high quality and actually like matching what you mean. And so people are incentivized to create content that is high quality, that like the create content that they know will be found by the right person. So like, you know, if I am a search researcher and I want to be found. By XR, I should blog about search and all the things I'm building because, because now we have a search engine like XR that's powerful enough to find them. And so the search engine will influence like the downstream internet in all sorts of amazing ways. Yeah. Uh, whatever the search engine optimizes for is what the internet looks like. Yeah.Swyx [00:33:01]: Are you familiar with the term? McLuhanism? No, it's not. Uh, it's this concept that, uh, like first we shape tools and then the tools shape us. Okay. Yeah. Uh, so there's like this reflexive connection between the things we search for and the things that get searched. Yes. So like once you change the tool. The tool that searches the, the, the things that get searched also change. Yes.Will [00:33:18]: I mean, there was a clear example of that with 30 years of Google. Yeah, exactly. Google has basically trained us to think of search and Google has Google is search like in people's heads. Right. It's one, uh, hard part about XR is like, uh, ripping people away from that notion of search and expanding their sense of what search could be. Because like when people think search, they think like a few keywords, or at least they used to, they think of a few keywords and that's it. They don't think to make these like really complex paragraph long requests for information and get a perfect list. ChatGPT was an interesting like thing that expanded people's understanding of search because you start using ChatGPT for a few hours and you go back to Google and you like paste in your code and Google just doesn't work and you're like, oh, wait, it, Google doesn't do work that way. So like ChatGPT expanded our understanding of what search can be. And I think XR is, uh, is part of that. We want to expand people's notion, like, Hey, you could actually get whatever you want. Yeah.Alessio [00:34:06]: I search on XR right now, people writing about learning in public. I was like, is it gonna come out with Alessio? Am I, am I there? You're not because. Bro. It's. So, no, it's, it's so about, because it thinks about learning, like in public, like public schools and like focuses more on that. You know, it's like how, when there are like these highly overlapping things, like this is like a good result based on the query, you know, but like, how do I get to Alessio? Right. So if you're like in these subcultures, I don't think this would work in Google well either, you know, but I, I don't know if you have any learnings.Swyx [00:34:40]: No, I'm the first result on Google.Alessio [00:34:42]: People writing about learning. In public, you're not first result anymore, I guess.Swyx [00:34:48]: Just type learning public in Google.Alessio [00:34:49]: Well, yeah, yeah, yeah, yeah. But this is also like, this is in Google, it doesn't work either. That's what I'm saying. It's like how, when you have like a movement.Will [00:34:56]: There's confusion about the, like what you mean, like your intention is a little, uh. Yeah.Alessio [00:35:00]: It's like, yeah, I'm using, I'm using a term that like I didn't invent, but I'm kind of taking over, but like, they're just so much about that term already that it's hard to overcome. If that makes sense, because public schools is like, well, it's, it's hard to overcome.Will [00:35:14]: Public schools, you know, so there's the right solution to this, which is to specify more clearly what you mean. And I'm not expecting you to do that, but so the, the right interface to search is actually an LLM.Swyx [00:35:25]: Like you should be talking to an LLM about what you want and the LLM translates its knowledge of you or knowledge of what people usually mean into a query that excellent uses, which you have called auto prompts, right?Will [00:35:35]: Or, yeah, but it's like a very light version of that. And really it's just basically the right answer is it's the wrong interface and like very soon interface to search and really to everything will be LLM. And the LLM just has a full knowledge of you, right? So we're kind of building for that world. We're skating to where the puck is going to be. And so since we're moving to a world where like LLMs are interfaced to everything, you should build a search engine that can handle complex LLM queries, queries that come from LLMs. Because you're probably too lazy, I'm too lazy too, to write like a whole paragraph explaining, okay, this is what I mean by this word. But an LLM is not lazy. And so like the LLM will spit out like a paragraph or more explaining exactly what it wants. You need a search engine that can handle that. Traditional search engines like Google or Bing, they're actually... Designed for humans typing keywords. If you give a paragraph to Google or Bing, they just completely fail. And so Exa can handle paragraphs and we want to be able to handle it more and more until it's like perfect.Alessio [00:36:24]: What about opinions? Do you have lists? When you think about the list product, do you think about just finding entries? Do you think about ranking entries? I'll give you a dumb example. So on Lindy, I've been building the spot that every week gives me like the top fantasy football waiver pickups. But every website is like different opinions. I'm like, you should pick up. These five players, these five players. When you're making lists, do you want to be kind of like also ranking and like telling people what's best? Or like, are you mostly focused on just surfacing information?Will [00:36:56]: There's a really good distinction between filtering to like things that match your query and then ranking based on like what is like your preferences. And ranking is like filtering is objective. It's like, does this document match what you asked for? Whereas ranking is more subjective. It's like, what is the best? Well, it depends what you mean by best, right? So first, first table stakes is let's get the filtering into a perfect place where you actually like every document matches what you asked for. No surgeon can do that today. And then ranking, you know, there are all sorts of interesting ways to do that where like you've maybe for, you know, have the user like specify more clearly what they mean by best. You could do it. And if the user doesn't specify, you do your best, you do your best based on what people typically mean by best. But ideally, like the user can specify, oh, when I mean best, I actually mean ranked by the, you know, the number of people who visited that site. Let's say is, is one example ranking or, oh, what I mean by best, let's say you're listing companies. What I mean by best is like the ones that have, uh, you know, have the most employees or something like that. Like there are all sorts of ways to rank a list of results that are not captured by something as subjective as best. Yeah. Yeah.Alessio [00:38:00]: I mean, it's like, who are the best NBA players in the history? It's like everybody has their own. Right.Will [00:38:06]: Right. But I mean, the, the, the search engine should definitely like, even if you don't specify it, it should do as good of a job as possible. Yeah. Yeah. No, no, totally. Yeah. Yeah. Yeah. Yeah. It's a new topic to people because we're not used to a search engine that can handle like a very complex ranking system. Like you think to type in best basketball players and not something more specific because you know, that's the only thing Google could handle. But if Google could handle like, oh, basketball players ranked by like number of shots scored on average per game, then you would do that. But you know, they can't do that. So.Swyx [00:38:32]: Yeah. That's fascinating. So you haven't used the word agents, but you're kind of building a search agent. Do you believe that that is agentic in feature? Do you think that term is distracting?Will [00:38:42]: I think it's a good term. I do think everything will eventually become agentic. And so then the term will lose power, but yes, like what we're building is agentic it in a sense that it takes actions. It decides when to go deeper into something, it has a loop, right? It feels different from traditional search, which is like an algorithm, not an agent. Ours is a combination of an algorithm and an agent.Swyx [00:39:05]: I think my reflection from seeing this in the coding space where there's basically sort of classic. Framework for thinking about this stuff is the self-driving levels of autonomy, right? Level one to five, typically the level five ones all failed because there's full autonomy and we're not, we're not there yet. And people like control. People like to be in the loop. So the, the, the level ones was co-pilot first and now it's like cursor and whatever. So I feel like if it's too agentic, it's too magical, like, like a, like a one shot, I stick a, stick a paragraph into the text box and then it spits it back to me. It might feel like I'm too disconnected from the process and I don't trust it. As opposed to something where I'm more intimately involved with the research product. I see. So like, uh, wait, so the earlier versions are, so if trying to stick to the example of the basketball thing, like best basketball player, but instead of best, you, you actually get to customize it with like, whatever the metric is that you, you guys care about. Yeah. I'm still not a basketballer, but, uh, but, but, you know, like, like B people like to be in my, my thesis is that agents level five agents failed because people like to. To kind of have drive assist rather than full self-driving.Will [00:40:15]: I mean, a lot of this has to do with how good agents are. Like at some point, if agents for coding are better than humans at all tests and then humans block, yeah, we're not there yet.Swyx [00:40:25]: So like in a world where we're not there yet, what you're pitching us is like, you're, you're kind of saying you're going all the way there. Like I kind of, I think all one is also very full, full self-driving. You don't get to see the plan. You don't get to affect the plan yet. You just fire off a query and then it goes away for a couple of minutes and comes back. Right. Which is effectively what you're saying you're going to do too. And you think there's.Will [00:40:42]: There's a, there's an in-between. I saw. Okay. So in building this product, we're exploring new interfaces because what does it mean to kick off a search that goes and takes 10 minutes? Like, is that a good interface? Because what if the search is actually wrong or it's not exactly, exactly specified to what you mean, which is why you get previews. Yeah. You get previews. So it is iterative, but ultimately once you've specified exactly what you mean, then you kind of do just want to kick off a batch job. Right. So perhaps what you're getting at is like, uh, there's this barrier with agents where you have to like explain the full context of what you mean, and a lot of failure modes happen when you have, when you don't. Yeah. There's failure modes from the agent, just not being smart enough. And then there's failure modes from the agent, not understanding exactly what you mean. And there's a lot of context that is shared between humans that is like lost between like humans and, and this like new creature.Alessio [00:41:32]: Yeah. Yeah. Because people don't know what's going on. I mean, to me, the best example of like system prompts is like, why are you writing? You're a helpful assistant. Like. Of course you should be an awful, but people don't yet know, like, can I assume that, you know, that, you know, it's like, why did the, and now people write, oh, you're a very smart software engineer, but like, you never made, you never make mistakes. Like, were you going to try and make mistakes before? So I think people don't yet have an understanding, like with, with driving people know what good driving is. It's like, don't crash, stay within kind of like a certain speed range. It's like, follow the directions. It's like, I don't really have to explain all of those things. I hope. But with. AI and like models and like search, people are like, okay, what do you actually know? What are like your assumptions about how search, how you're going to do search? And like, can I trust it? You know, can I influence it? So I think that's kind of the, the middle ground, like before you go ahead and like do all the search, it's like, can I see how you're doing it? And then maybe help show your work kind of like, yeah, steer you. Yeah. Yeah.Will [00:42:32]: No, I mean, yeah. Sure. Saying, even if you've crafted a great system prompt, you want to be part of the process itself. Uh, because the system prompt doesn't, it doesn't capture everything. Right. So yeah. A system prompt is like, you get to choose the person you work with. It's like, oh, like I want, I want a software engineer who thinks this way about code. But then even once you've chosen that person, you can't just give them a high level command and they go do it perfectly. You have to be part of that process. So yeah, I agree.Swyx [00:42:58]: Just a side note for my system, my favorite system, prompt programming anecdote now is the Apple intelligence system prompt that someone, someone's a prompt injected it and seen it. And like the Apple. Intelligence has the words, like, please don't, don't hallucinate. And it's like, of course we don't want you to hallucinate. Right. Like, so it's exactly that, that what you're talking about, like we should train this behavior into the model, but somehow we still feel the need to inject into the prompt. And I still don't even think that we are very scientific about it. Like it, I think it's almost like cargo culting. Like we have this like magical, like turn around three times, throw salt over your shoulder before you do something. And like, it worked the last time. So let's just do it the same time now. And like, we do, there's no science to this.Will [00:43:35]: I do think a lot of these problems might be ironed out in future versions. Right. So, and like, they might, they might hide the details from you. So it's like, they actually, all of them have a system prompt. That's like, you are a helpful assistant. You don't actually have to include it, even though it might actually be the way they've implemented in the backend. It should be done in RLE AF.Swyx [00:43:52]: Okay. Uh, one question I was just kind of curious about this episode is I'm going to try to frame this in terms of this, the general AI search wars, you know, you're, you're one player in that, um, there's perplexity, chat, GPT, search, and Google, but there's also like the B2B side, uh, we had. Drew Houston from Dropbox on, and he's competing with Glean, who've, uh, we've also had DD from, from Glean on, is there an appetite for Exa for my company's documents?Will [00:44:19]: There is appetite, but I think we have to be disciplined, focused, disciplined. I mean, we're already taking on like perfect web search, which is a lot. Um, but I mean, ultimately we want to build a perfect search engine, which definitely for a lot of queries involves your, your personal information, your company's information. And so, yeah, I mean, the grandest vision of Exa is perfect search really over everything, every domain, you know, we're going to have an Exa satellite, uh, because, because satellites can gather information that, uh, is not available publicly. Uh, gotcha. Yeah.Alessio [00:44:51]: Can we talk about AGI? We never, we never talk about AGI, but you had, uh, this whole tweet about, oh, one being the biggest kind of like AI step function towards it. Why does it feel so important to you? I know there's kind of like always criticism and saying, Hey, it's not the smartest son is better. It's like, blah, blah, blah. What? You choose C. So you say, this is what Ilias see or Sam see what they will see.Will [00:45:13]: I've just, I've just, you know, been connecting the dots. I mean, this was the key thing that a bunch of labs were working on, which is like, can you create a reward signal? Can you teach yourself based on a reward signal? Whether you're, if you're trying to learn coding or math, if you could have one model say, uh, be a grading system that says like you have successfully solved this programming assessment and then one model, like be the generative system. That's like, here are a bunch of programming assessments. You could train on that. It's basically whenever you could create a reward signal for some task, you could just generate a bunch of tasks for yourself. See that like, oh, on two of these thousand, you did well. And then you just train on that data. It's basically like, I mean, creating your own data for yourself and like, you know, all the labs working on that opening, I built the most impressive product doing that. And it's just very, it's very easy now to see how that could like scale to just solving, like, like solving programming or solving mathematics, which sounds crazy, but everything about our world right now is crazy.Alessio [00:46:07]: Um, and so I think if you remove that whole, like, oh, that's impossible, and you just think really clearly about like, what's now possible with like what, what they've done with O1, it's easy to see how that scales. How do you think about older GPT models then? Should people still work on them? You know, if like, obviously they just had the new Haiku, like, is it even worth spending time, like making these models better versus just, you know, Sam talked about O2 at that day. So obviously they're, they're spending a lot of time in it, but then you have maybe. The GPU poor, which are still working on making Lama good. Uh, and then you have the follower labs that do not have an O1 like model out yet. Yeah.Will [00:46:47]: This kind of gets into like, uh, what will the ecosystem of, of models be like in the future? And is there room is, is everything just gonna be O1 like models? I think, well, I mean, there's definitely a question of like inference speed and if certain things like O1 takes a long time, because that's the thing. Well, I mean, O1 is, is two things. It's like one it's it's use it's bootstrapping itself. It's teaching itself. And so the base model is smarter. But then it also has this like inference time compute where it could like spend like many minutes or many hours thinking. And so even the base model, which is also fast, it doesn't have to take minutes. It could take is, is better, smarter. I believe all models will be trained with this paradigm. Like you'll want to train on the best data, but there will be many different size models from different, very many different like companies, I believe. Yeah. Because like, I don't, yeah, I mean, it's hard, hard to predict, but I don't think opening eye is going to dominate like every possible LLM for every possible. Use case. I think for a lot of things, like you just want the fastest model and that might not involve O1 methods at all.Swyx [00:47:42]: I would say if you were to take the exit being O1 for search, literally, you really need to prioritize search trajectories, like almost maybe paying a bunch of grad students to go research things. And then you kind of track what they search and what the sequence of searching is, because it seems like that is the gold mine here, like the chain of thought or the thinking trajectory. Yeah.Will [00:48:05]: When it comes to search, I've always been skeptical. I've always been skeptical of human labeled data. Okay. Yeah, please. We tried something at our company at Exa recently where me and a bunch of engineers on the team like labeled a bunch of queries and it was really hard. Like, you know, you have all these niche queries and you're looking at a bunch of results and you're trying to identify which is matched to query. It's talking about, you know, the intricacies of like some biological experiment or something. I have no idea. Like, I don't know what matches and what, what labelers like me tend to do is just match by keyword. I'm like, oh, I don't know. Oh, like this document matches a bunch of keywords, so it must be good. But then you're actually completely missing the meaning of the document. Whereas an LLM like GB4 is really good at labeling. And so I actually think like you just we get by, which we are right now doing using like LLM

Ben Cates, NewsAdvance.com On Gretna, LCA, Altavista In State QFs by Ed Lane

Yonathan de León, alcalde de Arrecife, anuncia que el ayuntamiento ejercerá la Acusación Particular contra el pirómano que quemó 15 contenedores y tres vehículos en Altavista.

In our first How India™ podcast episode, host by Steve Davis talks to Jasmine Batra of Arrow Digital. Jasmine is the co-convener of the Technology Innovation and Startup National Industry Group, reflects on her first start-up, Employ India, launched in 1998. At a time when the internet was accessed through dial-up connections and search engines like Yahoo and AltaVista dominated, Jasmine, fresh out of her MBA, co-founded a job portal to connect Indian IT professionals with overseas opportunities. Despite the challenges of establishing trust in a nascent digital market, Employ India thrived with an innovative model where job seekers paid to be listed. The company quickly became cash-positive, receiving recognition as one of the top 10 job portals in India by the Economic Times. Jasmine transitioned into helping companies with digital marketing and visibility, founding Arrow Digital in 2007. With Google gaining prominence, she focused on growth marketing to ensure companies were seen online. Over the years, she has adapted to the evolving digital landscape, including the rise of platforms like YouTube and WhatsApp, which play key roles in India. Jasmine emphasises the diversity of India's market, explaining that each state has its own unique culture, language, and economic environment. This complexity requires companies to select the right launch location and adapt their strategies accordingly. She points out that while India can be price-sensitive, it also has an appetite for high-quality, premium products, as seen with luxury car sales. For companies entering the market, understanding cultural nuances and aligning marketing efforts with major festivals like Diwali is crucial. Now leading the National Industry Group, Jasmine assists start-ups in validating and commercialising in India. She stresses the importance of building relationships and leveraging chambers of commerce to navigate India's complex business landscape. Jasmine runs events, including pitch fests and delegations to India, to facilitate market entry for Australian companies. For those looking to explore opportunities in India, Jasmine advises reaching out through her website, jasminebatra.com, or the Australia India Chamber of Commerce. The National Industry Group offers support, market insights, and connections at both federal and state levels, helping companies better understand regional dynamics and establish a successful presence in the Indian market.See omnystudio.com/listener for privacy information.

Os preços de insumos e materiais básicos voltaram a subir no Brasil – e isso é uma má notícia não só para o produtor, mas também para o consumidor.  Geralmente, um aumento de custos para os empresários tende a ser repassado para os consumidores. Dessa forma, pressões inflacionárias sofridas pelos produtores podem antecipar o aumento de preços ao cliente final.   Com a alta dos preços ao produtor, qual a perspectiva para a inflação brasileira?  Entenda também: Os números do comércio e do varejo no Brasil; Mais sinais do corte de juros nos EUA; No Reino Unido, PIB em alta e inflação esfriando. No minuto 05:00, Claudia Moreno, responsável pela cobertura de Brasil no time de economia do C6 Bank, explica por que os preços de serviços também estão mais elevados. 

It's Friday, August 2nd, A.D. 2024. This is The World View in 5 Minutes written by Kevin Swanson and heard at www.TheWorldView.com.  Filling in for Adam McManus I'm Ean Leppin. Liberty Counsel Represents Kim Davis Liberty Counsel is taking hits for representing Kentucky County Clerk Kim Davis' religious liberty case at the 6th Circuit Court Appeals. At issue is the Liberty Counsel's challenge of the 2015 Obergefell decision and a potential reversal, in favor of religious liberty. Kim and her husband, as well as Liberty Counsel have been subjected to multiple, serious death threats. Liberty Counsel President Mat Staver released a statement Monday, noting that “Anyone who stands up to the hateful agenda of the LGBTQ Mafia is demonized. .   .. the LGBTQ left will not tolerate religious freedom and wants to destroy anyone who disagrees.” Biden Administration Announces Plea Deal with 9/11 Conspirators The Biden administration Department of Justice has announced a plea deal with alleged conspirators of the 9/11 attacks, which occurred 23 years ago. Khalid Sheikh Mohammed, Walid Bin ‘Attash, and Mustafa al Hawsawi will plead guilty to conspiracy and murder charges, but will not face the death penalty for the murders of 2,977 people on September 11, 2001. Senator JD Vance commenting on the deal told an audience yesterday, “"We need a president who kills terrorists, not negotiates with them.” And Speaker of the House, Mike Johnson, called the decision “unthinkable” and a “slap in the face” for families of those murdered by the terrorists. God said, “Whoever sheds man's blood, by man his blood shall be shed; for in the image of God He made man.” (Gen. 9:6) 60% of Americans Support Death Penalty 60% of Americans support the death penalty. There were 23 executions in the US last year down from 98 in 1999. The US Homicide rate has also increased since 2014, from 4.7 per 100,000, to 6.0 per 100,000 persons in 2023, which accounts for over 20,000 murders. Keep in mind, “The ruler is God's minister to you for good. But if you do evil, be afraid; for he does not bear the sword in vain; for he is God's minister, an avenger to execute wrath on him who practices evil.” Romans 13:4 20 Dead Over Venezuelan Protests Protests following the Venezuelan sham election over the weekend in which the communist dictator claimed victory - have resulted in 20 deaths, and 1,072 persons arrested by the regime, according to Effecto Cocuyo — an independent news source. The nation's prosecutor, Tarek William Saab has warned protestors, that they are facing up to 20-30 years in prison. A respected American polling organization, Edison Research, has issued its exit poll results for the Venezuelan election. Opposition candidate Edmundo González Urrutia of the Unitary Platform easily won by a margin of 65 to 31%. Younger (18-29 year old) voters were more likely to vote against the communist dictator, by a margin of 74% to 21%. Edison Research has been the sole provider of election data to the National Election Pool, consisting of ABC News, CBS News, CNN, and NBC News.  Also, AltaVista obtained results from about 1,000 polling stations, photographed them, analyzed them and then sent the results around the world. They also showed a landslide: 66 percent for González, 31 percent for Maduro. Debt in Africa Has Increased Substantially Africa's debt burden has increased substantially just since 2010. Reuters reports that Zambia, Ethiopia, and Ghana are now in default. And at least 20 African nations have taken on a heavy debt burden, as defined by the IMF, a condition that did not exist for these countries just 10 years ago.  Although, none of these nations are in as severe a condition as the United States - with a debt-to-GDP ratio — now at 122% up from 64% in 2008.  Japan, Sudan, Lebanon, and Greece have a higher debt-to-GDP ratio than the United States. “The debtor is servant to the lender.”  Proverbs 22:7. College Enrollment Dropping Undergraduate college enrollment has dropped another 852,000 students since 2019 - a 4.6% drop. Christain colleges are taking the hit.  A recent survey of 50 Christain colleges found 36 out of 50 Christian colleges had a net decrease in tuition income over the last 5 years, as reported by wng.org.  The college bubble has pretty much burst. . .In Minnesota, only 57% of high school graduates signed up for college on graduation, in 2022. That's down from 82% in 2011. Chik-fil-A Worker Fends off Armed Robber An armed robber broke into an Atlanta Chik Fil A last month, levelled a gun at employee Kevin Blair. . . and told him he was going to die if he didn't open the restaurant safe. By God's mercies, Blair fought off the armed robber for several harrowing minutes — in a desperate fight for his life, finally pushing him out the door of the restaurant. Blair talked about the struggle in an interview with WXIA TV — BLAIR: “I broke his glasses. I put my thumb into his eye.  He hit me several times.” WXIA TV ANNOUNCER: “Fighting off the intruder, the whole time thinking: BLAIR: “I want to see my kids.  That's really it.  Get through this.  Go see my kids.”  Police have arrested suspect Tommie Lee Williams in connection with the assault. And that's The World View in 5 Minutes on this Friday, August 2nd, in the year of our Lord 2024. Subscribe by iTunes or email to our unique Christian newscast at www.TheWorldView.com. Or get the Generations app through Google Play or The App Store. Filling in for Adam McManus I'm Ean Leppin. Seize the day for Jesus Christ. 

On this episode of Appointed, Senator Kim Pate speaks with Ottawa City Councillors, Theresa Kavanagh and Marty Carr. This fabulous duo successfully presented a motion on July 10, 2024, supporting a Guaranteed Livable Basic Income. They were inspired by the Ottawa Board of Health June 17, 2024 resolution supporting a Basic Income Guarantee for all people over the age of 17 as a means of addressing poverty, the number one social determinant of ill health.Kim and the Councillors discuss the importance of a Guaranteed Livable Basic Income, the potential it has to support safety, autonomy, the social determinants of health, and other inequities faced by Ottawa citizens and Canadians more broadly.Councillor Carr represents the area of Alta Vista, and Councillor Kavanagh is the councillor for the By Ward region of Ottawa.__________________________________Senator Pate's Guaranteed Livable Basic Income Fact Sheets can be read hereCity Council Motion to Support a Guaranteed Basic Income for Canadians available here & hereOttawa City Council Backs Basic Income can be watched hereBill S-233, An Act to develop a national framework for a guaranteed livable basic income can be read hereAn Op-Ed by Councillor Marty Carr can be found here

Here's what's happening today: CrowdStrike Update-via ABC NewsSecret Service Director Testifies-via AP NewsHarris for President-via AP News, NY TimesWatch this episode as a video at www.kimmoffat.com/thenewsRegister to vote, or check your registration at wearevoters.turbovote.orgTake the pledge to be a voter at raisingvoters.org/beavoterdecember. - on AmazonSubscribe to the Substack: kimmoffat.substack.comA full transcript (with links) is available at kimmoffat.com/hwh-transcriptsAs always, you can find me on Instagram/Twitter @kimmoffat and TikTok @kimmoffatishere

In this episode, host Jeremy Hitchcock sits down with Don Dodge, a tech and investment veteran with a storied career spanning groundbreaking startups and tech giants like Google and Microsoft. Join us as we explore Don's journey, from his early days at Forte Software, Alta Vista, and Napster to his pivotal roles at Microsoft and Google. We dive into his experiences with venture capital, the evolving landscape of startups, and his unique insights into dual-use technologies bridging the commercial and defense sectors. Don also shares his excitement about joining New North Ventures, emphasizing the opportunities in national security and commercial crossovers. This episode is packed with valuable lessons for entrepreneurs and investors alike, highlighting the importance of team dynamics, market understanding, and the changing game of building successful tech companies. 

In this episode, Hailey uncovers one of Lake Geneva's most beautiful attractions–the Geneva Lake Shore Path–that holds scattered hidden gems and untold history from centuries ago. On top of its natural beauty and design, showing off Geneva Lake, property owners go a step further adding unique features along the way. The Geneva Lake Shore Path most definitely shows off Lake Geneva's stunning beauty today, but it also holds much of the area's history. As Hailey leads the journey along the Path, she reveals more about Lake Geneva's roots, discovering the history first-hand.Read the blog here: https://discoverwisconsin.com/geneva-lake-shore-path-hidden-gems-history/Geneva Lake Shore Path: https://www.visitlakegeneva.com/things-to-do/shore-path/; The Miracle Path: https://www.facebook.com/themiraclepath/The Bobber: https://discoverwisconsin.com/blog/The Cabin Podcast: https://the-cabin.simplecast.com. Follow on social @thecabinpodShop Discover Wisconsin: shop.discoverwisconsin.com. Follow on social @shopdiscoverwisconsinDiscover Wisconsin: https://discoverwisconsin.com/. Follow on social @discoverwisconsinDiscover Mediaworks: https://discovermediaworks.com/. Follow on social @discovermediaworksVisit Lake Geneva: https://www.visitlakegeneva.com/. Follow on social @visitlakegeneva

Hat Tip to this week's creators: @edzitron, @bysarahkrouse, @dseetharaman, @JBFlint, @packyM, @KamalVC, @VaradanMonisha, @Claudiazeisberg, @IDTechReviews, @cjgustafson222, @NathanLands, @psawers, @lightspeedvp, @jaygoldberg, @avcContents* Editorial: Dear Sam, A Letter from a Founder to a Founder* Essays of the Week* Sam Altman Is Full Of S**t* Behind the Scenes of Scarlett Johansson's Battle With OpenAI* Sky voice actor says nobody ever compared her to ScarJo before OpenAI drama* Better Tools, Bigger Companies* The Pervasive, Head-Scratching, Risk-Exploding Problem With Venture Capital* Video of the Week* OpenAI vs Gemini 1.5* AI of the Week* Does AI have a gross margin problem?* OpenAI and Wall Street Journal owner News Corp sign content deal* Scale AI Raises $1B In Accel-Led Round; Hits $13.8B Valuation* The Awful State of AI in California* News Of the Week* It's Time to Believe the AI Hype* The 49-Year Unicorn Backlog* Humane, the creator of the $700 Ai Pin, is reportedly seeking a buyer* NVIDIA CRUSHES EARNINGS, AGAIN* Startup of the Week* SUNO'S HIT FACTORY* Warpcast of the Week* Be GenerousEditorial: Dear Sam, A Letter from a Founder to a Founder.This week let's break the pattern and write this as a letter to Sam Altman.Dear Sam,It's been a swings and roundabouts week for you at OpenAI.I had a week like that in the spring of 1998. I was at Internet World launching RealNames to the world. RealNames invented paid clicks on keywords. Our first partner was AltaVista, and Google was our second—calling the feature "I'm Feeling Lucky."It was the simplest technology ever. We had a keyword, bought by a customer. An example might be Disney buying "Bambi." They would buy it in every country and language they wanted and point it to a specific URL in each place. Search engines would look at the keywords you typed in (later browsers too) and if RealNames had it as a paid keyword, they would send the user to the site, with no search results. Just a direct navigation. RealNames got paid for the customer sent.At the launch, we used the example of the keyword “Bambi” to show how superior our keywords were compared to domain names. In those days, Bambi.com pointed to a porn site. Our launch demo showed that typing "Bambi" went to Disney, but typing "Bambi.com" did not. All was well except we altered our network settings the eve of the launch, and when we demoed the use of "Bambi" at the launch, it (you can guess) went to the porn site.Journalists wrote about RealNames as a scam and bad actors.Luckily, we had great partners, and within 12 hours the network issue was fixed, and all was well. But for 24 hours, I felt like the world was collapsing around me. On the one hand, we launched our company, mostly to great acclaim; on the other, we were being destroyed in the tech media.Sam, I know how this week must have felt. Your decision to pull the ‘Sky' voice was right. And despite the horrors of the first 24 hours, this will pass.That said, you mismanaged this entire thing. I'm sure you acted in good faith in wanting to embrace the “Her” meme. It is a good idea. And ‘Sky' was a good effort.It seems clear you had spoken to Scarlett Johansson and failed to reach an agreement. I'm prepared to believe you could not react fast enough to change the voice prior to the demo.But once it went awry, you needed to do more than wait for a legal challenge before pulling it, and you needed to say something before the actress. Not doing so means that many people, probably most, think you did the entire thing on purpose.Clearly, you did not preconceive this. If you did, then the fact that you were happy to pull the voice, and your knowledge that the actress was not prepared to have her voice used, would have stopped you before it got as far as it did. You would be very reckless to have thought you could get away with using a voice like hers without her permission.So, you need to either go on the record and get this behind you or ignore it and hope it goes away. I think now we have ‘ScarJo' as a word, the latter might prove difficult.Best Regards,Keith (A fellow Founder)Beyond ScarJo there are some great essays this week. Pack McCormick writes about why AI will lead to more jobs and bigger companies. In framing his case he says”Technologies are tools. I don't mean that in the normal way that people mean it to say that technology is neither good nor bad.Tools are good.Humans can build better things with tools than they can without them.But tools aren't the point. They're tools.Tools lead to new possibilities and those lead to new endeavors. Read his essay below.And a team made up of @KamalVC, @VaradanMonisha, @Claudiazeisberg have penned an essay called ‘The Pervasive, Head-Scratching, Risk-Exploding Problem With Venture Capital'. The main thesis is about investing in private companies versus public companies. They have a great graphic showing that the range of outcomes in Venture Capital is very wide compared to other asset classes:Venture Capital's top percentiles out-perform other asset classes, but most do not. The safest asset class is global equity (public company stock).Building on this they show that large Venture investors that invest across 500 or more companies can compete with less risky assets by diversification.This depicts a simulation of a manager doing 15 deals, compared to 500 and shows more deals equals less risk.I recommend reading the full piece, linked in the contents above and the headline below. I think they are right, but there is a better way of derisking. The advice they give below is better than traditional venture capital, but that is a low bar:To de-risk venture capital, CIOs simply need to acknowledge that VC math is different from public markets math. The importance of low-probability, excess-return-generating investments means that proper diversification requires a portfolio of at least 500 startups.It will take work to assemble such a portfolio. It is hard to do by investing directly. Current funds and funds-of-funds are rarely designed with diversification in mind. Instead, they concentrate funding in a small subset of ultra-popular entrepreneurs, sectors, and geographies, which risks driving down returns on capital, leaving higher-return strategies underfunded.Investors who allocate and diversify their funds wisely and accept the evidence will not only achieve better and less-volatile returns, but will also ultimately nudge GPs to finally design diversified funds.In my day job - also about de-risking venture - we use AI to reduce risk, removing companies that are highly unlikely to be successful. The remaining companies (about 7% of the full set of venture backed companies) out-perform the market in a narrower band of outcomes:Here is how the SignalRank Index compares to the S&P500 and the NASDAQ. We assume an investor puts $1 into the S&P, the NASDAQ and The SignalRank Index in each year from 2014-2019 and then show the returns from each (average and median in the case of SignalRank).The median outcome from venture investments is that the investor loses money. The average is a lot better. But almost no managers achieve the average. By using AI to reduce risk we get the average outcome in 2014 to be 4.31x the investment (the white numbers), compared to the S&P500 1.39 and the NASDAQ 1.89. SignalRanks Median outcome is 2.24.De-risking venture capital is important and the writers of the essay show that it is possible to de-risk by diversification. But we can do even better by both diversifying and using data intelligence to remove downside outliers.I will leave you with that thought. More next week This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit www.thatwastheweek.com/subscribe

On this episode, I covered "Sweet Sixteen" from season 4. Ron urges Leslie to take some time off of work to focus on her campaign; the gang throws a surprise birthday party for Jerry... but forget to invite him, Ann and Tom have another stupid argument; and Chris and Andy bond over Champion. Also, Orthodox Easter, more cicadas, and Alta Vista references. FILLER: Rebel Moon 2, Civil War (both 2024) CONTACT: citizensofpawnee@gmail.com and Instagram @citizensofpawneepodcast and @parksrecmemes Dance of Joy Podcast @danceofjoypod Intro/general nonsense (00:47) Rebel Moon 2 (13:42) Civil War (16:55) "Sweet Sixteen" (28:02) New episodes every Tuesday, please share!

Join Ryan for part one with the legendary Mark Roberge, former CRO of HubSpot and current venture capitalist. They dive deep into the world of AI companies, discussing the immense opportunities and potential pitfalls. Mark shares his unique perspective on identifying promising AI start-ups and provides invaluable insights for entrepreneurs and investors alike. Join 2,500+ readers getting weekly practical guidance to scale themselves and their companies using Artificial Intelligence and Revenue Cheat Codes.   Explore becoming Superhuman here: https://superhumanrevenue.beehiiv.com/ KEY TAKEAWAYS AI is a generational technology, potentially bigger than the internet, but currently in a massive hype cycle similar to the dot-com era. Most current AI ideas are basic integrations into existing workflows, not reimagining the workflow, and are likely to be disrupted. Joining an AI company now provides valuable experience, even if 90% fail, as the winners will scoop up experienced talent. The AI technology creating operational efficiency in the short term may not be the same as the disruptors redefining industries over the next decade. Understanding prompting, the foundation of AI language is crucial for developing micro-workflows, agents, and eventually autonomous swarms. The innovator's dilemma and appropriate beachhead selection are key concepts for AI startups to consider when building for the long-term vision. RevOps, once seen as supporting humans, may flip to AI doing the job with humans tweaking and refining. Technical issues like hallucinations and latency will likely be ironed out as the technology matures, revealing AI's true potential. BEST MOMENTS "I've never had more conviction for my students that come up to me. Like, 'Hey, what's up? Professor, like what, what should I do? Like, I want to be, I want to go into startups. Like, what should I do?' And I, I've always had like different opinions. I was like, do you have to go to AI? Like every tech company is going to, in like six years is going to be native AI. You have to get that experience now." "I think all the tea leaves are similar to like 1997. Let's not forget that in that time, we dialled up to the internet using AOL, launched a Netscape browser and did searches through AltaVista, right? Where are those companies? Lycos, right? Is, is, is, yeah, is OpenAI Netscape or is it Google? Like we got to figure that out, right?" "The AI technology that's going to create significant operational efficiency and growth this year and next for companies, especially these bigger companies that my CROs are from is not going to be the trillion dollar new company that disrupts the space." "I've kind of come to the conclusion that the AI companies, vendors, technology that is going to create the most operational efficiency, even for our start-ups in the next year is not going to be the same AI technology company, et cetera, that becomes the disruptor over the next decade and redefines the sales and martech sector." "RevOps has become huge, didn't even exist 15 years ago. And for the most part, RevOps is seen as an organization that supports the humans to do their job. And that flips with what you're saying. Is, at some point, it flips where RevOps does the job and humans there to tweak it." Ryan Staley Founder and CEO Whale Boss ryan@whalesellingsystem.com www.ryanstaley.io Saas, Saas growth, Scale, Business Growth, B2b Saas, Saas Sales, Enterprise Saas, Business growth strategy, founder, ceo: https://www.whalesellingsystem.com/closingsecrets

Ben Cates, NewsAdvance.com On No Hitters At Altavista, Special High School Lax Talent by Ed Lane

Amie Brunkow owns Alta Vista Meat Company in Alta Vista, Kansas. She joins the Inside Ag podcast to share her passion for business and meat sciences. Amie gives a glimpse into her life as a business owner, discloses the unique features she provides local ranchers and showcases the many benefits of a traditional meat locker. Find Alta Vista Meat Company at Alta Vista Meat Co

Here's what's happening today: FBI Informant Had Russian Ties-via AP NewsSCOTUS Leaves Powell Sanctions in Place-via CBS NewsNikki Haley Will Stay in the Race-via NBC NewsMissing 11-Year-Old's Body Found-via ABC NewsNex Benedict-via The IndependentA full transcript (with links) is available at kimmoffat.com/hwh-transcriptsAs always, you can find me on Instagram/Twitter @kimmoffat and TikTok @kimmoffatishere

This week, Jordi and G3 welcome Sultan Meghji back to the podcast. As long-time listeners know, when Sultan is the guest, green marbles start flying everywhere.This episode continues in that tradition, but it focuses narrowly on cybersecurity, as Jordi and Sultan believe that cybersecurity isn't getting the attention it deserves by the markets. In fact, while generative AI may have been the key factor driving the equity markets higher last year, 2024 is a new ball game. And cybersecurity – not LLMs – could represent the dominant narrative this year. But according to Jordi, the best way to participate in the “cyber surge” is through Bitcoin.Please check important disclosures at the end of the episode.Timestamps:What is Jordi's point of view on the national threat posed by TikTok? [9:12]How confident is Sultan on the U.S. possessing superior abilities to defend against cyber attacks? [21:46]Is the strong market performance of leading cybersecurity names a reflection of a growing concern of a cyber attack? [25:14]What is the role of decentralization, blockchain, and crypto in combatting cyber attacks? [34:05] What advice does Sultan offer on how to prevent being hacked? [42:18]Resources:Frontier Foundry's websiteUK report on state-sponsored cyber attackersFirst of its kind AI heist AI Voice ScamsWhat ever happened to Alta Vista?Disclosures: This podcast and associated content (collectively, the “Post”) are provided to you by Weiss Multi-Strategy Advisers LLC (“Weiss”). The views expressed in the Post are for informational purposes only and are subject to change without notice. Information in this Post has been developed internally and is based on market conditions as of the date of the recording from sources believed to be reliable. Nothing in this Post should be construed as investment, legal, tax, or other advice and should not be viewed as a recommendation to purchase or sell any security or adopt any investment strategy. Past performance is no guarantee of future results. You should consult your own advisers regarding business, legal, tax, or other matters concerning investments. Any health-related information shared on the podcast is not intended as medical advice or for use in self-diagnosis or treatment. Please consult a qualified healthcare professional before acting upon any health-related information on the podcast. Weiss has no control over information at any external site hyperlinked in this Post. Weiss makes no representation concerning and is not responsible for the quality, content, nature, or reliability of any hyperlinked site and has included hyperlinks only as a convenience. The inclusion of any external hyperlink does not imply any endorsement, investigation, verification, or ongoing monitoring by Weiss of any information in any hyperlinked site. In no event shall Weiss be responsible for your use of a hyperlinked site. This is not intended to be an offer or solicitation of any security. Please visit www.gweiss.com to...

Ben Cates, NewsAdvance.com On Altavista Bond With High School + LCA To LU by Ed Lane

Small-scale technologies like solar panels and on-site battery storage are empowering homeowners, businesses, and entire communities to become more energy independent. In this episode, we talk with Dick Bakker, Director of the Ottawa Renewable Energy Co-operative (OREC), about his personal switch to solar power, OREC's role as an advocate for renewable energy, and more. Related links   Ottawa Renewable Energy Co-operative: https://www.orec.ca/ Dick's article: https://www.orec.ca/wp-content/uploads/2024/01/Vistas-Jan-2024-2pages.pdf Hydro Ottawa: https://hydroottawa.com/en To subscribe using Apple Podcasts:  https://podcasts.apple.com/us/podcast/thinkenergy/id1465129405   To subscribe using Spotify: https://open.spotify.com/show/7wFz7rdR8Gq3f2WOafjxpl   To subscribe on Libsyn: http://thinkenergy.libsyn.com/ --- Subscribe so you don't miss a video: YouTube   Follow along on Instagram   Stay in the know on Facebook   Keep up with the posts on X --- Transcript: Dan Seguin  00:06 This is thinkenergy, the podcast that helps you better understand the fast changing world of energy through conversations with game changers, industry leaders, and influencers. So join me, Dan Seguin, as I explore both traditional and unconventional facets of the energy industry. Hey, everyone, welcome back. In today's era, there is a growing desire among residents to take charge of their energy consumption not only to manage costs, but also to actively generate their own power. Traditionally, electricity has been generated at large power plants and transmitted over extensive distances to homes and businesses, leaving consumers with little influence over the source of their electricity. However, advancements in small scale technologies such as solar panels and onsite battery storage are empowering homeowners, businesses, and entire communities to become energy self-sufficient. In addition to these technologies, the integration of smart thermostats, vehicle to grid charging stations and heat pumps is further reshaping the dialogue around energy generation, conservation, and being active participants in an emission free future. Today, Canadians have the opportunity to take control of virtually every aspect of their energy consumption and interaction. The landscape of energy is evolving, putting the power back into the hands of individuals and communities alike. So here's today's big question. What role will innovative technologies and decentralized energy solutions play in shaping the future energy independence for individuals and communities? Joining us today is Dick Bakker, an Ottawa area homeowner that recently published an article about his experience installing a solar panel system on his home. Dick is also the director of an auto renewable energy cooperative, so brings a unique perspective on other small scale renewable projects his organization has been involved in. Dick, welcome to the show.   Dick Bakker  02:34 Thank you very much.   Dan Seguin  02:36 Now, you recently published an article about the process of installing solar panels on your home. What inspired you and your family to make the switch to solar power? And why did you decide to share your experience in this article,   Dick Bakker  02:52 It was a long process, I actually had to go back to 98 when the ice storm hit Eastern, Northeastern the US and Canada. At that time, I was working in the internet equipment business. And I watched the world stop and became fascinated with how it happened. And that caused a restart and an interest in energy that I had from the 70s during the oil crisis. And I found the electricity grid to be very similar to the telecom industry, then in oh three. So in 98, we were out of power here for seven days. People across the road had power, so we're okay, but we just didn't have power in our house. We just live with them. Then in Oh, three the trees in Ohio shut down North America again. And I couldn't believe that that could happen again. But at that time, Ontario was the last jurisdiction in North America to come back on stream fully. It took us almost four weeks for the whole province to come back. But Quebec was lit up okay. And they actually had bars on the hunt in the hall side looking at the lights going off in Ontario. But I asked myself why the heck is this. And I realized very quickly that it was because of our big nuclear plants. They're so big, and so rigid. The premier at the time couldn't get the citizens of Ontario to turn off their air conditioning units because of the heatwave we were in. And Quebec was unaffected. Well, why? And I learned it is the centralized nature of Ontario's power grid, and the lack of demand management that we have here. Because of that, anyway, I became fascinated with electricity regulations, and all of that. And that eventually led to me becoming part of the Ottawa renewable energy cooperative, where I learned through hard knocks the problems of the electricity system, the predatory protective regulations, and this new idea called distributed energy resources. Anyway, long and short, I finally realized that we needed to do something at home. And that came about eventually to us putting solar on the house when certain regulations changed. I wrote the article so that I could share my experiences of how the Ontario electricity system works, what we can do about it, and I wrote it for the local community newspaper, the VISTAs, I live in Alta Vista. And through my work at Oreck and my own interests in this house and making it more efficient and cheaper to run, I learned an awful lot and that information should be spread, I thought, okay, Dick,   Dan Seguin  05:24 In your article, you mentioned the challenges you and your neighbors face during the durational storm that hit Ottawa in 2022. And the tornado in 2018. How did these experiences influence your decision to invest in solar and other distributed energy resources specifically?   Dick Bakker  05:46 Well, specific? A lot. They were instrumental. So I've lived in this house for 30 years and Alta Vista, we've been out of power for longer than five days, four times. In the 98 ice storm, the 2003 trees in Ohio that fell over and shut down North America, 2018 Tornado, and the 2022 Derecho. And then there was also another big ice storm in the spring of 23. But we'll leave that aside, it didn't affect us too much. So after the 2022 Derecho, my neighbor and I were discussing what had happened, were both out for 10 days, and he was beside himself because he didn't have anywhere to go. They want to get off the grid completely. And he knew I was involved in the Ottawa, renewable energy cooperative, or Rec. And I told him, You can't go off grid because it's not worthwhile. It's not effective, you're getting a subsidized price of electricity, which didn't, he didn't like hearing that. But I said, you're just we're just not paying enough for our electricity. We're getting it so cheap, it doesn't make sense to put solar on your roof. Besides, we both had trees in our cell site. So that was then I explained to him the centralized nature of the grid. 60% of our power comes from three nuclear sites. Bruce Darlington and Pickering. Pickering being 14%. The pension funds like to invest in big centralized power plants, big shiny objects that the world can see. And the long lines that bring the power from way over there to our little corner is like a cash stream that the incumbents want to keep. They're not interested in distributed energy resources, or D are spread around. But that's where we should be going that time in 2022. Knowing what I knew of the regulations and the orientation of the provincial government, I couldn't see ever having the potential to put solar on your house. Sorry, I couldn't see the financial justification of putting solar on the house. And on top of that, the present government is subsidizing our electricity bills to the tune of 7 billion a year five and a half billion of that is going to general subsidies to the middle class and upper class not targeted to the poor. So at some point that's going to rise. The rating agencies will correct that by threatening to downgrade Ontario's credit rating but all that to say it's still subsidized, so it's not worth putting it on. Then in 2023, January, the Ontario government came out with some changes and started encouraging net metering and local generation.   Dan Seguin  08:28 Okay, now, did you also discuss the changing landscape of Ontario's electricity rules, specifically mentioning the Ontario Energy Boards directive in 2023? What changed that, in your opinion, helped to facilitate the adoption of solar power and what challenges still exist for homeowners today?   Dick Bakker  08:51 Thanks, Dan. That directive from the Ontario Energy Board and 23 was was a game changer for the province. I don't think they realize what potential they unleash them. So from 2018 When the Conservative government took power, they had a big grid only mentality. They wanted big power plants and long lines to deliver the power to the homes and the rules around net metering, which is the only way you can put solar on your house and stay connected to the grid. That's where you generate power, consume it yourself, and trade credits for your over summer for your summer overproduction for your winter consumption or purchases from the grid. So that pricing scheme was basically rigged against the homeowner because homeowners were forced to go to the tiered pricing scheme. So just on that situation, and up until 2023. Net metering wasn't very cost effective because of the pricing, but it could work. Technically, the grid acts as a battery so you're never out of power. So that rule kept me way from thinking of solar on my house. Also, I had trees to the south of the house. So the best place to put the solar panels wouldn't be productive. I don't want to cut the trees down, because that keeps my air conditioning costs low, and they're nice. But then in 23, the province changed the rules around net metering, and came up with an ultra low overnight rate. So the key thing about net metering, they said the local distribution companies would have to give the net metering customer the option to pick their rate class. So you go to a time of use rate if you wish. And then you get value for your time value of electricity. So if you're producing an high rate, you get the high rate in your credits. Okay, so that's good, then they came up with an ultra low overnight time of use rate, third rate class to encourage every user to charge at night, not during the evening dinnertime when everybody's turning on lights and eaters and all their devices. So they want to reduce consumption during the peak hour, and increase consumption at the low hour. And if you produce solar during the four to 9pm, period at 28 cents, that's what you pay, you get credits for 28 cents, that is much better economics for the homeowner, the end user and the solar producer. That's when I realized that my house was actually ideal because I've got a very low sloped roof. The South Side is full of trees, but the north side is clear. And the North side's going to produce more during the four to eight o'clock pm in the summertime at 28 cents. So one hour of that can offset 10 hours at the 2.8 cents for the low rate. So that was one thing. The other thing is I have an EV. We have heat pumps. We just installed a heat pump water heater, so I can time shift my consumption to the low overnight rate, I think it's pretty good. I still think the cost of electricity is going to rise more. So my return on investment is only going to improve because putting all of this in is an insurance policy against that rising cost of electricity. You also asked what are the continuing challenges? The challenges for solar? on the residential side are buildings and trees. How's the building built? Which way are the roofs pointing? Where are the trees? What kind of shading do they throw? But the good thing is that in the summertime, the sun is very high in Canada, so the sun will come straight down more or less. And in the wintertime when there's no snow on your roof. Or even if there's a little bit of snow on the roof. Solar production is marginally better because it's cold. So the physics is better. So there's still lots of opportunity for solar even in this cold northern climate. The challenges are of course buildings and trees to a certain point the supply chain there aren't enough installers, electricians to do all the work that should be done can be done hydro Ottawa, a staff just to get the installations done the upgrades for the grid. But hydro Ottawa needs Ottawa residents to spend this money on their own Diyar so that you can meet your new targets for the year. So I think people who do this on their own are doing it for themselves, but indirectly they're doing it for the betterment of the overall grid, driving down the cost of electricity. Solar does not drive up the cost of electricity when producing nuclear plants drives up the cost of electricity. Okay.   Dan Seguin  13:40 Could you maybe provide more details on the cost and capacity of your solar panel system? What were the economic aspects of your investment, including any government incentives or rebates that may have influenced your decision?   Dick Bakker  13:56 So in my specific installation, I have 37 panels in total. 24 of them are on the north slope and 13 panels on the south slope. So total DC kilowatt of 14.43. That's going through a nine kilowatt inverter. I have no panels on the south slope because there are three big trees there. If I had panels there, it would probably be a third smaller for the same generation. So over 12 months, I expect to generate about 10,246 kilowatt hours. That's 78% of 2020 two's consumption and my electricity consumption includes 90% of our driving because I have an Eevee and a plug in hybrid Evie 90% of our driving 100% of our cooling 40% of our heating a little more than 40% this year because it's a warm winter and 100% of our lights and appliances. So I've got a gas station on my roof and I've got a furnace on my roof effectively because of the ultra low overnight time of use rate. I am confident that with time shifting I can cover 100% of my electricity purchases, not my connection charges 100% of my electricity cost with something like 78% of my electricity kilowatts, because of the time shifting between ultra low and peak rate, the overall cost was $30,478 for the equipment, plus HST electricity upgrade to 200 amp service, some internal wiring changes, and I reached angled under the panels on the north and east, I didn't do the South because it doesn't quite need it effectively, I future proof my house for 30 plus years of electricity, I've given myself 30 years plus of electricity, price insurance and forced savings. And I predict that the credit rating agencies at some point will force the province to reduce the subsidies we're giving to the middle and the upper class and electricity costs. And that'll drive up the electricity rates a little bit, not massively, and I'll be protected from that. or whoever's living here because I'm getting old. So I think the house value of homes that have solar are going to hold their value better than a new kitchen cabinet or a new, new whatever that the new owner pulls out and replaces, you know, you're not going to be replacing solar on a roof if it's reducing your utility bills.   Dan Seguin  16:23 Okay, now our batteries, shifting your energy use away from daytime usage, or other distributed energy resources a consideration?   Dick Bakker  16:33 Well, that's a very good question, because the one thing I haven't done in the house yet is put a battery and a disconnect Island. And that's the next thing I'm going to look at during the summertime, I do these things one at a time to make sure they work and see how they operate. So the next thing will be a battery probably in the garage, if it's appropriate. And the not sure the proper technical term islanding device to allow me to operate separate from the grid. And if I ever buy another car, it'll be an Eevee with to a charging, so that I'll be able to charge my house and the battery over the course of the year, so the battery will be there for a disaster. But over the course of the year, I'll be able to draw power from the solar on the roof, and from the grid at the low rate stored and discharge it to the grid during the peak rate. So that makes my neighbor's grid a little more resilient. And in a crisis, I can be Island as opposed to the noisy gas generators that are sitting around my neighborhood.   Dan Seguin  17:37 Shifting gears a bit now as the director of the auto renewable energy cooperatives since its creation in 2009. Can you share how it works? And what are some of the projects that your coop has built?   Dick Bakker  17:54 Sure, certainly. So OREC is a for profit, renewable energy Co Op that enables residents of Ottawa to be restricted to Ontario by certain rules that I won't get into. So it allows residents of Ottawa and mostly Eastern Ontario but Ontario to benefit from distributed energy resources in their own region, we build our own renewable energy generation. Presently, solar and wind, energy conservation assets, commercial building, lighting installation, retrofit projects that keep the electrons jobs and profits local. So we have 22 solar systems in place now, most of them or the feed in tariff contracts. Three of them are net metering projects, one at the Museum of Science and Tech, two at the French Catholic High School Board, Mere Blue and Paul Desmarais. And then 18 other feed in tariff contracts where we have a contract to sell the power to the grid. At a net metering project. We sell the power to the building. Then we also have two wind projects down in southwestern Ontario and three energy retrofit projects. We had five but two of them have finished their contractor. So the solar projects are on housing coops, burns, schools, museums, factories, and two of them are I'd say medium sized ground mounts, 500 kilowatt ground mounts, the two wind projects. One is a 2.3 megawatt project at Tiverton, just outside of the Bruce nuclear plant and a little funny story I like to tell everyone is that the Bruce nuclear plant doesn't supply power to the neighborhood. All the electricity from Bruce nuclear goes to Toronto on the transmission lines because they connect it to the distribution grid and Temperton that blows all the light bulbs so they feed Toronto and then it trickles all the way back to Tim Burton. The wind project that we have outside of Tim Burton is a standalone turbine and it feeds the distribution grid. So should heaven forbid should Bruce nuclear go down? Some of the people will have electricity coming from our wind turbine. The people that are working at Bruce nuclear will have power at home, not because of the nuclear plant. The second wind turbine is an 800 kilowatt project in Zurich directly south of there. That's a wonderful area for wind. Most of the wind projects in that area are large projects owned by American pension funds, feeding Toronto, all of the power is going on the transmission lines. So getting back to Oh, Rick in general. So we have solar wind and lighting retrofits at the IRA center, condo, and housing coops. All of our projects are revenue generating with proven technologies and solid counterparties. So pretty comfortable with the security of those assets. The board is made up of pretty experienced people, engineers, lawyers, business development, accountants comms people. I'm a bit of a generalist. But I have worked in telecom and technical fields my whole life, not as an engineer, we have 980 members, 500 of them, about half of them have invested over $11 million in equity and debt in our project since we started. And we've paid dividends every year since 2013. When our first project came online, we had repaid to our members over 3.5 million in dividends, interest and capital repayment with very little outside debt, we'd rather pair members than banks, no offense banks, but we want to keep the money within the family within the community. Our main function is to act as an investment cooperative for our members. So we spend most of our time looking for projects to build and or buy, and then raise the community capital to build, operate, repeat, get more projects, raise more capital, pay out the dividends and capital. But we do have to spend an awful lot of money on advocacy work to change the regulations, or maintain whatever regulations are, to promote distributed energy resources of all types. But the second core function that we want to do more of is utilize the knowledge of our 1000 members and create them. It's happening already organically, but we want to have more regular information sessions between our members who are doing things like I just did. We have the largest concentration of any 1000 People in the Ottawa Valley in the province. I think of people who have D er installed in their homes. So we have a lot of end users, battery users, people with knowledge of heat pumps and stuff like that. So we are a group of friends with knowledge of the ER.   Dan Seguin  22:34 Okay now, Dick, when did things really take off with the co-op? And are members seeing dividends?   Dick Bakker  22:42 Well, that's a good question. Because the first offering that we raised was in 2012. And we didn't know how it was going to go, it actually went better than we expected, our minimum requirement was to raise half a million dollars. And in those nine weeks that we had, we raised $970,000, and more cash than we actually needed for what we had to do. And ever since then, we've we're now on our 10th Raise, each raise has gone better than expected. We've always raised more cash than we had projects at that time. So for a period there, we were building up too much cash and didn't have enough projects for them. So projects come more harder than the money or the members, the membership has grown very well. And the equity in the cooperative has been very good. And I'm also proud to say that we've paid dividends every year, since 2013. In the last couple of years, it's been 4%. We'd like it to be higher, but we've had to build everything from scratch without any outside cash. We've just started our latest raise, it's going to close on August 28, I believe. And we're looking for new members with new equity, and that equity can be RRSP or TFSA. It's an investment in the portfolio of 27 existing projects, and the new projects that we're going to be building in the coming year.   Dan Seguin  24:11 Now, let's talk about the changing relationship between electricity consumers and producers. How do you see this evolving in the coming years? And what role do you think individuals and communities will play in the broader energy transition?   Dick Bakker  24:32 This is going to be the biggest change in our society in the coming years. I think we're going to move from being ratepayers with very little agency beyond paying our bills and turning off lights to prosumers or producer consumers who have the ability to produce electricity for conservation, which is what I'm doing or for profit and or for profit when the regulations in Ontario Are you allow hydro Ottawa to buy excess power from homeowners? Right now you can't. So we'll be able to conserve and profit from our assets on our roof. And we'll also be able to actively manage our consumption, again for conservation and profit. So right now we're able to reduce our demand and shift our demand from peak load to low load. But in the future, I'm pretty sure that Ontario will follow California and New York and allow for aggressive demand response programs. And what we'd like to do at some point in the future, as OREC is allow our members to pool their batteries and solar panels and air conditioners, so that we can turn down consumption as the grid gets choked or or constrained. So we just saw what happened in Alberta, they had no demand management program, they turned down some gas plants for renovation in the peak of winter, and then they got hit with a big demand. During a cold period. The only way they got out of their problem was begging their customers to turn down their home heating systems. The citizens responded, but the downtown office towers left their lights on all night. That's absurd. So going forward, I think that the LDCs will be paying people to turn down their demand, because we need the grid to be balanced. We don't need excess generation or excess demand or under demand, we need everything balanced. So a megawatt is as good as a megawatt.   Dan Seguin  26:52 Okay, thank you for that, in your opinion now. What is the city or province doing well, and what improvements need to be made? Now you gotta behave?   Dick Bakker  27:04 I'll try to behave. How long do we have? I don't want to rant. But it's hard not to. On the city site. If there's a climate emergency act like there is one, people should not be buying coffee from an idling car. Housing is energy, stopping natural gas expansion. The Better Homes program is a wonderful program of the city. Because it addresses the upfront costs of retrofitting and DTR and solar and all those things. It ties that cost to a 20 year loan fixed to the House tax bill, not to the person. I'm 68. I may not be in this house for 10 years, I tend to be here longer, but my intention and reality may be different. So we need to have the cost of long term assets spread over years. The Better Homes program says that the city should be encouraging solar and small wind for resilience purposes. Every large group should have solar and there should be wind turbines scattered throughout Eastern Ontario, not just in rural areas in batches of 50. There should be a couple of wind turbines in urban Ottawa with the proper setbacks. That's the city in the province. Every month Ontario's paying out $1.3 billion in gasoline and diesel costs. There's lots of money for the energy transition. You just have to shift it around. Let the nuclear plants run their course, don't shut them down early, but don't pour money down a sinkhole. We just announced today Pickering expansion, well Pickering retrofit, it's the oldest nuclear plant in North America. The province is in a pickle because they know the nukes will be late. The small modular reactors aren't small modular. They are big reactors, they can only go on the transmission lines. That demand is all over the province at the end of the distribution lines where we live and work and EVs and heat pumps are so just let the nuclear plants slow down or wear out. The Donsky Report to the Independent Electricity systems operator said the lowest cost of new energy in the provinces D er of all types. It's just regulations that are stopping it and it makes the province more resilient. So the province can have every city have a similar program to Otto as the Better Homes program. Secondly, remove the Ontario electricity rebate that's putting $5.5 million dollars of taxpayer money into the pockets of people who leave their lights on and put that money instead in the distribution lines allow every kind of virtual net metering in the province especially community solar gardens so that citizens could own the solar on a swimming pool hockey rink. Any facility that is used for a disaster recovery facility should be generating power day to day and then have the ability to island in a crisis and resilience See should be the first order of the electricity grid, proper costs but resiliency and localized and generally liberalize the rules around generation and distribution. Okay,   Dan Seguin  30:10 Does the co-op or its members have an objective to promote or advocate for renewable energy and distributed energy resources in the community or with local governments? Yes,   Dick Bakker  30:23 In every way, as a co op, and with other coops for community scale projects, 100 kilowatt to one or two megawatt is the size of projects that is natural for us. That's the kind of thing that citizens are going to be interested in and seeing and owning, but we are going to work in the bigger projects on the transmission side, but we're advocating for that all the time, spend a lot more time helping our members to act as individuals with information and examples, the whole idea of friends with knowledge to get them to put in their own home systems. So yes, we spend way too much time advocating on behalf of the ER.   Dan Seguin  31:03 Okay, now, are you seeing your co-op's focus areas reflected in government policy, either municipally or provincially? How do you ensure your voices are heard?   Dick Bakker  31:17 We're starting to see a focus on D er, but I'm not yet seeing action, hard, hard action on the ER except for a few exceptions. Hydro Ottawa with the IESO is right now focused on solar DERs as a conservation measure, there's a bunch of regulations around it. I won't get into that right now. So that's good. And the dusky report and the ultra low time of use rate, those are all very good things. But today, they've just announced the massive expenditure on Pickering, which locks us further into the centralized focus of the province. The orientation of all electricity grids is to build big things far away that will break at some point. We're here in Ottawa, and we see all these federal buildings, there's only a few of them that have sold on them. The federal government doesn't do a good job of buying from small organizations like us. So we've had lots of discussions with the feds, but they want to do massive things that the reporters can write about. We're advocating as ourselves and with other coops nationally and provincially in every province, because that's where electricity and Co Op law resides. And we have formed a national association called the Community Energy cooperatives Canada, which is based in Saskatoon right now and has 25 coops from across the country. The fastest growing area of renewable energy coops in Canada is Alberta because they have the most liberalized power grid. So that'll be our national voice. But it'll be a voice at the federal and more importantly, at the provincial level, because that's where electricity lives. We work a lot with the European res Co Op, who have been very successful in Europe to get the EU to pass a directive that says every citizen of the EU has the right to own, operate, store, share, and save their own renewable electricity. So if we get the federal government to encourage that, all they can do is bribe, encourage and embarrass the provinces. If we could get the federal government to pass a directive like that. That's EU directive 2018 -201. If anybody's interested, we get that kind of directive from the federal government. That'll put pressure and embarrassment on the provinces to loosen up their grids. Alberta and Nova Scotia have moved the furthest along in this area, Ontario and Quebec and Manitoba and Saskatchewan are the big laggards but we have to move that way and oh wreck with our friends in the other coops can push that. We're all voters. We're all voting with our money and our ballots, and the last thing, banks will notice the difference.   Dan Seguin  34:03 Lastly, Dick, we always end our interviews with some rapid fire questions.Are you ready?   Dick Bakker  34:12 Yes, sir.   Dan Seguin  34:13 Okay, what are you reading right now?   Dick Bakker  34:16 Well, two books, one is called treeing energy by Bill Nussey. It's all about the wonderful economics of Home DER technologies. And the other is by my favorite author Guy Vanderhaeghe, August Into Winter. And not a book but fascinating about Saskatchewan and rural Saskatchewan and Manitoba crime scene set in 39. With the Spanish Civil War in the coming world war two is the backdrop. It's great.   Dan Seguin  34:44 What would you name your boat if you had one? Or do you have one?   Dick Bakker  34:47 There ain't no easy road. Those are the words of a song I love called Jericho by Fred Eagle Smith. My wife gave me a paddle with this phrase on it a few years ago as a birthday present.   Dan Seguin  34:59 Next, who is someone that you admire?   Dick Bakker  35:01 Peggy my, my wife, mother of my children, business partner, best friend and a no BS problem solver.   Dan Seguin  35:09 Okay? What was the closest thing to real magic that you've witnessed   Dick Bakker  35:15 Birth of a child who grows into an adult who has a child. Now,   Dan Seguin  35:19 Now, as a result of the pandemic? Many of us are guilty of watching a little too much TV or movies. What is your favorite movie or show? What are you watching right now?   Dick Bakker  35:31 I'd have to say the Danish movie Borgan. It's a Danish TV series on politics and the trade offs and the personalities that shows the human side of difficult decision making. It's great.   Dan Seguin  35:46 Lastly, what is exciting you about your industry right now?   Dick Bakker  35:51 Well, the electricity industry has got the possibility to democratize energy to revitalize communities and especially rural communities. So with renewables and DDR and cooperatives, we can keep the electrons' jobs and profits local. Okay,   Dan Seguin  36:10 Dick, our listeners, if they want to learn more about you, how do they connect?   Dick Bakker  36:15 Probably the best way is to go online and check. www.orec.ca or orec website.   Dan Seguin  36:24 This is it. We've reached the end of another episode of The thinkenergy podcast. Thank you so much for joining me today. I hope you had a lot of fun. Cheers.   Dick Bakker  36:33 I did. Thank you very much, Dan. It's wonderful.   Dan Seguin  36:37 Thanks for tuning in for another episode of the thinkenergy podcast. Don't forget to subscribe and leave us a review wherever you're listening. And to find out more about today's guests or previous episodes, visit thinkenergypodcast.com I hope you'll join us again next time as we spark even more conversations about the energy of tomorrow.  

durée : 00:28:50 - Le Feuilleton - "Ça fait plus de vingt ans que j'habite ici et rien n'a changé, les pères ont transmis aux fils leurs armes, leurs maîtresses, leur mémoire."

durée : 00:28:50 - Le Feuilleton - "Ça fait plus de vingt ans que j'habite ici et rien n'a changé, les pères ont transmis aux fils leurs armes, leurs maîtresses, leur mémoire."

Songs include Tal Shia, Alta Vista and Mr. C T

We are running an end of year survey for our listeners. Let us know any feedback you have for us, what episodes resonated with you the most, and guest requests for 2024! RAG has emerged as one of the key pieces of the AI Engineer stack. Jerry from LlamaIndex called it a “hack”, Bryan from Hex compared it to “a recommendation system from LLMs”, and even LangChain started with it. RAG is crucial in any AI coding workflow. We talked about context quality for code in our Phind episode. Today's guests, Beyang Liu and Steve Yegge from SourceGraph, have been focused on code indexing and retrieval for over 15 years. We locked them in our new studio to record a 1.5 hours masterclass on the history of code search, retrieval interfaces for code, and how they get SOTA 30% completion acceptance rate in their Cody product by being better at the “bin packing problem” of LLM context generation. Google Grok → SourceGraph → CodyWhile at Google in 2008, Steve built Grok, which lives on today as Google Kythe. It allowed engineers to do code parsing and searching across different codebases and programming languages. (You might remember this blog post from Steve's time at Google) Beyang was an intern at Google at the same time, and Grok became the inspiration to start SourceGraph in 2013. The two didn't know eachother personally until Beyang brought Steve out of retirement 9 years later to join him as VP Engineering. Fast forward 10 years, SourceGraph has become to best code search tool out there and raised $223M along the way. Nine months ago, they open sourced SourceGraph Cody, their AI coding assistant. All their code indexing and search infrastructure allows them to get SOTA results by having better RAG than competitors:* Code completions as you type that achieve an industry-best Completion Acceptance Rate (CAR) as high as 30% using a context-enhanced open-source LLM (StarCoder)* Context-aware chat that provides the option of using GPT-4 Turbo, Claude 2, GPT-3.5 Turbo, Mistral 7x8B, or Claude Instant, with more model integrations planned* Doc and unit test generation, along with AI quick fixes for common coding errors* AI-enhanced natural language code search, powered by a hybrid dense/sparse vector search engine There are a few pieces of infrastructure that helped Cody achieve these results:Dense-sparse vector retrieval system For many people, RAG = vector similarity search, but there's a lot more that you can do to get the best possible results. From their release:"Sparse vector search" is a fancy name for keyword search that potentially incorporates LLMs for things like ranking and term expansion (e.g., "k8s" expands to "Kubernetes container orchestration", possibly weighted as in SPLADE): * Dense vector retrieval makes use of embeddings, the internal representation that LLMs use to represent text. Dense vector retrieval provides recall over a broader set of results that may have no exact keyword matches but are still semantically similar. * Sparse vector retrieval is very fast, human-understandable, and yields high recall of results that closely match the user query. * We've found the approaches to be complementary.There's a very good blog post by Pinecone on SPLADE for sparse vector search if you're interested in diving in. If you're building RAG applications in areas that have a lot of industry-specific nomenclature, acronyms, etc, this is a good approach to getting better results.SCIPIn 2016, Microsoft announced the Language Server Protocol (LSP) and the Language Server Index Format (LSIF). This protocol makes it easy for IDEs to get all the context they need from a codebase to get things like file search, references, “go to definition”, etc. SourceGraph developed SCIP, “a better code indexing format than LSIF”:* Simpler and More Efficient Format: SCIP utilizes Protobuf instead of JSON, which is used by LSIF. Protobuf is more space-efficient, simpler, and more suitable for systems programming. * Better Performance and Smaller Index Sizes: SCIP indexers, such as scip-clang, show enhanced performance and reduced index file sizes compared to LSIF indexers (10%-20% smaller)* Easier to Develop and Debug: SCIP's design, centered around human-readable string IDs for symbols, makes it faster and more straightforward to develop new language indexers. Having more efficient indexing is key to more performant RAG on code. Show Notes* Sourcegraph* Cody* Copilot vs Cody* Steve's Stanford seminar on Grok* Steve's blog* Grab* Fireworks* Peter Norvig* Noam Chomsky* Code search* Kelly Norton* Zoekt* v0.devSee also our past episodes on Cursor, Phind, Codeium and Codium as well as the GitHub Copilot keynote at AI Engineer Summit.Timestamps* [00:00:00] Intros & Backgrounds* [00:05:20] How Steve's work on Grok inspired SourceGraph for Beyang* [00:08:10] What's Cody?* [00:11:22] Comparison of coding assistants and the capabilities of Cody* [00:16:00] The importance of context (RAG) in AI coding tools* [00:21:33] The debate between Chomsky and Norvig approaches in AI* [00:30:06] Normsky: the Norvig + Chomsky models collision* [00:36:00] The death of the DSL?* [00:40:00] LSP, Skip, Kythe, BFG, and all that fun stuff* [00:53:00] The SourceGraph internal stack* [00:58:46] Building on open source models* [01:02:00] SourceGraph for engineering managers?* [01:12:00] Lightning RoundTranscriptAlessio: Hey everyone, welcome to the Latent Space podcast. This is Alessio, partner and CTO-in-Residence at Decibel Partners, and I'm joined by my co-host Swyx, founder of Smol AI. [00:00:16]Swyx: Hey, and today we're christening our new podcast studio in the Newton, and we have Beyang and Steve from Sourcegraph. Welcome. [00:00:25]Beyang: Hey, thanks for having us. [00:00:26]Swyx: So this has been a long time coming. I'm very excited to have you. We also are just celebrating the one year anniversary of ChatGPT yesterday, but also we'll be talking about the GA of Cody later on today. We'll just do a quick intros of both of you. Obviously, people can research you and check the show notes for more. Beyang, you worked in computer vision at Stanford and then you worked at Palantir. I did, yeah. You also interned at Google. [00:00:48]Beyang: I did back in the day where I get to use Steve's system, DevTool. [00:00:53]Swyx: Right. What was it called? [00:00:55]Beyang: It was called Grok. Well, the end user thing was Google Code Search. That's what everyone called it, or just like CS. But the brains of it were really the kind of like Trigram index and then Grok, which provided the reference graph. [00:01:07]Steve: Today it's called Kythe, the open source Google one. It's sort of like Grok v3. [00:01:11]Swyx: On your podcast, which you've had me on, you've interviewed a bunch of other code search developers, including the current developer of Kythe, right? [00:01:19]Beyang: No, we didn't have any Kythe people on, although we would love to if they're up for it. We had Kelly Norton, who built a similar system at Etsy, it's an open source project called Hound. We also had Han-Wen Nienhuys, who created Zoekt, which is, I think, heavily inspired by the Trigram index that powered Google's original code search and that we also now use at Sourcegraph. Yeah. [00:01:45]Swyx: So you teamed up with Quinn over 10 years ago to start Sourcegraph and you were indexing all code on the internet. And now you're in a perfect spot to create a code intelligence startup. Yeah, yeah. [00:01:56]Beyang: I guess the backstory was, I used Google Code Search while I was an intern. And then after I left that internship and worked elsewhere, it was the single dev tool that I missed the most. I felt like my job was just a lot more tedious and much more of a hassle without it. And so when Quinn and I started working together at Palantir, he had also used various code search engines in open source over the years. And it was just a pain point that we both felt, both working on code at Palantir and also working within Palantir's clients, which were a lot of Fortune 500 companies, large financial institutions, folks like that. And if anything, the pains they felt in dealing with large complex code bases made our pain points feel small by comparison. So that was really the impetus for starting Sourcegraph. [00:02:42]Swyx: Yeah, excellent. Steve, you famously worked at Amazon. And you've told many, many stories. I want every single listener of Latent Space to check out Steve's YouTube because he effectively had a podcast that you didn't tell anyone about or something. You just hit record and just went on a few rants. I'm always here for your Stevie rants. And then you moved to Google, where you also had some interesting thoughts on just the overall Google culture versus Amazon. You joined Grab as head of eng for a couple of years. I'm from Singapore, so I have actually personally used a lot of Grab's features. And it was very interesting to see you talk so highly of Grab's engineering and sort of overall prospects. [00:03:21]Steve: Because as a customer, it sucked? [00:03:22]Swyx: Yeah, no, it's just like, being from a smaller country, you never see anyone from our home country being on a global stage or talked about as a startup that people admire or look up to, like on the league that you, with all your legendary experience, would consider equivalent. Yeah. [00:03:41]Steve: Yeah, no, absolutely. They actually, they didn't even know that they were as good as they were, in a sense. They started hiring a bunch of people from Silicon Valley to come in and sort of like fix it. And we came in and we were like, Oh, we could have been a little better operational excellence and stuff. But by and large, they're really sharp. The only thing about Grab is that they get criticized a lot for being too westernized. Oh, by who? By Singaporeans who don't want to work there. [00:04:06]Swyx: Okay. I guess I'm biased because I'm here, but I don't see that as a problem. If anything, they've had their success because they were more westernized than the Sanders Singaporean tech company. [00:04:15]Steve: I mean, they had their success because they are laser focused. They copy to Amazon. I mean, they're executing really, really, really well for a giant. I was on a slack with 2,500 engineers. It was like this giant waterfall that you could dip your toe into. You'd never catch up. Actually, the AI summarizers would have been really helpful there. But yeah, no, I think Grab is successful because they're just out there with their sleeves rolled up, just making it happen. [00:04:43]Swyx: And for those who don't know, it's not just like Uber of Southeast Asia, it's also a super app. PayPal Plus. [00:04:48]Steve: Yeah. [00:04:49]Swyx: In the way that super apps don't exist in the West. It's one of the enduring mysteries of B2C that super apps work in the East and don't work in the West. We just don't understand it. [00:04:57]Beyang: Yeah. [00:04:58]Steve: It's just kind of curious. They didn't work in India either. And it was primarily because of bandwidth reasons and smaller phones. [00:05:03]Swyx: That should change now. It should. [00:05:05]Steve: And maybe we'll see a super app here. [00:05:08]Swyx: You retired-ish? I did. You retired-ish on your own video game? Mm-hmm. Any fun stories about that? And that's also where you discovered some need for code search, right? Mm-hmm. [00:05:16]Steve: Sure. A need for a lot of stuff. Better programming languages, better databases. Better everything. I mean, I started in like 95, right? Where there was kind of nothing. Yeah. Yeah. [00:05:24]Beyang: I just want to say, I remember when you first went to Grab because you wrote that blog post talking about why you were excited about it, about like the expanding Asian market. And our reaction was like, oh, man, how did we miss stealing it with you? [00:05:36]Swyx: Hiring you. [00:05:37]Beyang: Yeah. [00:05:38]Steve: I was like, miss that. [00:05:39]Swyx: Tell that story. So how did this happen? Right? So you were inspired by Grok. [00:05:44]Beyang: I guess the backstory from my point of view is I had used code search and Grok while at Google, but I didn't actually know that it was connected to you, Steve. I knew you from your blog posts, which were always excellent, kind of like inside, very thoughtful takes from an engineer's perspective on some of the challenges facing tech companies and tech culture and that sort of thing. But my first introduction to you within the context of code intelligence, code understanding was I watched a talk that you gave, I think at Stanford, about Grok when you're first building it. And that was very eye opening. I was like, oh, like that guy, like the guy who, you know, writes the extremely thoughtful ranty like blog posts also built that system. And so that's how I knew, you know, you were involved in that. And then, you know, we always wanted to hire you, but never knew quite how to approach you or, you know, get that conversation started. [00:06:34]Steve: Well, we got introduced by Max, right? Yeah. It was temporal. Yeah. Yeah. I mean, it was a no brainer. They called me up and I had noticed when Sourcegraph had come out. Of course, when they first came out, I had this dagger of jealousy stabbed through me piercingly, which I remember because I am not a jealous person by any means, ever. But boy, I was like, but I was kind of busy, right? And just one thing led to another. I got sucked back into the ads vortex and whatever. So thank God Sourcegraph actually kind of rescued me. [00:07:05]Swyx: Here's a chance to build DevTools. Yeah. [00:07:08]Steve: That's the best. DevTools are the best. [00:07:10]Swyx: Cool. Well, so that's the overall intro. I guess we can get into Cody. Is there anything else that like people should know about you before we get started? [00:07:18]Steve: I mean, everybody knows I'm a musician. I can juggle five balls. [00:07:24]Swyx: Five is good. Five is good. I've only ever managed three. [00:07:27]Steve: Five is hard. Yeah. And six, a little bit. [00:07:30]Swyx: Wow. [00:07:31]Beyang: That's impressive. [00:07:32]Alessio: So yeah, to jump into Sourcegraph, this has been a company 10 years in the making. And as Sean said, now you're at the right place. Phase two. Now, exactly. You spent 10 years collecting all this code, indexing, making it easy to surface it. Yeah. [00:07:47]Swyx: And also learning how to work with enterprises and having them trust you with their code bases. Yeah. [00:07:52]Alessio: Because initially you were only doing on-prem, right? Like a lot of like VPC deployments. [00:07:55]Beyang: So in the very early days, we're cloud only. But the first major customers we landed were all on-prem, self-hosted. And that was, I think, related to the nature of the problem that we're solving, which becomes just like a critical, unignorable pain point once you're above like 100 devs or so. [00:08:11]Alessio: Yeah. And now Cody is going to be GA by the time this releases. So congrats to your future self for launching this in two weeks. Can you give a quick overview of just what Cody is? I think everybody understands that it's a AI coding agent, but a lot of companies say they have a AI coding agent. So yeah, what does Cody do? How do people interface with it? [00:08:32]Beyang: Yeah. So how is it different from the like several dozen other AI coding agents that exist in the market now? When we thought about building a coding assistant that would do things like code generation and question answering about your code base, I think we came at it from the perspective of, you know, we've spent the past decade building the world's best code understanding engine for human developers, right? So like it's kind of your guide as a human dev if you want to go and dive into a large complex code base. And so our intuition was that a lot of the context that we're providing to human developers would also be useful context for AI developers to consume. And so in terms of the feature set, Cody is very similar to a lot of other assistants. It does inline autocompletion. It does code base aware chat. It does specific commands that automate, you know, tasks that you might rather not want to do like generating unit tests or adding detailed documentation. But we think the core differentiator is really the quality of the context, which is hard to kind of describe succinctly. It's a bit like saying, you know, what's the difference between Google and Alta Vista? There's not like a quick checkbox list of features that you can rattle off, but it really just comes down to all the attention and detail that we've paid to making that context work well and be high quality and fast for human devs. We're now kind of plugging into the AI coding assistant as well. Yeah. [00:09:53]Steve: I mean, just to add my own perspective on to what Beyang just described, RAG is kind of like a consultant that the LLM has available, right, that knows about your code. RAG provides basically a bridge to a lookup system for the LLM, right? Whereas fine tuning would be more like on the job training for somebody. If the LLM is a person, you know, and you send them to a new job and you do on the job training, that's what fine tuning is like, right? So tuned to our specific task. You're always going to need that expert, even if you get the on the job training, because the expert knows your particular code base, your task, right? That expert has to know your code. And there's a chicken and egg problem because, right, you know, we're like, well, I'm going to ask the LLM about my code, but first I have to explain it, right? It's this chicken and egg problem. That's where RAG comes in. And we have the best consultants, right? The best assistant who knows your code. And so when you sit down with Cody, right, what Beyang said earlier about going to Google and using code search and then starting to feel like without it, his job was super tedious. Once you start using these, do you guys use coding assistants? [00:10:53]Swyx: Yeah, right. [00:10:54]Steve: I mean, like we're getting to the point very quickly, right? Where you feel like almost like you're programming without the internet, right? Or something, you know, it's like you're programming back in the nineties without the coding assistant. Yeah. Hopefully that helps for people who have like no idea about coding systems, what they are. [00:11:09]Swyx: Yeah. [00:11:10]Alessio: I mean, going back to using them, we had a lot of them on the podcast already. We had Cursor, we have Codium and Codium, very similar names. [00:11:18]Swyx: Yeah. Find, and then of course there's Copilot. [00:11:22]Alessio: You had a Copilot versus Cody blog post, and I think it really shows the context improvement. So you had two examples that stuck with me. One was, what does this application do? And the Copilot answer was like, oh, it uses JavaScript and NPM and this. And it's like, but that's not what it does. You know, that's what it's built with. Versus Cody was like, oh, these are like the major functions. And like, these are the functionalities and things like that. And then the other one was, how do I start this up? And Copilot just said NPM start, even though there was like no start command in the package JSON, but you know, most collapse, right? Most projects use NPM start. So maybe this does too. How do you think about open source models? Because Copilot has their own private thing. And I think you guys use Starcoder, if I remember right. Yeah, that's correct. [00:12:09]Beyang: I think Copilot uses some variant of Codex. They're kind of cagey about it. I don't think they've like officially announced what model they use. [00:12:16]Swyx: And I think they use a range of models based on what you're doing. Yeah. [00:12:19]Beyang: So everyone uses a range of model. Like no one uses the same model for like inline completion versus like chat because the latency requirements for. Oh, okay. Well, there's fill in the middle. There's also like what the model's trained on. So like we actually had completions powered by Claude Instant for a while. And but you had to kind of like prompt hack your way to get it to output just the code and not like, hey, you know, here's the code you asked for, like that sort of text. So like everyone uses a range of models. We've kind of designed Cody to be like especially model, not agnostic, but like pluggable. So one of our kind of design considerations was like as the ecosystem evolves, we want to be able to integrate the best in class models, whether they're proprietary or open source into Cody because the pace of innovation in the space is just so quick. And I think that's been to our advantage. Like today, Cody uses Starcoder for inline completions. And with the benefit of the context that we provide, we actually show comparable completion acceptance rate metrics. It's kind of like the standard metric that folks use to evaluate inline completion quality. It's like if I show you a completion, what's the chance that you actually accept the completion versus you reject it? And so we're at par with Copilot, which is at the head of that industry right now. And we've been able to do that with the Starcoder model, which is open source and the benefit of the context fetching stuff that we provide. And of course, a lot of like prompt engineering and other stuff along the way. [00:13:40]Alessio: And Steve, you wrote a post called cheating is all you need about what you're building. And one of the points you made is that everybody's fighting on the same axis, which is better UI and the IDE, maybe like a better chat response. But data modes are kind of the most important thing. And you guys have like a 10 year old mode with all the data you've been collecting. How do you kind of think about what other companies are doing wrong, right? Like, why is nobody doing this in terms of like really focusing on RAG? I feel like you see so many people. Oh, we just got a new model. It's like a bit human eval. And it's like, well, but maybe like that's not what we should really be doing, you know? Like, do you think most people underestimate the importance of like the actual RAG in code? [00:14:21]Steve: I think that people weren't doing it much. It wasn't. It's kind of at the edges of AI. It's not in the center. I know that when ChatGPT launched, so within the last year, I've heard a lot of rumblings from inside of Google, right? Because they're undergoing a huge transformation to try to, you know, of course, get into the new world. And I heard that they told, you know, a bunch of teams to go and train their own models or fine tune their own models, right? [00:14:43]Swyx: Both. [00:14:43]Steve: And, you know, it was a s**t show. Nobody knew how to do it. They launched two coding assistants. One was called Code D with an EY. And then there was, I don't know what happened in that one. And then there's Duet, right? Google loves to compete with themselves, right? They do this all the time. And they had a paper on Duet like from a year ago. And they were doing exactly what Copilot was doing, which was just pulling in the local context, right? But fundamentally, I thought of this because we were talking about the splitting of the [00:15:10]Swyx: models. [00:15:10]Steve: In the early days, it was the LLM did everything. And then we realized that for certain use cases, like completions, that a different, smaller, faster model would be better. And that fragmentation of models, actually, we expected to continue and proliferate, right? Because we are fundamentally, we're a recommender engine right now. Yeah, we're recommending code to the LLM. We're saying, may I interest you in this code right here so that you can answer my question? [00:15:34]Swyx: Yeah? [00:15:34]Steve: And being good at recommender engine, I mean, who are the best recommenders, right? There's YouTube and Spotify and, you know, Amazon or whatever, right? Yeah. [00:15:41]Swyx: Yeah. [00:15:41]Steve: And they all have many, many, many, many, many models, right? For all fine-tuned for very specific, you know. And that's where we're heading in code, too. Absolutely. [00:15:50]Swyx: Yeah. [00:15:50]Alessio: We just did an episode we released on Wednesday, which we said RAG is like Rexis or like LLMs. You're basically just suggesting good content. [00:15:58]Swyx: It's like what? Recommendations. [00:15:59]Beyang: Recommendations. [00:16:00]Alessio: Oh, got it. [00:16:01]Steve: Yeah, yeah, yeah. [00:16:02]Swyx: So like the naive implementation of RAG is you embed everything, throw it in a vector database, you embed your query, and then you find the nearest neighbors, and that's your RAG. But actually, you need to rank it. And actually, you need to make sure there's sample diversity and that kind of stuff. And then you're like slowly gradient dissenting yourself towards rediscovering proper Rexis, which has been traditional ML for a long time. But like approaching it from an LLM perspective. Yeah. [00:16:24]Beyang: I almost think of it as like a generalized search problem because it's a lot of the same things. Like you want your layer one to have high recall and get all the potential things that could be relevant. And then there's typically like a layer two re-ranking mechanism that bumps up the precision and tries to get the relevant stuff to the top of the results list. [00:16:43]Swyx: Have you discovered that ranking matters a lot? Oh, yeah. So the context is that I think a lot of research shows that like one, context utilization matters based on model. Like GPT uses the top of the context window, and then apparently Claude uses the bottom better. And it's lossy in the middle. Yeah. So ranking matters. No, it really does. [00:17:01]Beyang: The skill with which models are able to take advantage of context is always going to be dependent on how that factors into the impact on the training loss. [00:17:10]Swyx: Right? [00:17:10]Beyang: So like if you want long context window models to work well, then you have to have a ton of data where it's like, here's like a billion lines of text. And I'm going to ask a question about like something that's like, you know, embedded deeply into it and like, give me the right answer. And unless you have that training set, then of course, you're going to have variability in terms of like where it attends to. And in most kind of like naturally occurring data, the thing that you're talking about right now, the thing I'm asking you about is going to be something that we talked about recently. [00:17:36]Swyx: Yeah. [00:17:36]Steve: Did you really just say gradient dissenting yourself? Actually, I love that it's entered the casual lexicon. Yeah, yeah, yeah. [00:17:44]Swyx: My favorite version of that is, you know, how we have to p-hack papers. So, you know, when you throw humans at the problem, that's called graduate student dissent. That's great. It's really awesome. [00:17:54]Alessio: I think the other interesting thing that you have is this inline assist UX that I wouldn't say async, but like it works while you can also do work. So you can ask Cody to make changes on a code block and you can still edit the same file at the same time. [00:18:07]Swyx: Yeah. [00:18:07]Alessio: How do you see that in the future? Like, do you see a lot of Cody's running together at the same time? Like, how do you validate also that they're not messing each other up as they make changes in the code? And maybe what are the limitations today? And what do you think about where the attack is going? [00:18:21]Steve: I want to start with a little history and then I'm going to turn it over to Bian, all right? So we actually had this feature in the very first launch back in June. Dominic wrote it. It was called nonstop Cody. And you could have multiple, basically, LLM requests in parallel modifying your source [00:18:37]Swyx: file. [00:18:37]Steve: And he wrote a bunch of code to handle all of the diffing logic. And you could see the regions of code that the LLM was going to change, right? And he was showing me demos of it. And it just felt like it was just a little before its time, you know? But a bunch of that stuff, that scaffolding was able to be reused for where we're inline [00:18:56]Swyx: sitting today. [00:18:56]Steve: How would you characterize it today? [00:18:58]Beyang: Yeah, so that interface has really evolved from a, like, hey, general purpose, like, request anything inline in the code and have the code update to really, like, targeted features, like, you know, fix the bug that exists at this line or request a very specific [00:19:13]Swyx: change. [00:19:13]Beyang: And the reason for that is, I think, the challenge that we ran into with inline fixes, and we do want to get to the point where you could just fire and forget and have, you know, half a dozen of these running in parallel. But I think we ran into the challenge early on that a lot of people are running into now when they're trying to construct agents, which is the reliability of, you know, working code generation is just not quite there yet in today's language models. And so that kind of constrains you to an interaction where the human is always, like, in the inner loop, like, checking the output of each response. And if you want that to work in a way where you can be asynchronous, you kind of have to constrain it to a domain where today's language models can generate reliable code well enough. So, you know, generating unit tests, that's, like, a well-constrained problem. Or fixing a bug that shows up as, like, a compiler error or a test error, that's a well-constrained problem. But the more general, like, hey, write me this class that does X, Y, and Z using the libraries that I have, that is not quite there yet, even with the benefit of really good context. Like, it definitely moves the needle a lot, but we're not quite there yet to the point where you can just fire and forget. And I actually think that this is something that people don't broadly appreciate yet, because I think that, like, everyone's chasing this dream of agentic execution. And if we're to really define that down, I think it implies a couple things. You have, like, a multi-step process where each step is fully automated. We don't have to have a human in the loop every time. And there's also kind of like an LM call at each stage or nearly every stage in that [00:20:45]Swyx: chain. [00:20:45]Beyang: Based on all the work that we've done, you know, with the inline interactions, with kind of like general Codyfeatures for implementing longer chains of thought, we're actually a little bit more bearish than the average, you know, AI hypefluencer out there on the feasibility of agents with purely kind of like transformer-based models. To your original question, like, the inline interactions with CODI, we actually constrained it to be more targeted, like, you know, fix the current error or make this quick fix. I think that that does differentiate us from a lot of the other tools on the market, because a lot of people are going after this, like, shnazzy, like, inline edit interaction, whereas I think where we've moved, and this is based on the user feedback that we've gotten, it's like that sort of thing, it demos well, but when you're actually coding day to day, you don't want to have, like, a long chat conversation inline with the code base. That's a waste of time. You'd rather just have it write the right thing and then move on with your life or not have to think about it. And that's what we're trying to work towards. [00:21:37]Steve: I mean, yeah, we're not going in the agent direction, right? I mean, I'll believe in agents when somebody shows me one that works. Yeah. Instead, we're working on, you know, sort of solidifying our strength, which is bringing the right context in. So new context sources, ways for you to plug in your own context, ways for you to control or influence the context, you know, the mixing that happens before the request goes out, etc. And there's just so much low-hanging fruit left in that space that, you know, agents seems like a little bit of a boondoggle. [00:22:03]Beyang: Just to dive into that a little bit further, like, I think, you know, at a very high level, what do people mean when they say agents? They really mean, like, greater automation, fully automated, like, the dream is, like, here's an issue, go implement that. And I don't have to think about it as a human. And I think we are working towards that. Like, that is the eventual goal. I think it's specifically the approach of, like, hey, can we have a transformer-based LM alone be the kind of, like, backbone or the orchestrator of these agentic flows? Where we're a little bit more bearish today. [00:22:31]Swyx: You want the human in the loop. [00:22:32]Beyang: I mean, you kind of have to. It's just a reality of the behavior of language models that are purely, like, transformer-based. And I think that's just like a reflection of reality. And I don't think people realize that yet. Because if you look at the way that a lot of other AI tools have implemented context fetching, for instance, like, you see this in the Copilot approach, where if you use, like, the at-workspace thing that supposedly provides, like, code-based level context, it has, like, an agentic approach where you kind of look at how it's behaving. And it feels like they're making multiple requests to the LM being like, what would you do in this case? Would you search for stuff? What sort of files would you gather? Go and read those files. And it's like a multi-hop step, so it takes a long while. It's also non-deterministic. Because any sort of, like, LM invocation, it's like a dice roll. And then at the end of the day, the context it fetches is not that good. Whereas our approach is just like, OK, let's do some code searches that make sense. And then maybe, like, crawl through the reference graph a little bit. That is fast. That doesn't require any sort of LM invocation at all. And we can pull in much better context, you know, very quickly. So it's faster. [00:23:37]Swyx: It's more reliable. [00:23:37]Beyang: It's deterministic. And it yields better context quality. And so that's what we think. We just don't think you should cargo cult or naively go like, you know, agents are the [00:23:46]Swyx: future. [00:23:46]Beyang: Let's just try to, like, implement agents on top of the LM that exists today. I think there are a couple of other technologies or approaches that need to be refined first before we can get into these kind of, like, multi-stage, fully automated workflows. [00:24:00]Swyx: It makes sense. You know, we're very much focused on developer inner loop right now. But you do see things eventually moving towards developer outer loop. Yeah. So would you basically say that they're tackling the agent's problem that you don't want to tackle? [00:24:11]Beyang: No, I would say at a high level, we are after maybe, like, the same high level problem, which is like, hey, I want some code written. I want to develop some software and can automate a system. Go build that software for me. I think the approaches might be different. So I think the analogy in my mind is, I think about, like, the AI chess players. Coding, in some senses, I mean, it's similar and dissimilar to chess. I think one question I ask is, like, do you think producing code is more difficult than playing chess or less difficult than playing chess? More. [00:24:41]Swyx: I think more. [00:24:41]Beyang: Right. And if you look at the best AI chess players, like, yes, you can use an LLM to play chess. Like, people have showed demos where it's like, oh, like, yeah, GPT-4 is actually a pretty decent, like, chess move suggester. Right. But you would never build, like, a best in class chess player off of GPT-4 alone. [00:24:57]Swyx: Right. [00:24:57]Beyang: Like, the way that people design chess players is that you have kind of like a search space and then you have a way to explore that search space efficiently. There's a bunch of search algorithms, essentially. We were doing tree search in various ways. And you can have heuristic functions, which might be powered by an LLM. [00:25:12]Swyx: Right. [00:25:12]Beyang: Like, you might use an LLM to generate proposals in that space that you can efficiently explore. But the backbone is still this kind of more formalized tree search based approach rather than the LLM itself. And so I think my high level intuition is that, like, the way that we get to more reliable multi-step workflows that do things beyond, you know, generate unit test, it's really going to be like a search based approach where you use an LLM as kind of like an advisor or a proposal function, sort of your heuristic function, like the ASTAR search algorithm. But it's probably not going to be the thing that is the backbone, because I guess it's not the right tool for that. Yeah. [00:25:50]Swyx: I can see yourself kind of thinking through this, but not saying the words, the sort of philosophical Peter Norvig type discussion. Maybe you want to sort of introduce that in software. Yeah, definitely. [00:25:59]Beyang: So your listeners are savvy. They're probably familiar with the classic like Chomsky versus Norvig debate. [00:26:04]Swyx: No, actually, I wanted, I was prompting you to introduce that. Oh, got it. [00:26:08]Beyang: So, I mean, if you look at the history of artificial intelligence, right, you know, it goes way back to, I don't know, it's probably as old as modern computers, like 50s, 60s, 70s. People are debating on like, what is the path to producing a sort of like general human level of intelligence? And kind of two schools of thought that emerged. One is the Norvig school of thought, which roughly speaking includes large language models, you know, regression, SVN, basically any model that you kind of like learn from data. And it's like data driven. Most of machine learning would fall under this umbrella. And that school of thought says like, you know, just learn from the data. That's the approach to reaching intelligence. And then the Chomsky approach is more things like compilers and parsers and formal systems. So basically like, let's think very carefully about how to construct a formal, precise system. And that will be the approach to how we build a truly intelligent system. I think Lisp was invented so that you could create like rules-based systems that you would call AI. As a language. Yeah. And for a long time, there was like this debate, like there's certain like AI research labs that were more like, you know, in the Chomsky camp and others that were more in the Norvig camp. It's a debate that rages on today. And I feel like the consensus right now is that, you know, Norvig definitely has the upper hand right now with the advent of LMs and diffusion models and all the other recent progress in machine learning. But the Chomsky-based stuff is still really useful in my view. I mean, it's like parsers, compilers, basically a lot of the stuff that provides really good context. It provides kind of like the knowledge graph backbone that you want to explore with your AI dev tool. Like that will come from kind of like Chomsky-based tools like compilers and parsers. It's a lot of what we've invested in in the past decade at Sourcegraph and what you build with Grok. Basically like these formal systems that construct these very precise knowledge graphs that are great context providers and great kind of guard rails enforcers and kind of like safety checkers for the output of a more kind of like data-driven, fuzzier system that uses like the Norvig-based models. [00:28:03]Steve: Jang was talking about this stuff like it happened in the middle ages. Like, okay, so when I was in college, I was in college learning Lisp and prologue and planning and all the deterministic Chomsky approaches to AI. And I was there when Norvig basically declared it dead. I was there 3,000 years ago when Norvig and Chomsky fought on the volcano. When did he declare it dead? [00:28:26]Swyx: What do you mean he declared it dead? [00:28:27]Steve: It was like late 90s. [00:28:29]Swyx: Yeah. [00:28:29]Steve: When I went to Google, Peter Norvig was already there. He had basically like, I forget exactly where. It was some, he's got so many famous short posts, you know, amazing. [00:28:38]Swyx: He had a famous talk, the unreasonable effectiveness of data. Yeah. [00:28:41]Steve: Maybe that was it. But at some point, basically, he basically convinced everybody that deterministic approaches had failed and that heuristic-based, you know, data-driven statistical approaches, stochastic were better. [00:28:52]Swyx: Yeah. [00:28:52]Steve: The primary reason I can tell you this, because I was there, was that, was that, well, the steam-powered engine, no. The reason was that the deterministic stuff didn't scale. [00:29:06]Swyx: Yeah. Right. [00:29:06]Steve: They're using prologue, man, constraint systems and stuff like that. Well, that was a long time ago, right? Today, actually, these Chomsky-style systems do scale. And that's, in fact, exactly what Sourcegraph has built. Yeah. And so we have a very unique, I love the framing that Bjong's made, that the marriage of the Chomsky and the Norvig, you know, sort of models, you know, conceptual models, because we, you know, we have both of them and they're both really important. And in fact, there, there's this really interesting, like, kind of overlap between them, right? Where like the AI or our graph or our search engine could potentially provide the right context for any given query, which is, of course, why ranking is important. But what we've really signed ourselves up for is an extraordinary amount of testing. [00:29:45]Swyx: Yeah. [00:29:45]Steve: Because in SWIGs, you were saying that, you know, GPT-4 tends to the front of the context window and maybe other LLMs to the back and maybe, maybe the LLM in the middle. [00:29:53]Swyx: Yeah. [00:29:53]Steve: And so that means that, you know, if we're actually like, you know, verifying whether we, you know, some change we've made has improved things, we're going to have to test putting it at the beginning of the window and at the end of the window, you know, and maybe make the right decision based on the LLM that you've chosen. Which some of our competitors, that's a problem that they don't have, but we meet you, you know, where you are. Yeah. And we're, just to finish, we're writing tens of thousands. We're generating tests, you know, fill in the middle type tests and things. And then using our graph to basically sort of fine tune Cody's behavior there. [00:30:20]Swyx: Yeah. [00:30:21]Beyang: I also want to add, like, I have like an internal pet name for this, like kind of hybrid architecture that I'm trying to make catch on. Maybe I'll just say it here. Just saying it publicly kind of makes it more real. But like, I call the architecture that we've developed the Normsky architecture. [00:30:36]Swyx: Yeah. [00:30:36]Beyang: I mean, it's obviously a portmanteau of Norvig and Chomsky, but the acronym, it stands for non-agentic, rapid, multi-source code intelligence. So non-agentic because... Rolls right off the tongue. And Normsky. But it's non-agentic in the sense that like, we're not trying to like pitch you on kind of like agent hype, right? Like it's the things it does are really just developer tools developers have been using for decades now, like parsers and really good search indexes and things like that. Rapid because we place an emphasis on speed. We don't want to sit there waiting for kind of like multiple LLM requests to return to complete a simple user request. Multi-source because we're thinking broadly about what pieces of information and knowledge are useful context. So obviously starting with things that you can search in your code base, and then you add in the reference graph, which kind of like allows you to crawl outward from those initial results. But then even beyond that, you know, sources of information, like there's a lot of knowledge that's embedded in docs, in PRDs or product specs, in your production logging system, in your chat, in your Slack channel, right? Like there's so much context is embedded there. And when you're a human developer, and you're trying to like be productive in your code base, you're going to go to all these different systems to collect the context that you need to figure out what code you need to write. And I don't think the AI developer will be any different. It will need to pull context from all these different sources. So we're thinking broadly about how to integrate these into Codi. We hope through kind of like an open protocol that like others can extend and implement. And this is something else that should be accessible by December 14th in kind of like a preview stage. But that's really about like broadening this notion of the code graph beyond your Git repository to all the other sources where technical knowledge and valuable context can live. [00:32:21]Steve: Yeah, it becomes an artifact graph, right? It can link into your logs and your wikis and any data source, right? [00:32:27]Alessio: How do you guys think about the importance of, it's almost like data pre-processing in a way, which is bring it all together, tie it together, make it ready. Any thoughts on how to actually make that good? Some of the innovation you guys have made. [00:32:40]Steve: We talk a lot about the context fetching, right? I mean, there's a lot of ways you could answer this question. But, you know, we've spent a lot of time just in this podcast here talking about context fetching. But stuffing the context into the window is, you know, the bin packing problem, right? Because the window is not big enough, and you've got more context than you can fit. You've got a ranker maybe. But what is that context? Is it a function that was returned by an embedding or a graph call or something? Do you need the whole function? Or do you just need, you know, the top part of the function, this expression here, right? You know, so that art, the golf game of trying to, you know, get each piece of context down into its smallest state, possibly even summarized by another model, right, before it even goes to the LLM, becomes this is the game that we're in, yeah? And so, you know, recursive summarization and all the other techniques that you got to use to like stuff stuff into that context window become, you know, critically important. And you have to test them across every configuration of models that you could possibly need. [00:33:32]Beyang: I think data preprocessing is probably the like unsexy, way underappreciated secret to a lot of the cool stuff that people are shipping today. Whether you're doing like RAG or fine tuning or pre-training, like the preprocessing step matters so much because it's basically garbage in, garbage out, right? Like if you're feeding in garbage to the model, then it's going to output garbage. Concretely, you know, for code RAG, if you're not doing some sort of like preprocessing that takes advantage of a parser and is able to like extract the key components of a particular file of code, you know, separate the function signature from the body, from the doc string, what are you even doing? Like that's like table stakes. It opens up so much more possibilities with which you can kind of like tune your system to take advantage of the signals that come from those different parts of the code. Like we've had a tool, you know, since computers were invented that understands the structure of source code to a hundred percent precision. The compiler knows everything there is to know about the code in terms of like structure. Like why would you not want to use that in a system that's trying to generate code, answer questions about code? You shouldn't throw that out the window just because now we have really good, you know, data-driven models that can do other things. [00:34:44]Steve: Yeah. When I called it a data moat, you know, in my cheating post, a lot of people were confused, you know, because data moat sort of sounds like data lake because there's data and water and stuff. I don't know. And so they thought that we were sitting on this giant mountain of data that we had collected, but that's not what our data moat is. It's really a data pre-processing engine that can very quickly and scalably, like basically dissect your entire code base in a very small, fine-grained, you know, semantic unit and then serve it up. Yeah. And so it's really, it's not a data moat. It's a data pre-processing moat, I guess. [00:35:15]Beyang: Yeah. If anything, we're like hypersensitive to customer data privacy requirements. So it's not like we've taken a bunch of private data and like, you know, trained a generally available model. In fact, exactly the opposite. A lot of our customers are choosing Cody over Copilot and other competitors because we have an explicit guarantee that we don't do any of that. And that we've done that from day one. Yeah. I think that's a very real concern in today's day and age, because like if your proprietary IP finds its way into the training set of any model, it's very easy both to like extract that knowledge from the model and also use it to, you know, build systems that kind of work on top of the institutional knowledge that you've built up. [00:35:52]Alessio: About a year ago, I wrote a post on LLMs for developers. And one of the points I had was maybe the depth of like the DSL. I spent most of my career writing Ruby and I love Ruby. It's so nice to use, but you know, it's not as performant, but it's really easy to read, right? And then you look at other languages, maybe they're faster, but like they're more verbose, you know? And when you think about efficiency of the context window, that actually matters. [00:36:15]Swyx: Yeah. [00:36:15]Alessio: But I haven't really seen a DSL for models, you know? I haven't seen like code being optimized to like be easier to put in a model context. And it seems like your pre-processing is kind of doing that. Do you see in the future, like the way we think about the DSL and APIs and kind of like service interfaces be more focused on being context friendly, where it's like maybe it's harder to read for the human, but like the human is never going to write it anyway. We were talking on the Hacks podcast. There are like some data science things like spin up the spandex, like humans are never going to write again because the models can just do very easily. Yeah, curious to hear your thoughts. [00:36:51]Steve: Well, so DSLs, they involve, you know, writing a grammar and a parser and they're like little languages, right? We do them that way because, you know, we need them to compile and humans need to be able to read them and so on. The LLMs don't need that level of structure. You can throw any pile of crap at them, you know, more or less unstructured and they'll deal with it. So I think that's why a DSL hasn't emerged for sort of like communicating with the LLM or packaging up the context or anything. Maybe it will at some point, right? We've got, you know, tagging of context and things like that that are sort of peeking into DSL territory, right? But your point on do users, you know, do people have to learn DSLs like regular expressions or, you know, pick your favorite, right? XPath. I think you're absolutely right that the LLMs are really, really good at that. And I think you're going to see a lot less of people having to slave away learning these things. They just have to know the broad capabilities and the LLM will take care of the rest. [00:37:42]Swyx: Yeah, I'd agree with that. [00:37:43]Beyang: I think basically like the value profit of DSL is that it makes it easier to work with a lower level language, but at the expense of introducing an abstraction layer. And in many cases today, you know, without the benefit of AI cogeneration, like that totally worth it, right? With the benefit of AI cogeneration, I mean, I don't think all DSLs will go away. I think there's still, you know, places where that trade-off is going to be worthwhile. But it's kind of like how much of source code do you think is going to be generated through natural language prompting in the future? Because in a way, like any programming language is just a DSL on top of assembly, right? And so if people can do that, then yeah, like maybe for a large portion of the code [00:38:21]Swyx: that's written, [00:38:21]Beyang: people don't actually have to understand the DSL that is Ruby or Python or basically any other programming language that exists. [00:38:28]Steve: I mean, seriously, do you guys ever write SQL queries now without using a model of some sort? At least a draft. [00:38:34]Swyx: Yeah, right. [00:38:36]Steve: And so we have kind of like, you know, past that bridge, right? [00:38:39]Alessio: Yeah, I think like to me, the long-term thing is like, is there ever going to be, you don't actually see the code, you know? It's like, hey, the basic thing is like, hey, I need a function to some two numbers and that's it. I don't need you to generate the code. [00:38:53]Steve: And the following question, do you need the engineer or the paycheck? [00:38:56]Swyx: I mean, right? [00:38:58]Alessio: That's kind of the agent's discussion in a way where like you cannot automate the agents, but like slowly you're getting more of the atomic units of the work kind of like done. I kind of think of it as like, you know, [00:39:09]Beyang: do you need a punch card operator to answer that for you? And so like, I think we're still going to have people in the role of a software engineer, but the portion of time they spend on these kinds of like low-level, tedious tasks versus the higher level, more creative tasks is going to shift. [00:39:23]Steve: No, I haven't used punch cards. [00:39:25]Swyx: Yeah, I've been talking about like, so we kind of made this podcast about the sort of rise of the AI engineer. And like the first step is the AI enhanced engineer. That is that software developer that is no longer doing these routine, boilerplate-y type tasks, because they're just enhanced by tools like yours. So you mentioned OpenCodeGraph. I mean, that is a kind of DSL maybe, and because we're releasing this as you go GA, you hope for other people to take advantage of that? [00:39:52]Beyang: Oh yeah, I would say so OpenCodeGraph is not a DSL. It's more of a protocol. It's basically like, hey, if you want to make your system, whether it's, you know, chat or logging or whatever accessible to an AI developer tool like Cody, here's kind of like the schema by which you can provide that context and offer hints. So I would, you know, comparisons like LSP obviously did this for kind of like standard code intelligence. It's kind of like a lingua franca for providing fine references and codefinition. There's kind of like analogs to that. There might be also analogs to kind of the original OpenAI, kind of like plugins, API. There's all this like context out there that might be useful for an LM-based system to consume. And so at a high level, what we're trying to do is define a common language for context providers to provide context to other tools in the software development lifecycle. Yeah. Do you have any critiques of LSP, by the way, [00:40:42]Swyx: since like this is very much, very close to home? [00:40:45]Steve: One of the authors wrote a really good critique recently. Yeah. I don't think I saw that. Yeah, yeah. LSP could have been better. It just came out a couple of weeks ago. It was a good article. [00:40:54]Beyang: Yeah. I think LSP is great. Like for what it did for the developer ecosystem, it was absolutely fantastic. Like nowadays, like it's much easier now to get code navigation up and running in a bunch of editors by speaking this protocol. I think maybe the interesting question is like looking at the different design decisions comparing LSP basically with Kythe. Because Kythe has more of a... How would you describe it? [00:41:18]Steve: A storage format. [00:41:20]Beyang: I think the critique of LSP from a Kythe point of view would be like with LSP, you don't actually have an actual symbolic model of the code. It's not like LSP models like, hey, this function calls this other function. LSP is all like range-based. Like, hey, your cursor's at line 32, column 1. [00:41:35]Swyx: Yeah. [00:41:35]Beyang: And that's the thing you feed into the language server. And then it's like, okay, here's the range that you should jump to if you click on that range. So it kind of is intentionally ignorant of the fact that there's a thing called a reference underneath your cursor, and that's linked to a symbol definition. [00:41:49]Steve: Well, actually, that's the worst example you could have used. You're right. But that's the one thing that it actually did bake in is following references. [00:41:56]Swyx: Sure. [00:41:56]Steve: But it's sort of hardwired. [00:41:58]Swyx: Yeah. [00:41:58]Steve: Whereas Kythe attempts to model [00:42:00]Beyang: like all these things explicitly. [00:42:02]Swyx: And so... [00:42:02]Steve: Well, so LSP is a protocol, right? And so Google's internal protocol is gRPC-based. And it's a different approach than LSP. It's basically you make a heavy query to the back end, and you get a lot of data back, and then you render the whole page, you know? So we've looked at LSP, and we think that it's a little long in the tooth, right? I mean, it's a great protocol, lots and lots of support for it. But we need to push into the domain of exposing the intelligence through the protocol. Yeah. [00:42:29]Beyang: And so I would say we've developed a protocol of our own called Skip, which is at a very high level trying to take some of the good ideas from LSP and from Kythe and merge that into a system that in the near term is useful for Sourcegraph, but I think in the long term, we hope will be useful for the ecosystem. Okay, so here's what LSP did well. LSP, by virtue of being like intentionally dumb, dumb in air quotes, because I'm not like ragging on it, allowed language servers developers to kind of like bypass the hard problem of like modeling language semantics precisely. So like if all you want to do is jump to definition, you don't have to come up with like a universally unique naming scheme for each symbol, which is actually quite challenging because you have to think about like, okay, what's the top scope of this name? Is it the source code repository? Is it the package? Does it depend on like what package server you're fetching this from? Like whether it's the public one or the one inside your... Anyways, like naming is hard, right? And by just going from kind of like a location to location based approach, you basically just like throw that out the window. All I care about is jumping definition, just make that work. And you can make that work without having to deal with like all the complex global naming things. The limitation of that approach is that it's harder to build on top of that to build like a true knowledge graph. Like if you actually want a system that says like, okay, here's the web of functions and here's how they reference each other. And I want to incorporate that like semantic model of how the code operates or how the code relates to each other at like a static level. You can't do that with LSP because you have to deal with line ranges. And like concretely the pain point that we found in using LSP for source graph is like in order to do like a find references [00:44:04]Swyx: and then jump definitions, [00:44:04]Beyang: it's like a multi-hop process because like you have to jump to the range and then you have to find the symbol at that range. And it just adds a lot of latency and complexity of these operations where as a human, you're like, well, this thing clearly references this other thing. Why can't you just jump me to that? And I think that's the thing that Kaith does well. But then I think the issue that Kaith has had with adoption is because it is more sophisticated schema, I think. And so there's basically more things that you have to implement to get like a Kaith implementation up and running. I hope I'm not like, correct me if I'm wrong about any of this. [00:44:35]Steve: 100%, 100%. Kaith also has a problem, all these systems have the problem, even skip, or at least the way that we implemented the indexers, that they have to integrate with your build system in order to build that knowledge graph, right? Because you have to basically compile the code in a special mode to generate artifacts instead of binaries. And I would say, by the way, earlier I was saying that XREFs were in LSP, but it's actually, I was thinking of LSP plus LSIF. [00:44:58]Swyx: Yeah. That's another. [00:45:01]Steve: Which is actually bad. We can say that it's bad, right? [00:45:04]Steve: It's like skip or Kaith, it's supposed to be sort of a model serialization, you know, for the code graph, but it basically just does what LSP needs, the bare minimum. LSIF is basically if you took LSP [00:45:16]Beyang: and turned that into a serialization format. So like you build an index for language servers to kind of like quickly bootstrap from cold start. But it's a graph model [00:45:23]Steve: with all of the inconvenience of the API without an actual graph. And so, yeah. [00:45:29]Beyang: So like one of the things that we try to do with skip is try to capture the best of both worlds. So like make it easy to write an indexer, make the schema simple, but also model some of the more symbolic characteristics of the code that would allow us to essentially construct this knowledge graph that we can then make useful for both the human developer through SourceGraph and through the AI developer through Cody. [00:45:49]Steve: So anyway, just to finish off the graph comment, we've got a new graph, yeah, that's skip based. We call it BFG internally, right? It's a beautiful something graph. A big friendly graph. [00:46:00]Swyx: A big friendly graph. [00:46:01]Beyang: It's a blazing fast. [00:46:02]Steve: Blazing fast. [00:46:03]Swyx: Blazing fast graph. [00:46:04]Steve: And it is blazing fast, actually. It's really, really interesting. I should probably have to do a blog post about it to walk you through exactly how they're doing it. Oh, please. But it's a very AI-like iterative, you know, experimentation sort of approach. We're building a code graph based on all of our 10 years of knowledge about building code graphs, yeah? But we're building it quickly with zero configuration, and it doesn't have to integrate with your build. And through some magic tricks that we have. And so what just happens when you install the plugin, that it'll be there and indexing your code and providing that knowledge graph in the background without all that build system integration. This is a bit of secret sauce that we haven't really like advertised it very much lately. But I am super excited about it because what they do is they say, all right, you know, let's tackle function parameters today. Cody's not doing a very good job of completing function call arguments or function parameters in the definition, right? Yeah, we generate those thousands of tests, and then we can actually reuse those tests for the AI context as well. So fortunately, things are kind of converging on, we have, you know, half a dozen really, really good context sources, and we mix them all together. So anyway, BFG, you're going to hear more about it probably in the holidays? [00:47:12]Beyang: I think it'll be online for December 14th. We'll probably mention it. BFG is probably not the public name we're going to go with. I think we might call it like Graph Context or something like that. [00:47:20]Steve: We're officially calling it BFG. [00:47:22]Swyx: You heard it here first. [00:47:24]Beyang: BFG is just kind of like the working name. And so the impetus for BFG was like, if you look at like current AI inline code completion tools and the errors that they make, a lot of the errors that they make, even in kind of like the easy, like single line case, are essentially like type errors, right? Like you're trying to complete a function call and it suggests a variable that you defined earlier, but that variable is the wrong type. [00:47:47]Swyx: And that's the sort of thing [00:47:47]Beyang: where it's like a first year, like freshman CS student would not make that error, right? So like, why does the AI make that error? And the reason is, I mean, the AI is just suggesting things that are plausible without the context of the types or any other like broader files in the code. And so the kind of intuition here is like, why don't we just do the basic thing that like any baseline intelligent human developer would do, which is like click jump to definition, click some fine references and pull in that like Graph Context into the context window and then have it generate the completion. So like that's sort of like the MVP of what BFG was. And turns out that works really well. Like you can eliminate a lot of type errors that AI coding tools make just by pulling in that context. Yeah, but the graph is definitely [00:48:32]Steve: our Chomsky side. [00:48:33]Swyx: Yeah, exactly. [00:48:34]Beyang: So like this like Chomsky-Norvig thing, I think pops up in a bunch of differ

Alta Vista is a small, craft winery that is producing high quality wines that are also affordable. The Malbec varietal has become so popular in the US that it is found in every off-premise account and most on-premise accounts. Additionally, Argentina is not just about Malbec anymore, they are making great Cabernet Sauvignon and Chardonnay as well. Be sure to mention all of these great options when selling! Nick Poletto is the Vice President of Education at Kobrand Corporation. Kobrand has been importing fine wine into the US since 1944. Kobrand is a family owned importer with quality wine as its main focus. Nick Poletto travels around the US teaching sales teams about wine and the many different producing regions. Nick has visited all of these properties around the globe and brings you the most complete information with the most important sales tips. For more information, visit our website at https://www.kobrandwineandspirits.com  Follow us on Instagram https://www.instagram.com/kobrandwines  For more wine education visit https://wine365.com  or view Nick's Wine Journal https://www.youtube.com/@nickyvino1  Good selling!

durée : 00:28:50 - Le Feuilleton - "Ça fait plus de vingt ans que j'habite ici et rien n'a changé, les pères ont transmis aux fils leurs armes, leurs maîtresses, leur mémoire."

Join us as we welcome internet marketing titan, Perry Belcher, to the AM/PM Podcast! Listen in as we journey through Perry's remarkable career path - from humble beginnings before turning to digital marketing. Perry's illustrious career even saw him get a personal call from none other than Jeff Bezos himself, a short story you don't want to miss!   The conversation continues with Perry reflecting on the rise and fall of his business and partnerships. His journey, marked by selling health supplements to launching a digital marketing business, and finally starting the Driven Mastermind and the War Room, is an insightful one for any entrepreneur. Our chat also covers the importance of joining a mastermind group, the benefits it can bring, and how it can help you gain a broad perspective of different industries.   Lastly, Perry shares fascinating insights about the role of AI in business, specifically in copywriting. From reducing labor costs to crafting compelling headlines and stories, the potential applications of AI are far-reaching. He also discusses misconceptions people have about AI and the opportunities it presents. Tune in for a riveting discussion about the intersection of AI, E-commerce, and internet marketing. In episode 365 of the AM/PM Podcast, Kevin and Perry discuss: 09:22 - Success in Real Estate and Selling 16:45 - Running Successful Events 23:30 - The Value of Networking and Collaboration 29:55 - Selling Event Recordings for Profit 34:19 - Cash Prize Incentives for Speakers 39:00 - Leveraging Email Lists for Business Success 42:06 - Artificial Intelligence And Its Impact On Internet Marketing 53:21 - Other Mindblowing AI Capabilities 57:27 - AI's Role in Various Industries 1:07:38 - Follow Perry on Facebook for Updates 1:09:46 - Kevin's Words Of Wisdom Kevin King: Welcome to episode 365 of the AEM PM podcast. My guest this week is none other than the famous Perry Belcher. If you don't know who Perry is, perry is one of the top internet marketers, probably one of the top copywriters in the world today. He's got his hands in all kinds of stuff, from newsletters to AI, to print on demand to funnels, to you name it. In marketing, Perry's either got tremendous amount of experience in it or he's heavily involved in it right now. We talked some shop today and just go kind of all over the place on some really cool, interesting topics. I think you're getting a lot from this episode, so I hope you enjoy it. And don't forget, if you haven't yet, be sure to sign up for the Billion Dollar Sellers Newsletter. It's at billiondollarsellerswithaness.com. It's totally free. New issue every Monday and Thursday. It's getting rave reviews from people in the industry and some of the top people in the industry as well as people just getting started. So it's got a little bit different take on it and just a lot of information. Plus, we have a little bit of fun as well in the newsletter. So hopefully you can join us at billiondollarsellers.com. Enjoy today's episode with Perry. Perry Belcher, welcome to the AM/PM Podcast. It's an honor to have you on here. How's?   Perry: it going, man, Dr King, esquire at all. I'm doing great, buddy, I'm doing great. I'm just trying to survive this hot, hot, hot summer that we're all having, you know.   Kevin King: Well, you're out there in Vegas. Y'all had floods, right. I was seeing some stuff on TikTok, like some of the casino garages and stuff were flooding.   Perry: Yeah, there were some floods out here, so it's been. We got like years worth of rain in two days or something like that, they said, which we could stand. It didn't hurt. But the hot weather out here is just the way that it is. You get used to it after a little while.   Kevin King: Yeah, it's the same in Austin. It's like 108, I think today, and I know you know, football season just recently started and everybody's complaining that they're doing a game. One of the first games was in the middle of the afternoon, like 2.30 in the afternoon and like man, half these people are going to be dying out there, you better have some extra medical. You know supposed to do these things at night in Texas during September.   Perry: My kid did in the middle of the day and he had some days that they were kids passing out, you know. So I don't miss the heat in Austin. I'll take the heat in Vegas instead. It's different kind of heat to me.   Kevin King: Yeah, it's not. It's more of a dry heat, not that, not that human heat that we have here. I'll take it so for those. There's some probably some people listening that don't know. They're like who's this? Perry Belcher character? I never heard of this Perry Belcher guy and if you haven't, you've probably been living on a rock in internet marketing, because Perry Belcher is one of the living legends out there and when it comes to internet marketing, it's not just he dabbles on Amazon, but it's Amazon's just a little piece of what he does. He does a ton of other stuff. So, and you've been doing this since you're like, you've been an entrepreneur since you're like I don't know, three years old. I heard you selling hot dogs. I mean, you've pretty much done, everything from run from selling hot dogs to running, I don't know jewelry, pear shops or something, to having little kiosk in the mall, to crazy kind of stuff. I mean, just for those that don't know who the heck you are, just give a little bit about your background.   Perry: Sure, I'm world famous in Kazakhstan. I started out, you know, I grew up really poor in little town in Kentucky, paducah. It's a sound of dead body makes when it hits the floor. And I'll as soon as I could. I stayed there until I could drive. I could drive a car. I got the heck out of there and went to the big city, nashville, you know, and I got into, you know, early on I got into retail and I owned 42 jewelry stores. At one time when I was really, really young, before I was old enough to buy beer, I owned 42 jewelry stores. Isn't that crazy? That's crazy. Not that I didn't buy beer, but as long as I was legally buying beer Exactly. You know. So I was in retail. I went out of, you know, eventually I made three different runs and retailed it, Okay, and then I got into manufacturing. I found I really enjoyed manufacturing Great deal. I still do a lot of manufacturing, as you know and then along, I guess about 1997, for those young whippersnappers that were born about then that are on in your Amazon crowd right In 1997, they invented this thing called the interwebs and Jeff Bezos started a store called Amazon and I sort of got. I sort of got all caught up in the web thing. And you probably don't know this story. It was a true story, Kevin. I got a call from Jeff Bezos when I owned craftstorecom, so this was in probably 1998 or 1999. I got a personal call from Jeff Bezos wanting to talk to me about buying craftstorecom and rolling it into the Amazon family. And then they were only selling books, they were bleeding I don't even know $100 million, a quarter, or some crazy number. And I'm like dude, you're, I'm reading about you, you're losing money, I'm making money. You know, I think you got this reversed. I probably should buy you. I swear to God, I said that. Yeah yeah, I said that that was about best I can figure about a $750 million mistake.   Kevin King: Well, it's funny you say that, because I mean we go back, we're old school when it comes to way, before you know all this internet marketing craze. We were doing old school marketing, you know, by by putting a postage stamp on an envelope and sending it out. And I remember I have a couple of similar stories back around that same time, early late 90s, early 2000s. The guy at MySpace had just started somewhere around in there and those guys reached out to me. I had a newsletter, an online newsletter going at the time, and they reached out to me to do something and I turned. I just ignored them. I was like what's this MySpace thing? I never heard of it.   Perry: I did the same thing with Jim Barksdale. You know who that was. Yeah, yeah, barksdale wanted to buy one of my companies and I blew them off, and he was Netscape you know they also used to do back you might remember this back.   Kevin King: I had several different websites and to get traffic back before there was Google and all these. You know, this SEO and all this stuff is basically as Alta Vista and you know, I love that, I love that Yahoo and all these guys and you could just just by putting stuff in the meta tags, you'd rank, you know on top of the crap out of yeah. You put a text down at the bottom and all the good, all the good, all the good all the good, all that kind of stuff. But I one of the things, what you might remember this there is what's called ring sites. So in order to get traffic, you go to some guy would figure out how to get people to his site and then it would be like next or previous, and you'd hit a button and it would go to the next, previous, and then we had a newsletter that was doing about 250,000 emails a day back before can spam and all that stuff and to get traffic to it. You know, we were getting on Howard Stern Show when he was on terrestrial radio and we were doing all kinds of crazy stuff. But I was working with a site called BOMAS B-O-M-I-S and they had one of these ring sites and we they were like one of our top sources of traffic and I just remember there's two guys there running out of their apartment or something. I talked to one of them. This is like probably around 2000 or so, ish, 2001. He said, hey, you're going to be dealing with me from now on. My buddy is moving on. I'm like all right. I said James is moving on. I said, ok, cool, what's he going to do? He said I don't know, some sort of encyclopedia or something. I'm not sure what he's going to do. He's got some some crazy idea. Turns out it was Jimmy Wells from Wikipedia. I was actually working with Jimmy Wells from Wikipedia before he was Jimmy Wells from Wikipedia. Isn't that crazy? It's crazy, I mean the stories that we can tell from the early days of the Internet.   Perry: When I look back, I just can't. You know my buddy's favorite saying, and I've adopted this I can't believe how stupid I was two weeks ago.   You know like you. Just you just realize you know just the boneheaded stuff that you did when there was so much opportunity. The first domain I ever bought this was like just when domain registrations came out I bought formulas, the number four you oh wow com, the most worthless domain anyone could ever own, when I could have probably bought internet.com Pretend to buy anything and I bought the most boneheaded stuff. You know.   Kevin King: Well, you remember the guy that he got in early he bought was at sex.com or something for, like you know, 10 bucks or whatever it cost to register it back then before there was a go daddy, yeah, and remember the fight like 20 years ago over that domain because it became like the most valuable domain on the entire Internet or something. Remember that huge fight about that.   Perry: It was. It was crazy, but I know there's been a bunch of those stories. Man, I've got some friends that really did well buying domain real estate early on. I bought a lot. I mean I've, over time, I still think domains are a bargain. I really do Most. For the most part, I own stuff like sewing.com and makeuptutorials.com and diyprojects.com. I still own some big stuff that we operate and I own a bunch of other big stuff that we don't operate and you know I'm buying after markets.   Now I bought conventions.com for a little over $400,000 two weeks before COVID Boy. That timing was extraordinary. You know what could go wrong. Conventions are impervious to depression and so anyway, yeah, so I started buying. You know I got a manufacturing and I immediately saw the benefit of online selling because you could cut out all the different layers of middlemen in the in between the consumer and the manufacturer. So I've been a manufacturer selling direct to consumer for a long time. And then I got. I got in business with Ryan Dice. After I got in a lot of trouble, almost went to jail in the supplement business scares me to death to this day. You know I lost everything I had, almost went to the clink, and when that all got settled out I went to business with Ryan Dice and we he turned me on really to the information selling world.   Kevin King: How'd you guys meet up? Was it at some events, or did you just meet up? Yeah, we met up.   Perry: Yeah, I'll tell you, the story is pretty funny story. So we met at a Yonix Silver event. We went to dinner with, you know, all these millionaires, you know in the room, the millionaire mastermind people, and we went to this big dinner and we had like 20 people at the dinner and when the check came it was like, well, I only had a salad, well, I only had the soup, and you know they're all dividing up checks and crap. And I'm like, come on and Ryan looked at me and I looked at him. He said do you just want to pay this bill and get the hell out of here? And I said, yeah, so we split the bill. And that's how we became friends, how we met. And then, you know, when I we knew each other through Yonix and then when I got in trouble in the supplement business, I mean, I had loads of friends when you're, when you're now and when you're when you're netting out half million dollars a month and you're flying all your friends on private jets, the Thomas and crap on the weekends, boy, you got lots of friends, you know. And as soon as the money ran out, well, guess what? The friends ran out. You know, you know everything was, you know. Nobody knew who I was. Then, you know, and Ryan called me and said hey, man, I got this business in Austin. It's doing a couple million dollars a year. If you'll come help me run it, I'll give you half of it. Oh, wow, and we did $9 million in the first seven months.   Kevin King: And that was a digital marketer. For those of you that don't know, that's correct.   Perry: Yeah, it was called touch tone publishing then, but eventually we rebranded it became digital marketer and then out of digital marketer came traffic and conversion summit and out of traffic and conversion summit came the war room mastermind and we ran all three of those for years. And digital we sold a TNC to a Claire and Blackstone Blackstone group about four years ago, I guess. Then I sold my interest in digital marketer to Ryan and Ryan, roland, richard about two years ago and then we dissolved war room about a year ago I guess they were going a different direction and and Kossim Islam and Jason Flylon I started driven mastermind so but yeah, it was a great, great run with. Those guys are super good, guys are super, super smart and we were business partners for 14 years long time. It's a long. That's a you know outlast a long time.   Kevin King: That's a long time in this business longer than all my marriages, almost divine, you know. So going just down. We'll talk about some of those in just a second, but just down that back what? What got you in trouble in the supplement business was it claims that you just didn't realize you couldn't be. Yeah, what was the it?   Perry: was kind of a combination. I was. I was legitimately a pharmaceutical manufacturer. We were an FDA pharmaceutical manufacturer. I got all the licensure and all that I got in trouble with the state had nothing to do with the federal. They called in federal, they called in DA, they called in everybody, like guys. Everything he's doing is correct. But the state took issue to some claims and what ended up happening? They realized that they had not. The thing is, once the state gets their tentacles into you and have your money, you know it's really hard to get rid of them, right? They're like a tick. But. But at the end of the day, the only thing that that that they actually that stuck was something called ways and measures. So that meant that my equipment wasn't precise enough to put the exact amount of product per bottle. So let's say it says it's two ounces right, mine might be 2.1 or 1.9 ounces right, and that's there's. There are state laws about that. They're called ways and measures laws. They're governed by the people who manage gas pumps, if you could believe it. But out of everything that they originally said that I was doing, they dropped everything else and that was the only thing that actually, at the end of the day, was it? But I had to settle it and they got all my money and all my stuff and left me three million dollars in debt. And when, when I went to Austin and we hustled hard, you know, for a couple of years, and I paid all that off, I didn't file bankruptcy on it and it was hilarious because I threw a Perry's broke party. Yeah, about two years in, when I got to zero, I got back to just broke. I wasn't three million dollars, right. I threw a giant Perry's broke party as maybe one of the most fun parties we've ever had. It was a little you're in.   Kevin King: Austin's, you do that out at Willie Nelson's ranch. Because, I was tapes, remember he did that when he got in trouble for seven million bucks and he did some sort of big ass fundraising party out. He has this like old ranch out West of Texas, west of Austin that's. It's got a studio lot on it, basically an old.   Perry: House. Then I just had it right over the house and we had a big pool party and, oh my Lord, so many drunk people. It was a lot of fun, it was good time, so I got a lot of friends at Austin and you'll talk digital marketer.   Kevin King: the conference from like. I think the first one's a few hundred people to what the? Now it's five, six thousand people, or yeah, we get the biggest internet for if you're an internet marketing, yeah, just in in general, it's not just Amazon, it's like across the board, it's the biggest one out there, I think.   Perry: Yeah, before the year before COVID, I think we had the biggest year was seventy two hundred. Oh wow, seventy two hundred, seventy eight hundred, I can't remember. They thought we were going to ten thousand the next year and they rented the Coliseum in San Diego instead of the hotels. And then, of course, covid yeah, and it was just a you know, two or three years we had sold just prior to that. So have we not have sold that first year of COVID? I think was probably around a five million dollar loss, but they had clear and had insurance for it, fortunately. So I don't think they. I don't. I don't know the exact damage, but I know it would have probably wiped us out and we've been because we had a refund. Tickets with In the venue would not have soft to hook and I was a big bunch of crap when it comes to running conferences.   Kevin King: I mean, I do my billion dollar solar summit. You do your events now, like you do. You've done the couple AI summits, you've done the Perry's weird event or whatever. You do quite a different things. You have the Whatever, whatever, whatever. You done like three of those which are fascinating. You do, you know, you have the driven mastermind and you're involved with digital market and our space. There's a ton of people it's almost gotten through Events for Amazon sellers, like everybody. Everybody in their dog wants to have an event and the vast majority of them suck. There's like seven people there they can't sell tickets that are losing their shirt. Very few of them actually make money. What is the key actually, if you want to do an event or you're thinking about that to actually making these things work, is it the long term play you gotta have? The upsell is at the.   Perry: Well, events, events are very, very much an uphill battle. That's the reason. When you go to sell one, they have a lot of value. If you go to, if you build an event to a thousand, two thousand people, it has a lot of value in the exit market because once an event hits a certain inflection point, they're insanely profitable. So you're so, like digital market, we lost money On TNC for probably the first four years that we did it. But the way we made up for it, we filmed all of the sessions and we sold them as individual products. So we built all of our. We had a thing that really made that thing magical, because every session had to be good enough to sell as a product. So it made the event itself, you know, great because you had to have executable do this, do this, do this, do this. It couldn't just be a fluffy talk, right. Every talk had to be good enough to sell as a product when Ryan and I were doing them. So for the first three or four years we didn't make hardly any money, but we generated a lot of product out of that. We sold throughout the year. So we, you know, we did make money a couple million dollars a year From the product sales and then over time, as the attendance goes up, the ticket prices tend to go up. You start at really low ticket prices and you ratchet ticket prices up as the event gets bigger and bigger, bigger, and you start taking on sponsors and we basically got to the point by the time that we sold. You don't really want to sell right, because the sponsors were paying for 80 90% of the cost to put on the event. Tickets were you then over a thousand dollars a ticket? We were selling 7000 tickets. You didn't really need to sell, you know, because you the event was paid for by the sponsors. The ticket sales money was just free money. And then whatever you do at the event, you know in sales is even more free money. But when you look at companies like Clary on the by these things, they don't care about the product creation, they don't care about selling at the event, they only care about tickets and they make a lot of money on hotel rooms. So they so in when, when they're promoting they got a lot of cash, so they'll buy all the hotel rooms in downtown San Diego a year before we, right before we, now we announced the dates, they buy all the rooms and then when you're buying your room from bookingcom or American Express or whatever, you're actually buying that ticket from Clary on, because Clary on in a lot of cases bought all the rooms in the city for $120 a night and then a year later you're paying 350 on AmEx and they just pay AmEx a commission, a 20% commission.   Kevin King: That's different than the way when I do like for a billion dollar so much in order to not have to pay you know, $3,000 to turn the Internet on in the ballroom, or to have to per day, or from not having to pay for the ballrooms or this or that. We have to do guarantees. Rather than buying the rooms up front, we have to guarantee that we're going to put 50 butts in the in these beds or whatever. If we don't, we get penalized, you know, yeah, right.   Perry: We did a little bit different model. Yeah, we did, we did too. You still have room blocks, you know, and the killer and the killer in the convention businesses contract negotiation and room blocks. You know, if you can get room blocks down, we did one recently at the ARIA and I didn't have a room block anywhere because the ARIA surrounded by like eight hotels within walking distance, so there's no reason to book a room block. Everybody could stay where they wanted within that complex and the room blocks Everybody could stay where they wanted within that complex. And then we got together and it didn't. It didn't create the problem, but you know they get you. Would they charge you more for F&B? So they, they're going to get you right. So I've got my own event center now I've got a 50 person event center. I think we're going to expand to 100 people and and I really prefer having smaller workshops anyway, they're they're more intimate, they're more effective and if you're going to sell something else to the attendees, the smaller the room, the higher your conversion rates will always be if you're offering something to the attendees.   Kevin King: That's true, yeah, so then you took it from there to the mastermind you did the war room for a long time and I know my buddies, Manny  and Guillermo, at Helium 10. They joined the war room about two years into working on helium 10. They said that was the number one life changing thing that they did.   Perry: They killed it to that.   Kevin King: I don't know the numbers, but I know it's. I see what he's spending and what he's doing, so I'm like it's some serious numbers. But they they attribute that to war room, because there was some. Y'all did one event and I think it was in Austin, actually around 2018 ish, and it was all about system. Whatever the talk was on that one, because they're quarterly, they were quarterly deals. I think it was all about systemizing and getting out your way and like cutting all the riffraff. I don't, but they said that was. It was game changing for them and made them tens of millions of dollars. So, but to join a war room was what 30 grand, I know driven was what you have now which I've been driven 30 grand.   Perry: Yeah, I've been to.   Kevin King: I've been to driven. I went to the one back in July which was excellent out in LA and and I love going to these. Those of you are listening. You know this is not an Amazon conference. A lot of us go to Amazon conferences, but I think the best conferences for me are actually the non Amazon conferences, because I go into something like a driven where there's yeah, there's a handful of Amazon people there, but there's also a bunch of Facebook people. There's also a bunch of domain people, there's SEO people, there's people that you know just have some sort of a shop in Baltimore that you know do internet marketing and you, you meet this range of people and for me it's brainstorming sessions. I'm uninterrupted. You know if I'm watching stuff online, even the recording of that, you know I got phone calls coming in, the dogs barking. You know wife's nagging, whatever it may be. You're interrupted. But you're sitting in a room from nine to five, obviously not in the room. You're sitting in a room From nine to five listening to people, these people talking a lot of it. You might already know, some of it may be new to you, but you're just in there. One guy says something, perry says something, and then Kazim says something, and then Jason says something, and whoever else the speaker says something, you start going. If I put all these things together and I can do this for my business, holy shit, this is freaking incredible. And so that's. These people look at me. And why the heck would I pay 25 or 30 grand to be in some sort of event? And if in the Amazon space, I personally wouldn't, because I'm going to be the one delivering most of the value in a lot of cases. And so why would I pay to join something? They should be paying me to come to it. But when you go to something where it's a cross section of people in the marketing world that all think like you but they do different things, I think that's the most valuable thing, would you? Would you agree?   Perry: I think honestly, I think in a good mastermind and that there's that good being in parenthesis and a good mastermind. I don't think you can lose money. I think it's almost impossible. I've made money in every mastermind I've ever been in you just, I like the idea of the diversity, right. I might learn something from a guy in the funeral industry that can be applied to somebody that's selling weight loss, right. You never know. And you know my benefit. I guess I've been around a long time, like you, kevin, I've been around the block a bunch and I've been fortunate enough to work with like hundreds and hundreds and hundreds of businesses Pretty intimately in the, in the, the war room and now driven setting, and you know I get to see what's working and what's not working from like a 10,000 foot view inside all these businesses. So for me personally it's a great benefit that I get to learn something from really diverse. You know I learned the other day I was talking to a friend of mine, a client, that that they're in the, they sell online, that you book an appointment, you know they call you in, whatever, and they're in an industry that I have no interest in, no knowledge of, right. But they figured out that if they once somebody's booked an appointment, if they put a zoom, a live zoom, on the thank you page with somebody sitting there going hey, kevin, so glad you booked your appointment. By the way, jimmy can take you right now if you want, right. That one thing those, those people are coming in that way, or converting nine times higher than the people who book a normal sales call. And the beautiful thing now is.   Kevin King: You can do that with AI. There's tools with AI where you could actually, when they fill in that form I'm registered, I'm Kevin air dot AI and all that yeah, several and one that you could actually and you could put in you upload a spreadsheet or tie it into. You know, through an API to your, your cell system, that Jenny is available and it can actually, as I'm typing in, kevin King it's in the background recording a video with with Perry saying hey, hi, kevin, this is Perry. I glad you just signed up. Jenny's available right now. It's all automated and all like holy cow how to help her is just sitting around it and you know the conversions on that go through the roof.   Perry: Oh, they're nutty and but that's something I learned from a person who's in the like the the trauma they. They serve trauma psychiatrists, that's their market and I'm like I would never know that in a million years. Right, but but how many other businesses or clients of mine could that one tactic be applicable to? The answers? A lot, right, so you. So, when you go into those rooms where you know to be in driven, you got to be doing at least a million a year, but I think our average is around seven million a year gross and, and some you know up to, you know there's there's some hundred million dollar Folks and big players in there. There's some big players there, but you but nobody's stupid, right? You're in a room full of really, really smart people when they're basically telling you what they're doing. I joke about. I get paid for people to tell me. I get paid for really smart people to tell me what they're doing. That's really working and what I right, what a great gig I got right. But, yeah, we've been doing it for a really long time there. Those groups masterminds are hard to keep together and Keep happy and all that there because they are, because they're intimate, people share a lot of details and sometimes you have personality, kind of little things. This is crazy nutty stuff. That happens that you, the only problem with those things are just, they're a, they're a bit to, they're a bit to manage and you know that, as far as the 30 grand goes, or 50 grand, or 70. I know a lot of people charge. I know a buddy mine charge is 70,000 a year. You know we act like that's a lot of money but everybody's got an idiot on their payroll that there's a more than 30 grand to, I promise you. Everybody does. Everybody has a dodo on their payroll that they should have fired a long time ago but he brings the doughnuts or something and you don't farm that. Would you rather have that dodo licking stamps four hours a day or would you rather, you know, have access to some of the smartest people and your peers and you know really Really that? Keep you accountable, keep you on your toes and keep you up to date, because we do a call every week along with the meeting. So I I'm not pitching it down, I don't. This is sound like I'm hey, go buy my thing, but no matter what the industry you're in, get into a mastermind group. If you can, it'll one that you can afford.   Kevin King: You know ours is out of reach for most people because they're they're not because it's they can afford it, because they just don't meet the minimum sales, like you said, like you know, if you're at a one million and you said the average is around seven, you know, for 30 grand a year, all you need is one, one little idea, one thing, just you, just the ROI could be immense on just one thing.   Perry: I've heard a hundred times and I got all my value for the year within the first two hours. The first meeting yeah, you know, I've heard that so many times because this Kevin King gets up and talks and says something really smart and you go. Well, that was worth it, right, I got. I learned a thing that I didn't know and and, like you said, when you're doing, the beauty is the reason we don't take people that aren't doing a lot of money yet. It's hard to ROI. But if you're already doing let's say you're doing seven million a year and you get an idea that gives you a 5% bump, right, let's 350 grand, yeah for an idea. And you, you know, you're in for a year. You're in for 52 calls and four live meetings and Intensives and networks and private calls and all kinds of stuff. It's you know and I'm not saying for us, just for any man mind if you get a good mastermind, you can't lose money if you, if you have a good enough business already that you can ROI.   Kevin King: One of the things that you do that's really cool too is, like you said. You know, with digital market and I agree that you know you're recording it, turning it into content you do that now. Well, you'll do a Like that, the weird event you you straight up say, hey, come out to this thing. Yeah, it's gonna be a hundred of you here, but I'm recording this. I'm gonna turn this into a product. Yeah, you turn it into six products. You know, and I didn't with my billion dollar seller summit. I didn't used to record those, but now that's half the prop. That's where the actual the profit is. It's actually in recording it and then selling it to the people that didn't come. But one of the cool things that you do, like it driven and some of your other events your AI event you did this. I think you do it. Probably pretty much everyone I've ever been to is at the end you say get the kick the cameras out of the room, turn everything off. Let's grab a bottle of wine. You sit up with the stage. You might bring a couple other your partners or the couple other speakers and it's just two hours, three hours. They're just shooting the shit of Q&A and, yeah, stuff that comes out of that Alone pays for the entire event.   Perry: Yeah, the unplugged we've we've been doing unplugged forever because at the end of most events, you know, you still have unanswered questions and I don't want people to have unanswered questions. But also some people just don't want to talk about, they don't feel comfortable talking about the particulars of their business on camera. Yeah, so you know, if they because you know, sometimes a lot of my students are also Gurus, right, and you know how gurus are they don't want to tell you that. Well, they don't want to tell you that they're having a hard time making the lease payment on Because they're pretty ill, hurt their image, right, I talk about all of my screw ups and Almost going to jail and going broke and all it, because you know it's real, that's the real of people. But but a lot of the guru guy, well, I can't say that because it was just destroying my image. So I like doing unplugged sessions a lot of times because they people feel a little more comfortable talking about their challenges and Without feeling like it changes their position. And I think sometimes, just, you know, people don't want to ask their question on a microphone in front of a thousand people for fear of embarrassment. And what if my questions? A dumb question. So when you're just sitting down Slugging back a beer and you know chatting they feel more comfortable asking the questions. They probably should be asking it we I've done that as a policy for a really long time. We do wicked smart and we do unplugged, and those are the two. You know we always ask for the best idea in the room, and that that was a funny story.   Wicked smart was invented the first year that Ryan and I did Traffin conversion summit. We programmed three days worth of content for a three-day event and At 11 o'clock on the third day we were out. We'd have anything else to talk about. We actually we had miscalculated our time and we have anything else to talk about. So we went to lunch and we said man, we got to fill all afternoon. What are we gonna do? And and and I don't know if Ryan or I are together, I think we pretty much together we came up with the idea let's just challenge people to come up and tell us the smartest thing They've learned in the last six months and how it affected their business, and let's give whoever gives the best idea. And I think the first person that came up, ryan or I won Jeff Mulligan's, a good friend of ours and he's from as a former boss tonight lives in New Hampshire and he always says wicked smart, that's wicked smart, you know. And yeah, and the first person came up and they did their thing was whoo, that's wicked smart and that's stuck. And that's how wicked smart got started. But we never did unplugged. I used to do unplugged with Andy Jenkins at Stompernet years ago when I would. I used to go speak for them every now and then and one of the things that I did was really, really cool was called unplugged and we just Andy and I, would sit down on the edge of the stage. I don't, andy was brilliant. I don't know if you ever knew him or not. He was absolutely a really really brilliant guy and he and I would sit on the edge of the stage and talk to people for hours. You know it was a lot of fun. So I kind of picked that up from Andy.   Kevin King: Yeah, I do that at the billion dollar source. I'm not do a hat contest, so the last day, what well? I do two things. I incentivize the speakers to bring it, so I put a cash prize on the speakers. So, because I don't want them doing the same presentation they just did it three other conferences or same thing they talked about on podcast I want them to bring their a game, so I put a five thousand dollar cash prize on the first and twenty five hundred on second. It's voted on the last day. I'm ineligible. I always speak last, so I'm ineligible.   But all the other speakers that I invite after the last one spoke, everybody votes On who they thought was the best speaker, deliver the best value, and then that person gets five grand. So it's become like an honor to do that and then, as a result, everybody is bringing next level stuff that they normally wouldn't talk about. Because, and then I publish the list of the and you know, if there's 15 speakers I Public, I start at number 10. I don't show number 11 through 15. I want to embarrass somebody totally, but I start at number 10 and go backwards and announce them up like it's. You know, like it's a billboard top 100 or something, casey casem or whatever and it works really really well because Everybody's. If you're not in the top 10 of a speaker, you're like you know you didn't do so well, you didn't resonate, and then you're not coming back if you need a spelling of my name for the check. You've been involved in AI for like seven years before. It was the cool thing to do, I think probably six yeah, probably six years.   Perry: I got. I spoke on AI at the largest TNC, that one before COVID. I spoke on AI and showed Jarvis and Well said labs and a bunch of those before Anybody or anything, and and everybody in the room was just blown away by it and I feel certain they didn't do anything at all when the dog, you know. But I was using it for copywriting and we were building services For and like this AI bot that were it'll be after this Heirs, but but this AI bot, you know, we're really concentrating more on the business models that you can apply AI to. So the first AI bot summit was all about Opening people's minds up to it, so they understood what it was, understanding how to use the tools and and really just grasping this. One thought of If you had 10,000 really smart people willing to work for you 24 hours a day for free, what would you have them do? That's always my question, because with AI and a little bit of robotics, that's what you have. You have an unlimited amount of Robotic slaves to do your bidding right, whatever you want, and they don't take breaks and they don't break up with a boyfriend and they don't sue you for, you know, workplace compliance issues and all that stuff and, and you're gonna see, I think it's already happening. It's just people aren't exposed to it in mainstream yet, but Corporate is projecting like huge profits over the next few years as they Diminish the amount of workers, physical workers they haven't replaced with AI Elon Musk whether you like him or not, you know, cut the workforce at Twitter by 90% and arguably, the experience for the end user hasn't changed.   Kevin King: Yeah right, yeah, it's, it's your event back in just to tell a quick little story. Then we'll go into this. But your event back in April. You're showing some business uses. You know you're talking about the army of 10,000. You showed something about a. You know here's a building, the payroll of this building and use AI and the payroll goes from I don't know some crazy number of a million dollars a month to 86 dollars a month or what some exaggerate there.   Perry: It's the Empire State Building and the payroll. The daily payroll in the Empire State Building is about I I'm gonna paraphrase, I don't remember the numbers, but it's about a million dollars or more a day and the average worker output 750 words of text a day in white collar America. So if you translate that into the cost of open AI to generate the same 750 words, it's about 42 bucks, I think yeah, it's like you know it's it's in 42 I mean for all of them, not for one of all of you know 42 bucks or 92, but it wasn't much.   Kevin King: It was less than less than 200 dollars, I think, to generate the same amount of work product one of the things that you talked about there were newsletters and like how AI can automate a lot of newsletters and and I'm a I'm gonna disagree with you a little bit there on where you can actually have. I think at that time you may have changed your tune now I'm not sure. But you're like let AI do all the writing, do everything. You can just put these things on autopilot and I think that's definitely possible, but the quality sucks and for the most part, unless you're just assembling links. But if, but, but. What you said there actually about newsletters got me thinking. It's backed on that same thing we're talking about earlier bringing this all together. Here is where, about going to events. It's like you know what I used to run a newsletter in the late 90s and early 2000s that we that had 250,000 daily subscribers. We crushed it as using that as a lead magnet to sell memberships, to sell physical products, to sell everything. What, if you know? And this Amazon product space, everybody's always trying to build audiences and they're always like go build a Facebook group, go Create a blog post and you, as you know, the most valuable asset in any business as your customer list, your email list, your Custom list and be able to use that when you want, as you please. And you can't do that on social media. You have no control with algorithms on Facebook, you know, have no control over how many people see your LinkedIn post or or anything. But with an email list or a customer base database, you do. I was like, wait a second, what if we took newsletters and did this with physical products and actually to build audiences? So if I'm selling a dog products and I happen to have sustainable dog products, I'm like what if I build an audience? A dog, the dog markets half of America. That's too big. Well, if I niche that down to some people who ends dogs and sustainability, create a newsletter for them. I'm not trying to sell them anything. This is not a promotional email from my company saying, hey, look at our latest product, here's our new things. But it's more of a about the dogs, about dog training, dog tips, food tips, whatever. And then occasionally spreeking on some affiliate links To test things or you maybe even get a sponsorship. So make this thing self-sustaining and when you're ready to launch a product, you have an avid, rabid, loyal fan base to launch that product to as like this is the way to actually build things. So we I started looking into it Devoured everything you you showed about newsletters. You even set up a special tele I think it was telegram Newsletter channel, devoured everything in there. I went out, devoured everything in the newsletter space for three months, like everything is like. I already know this stuff, but I want to re educate myself on the latest tools, the latest strategies, and I just launched one In August, august 14th for the Amazon space. That's that I already have an audience there. Let me figure this out. Let me, like, figure out what are the best tools, the best systems, and then I can spread this to across multiple industries, multiple things, and that's what we're doing now and it's hugely Successful so far. And and AI is a part of that. But I'm not letting AI write it. AI is more of the, the creative side. It's how it it will rewrite something. If I'm trying to think of a headline, I'm like what's a better way to say X, y, z? I'll type in what's a better way, you know, to say we're ten ways that there are funny and catchy, in the tone of Perry Belcher, whatever it may be, to say this you know, give me all these cool ideas and then I mix and match, or sometimes it nails it, or I'll write a. I do a six you, you talked about this and one of your things the six second video, and so the beginning of every one of my newsletters is a six second, basic six second story. It's a personal story About me. It's something about me meeting Michael Jordan, spending a night with him in a sweet and Atlantic City the day before the night before he first retired, and you know it's crazy. Stories are about my divorce or about you know, so you're a naked girl on the balcony. I know it's, it's edgy, crazy story. But then I tie that back into the physical products and I'll use AI sometimes, maybe to help tweak that. Or if we got it some scientific document from Amazon about how the algorithm works, I'll use it to read the document, summarize it and then, you know, rewrite it with a human touch and add personality to it. So that's where using AI in other industries. I think it is brilliant. Most people aren't getting that right now. Most people just think of it as this is a threat to my job, this is a threat to you, this is the terminators coming to kill me and take over the world.   Perry: So what about? Everything's a conspiracy theory.   Kevin King: Yeah, I mean AI. I was just had just had a chat in August, so it's my father's 82nd birthday and I was sitting there for an hour explaining AI to you know, an 82 year old and a 79 year old in their mind was just, they're just was blown. They're like how do you know all this? This is, this is like science fiction movies or something, and like this is what you can do with it. And most people don't understand that. What are your thoughts on on AI right now and how people are misunderstanding or misusing and what are the best opportunities out there?   Perry: Well, circling back to your newsletter thing that the AI sucks for newsletters, it depends on the kind of newsletter you're writing.   Kevin King: That's what I said. If it's a link, newsletter or something, you can do it.   Perry: If it's a, if it's an aggregated or what you call a link newsletter, what I call a curated newsletter, they add as a really good job at writing basically a tweet and then linking to the article, and you do that like eight or nine times and you got a newsletter. But did you see the one?   Kevin King: the hustle, I think it's. They did a study. Like people are saying that. I don't know if you saw this from the hustle, but the hustle actually hired a guy, he went out and he did Let me see if I can fully automate a newsletter 100% AI so they had their programmers do some stuff and they put it out. It was about the nineties. So they would take today. You know, if today is, you know, April 6th, no, august 6th 2023, they would do August 6th 1993. What happened on that day? You know? Jurassic.   Perry: Park, the whole movie.   Kevin King: But the thing is it was repeating itself. The way it was writing was like all it was just you got to have, you got to have ins that.   Perry: Do a final review. I mean you got to have a human still, do a final review. Yeah, we've got a system. So Chad, my partner Chad, built a software system we're about to launch actually it's called Letterman and it we manage 18 newsletters a day through it and we do it with three outsourcers.   Kevin King: And the way that we do it is we hand out the we handpick what we're going to talk about.   Perry: So basically, we have a bunch of API feeds that tell us these are the stories that are trending about this subject today, and then our guys can go in and just hit, click, click, yes, yes, yes, no, yes, yes, no, delay, delay, delay. So maybe for a future issue, and then it's going to pull together those links and drop them into our software and then the software reads the article and then writes a like a tweet, that tells them to go, that compels them to go read this article. The call to action is compelling them to read the article. Right?   Kevin King: So that's SDO, then something really. It's a. Or is it a newsletter? It's a newsletter.   Perry: So this all goes into a newsletter and basically like, for instance, financials, a great example. The capitalists is ours and we want them to be able to get the gist of, like the Wall Street Journal and three thumbs swipes. And even though we're only writing, there might be 10 links in here. Right, we're writing like 140 characters on each link, compelling you to go click the link, and AI is writing that.   Kevin King: Okay.   Perry: And then they're going over and reading the actual article on the original source, right, okay, so so it's expanded.   Kevin King: It's an expanded judge report or something. It's exactly what it is.   Perry: It's not. It's not even kind of like it. It's exactly what it is Now the opposite. That's only really useful if you have a news worthy topic. Yeah. News or financial or something that's not for entertainment, financial entertainment, sports, politics things that change every single day. But if you're in the Amazon space, you got to think about it more like a, a magazine.   Kevin King: That's what I do, yeah.   Perry: So what we'll do there is find a feature article or three features. Three feature articles is even better. So we'll, let's say, for instance, my things on Amazon, and I'm talking about optimizing the perfect Amazon listing, right? I don't know whatever, but I'd go find three, the three best articles I could possibly find on that subject anywhere in the world, feed them into the AI, have them read all three and then write me a new article. And oftentimes the way we keep it interesting, we have characters, ghost writers created that right in the style of whomever right. So, but I mean really detailed. But one of the things that we found, Kevin, that's killing right now that you might find is our email list. I'm on a mission to get my email list to never send a promotion ever.   Kevin King: That's what I'm on to. I'm on to yeah.   Perry: So the way I do it is by sending out content, so like Perry might send out an email. You're doing it every day right now.   Kevin King: I get an email from you every day on copywriting Big, long email right. Yeah, big long. No, I save them. They're valuable. I mean, some of them go into my swap file.   Perry: It's a subtle.   Kevin King: It's a subtle like you're staying top of mind. You're doing it. Dan Kennedy does it right now and there's a couple others. He's doing that with Russell, but I and they're valuable. You can just read that and never do another thing. But it's you're staying top of mind and then you'll put in something OPS, remember the AI summits coming or whatever that stuff works.   Perry: But what's about to happen with those lists and we're doing another list right now is, once you open that thing about headline writing right, I can fire off a straight up promotion to you.   Kevin King: Yeah, you're segmenting based on what I click and what I do open and read Instantly.   Perry: So you're opening reading my article, right? So you just read my article about headlines and then the. Then you close that article down close that email. The next email in your queue is from me going hey, fibs, copywriting course is 50% off today. Great deal, and you're already so pre-framed to that. The open, the open rate on that second email is like 70 to 80%. Yeah, yeah, we're doing that.   Kevin King: We're going to do that in the product space, where we will watch what people click and if they're always looking on the docs and story, we'll start feeding them more docs. And there's a tool out there, there's a what. There's a tool that does this for the AMA right now, that that does newsletters, where it automated it watches everything and automatically get basically creates a personalized feed in a newsletter we want to Instagram.   Perry: We basically want to Instagram the newsletter business. So if you're only opening dots and stuff, then we want to deliver dots and stuff to you. If you're only delivering lip plumper articles, then we want to deliver a lip plumper off offers to you and and make the newsletter more lip related.   Kevin King: If that's your thing you're into in a makeup space, we're talking about it for newsletters, for you know Amazon sellers, but you can do this for physical products. You can do this for any industry and then leverage off of that. You see that they're always by clicking on the docs and ads. Then you start driving them to your print on demand docs and t-shirts, or you start driving them to Amazon to buy docs and bowls or whatever it's there's a guy that sells drones on Amazon.   Perry: You should have a drone newsletter. You know. You absolutely should have a drone newsletter. We say when, when Perry and I are talking about newsletters there's a big misconception in my mind.   Kevin King: Maybe you have a little bit different take on it, but so many people have what they call a newsletter. You go to their website you know the drone maker, sign up for our newsletter and the newsletter is nothing but a promotional email. It's like hey, we just announced two new parts. We just announced this to me. That's not a newsletter. That's a good one. That's not a newsletter.   Perry: That's a good one. You're not going to get deliverability on it either I mean a newsletter provides value.   Kevin King: It's like 95% value, 5% promotional. It's valued, something you want to get it to where people look forward to getting it, not, oh God dang. I just got another freaking email from drones. Or us Delete, delete, delete. They like I got to open this because they may have some cool tactic in there on how to fly my drone, you know, or in heavy winds, or whatever. Whatever it may be. That's where you got to be thinking when you're doing this, and AI is a great tool. And I always remember something you said when just as a quick aside here, it's a quote I often re-quote you on this and credit to you but you always said, when it comes to selling products on Amazon, people don't buy products on Amazon. They buy photos, absolutely, and so can you talk about just for the Amazon people.   Perry: Nobody can buy a picture. Nobody can buy anything on the internet. It's impossible. All you can do is buy a picture or something that's. Or if you're writing copy, you're creating a mental picture of a thing, right? So yeah, I'm a big believer in product photography being a giant piece of what you do and making something that's demonstrable. If you can actually show how it works in a 30 second video clip, I think that's different than anything. You know that works more powerfully than anything, because you've got to, and design I think you're seeing now is becoming more and more important the quality of your design, because we don't have any way to trust companies, right? You don't really have a way. It used to be the old Dan Kennedy world and Dan at the time was right. You know, ugly sells and pretty doesn't, right? The truth is today, pretty outsells ugly, and that's just. We've proved it eight times, eight times over. Pretty outsells ugly, and especially if you're selling a physical good, right? So don't skimp on the amount of money you spend on photography and photo editing and all those things. I was in was in Kevin interesting thing I was in Guangzhou, China, and I went to this illustration company. They do illustrations, you know. Have you been to? You've been to Yiwu before? Yeah, I've been able. Ok, so you know, upstairs in Yiwu, like on the fourth and fifth floor, it's all service companies, web companies, and I found a company up there and they were doing watches so they would take a watch. You can't take a good enough photograph of a watch for that photograph to actually work in a magazine. It's an impossibility. So what they do is they take a picture of the watch and they pull it into an illustration computer and then there's a program just for jewelry that has all of these textures and paint brushes and all that and they actually build the watch on top of the photo. They build an illustration of the watch and if you ever pick up a magazine and really look at, get a magnifying glass and look at the picture of the Rolex on the back right, you can see where there's an illustration piece cut here or there. You don't see any of the photo. They completely overlay it. But sometimes it takes these guys two weeks to set on illustrator and replace every little pixel dot. Everything is a vector and then they send that off and that.   Kevin King: But now AI can do a lot of that.   Perry: Yeah, I don't know how much I would trust it to do that, but yeah, it probably can. It can certainly enhance the photos a lot. You're seeing AI photo enhancement become a really big deal. Have you seen that thing that takes? I mentioned it at AIBotson. I'm trying to think of the name of it now Topaz.   Kevin King: Yeah.   Perry: Topazai. Well, you can take your old video footage and it'll turn it into 4K footage. It looks pretty doggone good. I mean, you take an old piece of footage that you shot 10 years ago and you run it through there and it'll give you a whole face lift and make it really appear to be a 4K footage.   Kevin King: Yeah, as Remini does that for photos, you can have some old photo or even something you downloaded, some stock image you downloaded online. It's kind of low res because they want you to go pay for the high res. Just download the low res, run it through Remini and it'll upscale it. And upscaleio is another one. There's a bunch of them and some of it's like holy cow. This is amazing stuff.   Perry: Another year from now, probably most of the things that we're using services for now will be you know you don't have to. We're making a lot of money right now in the Philippines by our outsource company uses AI to do things for people. So if you wanted an illustration of a product or whatever, you could send it to man. We're going to charge X for that, but we're actually going to use tools that cut our labor time down by 80, 90%. We haven't got it to where we can cut it all the way out yet and we still hire art directors. You know, really, but it allows you to, instead of hiring 30 B minus designers and you know an art director, you use AI and you get three or three or so, three or four really high level art directors and you don't need all the carpenters anymore. Right, and if you've seen the way they're building houses now, with the brick laying machines and all that all the carpenters, all the framers that won't be a profession in another 24 months.   Kevin King: Well, that's the scare I think that general public has when it comes to AI is like, well, it's going to take my job and so I don't want that, but look what happened in the industrial revolution, look what happened when the wheel wasn't been it. People will adapt and if you don't adapt, you're going to get left behind. And I think right now, one of the biggest skills if you're listening to this and you're, you know, in high school or college or you're young and still trying to figure you need to learn how to do prompting Prompting. I think good prompting versus okay prompting can make a world of difference with AI. As this gets more sophisticated, being good at prompting is going to be a major skill set that's high in demand. Would you agree with that?   0:55:51 - Perry: I think so. It's funny though, you know. Now you can go to open AI and say write me a mid-journey prompt. Yeah you know this and use this camera lens and this but you don't want the camera lens.   Kevin King: That's where photographers and artists right now are.   Perry: You kind of don't. You can actually have open AI right the mid-journey prompt for you. It's crazy and a lot of people are doing that and I think that's. I think prompting is going to become easier and easier, but it's still going to require imagination.   Kevin King: You know.   Perry: No, no artificial intelligence engines ever going to be able to replace imagination. You know it's not going to happen. So I think that we're we're we're fine for, you know, a good long while. I don't see it being a problem, but there's good money to be made right now with just arbitrage. You know how it is, kevin. You've been around this business long enough. When, anytime, a market is inefficient, that's when all the money's made, right, and right now you got people who need things done. Nobody wants to work, right? So you know AI is just filling the slot perfectly, so we can offer services. Now that used to be. You know, like. We'll do unlimited video editing for $2,000 a month, right? Well, we're doing 90% of that video editing with AI. If we were doing it by hand, we'd have searched $10,000 a month, right, and the end of the day, the customer doesn't care. The customer's getting the desired product delivered within a timeline. They don't really care if you did it yourself or if a robot did it. And if they do care, well, it's probably not your kind of customer, right? So all the stuff that you guys go through of writing product descriptions and all your SEO, your keyword loading and your product photo enhancement and all the stuff that you do, I'd say within a year, probably. Right now, if you're studious you can do 90% of it?   Kevin King: Yeah, you can, but within a year. I mean, it's been a big thing. I just was in another mastermind with a big Chinese seller. He does $50 million a year or something. He's based in China and sells into the US and he said that AI has been a leveling ground for the Chinese sellers.   Perry: Yeah, of course.   Kevin King: Because now they used to, you'd have all that broken English and stuff on listings or they couldn't understand the culture to write it in the right way. And he said with AI, that advantage is gone for Westerners, so you got to step up your game and now it's in. Still, you have an advantage in branding or innovation or some other areas, but it's leveling the playing field for a lot of people.   Perry: Yeah, we found it. We found with Mid Journey packaging design.   Kevin King: Yeah.   Perry: It's been. Packaging design mockups have been amazing. We've come up with some really great packaging ideas that we wouldn't have come up with and for the most part you can send those over to your factories in China and get a reasonable.   Kevin King: When people are doing that for product. Now they'll come up with a product idea like, hey, I want to make a I don't know a new dog bowl. You'll have the AI create. You know, they'll give it some parameters. It needs to be this, it needs to be slow the dog down from eating or not slip on the floor, whatever Right and have the AI create a hundred different models of it. Just boom, boom, boom. Use 3D illustrations, put that into a tool like PickFu, let people vote on it and then, you know, have the top couple. You know, go to molding and make prototypes and then do some additional testing. You couldn't do that. That's just what you can do. Now is just some of the times, sometimes almost mind boggling.   Perry: And robotics have really taken down molding costs.   Kevin King: Yeah.   Perry: Back when you and I started, you know I want to custom mold for this. Well, it'll be $100,000. Now you know, six grand you know, whatever it lasts, you know, depending on what you're molding, but it's crazy how cheap molding costs have gotten.   Kevin King: So we're almost out of time here. Actually we've gone over, but just real quick before we wrap up. What are? What would you say are three things out there that you're seeing right now that either hot opportunities that people need to be paying attention to, or three big, or maybe even three big mistakes that people are making when it comes to trying to sell physical products to people.  

PhoneBoy talks to Adam Gray, CTO of Novacoast about how ChatGPT is used by threat actors to compromise systems, the GPT-4 System Card, where ChatGPT seems to be useful in general with respect to cyber security, ChatGPT writing legal briefs, what early search engines and ChatGPT have in common, and how the more some things change, the more they stay the same.

AND WE ARE BACK!We needed a moment, took one, and noticed a few things:It's not a depression, it's more of a "there is to much and I don't want to do it" frame of mind.We may need hugs.We are more than what you may see, think, and perceive - that goes for our own perception as well.We need tangible tools to encourage ourselves to live the organized, effective, and HAPPY life we want.We have some ideas, from our own lives, that work for us - and may work for you! Christine blows it out of the park with her "Mindset Concepts" for the week (check out the "Team Stine" page on Facebook), Susan discusses the MANY notebooks and digital concepts she puts to good use, and much much more.Our favorite: the "September To Remember" bingo board!  Whoa!Then we taste Alta Vista Terroir Select Cabernet Sauvignon from Argentina.  LAST SOUTH AMERICA WINE FOR THE MONTH OF SUSAN! This wine is a blend of two wine growing areas from the Uco Valley; an area with healthy vines, lower yields, high quality fruit and wines that show a generous body and a pretty balanced acidity. Tired of the same same domestic Cabs you are buying that may appear clunky (not all - don't come at me) or that do not show the elegance you are looking for? We have decided this Cab is our drink alone, charcuterie, light pasta, enjoyment Cab.  Taste along and let us know what you think!More about Alta Vista Terroir Selection Cabernet Sauvignon:https://altavistawines.com/wp-content/uploads/2016/10/FT-TS-CS-17-EN.pdfMore about Blue Zones:https://www.bluezones.comhttps://www.netflix.com/title/81214929More about Trello:https://play.google.com/store/apps/details?id=com.trello&hl=en_US&gl=US&pli=1More about Burke Decor:https://www.burkedecor.com/?gclid=Cj0KCQjw9fqnBhDSARIsAHlcQYSZupv4JtLZCx-iTOCFg2D7Z7x46sWNKewiMAvhTDM8s38LLywRptYaAttPEALw_wcBFind Christine D'Angelo:Instagram: @christine_dangelo_ Facebook: @Christine Casiero D'AngeloSet yourself up with Christine D"Angelo as a COACH on the 1st Phorm App!  https://www.1stphorm.app/StineD***JOIN Christine's Facebook Page "Team Shine Fitness": https://www.facebook.com/groups/teamstinefitness/Find Susan Pajak:Instagram: @winegirlgonewildFacebook: @Susan PajakTwitter: @spajakPersonal Blog: winegirlgonewild.comhttps://winegirlgonewild.comCome for a chat, stay for a sip, and leave us a comment!

We are all special, but do you deserve those special things? Entitlement is a tricky road to walk down; are you living that "you owe me" attitude, are you argumentative at every turn of the road, do you feet you have been neglected or treated unfairly? Or - is this just a buzz word and fall out from the aggravated world we are living in now? Who can say: listen and find out what we think!  Come for a chat, stay for a sip!Then we taste Cabernet Sauvignon from the winery Alta Vista located in Mendoza Argentina.  Here is the low down:Former Ownership: Famous Champagne House, Piper HeidsieckWinemaking Intent: French approach to viticulture and stylistic narrativeOld World Style in a New World Package: Elegant, full body, obvious dark berry fruit components but leading with earthy components as well.  Hand picked, one bottle per vine, well balanced, and forward with its personality! We pair this wine with burgers, arroz con pollo, chicken cutlets - Christine approved!Welcome to the month of Susan! (and happy anniversary to us - ONE YEAR!)More about Alta Vista Cabernet Sauvignon:https://www.kobrandwineandspirits.com/brand-page/alta-vista/Real AF with Andy Frisella Podcast, Episode 553:https://podcasts.apple.com/us/podcast/real-af-with-andy-frisella/id1012570406?i=1000622624036The Mel Robbins Podcast: From PMS to Menopause:https://podcasts.apple.com/us/podcast/the-mel-robbins-podcast/id1646101002?i=1000622473367Find Christine D'Angelo:Instagram: @christine_dangelo_ Facebook: @Christine Casiero D'AngeloSet yourself up with Christine D"Angelo as a COACH on the 1st Phorm App!  https://www.1stphorm.app/StineD***JOIN Christine's Facebook Page "Team Shine Fitness": https://www.facebook.com/groups/teamstinefitness/Find Susan Pajak:Instagram: @winegirlgonewildFacebook: @Susan PajakTwitter: @spajakPersonal Blog: winegirlgonewild.comhttps://winegirlgonewild.comCome for a chat, stay for a sip, and leave us a comment!

Pablo Molina, associate vice president of information technology and chief information security officer at Drexel University and adjunct professor at Georgetown University, leads the conversation on the implications of artificial intelligence in higher education.   FASKIANOS: Welcome to CFR's Higher Education Webinar. I'm Irina Faskianos, vice president of the National Program and Outreach here at CFR. Thank you for joining us. Today's discussion is on the record, and the video and transcript will be available on our website, CFR.org/Academic, if you would like to share it with your colleagues. As always, CFR takes no institutional positions on matters of policy. We are delighted to have Pablo Molina with us to discuss implications of artificial intelligence in higher education. Dr. Molina is chief information security officer and associate vice president at Drexel University. He is also an adjunct professor at Georgetown University. Dr. Molina is the founder and executive director of the International Applies Ethics in Technology Association, which aims to raise awareness on ethical issues in technology. He regularly comments on stories about privacy, the ethics of tech companies, and laws related to technology and information management. And he's received numerous awards relating to technology and serves on the board of the Electronic Privacy Information Center and the Center for AI and Digital Policy. So Dr. P, welcome. Thank you very much for being with us today. Obviously, AI is on the top of everyone's mind, with ChatGPT coming out and being in the news, and so many other stories about what AI is going to—how it's going to change the world. So I thought you could focus in specifically on how artificial intelligence will change and is influencing higher education, and what you're seeing, the trends in your community. MOLINA: Irina, thank you very much for the opportunity, to the Council on Foreign Relations, to be here and express my views. Thank you, everybody, for taking time out of your busy schedules to listen to this. And hopefully, I'll have the opportunity to learn much from your questions and answer some of them to the best of my ability. Well, since I'm a professor too, I like to start by giving you homework. And the homework is this: I do not know how much people know about artificial intelligence. In my opinion, anybody who has ever used ChatGPT considers herself or himself an expert. To some extent, you are, because you have used one of the first publicly available artificial intelligence tools out there and you know more than those who haven't. So if you have used ChatGPT, or Google Bard, or other services, you already have a leg up to understand at least one aspect of artificial intelligence, known as generative artificial intelligence. Now, if you want to learn more about this, there's a big textbook about this big. I'm not endorsing it. All I'm saying, for those people who are very curious, there are two great academics, Russell and Norvig. They're in their fourth edition of a wonderful book that covers every aspect of—technical aspect of artificial intelligence, called Artificial Intelligence: A Modern Approach. And if you're really interested in how artificial intelligence can impact higher education, I recommend a report by the U.S. Department of Education that was released earlier this year in Washington, DC from the Office of Education Technology. It's called Artificial Intelligence and Future of Teaching and Learning: Insights and Recommendations. So if you do all these things and you read all these things, you will hopefully transition from being whatever expert you were before—to a pandemic and Ukrainian war expert—to an artificial intelligence expert. So how do I think that all these wonderful things are going to affect artificial intelligence? Well, as human beings, we tend to overestimate the impact of technology in the short run and really underestimate the impact of technology in the long run. And I believe this is also the case with artificial intelligence. We're in a moment where there's a lot of hype about artificial intelligence. It will solve every problem under the sky. But it will also create the most catastrophic future and dystopia that we can imagine. And possibly neither one of these two are true, particularly if we regulate and use these technologies and develop them following some standard guidelines that we have followed in the past, for better or worse. So how is artificial intelligence affecting higher education? Well, number one, there is a great lack of regulation and legislation. So if you know, for example around this, OpenAI released ChatGPT. People started trying it. And all of a sudden there were people like here, where I'm speaking to you from, in Italy. I'm in Rome on vacation right now. And Italian data protection agency said: Listen, we're concerned about the privacy of this tool for citizens of Italy. So the company agreed to establish some rules, some guidelines and guardrails on the tool. And then it reopened to the Italian public, after being closed for a while. The same thing happened with the Canadian data protection authorities. In the United States, well, not much has happened, except that one of the organizations on which board I serve, the Center for Artificial Intelligence and Digital Policy, earlier this year in March of 2023 filed a sixty-four-page complaint with the Federal Trade Commission. Which is basically we're asking the Federal Trade Commission: You do have the authority to investigate how these tools can affect the U.S. consumers. Please do so, because this is your purview, and this is your responsibility. And we're still waiting on the agency to declare what the next steps are going to be. If you look at other bodies of legislation or regulation on artificial intelligence that can help us guide artificial intelligence, well, you can certainly pay attention to the U.S. Congress. And what is the U.S. Congress doing? Yeah, pretty much that, not much, to be honest. They listen to Sam Altman, the founder of ChatGPT, who recently testified before Congress, urging Congress to regulate artificial intelligence. Which is quite clever on his part. So it was on May 17 that he testified that we could be facing catastrophic damage ahead if artificial intelligence technology is not regulated in time. He also sounded the alarm about counterfeit humans, meaning that these machines could replace what we think a person is, at least virtually. And also warned about the end of factual evidence, because with artificial intelligence anything can be fabricated. Not only that, but he pointed out that artificial intelligence could start wars and destroy democracy. Certainly very, very grim predictions. And before this, many of the companies were self-regulating for artificial intelligence. If you look at Google, Microsoft, Facebook now Meta. All of them have their own artificial intelligence self-guiding principles. Most of them were very aspirational. Those could help us in higher education because, at the very least, it can help us create our own policies and guidelines for our community members—faculty, staff, students, researchers, administrators, partners, vendors, alumni—anybody who happens to interact with our institutions of higher learning. Now, what else is happening out there? Well, we have tons, tons of laws that have to do with the technology and regulations. Things like the Gramm-Leach-Bliley Act, or the Securities and Exchange Commission, the Sarbanes-Oxley. Federal regulations like FISMA, and Cybersecurity Maturity Model Certification, Payment Card Industry, there is the Computer Fraud and Abuse Act, there is the Budapest Convention where cybersecurity insurance providers will tells us what to do and what not to do about technology. We have state laws and many privacy laws. But, to be honest, very few artificial intelligence laws. And it's groundbreaking in Europe that the European parliamentarians have agreed to discuss the Artificial Intelligence Act, which could be the first one really to be passed at this level in the world, after some efforts by China and other countries. And, if adopted, could be a landmark change in the adoption of artificial intelligence. In the United States, even though Congress is not doing much, what the White House is trying to position itself in the realm of artificial intelligence. So there's an executive order in February of 2023—that many of us in higher education read because, once again, we're trying to find inspiration for our own rules and regulations—that tells federal agencies that they have to root out bias in the design and use of new technologies, including artificial intelligence, because they have to protect the public from algorithm discrimination. And we all believe this. In higher education, we believe in being fair and transparent and accountable. I would be surprised if any of us is not concerned about making sure that our technology use, our artificial technology use, does not follow these particular principles as proposed by the Organization for Economic Cooperation and Development, and many other bodies of ethics and expertise. Now, the White House also announced new centers—research and development centers with some new national artificial intelligence research institutes. Many of us will collaborate with those in our research projects. A call for public assessments of existing generative artificial intelligence systems, like ChatGPT. And also is trying to enact or is enacting policies to ensure that U.S. government—the U.S. government, the executive branch, is leading by example when mitigating artificial intelligence risks and harnessing artificial intelligence opportunities. Because, in spite of all the concerns about this, it's all about the opportunities that we hope to achieve with artificial intelligence. And when we look at how specifically can we benefit from artificial intelligence in higher education, well, certainly we can start with new and modified academic offerings. I would be surprised if most of us will not have degrees—certainly, we already have degrees—graduate degrees on artificial intelligence, and machine learning, and many others. But I would be surprised if we don't even add some bachelor's degrees in this field, or we don't modify significantly some of our existing academic offerings to incorporate artificial intelligence in various specialties, our courses, or components of the courses that we teach our students. We're looking at amazing research opportunities, things that we'll be able to do with artificial intelligence that we couldn't even think about before, that are going to expand our ability to generate new knowledge to contribute to society, with federal funding, with private funding. We're looking at improved knowledge management, something that librarians are always very concerned about, the preservation and distribution of knowledge. The idea would be that artificial intelligence will help us find better the things that we're looking for, the things that we need in order to conduct our academic work. We're certainly looking at new and modified pedagogical approaches, new ways of learning and teaching, including the promise of adaptive learning, something that really can tell students: Hey, you're not getting this particular concept. Why don't you go back and study it in a different way with a different virtual avatar, using simulations or virtual assistance? In almost every discipline and academic endeavor. We're looking very concerned, because we're concerned about offering, you know, a good value for the money when it comes to education. So we're hoping to achieve extreme efficiencies, better ways to run admissions, better ways to guide students through their academic careers, better way to coach them into professional opportunities. And many of this will be possible thanks to artificial intelligence. And also, let's not forget this, but we still have many underserved students, and they're underserved because they either cannot afford education or maybe they have physical or cognitive disabilities. And artificial intelligence can really help us reach to those students and offer them new opportunities to advance their education and fulfill their academic and professional goals. And I think this is a good introduction. And I'd love to talk about all the things that can go wrong. I'd love to talk about all the things that we should be doing so that things don't go as wrong as predicted. But I think this is a good way to set the stage for the discussion. FASKIANOS: Fantastic. Thank you so much. So we're going to go all of you now for your questions and comments, share best practices. (Gives queuing instructions.) All right. So I'm going first to Gabriel Doncel has a written question, adjunct faculty at the University of Delaware: How do we incentivize students to approach generative AI tools like ChatGPT for text in ways that emphasize critical thinking and analysis? MOLINA: I always like to start with a difficult question, so I very much, Gabriel Doncel, for that particular question. And, as you know, there are several approaches to adopting tools like ChatGPT on campus by students. One of them is to say: No, over my dead body. If you use ChatGPT, you're cheating. Even if you cite ChatGPT, we can consider you to be cheating. And not only that, but some institutions have invested in tools that can detect whether or something was written with ChatGPT or similar rules. There are other faculty members and other academic institutions that are realizing these tools will be available when these students join the workforce. So our job is to help them do the best that they can by using these particular tools, to make sure they avoid some of the mishaps that have already happened. There are a number of lawyers who have used ChatGPT to file legal briefs. And when the judges received those briefs, and read through them, and looked at the citations they realized that some of the citations were completely made up, were not real cases. Hence, the lawyers faced professional disciplinary action because they used the tool without the professional review that is required. So hopefully we're going to educate our students and we're going to set policy and guideline boundaries for them to use these, as well as sometimes the necessary technical controls for those students who may not be that ethically inclined to follow our guidelines and policies. But I think that to hide our heads in the sand and pretend that these tools are not out there for students to use would be—it's a disserve to our institutions, to our students, and the mission that we have of training the next generation of knowledge workers. FASKIANOS: Thank you. I'm going to go next to Meena Bose, who has a raised hand. Meena, if you can unmute yourself and identify yourself. Q: Thank you, Irina. Thank you for this very important talk. And my question is a little—(laughs)—it's formative, but really—I have been thinking about what you were saying about the role of AI in academic life. And I don't—particularly for undergraduates, for admissions, advisement, guidance on curriculum. And I don't want to have my head in the sand about this, as you just said—(laughs)—but it seems to me that any kind of meaningful interaction with students, particularly students who have not had any exposure to college before, depends upon kind of multiple feedback with faculty members, development of mentors, to excel in college and to consider opportunities after. So I'm struggling a little bit to see how AI can be instructive for that part of college life, beyond kind of providing information, I guess. But I guess the web does that already. So welcome your thoughts. Thank you. FASKIANOS: And Meena's at Hofstra University. MOLINA: Thank you. You know, it's a great question. And the idea that everybody is proposing right here is we are not—artificial intelligence companies, at least at first. We'll see in the future because, you know, it depends on how it's regulated. But they're not trying, or so they claim, to replace doctors, or architects, or professors, or mentors, or administrators. They're trying to help those—precisely those people in those professions, and the people they served gain access to more information. And you're right in a sense that that information is already on the web. But we've aways had a problem finding that information regularly on the web. And you may remember that when Google came along, I mean, it swept through every other search engine out there AltaVista, Yahoo, and many others, because, you know, it had a very good search algorithm. And now we're going to the next level. The next level is where you ask ChatGPT in human-natural language. You're not trying to combine the three words that say, OK, is the economics class required? No, no, you're telling ChatGPT, hey, listen, I'm in the master's in business administration at Drexel University and I'm trying to take more economic classes. What recommendations do you have for me? And this is where you can have a preliminary one, and also a caveat there, as most of these search engine—generative AI engines already have, that tell you: We're not here to replace the experts. Make sure you discuss your questions with the experts. We will not give you medical advice. We will not give you educational advice. We're just here, to some extent, for guiding purposes and, even now, for experimental and entertainment purposes. So I think you are absolutely right that we have to be very judicious about how we use these tools to support the students. Now, that said, I had the privilege of working for public universities in the state of Connecticut when I was the CIO. I also had the opportunity early in my career to attend public university in Europe, in Spain, where we were hundreds of students in class. We couldn't get any attention from the faculty. There were no mentors, there were no counselors, or anybody else. Is it better to have nobody to help you or is it better to have at least some technology guidance that can help you find the information that otherwise is spread throughout many different systems that are like ivory towers—emissions on one side, economics on the other, academics advising on the other, and everything else. So thank you for a wonderful question and reflection. FASKIANOS: I'm going to take the next question written from Dr. Russell Thomas, a senior lecturer in the Department of International Relations and Diplomatic Studies at Cavendish University in Uganda: What are the skills and competencies that higher education students and faculty need to develop to think in an AI-driven world? MOLINA: So we could argue here that something very similar has happened already with many information technologies and communication technologies. It is the understanding at first faculty members did not want to use email, or the web, or many other tools because they were too busy with their disciplines. And rightly so. They were brilliant economists, or philosophers, or biologists. They didn't have enough time to learn all these new technologies to interact with the students. But eventually they did learn, because they realized that it was the only way to meet the students where they were and to communicate with them in efficient ways. Now, I have to be honest; when it comes to the use of technology—and we'll unpack the numbers—it was part of my doctoral dissertation, when I expanded the adoption of technology models, that tells you about early adopters, and mainstream adopters, and late adopters, and laggards. But I uncovered a new category for some of the institutions where I worked called the over-my-dead-body adopters. And these were some of the faculty members who say: I will never switch word processors. I will never use this technology. It's only forty years until I retire, probably eighty more until I die. I don't have to do this. And, to be honest, we have a responsibility to understand that those artificial intelligence tools are out there, and to guide the students as to what is the acceptable use of those technologies within the disciplines and the courses that we teach them in. Because they will find those available in a very competitive work market, in a competitive labor market, because they can derive some benefit from them. But also, we don't want to shortchange their educational attainment just because they go behind our backs to copy and paste from ChatGPT, learning nothing. Going back to the question by Gabriel Doncel, not learning to exercise the critical thinking, using citations and material that is unverified, that was borrowed from the internet without any authority, without any attention to the different points of view. I mean, if you've used ChatGPT for a while—and I have personally, even to prepare some basic thank-you speeches, which are all very formal, even to contest a traffic ticket in Washington, DC, when I was speeding but I don't want to pay the ticket anyway. Even for just research purposes, you could realize that most of the writing from ChatGPT has a very, very common style. Which is, oh, on the one hand people say this, on the other hand people say that. Well, the critical thinking will tell you, sure, there are two different opinions, but this is what I think myself, and this is why I think about this. And these are some of the skills, the critical thinking skills, that we must continue to teach the students and not to, you know, put blinds around their eyes to say, oh, continue focusing only on the textbook and the website. No, no. Look at the other tools but use them judiciously. FASKIANOS: Thank you. I'm going to go next to Clemente Abrokwaa. Raised hand, if you can identify yourself, please. Q: Hi. Thanks so much for your talk. It's something that has been—I'm from Penn State University. And this is a very important topic, I think. And some of the earlier speakers have already asked the questions I was going to ask. (Laughs.) But one thing that I would like to say that, as you said, we cannot bury our heads in the sand. No matter what we think, the technology is already here. So we cannot avoid it. My question, though, is what do you think about the artificial intelligence, the use of that in, say, for example, graduate students using it to write dissertations? You did mention about the lawyers that use it to write their briefs, and they were caught. But in dissertations and also in class—for example, you have students—you have about forty students. You give a written assignment. You make—when you start grading, you have grading fatigue. And so at some point you lose interest of actually checking. And so I'm kind of concerned about that how it will affect the students' desire to actually go and research without resorting to the use of AI. MOLINA: Well, Clemente, fellow colleague from the state of Pennsylvania, thank you for that, once again, both a question and a reflection here. Listen, many of us wrote our doctoral dissertations—mine at Georgetown. At one point of time, I was so tired of writing about the same topics, following the wonderful advice, but also the whims of my dissertation committee, that I was this close from outsourcing my thesis to China. I didn't, but I thought about it. And now graduate students are thinking, OK, why am I going through the difficulties of writing this when ChatGPT can do it for me and the deadline is tomorrow? Well, this is what will distinguish the good students and the good professionals from the other ones. And the interesting part is, as you know, when we teach graduate students we're teaching them critical thinking skills, but also teaching them now to express themselves, you know, either orally or in writing. And writing effectively is fundamental in the professions, but also absolutely critical in academic settings. And anybody who's just copying and pasting from ChatGPT to these documents cannot do that level of writing. But you're absolutely right. Let's say that we have an adjunct faculty member who's teaching a hundred students. Will that person go through every single essay to find out whether students were cheating with ChatGPT? Probably not. And this is why there are also enterprising people who are using artificial intelligence to find out and tell you whether a paper was written using artificial intelligence. So it's a little bit like this fighting of different sources and business opportunities for all of them. And we've done this. We've used antiplagiarism tools in the past because we knew that students were copying and pasting using Google Scholar and many other sources. And now oftentimes we run antiplagiarism tools. We didn't write them ourselves. Or we tell the students, you run it yourself and you give it to me. And make sure you are not accidentally not citing things that could end up jeopardizing your ability to get a graduate degree because your work was not up to snuff with the requirements of our stringent academic programs. So I would argue that this antiplagiarism tools that we're using will more often than not, and sooner than expected, incorporate the detection of artificial intelligence writeups. And also the interesting part is to tell the students, well, if you do choose to use any of these tools, what are the rules of engagement? Can you ask it to write a paragraph and then you cite it, and you mention that ChatGPT wrote it? Not to mention, in addition to that, all the issues about artificial intelligence, which the courts are deciding now, regarding the intellectual property of those productions. If a song, a poem, a book is written by an artificial intelligence entity, who owns the intellectual property for those works produced by an artificial intelligence machine? FASKIANOS: Good question. We have a lot of written questions. And I'm sure you don't want to just listen to my voice, so please do raise your hands. But we do have a question from one of your colleagues, Pablo, Pepe Barcega, who's the IT director at Drexel: Considering the potential biases and limitations of AI models, like ChatGPT, do you think relying on such technology in the educational domain can perpetuate existing inequalities and reinforce systemic biases, particularly in terms of access, representation, and fair evaluation of students? And Pepe's question got seven upvotes, we advanced it to the top of the line. MOLINA: All right, well, first I have to wonder whether he used ChatGPT to write the question. But I'm going to leave it that. Thank you. (Laughter.) It's a wonderful question. One of the greatest concerns we have had, those of us who have been working on artificial intelligence digital policy for years—not this year when ChatGPT was released, but for years we've been thinking about this. And even before artificial intelligence, in general with algorithm transparency. And the idea is the following: That two things are happening here. One is that we're programming the algorithms using instructions, instructions created by programmers, with all their biases, and their misunderstandings, and their shortcomings, and their lack of context, and everything else. But with artificial intelligence we're doing something even more concerning than that, which is we have some basic algorithms but then we're feeling a lot of information, a corpus of information, to those algorithms. And the algorithms are fine-tuning the rules based on those. So it's very, very difficult for experts to explain how an artificial intelligence system actually makes decisions, because we know the engine and we know the data that we fed to the engine, but we don't know the real outcome how those decisions are being made through neural networks, through all of the different systems that we have and methods that we have for artificial intelligence. Very, very few people understand how those work. And those are so busy they don't have time to explain how the algorithm works for others, including the regulators. Let's remember some of the failed cases. Amazon tried this early. And they tried this for selecting employees for Amazon. And they fed all the resumes. And guess what? It turned out that most of the recommendations were to hire young white people who had gone to Ivy League schools. Why? Because their first employees were feeding those descriptions, and they had done extremely well at Amazon. Hence, by feeding that information of past successful employees only those were there. And so that puts away the diversity that we need for different academic institutions, large and small, public and private, from different countries, from different genders, from different ages, from different ethnicities. All those things went away because the algorithm was promoting one particular one. Recently I had the opportunity to moderate a panel in Washington, DC, and we had representatives from the Equal Employment Opportunity Commission. And they told us how they investigated a hiring algorithm from a company that was disproportionately recommending that they hired people whose first name was Brian and had played lacrosse in high school because, once again, a disproportionate number of people in that company had done that. And the algorithm realized, oh, this must be important characteristics to hire people for this company. Let's not forget, for example, with the artificial facial recognition and artificial intelligence by Amazon Rekog, you know, the facial recognition software, that the American Civil Liberties Union, decided, OK, I'm going to submit the pictures of all the congressmen to this particular facial recognition engine. And it turned out that it misidentified many of them, particularly African Americans, as felons who had been convicted. So all these artificial—all these biases could have really, really bad consequences. Imagine that you're using this to decide who you admit to your universities, and the algorithm is wrong. You know, you are making really biased decisions that will affect the livelihood of many people, but also will transform society, possibly for the worse, if we don't address this. So this is why the OECD, the European Union, even the White House, everybody is saying: We want this technology. We want to derive the benefits of this technology, while curtailing the abuses. And it's fundamental we achieve transparency. We are sure that these algorithms are not biased against the people who use them. FASKIANOS: Thank you. So I'm going to go next to Emily Edmonds-Poli, who is a professor at the University of San Diego: We hear a lot about providing clear guidelines for students, but for those of us who have not had a lot of experience using ChatGPT it is difficult to know what clear guidelines look like. Can you recommend some sources we might consult as a starting point, or where we might find some sample language? MOLINA: Hmm. Well, certainly this is what we do in higher education. We compete for the best students and the best faculty members. And we sometimes compete a little bit to be first to win groundbreaking research. But we tend to collaborate with everything else, particularly when it comes to policy, and guidance, and rules. So there are many institutions, like mine, who have already assembled—I'm sure that yours has done the same—assembled committees, because assembling committees and subcommittees is something we do very well in higher education, with faculty members, with administrators, even with the student representation to figure out, OK, what should we do about the use of artificial intelligence on our campus? I mentioned before taking a look at the big aspirational declarations by Meta, and Google, and IBM, and Microsoft could be helpful for these communities to look at this. But also, I'm a very active member of an organization known as EDUCAUSE. And EDUCAUSE is for educators—predominantly higher education educators. Administrators, staff members, faculty members, to think about the adoption of information technology. And EDUCAUSE has done good work on this front and continues to do good work on this front. So once again, EDUCAUSE and some of the institutions have already published their guidelines on how to use artificial intelligence and incorporate that within their academic lives. And now, that said, we also know that even though all higher education institutions are the same, they're all different. We all have different values. We all believe in different uses of technology. We trust more or less the students. Hence, it's very important that whatever inspiration you would take, you work internally on campus—as you have done with many other issues in the past—to make sure it really reflects the values of your institution. FASKIANOS: So, Pablo, would you point to a specific college or university that has developed a code of ethics that addresses the use of AI for their academic community beyond your own, but that is publicly available? MOLINA: Yeah, I'm going to be honest, I don't want to put anybody on the spot. FASKIANOS: OK. MOLINA: Because, once again, there many reasons. But, once again, let me repeat a couple resources. One is of them is from the U.S. Department of Education, from the Office of Educational Technology. And the article is Artificial Intelligence and Future of Teaching and Learning: Insights and Recommendations, published earlier this year. The other source really is educause.edu. And if you look at educause.edu on artificial intelligence, you'll find links to articles, you'll find links to universities. It would be presumptuous of me to evaluate whose policies are better than others, but I would argue that the general principles of nonbiased, transparency, accountability, and also integration of these tools within the academic life of the institution in a morally responsible way—with concepts by privacy by design, security by design, and responsible computing—all of those are good words to have in there. Now, the other problem with policies and guidelines is that, let's be honest, many of those have no teeth in our institutions. You know, we promulgate them. They're very nice. They look beautiful. They are beautifully written. But oftentimes when people don't follow them, there's not a big penalty. And this is why, in addition to having the policies, educating the campus community is important. But it's difficult to do because we need to educate them about so many things. About cybersecurity threats, about sexual harassment, about nondiscriminatory policies, about responsible behavior on campus regarding drugs and alcohol, about crime. So many things that they have to learn about. It's hard to get at another topic for them to spend their time on, instead of researching the core subject matter that they chose to pursue for their lives. FASKIANOS: Thank you. And we will be sending out a link to this video, the transcript, as well as the resources that you have mentioned. So if you didn't get them, we'll include them in the follow-up email. So I'm going to go to Dorian Brown Crosby who has a raised hand. Q: Yes. Thank you so much. I put one question in the chat but I have another question that I would like to go ahead and ask now. So thank you so much for this presentation. You mentioned algorithm biases with individuals. And I appreciate you pointing that out, especially when we talk about face recognition, also in terms of forced migration, which is my area of research. But I also wanted you to speak to, or could you talk about the challenges that some institutions in higher education would have in terms of support for some of the things that you mentioned in terms of potential curricula, or certificates, or other ways that AI would be woven into the new offerings of institutions of higher education. How would that look specifically for institutions that might be challenged to access those resources, such as Historically Black Colleges and Universities? Thank you. MOLINA: Well, very interesting question, and a really fascinating point of view. Because we all tend to look at things from our own perspective and perhaps not consider the perspective of others. Those who have much more money and resources than us, and those who have fewer resources and less funding available. So this is a very interesting line. What is it that we do in higher education when we have these problems? Well, as I mentioned before, we build committees and subcommittees. Usually we also do campus surveys. I don't know why we love doing campus surveys and asking everybody what they think about this. Those are useful tools to discuss. And oftentimes the thing that we do also, that we've done for many other topics, well, we hire people and we create new offices—either academic or administrative offices. With all of those, you know, they have certain limitations to how useful and functional they can be. And they also continue to require resources. Resources that, in the end, are paid for by students with, you know, federal financing. But this is the truth of the matter. So if you start creating offices of artificial intelligence on our campuses, however important the work may be on their guidance and however much extra work can be assigned to them instead of distributed to every faculty and the staff members out there, the truth of the matter is that these are not perfect solutions. So what is it that we do? Oftentimes, we work with partners. And our partners love to take—(inaudible)—vendors. But the truth of the matter is that sometimes they have much more—they have much more expertise on some of these topics. So for example, if you're thinking about incorporating artificial intelligence to some of the academic materials that you use in class, well, I'm going to take a guess that if you already work with McGraw Hill in economics, or accounting, or some of the other books and websites that they put that you recommend to your students or you make mandatory for your students, that you start discussing with them, hey, listen, are you going to use artificial intelligence? How? Are you going to tell me ahead of time? Because, as a faculty member, you may have a choice to decide: I want to work with this publisher and not this particular publisher because of the way they approach this. And let's be honest, we've seen a number of these vendors with major information security problems. McGraw Hill recently left a repository of data misconfigured out there on the internet, and almost anybody could access that. But many others before them, like Chegg and others, were notorious for their information security breaches. Can we imagine that these people are going to adopt artificial intelligence and not do such a good job of securing the information, the privacy, and the nonbiased approaches that we hold dear for students? I think they require a lot of supervision. But in the end, these publishers have the economies of scale for you to recommend those educational materials instead of developing your own for every course, for every class, and for every institution. So perhaps we're going to have to continue to work together, as we've done in higher education, in consortia, which would be local, or regional. It could be based on institutions of the same interest, or on student population, on trying to do this. And, you know, hopefully we'll get grants, grants from the federal government, that can be used in order to develop some of the materials and guidelines that are going to help us precisely embrace this and embracing not only to operate better as institutions and fulfill our mission, but also to make sure that our students are better prepared to join society and compete globally, which is what we have to do. FASKIANOS: So I'm going to combine questions. Dr. Lance Hunter, who is an associate professor at Augusta University. There's been a lot of debate regarding if plagiarism detection software tools like Turnitin can accurately detect AI-generated text. What is your opinion regarding the accuracy of AI text generation detection plagiarism tools? And then Rama Lohani-Chase, at Union County College, wants recommendations on what plagiarism checker devices you would recommend—or, you know, plagiarism detection for AI would you recommend? MOLINA: Sure. So, number one, I'm not going to endorse any particular company because if I do that I would ask them for money, or the other way around. I'm not sure how it works. I could be seen as biased, particularly here. But there are many there and your institutions are using them. Sometimes they are integrated with your learning management system. And, as I mentioned, sometimes we ask the students to use them themselves and then either produce the plagiarism report for us or simply know themselves this. I'm going to be honest; when I teach ethics and technology, I tell the students about the antiplagiarism tools at the universities. But I also tell them, listen, if you're cheating in an ethics and technology class, I failed miserably. So please don't. Take extra time if you have to take it, but—you know, and if you want, use the antiplagiarism tool yourself. But the question stands and is critical, which is right now those tools are trying to improve the recognition of artificial intelligence written text, but they're not as good as they could be. So like every other technology and, what I'm going to call, antitechnology, used to control the damage of the first technology, is an escalation where we start trying to identify this. And I think they will continue to do this, and they will be successful in doing this. There are people who have written ad hoc tools using ChatGPT to identify things written by ChatGPT. I tried them. They're remarkably good for the handful of papers that I tried myself, but I haven't conducted enough research myself to tell you if they're really effective tools for this. So I would argue that for the timing you must assume that those tools, as we assume all the time, will not catch all of the cases, only some of the most obvious ones. FASKIANOS: So a question from John Dedie, who is an assistant professor at the Community College of Baltimore County: To combat AI issues, shouldn't we rethink assignments? Instead of papers, have students do PowerPoints, ask students to offer their opinions and defend them? And then there was an interesting comment from Mark Habeeb at Georgetown University School of Foreign Service. Knowledge has been cheap for many years now because it is so readily available. With AI, we have a tool that can aggregate the knowledge and create written products. So, you know, what needs to be the focus now is critical thinking and assessing values. We need to teach our students how to assess and use that knowledge rather than how to find the knowledge and aggregate that knowledge. So maybe you could react to those two—the question and comment. MOLINA: So let me start with the Georgetown one, not only because he's a colleague of mine. I also teach at Georgetown, and where I obtained my doctoral degree a number of years ago. I completely agree. I completely agree with the issue that we have to teach new skills. And one of the programs in which I teach at Georgetown is our master's of analysis. Which are basically for people who want to work in the intelligence community. And these people have to find the information and they have to draw inferences, and try to figure out whether it is a nation-state that is threatening the United States, or another, or a corporation, or something like that. And they do all of those critical thinking, and intuition, and all the tools that we have developed in the intelligence community for many, many years. And artificial intelligence, if they suspend their judgement and they only use artificial intelligence, they will miss very important information that is critical for national security. And the same is true for something like our flagship school, the School of Foreign Service at Georgetown, one of the best in the world in that particular field, where you want to train the diplomats, and the heads of state, and the great strategical thinkers on policy and politics in the international arena to precisely think not in the mechanical way that a machine can think, but also to connect those dots. And, sure they should be using those tools in order to, you know, get the most favorable position and the starting position, But they should also use their critical thinking always, and their capabilities of analysis in order to produce good outcomes and good conclusions. Regarding redoing the assignments, absolutely true. But that is hard. It is a lot of work. We're very busy faculty members. We have to grade. We have to be on committees. We have to do research. And now they ask us to redo our entire assessment strategy, with new assignments that we need to grade again and account for artificial intelligence. And I don't think that any provost out there is saying, you know what? You can take two semesters off to work on this and retool all your courses. That doesn't happen in the institutions that I know of. If you get time off because you're entitled to it, you want to devote that time to do research because that is really what you sign up for when you pursued an academic career, in many cases. I can tell you one thing, that here in Europe where oftentimes they look at these problems with fewer resources than we do in the United States, a lot of faculty members at the high school level, at the college level, are moving to oral examinations because it's much harder to cheat with ChatGPT with an oral examination. Because they will ask you interactive, adaptive questions—like the ones we suffered when we were defending our doctoral dissertations. And they will realize, the faculty members, whether or not you know the material and you understand the material. Now, imagine oral examinations for a class of one hundred, two hundred, four hundred. Do you do one for the entire semester, with one topic chosen and run them? Or do you do several throughout the semester? Do you end up using a ChatGPT virtual assistance to conduct your oral examinations? I think these are complex questions. But certainly redoing our assignments and redoing the way we teach and the way we evaluate our students is perhaps a necessary consequence of the advent of artificial intelligence. FASKIANOS: So next question from Damian Odunze, who is an assistant professor at Delta State University in Cleveland, Mississippi: Who should safeguard ethical concerns and misuse of AI by criminals? Should the onus fall on the creators and companies like Apple, Google, and Microsoft to ensure security and not pass it on to the end users of the product? And I think you mentioned at the top in your remarks, Pablo, about how the founder of ChatGPT was urging the Congress to put into place some regulation. What is the onus on ChatGPT to protect against some of this as well? MOLINA: Well, I'm going to recycle more of the material from my doctoral dissertation. In this case it was the Molina cycle of innovation and regulation. It goes like this, basically there are—you know, there are engineers and scientists who create new information technologies. And then there are entrepreneurs and businesspeople and executives to figure out, OK, I know how to package this so that people are going to use it, buy it, subscribe to it, or look at it, so that I can sell the advertisement to others. And, you know, this begins and very, very soon the abuses start. And the abuses are that criminals are using these platforms for reasons that were not envisioned before. Even the executives, as we've seen with Google, and Facebook, and others, decide to invade the privacy of the people because they only have to pay a big fine, but they make much more money than the fines or they expect not to be caught. And what happened in this cycle is that eventually there is so much noise in the media, congressional hearings, that eventually regulators step in and they try to pass new laws to do this, or the regulatory agencies try to investigate using the powers given to them. And then all of these new rules have to be tested in courts of law, which could take years by the time it reaches sometimes all the way to the Supreme Court. Some of them are even knocked down on the way to the Supreme Court when they realize this is not constitutional, it's a conflict of laws, and things like that. Now, by the time we regulate these new technologies, not only many years have gone by, but the technologies have changed. The marketing products and services have changed, the abuses have changed, and the criminals have changed. So this is why we're always living in a loosely regulated space when it comes to information technology. And this is an issue of accountability. We're finding this, for example, with information security. If my phone is my hacked, or my computer, my email, is it the fault of Microsoft, and Apple, and Dell, and everybody else? Why am I the one paying the consequences and not any of these companies? Because it's unregulated. So morally speaking, yes. These companies are accountable. Morally speaking also the users are accountable, because we're using these tools because we're incorporating them professionally. Legally speaking, so far, nobody is accountable except the lawyers who submitted briefs that were not correct in a court of law and were disciplined for that. But other than that, right now, it is a very gray space. So in my mind, it requires everybody. It takes a village to do the morally correct thing. It starts with the companies and the inventors. It involves the regulators, who should do their job and make sure that there's no unnecessary harm created by these tools. But it also involves every company executive, every professional, every student, and professor who decides to use these tools. FASKIANOS: OK. I'm going to take—combine a couple questions from Dorothy Marinucci and Venky Venkatachalam about the effect of AI on jobs. Dorothy talks about—she's from Fordham University—about she read something about Germany's best-selling newspaper Bild reportedly adopting artificial intelligence to replace certain editorial roles in an effort to cut costs. Does this mean that the field of journalism communication will change? And Venky's question is: AI—one of the impacts is in the area of automation, leading to elimination of certain types of jobs. Can you talk about both the elimination of jobs and what new types of jobs you think will be created as AI matures into the business world with more value-added applications? MOLINA: Well, what I like about predicting the future, and I've done this before in conferences and papers, is that, you know, when the future comes ten years from now people will either not remember what I said, or, you know, maybe I was lucky and my prediction was correct. In the specific field of journalism, and we've seen it, the journalism and communications field, decimated because the money that they used to make with advertising—and, you know, certainly a bit part of that were in the form of corporate profits. But many other one in the form of hiring good journalists, and investigative journalism, and these people could be six months writing a story when right now they have six hours to write a story, because there are no resources. And all the advertisement money went instead to Facebook, and Google, and many others because they work very well for advertisements. But now the lifeblood of journalism organizations has been really, you know, undermined. And there's good journalism in other places, in newspapers, but sadly this is a great temptation to replace some of the journalists with more artificial intelligence, particularly the most—on the least important pieces. I would argue that editorial pieces are the most important in newspapers, the ones requiring ideology, and critical thinking, and many others. Whereas there are others that tell you about traffic changes that perhaps do not—or weather patterns, without offending any meteorologists, that maybe require a more mechanical approach. I would argue that a lot of professions are going to be transformed because, well, if ChatGPT can write real estate announcements that work very well, well, you may need fewer people doing this. And yet, I think that what we're going to find is the same thing we found when technology arrived. We all thought that the arrival of computers would mean that everybody would be without a job. Guess what? It meant something different. It meant that in order to do our jobs, we had to learn how to use computers. So I would argue that this is going to be the same case. To be a good doctor, to be a good lawyer, to be a good economist, to be a good knowledge worker you're going to have to learn also how to use whatever artificial intelligence tools are available out there, and use them professionally within the moral and the ontological concerns that apply to your particular profession. Those are the kind of jobs that I think are going to be very important. And, of course, all the technical jobs, as I mentioned. There are tons of people who consider themselves artificial intelligence experts. Only a few at the very top understand these systems. But there are many others in the pyramid that help with preparing these systems, with the support, the maintenance, the marketing, preparing the datasets to go into these particular models, working with regulators and legislators and compliance organizations to make sure that the algorithms and the tools are not running afoul of existing regulations. All of those, I think, are going to be interesting jobs that will be part of the arrival of artificial intelligence. FASKIANOS: Great. We have so many questions left and we just couldn't get to them all. I'm just going to ask you just to maybe reflect on how the use of artificial intelligence in higher education will affect U.S. foreign policy and international relations. I know you touched upon it a little bit in reacting to the comment from our Georgetown University colleague, but any additional thoughts you might want to add before we close? MOLINA: Well, let's be honest, one particular one that applies to education and to everything else, there is a race—a worldwide race for artificial intelligence progress. The big companies are fighting—you know, Google, and Meta, many others, are really putting—Amazon—putting resources into that, trying to be first in this particular race. But it's also a national race. For example, it's very clear that there are executive orders from the United States as well as regulations and declarations from China that basically are indicating these two big nations are trying to be first in dominating the use of artificial intelligence. And let's be honest, in order to do well in artificial intelligence you need not only the scientists who are going to create those models and refine them, but you also need the bodies of data that you need to feed these algorithms in order to have good algorithms. So the barriers to entry for other nations and the barriers to entry by all the technology companies are going to be very, very high. It's not going to be easy for any small company to say: Oh, now I'm a huge player in artificial intelligence. Because even if you may have created an interesting new algorithmic procedure, you don't have the datasets that the huge companies have been able to amass and work on for the longest time. Every time you submit a question to ChatGPT, the ChatGPT experts are using their questions to refine the tool. The same way that when we were using voice recognition with Apple or Android or other companies, that we're using those voices and our accents and our mistakes in order to refine their voice recognition technologies. So this is the power. We'll see that the early bird gets the worm of those who are investing, those who are aggressively going for it, and those who are also judiciously regulating this can really do very well in the international arena when it comes to artificial intelligence. And so will their universities, because they will be able to really train those knowledge workers, they'll be able to get the money generated from artificial intelligence, and they will be able to, you know, feedback one with the other. The advances in the technology will result in more need for students, more students graduating will propel the industry. And there will also be—we'll always have a fight for talent where companies and countries will attract those people who really know about these wonderful things. Now, keep in mind that artificial intelligence was the core of this, but there are so many other emerging issues in information technology. And some of them are critical to higher education. So we're still, you know, lots of hype, but we think that virtual reality will have an amazing impact on the way we teach and we conduct research and we train for certain skills. We think that quantum computing has the ability to revolutionize the way we conduct research, allowing us to do competitions that were not even thinkable today. We'll look at things like robotics. And if you ask me about what is going to take many jobs away, I would say that robotics can take a lot of jobs away. Now, we thought that there would be no factory workers left because of robots, but that hasn't happened. But keep adding robots with artificial intelligence to serve you a cappuccino, or your meal, or take care of your laundry, or many other things, or maybe clean your hotel room, and you realize, oh, there are lots of jobs out there that no longer will be there. Think about artificial intelligence for self-driving vehicles, boats, planes, cargo ships, commercial airplanes. Think about the thousands of taxi drivers and truck drivers who may end up being out of jobs because, listen, the machines drive safer, and they don't get tired, and they can be driving twenty-four by seven, and they don't require health benefits, or retirement. They don't get depressed. They never miss. Think about many of the technologies out there that have an impact on what we do. So, but artificial intelligence is a multiplier to technologies, a contributor to many other fields and many other technologies. And this is why we're so—spending so much time and so much energy thinking about these particular issues. FASKIANOS: Well, thank you, Pablo Molina. We really appreciate it. Again, my apologies that we couldn't get to all of the questions and comments in the chat, but we appreciate all of you for your questions and, of course, your insights were really terrific, Dr. P. So we will, again, be sending out the link to this video and transcript, as well as the resources that you mentioned during this discussion. I hope you all enjoy the Fourth of July. And I encourage you to follow @CFR_Academic on Twitter and visit CFR.org, ForeignAffairs.com, and ThinkGlobalHealth.org for research and analysis on global issues. Again, you send us comments, feedback, suggestions to CFRacademic@CFR.org. And, again, thank you all for joining us. We look forward to your continued participation in CFR Academic programming. Have a great day. MOLINA: Adios. (END)

Se in questo podcast possiamo raccontare di startup italiane di successo, una parte del merito va certamente a Massimiliano Magrini, founder e managing partner di United Ventures. Laureato in scienze politiche, ma da sempre appassionato di tecnologia, Massimiliano si può considerare uno dei fondatori del venture capital italiano.Dal 2000 ha guidato l'avvento in Italia dei motori di ricerca prima come country manager di Altavista e poi di Google.Massimiliano trae ispirazione dalla mentalità imprenditoriale americana, ma allo stesso tempo non accetta che l'Italia sia considerata un paese dove non si può fare innovazione.Così, nel 2009 lascia Google e fonda Annapurna Ventures, uno dei primi VC italiani, mentre nel 2013 crea United Ventures insieme al socio Paolo Gesess. Attraverso United Ventures, Massimiliano ha raccolto oltre 500 milioni, gestisce tre fondi, con un quarto in arrivo, e ha investito in oltre 30 startup, realizzando due exit di grande valore con FACEIT e Datrix.Ma oltre a FACEIT, sono tantissime le storie di successo del nostro podcast che sono state finanziate da United Ventures: da Moneyfarm a D-Orbit fino a Babaco ed Electra Vehicles. In questo episodio, Massimiliano fornisce consigli fondamentali per ogni founder in cerca di investitori, ci spiega gli errori da non fare davanti ad un VC e ci svela anche cos'è la prova dell'aeroporto… Ascoltando Massimiliano percepiamo tutta l'importanza e il fascino del ruolo del venture capitalist. Libro: Fuori dal gregge. Il pensiero divergente che crea innovazione SPONSOR Turnover è l'agenzia nata all'inizio del 2021 che aiuta le aziende a muoversi nel complesso mondo delle vendite online, in particolare su Amazon. Turnover gestisce gli account Amazon dei propri clienti per aiutarli a vendere di più e meglio. Per farlo, lavora su diversi ambiti, come l'ottimizzazione e gestione dei prodotti; lo sviluppo di asset come foto e Video ad hoc per questo canale; la creazione e coordinazione di campagne ADV; l'analisi e lo studio dei dati per implementare le strategie di crescita. Turnover è parte del Service Provider Network di Amazon e Verified Partner di Amazon Advertising, il programma di intermediari certificati Amazon. Maggiori informazioni su⁠⁠⁠ www.digitalturnover.it⁠⁠⁠. SOCIAL MEDIA Se vi piace il podcast, il modo migliore per dircelo o per darci un feedback (e quello che ci aiuta di più a farlo diffondere) è semplicemente lasciare una recensione a 5 stelle o un commento su Spotify o l'app di Apple Podcast. Ci ha aiuta davvero tantissimo, quindi non esitate :) Se volete farci delle domande o seguirci, potete farlo qui: Instagram ⁠⁠⁠⁠⁠⁠⁠@madeit.podcast⁠⁠⁠⁠⁠⁠⁠ LinkedIn ⁠⁠⁠⁠⁠⁠⁠@madeitpodcast⁠⁠

Description: Join us on our podcast episode, where we dive deep into the dynamic world of client acquisition, sales, and marketing with industry entrepreneur, Ed Forteau. From being an early adopter of leveraging technology for marketing to the founder of Email Open Rate Optimization, Ed has been at the forefront of the digital marketing revolution. In this series, we'll explore Ed's journey, embracing change and defying plateaus in sales growth, and uncovering the hidden pitfalls of client acquisition. We'll delve into the evolution of B2B sales and marketing from the era of AltaVista to the age of AI, and how Ed has consistently stayed ahead of the curve. This episode will focus on a different facet of Ed's career and insights: "Boosting B2B Success: The Art and Science of Client Acquisition" "The Journey of a Digital Marketing Pioneer" "Embracing Change: The Evolution of B2B Sales and Marketing" "Defying Plateaus: Overcoming the Stagnation in Sales Growth" "Uncovering the Hidden Pitfalls of Client Acquisition" Ed is ready to answer all the burning questions on the minds of entrepreneurs and marketers today. From emerging trends in client acquisition and marketing strategy, to common pitfalls in the client acquisition process that B2B businesses tend to fall into, and the critical role of email and targeting in client acquisition. We'll tackle challenging queries such as: What messaging mistakes are most businesses making? Is there a right time to press for a sale in the marketing process? What role does AI play in the future of sales and marketing? And for businesses struggling with client acquisition, Ed has a wealth of advice and strategies to share. As we delve into the future of sales and marketing, Ed will also share the next big thing for him and exciting projects on the horizon. Join us on this podcast episode for an enlightening conversation with Ed Forteau, the sales and marketing geek who turned his passion into a thriving business. Get ready to enhance your understanding and skills in client acquisition, sales, and marketing. Disclaimer: Not advice. Educational purposes only. Not an endorsement for or against. Results not vetted. Views of the guests do not represent those of the host or show. Do your due diligence. Click here to join PodMatch (the "AirBNB" of Podcasting): https://www.joinpodmatch.com/drchrisloomdphd We couldn't do it without the support of our listeners. To help support the show: CashApp- https://cash.app/$drchrisloomdphd Venmo- https://account.venmo.com/u/Chris-Loo-4 Buy Me a Coffee- https://www.buymeacoffee.com/chrisJx Thank you to our sponsor, CityVest: https://bit.ly/37AOgkp Click here to schedule a 1-on-1 private coaching call: https://www.drchrisloomdphd.com/book-online Click here to purchase my books on Amazon: https://amzn.to/2PaQn4p Follow our YouTube channel: https://www.youtube.com/chL1357 Thank you to our advertisers on Spotify. Financial Freedom for Physicians, Copyright 2023

Show Resources Here were the resources we covered in the episode: Top Rank Marketing's Site Lee Odden's LinkedIn Lee Odden's Instagram Follow AJ on LinkedIn   NEW LinkedIn Learning course about LinkedIn Ads by AJ Wilcox Youtube Channel Contact us at Podcast@B2Linked.com with ideas for what you'd like AJ to cover. A great no-cost way to support us: Rate/Review! Show Transcript AJ Wilcox Are you using influencer marketing community and thought leadership as part of your LinkedIn Ads strategy? We dive into these topics and more on this week's episode of the LinkedIn Ads Show. Welcome to the LinkedIn Ads Show. Here's your host, AJ Wilcox. AJ Wilcox Hey there LinkedIn Ads fanatics. Today I have a special treat for you. Lee Odden has specialized in B2B marketing for his almost entire career. And I'm grateful to call him a friend. He and I have spoken at many of the same B2B events. Although I have to admit he's oftentimes the keynote, while I'm one of the breakout speakers. But, I've always looked up to him as a mentor and a thought leader and a friend. Today, we have a wide ranging conversation covering everywhere from thought leadership to influencer marketing and building communities in B2B, and I promise as a B2B marketer, you'll get a lot out of this. I wanted to highlight a review here by Alessia Negro. She's a senior sales and marketing executive in the hospitality industry out of Dublin, Ireland. Alessia says, "Absolutely great podcast. I have learned a lot about LinkedIn from this podcast. I think whoever wants to learn LinkedIn ads should follow it." Alessia, thanks so much for the kind words. And I do agree. Although I realized I'm a bit biased. And if you're a regular listener, I want to feature you so make sure to leave a review. And I'll shout you out live here. All right. Without further ado, let's hit it. Alright, we've got Lee Odden, co founder of TopRank Marketing. Lee, thanks so much for joining us. We're super excited to have you here. Lee Odden It's great to be here, AJ. Good to see you. AJ Wilcox Alright, so first off, tell us a little bit more about yourself. Anything that I didn't cover in that rather short and sweet intro. Lee Odden I started in marketing, and basically the late 90s. I was actually got into the website sales game selling websites over the phone to small businesses, believe it or not, people would fax us their brochures and we'd make a website out of that. And then of course, people started asking how are we going to get traffic to these websites that you're making us? And that's where I learned about SEO, you know, about creating doorway pages and, you know, creating different web pages for different search engines AltaVista excite Lycos hotpot, you know, no one has heard of those. But then Google came on the scene and linking, and content became more important blogging, social media, and really, over the last 10 years really leaned into the whole idea of CO creating content with influencers. So ultimately, what we tried to do is help customers be the best answer help our clients, b2b tech companies mostly be the best answer for their customers. And it's worked out really well. And, you know, it's kind of funny, on the path from being a really small agency to serving small customers to now serving large enterprise brands. I took that advice myself, right. So I started blogging, and I started doing the kinds of things to make myself known in the industry as a magnet to the agency. You know what I mean by speaking at events and being active on social and connecting with people who were influential to make myself influential, you know, and it's been a fun ride. And when I'm not doing marketing stuff, I'm usually running running is kind of been my thing during COVID. Basically, when I'm actually training for my first marathon, which is going to happen in four weeks. So I'm pretty excited about that. Oh, that's so AJ Wilcox Exciting! Alright, so what's your preparation for the marathon look like right now? I know, you start to do a lot more mileage as you go. Are you doing like low high mileage right now? Lee Odden Yeah, my highest mileage will be tomorrow. But I did 23 miles last Saturday. And I'll probably do 23.5 or something like that tomorrow. I don't need to do much more. Because the whole race is 26. Yes. So yeah, I've been running more than 13 miles for the last seven weeks or so. I've even run a couple of half marathon races just as a workout, which is kind of a crazy idea. Because a year ago, I was wondering whether I could even run 13 miles, let alone run one quickly. And quickly is subjective quickly to me. Not quickly, compared to a lot of friends of mine that I have out there that are runners that are just smoking it, but it's a great way to get out. It's a disciplined thing. It's a lifestyle thing. Now for me, I feel odd if I haven't run that day. And so it's good for your health. And it's great to have goals, right? Yes, it's great to work towards a goal and having a commitment to a thing that takes a long time is incredibly rewarding when you've been able to reach those milestones, right? And also you connect with a community of other like minded people. And now I'm starting to get all these Instagram posts from running communities with these inside jokes about running and it's like, oh my god, this is so funny, but nobody else cares, nobody else cares. It's fun. AJ Wilcox Well, It's so fulfilling, I think to accomplish something physically, that is hard that you know, other people can't. Before a knee surgery, I used to be a runner. And so I do a half marathon every year and an Olympic trial. And I love those kinds of things. I loved accomplishing something physically, that's awesome. Plus, it gives you insight, because you're already community minded, which we'll talk into more, but it gives you something to compare and think about, as you're developing communities around an interest that people have, you can mentally I think, compare it to like, oh, for me, this is running. But for someone else this is whatever the community is. Lee Odden You know, that's really important insight. There's a lot of value in metaphor analogy, when trying to break down new or complex ideas into something familiar for people. Because in order for those ideas to be adopted, they have to be communicated in a way that people will be willing to receive. And so yeah, that's a great point running as metaphor, or, you know, overcoming challenges. A lot of people, I'm inspired by our fitness people, but what I'm inspired by them about isn't so much about the fitness it's about overcoming challenges and worldview, and, you know, being resilient and those sorts of things. AJ Wilcox Perfect. So jumping back here into B2B, how did you originally get interested in b2b? How did you land? Because my understanding is you're exclusively B2B with your clients, or do you have some VC? Lee Odden Exactly. So my agency top rank marketing did start as SEO and PR firm, I had previously mostly consumer experience in marketing doing SEO for consumer websites. And my partner, though, had been working mostly with B2B tech companies as a director of marketing, VP of marketing. And so over time, well, through her knowledge and experience, I learned a lot more about B2B. And the market was responding in terms of inbound interest in our services from B2B companies. And I have a fun story about the big leap early in our business, we were working remotely. And I was working in my unfinished basement, right. And I had a desk down there. And I got a call and someone asked me if I would come speak at their conference, this company was fortune 15 company. So I had been blogging, I spoke a little bit, people knew about me being able to talk about SEO and public relations. So that was a unique intersection. And they wanted me to come speak at their event. And I spoke one morning to their marketing people. And then they said, Come back tomorrow and talk to all of our PR people. So they were a company that was so big, they brought all their different businesses, all the marketers from their all their different businesses and had a little mini conference. And so the second day, I talked to all the PR people. And my contact said, Hey, one of our senior executives was in the room yesterday. And we wanted to know if we could engage your agency on an ongoing basis. So this is the fortune 15 company was a b2b business unit that wanted to hire us. And we were only four people I'm working out of my unfinished basement. Wow. So of course, I said, yes, absolutely, we can help you out. And we figured it out. And that was a fast track to learning about b2b and learning the language of large, complex organizations, because obviously, that's very different than working with a small or medium sized business. And that laid the groundwork for me and the people that I had to competently be able to serve other b2b companies. Right. So now, Adobe, and you know, we've had experience with companies like SAP and Oracle, I've done work for Microsoft, and a really large telecom that I cannot mention, ever, but you know, biggest one, and so on and so forth. So and LinkedIn, my God, what am I saying, we've been working with LinkedIn for gosh, almost 10 years now, providing content and SEO services. So it's been an interesting ride, and b2b is a great space, because there's so much opportunity to raise the bar, it doesn't have to mean boring, boring, there's a lot of exciting things you can do. And there's so much room for us to be able to do it with because of the longer sales cycles, emphasis on content and education and that sort of thing. AJ Wilcox Well, I'm glad you shared someof them, I get the feeling that a lot of the kinds of clients that you were with are the kind that don't want you saying that you work with them. So it's really good to understand, and from my understanding a lot of the community and many of the thought leaders within LinkedIn that we hear from, I won't say created by you that you're the one behind them, but you're kind of the inspiration there. Do you have any comment or anything to share on that? Lee Odden Well, certainly, it's one thing to make a decision that people within your company should have greater visibility that you want to grow their influence, that you want to facilitate social interaction. But it's another thing to do that with intent with intent in a way that will achieve a particular outcome. So that requires doing some homework developing a strategy and architecting Okay, exactly how are we going to execute this in a way that is best going to accomplish the goal that we're after? So it's not just about tweeting more, or doing a LinkedIn live every once in a while. It's like, okay, what's the topic? And what is the anchor topic? What are the derivative topics? What are the conversations that we can repurpose from that? Who are the content collaborators or influencers on that topic that we can connect with, not only for co creation, but distribution, and so architecting all that stuff is really where the most magic comes from. And then for some companies, we do write content for them. But like I said, a lot of the magic comes in through the strategy and the architecture of the all that, and then of course, the ongoing optimization of performance. And then of course, yeah, and then some content here and there. AJ Wilcox Beautiful. I love it. Alright, so talking about thought leadership, specifically, tell us about why B2B marketers should be investing in being seen as a thought leader. Lee Odden So if you mean is, like B2B marketer, as an executive at a company or B2B marketer, like you and me are B2B marketers. AJ Wilcox Yeah, I think a little bit of everything. I think, executives, I think the frontline workers, everyone in between, like, what's the value? Lee Odden Absolutely. So there's a lot of value in that. We all are familiar with the idea, I think Nielsen came out with this research about how people don't trust brands, and they don't trust advertising. And of course, that's been repeated, by different folks since and it still is a challenge in combination with the overwhelming amount of information available to us. I don't know about you, but my email inbox is more of a monster now than it ever was. I mean, just keeping up is crazy. The social channels and people that I follow, I made a lot of effort to craft who I curate, and listen to, but it's just overwhelming amounts of information. I can only imagine what it might be for other folks who haven't had expertise in curation, right. So being a thought leader being a source of truth, for people who are in need, and let's face it, if you're in marketing, you're in need of up to date information every day, right? Ours is a dynamic industry. And so it's super important and to be competent in our industry, it's really very important that we connect with people that are on the forefront of what's new, what's trending, what's relevant. And so being a thought leader helps you as a person who is capable of original thought, who has something to offer, because that's kind of a prerequisite here, and being able to provide value to others in the industry. So that manifests as community building. It manifests as, you know, customers coming to you saying, hey, AJ, I've been listening to your podcast, and I've heard you talk about LinkedIn ads in this way or that way. It's like, you know, we're actually now in a position to get some help. I'm sure that happens all the time. Natalie, I can tell you from personal perspective, I've had it happen. A million times, it feels like where people say, Haley, I saw you speak last week, I saw you speak 10 years ago. And this is people who want to hire my agency, because I'm when I speak, I tell stories about the work the agency does, as well as just best practices, but also people who want to work for me will say, Yeah, you know, I saw you speak, I've been following your blog. And I'm just wondering if there's an opening, oh, my goodness, this person's amazing. And they're coming to us, you know. So for other folks, if you're an executive at a company, if you want to be listened to, if you want to be relevant, it's not enough for your own brands marketing, to go and put out information you think your customers need to know, buyers are looking for sources of truth. buyers are looking for people, humans that they can relate to, that they can subscribe to, so to speak. And if you have subject matter experts, if you are a subject matter expert, and you have something to offer, then it makes sense for you to go down that thought leadership path and make a connection and create value for those folks out in the industry because you know what they desperately need it. And they're overwhelmed with other information. And so you can actually provide them a service. And guess what, what's going to come back to you is new business. What's going to come back to you is community what's going to come back to you is connection with people who can make things happen. And AJ Wilcox What I hear from you is you actually have to have something to say in order to be a thought leader. So don't strive to be a thought leader for thought leader sake. But strive to be a great business professional, a skilled expert in your field, and then take that to share with others. Lee Odden Yeah, and so there's two things I'd say too, if you don't have that yet, if you're not there yet. I feel like if you're Junior in New York career and you feel like thought leadership is in your future. Two things. One, if you don't have a lot of resources, you could document your journey towards thought leadership. And that actually could help you be a thought leader, as a junior person, you could connect with others who are already thought leaders, they could do things like interviews, you could do things like, get quotes from them, or whatever. And so you can document that journey as you are learning more and more about a particular subject matter. And, you know, hey, I experimented with this. And I found this as an outcome, or I talked to this person. And here's some insight that they shared, I noticed in the news, they're talking about this. And here's what I think about that. documenting that journey can actually help you become a thought leader. The other thing is, if you do have resources, you're at a brand. And you don't know how to do this, you can certainly hire an expert, like an agency, or a PR firm or someone like us who can help you develop a plan for thought leadership, maybe even provide some of the content. And that doesn't happen that often. But it can. AJ Wilcox That's a great point. All right. So we've talked about thought leadership as individuals, what about getting your B2B brand to be seen as a thought leadership brand? Do you have any thoughts? Lee Odden Yeah, absolutely. You know, for companies, it's a huge differentiator in a crowded marketplace to be thought of first, when companies have a problem, and they're thinking of solution providers, right. And thought leadership is something especially in B2B. But you know, in general, it's useful from a marketing PR perspective. But in B2B, especially, these are large considered purchases. So you're not just looking for the best solution, you're looking for a solution that you know, is going to be relevant, and maybe innovative and important for you in the long run. So if a brand invest in thought leadership, and what that means is you're articulating a point of view, and it's validated by third parties. So that means that industry publications and industry influencers that are validating the ideas that you're putting forth, you're putting out original research, you're pointing out points of view, you're creating opportunities for other people who are important voices and trusted voices in the industry to have conversations with your executives with people that represent your brand. And that kind of combination of information, helps people subscribe to your religion, so to speak, they subscribe to your point of view. And they start to rely in trust on you as the source of truth. So a great example is Edelman puts out the trust research. Edelman is a huge, huge agency, obviously, they have a lot of resources. But it's like, you know, you can rely on Edelman's research about how people trust brands or not, year after year, because they continue to put out that research and they have other marketing conversations around that. For our small part. You know, we put out a report on B2B influencer marketing. And it's really been a great way for people to know that there is data, thoughtfulness and expertise behind the fact that we're a source of solution when it comes to working with influencers or content in the B2B marketing space, because we are connecting with third party entities influencers and media to corroborate those ideas, right? So brands is thought leaders super important. If someone's got a problem, don't you want them to think of you first, as a solution? I totally agree. AJ Wilcox I think especially in a crowded marketplace, where you have 15 vendors that you can go to for a CRM and information security service, of course, you're gonna gravitate towards the one that you feel like you have the best relationship with. And I think that comes from the thought leadership that comes from community that comes from being a voice that people want to hear. Absolutely. So I think you've touched a little bit here on parts of the strategy. But if someone wanted to start becoming a thought leader, or having their company be seen as a thought leader, what are the steps, the components that you would tell them? Like, here's the strategy for how you actually start to implement this? Lee Odden That can be a really big answer. So I'll be succinct. If I can, I think the first thing you got to do is, you know, specify what is it that you want to be a thought leader about? There's got to be topic specificity. You can't be a thought leader about all things, right? That's just not resource practical. But you've got to identify that thing that sits at the intersection of how you want to be known and what customers are most interested in. And so be a thought leader about that thing, right? Because that represents what's in demand and relevant to your solutions. And as you make that determination, then all things flow from that. What kind of content will you create? What kind of connections will you make? What kind of cadence will you publish and interact at right? So you get an idea about the resources that are needed in order to put something like that into action, right? There's got to be some consistency and continuity of message from thought leader. So in other words, if I talk about 10 Different things over a period of time, it's like, well, yeah, he talks about a lot of stuff like I guess marketing. But if I'm talking about, you know, B2B content, B2B content marketing, b2b content marketing, I mean, I'm talking about derivative ideas around that concept. But really, it's like, wow, B2B content marketing or B2B influencer marketing, people will come to know you as that very specific thing. So you've got to have some choices made about the topic derivative topics, you've got to think about the publishing platform or platforms, you know, is it the company website, the blog, or social channel? As the center of your hub? And the spokes? Are your distribution channels? Okay, where am I going to amplify this? Am I going to amplify through email? Am I going to pull people in through ads? Am I going to do some media relations and talk about these stories with journalists in the industry? Am I going to connect with industry experts? Am I going to partner with them and collaborate with any of them on initiatives? We have what we like to call best answer strategies, how to be the best answer is really kind of a thought leadership play. And you can do things like this is a practical tactic things. Okay, so let's say you want to be the best answer for a particular thing. It's like, okay, Fast Track way to get on the radar of the most important people in the industry, about that topic is to do what we call an honoring post, right? So it's like a list where the 25 top cybersecurity experts, right, I'm a cybersecurity provider of some kind and finance, right. And so here are the top finance cybersecurity experts. I don't even know if I could find 25 of those, but I'll do my best. And then I reach out to them. And I start to create a relationship with those folks. And I'll simultaneous to that, I might do a small version research, what are the trends, what's happening, and I'll invite those folks to be a part of that research. I'll start a podcast. And as I gain momentum, I might do something like, you know, a list of disparate resources like books, conferences, communities, and so on, and so forth. So I become like this destination around the topic. And that seems like a lot of work. But guess what, there's a lot of competition. And to be the best answer, guess what you've kind of have to be the best source of information, I would just say, as far as a thought leadership strategy, whether it's just you, or whether it's your brand, you don't go it alone, that would be one of the biggest mistakes. And that's why I suggest the idea of connecting with other industry experts, and finding opportunities to collaborate with them, you can start a podcast and become a thought leader on the thing that the podcast is about, that's cool. But if you involve other people, that are also well known about that idea, and you can create collaboration opportunities that create mutual value for you and your collaborators, then everybody wins out true, not just you and the person or people you're collaborating with, but especially the audience that you're trying to attract and engage. AJ Wilcox Oh, so true. I think it's so easy. If you're, let's say scrolling through your LinkedIn feed, and you're saying, I want to be a thought leader, let me see what other people are doing. You see someone is running a live stream, and you're like, oh, maybe I need to run a weekly live stream, and then someone else is running a podcast, and then someone else is recording videos, screenshare and posting them to YouTube, you're like, oh, I have to have a YouTube channel. And when you start taking from all of these ideas, you overextend yourself, because yeah, there isn't a strategy. So what I appreciate so much about what you shared is like this actually is a specific strategy where you can put the blinders on and ignore some of the other methods that other people are doing work on your own. And then you won't overextend yourself, like you can actually do this. Lee Odden Something I learned a long time ago about, you know, being known about a thing is one element, one leg of the stool is document your success, and then duplicate. So be specific, like you're saying, be specific on a particular or very specific channel, grow community on that channel. And you'll get to a point where it's like, wow, now it makes sense for me to extend into some other channels, right. And you can duplicate what you learned in that one channel into others. And that way, you can manage resources appropriately. And you create that continuity of message. And you'll be putting forward the most effective tactics for communication, as opposed to being in a constant state of experimentation. AJ Wilcox Yes. And when you look at what everyone else is doing, a lot of times they have a team that's helping them. And so I think that you can do this all yourself, you know, by the time you've done what you've talked about, which is really building a community around one channel or one thing, by the time it's ready to start expanding into other channels and taking on more things. By that time. You've probably built up more of a team and it's a She realistic, you Lee Odden could do that. Yeah. And then just thinking of myself, I started blogging, my only thing was I just started blogging, I could not write. And I started blogging in that blogging. You know, I always made mention of other people, I was just a way to have a conversation. And then social media came on. And people that read the blog started following me on Twitter, and then LinkedIn. And then I started speaking, and then I wrote a book and you know, just grew organically 100% organically. I did have help with the book. But with the rest, I publish all my own social stuff, I do all my own speaking stuff. And so it does make sense, at least in my case to branch out, you can get help to facilitate and expedite that stuff. But as an individual person, I think topics specificity specific channel is a great thing to get started with and expand from there. AJ Wilcox Oh, amen to that. Here's a quick sponsor break, and then we'll dive back into the interview. The LinkedIn Ads Show is proudly brought to you by B2Linked.com, the LinkedIn Ads experts. AJ Wilcox Managing LinkedIn Ads is a massive time and money investment. You want a return on some of that investment? Consider booking a discovery call with B2Linked, the original LinkedIn ads performance agency, we've worked with some of the largest accounts over the past 12 years, and our unique scientific approach to ADS management, combined with our proprietary tools that allow us to confidently optimize and scale your LinkedIn Ads faster and more efficiently than any other agency in house team, or digital ads hire plus or official LinkedIn partners. Just navigate on over to B2Linked.com/apply. And we'd absolutely love the chance to get to work with you. All right, let's jump back into the interview with Leo. Alright, so let's shift gears now talking about influencer marketing, because I know you've done a lot with influencers. We probably see a lot of influencers in b2c we have for a long time, hate actors and celebrity involvement. But in B2B, I think this is a little fresher. And I think a little bit more new. What can you tell us about how B2B marketers should be thinking about leveraging influencers in B2B? Lee Odden We're kind of in a do more with less age, right? There's a lot of marketers pulling back right now B2B tech especially. And, you know, a lot of folks are looking for what are the most effective things I can be doing? Because the demands on delivering on mid to end to funnel KPIs, you know, forget about brand, right, are really high on people's lists. Well, the B2B Institute at LinkedIn did some research. And they found that at any given time, 95% of buyers are not in market, they're out market, right for any solution and only 5% are in a position like, yeah, we need a solution. And so that's not changed. Well, why influencers? Well, here's the thing, you know, you've got factors that are working against you, as a marketer, in this environment where you got to do more with less, we've got to produce, we've got to get results, we've got to react to this economic environment that we're in. And people are as buyers, you know, they are confronted with this information overload that we talked about before they are struggling to find single sources of truth. Who do they trust, they don't trust advertising. They don't trust brands a lot of the time, but they do trust people, they trust people that they follow. And so the idea of what would happen if you're able to connect with the most trusted voices in your industry? What if you're able to collaborate with them on creating content that was super targeted, super valuable to your buyers, and you're able to build a community of subscribers to a regular cadence of that kind of content, imagine how much more effective you'd be at reaching buyers that are actually going to pay attention, versus, you know, singularly relying on interruptive types of communications, right. And so influencers can play a very important role, not for everybody, but for a lot of companies that want to break through and want to attract and engage with buyers, that are really relying on industry experts that are influenced by people who are the thought leaders in the industry, right? And so there are very effective ways in which marketing programs can be put forward, that are creating content of value, of course, but our collaborations with these industry experts, so it's not just like every influencer is the same. It's not like I'm gonna think of a B2B marketing influencer and handling right? And Haley's, a wonderful, wonderful human being. But just because an Hanley is in a piece of ebook that we make or a video we make doesn't mean that's going to solve all problems. And handle is really broadly known as a very unique individual. She has both broad popularity, and she's actually super competent about her discipline. And so she's a unicorn in that way. So she's actually going to help satisfy bringing people in that don't haven't even heard of you, and actually start to consider you because she has that discipline, competency, about copywriting about content marketing, and that sort of stuff. But not all influencers are like that. So it's not just about working with influencers, it's about okay, how do we pair the right kinds of influencers for different stages of the buying cycle, for example, you know, the most popular influencers, those, you know, the pro influencers, they're doing keynotes. They're publishing books all the time. They're professionally famous in the B2B world, in their respective industry. Those are people you use to attract top of funnel types of outcomes. But then you work with people who are actual practitioners in an area, they can actually speak authoritatively on the discipline. So whether they're super user, whether they're a customer, or whatever, for those middle end to funnel types of content, assets, right. So you kind of line things up. So influencers are important, because everyone is influential, but everyone is influenced on a daily basis in some way. And you can architect programs that not only make your brand more relevant, and more credible, and can reach people that you're not reaching with your ads. But over time, you can build relationships with these most trusted voices in your industry in a way that they are starting to organically advocate for you. And that's priceless, right most valuable form of advertising word of mouth, if you can facilitate that as priceless. Oh, totally true. AJ Wilcox I have noticed in B2C influencers, usually they're paid off in some way. But in B2B, I see a lot of influencers, gotten through collaboration opportunities, and a lot of times money doesn't even have to exchange hands, you're doing something that's mutually beneficial to both of these companies are influencers audiences. Yeah, and you know, that's becoming more of an art to achieve that. Lee Odden So we started doing influencer marketing for B2B brands back in 2012. So back then, yes, it was true that the vast majority of influencer engagements were what we call organic. In the case of B2B influencer marketing, most of the influencers are contributing to content that lives on the brand channels. So if a person who's really well known in their particular subject matter area, could be seen in combination with a major brand, that's really credibility building for them, right. So they're creating influence for each other in that way. And so they'd be happy to do it for free, so to speak, obviously, it has to do with the ask too, if I ask someone for a quote, you know, no problem. But if I asked someone to write me 1000 word article, these days, that's probably going to cost something if I want them to write an article every month, that's gonna cost something. So it used to be maybe 90%, were organic 10% were paid way back in the day. Nowadays, it's like 60/40 60% might be organic activations, and 40% are paid. And that paid number is going up and up and up, as more people who are influential in the B2B space are creating media properties for themselves, right, they've got a really established podcast, they're part of a network of podcasts, you know, they're a blogger, or increasingly video assets of some kind, right? And so they're able to not only just say one nice thing about the brand, but they're actually able to put together a package of social distribution and content assets and this and that we're even, you know, do reports, and so on and so forth. So it's great, though, to have that mutual value, it's important to find something that you have in common with the influencer. And in some cases, they may want to contribute, because it's an easy Ask, and it aligns with what they want to do, and it doesn't cost you anything. And it could be that's the first thing you do. But then you might follow up and say, Wow, that was amazing. We'd really like to do to do this more robust thing, how much would that cost? And they're going to appreciate that. And trust me, when you pay an influencer. It's awesome, of course, for the influencer, but it's awesome for your brand, because now you've hired them. They are accountable to delivering to a specification. Whereas if you engage with people organically, and they say, Yeah, sure, I'll give you that thing by next Thursday. And if they don't, you have no recourse. You can't count on it. Right. They're not signing an agreement. So paying influencers is actually a really good thing. It's up to you to negotiate and to do your due diligence as far as who they are and their ability to deliver and have impact. AJ Wilcox Beautiful. I want to switch gears to talking about community because we've mentioned community several times. Yeah, you've put a lot of value and community over your career here in B2B, what role do you see communities playing in B2B. Lee Odden community is hugely important, I think because so many buyers are going to rely on their peers for recommendations. I mean, think you've probably had it happen I know I have. It's like, we go to remote work, and we need a new phone system that we can work, you know, so I don't know who to go to. So I asked a friend of mine, I go to a group community that I'm part of, and I say, Hey, does anybody know what's a good phone system? And this isn't real. But you know, I'm thinking of a silly example. And this happens every day. Right? And so being present with a community is important for b2b because it helps make you relevant, right? I talked about that expression being the best answer, right? Being a thought leader, being first choice means being where your customers are. And certainly your customers are part of different communities. So you have a couple of choices. You can be present in communities where your customers are, and you can exchange value, you can be a participant and you know, answer questions and interact or whatever your salespeople can or whoever, or you could create communities, right, you could say, Look, you know, we see that there's a common interest here, there's something that we can solve for. And it's not something we can do by ourselves. Why don't you join us at helping solve this problem together? Or why don't you join us on this journey to making our industry a better place, we actually are at the beginning of building a community around elevate b2b, right, elevate b2b marketing. And you know, we want to make b2b marketing a better place. And there are different messaging pillars that go along with that marketing, that is more experiential marketing, that is more inclusive, marketing that is more focused on connection, right community building, that sort of thing. So community is super important, I think, to be relevant, to be relevant. And first choice to customers in spaces where they're actually spending time. And where there can be a value exchange, right? It's one thing to provide useful content or utility to your customers through content marketing. It's another thing for you to create a place where as a brand, where your customers can connect with you, but also they can connect with each other. And there's a lot of momentum that can come from that. So I think community is super important. And when it comes to influencer marketing, same thing, you can engage with influencers on an individual basis, and that's fine. But when you can create a community of influencers that can connect with each other, in the context of your brand is solving Wow, now you have something really powerful right, that you can go to market with. And that can have a much bigger impact than these little one off campaigns, people are kind of dipping their toe in the water with here and there. Perfect. AJ Wilcox Alright, so lead, tell us what are those components to creating a community, especially as we're thinking about it for b2b? Lee Odden So as I mentioned, with, you know, thought leadership, I think that idea of topics specificity? Well, it's around the problems that you're trying to solve for, or the things that you stand for, that would be best served by the community. Right? So what's the glue that's going to hold this community together? What is the common interest that they have that aligns with that intersection of what it is that you stand for as a brand? And what's interesting to your customers, right? You also have to define who's going to be part of this community? How is the community get a function? There are other practical questions to be answered? Like, what platform are you going to use? And who are going to be your champions and your moderators? And what are the goals that you have for the community in terms of messaging penetration, in terms of size, in terms of engagement, and ultimately, you have to be accountable to some sort of ROI, right? And with any marketing initiative, those things all need to be defined, right? So sometimes community can start very intentionally. Yeah, there's communities is starting all the time where people just start in a LinkedIn, LinkedIn group or Facebook group or something like that. And it's just like, hey, and they invite a couple of other people who have a common interest. And it just starts organically and they start inviting, who do you know, that we can invite into this community and so on and so forth, or community could be an extension of an event? I think about marketing profs, and their marketing community, right, or Content Marketing Institute, has a whole community, but they also have an event, right? There's a marketing conference called PubCon that's been around. It's an SEO conference been around forever. And there's absolutely a community, you know, that is tied to that event where people you know, get to actually meet in person. So, you have to make choices about what is the purpose of the community, what are the topics that You're gonna cover what problems you're going to try and solve. And then you gotta identify some champions, some people who are going to help facilitate conversations, there's technology choices to make. And then obviously, you got to set up what kind of goals you're trying to achieve, not just for yourself, but the goals for the community itself. Well, selfishly, this AJ Wilcox is an awesome conversation, because we're actually just getting ready to launch our LinkedIn ads, courses and community all together. And so I'm paying special attention here. So thanks for the free advice. What are some of the phases that you'd actually go through in building and then scaling the community app? What should we keep in mind as we actually go to build this? Lee Odden Well, you know, again, you've got to think about some sort of milestone goals. Maybe the first phase is simply, you know, creating the architecture of the community. And as it relates to the major topics, the subtopics, and getting people involved that represent relevance and interest around those topics, inviting them to be actually be a part and then you've got to decide, okay, what are we going to offer them, right? You're so you're offering courses? Or you're offering opportunities for roundtables or discussions? Are you giving them visibility opportunities, and, you know, set some goals for that first milestone of having a certain level of participation. Maybe your first phase is very private. And no one knows about it, except, you know, those early invitees. And we've seen this demonstrated by the social networks that have all popped up all over the place over the last 20 years, or 15 years. And then maybe once you get to a certain threshold or milestone of participation, then you open it up, you know, more publicly as a phase out. And this is what I've observed being successful. One thing though, that, I think what happens is, there's a lot of excitement about anything new. And it's going to be really important to keep that excitement alive. And so you've got to allocate resources to a community manager, or managers that are not only going to be moderating stuff, but are going to be paying attention to what's the ebb and flow from a topic interest standpoint, from a content format standpoint, and adjusting and optimizing. Because if you do the same stuff, six months or a year into your community, it's probably going to peter out, right? If you're not responsive to where the community is growing and showing interest, you're, you're going to lose them probably. So hopefully, I think that's something to look forward to. That's something to anticipate from a face standpoint, maybe, you know, not every community has to start as an exclusive thing. That seems to work really well, though. And then it evolves into an inclusive thing, as there's something of more substance for people to experience, once you open the doors to all. I love it. AJ Wilcox Haley, just kind of off the wall question here for you specifically about LinkedIn ads. I mean, we've talked about communities, we've talked about thought leadership and influencers. One of the new features that's going to be coming out here in the next I would estimate one to three months that LinkedIn has are these thought leadership ads, where we're going to be able to boost personal posts, rather than rely on boosting or creating posts that come from the company for something like this coming out. What role do you see this playing? Do you have any feedback or thoughts or advice for us marketers? In thinking about the new ad format? Lee Odden I'm super excited about that. Because we know that things can get lost in the stream, but not so much about being able to put money behind a thought leadership posts for an individual. It's just like, you know, the targeting, how can we make sure how can we increase our ability for people that we want to see this thought leadership content? Well, other than through a feature like this, right. So I think that's super, super important. And, you know, we've already talked about the importance of individual thought leadership. And by the way, we did some research in our influencer marketing report about the value of executive influence and executive thought leadership at 65% of the companies that were engaging with building their executives, as thought leaders, said that that effort elevated the influence of the brand itself, right. So something like this being able to augment organic content at the individual level with LinkedIn thought leadership ads. Excellent, excellent opportunity. And, again, it'll really help people be able to be a little more intentional and targeted about what's showing up to who, and I think will really give those advertisers a big advantage over those who are relying on just the organic visibility that happens when you post normally I AJ Wilcox love that. All right, so final switch of gears here. What are you most excited about professionally right now? I'd love to ask the same question about what are you most excited about personally? Lee Odden Well, professionally, you know, we are celebrating our 22nd year in business this year. So that's amazing. That is amazing. Wow. And, you know, we made some strategic hires recently in marketing and sales. And we're launching a fresh brand fresh brand new website and blog will be launching with hundreds and hundreds of articles on content, search and influence, and a lot of really cool features, but a really well architected brand, and messaging and all that stuff. So I'm super excited about it. We haven't launched a new website in 10, fit 12 years, and we haven't really had a professional brand engagement with like a branding agency ever, never ever. So I'm super excited about that. Launching for influencer marketing here, late summer, and lots of other things happening. So I'm super excited about that great team. Yeah, an even bigger things plan that can't even talk about later on this year. So I'm super excited about that. I'd say at the intersection of personal and professional, I get to speak at a conference, the biggest conference for b2b in France, next month in Paris. Wow. And then a week later, I get to speak at the largest b2b conference in the UK, in London. So I'm super excited about that. And then I get to visit a client of ours in Geneva in between. So you know, that's pretty awesome. Get to do a little travel, get to do a little business abroad. And there you go. AJ Wilcox That's a lot to be excited about. All right. So I've caught a couple of the resources that you've kind of mentioned here. You talked about the Elevate b2b marketing community, you've talked about your influencer marketing report, I will put the links to those down below in the show notes for anyone. But as for resources, what would you like this LinkedIn ads audience to do what you want them to come follow you contact you in some way? Join the community, download a report, like, what do you have that we should be paying attention to? Lee Odden Absolutely check out top rank marketing.com. Our blog is there as well. With those actually, it's more than hundreds, it's 1000s of articles. But it's probably in the hundreds of those that are most recent and relevant. Yeah, toprank marketing.com, people obviously can connect with me on the socials, Twitter, LinkedIn, Instagram, to see all the crazy food that I eat and all the running that I'm doing and other thing and travel especially, that's always fun, le e o d, d, e n. Yeah. And that's where we're going to announce our report. And we've got a enterprise brand, influencer marketing report coming out very, very quickly, we have a search intent report that's out already. So lots of fun resources there. And also, I have to say, if you're in the LinkedIn ads audience, if you're not already, you've got to be subscribing to the LinkedIn marketing blog. The LinkedIn ads blog, for sure. Also, in the LinkedIn Collective is another great resource over at LinkedIn. That's a great example of LinkedIn, own thought leadership. And of course, the b2b Institute is another great resource at LinkedIn. And you know, my disclaimer is that yes, LinkedIn is a client. But these are resources I, myself personally, rely on quite a bit. AJ Wilcox Perfect. Well, Lee, thank you so much for sharing your mountain of knowledge here. I'm grateful to get to hear it. Everyone, please go follow Lea, check out the resources that he and his company have come up with Lee, thanks again so much. And we'd love to have you on for around two sometime down the road. Super. Thanks, AJ, I appreciate it. I've got the episode resources for you coming right up. So stick around. Thank you for listening to the LinkedIn Ads Show. Hungry for more? AJ Wilcox, take it away. AJ Wilcox All right, like we talked about with Lee, if you go to his site, top rank marketing.com. And we have links to that down in the show notes. You can get access to everything, all the reports and everything he was talking about. You'll also see his three social media handles there in the show notes, his LinkedIn, his Twitter and his Instagram links. So go follow him stay caught up on what he's doing. He's constantly sharing gold. I'm telling you, if you or anyone you know, is looking to learn more about LinkedIn ads, check out the course that I did on LinkedIn learning all about LinkedIn ads. It's by far the lowest cost and the highest production value course that there is out there. If this is your first time listening, welcome. Thanks for coming. Make sure to hit that subscribe button if you've liked what you've heard. If this is not your first time listening, though, please do go and rate and review us. Usually it's Apple podcasts, but anywhere where you can leave a review. That is by far the best way that you can say thanks for us putting out this content week after week with any questions, suggestions or corrections about what we talked about here, reach out to us at Podcast@B2Linked.com. And with that being said, we'll see you back here next week. Cheering you on in your LinkedIn Ads initiatives.

If you've listened to this podcast in the past, you probably know that I'm an avid fan of tech history. There's nothing like a legendary founding story of a company that completed disrupted an industry. Well, I have an amazing episode today that is going to cover a lot of ground at several ground-breaking, platform shift companies. I'm talking about the stories of legendary companies or products like AltaVista, Napster, Groove Networks… and the early days of tech pillars like TechCrunch, Y Combinator and more. Here are just a few tidbits that you'll learn from our conversation: * Did you know that AltaVista tried to hire the founders of Google while they were still at Stanford? * Did you know that Napster was actually trying to build a legit platform with a business model that would have allowed the music labels to welcome in and embrace the digital era? * Did you know that Y Combinator held their very first demo day in Cambridge, Massachusetts and can you guess what company demoed that day and is now a top 20 web property? Now that I think about it, Don and I go way back. Not only am I thankful for his time to share these great stories, but I am also thankful for all of his support. When I started VentureFizz, I used to syndicate content from investors and influencers in the tech industry and Don was one of our partners. Don was a prolific blogger and it definitely helped up build up our credibility. His blog was called The Next Big Thing and if you want to take another trip down memory lane, check it out, as it is still published. In this episode of our podcast, we also cover many other topics like Don's thought on AI, which he was blogging about years ago and what he's up to these days. This week's episode is sponsored by MarketMuse, a content intelligence platform that sets the standard for content quality. Their AI-powered software enables companies to create predictably better content at scale that increases traffic and engagement, cuts down your content process, and improves your search visibility. Get more out of your content with packages starting at $0/month. Plus you can get 20% off the MarketMuse Standard plan, which includes a ChatGPT integration, by using our code "VFIZZ20" at checkout. Go to marketmuse.com to get started!

Mike is CEO and co-founder of Yieldmo, and formerly was CEO of contextual ad network Quigo which was acquired by AOL in 2007 and became part of Advertising.comMore

Mike is CEO and co-founder of Yieldmo, and formerly was CEO of contextual ad network Quigo which was acquired by AOL in 2007 and became part of Advertising.comMore

In this episode, Galen talks with Aaron Hockel of AltaVista Strategic Partners, a marketing firm that helps contractors grow their businesses. They discuss the significance of comprehensive marketing plans, avoiding common social media blunders, and staying informed about industry trends. Aaron emphasizes the importance of custom marketing plans, partnering with experienced professionals, and staying up-to-date on evolving technologies such as AI-generated content. They also explore the need for a fast-loading and visually appealing website, as well as addressing the issue of disappearing Google reviews.  Who's The Guest? Aaron Hockel is a partner and vice president at AltaVista Strategic Partners. He is working with businesses to help them build and implement marketing plans that are designed to put warm sales opportunities into their pipelines. As a partner his duties to the firm cover two major areas: Service development and implementation: In this area Aaron is responsible for staying ahead of the latest marketing trends and ensuring our clients and prospective customers are doing the same.He develops new service channels and figures out how to implement them operationally before rolling them out to our portfolio of clients. Client acquisition and firm growth: In this area he is responsible for contributing to the overall growth of the firm's revenue stream and client base with a focus on diversifying their client portfolio across industries, geographies, and product lines.   Highlights The importance of having goals and a comprehensive view of the client's needs before executing a marketing plan The misuse of social media by some clients before working with Altavista, including inconsistency in posting and lack of targeting in ads Common errors in marketing and the importance of consistency and strategy The use of multichannel marketing to increase visibility and drive customer loyalty Example of utilizing multifaceted targeting techniques to tap into a specific local area The importance of focusing on improving websites every day and meeting the balance between core web vitals and content delivery Online reviews for home service businesses and how it can impact their growth. Predictions on Google's response to AI-generated content and how it may impact SEO and PPC Episode Resources Connect with Galen M. Hair https://insuranceclaimhq.com hair@hairshunnarah.com www.levelupclaim.com   Connect with Aaron Hockel https://www.altavistasp.com/ https://www.linkedin.com/in/aaronhockel 

 Former UVA safety and two-time Super Bowl Champion Juan Thornhill is headed to the Cleveland Browns out of free agency. Former Virginia safety and two-time Super Bowl Champion Juan Thornhill has a new home in the National Football League. After spending the first four seasons of his professional career with the Kansas City Chiefs, Thornhill signed a three-year, $21 million contract to join the Cleveland Browns, as first reported on Wednesday by the NFL Network's Tom Pelissero. Two-time Super Bowl winner Juan Thornhill agreed to terms with the #Browns on a three-year, $21 million contract with $14 million fully guaranteed at signing over the first two years, per sources. Thornhill started 52 games for the Kansas City Chiefs and is a big addition for the Cleveland Browns. After an impressive career as a defensive back for the UVA football program, the Altavista, Virginia native was selected by the Chiefs with the 63rd overall pick in the second round of the 2019 NFL Draft. Thornhill had an excellent rookie season with three interceptions, but tore his left ACL late in the year and watched from the sidelines as the Chiefs beat the 49ers in Super Bowl LIV. Kansas City reached the Super Bowl the next season and the AFC Championship Game the year after that, but came up short of winning it all.  This season, the Chiefs reached the mountaintop again and Juan Thornhill played a pivotal role, recording five tackles and a pass breakup in Kansas City's 38-35 victory over the Philadelphia Eagles to win Super Bowl LVII, earning Thornhill his second Super Bowl ring. Thornhill was brilliant in the entire postseason run for the Chiefs, grading at 90.5 on Pro Football Focus over his final five games, the top mark in the NFL among safeties during that period. In 65 total appearances and 52 starts over four seasons with the Chiefs, Thornhill recorded eight interceptions, one pick-six, 20 passes defended, one sack, one forced fumble, one fumble recovery, five tackles for loss, and 234 total tackles. In 2022, Thornhill had three interceptions, nine passes defended and 71 tackles. #BrownsBuilt BarBuilt Bar is a protein bar that tastes like a candy bar. Go to builtbar.com and use promo code “LOCKEDON15,” and you'll get 15% off your next order.Ultimate Football GMTo download the game just visit Ultimate-GM.com or look it up on the app stores. Our listeners get a 100% free boost to their franchise when using the promo LOCKEDON (ALL CAPS) in the game store.FanDuelMake Every Moment More. Don't miss the chance to get your No Sweat First Bet up to ONE THOUSAND DOLLARS in Bonus Bets when you go FanDuel.com/LOCKEDON.FANDUEL DISCLAIMER: 21+ in select states. First online real money wager only. Bonus issued as nonwithdrawable free bets that expires in 14 days. Restrictions apply. See terms at sportsbook.fanduel.com. Gambling Problem? Call 1-800-GAMBLER or visit FanDuel.com/RG (CO, IA, MD, MI, NJ, PA, IL, VA, WV), 1-800-NEXT-STEP or text NEXTSTEP to 53342 (AZ), 1-888-789-7777 or visit ccpg.org/chat (CT), 1-800-9-WITH-IT (IN), 1-800-522-4700 (WY, KS) or visit ksgamblinghelp.com (KS), 1-877-770-STOP (LA), 1-877-8-HOPENY or text HOPENY (467369) (NY), TN REDLINE 1-800-889-9789 (TN) Learn more about your ad choices. Visit podcastchoices.com/adchoices

 Former UVA safety and two-time Super Bowl Champion Juan Thornhill is headed to the Cleveland Browns out of free agency. Former Virginia safety and two-time Super Bowl Champion Juan Thornhill has a new home in the National Football League. After spending the first four seasons of his professional career with the Kansas City Chiefs, Thornhill signed a three-year, $21 million contract to join the Cleveland Browns, as first reported on Wednesday by the NFL Network's Tom Pelissero. Two-time Super Bowl winner Juan Thornhill agreed to terms with the #Browns on a three-year, $21 million contract with $14 million fully guaranteed at signing over the first two years, per sources. Thornhill started 52 games for the Kansas City Chiefs and is a big addition for the Cleveland Browns.  After an impressive career as a defensive back for the UVA football program, the Altavista, Virginia native was selected by the Chiefs with the 63rd overall pick in the second round of the 2019 NFL Draft. Thornhill had an excellent rookie season with three interceptions, but tore his left ACL late in the year and watched from the sidelines as the Chiefs beat the 49ers in Super Bowl LIV. Kansas City reached the Super Bowl the next season and the AFC Championship Game the year after that, but came up short of winning it all.  This season, the Chiefs reached the mountaintop again and Juan Thornhill played a pivotal role, recording five tackles and a pass breakup in Kansas City's 38-35 victory over the Philadelphia Eagles to win Super Bowl LVII, earning Thornhill his second Super Bowl ring. Thornhill was brilliant in the entire postseason run for the Chiefs, grading at 90.5 on Pro Football Focus over his final five games, the top mark in the NFL among safeties during that period.  In 65 total appearances and 52 starts over four seasons with the Chiefs, Thornhill recorded eight interceptions, one pick-six, 20 passes defended, one sack, one forced fumble, one fumble recovery, five tackles for loss, and 234 total tackles. In 2022, Thornhill had three interceptions, nine passes defended and 71 tackles.  #Browns Built Bar Built Bar is a protein bar that tastes like a candy bar. Go to builtbar.com and use promo code “LOCKEDON15,” and you'll get 15% off your next order. Ultimate Football GM To download the game just visit Ultimate-GM.com or look it up on the app stores. Our listeners get a 100% free boost to their franchise when using the promo LOCKEDON (ALL CAPS) in the game store. FanDuel Make Every Moment More. Don't miss the chance to get your No Sweat First Bet up to ONE THOUSAND DOLLARS in Bonus Bets when you go FanDuel.com/LOCKEDON. FANDUEL DISCLAIMER: 21+ in select states. First online real money wager only. Bonus issued as nonwithdrawable free bets that expires in 14 days. Restrictions apply. See terms at sportsbook.fanduel.com. Gambling Problem? Call 1-800-GAMBLER or visit FanDuel.com/RG (CO, IA, MD, MI, NJ, PA, IL, VA, WV), 1-800-NEXT-STEP or text NEXTSTEP to 53342 (AZ), 1-888-789-7777 or visit ccpg.org/chat (CT), 1-800-9-WITH-IT (IN), 1-800-522-4700 (WY, KS) or visit ksgamblinghelp.com (KS), 1-877-770-STOP (LA), 1-877-8-HOPENY or text HOPENY (467369) (NY), TN REDLINE 1-800-889-9789 (TN) Learn more about your ad choices. Visit podcastchoices.com/adchoices

In this episode of Pit Stops To Podium, we had a conversation about Product Marketing Fit with Andrew Goldner. He's the Co-Founder and CEO of GrowthX, a venture fund that helps first and then invests. They work alongside B2B founders in rising cities to help them find product-market fit. Andrew has been in the startup scene since 1998, based in NYC, Hong Kong, Singapore, Palo Alto and Nashville. He Kicked off his career when the Internet was first commercializing as a tech lawyer for Alta Vista, Yahoo, DoubleClick, Salesforce and others pioneers. Chapters: 00:00 - Intro 00:58 - Who's GrowthX? 02:57 - Who's Andrew Outside The Work? 06:47 - What Means "Slow Is Smooth, Smooth Is Fast"? 09:14 - Learning Precedes Revenue 12:47 - You're Not Selling Until You Find Product Market Fit 15:25 - What Being In The Green Zone Of PMF Means 16:57 - The 5 Steps To Confirm Product Market Fit 22:56 - How To Find Product Market Fit 28:58 - Engage With Andrew //ENGAGE WITH ANDREW Andrew's LinkedIn GrowthX LinkedIn Page //LINKS GrowthX - 5 Steps To Product Market Fit GrowthX - Do These Things To Find Product Market Fit GrowthX - The Revenue Accelerator // SUBSCRIBE! Subscribe to RevPartners YouTube Channel New "pit stops" every week. Join our growing community! // STAY AWESOME & DO IT BIG!! Website: revpartners.io Listen on Spotify and Apple Podcasts

Pode ser difícil de acreditar, mas houve um tempo em que o melhor jeito de encontrar coisas na internet não era o Google. E a comunicação instantânea não era feita por WhatsApp. Antes do aparecimento e da consolidação dessas soluções como líderes de mercado, várias outras brigavam por espaço. Hoje, formam um longo rastro de concorrentes deixados para trás.No episódio de hoje, relembramos algumas dessas empresas e produtos que dominavam antes dos atuais dominantes. Se fazia tempo que você não pensava em nomes como AltaVista, MSN (ou Windows Live Messenger?) e Nokia, este episódio é pra você. Dá o play e vem com a gente!ParticipantesThiago MobilonPaulo HigaEmerson AlecrimJosué de OliveiraCréditosProdução: Josué de OliveiraEdição e Sonorização: Ariel LiborioArte da capa: Vitor Pádua 

Today is a little different from our usual episodes because today we are going to talk about the other side of farming which is livestock operation. We have with us Abe Karels from Alta Vista, Iowa who grew up farming but found the opportunity to raise cattle along the way. This is an interesting episode because we are going to tackle animal health with proper nutrition so that it could add up to our farm's profitability. Come on and let's dig in!

TikTok has reshaped the Internet in under a three-year span, but if its parent company, ByteDance, has its way, the platform's dominance is just getting started. This week I brought Stan founder Denisha Kuhlor back onto the show to discuss TikTok's ambitious plans for total media domination.In the past few months, TikTok has announced plans for several new features — each aimed at competing with current media giants such as Google, Spotify, and Ticketmaster. Features include extending video-length capacity to 10 minutes, the TikTok Music streaming service, better internal search capabilities, and a ticketing platform, among many others.Recent history in Western culture is not kind to companies trying to be an all-in-one platform. Google and Facebook stumbles come to mind. To predict how TikTok might fare, Denisha and I hit the new features point-by-point, weighing TikTok's advantages and disadvantages at breaking into each. Here's our main talking points: [0:50] TikTok's masterplan[7:02] Prediction: 10-minute-long TikTok videos[11:50] Prediction: TikTok music streaming service[15:43] Prediction: Enhanced TikTok search[22:00] Prediction: SoundOn music distribution[25:42] Prediction: In-app ticketing [29:46] Are consumers creator or platform loyal?[33:18] TikTok's impact on creator economy [37:22] TikTok's geopolitical issuesListen: Apple Podcasts | Spotify | SoundCloud | Stitcher | Overcast | Amazon | Google Podcasts | Pocket Casts | RSSHost: Dan Runcie, @RuncieDan, trapital.coGuests: Denisha Kuhlor, @denishakuhlor  Sponsors: MoonPay is the leader in web3 infrastructure. They have partnered with Timbaland, Snoop Dogg, and many more. To learn more, visit moonpay.com/trapital Enjoy this podcast? Rate and review the podcast here! ratethispodcast.com/trapital Trapital is home for the business of hip-hop. Gain the latest insights from hip-hop's biggest players by reading Trapital's free weekly memo. TRANSCRIPTION[00:00:00] Denisha Kuhlor: It has become this trend where we have more affinity to the platform and the platform's ability to curate the content than some of these content creators themselves. And in a world where I think these content creators are so driven to following the algorithm and getting promoted by the algorithm, what they don't realize is kind of the uniformity in content that is created.  [00:00:30] Dan Runcie: Hey, welcome to The Trapital podcast. I'm your host and the founder of Trapital, Dan Runcie. This podcast is your place to gain insights from executives in music, media, entertainment, and more, who are taking hip-hop culture to the next level. [00:00:50] Dan Runcie: All right. We're joined again today by Denisha Kuhlor, who is the founder and CEO of Stan. And today we're going to talk all about TikTok. And TikTok has been a topic I know you and I have talked about offline, we've both covered it and have our opinions on it, but I want to talk today about talk's grand plan to try to take over everything. Just to name a few headlines from the past couple of months, TikTok is planning to extend into 10-minute long videos. It is launching its own music distribution service called SoundOn. It filed a trademark for its own streaming service called TikTok Music. They are enhancing their search function to identify key terms. They're also adding in a text-to-image option as well so that people can start to do that. And it sounds like a lot, the company has grown quite a bit, so it's understandable. But do we think that TikTok is going to be able to do all of these things? What's your thought? [00:01:47] Denisha Kuhlor: Yeah. So TikTok's been really interesting to watch these last few months and honestly, really from inception, my initial hunch is that it's hard to do a lot of things well. And as TikTok grows and somewhat through replication and also a bit through innovation, I do think they're going to struggle to really get to scale for all the new features that they want to launch. [00:02:11] Dan Runcie: Yeah. I think the tough thing with this, and it's something that has been ingrained with big tech companies for a while is when the big social network grows and they have this huge following. TikTok now is the fastest to reach 1 billion monthly active users. We can see the trajectory of it potentially getting to be as big as Facebook is now. And Facebook, of course, is another company that has tried and is still trying to do every possible thing under the sun. But I think the part that's important is there are a few examples when these companies have succeeded. Instagram copying Snapchat is of course the primary example that people often look back to, but more often than not, most of these attempts don't actually work that well. And one of the reasons they don't work as well is because they don't necessarily solve a true need that the core users are looking for to be solved from that app. And I think that's one of the important things about Instagram Story specifically because Instagram Stories copy Snapchat worked because Instagram already had a hub of influencers as its core users. And these core users wanted to be able to both post pictures, but they also didn't want to feel the pressure of needing to have this polished picture that was on their feed all of the time. So their thought was, okay, if they could copy this feed that they see Snapchat's doing, they already had the core users there and having something that's more ephemeral. It can go away in 24 hours was perfect. It worked as good as you could probably expect it to. And honestly, it worked better than Snapchat because Instagram already had the home base of those core users whereas Snapchat, at the time, they had a bit of penetration from Gen Z, a bit of DJ Khaled here and there, but it just wasn't to that same level. And I think when you look at a lot of the other attempts that Facebook has tried to copy from others and even Instagram as well with seeing with Reels, that's the piece that I go back to. If these successes and these copycat attempts haven't worked, it's usually because there's some type of disconnect between what the core users on that app are looking for and whether or not that new feature helps them do that.[00:04:23] Denisha Kuhlor: Totally. And I think it creates a culture even internally for these organizations of duplication versus innovation, right? So now you see these organizations going and seeking the desire to duplicate and get to market as quickly as possible, whereas before they had no choice but to be innovative. And to do that, I think they really had to listen to their users and the folks on the app. So it also just even changes, in a way, the culture of what the app is about because now folks are so used to see or expecting to see things that have already been done before, rather than excitement towards really where the platform could take things. [00:05:01] Dan Runcie: Yeah, it's interesting because, on one hand, I do understand the aspect of copying what's already successful. You see it's there and you know that you have those users on your platform already. So why not make an attempt, why not use your resources, especially because of how much money these companies print on ads, then, yeah, you could take the chance with Google having its Google X or Facebook opening up its own VC firm or in many ways, treating all these new initiatives as its own VC firm. But to your point, you do lose the innovation and that's exactly why these apps became relevant in the first place. They offered something newer. They did it in a truly unique way. And when you think about why TikTok has blown up, the genius of it is that For You page. They made it so frictionless to be able to stay entertained, to scroll. You don't have to think about who to follow. You don't need to do any of those things. And that is its biggest strength, but I think it also makes it very challenging to have any type of new feature that is harder or requires more user- input or more activity than the mindless scrolling that has worked in its favor up to this point.[00:06:11] Denisha Kuhlor: Yeah, I completely agree. I feel like the For You page really was the magic and to, in some ways, see them stray away from that, or even improving that in other ways does feel a little unfortunate. Some of the features that you listed, while exciting, I think are just not necessary in the sense that so many other folks are out there doing it. But it will be interesting to see how it fits within maybe the grand scheme or the grand vision for TikTok users and creators. I mean, when it comes down to maybe offering a more seamless experience, then it gets a little bit more interesting. But how big of a problem is that right now for creators, especially when you think about, like, some of the plays towards distribution and features around that? The problems don't seem prevalent enough to justify the investment. But maybe there's a grand vision within all of that, in which it makes more sense. [00:07:02] Dan Runcie: So let's break those down. Let's go through each of 'em. Let's start first with TikTok extending into 10-minute videos. I do feel like this is probably the least friction out of each of them, but what's your thought on this expansion and clearly a move to compete more directly with YouTube? [00:07:18] Denisha Kuhlor: Yeah, I think this one is interesting because it really, in some ways, is probably the least painful in the sense that if content is compelling enough, you could argue that an individual is just going to keep watching, if the initial, you know, piece of content is compelling enough. What actually is, like, somewhat fascinating to me is that in some ways you could argue that TikTok took away or has hindered people's ability to focus for that long of time. So going to like the corollary of now having 10-minute videos, I do think will be interesting 'cause it's like a different habit, right? Even just focusing on something for 10 minutes versus like 6 seconds is a very different habit. So to see how or to see what type of users actually adapt to that, I think it will be interesting. I do think though there'll probably be some niche communities that emerge as a result of that feature who do want to take deep dives much to content, right? There's folks that like read casually about the music industry and then folks that like really, I think, deep dive, much as a testament to Trapital's content. And so I do think, like, some interesting, like, subsections of the feature will rise. However, I think the bulk of the users aren't even, like, able to watch a video for that long.[00:08:33] Dan Runcie: I think that if it is extending the videos into that length, I agree with you. This is the least friction one. I think it does have the highest likelihood of success. But if I'm thinking about music videos specifically or something that ends up being at least that length, it changes the format to look like what the videos for the most successful YouTubers often look like and the science that goes behind that. I'm picturing what NBA YoungBoy does in the beginning of his videos or even someone like MrBeast. There's some hook there that gives you some tease and that keeps you engaged, just to make sure that you end up watching the whole thing to see what it is. So I feel like if artists start creating music videos or start creating videos in general to be more like 10 minutes. And I think the format of how those videos look will be a lot different and everything will be how do we keep people engaged so that, okay, if we keep them for the first 15 seconds, how do you get them for the next 15 seconds and after that. Like, you can't have these long buildups that I think you can have for certain types of videos on YouTube, just give the audience. But I think it will change things in that format. No different than when MTV blew up, there was a type of video vibe that people tried to go after. I think that if this is the route that TikTok is really trying to go. I think we may see videos lead more into that where, yeah, everyone does start creating videos where you may look like you're trying to be a YouTuber. You're trying to be a TikTok dancer, whatever it is. But I feel like that's where it could head if it's as successful as it could be. [00:10:04] Denisha Kuhlor: Yeah, that's actually really interesting in the sense that a lot of folks, like, point to their desire to use TikTok because it does feel like less polished, in a way more authentic. I was listening to a podcast where a TikToker says she makes more content on TikTok because she has to like, yeah, just be less prepared in a way or prepare herself to get on TikTok in the way you would for on Instagram. So I think if that does happen, it'll probably have an effect they don't want, which is a longer timeline to people creating and posting content. And like, just a harder barrier to entry because now folks will feel like, well, I don't have all the things needed to start a TikTok or to really start posting on TikTok, which is really against, I think what the platform did in its early days. [00:10:50] Dan Runcie: Right. Yeah, you're right. That whole instant, making it easy as possible is part of it. It almost brings me back to Vine to an extent. Maybe that's a better comparison for what this looks like 'cause of course someone like a MrBeast or NBA YoungBoy, they have big teams at this point. But some would be able to take a Vine and having this whole narrative story in that 6-second, 7-second clip, maybe it's getting a bit back to that. But even that takes time and there's clearly a difference between that. And, you know, while Vine was popular, it didn't blow up the way that TikTok has blown up. So I feel like you're right. It may change the app in a way that users aren't ready for. But we'll see. I obviously know that this is kind of what happens when you're trying to do everything. You're going to risk having some type of frustration that comes from the core users. [00:11:37] Denisha Kuhlor: Exactly. Much to your point, I do think there will be a really active, like, community or communities around that in which like 10 minutes of content works really well and TikTok is just like an easy medium to do that [00:11:50] Dan Runcie: For sure. All right. The second one here, this one I will be interested to dive into. TikTok Music and TikTok filing this trademark. It clearly wants to launch its own music streaming service. We had heard rumors about ByteDance, TikTok's parent company, wanting to do this, but how do you see this one playing out?[00:12:09] Denisha Kuhlor: Yeah, this one, I just kind of felt like, okay, like another music streaming service You know, one, I don't think people realize or really think through just like how complex streaming services are as a business. I mean, thankfully, you know, a lot of platforms have kind of, pioneered some of the heavy lifting that came with making deals with labels and really like getting the content onto the platform. But that's all still to be said that it's a very unique and complex business model that's driven on another party, right? And how another party feels about giving you access to your content? What does seem somewhat interesting about it is, in the same way that TikTok democratizes content creation and the barrier to entry to post, you could probably argue that it in a way, democratizes that for music, and so more artists are able to get more volume or traction as a result. And so I do think if they focus on maybe content from newer creators or newer musicians who don't necessarily have some of that on the platform, that could be interesting, like in terms of a new streaming platform being able to get access to these independent artists at rates that could be favorable. I think that's interesting, but I don't know if that works at scale. And frankly, like, songs from independent artists, I don't think, is enough to keep a consumer satiated. And there's an even harder barrier to entry to have two streaming platforms at once. [00:13:35] Dan Runcie: Yeah, this is the one I'm probably the most skeptical of its success and for very similar reasons. Say what you want about Spotify. I know people have a number of issues about how that platform is operated and how it distributes its money. But the fact that it's helped the music industry, A, get to this point, says something and just the type of deals it's been able to negotiate to make it all work the way that it has that's enabled all the other types of revenue-generating opportunities that have came from it. And then additionally, it's hard to get to that point. Again, you may not agree with all the decisions that they make, but it is very hard to get to that point. And while I understand, from a strategic perspective, why TikTok may initially want to do this. Of course, if you have and you own the top of funnel that exists in the industry today, why wouldn't you want to at least think about what it could be like to keep that attention on your platform? If your platform is where discovery is happening for both the new fans, for artists to get initial exposure, and for that, you know, the record labels are already seeing, I understand why you would want to think about keeping more of that in-house. But it is a lot tougher than they think for the reasons you mentioned. And also going back to the usability of the app streaming services are a type of consumer experience that requires much more active engagement. People don't just scroll through Spotify and Apple Music. You're going there actively to find something that you're looking for. I mean, I don't even know that many people that are actively relying on that discover weekly playlists to find anything. You're still searching for the things that you want. Even if you're looking for a playlist, it's probably that's much more catered toward the mood or genre that you have. So I think anything that requires that level of agency or action from the consumer side will always be a bit of a challenge for TikTok there. So yeah, I'm skeptical on the success of that one. Unless it tries to go more of the YouTube route of things, which ties back into the 10-minute video things that we talked about before. There's some potential there, but even there, I think there's still some question marks.[00:15:41] Denisha Kuhlor: Yeah. I'm aligned with you there. [00:15:43] Dan Runcie: Yeah. The next one is TikTok Search. A lot of us had seen the viral tweet that someone had. I don't Google I TikTok. And a lot of that spoke to how a lot of folks in Gen Z are looking for information and I get it, I've even done it myself, my wife and I were recently searching to buy a new mattress. And you know what? I didn't want to go through a Google Search and just read some sponsored content about a mattress. I wanted to see a video of someone unboxing this thing to see what it looks like. [00:16:10] Denisha Kuhlor: Exactly. [00:16:10] Dan Runcie: And TikTok was the quickest place to do that, even quicker than YouTube. YouTube's going to show me a mix of explainer videos and then also concept from the company. I just wanted to see some random person be like, oh, hey, here's what I think about this bed. And here's what I think about that bed. It was quick, it was easy. So I do think that that works, but I think there's a few caution flags with it. A, I still think that even though TikTok was able to offer that, there's still deeper search functionality that went into how Google got to be as good as it is, even going back thinking about 20 years ago about like why Google succeeded where Lycos and AltaVista and all those other go.com, .com era search engines didn't work. So I don't know if TikTok has all of that baked in to really go beyond just, you know, people like me looking for random purchases that they want to look through here or there or just want to look up a certain topic. And I also think the other bigger, more important pieces, the misinformation, and just being able to correct for that because that's already been an issue on TikTok. And I think that could potentially continue if there isn't some way to relegate what's happening in search. So, high likelihood success, but still some trepidation. [00:17:23] Denisha Kuhlor: Yeah, this one is one I'm actually a little bit more excited about. I do think it's really interesting, like in the sense of search, because it is something that we naturally do more. I first started to search on social media using Instagram. And I think they've even done a greater job of like adding more functionality to do that search, whether it's by location and showing you things surrounding that location or even venues or event spaces. So I think that it's a growing feature and a great feature. Like you said, the reviews, whether people sought them to be that way or just inherently more, right, they're showing you video. Most times they're talking through it and you can just consume and walk away with a more educated viewpoint for a time that's favorable, right? A 30-second or 1-minute video can really give you a lot of feedback about whatever you're searching. I think, honestly, this is where a TikTok should spend some time doubling down. I think we want to see more of that functionality from them playing around with maybe the highest use cases, whether that's locations or certain venues, or even like festivals. As I think about it, like, I see so often on TikTok, like you can see a certain event from multiple vantage points and understanding what it's like at a festival from someone in VIP versus general admission versus backstage, even, right? Like, Rolling Loud, you see, like, every single vantage point, even sometimes down to the artist manager with them. So I think, like, them doubling down on a few use cases that really highlight the immersiveness of search is something that excites me. And I think just naturally follows up on what the users are already doing on the platform. [00:19:02] Dan Runcie: That ties into the another announcement I saw from them about enhancing its ability to search for things locally, or being able to find things from that level because to your point if you are seeing multiple vantage points at Rolling Loud or at Coachella, you may want to meet up with someone that is there, or you might be able to see their vantage point. You might have times I've been to a music festival and it's like, where are you? I'm at the main stage. But what part of the main stage, you know, they got this quarter over here, they got that quarter, but if someone could just do a quick, like, boom. And maybe that could be even easier than them trying to send me a FaceTime video or something like that, where there's no service, but if they could at least post it up on TikTok or wherever, then it could be like, okay, I see your angle. I'll be there. I'll come see you in a minute.[00:19:44] Denisha Kuhlor: Yeah, I think that's great. And to that point, too, it kind of like puts on other users in terms of like, okay, wow, I didn't enjoy my experience at like Rolling Loud 'cause I was GA. So maybe VIP is worth it to me or I should consider doing that. And so I actually think more artists should be embracing and recognizing that search feature. The only thing is too is because so many people are using it, you in real-time, right, seem to get updates. So like, Kizz Daniel who's come under fire in Tanzania for not showing up to his performance. I already, like in my mind was like, well, Kizz Daniel was four hours late to his DC show. And how did I know that? Or DC or New York? I'm sorry, but how did I know that? Like, because I saw it on TikTok and so that's like twice in a row. So how likely am I as a fan to justify the cost of a ticket in the event that he is going to be near me? So I think it's like a good maybe transparency or accountability measure. But with that search, we maybe do sometimes need to recognize like, what do they say that like, people are most likely to post or leave reviews when they either have a really great experience or a really bad experience. And so sometimes you might not just get what the true experience is in the case of like a service-based search. [00:21:02] Dan Runcie: That's true. That's a really good point. And that goes back to the quality of the results and how they can find a way to measure that piece 'cause I think that's the piece that ties back into why Google has been good at what it does over its competitors. So that TikToks can actually survive and not, you know, become someone else that may do video search even better. [00:21:23] Denisha Kuhlor: Yeah. And maybe, you know, to the extent they would consider this, like, there's an opportunity for collaboration, right? Like Google's done a great job of, you know, when you ask certain questions, they have a definitive answer, but they also pull like multiple sources. And so what if, like, on a Google search, you search a restaurant and you're also seeing like TikToks in the area? I think the aggregation of that repository of information could be really great. And also a way for them to continue to like maintain their dominance in search. [00:21:55] Dan Runcie: Let's take a quick break to hear a word from this week's sponsor. [00:22:00] Dan Runcie: Definitely. So the next one, this one's interesting, music distribution. TikTok recently launched SoundOn, which is a service that in many ways is set to compete with a lot of the music distributors. And I think similarly, it could be seen as its opportunity to capture its top of funnel attention as well. You already have the artists, why not make it easier for artists to use your platform, to distribute the music that they have? What are your thoughts on this one? [00:22:31] Denisha Kuhlor: Yeah, and this isn't personal, but I'm just not really excited by music distribution. Nowadays, like in a lot of ways we're listening to a song on a streaming platform is a commodity, right? Like listening to Drake on Apple Music sounds the same as, like, Spotify. I feel that way with music distribution, like, as a consumer, the consumers have no idea, right? They just know they go to their streaming platform and the song is there. The reverse engineering of how it got there and the back end is really not of much interest to them. On the artist side or for them to do this, I think it requires a really deeper investment in artist education. And so I'm curious to see, you know, especially as they double down on creator programs and things of that nature how willing they are to invest both on a content and community side, but also a capital side, in artist education to incentivize users to distribute through that platform. When you think about switching costs in terms of getting set up on a new platform and just probably some of the like new things you have to adjust to by doing it. I feel very underwhelmed hearing about this, and I'm really curious to see how it goes. [00:23:39] Dan Runcie: I think you called it right in the beginning. Music distribution is a tough business. It is purely a commodity at this point. And I think you can win a few ways. You win by trying to achieve massive scale with it, which Distrokid clearly has just given everything else. But if you don't have that scale, you try to find something unique to position yourself with. I think we've seen that a bit with United Masters, but even that's a bit of a unique business model because, A, they've done a bunch of partnerships with different platforms and companies in sports and entertainment to try to use that as a way both to attract artists and give them an opportunity. But it's also attached to an ad agency with translation, which essentially can, you know, offset any costs or anything like that if there are already losses that come through with the business. So that part of it is unique there. But then even with some of the other services, I think a lot of them have adapted their business models over time because that customer service piece is so timely. It's so expensive. And yeah, when you have an artist that maybe generated less than $20,000 a year, and they're calling your service every other week because they're trying to feed their supporters and making sure that every one of their fans can get their music. How do you justify that cost when you want to be able to support the fans? But the economics of it don't make sense if you're also trying to compete with Distrokid where it costs very little money to be able to use their service on a regular basis. And the same could be said about TuneCore and the others. So it's a tough business to enter. [00:25:16] Denisha Kuhlor: And I think, you know, artists and management teams don't really have any particular affiliation to, you know, to like any platform. Maybe there are things that they like about certain platforms or that keep them there. But when you talk to artists and management teams, it's kind of just this is what we use, it works, it gets the job done. And it's not an area of the business as long as things are working, they're going to particularly spend a lot of time overly evaluating. [00:25:42] Dan Runcie: Right. The next piece of this and the next thing that TikTok's been trying to do is ticketing. And while this is less of a big initiative the way they have it right now, it's an integration with a Ticketmaster who, of course, owns most of the medium size to large venues from a ticketing operation, given their relationship with Live Nation. I have to imagine that TikTok's ultimate stream would actually be trying to do what we just saw from Spotify to try to launch its own ticketing service. But even that has plenty of issues and challenges there, but what's your take on at least this first step of the Ticketmaster integration for TikTok and where it could go from here?[00:26:23] Denisha Kuhlor: Yeah. On the ticketing side, it's interesting. And just like having a background in venture and tech and startups, like, I've seen a lot of folks try to solve ticketing in many areas, right? The curation that comes with ticketing, ticketing from all over the world and in different currencies, and just a better user experience overall. I will say while I don't think I'm, like, particularly mad at TikTok's, like, foray into ticketing, I do think it's a missed opportunity to probably focus on like events that have organically grown through the platform. And something that's like so interesting is I think you've seen more and more promoters or even event producers, like really like leverage TikTok to create those events and grow their followings in their community. And that's not what TikTok's ticketing platform is really targeted with as evidenced by, you know, a partnership with Ticketmaster. And so while I feel like it's somewhere in the direction, I do think it could be a bit more directionally accurate by focusing on, kind of the, yeah, the smaller organic events that just naturally have grown through TikTok and like TikTok partnering with those events to help users produce more content and like, it can truly be mutually beneficial in a way that I think some of those event organizers would welcome. And so while I could understand why they went for the validity and reputation that comes with a bigger brand, such as Ticketmaster, I think they could have got more bang for their buck with a smaller, more targeted partnership with folks that already found interesting use cases to grow ticketing for the respective events.[00:27:54] Dan Runcie: Yeah. I feel like there's a bit of a balance there because I hear you and I do think that it ideally would be, yeah, great for them to double down on the creative uses of the, especially some of the more emerging artists that are using this platform to bring folks together, right? Almost similar to what you may see people trying to do, whether it's seeing things virtually in Twitch or bringing those types of audiences in real life to particular things. I think that's really cool and unique. I do feel that for TikTok though, specifically with what we're seeing them do on the music side, in the back of my mind I always wonder, okay, if it weren't for Doja Cat and Megan Thee Stallion and some of these other major artists that are using the platform, what percentage of their impact is making up the overall pie of TikTok Music? Just thinking about that, they had that pie chart from a few years back about the genres and how hip-hop was over-indexed and how Megan Thee Stallion was the most popular artist. So if you're trying to cater to the biggest artist on your platform, you know, Megan Thee Stallion and Doja Cat are definitely at the Ticketmaster level of what they're doing. So if they are going to have an event, could you have something that keeps them in, right, because I think that the more organic things that we've seen likely are more of a direct competition to what we see from Eventbrite, let's say, which I think is much more in that sweet spot of everything from like a birthday party up until you get to like, you know, a small club concert or event, right? But then obviously Ticketmaster is everything else. So yeah, it's like, my heart wants to be like, oh yeah, stay with the types of cool events you've had. But also just thinking about how YouTube leaned into its biggest customers and like, if you're TikTok then yeah, it's the Megs, it's the Dojas, and ones like that. [00:29:38] Denisha Kuhlor: That is interesting and I think a good corollary. Maybe it does, like, trickle down on more of like a hybrid approach. Yeah, that's interesting. [00:29:46] Dan Runcie: I do think this taps back into something that you had mentioned before just about the platform itself and as this platform continues to grow, where does the loyalty sit for the consumer, right, whether it's with the artist or with the platform. [00:30:03] Denisha Kuhlor: Yeah. I think this is such a big thing, right, and that comes with building a fan base or even just like your notoriety on TikTok. You see the changes that were made to Instagram and kind of everybody from the Kardashians, right, calling them out. And I think it has become this trend where we have more affinity to the platform and the platform's ability to curate the content than some of these content creators themselves. And in a world where I think these content creators are so driven to following the algorithm and getting promoted by the algorithm, what they don't realize is kind of the uniformity in content that is created. Even when it comes down to, like, some of the events or experiences or those types of videos, sometimes, like if you've seen one, you've seen them all. And I think that's why there's other creators, whether it's, like, more comedian-focused or other topics that really excel because it forces them to kind of have to do something different, even if they do have to be relatable. And so unfortunately I think that, you know, artists who are employing TikTok and kind of using this, especially as they build their name and their brand, need to think a lot about like, okay, I have X amount of followers on TikTok, but the barrier to entry to get someone to follow you on TikTok looks very different compared to other platforms. And then taking that a step further, it's like, what does that mean? Because while people might like you, how willing are they to migrate to another platform? They ultimately have that ultimate affinity and loyalty, in my opinion, to TikTok. [00:31:38] Dan Runcie: I couldn't agree more, and it makes me think about how I use these apps today. For instance, we're recording this now August 26th, and this is a few hours after DJ Khaled released his album and Jay-Z had his four-minute-long verse on GOD DID. And I've seen everyone from ESPN's account to all of the hip-hop blogs and everyone else posting about this. And of course, you get it. And it's all these memes you see about people posting, okay, what Hov did on this track and they're getting photos of LeBron's best games or LeBron's game six against the Boston Celtics and things like that. But I bring that up because speaks to the uniformity of how all of these platforms or all these accounts on these platforms end up doing the same types of things because they know it works and they know what is effective. And it comes to the point now, when I'm scrolling through Instagram, I don't really know who the account is that is there that's posting something that I see unless it's something that's super specific to that person, right? Like if a friend is posting something from, you know, one of their kids starting in kindergarten, then it's like, okay, I know that that only comes from you. But if someone's posting something that's happening in media or something, that's happening in the news, you have to, like, look at that account at the top to really know who it is. And I think that's something they probably got from TikTok more so that, unless you're really looking to see where that account's coming from, it's a bit hidden now, right? Like that's part of, I know some of the frustration people had had, whether it's with Google searches or how social media was sharing links and they made all the links look the same, whether it's something from The Wall Street Journal or your friend's blog, right? And it kind of goes back to that point. [00:33:18] Denisha Kuhlor: Yeah. And it probably has like real implications for the creator economy now that we're talking about it, like, I think, you know, living in New York, I see, I like casually probably see a few TikTok creators a month and maybe even sometimes I follow them. But you know, what's interesting? Like, rather than noting, other than noting to myself, like, oh, I follow them on TikTok, kind of the like je nais se quois or like the magic of like, oh my God, like I'm seeing this person in real life feels like it's disappeared a bit to me in a way that used to exist with YouTube or some of these other platforms where it felt like a weird, like breaking of the screen. But now that everybody's behind the screen and as a result, even some of the content they're showing is so accessible. I do think it probably, like, leads to this dynamic of where we're just like, okay, let's just see interesting things. The people creating said interesting things are no more interesting in some ways than like you or they just did a great job at doing this. And I see that with, there's a lot of debate and, like, discourse around some of the lifestyle blogs or, you know, like people showing their lifestyle, like waking up in the morning, like obviously, you had to set the camera up before to do that. But a lot of folks in the comments argue like this is just a type of content. Like, it's a type of cinematography that people like to view and people like to see. And so as a result, these people are continuing to make these videos, but if that's just a type of content that people like to see, TikTok is simply going to provide that content all the time, regardless of really any affiliation to one creator, which makes it a lot tougher on these creators, I think, to build these networks and conversely artists.[00:34:55] Dan Runcie: Right, and this brings me back to the whole issues that people have with Web 2.0 to begin with and why they wanted to be able to solve some of this with Web 3.0. It's because the platforms commoditize your content, and then in return, they're the ones that hold the power. [00:35:10] Denisha Kuhlor: You know, I think though folks have to be honest. In some ways, it's what the user likes or what the, yeah, the users do like this because if not, you know, we're long past exclusives being standard in the industry, but if not the exclusives would've worked. Like having, you know, Chance the Rapper's album on Apple Music for two weeks, that would work. But the industry shied away from that because ultimately consumers cared more about choice and the ability to choose and experience and be exposed to all types of artists. And so I do think it's a dangerous game because it doesn't recognize like that's why malls exist, right? Like, you go and you want to go to multiple stores. And so I do think sometimes while I understand and recognize and very much like honor the need to, you know, differentiate and be able to have your core audience and provide to those things, I think we'd be remiss if we also didn't realize, like, natural human behavior comes from choice and like the brevity of choice. And so that's sometimes the interesting thing between Web 2.0 and even Web 3.0 and with crypto, for me, because ultimately, like, the barrier to entry is so high, right, to get someone, a true fan to download an artist NFT because that insinuates their true fans. And I think a lot of artists have actually had to face the music in some ways with realizing their perceived fan base isn't as big as they thought and the mechanism to realize that has been some of these drops.[00:36:40] Dan Runcie: Well said, well said. The engagement piece and what you need to have a true fan is harder than people think, so, yeah, I couldn't agree more. Well, we're getting to the tail end, but before we close things out, we have to talk about the elephant in the room for TikTok, and that is its geopolitical standing and all of the things that it wants to do while, whether or not they will be successful, a lot of it depends on the company's viability in the US and whether or not it's current status, especially given the fact that the Chinese government does have this data and there are unknown questions about what that means, what it can do with this data, how do you see this piece it?[00:37:22] Denisha Kuhlor: Yeah, I think it's tricky in some ways, because, you know, as consumers, we're now kind of privy to the implications of tech and big data. And even just being on our phones, being on our phones in general, what I will say is and a lot of the like research indicates that true, like avid TikTok users are just, like, hooked in a way where they don't or they might say to you they don't care. Now how much is that true, I guess we'll find out. But I do think it's concerning because maybe to some extent, we don't even fully realize everything and all the factors that are at play here, right? Like, you're just giving that summary, I'm like, whoa. But as a user coming on every day, you're not thinking about that. And so often with big data and some of these platforms, in a way, you don't realize just the implications it had until it was too late, right? Until we're now talking about the ramifications of a platform existing in that way. And so I think it's going to be really interesting to see how, seriously consumers want to take it and beyond consumers, like the US, the US in general. I mean, I would be remiss if I didn't say that some of the data is concerning, right, learning about some of the data TikTok has access to is concerning. But ultimately like as more and more people post and the ecosystem grows larger and there's now 10-minute videos and your favorite artists are on there and they have a streaming platform and all these things in this ecosystem, it starts to get hard to really stray away. And so I think that's going to be a challenge because it feels like it almost has to be a collective push for folks to disintermediate from the platform. But I'm really curious your thoughts on this, too.[00:39:00] Dan Runcie: Yeah. So the first attempt of this was in fall 2020. So it was around two years ago at this point, when Trump had tried to shut down TikTok. That didn't work for a number of reasons. There were a number of things going on in the world that the attention just wasn't there. And I don't think that the argument was made in a concise and effective way that could have necessarily gotten the job done. And TikTok had other challenges at the time, Kevin Mayer had his short term and then he had left shortly after. So there were a number of issues there. This though, I think that even though you're starting to hear some senators say certain things about it, I think things will be pretty mum, I would guess, until the 2022 midterm elections coming up just 'cause think from a strategic perspective, they want to keep momentum on things that they can confirm can get votes. So while I think I've probably heard more of the concern, if I'm being honest, coming from democratic senators, their biggest concern right now is okay, how can we continue to try to celebrate Joe Biden's victory so that they can not lose seat coming up with this election, I feel like. And because of that, like, we kind of see how this whole thing plays out. I do think though that we could be facing a potential situation where it's almost like the Facebook thing where people know that this is an issue, but it's not going to happen proactively. It happens reactively. It's going to be like, when shit hits the fan and then people are going to be like, oh shit, now we need to do something. [00:40:27] Denisha Kuhlor: Exactly. Exactly. Out of curiosity, how do you think TikTok, and I'm sure it'll vary, right, but how do you think TikTok is going to be used with the upcoming election cycle? [00:40:37] Dan Runcie: Oh, good question. I don't see it impacting 2022 as much, but I could see it playing a factor more so in 2024 because I just think that even though there's plenty at stake coming up with this election, the presidential elections always get more in place. I do think that, especially as this group of voters does tend to grow and as more and more older people do get on TikTok, a lot of the same types of activities and nefarious behavior that we saw on Facebook here is going to make its way onto. TikTok. The bigger challenge is though, I think, it's even tougher to navigate all those things. I mean, we even saw that there was misinformation back in 2020 when you had a lot of the Black Lives Matter uprisings and people, they were censoring certain things related to those hashtags. So I do think that those things are going to cause big problems. I think the difference though, and this is part of it is that when these issues happen for Facebook, it's one thing if you have mark Zuckerberg coming to Congress and it can kind of be this thing where he could be media training, he can kind of have these like, you know, haha moments where it's like, Senator, we sell ads, that's how we make money. That doesn't exactly work with the Chinese government in the same way 'cause I don't think that that type of congressional hearing would necessarily work in the same way. So it would have to be some type of, you know, harder crackdown that happens with it. So, yeah, it's tough. I feel like we're not going to see anything actually happen until shit does hit the fan. And unfortunately, that could be the 2024 presidential election in the US, but maybe it could be something sooner. [00:42:19] Denisha Kuhlor: Yeah, no, we'll definitely have to see how it plays out. I also think we could potentially see, like, new candidates that come to the result from easily being able to build followings on a platform like TikTok. So I'm curious to see what, like, TikTok- native candidates emerge as well. [00:42:36] Dan Runcie: Right, like kind of like how Obama was the Facebook champion in 2000. [00:42:41] Denisha Kuhlor: Exactly. [00:42:43] Dan Runcie: Yeah. It's funny, right? Because I feel like, you know, back then it was like, oh, look at all the great things that Facebook could do with 2008. And just, I think given some of the political leanings at the time, but then 2016 in many ways was a very opposite case with it. So I do feel like we're a bit more jaded and cynical of the powers of social media than we were then. But there is always a candidate that rises up with these things, that does these things, right? Like, I don't know, thinking back to the days of candidates that are just entering a different thing or new platform, whether it's Bill Clinton going on the Arsenio Hall Show playing his saxophone or something like that. Like, who's going to be that on TikTok? I don't know. I don't follow any politicians on TikTok. I'm sure they have accounts, but I'm sure they'll probably be doing that. And who knows? They'll probably have a debate on Hot Ones for all I know. [00:43:35] Denisha Kuhlor: It's definitely going to be interesting. [00:43:37] Dan Runcie: Yeah, for sure. All right. Well, this was great. We covered a bunch in this, so we'll definitely have to revisit this topic at some point. And we'll see how TikTok succeeds over this for the next few months. I think we both have our internal scorecards ready, but we'll definitely have to touch base on this again at some point. [00:43:54] Denisha Kuhlor: Agree. Thanks so much for having me.[00:43:56] Dan Runcie: For sure. Thank you.[00:43:58] Dan Runcie: If you enjoyed this podcast, go ahead and share it with a friend. Copy the link, text it to a friend, post it in your group chat, post it in your Slack groups, wherever you and your people talk, spread the word. That's how Trapital continues to grow and continues to reach the right people. And while you're at it, if you use Apple podcast, go ahead, rate the podcast. Give it a high rating and leave a review. Tell people why you liked the podcast. That helps more people discover the show. Thank you in advance. Talk to you next week.Advertising Inquiries: https://redcircle.com/brands

Our guest this week is Larry Siegel. He is the Gary P. Brinson director of research at the CFA Institute Research Foundation. Prior to that, he was director of research for the Ford Foundation's investment division for 15 years. Siegel began his career at Ibbotson Associates in 1979. He specializes in asset management and investment consulting and has served on various boards as both an advisor and a director. He has also served on the editorial board of the Financial Analysts Journal and currently serves on the editorial board of The Journal of Portfolio Management and TheJournal of Investing. Siegel is a prolific writer and has authored several critically acclaimed books in recent years, including Unknown Knowns: On Economics, Investing, Progress, and Folly as well as Fewer, Richer, Greener: Prospects for Humanity in an Age of Abundance. He earned his Bachelor of Arts from the University of Chicago and his MBA in finance at the University of Chicago Booth School of Business.BackgroundBioUnknown Knowns: On Economics, Investing, Progress, and Folly, by Laurence SiegelFewer, Richer, Greener: Prospects for Humanity in an Age of Abundance, by Laurence SiegelResearch"Lifetime Financial Advice: Human Capital, Asset Allocation, and Insurance," by Roger Ibbotson, Moshe Arye Milevsky, and Kevin Zhu, ResearchGate, January 2007.Popularity: A Bridge Between Classical and Behavioral Finance, by Roger Ibbotson, Thomas Idzorek, Paul Kaplan, and James Xiong, Jan. 15, 2019."Bursting the Bubble—Rationality in a Seemingly Irrational Market," by David F. DeRosa, SSRN, April 29, 2021."Equity Risk Premium Forum: Don't Bet Against a Bubble?," by Paul McCaffrey, CFA Institute, April 8, 2022.The Myth of Artificial Intelligence: Why Computers Can't Think the Way We Do, by Erik Larson, April 6, 2021."Value Investing: Robots Versus People," by Laurence Siegel, larrysiegel.org, June 30, 2017.Endowments and Investing Lessons"Don't Give Up the Ship: The Future of the Endowment Model," by Laurence Siegel, larrysiegel.org, April 7, 2021."Where's Tobin? Protecting Intergenerational Equity for Endowments: A New Benchmarking Approach," by M. Barton Waring and Laurence Siegel, larrysiegel.org, April 21, 2022."Debunking Nine and a Half Myths of Investing," by Laurence Siegel, larrysiegel.org, March 12, 2020.Inflation"Protecting Portfolios Against Inflation," by Eugene Podkaminer, Wylie Tollette, and Laurence Siegel, The Journal of Investing, April 2022."The Novelty of the Coronavirus: What It Means for Markets," by Laurence Siegel, larrysiegel.com, April 1, 2020."Will Demographic Trends Drive Higher Inflation and Interest Rates?" by Laurence Siegel, larrysiegel.com, Feb. 10, 2021.Other"Cliff Asness: Value Stocks Still Look Like a Bargain," The Long View podcast, Morningstar.com, May 31, 2022."Tom Idzorek: Exploring the Role of Human and Financial Capital in Retirement Planning," The Long View podcast, Morningstar.com, June 7, 2022.TranscriptJeff Ptak: Hi, and welcome to The Long View. I'm Jeff Ptak, chief ratings officer for Morningstar Research Services.Christine Benz: And I'm Christine Benz, director of personal finance and retirement planning for Morningstar.Ptak: Our guest this week is Larry Siegel. Larry is the Gary P. Brinson director of research at the CFA Institute Research Foundation. Prior to that, he was director of research at the Ford Foundation's investment division for 15 years. Larry began his career at Ibbotson Associates in 1979. He specializes in asset management and investment consulting and has served on various boards as both an advisor and a director. He has also served on the editorial board of the Financial Analysts Journal and currently serves on the editorial board of The Journal of Portfolio Management and The Journal of Investing. Larry is a prolific writer and has authored several critically acclaimed books in recent years, including Unknown Knowns: On Economics, Investing, Progress, and Folly as well as Fewer, Richer, Greener: Prospects for Humanity in an Age of Abundance. Larry earned his Bachelor of Arts from the University of Chicago and his MBA in finance at the University of Chicago, Booth School of Business.Larry, welcome to The Long View.Laurence Siegel: Thank you.Ptak: Thank you so much for joining us. We're really excited to chat with you. I wanted to start with your early career. You worked for Roger Ibbotson early in your career. In fact, you were Ibbotson's first employee if I'm not mistaken. Talk about Roger's influence on you and more broadly, the impact he has had on our understanding of markets and investing.Siegel: Roger was not only my first boss, he was my first finance professor at the University of Chicago. So, I got fed the Ibbotson—and to give credit where it's due, to Sinquefield—view of the markets early. I was 21 years old. And I would describe that view as that asset classes are what's important; that security, individual securities, are best viewed as components of asset classes, although when you get involved in the business, you realize that you have to understand the market at the security level, too; and that long-term performance is very strongly in favor of equities. So, at the time, pension funds, who were the main customers for Ibbotson Associates' work, had relatively little in equities, and one of our missions was to improve the returns of those funds and thus for the sponsors and the employees by holding more equities. This was in the early ‘80s. I was hired in 1979. So, you can see that was a good strategy.Benz: So, sticking with your background in your early career, you think young professionals should have a grounding in the humanities and liberal arts. Why is that?Siegel: Well, not every single one needs to, but the ones who are going to rise to the top in the business need a grounding in the common cultural heritage of the human race, and that's given by humanities and social sciences that the liberal arts broadly construed. Investors invest in businesses or governments, but mostly businesses, and businesses exist to serve the needs and wants of people, an ever-changing group of people around the world. So, without a deep understanding of human affairs—in other words, of the why of business—young investment professionals are likely to fall into some intellectual traps: short-termism, geographically narrow thinking, where you only think about your own country, and a bunch of other well-documented behavioral biases—you shouldn't do that.Ptak: Maybe a dumb question to follow up on that: Why doesn't the market do a better job of creating incentives to ensure that younger professionals—let's talk about those who are heading into finance and in investing in particular—that they have a liberal arts background and they're able to better avoid some of those traps? Why haven't those incentives really taken shape and why is it still so typical to see this procession of MBAs and people with the traditional finance background dominating finance and investing?Siegel: Well, if you're as old as me, I'm 68, you have observed that it used to. The market, when I was getting out of school, was in a very different position. There weren't many MBAs. It was an unpopular decision to go to business school. And most of the people who were accepted in business school had an Ivy Plus background where a liberal arts education is required in order to graduate. By Ivy Plus I mean the University of Chicago, Stanford, Northwestern, places like that, plus the Ivy League. So, this staffed the investment business with a fairly broadly educated group of people. What happened in the next 40 years is that business got too big. And the MBA programs mushroomed from a little specialty of a dozen or two dozen schools to something that everybody felt they had to get in order to get a job. So, it just became more of a trade school degree rather than an academic degree. And I'm sorry if I'm offending anybody here, but that's the way I see it. And the investment business became more of a trade. So, the market became less efficient, I think, because it just got so big that it had to pull in a lot of different people, including people who had specialized early because they wanted to be in finance because they were seeing people in finance made a lot of money.Benz: Speaking of specialization, do you think that the only way to truly specialize is to have had a generalist humanistic education first? In other words, are the most successful specialists people who trained as generalists first and is there any evidence for this?Siegel: I think there is among CEOs and maybe CIOs, chief investment officers. The greatest businesspeople in the world have generally had a pretty broad background and a lot of them started, the legend is in the mail room, but they may have started in engineering, accounting. They may have started in sales. Whatever they did, they found their way to the investment business through a kind of evolution over time. An organization needs foxes and hedgehogs. Isaiah Berlin, drawing on an ancient Greek story, said that there are two kinds of people of foxes who know a little about everything and hedgehogs who no one big thing. Einstein, for example, was a hedgehog. He really only cared about physics, and he was very productive. We would have a very different world without him. I am suggesting that you're better off looking for foxes, but you also want to have a few Einsteins in there, and an organization that consists entirely of foxes would be very unfocused and would be more like a college dorm than a business.Ptak: Wanted to shift and talk about something that seems like it's been an awfully short supply lately, which is optimism. You wrote a book called Fewer, Richer, Greener, evincing optimism about the global economy and humanity in general. Have you always been an optimistic person? Or has it gone back and forth or been situation dependent?Siegel: I've always been an optimistic person in terms of my intrinsic biases. I do know enough economic history and regular history to know that living conditions have improved so much in the last 250 years, and actually in the last 50, that you'd be kind of crazy to deny that things have improved. This is a bad year and a bad decade. And it's very easy to become pessimistic when you read the news or check the stock market or look at the world situation with wars and so forth. But underneath the surface of all this chaos and negativity, technology is continuing to advance at an amazing rate of speed. And what we really rely on for economic growth is improvements in technology, where I use the word technology to mean it very broadly. Technology is not just the gadgets or computing power. It's biology. It's social technology—my ability to gather together a bunch of people in a Zoom meeting from all over the world and have a board meeting. And as this technology has grown in the broad sense, we have made our lives much easier; work has gotten easier. We do less of it. The 80-hour work week has now become a specialty of doctors, lawyers, and CEOs. Coal miners—my father-in-law was a coal miner and he worked 80 hours a week in a coal mine when they let him. He would have preferred to work 40, but he needed the money. So, we have an economy in which we produce an awful lot without doing all that much, frankly. We have probably the easiest lives of any population that's ever existed.Benz: Optimism seems like one of those secret weapons in investing, in finance in that if you're optimistic, you're more likely to stick with it, stick with your plan, and markets have tended to reward people who have stuck with it over the longer term. But it's hard to be optimistic about the long term given how unknowable things are. So, is the equity-risk premium compensation for subjecting ourselves to that unknowability?Siegel: Yes. There are two kinds of risks. One is fluctuations in asset prices. We all know what that is. The market just went down 20% or 25%, and we're feeling it. And we might forget this, but it went down 34% in a month in the spring of 2020, which is a profound dislocation in the markets. And a few months later, we forgot it. The other kind of risk is actually more profound, and it's the possibility that our general expectations for assets are wrong. And if you look back, equities have returned about real 7%, 7% plus inflation. Going forward, it's pretty unlikely that they're going to do that over the next 20 or 30 years just because of the high prices. Even if economic growth were as rapid in the future as it was in the past, you want to pay less rather than more for the stocks. So, right now, they're selling at a premium to their historical average. That conventional asset-allocation input of equities generate 6.7% or 7% real is almost certainly too optimistic, and we've got to do what Jack Bogle said, which is budget for it. We can't all earn alpha and earn a higher return, because the net alpha in the market is 0, so we would all be trying to take it away from somebody else. We have to budget for lower returns.When you look at the bond market, it's even worse. Bonds seem to be priced to yield about real 0%  to real 1%. That's much lower than the historical average, about half the historical average.Ptak: You got that right. It looks like real yields across the yield curve 49 to 99 basis points as of yesterday, which would be July 11, so a pretty paltry real yield. I did want to, if I may, stick with the general topic of optimism and its nexus with investing, talk about that in the context of value investing. I sometimes wonder if value investing pays off because it's so repulsive over long stretches that it's almost impossible to be optimistic. That does, though, raise questions about the implications for its practical usability. For instance, if investors are likely to give up on it because they do find it so repulsive when it underperforms growth as it had done until relatively recently, they might miss out on some of that payoff, which can come in bunches. Or do you think that's off base? Do you think that value investing really is usable, you just have to stick with it long enough?Siegel: I think that value investing is usable. But you shouldn't concentrate your whole portfolio in it. What we've seen is that the pendulum has swung between value and growth in very long cycles and large cycles where value does much better or much worse for the entire time that data are available. Fama and French did this back to 1927 and you get these five- to 15-year swings, which is so long that people give up on either value or growth at exactly the wrong time. So, in 2007, value had outperformed massively, and it was a great time to buy growth stocks because we were just about to enter not a tech bubble but a period of tech innovation that produced huge returns for a decade and a half. Anybody who went against the grain, anybody who went against the tide and overweighted growth stocks did much better than the market from 2007 until a year or two ago. Now people are saying, only growth works, so value is disgusting. And the more disgusted you are, the more likely it is to work. I would overweight value right now, but not all the time.Benz: I wanted to ask about intuition. It's something that tends to be greatly valued in everyday life, but it can lead us astray when it comes to investing. For example, in March 2020, which you referenced earlier, few of us expected the great snap back in the markets because intuitively we knew the pandemic would be bad for humanity. Do you think intuition was a better model for investing before markets became so efficient or has it never really worked?Siegel: Well, informed intuition, if you've spent a lifetime in, let's say, engineering and you know something about the way that computers are put together or the internet is put together or something, you might have had the intuition that this was going to be a profound change in the way everybody did everything and you bought those stocks. But the problem is that most people who bought the stocks in the first tech wave, in the 1990s, bought them without knowing anything about the individual companies. They were right about the technology; they were wrong about the companies. So, you would now have a portfolio of AltaVista and Netscape and AOL and a bunch of other companies that had promised but they were just outcompeted by somebody else. So, I would rather hang my hat on analysis than intuition unless you just happen to be one of those people with special inside knowledge but that is obtained legally. But most people who think they have inside knowledge don't. So, I would try to avoid relying on intuition too much.Ptak: Wanted to shift and talk about your role at the CFA Institute. You have a lot of experience assessing research proposals in that role. What are the best pieces of research have in common based on your experience?Siegel: Well, they draw heavily on theory to make practical recommendations that can be implemented in the short to medium term. And going back to Roger Ibbotson, we published a piece in 2007 on lifetime financial advice that came from Roger with several colleagues. We are about to publish, but have not yet received the manuscript, the second installment of that from Paul Kaplan, Tom Idzorek, and a third author whose name I forget, and that will come out later this year or early next year. So, even though they're 15 years apart, the Ibbotson people have an integrated theory of investing insurance, annuities—all these different tools in order to provide people with a lifetime income that's secure and yet has the room for adding value through either asset allocation or security selection alpha. So, that's the kind of research I like most. We sometimes have also done pieces that step outside of the box of the Financial Analysts Journal or the Journal of Portfolio Management -type of research and look at a broader set of issues—for example, geopolitics, demography. There was a beautiful piece by David DeRosa on bubbles. He's against them. I don't know how he can be for or against bubbles. Either bubbles are or bubbles are not. But he takes the position that what we think are bubbles are mostly rational responses to circumstances and then when the circumstances change, the bubble bursts. But it wasn't a bubble; it was rational at the time. I don't know that I buy that 100%, but it sure was interesting reading his logic because he expresses it so well. So, these are the kinds of research I enjoy the most.I've also done some of my own research here. I am compiling for the CFA Institute Research Foundation a book on the equity risk premium, which was a symposium of 11 fairly famous people—Marty Leibowitz, Rob Arnott, Cliff Asness and so forth—which I led. I'm not one of the famous people, but I know them all socially, so I was able to get them to come. And I edited it with a co-editor, Paul McCaffrey, who is producing a book on that as we speak. It could come out in the next month.Ptak: I did want to ask you about what's become the new rage in investing research and portfolio management, which is combining quantitative and human-driven decisions. If you had to draw up a CFA curricula for a bot, how would it differ for the current human-based curricula? And on the flip side, how do you think the current human curricula ought to be reshaped to account for the rise of things like machine learning? Is that something you've given any consideration?Siegel: A little bit. I'm writing a book review right now for Advisor Perspectives, which is an industry newsletter, a very good one. And the review is of a book by Erik Larson that's called The Myth of Artificial Intelligence. I'm giving it a good review, so you can see where I'm going to come out. I believe that machine learning is a real thing. Machines can be programmed to learn, and that's a valuable tool in investment management. But when you step beyond that to the idea of artificial general intelligence, I think it's an illusion caused by very fast computers, very big data and very clever programmers who want to create that illusion. So, we have had 300 million years of evolution—not as human beings obviously but as animals—to develop a set of connections in our brains that actually are intelligent. Yet intelligence in the sense that we are talking about now didn't really emerge until the last 200,000 years. So, it is rare. It is fragile. And we don't know what it is. It's like Justice Potter Stewart said about pornography: We don't know what it is, but we know it when we see it. And to imagine that we're, as human beings, of one level of intelligence, whatever we are, can build a machine in a few decades of those 200,000 years that's more intelligent than we are with all that evolutionary heritage is frankly ridiculous. These machines are going to do what we tell them to do. But if we tell them using instructions that are crafted well enough, it will give the illusion of being intelligent. When I don't know how something works, like everybody else, I tend to think it's magic. I'm driving and there are two or three cars lined up at a red light, it immediately turns green and makes the other traffic stop because it's a smart red light, and all it's doing is counting the number of cars that are waiting for it to turn and changes the cycle, changes the frequency, according to the traffic instead of operating on a fixed time cycle. But it looks like a pretty smart red light when you haven't encountered it before and you say “Gee, that's really amazing.” Well, I think that AI as we're experiencing it now is kind of the same as that. It's just a technology that other people understand because they developed it, but we don't because we don't have the knowledge and so we feel like it's magic or intelligence, whichever you want to call it.Benz: There's been a lot written about the glut of skilled, highly trained professionals in the investing field. Can you talk about the level of competition you see now versus what you saw earlier in your career?Siegel: The industry has become way too big. Every stockbroker has become a financial advisor. Ninety-six percent of them ought to tell people buy, hold, diversify, and rebalance and minimize taxes, and then they have to fill in that outline through implementation. In other words, somebody has to do it; their clients aren't qualified to do it. But they should mostly be telling people to buy index funds and to use premixed asset-allocation decisions that conform to what somebody at the headquarters has decided is optimal. To add value for an individual, what you really need to do is be more like a psychologist and a life counselor who says, “You have too much debt, you're not saving enough; you have too many houses; at some point your assets become a liability.” Or you don't have a house at all, you are a renter—you might want to consider a house as a hedge against inflation. But telling them which securities to buy or micromanaging the list of mutual funds, to me, is a fool's errand for most people.Inside the business, that's the public-facing side. Inside the business there are too many security analysts, too many asset allocators, too many broker/dealers. And I think that competition has become more and more people fighting over fewer and fewer real alpha opportunities, and that's why the competition feels so fierce. It used to be an easy business. And it's not easy anymore because the market is more efficient, I guess.Ptak: Wanted to shift gears and talk about asset allocation, specifically the 60/40 portfolio. And my question for you, which is a question I think many are asking, is the 60/40 debt. It's having one of its worst years ever. But the paradox is that yields are now, albeit they're still paltry, they're now a little bit higher and valuations are a tad lower, which you'd think would boost the 60/40's future prospects. What's your take on the 60/40, Larry?Siegel: I think that it's a pretty good consensus outcome of people buying what's available in the market. If you look at the supply of securities, it has to be somewhere around 60/40 because everybody holds it, and the supply and demand have to equilibrate in the long run. But why do issuers produce that ratio? I think that the underlying reason is that for a very long period of history, bonds were a very good investment. If you didn't have 40% in bonds, you wanted to, because they were producing high real returns. And that period is roughly 1981 to 2007. It's a long time. From 1940 to 1981, bonds did terribly because interest rates were going up and up and up, and we didn't have a lot of 60/40 portfolios, but what we had was mostly 0 or 100. Institutions bought fixed income to fund their pension plans. They bought fixed income to fund if there were insurance companies. The big money was in fixed income and equities were this gravy—you sold some stocks to some rich people. And over time as the stock market went up and the bond market didn't go up, you had greater interest in equities, and the consultants who emerged from this world of pension funds settled on 60/40 as a consensus. And so, you've got what I call the standard model. The allocators picked from a list of active managers in each asset class, usually buy way too many of them, didn't have access to index funds or didn't want to buy them. And so, they compared the performance of their active managers to benchmarks, fired the underperforming ones, gave more money to the outperforming ones, and since these things tend to run in cycles, generally underperform the market. They also had to have an overall asset-allocation policy where 50/50 was the tradition that they'd been coming from, but they moved it up to 60/40 because the stock market was beating the bond market and it just stayed there. Stocks are risky. So, 70/30 or 80/20 seemed like it was too volumed. We're all human, and we do what we see the person next to us doing. I think it's really just consensus-building, although there is a supply aspect to it. You have to buy what's out there. And if we all decided to increase our allocation to equities, we couldn't. But we would just be buying them from each other. This is a point Cliff Asness made. He can usually be counted on for very good thinking.Benz: Our research has found that fund investors tend to do a really poor job of utilizing so-called liquid alternative funds. If you take the illiquidity and gates away from alternatives, do you think they can still work for individual investors in the form of liquid alternatives?Siegel: Well, the term liquid alternatives has changed over time. When I started hearing about liquid alternatives in the early to mid-90s, it meant hedge funds and to some extent managed-futures funds because the stuff they were buying was liquid, and then the illiquid alternatives were venture capital and private equity. Over time, liquid alternatives have come to mean liquid to the investor. And when you securitize an alternative investment, you've removed—so that you can trade it like a stock—you've removed the one thing that has tended to give alternative investments better returns, which is the lockup. If you can lock up somebody's money for a long time, you can take risks that don't necessarily pay off in the short run, but that may pay off in the long run. If you take that away, I would rather just invest in liquid nonalternatives, stocks, bonds, and some real estate. Although some people call real estate an alternative. It's the oldest asset class, so I'm reluctant to put it in the alternatives bucket.Ptak: Wanted to shift and talk about endowments. You spent a good chunk of your career in the endowment world. And as you know, a lot of ink has been spilled concerning debates over the endowment model. Some decried it as costly and complex, others defend it as path-breaking. What are the lessons an advisor or an individual investor should take away from the success of the endowment approach? And conversely, what are the lessons they need to unlearn, so to speak?Siegel: I'll start with the last one because it's so easy. The lesson they need to unlearn is that if David Swensen can do it, so can I. He and the people at other big endowments and foundations have access to the best funds because they come to you, you don't have to go ferret them out. The best people they can afford to hire, outstanding analysts and other chief investment officers who can make millions. And if they do lose money, they have this capability of withstanding some pain. A foundation, in particular, which doesn't have professors to pay, or buildings to maintain, or students to give scholarships to, has to pay out 5% of whatever it has at the time, so if it loses some of the assets, their liabilities go down too in a one-to-one correspondence and so, at some level, they don't care. Of course, they do care because it's always better to have more money to give away than less. But the foundation isn't going to be destroyed by a 20% decline in the market.Endowments are a little trickier because the liabilities are not so flexible. If you start paying your professors less, they will just go to another place that doesn't pay less. Students will do the same thing. But these institutions also have a lot of reserve in their fundraising ability. An ordinary individual investor doesn't have any of this backstop. If I want to raise funds, I have to work harder. I'm already working as hard as I can. And I don't have the option to reduce my liabilities by saying I'm just not going to pay them. So, individuals have to be inherently more conservative. You get older, life becomes a race against diminishing capabilities and your risk level has to go down as you get older. So, there's a lifecycle effect that institutions don't experience. So, I would say that's the main lesson is, endowments and foundations have generally done well, but they have some structural advantages over individuals. Unless you have a rich uncle—a university has a rich uncle—which is the alumni and yet that's not an unlimited resource any more than your rich uncle is. But it is a backstop for bad performance.Benz: One investing paradox is that success demands humility, but humility is a tough sell. What's the humblest thing an investor can do to boost their odds of success while also attracting clients? Is it to have a long time horizon?Siegel: Well, the humblest thing an investor can do is buy index funds. It says to the client, I don't know what stocks are going to do best, but other people collectively as a market make pretty good decisions, so I'm just going to trust them to say the prices are roughly right. And when you buy an index fund, you're making a bet that the prices are roughly right. They're obviously not exactly right. In terms of having a long time horizon, it can be humility, or it could be hubris. I can claim to have a long time horizon, but I don't know what liabilities I'm going to face tomorrow, so I better have a short time horizon with some of my investments and I could also live 30 more years, so I need to have a long time horizon with other parts of my portfolio. But the time horizon issue I don't see so much as humility versus hubris, but it's a planning tool that a lot of people don't use effectively.Ptak: One of your more popular pieces of writing in recent years was an article you wrote on investing myths. If I'm not mistaken, I think you've updated it a few times to this point, the most recent being in 2020. Why'd you write it, and how would you change it if you were to update the piece yet again today?Siegel: I wrote it because somebody in Brazil paid me to come down there and give a talk on Siegel's Nine Myths of Investing. So, when that gave me an outline I had to fill in. Most of the myths have changed over time. I've updated it every two to five years. And what would I change now? Well, first of all, you'd have to go back and look at what the myths are. I don't really think I have time to go over all of them. But the one that I would change today is that stocks and bonds are always negatively correlated, so each is a good hedge against the other. It's not true. It runs in cycles. There was a period where they were positively correlated in the ‘90s and then before that at some other time, and all of a sudden, it's back. So, with stock market down, the bond market is also down, and people say, "Diversification doesn't work." Well, first of all, nobody told you to go out and buy the longest bond. Diversification within the bond market works in the sense of holding some less-volatile, shorter-term securities. They sacrifice some yield in order to get that safety. Secondly, stocks and bonds will again be uncorrelated or negatively correlated someday. But this is not that day. And there are other assets. The one that comes to mind is the original alternative investment: cash. Right now, you're losing money in cash in real terms, because inflation is so high. But, on average, over time cash has paid a percent or so over the inflation rate. And then the other one is real estate. I keep coming back to real estate because it has become the unloved stepchild in the investment world. And other than their house, nobody has any. The last time I heard somebody talking about real estate as an investment was probably in the decade of the 2000s, and probably it was going up a lot. Then there was a crash. And the crash stuck in people's minds while real estate itself turned around and went up again. And there may yet be another crash, but it's just another asset class that should probably be in your toolkit.Other myths—I kind of went out on a limb in the last version of that article and started talking more about social and political issues. One is that we can transition to entirely green energy without disrupting the entire world economy. We can't. We either have to transition slowly, which may not be good enough, but I actually happen to think it is, because energy transitions have taken a half century or so—wood, coal, coal to oil, oil to natural gas, and so forth—and the next transition is not going to be all solar and wind. Nuclear power is going to be a vital and probably the most important part of it. So, if the myth that you're subscribing to is the, let's call it the European version, although that's not quite fair because they have plenty of nuclear power in Europe. It's not going to happen, but we're going to need all the energy we've got, because the world is getting richer fast. Growth rates in China are down to 5%. That's still huge. Indonesia is higher than that, and it's a country of 300 million people that most Americans couldn't find on a map. The energy demands are going to be huge from all these different parts of the world that are growing and becoming middle class. And so that myth is something I spent a little time on in the article and I would write more about it next time.Benz: You more or less predicted the spate of inflation we would have before it happened. In fact, one of the myths you wrote about in 2020 was that the government could borrow all it wanted without sparking inflation. What did you see then and what do you think people should be monitoring to assess how long high inflation will persist into the future?Siegel: My forecast at the time was based on basic economic history from the 1700 and 1800s, which is that when the government borrows more money than it can pay back, it's going to pay it back anyway but in cheaper dollars. And the way that you get cheaper dollars is to have inflation. Inflation is a transfer of resources, of real resources, from savers who are bondholders and cash holders, to borrowers, which in this case is the government itself. So, it's tax. So, when you have a budget—that's how government budgets, it's out of balance by a lot for a long time— you're going to have a lot of inflation, because it's the only way the government is going to be able to make those payments on the bonds. I didn't see anything in the economy other than the budget deficits. And it was so early that you could say, I was wrong. There's not much difference between being a decade and a half early and being outright wrong. So, I'll say I was wrong.What I didn't see was the supply catastrophe that came with COVID and our response to COVID. So, when you get a supply shock like the one we've just been through, prices are going to rise, and you don't even need an unbalanced government budget, you don't need budget deficits for prices to rise when there are shortages of things because by ships not being able to dock and workers not coming to work, we just have never seen anything like this. And so, I think the inflation rate will come down from these astronomical rates to something more normal, 2%, 3%, 4%, 5%, but we're not going to go back to zero to 2, because governments have over-leveraged, and deleveraging is always inflationary.Ptak: What role do you think top-down macro should play in an allocation and investing process? Obviously, it's hard to correctly make a macro bet, though we've just talked about one you did correctly make, but it's even harder to translate that into a successful investment. So, should most people just avoid macro and diversify and call it a day?Siegel: If you mean macro bets to guide your general asset-allocation philosophy, I think you should. In other words, if you believe, as I do, that global economic growth, while slowing, is going to be very large in absolute terms for a very long time. In other words, the absolute terms meaning the number of overall dollars, or whatever your currency is, generated by the world economy that you want to hold equities because bonds don't give you a claim to that growth. And they give you a very indistinct claim I wouldn't bank on it. But international investors say that when a country is growing rapidly, the currency goes up, so you get a little bit of diversification that way. But equities are much more powerful, and international equities are frankly cheap relative to the United States. So, that's a macro bet, and I'm recommending it. But again, I recommended it for a long time. I thought the U.S. was expensive. It hasn't been cheap since the 2007-08-09 period. So, you should make an evaluation of those conditions and implement it through your portfolio.In general, most Americans suffer from home country bias because the U.S. is so big that you can get a pretty diversified portfolio with just the S&P 500 actually, because that's a lot of stocks, and those are all the big caps. If you lived in Belgium, you would not be under the illusion that Belgium was the whole world. It's just you can reach the border in an hour from anywhere in the country. So, you've known since you were a little kid that there's a big world out there. We Americans just don't have that intuition. So, that's why I'm saying that international is a macro bet that is reasonable to make. Now, if by macro bets you think that you act like a hedge fund and you think that the pound is going to crash, and that oil is going to go to $70 and then back to $110. No, individual investors should not do that.Benz: People aren't very good at respectfully disagreeing these days. You're someone who seems unafraid of having a fulsome debate. Besides stepping away from social media and the internet, what are some things we can do to exchange differing views without becoming polarized?Siegel: Well, if I knew I would run for President. People have become dug in—I don't like it at all. Spend a quarter of your reading time reading points of view that you know in advance you're going to disagree with, see how that person expresses themselves and what arguments they make and trying to take their side mentally while you're reading it. Consider maybe I'm wrong, maybe they're right. If I name some names, that would be too obvious where my biases are. But I would read the moderates on the other side, because the extremists are extremists, and they overstate everything. That's about all I can think of other than be nice. If the people you care about and generally respect have different views from you, ask yourself why. It's not because they're crazy or stupid or evil. It's because they've looked at the same data in the broad sense. They've looked at the same world and come up with different conclusions. Try to think about why that might happen, and then picture them doing that to you. That's about all I have to say about that.Ptak: Well, that's great advice and I think a great way to close this conversation, which we very much enjoyed, Larry. Thanks so much for your time and insights. We very much enjoyed having you on The Long View.Siegel: Well, thank you very much.Benz: Thanks so much, Larry.Ptak: Thanks for joining us on The Long View. If you could, please take a minute to subscribe to and rate the podcast on Apple, Spotify, or wherever you get your podcasts.You can follow us on Twitter @Syouth1, which is, S-Y-O-U-T-H and the number 1.Benz: And @Christine_Benz.Ptak: George Castady is our engineer for the podcast and Kari Greczek produces the show notes each week.Finally, we'd love to get your feedback. If you have a comment or a guest idea, please email us at TheLongView@Morningstar.com. Until next time, thanks for joining us.(Disclaimer: This recording is for informational purposes only and should not be considered investment advice. Opinions expressed are as of the date of recording. Such opinions are subject to change. The views and opinions of guests on this program are not necessarily those of Morningstar, Inc. and its affiliates. Morningstar and its affiliates are not affiliated with this guest or his or her business affiliates unless otherwise stated. Morningstar does not guarantee the accuracy, or the completeness of the data presented herein. Jeff Ptak is an employee of Morningstar Research Services LLC. Morningstar Research Services is a subsidiary of Morningstar, Inc. and is registered with and governed by the U.S. Securities and Exchange Commission. Morningstar Research Services shall not be responsible for any trading decisions, damages or other losses resulting from or related to the information, data analysis or opinions or their use. Past performance is not a guarantee of future results. All investments are subject to investment risk, including possible loss of principal. Individuals should seriously consider if an investment is suitable for them by referencing their own financial position, investment objectives and risk profile before making any investment decision.)