Podcasts about Formal verification

Wouter Swierstra is a Math Bachelor's from the University of Utrecht, has done his PhD with Thorsten Altenkirch at the University of Nottingham, did a post-doc at Chalmers, has experience in the industry working on facilitating the design of embedded system using FP and currently is a Professor at the University of Utrecht and co-host of the Haskell Interlude Podcast. In this episode we talk about his trajectory into formal methods and functional programming. We talk about Datatypes a la Carte, the Expression Problem, Functional Pearls, Program Synthesis vs Program Calculation, and much more! 0:00 – Intro & Welcome 0:02:08 – Announcing the Type Theory Forall Merch Store! 1:12 – Early Influences: From Lenses to Logic 4:40 – Discovering Functional Programming in Utrecht 8:15 – On Monads, Papers, and Learning by Teaching 12:20 – What Makes a Paper ‘Beautiful'? 17:50 – PhD in Nottingham: Theory Meets Community 22:00 – Writing ‘Certified Programming with Dependent Types' 29:10 – Teaching Dependent Types: Challenges and Joys 34:00 – On Agda vs Coq: Philosophies and Use Cases 38:40 – Type-Driven Development in Practice 45:05 – The Power of Elegant Proofs 52:00 – Advice to Aspiring Researchers in Type Theory 1:03:00 – Beating C with Functional Programming 1:20:00 – Formal Verification and Loop Invariants 1:33:28 – Program Calculation vs Program Synthesis 1:39:00 – Formalizing Blockchain 2:01:38 – Final Thoughts Links Wouter Website Haskell Interlude Advanced FP Summer School ttforall twitch ttforall store Discount code for 10% off: typetheory

If you're in SF, join us tomorrow for a fun meetup at CodeGen Night!If you're in NYC, join us for AI Engineer Summit! The Agent Engineering track is now sold out, but 25 tickets remain for AI Leadership and 5 tickets for the workshops. You can see the full schedule of speakers and workshops at https://ai.engineer!It's exceedingly hard to introduce someone like Bret Taylor. We could recite his Wikipedia page, or his extensive work history through Silicon Valley's greatest companies, but everyone else already does that.As a podcast by AI engineers for AI engineers, we had the opportunity to do something a little different. We wanted to dig into what Bret sees from his vantage point at the top of our industry for the last 2 decades, and how that explains the rise of the AI Architect at Sierra, the leading conversational AI/CX platform.“Across our customer base, we are seeing a new role emerge - the role of the AI architect. These leaders are responsible for helping define, manage and evolve their company's AI agent over time. They come from a variety of both technical and business backgrounds, and we think that every company will have one or many AI architects managing their AI agent and related experience.”In our conversation, Bret Taylor confirms the Paul Buchheit legend that he rewrote Google Maps in a weekend, armed with only the help of a then-nascent Google Closure Compiler and no other modern tooling. But what we find remarkable is that he was the PM of Maps, not an engineer, though of course he still identifies as one. We find this theme recurring throughout Bret's career and worldview. We think it is plain as day that AI leadership will have to be hands-on and technical, especially when the ground is shifting as quickly as it is today:“There's a lot of power in combining product and engineering into as few people as possible… few great things have been created by committee.”“If engineering is an order taking organization for product you can sometimes make meaningful things, but rarely will you create extremely well crafted breakthrough products. Those tend to be small teams who deeply understand the customer need that they're solving, who have a maniacal focus on outcomes.”“And I think the reason why is if you look at like software as a service five years ago, maybe you can have a separation of product and engineering because most software as a service created five years ago. I wouldn't say there's like a lot of technological breakthroughs required for most business applications. And if you're making expense reporting software or whatever, it's useful… You kind of know how databases work, how to build auto scaling with your AWS cluster, whatever, you know, it's just, you're just applying best practices to yet another problem. "When you have areas like the early days of mobile development or the early days of interactive web applications, which I think Google Maps and Gmail represent, or now AI agents, you're in this constant conversation with what the requirements of your customers and stakeholders are and all the different people interacting with it and the capabilities of the technology. And it's almost impossible to specify the requirements of a product when you're not sure of the limitations of the technology itself.”This is the first time the difference between technical leadership for “normal” software and for “AI” software was articulated this clearly for us, and we'll be thinking a lot about this going forward. We left a lot of nuggets in the conversation, so we hope you'll just dive in with us (and thank Bret for joining the pod!)Timestamps* 00:00:02 Introductions and Bret Taylor's background* 00:01:23 Bret's experience at Stanford and the dot-com era* 00:04:04 The story of rewriting Google Maps backend* 00:11:06 Early days of interactive web applications at Google* 00:15:26 Discussion on product management and engineering roles* 00:21:00 AI and the future of software development* 00:26:42 Bret's approach to identifying customer needs and building AI companies* 00:32:09 The evolution of business models in the AI era* 00:41:00 The future of programming languages and software development* 00:49:38 Challenges in precisely communicating human intent to machines* 00:56:44 Discussion on Artificial General Intelligence (AGI) and its impact* 01:08:51 The future of agent-to-agent communication* 01:14:03 Bret's involvement in the OpenAI leadership crisis* 01:22:11 OpenAI's relationship with Microsoft* 01:23:23 OpenAI's mission and priorities* 01:27:40 Bret's guiding principles for career choices* 01:29:12 Brief discussion on pasta-making* 01:30:47 How Bret keeps up with AI developments* 01:32:15 Exciting research directions in AI* 01:35:19 Closing remarks and hiring at Sierra Transcript[00:02:05] Introduction and Guest Welcome[00:02:05] Alessio: Hey everyone, welcome to the Latent Space Podcast. This is Alessio, partner and CTO at Decibel Partners, and I'm joined by my co host swyx, founder of smol.ai.[00:02:17] swyx: Hey, and today we're super excited to have Bret Taylor join us. Welcome. Thanks for having me. It's a little unreal to have you in the studio.[00:02:25] swyx: I've read about you so much over the years, like even before. Open AI effectively. I mean, I use Google Maps to get here. So like, thank you for everything that you've done. Like, like your story history, like, you know, I think people can find out what your greatest hits have been.[00:02:40] Bret Taylor's Early Career and Education[00:02:40] swyx: How do you usually like to introduce yourself when, you know, you talk about, you summarize your career, like, how do you look at yourself?[00:02:47] Bret: Yeah, it's a great question. You know, we, before we went on the mics here, we're talking about the audience for this podcast being more engineering. And I do think depending on the audience, I'll introduce myself differently because I've had a lot of [00:03:00] corporate and board roles. I probably self identify as an engineer more than anything else though.[00:03:04] Bret: So even when I was. Salesforce, I was coding on the weekends. So I think of myself as an engineer and then all the roles that I do in my career sort of start with that just because I do feel like engineering is sort of a mindset and how I approach most of my life. So I'm an engineer first and that's how I describe myself.[00:03:24] Bret: You majored in computer[00:03:25] swyx: science, like 1998. And, and I was high[00:03:28] Bret: school, actually my, my college degree was Oh, two undergrad. Oh, three masters. Right. That old.[00:03:33] swyx: Yeah. I mean, no, I was going, I was going like 1998 to 2003, but like engineering wasn't as, wasn't a thing back then. Like we didn't have the title of senior engineer, you know, kind of like, it was just.[00:03:44] swyx: You were a programmer, you were a developer, maybe. What was it like in Stanford? Like, what was that feeling like? You know, was it, were you feeling like on the cusp of a great computer revolution? Or was it just like a niche, you know, interest at the time?[00:03:57] Stanford and the Dot-Com Bubble[00:03:57] Bret: Well, I was at Stanford, as you said, from 1998 to [00:04:00] 2002.[00:04:02] Bret: 1998 was near the peak of the dot com bubble. So. This is back in the day where most people that they're coding in the computer lab, just because there was these sun microsystems, Unix boxes there that most of us had to do our assignments on. And every single day there was a. com like buying pizza for everybody.[00:04:20] Bret: I didn't have to like, I got. Free food, like my first two years of university and then the dot com bubble burst in the middle of my college career. And so by the end there was like tumbleweed going to the job fair, you know, it was like, cause it was hard to describe unless you were there at the time, the like level of hype and being a computer science major at Stanford was like, A thousand opportunities.[00:04:45] Bret: And then, and then when I left, it was like Microsoft, IBM.[00:04:49] Joining Google and Early Projects[00:04:49] Bret: And then the two startups that I applied to were VMware and Google. And I ended up going to Google in large part because a woman named Marissa Meyer, who had been a teaching [00:05:00] assistant when I was, what was called a section leader, which was like a junior teaching assistant kind of for one of the big interest.[00:05:05] Bret: Yes. Classes. She had gone there. And she was recruiting me and I knew her and it was sort of felt safe, you know, like, I don't know. I thought about it much, but it turned out to be a real blessing. I realized like, you know, you always want to think you'd pick Google if given the option, but no one knew at the time.[00:05:20] Bret: And I wonder if I'd graduated in like 1999 where I've been like, mom, I just got a job at pets. com. It's good. But you know, at the end I just didn't have any options. So I was like, do I want to go like make kernel software at VMware? Do I want to go build search at Google? And I chose Google. 50, 50 ball.[00:05:36] Bret: I'm not really a 50, 50 ball. So I feel very fortunate in retrospect that the economy collapsed because in some ways it forced me into like one of the greatest companies of all time, but I kind of lucked into it, I think.[00:05:47] The Google Maps Rewrite Story[00:05:47] Alessio: So the famous story about Google is that you rewrote the Google maps back in, in one week after the map quest quest maps acquisition, what was the story there?[00:05:57] Alessio: Is it. Actually true. Is it [00:06:00] being glorified? Like how, how did that come to be? And is there any detail that maybe Paul hasn't shared before?[00:06:06] Bret: It's largely true, but I'll give the color commentary. So it was actually the front end, not the back end, but it turns out for Google maps, the front end was sort of the hard part just because Google maps was.[00:06:17] Bret: Largely the first ish kind of really interactive web application, say first ish. I think Gmail certainly was though Gmail, probably a lot of people then who weren't engineers probably didn't appreciate its level of interactivity. It was just fast, but. Google maps, because you could drag the map and it was sort of graphical.[00:06:38] Bret: My, it really in the mainstream, I think, was it a map[00:06:41] swyx: quest back then that was, you had the arrows up and down, it[00:06:44] Bret: was up and down arrows. Each map was a single image and you just click left and then wait for a few seconds to the new map to let it was really small too, because generating a big image was kind of expensive on computers that day.[00:06:57] Bret: So Google maps was truly innovative in that [00:07:00] regard. The story on it. There was a small company called where two technologies started by two Danish brothers, Lars and Jens Rasmussen, who are two of my closest friends now. They had made a windows app called expedition, which had beautiful maps. Even in 2000.[00:07:18] Bret: For whenever we acquired or sort of acquired their company, Windows software was not particularly fashionable, but they were really passionate about mapping and we had made a local search product that was kind of middling in terms of popularity, sort of like a yellow page of search product. So we wanted to really go into mapping.[00:07:36] Bret: We'd started working on it. Their small team seemed passionate about it. So we're like, come join us. We can build this together.[00:07:42] Technical Challenges and Innovations[00:07:42] Bret: It turned out to be a great blessing that they had built a windows app because you're less technically constrained when you're doing native code than you are building a web browser, particularly back then when there weren't really interactive web apps and it ended up.[00:07:56] Bret: Changing the level of quality that we [00:08:00] wanted to hit with the app because we were shooting for something that felt like a native windows application. So it was a really good fortune that we sort of, you know, their unusual technical choices turned out to be the greatest blessing. So we spent a lot of time basically saying, how can you make a interactive draggable map in a web browser?[00:08:18] Bret: How do you progressively load, you know, new map tiles, you know, as you're dragging even things like down in the weeds of the browser at the time, most browsers like Internet Explorer, which was dominant at the time would only load two images at a time from the same domain. So we ended up making our map tile servers have like.[00:08:37] Bret: Forty different subdomains so we could load maps and parallels like lots of hacks. I'm happy to go into as much as like[00:08:44] swyx: HTTP connections and stuff.[00:08:46] Bret: They just like, there was just maximum parallelism of two. And so if you had a map, set of map tiles, like eight of them, so So we just, we were down in the weeds of the browser anyway.[00:08:56] Bret: So it was lots of plumbing. I can, I know a lot more about browsers than [00:09:00] most people, but then by the end of it, it was fairly, it was a lot of duct tape on that code. If you've ever done an engineering project where you're not really sure the path from point A to point B, it's almost like. Building a house by building one room at a time.[00:09:14] Bret: The, there's not a lot of architectural cohesion at the end. And then we acquired a company called Keyhole, which became Google earth, which was like that three, it was a native windows app as well, separate app, great app, but with that, we got licenses to all this satellite imagery. And so in August of 2005, we added.[00:09:33] Bret: Satellite imagery to Google Maps, which added even more complexity in the code base. And then we decided we wanted to support Safari. There was no mobile phones yet. So Safari was this like nascent browser on, on the Mac. And it turns out there's like a lot of decisions behind the scenes, sort of inspired by this windows app, like heavy use of XML and XSLT and all these like.[00:09:54] Bret: Technologies that were like briefly fashionable in the early two thousands and everyone hates now for good [00:10:00] reason. And it turns out that all of the XML functionality and Internet Explorer wasn't supporting Safari. So people are like re implementing like XML parsers. And it was just like this like pile of s**t.[00:10:11] Bret: And I had to say a s**t on your part. Yeah, of[00:10:12] Alessio: course.[00:10:13] Bret: So. It went from this like beautifully elegant application that everyone was proud of to something that probably had hundreds of K of JavaScript, which sounds like nothing. Now we're talking like people have modems, you know, not all modems, but it was a big deal.[00:10:29] Bret: So it was like slow. It took a while to load and just, it wasn't like a great code base. Like everything was fragile. So I just got. Super frustrated by it. And then one weekend I did rewrite all of it. And at the time the word JSON hadn't been coined yet too, just to give you a sense. So it's all XML.[00:10:47] swyx: Yeah.[00:10:47] Bret: So we used what is now you would call JSON, but I just said like, let's use eval so that we can parse the data fast. And, and again, that's, it would literally as JSON, but at the time there was no name for it. So we [00:11:00] just said, let's. Pass on JavaScript from the server and eval it. And then somebody just refactored the whole thing.[00:11:05] Bret: And, and it wasn't like I was some genius. It was just like, you know, if you knew everything you wished you had known at the beginning and I knew all the functionality, cause I was the primary, one of the primary authors of the JavaScript. And I just like, I just drank a lot of coffee and just stayed up all weekend.[00:11:22] Bret: And then I, I guess I developed a bit of reputation and no one knew about this for a long time. And then Paul who created Gmail and I ended up starting a company with him too, after all of this told this on a podcast and now it's large, but it's largely true. I did rewrite it and it, my proudest thing.[00:11:38] Bret: And I think JavaScript people appreciate this. Like the un G zipped bundle size for all of Google maps. When I rewrote, it was 20 K G zipped. It was like much smaller for the entire application. It went down by like 10 X. So. What happened on Google? Google is a pretty mainstream company. And so like our usage is shot up because it turns out like it's faster.[00:11:57] Bret: Just being faster is worth a lot of [00:12:00] percentage points of growth at a scale of Google. So how[00:12:03] swyx: much modern tooling did you have? Like test suites no compilers.[00:12:07] Bret: Actually, that's not true. We did it one thing. So I actually think Google, I, you can. Download it. There's a, Google has a closure compiler, a closure compiler.[00:12:15] Bret: I don't know if anyone still uses it. It's gone. Yeah. Yeah. It's sort of gone out of favor. Yeah. Well, even until recently it was better than most JavaScript minifiers because it was more like it did a lot more renaming of variables and things. Most people use ES build now just cause it's fast and closure compilers built on Java and super slow and stuff like that.[00:12:37] Bret: But, so we did have that, that was it. Okay.[00:12:39] The Evolution of Web Applications[00:12:39] Bret: So and that was treated internally, you know, it was a really interesting time at Google at the time because there's a lot of teams working on fairly advanced JavaScript when no one was. So Google suggest, which Kevin Gibbs was the tech lead for, was the first kind of type ahead, autocomplete, I believe in a web browser, and now it's just pervasive in search boxes that you sort of [00:13:00] see a type ahead there.[00:13:01] Bret: I mean, chat, dbt[00:13:01] swyx: just added it. It's kind of like a round trip.[00:13:03] Bret: Totally. No, it's now pervasive as a UI affordance, but that was like Kevin's 20 percent project. And then Gmail, Paul you know, he tells the story better than anyone, but he's like, you know, basically was scratching his own itch, but what was really neat about it is email, because it's such a productivity tool, just needed to be faster.[00:13:21] Bret: So, you know, he was scratching his own itch of just making more stuff work on the client side. And then we, because of Lars and Yen sort of like setting the bar of this windows app or like we need our maps to be draggable. So we ended up. Not only innovate in terms of having a big sync, what would be called a single page application today, but also all the graphical stuff you know, we were crashing Firefox, like it was going out of style because, you know, when you make a document object model with the idea that it's a document and then you layer on some JavaScript and then we're essentially abusing all of this, it just was running into code paths that were not.[00:13:56] Bret: Well, it's rotten, you know, at this time. And so it was [00:14:00] super fun. And, and, you know, in the building you had, so you had compilers, people helping minify JavaScript just practically, but there is a great engineering team. So they were like, that's why Closure Compiler is so good. It was like a. Person who actually knew about programming languages doing it, not just, you know, writing regular expressions.[00:14:17] Bret: And then the team that is now the Chrome team believe, and I, I don't know this for a fact, but I'm pretty sure Google is the main contributor to Firefox for a long time in terms of code. And a lot of browser people were there. So every time we would crash Firefox, we'd like walk up two floors and say like, what the hell is going on here?[00:14:35] Bret: And they would load their browser, like in a debugger. And we could like figure out exactly what was breaking. And you can't change the code, right? Cause it's the browser. It's like slow, right? I mean, slow to update. So, but we could figure out exactly where the bug was and then work around it in our JavaScript.[00:14:52] Bret: So it was just like new territory. Like so super, super fun time, just like a lot of, a lot of great engineers figuring out [00:15:00] new things. And And now, you know, the word, this term is no longer in fashion, but the word Ajax, which was asynchronous JavaScript and XML cause I'm telling you XML, but see the word XML there, to be fair, the way you made HTTP requests from a client to server was this.[00:15:18] Bret: Object called XML HTTP request because Microsoft and making Outlook web access back in the day made this and it turns out to have nothing to do with XML. It's just a way of making HTTP requests because XML was like the fashionable thing. It was like that was the way you, you know, you did it. But the JSON came out of that, you know, and then a lot of the best practices around building JavaScript applications is pre React.[00:15:44] Bret: I think React was probably the big conceptual step forward that we needed. Even my first social network after Google, we used a lot of like HTML injection and. Making real time updates was still very hand coded and it's really neat when you [00:16:00] see conceptual breakthroughs like react because it's, I just love those things where it's like obvious once you see it, but it's so not obvious until you do.[00:16:07] Bret: And actually, well, I'm sure we'll get into AI, but I, I sort of feel like we'll go through that evolution with AI agents as well that I feel like we're missing a lot of the core abstractions that I think in 10 years we'll be like, gosh, how'd you make agents? Before that, you know, but it was kind of that early days of web applications.[00:16:22] swyx: There's a lot of contenders for the reactive jobs of of AI, but no clear winner yet. I would say one thing I was there for, I mean, there's so much we can go into there. You just covered so much.[00:16:32] Product Management and Engineering Synergy[00:16:32] swyx: One thing I just, I just observe is that I think the early Google days had this interesting mix of PM and engineer, which I think you are, you didn't, you didn't wait for PM to tell you these are my, this is my PRD.[00:16:42] swyx: This is my requirements.[00:16:44] mix: Oh,[00:16:44] Bret: okay.[00:16:45] swyx: I wasn't technically a software engineer. I mean,[00:16:48] Bret: by title, obviously. Right, right, right.[00:16:51] swyx: It's like a blend. And I feel like these days, product is its own discipline and its own lore and own industry and engineering is its own thing. And there's this process [00:17:00] that happens and they're kind of separated, but you don't produce as good of a product as if they were the same person.[00:17:06] swyx: And I'm curious, you know, if, if that, if that sort of resonates in, in, in terms of like comparing early Google versus modern startups that you see out there,[00:17:16] Bret: I certainly like wear a lot of hats. So, you know, sort of biased in this, but I really agree that there's a lot of power and combining product design engineering into as few people as possible because, you know few great things have been created by committee, you know, and so.[00:17:33] Bret: If engineering is an order taking organization for product you can sometimes make meaningful things, but rarely will you create extremely well crafted breakthrough products. Those tend to be small teams who deeply understand the customer need that they're solving, who have a. Maniacal focus on outcomes.[00:17:53] Bret: And I think the reason why it's, I think for some areas, if you look at like software as a service five years ago, maybe you can have a [00:18:00] separation of product and engineering because most software as a service created five years ago. I wouldn't say there's like a lot of like. Technological breakthroughs required for most, you know, business applications.[00:18:11] Bret: And if you're making expense reporting software or whatever, it's useful. I don't mean to be dismissive of expense reporting software, but you probably just want to understand like, what are the requirements of the finance department? What are the requirements of an individual file expense report? Okay.[00:18:25] Bret: Go implement that. And you kind of know how web applications are implemented. You kind of know how to. How databases work, how to build auto scaling with your AWS cluster, whatever, you know, it's just, you're just applying best practices to yet another problem when you have areas like the early days of mobile development or the early days of interactive web applications, which I think Google Maps and Gmail represent, or now AI agents, you're in this constant conversation with what the requirements of your customers and stakeholders are and all the different people interacting with it.[00:18:58] Bret: And the capabilities of the [00:19:00] technology. And it's almost impossible to specify the requirements of a product when you're not sure of the limitations of the technology itself. And that's why I use the word conversation. It's not literal. That's sort of funny to use that word in the age of conversational AI.[00:19:15] Bret: You're constantly sort of saying, like, ideally, you could sprinkle some magic AI pixie dust and solve all the world's problems, but it's not the way it works. And it turns out that actually, I'll just give an interesting example.[00:19:26] AI Agents and Modern Tooling[00:19:26] Bret: I think most people listening probably use co pilots to code like Cursor or Devon or Microsoft Copilot or whatever.[00:19:34] Bret: Most of those tools are, they're remarkable. I'm, I couldn't, you know, imagine development without them now, but they're not autonomous yet. Like I wouldn't let it just write most code without my interactively inspecting it. We just are somewhere between it's an amazing co pilot and it's an autonomous software engineer.[00:19:53] Bret: As a product manager, like your aspirations for what the product is are like kind of meaningful. But [00:20:00] if you're a product person, yeah, of course you'd say it should be autonomous. You should click a button and program should come out the other side. The requirements meaningless. Like what matters is like, what is based on the like very nuanced limitations of the technology.[00:20:14] Bret: What is it capable of? And then how do you maximize the leverage? It gives a software engineering team, given those very nuanced trade offs. Coupled with the fact that those nuanced trade offs are changing more rapidly than any technology in my memory, meaning every few months you'll have new models with new capabilities.[00:20:34] Bret: So how do you construct a product that can absorb those new capabilities as rapidly as possible as well? That requires such a combination of technical depth and understanding the customer that you really need more integration. Of product design and engineering. And so I think it's why with these big technology waves, I think startups have a bit of a leg up relative to incumbents because they [00:21:00] tend to be sort of more self actualized in terms of just like bringing those disciplines closer together.[00:21:06] Bret: And in particular, I think entrepreneurs, the proverbial full stack engineers, you know, have a leg up as well because. I think most breakthroughs happen when you have someone who can understand those extremely nuanced technical trade offs, have a vision for a product. And then in the process of building it, have that, as I said, like metaphorical conversation with the technology, right?[00:21:30] Bret: Gosh, I ran into a technical limit that I didn't expect. It's not just like changing that feature. You might need to refactor the whole product based on that. And I think that's, that it's particularly important right now. So I don't, you know, if you, if you're building a big ERP system, probably there's a great reason to have product and engineering.[00:21:51] Bret: I think in general, the disciplines are there for a reason. I think when you're dealing with something as nuanced as the like technologies, like large language models today, there's a ton of [00:22:00] advantage of having. Individuals or organizations that integrate the disciplines more formally.[00:22:05] Alessio: That makes a lot of sense.[00:22:06] Alessio: I've run a lot of engineering teams in the past, and I think the product versus engineering tension has always been more about effort than like whether or not the feature is buildable. But I think, yeah, today you see a lot more of like. Models actually cannot do that. And I think the most interesting thing is on the startup side, people don't yet know where a lot of the AI value is going to accrue.[00:22:26] Alessio: So you have this rush of people building frameworks, building infrastructure, layered things, but we don't really know the shape of the compute. I'm curious that Sierra, like how you thought about building an house, a lot of the tooling for evals or like just, you know, building the agents and all of that.[00:22:41] Alessio: Versus how you see some of the startup opportunities that is maybe still out there.[00:22:46] Bret: We build most of our tooling in house at Sierra, not all. It's, we don't, it's not like not invented here syndrome necessarily, though, maybe slightly guilty of that in some ways, but because we're trying to build a platform [00:23:00] that's in Dorian, you know, we really want to have control over our own destiny.[00:23:03] Bret: And you had made a comment earlier that like. We're still trying to figure out who like the reactive agents are and the jury is still out. I would argue it hasn't been created yet. I don't think the jury is still out to go use that metaphor. We're sort of in the jQuery era of agents, not the react era.[00:23:19] Bret: And, and that's like a throwback for people listening,[00:23:22] swyx: we shouldn't rush it. You know?[00:23:23] Bret: No, yeah, that's my point is. And so. Because we're trying to create an enduring company at Sierra that outlives us, you know, I'm not sure we want to like attach our cart to some like to a horse where it's not clear that like we've figured out and I actually want as a company, we're trying to enable just at a high level and I'll, I'll quickly go back to tech at Sierra, we help consumer brands build customer facing AI agents.[00:23:48] Bret: So. Everyone from Sonos to ADT home security to Sirius XM, you know, if you call them on the phone and AI will pick up with you, you know, chat with them on the Sirius XM homepage. It's an AI agent called Harmony [00:24:00] that they've built on our platform. We're what are the contours of what it means for someone to build an end to end complete customer experience with AI with conversational AI.[00:24:09] Bret: You know, we really want to dive into the deep end of, of all the trade offs to do it. You know, where do you use fine tuning? Where do you string models together? You know, where do you use reasoning? Where do you use generation? How do you use reasoning? How do you express the guardrails of an agentic process?[00:24:25] Bret: How do you impose determinism on a fundamentally non deterministic technology? There's just a lot of really like as an important design space. And I could sit here and tell you, we have the best approach. Every entrepreneur will, you know. But I hope that in two years, we look back at our platform and laugh at how naive we were, because that's the pace of change broadly.[00:24:45] Bret: If you talk about like the startup opportunities, I'm not wholly skeptical of tools companies, but I'm fairly skeptical. There's always an exception for every role, but I believe that certainly there's a big market for [00:25:00] frontier models, but largely for companies with huge CapEx budgets. So. Open AI and Microsoft's Anthropic and Amazon Web Services, Google Cloud XAI, which is very well capitalized now, but I think the, the idea that a company can make money sort of pre training a foundation model is probably not true.[00:25:20] Bret: It's hard to, you're competing with just, you know, unreasonably large CapEx budgets. And I just like the cloud infrastructure market, I think will be largely there. I also really believe in the applications of AI. And I define that not as like building agents or things like that. I define it much more as like, you're actually solving a problem for a business.[00:25:40] Bret: So it's what Harvey is doing in legal profession or what cursor is doing for software engineering or what we're doing for customer experience and customer service. The reason I believe in that is I do think that in the age of AI, what's really interesting about software is it can actually complete a task.[00:25:56] Bret: It can actually do a job, which is very different than the value proposition of [00:26:00] software was to ancient history two years ago. And as a consequence, I think the way you build a solution and For a domain is very different than you would have before, which means that it's not obvious, like the incumbent incumbents have like a leg up, you know, necessarily, they certainly have some advantages, but there's just such a different form factor, you know, for providing a solution and it's just really valuable.[00:26:23] Bret: You know, it's. Like just think of how much money cursor is saving software engineering teams or the alternative, how much revenue it can produce tool making is really challenging. If you look at the cloud market, just as a analog, there are a lot of like interesting tools, companies, you know, Confluent, Monetized Kafka, Snowflake, Hortonworks, you know, there's a, there's a bunch of them.[00:26:48] Bret: A lot of them, you know, have that mix of sort of like like confluence or have the open source or open core or whatever you call it. I, I, I'm not an expert in this area. You know, I do think [00:27:00] that developers are fickle. I think that in the tool space, I probably like. Default towards open source being like the area that will win.[00:27:09] Bret: It's hard to build a company around this and then you end up with companies sort of built around open source to that can work. Don't get me wrong, but I just think that it's nowadays the tools are changing so rapidly that I'm like, not totally skeptical of tool makers, but I just think that open source will broadly win, but I think that the CapEx required for building frontier models is such that it will go to a handful of big companies.[00:27:33] Bret: And then I really believe in agents for specific domains which I think will, it's sort of the analog to software as a service in this new era. You know, it's like, if you just think of the cloud. You can lease a server. It's just a low level primitive, or you can buy an app like you know, Shopify or whatever.[00:27:51] Bret: And most people building a storefront would prefer Shopify over hand rolling their e commerce storefront. I think the same thing will be true of AI. So [00:28:00] I've. I tend to like, if I have a, like an entrepreneur asked me for advice, I'm like, you know, move up the stack as far as you can towards a customer need.[00:28:09] Bret: Broadly, but I, but it doesn't reduce my excitement about what is the reactive building agents kind of thing, just because it is, it is the right question to ask, but I think we'll probably play out probably an open source space more than anything else.[00:28:21] swyx: Yeah, and it's not a priority for you. There's a lot in there.[00:28:24] swyx: I'm kind of curious about your idea maze towards, there are many customer needs. You happen to identify customer experience as yours, but it could equally have been coding assistance or whatever. I think for some, I'm just kind of curious at the top down, how do you look at the world in terms of the potential problem space?[00:28:44] swyx: Because there are many people out there who are very smart and pick the wrong problem.[00:28:47] Bret: Yeah, that's a great question.[00:28:48] Future of Software Development[00:28:48] Bret: By the way, I would love to talk about the future of software, too, because despite the fact it didn't pick coding, I have a lot of that, but I can talk to I can answer your question, though, you know I think when a technology is as [00:29:00] cool as large language models.[00:29:02] Bret: You just see a lot of people starting from the technology and searching for a problem to solve. And I think it's why you see a lot of tools companies, because as a software engineer, you start building an app or a demo and you, you encounter some pain points. You're like,[00:29:17] swyx: a lot of[00:29:17] Bret: people are experiencing the same pain point.[00:29:19] Bret: What if I make it? That it's just very incremental. And you know, I always like to use the metaphor, like you can sell coffee beans, roasted coffee beans. You can add some value. You took coffee beans and you roasted them and roasted coffee beans largely, you know, are priced relative to the cost of the beans.[00:29:39] Bret: Or you can sell a latte and a latte. Is rarely priced directly like as a percentage of coffee bean prices. In fact, if you buy a latte at the airport, it's a captive audience. So it's a really expensive latte. And there's just a lot that goes into like. How much does a latte cost? And I bring it up because there's a supply chain from growing [00:30:00] coffee beans to roasting coffee beans to like, you know, you could make one at home or you could be in the airport and buy one and the margins of the company selling lattes in the airport is a lot higher than the, you know, people roasting the coffee beans and it's because you've actually solved a much more acute human problem in the airport.[00:30:19] Bret: And, and it's just worth a lot more to that person in that moment. It's kind of the way I think about technology too. It sounds funny to liken it to coffee beans, but you're selling tools on top of a large language model yet in some ways your market is big, but you're probably going to like be price compressed just because you're sort of a piece of infrastructure and then you have open source and all these other things competing with you naturally.[00:30:43] Bret: If you go and solve a really big business problem for somebody, that's actually like a meaningful business problem that AI facilitates, they will value it according to the value of that business problem. And so I actually feel like people should just stop. You're like, no, that's, that's [00:31:00] unfair. If you're searching for an idea of people, I, I love people trying things, even if, I mean, most of the, a lot of the greatest ideas have been things no one believed in.[00:31:07] Bret: So I like, if you're passionate about something, go do it. Like who am I to say, yeah, a hundred percent. Or Gmail, like Paul as far, I mean I, some of it's Laura at this point, but like Gmail is Paul's own email for a long time. , and then I amusingly and Paul can't correct me, I'm pretty sure he sent her in a link and like the first comment was like, this is really neat.[00:31:26] Bret: It would be great. It was not your email, but my own . I don't know if it's a true story. I'm pretty sure it's, yeah, I've read that before. So scratch your own niche. Fine. Like it depends on what your goal is. If you wanna do like a venture backed company, if its a. Passion project, f*****g passion, do it like don't listen to anybody.[00:31:41] Bret: In fact, but if you're trying to start, you know an enduring company, solve an important business problem. And I, and I do think that in the world of agents, the software industries has shifted where you're not just helping people more. People be more productive, but you're actually accomplishing tasks autonomously.[00:31:58] Bret: And as a consequence, I think the [00:32:00] addressable market has just greatly expanded just because software can actually do things now and actually accomplish tasks and how much is coding autocomplete worth. A fair amount. How much is the eventual, I'm certain we'll have it, the software agent that actually writes the code and delivers it to you, that's worth a lot.[00:32:20] Bret: And so, you know, I would just maybe look up from the large language models and start thinking about the economy and, you know, think from first principles. I don't wanna get too far afield, but just think about which parts of the economy. We'll benefit most from this intelligence and which parts can absorb it most easily.[00:32:38] Bret: And what would an agent in this space look like? Who's the customer of it is the technology feasible. And I would just start with these business problems more. And I think, you know, the best companies tend to have great engineers who happen to have great insight into a market. And it's that last part that I think some people.[00:32:56] Bret: Whether or not they have, it's like people start so much in the technology, they [00:33:00] lose the forest for the trees a little bit.[00:33:02] Alessio: How do you think about the model of still selling some sort of software versus selling more package labor? I feel like when people are selling the package labor, it's almost more stateless, you know, like it's easier to swap out if you're just putting an input and getting an output.[00:33:16] Alessio: If you think about coding, if there's no ID, you're just putting a prompt and getting back an app. It doesn't really matter. Who generates the app, you know, you have less of a buy in versus the platform you're building, I'm sure on the backend customers have to like put on their documentation and they have, you know, different workflows that they can tie in what's kind of like the line to draw there versus like going full where you're managed customer support team as a service outsource versus.[00:33:40] Alessio: This is the Sierra platform that you can build on. What was that decision? I'll sort of[00:33:44] Bret: like decouple the question in some ways, which is when you have something that's an agent, who is the person using it and what do they want to do with it? So let's just take your coding agent for a second. I will talk about Sierra as well.[00:33:59] Bret: Who's the [00:34:00] customer of a, an agent that actually produces software? Is it a software engineering manager? Is it a software engineer? And it's there, you know, intern so to speak. I don't know. I mean, we'll figure this out over the next few years. Like what is that? And is it generating code that you then review?[00:34:16] Bret: Is it generating code with a set of unit tests that pass, what is the actual. For lack of a better word contract, like, how do you know that it did what you wanted it to do? And then I would say like the product and the pricing, the packaging model sort of emerged from that. And I don't think the world's figured out.[00:34:33] Bret: I think it'll be different for every agent. You know, in our customer base, we do what's called outcome based pricing. So essentially every time the AI agent. Solves the problem or saves a customer or whatever it might be. There's a pre negotiated rate for that. We do that. Cause it's, we think that that's sort of the correct way agents, you know, should be packaged.[00:34:53] Bret: I look back at the history of like cloud software and notably the introduction of the browser, which led to [00:35:00] software being delivered in a browser, like Salesforce to. Famously invented sort of software as a service, which is both a technical delivery model through the browser, but also a business model, which is you subscribe to it rather than pay for a perpetual license.[00:35:13] Bret: Those two things are somewhat orthogonal, but not really. If you think about the idea of software running in a browser, that's hosted. Data center that you don't own, you sort of needed to change the business model because you don't, you can't really buy a perpetual license or something otherwise like, how do you afford making changes to it?[00:35:31] Bret: So it only worked when you were buying like a new version every year or whatever. So to some degree, but then the business model shift actually changed business as we know it, because now like. Things like Adobe Photoshop. Now you subscribe to rather than purchase. So it ended up where you had a technical shift and a business model shift that were very logically intertwined that actually the business model shift was turned out to be as significant as the technical as the shift.[00:35:59] Bret: And I think with [00:36:00] agents, because they actually accomplish a job, I do think that it doesn't make sense to me that you'd pay for the privilege of like. Using the software like that coding agent, like if it writes really bad code, like fire it, you know, I don't know what the right metaphor is like you should pay for a job.[00:36:17] Bret: Well done in my opinion. I mean, that's how you pay your software engineers, right? And[00:36:20] swyx: and well, not really. We paid to put them on salary and give them options and they vest over time. That's fair.[00:36:26] Bret: But my point is that you don't pay them for how many characters they write, which is sort of the token based, you know, whatever, like, There's a, that famous Apple story where we're like asking for a report of how many lines of code you wrote.[00:36:40] Bret: And one of the engineers showed up with like a negative number cause he had just like done a big refactoring. There was like a big F you to management who didn't understand how software is written. You know, my sense is like the traditional usage based or seat based thing. It's just going to look really antiquated.[00:36:55] Bret: Cause it's like asking your software engineer, how many lines of code did you write today? Like who cares? Like, cause [00:37:00] absolutely no correlation. So my old view is I don't think it's be different in every category, but I do think that that is the, if an agent is doing a job, you should, I think it properly incentivizes the maker of that agent and the customer of, of your pain for the job well done.[00:37:16] Bret: It's not always perfect to measure. It's hard to measure engineering productivity, but you can, you should do something other than how many keys you typed, you know Talk about perverse incentives for AI, right? Like I can write really long functions to do the same thing, right? So broadly speaking, you know, I do think that we're going to see a change in business models of software towards outcomes.[00:37:36] Bret: And I think you'll see a change in delivery models too. And, and, you know, in our customer base you know, we empower our customers to really have their hands on the steering wheel of what the agent does they, they want and need that. But the role is different. You know, at a lot of our customers, the customer experience operations folks have renamed themselves the AI architects, which I think is really cool.[00:37:55] Bret: And, you know, it's like in the early days of the Internet, there's the role of the webmaster. [00:38:00] And I don't know whether your webmaster is not a fashionable, you know, Term, nor is it a job anymore? I just, I don't know. Will they, our tech stand the test of time? Maybe, maybe not. But I do think that again, I like, you know, because everyone listening right now is a software engineer.[00:38:14] Bret: Like what is the form factor of a coding agent? And actually I'll, I'll take a breath. Cause actually I have a bunch of pins on them. Like I wrote a blog post right before Christmas, just on the future of software development. And one of the things that's interesting is like, if you look at the way I use cursor today, as an example, it's inside of.[00:38:31] Bret: A repackaged visual studio code environment. I sometimes use the sort of agentic parts of it, but it's largely, you know, I've sort of gotten a good routine of making it auto complete code in the way I want through tuning it properly when it actually can write. I do wonder what like the future of development environments will look like.[00:38:55] Bret: And to your point on what is a software product, I think it's going to change a lot in [00:39:00] ways that will surprise us. But I always use, I use the metaphor in my blog post of, have you all driven around in a way, Mo around here? Yeah, everyone has. And there are these Jaguars, the really nice cars, but it's funny because it still has a steering wheel, even though there's no one sitting there and the steering wheels like turning and stuff clearly in the future.[00:39:16] Bret: If once we get to that, be more ubiquitous, like why have the steering wheel and also why have all the seats facing forward? Maybe just for car sickness. I don't know, but you could totally rearrange the car. I mean, so much of the car is oriented around the driver, so. It stands to reason to me that like, well, autonomous agents for software engineering run through visual studio code.[00:39:37] Bret: That seems a little bit silly because having a single source code file open one at a time is kind of a goofy form factor for when like the code isn't being written primarily by you, but it begs the question of what's your relationship with that agent. And I think the same is true in our industry of customer experience, which is like.[00:39:55] Bret: Who are the people managing this agent? What are the tools do they need? And they definitely need [00:40:00] tools, but it's probably pretty different than the tools we had before. It's certainly different than training a contact center team. And as software engineers, I think that I would like to see particularly like on the passion project side or research side.[00:40:14] Bret: More innovation in programming languages. I think that we're bringing the cost of writing code down to zero. So the fact that we're still writing Python with AI cracks me up just cause it's like literally was designed to be ergonomic to write, not safe to run or fast to run. I would love to see more innovation and how we verify program correctness.[00:40:37] Bret: I studied for formal verification in college a little bit and. It's not very fashionable because it's really like tedious and slow and doesn't work very well. If a lot of code is being written by a machine, you know, one of the primary values we can provide is verifying that it actually does what we intend that it does.[00:40:56] Bret: I think there should be lots of interesting things in the software development life cycle, like how [00:41:00] we think of testing and everything else, because. If you think about if we have to manually read every line of code that's coming out as machines, it will just rate limit how much the machines can do. The alternative is totally unsafe.[00:41:13] Bret: So I wouldn't want to put code in production that didn't go through proper code review and inspection. So my whole view is like, I actually think there's like an AI native I don't think the coding agents don't work well enough to do this yet, but once they do, what is sort of an AI native software development life cycle and how do you actually.[00:41:31] Bret: Enable the creators of software to produce the highest quality, most robust, fastest software and know that it's correct. And I think that's an incredible opportunity. I mean, how much C code can we rewrite and rust and make it safe so that there's fewer security vulnerabilities. Can we like have more efficient, safer code than ever before?[00:41:53] Bret: And can you have someone who's like that guy in the matrix, you know, like staring at the little green things, like where could you have an operator [00:42:00] of a code generating machine be like superhuman? I think that's a cool vision. And I think too many people are focused on like. Autocomplete, you know, right now, I'm not, I'm not even, I'm guilty as charged.[00:42:10] Bret: I guess in some ways, but I just like, I'd like to see some bolder ideas. And that's why when you were joking, you know, talking about what's the react of whatever, I think we're clearly in a local maximum, you know, metaphor, like sort of conceptual local maximum, obviously it's moving really fast. I think we're moving out of it.[00:42:26] Alessio: Yeah. At the end of 23, I've read this blog post from syntax to semantics. Like if you think about Python. It's taking C and making it more semantic and LLMs are like the ultimate semantic program, right? You can just talk to them and they can generate any type of syntax from your language. But again, the languages that they have to use were made for us, not for them.[00:42:46] Alessio: But the problem is like, as long as you will ever need a human to intervene, you cannot change the language under it. You know what I mean? So I'm curious at what point of automation we'll need to get, we're going to be okay making changes. To the underlying languages, [00:43:00] like the programming languages versus just saying, Hey, you just got to write Python because I understand Python and I'm more important at the end of the day than the model.[00:43:08] Alessio: But I think that will change, but I don't know if it's like two years or five years. I think it's more nuanced actually.[00:43:13] Bret: So I think there's a, some of the more interesting programming languages bring semantics into syntax. So let me, that's a little reductive, but like Rust as an example, Rust is memory safe.[00:43:25] Bret: Statically, and that was a really interesting conceptual, but it's why it's hard to write rust. It's why most people write python instead of rust. I think rust programs are safer and faster than python, probably slower to compile. But like broadly speaking, like given the option, if you didn't have to care about the labor that went into it.[00:43:45] Bret: You should prefer a program written in Rust over a program written in Python, just because it will run more efficiently. It's almost certainly safer, et cetera, et cetera, depending on how you define safe, but most people don't write Rust because it's kind of a pain in the ass. And [00:44:00] the audience of people who can is smaller, but it's sort of better in most, most ways.[00:44:05] Bret: And again, let's say you're making a web service and you didn't have to care about how hard it was to write. If you just got the output of the web service, the rest one would be cheaper to operate. It's certainly cheaper and probably more correct just because there's so much in the static analysis implied by the rest programming language that it probably will have fewer runtime errors and things like that as well.[00:44:25] Bret: So I just give that as an example, because so rust, at least my understanding that came out of the Mozilla team, because. There's lots of security vulnerabilities in the browser and it needs to be really fast. They said, okay, we want to put more of a burden at the authorship time to have fewer issues at runtime.[00:44:43] Bret: And we need the constraint that it has to be done statically because browsers need to be really fast. My sense is if you just think about like the, the needs of a programming language today, where the role of a software engineer is [00:45:00] to use an AI to generate functionality and audit that it does in fact work as intended, maybe functionally, maybe from like a correctness standpoint, some combination thereof, how would you create a programming system that facilitated that?[00:45:15] Bret: And, you know, I bring up Rust is because I think it's a good example of like, I think given a choice of writing in C or Rust, you should choose Rust today. I think most people would say that, even C aficionados, just because. C is largely less safe for very similar, you know, trade offs, you know, for the, the system and now with AI, it's like, okay, well, that just changes the game on writing these things.[00:45:36] Bret: And so like, I just wonder if a combination of programming languages that are more structurally oriented towards the values that we need from an AI generated program, verifiable correctness and all of that. If it's tedious to produce for a person, that maybe doesn't matter. But one thing, like if I asked you, is this rest program memory safe?[00:45:58] Bret: You wouldn't have to read it, you just have [00:46:00] to compile it. So that's interesting. I mean, that's like an, that's one example of a very modest form of formal verification. So I bring that up because I do think you have AI inspect AI, you can have AI reviewed. Do AI code reviews. It would disappoint me if the best we could get was AI reviewing Python and having scaled a few very large.[00:46:21] Bret: Websites that were written on Python. It's just like, you know, expensive and it's like every, trust me, every team who's written a big web service in Python has experimented with like Pi Pi and all these things just to make it slightly more efficient than it naturally is. You don't really have true multi threading anyway.[00:46:36] Bret: It's just like clearly that you do it just because it's convenient to write. And I just feel like we're, I don't want to say it's insane. I just mean. I do think we're at a local maximum. And I would hope that we create a programming system, a combination of programming languages, formal verification, testing, automated code reviews, where you can use AI to generate software in a high scale way and trust it.[00:46:59] Bret: And you're [00:47:00] not limited by your ability to read it necessarily. I don't know exactly what form that would take, but I feel like that would be a pretty cool world to live in.[00:47:08] Alessio: Yeah. We had Chris Lanner on the podcast. He's doing great work with modular. I mean, I love. LVM. Yeah. Basically merging rust in and Python.[00:47:15] Alessio: That's kind of the idea. Should be, but I'm curious is like, for them a big use case was like making it compatible with Python, same APIs so that Python developers could use it. Yeah. And so I, I wonder at what point, well, yeah.[00:47:26] Bret: At least my understanding is they're targeting the data science Yeah. Machine learning crowd, which is all written in Python, so still feels like a local maximum.[00:47:34] Bret: Yeah.[00:47:34] swyx: Yeah, exactly. I'll force you to make a prediction. You know, Python's roughly 30 years old. In 30 years from now, is Rust going to be bigger than Python?[00:47:42] Bret: I don't know this, but just, I don't even know this is a prediction. I just am sort of like saying stuff I hope is true. I would like to see an AI native programming language and programming system, and I use language because I'm not sure language is even the right thing, but I hope in 30 years, there's an AI native way we make [00:48:00] software that is wholly uncorrelated with the current set of programming languages.[00:48:04] Bret: or not uncorrelated, but I think most programming languages today were designed to be efficiently authored by people and some have different trade offs.[00:48:15] Evolution of Programming Languages[00:48:15] Bret: You know, you have Haskell and others that were designed for abstractions for parallelism and things like that. You have programming languages like Python, which are designed to be very easily written, sort of like Perl and Python lineage, which is why data scientists use it.[00:48:31] Bret: It's it can, it has a. Interactive mode, things like that. And I love, I'm a huge Python fan. So despite all my Python trash talk, a huge Python fan wrote at least two of my three companies were exclusively written in Python and then C came out of the birth of Unix and it wasn't the first, but certainly the most prominent first step after assembly language, right?[00:48:54] Bret: Where you had higher level abstractions rather than and going beyond go to, to like abstractions, [00:49:00] like the for loop and the while loop.[00:49:01] The Future of Software Engineering[00:49:01] Bret: So I just think that if the act of writing code is no longer a meaningful human exercise, maybe it will be, I don't know. I'm just saying it sort of feels like maybe it's one of those parts of history that just will sort of like go away, but there's still the role of this offer engineer, like the person actually building the system.[00:49:20] Bret: Right. And. What does a programming system for that form factor look like?[00:49:25] React and Front-End Development[00:49:25] Bret: And I, I just have a, I hope to be just like I mentioned, I remember I was at Facebook in the very early days when, when, what is now react was being created. And I remember when the, it was like released open source I had left by that time and I was just like, this is so f*****g cool.[00:49:42] Bret: Like, you know, to basically model your app independent of the data flowing through it, just made everything easier. And then now. You know, I can create, like there's a lot of the front end software gym play is like a little chaotic for me, to be honest with you. It is like, it's sort of like [00:50:00] abstraction soup right now for me, but like some of those core ideas felt really ergonomic.[00:50:04] Bret: I just wanna, I'm just looking forward to the day when someone comes up with a programming system that feels both really like an aha moment, but completely foreign to me at the same time. Because they created it with sort of like from first principles recognizing that like. Authoring code in an editor is maybe not like the primary like reason why a programming system exists anymore.[00:50:26] Bret: And I think that's like, that would be a very exciting day for me.[00:50:28] The Role of AI in Programming[00:50:28] swyx: Yeah, I would say like the various versions of this discussion have happened at the end of the day, you still need to precisely communicate what you want. As a manager of people, as someone who has done many, many legal contracts, you know how hard that is.[00:50:42] swyx: And then now we have to talk to machines doing that and AIs interpreting what we mean and reading our minds effectively. I don't know how to get across that barrier of translating human intent to instructions. And yes, it can be more declarative, but I don't know if it'll ever Crossover from being [00:51:00] a programming language to something more than that.[00:51:02] Bret: I agree with you. And I actually do think if you look at like a legal contract, you know, the imprecision of the English language, it's like a flaw in the system. How many[00:51:12] swyx: holes there are.[00:51:13] Bret: And I do think that when you're making a mission critical software system, I don't think it should be English language prompts.[00:51:19] Bret: I think that is silly because you want the precision of a a programming language. My point was less about that and more about if the actual act of authoring it, like if you.[00:51:32] Formal Verification in Software[00:51:32] Bret: I'll think of some embedded systems do use formal verification. I know it's very common in like security protocols now so that you can, because the importance of correctness is so great.[00:51:41] Bret: My intellectual exercise is like, why not do that for all software? I mean, probably that's silly just literally to do what we literally do for. These low level security protocols, but the only reason we don't is because it's hard and tedious and hard and tedious are no longer factors. So, like, if I could, I mean, [00:52:00] just think of, like, the silliest app on your phone right now, the idea that that app should be, like, formally verified for its correctness feels laughable right now because, like, God, why would you spend the time on it?[00:52:10] Bret: But if it's zero costs, like, yeah, I guess so. I mean, it never crashed. That's probably good. You know, why not? I just want to, like, set our bars really high. Like. We should make, software has been amazing. Like there's a Mark Andreessen blog post, software is eating the world. And you know, our whole life is, is mediated digitally.[00:52:26] Bret: And that's just increasing with AI. And now we'll have our personal agents talking to the agents on the CRO platform and it's agents all the way down, you know, our core infrastructure is running on these digital systems. We now have like, and we've had a shortage of software developers for my entire life.[00:52:45] Bret: And as a consequence, you know if you look, remember like health care, got healthcare. gov that fiasco security vulnerabilities leading to state actors getting access to critical infrastructure. I'm like. We now have like created this like amazing system that can [00:53:00] like, we can fix this, you know, and I, I just want to, I'm both excited about the productivity gains in the economy, but I just think as software engineers, we should be bolder.[00:53:08] Bret: Like we should have aspirations to fix these systems so that like in general, as you said, as precise as we want to be in the specification of the system. We can make it work correctly now, and I'm being a little bit hand wavy, and I think we need some systems. I think that's where we should set the bar, especially when so much of our life depends on this critical digital infrastructure.[00:53:28] Bret: So I'm I'm just like super optimistic about it. But actually, let's go to w

Welcome to The GeekNarrator podcast! In this episode, host Kaivalya Apte goes deeper into the practical applications of formal methods with Jack Vanlightly, a principal technologist at Confluent. With years of experience in distributed systems, Jack discusses his journey and how formal methods have been instrumental in system design verification and bug detection. The conversation covers Jack's background, his process of using formal methods, the significance of modelling, verification, documentation, and systems learning, as well as the future evolution of tooling and its applications. Tune in to understand the intricacies of how formal methods can transform your approach to distributed systems! Chapters: 00:00 Introduction to the episode 00:37 Meet Jack VanLightly: Principal Technologist at Confluent 02:17 Jack's Journey into Distributed Systems 04:29 Discovering the Power of Formal Methods 08:11 Modeling and Simulation in Formal Methods 13:43 Verification and Safety Properties 19:02 Documentation and Communication Challenges 20:43 Formal Methods as a Systems Learning Tool 24:26 Practical Applications and Case Studies 56:38 Future of Formal Verification and Closing Thoughts Jack's Blog: https://jack-vanlightly.com/ Become a member of The GeekNarrator to get access to member only videos, notes and monthly 1:1 with me. Like building stuff? Try out CodeCrafters and build amazing real world systems like Redis, Kafka, Sqlite. Use the link below to signup and get 40% off on paid subscription. https://app.codecrafters.io/join?via=geeknarrator If you like this episode, please hit the like button and share it with your network. Also please subscribe if you haven't yet. Database internals series: https://youtu.be/yV_Zp0Mi3xs Popular playlists: Realtime streaming systems: https://www.youtube.com/playlist?list=PLL7QpTxsA4se-mAKKoVOs3VcaP71X_LA- Software Engineering: https://www.youtube.com/playlist?list=PLL7QpTxsA4sf6By03bot5BhKoMgxDUU17 Distributed systems and databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4sfLDUnjBJXJGFhhz94jDd_d Modern databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4scSeZAsCUXijtnfW5ARlrsN Stay Curios! Keep Learning!

Navid Hashemi recently defended his PhD at USC and is about to begin a post-doc at Vanderbilt.  His research focuses on the intersection of Artificial Intelligence and Temporal Logics, with applications in Formal Verification of Learning Enabled Systems and Neurosymbolic Reinforcement Learning.  Today Navid joined us for a really exciting presentation about his work on metrizable logics for reinforcement learning, and a technique for verification thereof based on the over-approximation of reachable sets using ReLU.

Professor Swarat Chaudhuri from the University of Texas at Austin and visiting researcher at Google DeepMind discusses breakthroughs in AI reasoning, theorem proving, and mathematical discovery. Chaudhuri explains his groundbreaking work on COPRA (a GPT-based prover agent), shares insights on neurosymbolic approaches to AI. Professor Swarat Chaudhuri: https://www.cs.utexas.edu/~swarat/ SPONSOR MESSAGES: CentML offers competitive pricing for GenAI model deployment, with flexible options to suit a wide range of models, from small to large-scale deployments. https://centml.ai/pricing/ Tufa AI Labs is a brand new research lab in Zurich started by Benjamin Crouzier focussed on ARC and AGI, they just acquired MindsAI - the current winners of the ARC challenge. Are you interested in working on ARC, or getting involved in their events? Goto https://tufalabs.ai/ TOC: [00:00:00] 0. Introduction / CentML ad, Tufa ad 1. AI Reasoning: From Language Models to Neurosymbolic Approaches [00:02:27] 1.1 Defining Reasoning in AI [00:09:51] 1.2 Limitations of Current Language Models [00:17:22] 1.3 Neuro-symbolic Approaches and Program Synthesis [00:24:59] 1.4 COPRA and In-Context Learning for Theorem Proving [00:34:39] 1.5 Symbolic Regression and LLM-Guided Abstraction 2. AI in Mathematics: Theorem Proving and Concept Discovery [00:43:37] 2.1 AI-Assisted Theorem Proving and Proof Verification [01:01:37] 2.2 Symbolic Regression and Concept Discovery in Mathematics [01:11:57] 2.3 Scaling and Modularizing Mathematical Proofs [01:21:53] 2.4 COPRA: In-Context Learning for Formal Theorem-Proving [01:28:22] 2.5 AI-driven theorem proving and mathematical discovery 3. Formal Methods and Challenges in AI Mathematics [01:30:42] 3.1 Formal proofs, empirical predicates, and uncertainty in AI mathematics [01:34:01] 3.2 Characteristics of good theoretical computer science research [01:39:16] 3.3 LLMs in theorem generation and proving [01:42:21] 3.4 Addressing contamination and concept learning in AI systems REFS: 00:04:58 The Chinese Room Argument, https://plato.stanford.edu/entries/chinese-room/ 00:11:42 Software 2.0, https://medium.com/@karpathy/software-2-0-a64152b37c35 00:11:57 Solving Olympiad Geometry Without Human Demonstrations, https://www.nature.com/articles/s41586-023-06747-5 00:13:26 Lean, https://lean-lang.org/ 00:15:43 A General Reinforcement Learning Algorithm That Masters Chess, Shogi, and Go Through Self-Play, https://www.science.org/doi/10.1126/science.aar6404 00:19:24 DreamCoder (Ellis et al., PLDI 2021), https://arxiv.org/abs/2006.08381 00:24:37 The Lambda Calculus, https://plato.stanford.edu/entries/lambda-calculus/ 00:26:43 Neural Sketch Learning for Conditional Program Generation, https://arxiv.org/pdf/1703.05698 00:28:08 Learning Differentiable Programs With Admissible Neural Heuristics, https://arxiv.org/abs/2007.12101 00:31:03 Symbolic Regression With a Learned Concept Library (Grayeli et al., NeurIPS 2024), https://arxiv.org/abs/2409.09359 00:41:30 Formal Verification of Parallel Programs, https://dl.acm.org/doi/10.1145/360248.360251 01:00:37 Training Compute-Optimal Large Language Models, https://arxiv.org/abs/2203.15556 01:18:19 Chain-of-Thought Prompting Elicits Reasoning in Large Language Models, https://arxiv.org/abs/2201.11903 01:18:42 Draft, Sketch, and Prove: Guiding Formal Theorem Provers With Informal Proofs, https://arxiv.org/abs/2210.12283 01:19:49 Learning Formal Mathematics From Intrinsic Motivation, https://arxiv.org/pdf/2407.00695 01:20:19 An In-Context Learning Agent for Formal Theorem-Proving (Thakur et al., CoLM 2024), https://arxiv.org/pdf/2310.04353 01:23:58 Learning to Prove Theorems via Interacting With Proof Assistants, https://arxiv.org/abs/1905.09381 01:39:58 An In-Context Learning Agent for Formal Theorem-Proving (Thakur et al., CoLM 2024), https://arxiv.org/pdf/2310.04353 01:42:24 Programmatically Interpretable Reinforcement Learning (Verma et al., ICML 2018), https://arxiv.org/abs/1804.02477

In this thought-provoking discussion with Can Gurel (Delphi), the NEAR co-founders share insights on Illia's work co-authoring the groundbreaking "Attention is All You Need" paper that introduced transformers, their vision for user-owned AI and democratizing access to AI capabilities and more. Key insights include: - How NEAR's original focus on AI developer tools led them to discover the need for better payment infrastructure for distributed workforces - Their framework for user-owned AI that optimizes for individual success rather than platform profits - Technical challenges and solutions for distributed AI training and inference - The importance of private computing for enabling user-controlled AI systems The conversation provides valuable perspective on the intersection of AI and crypto from pioneers who have worked at the cutting edge of both fields, offering insights into how decentralized technologies could enable more democratic and user-centric AI development. "The reality is we kind of have like a little bit of a model that some data we don't want to share... Your assistant will be able to help you with filing your taxes and finding ways to file receipts for expenses, and it will need to have access to all of this. What you don't want is all of this data end up leaking or showing up in somebody else's chat." — Illia Polosukhin, NEAR Protocol Watch more sessions from Crypto x AI Month here: https://delphidigital.io/crypto-ai --- Crypto x AI Month is the largest virtual event dedicated to the intersection of crypto and AI, featuring 40+ top builders, investors, and practitioners. Over the course of three weeks, this event brings together panels, debates, and discussions with the brightest minds in the space, presented by Delphi Digital. Crypto x AI Month is free and open to everyone thanks to the support from our sponsors: https://olas.network/ https://venice.ai/ https://near.org/ https://mira.foundation/ https://www.theoriq.ai/ --- Follow the Speakers: - Can Gurel on Twitter/X ► https://x.com/cannngurel - Illia Polosukhin on Twitter/X ► https://x.com/ilblackdragon - Alex Skidanov on Twitter/X ► https://x.com/alexskidanov --- Chapters 00:00 Introduction and Sponsor Acknowledgments 00:48 Introduction of Illia and Alex from NEAR 03:11 Origins of the Transformers Paper 08:07 Discussion on LLMs and Model Understanding 11:05 Alpha Go and Self-Improvement in AI 15:16 NEAR's Evolution from AI to Blockchain 19:37 The Story Behind NEAR's Foundation 26:32 Vision for User-Owned AI 30:31 Analysis of the Crypto-AI Stack 35:28 Data Labeling and Trust Systems 39:33 Private Inference and Edge Computing 44:17 User Privacy and Trust in AI 47:44 Edge Computing Challenges 49:54 Novel Data Sources and Synthetic Data 52:40 AI Agent Security and DeFi Applications 58:43 Formal Verification in AI Systems 1:03:14 Distributed Training Challenges 1:11:04 AI Talent and Crypto Industry 1:13:44 Rating Future Technologies and Closing Thoughts Disclaimer All statements and/or opinions expressed in this interview are the personal opinions and responsibility of the respective guests, who may personally hold material positions in companies or assets mentioned or discussed. The content does not necessarily reflect the opinion of Delphi Citadel Partners, LLC or its affiliates (collectively, “Delphi Ventures”), which makes no representations or warranties of any kind in connection with the contained subject matter. Delphi Ventures may hold investments in assets or protocols mentioned or discussed in this interview. This content is provided for informational purposes only and should not be misconstrued for investment advice or as a recommendation to purchase or sell any token or to use any protocol.

The Ethereum Foundation has been very active in the public discourse, notably with Vitalik's return back to X and engaging in the technical debates about the Ethereum roadmap. Following our recent podcast with Justin Drake about Ethereum's roadmap, we set out to understand one key component of that discussion in further detail. This component? Real-time proving, formal verification, and building a more resilient smart contract environment for Ethereum. While it might not get as much spotlight as the latest protocol launch or governance upgrade, formal verification is a crucial foundation for ensuring that the infrastructure we rely on is secure, efficient, and free from catastrophic errors (and thus, exploits). It's about verifying that the code we trust with billions of dollars works exactly as intended, which is especially important as we transition from optimistic rollups to full ZK rollups, with no backup. In this discussion with Alexander Hicks from the Ethereum Foundation, we explore the Foundation's ongoing efforts to apply formal verification to ZKVMs. This approach is designed to make ZKVMs more scalable, secure, and ultimately more reliable for users and developers alike. Formal verification ensures that every step of a complex system like a ZKVM is mathematically proven to be correct, offering a new layer of security to the onchain world. Alexander walks us through his journey from computer security and mathematics into the blockchain space, and how this background helped shape his work on formal verification at Ethereum. With a $20 million budget for builders and a long-term focus on sustainable solutions, Ethereum's formal verification program is aiming to safeguard not only ZK rollups but potentially Ethereum L1 itself. Whether you're a founder, builder, developer or an onchain maxi looking to get involved this one is for you. Website: https://therollup.co/ Spotify: https://open.spotify.com/show/1P6ZeYd.. Podcast: https://therollup.co/category/podcast Follow us on X: https://www.x.com/therollupco Follow Rob on X: https://www.x.com/robbie_rollup Follow Andy on X: https://www.x.com/ayyyeandy Join our TG group: https://t.me/+8ARkR_YZixE5YjBh The Rollup Disclosures: https://therollup.co/the-rollup-discl

In the past two years there has been increased interest in formal verification-based approaches to AI safety. Formal verification is a sub-field of computer science that studies how guarantees may be derived by deduction on fully-specified rule-sets and symbol systems. By contrast, the real world is a messy place that can rarely be straightforwardly represented in a reductionist way. In particular, physics, chemistry and biology are all complex sciences which do not have anything like complete symbolic rule sets. Additionally, even if we had such rules for the natural sciences, it would be very difficult for any software system to obtain sufficiently accurate models and data about initial conditions for a prover to succeed in deriving strong guarantees for AI systems operating in the real world.Practical limitations like these on formal verification have been well-understood for decades to engineers and applied mathematicians building real-world software systems, which makes [...] ---Outline:(01:23) What do we Mean by Formal Verification for AI Safety?(12:13) Challenges and Limitations(37:58) What Can Be Hoped-For?--- First published: August 19th, 2024 Source: https://www.lesswrong.com/posts/B2bg677TaS4cmDPzL/limitations-on-formal-verification-for-ai-safety --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Limitations on Formal Verification for AI Safety, published by Andrew Dickson on August 20, 2024 on LessWrong. In the past two years there has been increased interest in formal verification-based approaches to AI safety. Formal verification is a sub-field of computer science that studies how guarantees may be derived by deduction on fully-specified rule-sets and symbol systems. By contrast, the real world is a messy place that can rarely be straightforwardly represented in a reductionist way. In particular, physics, chemistry and biology are all complex sciences which do not have anything like complete symbolic rule sets. Additionally, even if we had such rules for the natural sciences, it would be very difficult for any software system to obtain sufficiently accurate models and data about initial conditions for a prover to succeed in deriving strong guarantees for AI systems operating in the real world. Practical limitations like these on formal verification have been well-understood for decades to engineers and applied mathematicians building real-world software systems, which makes it puzzling that they have mostly been dismissed by leading researchers advocating for the use of formal verification in AI safety so far. This paper will focus-in on several such limitations and use them to argue that we should be extremely skeptical of claims that formal verification-based approaches will provide strong guarantees against major AI threats in the near-term. What do we Mean by Formal Verification for AI Safety? Some examples of the kinds of threats researchers hope formal verification will help with come from the paper "Provably Safe Systems: The Only Path to Controllable AGI" [1] by Max Tegmark and Steve Omohundro (emphasis mine): Several groups are working to identify the greatest human existential risks from AGI. For example, the Center for AI Safety recently published 'An Overview of Catastrophic AI Risks' which discusses a wide range of risks including bioterrorism, automated warfare, rogue power seeking AI, etc. Provably safe systems could counteract each of the risks they describe. These authors describe a concrete bioterrorism scenario in section 2.4: a terrorist group wants to use AGI to release a deadly virus over a highly populated area. They use an AGI to design the DNA and shell of a pathogenic virus and the steps to manufacture it. They hire a chemistry lab to synthesize the DNA and integrate it into the protein shell. They use AGI controlled drones to disperse the virus and social media AGIs to spread their message after the attack. Today, groups are working on mechanisms to prevent the synthesis of dangerous DNA. But provably safe infrastructure could stop this kind of attack at every stage: biochemical design AI would not synthesize designs unless they were provably safe for humans, data center GPUs would not execute AI programs unless they were certified safe, chip manufacturing plants would not sell GPUs without provable safety checks, DNA synthesis machines would not operate without a proof of safety, drone control systems would not allow drones to fly without proofs of safety, and armies of persuasive bots would not be able to manipulate media without proof of humanness. [1] The above quote contains a number of very strong claims about the possibility of formally or mathematically provable guarantees around software systems deployed in the physical world - for example, the claim that we could have safety proofs about the real-world good behavior of DNA synthesis machines, or drones. From a practical standpoint, our default stance towards such claims should be skepticism, since we do not have proofs of this sort for any of the technologies we interact with in the real-world today. For example, DNA synthesis machines exist today and do not come with f...

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Limitations on Formal Verification for AI Safety, published by Andrew Dickson on August 20, 2024 on LessWrong. In the past two years there has been increased interest in formal verification-based approaches to AI safety. Formal verification is a sub-field of computer science that studies how guarantees may be derived by deduction on fully-specified rule-sets and symbol systems. By contrast, the real world is a messy place that can rarely be straightforwardly represented in a reductionist way. In particular, physics, chemistry and biology are all complex sciences which do not have anything like complete symbolic rule sets. Additionally, even if we had such rules for the natural sciences, it would be very difficult for any software system to obtain sufficiently accurate models and data about initial conditions for a prover to succeed in deriving strong guarantees for AI systems operating in the real world. Practical limitations like these on formal verification have been well-understood for decades to engineers and applied mathematicians building real-world software systems, which makes it puzzling that they have mostly been dismissed by leading researchers advocating for the use of formal verification in AI safety so far. This paper will focus-in on several such limitations and use them to argue that we should be extremely skeptical of claims that formal verification-based approaches will provide strong guarantees against major AI threats in the near-term. What do we Mean by Formal Verification for AI Safety? Some examples of the kinds of threats researchers hope formal verification will help with come from the paper "Provably Safe Systems: The Only Path to Controllable AGI" [1] by Max Tegmark and Steve Omohundro (emphasis mine): Several groups are working to identify the greatest human existential risks from AGI. For example, the Center for AI Safety recently published 'An Overview of Catastrophic AI Risks' which discusses a wide range of risks including bioterrorism, automated warfare, rogue power seeking AI, etc. Provably safe systems could counteract each of the risks they describe. These authors describe a concrete bioterrorism scenario in section 2.4: a terrorist group wants to use AGI to release a deadly virus over a highly populated area. They use an AGI to design the DNA and shell of a pathogenic virus and the steps to manufacture it. They hire a chemistry lab to synthesize the DNA and integrate it into the protein shell. They use AGI controlled drones to disperse the virus and social media AGIs to spread their message after the attack. Today, groups are working on mechanisms to prevent the synthesis of dangerous DNA. But provably safe infrastructure could stop this kind of attack at every stage: biochemical design AI would not synthesize designs unless they were provably safe for humans, data center GPUs would not execute AI programs unless they were certified safe, chip manufacturing plants would not sell GPUs without provable safety checks, DNA synthesis machines would not operate without a proof of safety, drone control systems would not allow drones to fly without proofs of safety, and armies of persuasive bots would not be able to manipulate media without proof of humanness. [1] The above quote contains a number of very strong claims about the possibility of formally or mathematically provable guarantees around software systems deployed in the physical world - for example, the claim that we could have safety proofs about the real-world good behavior of DNA synthesis machines, or drones. From a practical standpoint, our default stance towards such claims should be skepticism, since we do not have proofs of this sort for any of the technologies we interact with in the real-world today. For example, DNA synthesis machines exist today and do not come with f...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Limitations on Formal Verification for AI Safety, published by Andrew Dickson on August 19, 2024 on The AI Alignment Forum. In the past two years there has been increased interest in formal verification-based approaches to AI safety. Formal verification is a sub-field of computer science that studies how guarantees may be derived by deduction on fully-specified rule-sets and symbol systems. By contrast, the real world is a messy place that can rarely be straightforwardly represented in a reductionist way. In particular, physics, chemistry and biology are all complex sciences which do not have anything like complete symbolic rule sets. Additionally, even if we had such rules for the natural sciences, it would be very difficult for any software system to obtain sufficiently accurate models and data about initial conditions for a prover to succeed in deriving strong guarantees for AI systems operating in the real world. Practical limitations like these on formal verification have been well-understood for decades to engineers and applied mathematicians building real-world software systems, which makes it puzzling that they have mostly been dismissed by leading researchers advocating for the use of formal verification in AI safety so far. This paper will focus-in on several such limitations and use them to argue that we should be extremely skeptical of claims that formal verification-based approaches will provide strong guarantees against major AI threats in the near-term. What do we Mean by Formal Verification for AI Safety? Some examples of the kinds of threats researchers hope formal verification will help with come from the paper "Provably Safe Systems: The Only Path to Controllable AGI" [1] by Max Tegmark and Steve Omohundro (emphasis mine): Several groups are working to identify the greatest human existential risks from AGI. For example, the Center for AI Safety recently published 'An Overview of Catastrophic AI Risks' which discusses a wide range of risks including bioterrorism, automated warfare, rogue power seeking AI, etc. Provably safe systems could counteract each of the risks they describe. These authors describe a concrete bioterrorism scenario in section 2.4: a terrorist group wants to use AGI to release a deadly virus over a highly populated area. They use an AGI to design the DNA and shell of a pathogenic virus and the steps to manufacture it. They hire a chemistry lab to synthesize the DNA and integrate it into the protein shell. They use AGI controlled drones to disperse the virus and social media AGIs to spread their message after the attack. Today, groups are working on mechanisms to prevent the synthesis of dangerous DNA. But provably safe infrastructure could stop this kind of attack at every stage: biochemical design AI would not synthesize designs unless they were provably safe for humans, data center GPUs would not execute AI programs unless they were certified safe, chip manufacturing plants would not sell GPUs without provable safety checks, DNA synthesis machines would not operate without a proof of safety, drone control systems would not allow drones to fly without proofs of safety, and armies of persuasive bots would not be able to manipulate media without proof of humanness. [1] The above quote contains a number of very strong claims about the possibility of formally or mathematically provable guarantees around software systems deployed in the physical world - for example, the claim that we could have safety proofs about the real-world good behavior of DNA synthesis machines, or drones. From a practical standpoint, our default stance towards such claims should be skepticism, since we do not have proofs of this sort for any of the technologies we interact with in the real-world today. For example, DNA synthesis machines exist today and do no...

Summary In this week's episode, Anna (https://x.com/AnnaRRose) chats with Jens Groth (https://x.com/JensGroth16) and Daniel Marin (https://x.com/danielmarinq) from Nexus (https://nexus.xyz/). They catch up on all things Groth16 with the author himself before diving into a variety topics, such as formal verification in the context of ZKPs, the Nexus architecture, the benefits and challenges of building a system from the ground up, folding and IVC plus the properties these offer in a zkVM context and much more. Here's some additional links for this episode: ZKProof Conference in Berlin (https://zkproof.org/events/zkproof-6-berlin/) Nova: Recursive Zero-Knowledge Arguments from Folding Schemes by Kothapalli, Setty, and Tzialla (https://eprint.iacr.org/2021/370.pdf) Nexus zkVM (https://nexus.xyz/) Episode 284: Using Formal Verification on ZK Systems with Jon Stephens (https://zeroknowledge.fm/284-2/) Jens Groth Publication List (http://www0.cs.ucl.ac.uk/staff/j.groth/) Nexus Docs (https://docs.nexus.xyz/) Nexus 1.0 Machine (https://nexus.xyz/zkvm-v1) Enabling General-Purpose Verifiable Computing | Daniel Marin (Oct 2023) on YouTube (https://www.youtube.com/watch?v=G4ziNNC4zHM) Nexus 2.0 (https://nexus.xyz/zkvm) SETI@home (https://setiathome.berkeley.edu/) zkSummit12 is happening in Lisbon on Oct 8th! Applications to speak or attend are now open at zksummit.com (https://www.zksummit.com/), speaker applications close TODAY (Aug 14th) and early bird tickets for attendance are limited! Launching soon, Namada (https://namada.net/) is a proof-of-stake L1 blockchain focused on multichain, asset-agnostic privacy, via a unified shielded set. Namada is natively interoperable with fast-finality chains via IBC, and with Ethereum using a trust-minimized bridge. Follow Namada on Twitter @namada (https://twitter.com/namada) for more information and join the community on Discord (http://discord.gg/namada). Aleo (http://aleo.org/) is a new Layer-1 blockchain that achieves the programmability of Ethereum, the privacy of Zcash, and the scalability of a rollup. As Aleo is gearing up for their mainnet launch in Q1, this is an invitation to be part of a transformational ZK journey. Dive deeper and discover more about Aleo at http://aleo.org/ (http://aleo.org/). If you like what we do: * Find all our links here! @ZeroKnowledge | Linktree (https://linktr.ee/zeroknowledge) * Subscribe to our podcast newsletter (https://zeroknowledge.substack.com) * Follow us on Twitter @zeroknowledgefm (https://twitter.com/zeroknowledgefm) * Join us on Telegram (https://zeroknowledge.fm/telegram) * Catch us on YouTube (www.youtube.com/channel/UCYWsYz5cKw4wZ9Mpe4kuM_g)

ARC's current research focus can be thought of as trying to combine mechanistic interpretability and formal verification. If we had a deep understanding of what was going on inside a neural network, we would hope to be able to use that understanding to verify that the network was not going to behave dangerously in unforeseen situations. ARC is attempting to perform this kind of verification, but using a mathematical kind of "explanation" instead of one written in natural language.To help elucidate this connection, ARC has been supporting work on Compact Proofs of Model Performance via Mechanistic Interpretability by Jason Gross, Rajashree Agrawal, Lawrence Chan and others, which we were excited to see released along with this post. While we ultimately think that provable guarantees for large neural networks are unworkable as a long-term goal, we think that this work serves as a useful springboard towards alternatives.In this [...]The original text contained 1 footnote which was omitted from this narration. --- First published: June 25th, 2024 Source: https://www.lesswrong.com/posts/SyeQjjBoEC48MvnQC/formal-verification-heuristic-explanations-and-surprise --- Narrated by TYPE III AUDIO.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Formal verification, heuristic explanations and surprise accounting, published by Jacob Hilton on June 25, 2024 on The AI Alignment Forum. ARC's current research focus can be thought of as trying to combine mechanistic interpretability and formal verification. If we had a deep understanding of what was going on inside a neural network, we would hope to be able to use that understanding to verify that the network was not going to behave dangerously in unforeseen situations. ARC is attempting to perform this kind of verification, but using a mathematical kind of "explanation" instead of one written in natural language. To help elucidate this connection, ARC has been supporting work on Compact Proofs of Model Performance via Mechanistic Interpretability by Jason Gross, Rajashree Agrawal, Lawrence Chan and others, which we were excited to see released along with this post. While we ultimately think that provable guarantees for large neural networks are unworkable as a long-term goal, we think that this work serves as a useful springboard towards alternatives. In this post, we will: Summarize ARC's takeaways from this work and the problems we see with provable guarantees Explain ARC's notion of a heuristic explanation and how it is intended to overcome these problems Describe with the help of a worked example how the quality of a heuristic explanation can be quantified, using a process we have been calling surprise accounting We are also sharing a draft by Gabriel Wu (currently visiting ARC) describing a heuristic explanation for the same model that appears in the above paper: max_of_k Heuristic Estimator Thanks to Stephanie He for help with the diagrams in this post. Thanks to Eric Neyman, Erik Jenner, Gabriel Wu, Holly Mandel, Jason Gross, Mark Xu, and Mike Winer for comments. Formal verification for neural networks In Compact Proofs of Model Performance via Mechanistic Interpretability, the authors train a small transformer on an algorithmic task to high accuracy, and then construct several different formal proofs of lower bounds on the network's accuracy. Without foraying into the details, the most interesting takeaway from ARC's perspective is the following picture: In the top right of the plot is the brute-force proof, which simply checks every possible input to the network. This gives the tightest possible bound, but is very long. Meanwhile, in the bottom left is the trivial proof, which simply states that the network is at least 0% accurate. This is very short, but gives the loosest possible bound. In between these two extremes, along the orange Pareto frontier, there are proofs that exploit more structure in the network, leading to tighter bounds for a given proof length, or put another way, shorter proofs for a given bound tightness. It is exciting to see a clear demonstration that shorter proofs better explain why the neural network has high accuracy, paralleling a common mathematical intuition that shorter proofs offer more insight. One might therefore hope that if we understood the internals of a neural network well enough, then we would be able to provide provable guarantees for very complex behaviors, even when brute-force approaches are infeasible. However, we think that such a hope is not realistic for large neural networks, because the notion of proof is too strict. The basic problem with provable guarantees is that they must account for every possible way in which different parts of the network interact with one another, even when those interactions are incidental to the network's behavior. These interactions manifest as error terms, which the proof must provide a worst-case bound for, leading to a looser bound overall. The above picture provides a good demonstration of this: moving towards the left of the plot, the best bound gets looser and looser. Mo...

The GoFetch side channel in Apple CPUs, OpenSSF's plan for secure software developer education, fuzzing vs. formal verification as a security strategy, hard problems in InfoSec (and AppSec), and more! Show Notes: https://securityweekly.com/asw-278

The GoFetch side channel in Apple CPUs, OpenSSF's plan for secure software developer education, fuzzing vs. formal verification as a security strategy, hard problems in InfoSec (and AppSec), and more! Show Notes: https://securityweekly.com/asw-278

In this episode, we talk about formal verification (FV) with Dr Ashish Darbari, founder, and CEO of Axiomise. We talk about the role of FV in enabling designs to be shipped bug-free, how FV helps developers and where it fits in the design flow, whether it's in embedded AI, IoT or high-performance computing (HPC). We also learn about Axiomise – from the company's founding in 2017 to its impact in today's embedded world.  

Anatoly Yakovenko joins us for the most information-packed episode on Solana we've ever listened to. It's a masterclass exploration of Solana's architecture and roadmap. We discuss the Saga Chapter 2, fee market optimizations, Jito and MEV, validator profitability, Solana's end game, Firedancer and client diversity, token extensions and more! - - Timestamps (00:00) Introduction (01:01) Saga Chapter 2 (10:00) Access Protocol Lab (11:01) DAS London Plug (11:58) Solana Fee Market Optimizations (20:41) Jito's Role (22:42) Scheduler Upgrades (1.17 and 1.18) (25:17) Dynamic Write Lock Fees (29:55) Wen Slashing? (33:47) Solana Consensus and Formal Verification (36:41) The Value of Client Diversity: Liveness vs Safety (41:10) Are Solana Vote Transactions Just for Pumping TPS? (44:17) Solana's End Game (47:59) Is Economic Security a Meme? (53:10) Validator Profitability and Inflation (58:32) Commoditized Data Layers and L2s (01:03:18) Firedancer Update (01:10:02) Solana's New Token Extensions (01:13:46) Squads Protocol Feature Requests + Jacked Toly (01:18:18) Advice for Solana Founders - - Access Protocol is the best way to discover premium content from crypto's top publishers and independent creators. Access Protocol has reinvented content monetization, meaning you can access this premium content without endless ads or hard-to-cancel subscriptions.  Access Protocol is crypto-native, built on Solana, and already has 225K users reading content, receiving NFTs and interacting with creators! Use this link to check out Access Protocol today:https://bit.ly/AccessProtocol_Lightspeed - - Join us at DAS (Digital Asset Summit) in London this March! DAS is the #1 institutional conference in crypto, hosted by Blockworks. Use the link below to learn more, and use LIGHTSPEED10 to get 10% off your ticket! Sign up now because the price goes up every month. See you there! Learn more + get your ticket here: https://blockworks.co/event/digital-asset-summit-2024-london/home - - Follow Anatoly: https://twitter.com/aeyakovenko Follow Mert: https://twitter.com/0xMert_ Follow Garrett: https://twitter.com/GarrettHarper_ Follow Lightspeed: https://twitter.com/Lightspeedpodhq Subscribe on YouTube: https://bit.ly/43o3Syk Subscribe on Apple: https://apple.co/3OhiXgV Subscribe on Spotify: https://spoti.fi/3OkF7PD Get top market insights and the latest in crypto news. Subscribe to Blockworks Daily Newsletter: https://blockworks.co/newsletter/ - - Resources Anatoly on Lightspeed https://spoti.fi/4bacqhA - -  Disclaimers: Lightspeed was kickstarted by a grant from the Solana Foundation. Nothing said on Lightspeed is a recommendation to buy or sell securities or tokens. This podcast is for informational purposes only, and any views expressed by anyone on the show are solely our opinions, not financial advice. Mert, Garrett and our guests may hold positions in the companies, funds, or projects discussed.

with @alive_eth @danboneh @smc90This week's all-new episode covers the convergence of two important, very top-of-mind trends: AI (artificial intelligence) & blockchains/ crypto. These domains together have major implications for how we all live our lives everyday; so this episode is for anyone just curious about, or already building in the space. The conversation covers topics ranging from deep fakes, bots, and the need for proof-of-humanity in a world of AI; to big data, large language models like ChatGPT, user control, governance, privacy and security, zero knowledge and zkML; to MEV, media, art, and much more. Our expert guests (in conversation with host Sonal Chokshi) include: Dan Boneh, Stanford Professor (and Senior Research Advisor at a16z crypto), a cryptographer who's been working on blockchains for over a decade and who specializes in cryptography, computer security, and machine learning -- all of which intersect in this episode;Ali Yahya, general partner at a16z crypto, who also previously worked at Google -- where he not only worked on a distributed system for a fleet of robots (a sort of "collective reinforcement learning") but also worked on Google Brain, where he was one of the core contributors to the machine learning library TensorFlow built at Google.The first half of the hallway-style conversation between Ali & Dan (who go back together as student and professor at Stanford) is all about how AI could benefit from crypto, and the second half on how crypto could benefit from AI... the thread throughout is the tension between centralization vs. decentralization.  So we also discuss where the intersection of crypto and AI can bring about things that aren't possible by either one of them alone...pieces referenced in this episode/ related reading:The Next Cyber Reasoning System for Cyber Security (2023) by Mohamed Ferrag, Ammar Battah, Norbert Tihanyi, Merouane Debbah, Thierry Lestable, Lucas CordeiroA New Era in Software Security: Towards Self-Healing Software via Large Language Models and Formal Verification (2023) by  Yiannis Charalambous, Norbert Tihanyi, Ridhi Jain, Youcheng Sun, Mohamed Ferrag, Lucas CordeiroFixing Hardware Security Bugs with Large Language Models (2023) by Baleegh Ahmad, Shailja Thakur, Benjamin Tan, Ramesh Karri, Hammond PearceDo Users Write More Insecure Code with AI Assistants? (2022) by Neil Perry, Megha Srivastava, Deepak Kumar, Dan BonehAsleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributions (2021) by Hammond Pearce, Baleegh Ahmad, Benjamin Tan, Brendan Dolan-Gavitt, Ramesh KarriVoting, Security, and Governance in Blockchains (2019) with Ali Yahya and Phil Daian    As a reminder: none of the following should be taken as investment, legal, business, or tax advice; please see a16z.com/disclosures for more important information -- including to a link to a list of our investments – especially since we are investors in companies mentioned in this episode. Stay Updated: Find a16z on Twitter: https://twitter.com/a16zFind a16z on LinkedIn: https://www.linkedin.com/company/a16zSubscribe on your favorite podcast app: https://a16z.simplecast.com/Follow our host: https://twitter.com/stephsmithioPlease note that the content here is for informational purposes only; should NOT be taken as legal, business, tax, or investment advice or be used to evaluate any investment or security; and is not directed at any investors or potential investors in any a16z fund. a16z and its affiliates may maintain investments in the companies discussed. For more details please see a16z.com/disclosures.

In this episode of Crypto 101 we are joined by Monier Jalal of CertiK! CertiK is a pioneer in blockchain security, utilizing best-in-class Formal Verification and AI technology to secure and monitor blockchains, smart contracts, and Web3 apps. Monier explains how CertiK is building and working with companies in Web2, Web3, and helping traditional finance institutions navigate the Web3 landscape.Get your FREE copy of "Crypto Revolution" and start making big profits from buying, selling, and trading cryptocurrency today: https://www.cryptorevolution.com/freeSubscribe to YouTube for Exclusive Content:https://www.youtube.com/@crypto101podcastFollow us on social media for leading-edge crypto updates and trade alerts:https://twitter.com/Crypto101Podhttps://instagram.com/crypto_101Guest Links:https://www.certik.com/ *This is NOT financial, tax, or legal advice*Boardwalk Flock LLC. All Rights Reserved 2023. ▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬Fog by DIZARO https://soundcloud.com/dizarofrCreative Commons — Attribution-NoDerivs 3.0 Unported — CC BY-ND 3.0 Free Download / Stream: http://bit.ly/Fog-DIZAROMusic promoted by Audio Library https://youtu.be/lAfbjt_rmE8▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬Advertising Inquiries: https://redcircle.com/brandsPrivacy & Opt-Out: https://redcircle.com/privacy

with @alive_eth @danboneh @smc90This week's all-new episode covers the convergence of two important, very top-of-mind trends: AI (artificial intelligence) & blockchains/ crypto. These domains together have major implications for how we all live our lives everyday; so this episode is for anyone just curious about, or already building in the space. The conversation covers topics ranging from deep fakes, bots, and the need for proof-of-humanity in a world of AI; to big data, large language models like ChatGPT, user control, governance, privacy and security, zero knowledge and zkML; to MEV, media, art, and much more. Our expert guests (in conversation with host Sonal Chokshi) include: Dan Boneh, Stanford Professor (and Senior Research Advisor at a16z crypto), a cryptographer who's been working on blockchains for over a decade and who specializes in cryptography, computer security, and machine learning -- all of which intersect in this episode;Ali Yahya, general partner at a16z crypto, who also previously worked at Google -- where he not only worked on a distributed system for a fleet of robots (a sort of "collective reinforcement learning") but also worked on Google Brain, where he was one of the core contributors to the machine learning library TensorFlow built at Google.The first half of the hallway-style conversation between Ali & Dan (who go back together as student and professor at Stanford) is all about how AI could benefit from crypto, and the second half on how crypto could benefit from AI... the thread throughout is the tension between centralization vs. decentralization.  So we also discuss where the intersection of crypto and AI can bring about things that aren't possible by either one of them alone...pieces referenced in this episode/ related reading:The Next Cyber Reasoning System for Cyber Security (2023) by Mohamed Ferrag, Ammar Battah, Norbert Tihanyi, Merouane Debbah, Thierry Lestable, Lucas CordeiroA New Era in Software Security: Towards Self-Healing Software via Large Language Models and Formal Verification (2023) by  Yiannis Charalambous, Norbert Tihanyi, Ridhi Jain, Youcheng Sun, Mohamed Ferrag, Lucas CordeiroFixing Hardware Security Bugs with Large Language Models (2023) by Baleegh Ahmad, Shailja Thakur, Benjamin Tan, Ramesh Karri, Hammond PearceDo Users Write More Insecure Code with AI Assistants? (2022) by Neil Perry, Megha Srivastava, Deepak Kumar, Dan BonehAsleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributions (2021) by Hammond Pearce, Baleegh Ahmad, Benjamin Tan, Brendan Dolan-Gavitt, Ramesh KarriVoting, Security, and Governance in Blockchains (2019) with Ali Yahya and Phil Daian    As a reminder: none of the following should be taken as investment, legal, business, or tax advice; please see a16z.com/disclosures for more important information -- including to a link to a list of our investments – especially since we are investors in companies mentioned in this episode.

This week Anna Rose (https://twitter.com/annarrose) chats with Jon Stephens (https://twitter.com/FormallyJon), Computer Science Ph.D. student in the UToPiA group (https://utopia.cs.utexas.edu/) at UT Austin and co-founder of Veridise (https://veridise.com/). Veridise is a blockchain auditing firm that audits smart contracts and ZK systems. They discuss what led Jon to work on system security, what tools are available to test the security of ZK systems and the process of performing formal verification on ZK systems. They also cover general ZK security, why this topic matters and ways we can incentivise ethical disclosures when bugs and vulnerabilities are found. Additional reading for this week's episode; SmartPulse: Automated Checking of Temporal Properties in Smart Contracts by Stephens, Ferles, Mariano, Lahiri, and Dillig (https://www.cs.utexas.edu/~isil/SmartPulse.pdf) Certifying Zero-Knowledge Circuits with Refinement Types by J. Liu, Kretz, H. Liu, Tan, Wang, Sun, Pearson, Miltner, Dillig, and Feng (https://eprint.iacr.org/2023/547.pdf) Practical Security Analysis of Zero-Knowledge Proof Circuits by Wen, Stephens, Chen, Ferles, Pailoor, Charbonnet, Dillig and Feng (https://eprint.iacr.org/2023/190.pdf) Episode 67: Formal Verification with Martin Lundfall (https://zeroknowledge.fm/67-2/) Episode 70: Digging into DAI with Rune Christensen from Maker (https://zeroknowledge.fm/70-2/) Episode 255: Verifying Consensus On-Chain with Succinct (https://zeroknowledge.fm/255-2/) Boogie: An Intermediate Verification Language (https://www.microsoft.com/en-us/research/project/boogie-an-intermediate-verification-language/) Circom-lib (https://docs.circom.io/circom-language/circom-insight/circom-library/) How Coders Hacked Back to ‘Rescue' $208 Million in Ethereum (https://www.vice.com/en/article/qvp5b3/how-ethereum-coders-hacked-back-to-rescue-dollar208-million-in-ethereum) zkSummit 10 is happening in London on September 20, 2023! Apply to attend now -> zkSummit 10 Application Form (https://9lcje6jbgv1.typeform.com/zkSummit10) Polygon Labs (https://polygon.technology/) is thrilled to announce Polygon 2.0: The Value Layer for the Internet (https://polygon.technology/roadmap). Polygon 2.0 and all of our ZK tech is open-source and community-driven. Reach out to the Polygon community on Discord (https://discord.gg/0xpolygon) to learn more, contribute, or join in and build the future of Web3 together with Polygon! Anoma's (https://anoma.net/) first fractal instance, Namada (https://namada.net/), is launching soon! The MASP circuit's latest update enables shielded set rewards directly in the shielded set, a novel feature that funds privacy as a public good. Follow Namada on twitter @namada (https://twitter.com/namada) for more information and join the community on Discord discord.gg/namada (https://discord.com/invite/namada). If you like what we do: * Find all our links here! @ZeroKnowledge | Linktree (https://linktr.ee/zeroknowledge) * Subscribe to our podcast newsletter (https://zeroknowledge.substack.com) * Follow us on Twitter @zeroknowledgefm (https://twitter.com/zeroknowledgefm) * Join us on Telegram (https://zeroknowledge.fm/telegram) * Catch us on YouTube (https://zeroknowledge.fm/)

It's the Season 10 finale of the Elixir Wizards podcast! José Valim, Guillaume Duboc, and Giuseppe Castagna join Wizards Owen Bickford and Dan Ivovich to dive into the prospect of types in the Elixir programming language! They break down their research on set-theoretical typing and highlight their goal of creating a type system that supports as many Elixir idioms as possible while balancing simplicity and pragmatism. José, Guillaume, and Giuseppe talk about what initially sparked this project, the challenges in bringing types to Elixir, and the benefits that the Elixir community can expect from this exciting work. Guillaume's formalization and Giuseppe's "cutting-edge research" balance José's pragmatism and "Guardian of Orthodoxy" role. Decades of theory meet the needs of a living language, with open challenges like multi-process typing ahead. They come together with a shared joy of problem-solving that will accelerate Elixir's continued growth. Key Topics Discussed in this Episode: Adding type safety to Elixir through set theoretical typing How the team chose a type system that supports as many Elixir idioms as possible Balancing simplicity and pragmatism in type system design Addressing challenges like typing maps, pattern matching, and guards The tradeoffs between Dialyzer and making types part of the core language Advantages of typing for catching bugs, documentation, and tooling The differences between typing in the Gleam programming language vs. Elixir The possibility of type inference in a set-theoretic type system The history and development of set-theoretic types over 20 years Gradual typing techniques for integrating typed and untyped code How José and Giuseppe initially connected through research papers Using types as a form of "mechanized documentation" The risks and tradeoffs of choosing syntax Cheers to another decade of Elixir! A big thanks to this season's guests and all the listeners! Links and Resources Mentioned in this Episode: Bringing Types to Elixir | Guillaume Duboc & Giuseppe Castagna | ElixirConf EU 2023 (https://youtu.be/gJJH7a2J9O8) Keynote: Celebrating the 10 Years of Elixir | José Valim | ElixirConf EU 2022 (https://youtu.be/Jf5Hsa1KOc8) OCaml industrial-strength functional programming https://ocaml.org/ ℂDuce: a language for transformation of XML documents http://www.cduce.org/ Ballerina coding language https://ballerina.io/ Luau coding language https://luau-lang.org/ Gleam type language https://gleam.run/ "The Design Principles of the Elixir Type System" (https://www.irif.fr/_media/users/gduboc/elixir-types.pdf) by G. Castagna, G. Duboc, and J. Valim "A Gradual Type System for Elixir" (https://dlnext.acm.org/doi/abs/10.1145/3427081.3427084) by M. Cassola, A. Talagorria, A. Pardo, and M. Viera "Programming with union, intersection, and negation types" (https://www.irif.fr/~gc/papers/set-theoretic-types-2022.pdf), by Giuseppe Castagna "Covariance and Contravariance: a fresh look at an old issue (a primer in advanced type systems for learning functional programmers)" (https://www.irif.fr/~gc/papers/covcon-again.pdf) by Giuseppe Castagna "A reckless introduction to Hindley-Milner type inference" (https://www.lesswrong.com/posts/vTS8K4NBSi9iyCrPo/a-reckless-introduction-to-hindley-milner-type-inference) Special Guests: Giuseppe Castagna, Guillaume Duboc, and José Valim.

In episode 74 of The Gradient Podcast, Daniel Bashir speaks to Professor Talia Ringer.Professor Ringer is an Assistant Professor with the Programming Languages, Formal Methods, and Software Engineering group at the University of Illinois at Urbana Champaign. Their research leverages proof engineering to allow programmers to more easily build formally verified software systems.Have suggestions for future podcast guests (or other feedback)? Let us know here or reach us at editor@thegradient.pubSubscribe to The Gradient Podcast:  Apple Podcasts  | Spotify | Pocket Casts | RSSFollow The Gradient on TwitterOutline:* (00:00) Daniel's long annoying intro* (02:15) Origin Story* (04:30) Why / when formal verification is important* (06:40) Concerns about ChatGPT/AutoGPT et al failures, systems for accountability* (08:20) Difficulties in making formal verification accessible* (11:45) Tactics and interactive theorem provers, interface issues* (13:25) How Prof Ringer's research first crossed paths with ML* (16:00) Concrete problems in proof automation* (16:15) How ML can help people verifying software systems* (20:05) Using LLMs for understanding / reasoning about code* (23:05) Going from tests / formal properties to code* (31:30) Is deep learning the right paradigm for dealing with relations for theorem proving? * (36:50) Architectural innovations, neuro-symbolic systems* (40:00) Hazy definitions in ML* (41:50) Baldur: Proof Generation & Repair with LLMs* (45:55) In-context learning's effectiveness for LLM-based theorem proving* (47:12) LLMs without fine-tuning for proofs* (48:45) Something ~ surprising ~ about Baldur results (maybe clickbait or maybe not)* (49:32) Asking models to construct proofs with restrictions, translating proofs to formal proofs* (52:07) Methods of proofs and relative difficulties* (57:45) Verifying / providing formal guarantees on ML systems* (1:01:15) Verifying input-output behavior and basic considerations, nature of guarantees* (1:05:20) Certified/verifies systems vs certifying/verifying systems—getting LLMs to spit out proofs along with code* (1:07:15) Interpretability and how much model internals matter, RLHF, mechanistic interpretability* (1:13:50) Levels of verification for deploying ML systems, HCI problems* (1:17:30) People (Talia) actually use Bard* (1:20:00) Dual-use and “correct behavior”* (1:24:30) Good uses of jailbreaking* (1:26:30) Talia's views on evil AI / AI safety concerns* (1:32:00) Issues with talking about “intelligence,” assumptions about what “general intelligence” means* (1:34:20) Difficulty in having grounded conversations about capabilities, transparency* (1:39:20) Great quotation to steal for your next thinkpiece + intelligence as socially defined* (1:42:45) Exciting research directions* (1:44:48) OutroLinks:* Talia's Twitter and homepage* Research* Concrete Problems in Proof Automation* Baldur: Whole-Proof Generation and Repair with LLMs* Research ideas Get full access to The Gradient at thegradientpub.substack.com/subscribe

In season 3 episode 6 of The IoT Podcast we connect with Ashish Darbari - Founder & CEO at Axiomise to discover how formal verification is being used to improve the quality, performance and security of IoT devices. Sit back, relax, tune in and be the first to discover... The IoT Podcast intro (00:00) Ashish's technology journey (01:30) How Axiomise was founded (05:39) The difference between formal and traditional verification (09:40) Proof, stimulus and debug (15:59) Challenges and misconceptions around formal (17:46) Best practices and guidelines (20:18) How has Formal Verification helped improve the quality, performance and security of IoT devices? (30:00) What's the future for formal verification? (36:09) Quick-fire questions (46:19) Thank you to today's episode sponsor Akenza.io, sign up for a 30-day free trial of their self-service platform: https://auth.akenza.io/register?utm_medium=referral&utm_source=5vmedia&utm_campaign=theiotpodcast ABOUT THE GUEST Ashish Darbari is the Founder and CEO of Axiomise, the world's only formal verification training, consulting & services company that specializes in enabling formal verification in the semi-conductor industry. The vision of Axiomise is to enable all designers and verification engineers to use formal verification for the right reasons. Connect with Ashish: https://www.linkedin.com/in/ashish-darbari/ Find out more about Axiomise: https://www.axiomise.com/ SUBSCRIBE TO THE IOT PODCAST: https://linktr.ee/theiotpodcast  Sign Up for exclusive email updates: https://theiotpodcast.com/  Contact us to become a guest/partner: https://theiotpodcast.com/contact/ Connect with host Brad King-Taylor: https://www.linkedin.com/in/brad5values/

SCION (Scalability, Control, and Isolation on Next-Generation Networks) ist ein Konzept für eine sichere Internet-Architektur, die zahlreiche Sicherheitsprobleme des heutigen Internets vermeiden helfen soll. In dieser Folge sprechen wir mit Prof. Adrian Perrig vom Institut für Informationssicherheit an der ETH Zürich, dem massgeblichen Entwickler von SCION. Adrian beschreibt SCIONs Kernkonzepte und erläutert, wie diese eingesetzt werden können, um sicheres, aber auch skalierbareres und zuverlässigeres Internet-Routing zu ermöglichen und wie Anwendungen davon profitieren können, z.B. durch aktives Multipath-Forwarding in Endsystemen. Darüber hinaus sprechen wir noch über heutige und zukünftige Einsatzfelder, die Zukunft von SCION und mögliche Standardisierungsstrategien. Mehr zu Neulich im Netz auf https://www.neulich-im.net/ music by scottholmesmusic.com Quellen: Neulich im Netz: Die (Un)Sicherheit des globalen Routings, SCION Homepage, SCION Association, SCION Software, The Complete Guide to SCION – From Design Principles to Formal Verification, C. Krähenbühl, et al. "Deployment and Scalability of an Inter-Domain Multi-Path Routing Infrastructure", In Proceedings of CoNEXT 21, Virtual Event, December 2021, A. Davidson, et al., "Tango or square dance? how tightly should we integrate network functionality in browsers?" In Proceedings of the 21st ACM Workshop on Hot Topics in Networks (HotNets '22). Association for Computing Machinery, New York, NY, USA, 205–212. (pre-print) --- Send in a voice message: https://podcasters.spotify.com/pod/show/neulich-im-netz/message

The Move Prover (MVP) is a formal verifier for smart contracts written In the Move programming language. MVP has an expressive specification language, and is fast and reliable enough that it can be run routinely by developers and in integration testing. Besides the simplicity of smart contracts and the Move language, three implementation approaches are responsible for the practicality of MVP: (1) an alias-free memory model, (2)fine-grained invariant checking, and (3) monomorphization. The entirety of the Move code for the Diem blockchain has been extensively specified and can be completely verified by MVP in a few minutes. Changes in the Diem framework must be successfully verified before being integrated into the open source repository on GitHub.

The Move Prover (MVP) is a formal verifier for smart contracts written In the Move programming language. MVP has an expressive specification language, and is fast and reliable enough that it can be run routinely by developers and in integration testing. Besides the simplicity of smart contracts and the Move language, three implementation approaches are responsible for the practicality of MVP: (1) an alias-free memory model, (2)fine-grained invariant checking, and (3) monomorphization. The entirety of the Move code for the Diem blockchain has been extensively specified and can be completely verified by MVP in a few minutes. Changes in the Diem framework must be successfully verified before being integrated into the open source repository on GitHub. About the speaker: Dr. Meng Xu is an Assistant Professor in the Cheriton School of Computer Science at the University of Waterloo, Canada. His research is in the area of system and software security, with a focus on delivering high-quality solutions to practical security programs, especially in finding and patching vulnerabilities in critical computer systems. This usually includes research and development of automated program analysis/ testing / verification tools that facilitate the security reasoning of critical programs.

Rust で使える静的検証ツールの論文を向井が読みました。

מי הוא מהנדס הפורמל ומה הוא עושה? איך נראה יום עבודה שלו? עם מי הוא עובד? למה צריך וריפיקציה רגילה אם קיים פורמל? מי קדם למי?בפרק אירחנו את אור דרוקר שענה לנו על השאלות האלה ועל עוד המון אחרות.אור בעל 4 שנים של ניסיון בתחום והציג את התחום בצורה יפה.הצטרפו אלינו לפרק שבו התראיין אצלנו מהנדס פורמל ושאלנו את השאלות שחשבנו שתרצו תשובות עליהן. אם יש לכם עוד שאלות שלא ענינו בפרק, כתבו לנו :)podcasthardreset@gmail.com

In this week's Fish Fry podcast, Ashish Darbari (Founder and CEO at Axiomise) joins me to chat about the past, present and future of formal verification. Ashish and I explore the three pillars of formal verification, how the perception of formal verification as changed over the years, and why we are seeing the increased adoption of formal verification today. Also this week, I delve into the details of a new immune-system-on-a-chip developed by the Wyss Institute at Harvard university.  

BEST PLACE TO BUY CRYPTO MINERS: https://minersdeals.com/ BEST CRYPTO TRADING TRAINING: https://benanalyst.krtra.com/t/Kbmer2... TEENAGERS MAKING $35K MINING CRYPTO: https://youtu.be/QdBAVCr5Xz0 "How much money can you make with Goldshell KD6" "Goldshell KD6 profitability" "Goldshell KD6 reviews" "Goldshell KD6" MAKE $20K/MONTH WITH HIGH TICKET AFFILIATE MARKETING: https://bit.ly/3HmUqBF BECOME BUSINESS ANALYST: https://sfbatraining.com/ BLUEPRINT TO MAKE $20/MONTH ONLINE: https://www.youtube.com/watch?v=dhD8M... HOW TO VISUALIZE: https://youtu.be/J1at4WRuSiM PLAYLISTS: HIGH TICKET AFFILIATE MARKETING TRAINING: https://www.youtube.com/watch?v=sifzx... INSPIRATION & MINDSET: https://www.youtube.com/watch?v=jQOKG... HOW TO MAKE MONEY IN THE US: https://www.youtube.com/watch?v=CilMF... SALESFORCE BUSINESS ANALYST TUTORIAL: https://www.youtube.com/watch?v=tslPG... BUILD BUSINESS CREDIT TO GET FUNDING: https://www.youtube.com/watch?v=qAgKL... HOW TO START A BUSINESS WITH NO MONEY: https://www.youtube.com/watch?v=x7vNr... IMMIGRATION: https://www.youtube.com/watch?v=2R3Bw... GoldShell KD6 is expected to be released in April 2022. It is the most powerful Kadena miner at the moment, with a has rate of 26.3 TH /s and a maximum power consumption of 2630W. The market shows that the miner's hash rate will generate around $ 12,000 in monthly profit. Which means the device can generate a yearly profit of $144,000 These values are just estimations and can go up or down depending on the market. The device is used to mine Kadena. Kadena is a public blockchain that aims to optimize for scalability and features a new smart contract language, dubbed Pact, which comes equipped with formal verification and upgradeable smart contracts. Kadena's native smart contract language, Pact, is designed to improve upon common flaws observed in Ethereum's Solidity, particularly its susceptibility to unbounded loops and lack of Formal Verification. Pact smart contracts can also be upgraded at any time without requiring a hard fork. GoldShell KD6 is a device that looks exactly like other Goldshell units. It weighs 8500g and has a size of 200 x 264 x 290mm. The miner uses the ethernet interface, with a voltage of 176V - 264V. It also comes with 2 fans. GoldShell KD6 uses Kadena algorithm, which offers an evolving BFT consensus mechanism, which makes it unique. The algorithm is not as secure as the Bitcoin algorithm, but it is quite acceptable. Although there are a few flaws here and there, the algorithm provides good profitability. Mostly the units using Kadena's algorithm are still profitable today. It is also fast and secure, which protects users against attacks by 51%. GoldShell KD6 is pretty loud compared to other Miners on the market, with a noise level of 80 decibels. Its temperature ranges from 5 - 45 °C and humidity ranges from 5 - 95 %. We believe there will be a 180 days warranty period just like most Goldshell devices on the market. "How much money can you make with Goldshell KD6" "Goldshell KD6 profitability" "Goldshell KD6 reviews" "Goldshell KD6 26.3TH" "Goldshell KD6"

In this episode, we chat with Dr. Kathleen Fisher, who was chair of the Computer Science department at Tufts University at the time of the interview. We talk about Kathleen's experience in applying formal methods and PL theory to solve significant practical problems throughout her career. Equally important, we discuss how it came to be that she is practically a pro at golf!Watch all our episodes on the Building Better Systems youtube channel.Dr. Kathleen Fisher: https://www.darpa.mil/staff/dr-kathleen-fisher HACMS: https://www.darpa.mil/program/high-assurance-cyber-military-systems PADS: https://pads.cs.tufts.edu/about.html From Dirt to Shovels paper: https://www.cs.princeton.edu/~dpw/papers/learningpopl08-final.pdf Hancock: https://dl.acm.org/doi/abs/10.1145/331960.331981PLMW: http://sigplan.org/Conferences/PLMW/ CRAW: https://cra.org/cra-wp/ NSF Broadening Participation in Computing: https://beta.nsf.gov/funding/opportunities/broadening-participation-computing-bpc-0 Joey Dodds: https://galois.com/team/joey-dodds/ Shpat Morina: https://galois.com/team/shpat-morina/ Galois, Inc.: https://galois.com/ Contact us: podcast@galois.com

Find Certik on Twitter: https://twitter.com/certik_io

This week we discuss our new formal verification course launched on 6 April, last week. If you're looking to understand how to apply formal methods, especially for industrial projects in VLSI, then we have something for you.

How do we know when the proof is the valid proof? Can we always see the proof? Are visible proofs required for verification? Can you trust invisible proofs for sign-off? Welcome to formal verification! Tune in to this week's podcast to learn more.

Dan Guido, CEO of Trail of Bits, walks us through how they work with customers to make long-term improvements in security and software quality. He also describes what blockchain has done right, and how the rest of the software world should learn from them.You can watch this episode on our Youtube Channel. https://youtube.com/c/BuildingBetterSystemsPodcastJoey Dodds: https://galois.com/team/joey-dodds/ Shpat Morina: https://galois.com/team/shpat-morina/  Dan Guido: https://www.linkedin.com/in/danguido/Trail of Bits blog: https://blog.trailofbits.com/Galois, Inc.: https://galois.com/ Contact us: podcast@galois.com

Learn how to sign-off formal verification using six dimensions of coverage. Metric-driven verification is important, but we need to consider all aspects when using formal verification including qualitative and quantitative methods. We made it easy for you to use the six dimensions of coverage to sign-off RISC-V verification. Find out about it in more detail next week at the RISC-V summit.

Dr. Darbari talks about a new coverage solution for formal verification - scenario coverage. He describes why you need it, what it is, and how this has been used to verify the latest core from the OpenHW group - CVE4. Let's cover our formal verification properly.

Video of this podcast can be found on our Youtube channel. Jean Yang: https://www.linkedin.com/in/jean-yang-96575030/Akita Software: https://www.akitasoftware.com/Galois, Inc.: https://galois.com/Joey Dodds: https://galois.com/team/joey-dodds/Shpat Morina:  https://galois.com/team/shpat-morina/Contact us: marketing@galois.com

It’s a software engineer’s dream: A compiler that can take idiomatic high-level code and output maximally efficient instructions. Ron’s guest this week is Greta Yorsh, who has worked on just that problem in a career spanning both industry and academia. Ron and Greta talk about some  of the tricks that compilers use to make our software faster, ranging from feedback-directed optimization and super-optimization to formal analysis.

iOS 14 & Android 11 security features, DuckDuckGo gets big.The most important iOS 14 privacy & security featuresAll of Android 11's new privacy & security featuresDuckDuckGo usage growth goes exponentialLAN attack bug fixed in Firefox 79 for AndroidGoodbye Forever Firefox Send and Notes... Oh, how we loved yeMicrosoft's catastrophic Zerologon vulnerabilityWhy we're headed toward formal verification of security protocolsWe invite you to read our show notes at https://www.grc.com/sn/SN-785-Notes.pdf Hosts: Steve Gibson and Leo Laporte Download or subscribe to this show at https://twit.tv/shows/security-now. You can submit a question to Security Now! at the GRC Feedback Page. For 16kbps versions, transcripts, and notes (including fixes), visit Steve's site: grc.com, also the home of the best disk maintenance and recovery utility ever written Spinrite 6. Sponsors: extrahop.com/SECURITYNOW Wasabi.com offer code SECURITYNOW securityscorecard.com/twit

iOS 14 & Android 11 security features, DuckDuckGo gets big.The most important iOS 14 privacy & security featuresAll of Android 11's new privacy & security featuresDuckDuckGo usage growth goes exponentialLAN attack bug fixed in Firefox 79 for AndroidGoodbye Forever Firefox Send and Notes... Oh, how we loved yeMicrosoft's catastrophic Zerologon vulnerabilityWhy we're headed toward formal verification of security protocolsWe invite you to read our show notes at https://www.grc.com/sn/SN-785-Notes.pdf Hosts: Steve Gibson and Leo Laporte Download or subscribe to this show at https://twit.tv/shows/security-now. You can submit a question to Security Now! at the GRC Feedback Page. For 16kbps versions, transcripts, and notes (including fixes), visit Steve's site: grc.com, also the home of the best disk maintenance and recovery utility ever written Spinrite 6. Sponsors: extrahop.com/SECURITYNOW Wasabi.com offer code SECURITYNOW securityscorecard.com/twit

iOS 14 & Android 11 security features, DuckDuckGo gets big.The most important iOS 14 privacy & security featuresAll of Android 11's new privacy & security featuresDuckDuckGo usage growth goes exponentialLAN attack bug fixed in Firefox 79 for AndroidGoodbye Forever Firefox Send and Notes... Oh, how we loved yeMicrosoft's catastrophic Zerologon vulnerabilityWhy we're headed toward formal verification of security protocolsWe invite you to read our show notes at https://www.grc.com/sn/SN-785-Notes.pdf Hosts: Steve Gibson and Leo Laporte Download or subscribe to this show at https://twit.tv/shows/security-now. You can submit a question to Security Now! at the GRC Feedback Page. For 16kbps versions, transcripts, and notes (including fixes), visit Steve's site: grc.com, also the home of the best disk maintenance and recovery utility ever written Spinrite 6. Sponsors: extrahop.com/SECURITYNOW Wasabi.com offer code SECURITYNOW securityscorecard.com/twit

iOS 14 & Android 11 security features, DuckDuckGo gets big.The most important iOS 14 privacy & security featuresAll of Android 11's new privacy & security featuresDuckDuckGo usage growth goes exponentialLAN attack bug fixed in Firefox 79 for AndroidGoodbye Forever Firefox Send and Notes... Oh, how we loved yeMicrosoft's catastrophic Zerologon vulnerabilityWhy we're headed toward formal verification of security protocolsWe invite you to read our show notes at https://www.grc.com/sn/SN-785-Notes.pdf Hosts: Steve Gibson and Leo Laporte Download or subscribe to this show at https://twit.tv/shows/security-now. You can submit a question to Security Now! at the GRC Feedback Page. For 16kbps versions, transcripts, and notes (including fixes), visit Steve's site: grc.com, also the home of the best disk maintenance and recovery utility ever written Spinrite 6. Sponsors: extrahop.com/SECURITYNOW Wasabi.com offer code SECURITYNOW securityscorecard.com/twit

iOS 14 & Android 11 security features, DuckDuckGo gets big.The most important iOS 14 privacy & security featuresAll of Android 11's new privacy & security featuresDuckDuckGo usage growth goes exponentialLAN attack bug fixed in Firefox 79 for AndroidGoodbye Forever Firefox Send and Notes... Oh, how we loved yeMicrosoft's catastrophic Zerologon vulnerabilityWhy we're headed toward formal verification of security protocolsWe invite you to read our show notes at https://www.grc.com/sn/SN-785-Notes.pdf Hosts: Steve Gibson and Leo Laporte Download or subscribe to this show at https://twit.tv/shows/security-now. You can submit a question to Security Now! at the GRC Feedback Page. For 16kbps versions, transcripts, and notes (including fixes), visit Steve's site: grc.com, also the home of the best disk maintenance and recovery utility ever written Spinrite 6. Sponsors: extrahop.com/SECURITYNOW Wasabi.com offer code SECURITYNOW securityscorecard.com/twit

iOS 14 & Android 11 security features, DuckDuckGo gets big.The most important iOS 14 privacy & security featuresAll of Android 11's new privacy & security featuresDuckDuckGo usage growth goes exponentialLAN attack bug fixed in Firefox 79 for AndroidGoodbye Forever Firefox Send and Notes... Oh, how we loved yeMicrosoft's catastrophic Zerologon vulnerabilityWhy we're headed toward formal verification of security protocolsWe invite you to read our show notes at https://www.grc.com/sn/SN-785-Notes.pdf Hosts: Steve Gibson and Leo Laporte Download or subscribe to this show at https://twit.tv/shows/security-now. You can submit a question to Security Now! at the GRC Feedback Page. For 16kbps versions, transcripts, and notes (including fixes), visit Steve's site: grc.com, also the home of the best disk maintenance and recovery utility ever written Spinrite 6. Sponsors: extrahop.com/SECURITYNOW Wasabi.com offer code SECURITYNOW securityscorecard.com/twit

Dr. Darbari demystifies the topic of architectural formal verification with the focus on RISC-V. He describes the similarities with simulation-based compliance testing and key benefits of using formalISA and formal verification for architectural compliance. A brand-new blog on this topic is available from Tech Design Forums.

In this podcast, Dr. Ashish Darbari outlines the ten reasons why formal verification should be used. Save money, find more bugs, find bugs quicker, prove bug absence, ship safe and secure chips.

In this podcast, Dr. Darbari talks about the connection between constraints and coverage in formal verification. He discusses why the two topics are closely connected using the concepts of controllability and observability and why proof-core and COI coverage are not the best mechanisms to sign-off formal verification, especially for inconclusive proofs.

In this week's episode we talk to OXBC member Lewis Harland about his journey to blockchain via a psychology degree and some great connections with other OXBC members, as well as his new newsletter Formal Verification.

In episode 3057 I talked about formal verification of software and forgot to mention why one would want to do it. This episode hopefully answers to that. While formal verification is powerful tool, it’s also rather cumbersome and slow to use. In some cases you’re better off with traditional ways of testing.

What happens when you apply formal verification to find architectural flaws in processors? In this podcast, Dr. Ashish Darbari talks about an interesting way of using Axiomise ISA formal proof kit to find bugs in RISC-V cores. He describes how by using the combination of automated formal properties from the Axiomise proof kit together with constraints we can not only find bugs but also root-cause the precise nature of simulation resistant bugs. You might like this podcast if you ever wondered how constraints together with automated formal can be used to address the complex challenges of finding corner-case bugs in your CPU designs.

One of the biggest challenges with formal verification is scoping out what constraints are needed, and how they will be coded in formal verification for efficient predictable results. In this podcast, we discuss the role of constraints in formal verification.

In this podcast, Dr. Darbari talks about the role of coverage in formal verification and sign-off. We examine why coverage is important and what can be done to sign-off the verification with confidence. We discuss the interaction between structural coverage, functional coverage in simulation, and what happens for formal verification, and what should happen?

Dr. Darbari talks about why processors need formal verification in the latest podcast. He describes why processors are complex, and why formal verification is a necessity.

In this podcast, Dr. Ashish Darbari talks about testing and formal verification for SoCs. He describes the basics of simulation-based-verification techniques such as constrained random verification, directed testing, emulation, and formal verification. 

In this episode me and Rikard Hjort talked about what Formal Verification is and how Smart Contracts gave a new lease of life to Formal Verification. Rikard is a student of computer science at Chalmers University in Gothenburg and he also works for Runtime Verification, a software company focused on verification-based tools. Runtime Verification: https://runtimeverification.com Runtime's Firefly: https://runtimeverification.com/firefly/ Prototype Formal Semantics of WebAssembly in K: https://github.com/kframework/wasm-semantics My article on Coinfomania: Security Tokens on Tezos: Higher Demand?

Jana talks about formal verification of computer systems and synthesizing controllers from models.We get an introduction to the relatively new, especially when applied to robotics, field of formal verification. Jana talks about the requirements and limits of formal verification and how she feels we are ready to start merging the computer science process with regulatory and business processes.Jana also describes how she worked on an autonomous golf cart in Singapore where the controller was synthesized.This podcast is part of the Wevolver network. Wevolver is a platform & community providing engineers informative content to help them innovate.Learn more at Wevolver.comPromote your company in our podcast?If you are interested in sponsoring the podcast, you can contact us at richard@wevolver.com

This is the spoken version of EOS Nation's EOS Hot Sauce, a weekly published series covering the hottest topics in the EOS Ecosystem. Brought to you by EOS Nation. Find the written version filled with relevant links here: eosnation.io/eos-hot-sauce-26

Dan Guido, CEO of Trail of Bits, comes on the air with us to talk about how tooling is evolving in smart contract security, the landscape of security auditing today, and we have the opportunity to congratulate him on being named a leader in Forrester Research's Midsize Cybersecurity Consulting Services, Q2 2019 report. We go over advancements in fuzzing, static analysis, formal verification, and some interesting and unexpected problems found in smart contracts. Links Forrester Wave Blogpost What blockchain got right - slides zeppelin forum for DSChief bug Anatomy of unsafe contracts Trail of Bits Twitter Donate https://donate.hashingitout.stream

Dan Guido, CEO of Trail of Bits, comes on the air with us to talk about how tooling is evolving in smart contract security, the landscape of security auditing today, and we have the opportunity to congratulate him on being named a leader in Forrester Research's Midsize Cybersecurity Consulting Services, Q2 2019 report. We go over advancements in fuzzing, static analysis, formal verification, and some interesting and unexpected problems found in smart contracts. Links Forrester Wave Blogpost What blockchain got right - slides Zeppelin forum for DSChief bug Anatomy of unsafe smart contracts Trail of Bits Twitter Donate https://donate.hashingitout.stream

Audio of a recent conference talk relating Math Mutation topics to the verification of processor designs. (Send feeback to erik@mathmutation.com)

We bring you another brilliant mind on the show for this episode! Russell O'Connor, developer of Blockstream's Simplicity programming language for Bitcoin, dives deep into formal verification topics. We learn about the language design principles driving Simplicity, how formal verification plays an integral role in making the language suitable for securely automating Bitcoin transactions, and the challenges in creating a smart contract script for Bitcoin. We get a glimpse into the future of blockchain automation, and he elucidates what is being done right and what could be done better in blockchain platforms to place security first in smart contract design. Links Simplicity's Github Simplicity Publication -- PLAS 2017 Simplicity presentation -- BPASE 2018 (slides) Illustration of formal verification using Simplicity -- Scale by the Bay 2018 (slides) Enhancing Bitcoin Transactions with Covenants -- The Financial Crypography 2017 Workshop Andrew Appel's Verified Software Toolchain for formal verification of C The Coq proof assistant Mooly Sagiv's Modularity for Decidability: Implementing and Semi-Automatically Verifying Distributed Systems

We bring you another brilliant mind on the show for this episode! Russell O'Connor, developer of Blockstream's Simplicity programming language for Bitcoin, dives deep into formal verification topics. We learn about the language design principles driving Simplicity, how formal verification plays an integral role in making the language suitable for securely automating Bitcoin transactions, and the challenges in creating a smart contract script for Bitcoin. We get a glimpse into the future of blockchain automation, and he elucidates what is being done right and what could be done better in blockchain platforms to place security first in smart contract design. Links Simplicity's Github Simplicity Publication -- PLAS 2017 Simplicity presentation -- BPASE 2018 (slides) Illustration of formal verification using Simplicity -- Scale by the Bay 2018 (slides) Enhancing Bitcoin Transactions with Covenants -- The Financial Crypography 2017 Workshop Andrew Appel's Verified Software Toolchain for formal verification of C The Coq proof assistant Mooly Sagiv's Modularity for Decidability: Implementing and Semi-Automatically Verifying Distributed Systems

We are excited to have the Kadena team back on with an update. The erudite Stuart Popejoy and brilliant Emily Pillmore speak more on Pact, the smart contract language with built-in formal verification used in the Kadena Chainweb. We learn more about formal verification: what it means, how it works, and a bit of its limitations. They also give us an update on Chainweb itself and talk us through some of the challenges since we last spoke... AND their innovated solutions to those challenges! Links: http://kadena.io https://kadena.io/en/resources/ https://twitter.com/kadena_io https://twitter.com/sirlensalot https://twitter.com/emi1ypi EVM is fundamentally unsafe Turing Complete vs Turing Incomplete Sponsorship: We'd like to thank our sponsor for this episode Trail of Bits for supporting us. Now go sign up for automatic github-integrated smart contract security at www.crytic.io !

We are excited to have the Kadena team back on with an update. The erudite Stuart Popejoy and brilliant Emily Pillmore speak more on Pact, the smart contract language with built-in formal verification used in the Kadena Chainweb. We learn more about formal verification: what it means, how it works, and a bit of its limitations. They also give us an update on Chainweb itself and talk us through some of the challenges since we last spoke... AND their innovated solutions to those challenges! show links: http://kadena.io https://kadena.io/en/resources/ https://twitter.com/kadena_io https://twitter.com/sirlensalot https://twitter.com/emi1ypi EVM is fundamentally unsafe Turing Complete vs Turing Incomplete Sponsorship: We'd like to again thank [Trail of Bits] for supporting us. Go integrate high quality security into your smart contract codebase with Crytic! Stay safe out there.

We spoke with Edison Lim, Application Lead at Zilliqa about their protocol. We consider Zilliqa’s approach to scaling and energy usage. We discuss their proof of work consensus model for miner verification, and their approach to sharded consensus groups. We take a technical look at their smart contract language Scilla and its roots in OCaml, and why formal verification is important. Topics: What Zilliqa is On smart contract language Different consensus mechanism What problems are they solving Practical Byzantine Fault Tolerance What their consensus model is Types of business or markets that benefit from their solution Links: Zilliqa - https://zilliqa.com/ For Developers - Zilliqa - https://zilliqa.com/for-developers.html Scilla Overview Paper - https://ilyasergey.net/papers/scilla-overview.pdf What is Practical Byzantine Fault Tolerance? - https://blockonomi.com/practical-byzantine-fault-tolerance/ Ecosystem Partners - Zilliqa - https://zilliqa.com/ecosystem-partners.html

In this week's episode, we sit down with Martin Lundfall (https://twitter.com/martinlundfall) from Dapphub & MakerDAO to discuss formal verification (https://en.wikipedia.org/wiki/Formal_verification) - a topic request that comes directly from the Zero Knowledge audience. We cover what formal verification is, where it makes sense, where it doesn't, the k-framework, how different this approach is to the "move fast and break things" approach, and much more. Here are some of articles and videos we mention: * Is a type a lifebuoy or a lamp? (https://skillsmatter.com/skillscasts/8893-is-a-type-a-lifebuoy-or-a-lamp) * https://github.com/dapphub/klab * https://github.com/kframework/evm-semantics * https://github.com/kframework/k * https://jellopaper.org/ * https://dapphub.chat/ Thanks to Web3Foundation for being our sponsor on this week's episode. For more about the Web3Foundation grants, check out their blog: https://medium.com/web3foundation/introducing-web3-foundation-grants-442f6753926a and grants.web3.foundation (https://github.com/w3f/Web3-collaboration/blob/master/grants/grants.md) If you like what we do: Follow us on Twitter - @zeroknowledgefm Join us on Telegram - https://t.me/joinchat/B_81tQ57-ThZg8yOSx5gjA Support our Gitcoin Grant - https://gitcoin.co/grants/38/zero-knowledge-podcast Support us on Patreon - https://www.patreon.com/zeroknowledge Or directly here: ETH: 0xC0FFEE1B5083230a5154F55f253B6b6ae8F29B1a BTC: 1cafekGa3podM4fBxPSQc6RCEXQNTK8Zz

Beschreibung: In dieser Folge unterhalten wir uns über Wireguard, ein neues VPN-Protokoll das von Jason A. Donenfeld designt und entwickelt wurde. Auch wenn wir versuchen objektiv über das VPN-Protokoll zu sprechen, können wir unsere Begeisterung schwer verbergen. Ein komplettes VPN-Protokoll in weniger als 4000 Zeilen Code und basierend auf moderner Kryptografie. Zusätzlich ist Wireguard so aufgebaut, dass keine dynamischen Speicherallozierung notwendig ist und auch kein Parsing. Shownotes: Homepage von Wireguard Whitepaper von Wireguard WP: Virtual Private Network Linus Torvalds über Wireguard auf der LKML Paper: A Cryptographic Evaluation of IPsec von Ferguson & Schneier Schneier on Security WP: Niels Ferguson Slides: On the Possibility of a Back Door in the NIST SP800-90 Dual EC PRNG New York Times provides new details about NSA backdoor in crypto spec Homepage von Jason A. Donenfeld (zx2c4) Pass: the standard unix password manager Liste von Exploits von zx2c4 The padata parallel execution mechanism Erste Performance-Tests von Wireguard Reddit: WireGuard benchmark between two servers with 10 Gb ethernet FriVPN: Multithreaded OpenVPN client (WIP) von Hunz Curve25519: A state-of-the-art Diffie-Hellman function BLAKE2: simpler, smaller, fast as MD5 The ChaCha family of stream ciphers The Poly1305-AES message-authentication code Formal Verification of the WireGuard Protocol von J. Donenfeld und K. Milner TAI64N Zeitstempel von Daniel J. Bernstein Security Analysis of WireGuard (MIT 6.857 Final Project) von Andrew He, Baula Xu und Jerry Wu Noise Protocol Framework Vortrag von Trevor Perrin über das Noise Protocol auf dem 34C3 WP: Diffie–Hellman key exchange Fun with NULL pointers, part 1 Mosh: the mobile shell Mullvad, Schwedischer VPN Provider der auch Wireguard Server betreibt Wireguard Demo Server

Rob and Jason are joined by Matt Fernandez from Intel Labs to discuss Formal Verification.   Matthew Fernandez is a Research Scientist with Intel Labs. Matt began his programming career building Windows GUI applications and designing databases, before moving into operating system architecture and security. He has a PhD in formal verification of operating systems from the University of New South Wales in Australia, and worked with the Australian research group Data61. In the past, he has worked on compilers, device drivers and hypervisors, and now spends his days exploring new tools and techniques for functional correctness and verification of security properties. On the weekends, you can usually find Matt in a park with a good book, hunting for good coffee or helping a newbie debug their code. He hopes to avoid saying “monad” on this podcast. News C++17 in Detail now available Cross-language interfaces between C and C++ Spaceship Operator Links The sel4 Microkernel Isabelle - Generic Proof Assistant The Coq Proof Assistant Dafny - Microsoft language and program verifier Z3 Theorem Prover Sponsors Backtrace Patreon CppCast Patreon Hosts @robwirving @lefticus  

With the rise of smart contract technology, we’ve become acutely aware of the need for smart contract code to accurately reflect the intentions of its author; and for the code to have certain (safe) behaviors in all circumstances. Creating the languages and software tools to enable ordinary developers to write safe contracts has become an intense research endeavor in the cryptocurrency space. Scilla is a Turing incomplete intermediate level language; inspired from the paradigms of functional programming and formal verification; that makes it easy for smart contract developers to automatically prove statements about smart contract behavior. For example, Scilla could allow a future multi-signature smart contract author to mathematically prove that funds in that contract would always be retrievable by certain addresses (and never get stuck like the Parity incident). The ability to mathematically prove such safety properties of the smart contract has the potential to be an enabling invention prior to widescale use of this technology. In this episode, we are joined by Dr. Amrit Kumar and Dr. Ilya Sergey to discuss Scilla, the smart contract language of the upcoming Zilliqa blockchain. In a previous episode, we’ve already covered the vision and technical approach of Zilliqa to solve the transaction scalability problem of permissionless blockchains. This episode focuses specifically on their smart contract language development efforts. Topics covered in this episode: Updated on Zilliqa’s progress since our last episode The technology of mechanised proofs Dr. Ilya Serger’s effort to mechanically prove safety properties of a blockchain consensus network Aims of the Scilla language Future capabilities enabled by the Scilla language Developer experience and perspective using formal verification tools How Scilla compares to Michelson, Tezos’ approach to smart contract languages with a similar end goal Current state of development of Scilla, and next milestones Episode links: Our previous episode on the Zilliqa platform Ilya Sergey's paper on mechanising blockchain consensus Scilla whitepaper Michelson, Tezos platform's smart contract language Zilliqa blog for updates on platform development Coq, a formal proof management system This episode is hosted by Meher Roy and Sunny Aggarwal. Show notes and listening options: epicenter.tv/238

Link to the website: https://codepodcast.com/posts/2018-03-12-episode-7-300m-worth-of-bugs/ Imagine – your company's code and data are exposed. How long will it take for malicious hackers to find vulnerabilities? To steal users' personal information? For developers that build on Ethereum that situation is not a distant possibility, it's an everyday reality. All the code, the state and the calls to their programs are publicly accessible and live forever on the blockchain. Add to it the fact that their code will manipulate money. Getting rid of *all* the bugs and holes becomes crucial. In this episode we'll talk about software that finds bugs in other software. Specifically ways of verifying Ethereum smart contracts. The story begins in the summer of 2017 when someone is able to steal $30M worth of ether. --- Episode was produced by [Andrey Salomatin](https://flpvsk.com). ## Support the podcast If you get value from the podcast, please consider supporting us on https://codepodcast.com/patreon Alternatively, you can also send us eth to this address: 0x730075d42c3BC0EA38c23A6D0D9611E9d78C5Af0 ## Guests * [Santiago Palladino](https://twitter.com/smpalladino) * [Matt Condon](https://twitter.com/mattgcondon) * [Yoichi Hirai](https://twitter.com/pirapira) ### Links * [Ethereum](https://ethereum.org/) * [Ethereum Development Tutorial](https://github.com/ethereum/wiki/wiki/Ethereum-Development-Tutorial) * [Parity](https://www.parity.io/) * EVM-compatible languages * [Solidity](https://github.com/ethereum/solidity) * [Serpent](https://github.com/ethereum/serpent) * [Vyper](https://github.com/ethereum/vyper) * [Bamboo](https://github.com/pirapira/bamboo) * Wiki: ["Abstract interpretation"](https://en.wikipedia.org/wiki/Abstract_interpretation) * Symbolic execution * Article ["Introducing Mythril: A framework for bug hunting on the Ethereum blockchain"](https://hackernoon.com/introducing-mythril-a-framework-for-bug-hunting-on-the-ethereum-blockchain-9dc5588f82f6) * [Manticore](https://github.com/trailofbits/manticore) * Wiki: ["Formal Verification"](https://en.wikipedia.org/wiki/Formal_verification) * [The Hydra Project](https://thehydra.io/) ### Links: Santiago * [OpenZeppelin website](https://openzeppelin.org/) * [OpenZeppelin Slack](https://slack.openzeppelin.org/) * [ZepellinOS](https://zeppelinos.org/) * Article ["The Parity Wallet Hack Explained"](https://blog.zeppelin.solutions/on-the-parity-wallet-multisig-hack-405a8c12e8f7) ### Links: Matt * [XLNT website](https://xlnt.co/) * Article ["Getting Up to Speed on Ethereum"](https://medium.com/@mattcondon/getting-up-to-speed-on-ethereum-63ed28821bbe) * Article ["Announcing the Steak Network"](https://medium.com/truebit/announcing-the-steak-network-c3d44290d53d) ### Links: Yoichi * Gist ["Formal Verification of Ethereum Contracts"](https://github.com/pirapira/ethereum-formal-verification-overview) * [Bamboo](https://github.com/pirapira/bamboo) * [A Lem formalization of EVM and some Isabelle/HOL proofs](https://github.com/pirapira/eth-isabelle) * Video ["Formal verification of EVM bytecodes"](https://www.youtube.com/watch?v=Mzh4fyoaBJ0) * Video ["Formal Verification of Smart Contracts"](https://www.youtube.com/watch?v=cCUGMAnCh7o) ### Music [Mid-Air!](https://soundcloud.com/mid_air)

Sebastian Ullrich (KIT, Germany), gives the fourth talk in the second panel, Dependently Typed Programming, on the 3rd day of the ICFP conference. Co-written by Gabriel Exner (Vienna University of Technology, Austria), Jared Roesch (University of Washington, USA), Jeremy Avigad (Carnegie Mellon University, USA), Leonardo De Moura (Microsoft Research). Dependent type theory is a powerful framework for interactive theorem proving and automated reasoning, allowing us to encode mathematical objects, data type specifications, assertions, proofs, and programs, all in the same language. Here we show that dependent type theory can also serve as its own metaprogramming language, that is, a language in which one can write programs that assist in the construction and manipulation of terms in dependent type theory itself. Specifically, we describe the metaprogramming language currently in use in the Lean theorem prover, which extends Lean's object language with an API for accessing natively implemented procedures and provides ways of reflecting object-level expressions into the metalanguage. We provide evidence to show that our language is performant, and that it provides a convenient and flexible way of writing not only small-scale interactive tactics, but also more substantial kinds of automation.

In this podcast Charles Humble talks to Sylvan Clebsch, who is the designer of the actor-model language Pony programming and now works at Microsoft Research in Cambridge in the Programming Language Principles group. They talk about the inspirations behind Pony, how the garbage collector avoids stop-the-world pauses, the queuing systems, work scheduler, and formal verification. Why listen to this podcast: * Pony scales from a Raspberry Pi through a 64 core half terabyte machine to a 4096 core SGI beast * An actor has a 256-byte overhead, so creating hundreds of thousands of actors is possible * Actors have unbounded queues to prevent deadlock * Each actor garbage collects its own heap, so global stop-the-world pauses are not needed * Because the type system is data-race free, it’s impossible to have concurrency problems in Pony More on this: Quick scan our curated show notes on InfoQ http://bit.ly/2tZXcKE You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Want to see extented shownotes? Check the landing page on InfoQ: http://bit.ly/2tZXcKE

While designing computer systems and their underlying protocols, architects impose functionality, security, and privacy requirements or policies with which the designed systems and protocols should comply with. These requirements and policies are generally written in natural language and more often than not they are not complied with in the implementations due to ambiguity, misinterpretation of the requirements, or developer errors. Non-compliance with the requirements can not only have security, privacy, and utility consequences but also can have safety implications. One possible solution is to express the requirements in some formal language. In addition to eliminating ambiguities and misinterpretations of the requirements, this also enables application of formal verification techniques to check for compliance of the implementation against the desired requirements or the policies. Formal verification techniques can be applied for checking compliance in potentially three different settings. In the first setting,compliance checking is performed statically before a system or a protocol is deployed. In the second setting, a runtime monitor can be deployed alongside the system or the protocol, and the monitor provably disallows the system or the protocol to take non-compliant actions. Finally, compliance can be be checked in a post-hoc fashion by capturing all the relevant runtime events in an audit log which can then be scrutinized for non-compliance. In this talk, I will present demonstrative examples of using formal verification techniques for compliance checking in each of these settings. About the speaker: Omar Chowdhury is a Post-Doctoral Research Associate at the Department of Computer Science at Purdue University. Before joining Purdue, he was a Post-doctoral Research Associate at Cylab, Carnegie Mellon University. He received his Ph.D. in Computer Science from the University of Texas at San Antonio. His research interest broadly lies in the field of Computer Security and Privacy. He is specifically interested in applying formal verification techniques for developing efficient compliance checking mechanism for computer information systems with respect to applicable privacy regulations like HIPAA and GLBA. He has won the best paper award The ACM Symposium on Access Control Models and Technologies (SACMAT). He has also served as a program committee member in ACM SACMAT and ACM CCS.

I shamelessly plug my new Formal Verification book. (Send feeback to erik@mathmutation.com)