Podcasts about xhr

8.30 bats around some final ‘24 MLB regular season topics (0:38), peeks at the TGFBI standings (5:14), (6:56), starts modeling xHR - expected HR (11:42), and reviews Justin Martinez (24:45).

Luca Casonato is the tech lead for Deno Deploy and a TC39 delegate. Deno is a JavaScript runtime from the original creator of NodeJS, Ryan Dahl. Topics covered: What's a JavaScript runtime How V8 is used Why Deno was created The W3C WinterCG for server-side JavaScript Why it's difficult to ship new features in Node The benefits of web standards Creating an all-inclusive toolset like Rust and Go Deno's node compatibility layer Use cases for WebAssembly Benefits and implementation of Deno Deploy Reasons to deploy on the edge What's coming next Luca Luca Casonato @lcasdev Deno Homepage Deploy Showcase Subhosting Fresh web framework The anatomy of an Isolate Cloud Deno Users Netlify Edge Functions Deno at Slack GitHub Flat Data Shopify Oxygen Other related links Cache Web API V8 (JavaScript and WebAssembly engine) TC39 (JavaScript specification group) Web-interoperable Runtimes Community Group (WinterCG) Cloudflare Workers (Deno Deploy competitor) How Cloudflare KV works CockroachDB (Distributed database) XKCD Standards Comic Transcript You can help edit this transcript on GitHub. [00:00:07] Jeremy: Today I'm talking to Luca Casonato. He's a member of the Deno Core team and a TC 39 Delegate. [00:00:06] Luca: Hey, thanks for having me. What's a runtime? [00:00:07] Jeremy: So today we're gonna talk about Deno, and on the website it says, Deno is a runtime for JavaScript and TypeScript. So I thought we could start with defining what a runtime is. [00:00:21] Luca: Yeah, that's a great question. I think this question actually comes up a lot. It's, it's like sometimes we also define Deno as a headless browser, or I don't know, a, a JavaScript script execution tool. what actually defines runtime? I, I think what makes a runtime a runtime is that it is a, it's implemented in native code. It cannot be self-hosted. Like you cannot self-host a JavaScript runtime. and it executes JavaScript or TypeScript or some other scripting language, without relying on, well, yeah, I guess it's the self-hosting thing. Like it's, it's essentially a, a JavaScript execution engine, which is not self-hosted. So yeah, it, it maybe has IO bindings, but it doesn't necessarily need to like, it. Maybe it allows you to read the, from the file system or, or make network calls. Um, but it doesn't necessarily have to. It's, I think the, the primary definition is something which can execute JavaScript without already being written in JavaScript. How V8 and JavaScript runtimes are related [00:01:20] Jeremy: And when we hear about JavaScript run times, whether it's Deno or Node or Bun, or anything else, we also hear about it in the context of v8. Could you explain the relationship between V8 and a JavaScript run time? [00:01:36] Luca: Yeah. So V8 and, and JavaScript core and Spider Monkey, these are all JavaScript engines. So these are the low level virtual machines that can execute or that can parse your JavaScript code. turn it into byte code, maybe turn it into, compiled machine code, and then execute that code. But these engines, Do not implement any IO functions. They do not. They implement the JavaScript spec as is written. and then they provide extension hooks for, they call these host environments, um, like environments that embed these engines to provide custom functionalities to essentially poke out of the sandbox, out of the, out of the virtual machine. Um, and this is used in browsers. Like browsers have, have these engines built in. This is where they originated from. Um, and then they poke holes into this, um, sandbox virtual machine to do things like, I don't know, writing to the dom or, or console logging or making fetch calls and all these kinds of things. And what a runtime essentially does, a JavaScript runtime is it takes one of these engines and. It then provides its own set of host APIs, like essentially its own set of holes. It pokes into the sandbox. and depending on what the runtime is trying to do, um, the weight will do. This is gonna be different and, and the sort of API that is ultimately exposed to the end user is going to be different. For example, if you compare Deno and node, like node is very loosey goosey, about how it pokes holds into the sandbox, it sort of just pokes them everywhere. And this makes it difficult to enforce things like, runtime permissions for example. Whereas Deno is much more strict about how it, um, pokes holds into its sandbox. Like everything is either a web API or it's behind in this Deno name space, which means that it's, it's really easy to find, um, places where, where you're poking out of the sandbox. and really you can also compare these to browsers. Like browsers are also JavaScript run times. Um, they're just not headless. JavaScript run times, but JavaScript run times that also have a ui. and. . Yeah. Like there, there's, there's a whole Bunch of different kinds of JavaScript run times, and I think we're also seeing a lot more like embedded JavaScript run times. Like for example, if you've used React Native before, you, you may be using Hermes as a, um, JavaScript engine in your Android app, which is like a custom JavaScript engine written just for, for, for React native. Um, and this also is embedded within a, like react native run time, which is specific to React native. so it's also possible to have run times, for example, that are, that can be where the, where the back backing engine can be exchanged, which is kind of cool. [00:04:08] Jeremy: So it sounds like V8's role, one way to look at it is it can execute JavaScript code, but only pure functions. I suppose you [00:04:19] Luca: Pretty much. Yep. [00:04:21] Jeremy: Do anything that doesn't interact with IO so you think about browsers, you were mentioning you need to interact with a DOM or if you're writing a server side application, you probably need to receive or make HTTP requests, that sort of thing. And all of that is not handled by v8. That has to be handled by an external runtime. [00:04:43] Luca: Exactly Like, like one, one. There's, there's like some exceptions to this. For example, JavaScript technically has some IO built in with, within its standard library, like math, random. It's like random number. Generation is technically an IO operation, so, Technically V8 has some IO built in, right? And like getting the current date from the user, that's also technically IO So, like there, there's some very limited edge cases. It's, it's not that it's purely pure, but V8 for example, has a flag to turn it completely deterministic. which means that it really is completely pure. And this is not something which run times usually have. This is something like the feature of an engine because the engine is like so low level that it can essentially, there's so little IO that it's very easy to make deterministic where a runtime higher level, um, has, has io, um, much more difficult to make deterministic. [00:05:39] Jeremy: And, and for things like when you're working with JavaScript, there's, uh, asynchronous programming [00:05:46] Luca: mm-hmm. Concurrent JavaScript execution [00:05:47] Jeremy: So you have concurrency and things like that. Is that a part of V8 or is that the responsibility of the run time? [00:05:54] Luca: That's a great question. So there's multiple parts to this. There's the part, um, there, there's JavaScript promises, um, and sort of concurrent Java or well, yes, concurrent JavaScript execution, which is sort of handled by v8, like v8. You can in, in pure v8, you can create a promise, and you can execute some code within that promise. But without IO there's actually no way to defer time, uh, which means that in with pure v8, you can either, you can create a promise. Which executes right now. Or you can create a promise that never executes, but you can't create a promise that executes in 10 seconds because there's no way to measure 10 seconds asynchronously. What run times do is they add something called an event loop on top of this, um, on top of the base engine and that event loop, for example, like a very simple event loop, for example, might have a timer in it, which every second looks at if there's a timer schedule to run within that second. And if it does, if, if that timer exists, it'll go call out to V8 and say, you can now execute that promise. but V8 is still the one that's keeping track of, of like which promises exist, and the code that is meant to be invoked when they resolve all that kind of thing. Um, but the underlying infrastructure that actually invokes which promises get resolved at what point in time, like the asynchronous, asynchronous IO is what this is called. This is driven by the event loop, um, which is implemented by around time. So Deno, for example, it uses, Tokio for its event loop. This is a, um, an event loop written in Rust. it's very popular in the Rust ecosystem. Um, node uses libuv. This is a relatively popular runtime or, or event loop, um, implementation for c uh, plus plus. And, uh, libuv was written for Node. Tokio was not written for Deno. But um, yeah, Chrome has its own event loop implementation. Bun has its own event loop implementation. [00:07:50] Jeremy: So we, we might go a little bit more into that later, but I think what we should probably go into now is why make Deno, because you have Node that's, uh, currently very popular. The co-creator of Deno, to my understanding, actually created Node. So maybe you could explain to our audience what was missing or what was wrong with Node, where they decided I need to create, a new runtime. Why create a new runtime? (standards compliance) [00:08:20] Luca: Yeah. So the, the primary point of concern here was that node was slowly diverging from browser standards with no real path to, to, to, re converging. Um, like there was nothing that was pushing node in the direction of standards compliance and there was nothing, that was like sort of forcing node to innovate. and we really saw this because in the time between, I don't know, 2015, 2018, like Node was slowly working on esm while browsers had already shipped ESM for like three years. , um, node did not have fetch. Node hasn't had, or node only at, got fetch last year. Right? six, seven years after browsers got fetch. Node's stream implementation is still very divergent from, from standard web streams. Node was very reliant on callbacks. It still is, um, like promises in many places of the Node API are, are an afterthought, which makes sense because Node was created in a time before promises existed. Um, but there was really nothing that was pushing Node forward, right? Like nobody was actively investing in, in, in improving the API of Node to be more standards compliant. And so what we really needed was a new like Greenfield project, which could demonstrate that actually writing a new server side run. Is A viable, and b is totally doable with an API that is more standards combined. Like essentially you can write a browser, like a headless browser and have that be an excellent to use JavaScript runtime, right? And then there was some things that were I on top of that, like a TypeScript support because TypeScript was incredibly, or is still incredibly popular. even more so than it was four years ago when, when Deno was created or envisioned, um, this permission system like Node really poked holes into the V8 sandbox very early on with, with like, it's gonna be very difficult for Node to ever, ever, uh, reconcile this, this. Especially cuz the, some, some of the APIs that it, that it exposes are just so incredibly low level that like, I don't know, you can mutate random memory within your process. Um, which like if you want to have a, a secure sandbox like that just doesn't work. Um, it's not compatible. So there was really needed to be a place where you could explore this, um, direction and, and see if it worked. And Deno was that. Deno still is that, and I think Deno has outgrown that now into something which is much more usable as, as like a production ready runtime. And many people do use it, in production. And now Deno is on the path of slowly converging back with Node, um, in from both directions. Like Node is slowly becoming more standards compliant. and depending on who you ask this was, this was done because of Deno and some people said it would had already been going on and Deno just accelerated it. but that's not really relevant because the point is that like Node is becoming more standard compliant and, and the other direction is Deno is becoming more node compliant. Like Deno is implementing node compatibility layers that allow you to run code that was originally written for the node ecosystem in the standards compliant run time. so through those two directions, the, the run times are sort of, um, going back towards each other. I don't think they'll ever merge. but we're, we're, we're getting to a point here pretty soon, I think, where it doesn't really matter what runtime you write for, um, because you'll be able to write code written for one runtime in the other runtime relatively easily. [00:12:03] Jeremy: If you're saying the two are becoming closer to one another, becoming closer to the web standard that runs in the browser, if you're talking to someone who's currently developing in node, what's the incentive for them to switch to Deno versus using Node and then hope that eventually they'll kind of meet in the middle. [00:12:26] Luca: Yeah, so I think, like Deno is a lot more than just a runtime, right? Like a runtime executes JavaScript, Deno executes JavaScript, it executes type script. But Deno is so much more than that. Like Deno has a built-in format, or it has a built-in linter. It has a built-in testing framework, a built-in benching framework. It has a built-in Bundler, it, it like can create self-hosted, um, executables. yeah, like Bundle your code and the Deno executable into a single executable that you can trip off to someone. Um, it has a dependency analyzer. It has editor integrations. it has, Yeah. Like I could go on for hours, (laughs) about all of the auxiliary tooling that's inside of Deno, that's not a JavaScript runtime. And also Deno as a JavaScript runtime is just more standards compliant than any of the other servers at Runtimes right now. So if, if you're really looking for something which is standards complaint, which is gonna like live on forever, then it's, you know, like you cannot kill off the Fetch API ever. The Fetch API is going to live forever because Chrome supports it. Um, and the same goes for local storage and, and like, I don't know, the Blob API and all these other web APIs like they, they have shipped and browsers, which means that they will be supported until the end of time. and yeah, maybe Node has also reached that with its api probably to some extent. but yeah, don't underestimate the power of like 3 billion Chrome users. that would scream immediately if the Fetch API stopped working Right? [00:13:50] Jeremy: Yeah, I, I think maybe what it sounds like also is that because you're using the API that's used in the browser places where you deploy JavaScript applications in the future, you would hope that those would all settle on using that same API so that if you were using Deno, you could host it at different places and not worry about, do I need to use a special API maybe that you would in node? WinterCG (W3C group for server side JavaScript) [00:14:21] Luca: Yeah, exactly. And this is actually something which we're specifically working towards. So, I don't know if you've, you've heard of WinterCG? It's a, it's a community group at the W3C that, um, CloudFlare and, and Deno and some others including Shopify, have started last year. Um, we're essentially, we're trying to standardize the concept of what a server side JavaScript runtime is and what APIs it needs to have available to be standards compliant. Um, and essentially making this portability sort of written down somewhere and like write down exactly what code you can write and expect to be portable. And we can see like that all of the big, all of the big players that are involved in, in, um, building JavaScript run times right now are, are actively, engaged with us at WinterCG and are actively building towards this future. So I would expect that any code that you write today, which runs. in Deno, runs in CloudFlare, workers runs on Netlify Edge functions, runs on Vercel's Edge, runtime, runs on Shopify Oxygen, is going to run on the other four. Um, of, of those within the next couple years here, like I think the APIs of these is gonna converge to be essentially the same. there's obviously gonna always be some, some nuances. Um, like, I don't know, Chrome and Firefox and Safari don't perfectly have the same API everywhere, right? Like Chrome has some web Bluetooth capabilities that Safari doesn't, or Firefox has some, I don't know, non-standard extensions to the error object, which none of the other runtimes do. But overall you can expect these front times to mostly be aligned. yeah, and I, I think that's, that's really, really, really excellent and that, that's I think really one of the reasons why one should really consider, like building for, for this standard runtime because it, it just guarantees that you'll be able to host this somewhere in five years time and 10 years time, with, with very little effort. Like even if Deno goes under or CloudFlare goes under, or, I don't know, nobody decides to maintain node anymore. It'll be easy to, to run somewhere else. And also I expect that the big cloud vendors will ultimately, um, provide, manage offerings for, for the standards compliant JavaScript on time as well. Is Node part of WinterCG? [00:16:36] Jeremy: And this WinterCG group is Node a part of that as well? [00:16:41] Luca: Um, yes, we've invited Node, um, to join, um, due to the complexities of how node's, internal decision making system works. Node is not officially a member of WinterCG. Um, there is some individual members of the node, um, technical steering committee, which are participating. for example, um, James m Snell is, is the co-chair, is my co-chair on, on WinterCG. He also works at CloudFlare. He's also a node, um, TSC member, Mateo Colina, who has been, um, instrumental to getting fetch landed in Node, um, is also actively involved. So Node is involved, but because Node is node and and node's decision making process works the way it does, node is not officially listed anywhere as as a member. but yeah, they're involved and maybe they'll be a member at some point. But, yeah, let's. , see (laughs) [00:17:34] Jeremy: Yeah. And, and it, so it, it sounds like you're thinking that's more of a, a governance or a organizational aspect of note than it is a, a technical limitation. Is that right? [00:17:47] Luca: Yeah. I obviously can't speak for the node technical steering committee, but I know that there's a significant chunk of the node technical steering committee that is, very favorable towards, uh, standards compliance. but parts of the Node technical steering committee are also not, they are either indifferent or are actively, I dunno if they're still actively working against this, but have actively worked against standards compliance in the past. And because the node governance structure is very, yeah, is, is so, so open and let's, um, and let's, let's all these voices be heard, um, that just means that decision making processes within Node can take so long, like. . This is also why the fetch API took eight years to ship. Like this was not a technical problem. and it is also not a technical problem. That Node does not have URL pattern support or, the file global or, um, that the web crypto API was not on this, on the global object until like late last year, right? Like, these are not technical problems, these are decision making problems. Um, and yeah, that was also part of the reason why we started Deno as, as like a separate thing, because like you can try to innovate node, from the inside, but innovating node from the inside is very slow, very tedious, and requires a lot of fighting. And sometimes just showing somebody, from the outside like, look, this is the bright future you could have, makes them more inclined to do something. Why it takes so long to ship new features in Node [00:19:17] Jeremy: Do, do you have a sense for, you gave the example of fetch taking eight years to, to get into node. Do you, do you have a sense of what the typical objection is to, to something like that? Like I, I understand there's a lot of people involved, but why would somebody say, I, I don't want this [00:19:35] Luca: Yeah. So for, for fetch specifically, there was a, there was many different kinds of concerns. Um, one of the, I, I can maybe list two of them. One of them was for example, that the fetch API is not a good API and as such, node should not have it. which is sort of. missing the point of, because it's a standard API, how good or bad the API is is much less relevant because if you can share the API, you can also share a wrapper that's written around the api. Right? and then the other concern was, node does need fetch because Node already has an HTTP API. Um, so, so these are both kind of examples of, of concerns that people had for a long time, which it took a long time to either convince these people or, or to, push the change through anyway. and this is also the case for, for other things like, for example, web, crypto, um, like why do we need web crypto? We already have node crypto, or why do we need yet another streams? Implementation node already has four different streams implementations. Like, why do we need web streams? and the, the. Like, I don't know if you know this XKCD of, there's 14 competing standards. so let's write a 15th standard, to unify them all. And then at the end we just have 15 competing standards. Um, so I think this is also the kind of concern that people were concerned about, but I, I think what we've seen here is that this is really not a concern that one needs to have because it ends up that, or it turns out in the end that if you implement web APIs, people will use web APIs and will use web APIs only for their new code. it takes a while, but we're seeing this with ESM versus require like new code written with require much less common than it was two years ago. And, new code now using like Xhr, whatever it's called, form request or. You know, the one, I mean, compared to using Fetch, like nobody uses that name. Everybody uses Fetch. Um, and like in Node, if you write a little script, like you're gonna use Fetch, you're not gonna use like Nodes, htp, dot get API or whatever. and we're gonna see the same thing with Readable Stream. We're gonna see the same thing with Web Crypto. We're gonna see, see the same thing with Blob. I think one of the big ones where, where Node is still, I, I, I don't think this is one that's ever gonna get solved, is the, the Buffer global and Node. like we have the Uint8, this Uint8 global, um, and like all the run times including browsers, um, and Buffer is like a super set of that, but it's in global scope. So it, it's sort of this non-standard extension of unit eight array that people in node like to use and it's not compatible with anything else. Um, but because it's so easy to get at, people use it anyway. So those are, those are also kind of problems that, that we'll have to deal with eventually. And maybe that means that at some point the buffer global gets deprecated and I don't know, probably can never get removed. But, um, yeah, these are kinds of conversations that the no TSE is going have to have internally in, I don't know, maybe five years. Write once, have it run on any hosting platform [00:22:37] Jeremy: Yeah, so at a high level, What's shipped in the browser, it went through the ECMAScript approval process. People got it into the browser. Once it's in the browser, probably never going away. And because of that, it's safe to build on top of that for these, these server run times because it's never going away from the browser. And so everybody can kind of use it into the future and not worry about it. Yeah. [00:23:05] Luca: Exactly. Yeah. And that's, and that's excluding the benefit that also if you have code that you can write once and use in both the browser and the server side around time, like that's really nice. Um, like that, that's the other benefit. [00:23:18] Jeremy: Yeah. I think that's really powerful. And that right now, when someone's looking at running something in CloudFlare workers versus running something in the browser versus running something in. it's, I think a lot of people make the assumption it's just JavaScript, so I can use it as is. But it, it, there are at least currently, differences in what APIs are available to you. [00:23:43] Luca: Yep. Yep. Why bundle so many things into Deno? [00:23:46] Jeremy: Earlier you were talking about how Deno is more than just the runtime. It has a linter, formatter, file watcher there, there's all sorts of stuff in there. And I wonder if you could talk a little bit to the, the reasoning behind that [00:24:00] Luca: Mm-hmm. [00:24:01] Jeremy: Having them all be separate things. [00:24:04] Luca: Yeah, so the, the reasoning here is essentially if you look at other modern run time or mo other modern languages, like Rust is a great example. Go is a great example. Even though Go was designed around the same time as Node, it has a lot of these same tools built in. And what it really shows is that if the ecosystem converges, like is essentially forced to converge on a single set of built-in tooling, a that built-in tooling becomes really, really excellent because everybody's using it. And also, it means that if you open any project written by any go developer, any, any rest developer, and you look at the tests, you immediately understand how the test framework works and you immediately understand how the assertions work. Um, and you immediately understand how the build system works and you immediately understand how the dependency imports work. And you immediately understand like, I wanna run this project and I wanna restart it when my file changes. Like, you immediately know how to do that because it's the same everywhere. Um, and this kind of feeling of having to learn one tool and then being able to use all of the projects, like being able to con contribute to open source when you're moving jobs, whatever, like between personal projects that you haven't touched in two years, you know, like being able to learn this once and then use it everywhere is such an incredibly powerful tool. Like, people don't appreciate this until they've used a runtime or, or, or language which provides this to them. Like, you can go to any go developer and ask them if they would like. There, there's this, there's this saying in the Go ecosystem, um, that Go FMT is nobody's favorite, but, or, uh, wait, no, I don't remember what the, how the saying goes, but the saying essentially implies that the way that go FMT formats code, maybe not everybody likes, but everybody loves go F M T anyway, because it just makes everything look the same. And like, you can read your friend's code, your, your colleagues code, your new jobs code, the same way that you did your code from two years ago. And that's such an incredibly powerful feeling. especially if it's like well integrated into your IDE you clone a repository, open that repository, and like your testing panel on the left hand side just populates with all the tests, and you can click on them and run them. And if an assertion fails, it's like the standard output format that you're already familiar with. And it's, it's, it's a really great feeling. and if you don't believe me, just go try it out and, and then you will believe me, (laughs) [00:26:25] Jeremy: Yeah. No, I, I'm totally with you. I, I think it's interesting because with JavaScript in particular, it feels like the default in the community is the opposite, right? There's so many different ways. Uh, there are so many different build tools and testing frameworks and, formatters, and it's very different than, like you were mentioning, a go or a Rust that are more recent languages where they just include that, all Bundled in. Yeah. [00:26:57] Luca: Yeah, and I, I think you can see this as well in, in the time that average JavaScript developer spends configuring their tooling compared to a rest developer. Like if I write Rust, I write Rust, like all day, every day. and I spend maybe two, 3% of my time configuring Rust tooling like. Doing dependency imports, opening a new project, creating a format or config file, I don't know, deleting the build directory, stuff like that. Like that's, that's essentially what it means for me to configure my rest tooling. Whereas if you compare this to like a front-end JavaScript project, like you have to deal with making sure that your React version is compatible with your React on version, it's compatible with your next version is compatible with your ve version is compatible with your whatever version, right? this, this is all not automatic. Making sure that you use the right, like as, as a front end developer, you developer. You don't have just NPM installed, no. You have NPM installed, you have yarn installed, you have PNPM installed. You probably have like, Bun installed. And, and, and I don't know to use any of these, you need to have corepack enabled in Node and like you need to have all of their global bin directories symlinked into your or, or, or, uh, included in your path. And then if you install something and you wanna update it, you don't know, did I install it with yarn? Did I install it with N pNPM? Like this is, uh, significant complexity and you, you tend to spend a lot of time dealing with dependencies and dealing with package management and dealing with like tooling configuration, setting up esent, setting up prettier. and I, I think that like, especially Prettier, for example, really showed, was, was one of the first things in the JavaScript ecosystem, which was like, no, we're not gonna give you a config where you, that you can spend like six hours configuring, it's gonna be like seven options and here you go. And everybody used it because, Nobody likes configuring things. It turns out, um, and even though there's always the people that say, oh, well, I won't use your tool unless, like, we, we get this all the time. Like, I'm not gonna use Deno FMT because I can't, I don't know, remove the semicolons or, or use single quotes or change my tab width to 16. Right? Like, wait until all of your coworkers are gonna scream at you because you set the tab width to 16 and then see what they change it to. And then you'll see that it's actually the exact default that, everybody uses. So it'll, it'll take a couple more years. But I think we're also gonna get there, uh, like Node is starting to implement a, a test runner. and I, I think over time we're also gonna converge on, on, on, on like some standard build tools. Like I think ve, for example, is a great example of this, like, Doing a front end project nowadays. Um, like building new front end tooling that's not built on Vite Yeah. Don't like, Vite's it's become the standard and I think we're gonna see that in a lot more places. We should settle on what tools to use [00:29:52] Jeremy: Yeah, though I, I think it's, it's tricky, right? Because you have so many people with their existing projects. You have people who are starting new projects and they're just searching the internet for what they should use. So you're, you're gonna have people on web pack, you're gonna have people on Vite, I guess now there's gonna be Turbo pack, I think is another one that's [00:30:15] Luca: Mm-hmm. [00:30:16] Jeremy: There's, there's, there's all these different choices, right? And I, I think it's, it's hard to, to really settle on one, I guess, [00:30:26] Luca: Yeah, [00:30:27] Jeremy: uh, yeah. [00:30:27] Luca: like I, I, I think this is, this is in my personal opinion also failure of the Node Technical Steering committee, for the longest time to not decide that yes, we're going to bless this as the standard format for Node, and this is the standard package manager for Node. And they did, they sort of did, like, they, for example, node Blessed NPM as the standard, package manager for N for for node. But it didn't innovate on npm. Like no, the tech nodes, tech technical steering committee did not force NPM to innovate NPMs, a private company ultimately bought by GitHub and they had full control over how the NPM cli, um, evolved and nobody forced NPM to, to make sure that package install times are six times faster than they were. Three years ago, like nobody did that. so it didn't happen. And I think this is, this is really a failure of, of the, the, the, yeah, the no technical steering committee and also the wider JavaScript ecosystem of not being persistent enough with, with like focus on performance, focus on user experience, and, and focus on simplicity. Like things got so out of hand and I'm happy we're going in the right direction now, but, yeah, it was terrible for some time. (laughs) Node compatibility layer [00:31:41] Jeremy: I wanna talk a little bit about how we've been talking about Deno in the context of you just using Deno using its own standard library, but just recently last year you added a compatibility shim where people are able to use node libraries in Deno. [00:32:01] Luca: Mm-hmm. [00:32:01] Jeremy: And I wonder if you could talk to, like earlier you had mentioned that Deno has, a different permissions model. on the website it mentions that Deno's HTTP server is two times faster than node in a Hello World example. And I'm wondering what kind of benefits people will still get from Deno if they choose to use packages from Node. [00:32:27] Luca: Yeah, it's a great question. Um, so I think a, again, this is sort of a like, so just to clarify what we actually implemented, like what we have is we have support for you to import NPM packages. Um, so you can import any NPM package from NPM, from your type script or JavaScript ECMAScript module, um, that you have, you already have for your Deno code. Um, and we will under the hood, make sure that is installed somewhere in some directory globally. Like PNPM does. There's no local node modules folder you have to deal with. There's no package of Jason you have to deal with. Um, and there's no, uh, package. Jason, like versioning things you need to deal with. Like what you do is you do import cowsay from NPM colon cowsay at one, and that will import cowsay with like the semver tag one. Um, and it'll like do the sim resolution the same way node does, or the same way NPM does rather. And what you get from that is that essentially it gives you like this backdoor to a callout to all of the existing node code that Isri been written, right? Like you cannot expect that Deno developers, write like, I don't know. There was this time when Deno did not really have that many, third party modules yet. It was very early on, and I don't know the, you either, if you wanted to connect to Postgres and there was no Postgres driver available, then the solution was to write your own Postgres driver. And that is obviously not great. Um, (laughs) . So the better solution here is to let users for these packages where there's no Deno native or, or, or web native or standard native, um, package for this yet that is importable with url. Um, specifiers, you can import this from npm. Uh, so it's sort of this like backdoor into the existing NPM ecosystem. And we explicitly, for example, don't allow you to, create a package.json file or, import bare node specifiers because we don't, we, we want to stay standards compliant here. Um, but to make this work effectively, we need to give you this little back door. Um, and inside of this back door. All hell is like, or like everything is terrible inside there, right? Like inside there you can do bare specifiers and inside there you can like, uh, there's package.json and there's crazy node resolution and underscore underscore DIRNAME and common js. And like all of that stuff is supported inside of this backdoor to make all the NPM packages work. But on the outside it's exposed as this nice, ESM only, NPM specifiers. and the, the reason you would want to use this over, like just using node directly is because again, like you wanna use TypeScript, no config, like necessary. You want to use, you wanna have a formatter you wanna have a linter, you wanna have tooling that like does testing and benchmarking and compiling or whatever. All of that's built in. You wanna run this on the edge, like close to your users and like 30 different, 35 different, uh, points of presence. Um, it's like, Okay, push it to your git repository. Go to this website, click a button two times, and it's running in 35 data centers. like this is, this is the kind of ex like developer experience that you can, you do not get. You, I will argue that you cannot get with Node right now. Like even if you're using something like ts-node, it is not possible to get the same level of developer experience that you do with Deno. And the, the, the same like speed at which you can iterate, iterate on your projects, like create new projects, iterate on them is like incredibly fast in Deno. Like, I can open a, a, a folder on my computer, create a single file, may not ts, put some code in there and then call Deno Run may not. And that's it. Like I don't, I did not need to do NPM install I did not need to do NPM init -y and remove the license and version fields and from, from the generated package.json and like set private to true and whatever else, right? It just all works out of the box. And I think that's, that's what a lot of people come to deno for and, and then ultimately stay for. And also, yeah, standards compliance. So, um, things you build in Deno now are gonna work in five, 10 years, with no hassle. Node shims and testing [00:36:39] Jeremy: And so with this compatibility layer or this, this shim, is it where the node code is calling out to node APIs and you're replacing those with Deno compatible equivalents? [00:36:54] Luca: Yeah, exactly. Like for example, we have a shim in place that shims out the node crypto API on top of the web crypto api. Like sort of, some, some people may be familiar with this in the form of, um, Browserify shims. if anybody still remembers those, it's essentially. , your front end tooling, you were able to import from like node crypto in your front end projects and then behind the scenes your web packs or your browser replies or whatever would take that import from node crypto and would replace it with like the shim that was essentially exposed the same APIs node crypto, but under the hood, wasn't implemented with native calls, but was implemented on top of web crypto, or implemented in user land even. And Deno does something similar. there's a couple edge cases of APIs that there's, where, where we do not expose the underlying thing that we shim to, to end users, outside of the node shim. So like there's some, some APIs that I don't know if I have a good example, like node nextTick for example. Um, like to properly be able to shim node nextTick, you need to like implement this within the event loop in the runtime. and. , you don't need this in Deno, because Deno, you use the web standard queueMicrotask to, to do this kind of thing. but to be able to shim it correctly and run node applications correctly, we need to have this sort of like backdoor into some ugly APIs, um, which, which natively integrate in the runtime, but, yeah, like allow, allow this node code to run. [00:38:21] Jeremy: A, anytime you're replacing a component with a, a shim, I think there's concerns about additional bugs or changes in behavior that can be introduced. Is that something that you're seeing and, and how are you accounting for that? [00:38:38] Luca: Yeah, that's, that's an excellent question. So this is actually a, a great concern that we have all the time. And it's not just even introducing bugs, sometimes it's removing bugs. Like sometimes there's bugs in the node standard library which are there, and people are relying on these bugs to be there for the applications to function correctly. And we've seen this a lot, and then we implement this and we implement from scratch and we don't make that same bug. And then the test fails or then the application fails. So what we do is, um, we actually run node's test suite against Deno's Shim layer. So Node has a very extensive test suite for its own standard library, and we can run this suite against, against our shims to find things like this. And there's still edge cases, obviously, which node, like there was, maybe there's a bug which node was not even aware of existing. Um, where maybe this, like it's is, it's now standard, it's now like intended behavior because somebody relies on it, right? Like the second somebody relies on, on some non-standard or some buggy behavior, it becomes intended. Um, but maybe there was no test that explicitly tests for this behavior. Um, so in that case we'll add our own tests to, to ensure that. But overall we can already catch a lot of these by just testing, against, against node's tests. And then the other thing is we run a lot of real code, like we'll try run Prisma and we'll try run Vite and we'll try run NextJS and we'll try run like, I don't know, a bunch of other things that people throw at us and, check that they work and they work and there's no bugs. Then we did our job well and our shims are implemented correctly. Um, and then there's obviously always the edge cases where somebody did something absolutely crazy that nobody thought possible. and then they'll open an issue on the Deno repo and we scratch our heads for three days and then we'll fix it. And then in the next release there'll be a new bug that we added to make the compatibility with node better. so yeah, but I, yeah. Running tests is the, is the main thing running nodes test. Performance should be equal or better [00:40:32] Jeremy: Are there performance implications? If someone is running an Express App or an NextJS app in Deno, will they get any benefits from the Deno runtime and performance? [00:40:45] Luca: Yeah. It's actually, there is performance implications and they're usually. The opposite of what people think they are. Like, usually when you think of performance implications, it's always a negative thing, right? It's always okay. Like you, it's like a compromise. like the shim layer must be slower than the real node, right? It's not like we can run express faster than node can run, express. and obviously not everything is faster in Deno than it is in node, and not everything is faster in node than it is in Deno. It's dependent on the api, dependent on, on what each team decided to optimize. Um, and this also extends to other run times. Like you can always cherry pick results, like, I don't know, um, to, to make your runtime look faster in certain benchmarks. but overall, what really matters is that you do not like, the first important step for for good node compatibility is to make sure that if somebody runs your code or runs their node code in Deno or your other run type or whatever, It performs at least the same. and then anything on top of that great cherry on top. Perfect. but make sure the baselines is at least the same. And I think, yeah, we have very few APIs where we behave, where we, where, where like there's a significant performance degradation in Deno compared to Node. Um, and like we're actively working on these things. like Deno is not a, a, a project that's done, right? Like we have, I think at this point, like 15 or 16 or 17 engineers working on Deno, spanning across all of our different projects. And like, we have a whole team that's dedicated to performance, um, and a whole team that's dedicated node compatibility. so like these things get addressed and, and we make patch releases every week and a minor release every four weeks. so yeah, it's, it's not a standstill. It's, uh, constantly improving. What should go into the standard library? [00:42:27] Jeremy: Uh, something that kind of makes Deno stand out as it's standard library. There's a lot more in there than there is in in the node one. [00:42:38] Luca: Mm-hmm. [00:42:39] Jeremy: Uh, I wonder if you could speak to how you make decisions on what should go into it. [00:42:46] Luca: Yeah, so early on it was easier. Early on, the, the decision making process was essentially, is this something that a top 100 or top 1000 NPM library implements? And if it is, let's include it. and the decision making is still short of based on that. But right now we've already implemented most of the low hanging fruit. So things that we implement now are, have, have discussion around them whether we should implement them. And we have a process where, well we have a whole team of engineers on our side and we also have community members that, that will review prs and, and, and make comments. Open issues and, and review those issues, to sort of discuss the pros and cons of adding any certain new api. And sometimes it's also that somebody opens an issue that's like, I want, for example, I want an API to, to concatenate two unit data arrays together, which is something you can really easily do node with buffer dot con cat, like the scary buffer thing. and there's no standards way of doing that right now. So we have to have a little utility function that does that. But in parallel, we're thinking about, okay, how do we propose, an addition to the web standards now that makes it easy to concatenate iterates in the web standards, right? yeah, there's a lot to it. Um, but it's, it's really, um, it's all open, like all of our, all of our discussions for, for, additions to the standard library and things like that. It's all, all, uh, public on GitHub and the GitHub issues and GitHub discussions and GitHub prs. Um, so yeah, that's, that's where we do that. [00:44:18] Jeremy: Yeah, cuz to give an example, I was a little surprised to see that there is support for markdown front matter built into the standard library. But when you describe it as we look at the top a hundred thousand packages, are people looking at markdown? Are they looking at front matter? I, I'm sure there's a fair amount that are so that that makes sense. [00:44:41] Luca: Yeah, like it sometimes, like that one specifically was driven by, like, our team was just building a lot of like little blog pages and things like that. And every time it was either you roll your own front matter part or you look for one, which has like a subtle bug here and the other one has a subtle bug there and really not satisfactory with any of them. So, we, we roll that into the standard library. We add good test coverage for it good, add good documentation for it, and then it's like just a resource that people can rely on. Um, and you don't, you then don't have to make the choice of like, do I use this library to do my front meta parsing or the other library? No, you just use the one that's in the standard library. It's, it's also part of this like user experience thing, right? Like it's just a much nicer user experience, not having to make a choice, about stuff like that. Like completely inconsequential stuff. Like which library do we use to do front matter parsing? (laughs) [00:45:32] Jeremy: yeah. I mean, I think when, when that stuff is not there, then I think the temptation is to go, okay, let me see what node modules there are that will let me parse the front matter. Right. And then it, it sounds like probably ideally you want people to lean more on what's either in the standard library or what's native to the Deno ecosystem. Yeah. [00:46:00] Luca: Yeah. Like the, the, one of the big benefits is that the Deno Standard Library is implemented on top of web standards, right? Like it's, it's implemented on top of these standard APIs. so for example, there's node front matter libraries which do not run in the browser because the browser does not have the buffer global. maybe it's a nice library to do front matter pricing with, but. , you choose it and then three days later you decide that actually this code also needs to run in the browser, and then you need to go switch your front matter library. Um, so, so those are also kind of reasons why we may include something in Strand Library, like maybe there's even really good module already to do something. Um, but if there's certain reliance on specific node features that, um, we would like that library to also be compatible with, with, with web standards, we'll, uh, we might include in the standard library, like for example, YAML Parser, um, or the YAML Parser in the standard library is, is a fork of, uh, of the node YAML module. and it's, it's essentially that, but cleaned up and, and made to use more standard APIs rather than, um, node built-ins. [00:47:00] Jeremy: Yeah, it kind of reminds me a little bit of when you're writing a front end application, sometimes you'll use node packages to do certain things and they won't work unless you have a compatibility shim where the browser can make use of certain node APIs. And if you use the APIs that are built into the browser already, then you won't, you won't need to deal with that sort of thing. [00:47:26] Luca: Yeah. Also like less Bundled size, right? Like if you don't have to shim that, that's less, less code you have to ship to the client. WebAssembly use cases [00:47:33] Jeremy: Another thing I've seen with Deno is it supports running web assembly. [00:47:40] Luca: Mm-hmm. [00:47:40] Jeremy: So you can export functions and call them from type script. I was curious if you've seen practical uses of this in production within the context of Deno. [00:47:53] Luca: Yeah. there's actually a Bunch of, of really practical use cases, so probably the most executed bit of web assembly inside of Deno right now is actually yes, build like, yes, build has a web assembly, build like yeses. Build is something that's written and go. You have the choice of either running. Um, natively in machine code as, as like an ELF process on, on Linux or on on Windows or whatever. Or you can use the web assembly build and then it runs in web assembly. And the web assembly build is maybe 50% slower than the, uh, native build, but that is still significantly faster than roll up or, or, or, or I don't know, whatever else people use nowadays to do JavaScript Bun, I don't know. I, I just use es build always, um, So, um, for example, the Deno website, is running on Deno Deploy. And Deno Deploy does not allow you to run Subprocesses because it's, it's like this edge run time, which, uh, has certain security permissions that it's, that are not granted, one of them being sub-processes. So it needs to execute ES build. And the way it executes es build is by running them inside a web assembly. Um, because web assembly is secure, web assembly is, is something which is part of the JavaScript sandbox. It's inside the JavaScript sandbox. It doesn't poke any holes out. Um, so it's, it's able to run within, within like very strict security context. . Um, and then other examples are, I don't know, you want to have a HTML sanitizer, which is actually built on the real HTML par in a browser. we, we have an hdml sanitizer called com or, uh, ammonia, I don't remember. There's, there's an HTML sanitizer library on denoland slash x, which is built on the html parser from Firefox. Uh, which like ensures essentially that your html, like if you do HTML sanitization, you need to make sure your HTML par is correct, because if it's not, you might like, your browser might parse some HTML one way and your sanitizer pauses it another way and then it doesn't sanitize everything correctly. Um, so there's this like the Firefox HTML parser compiled to web assembly. Um, you can use that to. HTML sanitization, or the Deno documentation generation tool, for example. Uh, Deno Doc, there's a web assembly built for it that allows you to programmatically, like generate documentation for, for your type script modules. Um, yeah, and, and also like, you know, deno fmt is available as a WebAssembly module for programmatic access and a Bunch of other internal Deno, programs as well. Like, or, uh, like components, not programs. [00:50:20] Jeremy: What are some of the current limitations of web assembly and Deno for, for example, from web assembly, can I make HTTP requests? Can I read files? That sort of thing. [00:50:34] Luca: Mm-hmm. . Yeah. So web assembly, like when you spawn as web assembly, um, they're called instances, WebAssembly instances. It runs inside of the same vm, like the same, V8 isolate is what they're called, but. it does not have it, it's like a completely fresh sandbox, sort of, in the sense that I told you that between a runtime and like an engine essentially implements no IO calls, right? And a runtime does, like a runtime, pokes holds into the, the, the engine. web assembly by default works the same way that there is no holes poked into its sandbox. So you have to explicitly poke some holes. Uh, if you want to do HTTP calls, for example, when, when you create web assembly instance, it gives you, or you can give it something called imports, uh, which are essentially JavaScript function bindings, which you can call from within the web assembly. And you can use those function bindings to do anything you can from JavaScript. You just have to pass them through explicitly. and. . Yeah. Depending on how you write your web assembly, like if you write it in Rust, for example, the tooling is very nice and you can just call some JavaScript code from your Rust, and then the build system will automatically make sure that the right function bindings are passed through with the right names. And like, you don't have to deal with anything. and if you're writing go, it's slightly more complicated. And if you're writing like raw web assembly, like, like the web assembly, text format and compiling that to a binary, then like you have to do everything yourself. Right? It's, it's sort of the difference between writing C and writing JavaScript. Like, yeah. What level of abstraction do you want? It's definitely possible though, and that's for limitations. it, the same limitations as, as existing browsers apply. like the web assembly support in Deno is equivalent to the web assembly support in Chrome. so you can do, uh, many things like multi-threading and, and stuff like that already. but especially around, shared mutable memory, um, and having access to that memory from JavaScript. That's something which is a real difficulty with web assembly right now. yeah, growing web assembly memory is also rather difficult right now. There's, there's a, there's a couple inherent limitations right now with web assembly itself. Um, but those, those will be worked out over time. And, and Deno is like very up to date with the version of, of the standard, it, it implements, um, through v8. Like we're, we're, we're up to date with Chrome Beta essentially all the time. So, um, yeah. Any, anything you see in, in, in Chrome beta is gonna be in Deno already. Deno Deploy [00:52:58] Jeremy: So you talked a little bit about this before, the Deno team, they have their own, hosting. Platform called Deno Deploy. So I wonder if you could explain what that is. [00:53:12] Luca: Yeah, so Deno has this really nice, this really nice concept of permissions which allow you to, sorry, I'm gonna start somewhere slightly, slightly unrelated. Maybe it sounds like it's unrelated, but you'll see in a second. It's not unrelated. Um, Deno has this really nice permission system which allows you to sandbox Deno programs to only allow them to do certain operations. For example, in Deno, by default, if you try to open a file, it'll air out and say you don't have read permissions to read this file. And then what you do is you specify dash, dash allow read um, maybe you have to give it. they can either specify, allow, read, and then it'll grant to read access to the entire file system. Or you can explicitly specify files or folders or, any number of things. Same goes for right permissions, same goes for network permissions. Um, same goes for running subprocesses, all these kind of things. And by limiting your permissions just a little bit. Like, for example, by just disabling sub-processes and foreign function interface, but allowing everything else, allowing reeds and allowing network access and all that kind of stuff. we can run Deno programs in a way that is significantly more cost effective to you as the end user than, and, and like we can cold start them much faster than, like you may be able to with a, with a more conventional container based, uh, system. So what, what do you, what Deno Deploy is, is a way to run JavaScript or Deno Code, on our data centers all across the world with very little latency. like you can write some JavaScript code which execute, which serves HTTP requests deploy that to our platform, and then we'll make sure to spin that code up all across the world and have your users be able to access it through some URL or, or, or some, um, custom domain or something like that. and this is some, this is very similar to CloudFlare workers, for example. Um, and it's like Netlify Edge functions is built on top of Deno Deploy. Like Netlify Edge functions is implemented on top of Deno Deploy, um, through our sub hosting product. yeah, essentially Deno Deploy is, is, um, yeah, a cloud hosting service for JavaScript, um, which allows you to execute arbitrary JavaScript. and there there's a couple, like different directions we're going there. One is like more end user focused, where like you link your GitHub repository and. Like, we'll, we'll have a nice experience like you do with Netlify and Versace, that word like your commits automatically get deployed and you get preview deployments and all that kind of thing. for your backend code though, rather than for your front end websites. Although you could also write front-end websites and you know, obviously, and the other direction is more like business focused. Like you're writing a SaaS application and you want to allow the user to customize, the check like you're writing a SaaS application that provides users with the ability to write their own online store. Um, and you want to give them some ability to customize the checkout experience in some way. So you give them a little like text editor that they can type some JavaScript into. And then when, when your SaaS application needs to hit this code path, it sends a request to us with the code, we'll execute that code for you in a secure way. In a secure sandbox. You can like tell us you, this code only has access to like my API server and no other networks to like prevent data exfiltration, for example. and then you do, you can have all this like super customizable, code in inside of your, your SaaS application without having to deal with any of the operational complexities of scaling arbitrary code execution, or even just doing arbitrary code execution, right? Like it's, this is a very difficult problem and give it to someone else and we deal with it and you just get the benefits. yeah, that's Deno Deploy, and it's built by the same team that builds the Deno cli. So, um, all the, all of your favorite, like Deno cli, or, or Deno APIs are available in there. It's just as web standard is Deno, like you have fetch available, you have blob available, you have web crypto available, that kind of thing. yeah. Running code in V8 isolates [00:56:58] Jeremy: So when someone ships you their, their code and you run it, you mentioned that the, the cold start time is very low. Um, how, how is the code being run? Are people getting their own process? It sounds like it's not, uh, using containers. I wonder if you could explain a little bit about how that works. [00:57:20] Luca: Yeah, yeah, I can, I can give a high level overview of how it works. So, the way it works is that we essentially have a pool of, of Deno processes ready. Well, it's not quite Deno processes, it's not the same Deno CLI that you download. It's like a modified version of the Deno CLI based on the same infrastructure, that we have spun up across all of our different regions across the world, uh, across all of our different data centers. And then when we get a request, we'll route that request, um, the first time we get request for that, that we call them deployments, that like code, right? We'll take one of these idle Deno processes and will assign that code to run in that process, and then that process can go serve the requests. and these process, they're, they're, they're isolated and they're, you. it's essentially a V8 isolate. Um, and it's a very, very slim, it's like, it's a much, much, much slimmer version of the Deno cli essentially. Uh, which the only thing it can do is JavaScript execution and like, it can't even execute type script, for example, like type script is we pre-process it up front to make the the cold start faster. and then what we do is if you don't get a request for some amount of. , we'll, uh, spin down that, um, that isolate and, uh, we'll spin up a new idle one in its place. And then, um, if you get another request, I don't know, an hour later for that same deployment, we'll assign it to a new isolate. And yeah, that's a cold start, right? Uh, if you have an isolate which receives, or a, a deployment rather, which receives a Bunch of traffic, like let's say you receive a hundred requests per second, we can send a Bunch of that traffic to the same isolate. Um, and we'll make sure that if, that one isolate isn't able to handle that load, we'll spin it out over multiple isolates and we'll, we'll sort of load balance for you. Um, and we'll make sure to always send to the, to the point of present that's closest to, to the user making the request. So they get very minimal latency. and they get we, we've these like layers of load balancing in place and, and, and. I'm glossing over a Bunch of like security related things here about how these, these processes are actually isolated and how we monitor to ensure that you don't break out of these processes. And for example, Deno Deploy does, it looks like you have a file system cuz you can read files from the file system. But in reality, Deno Deploy does not have a file system. Like the file system is a global virtual file system. which is, is, uh, yeah, implemented completely differently than it is in Deno cli. But as an end user you don't have to care about that because the only thing you care about is that it has the exact same API as the Deno cli and you can run your code locally and if it works there, it's also gonna work in deploy. yeah, so that's, that's, that's kind of. High level of Deno Deploy. If, if any of this sounds interesting to anyone, by the way, uh, we're like very actively hiring on, on Deno Deploy. I happen to be the, the tech lead for, for a Deno Deploy product. So I'm, I'm always looking for engineers, to, to join our ranks and, and build cool distributed systems. Deno.com/jobs. [01:00:15] Jeremy: for people who aren't familiar with the isolates, are these each run in their own processes, or do you have a single process and that has a whole Bunch of isolates inside it? [01:00:28] Luca: in, in the general case, you can say that we run, uh, one isolate per process. but there's many asterisks on that. Um, because, it's, it's very complicated. I'll just say it's very complicated. Uh, in, in the general case though, it's, it's one isolate per process. Yeah. Configuring permissions [01:00:45] Jeremy: And then you touched a little bit on the permissions system. Like you gave the example of somebody could have a website where they let their users give them code to execute. how does it look in terms of specifying what permissions people have? Like, is that a configuration file? Are those flags you pass in? What, what does that look? [01:01:08] Luca: Yeah. So, so that product is called sub hosting. It's, um, slightly different from our end user platform. Um, it's essentially a service that allows you to, like, you email us, well, we'll send you a, um, onboard you, and then what you can do is you can send HTTP requests to a certain end point with a, authentication token and. a reference to some code to execute. And then what we'll do is, we'll, um, when we receive that HTTP request, we'll fetch the code, it's spin up and isolate, execute the code. execute the code. We serve the request, return you the response, um, and then we'll pipe logs to you and, and stuff like that. and the, and, and part of that is also when we, when we pull the, um, the, the code for to spin up the isolate, that code doesn't just include the code that we're executing, but also includes things like permissions, and, and various other, we call this isolate configuration. Um, you can inspect, this is all public. we have public docs for this at Deno.com/subhosting. I think. Yes, Deno.com/subhosting. [01:02:08] Jeremy: And is that built on top of something that's a part of the public Deno project, the open source part? Or is this specific to this sub hosting

Chris Toomey is back! (For an episode.) He talks about what he's been up to since handing off the reins to Joël. He's been playing around with something at Sagewell that he enjoys. At the core of it? Serializers. Primalize gem (https://github.com/jgaskins/primalize) Derek's talk on code review (https://www.youtube.com/watch?v=PJjmw9TRB7s) Inertia.js (https://inertiajs.com/) Phantom types (https://thoughtbot.com/blog/modeling-currency-in-elm-using-phantom-types) io-ts (https://gcanti.github.io/io-ts/) dry-rb (https://dry-rb.org/) parse don't validate (https://lexi-lambda.github.io/blog/2019/11/05/parse-don-t-validate/) value objects (http://wiki.c2.com/?ValueObject) broader perspective on parsing (https://thoughtbot.com/blog/a-broader-take-on-parsing) Enumerable#tally (https://medium.com/@baweaver/ruby-2-7-enumerable-tally-a706a5fb11ea) RubyConf mini (https://www.rubyconfmini.com/) where.missing (https://boringrails.com/tips/activerecord-where-missing-associations) Transcript: JOËL: Hello and welcome to another episode of The Bike Shed, a weekly podcast from your friends at thoughtbot about developing great software. I'm Joël Quenneville. And today, I'm joined by a very special guest, former host Chris Toomey. CHRIS: Hi, Joël. Thanks for having me. JOËL: And together, we're here to share a little bit of what we've learned along the way. So, Chris, what's new in your world? CHRIS: Being on this podcast is new in my world, or everything old is new again, or something along those lines. But, yeah, thank you so much for having me back. It's a pleasure. Although it's very odd, it feels somehow so different and yet very familiar. But yeah, more generally, what's new in my world? I think this was probably in development as I was winding down my time as a host here on The Bike Shed, but I don't know that I ever got a chance to talk about it. There has been a fun sort of deep-in-the-weeds technical thing that we've been playing around with at Sagewell that I've really enjoyed. So at the core of it, we have serializers. So we take some data structures in our Ruby on Rails code base, and we need to serialize them to JSON to send them to the front end. In our case, we're using Inertia, so it's not quite a JSON API, but it's fine to think about it in that way for the context of this discussion. And what we were finding is our front end has TypeScript. So we're writing Svelte, which is using TypeScript. And so we're stating or asserting that the types like, hey, we're going to get this data in from the back end, and it's going to have this shape to it. And we found that it was really hard to keep those in sync to keep, like, what does the user mean on the front end? What's the data that we're going to get? It's going to have a full name, which is a string, except sometimes that might be null. So how do we make sure that those are keeping up to date? And then we had a growing number of serializers on the back end and determining which serializer we were actually using, and it was just...it was a mess, to put it lightly. And so we had explored a couple of different options around it, and eventually, we found a library called Primalize. So Primalize is a Ruby library. It is for writing JSON serializers. But what's really interesting about it is it has a typing layer. It's like a type system sort of thing at play. So when you define a serializer in Primalize, instead of just saying, here are the fields; there is an ID, a name, et cetera, you say, there is an ID, and it is a string. There is a name, and it is a string, or an optional string, which is the even more interesting bit. You can say array. You can say object. You can say an enum of a couple of different values. And so we looked at that, and we said, ooh, this is very interesting. Astute listeners will know that this is probably useless in a Ruby system, which doesn't have types or a compilation step or anything like that. But what's really cool about this is when you use a Primalize serializer, as you're serializing an object, if there is ever a type mismatch, so the observed type at runtime and the authored type if those ever mismatch, then you can have some sort of notification happen. So in our case, we configured it to send a warning to Sentry to say, "Hey, you said the types were this, but we're actually seeing this other thing." Most often, it will be like an Optional, a null sneaking through, a nil sneaking through on the Ruby side. But what was really interesting is as we were squinting at this, we're like, huh, so now we're going to write all this type information. What if we could somehow get that type information down to the front end? So I had a long weekend, one weekend, and I went away, and I wrote a bunch of code that took all of those serializers, ran through them, and generated the associated TypeScript interfaces. And so now we have a build step that will essentially run that and assert that we're getting the same thing in CI as we have committed to the codebase. But now we have the generated serializer types on the front end that match to the used serializer on the back end, as well as the observed run-time types. So it's a combination of a true compilation step type system on the front end and a run-time type system on the back end, which has been very, very interesting. JOËL: I have a lot of thoughts here. CHRIS: I figured you would. [laughs] JOËL: But the first thing that came to mind is, as a consultant, there's a scenario with especially smaller startups that generally concerns me, and that is the CTO goes away for a weekend and writes a lot of code... CHRIS: [laughs] JOËL: And brings in a new system on Monday, which is exactly what you're describing here. How do you feel about the fact that you've done that? CHRIS: I wasn't ready to go this deep this early on in this episode. JOËL: [laughs] CHRIS: But honestly, that is a fantastic question. It's a thing that I have been truly not struggling with but really thinking about. We're going to go on a slight aside here, but I am finding it really difficult to engage with the actual day-to-day coding work that we're doing and to still stay close to the codebase and not be in the way. There's a pattern that I've seen happen a number of times now where I pick up a piece of work that is, you know, one of the tickets at the top of the backlog. I start to work on it. I get pulled into a meeting, then another meeting, then three more meetings. And suddenly, it's three days later. I haven't completed this piece of work that was defined to be the next most important piece of work. And suddenly, I'm blocking the team. JOËL: Hmmm. CHRIS: So I actually made a rule that I'm not allowed to own critical path work, which feels weird because it's like, I want to be engaged with that work. So the counterpoint to that is I'm now trying to schedule pairing sessions with each of the developers on the team once a week. And in that time, I can work on that sort of stuff with them, and they'll then own it and run with it. So it makes sure that I'm not blocking on those sorts of things, but I'm still connected to the core work that we're doing. But the other thing that you're describing of the CTO goes away for the weekend and then comes back with a new harebrained scheme; I'm very sensitive to that, having worked on; frankly, I think the same project. I can think of a project that you and I worked on where we experienced this. JOËL: I think we're thinking of the same project. CHRIS: So yes. Like, I'm scarred by that and, frankly, a handful of experiences of that nature. So we actually, I think, have a really healthy system in place at Sagewell for capturing, documenting, prioritizing this sort of other work, this developer-centric work. So this is the feature and bug work that gets prioritized and one list over here that is owned by our product manager. Separately, the dev team gets to say, here are the pain points. Here's the stuff that keeps breaking. Here are the things that I wish was better. Here is the observability hard-to-understand bits. And so we have a couple of different systems at play and recurring meetings and sort of unique ceremonies around that, and so this work was very much a fallout of that. It was actually a recurring topic that we kept trying a couple of different stabs at, and we never quite landed it. And then I showed up this one Monday morning, and I was like, "I found a thing; what do we think?" And then, critically, from there, I made sure I paired with other folks on the team as we pushed on the implementation. And then, actually, I mentioned Primalize, the library that we're using. We have now since deprecated Primalize within the app because we kept just adding to it so much that eventually, we're like, at this point, should we own this stuff? So we ended up rewriting the core bits of Primalize to better fit our use cases. And now we've actually removed Primalize, wonderful library. I highly recommend it to anyone who has that particular use case but then the additional type generation for the front end. Plus, we have some custom types within our app, Money being the most interesting one. We decided to model Money as our first-class consideration rather than just letting JavaScript have the sole idea of a number. But yes, in a very long-winded way, yes, I'm very sensitive to the thing you described. And I hope, in this case, I did not fall prey to the CTO goes away for the weekend and made a thing. JOËL: I think what I'm hearing is the key difference here is that you got buy-in from the team around this idea before you went out and implemented it. So you're not off doing your own things disconnected from the team and then imposing it from on high. The team already agreed this is the thing we want to do, and then you just did it for them. CHRIS: Largely, yes. Although I will say there are times that each developer on the team, myself included, have sort of gone away, come back with something, and said, "Hey, here's a WIP PR exploring an area." And there was actually...I'm forgetting what the context was, but there was one that happened recently that I introduced. I was like; I had to do this. And the team talked me out of it, and I ended up closing that PR. Someone else actually made a different PR that was an alternative implementation. I was like, no, that's better; we should absolutely do that. And I think that's really healthy. That's a hard thing to maintain but making sure that everyone feels like they've got a strong voice and that we're considering all of the different ways in which we might consider the work. Most critically, you know, how does this impact users at the end of the day? That's always the primary consideration. How do we make sure we build a robust, maintainable, observable system, all those sorts of things? And primarily, this work should go in that other direction, but I also don't want to stifle that creative spark of I got this thing in my head, and I had to explore it. Like, we shouldn't then need to never mind, throw away the work, put it into a ticket. Like, for as long as we can, that more organic, intuitive process if we can retain that, I like that. Critically, with the ability for everyone to tell me, "No, this is a bad idea. Stop it. What are you doing?" And that has happened recently. I mean, they were kinder about it, but they did talk me out of a bad idea. So here we are. JOËL: So you showed up on Monday morning, not with telling everyone, "Hey, I merged this thing over the weekend." You're showing up with a work-in-progress PR. CHRIS: Yes, definitely. I mean, everything goes through a PR, and everything has discussion and conversation around it. That's a strong, strong like Derek Prior's wonderful talk Building a Culture of Code Review. I forget the exact name of it. But it's one of my favorite talks in talking about the utility of code review as a way to share ideas and all of those wonderful things. So everything goes through code review, and particularly anything that is of that more exploratory architectural space. Often we'll say any one review from anyone on the team is sufficient to merge most things but something like that, I would want to say, "Hey, can everybody take a look at this? And if anyone has any reservations, then let's talk about it more." But if I or anyone else on the team for this sort of work gets everybody approving it, then cool, we're good to go. But yeah, code review critical, critical part of the process. JOËL: I'm curious about Primalize, the gem that you mentioned. It sounds like it's some kind of validation layer between some Ruby data structure and your serializers. CHRIS: It is the serializer, but in the process of serializing, it does run-time type validation, essentially. So as it's accessing, you know, you say first name. You have a user object. You pass it in, and you say, "Serializer, there's a first name, and it's a string." It will call the first name method on that user object. And then, it will check that it has the expected type, and if it doesn't, then, in our case, it sends to Sentry. We have configured it...it's actually interesting. In development and test mode, it will raise for a type mismatch, and in production mode, it will alert Sentry so you can configure that differently. But that ends up being really nice because these type mismatches end up being very loud early on. And it's surprisingly easy to maintain and ends up telling us a lot of truths about our system because, really, what we're doing is connecting data from many different systems and flowing it in and out. And all of the inputs and outputs from our system feel very meaningful to lock down in this way. But yeah, it's been an adventure. JOËL: It seems to me there could almost be two sets of types here, the inputs coming into Primalize from your Ruby data structures and then the outputs that are the actual serialized values. And so you might expect, let's say, an integer on the Ruby side, but maybe at the serialization level, you're serializing it to a string. Do you have that sort of conversion step as part of your serializers sometimes, or is the idea that everything's already the right type on the Ruby side, and then we just, like, to JSON it at the end? CHRIS: Yep. Primalize, I think, probably works a little closer to what you're describing. They have the idea of coercions. So within Primalize, there is the concept of a timestamp; that is one of the types that is available. But a timestamp is sort of the union of a date, a time, or I think they might let through a string; I'm not sure if there is as well. But frankly, for us, that was more ambiguity than we wanted or more blurring across the lines. And in the implementation that we've now built, date and time are distinct. And critically, a string is not a valid date or time; it is a string, that's another thing. And so there's a bunch of plumbing within the way you define the serializers. There are override methods so that you can locally within the serializer say, like, oh, we need to coerce from the shape of data into this other shape of data, even little like in-line proc, so we can do it quickly. But the idea is that the data, once it has been passed to the serializer, should be up the right shape. And so when we get to the type assertion part of the library, we expect that things are in the asserted type and will warn if not. We get surprisingly few warnings, which is interesting now. This whole process has made us pay a little more intention, and it's been less arduous simultaneously than I would have expected because like this is kind of a lot of work that I'm describing. And yet it ends up being very natural when you're the developer in context, like, oh, I've been reading these docs for days. I know the shape of this JSON that I'm working with inside and out, and now I'll just write it down in the serializer. It's very easy to do in that moment, and then it captures it and enforces it in such a useful way. As an aside, as I've been looking at this, I'm like, this is just GraphQL, but inside out, I'm pretty sure. But that is a choice that we have made. We didn't want to adopt the whole GraphQL thing. But just for anyone out there who is listening and is thinking, isn't this just GraphQL but inside out? Kind of. Yes. JOËL: I think my favorite part of GraphQL is the schema, which is not really the selling point for GraphQL, you know, like the idea that you can traverse the graph and get any subset of data that you want and all that. I think I would be more than happy with a REST API that has some kind of schema built around it. And someone told me that maybe what I really just want is SOAP, and I don't know how to feel about that comment. CHRIS: You just got to have some XML, and some WSDLs, and other fun things. I've heard people say good things about SOAP. SOAP seems like a fine idea. If anything, I think a critical part of this is we don't have a JSON API. We have a very tightly coupled front end and back end, and a singular front end, frankly. And so that I think naturally...that makes the thing that I'm describing here a much more comfortable fit. If we had multiple different downstream clients that we're trying to consume from the same back end, then I think a GraphQL API or some other structured JSON schema, whatever it is type of API, and associated documentation and typing layer would be probably a better fit. But as I've said many a time on this here, Bike Shed, Inertia is one of my favorite libraries or frameworks (They're probably more of a framework.) one of my favorite technological approaches that I have ever found. And particularly in buildings Sagewell, it has allowed us to move so rapidly the idea that changes are, you know, one fell swoop changes everything within the codebase. We don't have to think about syncing deploys for the back end and the front end and how to coordinate across them. Our app is so much easier to understand by virtue of that architecture that Inertia implies. JOËL: So, if I understand correctly, you don't serialize to JSON as part of the serializers. You're serializing directly to JavaScript. CHRIS: We do serialize to JSON. At the end of the day, Inertia takes care of this on both the Rails side and the client side. There is a JSON API. Like, if you look at the network inspector, you will see XHR requests happening. But critically, we're not doing that. We're not the ones in charge of it. We're not hitting a specific endpoint. It feels as an application coder much closer to a traditional Rails app. It just happens to be that we're writing our view layer. Instead of an ERB, we're writing them in Svelte files. But otherwise, it feels almost identical to a normal traditional Rails app with controllers and the normal routing and all that kind of stuff. JOËL: One thing that's really interesting about JSON as an interchange format is that it is very restrictive. The primitives it has are even narrower than, say, the primitives that Ruby has. So you'd mentioned sending a date through. There is no JSON date. You have to serialize it to some other type, potentially an integer, potentially a string that has a format that the other side knows how it's going to interpret. And I feel like it's those sorts of richer types when we need to pass them through JSON that serialization and deserialization or parsing on the other end become really interesting. CHRIS: Yeah, I definitely agree with that. It was a struggling point for a while until we found this new approach that we're doing with the serializers in the type system. But so far, the only thing that we've done this with is Money. But on the front end, a while ago, we introduced a specific TypeScript type. So it's a phantom type, and I believe I'm getting this correct. It's a phantom type called Cents, C-E-N-T-S. So it represents...I'm going to say an integer. I know that JavaScript doesn't have integers, but logically, it represents an integer amount of cents. And critically, it is not a number, like, the lowercase number in the type system. We cannot add them together. We can't -- JOËL: I thought you were going to say, NaN. CHRIS: [laughs] It is not a number. I saw a n/a for not applicable somewhere in the application the other day. I was like, oh my God, we have a NaN? It happened? But it wasn't, it was just n/a, and I was fine. But yeah, so we have this idea of Cents within the application. We have a money input, which is a special input designed exactly for this. So to a user, it is formatted to look like you're entering dollars and cents. But under the hood, we are bidirectionally converting that to the integer amount of cents that we need. And we strictly, within the type system, those are cents. And you can't do math on Cents unless you use a special set of helper functions. You cannot generate Cents on the fly unless you use a special set of helper functions, the constructor functions. So we've been really restrictive about that, which was kind of annoying because a lot of the data coming from the server is just, you know, numbers. But now, with this type system that we've introduced on the Ruby side, we can assert and enforce that these are money.new on the Ruby side, so using the Money gem. And they come down to the front end as capital C Cents in the type system on the TypeScript side. So we're able to actually bind that together and then enforce proper usage sort of on both sides. The next step that we plan to do after that is dates and times. And those are actually almost weirder because they end up...we just have to sort of say what they are, and they will be ISO 8601 date and time strings, respectively. But we'll have functions that know this is a date string; that's a thing. It is, again, a phantom type implemented within our TypeScript type system. But we will have custom functions that deal with that and really constrain...lock ourselves down to only working with them correctly. And critically, saying that is the only date and time format that we work with; there is no other. We don't have arbitrary dates. Is this a JSON date or something else? I don't know; there are too many date syntaxes. JOËL: I like the idea of what you're doing in that it sounds like you're very much narrowing that sort of window of where in the stack the data exists in the sort of unstructured, free-floating primitives that could be misinterpreted. And so, at this point, it's almost narrowed to the point where it can't be touched by any user or developer-written code because you've pushed the boundaries on the Rails side down and then on the JavaScript side up to the point where the translation here you define translations on one side or, I guess, a parser on one side and a serializer on the other. And they guarantee that everything is good up until that point. CHRIS: Yep, with the added fun of the runtime reflection on the Ruby side. So it's an interesting thing. Like, TypeScript actually has similar things. You can say what the type is all day long, and your code will consistently conform to that asserted type. But at the end of the day, if your JSON API gets in some different data...unless you're using a library like io-ts, is one that I've looked at, which actually does parsing and returns a result object of did we parse to the thing that you wanted or did we get an error in that data structure? So we could get to that level on the client side as well. We haven't done that yet largely because we've essentially pushed that concern up to the Ruby layer. So where we're authoring the data, because we own that, we're going to do it at that level. There are a bunch of benefits of defining it there and then sort of reflecting it down. But yeah, TypeScript, you can absolutely lie to yourself, whereas Elm, a language that I know you love dearly, you cannot lie to yourself in Elm. You've got to tell the truth. It's the only option. You've got to prove it. Whereas in TypeScript, you can just kind of suggest, and TypeScript will be like, all right, cool, I'll make sure you stay honest on that, but I'm not going to make you prove it, which is an interesting sort of set of related trade-offs there. But I think we found a very comfortable resting spot for right now. Although now, we're starting to look at the edges of the Ruby system where data is coming in. So we have lots of webhooks and other external partners that we're integrating with, and they're sending us data. And that data is of varying shapes. Some will send us a payload with the word amount, and it refers to an integer amount of cents because, of course, it does. Some will send us the word amount in their payload, and it will be a floating amount of dollars. And I get a little sad on those days. But critically, our job is to make sure all of those are the same and that we never pass dollars as cents or cents as dollars because that's where things go sad. That is job number one at Sagewell in the engineering team is never get the decimal place wrong in money. JOËL: That would be a pretty terrible mistake to make. CHRIS: It would. I mean, it happens. In fintech, that problem comes up a lot. And again, the fact that...I'm honestly surprised to see situations out there where we're getting in floating point dollars. That is a surprise to me because I thought we had all agreed sort of as a community that it was integer cents but especially in a language that has integers. JavaScript, it's kind of making it up the whole time. But Ruby has integers. JSON, I guess, doesn't have integers, so I'm sort of mixing concerns here, but you get the idea. JOËL: Despite Ruby not having a static type system, I've found that generally, when I'm integrating with a third-party API, I get to the point where I want something that approximates like Elm's JSON decoders or io-ts or something like that. Because JSON is just a big blob of data that could be of any shape, and I don't really trust it because it's third-party data, and you should not trust third parties. And I find that I end up maybe cobbling something together commonly with like a bunch of usage of hash.fetch, things like that. But I feel like Ruby doesn't have a great approach to parsing and composing these validators for external data. CHRIS: Ruby as a language certainly doesn't, and the ecosystem, I would say, is rather limited in terms of the options here. We have looked a bit at the dry-rb stack of gems, so dry-validation and dry-schema, in particular, both offer potentially useful aspects. We've actually done a little bit of spiking internally around that sort of thing of, like, let's parse this incoming data instead of just coercing to hash and saying that it's got probably the shape that we want. And then similarly, I will fetch all day instead of digging because I want to be quite loud when we get it wrong. But we're already using dry-monads. So we have the idea of result types within the system. We can either succeed or fail at certain operations. And I think it's just a little further down the stack. But probably something that we will implement soon is at those external boundaries where data is coming in doing some form of parsing and validation to make sure that it conforms to unknown data structure. And then, within the app, we can do things more cleanly. That also would allow us to, like, let's push the idea that this is floating point dollars all the way out to the edge. And the minute it hits our system, we convert it into a money.new, which means that cents are properly handled. It's the same type of money or dollar, same type of currency handling as everywhere else in the app. And so pushing that to the very edges of our application is a very interesting idea. And so that could happen in the library or sort of a parsing client, I guess, is probably the best way to think about it. So I'm excited to do that at some point. JOËL: Have you read the article, Parse, Don't Validate? CHRIS: I actually posted that in some code review the other day to one of the developers on the team, and they replied, "You're just going to quietly drop one of my favorite articles of all time in code review?" [laughs] So yes, I've read it; I love it. It's a wonderful idea, definitely something that I'm intrigued by. And sort of bringing dry-monads into Ruby, on the one hand, feels like a forced fit and yet has also been one of the other, I think strongest sort of architectural decisions that we've made within the application. There's so much imperative work that we ended up having to do. Send this off to this external API, then tell this other one, then tell this other one. Put the whole thing in a transaction so that our local data properly handles it. And having dry-monads do notation, in particular, to allow us to make that manageable but fail in all the ways it needs to fail, very expressive in its failure modes, that's been great. And then parse, don't validate we don't quite do it yet. But that's one of the dreams of, like, our codebase really should do that thing. We believe in that. So let's get there soon. JOËL: And the core idea behind parse, don't validate is that instead of just having some data that you don't trust, running a check on it and passing that blob of now checked but still untrusted data down to the next person who might also want to check it. Generally, you want to pass it through some sort of filter that will, one, validate that it's correct but then actually typically convert it into some other trusted shape. In Ruby, that might be something like taking an amorphous blob of JSON and turning it into some kind of value object or something like that. And then anybody downstream that receives, let's say, money object can trust that they're dealing with a well-formed money value as opposed to an arbitrary blob of JSON, which hopefully somebody else has validated, but who knows? So I'm going to validate it again. CHRIS: You can tell that I've been out of the podcasting game for a while because I just started responding to yes; I love that blog post without describing the core premise of it. So kudos to you, Joël; you are a fantastic podcast host over there. I will say one of the things you just described is an interesting...it's been a bit of a struggle for us. We keep sort of talking through what's the architecture. How do we want to build this application? What do we care about? What are the things that really matter within this codebase, and then what is all the other stuff? And we've been good at determining the things that really matter, thinking collectively as a group, and I think coming up with some novel, useful, elegant...I'm saying too many positive adjectives for what we're doing. But I've been very happy with sort of the thing that we decide. And then there's the long-tail work of actually propagating that change throughout the rest of the application. We're, like, okay, here's how it works. Every incoming webhook, we now parse and yield a value object. That sentence that you just said a minute ago is exactly what I want. That's like a bunch of work. It's particularly a bunch of work to convert an existing codebase. It's easy to say, okay, from here forward, any new webhooks, payloads that are coming in, we're going to do in this way. But we have a lot of things in our app now that exist in this half-converted way. There was a brief period where we had three different serializer technologies at play. Just this week, I did the work of killing off the middle ground one, the Primalized-based thing, and we now have only our new hotness and then the very old. We were using Blueprinter as the serializer as the initial sort of stub. And so that still exists within the codebase in some places. But trying to figure out how to prioritize that work, the finishing out those maintenance-type conversions is a tricky one. It's never the priority. But it is really nice to have consistency in a codebase. So it's...yeah, do you have any thoughts on that? JOËL: I think going back to the article and what the meaning of parsing is, I used to always think of parsing as taking strings and turning them into something else, and I think this really broadened my perspective on the idea of parsing. And now, I think of it more as converting from a broader type to a narrower type with failures. So, for example, you could go from a string to an integer, and not all strings are valid integers. So you're narrowing the type. And if you have the string hello world, it will fail, and it will give you an error of some type. But you can have multiple layers of that. So maybe you have a string that you parse into an integer, but then, later on, you might want to parse that integer into something else that requires an integer in a range. Let's say it's a percentage. So you have a value object that is a percentage, but it's encoded in the JSON as a string. So that first pass, you parse it from a string into an integer, and then you parse that integer into a percentage object. But if it's outside the range of valid percentage numbers, then maybe you get an error there as well. So it's a thing that can happen at multiple layers. And I've now really connected it with the primitive obsession smell in code. So oftentimes, when you decide, wait, I don't want a primitive here; I want a richer type, commonly, there's going to be a parsing step that should exist to go from that primitive into the richer type. CHRIS: I like that. That was a classic Joël wildly concise summary of a deeply complex technical topic right there. JOËL: It's like I'm going to connect some ideas from functional programming and a classic object-oriented code smell and, yeah, just kind of mash it all together with a popular article. CHRIS: If only you had a diagram. Podcast is not the best medium for diagrams, but I think you could do it. You could speak one out loud, and everyone would be able to see it in their mind's eye. JOËL: So I will tell you what my diagram is for this because I've actually created it already. I imagine this as a sort of like pyramid with different layers that keep getting smaller and smaller. So the size of type is sort of the width of a layer. And so your strings are a very wide layer. Then on top of that, you have a narrower layer that might be, you know, it could be an integer, or you could even if you're parsing JSON, you first start with a string, then you parse that into a Ruby hash, not all strings are valid hashes. So that's going to be narrower. Then you might extract some values out of that hash. But if the keys aren't right, that might also fail. You're trying to pull the user out of it. And so each layer it gets a richer type, but that richer type, by virtue of being richer, is narrower. And as you're trying to move up that pyramid at every step, there is a possibility for a failure. CHRIS: Have you written a blog post about this with said diagram in it? And is that why you have that so readily at hand? [laughs] JOËL: Yes, that is the case. CHRIS: Okay. Yeah, that made sense to me. [laughs] JOËL: We'll make sure to link to it in the show notes. CHRIS: Now you have to link to Joël blog posts, whereas I used to have to link to them [chuckles] in almost every episode of The Bike Shed that I recorded. JOËL: Another thing I've been thinking about in terms of this parsing is that parsing and serializing are, in a sense, almost opposites of each other. Typically, when you're parsing, you're going from a broad type to a narrow one. And when you're serializing, you're going from a narrow type to a broader one. So you might go from a user into a hash into a string. So you're sort of going down that pyramid rather than going up. CHRIS: It is an interesting observation and one that immediately my brain is like, okay, cool. So can we reuse our serializers but just run them in reverse or? And then I try and talk myself out of that because that's a classic don't repeat yourself sort of failure mode of, like, actually, it's fine. You can repeat a little bit. So long as you can repeat and constrain, that's a fine version. But yeah, feels true, though, at the core. JOËL: I think, in some ways, if you want a single source of truth, what you want is a schema, and then you can derive serializers and parsers from that schema. CHRIS: It's interesting because you used the word derive. That has been an interesting evolution at Sagewell. The engineering team seems to be very collected around the idea of explicitness, almost the Zen of Python; explicit is better than implicit. And we are willing to write a lot of words down a lot of times and be happy with that. I think we actually made the explicit choice at one point that we will not implement an automatic camel case conversion in our serializer, even though we could; this is a knowable piece of code. But what we want is the grepability from the front end to the back end to say, like, where's this data coming from? And being able to say, like, it is this data, which is from this serializer, which comes from this object method, and being able to trace that very literally and very explicitly in the code, even though that is definitely the sort of thing that we could derive or automatically infer or have Ruby do that translation for us. And our codebase is more verbose and a little noisier. But I think overall, I've been very happy with it, and I think the team has been very happy. But it is an interesting one because I've seen plenty of teams where it is the exact opposite. Any repeated characters must be destroyed. We must write code to write the code for us. And so it's fun to be working with a team where we seem to be aligned around an approach on that front. JOËL: That example that you gave is really interesting because I feel like a common thing that happens in a serialization layer is also a form of normalization. And so, for example, you might downcase all strings as part of the serialization, definitely, like dates always get written in ISO 8601 format whenever that happens. And so, regardless of how you might have it stored on the Ruby side, by the time it gets to the JSON, it's always in a standard format. And it sounds like you're not necessarily doing that with capitalization. CHRIS: I think the distinction would be the keys and the values, so we are definitely doing normalization on the values side. So ISO 8601 date and time strings, respectively that, is the direction that we plan to go for the value. But then for the key that's associated with that, what is the name for this data, those we're choosing to be explicit and somewhat repetitive, or not even necessarily repetitive, but the idea of, like, it's first_name on the Ruby side, and it's first capital N name camel case, or it's...I forget the name. It's not quite camel case; it's a different one but lower camel, maybe. But whatever JavaScript uses, we try to bias towards that when we're going to the front end. It does get a little tricky coming back into the Ruby side. So our controllers have a bunch of places where they need to know about what I think is called lower camel case, and so we're not perfect there. But that critical distinction between sort of the names for things, and the values for things, transformations, and normalizations on the values, I'm good with that. But we've chosen to go with a much more explicit version for the names of things or the keys in JSON objects specifically. JOËL: One thing that can be interesting if you have a normalization phase in your serializer is that that can mean that your serializer and parsers are not necessarily symmetric. So you might accept malformed data into your parser and parse it correctly. But then you can't guarantee that the data that gets serialized out is going to identically match the data that got parsed in. CHRIS: Yeah, that is interesting. I'm not quite sure of the ramifications, although I feel like there are some. It almost feels like formatting Prettier and things like that where they need to hold on to whitespace in some cases and throw out in others. I'm thinking about how ASTs work. And, I don't know, there's interesting stuff, but, again, not sure of the ramifications. But actually, to flip the tables just a little bit, and that's an aggressive terminology, but we're going to roll with it. To flip the script, let's go with that, Joël; what's been up in your world? You've been hosting this wonderful show. I've listened in to a number of episodes. You're doing a fantastic job. I want to hear a little bit more of what's new in your world, Joël. JOËL: So I've been working on a project that has a lot of flaky tests, and we're trying to figure out the source of that flakiness. It's easy to just dive into, oh, I saw a flaky Test. Let me try to fix it. But we have so much flakiness that I want to go about it a little bit more systematically. And so my first step has actually been gathering data. So I've actually been able to make API requests to our CI server. And the way we figure out flakiness is looking at the commit hash that a particular test suite run has executed on. And if there's more than one CI build for a given commit hash, we know that's probably some kind of flakiness. It could be a legitimate failure that somebody assumed was flakiness, and so they just re-run CI. But the symptom that we are trying to address is the fact that we have a very high level of people re-verifying their code. And so to do that or to figure out some stats, I made a request to the API grouped by commit hash and then was able to get the stats of how many re-verifications there are and even the distribution. The classic way that you would do that is in Ruby; you would use the GroupBy function from enumerable. And then, you would transform values instead of having, like, say; each commit hash then points to all the builds, an array of builds that match that commit hash. You would then thumb those. So now you have commit hashes that point to counts of how many builds there were for that commit hash. Newer versions of Ruby introduced the tally method, which I love, which allows you to basically do all of that in one step. One thing that I found really interesting, though, is that that will then give me a hash of commit hashes that point to the number of builds that are there. If I want to get the distribution for the whole project over the course of, say, the last week, and I want to say, "How many times do people run only one CI run versus running twice in the same commit versus running three times, or four times, or five or six times?" I want to see that distribution of how many times people are rerunning their build. You're effectively doing that tally process twice. So once you have a list of all the builds, you group by hash. You count, and so you end up with that. You have the Ruby hash of commit SHAs pointing to number of times the build was run on that. And then, you again group by the number of builds for each commit SHA. And so now what you have is you'll have something like one, and then that points to an array of SHA one, SHA two, SHA three, SHA four like all the builds. And then you tally that again, or you transform values, or however, you end up doing it. And what you end up with is saying for running only once, I now have 200 builds that ran only once. For running twice in the same commit SHA, there are 15. For running three times, there are two. For running four times, there is one. And now I've got my distribution broken down by how many times it was run. It took me a while to work through all of that. But now the shortcut in my head is going to be you double tally to get distribution. CHRIS: As an aside, the whole everything you're talking about is interesting and getting to that distribution. I feel like I've tried to solve that problem on data recently and struggled with it. But particularly tally, I just want to spend a minute because tally is such a fantastic addition to the Ruby standard library. I used to have in sort of like loose muscle memory transform value is grouped by ampersand itself, transform values count, sort, reverse to H. That whole string of nonsense gets replaced by tally, and, oof, what a beautiful example of Ruby, and enumerable, and all of the wonder that you can encapsulate there. JOËL: Enumerable is one of the best parts of Ruby. I love it so much. It was one of the first things that just blew my mind about Ruby when I started. I came from a PHP, C++ background and was used to writing for loops for everything and not the nice for each loops that a lot of languages have these days. You're writing like a legit for or while loop, and you're managing the indexes yourself. And there's so much room for things to go wrong. And being introduced to each blew my mind. And I was like, this is so beautiful. I'm not dealing with indexes. I'm not dealing with the raw implementation of the array. I can just say do a thing for each element. This is amazing. And that is when I truly fell in love with Ruby. CHRIS: I want to say I came from Python, most recently before Ruby. And Python has pretty nice list comprehensions and, in fact, in some ways, features that enumerable doesn't have. But, still, coming to Ruby, I was like, oh, this enumerable; this is cool. This is something. And it's only gotten better. It still keeps growing, and the idea of custom enumerables. And yeah, there's some real neat stuff in there. JOËL: I'm going to be speaking at RubyConf Mini this fall in November, and my talk is all about Enumerators and ranges in enumerable and ways you can use those to make the APIs of the objects that you create delightful for other people to use. CHRIS: That sounds like a classic Joël talk right there that I will be happy to listen to when it comes out. A very quick related, a semi-related aside, so, tally, beautiful addition to the Ruby language. On the Rails side, there was one that I used recently, which is where.missing. Have you seen where.missing? JOËL: I have not heard of this. CHRIS: So where.missing is fantastic. Let's assume you've got two related objects, so you've got like a has many blah, so like a user has many posts. I think you can...if I'm remembering it correctly, it's User.where.missing(:posts). So it's where dot missing and then parentheses the symbol posts. And under the hood, Rails will do the whole LEFT OUTER JOIN where the count is null, et cetera. It turns into this wildly complex SQL query or understandably complex, but there's a lot going on there. And yet it compresses down so elegantly into this nice, little ActiveRecord bit. So where.missing is my new favorite addition into the Rails landscape to complement tally on the Ruby side, which I think tally is Ruby 2.7, I want to say. So it's been around for a while. And where.missing might be a Ruby 7 feature. It might be a six-something, but still, wonderful features, ever-evolving these tool sets that we use. JOËL: One of the really nice things about enumerable and family is the fact that they build on a very small amount of primitives, and so as long as you basically understand blocks, you can use enumerable and anything in there. It's not special syntax that you have to memorize. It's just regular functions and blocks. Well, Chris, thank you so much for coming back for a visit. It's been a pleasure. And it's always good to have you share the cool things that you're doing at Sagewell. CHRIS: Well, thank you so much, Joël. It's been an absolute pleasure getting to come back to this whole Bike Shed. And, again, just to add a note here, you're doing a really fantastic job with the show. It's been interesting transitioning back into listener mode for the show. Weirdly, I wasn't listening when I was a host. But now I've regained the ability to listen to The Bike Shed and really enjoy the episodes that you've been doing and the wonderful spectrum of guests that you've had on and variety of topics. So, yeah, thank you for hosting this whole Bike Shed. It's been great. JOËL: And with that, let's wrap up. The show notes for this episode can be found at bikeshed.fm. This show is produced and edited by Mandy Moore. If you enjoyed listening, one really easy way to support the show is to leave us a quick rating or even a review in iTunes. It really helps other folks find the show. If you have any feedback, you can reach us at @_bikeshed, or reach me at @joelquen on Twitter, or at hosts@bikeshed.fm via email. Thank you so much for listening to The Bike Shed, and we'll see you next week. Byeeeeeeeeeee!!!!!!!! ANNOUNCER: This podcast was brought to you by thoughtbot. thoughtbot is your expert design and development partner. Let's make your product and team a success.

Nate Berkopec is the author of the Complete Guide to Rails Performance, the creator of the Rails Performance Workshop, and the co-maintainer of Puma. He talks with Steph about being known as "The Rails Speed Guy," and how he ended up with that title, publishing content, working on workshops, and also contributing to open source projects. (You could say he's kind of a busy guy!) Speedshop (https://www.speedshop.co/) Puma (https://github.com/puma/puma/commits/master?author=nateberkopec) The Rails Performance Workshop (https://www.speedshop.co/rails-performance-workshop.html) The Complete Guide to Rails Performance (https://www.railsspeed.com/) How To Use Turbolinks to Make Fast Rails Apps (https://www.speedshop.co/2015/05/27/100-ms-to-glass-with-rails-and-turbolinks.html) Sidekiq (https://sidekiq.org/) Follow Nate Berkopec on Twitter (https://twitter.com/nateberkopec) Visit Nate's Website (https://www.nateberkopec.com/) Sign up for Nate's Speedshop Ruby Performance Newsletter (https://speedshop.us11.list-manage.com/subscribe?u=1aa0f43522f6d9ef96d1c5d6f&id=840412962b) Transcript: STEPH: All right. I'll kick us off with our fancy intro. Hello and welcome to another episode of The Bike Shed, a weekly podcast from your friends at thoughtbot about developing great software. I'm Steph Viccari. And this week, Chris is taking a break. But while he's away, I'm joined by Nate Berkopec, who is the owner of Speedshop, a Ruby on Rails performance consultancy. And, Nate, in addition to running a consultancy, you're the co-maintainer of Puma. You're also an author as you wrote a book called The Complete Guide to Rails Performance. And you run the workshop called The Rails Performance Workshop. So, Nate, I'm sensing a theme here. NATE: Yeah, make code go fast. STEPH: And you've been doing that for quite a while, haven't you? NATE: Yeah. It's pretty much been since 2015, or so I think. It all started when I actually wrote a blog post about Turbolinks that got a lot of pick up. My hot take at the time was that Turbolinks is actually a good thing. That take has since become uncontroversial, but it was quite controversial in 2015. So I got a lot of pick up on that, and I realized I liked working on performance, and people seem to want to hear about it. So I've been in that groove ever since. STEPH: When you started down the path of really focusing on performance, were you running your own consultancy at that point, or were you working for someone else? NATE: I would say it didn't really kick off until I actually published The Complete Guide to Rails Performance. So after that came out, which was, I think, March of 2016…I hope I'm getting that right. It wasn't until after that point when it was like, oh, I'm the Rails performance guy now. And I started getting emails inbound about that. I didn't really have any time when I was actually working on the CGRP to do that sort of thing. I just made that my full-time job to actually write, and market, and publish that. So it wasn't until after that that I was like, oh, I'm a performance consultant now. This is the lane I've driven myself into. I don't think I really had that as a strategy when I was writing the book. I wasn't like, okay, this is what I'm going to do. I'm going to build some reputation around this, and then that'll help me be a better consultant with this. But that's what ended up happening. STEPH: I see. So it sounds like it really started more as a passion and something that you wanted to share. And it has manifested to this point where you are the speed guy. NATE: Yeah, I think you could say that. I think when I started writing about it, I just knew...I liked it. I liked the work of performance. In a lot of ways, performance is a much more concrete discipline than a lot of other sub-disciplines of programming where I joke my job is number go down. It's very measurable, and it's very clear when you've made a difference. You can say, “Hey, this number was this, and now it's this. Look what I did.” And I always loved that concreteness of performance work. It makes it actually a lot more like a real kind of engineering discipline where I think of performance engineering as clarifying requirements and the limitations and then building a project that meets the requirements while staying within those limitations and constraints. And that's often not quite as clear for other disciplines like general feature work. It's kind of hard to say sometimes, like, did you actually make the user's life better by implementing such and such? That's more of a guess. That's more of a less clear relationship. And with performance, nobody's going to wake up ten years from today and wish that their app was slower. So we can argue about the relative importance of performance in an application, but we don't really argue about whether or not we made it faster because we can prove that. STEPH: Yeah. That's one area that working with different teams (as I tend to shift the clients that I'm working with every six months) where we often push hard around feature work to say, “How can we measure this? How can we know that we are delivering something valuable to users?” But as you said, that's really tricky. It's hard to evaluate. And then also, when you add on the fact that if I am leaving that project in six months, then I don't have the same insights to understand how something went for that team. So I can certainly appreciate the satisfaction that comes from knowing that, yes, you are delivering a faster app. And it's very measurable, given the time that you're there, whether it's a short time or if it's a long time that you're with that team. NATE: Yeah, totally. My consulting engagements are often really short. I don't really do a lot of super long-term stuff, and that's usually fine because I can point to stuff and say, “Yep. This thing was at A, and now it's at B. And that's what you hired me to do, so now it's done.” STEPH: I am curious; given that you have so many different facets where you are running your consultancy, you are also often publishing a lot of content and working on workshops and then also contributing to open source projects. What does a typical week look like for you? NATE: Well, right now is actually a decent example. I have client work two or three days a week. And I'm actually working on a new product right now that I'm calling Sidekiq in Practice, which is a course/workshop about scaling Sidekiq from zero to 1000 jobs per second. And I'll spend the other days of the week working on that. My content is...I always struggle with how much time to spend on blogging specifically because it takes so much time for me to come up with a post and publish that. But the newsletter that I write, which I try to write two once a week, I haven't been doing so well with it lately. But I think I got 50 newsletters done in 2020 or something like that. STEPH: Wow. NATE: And so I do okay on the per-week basis. And it's all content I've never published anywhere else. So that actually is like 45 minutes of me sitting down on a Monday and being like rant, [chuckles] slam keyboard and rant and then hit send. And my open source work is mostly 15 minutes a day while I'm drinking morning coffee kind of stuff. So I try to spread myself around and do a lot of different stuff. And a lot of that means, I think, pulling back in terms of thinking how much you need to spend on something, especially with newsletters, email newsletters, it was very easy to overthink that and spend a lot of time revising and whatever. But some newsletter is better than no newsletter. And especially when it comes to content and marketing, I've learned that frequency and regularity is more important than each and every post is the greatest thing that's ever come out since sliced bread. So trying to build a discipline and a practice around doing that regularly is more important for me. STEPH: I like that, some newsletter is better than no newsletter. I was listening to your chat with Brittany Martin on the Ruby on Rails podcast. And you said something very honest that I appreciated where you said, “Writing is really hard, and writing sucks.” And that made me laugh in the moment because even though I do enjoy writing, I still find it very hard to be disciplined, to sit down and make it happen. And then you go into that editor mode where you critique everything, and then you never really get it published because you are constantly fixing it. It sounds like...you've mentioned you set aside about 45 minutes on a Monday, and you crank out some work. How do you work through that inner critic? How do you get past it to the point where then you just publish? NATE: You have to separate the steps. You have to not do editing and first drafting at the same time. And the reason why I say it sucks and it's hard is because I think a lot of people don't do a lot of regular writing, maybe get intimidated when they try to start. And they're like, “Wow, this is really hard. This is not fun.” And I'm just trying to say that's everybody's experience and if it doesn't get any better, because it doesn't, [chuckles] there's nothing wrong with you, that's just writing, it's hard. For me, especially with the newsletter, I just have to give myself permission not to edit and to just hit send when I'm done. I try to do some spell checking,, and that's it. I just let it go. I'm not going back and reading it through again and making sure that I was very clear and cogent in all my points and that there's a really good flow through that newsletter. I think it comes with a little bit of confidence in your own ideas and your own experience and knowledge, believing that that's worth sharing and that's worth somebody's time, even if it's not a perfect expression of what's in your head. Like, a 75% expression is good enough, especially in a newsletter format where it's like 500 to 700 words. And it's something that comes once a week. And maybe not everyone's amazing, but some of them are, enough of them are that people stay subscribed. So I think a combination of separating editing and first drafting and just having enough confidence and the basis where you have to say, “It doesn't have to be perfect every single time.” STEPH: Yeah, I think that's something that I learned a while back to apply to my coding process where I had to separate those two steps of where I have to let the creator in me just create and write some code and make it work, and then come back to the editing process, and taking a similar approach with writing. As you may be familiar with thoughtbot, we're big advocates when it comes to sharing content and sharing things that we have learned throughout the week and different projects that we're working on. And often when people join thoughtbot, they're very excited to contribute to the blog. But it is daunting for that first post because you think it has to be this really grand novel. And it has to be something that is really going to appeal to everybody, and it's going to help everyone. And then over time, you learn it's like, oh well, actually it can be this very just small thing that I learned that maybe only helps 20 people, but it still helped those 20 people. And learning to publish more frequently versus going for those grand pieces is more favorable and often more helpful for people. NATE: Yeah, totally. That's something that is difficult for people at first. But everything in my experience has led me to believe that frequency and regularity is just as, if not more important than the quality of any individual piece of content that I put out. So that's not to say that...I guess it's weird advice to give because people will take it too far the other way and think that means he's saying quality doesn't matter. No, of course, it does, but I think just everyone's internal biases are just way too tuned towards this thing must be perfect. I've also learned we're just really bad judges internally of what is useful and good for people. Stuff that I think is amazing and really interesting sometimes I'll put that out, and nobody cares. [chuckles] And the other stuff I put out that's just like the 45-minute banging out newsletter, people email me back and say, “This is the most helpful thing anyone's ever read.” So that quality bias also assumes that you know what is good and actually we're not really good at that, knowing every time what our audience needs is actually really difficult. STEPH: That's totally fair. And I have definitely run into that too, where I have something that I'm very proud of and excited to share, and I realize it relates to a very small group of people. But then there's something small that I do every day, and then I just happen to tweet about it or talk about it, and suddenly that's the thing that everybody's really excited about. So yeah, you never know. So share it all. NATE: Yeah. And it's important to listen. I pay attention to what people get interested in from what I put out, and I will do more of that in the future. STEPH: You mentioned earlier that you are working on another workshop focused on Sidekiq. What can you tell me about that? NATE: So it's meant to be a guide to scaling Sidekiq from zero to 1000 requests per second. And it's meant to be a missing guide to all the things that happen, like the situations that can crop up operationally when you're working on an application that does a lot of work with Sidekiq. Whereas Mike Sidekiq, Wiki, or the docs are great about how do, you do this? What does this setting mean? And the basics of getting it just running, Sidekiq in practice, is meant to be the last half of that. How do you get it to run 1,000 jobs per second in a day-to-day application? So it's the collected wisdom and collected battle scars from five years of getting called in to fix people's Sidekiq installations and very much a product of what are the actual problems that people experience, and how do you fix and deal with those? So stuff about memory and managing Sidekiq memory usage, how to think about queues. Like, what should your queue structure be? How many should you have? Like, how do you organize jobs into queues, and how do you deal with problems like some client is dropping 10,000, 20,000 jobs into a queue. And now the other jobs I put in that queue have 20,000 jobs in front of them. And now this other job I've got will take three hours to get through that queue. How do you deal with problems like that? All the stuff that people have come to me over the years and that I've had to help them fix. STEPH: That sounds really great. Because yeah, I find that teams who are often in this space with Sidekiq we just let it run until there's a fire. And then suddenly, we start to care as to how it's processing, and we care about our queue structure and how many workers that we have that are pulling from that queue. So that sounds really helpful. When you're building a workshop, do you often go back to any of those customers and pull more ideas from them, or do you find that you just have enough examples from your collective work with clients that that itself creates a course? NATE: Usually, pretty much every chapter in the workshop I've probably implemented like three-plus times, so I don't really have to go back to any individual customer. I have had some interesting stuff with my current client, Gusto. And Gusto is going through some background job reorganization right now and actually started to implement a lot of the things that I'm advocating in the workshop actually without talking to me. It was a good validation of hey, we all actually think the same here. And a lot of the solutions that they were implementing were things that I was ready to put down into those workshops. So I'd like to see those solutions implemented and succeed. So I think a lot of the stuff in here has been pretty battle-tested. STEPH: For the Rails Performance Workshop, you started off doing those live and in-person with teams, and then you have since switched to now it is a CLI course, correct? NATE: That's correct. Yep. STEPH: I love that very much. When you've talked about it, it does feel very appropriate in terms of developers and how we like to consume content and learn. So that is really novel and also, it seems like a really nice win for you. So then other people can take this course, but you are no longer the individual that has to deliver it to their team, that they can independently take the course and go through it on their own. Are you thinking about doing the same thing for the Sidekiq course, or what are your plans for that one? NATE: Yeah, it's the exact same structure. So it's going to be delivered via the command line. Although I would say Sidekiq in practice has more text components. So it's going to be a combination of a very short manual or book, and some video, and some hands-on exercises. So, an equal blend between all three of those components. And it's a lot of stuff that I've learned over having to teach; I guess intermediate to advanced programming concepts for the last five years now that people learn at different paces. And one of the great things about this kind of format is you can pick it up, drop it off, and move at your own speed. Whereas a lot of times when I would do this in person, I think I would lose people halfway through because they would get stuck on something that I couldn't go back to because we only had four hours of the day. And if you deliver it in a class format, you're one person, and I've got 24 other people in this room. So it's infinitely pausable and replayable, and you can go back, or you can just skip ahead. If you've got a particular problem and you're like, hey, I just want to figure out how to fix such and such; you can do that. You can just come in and do a particular thing and then leave, and that's fine. So it's a good format that way. And I've definitely learned a lot from switching to pre-recorded and pre-prepared stuff rather than trying to do this all live in person. STEPH: That is one of the lessons that I've learned as well from the couple of workshops that I've led is that doing them in person, there's a lot of energy. And I really enjoy that part where I get to see people respond to the content. And then I get a lot of great feedback from people about what type of questions they have, where they are getting stuck. And that part is so important to me that I always love doing them live first. But then you get to the point, as you'd mentioned, where if you have a room full of 20 people and you have two people that are stuck, how do you help them but then still keep the class going forward? And then, if you are trying to tailor this content for a wide audience…so maybe beginners could take the Rails Performance Workshop, or they could also take the Sidekiq course. But you also want the more senior engineers to get something out of it as well. It's a very challenging task to make that content scale for everyone. NATE: Yeah. What you said there about getting feedback and learning was definitely something that I got out of doing the Rails Performance Workshop in person like three dozen times, was the ability to look over people's shoulders and see where they got stuck. Because people won't email me and say, “Hey, this thing is really confusing.” Or “It doesn't work the way you said it does for me.” But when I'm in the same room with them, I can look over their shoulder and be like, “Hey, you're stuck here.” People will not ask questions. And you can get past that in an in-person environment. Or there are even certain questions people will ask in person, but they won't take the time to sit down and email me about. So I definitely don't regret doing it in person for so long because I think I learned a lot about how to teach the material and what was important and how people...what were the problems that people would encounter and stuff like that. So that was useful. And definitely, the Rails Performance Workshop would not be in the place that it is today if I hadn't done that. STEPH: Yeah, helping people feel comfortable asking questions is incredibly hard and something I've gone so far in the past where I've created an anonymous way for people to submit questions. So during class, even if you didn't want to ask a question in front of everybody, you could submit a question to this forum, and I would get notified. I could bring it up, and we could answer it together. And even taking that strategy, I found that people wouldn't ask questions. And I guess it circles back to that inner critic that we have that's also preventing us from sharing knowledge that we have with the world because we're always judging what we're going to share and what we're going to ask in front of our peers who we respect. So I can certainly relate to being able to look over someone's shoulder and say, “Hey, I think you're stuck. We should talk. Let me walk you through this or help you out.” NATE: There are also weird dynamics around in-person, not necessarily in a small group setting. But I think one thing I really picked up on and learned from RailsConf2021 which was done online, was that in-person question asking requires a certain amount of confidence and bravado that you're not...People are worried about looking stupid, and they won't ask things in a public or semi-public setting that they think might make them look dumb. And so then the people that do end up asking questions are sometimes overconfident. They don't even ask a question. They just want to show off how smart they are about a particular issue. This is more of an issue at conferences. But the quality of questions that I got in the Q&A after RailsConf this year (They did it as Discord chats.) was way better. The quality of questions and discussion after my RailsConf talk was miles better than I've ever had at a conference before. Like, not even close. So I think experimenting with different formats around interaction is really good and interesting. Because it's clear there's no perfect format for everybody and experimenting with these different settings and different methods of delivery has been very useful to me. STEPH: Yeah, that makes a ton of sense. And I'm really glad then for those opportunities where we're discovering that certain forums will help us get more feedback and questions from people because then we can incorporate that and to future conferences where people can speak up and ask questions, and not necessarily be the one that's very confident and enjoys hearing their own voice. For the Rails Performance Workshop, what are some of the general things that you dive into for that workshop? I'm curious, what is it like to attend that workshop? Although I guess one can't attend it anymore. But what is it like to take that workshop? NATE: Well, you still can attend it in some sense because I do corporate bookings for it. So if you want to buy 20 seats, then I can come in and basically do a Q&A every week while everybody takes the workshop. Anyway, I still do that. I have one coming up in July, actually. But my overall approach to performance is to always start with monitoring. So the course starts with goals and monitoring and understanding where you want to go and where you are when it comes to performance. So the first module of the Rails Performance Workshop is actually really a group exercise that's about what are our performance requirements and how can we set those? Both high-level and low-levels. So what is our goal for page load time? How are we going to measure that? How are we going to use that to back into lower-level metrics? What is our goal for back-end response times? What is our goal for JavaScript bundle sizes? That all flows from a higher-level metric of how fast you want the page to load or how fast you want a route to change in a React app or something, and it talks about those goals. And then where should you even start with where those numbers should be? And then how are you going to measure it? What are the browser events that matter here? What tools are available to help you to get that data? Because without measurement, you don't really have a performance practice. You just have people guessing at what stuff is faster and what is not. And I teach performance as a scientific process as science and engineering. And so, in the scientific method, we have hypotheses. We test those hypotheses, and then we learn based on those tests of our hypotheses. So that requires us to A, have a hypothesis, so like, I think that doing X makes this faster. And I talk about how you generate hypotheses using profiling, using tools that will show you where all the time goes when you do this particular operation of your software—and then measuring what happens when you do that? And that's benchmarking. So if you think that getting rid of method X or changing method X will speed up the app, benchmarking tells you did you actually speed it up or not? And there are all sorts of little finer points to making sure that that hypothesis and that experiment is tested in a valid way. I spend a lot of time in the workshop yapping about the differences between development/local environments and production environments and which ones matter. Because what differences matter, it's not often the ones that we think about, but instead it's differences like actually in Rails apps the asset packaging and asset pipeline performs very differently in production than it does in development, works very differently. And it makes it one of the primary reasons development is slower than production, so making sure that we understand how to change those settings to more production-like settings. I talk a lot about data. It's the other primary difference between development and production is production has a million users, and development has 10. So when you call things like User.all, that behavior is very different in production than it is locally. So having decent production-like data is another big one that I like to harp on in the workshops. So it's a process in the workshop of you just go lesson by lesson. And it's a lot of video followed up by hands-on exercises that half of them are pre-baked problems where I'm like, hey, take a look at this Turbolinks app that I've given you and look at it in DevTools. And here's what you should see. And then the other half is like, go work on your application. And here are some pull requests I think you should probably go try on your app. So it's a combination of hands-on and videos of the actual experience going through it. STEPH: I love how you start with a smaller application that everyone can look at and then start to learn how performant is this particular application that I'm looking at? Versus trying to assess, let's say, their own application where there may be a number of other variables that they have to consider. That sounds really nice. You'd mentioned one of the first exercises is talking about setting some of those goals and perhaps some of those benchmarks that you want to meet in terms of how fast should this page load, or how quickly should a response from the API be? Do you have a certain set of numbers for those benchmarks, or is it something that is different for each product? NATE: Well, to some extent, Google has suddenly given us numbers to work with. So as of this month, I think, June 2021, Google has started to use what they're calling Core Web Vitals in their ranking of search results. They've always tried to say it's not a huge ranking factor, et cetera, et cetera, but it does exist. It is being used. And that data is based on Chrome user telemetry. So every time you go to a website in Chrome, it measures three metrics and sends those back to Google. And those three metrics are Largest Contentful Paint (LCP), First Input Delay (FID), and Cumulative Layout Shift (CLS). And First Input Delay and Cumulative Layout shift are more important for your single-page apps kind of stuff. It's hard to screw those up with a Golden Path Rails app that just does Turbolinks or Hotwire or whatever. But Largest Contentful Paint is an easy one to screw up. So Google's line in the sand that they've drawn is 2.5 seconds for Largest Contentful Paint. So that's saying that from clicking on your website in a Google search result, it should take 2.5 seconds for the page to paint the largest component of that new page. That's often an image or a video or a large H1 tag or something like that. And that process then will help you to...to get to 2.5 seconds in Largest Contentful Paint; there are things that have to happen along the way. We have to download and execute all JavaScript. We have to download CSS. We have to send and receive back-end responses. In the case of a simple Hotwire app, it's one back-end response. But in the case of a single-page app, you got to download the document and then maybe download several XHR fetches or whatever. So there's a chain of events that has to happen there. And you have to walk that back now from 2.5 seconds in Largest Contentful Paint. So that's the line that I'm seeing getting drawn in the sand right now with Google's Core Web Vitals. So pretty much any meaningful web application performance metric can be walked back from that. STEPH: Okay. That's super helpful. I wasn't aware of the Core Web Vitals and that particular stat that Google is using to then rank the sites. I was going to ask, this kind of blends in nicely into when do you start caring about performance? So if you have a new application that you are just starting to get to market, based on the fact that Google is going to start ranking you right away, you do have to care some right out of the gate. But I am curious, when do you start caring more about performance, and are there certain tools and benchmarking that you want to have in place from day one versus other things that you'll say, “Well, we can wait until we have X numbers of users or other conditions before we add more profiling?” NATE: I'd say as an approach, I teach people not to have a performance strategy of monitoring. So if your strategy is to have dashboards and look at them regularly, you're going to lose. Eventually, you're not going to look at that dashboard, or more often, you just don't understand what you're looking at. You just install New Relic or Datadog or whatever, and you don't know how to turn a dashboard into actual action. Also, it seems to just wear teams out, and there's no clear mechanism when you just have a dashboard of turning that into oh, well, this has to now be something that somebody on our team has to go work on. Contrast that with bugs, so teams usually have very defined processes around bugs. So usually, what happens is you'll get an Exception Notification through Sentry or Bugsnag or whatever your preferred Exception Notification service is. That gets read by a developer. And then you turn that into a Jira ticket or a Kanban board or whatever. And then that is where work is done and prioritized. Contrast that with performance; there's often no clear mechanism for turning metrics into stuff that people actually work on. So understanding at your organization how that's going to work and setting up a process that automatically will turn performance issues into actual work that people get done is important. The way that I generally teach people to do this is to focus instead of dashboards and monitoring, on alerts, on automated thresholds that get tripped and then sends somebody's an email or put something in the Kanban board or whatever. It just has to be something that automatically gets fired. Different tools have different ways of doing this. Datadog has pretty much built their entire product around monitoring and what they call monitors. That's a perfectly fine way to do it, whatever your chosen performance monitoring tool, which I would say is a required thing. I don't think there's really any good excuse in 2021 for not having a performance monitoring tool. There are a million different ways to slice it. You can do it yourself with OpenTelemetry and then like statsD, I don't know, or pay someone else like everyone else does for Datadog or New Relic or AppSignal or whatever. But you got to have one installed. And then I would say you have to have some sort of automated alerting. Now that alerting means that you've also decided on thresholds. And that's the hard work that doesn't get done when your strategy is just monitoring. So it's very easy to just install a dashboard and say, “Hey, I have this average page time load dashboard. That means I'm paying attention to performance.” But if you don't have a clear answer to what number is good and what number is bad, then that dashboard cannot be turned into real action. So that's why I push monitoring so hard is because it allows people to ignore performance is all that matters, and it forces you to make the decision upfront as to what number matters. So that is what I would say, install some kind of performance monitoring. I don't really care what kind. Nowadays, I also think there's probably no excuse to not have Real User Monitoring. So there's enough GDPR compliance Real User Monitoring now that I think everyone should be using it. So for industry terms, Real User Monitoring is just performance monitoring in the browser. So it's just users' browser APIs and sends those back to you or your third-party provider, so having that so you actually are collecting back-end and front-end performance metrics. And then making decisions around what is bad and what is good. Probably everybody should just start with a page load time monitor, Largest Contentful Paint monitor. And if you've got a single-page app, probably hooking up some stuff around route changes or whatever your app...because you don't actually have page loads on every single time you navigate. You have to instrument whatever those interactions are. So having those up and then just drawing some lines that say, “Hey, we want our React route changes to always be one second or less.” So I will set an alert that if the 95th percentile is one second or more, I'm going to get alerted. There's a lot of different ways to do that, and everybody will have different needs there. But having a handful of automated monitors is probably a place to start. STEPH: I like how you also focus on once you have decided those thresholds and have that monitoring in place, but then how do you make it actionable? Because I have certainly been part of teams where we get those alerts, but we don't necessarily...what you just mentioned, prioritize that work to get done until we have perhaps a user complaint about it. Or we start actually having pages that are timing out and not loading, and then they get bumped up in the priority queue. So I really like that idea that if we agree upon those thresholds and then we get alerted, we treat that alert as if it is a user that is letting us know that a page is too slow and that they are unable to use our application, so then we can prioritize that work. NATE: And it's not all that dissimilar to bugs, really. And I think most teams have processes around correctness issues. And so, all that my strategy is really advocating for is to make performance fail loudly in the same way that most exceptions do. [chuckles] Once you get to that point, I think a lot of teams have processes around prioritization for bugs versus features and all that. And just getting performance into that conversation at least tends to make that solve itself. STEPH: I'm curious, as you're joining teams and helping them with their performance issues, are there particular buckets or categories of performance issues that are the most common in terms of, let's say, 50% of issues are SQL-related N+1 issues? What tends to be the breakdown that you see? NATE: So, when it comes to why something is slow in a Ruby application, I teach a method that I call DRM. And that doesn't have anything to do with actual DRM. It's just memorable because it reminds me of things I don't like. DRM stands for Database Ruby and Memory and in that order. So the most common issue is database, the second most common issue is issues with your Ruby code. The least common issue is memory. Specifically, I'm talking about allocation of objects, creating lots of objects. So probably 80% of your issues are in some way database-related. In Rails, it's 50% of those are probably N+1. And then 30% of database issues are probably what I would call unnecessary SQL. So it's not necessarily N+1, but it's a SQL query for information that you already had, or you could do in a more efficient way. So a common thing for unnecessary SQL would be people will filter an ActiveRecord::Collection like ten different ways when they could have just loaded the whole collection, filtered it with Ruby in the ten different ways afterwards, and that works really well if the collection that you're loading is like 10, 20. Turning that into one database query, plus a bunch of calls to innumerable methods is often way faster than doing that as ten separate database queries. Also, that tends to be a more robust approach. This doesn't happen in most companies, but what could happen is the database is like a shared resource. It's a resource that everybody is affected by. So a performance degradation to the database is the worst possible scenario because everything is affected. But if you screw up what's happening at an individual Rails process, then only that Rails process is affected. The blast radius is tiny. It's just that one request. So doing less stuff in the database while it can actually seem like, oh, that doesn't feel right. I'm supposed to do a lot of stuff in the database. It actually can reduce blast radiuses of performance issues because you're not doing it on this database that everyone has to have access to. There are a lot of areas of gray here. And I talk a lot in all my other material like why -- There's a lot of nuance here. So database is the main stuff. Issues in how you write your Ruby code is probably the other one. Usually, that's just what I would call code that goes bump in the night. It's code that you don't know is running but actually is. Profilers are what help us figure that out. So oftentimes, I'll have someone open up a profiler on their controller action for the first time. And they're like, wait a minute, I had no idea that such and such was running during this controller action, and actually, we don't need to do that at all. So why is it here? So that's the second most common issue. And then the third issue that really doesn't actually come up all that often is object allocation, numbers of objects that get created. So primarily, this is a problem in index actions or actions transactions that deal with big collections. So in Ruby, we often get overly focused on garbage collection, but garbage collection doesn't take any time if you just don't create objects. And object creation itself takes time. So looking at code through the lens of what object does this code create? And trying to get rid of those object allocations can often be a pretty productive way to make stuff faster. STEPH: You said a lot of amazing things there. So I'm debating on which one to follow up on. I think the one that stuck out to me the most where I have felt pain around this is you mentioned identifying code that goes bump in the night or code that is running, but it doesn't need to be run. And that is something that I've run into with applications where we have a code path that seems important, but yet I can't prove that it's being executed and exactly why it's there and what flow it's supporting. And I'm curious, do you have any tips or tricks in how you've helped teams identify that this code path isn't used and it's something that we can remove and then that itself will help speed up the performance of that particular endpoint? NATE: Like, there's no performance cost to like 100 models in an application that never actually get used. There's really no performance downside to code in an app that doesn't actually ever get run. But instead, what happens is code gets added into callbacks that usually is probably the biggest offender that's like, always do this thing after you do X. But then, two years later, you don't always need to do that thing after you do X. So the callbacks always run, but sometimes requirements change, and they don't always need to be run. So usually, it's enough to just pop the profiler now on something. And I have people look at it, and they're like, “I don't know why any of this is happening.” Like, it's usually a pretty big Eureka moment once we look at a flame graph for the first time and people understand how to read those, and they understand what they're looking at. But sometimes there's a bit of a process where especially in a bigger app where it's like, “Such and such is running, and this was an entire other team that's working on this. I have no idea what this even does.” So on bigger apps, there's going to be more learning that has to get done there. You have to learn about other parts of the application that maybe you've never learned about before. But profiling helps us to not only see what code is running but also what that relative importance is. Like, okay, maybe this one callback runs, and you don't know what it does, and it's probably unnecessary. But if it only takes 1% of the total time to run this action, that's probably less important than something that takes 20% of total time. And so profilers help us to not only just see all the code that's being run but also to know where that time goes and what time corresponds to what parts of the code. STEPH: Yeah, that's often the code that makes me the most nervous is where it's code that I suspect is being run or maybe being run, but I don't understand why it's there and then figuring out if it can be removed and then figuring out ways to perhaps even log when a call is being made to that code to determine if it's truly in use or not or at least supported by a code path that a user is hitting. You have a blog post that I read recently that I really appreciated that talks about essentially gaming benchmarking where you talk about the importance of having context around benchmarks. So if someone says, “I've improved something where it is now 10% faster.” It's like, well, what is that 10% relative to? And if it's a tool that other people are using, what does that mean for them? Or did you improve something that was already very fast, and you made it 10% faster? Was that a really valuable use of your time? NATE: Yeah. You know, something that I read recently that made me think of that again was this Hacker News post that went viral. That was like, how I optimize an AWS EC2 instance to take 1.5 million requests per second on my JSON API. And out of the box, it was like 500 requests per second, and then he got it to 1.5 million. And the whole article was presented with relative numbers. So it was like, “I made this change, and things got 33% faster. And if you do the whole thing right, 500 to 1.5 million requests per second, it's like my app is three times faster now,” or whatever. And that's true, but it would probably be more accurate to say, “I've taken three-millionth of a second out of every request in my app.” That's two ways of saying the same thing because latency and throughput are just related that way. But it's probably more accurate and more useful to say the absolute number, but it doesn't make for great blog posts, so that doesn't tend to get said. The kinds of improvements that were discussed in this article were really, really low-level stuff. That was like if you turn off...I think it was like turn off iptables or something like that. And it's like, that shaves a microsecond off of every time we make a syscall or something. And that is useful if your performance goal is to serve 1.5 million requests per second Hello World responses off of my EC2 instance, which is what this person admittedly was doing. But there's a tendency to walk that back to if I do all things in this article, my application will be three times faster. And that's just not what the evidence says. It's not what you were told. So there's just a tendency to use relative numbers when absolute numbers would be more useful to giving you the context of like, oh, well, this will improve my app or it won't. We get this a lot in Puma. We get benchmarks that are like, hey, this thing is going to help us to do 50,000 requests per second in Puma instead of 10,000. And another way of saying that is you took a couple of nanoseconds off of the overhead of every single request to Puma. And most Puma applications have a hundred millisecond response time. So it's like, yeah, I guess it's cool that you took a nanosecond off, and I'm sure it's going to help us have cool benchmarks, but none of our users are going to care. No one that's used Puma is going to care that their requests are one nanosecond faster now. So what did we really gain here? STEPH: Yeah, it makes sense that people would want to share those more...I want to call them sparkly stats and something that catches your attention, but they're not necessarily something that's going to translate to us in the way that we hoped that they will in terms of it's not going to speed up our app 30% or have those same rewards or benefits. Speaking of Puma, how is it being a co-maintainer of Puma? And how do you balance that role with all of your other work? NATE: Actually, it doesn't take all that much of my time. I try to spend about 15 minutes a day on it. And that's really possible because of the philosophy I have around open-source maintenance. I think that open source projects are fundamentally about collaboration and about sharing our hard-fought extractions and fixes and knowledge together. And it's not about a single super contributor or super maintainer who is just out of the goodness of their heart releasing all of their incredible work and time into the public domain or into a free software license. Puma is a pretty popular piece of Ruby software, so a lot of people use it. And I have things on my back burner of if I ever got 20 hours to work on Puma, here's stuff I would do. But there are a lot of other people that have more time than me to work on Puma. And they're just as smart, and they have other tools they've got in their locker that I don't have. And I realized that it was more important that I actually find ways to recruit and then unblock those people than it was for me to devote as much time as I could to Puma. And so my work on Puma now is really just more like management than anything else. It's more trying to recruit new contributors and trying to give them what they need to help Puma. And contributing to open source is a really fraught experience for a lot of people, especially their first time. And I think we should also be really conscious of that. Like, 95% of software developers have really never contributed to open source in a meaningful way. And that's a huge talent pool of people that could be helping us that aren't. So I'm less concerned about the problems of the 5% that are currently contributing than I am about why there are 95% of us that don't do anything. So that's what gets me excited to work on Puma now, is trying to change that ratio. STEPH: I really like that mindset of where you are there to provide guidance but then essentially help unblock others as they're making contributions to the project but then still be there to have the history and full context and also provide a path forward of a good direction for Puma to head. In regards to encouraging more people to contribute to open source projects, I've often heard people say how challenging that is, where they have an open-source project that they would really love people to contribute to but finding people is really hard or just letting people know that they're interested in contributions. Have you had any strategies that have been successful for you in encouraging people to contribute? NATE: Yeah. So first thing, the easiest thing is we have a contributing.md file. So that's something I think more projects should adopt is have an actual file in your project that says everything about how to contribute. Like, what kinds of contributions do you want? Different projects have different things that they want. Like, Rails doesn't want to refactor PRs. Don't send a refactor PR to Rails because they'll reject it. Puma, I'm happy to accept those. So letting people know like, “Hey, here's how we work here. Here's the community we're creating, and here's how it works. Here's how to get involved.” And I think of it as hanging out the shingle and saying, “Yes, I want your contributions. Here's how to do it.” That alone puts you a step above other projects. The second thing I would say is you need to have contributor-only communication channels. So we have Matrix chat. So Matrix is like this successor to IRC. So we have a chat channel basically, but it's like contributors only. I don't enforce that, but I just don't want support requests in there. I don't want people coming in there and being like, “My Puma config doesn't work.” And instead, it's just for people that want to contribute to Puma, and that want to help out. If you have a question and come in there, anyone can answer it. And then finally, another thing that I've had success with is doing one-on-one stuff. So I will actually...I have a Calendly invite that I think is in contributing.md now that you can just book 30 minutes with me anytime about contributing to Puma. And I will get on a Zoom call with you and talk to you about what are your concerns? Where do I think you can help? And I give my time away that way. The way I see it is like if I do that 20 times and I create one super contributor to Puma, that is worth more than me spending 10 hours on Puma because that person can contribute 100, 200, 1,000 hours over their lifetime of contributing to Puma. So that's actually a much more higher leverage contribution, really from my perspective. It's actually helping other people contribute more. STEPH: Yeah, that's huge to offer people to say, “Hey, you can book time with me, and I will walk you through and let you know where you can start making an impactful contribution right away,” or “Here are some areas that I think you'd be interested, to begin with.” That seems like such a nice onboarding for someone who says, “I'm interested, but I'm nervous,” or “I'm just not sure about where to get started.” Also, I love your complaint department voice for the person who their Puma config doesn't work. That was delightful. [chuckles] NATE: I think it's a little bit part of my open-source philosophy that, especially at a certain scale like Puma is at that we really kind of over-prioritize users. And I'm not really here to do support; I'm here to make the project better. And users don't actually contribute to open source projects. Users use the thing, and that's great. That's the whole reason we're open-sourcing is so more people use it. But it's important not to prioritize that over people who want to make the project better. And I think a lot of times; people get caught up in this almost clout chasing of getting the most GitHub stars that they think they need and users they think they need. And you don't get paid for having users, and the product doesn't get any better either. So I don't prioritize users. I prioritize the quality of the project and getting contributors. And that will create a better project, which will then create more users. So I think it's easy to get sidetracked by people that ask for your time when they're not giving anything back to the project in return. And especially at Puma's scale, we have enough people that want my time or the time of other maintainers at Puma so that they can contribute to the project. And putting user support requests ahead of that is not good for the project. It's not the biggest, long-term value increase we could be making, so I don't prioritize them anymore. STEPH: Yep. That sounds like more the pursuit of sparkly stats and looking for all those GitHub stars or all of those likes. Well, Nate, if you're game, I have two listener questions that I'd like to run by you because I shared with some folks that you are going to be on The Bike Shed today. And they're very excited and have two questions that they'd like me to run by you. How does that sound? NATE: Yeah, all right. STEPH: So the first question is, are there any paradigms or trends in Rails that inherently hurt performance? NATE: Yeah. I get this question a lot, and I will preface it with saying that I'm the performance guy, and I'm not the software design guy. And I get a lot of questions about does such and such software design...how does that impact performance? And usually, there's like a way to do anything in a performant way. And I'm just here to help people to find the performant way and not to prescribe “You must always do X, Y, or Z,” or “ActiveRecord is bad. Never use it.” That's not my job here. And in my experience, there's a fast way to do almost anything. Now, one thing that I think is dying, I guess, or one approach or one common...I don't know what to call it. One common mistake that is clearly wrong is to not do any form of server-side rendering in a web application. So I am anti-client-side app. But there are ways to do that and to do it quickly. But rendering a basically blank document, which is what most of these applications will do when they're using Rails as a back-end…you'll serve this basically blank document or a document with maybe some Chrome in it. And then, the client-side app has to execute, compile JavaScript, make XHR requests, and then render the page. That is just by definition slower than serving somebody a server-side rendered page. Now, I am 100% agnostic on how you want to generate that server-side rendering. There are some people that are working on better ways to do that with Rails and client-side apps. Or you could just go the Hotwire Turbolinks way. And it's more progressive enhancement where the back-end is always just serving the server-side rendered response. And then you do some JavaScript on top of that. So I think five years from now, nobody will be doing this approach of serving blank documents and then booting client-side apps into that. Or at least it will be seen as outdated enough that you should never design a project that way anymore. It's one of those few things where it's like, yeah, just by definition, you're adding more steps into a rendering flow. That means, by necessity, it has to be slower. So I think everybody should be thinking about server-side rendering in their project. Again, I'm totally agnostic on how you want to implement that. With React, whatever front-end flavor of the month you want to go with, there's plenty of ways to do that, but I just think you have to be prioritizing that now. STEPH: All right. Well, I like that five-year projection of where we're headed. I have found that it's often the admin-side where people will still bring in a lot of JavaScript rendering, just to touch on a bit of what you're saying, in terms of let's favor the server-rendered HTML versus over-optimizing a space that one, probably isn't a profitable space in terms that we do want our admins to have a great experience for our product. But if they are not necessarily our users, then it also doesn't need to be anything that is over the top or fancy or probably uses a lot of JavaScript. And instead, we can start simple. And there's a number of times that I've been on projects where we have often walked the admin back to be more server-rendered because we got to a point where someone was very excited to make the admin very splashy and quick but then couldn't keep up with the requests because then they were having to prioritize the user experience first. So it was almost like optimizing the admin, but then it got left out in the cold. So then it's just sort of this poor experience. NATE: Yes. Shopify famously walked back their admin from I think it was Backbone to Turbolinks. And I think that that has now moved back to React is my understanding. But Shopify is a huge company, so they have plenty of time and resources to be able to do that. But I just remember that happening at the time where I was like, oh wow, they just rolled the whole thing back to Turbolinks again. And now, with the consolidation that's gone on in the React world, it's a little bit easier to pipe a server-side rendering into a React app. Whereas with Backbone, it was like no one knew what you were doing. So there was less knowledge about how to server-side render this stuff. Now it doesn't seem to be so much of a problem. But yeah, I mean, Rails is really good at CRUD apps, and admin is like 99% CRUD. And adhering as closely as possible to the Rails Golden Path there in an admin seems to be the most productive way to work on that kind of feature. STEPH: All right. Ready for your second question? NATE: Yes. STEPH: Okay. This one's a bit more in-depth. They also mentioned a particular project name. So I am going to swap it out with a different name. So on project cinnamon roll, we found a really gnarly time-consuming API endpoint that's getting hammered. And on a first pass, we addressed a couple of N+1 issues and tuned the performance, and felt pretty confident that they had addressed the issue. But it was still fairly slow. So then they took some additional incremental steps. So they swapped out to use OJ for serialization that shaved off an additional 10% but was still slow. They also went the route of going straight to Rails cache with a one-minute expiration. So that way, they could avoid mucking with cache busting because they confirmed with the client that data could be slightly stale. And this was great. It worked out well. So it dropped their average response time down to less than 70 milliseconds. With all that said, that journey took a few hours over a few days, and multiple production deploys. And had they gone straight to the cache, then they would have had a 15-minute fix with a single deploy. So this person's wondering, are there any other examples like that where, rather than taking these incremental seemingly obvious performance whims, there are situations where you want to be much more direct with your path? NATE: I guess I'd say that profiling can help you to understand and form better hypotheses about what will make things faster and what won't. Because a profiler can't really lie to you about where time goes, either you spent 20% of your time in this method, or you didn't. So I don't spend any time in any of my material talking about what JSON serializer you use. Because really that's actually never...that's really never anybody's bottleneck. It's never a huge proportion of people's total percentage of time. And I know that because I've looked at enough profiles that the issues are usually in other places. So I would say that if your hypotheses that you're generating are not working, it's because you're not generating good enough hypotheses. And profiling is the place to do that. So having profilers running in production is probably the biggest level-upscale-like that most teams could take. So having profilers that you can access as on production servers as a user is probably the biggest level up that you could make to generating the hypotheses because that'll have real production data, real production servers, real production environment. And it's pretty common now that pretty much every team that I work with either has that already, or we work on implementing it. It's something that I've seen in production at GitHub and Shopify. You can do it yourself with rack-mini-profiler. It's all about setting up the authorization, just making sure that only authorized users get to see every single SQL query generated in the flame graph and all that. But other than that, there's no reason you shouldn't do it. So I would say that if you're not generating the right hypotheses or you don't...if the last hypothesis out of 10 is the one that works, you need better hypotheses, and the best way to do that is better profiling. STEPH: Okay, better profiling. And yeah, it sounds like there's also a bit of experience in there in terms of things that you're used to seeing, that you've noticed that could be outliers in terms of that they're not necessarily the thing that you want to improve. Like you mentioned spending time on how you're serializing your JSON is not somewhere that you would look. But then there are other areas that you've gained experience that you know would be likely more beneficial to then focus on to form that hypothesis. NATE: Yeah, that's a long way of saying experience pays off. I've had six years of doing this every single day. So I'm going to be pretty good at...that's what I get paid for. [laughs] So if I wasn't very good at that, I probably wouldn't be making any money at it. STEPH: [laughs] All right. Well, thanks, Nate, so much for coming on the show today and talking so much about performance. On that note, I think it's a good place for us to wrap up. If people are interested in following along with what you're working on and they want to keep up with your latest and greatest workshops that are coming out, where can they find you on the internet? NATE: speedshop.co is my site. @nateberkopec on Twitter. And speedshop.co has a link to my newsletter, which is where I'm actively thinking every week and publishing stuff too. So if you want to get the drip of news and thoughts, that's probably the best place to go. STEPH: Perfect. All right. Well, thank you so much. NATE: No problem. STEPH: The show notes for this episode can be found at bikeshed.fm. CHRIS: This show is produced and edited by Mandy Moore. STEPH: If you enjoyed listening, one really easy way to support the show is to leave us a quick rating or a review in iTunes as it helps other people find the show. CHRIS: If you have any feedback for this or any of our other episodes, you can reach us @bikeshed on Twitter. And I'm @christoomey. STEPH: And I'm @SViccari. CHRIS: Or you can email us at hosts@bikeshed.fm. STEPH: Thanks so much for listening to The Bike Shed, and we'll see you next week. Together: Bye. Announcer: This podcast was brought to you by thoughtbot. thoughtbot is your expert design and development partner. Let's make your product and team a success.

Surprise Gore Heads, this episode is more of an introduction to our new show than our old. Not really a new show so to speak but more of a rebranding of XHR into a new show. Same humor, same fun, same us, new look but we're now called "The GoreHead Podcast".  Here are some links for you Spotify https://open.spotify.com/show/1mz8GdIlBNS7fpRLZdFyIl Anchor https://anchor.fm/gorehead-podcast --- Send in a voice message: https://anchor.fm/extremehorrorreplay/message

This week Steph's taking a quick break, but while she's off, Chris is joined by a special guest - Jonathan Reinink. Jonathan is the creator of Inertia.js. Inertia.js (https://inertiajs.com/) lets you quickly build modern single-page React, Vue and Svelte apps using classic server-side routing and controllers, and listeners of the show will certainly have heard Chris rave about it on previous episodes. Chris and Jonathan dig into what makes Inertia unique as compared to frameworks like Phoenix LiveView, Laravel Livewire, and Rails' Hotwire & Turbo. They also discuss how Inertia embraces the URL, the unique "protocol" nature of Inertia, and how to consider Inertia alongside native mobile applications. Throughout the conversation, Jonathan's consistent philosophy of wanting to build robust, performant, and delightful applications shines through. Jonathan Reinink on twitter (https://twitter.com/reinink) Inertia.js (https://inertiajs.com/) Eloquent Performance Patterns (https://eloquent-course.reinink.ca/) TailwindCSS (https://tailwindcss.com/) Church Social (https://churchsocial.com/) Jonathan on Full Stack Radio (https://twitter.com/adamwathan/status/1192152495407222784?s=21) Foundational blog post: Server-side apps with client-side rendering (https://reinink.ca/articles/server-side-apps-with-client-side-rendering) Laravel Livewire (https://laravel-livewire.com/) Alpine.js (https://github.com/alpinejs/alpine) Phoenix LiveView (https://github.com/phoenixframework/phoenix_live_view) Hotwire Turbo (https://turbo.hotwire.dev/) The Inertia Protocol (https://inertiajs.com/the-protocol) Transcript: CHRIS TOOMEY: I am seeing what I believe to be the relevant things. JONATHAN REINIK: Let's dance. CHRIS: Hello and welcome to another episode of The Bike Shed, a weekly podcast from your friends at thoughtbot about developing great software. I'm Chris Toomey. And this week, Steph is taking a quick break, but while she's away, I'm joined by a special guest, Jonathan Reinik. Jonathan Reinik is the creator of Inertia.js. And listeners of the show will know that that is increasingly one of my favorite frameworks and, frankly, just ways to build applications on the internet. Jonathan is also the creator of the Eloquent Performance Patterns course, which teaches the Eloquent ORM, which is the ORM in Laravel but really digs into deep performance and database things, so really covering that back end as well. Jonathan also collaborated on the development of Tailwind CSS, a utility-first CSS framework which again is something that I have spoken of in very high terms on this podcast. And lastly, Jonathan currently runs his own SaaS business called Church Social. So really, Jonathan is a bit of a quadruple threat covering back end and front end design and entrepreneurship. So pretty much everything you want to see. And frankly, I've been so impressed by the breadth and the depth of Jonathan's work and just the deep way that he is thinking about building applications. So I am absolutely thrilled to have him on the show today. So without further ado, Jonathan, thank you so much for joining me. JONATHAN: Thanks so much, Chris. That was a very kind introduction, and yeah, it's awesome to be on The Bike Shed. I've been a long-time listener, and as I've said to you, I really appreciate the support that you've given to my work over the years. So yeah, it's awesome to be here. CHRIS: That's interesting. We're measuring it in years now, but it's a very sincere thing for me. I think with Inertia, you've built something that is both very unique and a special approach to how we build things, but it's also built from very familiar pieces and allows us to reuse the deep amounts of knowledge that we have in the Rails community or the Laravel community. But actually backing up just a little bit, because we're going to dive deep on Inertia.js today, for folks that are not as familiar or have only heard me mention it in passing, there's a wonderful episode of Full Stack Radio where Jonathan and Adam Wathan talked about Inertia.js, and I think gave a very good foundational summary. So we'll link to that in the show notes in case anyone wants to dig in a little bit more. Likewise, Jonathan has a really fantastic blog post called Server-side Apps With Client-side Rendering, which, as far as I can tell, is the manifesto that began this whole journey for you. And I really love that you have done so much of the work in public, and people can see the history of how this idea has evolved and really crystallized into what now is a very production-ready framework in sort of the way to build things. But I would love to hear right now just for anyone who is not as familiar and also just to hear how you summarize it at this point in time. What does your introductory elevator pitch for Inertia.js sound like in April 2021? JONATHAN: That's a great question. So it's hard to answer this without unpacking a lot of different things. And you mentioned the podcast with Adam; I think that's good because it goes in a lot of the technical detail of how Inertia works and why I created it. But the elevator pitch these days for when I talk to someone about it, it's generally I explain it as a way of building modern web apps. And in particular, when I say modern, I mean web apps that have a lot of JavaScript, so frameworks like Vue or React or Svelte, so applications that are built using those tools. And the key thing that Inertia offers is for you to develop these modern applications without having to first build an API sort of the way historically if you ever wanted to use one of these modern web stacks like Vue or React or Svelte, you could use them within Laravel or Rails by inserting them into your Vue or into your Blade templates or your ERB templates. But it was difficult if you ever wanted to turn it into a legitimate single-page application. And anytime you would ask that question, if you go out on the internet and say, “Hey, how do I build SPA with, say, Vue and Laravel or React and Rails?” The answer was always, “Well, you need to build an API. You need to build an API.” That was always the missing piece because that's the way that everyone in the Jamstack era that we're in that's the way that everyone's building their applications that are like these heavy client-side applications. And I totally get the need for those style apps and the place for those style apps. But I really missed this way that you could build an application with Rails or with Laravel where you could just literally spin up a new app, create some routes within your server-side framework, create some controllers, create some views, and have a working application within minutes really. You could have something being displayed on the screen within minutes with these classic monolith applications. And if you wanted to do the same thing, if you wanted to get an app up and running in minutes with Vue or React as your completely client-side SPA scenario, it just wasn't working because as soon as you say, “Well, in order to do that, you're going to need to have a back end Rails or a Lavarel application, and then a client-side Vue or React application. And then you're going to have to create this API that connects the two together.” There's just a lot more work that goes into that. It's not only the work of actually creating the API; I find a lot of the decisions that come along with building an API; it's like, okay, what does the abstraction look like? Am I going to build it with REST, or am I going to build it as a GraphQL API? And all the decisions that come along with designing and architecting that which again has its place. But there's just something awesome about saying, “Here's a new route. Here's the view that I want to render. And here's the data that I want that view to have,” and just go off and do it, and it's done. And some people ask me, “Well, with Inertia, if you're not building an API, what happens someday if you need an API?” And they frame it like, well, this is a terrible decision. You should be starting with an API. But for me, the reality is that so many of the web applications that I was building and that I've seen other people building is they had already made the decision not to use an API because they had already made the decision that they wanted to use Laravel as a monolith app that had their controllers and the routes and their views all within that and the same thing with Rails. So if you've made that decision to build a monolith app with Laravel or Rails, you've already made the decision to not build it with an API. I was coming in from the other way. It's like, I just want to build an app the way I've always built in Laravel, and I don't want to have to build this API. Of course, there are times where you do need an API, which I think we're going to talk about maybe a little bit later if I don't ramble on too long, where it does make sense to have an API. But yeah, that's kind of the elevator pitch. I think maybe to close off that thought is that I really, really enjoy having a tight coupling between my routes and my data layer and my views, which, again, I appreciate that. That probably sounds like blasphemy in modern web development. But for me, I think it's so empowering when you say, “Hey, I have a controller that's given me some data, and I have a view that's rendering the data, and those two know about each other, and those two depend on each other.” You can work so fast because I'm not thinking, okay, well, I have this API endpoint that returns a user, and that has their first name and their last name and their email. But I also need to think about it in the situation in the future where I might need this attribute or that attribute or some other attribute and make sure I have all that figured out ahead of time or at least have a way to add it in later. And all of that thinking that goes into designing an API, I find that that adds a lot of overhead. And then maybe related to that also is the amount of times that you're rendering a view within your application that needs data from multiple different places. And to me, this is like one of the huge performance benefits that you get with a tool like Inertia is with, say a REST API, GraphQl solves this, but with a REST API, you're often getting too much data for what you actually need for the page, or you're often making more than one HTTP request because you say, “Well, on this particular dashboard, I need some user information. So I have to hit the user endpoint. I need maybe the latest product sales data, so I need to hit that endpoint.” And you're dealing with these performance issues that you get with a REST API that with Inertia, you don't have that problem because it's just going back to classic Laravel Blade views or Rails and ERB templates. Am I saying that right, ERB template? CHRIS: Mm-hmm. JONATHAN: In those situations, you say, “Well, if I need data from three different places, well, I'll just grab data from three different places and send it to the view, and that's fine. And I can do that in the most efficient way and get the data that I need specifically for that view.” So anyway, that's some of the thinking that drove me to build Inertia and some of the things that I was going for. And yeah, it was an evolution. It really came out of me using Turbolinks and really appreciating what Turbolinks gave me but taking it to that next step where it's like Turbolinks, except it's built with the same principles as Turbolinks but built for modern client-side frameworks like Vue and React. CHRIS: Yeah, that all feels very familiar to me. And in my experience, I've now worked with Inertia on a handful of projects, but in particular, I have just a small personal app that I use to manage different aspects of my life. And it's been my playground for different technologies. And I've migrated it through a bunch of different versions where it used to just be a Rails app. And I was like, oh no, the thing that I need to do to be on the cutting edge is to turn this into -- it's a Rails app on the back end with an API, and then it's a React app. It's separately deployed, but those two talk to each other. And what you were talking about of the deep coupling, I think that coupling exists whether we want it to or not. And so embracing that and revisiting when I eventually migrated that application to use Inertia and the client-side stuff folded back into the core codebase. Now deployments all go out in sync. And that turns out to actually be a really nice thing and a non-trivial thing to solve otherwise. As a developer of one on this particular project, the amount of complexity that was removed from the app when I switched it over to Inertia was amazing. I got to remove client-side routing. I got to remove client-side state management, which I think I was using Redux at the time. I got to remove some form helpers that I had. I think I might've been using Formik or React Use Former, any of those. But there are so many different little pieces that you ended up cobbling together to make an application. And it was amazing to me as I moved to Inertia, where I was like; actually, I don't need any of those, and routes suddenly are defined in one place in Rails in a familiar way. But things like redirects all work -- It feels just like a Rails app but with the extra abilities that a front-end client-side framework gives you when you want those when you need those. But otherwise, it really does feel like I'm rendering to an ERB template. It just happens to be that that template is rendering on the client-side and is written in React or Svelte or whatever the front-end framework is. But it almost feels like progressive enhancement. I'm borrowing a term, and it's not actually applicable here, but it really feels like that. It's like, oh, it's a Rails app, but I just want to make it a little bit fancier, and Inertia does that in such a fantastic way. And actually pivoting just a bit, as far as I can tell, there seems to be an explosion of thinking in this space. There are a handful of frameworks, namely Laravel Livewire, which is often paired with Alpine.js. Elixir has Phoenix LiveView, and then Rails has the Hotwire suite, which Turbo and Stimulus are the most pointed considerations. But interestingly, I think all of those frameworks, which I think are trying to provide a very similar experience, tend to keep things on the server-side, so using the Laravel Blade templates or the ERB in Rails. But you've taken the different approach to say, “No, let's embrace this front-end technology where it makes sense.” And again, there are a lot of pieces that can fall in, and I don't need the Redux and the React Router and all of those things but still use that client-side framework to be the rendering engine. And so I'm intrigued if you can talk a little bit more about that and that trade-off because I think it really differentiates Inertia and its approach. I personally found it to be fantastic, but I'd love to hear a little bit more about your thinking on that. JONATHAN: The thing about modern websites and web apps, in particular, is it doesn't matter how you slice it; we need JavaScript. So if you disagree with me there, then everything I say from this point on will not make sense to you. But I think we can all agree that modern web apps need JavaScript. JavaScript is the programming language of the web of the browser that allows us to do whatever magical things that we want to do. And if you look at tools like Phoenix LiveView, Laravel Livewire, and even the new Turbo stuff from the folks over at Basecamp, they all are embracing JavaScript in the same way. It's just they're framing it in a different way. I would say, especially with Livewire and LiveView, they're almost creating like an abstraction between the server and the way you write things on the server and the client-side. And they're almost hiding the JavaScript, which is really, really cool. I think it's such an interesting thing to try to do where somebody who's not familiar with JavaScript and not familiar with Vue, React, and Svelte can go and do things, write server-side code that gets rendered server-side. And then there's some JavaScript that these libraries insert that allow you to do more interesting things, whatever those things might be, show a drop-down, or drag and drop or validate a form or submit a form without actually submitting it fully to the server but submit it over XHR instead, all these kinds of things. But the point is they're embracing JavaScript just in a different way. And same with Turbo, Turbo gives you a way to write JavaScript for an application that mostly has server-side rendered HTML. So I think it's important to just recognize that JavaScript is key in all these frameworks. With Inertia, it's the same idea in that Inertia wants to embrace that classic way of building applications using the classic server-side monolith application framework like Laravel or Rails. But the difference is it acknowledges or embraces these existing client-side frameworks that have really grown in popularity. And the three, again, I keep mentioning them, Vue, React, or Svelte. Svelte being an up-and-coming one that's not nearly as popular yet, but it seems to be gaining a lot of steam. CHRIS: It's on the rise. That's my long [inaudible 14:46] JONATHAN: Yes, and people keep saying that. So anyway, Inertia basically said, “Hey, we want to keep building server-side apps. We want to keep building monolith apps similar to these other tools, except what we're going to do is we're going to embrace the fact that there's this really, really amazing tooling that's been developed for the client-side.” And it just doubles down on that. So for me, the reason that I ended up here was because, in my own SaaS application, it was a Laravel application that started with mostly Blade views initially. And then, over the years of building it, which has been many, many years, I've slowly added more and more Vue components within my app. And initially, the way I did it is those Vue components would just be inserted in as regular HTML tags in my server-side rendered templates. And then, when the page renders, those Vue components would boot up and do whatever they need to do. So for me, when I was building Inertia, I had already fallen in love with Vue, in particular, and having all the power of these client-side frameworks. And there is so much there. It's not just Vue, React, and Svelte; it's all the amazing tooling that's available out there that you can add on top of it. And this is the thing I often tell people that Inertia isn't -- we say right on the homepage, “Inertia isn't a framework.” And the reason why I say that is because I don't want people to think of Inertia as an alternative to Vue, React, and Svelte. Do you know what is a better way to frame it? It's actually more of an alternative to React Router or Vue Router; that's really more what it is, where you can say, “All my routings are handled server-side,” and that has all kinds of interesting implications. But it's more of a router, and it just so happens to pass along that routing control over to the server. Anyway, so that's really for me what differentiates Inertia from those other tools is because it's really doubled down on these client-side frameworks. And I think the reason why Inertia has been relatively popular is because people know Vue and people know React. And when it comes to then working with Inertia, it's not some new thing that they have to learn. It's an existing set of tools that they're already super comfortable with. And in so many ways, when you're building an Inertia app, you're kind of building a classic Vue app or a classic React app or a classic Svelte app. It's just there's a bunch of pieces missing. Like you said, it's like a bunch of the client-side state management stuff, which nobody likes anyway, is gone. The other thing that's gone is client-side routing. You don't have this back-end routing is over here now, and client-side routing is over here, and I have two different routing definitions. It's like, no, that's all just server-side now in one place. The other amazing thing you get is you mentioned redirects and that whole HTTP layer you get just along with Inertia for free because it's just part of your server-side stack. And one key aspect of that is auth. You can just use good old-fashioned nothing is better than session auth. Like, it just works. And, so whatever your typical solution for doing session auth in Laravel or Rails or whatever server-side framework you're using, all this stuff just works. So anyway, coming full circle on your question, the reason why Inertia has gone this way is because I really think that there's a huge amount of value with using these modern frameworks. And we just doubled down on using them. CHRIS: Yeah, that resonates with my experiences using Inertia and in contrast to the other frameworks. Everyone seems to be trying to get to the same place of providing a mechanism to have more almost app-like functionality but still using the traditional server-side technologies. But I think Inertia has chosen an approach to that that is unique in that category and really has provided a fantastic outcome. I've been very frankly surprised by the fidelity of experience and how app-like I can get something to feel when building with Inertia while still using all the same technologies. And the fact that I can use just traditional server-side auth and redirects and things like that is just so nice, and everything feels right. There's an experience that I've had on many applications that are, say, a React client-side bundle that gets sent down and then boots up, and then the layout starts to render. And as its data fetching, it gets like a 402 response or something like that in that data fetching. And then it's like, oh no, I need to hard redirect you over here on the client-side to this other page. And there's this junk of semi-filled-out layout, and then suddenly you're on the login page. And again, with Inertia, it looks like a normal server-side rendered app, but it isn't in the ways that really matter to us. And it is one of those things where the more I played with it, the more there's an experience of interacting with Inertia that it surprises me consistently how nice it is to work with it and yet it's so much easier to maintain an application using it. I know I'm raving here, but I am really a big fan of this for everyone listening in the audience. JONATHAN: [laughs] CHRIS: And actually to continue on one of the things you were saying there, one of the things that stands out to me in Inertia is the way that it embraces URLs and to a certain degree, that seems like a purposeful thing, but it also seems like it just naturally falls out of how Inertia works because we're no longer using a client-side state management technology, the way to manipulate state is through the URL. If you want to see a different version of the to-do list you're looking at when you click on that link, you change the URL and the state changes in response to that. And so everything is fundamentally kept in sync, but URLs are very much at the center of the architecture, and I really love that so much. I think URLs are often forgotten in client-side frameworks or underserved or underused. And it turns out in my experience as a user and both having served many users, people love to command-click on links. They love to right-click open a new tab. They love to be able to reload and see the same thing on the screen when they reload the page. They love to be able to bookmark. These are all really wonderful things that come out of working on the web. And the fact that Inertia has a pit of success around having URLs and have that be the way that we drive state is just so fantastic. So I'm wondering how much of that was very purposeful on your part versus how much of that fell out of the architecture. JONATHAN: That is very much something that fell out of the architecture. I say that not to say that I don't value URLs; I absolutely did. That's the way every single one of my Laravel built apps worked. It always starts in the route file. You hit the route file, you define a new route, and it goes from there. So I absolutely think that the URLs are critical. But the fact that it just ended up working out so nicely was, yeah, I'm going to say it was a bit of luck, a bit of coincidence. I find this is what's so interesting when you start pushing on a new way of things; you initially don't really know where it's going to end. It's like you have some ideas of how the tool can work and where it might go, but I think there's a lot of unknowns that you just figure out after a while. So the thing I said earlier actually about the fact that Inertia in a lot of ways is like a client-side router; it's, it's a routing library, to put it that way. I had been working on Inertia for a year and a half, and then a buddy of mine, Taylor Otwell, the creator of Laravel, he and I were chatting, and he said to me at one point he's like, “Oh, you know what? Inertia is actually super simple. It's really just a routing library.” And it was like, bam. It was kind of that moment; it's like, oh yeah, I hadn't thought of it like that at all. But when he said that, it made a ton of sense to me. So it's just this interesting progression the more you work on something, and the more you push on the edges, you learn what's possible and what it even is. I had this interesting experience, so remembering that Inertia came from Turbolinks. So I had my whole app built with Laravel ton of server-side rendered templates with Blade with view mixed in. And I had the SPA mode by clicking around using Turbolinks. So when I decided to try building Inertia, I removed Turbolinks, and all these requests now happened over XHR but using this preset JSON structure that powers Inertia. I really, in my mind, had this idea that it was only for GET requests, for GET visits; it was just for that. So the initial version of Inertia, there was no Inertia.post or Inertia.put or anything like that. It just wasn't something I even thought was possible. But then I remember, and this is often how it goes; I was out for a hike that day to get away from the computer for a little while and just let your brain drift; I'm sure you can relate to that. I was like, wait a minute; I could totally just support POST, PUT, PATCH, DELETE. And that was such an aha moment for me where I just realized that it was so much more than what I originally thought it was. And then the whole thing from that I remember it was a bit of like a waterfall effect after that where I remember running home out on that hike and hacked it together, and then it was like, okay, well, if I submit a form using POST, well, okay, I'm on the create user page. And I submit this form using Inertia.post to the user's endpoint. I'm like, well, how do I now end up back at the user index page or whatever page, maybe the user edit page. I'm like, wait a minute, I can just return a redirect back to the user index page, and it's literally going to return an Inertia response from the user index page. And then the way Inertia works is it dynamically swaps the page component client-side. And it was just like, oh, this is way too cool. And this really drives my thinking now that it's become a little bit more clear to me is that it really it's all based on HTTP using headers and normal HTTP stuff like redirects are such a critical piece of the story. But to me, that's super neat that, in a way, it's like a throwback to the fundamentals of the web and the browser and the fact that Inertia can just use those things,,, and it doesn't have to be fancy in a lot of the ways. It can just rely on those existing core pieces of the browser. So, yeah. CHRIS: It really is interesting to me how it feels like progressive enhancement in that way where you're building on top of these core fundamentals of HTTP and requests and redirects and status codes and things of that nature. Particularly interesting to me was it took me a while, I'm going to be honest, to figure out forms and particularly validation errors in Inertia. And that is entirely my fault. You have absolutely fantastic documentation. I am so impressed by the quality and the density of the documentation that you have that really covers everything. If we're being honest, I hadn't read the page, but I was doing form posting and then the subsequent errors and how you deal with that. I was doing it in a very traditional Rails way which if we're being honest, that is not a fundamental of how HTTP works. Rails just chose an option of oh, if you POST but we don't create the object because there's a validation error, then we're going to stay on the URL of the POST, so the collection route, but we're going to re-render the form in line. And that's a choice that Rails made that is interesting because at that point, if a user reloads the page, then things are weird, and it's not going to reload. They're not going to see the same thing after that reload, or it's going to try to repost or et cetera, et cetera. There's a bunch of edge cases there that sort of fall out. Whereas with Inertia, you end up redirecting back, and there's this interesting handshake of the errors, but from an end-user experience, it is absolutely fantastic where you stay on the form; the URL does not change. Technically, there's a POST and a redirect back under the hood, but Inertia just handles all of that for you. And you end up with sort of in-line validation errors. But you don't clear out any fields, and there are just wonderful things that fall out of it that again took me a while to get there, but it was another one of those oh, wow, this just naturally falls out of the architecture, but it's so nice and such a nice incremental advance on top of frankly, the stuff that I was doing in Rails historically. JONATHAN: So the way that Laravel works and it's always worked this way is when you make a request using POST or PATCH or DELETE or whatever to an endpoint, and that endpoint does its validation in the event that that validation fails, this is just like built-in standard like Laravel Stock behavior. It automatically redirects you back to the endpoint that you were on. So if you're on the create page or the edit page, it automatically redirects. That's just Laravel behavior. And what it does is it takes those errors that come out of the validator, it flashes them to the session, and then when the forum page reloads, you have those errors available to you in the session. Now, of course, if you're building like a classic server-side rendered application and you redirect now back to your form, you have to repopulate old form inputs, which is not a lot of fun, which you don't have any of that stuff with Inertia because Inertia allows you to preserve your state. But anyway, that's a separate thing. But for me, it's like you build a tool a little bit like in your own silo and the world that you know, and for me, that's Laravel. But there are also ideas that you get that just come from the tooling that you use, and the fact that Taylor Otwell made that decision in Laravel at one point is absolutely what now dictates how is the go-to way to do it in Inertia, just because it works so nicely. CHRIS: I wonder if there's been any consideration in the Rails world to adopt that because I think from an experience perspective, it feels like it's a better thing. It feels like it has the same robustness and guarantees that I would expect. But yeah, that's interesting. It makes sense that that was just naturally there because, again, it didn't feel like the obvious correct thing that Rails was doing. It was always a little bit odd and so interesting that Laravel was already there. But then Inertia can take it that one step further. But taking a slightly higher level view of all of this, one of the things that's really interesting about Inertia to me, especially in contrast to some of the other frameworks that we've been talking about like the Livewires in the LiveViews is Inertia is almost at its core a protocol more than it is…it's a sum of pieces, and with Inertia, you have a server-side adapter, so there's the Rails adapter and the Laravel adapter. And then, on the client-side, you have a separate either Vue or React or Svelte. So those are the officially supported ones on both sides, but there's also been a swell of community support. And so there's a Django one, which I'm not sure if it's currently maintained, but I just saw a Clojure one the other day. There's a Java Spring Boot. So those are all server-side adapters. I haven't seen as much on the client-side, but I imagine there are at least a handful of them out there. And it's so interesting to me that there's this core idea that you define this protocol of communicating back and forth from the server to the client and now this collection of things that are growing around that. And I wonder again, how much was that purposeful versus how much did that just happen? And then further to add a second question to complicate things, how are you thinking about managing that community? Because my sense is that this could allow for Inertia to be so much of a bigger tent and really bring in the best ideas from all of these different communities and end up with something at the core of this Inertia thing that is the best of every community and all of that. So yeah, a lot of questions there, but I'll hand it over to you because I'm super interested. JONATHAN: So I think when I first got going, it was Laravel and Vue; those were the tools that I worked with. And often, the best software and the best open-source software in my mind comes out of trying to solve something for your own needs. So that's really where Inertia came from and specifically for Laravel and Vue. But I quickly realized early on that it didn't have to be just a Vue and Laravel thing. So intentionally early on, I had this idea of trying to build it with multiple adapters, and I had this idea that you could build as many server-side adapters as you want and as many client-side adapters as you want, and maybe we'll officially maintain a certain amount of those, which is what we do right now. We officially maintain the Vue, the React, the Svelte adapters. And then, we also maintain officially the Laravel and the Rails server-side adapters. So that was, I would say, pretty intentional. And it's crazy how many server-side adapters people have been able to put together. Somebody wrote a ColdFusion server-side adapter for Inertia. I had no idea ColdFusion was even a thing anymore; yeah, legit. There are node ones; there are Phoenix ones; if you can believe it, there's a WordPress one, which I'm not even totally sure even how that works. There is ASP.NET. CHRIS: [chuckles] JONATHAN: Like, there's a whole bunch of them. And it's actually despite of me, not because of me, that this has happened because I am yet to write a good here's how to build an Inertia server-side adapter in the language and framework of your choice guide. It's been on my to-do list. I have a bunch of things I want to do. So it's still something I want to write, but people what they're doing is they're just reverse engineering what we're doing in Laravel and Rails and these other adapters, and they're figuring out how to do it in their own server-side language and framework. So that's been really, really cool. On the flip side, on the client side, I'm starting to realize more and more that that's actually where the most important work for us as the maintainers of Inertia that we need to focus our efforts on because it's non-trivial to create these client-side adapters. And for us, we actually have four of them now because we have React and Svelte, but then we have Vue 2 and Vue 3. And they're different enough those frameworks that we actually had to create a separate adapter. So that's really where all our work is. The core of Inertia is actually ridiculously short, like the whole file, like the whole core Inertia adapter is 150 to 200 lines of code. And maybe it's a bit more than that, but it was that for a long time. It might be 300 or 400 now. It's very short. Even honestly, the client-side adapters are pretty short too. It's just that it's more difficult to make these client-side adapters because you get to learn all the intricacies of how each one of these frameworks handles their rendering. The core behavior that Inertia uses is the fact that you can dynamically swap components. So we dynamically swap page components when you visit from one page to the next and the details that come along with that. Anyway, so I've realized that moving forward, my job is going to be to make sure that the client-side adapters are awesome and then letting the community drive the server-side adapters a little bit more and providing some better guides on how to do that. But yeah, for now, it's like if we can get it working in Laravel and Rails, we should be able to get that functionality working in any server-side adapter. And because it's all again just based on HTTP, that's the language, that's the protocol like you say. That's the thing that matters between all these web frameworks, which they all, of course, support since they're web frameworks. CHRIS: I think you're not giving yourself nearly enough credit for the support that you've given to the server-side frameworks because you do actually have a page in the documentation called the protocol that does a great job of at least summarizing it at that HTTP level. But at the end of the day, again, like the job of someone implementing it is to then map that into their given language and framework of choice. But yeah, the documentation is impressive in just how much you put in there and how much care you obviously put into it and lots of nice, subtle details that are covered very well in that. So that again, if you read it, unlike me, then you get to know everything; eventually, I got there. I think I've read the whole thing now. But there's a lot there, and you cover all of the details. But actually looping back to a topic that you hinted at earlier, but this is something that I've been pressing up against lately is I absolutely love building web apps in Inertia, but there's often the need to bring in a mobile app, and we want native mobile for various reasons. I love the idea of progressive web apps, and I want to push that envelope as much as I can. But as an example, right now, iOS does not support push notifications to PWA. So if that's a key feature that we want, then we're dead in the water or if there are certain GPS things. There are a bunch of true platform native things that we just can't get. And so I'm now contemplating building out an app alongside my Inertia web stuff, but I want to build a React Native app, and I'm wondering, to a certain degree, does this invalidate some of my ideas? I know you hinted at this earlier, but I think I'm still convinced of the utility of Inertia on the web. But I think I need a different paradigm to build for a mobile app, and I'm trying to decide where that line falls. I'm also wondering if I can just get away with embedding a bunch of web views and reusing my web logic because, again, if I'm building all of this, I'm going to build it in a mobile responsive way. I don't want to rebuild the core page functionality of my app just to put it on mobile. Maybe mobile folks would tell me I'm wrong there, but I'm interested in maybe wrapping it and getting access to those platform features. But yeah, I'm interested in what your thoughts are there. JONATHAN: Well, embedding a web view within a native app has been proven to work just as DHH, obviously. But yeah, there are definitely people who disagree with that approach and feel like you should build a legitimately native app. Let's say that we're going to legitimately build a real native app. We want to have an Android and iOS app. So I actually ran into this myself for my own SaaS application, and I solved it by building a native app using React Native so React Native obviously being an abstraction on top of iOS and Android and all the tooling there, which is such an amazing platform. It was just a real joy to work with. And I don't even hardly work with React, and I was able to get a nice, high-quality native feeling app built relatively easily. But I had to come to grips with this very question because, like I've been saying all along like, “Inertia is great because you don't need to build an API. Yay, this is amazing. This is what you should do. Oh, crap. I need an API.” And I had those questions like, okay, well, does this invalidate everything that I've been doing? So I was thinking about it, and in the end, what I did is I just built a light API alongside my Inertia application. So what it is is I think I have seven endpoints, and they're just REST endpoints that are designed specifically for my native app. And this works honestly so well. And I think I've explained to you a little bit in a previous conversation, so I'll repeat myself a little bit here for the benefit of the listeners. The reason why I think it's completely legitimate to have Inertia and build your entire web app that way and then have a companion API alongside it in the same monolith app (let's be clear: it's in the same application. It's in my Laravel app, or it would be in your Rails application) is because it just extends a core principle for me of what Inertia is. And that core principle is a tight coupling between my data layer so my controllers, and my views. So if we take that thinking where we say, well, Inertia in an Inertia web app when we have an endpoint, we hit the controller, we load data from the database, we pass that very specific data to the view, which is Vue or React or Svelte and it renders it. And there's a very tight coupling between the two. And I treated my native app in the exact same way. I said, “Okay, I need an API because obviously, the native app on iOS and Android has to make an HTTP request to get this data somehow. But instead of trying to create this super generalized API that could theoretically be used for anything, I use the same principle that said I'm going to allow myself to create an API that has a really tight coupling between the screens in my native apps and the actual data that's coming from those API endpoints. And this worked out really, really, really well. I don't have to deal with a lot of the issues that you run into when trying to create a more generalized API because I could just say, “Hey, I have this calendar page, and I want that calendar page in my particular app. I want it to show people's birthdays, and I want it to show wedding anniversaries, and I want it to show custom events and these things that we have called schedule reminders.” So it's data from four different endpoints. I didn't try to say, “Well, I'm going to go and create now my event's endpoint, and my birthday's endpoint, and my anniversary's endpoint, and my schedule reminder's endpoint,” and now have all that work to do in my native app to okay, we'll hit all these different endpoints and merge it all together; it wasn't like that at all. I created a calendar endpoint that returned all the data that's needed for that screen. And I basically applied that thinking through my whole native app, and it was really a joy to work this way. So I think that approach works really well if you have an app that doesn't have complete feature parity with your web app. And I think if you had a native app that needed absolute feature parity between the native app and the web app, then my thinking might be a little bit different on this. But in my experience, so often, native apps have a vastly reduced subset of the features that the web app has in particular, even if not for the core functionality of the application but just for the administrative side of it. There's a whole bunch of stuff that you tend to have in a web app around administrative stuff that you literally never need. And I mean administrative both in terms of it's a multi-tenant style app, which most apps are so in terms of the user's administrative functionality and in terms of the system level, the software owner administration. If you build your whole web app to be built on top of an API, all that administrative stuff that really doesn't need to exist in both places, you now have to make it exist in your API because you've made that decision to build it that way. Whereas if you just stick with Inertia on the web and just build it using that classic monolith way where you get data from the controls and send it to your Blade views or in this situation, client-side page views, and then you just expose the stuff that you actually need natively, for me personally, it's worked out so well. And if I look at my own web app, the amount of controllers that I have for the whole web app, I have like 100; it's a very big app. And for my native app, I have about 10. So that was like, I'm so glad that I didn't have to create 100 of these in both places. And then some people will be like you might be thinking, well, now I have duplication. I have duplication in some of my API endpoints and my web endpoints, and that's true. I would say first that duplication isn't always a bad thing. I think more duplication in our web apps would actually probably lead -- I feel like we run away from duplication too quickly. I don't think duplication is as bad as software developers often think it is. But even then, if with the duplication you can't live with yourself, there are still ways to solve duplication. So Laravel, for instance, has this concept of they're called API resources, which is basically they're essentially transformers. You give it some models, and it transforms that model into some other states, some other design. So there's nothing stopping you. And I even did this myself within my server-side application within Laravel to have an API resource to have a transformer that's used by both my Inertia controller and my API controller in a couple of situations and for me, only when it makes sense. I'm not going to do it all the time because I found that most often, I wanted the data in a slightly different format in my native apps than I'd want it into my web app. So quite often, that didn't happen. But I'm just saying if you're scared of duplication, there are totally ways to solve it. And we can solve this in our existing frameworks. Laravel or Rails has ways to allow us to abstract some of that stuff and reuse it in multiple places. So, yeah, that's my long-winded answer to how I've approached doing the native apps sort of thing. I think that tight coupling between the data and the screen I think that's a really nice thing, and you just can build faster. And just like you can build faster with Inertia on the website, you can build fast [inaudible 43:19] CHRIS: Frankly, that answer makes a ton of sense one and two, makes me feel better about the path that I'm on because, again, I'm really desperate to cling to Inertia for the web side of things. So I love what you're saying. And again, it really resonates with me and how you're thinking about building. There's also I really appreciate a subtle common theme that I've seen in a bunch of things that you've said where you're like; let me poke at best practices a bit and see what falls out. What if we were to actually embrace the coupling between our data and our view layer? And it's like, actually some really nice things happen there. And actually, going back to an earlier project that you worked on, Tailwind CSS is one of those projects that when you first see it, you're like, well, that's obviously wrong. That's definitely an incorrect way to do things. But then you explore it, and you're like, well, I mean, I know there are trade-offs here, but actually, in my experience and I'm sure in your experience, Tailwind is absolutely fantastic. And the trade-offs you totally win in the long game, and it's maintainable, and it's understandable. And you can continue to develop on top of it in a way that I've never found with any other CSS framework. But again, at first glance, you're like, ooh, that's not right. That can't be right. JONATHAN: 100%, exactly. I think it's fun to push back and just experiment with different things. And for me, I think a lot of my decisions, too, come back to the fact that I'm running a SaaS application as one person, and I need to be able to move fast. I don't want to have two different servers and two different repos. I want to be able to build my applications as fast as I can, as a single developer, a single founder. And so I think the things that I push against and try and experiment with come out of me trying to find the simplest ways to maintain things. So Tailwind, that's really Adam's brainchild. I came along in the first six months or so; me and him built it. I was really just helping him flesh out his idea there, and that was super fun. But yeah, I had the exact same experience as you. Adam was telling me about this, and I'm like, that sounds pretty terrible. Like, I have CSS figured out already. And then it was like, oh man, this is amazing. Fun little fact, my SaaS app, me and him, were both working on web apps at that time. So my SaaS app was one of the first Tailwind applications ever because I and Adam were literally both building our own apps while building Tailwind CSS. But anyway, so yeah, it comes out of not me trying to be like, I know better than other people; it's not that at all. It's more just I'm trying to find a way to survive as a business and trying to build at the same time, not only survive but also I want to build awesome products. I don't want to build software that is just kind of okay. I love striving to make software that's just exceptional that delights people that works the way someone expects it to work. And I just think that there's so much broken software out there. There's a lot of bad software. And don't get me wrong, I've created a lot of bad software, too. But I really try to hold myself to a high standard. And really, for me, that comes down to not necessarily what some purist says that “This is how you need to do it.” It comes more down to like, okay, let me see the results. How fast does the webpage open? What's the performance? You mentioned my course earlier. I'm really, really interested in database performance and how to use databases more intelligently to deliver really fast web applications. And that matters to me because customers hate waiting. They hate it. And that was even part of what drove me to create Inertia because I hated this. I was working for a company, and we had built the right way where we have an API and the client separate And we went down that road. And that was a big team with 20 to 30 developers in the end. And I was just like -- I shouldn't say, “I was,” but we, in general, were not happy with what happened because just the way that the app was built and the way that single views were hitting the API. You could probably argue that this was like we were doing something wrong, but the paradigm didn't lend us to doing it right, in my opinion. So we'd have pages that were hitting the REST API with sometimes 10, 20 HTTP requests just to get the data. And you're dealing with all the loading states of all this stuff. And of course, there were probably better ways to design, but we were trying to ship a product there too. We were trying to get it out the door and make happy customers. And I didn't feel like that way was helping us. I think GraphQL, just as an aside, is a huge step forward where you can say, “Hey, here's all my data in an API, but I'm not going to hit the user's endpoint just to get back whatever you decide to give me.” I can be much more intentional about saying, “Hey, I want this data and then pull in this relationship for that data and this other piece of data.” And I think that's really, really cool. But I think the problem there again is you need to build that GraphQL API, and that's non-trivial, not to mention you probably have to figure out OAuth, which is pretty much always a game-stopper for me because if I never have to work with OAuth in my life [laughs] I'll be totally okay with that. I know it has its place, but yeah. CHRIS: There's a clear passion and a desire that you're describing there to just build good things and the belief that it can be done. And then, as someone who has really benefited from your work, I thank you for carrying that torch and for pushing the envelope. And like you said, having that high standard and holding yourself to it but then hopefully bringing the rest of us along, and I really appreciate that. But I think with that, that's probably a perfect time to wrap up. If folks want to follow more of what you are working on, where can they find what you're up to on the internet? JONATHAN: I'm on Twitter, the classic place to go for following someone in tech, so twitter.com/reinink, my last name. That's R-E-I-N-I-N-K. So that's where even if I have stuff shared elsewhere on the web, that's where it starts. CHRIS: Perfect. We'll include links to your Twitter as well as everything else that we've mentioned in this episode in the show notes. So folks that do want to keep up or investigate further listening to that other podcast episode that I mentioned will have all of that available. But with that, thank you so much for your time, and yeah, again, really appreciate you joining. JONATHAN: Thanks so much, Chris. Pleasure to be here. CHRIS: The show notes for this episode can be found at bikeshed.fm. If you enjoyed listening, one really easy way to support the show is to leave us a quick rating or even a review on iTunes, as it really helps other folks find the show. If you have any feedback for this or any of our other episodes, you can reach us at _bikeshed on Twitter. Or you can reach me @christoomey, or you can e-mail hosts@bikeshed.fm. Thanks so much for listening to The Bike Shed, and we'll see you next week. This podcast was brought to you by thoughtbot. thoughtbot is your expert design and development partner. Let's make your product and team a success.

Angular is a feature-rich and opinionated framework. Opinionated, for example, in terms of fetching data via XHR. Most, and perhaps all, Angular developers reach for the HttpClientModule. Most of us don't have to even think about that decision. While the debate will likely never end over JavaScript frameworks, Angular developers may never end their debate over what forms implementation to use. Do you use reactive forms (which are arguably not really reactive) or template-driven forms? The Angular show panelists want to take you on a learning journey into the depths of this debate in a healthy, fun, and educational way.If you read the Angular documentation you might be inclined to believe that Angular's template-driven forms approach is where you start, but real Angular devs use reactive forms. I mean, the name reactive is hip. And template-driven forms, that's so AngularJS-y. Why would I want to use template-driven forms?We have the perfect guest to answer that question for you. Ward Bell, a Google Developer Expert in Angular and President/Co-Founder at IdeaBlade, has put template-driven forms through its paces for many years, and in our opinion, is one of the foremost experts on using template-driven forms in Angular. Grab your kombucha, coffee, running shoes, or really whatever you want, and join us as we learn from the master on template-driven forms. By the end of the show, you might be asking yourself why you aren't already using template-driven forms in your Angular applications.Bonus notes from Ward after the show:I talked through a couple of approaches during the show but I didn't have an actual sample to refer to... I just updated the StackBlitz sample with an example of both approaches. Color me “obsessive”.https://stackblitz.com/edit/angular-kkatri?file=src%2Fapp%2Fhero-form%2Fhero-form.component.htmlThe hero now has both a “Power” and a “Power Qualifier” (category and subcategory). Both are required. You pick a power and the list of qualifiers changes accordingly. A special “Select a power qualifier” appears until you pick a qualifier. This demonstrates the “disabled control” approach.One of the selectable powers is “Other”. It has no pre-defined qualifiers. Instead, you must enter a free-form description of the “other power”; what you enter is bound to the hero's qualifier field.When you pick “Other”, the “Power Qualifier” selector is removed from the form and a required free-form input control takes its place. This is the alternative “ngIf” approach that I mentioned.Click the “New” button to see how it works when Hero has no power.Show NotesAbstractControl: https://angular.io/api/forms/AbstractControlConnect with us:Aaron Frost - @aaronfrostJennifer Wadella - @likeOMGitsFEDAYBrian Love - @brian_loveWard Bell - @wardbell

Take a trip Down Under with this brilliant little gem from Australia. In his first ever appearance on XHR, the King of Creepy Crap, Daniel from The Creepy Crap Podcast is here. Filled with brutal kills, excellent bad guys and an actual original story, this film rocks. --- This episode is sponsored by · Anchor: The easiest way to make a podcast. https://anchor.fm/app --- Send in a voice message: https://anchor.fm/extremehorrorreplay/message Support this podcast: https://anchor.fm/extremehorrorreplay/support

In this Hasty Treat, Scott and Wes talk about how to develop for slow internet connections. LogRocket - Sponsor LogRocket lets you replay what users do on your site, helping you reproduce bugs and fix issues faster. It’s an exception tracker, a session re-player and a performance monitor. Get 14 days free at logrocket.com/syntax. Show Notes 02:26 - Types of Slow connections Slow upload Slow download This could be that your kids are watching a show and your partner is on a zoom call High latency Intermittent 07:47 - Loading UI Show loading UI for everything, but after X seconds Page transitions on SPAs Form switches - enter country, get list of provinces/shipping/whatever 11:58 - Account for XHR failures Sometimes I’m in a loading state forever State machines will account for this 13:26 - “Still working” timeouts Some sort of timeout so the user can get feedback Links Sentry Sapper Backblaze Tweet us your tasty treats! Scott’s Instagram LevelUpTutorials Instagram Wes’ Instagram Wes’ Twitter Wes’ Facebook Scott’s Twitter Make sure to include @SyntaxFM in your tweets

In today's episode, I talk about something new I learned last week: that `fetch()` can actually be more performant than XHR for making API calls.In today's episode, I talk about something new I learned last week: that fetch() can actually be more performant than XHR for making API calls.Show Notes & Transcript →

In today's episode, I talk about something new I learned last week: that `fetch()` can actually be more performant than XHR for making API calls. Links My switch from XHR to fetch() - https://gomakethings.com/how-to-use-the-fetch-api-with-vanilla-js/ Alex Russell on why fetch() can be faster - https://twitter.com/slightlylate/status/1185017213268287488

Welcome back to anothe episode of Benched with Bubba. On BwB EP 163 Bubba will be joined by James McCool. They will talk about the long list of Fantasy Baseball injuries, dig into Jame's new xHR stat and then talk about some pitchers that have some positive or negative regression coming. Rundown- MLB News- Lucas Giolito, Marco Estrada, Nick Pivetta, Blake Swihart, Harrison Bader & Tyler O'Neil, Greg Bird, David Robertson, Freddy Peralta, Blake Snell, Tyler Skaggs New xHR Stat- all kind of conversation Pitchers for positive/negative regression- Corbin Burnes, Reynaldo Lopez, Miles Mikolas, Aaron Nola, Jack Flaherty, Corey Kluber, Brad Keller, Matt Boyd, Luis Castillo, Tyler Glasnow, Trevor Williams As always, thanks for listening to another episode of Benched with Bubba. We'd appreciate it if you could give a rating and review on iTunes.

Welcome back to anothe episode of Benched with Bubba. On BwB EP 163 Bubba will be joined by James McCool. They will talk about the long list of Fantasy Baseball injuries, dig into Jame's new xHR stat and then talk about some pitchers that have some positive or negative regression coming.Rundown-MLB News- Lucas Giolito, Marco Estrada, Nick Pivetta, Blake Swihart, Harrison Bader & Tyler O'Neil, Greg Bird, David Robertson, Freddy Peralta, Blake Snell, Tyler SkaggsNew xHR Stat- all kind of conversationPitchers for positive/negative regression- Corbin Burnes, Reynaldo Lopez, Miles Mikolas, Aaron Nola, Jack Flaherty, Corey Kluber, Brad Keller, Matt Boyd, Luis Castillo, Tyler Glasnow, Trevor WilliamsAs always, thanks for listening to another episode of Benched with Bubba. We'd appreciate it if you could give a rating and review on iTunes.

Welcome back to anothe episode of Benched with Bubba. On BwB EP 163 Bubba will be joined by James McCool. They will talk about the long list of Fantasy Baseball injuries, dig into Jame's new xHR stat and then talk about some pitchers that have some positive or negative regression coming. Rundown- MLB News- Lucas Giolito, Marco Estrada, Nick Pivetta, Blake Swihart, Harrison Bader & Tyler O'Neil, Greg Bird, David Robertson, Freddy Peralta, Blake Snell, Tyler Skaggs New xHR Stat- all kind of conversation Pitchers for positive/negative regression- Corbin Burnes, Reynaldo Lopez, Miles Mikolas, Aaron Nola, Jack Flaherty, Corey Kluber, Brad Keller, Matt Boyd, Luis Castillo, Tyler Glasnow, Trevor Williams As always, thanks for listening to another episode of Benched with Bubba. We'd appreciate it if you could give a rating and review on iTunes.

A new episode of "Stadiumx XHR" be sure to subscribe to kick off your weekend with the hottest dance music and DJs from across the globe. This show contains a Stadiumx selection of the E.D.M. finest and there latest updates, tracks and remixes.

We'd like to thank you all the support and love this year! Enjoy this special mix with the best Stadiumx songs and remixes from 2018! Happy New Year!

A new episode of "Stadiumx XHR" be sure to subscribe to kick off your weekend with the hottest dance music and DJs from across the globe. This show contains a Stadiumx selection of the E.D.M. finest and there latest updates, tracks and remixes.

A new episode of "Stadiumx XHR" be sure to subscribe to kick off your weekend with the hottest dance music and DJs from across the globe. This show contains a Stadiumx selection of the E.D.M. finest and there latest updates, tracks and remixes.

Panel Brendan Eich Joe Eames Aaron Frost AJ ONeal Jamison Dance Tim Caswell Charles Max Wood Discussion 01:57 – Brendan Eich Introduction JavaScript [Wiki] Brendan Eich [Wiki] 02:14 – Origin of JavaScript Java Netscape Jim Clark Marc Andreesen NCSA Mosaic NCSA HTTPd Lynx (Web Browser) Lou Montulli Silicon Graphics Kernel Tom Paquin Kipp Hickman MicroUnity Sun Microsystems Andreas Bechtolsheim Bill Joy Sun-1 Scheme Programming Language Structure and Interpretation of Computer Programs – 2nd Edition (MIT Electrical Engineering and Computer Science) by Harold Abelson, Gerald Jay Sussman & Julie Sussman Guy Steele Gerald Sussman SPDY Rob McCool Mike McCool Apache Mocha Peninsula Creamery, Palo Alto, CA Main () and Other Methods (C# vs Java) Static in Java, Static Variables, Static Methods, Static Classes 10:38 – Other Languages for Programmers Visual Basic Chrome Blacklist Firefox 12:38 – Naming JavaScript and Writing VMs Canvas Andrew Myers 16:14 – Envisioning JavaScript’s Platform Web 2.0 AJAX Hidaho Design Opera Mozilla Logo Smalltalk Self HyperTalk Bill Atkinson HyperCard Star Wars Trench Run 2.0 David Ungar Craig Chambers Lars Bak Strongtalk TypeScript HotSpot V8 Dart Jamie Zawinski 24:42 – Working with ECMA Bill Gates Blackbird Spyglass Carl Cargill Jan van den Beld Philips Mike Cowlishaw Borland David M. Gay ECMAScript Lisp Richard Gabriel 31:26 – Naming Mozilla Jamie Zawinski Godzilla 31:57 – Time-Outs 32:53 – Functions Clojure John Rose Oracle Scala Async.io 38:37 – XHR and Microsoft Flash Hadoop Ricardo Jenez Ken Smith Brent Noorda Ray Noorda .NET Shon Katzenberger Anders Hejlsberg NCSA File Formats 45:54 – SpiderMonkey Chris Houck Brendan Eich and Douglas Crockford – TXJS 2010 Douglas Crockford JavaScript: The Good Parts by Douglas Crockford TXJS.com ActionScript Flex Adobe E4X BEA Systems John Schneider Rhino JScript roku Waldemar Horwat Harvard Putnam Math Competition Chris Wilson Silverlight Allen Wirfs-Brock NDC Oslo 2014 JSConf Brendan JSConf Talks 59:58 – JavaScript and Mozilla GIP SSLeay Eric A. Young Tim Hudson Digital Styles Raptor Gecko ICQ and AIM PowerPlant CodeWarrior Camino David Hyatt Lotus Mitch Kapor Ted Leonsis Mitchell Baker David Baren Phoenix Tinderbox Harmony 1:14:37 – Surprises with Evolution of JavaScript Ryan Dahl node.js Haskell Elm Swift Unity Games Angular Ember.js Dojo jQuery react ClojureScript JavaScript Jabber Episode #107: ClojureScript & Om with David Nolen MVC 01:19:43 – Angular’s HTML Customization Sweet.js JavaScript Jabber Episode #039: Sweet.js with Tim Disney TC39 Rick Waldron 01:22:27 – Applications with JavaScript SPA’s Shumway Project IronRuby 01:25:45 – Future of Web and Frameworks LLVM Chris Lattner Blog Epic Games Emscripten Autodesk PortableApps WebGL 01:29:39 – ASM.js Dart.js John McCutchen Monster Madness Anders Hejlsberg, Steve Lucco, Luke Hoban: TypeScript 0.9 – Generics and More (Channel 9, 2013) Legacy 01:32:58 – Brendan’s Future with JavaScript Picks hapi.js (Aaron) JavaScript Disabled: Should I Care? (Aaron) Aaron’s Frontend Masters Course on ES6 (Aaron) Brendan’s “Cool Story Bro” (AJ) [YouTube] Queen – Don't Stop Me Now (AJ) Trending.fm (AJ) WE ARE DOOMED soundtrack EP by Robby Duguay (Jamison) Hohokum Soundtrack (Jamison) Nashville Outlaws: A Tribute to Mötley Crüe (Joe) Audible (Joe) Stripe (Chuck) Guardians of the Galaxy (Brendan)

Panel Brendan Eich Joe Eames Aaron Frost AJ ONeal Jamison Dance Tim Caswell Charles Max Wood Discussion 01:57 – Brendan Eich Introduction JavaScript [Wiki] Brendan Eich [Wiki] 02:14 – Origin of JavaScript Java Netscape Jim Clark Marc Andreesen NCSA Mosaic NCSA HTTPd Lynx (Web Browser) Lou Montulli Silicon Graphics Kernel Tom Paquin Kipp Hickman MicroUnity Sun Microsystems Andreas Bechtolsheim Bill Joy Sun-1 Scheme Programming Language Structure and Interpretation of Computer Programs – 2nd Edition (MIT Electrical Engineering and Computer Science) by Harold Abelson, Gerald Jay Sussman & Julie Sussman Guy Steele Gerald Sussman SPDY Rob McCool Mike McCool Apache Mocha Peninsula Creamery, Palo Alto, CA Main () and Other Methods (C# vs Java) Static in Java, Static Variables, Static Methods, Static Classes 10:38 – Other Languages for Programmers Visual Basic Chrome Blacklist Firefox 12:38 – Naming JavaScript and Writing VMs Canvas Andrew Myers 16:14 – Envisioning JavaScript’s Platform Web 2.0 AJAX Hidaho Design Opera Mozilla Logo Smalltalk Self HyperTalk Bill Atkinson HyperCard Star Wars Trench Run 2.0 David Ungar Craig Chambers Lars Bak Strongtalk TypeScript HotSpot V8 Dart Jamie Zawinski 24:42 – Working with ECMA Bill Gates Blackbird Spyglass Carl Cargill Jan van den Beld Philips Mike Cowlishaw Borland David M. Gay ECMAScript Lisp Richard Gabriel 31:26 – Naming Mozilla Jamie Zawinski Godzilla 31:57 – Time-Outs 32:53 – Functions Clojure John Rose Oracle Scala Async.io 38:37 – XHR and Microsoft Flash Hadoop Ricardo Jenez Ken Smith Brent Noorda Ray Noorda .NET Shon Katzenberger Anders Hejlsberg NCSA File Formats 45:54 – SpiderMonkey Chris Houck Brendan Eich and Douglas Crockford – TXJS 2010 Douglas Crockford JavaScript: The Good Parts by Douglas Crockford TXJS.com ActionScript Flex Adobe E4X BEA Systems John Schneider Rhino JScript roku Waldemar Horwat Harvard Putnam Math Competition Chris Wilson Silverlight Allen Wirfs-Brock NDC Oslo 2014 JSConf Brendan JSConf Talks 59:58 – JavaScript and Mozilla GIP SSLeay Eric A. Young Tim Hudson Digital Styles Raptor Gecko ICQ and AIM PowerPlant CodeWarrior Camino David Hyatt Lotus Mitch Kapor Ted Leonsis Mitchell Baker David Baren Phoenix Tinderbox Harmony 1:14:37 – Surprises with Evolution of JavaScript Ryan Dahl node.js Haskell Elm Swift Unity Games Angular Ember.js Dojo jQuery react ClojureScript JavaScript Jabber Episode #107: ClojureScript & Om with David Nolen MVC 01:19:43 – Angular’s HTML Customization Sweet.js JavaScript Jabber Episode #039: Sweet.js with Tim Disney TC39 Rick Waldron 01:22:27 – Applications with JavaScript SPA’s Shumway Project IronRuby 01:25:45 – Future of Web and Frameworks LLVM Chris Lattner Blog Epic Games Emscripten Autodesk PortableApps WebGL 01:29:39 – ASM.js Dart.js John McCutchen Monster Madness Anders Hejlsberg, Steve Lucco, Luke Hoban: TypeScript 0.9 – Generics and More (Channel 9, 2013) Legacy 01:32:58 – Brendan’s Future with JavaScript Picks hapi.js (Aaron) JavaScript Disabled: Should I Care? (Aaron) Aaron’s Frontend Masters Course on ES6 (Aaron) Brendan’s “Cool Story Bro” (AJ) [YouTube] Queen – Don't Stop Me Now (AJ) Trending.fm (AJ) WE ARE DOOMED soundtrack EP by Robby Duguay (Jamison) Hohokum Soundtrack (Jamison) Nashville Outlaws: A Tribute to Mötley Crüe (Joe) Audible (Joe) Stripe (Chuck) Guardians of the Galaxy (Brendan)

Panel Brendan Eich Joe Eames Aaron Frost AJ ONeal Jamison Dance Tim Caswell Charles Max Wood Discussion 01:57 – Brendan Eich Introduction JavaScript [Wiki] Brendan Eich [Wiki] 02:14 – Origin of JavaScript Java Netscape Jim Clark Marc Andreesen NCSA Mosaic NCSA HTTPd Lynx (Web Browser) Lou Montulli Silicon Graphics Kernel Tom Paquin Kipp Hickman MicroUnity Sun Microsystems Andreas Bechtolsheim Bill Joy Sun-1 Scheme Programming Language Structure and Interpretation of Computer Programs – 2nd Edition (MIT Electrical Engineering and Computer Science) by Harold Abelson, Gerald Jay Sussman & Julie Sussman Guy Steele Gerald Sussman SPDY Rob McCool Mike McCool Apache Mocha Peninsula Creamery, Palo Alto, CA Main () and Other Methods (C# vs Java) Static in Java, Static Variables, Static Methods, Static Classes 10:38 – Other Languages for Programmers Visual Basic Chrome Blacklist Firefox 12:38 – Naming JavaScript and Writing VMs Canvas Andrew Myers 16:14 – Envisioning JavaScript’s Platform Web 2.0 AJAX Hidaho Design Opera Mozilla Logo Smalltalk Self HyperTalk Bill Atkinson HyperCard Star Wars Trench Run 2.0 David Ungar Craig Chambers Lars Bak Strongtalk TypeScript HotSpot V8 Dart Jamie Zawinski 24:42 – Working with ECMA Bill Gates Blackbird Spyglass Carl Cargill Jan van den Beld Philips Mike Cowlishaw Borland David M. Gay ECMAScript Lisp Richard Gabriel 31:26 – Naming Mozilla Jamie Zawinski Godzilla 31:57 – Time-Outs 32:53 – Functions Clojure John Rose Oracle Scala Async.io 38:37 – XHR and Microsoft Flash Hadoop Ricardo Jenez Ken Smith Brent Noorda Ray Noorda .NET Shon Katzenberger Anders Hejlsberg NCSA File Formats 45:54 – SpiderMonkey Chris Houck Brendan Eich and Douglas Crockford – TXJS 2010 Douglas Crockford JavaScript: The Good Parts by Douglas Crockford TXJS.com ActionScript Flex Adobe E4X BEA Systems John Schneider Rhino JScript roku Waldemar Horwat Harvard Putnam Math Competition Chris Wilson Silverlight Allen Wirfs-Brock NDC Oslo 2014 JSConf Brendan JSConf Talks 59:58 – JavaScript and Mozilla GIP SSLeay Eric A. Young Tim Hudson Digital Styles Raptor Gecko ICQ and AIM PowerPlant CodeWarrior Camino David Hyatt Lotus Mitch Kapor Ted Leonsis Mitchell Baker David Baren Phoenix Tinderbox Harmony 1:14:37 – Surprises with Evolution of JavaScript Ryan Dahl node.js Haskell Elm Swift Unity Games Angular Ember.js Dojo jQuery react ClojureScript JavaScript Jabber Episode #107: ClojureScript & Om with David Nolen MVC 01:19:43 – Angular’s HTML Customization Sweet.js JavaScript Jabber Episode #039: Sweet.js with Tim Disney TC39 Rick Waldron 01:22:27 – Applications with JavaScript SPA’s Shumway Project IronRuby 01:25:45 – Future of Web and Frameworks LLVM Chris Lattner Blog Epic Games Emscripten Autodesk PortableApps WebGL 01:29:39 – ASM.js Dart.js John McCutchen Monster Madness Anders Hejlsberg, Steve Lucco, Luke Hoban: TypeScript 0.9 – Generics and More (Channel 9, 2013) Legacy 01:32:58 – Brendan’s Future with JavaScript Picks hapi.js (Aaron) JavaScript Disabled: Should I Care? (Aaron) Aaron’s Frontend Masters Course on ES6 (Aaron) Brendan’s “Cool Story Bro” (AJ) [YouTube] Queen – Don't Stop Me Now (AJ) Trending.fm (AJ) WE ARE DOOMED soundtrack EP by Robby Duguay (Jamison) Hohokum Soundtrack (Jamison) Nashville Outlaws: A Tribute to Mötley Crüe (Joe) Audible (Joe) Stripe (Chuck) Guardians of the Galaxy (Brendan)

In this episode, I discuss why I still use XHR instead of the Fetch API.In this episode, I discuss why I still use XHR instead of the Fetch API.Show Notes & Transcript →

In this episode, I discuss why I still use XHR instead of the Fetch API. Links The original article: https://gomakethings.com/why-i-still-use-xhr-instead-of-the-fetch-api/ Promise-based XHR: https://gomakethings.com/promise-based-xhr/ CSS-Tricks primer on Fetch: https://css-tricks.com/using-fetch/ MDN on Fetch: https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API/Using_Fetch Atomic Promisified XHR Plugin: https://github.com/cferdinandi/atomic

Justin Searls chats with us on testing Asynchronous JavaScript as well as best practices for Continuous Integration, Unit testing vs. Integration testing, and tools we can use to help us understand how to test code. Resources Test Double - https://testdouble.com Talks & articles by Justin: http://blog.testdouble.com/posts/2016-06-05-happier-tdd-with-testdouble-js.html http://blog.testdouble.com/posts/2015-11-16-how-to-stop-hating-your-tests.html http://blog.testdouble.com/posts/2016-09-16-surgical-refactors-with-suture.html http://blog.testdouble.com/posts/2016-12-01-a-creativity-talk.html Justin on JSJ - https://devchat.tv/js-jabber/226-jsj-test-doubles-with-justin-searls Supertest - https://github.com/visionmedia/supertest Pretender - https://github.com/pretenderjs/pretender Sinon - http://sinonjs.org/ Testim - https://www.testim.io/ Cabybara - http://teamcapybara.github.io/capybara/ TestDouble's alternative to Sinon - http://blog.testdouble.com/posts/2016-03-13-testdouble-vs-sinon.html Testem - https://github.com/testem/testem Guests Justin Searls (@searls) Jan Tenpas (@jtp4) Panel Danny Blue (@dee_bloo) Erik Isaksen (@eisaksen)

Llegamos tarde al tema polémico de hace meses, pero lo compensamos subiendo un audio de pésima calidad. Es Safari realmente "el nuevo Internet Explorer" o estamos sufriendo algunos de memoria selectiva y reducción al absurdo? Internet Explorer, Safari, Chrome y los intereses y razonamientos detrás de lo que cada uno decide implementar. Enlaces Safari is the New IE (Nolan Lawson) http://tinyurl.com/qb4yuqs Sadari is the new IE II (Nolan Lawson) http://tinyurl.com/pxfx2cb Microsoft crea XHR en IE, el primer paso hacia la web 2.0 (2000) http://tinyurl.com/ogndthn Apple crea canvas en Webkit para Dashboard (2004) http://tinyurl.com/q7wc7tw El cómic de Google Chrome por Scott McCloud (2008): http://tinyurl.com/69o5sm 10 cosas de WIndows 10 (Gabriela González) http://tinyurl.com/o9bxeov Randall Comic Munroe inventa el Navegador Webhttps://xkcd.com/1367/ XKCD en lo irrelevante de ser Mac o PC https://xkcd.com/934/ Don Melton y su creación: Safari http://tinyurl.com/o69xwtd Don Melton contesta a Nolan Lawson http://tinyurl.com/qeaufd5 Respuesta de Hacker News http://tinyurl.com/q6zgq9c La persistencia de IE 6 y la resistencia al cambio http://tinyurl.com/q8cku3e Microsoft pide a Apple que Webkit no sea el próximo IE (2012) http://tinyurl.com/nga2x8m El navegador más dañino de la historia: IE 6 http://tinyurl.com/nfpcem2

Anselm, Hans, Peter und Stefan versammelten sich in vorösterlicher Stimmung um in Eintracht mal wieder festzustellen, wie kaputt doch alles ist. Schaunotizen [00:00:12] That’s so fetch! Jake Archibald verteidigt die unter Beschuss geratene Fetch-API. Fetch soll als Ersatz für das alterwürdige XHR dienen, wird aber unter anderem kritisert, weil man laufende Requests (noch) nicht abbrechen […]

Yehuda Katz and Tom Dale join us to talk about the road to Ember 2.0 and "Fast Boot". They share insight about why they stick to a 6 week release cycle, and why they think JS frameworks might be the future of all web apps (especially content sites). We also chat about what "indie open source" means, and exactly how much design goes into the Ember project and community. Yehuda Katz (Twitter) Tom Dale (Twitter) Tom Dale's Klout score is 66 Tilde.io Erik Bryn Yehuda at Hack Summit: "Indie OSS" HTMLBars FastBoot: Ember's Server-Side Rendering solution Tom on Shop Talk Show on Server-Side Rendering Rendr JS Yehuda's RailsConf keynote: "10 Years!" Skylight.io: Make your Rails apps faster with actionable insights Transcript: FILE NAME: The Frontside 16 - Yehuda Katz & Tom Dale Talk About Javascript DURATION: 55:59 minutes CHARLES: Everybody, welcome to Frontside the Podcast, Episode 16. We've got Brandon and Stanley here with me on the podcast and some very special guests who need no introduction, so I'll let them introduce themselves. TOM: Maybe we just start with Yehuda because he's the most famous. Are we starting by number of Twitter followers? STANLEY: Actually, Cloud score. TOM: Yeah, it's Cloud score based. YEHUDA: I think Tom has a higher Cloud score than me. BRANDON: All right, Tom. Go for it. TOM: Hey, what's up? I'm Tom. I had the idea for Ember JS in the shower. [Laughter] YEHUDA: Hey. I'm Yehuda. I work on standard stuff and Ember a lot these days, and now Rust also. TOM: Yehuda just joined the Rust core team. I don't know if you guys saw that. BRANDON: I did see that. Rust is a programming language. YEHUDA: Programming language. STANLEY: Are we going to get to talk about that later? BRANDON: Sure. TOM: We can talk about whatever you want. I'm not going to have any insights on that. YEHUDA: We'll have some insights. TOM: Yeah. I don't know if you guys ever saw the Pokemon Movie, but basically Yehuda is reenacting that with core teams. You've got to catch them all. [Laughter] BRANDON: That's great. STANLEY: That's not just the movie, Tom. That's literally everything around Pokemon. TOM: Oh, okay. STANLEY: That is the tagline. TOM: I will definitely defer. You seem like an expert here, Stanley. STANLEY: You know; I know the most important facts of all time. BRANDON: Stanley is on the Pokemon core team actually. STANLEY: I actually just made a new Pokemon that's like a guitar, a chair, and a microwave put together. BRANDON: What's it called? STANLEY: Rock-On. TOM: That is the worst Pokemon name I have ever heard. [Laughter] TOM: Oh, my gosh. BRANDON: All right, well, off to an auspicious start here. So the two of you and Leah formed the original core team for Ember. Is that fair to say or were there other people involved at that time when you kind of were switching SproutCore 2.0 to Ember? YEHUDA: I don't think I would call the original group that switched SproutCore over to Ember necessarily the core team. I would say that there was a bunch of people that were working on what was called SproutCore 2 at the time at Strobe, and it doesn't -- what we were doing there doesn't really meet my requirements for a good core team or a good Indie Open Source project. But one of the things that we did after switching over to being Ember and announcing the separate project was to make the core team more like what I would want. TOM: Well, I would say that there was never one moment where we were like, hey, let's create a core team. I think one thing that I learned from Yehuda about managing an open source project is that it is extremely important to start delegating way before you feel ready or comfortable. So there was a point early on where we were just totally overwhelmed as people started using it and people came along and were interested. And so we just gave them commit bit without really thinking about the bureaucracy of it or the structure of it. And then it definitely got to a point where it was like, "Why is there no core team yet?" because there's a ton of people with commit. So we should probably think about this a little bit more. YEHUDA: Of the people who are on the core team now, Erik, Chris, and Steph were all involved extremely early. I think Erik Bryn was the second contributor. Well, the first contributor after people working at Strobe, and Chris and Steph got involved really early because they were building an app on Ember that was very, very mobile focused. Well, it's a mobile app, and so they needed heavy performance, and we were not necessarily focusing on that, so they got involved pretty early also. BRANDON: And you gave a really awesome talk about this recently at Hack Summit. We'll throw the link to that in the show notes. I thought it was terrific, and I thought there was a lot of amazing ideas that were clearly born of painful experience. And I want to talk about that in a moment and kind of basically running and maintaining an open source project that keeps the open source ethos, I think, was kind of the thrust of the talk and keeping the Web and Open Source Indie. But before we jump into that, I wanted to get kind of a -- without going into, like, a full state of the union, there's a really lot going on in the world of Ember right now. It can be actually kind of hard for individual developers to just keep up with the news of it. There are just so many cool things happening at once. And there are a few things in particular that I wanted to get an update about. You guys are doing some really interesting stuff right now, but some things that are shipping soon: HTMLBars is actually happening. YEHUDA: It's in the beta. BRANDON: Yeah, it's in the beta, so people -- we tested it in our app already, and with one exception with, I guess, Ember list view isn't quite ready for it yet, but. YEHUDA: I think that's ready now too. BRANDON: Oh, really? TOM: Yeah, that just got updated. BRANDON: So, yeah, it was phenomenal. I mean it just works. Like, it was pretty amazing. So the benefit to users for that has been kind of already gradually been implemented where the metamorph tags in the DOM were gone. TOM: Right. BRANDON: What else can people expect to see once HTMLBars is in place? YEHUDA: So let me just reiterate a thing that you just said that maybe people weren't clear about before I let Tom a little bit about HTMLBars, which is, one of our major goals now and probably forever is to continue to update things incrementally and without breaking apps. And that's something that takes a lot of effort, so I want to reiterate it because it's probably the driving force of everything that we do, so like you said, there's a lot of news. We've been talking about a lot of stuff. We can talk for hours on it, probably. But the key thing is that a lot of times you hear there's a lot of news in a project, and it feels overwhelming. It feels like, oh, my God, if there's that much news, it probably means I'm going to have to spend the next five years catching up with all the things that are happening. And with Ember, our major goal is to make sure that all that, all those new features don't affect your app. I mean there will be a 2.0, and at 2.0 there may be some breaking changes, but even with the 2.0, all the breaking changes will land before 2.0. They'll land on the 1x brands together with deprecation warnings so you'll learn about them as you upgrade. TOM: Yeah. YEHUDA: And so I think this is the driving force of everything we're doing now. TOM: Yeah, I think, with 2.0, it's not like oh, my gosh, there's all this new stuff I have to learn. Instead, what it's going to be is us removing stuff that you probably never even learned about anyway. YEHUDA: Or that we told you in 1.10 or 1.11 to switch away from and you had plenty of releases to remove. TOM: Yeah, you'll have ample warning, and you'll definitely -- it's not going to blindside anyone. But I think this is exactly the point is we're on a six-week release cycle, and it is impossible to do big bang stuff in six weeks. Right? Think about any big software project anyone listening to this has worked on. It's hard to build a huge thing in six weeks, which maybe seems like a limitation, but actually I think we both see that as a huge strength, which is that it really forces you, as an engineer, to think about, okay, I want to move mountains. But I need to do it six weeks at a time, so how do I basically touch back down to reality as often as possible? With HTMLBars, if you think about it, it's a pretty dramatic thing, right? We're basically entirely replacing the rendering engine of this pretty large JavaScript framework, which in some sense is like trying to change the engine on a 747 mid flight. And the only way that we can get away with doing that is to do it very incrementally. And the only way we can do it without breaking people's apps, I should say, is to do it incrementally. Step one was metal-views, which removed metamorphs. But, fundamentally, all view rendering was still string concatenation. And then the next step after that was to get actual HTMLBars in with creating DOM instead of creating strings. YEHUDA: There was actually another step in between, which was to change all the -- so the internal, whenever you use a curlies and handlebars, those curlies, in the old version of the templating engine, would go and they would have ad hoc code to observe something, observe paths, and there was all this complicated code at each point where a curlies was used anywhere in the templating engine, and the new system -- and this is again behind the scenes, so it's easy for even Tom to have forgotten about it. We refactored everything internally so that it used a stream-based system so that all the parts of the templating engine don't have to know exactly how that's all structured internally. And that makes it a lot easier to do performance optimizations, but also makes it a lot easier to avoid mistakes and bugs that cropped up from time-to-time. So that was another step in the direction that was necessary to get all the way to the end. TOM: So now that's in, and I think the next step is to actually -- the next step will be now that we've got HTMLBars integrated in a backwards compatible way, the next step is introducing nice, new syntax. So just one example of this is this gives us the ability to remove the bind-attr helper. Most people that I've noticed when I'm teaching them Ember intuitively think that they should be able to do attrf equals curly, curly URL, and that doesn't work together. You have to do bind-attr. But because HTMLBars is a much smarter parser, we're able to have that context, and we can actually just dramatically simplify the whole model. BRANDON: Okay. And you also answered another question I had, which was, there are a lot of people basically talking about how they should be building apps for 2.0 friendliness, but it sounds like if they stay with the point releases, there'll be very little work involved in moving to 2.0. So you don't have to kind of like today sit and architect your app in a certain way as long as you're staying up to date with the point releases. Would you say that's relatively accurate? TOM: Yeah, I think so. 2.0 is not going to have any new features, and one feature that Teddy Zeenny is working on for the Ember Inspector, you know the Chrome and Firefox plugin for the developer tools, is adding a deprecations pane. So what we expect to happen is that people should just keep upgrading their apps on the 1x point releases and then, every upgrade, you may see one or more deprecation warnings pop up, either in the console or in this pane in the developer tools. And you just fix those at your leisure. Then when 2.0 comes out, all that will happen is that the payload size of Ember will shrink. YEHUDA: Yeah. I think another way to put all that is, when we looked at React, so we looked at React a lot as part of the run-up to 2.0 for the past, like, six months. And when we looked at React, initially we saw a programming model that actually wasn't that different from how we thought people should build Ember apps, so things like you should have data flowing down from a single point and, in Ember, that single point is the model hook in your route, and then we always thought about data flowing down. And you should have events bubbling up, and you should use actions mostly for communication. I remember Erik Bryn very, very early on said, "I think people are overusing data bindings. People should use events more." And that's when we started adding the evented system to a bunch of the parts of the framework. TOM: Actions. Actions weren't there originally … two-way bindings. YEHUDA: Well, we added actions, but we also added, like, Ember.evented. TOM: Yeah. YEHUDA: And I think we kind of knew all this, right? And so idiomatically the way that Ember 2.0 works is actually not that different from how we thought Ember 1.x should work, but I think one of the things that ended up happening is that because data bindings are so -- two-way data bindings can be very nice and clever, a lot of times people would reach for two-way data mining because it was the first thing that was sitting there. And then they would end up building somewhat complicated systems that rely on communication through two-way data mining. TOM: The syntactic sugar pushed you in that direction. YEHUDA: And so a lot of what Ember 2.0 is is about making syntactic sugar more honest about what is the right default, and that does mean that there may be applications that were heavily relying on communication, especially communication channels through two-way data bindings. And that will work much less nicely in Ember 2.0, and it might feel painful to upgrade if you're trying to be both idiomatic and upgrade to 2.0 at the same time. But I think, for most people who are using actions and were using data down through the model hook, I think a lot of it will feel very familiar, will feel very much like how you've been doing things all along. CHARLES: Cool. I actually had a question about HTMLBars landing, and that's when we upgrade our apps, everything should just work seamless. Like Brandon said, we actually did a spike of that, and it mostly seems to be the case. You said that there are no new features, but are there more, like, newer development stories around? Like, given that the templating engine or the view layer understands the DOM, what power or APIs will users have to interact with it, like to do animations or bring things in and out and stuff like that? YEHUDA: Oh, yeah. CHARLES: Is there any of that stuff planned? YEHUDA: I don't think we meant to suggest that there are no new features in number 2.0. Just that they will land in the 1x series, I think, is the point. CHARLES: Ah, right, right. I see. TOM: I think probably the biggest feature is just speed. And I think, also, what HTMLBars' architecture unlocks is the ability to better integrate with other libraries by adopting kind of a diffing approach similar to what React does with the virtual DOM. Basically, in my mind, HTMLBars is all about a bunch of infrastructure work that allows us to make the programming model feel more natural for people who are…. YEHUDA: One way to think about it is that it's the first templating engine that was ever built specifically for Ember. Handlebars templating engine before was built as a general-purpose templating engine, and we pushed it as far as it could go to be real useful for Ember. But things like bind-attr and the metamorph tags kind of crept in as necessary because the templating engine wasn't really built for this purpose, the exact purpose that Ember was designed for. And the HTMLBars engine, part of it is that it's DOM based, and part of it is that it supports diffing, and part of it is that it's faster. But I think, ultimately, it allowed us to look at a lot of areas that are annoying about using templates in Ember and make them nicer. And, yeah, so I think that it's -- TOM: I'd say it also unlocks things like we're working on server side rendering right now, and a lot of that is due to the power of HTMLBars because we can run it in so many different environments, and we can model all of these things as streams internally. It gives us a lot of flexibility about what we can do with your applications. You know, we can do things like server side render your applications even though, of course, you never designed your app for that case in mind. But because of how expressive the templating and, in fact, the entire application architecture is, and because we all agree as a community that this is how we architect our apps, it unlocks a lot more stuff. And I think there'll be more things like server side rendering in the future that we all benefit from by sharing this very declarative application structure and very declarative templating language. YEHUDA: Yeah, I mean honestly, under the hood, the fact a lot of the innovations of the templating engine are not things that any user will ever see directly because they're just implementation. And if we wanted to go around pimping things like streams or the way that the diffing algorithm works internally and the way we clone fragments and all this stuff, we could probably spend a lot of time pimping it, and maybe that would even make a lot of people, some people happy. But I think you'll see, over the next year or so, these things will all lead towards better features or to more features that will be nice, and that's how I think we'd like to roll -- TOM: I think Web components integration is a big one. YEHUDA: Ah, yeah, that's a good point. TOM: I think HTMLBars makes it very easy. And so, in terms of actual improvements coming on top of HTMLBars, the component syntax, instead of being curly, curly to indicate that you want a component, you'll be able to use just regular angle brackets, so that'll be nice. And another thing that we're really keen to get rid of is: You know how today when you're building an Ember component if you want to customize the element associated with that component, you have to say, like, tag name, class name, bindings. You guys know what I'm talking about? BRANDON: Mm-hmm. TOM: So that's kind of annoying because all of those special properties on the component class that you used to customize the element are all duplicating features that already exist in the templating language. So it's just kind of this weird bifurcation of the programming model where, if you want to customize the element for this top level, do it in JavaScript. Everything else, do it in the template. So HTMLBars will allow us to allow you to customize your component element purely in the template, and you won't have to -- basically there are far more cases now where you'll be able to get away with a component that's just a template file, and you'll reduce the number of JavaScript classes in your app. YEHUDA: Yeah, I think, from a high level, the biggest -- the high level of the internal technical improvements are largely that it allows for contextual work. So the old templating engine wouldn't necessarily know that you're inside of an attrf when you have curlies, so we would have to spit out a bunch of extra stuff and maybe make you use extra helpers. It wouldn't necessarily know that you're in image FRC or a video tag or a component. It wouldn't be able to tell if you were using a polymer component that implements the bind protocol, right? But the new templating engine basically lets us see exactly what's happening at every curly, and that just has a large number of positive effects. BRANDON: So you said something else that I felt like was a good segue into the next part of the discussion that this basically allows you to do server side rendering, which you guys are really, like, in the thick of it right now. But for me, a lot of the talk I've seen a about server side rendering comes across as a little inside baseball. There's this sort of discussion between people who are really into React because they're suddenly doing a lot of stuff with server side rendering, and they're seeing some benefits out of it. And you see this stuff kind of pop up in the JavaScript community, but I'm curious to know if you guys can explain a little bit about the benefit of server side rendering that this is a major new feature coming to Ember now. YEHUDA: I'll let Tom maybe give the full pitch, but I think one thing that I feel somewhat strongly about is that, for most people, if you don't have a system that actually gives you something that works pretty well out of the box, in other words it doesn't ask you to do a lot of the work yourself, the idea of isomorphic server side rendering where you run your same app on the client and the server, I don't think that that ends up being worth it. And if you look at a lot of apps that use Ember today, a lot of them have spent relatively little time building non-isomorphic solutions for specifically SEO that have been very, very cheap in terms of time and very, very effective. But I think there's the: I want to not have to write that, even that little bit, and I think that that you get a lot of benefits out of if you just have your framework do it. In other words, if it's not just like your framework does 20% and you do 80%. If it's your framework does 95% and you do maybe a little bit, or you have some constraints, I think that is worth something. And I think the rehydration of fast boot is worth something, although that has even more issues and even a larger percentage that most people have to do on their own. Basically, I think the TLDR for me is that I've always saw server side rendering as indeed somewhat inside baseball because, for most people historically, and I think this is even true largely about React, the solution that you're offered is the framework will do a little bit for you, and you have to go figure out a lot of the details about how to make this work for real. And I think most people just don't have the time to figure all those details out. TOM: So it's been kind of interesting to watch this play out over the last year or two because I think there's been a big split between the JavaScript application community and old school people that create content for the Web who are really keen on this idea of progressive enhancement. And so there's kind been this split. And, for me, for a long time, I personally didn't really care about this case because, in my mind, JavaScript apps were really good for what I'll call workspace apps, which is most of them are behind a login. You log in. You have these very rich interactions. You're editing something. You know, I worked on the iCloud apps. It's a feature, not a bug, that Google can't index your calendar and your email, right? So to me that was the use case for JavaScript apps. But that was until I saw content sites. Like, I remember the first time thinking, "Wow, maybe JavaScript apps are actually the future of all Web applications," was when I saw Bustle, actually. When I saw Bustle, it was just a content site. It was just news articles. And if you would've asked me, I would have said you should probably use Rails or some other server rendered framework for this. But then I saw it, and it felt just like a website. I couldn't actually tell the difference other than how damn fast it was. And I kind of had to step back and question all of my opinions about how people should be building these kinds of applications. And especially for content sites, I think that the server rendering is really important, right, because historically your user has had to download all of this JavaScript and all that JavaScript had to be downloaded and evaluated and run before they saw anything. So having the ability to bootstrap that process on the server and have the first bits that the user starts downloading not be the JavaScript payload, but be HTML and CSS, so that the first thing that they see is useful, I think that's really going to change how people build applications because you get the benefits of server rendering while still retaining the kind of interactions that you can build with something like Ember that you just can't do with Rails. YEHUDA: But again, I think people trying to do this on their own and taking maybe a half solution and then trying to figure out how to make this work reliably ends up producing things that are pretty buggy and feel pretty bad on first boot, or they end up requiring tremendous amounts of engineering resources. And it's possible that, like, huge companies can make this work. But I mostly think about startups, and startups simply do not have the engineering resources to take on the server side rendering task on their own, so this is why I think we care about it for Ember because, as Tom said before, Ember is already pushing you down a pretty conventional path, and we think -- our hope is that by having people do that conventional path and us taking charge of server side rendering will have something that works mostly out of the box for a lot of users. Again, assuming they follow some additional constraints about what you're allowed to do on the server. TOM: And I think we should be clear here because there's, as always happens, there's ambiguity in the terminology. So first is the term isomorphic app, which I don't really love that term, but I guess we better get used to it, Yehuda. YEHUDA: Yeah. TOM: But there's really a spectrum. On the one side of the spectrum you have something like Airbnb has a library for Backbone called Rendr (with no e - well, one e, but not the second e). And that kind of lets you wire up some of the server side rendering. But again, it's very, very manual. And the whole purpose of this is just to make sure that the first thing that you deliver to the browser is HTML and not JavaScript so that the user, even if they don't have JavaScript enabled or they're on a slow connection or whatever, they get something useful. But then on the other side, you have things like Meteor or Asana where the whole idea is -- and to me, I'll be honest with you. It strikes me as extremely silly, but the idea is that you're writing both your server and your client in the same code base, and then you're deploying them both. You describe it, and it sounds like this magic bullet, but it also just seems really silly to me. YEHUDA: Well, I think the fact that even the first releases of Meteor had if Meteor.isClient and if Meteor.isServer, and even the first demos had large blocks of content…. TOM: Yeah. Conditionals. Yeah. YEHUDA: Basically means that people hadn't really figured it out exactly right yet. TOM: Yeah, the point is that even if you have a client side app, your server still has a lot of responsibilities, especially around data access to the database, authorization, authentication, and so on. And putting that in the code base with your UI and your drag and drop code just does not make any sense to me. So I want to be clear to everyone listening that that is not what we're going for. The idea is not that you're writing your json API server in Ember. The point is that you're writing the same old app. In fact, we hope that you don't have to make any changes to your app. And you can deploy it, and it will do a render using the same code. It's basically like your app running in a browser on the server, and then we'll have some way of -- YEHUDA: Except not actually a browser. TOM: -- it's actually a transferring state. YEHUDA: Except, importantly, it's much, much cheaper than being an actual browser. We're not using phantom JS or a zombie or anything. TOM: Right. Conceptually a browser, but we don't want to pay the cost. Phantom JS is a source of great pain for many people, ourselves included, so we want a pure JavaScript environment. But conceptually it's a browser. YEHUDA: I think the reason I hopped in there is I just want to be clear that the goal of conceptually a browser is actually not to be a browser, but to make Ember itself internally abstract enough so that the most expensive parts of being a browser can be replicated in a cheaper way on the server. Obviously the most important part of that is the DOM. And that's the part that React worked out for server side rendering is use the virtual DOM on the server and not a real DOM, right? And that means you don't need real phantom JS and the full scope of DOM. But there's also other stuff like how your routing works and how the model hook runs and how it makes requests, how it makes "XHRs" to get the data in the first place. Right? There are a lot of details if you think about what it takes to boot up an app in the first place. For us, the goal is to go through that whole process of booting up an app all the way through, but not including the did insert element hooks in your DOM and make sure that all that stuff doesn't require -- it has really constrained and clear abstractions, right? So rendering has a render abstraction and routing has a location abstraction, and the DOM has the HTML bars, little dom, lower case dom abstraction, right? So instead of saying we're running it inside of something that pretends to be a browser, we're saying Ember doesn't actually care whether it's in a browser or not, but it has very, very clear abstractions for what it means to be a browser. BRANDON: Okay, so is there--? It sounds like it could be a little like -- is this a little ways off? It sounds like there are a lot of benefits. Like, if you see the hand rolled stuff that Discourse has done, clearly this is something that Bustle cares enough about to sponsor, this is probably a little ways off for Ember developers. Is there anything that you want people doing Ember to know right now about server side rendering in terms of how it's going to affect them or some of the technical stuff that you're learning through this or anything like that? TOM: I think we've been thinking about this particular problem for a very long time. And in fact, we've intentionally designed the architecture of Ember specifically to handle this case, even from the beginning, even like three years ago. We knew that this is something that we wanted or at least we didn't want to paint ourselves into a corner around. So I think there are two aspects of this, and one of them, I think, is pretty well bounded and pretty straightforward. The other one is definitely going to require a little bit more research. The first step is simply getting rendering happening on the server. So because we designed Ember for this case, we were actually able to get Ember as a framework booting up in node in about a day's worth of work. So a couple things have crept in. There were areas where we had been a little bit sloppy and introduced dependencies on things like document.body, jQuery, things like that. So it was about a day to kind of encapsulate those, make sure that they weren't hard requirements for booting the framework, and that was about a day's worth of work. And then, by the end of the week -- we've only been working on this for about a week now -- at the end of the week, we actually had an app booting in node and handling route requests. So in terms of progress, it's been really great. But I guess all that we're saying is that, in a week, we were able to capitalize on the last three years of work we've been doing. That's not as impressive as it sounds. YEHUDA: It was nice that there were relatively few regressions, right? TOM: Yeah. YEHUDA: That the list of things where people were accidentally doing things that assumed the browser was actually relatively small. TOM: Yeah. And the way that we did that is basically introducing an abstraction that provides your environment to you, so a node that's different than in a browser. So that's the first part is to get the app booting, and the second part is to get it rendering. I think what's really cool is that, even though HTMLBars, of course, is a DOM based rendering engine, or despite that, we still use an abstraction around DOM access. What we're going to be able to do when we start, again, in earnest tomorrow, on Monday, is basically replace that DOM helper that we used to create DOM in the browser with something that we'll be able to build up strings so that you can build up your HTML on the server and serve that to the client. That's step one: rendering. I think we'll be able to make quite quick progress on that. I would guess probably about -- I don't know if I should give timelines here. I think we're ballparking around a month before we can at least beta it, the server rendering. BRANDON: That's a little faster timeline than I had assumed. TOM: That's purely rendering. I want to make clear that that is purely for things like SEO, for Web crawlers, for curl, things like that. Then the next phase after that, and this is where it gets into the tricky bit, is being able to -- YEHUDA: Rehydration. TOM: -- is rehydration, what people call it. What we call fast boot. And with fast boot the idea is that whatever state that we use to build up your application on the server, we also transfer that state, not just the HTML, not just the output, but the state that we use to build that output is somehow seamlessly transferred from the server to the client. And the client basically just takes the HTML that we've given it and reconnects all these bindings and goes from there, so it's totally seamless to the user. YEHUDA: And I think there's some complexity there. For example, there may be some part of your page that you can't actually render on the server because it says, like, "Hello, authenticated user," with the user's name. So thinking about ways to make sure that you can mark those as needing to be rendered on the client without causing jank. There's a bunch of stuff like that where, at first glance, it seems not too bad, but I guess the high level if there's a determinism issue, right? So the goal is to make sure that roughly what you did on the server is the same thing that you do on the client because if it's too far off, then you end up having to throw -- no matter how smart our algorithm is, we're going to end up having to throw away everything and replace it again. The idea is how to structure your app, how to structure the way that you set up your app in Ember CLI so that you tend to be putting out output that's deterministic on both sides. And, like Tom said, I think state is maybe overbroad. It's mostly modeled batter, right? Making sure that the model batter that you got on the server is equivalent and transferred together with the HTML payload so that the model hooks that you have in your client will not have to go make another XHR. They'll just have the data that the server already collected, and then hopefully rerender an equivalent DOM on the clients with relatively low…. TOM: There are just a lot of tricky cases, like imagine you have a bound helper that shows relative dates, like 30 seconds ago. So you have a clock on the server, and then you have a clock on the client. How do you reconcile those two? YEHUDA: Yeah, so the good news there -- the good news with all that, without getting too much into the weeds, is that the HTMLBars' engine is already broken up between rendering the parts of your DOM that are static, that are like the hello, the text hello inside of an H1, and updating the static parts with dynamic content. And today that's purely a performance optimization, and so that we could use the same document fragment over and over again after cloning, but that will also allow us to use server rendered content where, instead of expecting to have an empty text node that is to be filling in with dynamic content, we'll have a filled in text node. And we see, in that case, Tom, that the time that is in there already is not the correct time. It's not the equivalent time, and so we'll update it. So that's sort of like the React diffing strategy. I'm less worried about, like, there's a single text node with the wrong content because I know we can deal with that. And I'm a lot -- although if it's too much changes, it will look very bad. It will look very janky. I'm more worried about, like, this entire conditional change. So you're looking at something and then, like, your sidebar swaps out for a different sidebar, which I think would be a pretty unacceptable UI. CHARLES: Obviously there are a lot of different server side environments that people use. What requirements of the server is there going to be if I'm using a Rails app or something in Python or Java or whatnot? How am I going to interface to this? TOM: Well, you definitely need a JavaScript runtime, right, because your Ember app is written in JavaScript. Unless you're planning on pouring it into Ruby or whatever, we're definitely going to require JavaScript runtime. And I think we're starting with just supporting node. But what I would like to see, and maybe you guys can write this, is a Rails gem that you can install that will install dependencies in everything and basically get set up in a production environment. YEHUDA: I think one thing people often forget is you can definitely -- you could you have a node app running. People try to embed JavaScript. They try to use, like, The Ruby Racer, and embedding JavaScript is quite a disaster. BRANDON: [Laughter] I'm sorry. I don't know if you know. Charles wrote The Ruby Racer, and it is a disaster. STANLEY: It's cool. It's a disaster. YEHUDA: So let me be clear. It's a great idea. I use it a lot, but it just fundamentally doesn't now work. Right? It fundamentally cannot -- you cannot embed two VGCs in the same process. BRANDON: Right. YEHUDA: Okay. BRANDON: This is the greatest moment in the history of our podcast. I just want to say that. CHARLES: No, I know. There's no way to collect…. YEHUDA: Yes, exactly. CHARLES: -- for example. YEHUDA: I was about to use cycles as an example. CHARLES: The APIs just don't exist. YEHUDA: Yeah, so basically cycles, so this is why. The reason why I feel strongly about this is that we use Rust for Skylight, which is our product. And we need to embed something, and I would never in a million years embed something with a garbage collector. And I think The Ruby Racer was a pretty good -- I think, for constrained cases, it works fine if you know what you're doing, but people basically tried to use it as, "Oh, I'm just going to write part of my program in JavaScript. And, by the way, both languages have closures, so good luck," basically. The Ruby Racer is cool, but I would not -- I don't think that that's the correct strategy for having long-running JavaScript programs like Ember, like complicated stuff like Ember. I think a better thing for people to do would be to figure out a simple IPC protocol so that they could run their Ember app and then have a simple way of talking over a socket or something with the node app, so booting up your Rails app will also boot up the node app. And, if necessary, and you're serving through your Rails app, you could communicate. TOM: I think, to be clear, the Ember app, even when running on the server, still talks to the database server using json, right? So it's the exact same data marshalling flow. It's just presumably it'll be a lot faster because probably your API server and your application server are in the same data…. YEHUDA: Well, at a minimum, it'll be a lot more consistent, right? TOM: Yes. YEHUDA: I think when people -- when Twitter complained about somebody from some country connecting and getting a really slow connection, the issue there was that they were downloading the app shell and then who knows how long it took to download the json payload, right? But if the json payload is coming from the same data center, then it's, by definition, going to be downed within some range, reasonable range. BRANDON: Okay. Awesome. I'd like to shift gears and spend a couple minutes talking about something that most of the questions that I had originally for this were actually answered in your talk. But I'd like to go over it just a little bit. You alluded earlier to the way that you're running Ember as an ideal, sort of your ideals, discovering your ideals about open source projects and the Indie Web. And I think it's a really important topic, and I want to ask a little bit about that because I don't think a lot of people understand this. I think, especially I see in the JavaScript community, a lot more people establishing open source projects that are corporate run and that being considered a benefit rather than a drawback. And I've seen you push back on that a little bit, and I wanted to ask you, Yehuda, about what your definition of the Indie Web is and why you specifically run the Ember project the way that you guy run it. YEHUDA: Sure. I think there are basically two parts of what it means to be Indie, and the second one sort of derives out of the first one, but I think it's -- you wouldn't necessarily arrive at it yourself, so I'll make it explicit. The first one is that you have a diversified core team. What I mean by that is essentially diverse in all the ways you could possibly think of, and things that I learned from other projects are, like, functionally diverse. So make sure that if there's a person on your team -- if there's a person around somewhere running your events or doing documentation or doing evangelism or running user groups, make sure that if there's a person who is very skilled at doing that that they are the top person on your team in charge of that. The counter, the alternative that I've seen a lot in the open source community is that the person running events essentially reports to the core team, right? There's a person who is maybe a professional event runner, and they are not in charge of events. They're in charge of coming up with ideas for events that they run by the core team, and the core team decides yes or no. The problem is the core team has no skills in doing that, so this person ends up spending huge amounts of time being frustrated trying to explain to the core team something or other, right? That would be the equivalent of somebody on the core team writing some area of code, having to come to the rest to the core team and talking about really tiny, nitty-gritty implementation details. Of course, unlike code where I think people have an intuitive sense that you're deep in the weeds of some performance thing, and I don't really understand that. In a lot of areas that are not code, code people have a very high sense of their own understanding, right? The core teams, I've seen a lot of core teams that people come and they say, well, I happen to know a lot about how people want to read docs in this country and so I'm going to help with the translation effort. All of a sudden the core team is an expert on, like, Indian documentation. And they have all these opinions that are totally unwarranted. Step one is have a functionally diverse group of people, and have people that are not necessarily hardcore coders, but are talented and professionals in their area. Have them do that work at the top level. And I spent a little time on this just now because I feel strongly that this is an area that people miss, and it's just an area that code people are too high on their own supply. They're too impressed with their own skills. They cause a lot of pain for the people who are good at areas that are not hard-core code, so that's one. Two is be diverse in terms of the set of people that own decision-making in your project, so this is what you were talking about with the company run open source, and this is something companies can do. I've seen, for example, I worked with the Rust Project, and the Rust Project originally started at Mozilla. But they've been spending tremendous amounts of effort to try to increase the set of people who are on the core team of Rust that are not working at Mozilla. And this is something that maybe takes a lot of effort once you have an established project at a company. There's all the internal company politics you have to deal with. But the reality is that if you have a project that's at one company, you're kind of at the whims and the mercies of that one company's how they do resourcing, whether they think it's important, what their actual goals are. Maybe the thing that they built the project for doesn't necessarily match what the community is doing, right? So become diverse in the companies that own it. I think projects like Rails, Postgres, Ember, increasingly Rust are good examples of this. And, of course, I mean the regular meaning of the word diverse here, just because if you become more diverse in all areas, you actually find yourself being more function diverse just because of who ends up being in what areas. You end up finding yourself having a lot better insight. You end up sitting in a room, and when there are people of diverse backgrounds, the kinds of questions that people ask. And I can only say this. This is the thing that I've noticed personally. It's not a thing that I can empirically measure. I've just noticed that the kinds of questions that people ask, when you're diverse in all kinds of ways, ends up being -- they end up being stronger, better, and they end up pushing back on the kinds of decisions that you can make as a monoculture with group think, right? The harder it is to have group think, the harder it is for everyone to sit around and say, "You know what we're going to do? We're going to rewrite everything," right? That kind of thing is hard to do when you have a lot of people with a lot of different backgrounds with a lot of different interests. So that's, I think, the higher order bit of what in the open source means is be very diversified in a lot of different ways. But there's a specific thing that I think comes out of it that is very important, which is to do the work that you're doing incrementally. Again, the reason I think that this is a derive is that if you have a lot of people with a lot of different interests with a lot of different projects, I think it becomes very difficult for you to do full on rewrites because everybody has interests, and maybe a few people are excited about doing a rewrite, but everyone else says, "Oh, my God. What about my app?" Then you have the docs guy saying, "Oh, my God. Now we have to maintain two sets of docs?" And you have the evangelism guy hearing all the pushback from the community about the rewrite. So it becomes very difficult for you to have this situation where you're not doing things incrementally. But I think, in my talk, I spend some time on what it means to do things incrementally and what the benefits are and how to adopt the six-week release cycle and all that. I talked about that in more detail because, even though I think it's a natural consequence of being diversified, it's also something that you have to think about if you want to do it well. BRANDON: Yeah. It seems to me that that would require, like it's kind of a chicken and egg deal, but to me it seems like there was a lot of discipline in switching to a six-week release cycle, and that's important. It seems, for us at least as consumers of Ember as an open source framework, that's benefited us greatly. We find the framework much, much easier to keep up with on the six-week release cycles than pretty much most other open source projects we've worked with. YEHUDA: Which is kind of like a paradox, right, because you expect that six-week release cycle means you can never keep up with it because it's always changing. But in fact, six-week release cycle means you don't have time to ever change that much. BRANDON: Well, and even for apps that go dormant for a while, we find that, okay; we'll bring it up to one. Bring it up to the next one. The upgrade path becomes extremely obvious. YEHUDA: Yep, exactly, and there's deprecation warnings, and there's ... I think people should watch my talk on this specific area because it took me 15 minutes or something to lay out the whole thing. But I think the basic idea of just doing things incremental, and incrementally has this bad sense. It's like, oh, maybe you're hunting for the local maximum or something. All I mean by incrementally is the same way that the human body replaces its cells, right? You don't do it all at once. Maybe over an extended period of time, the thing that you're looking at is completely different. But because you did it a little at a time, you were able to move the whole community together in the direction instead of people, many, many people getting left behind, which is what used to happen a lot with Rails. I think Rails has gotten better at this over time. But it used to happen, like Rails 3 came out and a lot of people were stuck on Rails 2.3. BRANDON: I think when people hear incrementally, they can think about possibly incrementally be led in circles, right? You could incrementally be every day wake up and decide to change one degree or the other. What matters is if you have a compass that's pointing somewhere. For the Ember project, that incremental stuff doesn't work unless it's pointed somewhere very clear. And you and Tom are basically the keepers of that vision. And I wanted to ask about that, what the vision of the framework is that keeps guiding everybody. Is it sort of implicit to the project? As you use it, you recognize it. Or is it something that you've explicitly outlined somewhere? YEHUDA: I'll take this for a second and then Tom can jump in. I think, fundamentally, the main vision is just we ultimately wanted to have a full stack solution that solved the majority, the vast majority of the problems that a regular person would have writing a Web app, but we knew from the beginning that if we tried to take on that whole project, I mean even Meteor, who took on that whole project, is still struggling to have to complete the vision. And they tried to sort of boil the ocean. And so, we knew from the beginning that we were going to get it wrong if we tried to do a CLI tool and a framework and a data library and all this stuff all at once. And so I think we started off with the V in MVC, basically, right at the beginning and added routing and other stuff over time. But the mission always has been to build the thing that's a full stack of what you need to build front-end tools. Tom, you can take it from there. TOM: Yeah. I think, in the JavaScript world, I think, because JavaScript for so long was treated as a toy language that people didn't do serious stuff in, it attracted a certain type of developer who is -- which is a totally legitimate opinion, but they tended to be kind of hackers who would do these one-off experiments. And because of that, the notion of convention over configuration and having shared solutions is still a somewhat surprisingly controversial opinion in the JavaScript community. And so I think the role, as you said, for Yehuda and I, is to basically be willing to stand up and take the tomatoes that get thrown at us and say, "You know what? No, there is benefit in having a shared solution, especially when it's not just one-off hackers in their basement, but when it's a team of engineers working at a company, and you have a product that you need to ship, and it needs to have good interaction. It needs to be done yesterday. Those people need tools too. People like that deserve tools." And I think that's our goal is to have a framework that will last for at least the next ten years that is willing to incorporate good ideas as they come out and as they're embedded, move the community together, as Yehuda was saying, but without chasing the hype dragon where every six months: "Throw away everything you know because the next big thing is coming out. Rewrite your apps." I see Ember as a way of kind of tempering that instinct for engineers to chase the new and shiny constantly in a way that basically we have a community that agrees together what the next big thing is, and then we start moving towards that. I think, right now, major things that we're thinking about are one server rendering, as we talked about. Getting the CLI tools in place, that's a thing that we've wanted for a long time. But as Yehuda said, we didn't want to try boiling the ocean. And then the new HTMLBars view layer, which unlocks a lot of the cool things that React is able to do around, like, DOM diffing and so on. YEHUDA: Yeah. I usually tell people -- recently, I've come to a line, which is, if you want to tell me that there's not a place for shared solutions in some area or some abstraction, I think the burden is on you, actually. I think people in the JavaScript community, and there's a small group in the Rails community that feels the same way, they assume that the burden is on the person who is trying to abstract, right? If there's a common problem that a lot of people have, they think it's your job to prove that abstraction is a good idea. I think it is your job to prove that a particular abstraction is a good idea. But I think my de facto, my default position is that if a lot of people are solving the same problem that there's a shared solution worth hunting for. And I would say the JavaScript community is really -- big chunks of the JavaScript community are pretty anti this approach. But I think it's really the only way that you could build -- like Tom said, the only way you could build projects with large teams is to have some sense of what the shared answer is and to not have it be some genius on the fourth floor somewhere that does everything. And if you want to make any changes, you have to go to them. I've seen a lot of companies that work like this, and it works fine. Anther facet of this is that pretty much every company deciding to adopt, like, Ember, Angular, React, or Backbone, whatever, they do like this "taste test," right? A taste test is like a two-week project where they see which one is faster. By definition, the taste test doesn't successfully analyze what happens over a longer period or when you have a bigger set of developers, right? It's by definition optimized for short projects with a small number of developers, and so -- TOM: And usually it's the guy on the fourth floor conducting the taste test. YEHUDA: He's either conducting it or he is actively attacking it, right? He is saying we should not -- for example, Firefox OS refused to adopt any framework for an extended period despite the complaints of many people on the Firefox OS team because of the fact that they had a religion against frameworks. They didn't like frameworks as a concept. I've heard this from large numbers of people working on the periphery of Firefox OS and many complaints, right? And so I think we, Ember, one of the things we had to learn was that we can't get away with just saying that. We can't get away with saying, oh, you'll learn. If you just use Ember for a year, you'll figure it out. We had to really improve the getting started experience. But I think, on the flipside of that, there's no way that we could ever -- even if we get to be as nice as to optimal getting started framework, getting started tool, we're always going to have benefits that are not part of that that are very difficult to see when you're doing a quick analysis. Actually, recently we've seen a lot more big companies come out and talk about the benefits of Ember for long-lived projects, and I think that helps a lot just having people testify that they used Ember for a year within a team that wasn't three people, and they found it to be productive in these specific ways. I think that's helped a lot of people feel comfortable. BRANDON: That's been my experience, certainly, as well, just seeing that increase. I think everybody should -- honestly, I believe everybody listening to this should drop what they're doing and go watch the Hack Summit talk. I thought it was phenomenal, and I think it made me think a little differently because it's a little confusing out there, like what some of the tradeoffs are in these open source projects that are run in that kind of echo chamber. And the fact that you guys work so hard to pierce the echo chamber is really cool. I know that there may be some technical questions. We don't get a chance to have you guys on the podcast very often, obviously, so I wonder if anybody has any more questions. STANLEY: Before we get back to technical questions, I just want to cut in and say I really like Yehuda's talk from Rails Conf as well. It was really eye opening for me as somebody new to the Rails community, even though Rails has been around for a while, and kind of understanding the value of shared solutions and kind of the philosophy behind that. YEHUDA: Yeah, that was definitely the first time I tried to formulate a general theory for what shared solutions look like and why they're good, and essentially why Rails hit the nail on the head with the right amount of shared solutions and where the experimentation is happening and all that stuff. STANLEY: Cool. Back to you, Charles. CHARLES: I just had a quick question I wanted, before we wrap up. I had another question about the server side rendering, kind of a general one. I know I've definitely been burned by server side rendering in the past, you know, because it's been something that people talk about on and off, it seems like, for the last five, six years. I remember when mustache first came out. The first time that I tried it, it's like, oh, I've got this templating thing that I can run inside on my Rail server, and I can run it. There's a JavaScript implementation too. One of the things that I found was I was able to get up running very quickly, but then I felt like I was eaten alive by the edge cases. It was actually -- I think it was a blog post that you wrote, Tom, where you were talking about kind of the justification for resolving, always resolving RSVP promises asynchronously. Because, to not do so, have a different context, different stack sitting on top of the resolution was like releasing Zalgo into your application. I actually, when I read that thing about promises, it actually made me harken back to my experience with server side rendering. I was like, oh, with server side rendering I was releasing Zalgo onto my client. I had a very different context. TOM: Yeah, the thing you're describing is sharing templates across two different apps, right? CHARLES: Right. TOM: The data model diverges, and then the things that you need diverge. That specifically is something that we are going to avoid. The idea isn't, oh, you can reuse your model on the server, and you reuse your templates on the server. The point is it's your app, the same exact app, same code base that you would run in the user's browser just happens to be running on the server. YEHUDA: And I think there's also -- I think there's another important point, which is that if you look at how people do SSR, I think historically people have said, "Well, I'm going to use a view layer that's very good at SSR." And then you would have this pile of hacks that was involved in booting up your app. Honestly, Ember, in the beginning, had piles of hacks used to booting up your app. I definitely remember that period. Now the view layer maybe is very good at running on the client server and, like you said, originally was just like the template. But now maybe the whole view layer is good at it. But now the process of actually booting things is a source of non-determinism. You're saying Zalgo. I'm saying non-determinism. Zalgo is a better word. CHARLES: [Laughter] YEHUDA: And Ember has definitely held off on tackling server side rendering seriously until we felt confident that we had the full stack handled. In other words that, as a framework, we had the whole lifecycle handled. Then actually, if you look at technically what we've been doing recently, a lot of it is like separating out. We have an application right now, an application only -- there's only ever one instance of it. Now we're saying, well, there could be multiple sessions. And so we're really looking at the whole lifecycle of the application and, because we own the whole lifecycle of the application, we can actually feel confident that the path that we're going through is correct, so that's one part of it. The other part of it is essentially what React figured out at a high level. I think what we're doing is equivalent, which is you don't necessarily assume that you got exactly the same thing on the client server because probably in practice there's always going to be some thing or other, like the clock case that Tom talked about, or the hello Yehuda case that I talked about before, the authentication case. What you do is you rerender the template, and you don't say if it's not exactly the same, throw it away and start over. You say parts that are the same you can keep, and parts that are different you replace. Therefore, it's not so much a whack-a-mole problem. It's more like how much replacement can I tolerate and have it not feel janky, right? So you go use it, and if you see that there's an area that's popping in and replacing, that's an area that you have to go and figure out so, first of all, it will work. Right? It will work. It will not be broken. It might feel a little bad, and that's an area for you to go and improve the Zalgoishness of your solution. In practice, in Ember, it will almost always be something weird like you're relying on a non-deterministic DOM API or something like this, or you're relying on some XHR that even though we serialized it, you're getting a push notification, and it's different, and it happens quickly, so it's janked. Right? I think the basic point of try to control everything and also only replace things that are needed will get you to a much better starting place out of the gate with Ember than you will have if you try to do the old solutions. It may turn out to be a lot of work. And if it turns out to be a lot of work regardless, then I think it will still, even with Ember, be a thing that is used by people who really need it and not so much by people who don't. But I'm hopeful that the fact that we own the whole lifecycle of your application will let it be useful for a bigger set of people than people who are desperately in need of it as a solution. BRANDON: Awesome, so I think we're going to wrap up. Is there anything you guys want to give a shout-out to or anyone? YEHUDA: Please sign up for Skylight to make your Rails apps faster. TOM: Yeah, please. Make my Christmas a merry one. YEHUDA: We didn't talk about this at all. TOM: And sign up for Skylight. YEHUDA: We didn't talk about this at all, but Tom and I, much of our day job is working on Skylight. And if you watched my talk at Hack Summit, one of the things that I advocate and I really feel it in my gut because of Skylight is I advocate spending, even if you're full time in open source, spending some time, even a day a week or two days a week, would help a lot working on something that you have to maintain over the long haul because, like I said before, maintenance over the long haul is very difficult for you to market. It's something that you have to feel in your heart. If you're working on an open source project, work on -- use it for a real thing that you spend significant time on that you have to maintain over the long haul so you could feel, in a year, whether you're practice is holding up to being maintainable. And, yes, now that I've talked about Skylight for a second, please sign up. This is how we fund our ability to do any open source. Working on Skylight has definitely been the most enjoyable thing I've done in my career so far in the sense that I've had a lot of control over it, but it's also the most harrowing in the sense that we are responsible for getting all the revenue, so please sign up. BRANDON: All right, well, thanks very much, you all, for coming on. Everybody, go sign up for Skylight. It's very cool, very beautiful, and very actionable insight for Rails apps. Tom, Yehuda, thank you so much for joining us on this. It's been super enlightening. Again, everybody, please go watch Yehuda's talk on keeping the Web Indie. And if you've got a little extra time, the Rails Conf talk on layers of abstraction is also, I found, something that changed my views on a lot of stuff as well. Thanks again, both of you all, for coming on. YEHUDA: Thanks a lot. TOM: Thank you, guys. BRANDON: All right, talk to you later. CHARLES: Thank you, guys.

We chat with James Mickens, researcher and most likely funniest man at Microsoft, live on stage during Øredev 2014. Topics include C development, the purity of incrementation, death by specifications, scandinavian death metal and its font choices and also British football, distributed systems and the problems you encounter dealing with them. The downsides of being stuck alone in a set of universes is that Stack overflow can’t help you. And how should we fix the Javascript and web browser technology world? Comments on the internet? No. This recording exists as good as it is thanks to Stephen Chin of nighthacking.com for providing and masterfully wrangling all the necessary technology. Comments, thoughts or suggestions? Discuss this episode at Techworld! Links Freedom fries Microsoft research Distributed systems James' talk at Øredev 2014 Handling the zombie apocalypse - James' article “The night watch” James' talk at Monitorama 2014 Satya Nadella - CEO of Microsoft Nuclear proliferation Segfault Logical shift Logical AND Malmö Logical OR W3C - World wide web consortium Web components Web components templates Gorgoroth Fibonacci Factorial Fizz buzz Sepultura King diamond British steel Notepad Web workers Crom - James' system for speculative Javascript execution Speculative execution Total recall Firebug Galactus Odin The Shellshock bug Turtles all the way down Jquery Angular XHR MDN - Mozilla developer network, great source of Javascript and browser API documentation Git The grandfather paradox Polyfill asm.js NoSQL Mongodb Bare metal Just in time compiler Maslow’s hierarchy of needs let-satement Zeus Quirksmode.org Yo Leviticus Registry ASCII art About:config Max TCP connections Shockwave Singularity DVR - Digital video recorder Futures Bendgate Captain America’s shield Georgia Tomfoolery 4chan Huge sloths Titles This is just Hollywood stuff We’ll edit this out with CGI What happens on the set #freedom #america What prevents me from being a happy person in life It cheapens the art Homelessness is bad too At least you have hope Do you like iteration? I only increment variables Only go forward Black Gandalf Not just fair or balanced Otto von Hyphen We laugh to stop from crying Darkness is delightful The great thing about scandinavian death metal What would Beelzebub do? That TV is in the cloud now Like jazz musicians Like trying to write Inception 2 The left hand of satan Multiple speculative universes What universe am I in? A low-rent Stephen Hawking Firebug had no notion of my separate universes The Odin object In the regular development world Chicanery all the way down Function calls are not our strong point Not nothing will happen XHR:s over passenger pidgeon LOL I took a hard dependency on it It’s very difficult to make a joke in this space We don’t even issue writes devnulldb But we have hoverboards Tread carefully on the polyfills Close the tab and reboot the machine This is such a character builder Mumblefoo.js Fast Javascript, and cancer No-lock cancer Asynchronous cancer That kitten on a tradmill is not going to watch itself Another special type of disaster Folk wisdom on the web 127i content Emu futures If I had a website, I’d run it like Singapore Every computer should come with an old person This whole alternate semantic reality

Panel Christian Johansen (twitter github blog) Joe Eames (twitter github blog) Merrick Christensen (twitter github) Jamison Dance (twitter github blog) Discussion 00:45 - Christian Johansen Test-Driven JavaScript Development Sinon.JS Gitorious 01:26 - Sinon.JS 02:22 - Stubs, Mocks and Spies Mocks Aren’t Stubs: Martin Fowler Mocha 10:47 - History of Sinon.JS 12:25 - XHR, HML, HTTP 13:36 - Mocking the Clock Set Time Out 17:22 - Test-Driven JavaScript Development Andrea Giammarchi @WebReflection The Pragmatic Bookshelf Screencasts 21:43 - Test Framework Buster.JS js-test-driver 24:17 - Other Mocking Libraries mockjax 26:24 - Mocking Properties 27:22 - Matchers 30:46 - Sinon.JS Gotchas 33:10 - State of Test-Driven Development in JavaScript Strategies for Testing Picks Jack Reacher (Joe) Torchlight II (Joe) Effective JavaScript by David Herman (Merrick) Rdio (Merrick) Adventure Time (Jamison) How to implement an algorithm from a scientific paper: Emmanuel Goossaert (Jamison) Advanced Vim registers (Jamison) Emacs Rocks! (Christian) Simple Made Easy (Christian) LEGO Lord of the Rings (Christian) Testing Clientside JavaScript (Joe) Transcript MERRICK:  Classy guy. [Hosting and bandwidth provided by the Blue Box Group. Check them out at Bluebox.net.] [This episode is sponsored by Component One, makers of Wijmo. If you need stunning UI elements or awesome graphs and charts, then go to Wijmo.com and check them out.] JAMISON:  Hello friends. Welcome to JavaScript Jabber. This is Episode number 43. Today, we have Joe Eames. JOE:  Howdy! JAMISON:  Merrick Christensen. MERRICK:  Hey guys! JAMISON:  And Christian Johansen. And also me, Jamison Dance. But Christian is the special guest today. Do you want to talk a little bit about yourself? Introduce yourself for those of us that don’t know you? CHRISTIAN:  Yeah, sure. First of all, hi! I'm in Oslo, Norway up in the cold north. So, I wrote a book about testing JavaScript a couple of years back called ‘Test-Driven JavaScript Development’. And I've done a few open source libraries. Perhaps the one that most people know about is Sinon.JS. And currently, I work at Gitorious.org. So, that’s the brief introduction about me, I guess. JAMISON:  Great! Chuck is gone today. He’s at CES, I believe. So, that’s why I'm filling in for him. I think we want to talk mainly about Sinon.JS today. Do you want to just give an overview of it? CHRISTIAN:  Sure. JAMISON:  For those who have never heard of Sinon.JS, what is it? CHRISTIAN:  Sinon.JS is a stubbing and mocking library which means that when you're writing automated tests for your JavaScript, Sinon provides a tool kit to help you test functions and callbacks and stuff like that, to track how they're being used throughout the system. And then, it also provides some utilities to test asynchronous stuff through timers, like Set Time Out and Set Interval and those kinds of things. And it also has a fake XMLHttpRequest implementation. So, it allows you to test your client side JavaScript completely decoupled from the server and it gives you an API to mimic the role of the server in your tests. So, you can focus a test on how the client side reacts to various kind of behavior from the server. JAMISON:  So, you talked about stubbing and mocking. And I think, that means we have to get into the hairy discussion of the difference between stubs and mocks? MERRICK:  And spies. JAMISON:  And spies, yeah. Do you want to explain that a little bit? CHRISTIAN:  Sure. I can explain my take on it because I know there are more than just mine. MERRICK:  Sure. CHRISTIAN:  I'm using the terminology pretty much like Martin Fowler did and he has a famous article called ‘Spies are Not Mocks’ or something like that. So,

Panel Christian Johansen (twitter github blog) Joe Eames (twitter github blog) Merrick Christensen (twitter github) Jamison Dance (twitter github blog) Discussion 00:45 - Christian Johansen Test-Driven JavaScript Development Sinon.JS Gitorious 01:26 - Sinon.JS 02:22 - Stubs, Mocks and Spies Mocks Aren’t Stubs: Martin Fowler Mocha 10:47 - History of Sinon.JS 12:25 - XHR, HML, HTTP 13:36 - Mocking the Clock Set Time Out 17:22 - Test-Driven JavaScript Development Andrea Giammarchi @WebReflection The Pragmatic Bookshelf Screencasts 21:43 - Test Framework Buster.JS js-test-driver 24:17 - Other Mocking Libraries mockjax 26:24 - Mocking Properties 27:22 - Matchers 30:46 - Sinon.JS Gotchas 33:10 - State of Test-Driven Development in JavaScript Strategies for Testing Picks Jack Reacher (Joe) Torchlight II (Joe) Effective JavaScript by David Herman (Merrick) Rdio (Merrick) Adventure Time (Jamison) How to implement an algorithm from a scientific paper: Emmanuel Goossaert (Jamison) Advanced Vim registers (Jamison) Emacs Rocks! (Christian) Simple Made Easy (Christian) LEGO Lord of the Rings (Christian) Testing Clientside JavaScript (Joe) Transcript MERRICK:  Classy guy. [Hosting and bandwidth provided by the Blue Box Group. Check them out at Bluebox.net.] [This episode is sponsored by Component One, makers of Wijmo. If you need stunning UI elements or awesome graphs and charts, then go to Wijmo.com and check them out.] JAMISON:  Hello friends. Welcome to JavaScript Jabber. This is Episode number 43. Today, we have Joe Eames. JOE:  Howdy! JAMISON:  Merrick Christensen. MERRICK:  Hey guys! JAMISON:  And Christian Johansen. And also me, Jamison Dance. But Christian is the special guest today. Do you want to talk a little bit about yourself? Introduce yourself for those of us that don’t know you? CHRISTIAN:  Yeah, sure. First of all, hi! I'm in Oslo, Norway up in the cold north. So, I wrote a book about testing JavaScript a couple of years back called ‘Test-Driven JavaScript Development’. And I've done a few open source libraries. Perhaps the one that most people know about is Sinon.JS. And currently, I work at Gitorious.org. So, that’s the brief introduction about me, I guess. JAMISON:  Great! Chuck is gone today. He’s at CES, I believe. So, that’s why I'm filling in for him. I think we want to talk mainly about Sinon.JS today. Do you want to just give an overview of it? CHRISTIAN:  Sure. JAMISON:  For those who have never heard of Sinon.JS, what is it? CHRISTIAN:  Sinon.JS is a stubbing and mocking library which means that when you're writing automated tests for your JavaScript, Sinon provides a tool kit to help you test functions and callbacks and stuff like that, to track how they're being used throughout the system. And then, it also provides some utilities to test asynchronous stuff through timers, like Set Time Out and Set Interval and those kinds of things. And it also has a fake XMLHttpRequest implementation. So, it allows you to test your client side JavaScript completely decoupled from the server and it gives you an API to mimic the role of the server in your tests. So, you can focus a test on how the client side reacts to various kind of behavior from the server. JAMISON:  So, you talked about stubbing and mocking. And I think, that means we have to get into the hairy discussion of the difference between stubs and mocks? MERRICK:  And spies. JAMISON:  And spies, yeah. Do you want to explain that a little bit? CHRISTIAN:  Sure. I can explain my take on it because I know there are more than just mine. MERRICK:  Sure. CHRISTIAN:  I'm using the terminology pretty much like Martin Fowler did and he has a famous article called ‘Spies are Not Mocks’ or something like that. So,

Panel Christian Johansen (twitter github blog) Joe Eames (twitter github blog) Merrick Christensen (twitter github) Jamison Dance (twitter github blog) Discussion 00:45 - Christian Johansen Test-Driven JavaScript Development Sinon.JS Gitorious 01:26 - Sinon.JS 02:22 - Stubs, Mocks and Spies Mocks Aren’t Stubs: Martin Fowler Mocha 10:47 - History of Sinon.JS 12:25 - XHR, HML, HTTP 13:36 - Mocking the Clock Set Time Out 17:22 - Test-Driven JavaScript Development Andrea Giammarchi @WebReflection The Pragmatic Bookshelf Screencasts 21:43 - Test Framework Buster.JS js-test-driver 24:17 - Other Mocking Libraries mockjax 26:24 - Mocking Properties 27:22 - Matchers 30:46 - Sinon.JS Gotchas 33:10 - State of Test-Driven Development in JavaScript Strategies for Testing Picks Jack Reacher (Joe) Torchlight II (Joe) Effective JavaScript by David Herman (Merrick) Rdio (Merrick) Adventure Time (Jamison) How to implement an algorithm from a scientific paper: Emmanuel Goossaert (Jamison) Advanced Vim registers (Jamison) Emacs Rocks! (Christian) Simple Made Easy (Christian) LEGO Lord of the Rings (Christian) Testing Clientside JavaScript (Joe) Transcript MERRICK:  Classy guy. [Hosting and bandwidth provided by the Blue Box Group. Check them out at Bluebox.net.] [This episode is sponsored by Component One, makers of Wijmo. If you need stunning UI elements or awesome graphs and charts, then go to Wijmo.com and check them out.] JAMISON:  Hello friends. Welcome to JavaScript Jabber. This is Episode number 43. Today, we have Joe Eames. JOE:  Howdy! JAMISON:  Merrick Christensen. MERRICK:  Hey guys! JAMISON:  And Christian Johansen. And also me, Jamison Dance. But Christian is the special guest today. Do you want to talk a little bit about yourself? Introduce yourself for those of us that don’t know you? CHRISTIAN:  Yeah, sure. First of all, hi! I'm in Oslo, Norway up in the cold north. So, I wrote a book about testing JavaScript a couple of years back called ‘Test-Driven JavaScript Development’. And I've done a few open source libraries. Perhaps the one that most people know about is Sinon.JS. And currently, I work at Gitorious.org. So, that’s the brief introduction about me, I guess. JAMISON:  Great! Chuck is gone today. He’s at CES, I believe. So, that’s why I'm filling in for him. I think we want to talk mainly about Sinon.JS today. Do you want to just give an overview of it? CHRISTIAN:  Sure. JAMISON:  For those who have never heard of Sinon.JS, what is it? CHRISTIAN:  Sinon.JS is a stubbing and mocking library which means that when you're writing automated tests for your JavaScript, Sinon provides a tool kit to help you test functions and callbacks and stuff like that, to track how they're being used throughout the system. And then, it also provides some utilities to test asynchronous stuff through timers, like Set Time Out and Set Interval and those kinds of things. And it also has a fake XMLHttpRequest implementation. So, it allows you to test your client side JavaScript completely decoupled from the server and it gives you an API to mimic the role of the server in your tests. So, you can focus a test on how the client side reacts to various kind of behavior from the server. JAMISON:  So, you talked about stubbing and mocking. And I think, that means we have to get into the hairy discussion of the difference between stubs and mocks? MERRICK:  And spies. JAMISON:  And spies, yeah. Do you want to explain that a little bit? CHRISTIAN:  Sure. I can explain my take on it because I know there are more than just mine. MERRICK:  Sure. CHRISTIAN:  I'm using the terminology pretty much like Martin Fowler did and he has a famous article called ‘Spies are Not Mocks’ or something like that. So,