POPULARITY
This week we talk to Charles Lowell, a developer and consultant who has created a library called Effection. Effection is a library that allows you to write structured concurrency code in JavaScript. What is structured concurrency and how could it be useful for you? Find out in this episode!https://frontside.com/effectionhttps://github.com/thefrontside/effectionhttps://frontside.com/effection/contrib/https://frontside.com/Become a paid subscriber our patreon, spotify, or apple podcasts for the ad-free episode.https://www.patreon.com/devtoolsfmhttps://podcasters.spotify.com/pod/show/devtoolsfm/subscribehttps://podcasts.apple.com/us/podcast/devtools-fm/id1566647758https://www.youtube.com/@devtoolsfm/membership
Coté talks with Charles Lowell and Taras Mankovski about the Backstage work they've been doing with clients. First, they talk about what Backstage is in the first place. Obviously, it's an internal developer portal...thing...but what does that mean beyond, like, a wiki? Check out The Frontside, and also, Taras' talk at BackstageCon. Watch the video of this recording if you prefer that kind of thing.
Coté talks with Charles Lowell and Taras Mankovski about the Backstage work they've been doing with clients. First, they talk about what Backstage is in the first place. Obviously, it's an internal developer portal...thing...but what does that mean beyond, like, a wiki? Check out The Frontside, and also, Taras' talk at BackstageCon. Watch the video of this recording if you prefer that kind of thing.
When it seems like everyone around you has worked in the same field for a really long time, making a career pivot with confidence can be tricky. But not everyone's been coding since their early college days like Robbie and Chuck. Kara Luton started on track to become a professional ballerina. After college and a stint in music publicity, burnout prompted Kara to make a hard left and begin a career in tech. With all the developer bootcamps and online resources now available, making the switch has never been more accessible. Not to mention, the skills Kara learned as a ballerina and a music publicist helped shape the developer she is today. From staying dedicated and detail-oriented, learning to write and learning from burnout, Kara wouldn't change anything about her unconventional path to software. In this episode, Chuck and Robbie talk with Kara about her experience learning and relearning Ember, why she loves the Ember community, her advice for those looking to switch careers, Kara's cool home office, and why every developer has something valuable to offer. Key Takeaways [00:58] - A brief introduction to Kara. [03:16] - A whiskey review. [08:51] - Kara's non-traditional path to tech. [15:57] - Kara's experience in a bootcamp and her thoughts on bootcamps as a developer launchpad. [17:34] - How Kara found Ember. [23:10] - Kara's advice for people looking to make a career pivot. [28:44] - Why Kara's looking forward to contributing to open source projects. [32:30] - How Kara's home office setup has evolved. [37:57] - Kara's thoughts on NFTs. [40:17] - Why Kara loves animals and a deep dive on her two pet dogs. [47:48] - More of Kara's hobbies outside of the web and a chat about Marvel movies. [58:48] - A soccer and sports-themed whatnot. Quotes [15:20] - “Ballet, it's very detail-oriented and I feel like that's something that's really helped me in my career as a developer, like missing a semicolon or understanding the different syntaxes — it's really helped me a lot. I'm really really grateful for my time doing ballet.” ~ Kara Luton [29:37] - “Contributing to the framework that you use will give you such good knowledge of it, even if it's something small.” ~ Kara Luton [31:59] - “You never know if something you say, the way you phrase something, will just make it click for somebody in a way that they haven't understood it before. I really really recommend people writing blog posts.” ~ Kara Luton Links Kara Luton CrowdStrike Glimmer.js Three Chord Bourbon Strange Collaboration Nelson's Green Brier Distillery Nashville Predators Joffrey Ballet School Summer Intensives Belmont University freeCodeCamp Codecademy Ember.js Ryan Tablada Rock & Roll with Ember.JS Ember Octane Dev.to Ed Faulkner CodeNewbie Jessica Rose Alex Graul National Geographic Dependabot Ember Learning Team The Ember Times The Difference Between Regular Functions and Arrow Functions Introduction to CSS Grid: What You Should Know Kara's work from home setup Yeti Blue Microphones Amazon Quartet Whiteboards Lume Cube The Frame TV Damien Hirst's NFT Bored Ape Yacht Club Web3 is going just great Taras Mankovski Whiskey Web and Whatnot: Alternatives to Relay, the GraphQL Stack, and Adulthood with Charles Lowell and Taras Mankovski Connect with our hosts Robbie Wagner Chuck Carpenter Ship Shape Subscribe and stay in touch Apple Podcasts Spotify Google Podcasts Whiskey Web and Whatnot Top-Tier, Full-Stack Software Consultants This show is brought to you by Ship Shape. Ship Shape's software consultants solve complex software and app development problems with top-tier coding expertise, superior service, and speed. In a sea of choices, our senior-level development crew rises above the rest by delivering the best solutions for fintech, cybersecurity, and other fast-growing industries. Check us out at shipshape.io.
Just because something is widely used doesn't always mean it's your best solution. Frontside Founder Charles Lowell and CEO Taras Mankovski, stumbled into an alt GraphQL stack simply because the nature of a product didn't mesh with Apollo. After happening upon two up-and-coming technologies, GraphQL modules and Envelope, a solution was born, as was a newfound flexibility with GraphQL stacks. In this episode, Charles and Taras talk with Chuck and Robbie about their accidental developer discovery, the drawbacks of UI libraries, what a Relay alternative looks like, what in the world Pact is, and why adulthood is vastly overrated. Key Takeaways [00:48] - An introduction to the Frontside guys. [02:29] - A whiskey review. [08:46] - How Charles and Taras discovered a less-than-ordinary GraphQL stack. [18:39] - Why JSON:API doesn't always make sense. [23:11] - Taras' criteria for a valuable alternative to Relay. [25:04] - What is Pact? [28:30] - An NFT chat, and why adulthood is vastly overrated. [41:45] - Charles' and Taras' hobbies outside of the web and the best way to bond with your baby. [54:38] - A few last-minute mentions. Quotes [21:04] - “Relay is complex, it's difficult, and it's not as magical as other things that I've used. So I actually don't think that the primary benefit is to the clients that consume it, ironically. I think the benefit is to the developers that are trying to understand.” ~ Charles Lowell [56:20] - “The combination of testing and simulation and the developer experience stuff, and the emergence of developer experience as an area of focus is exciting and interesting in the same way that web and Ember was when it started. Just that sense of, we're discovering something new and there are people who are actively trying to solve a problem.” ~ Taras Mankovski Links Charles on Twitter Taras on Twitter Frontside The Balvenie Doublewood 12 The Singleton of Glendullan Liberty GraphQL Apollo Discord Envelope JSON:API runspired Whiskey Web and Whatnot: Discovering Ember, Adopting Orbit, and Unlocking Optimization with Chris Thoburn (runspired) Ember Data Orbit Relay Pact Swach Blockchain Web3 No JS Rails The Guild Hive GraphQL CodeGen The Sandbox Snoop Dog Second Life Linden Lab Duolingo Casamigos Deno GoJS Backstage Connect with our hosts Robbie Wagner Chuck Carpenter Ship Shape Subscribe and stay in touch Apple Podcasts Spotify Google Podcasts Whiskey Web and Whatnot Top-Tier, Full-Stack Software Consultants This show is brought to you by Ship Shape. Ship Shape's software consultants solve complex software and app development problems with top-tier coding expertise, superior service, and speed. In a sea of choices, our senior-level development crew rises above the rest by delivering the best solutions for fintech, cybersecurity, and other fast-growing industries. Check us out at shipshape.io.
"I don't care about networking...and load balancing." There's a lot of new concepts and stuff to learn when it comes to developing applications that will run on kubernetes. In this episode, Coté talks with Charles Lowell (https://twitter.com/cowboyd) about his experience. Also, we imagine measuring the humidity of mayonnaise. If you need some excellent app coding, check out Charle's company, Frontside (https://frontside.io/)! They also have a podcast (https://frontside.io/podcast) where they discuss recent programming frameworks and idea, and relating coding cool stuff. You may recall that Charles was the co-host of Coté's first podcast empire, DrunkAndRetired.com (https://archive.org/details/DrunkAndRetired). Special Guest: Charles Lowell.
Philippe De Ryck joins the show to talk all things security: the importance and why you should be taking active steps, how to do it in your codebase effectively, and what can happen during a breach. Philippe De Ryck: Pragmatic Web Security Resources: OWASP Top 10 OWASP Top 10 Proactive Controls Please join us in these conversations! If you or someone you know would be a perfect guest, please get in touch with us at contact@frontside.io. Our goal is to get people thinking on the platform level which includes tooling, internalization, state management, routing, upgrade, and the data layer. This show was produced by Mandy Moore, aka @therubyrep of DevReps, LLC. Transcript: CHARLES: Hello and welcome to The Frontside Podcast, a place where we talk about user interfaces and everything that you need to know to build them right. My name is Charles Lowell, a developer here at The Frontside. Joining me, also hosting today is Taras Mankovsky. Hello, Taras. TARAS: Hello, hello. CHARLES: And as always, we're going to be talking about web platforms, UI platforms, and the practices that go into them. And here to talk with us today about a pillar of the platform that I certainly don't know that much about, and so, I'm actually really happy to have this guest on to talk about it is Philippe De Ryck who owns his own company called Pragmatic Web Security. I understand you do trainings and are just generally involved in the small space. So, welcome, Philippe. PHILIPPE: Hi. Nice to meet you. CHARLES: Wow! I almost even don't even know where to start with this subject because I'm kind of like the hippie developer mindset where it's like, "LaÖlaÖlaÖlaÖlaÖ we're in this open land and nothing's ever bad going to happen and we're just going to put code out there," and nobody would ever take advantage of any holes or anything like that. And I think that a lot of developers share that mentality and that's how we end up with major, major security breaches. And so, like I said, this is something that I'm actually very eager to learn but I almost even don't know where to start. I need training, man. [Laughter] PHILIPPE: Well, that's good to hear. No, you're totally right about that. If you're not into security, it seems like this fast space for a lot is happening and you don't really know how or why and what really matters to you and should I even be worried about this. And let me start by addressing the very first thing. Yes, you should be worried because maybe you're not building something that somebody cares about but you always have something that somebody wants, even the simplest of attacks always targets a valuable resource. Just to give you a very simple idea today, cryptocurrency is all the hype and you have a taker that's just aiming to misuse your users' computers to mine crypto coins because it essentially saves them a bunch on electricity cost. So, there's always something to grab. Usually, it's data or services or worse. But even in the most minimal cases, you have hardware, you have devices, you have network capacity that somebody might want to abuse. So yes, security, I would say, always matters. CHARLES: What's the best way to get started? You said understanding that everything we do, we're holding onto resources that might be valuable, that someone might want to seize but I'm just getting started with my application. I don't know anything about security. Where do I get started on just understanding the space? And then before I even look at tools that I wantÖ PHILIPPE: You want the honest answer for that? [Laughter] PHILIPPE: The honest answer is probably hire someone who has security knowledge. I don't mean this in a bad way. I've come a very long way in my career doing what I do now. And if I look at that, if you are aiming as a developer with no knowledge about security to build a secure application, it's going to be very hard. There's a lot of things you need to know, intrinsic knowledge. These are not things you can simply read a small book in a week, you know all of these security things that you'll know what to do. So, if you have no previous experience at all, I suggest to find some help. CHARLES: Right. It's like saying, "Hey, you've never written a data layer before but you want to go out and you want to write a massively distributed system where you have all these notes talking to each other. You're not going to read the O'Reilly book 'How to Build Distributed Systems' in a week and go out and do the same thing." It's the same thing with security. You need to understand the entire context. And there's no substitute for experience. PHILIPPE: Sorry, I actually like that comparison because in a sense, you're right, it's like these other very complex topics you don't expect to learn that in a week or a month and right a functioning data layer. But the difference is if you fail at writing that data layer, your application is probably not going to work. While if you fail at securing the application or seeing potential vulnerabilities, it's still going to work just a bit more than you anticipated. It's going to result leaking all your data to an attacker or opening all doors so that they can gain access to your server, stuff like that. So, I would say that the consequences of not getting it right are, at least in the beginning, very invisible. It's only after things happened that it's like, "Oh, crap!" And you should pay attention to that. CHARLES: Yeah. And then you have these back doors and these leaks that are set in stone and may be very hard to change. PHILIPPE: Yeah, absolutely. And honestly, the worst part of the breach is for a company, it might be reputation damage. But what you really should be worried about is all that personal information that's being leaked to, most cases, publicly leaked to anyone but in other cases, it's sold on [inaudible] markets and actually abused by people looking for someone else's identity or credit card information or stuff like that. And the companies usually get away with some bad press, maybe a small dip in their stock price but eventually, they'll bounce back. But it's the users that suffer from these breaches for a very, very long time. TARAS: What do you see the kind of hot zones around concerns that companies have around security? Because I imagine it's hard to be concerned about everything, so they're probably thinking about specific things like this thing worries us. Like what kind of things do you see companies and teams need to worry about? PHILIPPE: That's an interesting question. You have all different kinds of companies and all different levels of security awareness and what they're worrying about. I would say if you have the companies that are not very good at or don't have very much security knowledge, they're probably not to worry about things because otherwise, they would have started investing in improving their practices. If you look at the companies that are at least very aware of the landscape, I'm not saying that anybody here is perfect, but some of the companies are actually doing quite a good job. One of the most interesting challenges today is dealing with dependencies. So, all of your packages, you depend on npm, Maven, Python packages, Ruby gems, and so on. All of them form a huge attack factor in most applications today. That's definitely a problem that a lot of companies struggle with and it's very hard to find good solutions. TARAS: GitHub recently, I saw their vulnerability alert service that I've been getting a lot of notifications from on some of the open source libraries that we use. They have a lot of dependencies. And a lot of projects have the same dependencies. So, the moment that one notification goes on, it like lights up on all of the GitHub repos that I have. So, I have been going through and like updating dependencies and all those libraries. PHILIPPE: Yeah, that's a very good example. Absolutely. A lot of the projects we build today usually start out by installing a bunch of dependencies. Even before you've written the first line of code, you already have a massive code base that you are relying upon. And a single vulnerability in a code base might be enough, it's not always the case, but it might be enough to compromise your application. And that leaves you, as a developer, in a very hard place because you haven't written any lines of code yet. You have built a vulnerable application and that starting point can be very terrifying. So, there's a lot of reports on this. And actually, if you want some numbers, 78% of the vulnerabilities discovered in existing applications like the ones you mentioned. If GitHub alerts you like, "Hey, there's a problem in one of your dependencies," it's often even an indirect dependency, meaning that you include a framework, if you include React or Express or whatever you're building, that one of your dependencies of one of those projects actually has a vulnerability. If you look at the trees of these packages, they get quite big that it's not dozens but it's thousands of packages that we're talking about. CHARLES: Yeah that's the other thing is how do you know how to interpret these security vulnerabilities because some of them, we get a lot of security vulnerabilities for node packages but we only use them for our development tools to build frontend. So, if we're building a React application and there's some security vulnerability in some node packages that we're using in our build tool, then that doesn't actually get deployed to the frontend. So, maybe it's not a concern but if we were actually using it to build a server, then it would be absolutely critical. And so, how do you evaluate because the same security vulnerability is not a vulnerability in one context, but might be in another or maybe I'm thinking about it wrong. You see what I mean? PHILIPPE: Yeah, sure. I totally get what you mean. Actually, I have observed the same things. I also get these security alerts on my projects, and sometimes it's devDependency, so it seems like we don't need to care about that. You're right in the sense that you have to assess the criticality of such a report. So, they will have a rating, a severity rating saying like, "This is a minor issue," or, "This is a major issue," that should be a first indication. And then a second thing to look at is, of course, how are these things used in practice. It's not because it's a devDependency that it's not exploitable because it all depends on what is the vulnerability. If there's an intentional malicious backdoor in the library and you're building that on your build server, it might give an attacker access to your build server. So, that might not be something you actually want to do. So in that case, it does matter. Of course, if it's only stuff you run locally, you can say like, "OK, this is less important." But usually, updating or fixing these vulnerabilities also requires less effort because there's no building and deploying to production servers either. So, it's a matter of staying up-to-date with these. And one of the things that people struggle with is handling this in a lot of different applications. You mentioned you had a lot of GitHub repos and the vulnerability starts popping up in all of them and you have to fix and update all of them. You can imagine that major companies struggle with that, as well, especially if you have quite a few different technologies. Managing all of that is insanely hard. CHARLES: Right, because you just usually look at it and you're like, "Oh, I've got to download this." And maybe, "I haven't used it this repo for a while. I've got to clone it up, I've got to update the dependency. I've got to make sure I run all my tests locally, then run all the tests in CI and make sure I didn't break anything by upgrading. I might have fixed closed security hole but broken my functionality." And so, make sure that that is all intact and then push it out to production. Even on the small, it's like I'm looking, "OK, maybe this is going to take me 30 to 45 minutes." But if you have four or five of those things, you're looking at half your day or maybe even the whole day being gone and that's if you have the processes in place to do those automated verification. If you have a very high confidence in your deployment pipeline which I don't think a lot of places have. So, it sounds like these are complementary, like you really need in order to keep a secure application, you have to keep it up-to-date because I think what I'm hearing is you should just evaluate all the threats. You should fix it if you can. The first part of my question is, am I kidding myself when I say, "Oh, I can ignore this one because it's just local or it's just a devDependency." PHILIPPE: The answer to that question is briefly, I would say they are less critical. CHARLES: That's cool. PHILIPPE: In general, the rule is update if you can. And actually some of the tools out there that monitor vulnerabilities, they will automatically create a pull request in your repo saying to upgrade to this version and then you can automatically run your tests if you have them, and you can very quickly see whether some conflicts are generated by updating that dependency - yes or no. And in most cases, if it's a minor version bump, it's going to work as expected and you can easily push out the new version without that vulnerability. So, I would say fix if you can. If it goes quickly, then definitely fix them. But I would focus on non-devDependencies first instead of devDependencies. CHARLES: Yeah. PHILIPPE: Second thing I wanted to add is you paint a very grim picture saying you have to spend a lot of time updating these issues and I can totally understand that happening the very first time you look into this. There's going to be some stuff in there, I can guarantee that. But if you do this regularly, the effort becomes less and less because once you have up-to-date libraries, the problem is bad but it's not like we have 50 new vulnerabilities every day, fortunately. CHARLES: Right. PHILIPPE: So, once you have done that, it's going to be a bit less intensive than you might anticipate at first glance. Of course, if you're using these projects, if you're reusing the same library, then you'll have to update them everywhere. That's the downside, of course. CHARLES: It's probably a little bit dangerous to be assessing the criticality of the security threats yourself if you're not an expert, and kind of in the same way, it's dangerous to be assessing an architecture if you don't have an expertise in our architecture, I guess is the thing, because you might not understand the threat. PHILIPPE: Yeah, that's, again, absolutely true. It again depends very much on how it's deployed and what it's used for. That's going to be one important aspect. Another thing that might be very useful is, how deep is the dependency that creates the vulnerability or has the vulnerability? Because for example, if you have your tree of dependencies, if you dependency is like five or six levels deep, the chances of malicious data are reaching that specific vulnerability, and that specific library is going to be fairly small. Because usually, libraries have a lot of features and you only use part of them in your application. So, the other one is address of the features is just sitting there and if it's never used and it's also not exploitable. So, that might play a role as well. I saw a presentation about a month or two months ago from how Uber manages these things and they struggled with a lot of those things as well. And they eventually decided that they really care about vulnerabilities going three levels deep. And something that goes deeper is considered to be less relevant or less urgent to update because chances of exploitability are going to be very small. CHARLES: That's actually really interesting. TARAS: One thing that got me thinking about something that is actually happening right now. A friend of mine has a WordPress site that was hacked. But what's interesting about WordPress, I think the fact that WordPress site was hacked is not really a surprise but I think what's interesting about that is that the frequency and the sophistication of these attacks has increased. The tooling has improved also in the WordPress ecosystem. But at the same time, I think there is actually more people that are aware of the kind of exploits that could be done. There are a lot of people going after WordPress sites, but it kind of makes me think that there's probably going to be a time when the vectors of attack for web applications are going to become pretty well known as well. Because of the architecture, there are a fewer of them. But as the awareness of the actual architecture becomes more common, I think the angles of attack are going to become more interesting. Like one of the things that I was reading about a couple days ago is that there are some researchers that found a way to attract users based on a combination of JavaScript APIs that are available in the browser. So, they are actually able to fingerprint users based on the kind of things that they're using, the application for [inaudible] extensions they have installed. I think people are going to get more creative. And that's kind of scary because we've seen this happen already in WordPress and people are greedy. So, there are going to be ways. I think there's going to be more people looking at how to get into and how to exploit these vulnerabilities. PHILIPPE: Yeah. That's actually a couple of very good examples that illustrate the underlying issue. So, this browser-based tracking of users, it's called browser fingerprinting and it's been going on for a while. Back when I did my PhD, I had colleagues working on those things and you have other people at universities doing research on this. And yes, you can use things like JavaScript APIs in the browser to identify a particular user with a very high probability. It's not perfect but it's usually enough to identify a user for ad tracking or those purposes. By the way, these things also have a legitimate purpose. So, they are also used to keep track of a particular user to prevent things like session hijacking or detect problem logins or stuff like that, so they can also have a legitimate use case next to tracking. But they very clearly show how security will always be a cat and mouse game. Tracking used to be easy. You just set a cookie in a browser and the cookie was there next time and you knew who the user was. And then, users became a bit more savvy. You had browser extensions trying to block listings because let's be honest, they're kind of shady. So, users probably don't want that. And then the attacker started moving towards other things and getting more advanced. And you see that in other areas of security, as well. So, I consider that a good thing because as we make things harder for attackers, they will have to get more creative and it will become more difficult to exploit or to take advantage of applications. That's the good side. The bad side or the dark side of that equation is that unfortunately, the vulnerabilities are not going away. It's not because we now have these somewhat more advanced attacks using advanced features or even CView-based vulnerabilities that the old things like SQL injection and [inaudible] have disappeared in applications. That's also not true and that means that it makes it just a bit more harder for everyone on the defensive side to build more secure applications. You're going to have to know about the old stuff and you have to learn about the new stuff. CHARLES: Again, we come back to that idea. It's all a bit overwhelming. Aside from the solution of like, "Hey, let's hire Phillippe. Let's hire some other security expert." We were actually in your training, and obviously, I don't want to divulge all the secrets or whatever. If we were to attend your training, what do you see is the most important thing for people to know? PHILIPPE: There's no secrets there. [Chuckles] What I teach is web security. I kind of like to think I teach that in a very structured and methodical way. But in the end, there's no secrets and I don't mind talking about this here on the podcast because I honestly believe that everyone should know as much as they can about security. What do I teach? I can talk about specifics but I can also talk about generic things. One of the general takeaways is that one of the best things in my opinion that a developer can do is realize when they don't know something and actually admit that they don't know something, instead of just doing something. Maybe having like a brief thought like, "Hmm, is this secure? Well, it's probably good. I'm going to deploy it anyway. We'll see what happens." That is not the right way of doing things. If you do something and you recognize like, "Hey, this might be security sensitive. We're dealing with customer information here. We're dealing with healthcare information. We might want to look at what plays a role here," and then you can go ask someone who does. You probably have a colleague with a bit more security knowledge, so you can ask him like, "Hey Jim, or whatever your name is, do you think that this is OK or should we do something special here?" Very much like you are doing, asking me questions right here. That's one important takeaway that I hope everyone leaves with after a training class because not knowing something and realizing that you don't know it allows you to find someone who actually does. That still leaves us with that point which you wanted to sidestep. CHARLES: [Chuckles] PHILIPPE: A second thing is to realize that security is not a target. It's not something you're going to hit. It's not a holy goal that after working really hard for two years, you're going to hit this security milestone and you're done. It's always going to be a cat and mouse game. It's always going to be a moving target but that's OK. That's how things are. And that's the same with all other things in the world essentially. It's an evolving topic and you'll need to be ready to evolve with that as well. TARAS: One of the challenges that I see into quite often in teams is that at the individual level, people really try to do their best, maybe the best of their abilities. But it's often, when it comes to being part of a group, it's often like they do best within the kind of cultural environment that exists. I'm curious if you've seen good or kind of environments or cultures for engineering teams that are conducive to good security. Are there kind of systems or processes the companies put in place that you've seen to be very effective in preventing problems? Have you encountered anything like this? PHILIPPE: Ideally, you have developers that are very well educated about security but honestly, it's going to be insanely hard to find these people because a developer not only has to be educated about security, they also need to know about UI design and JavaScript frameworks and other frameworks and all of these things. And it's virtually impossible to find someone up-to-date on all of these things. So, what most companies do today that seems to work quite well, even though it's very hard to judge whether it's working or not, is they work with security champions. So, you typically have a dev team and within a dev team, you would have a security champion, one or two or five, depends on how large your teams are, of course, that is knowledgeable about security. So, that developer has some knowledge. He's not an expert but he knows about common attacks and common dangers and how to potentially address them in the application. So, having that person embedded in the team allows the team to be security aware because when you have a team meeting like, "Hey, how are we going to solve this particular problem?" That person will be able to inject security knowledge like, "Hey, that seems like a good idea but if we're using SQL in the backend, we need to ensure that we don't suffer from SQL injection." Or if you're using a NoSQL database, it's going to be NoSQL injection and so on. And that already elevates the level of security in the team. And then, of course, security champions themselves are not going to be security experts. They're mainly developers just with a security focus. So, they should be able to escalate problems up to people with more knowledge, like a security team which can be a small security team within their organization that people can easily reach out to, to ask like, "Hey, we're doing something here and I know that this is security relevant and I'm not entirely sure what's happening here. So, can we get a review of this part of your application?" Or, "Can you guys sit on the meeting to see what's going on and what's happening there?" And I think that structure also makes sense. It's still going to be hard to build secure applications because there's still a lot of things to address, but at least, your teams get some awareness. And then of course, you can help your security champions to become better and they will get better over time. You can augment them with the security architects. You can train your security champions separately with more in-depth knowledge and so on. And that veteran or that setup seems to work quite well in many large organizations today. CHARLES: Yeah. I like that. It gets me to thinking, so having the having the security champions, having people who have this as part of, not their specialization, but at least part of their focus, being in the room, being part of the conversation because we try and do that and provide that service when it comes to UI but we also have a bunch of processes that kind of automate the awareness of quality. So, the classic one is your CI pipeline, your deployment pipeline. So, you're automating your advancement to production. You're automating your QA. It's still no substitute for having someone who's thinking about how to have that quality outcome but you still have some way of verifying that the outcome is quality. Are there tools out there that you can do to kind of keep your project on the security Rails. I'm thinking something that we we've done recently is having preview apps, so that we get a tight feedback loop of being able to deploy a preview version of your application that's on a branch but it's talking to a real backend. There's a lot of more software and services that are supporting this and it's kind of become an integral part of our workflow. So, testing automated deployment preview apps, there's this kind of suite of tools to make sure that the feedback loops are tight and that the quality is verified even though you have people, you also have people guiding that quality. It's just making sure that the standards are met. Is there a similar set of tools and processes in the security space so that we've got these champions out there, they're being part of the conversations. They're making suggestions but they can't be everywhere at once. And is there a way to make sure that the kind of the ways that they're guiding the application, just verifying that the application is going in that direction? Or an alarm bell has sounded. We mentioned one which is the automated pull request with the, "Hey, you got this dependency and there was a pull request." Are there more things like that, I guess, is what I'm saying. PHILIPPE: Yes, there are. But I would dare to say not enough. So yes, you have some security tools you can integrate in your pipeline that do some automated scanning and they tried to find certain issues and alert you of those issues. So, these things do exist but they have their limitations. A tool can scan an application. Some of the findings are going to be easy and fairly trivial, but it's good to have the check in place nonetheless. But some of the more advanced issues are very likely to be undetectable by those automated tools because they require a large amount of skill and expertise to actually craft and exploit to abuse that particular feature in an application. So, we do have some limitations but I like discretion because I do believe that we need to leverage these mechanisms to ensure that we can improve the security quality of our applications. A very simple thing you can do is you can run an automated dependency check when you build the application and you can use that to decide to halt deployment when it's a severe vulnerability or go ahead anyway when you consider this to be acceptable because if you automate all of those things, things can go wrong as well. We can talk about that in a second. So yeah, these things can be done. But what I strongly encourage people to do to ensure that they can kind of improve the code quality is to flag certain known bad code patterns. So if you're building an Angular or a React application, if you're using functions that output go directly into the template, that's going to be very dangerous. So, we know these functions in Angular, they're called bypassSecurityTrustHtml, bypass security should be kind of a trigger and this kind of security irrelevant. And in React, that property is called Dangerously Set innerHTML, also indicating like a 'developer watch out what you're doing'. So, what you could do is you could set up code scanning tools that actually flag these things whenever they appear in application because sometimes people make mistakes. You hire an intern and they don't really know the impact of using that property and they use it anyway which would cause cross-site scripting vulnerability. If you're code scanning to flag these things ensures that it doesn't get pushed to production unless it's a benign case which is actually approved to be in there, then you can definitely stop some of these attacks coming on for sure or some of these vulnerabilities happening. TARAS: I think the hardest thing to understand is when someone doesn't understand what they're doing that what they will create is so cryptic that I think any tool that tries to figure out what it is that person is doing I think will have a really hard time. The person making the thing doesn't understand what they're doing, then the system is not going to understand what they're doing which makes me think that one of the things that we think about a lot at Frontside is this idea of trying to understand the system from the outside as kind of looking at a system as a black box and wonder what kind of tools are available specifically for inspecting the application from the outside, like as if somehow understanding what the application is doing based on what's actually going on inside of the runtime and then notifying someone that there could be something off in the application, but through exercising the [inaudible] things like, for example, memory leaks is not something you can catch unless you have a test suite that has like a thousand tests and then you will see over time that your application is actually leaking memory. But if you run individual tests, you'll never see that. I wonder if there's anything like that for security where at runtime, there's actually a way to understand that there might be some kind of a pattern that's incorrect in the application. PHILIPPE: If only, if only. It depends on who you ask. There is such a concept that's called Dynamic Application Security Testing. Essentially, what you do there is you run the application, you feed it all kinds of inputs, and you monitor when something bad happens. And that means that you have detected vulnerability. So, these things do exist. But unfortunately, their efficiency is not always that good. It very much depends on what kind of security problems you're trying to detect. And they can, for example, detect some instances of things like cross-site scripting or SQL injection or things like that. But there will always be limitations. I've seen tools like that being run as an application where you actually know there's a vulnerability because it has been exploited. There is a manual written exploits and the tool still doesn't find any vulnerabilities which is not surprising, because these things are really hard to make an abstraction of to be able to find that in an automated way with a tool. If you would have such a tool that would be, I think, that [inaudible] would be a lot better. I think there's a lot of funders working on that. But at the moment, those tools are not going to be our savior to build more secure applications. CHARLES: Yes. I mean, it's kind of like linting, right? Or you can make tests. We've been through this kind of all the features or the aspects that we want our application to have, whether it be accessibility. There's certainly a very comprehensive suite of lint level checks that you can run to make sure that your application is accessible. You can run a suite of three thousand things and if it triggers any of these things, then yes, your application won't be accessible but it's not a substitute for thinking through the accessibility architecture. The same thing goes with code linting. You're not going to solve bugs with a linter that makes sure that it's formatted and that you're declaring your variables right and that you're not shadowing things. But you can definitely eliminate a whole classes of things that might be put in there just for maybe even you know what you're doing and you're just forgetful. PHILIPPE: Yes, these rules exist, as well. They're not extensive but there are linting rules for Angular used for security, for example. But the problem in linting is that they are very useful to find potential instances of security relevant features or security relevant functionality. But the linting rule alone cannot decide whether something is OK or not. Just to give you a very simple example, if you use the bypassSecurityTrustHtml function, if you give that function a static snippet of HTML, that's going to be fine unless you write your own attack essentially. But if you feed that function user inputs, you're going to be in a lot of trouble. And making that distinction with a linter is going to be difficult unless you have a static string in the arguments. But if once you start having that from variables to dynamically decide to have a different code path, then that's going to be very, very difficult to decide automatically. So, yes, you can use that to find the places in the application where you should be looking for, in this example, a cross-site scripting in Angular but the linting alone is not going to give you an answer whether this is OK or not. That still requires knowledge of how Angular handles this things, what happens, and how you can do things safely. TARAS: Sounds like we keep going back to nothing beats having knowledgeable developers. PHILIPPE: Yes. Unfortunately, that is true. However, with that said, I want to highlight that frameworks like Angular, well mainly Angular, make things a lot better for developers because yes, you still need knowledgeable developers but the ways to introduce a cross-site scripting vulnerability in an Angular application are actually very, very limited. It's not going to be one, but there's going to be maybe three or four things you need to be aware of, and then you should be set. While if you would have done the same for PHP, it's going to be 50,000 things you need to be aware of that are potentially dangerous. So, yes, frameworks and libraries and all of these abstractions make it a lot better and I really like that. That's why I always refer to abstract things away in a library so that you actually have the ability to look for this dangerous code patterns using linting rules in your code base and that you can, at least, inspect the go to see whether it's OK or not, even though you might not be able to make an automatic decision. You, at least, know where to look and what to approve or how to change in the code base. TARAS: I think that's one of the things that oftentimes is not taken into account that the frameworks are different. And I think of big differences in how much -- like right now, the most popular framework, I think, React. But it's such a thin layer, it's such a small part of the framework that you can hardly call it a framework. But it is something that companies rely on. But then when you consider how much of that code that you need to write, to make React into a complete framework for your company, the amount of code that your team has to write versus the amount of code that your team has to write when you use something like Angular or Ember, there's definitely a lot less parts of the framework that you need to write or a lot less parts of the framework you need to choose from what's available in the ecosystem. Like in Angler and Ember, and I'm not sure what the story is with the view, but the pieces, they come from kind of a trusted source and they've been kind of battle tested against a lot of applications. But I don't think that enters into consideration when companies are choosing between Angular or whatever that might be because they're thinking like what is going to be easiest for us. What is going be [inaudible] for developers? They're not thinking about how much of the framework are we going to need to put together to make this work. CHARLES: I can say it sounds, Taras, like almost what you're saying is by using the frameworks that have been battle tested, you actually get to avail yourself of code that actually has security champions kind of baked into it, right? Is that what you were saying? You keep coming back to 'you need developers who are knowledgeable about security', and if you're using kind of a larger framework that covers more use cases, you're going to get that. Do you think that that is generally true, Philippe? PHILIPPE: Yeah. I think it is and that's why I mentioned that I liked Angular before because Angular actually does offer a full framework. And because they do that, they made a lot of choices for developers and some of these choices have a very, very big and positive impact on security. On the other hand, if you make those decisions, you become an opinionated framework and some people don't like that. They actually want the freedom to follow their own paths and then a less full featured framework like React might be an easier way to go. CHARLES: But I think what happens is folks don't enter into that decision with their eyes open to the fact that they then now need to be their own security champion because they just don't even see it. We said the most dangerous thing is the things that you don't know. PHILIPPE: Yeah, absolutely. And I totally agree. That's something that's at least a couple of years and probably still today, many companies moving into this space struggle like, "Which framework do we choose and why do we choose one or the other and which one will still be there in three years because we don't want to switch to another thing in three years," which is risky to our developers. I like that you said that Angular has this security champion knowledge built in because in Angular 2 and every version behind it, but the new version of Angular essentially, they spent a lot of time on security and they learned from their mistakes in the first version because there were some and they took that and they built a more robust framework with security built in by design or by out-of-the-box. Angular offers, for example, very strong protection against cross-site scripting. It's just there, it's always on and unless you actively sidestep it, it's going to protect you. And that's one of the things I really like about Angular and how they did that. CHARLES: Yeah, that's one of the things that I really like too because I remember there was a blog post back, this is probably, I don't know, almost 10 years ago now, maybe seven or eight years, where someone was comparing why they were more interested in using, their servers were implemented in Ruby and why it was better to use Rails than just Sinatra which is just a very, very, very lightweight HTTP framework. And one of the things that he was pointing to was this new vulnerability was discovered and if you were using Rails, the middle way where the middle square stack is managed by the framework, you just upgrade a minor version of Rails. And now, by default, there's this middleware that prevents this entire class of attack. PHILIPPE: Was that a cross-site request forgery? CHARLES: I think it might have been. PHILIPPE: I think Rails was one of the first to offer built in automatically on support for that. So yeah, that was a very good early example of how that can work really well. CHARLES: And the advantage from the developers' standpoint, because the contrast that, if you'd been writing your application in Sinatra which is this is very, very low level based right on top of rack and you're managing the middleware stack yourself and there are no opinions, then not only do you have to like fix this security vulnerability, you have to understand it. You have to get to do a lot of research to really come up with what's going on, how is this going to affect my application and then I can deploy a fix. And that's like a huge amount of time, whereas you have the freedom to not even understand the attack. I mean, it's always better to understand but you can defer that understanding invariably knowing that you're kind of invulnerable to it. And I think for people who enjoy kind of pretending, not pretending, but that the security world doesn't exist and say, "Hey, I want to focus and specialize on these other areas and attain deep knowledge there." It's very reassuring to know that if a defense for a novel attack comes out, I can avail myself of it just by bumping a version number. PHILIPPE: Yeah, absolutely. If you have everything in place to actually upgrade to that version that fixes those, that's a preferable solution. Towards the future, I believe it's going to be crucial to ensure that we can actually keep things up-to-date because everything that's being built today is going to require continuous updates for the lifetime of the application. I definitely hope that the frameworks get better and more secure and start following these patterns of naming the potentially insecure functions with something that indicates that they are insecure. I think that's definitely a good way forward. CHARLES: Yeah. Can I ask one more question? Because this is something that is always something that I wonder about whenever you talk about any aspect of a system. And part of it is folks will not appreciate good architecture until they've experienced some sort of pain associated with not having that architecture in place. Their project fails because they couldn't implement a set of features without it taking months and years and they just ran out of runway, ran out of deadline. Those types of people who've been on those projects appreciate having a nimble system internally, good tooling. Folks who have experienced good tooling understand how much time they could save, and so, have a very low tolerance for bad tooling. A tool takes too long or is misbehaved or is not well put together, they just can't stand because they know how much time they're losing with security. Is there a way to get people to care about it without having some sort of breach, without having gotten smacked in the face? When you do your trainings, is it generally, "Hey, someone has experienced a breach here, and so they want to bring you in." Or is there some way to get people raise awareness of the problems they don't have to experience that pain but can just experience only the benefit? PHILIPPE: That's, again, a very good question and that's also a very good illustration of why security is so hard. Because if you get everything right, nothing happens. [Laughter] PHILIPPE: Or it might be if nothing happens, that nobody cares enough to actually try something against your application. So, there's no positive confirmation if you've done a good job. You can keep putting things off but eventually, there's going to be vulnerability and it's a matter of how you respond to it. We recently had a cross-site scripting in Google's homepage, one of the most visited pages on the web. And somebody figured out that there were some weird browser thing that could be abused and that resulted in a vulnerability on, let's say, such a simple page. So, even there, things can go wrong. So, what would be a good way to draw with some awareness about this is I would recommend following some simple new resources or some Twitter feeds. I have some security relevant articles there but plenty of other people in the industry have as well. And when you read such an article about security incidents, just think about whether this could happen to you or not. And that should probably scare the shit out of you. Simple examples like the Equifax breach, one of the biggest, most impactful breaches of the past few years happened because of an Apache library that was not updated. I think the Apache library, they had a known vulnerability in there. We knew about it. We had a patch, yet it took too long to install that patch and the attackers abused that vulnerability. This is something that probably can happen to each and every one of us because the attacks started, I think, 72 hours after the vulnerability and the patch had been published. So, ask yourself, "Would I have updated my servers in three days after I got that vulnerability report on GitHub?" Yes or no. And if the answer is no, then the same thing can happen to you. Other cases: Magecart is a very big problem, people injecting credit card skimming malware in the JavaScript library. Are you including third party JavaScript libraries? If yes, then chances are that this can happen to you. And there's nothing preventing someone from exploiting that. It's probably just because you got lucky that nobody tried to do that. And the same thing you see now with all these attacks npm packages where people actively try to get you to install a malicious package as one of your dependencies. And again, everybody can fall victim to these things. So, if you read the articles with that mindset, I probably guess that your security awareness will grow rapidly and you will start caring about that very fast. CHARLES: Yeah. TARAS: Lots to think about. CHARLES: Yeah, there's lots to think about because the next thing that occurs to me is how do you even know if you've been targeted. Because a good attacker is not even going to let you know. PHILIPPE: Yeah. CHARLES: It's just better to siphon off your blood, like you said, than to kill the -- you want to be a vampire bat and come to the same cow every night and just take a little bit of blood rather than the lion that kills the cow and then the cow's gone. PHILIPPE: I would say constant monitoring is going to be crucial and you need that data for all kinds of different purposes. You need to monitor everything that happens, first of all, for a post-mortem analysis. If something happens, you want to be able to see how bad it was. This user apparently got a full admin access and if you have decent monitoring, you will be able to retrace his steps to see what they did or did not get. So, that is one very good use case. A second use case is you can use that data to detect attacks. Usually when the attacks are noisy, it's an automated scanning tool but it might be an attacker trying to do things. Again, that may be something very useful for you to act on to see if there is a problem to prevent that user from connecting, or so on. And then, another very good use case of these things is actually inspecting the logs manually as an ops engineer or whatever, who is responsible for doing that, because that might again yield new insights. I've been talking to someone who said that they discovered an abuse of one of their APIs just by looking at the logs manually and detecting a strange pattern and looking and digging deeper into it. And the automated monitoring tools that they had installed that trigger on certain events like a mass amount of requests to the authentication and stuff like that, they did not catch this particular abuse. So, I would say monitoring there is absolutely crucial, for sure. TARAS: So, the takeaway is higher attentive knowledgeable developers who will learn about security. PHILIPPE: I would say the takeaway is security knowledge is essential for every developer. So, I encourage every developer to at least have a little bit of interest in security. I'm not saying that everyone should be a security expert. We should at least know about that the most common vulnerabilities in web applications, what they mean, what they might result in, and what to be on the lookout for. So yes, I think that's one of the crucial things to start with. And then within an organization, you should have someone to fall back on in case that there are security relevant things that you actually can talk to someone who does see a bigger picture or maybe the full security picture to decide whether these things are a problem or not. I think we're closing or nearing the end here, but one of the things we haven't talked about is how to actually get started in security. What if you are interested in security after hearing this podcast and you want to get started? I want to give you just a few pointers so that you actually know where to look. One of the first things to look at is OWASP. And OWASP is the Open Web Application Security Project. It's essentially a nonprofit that has the mission to improve the security posture or knowledge of developers, and they have a lot of resources on various different topics. They have a lot of tools available and things like that. What you want to start with as a developer is the OWASP Top 10, which is a list of the 10 most common vulnerabilities that exist in applications, just to open your eyes like these things exist in applications today and are definitely a problem. And then, there's a complementary Top 10 called the Proactive Controls and that's about how you, as a developer, can actually prevent these things. So, what should we know about implementing security, which guidelines should we follow. And these two documents are a very good place to start. And then there is a huge community that's actually mostly very eager to help people figure out the right way of doing things and solving these problems we have in our ecosystems. TARAS: Awesome. That's great. Thank you very much. CHARLES: Yeah. I'll take that in. That is really, really helpful. Well, thank you very much, Philippe, for coming on and talking about security. I actually feel a lot better rather than usually I'm thinking about securities kind of stresses me out. [Laughs] PHILIPPE: You can bury the problem but it's going to return in the future anyway, so you might as well get onboard and start learning. It's not that scary if you actually -- it's a lot of fun. So, you should learn about security. CHARLES: Well, I am looking forward to diving in. If anyone wants to get in touch with you, how would they do that on Twitter or Email? PHILIPPE: Yeah, sure. I'm on Twitter. I'm always happy to chat about security. You can reach me by Email as well. I'm very easy to reach. And I'll be happy to help out people with questions. Sure. CHARLES: All right. Thank you so much, Philippe. Thank you for listening. If you or someone you know has something to say about building user interfaces that simply must be heard, please get in touch with us. We can be found on Twitter at @TheFrontside or over just plain old Email at contact@frontside.io. Thanks and see you next time.
Luke Melia, Aaron Chambers, and Mattia Gheda john Taras and Charles to discuss all things deployment! Luke Melia: Luke has been working with Ember since it was under early development as Sproutcore 2.0. Ember.js powers a SaaS company he co-founded, Yapp, and they funded their business for a couple of years doing Ember consulting under the Yapp Labs moniker. They're full-time on product now, and his engineering team at Yapp (currently 3 people) maintains around 6 Ember apps. Luke helps to maintain a bunch of popular addons, including ember-cli-deploy, ember-modal-dialog, ember-wormhole, ember-tether, and more. He started the Ember NYC meetup in 2012 and continues to co-organize it today. Aaron Chambers: Aaron Chambers: Aaron is the co-author of EmberCLI Deploy and is currently an Engineer at Phorest Salon Software, helping them move their desktop product to the web platform. He's been using Ember for 5 years and maintains a number of plugins in the EmberCLI Deploy ecosystem. Aaron loves trying to work out how we can ship JS apps faster, more reliably and with more confidence. Mattia Gheda: Mattia is a Software Engineer, Ember hacker, Ruby lover and Elixir aficionado. Currently he works as Director of Development for Precision Nutrition where Ember, Ruby and Elixir power several applications. He loves meetups, organizes Ember.js Toronto and co-organizes Elixir Toronto. Resource: Immutable Web Apps Please join us in these conversations! If you or someone you know would be a perfect guest, please get in touch with us at contact@frontside.io. Our goal is to get people thinking on the platform level which includes tooling, internalization, state management, routing, upgrade, and the data layer. This show was produced by Mandy Moore, aka @therubyrep of DevReps, LLC. Transcript: CHARLES: Hello and welcome to The Frontside Podcast, a place where we talk about user interfaces and everything that you need to know to build them right. My name is Charles Lowell, a developer here at the Frontside. With me also co-hosting today is Taras Mankovsky. Hey, Taras. TARAS: Hello, everyone. CHARLES: Today, we have three special guests that we're going to be talking to. We have Aaron Chambers, Luke Melia, and Mattia Gheda who originally met collaborating on fantastic open source library that we, at the Frontside, have used many, many times that saved us countless hours, saved our clients hundreds of thousands of dollars, if not more. ember-cli-deploy. We're gonna be talking not about that library in particular but around the operations that happen around UI. So, welcome you all. LUKE: Thanks, it's great to be here. CHARLES: Like I said, I actually am really excited to have you all on because when we talk about the platform that you develop your UI on, something that often gets short shrift in communities outside of the Ember community is how do I actually deliver that application into users' hands. Because obviously, we don't want it to be working just on our laptop. We want it to be delivered to our users and there are myriad ways that that can happen and it's only gotten more complex since the last time we talked which must have been like three or four years ago. I kind of just have to ask, I think that what you all were talking about then was cutting edge is still cutting edge now but there must have been some pretty incredible developments like in the last three or four years. What have been kind of the new insights that you all have? LUKE: I think that what we realized as we got started with ember-cli-deploy and a project kind of came together as a combination of a few different open source efforts, something that Aaron was working on, something that our collaborator Mike was working on. We decided to come together under one umbrella, joined forces. And what we realized pretty soon is that deployment needs vary a ton between companies. And so, we are coming from this background in Ember community where we had this attitude where nobody is a special snowflake. We all kind of have the same needs for 90% of what we do. And that's true. I really believe in a lot of that Ember [ethos]. But when it comes to deployment, you know what? A lot of companies are special snowflakes or it's at least is much more fragmented than kind of our needs around on the JavaScript side. And so, what we decided to do was to try to evolve ember-cli-deploy into a platform essentially, an ecosystem that could let people mix and match plug-ins to do in their organization without locking them into an opinion that might simply be a non-starter in their org. CHARLES: It's hard enough to have opinions just around the way that your JavaScript code is structured but when it comes to rolling out your app, it really does encompass the entire scope of your application. So, it has to take account of your server. It has to take account of your user base. It has to take account of all the different processes that might be running all over, distributed around the Internet. Maybe somewhere on AWS, maybe somewhere on Legacy servers but it has to consider that in its entirety. So, it's having opinions that span that scope is particularly difficult. LUKE: Yes. And so, you mentioned a bunch of technical details which are absolutely forcing factors for a lot of words in how they do their deployments but what we found in talking to people that there are also people in political aspects to deployment in many cases. Engineers kind of own the JavaScript code that's running within their app, more or less. But when it comes to pushing the app into the world and a lot of companies, that means they're interacting with sysadmins, ops folks, people who have very strong opinions about what is an allowable and supportable way to get those deployments done and to have that stuff exist in production. And so, we needed to come up with an architecture that was going to support all these kind of varied use cases. And so, we came up with this system of essentially a deployment pipeline and plugins that can work at various stages of that pipeline. And that ecosystem has now grown quite a bit. It's actually, I don't know Aaron if you and Mattia would agree but I think it's probably the best decision that we made in this project because that ecosystem has grown and evolved without us needing to do a ton of work in maintenance. And it's been really great. I think Mattia, you pulled some of the current numbers there. MATTIA: Yeah. I pulled some numbers just yesterday and we have currently 150 different plugins attached to themselves, to different parts of the pipeline. So some of them are about how to build the assets, some of them are about how to compress them, some of them are about shipping. And they allow people to ship it with different ways like we are seeing [inaudible] with just simply Amazon APIs or Azure APIs and some of them even are about just how to visualize data about your deployments or how to give feedback to the user about what was deployed and represent the information. And then this is kind of a bit more detail, probably specific to us but we also added this idea of plugin packs. So, in order to help people define their deployment story, we created this ideal of plugin packs. Plugin packs are simply group of plugins. So, plugins grouped together. As a user, if you want to implement what we call a deploy strategy, you can simply install a plugin pack and that will give you all the plugins that allow you to deploy in a specific way. And that's kind of like an optimization that we added just to make it easier for people to share deployment strategies, share ways of deploying applications. CHARLES: Right. It's almost like an application within the framework. MATTIA: Yeah, exactly. But to stay on the community side, I think that the interesting part about what Luke was saying which was a great success for us is that all we maintain as the core team for this project is the core infrastructure, so the pipeline and a thousand of plugins. Everything else is community-based. And often, even in my day-to-day work, I end up using plugins that I didn't write and that I don't even maintain. But because the underpinning of it were designed especially by Luke and I are flexible enough. It just like has been very, very stable and very, very reliable for many years. So, I will say definitely the idea of like in the spirit of what Ember is, I guess, creating a shared ecosystem where people can add what they want and extend what was provided has been the one single biggest win of this project. CHARLES: One of the things that I'm curious about is we've talked about how you're allowing for and kind of embracing the fragmentation that happens in people's kind of the topology of their infrastructure. What do you see is the common threads that really bind every single good deployment strategy together? MATTIA: My biggest thing here, and we actually have some shared notes about this, but my biggest thing about this is the idea that building and deploying an application for me is divided into three parts. There's building part where you have to decide how to compile your JavaScript application and how to produce some sort of artifact. There is the shipping part where it's about deciding where you're going to put the artifact. And then there is the serving part which is how you show it and deliver it to your users. I think that these three are the underpinnings of any deploy strategy. What we did with this project is just acknowledging that and give each one of these a place. And so, the entirety of what we do in what Luke defined as a pipeline is simply give you a way to customize how you build, customize how you ship, and then customize how you serve. So yeah, I think that that's kind of the root. And the question that everybody that wants to deploy a modern JavaScript application have to ask themselves is how do I want to build it, how do I want to ship it, and how will I serve it to my users. And these things are completely independent one from the order in the sense that you can have something build it, something ship it and something serve it, and that's what we end up doing in most of our deployments, I find. CHARLES: It's good to think about those things as soon as you possibly can and make sure that you have all three of those bases covered really before you start adding a whole ton of features. TARAS: Sprint 0, right? LUKE: In Agile, we call Sprint 0 the phase the thing you do right in the beginning. You've got a skeleton of an app and then you get the deployment infrastructure going, you have the test infrastructure going, so that there is no task within your actual feature development where you have to do those things. And I think that can be a valuable concept to embrace. I would just add to Mattia's three points that for deployments, to me, some very simple qualities of a good deployments are repeatability. You need to be able to reliably and consistently run your deployment process. Sounds simple but there's plenty of operations that have run up way too long on manual deployments. So, we don't want to see those rollback capabilities if you have a deployment that you realize was a mistake right after it gets into production. I'm sure none of us have ever experienced that. CHARLES: That never happens to me. LUKE: You want to have a method to roll that code back. That's something that can be remarkably complex to do. And so, having some guardrails and some support mechanisms to do that like ember-cli-deploy provides can be really useful. But whatever your approach is, I think that's a necessary quality. And then I think we start to step into kind of more advanced capabilities that a good deployment architecture can provide when we start to think about things like personalization, A/B testing, feature flags, these kinds of things. And that requires more sophistication, but you cannot build that on a deployment foundation that's not solid. AARON: I think for me, one of the things I've been really thinking about a lot lately, it's a bit of a mindset shift, I think, to get to where the things Mattia was talking about separating those different parts of deployment. And so, I really start to realize the traditional mindset around deployments like I build some stuff and I ship it to the server and then the users get it. But if we can actually stop and actually split our understanding of deployment into two separate phases. One is the building and the physical shipping of the files; and the other one's actually making them available to people. You open up this whole world of our features that you wouldn't normally have. So to be able to actually physically put stuff in production but not yet have it active, as in users don't see it yet but you can preview those versions in production against production databases. And then at some point after the fact, decide, "Okay, I'm now going to route all my users to this new thing," And to be able to do that really easily is massively, massively powerful. And so, to me, the thing I've been thinking about lately is it is a small mindset shift away from packaging everything up and pushing and overwriting what's currently there to being something, again like Luke said, immutable deployments where everything we build and ship sits next to all the other versions and we just decide which one we want to use to look at it any time which leads into then, I guess, A/B testing, feature flags and things. So I guess deployment really is not so much about the physical shipping, that's one part of it. To me, deployment now is shipping of stuff, as in physical deployment and then the releasing it or enabling it or activating it to users. CHARLES: Or routing it. Sounds like what you're describing is an extraordinarily lightweight process. AARON: It is, yeah. CHARLES: To actually route traffic to those files. AARON: It is. It's incredibly lightweight. That's the amazing thing about it. When you think about it, you're building a few JavaScript files and CSS files and images and putting them on a CDN, and then you just need a tiny web server that basically decides which version of the app you want to serve to people. There's not much to it at all, really. CHARLES: I mean that's absolutely fascinating, though the capability that you have when you have the ability to have these versions, the same versions or different versions of your application sitting along next to each other and being able to route traffic. But it also seems to me like it introduces a little bit of complexity around version matching because only certain versions are going to be compatible with certain versions of your API. You have different versions of your API talking to the -- so the simplicity of having kind of mutable deployments, so to speak, is that everything is in sync and you don't have to worry about those version mismatches. Is that a problem or this could just be me worrying about nothing? But that's kind of the thing that just immediately jumps out to me is like are there any strategies to manage that complexity? LUKE: To me, what you're describing, I kind of think of as a feature not a bug. And what I mean by that is that it is very simple to have a mental model of, "Oh, I have a version of my JavaScript code that works with this version of my API." And as long as I kind of deploy those changes together, I'm good to go. The reality is that that's impossible. The JavaScript apps that we write today, people are using anywhere from two seconds at a time to two days at a time. It's not uncommon these days to have some of these dashboard apps. People literally live-in for their job eight hours a day, nine hours a day, keep the browser tab open and come in the next morning and continue. And so, obviously there are some mechanisms we could use to force them to reload that kind of thing. But at some point in most apps, you're going to have a slightly older version of your JavaScript app talking to a slightly newer version of your API for either the span of a minute or perhaps longer depending on their strategies. So to me, the process of thinking about that and at least being aware of that as an engineer thinking about how your code is going to get from your laptop into the world, I think it's an important step that we not paper over that complexity and that we kind of embrace it and say, "Hey, this is part of life." And so, we need to think about just like we need to think about how your database migrations get into production. That's not something that you can paper over and just have a process that it's going to take care of for you. It requires thought. And I think that this, in the same token, how different versions of your JavaScript app are going to interact with your API requires thought. An exact parallel also had different versions of a native mobile app that go into the app store. How did those interact with different versions of your API? So, I think you're right. There's complexity there. There's ways that we can try to mitigate... CHARLES: Keep repeating ourselves if we think that that [inaudible] actually doesn't exist even in the simple case? MATTIA: Yeah. I think that that's to reiterate what Luke is saying. That's exactly the point. You can pretend it's not there but it is and you have no way to avoid it. Once you ship something to a browser, you have no control over it anymore. And so, you have to assume that somebody is going to be using it. LUKE: Aaron, I think you too, I don't know if you can share it. But you recently told us some stories about kind of what you encountered in your work about this and of how long people were using versions and stuff. AARON: Yeah. Something that we hadn't sort of put a lot of thought into. But the last place I worked at, we had quite a long lived app and we're using feature flags and we're using launch [inaudible] something and it gives you a list of flags and when they were last requested. And there were also flags that we removed from the code and it was just a matter of waiting until all the users had the most current version of the app and weren't requesting the flag anymore. But this one flag just kept getting requested for months and we just could not work out why. It really sort of opened my eyes up to this exact problem that these long lived apps set in the browser and if you have someone that just doesn't reload the browser or restart the machine or anything, your app can live a lot longer than maybe you actually realize it is. So we're shipping bug fixes, we're shipping new features, and we're all patting ourselves in the back. We fixed this bug but have we really? If your users haven't reloaded the app and gotten the latest version, then you haven't actually fixed the bug for some number of people. And it's really hard to tell them as you think about this and put things in place, really hard to tell what versions are out in the world, how many people are using this buggy version still. CHARLES: Yeah, that's an excellent point. I haven't even thought about that. I mean, what is the countermeasure? AARON: We hadn't made it until we came across [crosstalk]. CHARLES: It's nothing quite like getting smacked in the face of the problem to make you aware of it. AARON: That's right. CHARLES: So, what's the strategy to deal with that? AARON: I guess for me, my learnings from that would be from very early on thinking about how we're going to encourage people to reload, to start with, and maybe even have the ability to force a reload and what that means but then that has gotchas as well. You don't want to just reload something when a user is in the middle of writing a big essay or something like that. But definitely thinking about it from the start is one of the things you've got to think about from the start. But I guess something that I'd like to implement and I've kind of thought through but not really explored yet, but the ability to see what versions are out there in the world and there are things I've been thinking about in terms of this little server that serves different versions. Maybe we can start having that kind of tracking what versions are out there and who's using what and being able to see because it would be great to be out to see a live chart or a dashboard or something that sort of shows what versions are out there, which ones we need to be aware of that are still there and even what users are using and what versions they can maybe even move them on, if we need to. But there's definitely a bunch of things that aren't immediately obvious. And I don't know how many people actually think about this early on, but it's critical to actually think about it early on. MATTIA: Yeah. I was going to share what we do which is very similar to what Aaron said just maybe for the listeners to have some context. The first thing you can do is basically what Gmail does, which is every time a web app sends a request, an [inaudible], it will send the version of the app with the request and the backend can check. And the backend can check if you are sending a request from the same version that is the most recent deployed version. And if it's not, this sends back a header and the same way that Gmail does, it will display a pop up that is like, "Hey, we have a new version if you want reload." And on top of that [inaudible] is that we have a dead man's switch. So if we accidentally deploy a broken version, a part of this process, the frontend application tracks in the headers. And if a special header is sent, it force reloads, which is not nice for the user but it's better and sometimes is critical to do so. CHARLES: Right. I remember that was that case where Gmail released something. They were doing something with broken service workers and the app got completely and totally borked. I remember that my Twitter blew up, I don't know, about a year ago I think, and one of the problems I don't think they had was they did not have that capability. MATTIA: I mean, you learned this the hard way sadly. But I think these two things are definitely crucial. And the third one, [inaudible]. I thought, Luke, you had the ability that Aaron was talking about like tracking versions in the world. And I think that's more useful for stats so that you know how often your users update. And then you can make the design decisions based on that and based on how much you want to support in the past. LUKE: Yeah. We haven't implemented that but it reminds me whenever there's a new iOS version, we do a bunch of mobile work in the app and we're always looking at that adoption curve that's published. A few different analytic services publish it and say, "OK. How fast is iOS 12 adoption? How fast are people leaving behind the old versions?" And that helps to inform how much time you're spending doing bug fixes on old version versus just telling people, "Hey, this is fixed in the new OS. Go get it." But if you are able to see that for your own JavaScript apps, I think that would be pretty hot. CHARLES: Yeah. Crazy thought here but it almost makes me wonder if there's something to learn from the Erlang community because this is kind of a similar problem. They solved 20 years ago where you have these very, very long running processes. Some of them there's some telephone servers in Sweden. They've been running for over a decade without the process ever coming down. And yet they're even upgrading the version of Erlang that the VM is running. And they have the capability to even upgrade a function like a recursive function as it's running. And there's just a lot of -- I don't know what the specific lessons are but I wonder if that's an area for study because if there's any community that has locked in on hot upgrade, I feel like it's that one. LUKE: That's a terrific analogy. I bet we could learn a ton. Just hearing that kind of makes me think about how kind of coursed our mental model is about updates to our JavaScript apps. We talked to Aaron, we're talking about kind of this idea of a mutable apps and you have different versions side by side. But the idea of being able to kind of hot upgrade a version with running code in a browser, now that's an ambitious idea. That's something I'd say, "Wow! That kind of thing would be a game changer." CHARLES: Yeah. AARON: Makes me feel like we got a whole bunch of work to do. CHARLES: We welcome them. I'm always happy to give people plenty more work to do. No, but how they manage even being able to do migrations on the memory that's running. I don't know if it's something that's going to be achievable but it sounds kind of like that's the direction that we're heading. LUKE: It does, as these apps get more complex and they continue to live longer. The idea, the work arounds that Mattia mentioned about kind of showing a message and having a dead man's switch, these are all certainly useful. And today, I would say even like the best practice but they were not what you would want to do if you could magically design any system. If you're taking a magical approach, the app will just be upgraded seamlessly as a user was using it. And they would be none the wiser, the bugs would be fixed. End of story. There's no interruption to their workflow. At least for me personally, I don't really think about that as a possibility but I love the Erlang story and analogy and to say maybe that is a possibility, what would it take? I would obviously take a collaboration across your JavaScript framework, perhaps even JavaScript language features and browser runtime features as well as your backend and deployment mechanism. But I think it's a great avenue for some creative thinking. CHARLES: I'm curious because when we're talking about this, I'm imagining the perfect evergreen app but there's also feels like there's maybe even a tension that arises because one of the core principles of good UI is you don't yank the rug out from underneath the user. They need to, at some point and we've all been there when the application does something of its own agency, that feels bad. It feels like, "Nope, this is my workspace. I need to be in control of it." The only way that something should move from one place to the other without me being involved is if it's part of some repeatable process that I kicked off. But obviously, things like upgrading the color of a button or fixing a layout bug, those are things that I'm just going to want to have happen automatically. I'm not going to worry about it. But there is this kind of a gradation of features and at what point do you say, "You know what? Upgrading needs to be something that the user explicitly requires," versus, "This is something that we're just going to push. We're going to make that decision for them." LUKE: Yeah, it's a great question. One of the things that I'm curious what you all think is when you think about the mental model that our users have of working in a browser app, do you think that there is a mental model of, "Oh, when I refresh, I might get a new version." Do people even think about that? Or are they just like your example about a button color changing as kind of a minor thing. I don't even know if I could endure stack. We've all been I think in situations where you do a minor redesign and all of a sudden, all hell breaks loose and users are in revolt. Take the slack icon. So, I think it's a fascinating question. CHARLES: I don't know. What's the answer? Do you always ask for an upgrade just observing? I don't have any data other than observing people around me who use web applications who don't understand how they actually work under the covers. I don't think it's the expectation that this code, this application is living and changing underneath their feet. I think the general perception is that the analogy to the desktop application where you've got the bundled binary and that's the one you're running is that's the perception. AARON: I'd say the difference there is that, and with all these new ways of deploying, we're shipping small things faster in multiple, multiple times a day or even an hour. So it's not the sort of thing you really want time to use. There has been an update, need to upgrade as well. And that's the difference between the desktop mentality. And if that's the mentality they have, it sits quite a bit of a shift, I guess. MATTIA: It makes me think that one of the tools that the users -- if you take a look at the general public, there's probably one tool that everybody can relate to which is Facebook. So, I think if there is a way to say what do people generally expect. There is a business user which I think we are often most familiar with but the general public, probably what they're most familiar with is what happens in Facebook. And I don't use Facebook almost. I haven't used it for a couple of years but I wonder how much of what people experience in Facebook actually impacts the expectations around how applications should behave. LUKE: I think that's a really good question. I do think your underlying point of you have to know your own users I think is an important one also. Obviously, some folks are going to be more technical than others or some audiences will be more technical than others. But I would even question, Charles, your suggestion that people think of it kind of as like a binary that it stays the same until you refresh. I think people have an idea that web apps improve over time or sometimes they get bugs but hopefully that they improve and change over time, and that there is a tradeoff there that means sometimes there's something new to learn but at the same time, you get new features. But I don't know that people necessarily associate that with and it happens when I hit reload or it only happens when I open a new browser, like I don't know that it's that clear for people in their head. CHARLES: Right. I can see that. But the question is if the evolution is too stark, I think people tend to get annoyed. If they're in the middle of a workflow or in the middle of a use case and something changes, then it gives it a feeling of instability and non-determinism which I think can be unsettling. LUKE: Definitely. We all value, as engineers, we value getting into that flow state so much of like, "Oh man, I'm being productive. I don't have any distractions." And you kind of owe that to your users also to be able to let them get into that state with your app and not be throwing up, "Hey, there's new stuff. Reload." "I'm in the middle of something. Sorry." CHARLES: Yeah. I definitely do the same thing. Sometimes, I let iOS be bugging me to upgrade for a month until I finally start to feel guilty about security and actually do the upgrade. LUKE: Right. CHARLES: Although once they started doing it at night, it actually made it a lot better. LUKE: That's an interesting idea, too. I think there's a natural tension between the lower integration risk that we have as the engineering teams of shipping very frequently. Aaron mentioned shipping a dozen times a day. We certainly have been there and done that as well. I would say on average, we ship a few times a day. But the reason that we do that is because we know the faster we get code into production, the faster we can trust it. So it passes all your tests, it passes your [inaudible], but you don't really know if you're being honest. You don't really know until there's thousands of people using it in production. And yet, this conversation makes me think about there is a tension between how frequently you do that versus your users' kind of comfort level and expectations. CHARLES: And maybe there is a thing where you can kind of analyze on a per user basis how often they're active in the application and try to push updates on times that are customized to them. LUKE: Like when a user has been idle for 30 minutes or something like that. CHARLES: Yep. Or even like track trends over months and see when they're most likely to be idle and schedule it for them. LUKE: Good point. CHARLES: Something like that. TARAS: I have an idea. We should introduce screen savers into web apps. And so when the user stops using the app, just turn on screen saver and do the upgrade. LUKE: I can see the VC patch. It's after dark but for the web. CHARLES: Enter install flying toaster. AARON: It does that to open up the idea as well of that automated checking that things are okay well after the fact because it's all right to sit there maybe activate something and sit there even for an hour and make sure there's no bug request coming in. But if no one's actually received your app, then of course it's not going to come in. And it's very easy to kind of move onto the next thing and forget about that. I guess it's not something that I've ever put a lot of time in and I haven't worked anywhere that's had really great automated checking to see is everything still okay. And I guess it's an interesting thing to start thinking about as well an important thing. CHARLES: Like actually bundling in diagnostics in with your application to get really fine grained information about kind of status and availability inside the actual app. AARON: I'm not exactly sure, really. I guess I maybe wasn't thinking about inside the app but I don't know what it looks like exactly. But there is that element of shipping fast and getting stuff out there. But are we really making sure it all works later on when everyone's actually using it. LUKE: Yeah. This by the way, I just wanted to say when we were talking earlier what are the essential qualities around deploying an app. And this reminds me that one thing that we didn't mention but is very simple is your app should have a version. And it should be unique and traceable back to what was the GitHub, git commit that was the origin of that code. It's just a very simple idea but if you're going to be analyzing errors in production when you have multiple different versions of your JavaScript app running, you're going to need to know what version caused this error and then how do I trace it back and make sure that the code that I'm trying to debug is actually the code that was running when this error happened. CHARLES: Do you only use just [inaudible] or do you assign like a build number or using SemVer? What's a good strategy? LUKE: In our case, we use git tags. And so, our CI deployment process for our Ember apps basically looks like this is we work on a PR, we'll merge it to master. If it builds, the master gets deployed into our QA environment automatically using ember-cli-deploy from our CI server. And then once we're happy with how things are on QA, we do git tag or actually I use an ember addon called ember release that does that tagging for me and I'll tag it either in patch minor or major, roughly [inaudible] although it doesn't matter that much in the case of apps. When there is a new tag that builds green on CI, that gets deployed automatically into production by ember-cli-deploy. And so, that's kind of a basic flow. That tagging, just to be clear, the SemVer tag is just going to be number.number.number. You can get more sophisticated than that and I think both Aaron and Mattia have a system where even in the PR stage, there's automatic deployments happening. So maybe one of you want to mention that. AARON: We're slightly different. Every time we push the pull request, that gets deployed to production. We're able to preview up pull requests in production before we even merge into master which we find super useful to send out links to stakeholders and maybe people that have raised bugs just to get them to verify things are fixed. And then at the point that it's all Google merged, the pull request to master which will automatically do another deploy which is the thing we'll ultimately activate, we activate it manually after the fact. We just do a little bit of sanity checking but we could automatically activate that on merge to master as well. But yeah, the being able to preview a pull request in production is super powerful for us. CHARLES: That is definitely a nice capability. It's hard. It's one of those certain workflows or patterns or tools that you remember life before them and then after them and it's very hard to go back to life before. I would definitely say kind of the whole concept of preview applications is one of those. AARON: Absolutely. It's a daunting concept if you're not there, previewing something that's essentially a work in progress and production. And there are some things you want to be careful with, obviously. But for the most part, it's a super valuable thing. As you say, it's a world where once you're there, it's very hard to step back and not be there again. CHARLES: So I had a question about, we talked about I think it was 162 plugins around the ember-cli-deploy community. What for you all has been the most surprising and delightful plugin to arrive that you never imagined? MATTIA: That is a good one. I'm pulling up the list. What I can tell you, for me, it's not about a specific plugin. The surprising part was the sheer amount of different strategies that people use for the shipping part. At least, I found that the build part is similar for most people, like most people want to do the things that you're supposed to do. So, you want to build your application and then you want a minify it in some way. And there's a bunch of options there from gzip to more recent technologies. But the way people deliver it to servers and the difference in the solutions, that I think for me has been the biggest thing, where people that ship [inaudible] are people that ship directly to Fastly, people that use FTP files, people that use old FTP, people that use our sync, people that do it over SSH. We have people that ship stuff directly to a database because some databases actually have great support for large files. So we even use it as a storage. We have people that do it in MySequel, people that do it in [inaudible]. CHARLES: It's actually storing the build artifacts inside of a database? MATTIA: Yes, I've seen them in that. It's kind of interesting like the solutions that people ended up using. And so for me, I think that that's been the most fascinating part. Because as we were saying at the beginning, I'm just seeing now, we even have one for ZooKeeper. I don't have an idea what this does but it's probably related to some sort of registration around the seven-day index. That, to me, I think has been the biggest surprise. Everybody ends up working in a different environment. And so, that flexibility that users need has been by far the most surprising one. AARON: I think that's also been one of the challenging things, one of the enlightening things for me. I think in the Ember ecosystem, addons and even just Ember itself, it's all about convention of configuration and doing a lot of the stuff that you do for you. I think people expect them to see a lot of point to do the same thing. But the key thing here is it just really automates all the things you would do manually and you need to understand exactly what you want your deployment strategy to be before use them to see a lot of [inaudible] could do for you. You need to decide do I want to install my assets on a CDN and do I want to install my index in Redis or in console or in S3 bucket. You need to know all these things and have decided on all these things and then ember-cli-deploy will make that really easy for you. And this is one of the educational things, I think we still haven't even nailed because there are always people that want to know why this doesn't work. But deployments are complex thing and as what you were saying Mattia, there are so many different variables and variations on doing this that there's no sensible configuration ember-cli-deploy could really provide out of the box, I guess. And so, that's why we ended up with a pipeline that gives you the tools to be flexible enough to support your strategy. LUKE: I think the closest that we come to the convention is that if any app is using ember-cli-deploy, you can run ember deployment targets or ember deploy prod and ember deploy QA and you can expect that that's going to work. What you don't know is how has it happened to be configured in this project. Charles, your question about kind of the most surprising thing that's come out of the ecosystem. For me, I would say -- Mattia mentioned plugin packs earlier which are groups of plugins that kind work together well. And so, we've seen some plugin packs like you might expect, like an AWS pack for deploying to AWS. But the more interesting ones to me that we've seen a lot of, companies open source their plugin packs. So what you naturally fall into as a company that's adopting ember-cli-deploy that has multiple ember apps is that you are going to develop your own plugin pack for internal use because generally speaking, companies follow the same deployment pattern for each of their apps. There's usually not any reason to vary that. So then the new thing that happen on top of that is people said, "Why don't I make this open source so other people can kind of see how we do it?" And that's been a really delightful part of the process to kind of get a peek into how other organizations are orchestrating their deployments. And if people are curious about kind of looking at that themselves, you can go on NPM and look for keyword ember-cli-deploy-plugin-pack and pull up all of those. And you can kind of poke around and see what different companies have open source there. CHARLES: I actually love all three of those answers because it really is for me when you have a constellation of people around a particular problem, it's the surprising solutions that emerge that are some of the most exciting that would have lain hidden otherwise. It would have been kind of buried beneath the source of company A or company B or company C but actually having it all out in the open so that you can inspect it and say, "Wow, where has this solution been all my life?" Something that you have never imagined yourself. LUKE: It's so funny that you mentioned that because that actually is the origin story way back, I'm talking like 2013 probably when we were very early Ember adopters and we were trying to figure out how do we deploy this thing. We're deploying it with our Rails app, like literally deploying the Ruby code and the JavaScript code together which took forever which is a disaster. And I heard through the grapevine, just exactly what you're saying Charles, where the good ideas are kind of hidden inside of a company. I heard through the grapevine that Square had this approach that they were using where they would deploy their JavaScript assets and then deploy their index HTML file, the contents of that file, into a database, it's Redis in their case, and serve then it out of there. And it empowered all of these interesting situations of like having multiple versions, being able to preview the release, et cetera. And so we then set out to copy that idea because there was nothing open source. So we had to create it ourselves which we did in Ruby which we made it inaccessible to many JavaScript shops in the first place. Then the evolution of that kind of over time and of Mattia and Aaron and Mike and the rest of the community kind of talking together has now moved this into the open source sphere where these ideas are more accessible and we've created an ecosystem encouraging these ideas to stay out in the open. It's so true that there's just gems of ideas that have been created by really brilliant engineers inside of companies that could be benefiting so many people, they just haven't seen the light of day yet. CHARLES: Yeah, that leads me to my next question. I would say most of the ideas that we've been talking about today really, except for the build part, how Ember specific are they? Obviously, the ember-cli is a great resource and has a lot of great opinions for actually building the JavaScript assets themselves. But the second two phases of the pipeline really can vary freely, if I understand correctly. And so, have you all ever thought about trying to maybe kind of abstract these processes and these plugins so that these same ideas don't remain not just inside of a company but inside a community that spans a set of companies so that it's available for a wider audience? How integrated is it with Ember and what kind of effort would that be? LUKE: That's a great question. Aaron or Mattia, one of you guys wanna talk a little bit about the history here? MATTIA: Yeah, sure. We've been thinking about this as well. In the past, a guy on the ember-cli-deploy team, Pepin, has actually started this effort and it kind of prototyped this very idea of separating the ember-cli-deploy part from Ember CLI itself and make it a bit more generic. And he started a project called Deploy JS which I can give you a link for the show notes later. I don't think that the project is currently maintained but definitely the start of the effort is there. And the funny thing is that it was surprisingly easy. I think that we didn't get there mostly because we just all use Ember at work. As you know, open source is mostly motivated by the needs that an individual or a group of people have. But if any of the listeners were very interested in this, I think they should definitely get in touch and we will be happy to talk to them and see what can be done here. AARON: And also if you look, there's actually a plugin called ember-cli-deploy-create-react-app and there's also ember-cli-build-view. So it can and is being used to deploy non-Ember apps which I think is super interesting because the only real Ember part of it is just using the CLI to discover the addons and plugins. And from then on, it's really out of the hands of Ember. But it sort of leads into a little bit, Luke mentioned this concept of immutable web apps. And I've been thinking a lot about this lately because a deployment strategy that ember-cli-deploy use as an example a lot and it's kind of become [inaudible] ember-cli-deploy in ways, the lightning approach which is this whole idea of splitting or putting your assets in CDN and your index HTML separately maybe in Redis and serving that. I've been trying to work out how I can talk about this to the wider JavaScript community in a non-Ember way and knowing full well that the concept of lightning deployment means nothing to anyone outside of Ember. Just by chance, I was just talking to some people and this terminology of immutable deployments kind of rose. I started searching around and I come across a website called ImmutableWebApps.org which was just scarily the same as what we've been talking about for the last three or four years with ember-cli-deploy. And a way to boil down at a framework-agnostic level, what are the key points that you need to consider when building a JavaScript web app to make it immutable. And it was just really amazing seeing it. This website was put up 3 weeks before I did my Google search coincidentally. And it's basically word for word what we have been talking about the last three or four years, So, someone else in the other side of the world's been coming up with the same ideas in their company like you were talking about earlier and we've reached out to him. I guess that, to me, is sort of the way forward that I want to sort of pursue to try and get these ideas out in a framework-agnostic way to the rest of community and say, "Hey, we thought about deployment in this way. Have you thought about building your app in this way to give you these sorts of capabilities?" CHARLES: I think the wider community could definitely benefit from that because most of the blog posts and talks I've seen that concern themselves with deployment of single page applications, it's still much more of the tutorial phase. Like, "This is how I achieve getting this deployment strategy." Not, "This is how I repeatedly encode it as a program," and leverage it that way. And so, definitely getting that message out to a wider audience, I think it's a -- what's the word? It's an underserved market. LUKE: Yeah. I really like this idea also. I think about this ImmutableWebApps.org, if you look at it. It's sort of a manifesto with conceptual description of what are the qualities that your app has to have to qualify as an immutable web app which I think is kind of a funny idea but one that people can start to get their heads around and compare that description to their own apps and say how do I hit this or fall short. And it reminds me a lot of kind of the idea of Twelve-Factor App which is an idea that I think came out of Heroku originally. And it's an idea of a backend app that is portable to be able to easily move across different hosts and easily be scalable to different instances of [inaudible] if your app obeys all of these things then it's going to do well under those circumstances, that will satisfy those needs. So, I think it's a great way of thinking and probably maybe even a better entree into the conversation with the wider community than a library, this is certainly a library called ember-cli-deploy. CHARLES: This is a fantastic discussion. It's definitely reminded me of some of the best practices that I haven't thought about in a long time and definitely opened my eyes to some new ones and some new developments. So often, we can be focused on how our apps work internally, like how the JavaScript code works that we can just kind of -- what's the saying -- lose sight of the forest through the trees or I can't remember. It's like too busy looking down the end of your nose to see past your face. I've probably mangled both of those adages, but maybe 60% of two mangled adages is at least equal to one. This is something that we need to be thinking about more, that everybody needs to be thinking about more. This is actually a very exciting, very useful problem space and I'm just really grateful that you guys came on to talk to us about it. So, thank you, Mattia. Thank you, Luke. Thank you, Aaron. MATTIA: Thank you so much. It was a great time. AARON: It was a pleasure. Thanks for having us. LUKE: Thanks so much. CHARLES: Thank you for listening. If you or someone you know has something to say about building user interfaces that simply must be heard, please get in touch with us. We can be found on Twitter at @TheFrontside or over just plain old email at contact@frontside.io. Thanks and see you next time.
Joe joins the panelists to talk about pull request etiquette. Joe Leblanc: Joe first learned to code on a Zenith computer his dad brought home from work. It had this built in blue LCD monitor and ran on 5 1/4" floppy disks. He used spreadsheets for work and Joe was interested. They spent about an hour going over macros together and he took off from there. Long after the Zenith died, the open-source content management system Joomla! landed in the center of his attention. Joe found himself writing a book about Joomla programming, authoring video tutorials about Joomla for lynda.com, giving Joomla talks, and helping organize Joomla conferences. Since his time in the Joomla community, he's picked up Node, Rails, React, and other frameworks. He's currently coding at True Link Financial and working on a few hobby-projects as well. Please join us in these conversations! If you or someone you know would be a perfect guest, please get in touch with us at contact@frontside.io. Our goal is to get people thinking on the platform level which includes tooling, internalization, state management, routing, upgrade, and the data layer. This show was produced by Mandy Moore, aka @therubyrep of DevReps, LLC. Transcript: CHARLES: Hello and welcome to The Frontside Podcast, the place where we talk about user interfaces and everything that you need to know to build them right. My name is Charles Lowell, a developer here at The Frontside. And with me also is Taras. Hello, Taras. TARAS: Hello, hello. CHARLES: Today, we're going to be continuing our conversation about platforms, as always, but in particular the pillar of your platform that has to do with how you collaborate on code. It's an important one. And so, we're spending some time on it. And with us to talk about this today is Joe LeBlanc who is a senior software engineer at True Link Financial in our very own beloved Austin, Texas. Hey, Joe. JOE: Hi. CHARLES: Thanks for coming on the show. We're going to be talking about collaboration and I was thinking we could kick off the discussion talking about pull requests because that's typically one of the ways that we collaborate on code. That's really where the rubber tends to hit the road. You have particular interest in the dynamics of a pull request. What experience kind of led you to that interest? JOE: My background has been doing a lot of work as a freelancer. And when you work as a freelancer or do agency work, a lot of times you are really the only software developer that is around or you're working with maybe one or two other people. And I didn't get a lot of opportunity for pull request reviews. Mine was in that space. And then when I moved to a full time job at a place where I was one among maybe half a dozen or a dozen engineers, that's when I really began to get interested in how I could be better at giving pull request reviews and also submitting pull requests that people want to review. CHARLES: What made you notice the need for this? JOE: What would happen is someone would submit a pull request and there would suddenly be just dozens and dozens of comments coming in that were just kind of difficult to keep track of and often were maybe talking about things that didn't necessarily need to be reviewed as a part of that pull request or addressed, maybe things that could be caught by a linter. CHARLES: Right. JOE: And other tools that are a little bit easier to receive feedback from. Like it's easier to have a tool tell you your spacing is off than have me tell that to you. CHARLES: Right. And really that seems like the spacing is off, that's kind of like if you need to deliver that feedback, that's kind of like not what you want to lead with. If you don't have linting in place like after all other issues are sorted out. JOE: Yeah. CHARLES: But it sounds like what you're describing is people who have kind of swarmed over a pull request each with their own kind of pet peeve issue. JOE: Yeah. And then you're just left with this long list of comments to go back and address and you're pushing up more and more commits. And by the end of it, you could have more than 100 comments on this PR. And you thought that you were going to get this done in a day or two, and then suddenly it's the end of the sprint and it's like, "Oh fine, then I get to merge this." CHARLES: And typically, it's actually, in my opinion, an anti-pattern when you have like 10 mergers that happen on the second last day of the sprint or the last day of the sprint. There's always issues with that, right? Ideally, you are merging code throughout the course of the sprint. It kind of like defeats the purpose almost. I guess you're doing your integration in shorter periods but even so, tons of stuff is bound to break when everybody's pushing and everyone's rushing. JOE: Yup. TARAS: So, what would happen in that situation, in your experience, when you have this pull request that a lot of people are commenting on and some of the comments could be addressed by linters. Would you guys go back and actually implement linting? Did that happen? Would you put linting in place so that you'd have consistent formatting on the projects to reduce that kind of feedback? What would be the resolution or a way to improve on a process so you could actually get better feedback? JOE: Yes, we did install a linter and that's something that you can either run locally or you can also run in your continuous integration environment. And it's really good to run it locally first because then you can just catch it before you even submit the PR. And that was something that definitely helped cut down on these sorts of comments and even debates over style. You just have this one thing that is maintaining that style or rather telling you when you're wrong for you, and then you don't have to have all these comments coming in as you submit your PR. And another thing that we would do is we would use, even beyond just things like whitespace and trailing newlines and those sorts of issues, sometimes you would have some of these issues that would come in that wouldn't necessarily be a trailing newline or whitespace issue or something like that but it would be purely someone's sense of style. And in those situations, we came up with a bit of a standard where if you wanted to make a comment that was strictly style that wasn't something that you necessarily wanted to block the pull request over, we would use emojis that had specific [inaudible] thing. So in our case, we would use either a beer emoji or the cocktail emoji to kind of say, "Here, take this with a drink." CHARLES: [Laughs] JOE: This is not something that's super serious. It's a suggestion, "Here, let's discuss this over beers." [Laughter] TARAS: It's a really nice approach to friendlify the actual pull request because people can get really -- I think another way people would say this is like when you say that it's not 'I believe it whole strongly'. How does that go, Charles? CHARLES: It's strong opinions, weakly held. JOE: Yes. CHARLES: Right. You're kind of making an assertion about the way things ought to be or often missing from that is how important it is to you. You've made this thing like it should be like this. That's a very black and white statement, very firmly cut between right and wrong. But how important it is to you if it goes the other way is often missing from the conversation. TARAS: So Joe, what would happen once you made this change? Did you see a shift in the kind of feedback you were getting? What did it look like after this? JOE: After we started implementing the ideas of putting on emojis to signify things that were not super serious, it became a lot easier to just picture all those comments and define the things that really needed to address. And it was a lot faster. I was just able to address these very specific problems and do that first. And then if I had time later, come back and address those style issues. CHARLES: Right. Was the kind of the good feedback there the whole time but it was just obscured by the noise of these kind of exterior issues or side issues? Or did you find that because people's thoughts were more given over to the heart of the matter, that you actually got more comments directly relating to kind of the meat of the implementation? JOE: I think it was really more that the good comments really were buried in there. A lot of times when you see critical comments that seem to be petty, that can really trigger your emotions. And so, I feel like a lot of the times the emotional state that we would get into would prevent us from seeing the good comments that people were leaving, and we would get a little too angry about things that really weren't that big of a deal. CHARLES: Would the tension kind of noticeably rise? JOE: Yeah. I actually remember several instances at my job where -- this is when I was working in office and in the same room as all my co-workers. And one of my co-workers would know that I was reading his review by the number of sighs that were coming out of me. [Laughter] JOE: Yes, I definitely had this reaction that people could tell. Implementing things like these emojis definitely helped. They didn't always fix everything, though. One thing that would happen too is even after filtering out kind of the noise comments and getting into the meat of the matter, we would still wind up in situations where I would want to use one design pattern and another engineer would come along and say, "No, you should do something different," and we would have these arguments. CHARLES: Right. And those are harder to resolve too because those are strong opinions strongly held. JOE: Yes. CHARLES: But it's still a step forward, right? You've eliminated the passionate arguments about the strong opinions weakly held. And so you're able to now grapple with 'how do you resolve these arguments' of these stronger opinions that are more strongly held. JOE: Yeah. Part of the way that we handled this was we came up with a way of being able to have a debate about something on a pull request, yet also keep in mind that we're working under deadlines and that ultimately, it might not be that big of a deal. TARAS: I don't know if you experienced this but there's this kind of a feeling where you know you have a deadline that you want to reach and this feedback is kind of not really helpful to that end goal. But at the same time, there feels like a trade off for you, like trading off quality by accepting something that people don't agree being fully ready. JOE: Yeah. TARAS: But then there is, I guess the tests seem to be like you have some tests and the tests are passing. So, thumbs up. [Laughs] CHARLES: Yeah. Well, here's the thing. This thought just occurred to me when I heard that is when we are reviewing code, probably the most important code to review is actually not the implementation but it's the tests. Because why do people get so care mad about the code being right? It's because they know how expensive it is to get it wrong. And we all have our different opinions of what's right and wrong and what's going to cost us more. But at the end of the day, we're all trying to save us, save ourselves and our team, pain in the future. The tool that we have to do that is the code review. We're trying to make sure the code is "good". But usually what ends up happening, and I know I'm certainly guilty of this, is I'm focusing on the implementation and I look in and say, "Oh yes, there are tests." I don't really look at the test and say, "Is this a good test?" Because a good test is going to lower that cost that we're so afraid of in the future, and that drives us to want to make sure the code is as close to what we conceive of as perfect as it's likely to get and still hit our deadline. It sounds to me like maybe what we really ought to be doing is reviewing the quality of the test because if you have an excellent test, then the shape of the code almost is unimportant or changing the shape of the code, more importantly, is very cheap. So if you do find that the implementation is suboptimal, the cost of rejiggering it into a better configuration is going to be very low. JOE: I've discovered so many times where you are running a test that was written previously either by somebody else or by yourself, and you realize that the test is testing the wrong thing entirely. CHARLES: Yeah, coverage is spotty. It covers 10% of your cases. JOE: It can also might have a stub or a mock in there somewhere that is hiding something that you weren't counting on. Or it could just be you pull up this screen and test then it saves but you're not really testing what it's saving. CHARLES: Right. Can I actually proffer a suggestion? At least I'm going to try and hold myself to going forward, is that when I review a pull request or some proposed change to a code base, I'm going to review the test first. JOE: Ooh, that's a great idea. CHARLES: I'm going to try and start with the test and avert my eyes from the implementation, no matter how curious I am how this person accomplished the solution. I'm going to hide from the solution and focus on the verification. And if I have faith, first and foremost, in the verification, then my faith in the implementation is secondary. TARAS: You know, Charles, I really like this. I think it'd be really interesting to try this out and see what kind of impact it creates. It makes me think that there is actually something really useful in here in terms of providing feedback because the challenge is that when you're giving feedback to someone on a pull request, ideally that feedback is going to be useful and it's going to give the person something to think about in regards to the work that they did. And I think a good way of pushing to work back onto the person in a way that allows them space to internalize what they need to do is identifying cases that might be missing because a lot of times, we test happy paths. We don't test the things that we didn't think about. And that's where the devil is in details. It's in the things that we don't test. Those are the kind of problems that we're trying to prevent with having good code. But really the best way to kind of flush those things out is to make sure we have the right test coverage for it. So yeah, I think if you add to that just when you're looking at a pull request is to go, is a test coverage missing the following cases and actually surfacing not. I'm actually thinking like a danger task. A danger is a tool for adding information to pull requests so that it's kind of like an automated mechanism to comment on pull requests. And so I'm thinking like having something that does what [JAS] does. [JAS] has this thing where they will check based on the last git commit. It will figure out what are the tests that have been added since the last commit and then it will actually show you. So when you run tests, it only runs the tests that you kind of impacted which is something that could be used to surface tests that were written in this commit. I mean, we also have that information in the actual commit itself but being able to see 'these are the tests that were written or added' and then you could use that and figure out what else is missing from this. This could be like a way to kind of add onto this process, something a little bit more automated so you could actually highlight this information very easily in the commit itself. CHARLES: It's funny, my mind was wandering off towards GitHub. When do you use danger and when do you use GitHub actions? GitHub actions have been kind of like brewing and fermenting in my mind. TARAS: One nice thing is that I mean, danger is a little awkward in that it creates one single commit at a topic. It creates a comment first time you commit and then it keeps pushing information into a comment which shows up at the top of your comment thread. CHARLES: Right. TARAS: But that's not how you read comments. You read like top to bottom. You don't read top to bottom to top. And so, I think the nice thing about the actions is that because it's using checks like GitHub checks that actually is part of a different area, it's part of the validation area as opposed to part of the actual comment conversation. This seems like a more appropriate place to put their information rather than the first comment. CHARLES: Yeah. So, do we decide on kind of what the ultimate resolution is for when you have these high order conflicts? JOE: Yeah. So that's one thing that we came up with that I'm really proud of. And so what we do and at least at that job that I was at, what we did in those situations was we said, "OK, if you've gone back and forth on this two or three times and there's still not an agreement as to what should be done or there is no compromise, what you should do is let the author of the pull request go ahead and merge that code," because you want to merge that code, get it out to production, make sure it's serving our users and our clients. And then what you want to do is take that conflict and record it somewhere to specify this interpersonal conflict not a code conflict or a merge conflict or anything like that. So you take this interpersonal conflict, the topic that was being discussed and you put that -- we were just putting it in a Google doc and leaving it there. And then every couple of weeks or maybe every couple of sprints, we would go through that Google doc and just talk about topics that had come up. And then really more often than not, what we would find is that we didn't care about the topic anymore. CHARLES: [Laughs] It's amazing how time has a way of cooling passion over things that really don't matter. JOE: Yeah. And for the things that did matter, we usually have like a really good discussion. Sometimes, we even came up with something different entirely based on just having more people in the conversation and thinking about this problem. TARAS: I like this because there's a bit of process around it. It's kind of like a retrospective on pull request passions. JOE: Yes. TARAS: It sounds like a healthy thing to do for a team, especially. And I think just allocating some breathing room to go through these kind of things could be really important cumulatively over time, especially. CHARLES: Yeah, definitely. It sounds almost like the pull request is then ongoing. You have this collaboration that happens kind of at the front of the change but then is rippling outwards and onwards and hopefully then has an impact on future changes, like if you're disagreeing. So, would you qualify that the types of issues that end up living in this Google document as architectural issues or just, 'hey this is the way we need to talk to each other and this was off and we need to fix this' or is it both? JOE: It was mainly more architectural decisions that was coming up in this, sometimes like code style issues as well. Not so much the interpersonal issues, like the interpersonal issues would come during retros. CHARLES: OK, because it sounds to me like it's not. Then you have the added benefit that your architecture does get to improve because clearly if there's some disagreement about this, then there's some sort of tension there. There's someone perceiving that some problem is not being resolved by a particular implementation. And so it's good to just at least surface this issue again and again. What was your experience with kind of revisiting the architectural issues? Was it mostly 'well, this wasn't really an issue' or is it 'wow, I'm really glad we took note of this because now we have these three ways of thinking about things' and it turns out that given three or four months more experience, this is something that we should be doing. JOE: Yeah, it is definitely a mix of both but I'm kind of leaning more towards the latter. Like we're just glad that we bring things up. Occasionally, there will be one or two topics that will come up that we still would resolve and we would have to agree to disagree on something. Or in some cases, I think we would allow one of the lead engineers to just make the final decision on something. But that was pretty rare. CHARLES: So, apart from appealing to either time, the passage of time or just kind of taking a brain dump of all the different architectural options or deferring to a more senior engineer and just kind of asking them to step in and make the call, are there any processes that you can put in place say to kind of mechanically come up with a decision? Like any set of values that you can use as a ruler to kind of line up a set of things to kind of attach weight to one particular solution? For example, one of the things that I always think of is when I have internal conflict about a decision, there's kind of a part of me that feels like this might be the right way and maybe another way is the right way. And so, a tool that I will use is what's internally consistent with the rest of the system. So, even if I've had an insight that something really ought to be framed,one solution should be framed in a certain way. If there's another solution that's more internally consistent with the way things are existing, then I'll use that as a discriminator to say, "OK, that's one thing that I can use to measure a solution against, that I'm not emotionally tied to." It feels more objective. And if you have those tools, you can kind of pile them up and see which pile ends up being higher for each solution. Do you see what I mean? JOE: I definitely lean more towards keeping things consistent with what's already there. Personally, I probably do that to a fault. Sometimes I need to branch out a little more and get comfortable with possibly breaking something. But you always need to weigh whether it's something that is acceptable to break. Like a lot of the systems I've worked with involve money and it's really important that people either have money when they need it or are charged the correct amount when you charge them with things. CHARLES: They get really mad when you charge them too much. [Laughs] JOE: Isn't it amazing? [Laughs] CHARLES: Yeah. JOE: It could be that you are OK with breaking this other end of it where you might charge them the correct amount but the fulfillment of that product or the service or whatever you're selling, you might have a little more wiggle room in that part of the code base. Where if you break something, somebody calls in and says it's broken, and then you're just able to fix it. CHARLES: Is there anything else? If folks are struggling with the way their pull requests work or their code reviews work and they're experiencing friction and tension with their teammates, is there anything else they can do? JOE: What I would suggest is definitely look at the pull request template feature that is in GitHub and make use of that. Because what you can do with that is come up with several sections of the pull request template and say, "These are the questions that we want to ask before we submit a PR." And if you have those questions and those sections right when you go on that template and people read through that as they're making their pull request, they can often catch problems before they get to another reviewer. I can't tell you how many times I have submitted a pull request, started to write the description and [inaudible] these questions and realized, "Oh, this is wrong," or it's broken or it's totally the wrong thing. CHARLES: Right. It's funny and I feel like you want to be asking those questions not when you're submitting the pull request but before you're even writing the code. JOE: Yes. CHARLES: But a lot of times, we don't do that until afterwards and you're like, "Oh man!" Just this process of forcing myself to think about the problem in this way or think about it holistically totally recasts my implementation. JOE: Yeah, things like, "How can this break after we merge it?" [Laughter] CHARLES: Yeah, that's actually a great question, that of one you're talking on pull request. JOE: Oh yes, definitely. CHARLES: Like imagining the [inaudible] scenarios. JOE: Yep. CHARLES: Man, it's almost like you want to read the -- what are the other questions that you have? JOE: So the ones that we have are, first of all, what is this pull request? How does it fulfill the requirements for the ticket? And then how can this break after we merge it? Are there any post-merge tasks that need to be run? If you need to get on a server and run a task or do something after it has gone to production, that's a place to document that there. And then the final question is, how is this tested? CHARLES: Right. That's a good one. And like I said, I think that's the one where I'm going to be starting when I'm both thinking about how my changes will be reviewed and how I review changes. JOE: Maybe I should consider moving that to the top position. CHARLES: [Laughs] I mean, it is like. I'm just thinking about it and it does seem like we go straight to the implementation because that's what's fascinating to us. And so we might have way too much bias for the importance of the implementation over the importance of the test because I have to say I am so guilty of when I'm reviewing. Just looking for the fact that it has a test and not looking for what the test does. JOE: Yeah. CHARLES: And only referring back to the test and trying to understand how the test accomplishes its task. If I don't understand the implementation, I'm like, "Oh, let me look at the test and see how this code is used to the test." That might be problematic but I know that's definitely a little personal goal that's coming out of this podcast for me. TARAS: Excellent. CHARLES: All right. Joe, do you have anything else that's coming up? Any talks? Any meet-ups that you're going to? Any announcements? Anything that you want to plug? JOE: I have a website at JLLeBlanc.com and that's where I am. CHARLES: All right. Well, fantastic. Thank you so much for coming on. JOE: All right. Thank you. CHARLES: Thank you for listening. If you or someone you know has something to say about building user interfaces that simply must be heard, please get in touch with us. We can be found on Twitter at @TheFrontside or over just plain old email at Contact@Frontside.io. Thanks and see you next time.
Sam joins the panelists to talk about frontend and backend team collaboration. Resources: Worse is better The Tao of Microservices by Richard Rodger Sam Joseph is a CoFounder of AgileVentures, a charity that helps groups of volunteers gather online to develop open source solutions for other charities all around the world. Sam's been mucking about with computers since the early 80s and followed the traditional education system through to a PhD in Neural Nets. Next he went all industry, researching mobile agents at Toshiba in Japan, going freelance and then swung back to academia to research peer to peer system and collaborative systems. He now spends the majority of his time trying to make AgileVentures a sustainable charity enterprise, with occasional moonlighting as a contract programmer. Check out his blog at nonprofits.agileventures.org. Please join us in these conversations! If you or someone you know would be a perfect guest, please get in touch with us at contact@frontside.io. Our goal is to get people thinking on the platform level which includes tooling, internalization, state management, routing, upgrade, and the data layer. This show was produced by Mandy Moore, aka @therubyrep of DevReps, LLC. TRANSCRIPT: CHARLES: Welcome to The Frontside Podcast, the place where we talk about user interfaces and everything that you need to know to build it right. My name is Charles Lowell, a developer here at the Frontside. With today also is TARAS: Mankovski. TARAS:: Hello, hello. CHARLES: Hey, TARAS:. Today we're going to be continuing our theme when we think about UI platforms and web platforms, continuing the theme of collaboration and with us to talk about this is Sam Joseph. Welcome, Sam. SAM: Hi, thanks for having me. CHARLES: We've already talked a great deal about how the way in which your team collaborates and the communication that happens between your team and between the different pieces of software, your system, form one of the pillars of the platform that you can't just take lightly. You need to actually be intentful about that. I was thinking we could kind of start today's discussion, kind of talking about some of those collaborations. One that we've probably all encountered, which is usually teams will be split into people who are focused on frontend, people who are focused on backend systems, kind of the services that make sure that all of the nodes that are running on our laptops and our desktops and stuff are running smoothly and error-free and obviously, those two groups of people can sometimes arrive with different sets of priorities and how do we resolve those priorities to make sure that that communication flows freely. TARAS:: What's interesting about these frontend and the backend teams is that our users are not seeing that separation. They only see one thing. They only touch one thing. They actually see as one group but there's tends of be this kind of split between the frontend and the backend. It's kind of interesting that how the user get into this. SAM: Yeah. Obviously in some teams, there's a very clear cut distinction between people at the backend and working with the components that are serving JSON over the API and there are some people who are very, very focused on the frontend and drilling CSS and a number of bits and pieces or even just staying explicitly on the design or UX design and there's a mythical full stack developer who is up and down the platform. It doesn't run exactly in parallel but there is this key thing which is almost how much sympathy or empathy can you have for another person who is not you, trying to use something that you set up. If there was a direct parallel, you'd say, "Obviously, all people who will be working on the frontend are more of that sort of person and perhaps, the people on the backend are not so much that sort of person," but actually, I think you can have people who are doing backend stuff and they're designing API is very, very thoughtfully or the kind of people that consumes those APIs and sometimes, you can have people who are very, very focused on the design and the aesthetics when not necessarily so plugged into how will someone else use this, how will it fits into their lifestyle, which might be very different from my own, so that's maybe another axis, if you know what I mean apart from this sort of pure technical [inaudible]. Does that makes any sense? TARAS:: Yeah. What's interesting is that everyone is trying to do a great job. Everyone is setting out to do something really good. What people's way of expressing good might be different, so if someone could be really focused on the quality of their code like they want to do their version of doing a really, really good job is doing the best code they could write. Sometimes that doesn't necessarily equate to the best user experience. I think everyone that I've met -- engineers who are writing code, almost everyone that I know is trying to do a really good job. If everyone is doing a good job, it doesn't necessarily always equate to the best user experience. CHARLES: I think we had a reference that everyone wants to make sure that their system is of the highest quality possible but quality in of itself is not an absolute value. It's relative. In other words, a good definition of quality is how well a system is fit to a purpose. If it fits very well to that purpose, then we say it's of high quality but if it fits very poorly to a particular purpose, then we say, "This is a system of low quality," but what constitutes something of high quality is relative to the purpose. The question is, what is the purpose of writing the frontend system? What is the purpose of writing the backend system? So it seems to me you're going to have a lot of dissonance if the purpose is divergent but if they both share the same purpose, then that is kind of standard quality on both sides. Does that make sense? SAM: That makes sense and what even more makes it tricky is that actually, purposes can evolve over time and so the thing with the frontend that needed to do with the business today is different tomorrow and the day after. The trick then is sort of the frontend and the backend, as people try to do a great their job, there's this question of how far ahead are we looking so you can talk about important things like sort of asking close modification about [inaudible] extension and there's a [inaudible] of different coding heuristics as best practice that say, you should kind of go in this direction. Sometimes, that will be the hill they want to die on. It's like, "This code needs to be this way because of this thing," and the idea is that it's sort of future-proofing them but I think the messy reality is that sometimes, the thing that you put in place to future-proof, the whole system gets canned. In two weeks, the business changes and what have you, so the extra effort that you are pushing in on one day to protect yourself against changes in the future actually gets lost and perhaps, if only you'd be able to deliver a shortcut pack, this one feature that that will got the next line of funding in, either one advocates cutting corners but... you know what I mean? Like --? CHARLES: Yeah, absolutely. I just wanted to chime in and vehemently agree with you that the purpose can change radically, especially if you're in an evolved environment. What constitutes the good code or the good quality solution can vary just as radically because it needs to fit that purpose. TARAS:: When we talk about frontend and the backend, it seems there's this relationship, which is kind of positional like there's the frontend and then there's the backend. One is in sequence from front to back. It's the direction with the frontend. It's closer maybe to the user where the backend is not. But I think in practice, it's probably more like the left and the right hand where when you have to do two things together, you actually need to do two things to coordinate together. What you describe about and this is I think where taking shortcuts sometimes can actually be a good thing is that if you say, "I'm going to spend the next few weeks on a backend and doing this specific thing," and then you find out that 10 days into the iteration that it doesn't fit, the better approach to synchronize would be to stop the action and realign. But if you persevere, then you might overstep and you actually be out of sync. It makes me think that the art of creating cohesive organizations from development perspective is to create context where the two kind of work in synergy together. I think that's where a good tooling that allows its synergy to exist. I think that's a lot of the work of actually creating systems where the user experience is kind of cohesive and integrated. CHARLES: Actually, there was something that you just mentioned in there but it's actually a little nugget that I'm curious to explore and that is if you're 10 days in to like a 14-day piece of work and you realize that it isn't right and it doesn't fit with the whole, right action is to throw it away. I feel like software organizations and this might be a little bit off topic but it's something that really is interesting to me, so please indulge me. I feel like we see the product being the kind of the code output, even in Agile environments and there is an aversion to throwing away code that has been written. There's the, I would say, incentive is to go ahead and persevere because developer time is so expensive. These are days out of people lives, so they've already invested 10 days. Why not just have four more days just so you have it. When in fact, you actually have more if you just throw that work in the trash because it's an obstruction that's not needed. It's a piece of weight. It's actually something that you now are going to own and it's actually going to be cheaper in the long run and it can be more beneficial to your organization to not own it. Do you all see that happen kind of play out where people become attached to software that they've invested and will make the decision to hold on to it? Say, we'll spend two more days to complete it, even though it's the wrong thing, like identifying which things need to be thrown away? I feel like we don't actually do that very aggressively -- to say, "You know what? We need to not complete this work." SAM: I think that is a really tricky one because I think whatever people might say, when you spend time working on something, you become emotionally attached to it. I would say strongly that how logical or rational or whatever sort of a person you are, my experience is being that people working on things want to see them used. I think in a situation with version control where you can keep things on a branch, they can be useful explorations, things don't have to be thrown away in their entirety. Our charity is doing work for the National Health Service in the UK where there's a lot of employees in Europe and I've noticed the user interface that we're supplying them and I was suggesting some sort of minor tweaks and one of the developers has sort of run with the entire, big, advanced search feature that partially solves the problem but brings in a lot of other bits and pieces. He does some good work there. It's a great work but I think he kind of ran further than I was expecting down that line and I think we have now found a much simpler solution that rather than bringing an entire cabinet, it's a little shading off the side of the existing one. But I think that doesn't have to be a loss and I was reassuring him that the work was valuable and that there is interesting learnings there and potentially, we might use it in the future version of the project and now of course, the way that stuff is moving on, just how it's going to branch, it doesn't mean you can then sort of magically lose some of the work. In that case of doing this 10 days out of 14 and so on, the real question is can you get any value from switching somebody that fast? If they spent 10 days in and they realized they don't need that thing, it's like can you switch them quickly onto something else where they'll do four days or might they as well, just finish up there and leave that on a branch. That's a pull request that gets closed and it's there to go back to. That's kind of depends team by team about how quickly you can repurpose people missed rigs, if you know what I mean. CHARLES: I absolutely agree but the key thing is leaving it on a branch and not integrating it into production, like not actually deploying it because I feel like when we see a feature, it's like we've got to get this thing done and it's done and we're just going to get it in versus now that we've learned how to do this, let's put that on a branch and let's rewrite it and let's take a different approach, rather than just being married to the idea that we're going to get it right the first time or that now is the time for this feature because we understand the complexity of what it actually takes. It is a tricky question. I'm wondering if our processes don't include enough implicit experimentation. We talked a little bit about this on a prior podcast that if they don't include some incentive to not deploy features just because you have them. TARAS:: I think from a business perspective, there's not a lot of model to evaluate a certain thing. We don't have really any effective way of evaluating learning. You can measure code by the numbers of lines of code that somebody wrote and how much of code was shipped but you can't really evaluate how much was learned and how much was persisted. I think a lot of this work around supporting effective collaboration is in building a cumulative systems of knowledge like why is a good Git history useful is because it has a built in mechanism for understanding history. It gives you a way to return back to time in your project and understand the context of that specific change and I think this is something that really good teams do this really well. They will respect this because when it's necessary, it is valuable to have this. When you don't have it and you are a year into the project and something happened and then, you are using the only thing you have, which is your troubleshooting skills or you are trying to figure something, as supposed to going back and relying on that history, on that knowledge that's built into the system, what I'm trying to get to is there is an element that is inherent to our development process, which is we don't have a way to quantify it. We have no way to really evaluate it. I think this is the problem that makes it difficult to throw away work because you can measure the amount of time they'll spend but you can't measure the amount of learning that was acquired. CHARLES: Right. Thatís true. SAM: I think if you have a positive team environment, if you have your team that is stable in there -- your contacts are coming in, the money is there, everybody is there and you've got the team, you don't necessarily be able to measure the learning because the output of the team will be good. They're kind of doing that in the background but I think individual comments are not going to want to pay for learning experience. It's certainly not at the rate of software developer's cost. Although, the throwing-aways, if you're familiar with the [inaudible] book, which I think is a [inaudible] is the author, you know, build one, to throw away, you will anyway. We have a frontend mob that we run weekly doing frontend stuff and CSS and so on and we've done like it's a mockup for the client and there's this question about in my [inaudible] and so Iím saying to other developers there, "Let's not get too attached to this. We may need to throw it away and it will be a great learning to sort of restart on that." The [inaudible] to look quite nicely and the client might get attached to it. They want to ship it and I'm like, "Well, actually there's a lot of other stuff that needs to happen," and so, it's a minefield. It really, really is. TARAS:: I think some of the business relationships that we have to this kind of client consulting company relationship, that relationship might not always create the fit to purpose team for specific challenge. You can have a company that has a lot of employees but their team and their team dynamics might not be fit to purpose to the problem they're trying to solve. If you're building a platform over a long time, you want to be creating that space where that learning gets accumulated over time. It doesn't matter what necessarily the relationship is between the companies that are participating in this process. I think what matter is what are you producing as a result. If you're working together with people from different companies and they're working together and they're building this kind of an environment where you can collaboratively build things together and then people learning from the output and that knowledge is being carried over and people are being able to stack their knowledge continuously over time, if you're creating that environment and you're building a large platform for example, then you are creating a build to purpose kind of technology team to fit what you're looking to accomplish and that'll include consulting companies. I think those team, they're kind of secondary but I think it's just [inaudible] to that. It's like building a quality development organization to fit the purpose of building whatever it is that you're trying to build. If you're building something small, you might have a small team that'll able to build that effectively. If you have something big, you could have a big team that is building that effectively but building that team in a way that is appropriate to what you're trying to accomplish, I think that's the real challenge that company struggles to do. CHARLES: Yeah. In the context of these real teams, I guess what can you put in place given that you have backend teams, frontend teams and each one of these needs to be specialized. This is kind of the power of the way that people work is that we're allowed to specialize and there's power in specialization. You can have someone who can be super focused on making sure that you have these high throughput backend systems that are resilient and fault tolerant and all these wonderful things, so that they don't necessarily have to have the entire context of the system that they're working on inside their head at a given time and the same thing someone working on CSS or working on a frontend system architecture, which has grown to be a very complex problem domain in its own right. But these teams can be working in a context with a purpose is whip-lashing around and changing quite drastically and so, how do you then keep that in sync? Because we've identified purpose as being actually something that's quite a dynamic value. How do you keep these teams keyed in and focused and so, that they're kind of locked in on that similar purpose of they're going to be adapting the systems for work that they're responsible for and specialties that they're responsible for to match that? SAM: It's a good question and I have my own bias view there -- CHARLES: I would love to hear it. SAM: I can't bear working on things where I don't understand in great detail how that end user has experienced or what is the effect overall that it's trying to be achieved. You know, I've been working in boards between the industry and the charity for like 30 years and I know people are commenting like, "You were the person who wants to understand everything." I think there were some people who are maybe quite satisfied who get ticket off the general board or what have you and work on that thing and if the API specs have filled in, so that's it. Go home and they weren't losing a sleep over it. But I think if youíre going to be dealing with these changes and sympathetic that the changes is coming down the path, I think you need to have some degree of empathy with the people who are driving the changes. Do you know what I mean? CHARLES: Oh, absolutely yeah. SAM: You know, as we've mentioned already, I think people get attached to what they build and I don't think you can do anything about that. They will get attached to what they build. CHARLES: Yeah, we've all experienced it. It's so true. It's impossible to [inaudible]. SAM: Yeah. We can try and target it but it's sort of part of our nature. I think there are sort of the tools of the design sprint, of the design jam, of these sorts of things where if you can build to some level is obviously that it can't be shipped but can actually tease out the needs of the user, the designs sprint, the Google team, they have an example with kind of like this robot for hotels where it's like the whole thing is they can have robots that can deliver toothpaste to your door if you run out of toothpaste. In this design, obviously, that would be a huge undertaking to make that as a sort of our production system but they used like a remote control of a robot to simulate all of the key touch points when the client of that hotel, will actually interacts with that robot. I think the same is true when you're building these systems if you can, again going to touch back on we don't fit a code in that way that I think an ever better thing is not be building the code in the first place and to build an almost a non-code system and then, maybe some of the backend are not going to be interested in the results of what people have learned through this kind of user experience trial before they [inaudible] forever but I think it's seeing other people try to use the end system that puts you in place where you can be empathetic. I argue for the bias point of view that I think people on the backend of Netflix, trying to optimizing the streams in that or the other, they need to spend some time watching Netflix or at least watching the comedy of Netflix in order to empathize of how their work relates to the end experience and then, when those end experience needs changed, it's important for them to make change on the backend. Do you know what I mean? CHARLES: Right, exactly. They have to watch through that kind of glass window where they can actually help the person. They can't give them any information. The only levers of control they have is through their own work, making the thing changes that they need to make on the backend, so that one of those users is going to have a good user experience. This idea reminds me of something which I believe is the case at Heroku where they rotate everybody in the company through pager duty, which I think is kind of a brilliant idea where the entire team is responsible for providing technical support to the end users. When a problem arises, you get to understand what it's like to be someone who's trying to use the system. You get to exposed to the satisfaction, whether an issue was resolved or when it's working correctly or your pager is quiet for your entire shift but you can also get exposed to the frustration that you're engendering in the users if something doesn't go quite according to plan. SAM: Yeah. I'm not [inaudible]. TARAS:: There is movement in this direction because there is actually a term that has been floating around. It's like a product engineer. It's someone who thinks about the product. These kind of people tend to have the highest value in Silicon Valley as someone who thinks about the product as the outcomes supposed to their code as the outcome. Even in the Agile space, there seems to be movement in that direction because I think one of the challenges, like to do that, you have to understand how you fit into the bigger picture. I could see it being really difficult. Imagine if you're working at a bank or something and being pager duty at a bank would be impossible, simply because their organization is either too big or too sensitive. The way that their company is dealing with that is they have these group of people where you have a product manager working closely with the frontend engineer and the backend engineer so there's this exchange that is happening where people get to understand the consequences of their actions a little bit more. They understand how things fit together. There is definitely a movement happening in that direction that I think just the more, the better because one of the big differences now is the things that we make are very palatable to the users and the quality of the user experience that users are now expecting is much higher than they were in the past. I think the world is changing. CHARLES: Yeah. We mentioned this when we were talking before the show but going to the University of Michigan as I did, I was in school with a bunch of mechanical engineers. Their goal was to go work for auto companies: Chevrolet and Ford and everything like that but their mindset was very much, I think the product engineering mindset. All of them loved cars and wanted to be part of building the coolest, most comfortable, most responsive cars. For whatever definition of a good car -- there are a lot of them -- they wanted to design good cars but they were into cars and they were into the experience of driving a car, so what's the equivalent then of that product engineer in software? I think that every conversation that I had with any of the mechanical engineers who were going into the auto industry, I'm sure there are some of them that were out there but it wasn't about carburetors or whatever. They really wanted to just get into the building of cars. In some aspects, I'm sure they all integrated in some way or another but it was a group that was very focused on that outcome. SAM: In the UK, I'm hearing this term, 'service designer' a lot that's coming up. Are saying that, Charles, you get the feeling that even getting into software, I'm not so excited about using their own? Like the car, I want this amazing car and then I can drive the car and I'll experience the car --? CHARLES: Right and other people will experience the car that I've helped design. SAM: Right and in some way, in software, it's almost like the software itself there's this kind of mathematical beauty of its own and it's like always been more important than the other software heads around me like the software that I wrote. I mean, the user? At the conference and all the software guys love it, that's the key objective, isn't it? Yeah. I work with a lot of learning developers who are rightly focused on wanting to improve their skills and wanting to level up in particular tech stacks that are the ones that will lead to jobs and the future and financial stability and so on but I kind of wish I could inculcate in them this desire that the user experience was the higher goal than which bit of code does this or the other. Maybe, I'm being unfair to them when I say that. Coding is hard. There's a lot strange concepts to grasp than sort of grappling with those concepts -- how does this work or why does this work or should I use this function or should I use this method or what have you. I don't know. I think [inaudible] bit a wall when I'm sort of saying like, "Let's think about it through the user perspective, what does the user needs here." It feels like they want the safer answer of what's the correct solution here, how should this be refactored because you need a skinny controller or a fat model or whatever happens to be. I don't know if that's -- CHARLES: Right, like what's going to make my life easier, in a sense of if I'm going to be maintaining this code and it's important. It's very important. You want to be happy in your job, you want to be happy in your work and so, you want to have the skinny control or the fat model because it's going to lower the risk of pain in your future, supposedly. I definitely understand but that can't be the only incentive. I think it's what I'm hearing. The thought just occurred to me, I actually don't know that much about game development. I know the game industry is certainly has got its problems and I don't know much about the development culture inside the game industry. I do wonder what it's like because it does seem to be somewhat analogous to something like the car industry, where if you're a developer in the game industry, you're probably extremely focused on the user experience. You just have to be. It's so incredibly saturated and competitive for just eyeballs and thumb on controllers. Like I said, I don't really know anything about the game industry when it comes to development but I'm wondering if there's an analogy there. SAM: Yeah. It sounds strong to me and as much of my cousin who works in the game industry and I sort of taught game programming and work through a lot of it. The game industry has got these sort of user testing experience baked in and it kind of like repeat, repeat, repeat, repeat. There's this sort of constant cycle of doing that and in the somewhat wider software industry, in a certain extent, pay lip service to that. CHARLES: The game industry, they live and die by that. The code lives and dies by how it actually plays with real users. SAM: Yes and it feels like in the general world, there's a lot more user interfaces that they just sort of struggle on whatever market dynamics people need to use, these existing vested interest, these banks, these supermarkets and what have you. I guess the market is too big, almost. It's not covered enough but do we really want our industry to be so cutthroat? I don't know. CHARLES: Yeah. We've kind of seen an example of a couple of industries: the car industry, the game industry which is kind of adjacent to where most software development happens but they have this concept of exhaustive user testing, kind of the pager duty if you will, where you get to experience that. So how do we, on our team and when I say our team, I mean anyone who happens to be listening, what's the equivalent of pager duty for the applications that we write? How can we plug ourselves into that cycle of user-ship so we can actually experience it in a real and repeatable way. SAM: In the work that we're doing with the NHS, they have a similar program. There's a lot of people who are working purely death by desk jobs but they do have a framework for us to go and observe in the hospitals and emergency rooms and so on. I guess the 'how can we achieve that outside of those clients who have those framework in place,' it seems like maybe we need a version of the cycle where a different person gets to be the product owner and tries to represent what the users are experience each week. I think almost though, it seems like we need more of that thing you mentioned, Charles which is like kind of being behind the one-way silvered mirror as some sort of framework that connects the loop between some of the individual developers are doing and their experience of how the users are seeing things. I think that's going to be difficult to introduce. Just to sort of back up a little bit with them, I see this kind of idea of Agile, which says, "Let's be using the software which is the thing that's kind of changing and evolving unless you already deployed and you are in the maintenance mode from the beginning and you're getting that thing out there so you have your one-week or two-week or three-week cycles where you keep on having touch points," and I think that's better than touching once every two years. As malleable as modern software potentially is, it's too sticky. It's like you were saying Charles before with that deploying or whatever, these are all gyros and so, you kind of need a really good mechanism for mocking out interfaces and having users experience and there's a couple of [inaudible], vision and there's something else that's sophisticated systems on the UX space where you can put together sort of a simulation of your interface relatively cheaply and you can run these things and then have sort of video capture of the users interacting with the system. I think the difficult part that we have in the software industry is we are in this thing, where there's not enough people wanting to be software developers, partly with I guess the cars and the games, is there's so many people want to be game designers that the industry can kind of set the terms but we're in this inverse situation here with software developers where software developers are so much in demand, they can kind of say, "Don't put too much pressure on me. I'll kind of like, I'll go off to different company." You know, we kind of have to like herd the cats, as they say into allowing these high powered software developers to go in the direction as they want to go in and so, it may be difficult to impose something like that, where you're getting software developers to really experience the end user's pain. I guess what one has to do is somehow, create a narrative to sell the excitement of the end user experience and the beautiful end user experience to software developers such that they're fully out of themselves and say, "Yes, I want to see how the users are using my software." Do you know what I mean? CHARLES: Yeah. TARAS:: Yeah. Because a lot of company had this process where there'll be a product manager, there'll be a designer, they'll design through mock ups and they'll just hand it over to developers to build. There could be some backend, some frontend. I think if you're doing that, there is no emotional engagement between the creators of the actual implementation and the people who are going to be using that. One way to do that is to try to add more of this actual 'however you do it.' You know, introduce more of the personal experience of the users to go along with the actual mock ups so that people understand like, there's actually be a person on the other end that are going to experience this and create some kind of emotional engagement between these two end. How you do that, I think that's a big question but I think if the organization is simply just throwing designs over to development, that's where the part of the problem is and trying to -- CHARLES: Well, actually, that's a fantastic point because ultimately, we've talked about emotional engagement and attachment to the software being a liability but the reason it exists and the reason it's intrinsic to the way we do things is that's how things get built. We get emotional attachment to it. It's the impetus. It's the driving force. It's literally the emotive force causes us to go forth and do a thing. It's why we're going to do it as a good job. Like you said TARAS:, if you just kind of throwing your tickets over the wall, there is no emotional engagement and so people will look for it wherever they can find it. In the absence of an emotional engagement, they will create their own which is good quality software that's 'clean' code because people need to find that meaning and purpose in their work. Maybe the answer is trying to really help them connect that emotional experience back to that purpose of the user experience, so that there is no vacuum to fill with kind of synthetic purpose, if you will. SAM: That's a great point. The charity in AgileVentures that I run, when we get things right, that's the thing that happens in that. We go for transparency at open source. We have these regular cycles where the charity client is using the software in a Hangout with us, with the developers who work on these things and the developers whether they're in a Hangout Live, whether they're watching the video a week later, they see the charity end user struggle with the feature that volunteer developers have been working on and they make that attachments. It's more than just 'I want to learn and level up in this thing.' It's like 'I want to make this feature work for this end user' and we're very lucky to have these charities who allow us to do that level of transparency. The difficulty often comes that I would see in our paid projects where I would love to be recording the key stakeholders using the system but for whatever political reason, you can't always get that. I think that process of actually connecting the developer with the person who using it and doing that reliably, so that they can have that empathy and then get that emotional connection. It's just tricky in the real world. CHARLES: It's very hard. One thing that we've deployed on past projects, which I think has worked fairly well is kind of putting a moratorium on product owners writing stories or writing tickets and actually having the developers collaborate with the product owners to write the tickets. In other words, instead of kind of catching a ticket that's thrown over the wall, really making sure that they understand what is the context under which this thing is being developed and then almost as in a code review sense, having the product owner saying, "Actually, the reason we're doing it is this," and having a review process where the developer is actually creating the story in support from the product owner. Because ultimately if they have that context, then coming up with the implementation is going to be much easier. It's really is about facilitating that upfront learning than being able to do the actual work. That's something but a lot of organizations are resistant to that because the product owners really want to say like, "No, I want it to go this way and I want to just hand this to a developer and I want them to do it." It's kind of wrestling in control and saying, "You've got veto power over this. You're the editor but we really want to make sure that you're on same page with the developer." In cases where we have been able to kind of have product owners make that shift where the developers are owning the story, oh sorry, or the primary authors of the story and more of the code reviewers of the story, if you will, implementation just seems to go so much more smoothly. The questions, the key points kind of come out of the front of the process and by the time you start actually working on the thing, the manager or the product owners has the confidence that the developer understands what is involved in making it work. SAM: Getting that done, what one will say something as tricky to do, I think in the ideal world, you have more of the developers involved in the design sprints or the design jams. The logistics of it are you can't really afford to have all those developers in all of those soft meetings where they are coding away. That sounds a great medium there. I think there's various organizations that do sort of their kick offs where their stories are kind of if not code designed, then there's cooperative voting on the complexities of the stories and making sure that they have folks understand the stories that they're working on and there might have some very different organization that are going to have different constraints on how much time that the organization feels, the different people can be allowed to spent on different sorts of activities. It's sort of tricky, isn't it? CHARLES: It is definitely tricky because any time that you allocate for people, that's the most expensive resource that you have, so you want to be smart about it. SAM: Which comes back to that issue then again of repurposing people. If you've got that feature that you go over 10 days, can you say to that person, "Can I switch you over onto this other thing for four days?" Maybe logically, that would be a better outcome for the sprint but maybe emotionally, that person is so attached to that. They are not fighting that front. The biggest trouble for me, I think over the last 30 years is I assumed that the logical stuff was tantamount. These things like the skinny controller and the fat model, we'd agreed on that that was correct and so, that's why I should pushed for, that's why I should fight for because it's the truth or whatever and actually, maybe I'll sling back around in another 30 years, you can choose your battles because the level of emotional pressure on people, then the whole thing just explodes and nothing gets done. You know what I mean? TARAS:: Sam, I have experienced something very similar and I think about this all the time. It's just how many perfect things are perfectly acceptable to people in using a different lens when I think about technology because the more skilled you are, I think quite often, people become more pedantic about how they approach things but in practice, the more things that you see that are not written but you realize how really imperfect they are and how in many ways, they fit the world very well, people use it. It's being used by many people in its imperfect state, so there is something like this hole between perfect implementation and the role of this perfect implementation in the world that I think that duality is really interesting. CHARLES: Yeah. There's a fantastic paper written by, I think it was Richard Garfield back in the late-90s or early 2000s called 'Less is More,' where he kind of talks about this exact tension and he calls it the MIT school of software and the Berkeley school of software and I think Garfield came from the MIT school but basically, the whole thing was saying the Berkeley School is actually right and the name of the paper is 'Worst is Better.' It's a really interesting essay but just talking about working things that are in people's hands will beat the best design every single time because those are the systems that get used and improved. There's a lot more to it than that. He talks about this exact fundamental tension and obviously, no system exists on either one of those perfect poles -- the MIT school or the Berkeley school. I think what he was trying to point out is that exact fundamental tension that we're talking about, the quest for the 'correct solution' and the quest for a solution. I'm actually have to go and re-read it. I remember it being an intriguing paper. SAM: I guess the thing that makes me think of is sort of related to philosophy value. I read recently about this, attachment to outcomes. I'm going to segue back but we've got this discussion in one of our teams about, this is sort of microservices versus monoliths and in reading this, I doubt microservices, which is actually pretty radical in some of its suggestions that they're talking about it in Greater Than Code Slack where it's sort of saying that actually, if you keep your microservices small enough, then a lot of the things like code quality become actually kind of irrelevant it sort of almost argues that a lot of the things that we like about the Agile and these things are our ceremonies that are only necessary because we trying to feed these monoliths. I'm not saying it is true. Iím just saying it's a pretty radical position that itís taking and it certainly, captures the minds of some people in our organization. Some things that I've been trying to make some simple changes just to smooth off some of the edges of the platform that we're working with and I had this respect of like, "No, we can't just make those simultaneous. We need to move to microservices." They're great and it will be perfect and they will allow expansion and so on and my immediate reaction is sort of, "I'm glad we don't have to build microservices because [inaudible]. It's all about attachment to outcomes, so am I attached trying to automate part of my week that I think will have some positive goals in the future? No one's got a crystal ball. That's only a guess on my part. If I've got some folks who are excited about doing this thing with microservices, maybe I should empower them in that and I should started a mobile microservices and we kind of playing with these set of framework and so on. It's interesting stuff as I'm working my way through the book. At the same time, while we've been doing that, I've actually delegated it to someone else to sort out this sort of thing and I've actually kind of addressed the problem what we would need the microservices for through a different mechanism but still, the microservices thing rolls on and I kind of think, "Well, is that all a complete waste?" But then actually maybe, in two years down the line, it will turn out and that will be a beautiful thing that will enable things. The further that I go on, the more I say, "I just don't know," and actually, if I can detach myself from caring too much about the outcomes one way or the other, I think it's both long and enjoy myself but it's a tricky thing when you got to pay the rent and pay the bills and so on. I don't see any resolutions to that. I love people being able to use things and get stuff done. I'm so excited about that and I guess, I will keep pushing all of the developers that I'm with towards trying to have a better empathy or understanding of their end users because I think software is more fun when you're connected to the end people who are using that. CHARLES: Absolutely. Microservices reminds me kind of the experience that we've had with the microstates library. Because we've kind of identified this one very small slice of a problem, a 'refactor,' it's basically a ground up rewrite. If you've got a library that is a couple of hundred lines of code but it presents a uniform API, then you can rewrite it internally. You can refactor it by doing a ground up rewrite and the cardinal rule or the cardinal sin is you're never supposed to do a rewrite. SAM: Yes and that's the microservices that they're saying. It's like you should rewrite everything all the time, basically. CHARLES: Exactly. You should always be rewriting it. SAM: Yeah. Should be able to throw it away because it's a microservice and it can be rewritten in a week and if you're throwing one away, it's only a week worth of work and you can keep on moving away. It is an amazing idea. CHARLES: I have to read that book. Anyhow, we're going to have you on again to explore -- SAM: Well, let's do another one on microservices. It was a great fun. Definitely, it's time to wrap up but I really appreciate you having me on the show and being able to discuss all these topics. CHARLES: Fantastic and I can't wait to talk about microservices. This might be the impetus that I need to finally actually go learn about microservices in depth. SAM: I'll recommend Richard Rodger's book, 'The Tao of Microservices.' CHARLES: All right. Fantastic. Anything we should mention, any upcoming engagements or podcast that you're going to be on? SAM: Iím always on the lookout for charities, developers, people who want to help, you can find us at AgileVentures.org. We're busy trying to help great causes. We're trying to help people learn about software development and getting involved in Agile and kind of like experience the real software in action, software development where you ideally interact with the end charity users and see how they're benefitting from the product. We love any support and help. You can get involved in that, if you want to give a little bit to open source and open development. We go for transparency. We have more mob programming sessions and scrum and meetings online/in-house every day, so just come and check out AgileVentures.org and maybe, see you [inaudible]. CHARLES: Yeah and if they wanted to say, reach out to you over email or Twitter, how would they get in touch? SAM: It's Sam@AgileVentures.org and I am at @tansakuu on Twitter. Hit me on Twitter or just at Sam@AgileVentures.org. CHARLES: All right. Well, fantastic. Thank you, Sam. Also, if you need any help with your frontend platform, you know where to get in touch with us. We're at @TheFrontside on Twitter or Info@Frontside.io. Thank you, TARAS:. Thank you, Sam and we will see everybody next time. Thank you for listening. If you or someone you know has something to say about building user interfaces that simply must be heard, please get in touch with us. We can be found on Twitter at @TheFrontside or over just plain old email at Contact@Frontside.io. Thanks and see you next time.
Jacob joins the panelists to talk about team collaboration based on his RubyConf 2017 talk, Code Reviews: Honesty, Kindness, Inspiration: Pick Three. Jacob Stoebel is a software developer living in Berea, KY. He spends his days writing web applications in Ruby, JavaScript, and Python, working with data, and leveling up as a software engineer. He works and studies at Berea College. You can find out more about Jacob at jstoebel.com. Please join us in these conversations! If you or someone you know would be a perfect guest, please get in touch with us at contact@frontside.io. Our goal is to get people thinking on the platform level which includes tooling, internalization, state management, routing, upgrade, and the data layer. This show was produced by Mandy Moore, aka @therubyrep of DevReps, LLC. TRANSCRIPT: CHARLES: Hello and welcome to The Frontside Podcast, the place where we talk about user interfaces and everything that you need to know to build it right. My name is Charles Lowell, a developer at the Frontside. With me today from Frontside also, is Taras Mankovski. TARAS: Hello, hello. CHARLES: Hello, Taras and today, we're going to be talking like we do every time about a piece of the platform that you used to develop user interfaces frontside at your company or organization or wherever it is that you build software. Today we're going to be talking about a piece of the platform that's very, very critical that often gets short shrift or is excluded entirely from what people think of when they think about their tech stack and that's how we as teams collaborate to build and maintain and produce the quality software that we can. With us today is Jacob Stoebel. Welcome, Jacob. JACOB: Hello. CHARLES: Now, what is it that you do, in your day-to-day? JACOB: I'm a full-stack developer for a little company called ePublishing and I mostly work in Rails and React. CHARLES: Rails and React and so, when we were searching for people to talk about how we collaborate these teams, Mandy suggested you because of a talk that you've given at a RubyConf, specifically about code reviews, which I think are actually a huge piece of the collaboration process because it's a major forum where team members get to interact with each other and it's the gateway for making sure code quality is maintained but more than that, I think it's a learning -- a place where we learn. I learned so much both as a reviewer and as someone who is submitting my work and so, it's actually a very important part of the software development process. You have a lot of great examples of how to not do code reviews. JACOB: Yeah, I think I may have been a little bit too indulgent in that talk. I had a lot of fun. I did some research from other people, mainly from anecdotes. I had research from talking to people about really, all the anti-patterns that come out of code reviews. It seems like every few weeks, I'll see a tweet that says something along the lines about how code reviews are broken. I don't really know about that and I have to say, I think I'm kind of lucky at my job that I think they're done in a way that really leaves me feeling pretty positive and that's certainly a good thing but I think what it comes down to -- I'm going to sort of talk about where these ideas come from in a minute -- is that we often have code reviews that for one -- and you can tell me how this is for you too -- often the code review is happening at a point so late in the process, where the feedback that you get may not be actionable. Have you experienced that? JACOB: Before I answer that question, just to kind of echo the sentiment and maybe I'm being presumptuous, I feel like the code reviews that we do are actually very positive, so I haven't got to experience firsthand. Although I have seen conversations on GitHub where it looks kind of like a Celebrity Chef, where you have someone doing the code reviews like Gordon Ramsay up there just screaming and someone has put this plate of food in front of them and kind of picking it apart. That one is extreme but this is actually something that I struggle with, what you were talking about, what is the appropriate point at which to get feedback. I agree that you want to get feedback as soon as possible and sometimes, when you've invested weeks and weeks into something or you're like at Mile 100 and they're like, "You know, at Mile 2, you were supposed to turn right," and now, you're off in the forest and you've been tracking 98 miles in the wrong direction. JACOB: Yeah and the work is due, right? This needs to get ships tomorrow. CHARLES: Right, so you've got massive pressure. This is something that I struggle with myself is when is an appropriate time to really try and be public about what it is that you're doing. JACOB: Yeah. I think that is a really good question and I think what you're getting at and I would agree is that, the sooner, the better and when you can tighten the intervals between feedback is probably better. I'll just take a step back and I'm going to take a longer route to go and get to my point. I am a career changer and before I was in this career, I was a high school theater teacher, so it was really different and I won't give answer why I changed other than this is the greatest [inaudible]. But one thing that I really struggled with is I was working with teenagers and I really wanted to see them grow and improve but at the same time, these were kids and they have fragile egos and I don't want to tear them down, so I came across this really interesting framework for feedback. It is called the Liz Lerman's Critical Response Process and I give credit to her in the talk. This comes out of the dance world. Liz Lerman is a pretty accomplished dancer and choreographer and what she found is that in the dance world and I think it's not too dissimilar from our industry is that the feedback received was often given in a way that was leaving people feel really torn down and mostly, not feeling inspired to go back to their work and make it better. It really felt like feedback is about giving you a grade. It's like, "I'm going to tell you how good of a job you do," and there's certainly a time and place for that but the inspiring question -- I guess the rhetorical question -- that she made is, "Shouldn't the big point about feedback be that it makes you so excited about the work you're doing that you just can't wait to go back to your keyboard and keep working on it," and she found or in the case, back to the dance studio, it really sort of structures this framework for how people can give feedback to a creator and that could be a creator of anything. You mentioned cooking. This could be about food as well that sort of set some guardrails for how we can give feedback that is useful, it's inspiring and it's kind. I'm going to really distinguish between kind and nice. Nice would be, here I'm going to say things that are only pleasant about your work but what I mean by kind is feedback that is really taking care of your team and making them feel like they are respected and cared for as human beings so they don't go home every Friday afternoon and cry on their couch and just drag and coming back on a Monday. That's kind of the basic idea of why we need, maybe a kind of a framework for getting feedback. CHARLES: I agree totally. It's almost like you need to conceive of your feedback is not a gate to quality but a gift to embolden somebody. It's like they've just been doing battle with this code, with this problem, they've been grappling with it and it needs someone to wipe their brow and maybe give them a stiff drink, so that they can get back into the ring and be invigorated. JACOB: Yeah. TARAS: What I'm hearing in this conversation so far is it's kind of like a tone or way of communicating to the person receiving the feedback but sometimes, no matter what your tone is, depending on how your team is set up, depending on the context of your actual code review, it can still kind of land in the wrong place? Have you experienced that where the team conditions impact your ability to actually provide feedback? JACOB: I think I know what you mean. I think what you're talking about is this sort of the organizational structures that are set up are sort of lend themselves to certain modes of feedback and discourage others. I will give this example that I've heard from numerous people and I think this is what you're getting at is feedback that's given at the end. Just like I said, feedback that is given the day before, it has to be shared. Or feedback that it's almost like if we work in an environment where sort of like the hair on fire environment where it's like everything was due yesterday, that's not an environment that's going to be conducive to slowing down, taking a step back and saying like, "Let's point out what we think this work is doing? What questions we have about it? Who has opinions about the direction that's being taken on it?" And really zoom out a little bit more. CHARLES: Do you have any concrete examples of how that early stage feedback has taken place at your work? Is it just over Slack? Maybe, the whole people need to step back from the idea that working on one-week iterations or two-week iterations and at the end of the iteration, everything is due and everything will get merged at that point. How do you kind of break up that structure to say, "No, we're going to try and have checkpoints and milestones?" What are deliverables that you can decompose the work into that fit inside the framework, that are inside the big deliverables, which is sensibly, the feature that you're working on? JACOB: I think first of all, it's the way this thing goes about. I don't think this framework works over Slack. I think it has to be immediate, I think it has to be either in personal or on Skype or Hangouts or something. One of the points of this type of feedback is really about checking in with each other as a team and I think what's great about Slack is that it lets you leave a message and walk away and the unfortunate thing about Slack is it's not meant for sort of attaching how people feel about things or what people's reactions were to them. I think you all have to be in the same space, hopefully with your camera on, if possible and really, just sort of checking in with each other as a team. I can point to times with my team that I think that's really worked out. I think you made a good point when it comes to really getting started with a project because there are times where I made the mistake of not trying to get feedback from my team early on when I was going to start a project and as you can imagine, did that really cost me? It's really getting feedback from my team and my supervisor about the direction this is going to want to go and I'll say from experience, I work on legacy codebases and as you can probably imagine, it's easy to paint yourself into a corner and what the thing about legacy code is that you don't know what pitfalls you're working into. You can get started and you can sort of find into the process, there is some reality about this code that you didn't know and it is really getting in the way of your work that had you known about it, it would have saved you a lot of trouble because you could have planned your way around it. Then the fortunate thing is drawing on the wisdom from people who they know about it because they struggled with it already if they've been around longer than I have. I think that's a really good point. This is a good thing to do. As you're getting started and you can be sharing this is my just initial idea of how I'm going to go about this. What my tech stack is or my understanding of the problem space, all of the above and to really sort of check your assumptions and see if they really check out. CHARLES: I want to circle back a little bit to something you'd mentioned before and that is you want to be kind in a code review and I would say, you probably want to be kind anytime you're giving feedback or interacting with your teammates but what's the process that you can go through if you find yourself struggling? How can I deliver this message that I want to deliver and have it come across this kind? What's a process or checklist that I can go so that I don't open my mouth up and say something that I can't take back or that is going to land wrong? JACOB: I'm glad you asked that and I think that this is a great way to introduce the guidelines for the critical response process. There's basically four steps to it and the way it's structured is you have the person who has created the thing. It could be the person or the team that has created the feature. You have other people who are responders to it. They could be people within your team or they could be others in the organization who are invested in your success. They're not here to tear you down or give you a grade. They're here because they want to see the project do well and then, there's going to be someone who is a facilitator. The way it works and I think this is directly addressing your question is there are guardrails that are going to help you not steer into territory that is going to leave someone feeling torn down. The first step to this process is all the responders are going to say statements of meaning about the work and what I mean by that is you're going to make things that stand out to you that are not attached with an opinion. You could say, for example, "This project is using React hooks. That's something I noticed." You're not going to say, "I think that's a good idea or a bad idea," but you're going to say, "This is what I'm noticing," and that is something that can get jotted down because that's going to be fodder for discussion later on like, "Let's talk about this new feature in React," and let's talk about it later. We can talk about if we think that's a good thing or not or what the implications are for that. The next step after that is that the person who has made the work of the team, they get a chance to ask questions of everybody else about what they thought. You've probably noticed that in a lot of pull requests, someone puts their work out there and then, the next thing that happens is everybody starts commenting on the work and saying what they think about it. This flips it. At first, you as the creator, start asking questions or you can say, "This thing over here, I think it's maybe a little wonky or it's a little hacky but I couldn't think of a better way to do it. What do you think? Do you think it's right? Do you think it's a bad idea?" Or this bit over here, "I'm pulling in the third party library. I think maybe, it's worth it but what do you think? Is it not worth it?" CHARLES: I like that because ultimately, the people who have created the thing are the most familiar with the problem space. They've spent a lot of time thinking about it and so, they can actually direct the conversation to the parts of the implementation that really are the most iffy and the most unknown because everybody is going to feel this need to comment but it's the classic case of bikeshedding really, the true experts or the people who've just been spent all this time implementing and so, people will comment up to their greatest point of familiarity with the problem, which could actually be not that great. Can you say like, "Can you really focus your mental energy on this?" I like that a lot. JACOB: Yeah and really, it sets a good tone. The assumption behind all of this is that the creator, like you said knows the work best and really ought to be in the driver seat when it comes to feedback, so it really sets the tone there and say like, "The creator knows what's most important. Let's give them the first opportunity to frame that," and I know plenty of times like if I'm asked to review a PR and I'm not given any kind of prompt or direction, I will probably scroll to the file. Maybe I'll find the file in that PR that I'm actually familiar with or I'll look for some pattern that's something that's like, "I understand what's going on here. I can give feedback," and if it's all that other thing over there, I'm completely confused, I'm going to ignore it. But the creator could illuminate me a little bit, then I would understand a little bit more and then, I'm in a position to comment on that thing that I otherwise wouldn't have understood. That's the second step. The third step is now the responders get a chance to ask their questions but the important thing about that is that there are neutral questions. The assumption here is that the responders need to better understand the context from which this code was written in order to be able to give their opinion. Here's an example coming from the Rails world. I could say, "Tell me your thought process for using Factory Bot," in this example. If anyone doesn't know that is somewhat of a contentious issue in the Rails community, rather than just coming out guns blazing and say like, "I think this was a bad idea." It's like, "Tell me your process because I want to understand the context you came from and then we can together evaluate if coming from that context makes sense or if it doesn't," so neutral question. They have to be neutral questions. By the way and we've probably all heard this before, this is not a neutral question, "What were you thinking when you decided to do blah-blah-blah-blah-blah." Everyone knows what that means. CHARLES: Right. I was going to say, you can be very, very passive-aggressive with questions. JACOB: Yeah, exactly and this is a place where having a facilitator can really help -- facilitator who is not one of the creators. Someone can just say, "Let's back up. Can you rephrase that without the opinion embedded? Can you just say that again?" and with that again, those are some of the guardrails that keep us on track. After the responders have been able to ask their questions how hopefully everyone has a better understanding of the context from which the code was written, then it's time for opinions with consent of the creators. The way it works is the responders can say, "I have an opinion about using React hooks in this codebase. Would you like to hear it?" The responders can say, "Yes, please. Let me know," or they can say, "No, thank you," because the responders, having the best knowledge of the context, might know that that feedback is not useful. Maybe, they have a really good reason for using React hooks or maybe, they know what's coming in the future or maybe, they know if there's no time to fix it now or it's not worth fixing now. They know the tradeoffs best and so again, those are guardrails and it puts the creator in the place of saying, "That's actually not useful feedback right now. Let's just not use it." They can say, "Yes, please tell me," or they can say, "No, thank you. Let's not talk about that." CHARLES: In the context of a pull request, the process you're describing could be played out in any number of media but in the context of a pull request, the creator is the person actually submitting the code, how do you handle the issue of who pushes the merge button if there's still some opinions that haven't been voiced? For example, a creator says, "No, I don't think that's helpful feedback," is the assumption of then the creator can go ahead and just push the merge button? Or is it basically saying, "I don't want to hear your opinion," relatively rare? JACOB: That is a great question. I think I tried to get at this in the talk. Before one thing, I am probably, like most people don't have the option to just say to my manager, "No, I don't want to hear your opinion." I get that. There is a contrast between the pure version of the feedback process and then reality. They have to balance. But teams can sort of work out the way they give feedback. I have example of an anecdote that someone shared with me once, when I was doing research for this talk. There was this sort of agreement with the manager that their manager wouldn't give feedback on Friday afternoon. They just wouldn't. Everyone preferred that. Everyone sort of wanted, say on Friday afternoons, I'm really going to be just focusing on winding down for the week and I don't want to get dumped on a bunch of feedback. The point being, even though you can't just blanket-ignore feedback, you can work out circumstances with your team for the best way that feedback can be given and circumstances under which, it can be politely declined. CHARLES: I see. TARAS: I'm curious about a different part of this because a lot of this is how to give feedback but I'm really curious about the why part. I think many of us take it for granted that good code reviews are very valuable because a lot of teams that I've encountered that are taking on really big challenges but didn't have a code review process and so, one of things I'm kind of curious is for people or for teams that don't have it in place, what kind of symptoms can they observe in their daily operations that would suggest that maybe code review is something that they need to put in place. CHARLES: Taras, you're talking about kind of the places where we've seen where the culture is just push everything to a branch, there's a 1000 commits there, open up a pull request and no description, no name. It's like, "Here's this thing. I'm taking comments for the next three hours and if everything goes well, let just merge." Are you talking about those kind of situations where really a big culture of pull request or just feedback around change is very, very nascent.? TARAS: Yeah, a lot of times, it's the 'ship it' culture, like this is getting in the way of shipping it. CHARLES: So you're saying like how you sell the entire idea? TARAS: Yeah and if someone is listening who is noticing that, a lot of people would know that we're not really doing code reviews but what are the symptoms that they could be observing that could say like really, this is we need to change, like this can't continue. JACOB: Yeah. One thing that occurs to me as if there's all high level of surprise when you read it, when pull requests are read, it's like, "Oh, you went in that direction." I think that could be an indication that maybe, we could have checked in at the halfway point or even sooner because there seems to be differences of perspective on where this is going or where it should end up that's why [inaudible]. TARAS: At what point do you think people would see this? Is this something that would happen kind of way down the road? Like actually, "How did this end up in the codebase?" CHARLES: That's surprising except deferred even further, right? JACOB: Yeah, who wrote this last few. In having that kind of feedback process, let's sort of take a look at where this project has come in the last few months and see if we can sort of learn from what went well and what didn't. CHARLES: This is related to the concept of surprise, if you see a proliferation of many different patterns to accomplish the same thing, that means the communication is not there. People are not learning from each other and not kind of creating their own code culture together. If that's missing and that manifests itself in all kinds of ways, in bugs, in weird development set up that takes me two hours to get this local environment set up and things like that, if you're seeing those things, chances are you need some way to come together in a code review culture is really, really good for that. JACOB: Yeah and I think in the ideal code review culture, everyone that's sort of brought on board is saying, "I am going to share responsibility in this patch, doing what it's supposed to do and not breaking anything or not burning down the world." You mentioned the 'ship it' culture, I could imagine toxic cultures where the person who shipped it, if it broke everything, it's on them to fix it and they have to get woken up or whatever and the idea about feedback is now we're sort of forming a community around what was done, so it's like the person that push the merge button isn't the only person involved. It's like if we have a culture where we say everyone that participated in the PR is collectively sharing the consequences and that will happen eventually and say like, everyone who's on this thread, it's on all of us if something goes wrong to fix it. I think that is certainly something that one would hope you'd see. TARAS: I'm curious, where do you see this kind of observations usually come from because sometimes, I can imagine there, being a developer, coming on the team and they're like, "Why wouldn't I do code reviews?" and they're like, "Well, we have always not done code reviews," but then there could be someone like a product manager or somebody who's like, "Why wouldn't I do code reviews? We should start code reviews." Have you seen ways of introducing these ideas to teams that have worked out well? JACOB: Yeah and I should probably say that, I'm not a manager, I'm certainly not an expert in how this works so I actually don't have a really great example, personally. I can churn example from working in a previous team, where everyone secretly wanted to be more collaborative but didn't know how to do it because if I'm the first person that puts myself out there and no one knows else knows how to collaborate with me, how to reciprocate, then what was the point? For the top 5% of teams that are just really have great energy together, they don't need a framework to do this sort of thing. They're just doing it naturally. I think for everybody else, we need some kind of guidelines to do this because for better or worse, this is the industry that we're in right now. It isn't built around us. We don't know how to function this way and it's no one's fault. It's just sort of that's the way we've all sort of learn to work in this industry and I suspect we're not the only industry like this but we need some kind of guidelines to do it. TARAS: Maybe it's a difficult question to answer. It varies probably from team-to-team. JACOB: Yeah. CHARLES: Yeah and maybe, we can kind of shift the question just a little bit because I have one that sits alongside not quite the same question but I think related and can bridge it and maybe we can find the answer there. We've talked about a little bit and there's definitely more to unpack there, how to give feedback that's kind and honest and I think to... What was the third? JACOB: Inspiring. CHARLES: Inspiring, that's right. What about from the flip side? This is something that actually comes up with us as consultants but I think it's something that other people will encounter in their jobs too, is what do you do when you are the creator and you're trying to present or share and ultimately solicit feedback from a stakeholder, the CEO of your company, one of your clients, one of your customers and you see them engaging in kind of a mode of feedback that's less than constructive. They're nitpicking on things that aren't really important and maybe, this could be completely and totally inadvertent. Their intention could be that they're trying to help you out but it's really not being helping at all and it's kind of like either tearing you down or just not being productive and it makes you feel... What's the word I'm looking for? Just brings an air of contention to the conversation that's not really helpful to producing the best result for everybody involved. How do you, as the creator actually engage with those people in a positive way and kind of help establish the guardrails and be that agent of change when those guardrails don't exist in their minds yet? JACOB: Yeah, how do you do it, right? It's probably rare that there's going to be an environment where someone's going to say, "We're going to do this framework for feedback," and everyone are onboard from Day 1. I think one of the things that you're probably getting at is the frustration that happens when people start giving feedback. You have this perception that they're giving you feedback that's unuseful and they're only giving it because that's the only thing they can think of or -- CHARLES: Exactly. They got to say something, they need to contribute their two cents to the conversation and some people are trying to be helpful and just aren't. Some people are just trying to appear smart. JACOB: Yeah and it's like, "Oh, boy. They're just really off." CHARLES: But you can derail the main narrative that you're trying to establish and get off in the weeds, kind of skirmishing with these people and after that happens, you're like, "Wait, no. I don't want to end up over there. I was trying to tell a story." JACOB: When I gave this talk once, one idea that someone threw out was when you make a PR, mark up your own code first with everything that you want to draw people's attention to. I think you were getting at this as like, "One frustrating thing is when people getting feedback seem to have less invested than you do." They sort of just flying by and dumping on you and they probably, actually couldn't care less and that can be frustrating. When you're engaging with people that maybe don't know how to get deeply involved in it yet or maybe don't have the time to, you can sort of gently nudge them to sort of like, "I want to talk about this part of it." It's almost like you're giving the answers to the test, which was like, "If you want to be part of the smart conversation, comment over here," and I think from the responders perspective, just speaking personally, I would appreciate that. It gives me an opportunity to feel like I'm actually being useful. We can all probably point to times where all the code review we have to do feels like homework that isn't the best use for our time. It's like, maybe the responder can make us feel like our opinions matter and they will be put to good use when you tell them, "If you comment here, it will probably be put to good use." I think the sort of the way to get started is the responder can just say, "I would really like feedback over here," and I can't speak for everybody but I would suspect that more people than you think would be more than happy to be gently guided in what feedback they should get. CHARLES: Right. I'm thinking how you would do this, for example in the context of a demo that you're giving to stakeholders because there's maybe the inkling of the idea to do that but it's often presented as an apology like, "This screen doesn't work," or, "Your handling isn't quite right. Sorry we're going to fix that." Look at the stuff that's over here and maybe, the way to frame that is a really hard problem based on the legacy architecture that we have for displaying errors and I'm not quite sure how to do it in a robust way and I could really use some feedback there. It's an area of exploration or something that we really need to focus on. You put that out there as I'm demoing this main functionality right now. JACOB: Yes. You sort of say like, "Here's something to watch out for and please, let us know what you think," and then after you could say like, "As a team, we thought up these three possible solutions but we didn't want to move on them because we wanted to get your feedback first, so please impart on us what is the right way to do it." You know, flattery goes a long way. TARAS: I have this imagery coming up in my mind as I'm hearing you guys talk about this that I keep seeing this kind of difference between a line and a triangle, where a line is kind of a tug and pull between, "Did I do this right? No, I didn't do this right." I think those are kind of a bad code review because it's very personal. It's not really where you kind of want to go and then the other way is like a triangle where the end goal that you're trying to get to is somewhere that is beyond both places where you let two people: the recipient of the code review and the giver of the feedback, the end goal that you want to get to is actually someplace else. When we deliver the code and we say, "My code is done," then it invites this kind of linear feedback where it's like, "No, you didn't do it right," but in the other way, "This is where I got to so far. Tell me where you think we could take it next. If you don't think we need to change anything, then we're done. We can merge this." JACOB: Yes. The very intelligent Jessica Kerr said recently on another podcast, there's no such thing as done but you can always make it better. I think you're getting at that point. That's a great question, by the way to ask of your responders -- where should we go next? Where do you see this going next? What will make it better? What would make it more robust? What would make us happier six months down the line when we're looking back on this? But I think of it as the firing range. That's the analogy I gave where it was like, "You put a pull request out and you assert that it is done and if no one can find fault with it, then it's done," and I just don't think that that makes sense. I don't think, really most people actually want to work that way. It's maybe not necessarily even healthy but I think everyone can find, at least I hope, a process that invites them to come into the process and share the way they see things. It can hopefully be enriching and just make everyone feel better along the way. TARAS: This sounds to me like the inspiration part of this conversation because one thing I really like about working at Frontside, I'm an to experienced developer but I know that Frontside is dedicated to doing something much greater than I can do as an individual. When I ask, "What do you think about this?" then I know the feedback is going to be because there something always that is just beyond the reach that could be a little bit better than what we have right now and it's not until I can actually get the feedback from Charles, from Jeffrey and hear, "What do you guys think? Is this it?" and they're like, "What about this?" and I know that the next step is going to be a little bit or maybe even a lot better than what I have right now. I think that's the part that inspires me. I know I have actually gone a little bit further beyond my personal ability to get us to that vision of like this being much better than we could do as individuals. JACOB: Yes and as opposed, "Oh, I screw it up and now, I have to fix it." The framework of find all the errors that I made, even though there may be errors. Let's be honest, there could be things that would be really bad, that would be a security problem but for a growth mindset, to think about it, it's like, "This is a way to make it better." CHARLES: One of the things that you can do to help that is try and find inside every change, try and see the potential that hasn't yet been realized but is enabled by this change, like what exciting paths does this change unlock in your mind, right? And then you can share that. That's a great way to 'inspiration is infectious,' so if you can find inspiration to change, then you might be able to share that with the creators. If something is very personally exciting to you when you see something, maybe spend a little time searching for what you find to be exciting about it. JACOB: Yeah, nice. That's great. CHARLES: Yeah, we might have touched on that. I just bring that up because so often, I'll see the changes that people on my team are submitting and sometimes, it can feel like a cloud burst of like, "We could do this or we could do this or we could do this," and it's great to get excited. JACOB: Yeah. I can share -- recently, there was an example. My coworker opened a pull request and it unlocked something for me where I said like, "You know what? This is making me think about this other thing that I've always hated doing with our legacy codebase. We should do that thing you're doing more because boy, would it make life easier?" and I wonder if it would be worth the time in refactoring X, Y, Z because then I would never ask you A, B, C again and I would be so much happier. TARAS: It makes me think that we need to revisit our pull request template like what would that like? What would be the sections on a pull request template that would facilitate this kind of way? CHARLES: I don't know. I know we're almost a time but before we even get into that, this is a question I have because we talk about pull requests templates, how do you match the amount of process with the scope of the change? Because it's probably a little bit heavy weight if I want to fix a typo and say, "Now we're going to have the creators lay out their case that they're going to ask questions of the reviewers and then the reviewers are going to ask questions and once everybody's been able to write down things that they observe the questions that they have, completely and totally divorced from opinion, now we can talk about opinion that is welcomed." If you're capitalizing one letter, that's probably a little bit heavy weight but on the flip side, it's probably absolutely warranted if it's a major feature that's going to be affecting a huge portions of your revenue stream and so, this is one of the problems I have with pull requests templates is they are one-size, fits-all. Sometimes, a pull request template feels like it's supporting you and sometimes, it feels like the epitome of busy work and I have to spend all this time deleting the sections for the pull requests template. I wish there were ways you could choose different pull requests templates. Maybe, there are. JACOB: So do I. CHARLES: That's a little bit of a quibble that I have is the amount of ritual and ceremony that you have to go through is fixed with a pull request template but it seems like you want to match the amount of process to the scope of the change. JACOB: Yeah, you do and the flipside is also joy. You don't want to give someone too little homework when you really needed the same work. Maybe there's a way to say, when a pull request is opened, the person who opened it or the manager or maybe someone else, has the option to sort of tag the pull request as saying, "This is going to be a bigger conversation. We cannot merge it until we have a face-to-face conversation first with these various stakeholders." Maybe, one of the questions, the first form that everyone gets for every PR gets is what level of PR is this? Is this a quick change? And what people that you just need some quick feedback on? Or is this the long form that you need where there needs to be a sit down with these people? TARAS: Actually, I was thinking what's the adjustment that could be made for pull request is to say, "Here's what a complete pull request looks like. You can opt into whatever section you want," so you decide based on your pull request, what the actual sections of this template are appropriate but then someone can, on the flip side say, "Look, I would really like to learn about the motivations of this pull request. Can you add motivation section?" and understand that from your pull requests. JACOB: Absolutely. You filled out Section A, please answer Section B and C. Yeah, cool. TARAS: Yeah. Because I think what part of the challenge is that we have people look to us, especially people who have a lot of experience, or developers look to us to know what is the right thing to do sometimes or often and I think it's helpful to be a little bit more gentle and saying, "You can decide what is the right amount of detail to set the right conditions for this code review but I will give you some directions for what are things you can consider in including. JACOB: Absolutely. CHARLES: All right. We're a little bit over time, so we should probably go ahead and wrap it up. You are absolutely right, Jacob. This is a topic that keeps on giving. We didn't really move much beyond the pull request and feedback and stuff but we moved a lot around within that topic because it's a big one. Jacob, is there anything that we should mention? Any upcoming talks, meetups? JACOB: No, I have a one-year old and it's all about work and family in this part of the life, so I have nothing to speak of at this point. You can come find me on Twitter as JStoebel. CHARLES: Okay, awesome. Well, thank you so much, Jacob for coming and talking to us. This is a very rich topic. We only really scratched the surface. That's it for our episode today and we'll see you next time. Thank you for listening. If you or someone you know has something to say about building user interfaces that simply must be heard, please get in touch with us. We can be found on Twitter at @TheFrontside or over just plain old email at Contact@Frontside.io Thanks and see you next time.
Sponsors Sentry use the code “devchat” for 2 months free on Sentry small plan Clubhouse CacheFly Host: Charles Max Wood Special Guest: Charles Lowell Episode Summary In this episode of My Ruby Story, Charles hosts Charles Lowell, founder and developer at The Frontside Software based in Austin, TX. Listen to Charles on the podcast JavaScript Jabber on this episode. Links JavaScript Jabber 337: Microstates.js – Composable State Primitives for JavaScript with Charles Lowell & Taras Mankovski Charles Lowell’s Twitter Charles Lowell’s GitHub Charles Lowell’s Frontside Bio https://devchat.tv/my-javascript-story/ https://www.facebook.com/DevChattv Picks Charles Lowell: Yousician App Charles Max Wood: Parade of Homes - St. George, Utah Vrbo.com
Sponsors Sentry use the code “devchat” for 2 months free on Sentry small plan Clubhouse CacheFly Host: Charles Max Wood Special Guest: Charles Lowell Episode Summary In this episode of My Ruby Story, Charles hosts Charles Lowell, founder and developer at The Frontside Software based in Austin, TX. Listen to Charles on the podcast JavaScript Jabber on this episode. Links JavaScript Jabber 337: Microstates.js – Composable State Primitives for JavaScript with Charles Lowell & Taras Mankovski Charles Lowell’s Twitter Charles Lowell’s GitHub Charles Lowell’s Frontside Bio https://devchat.tv/my-javascript-story/ https://www.facebook.com/DevChattv Picks Charles Lowell: Yousician App Charles Max Wood: Parade of Homes - St. George, Utah Vrbo.com
Sponsors Sentry use the code “devchat” for 2 months free on Sentry small plan Clubhouse CacheFly Host: Charles Max Wood Special Guest: Charles Lowell Episode Summary In this episode of My Ruby Story, Charles hosts Charles Lowell, founder and developer at The Frontside Software based in Austin, TX. Listen to Charles on the podcast JavaScript Jabber on this episode. Links JavaScript Jabber 337: Microstates.js – Composable State Primitives for JavaScript with Charles Lowell & Taras Mankovski Charles Lowell’s Twitter Charles Lowell’s GitHub Charles Lowell’s Frontside Bio https://devchat.tv/my-javascript-story/ https://www.facebook.com/DevChattv Picks Charles Lowell: Yousician App Charles Max Wood: Parade of Homes - St. George, Utah Vrbo.com
In this internal episode, Charles and Wil talk about testing issues and BigTest solutions. Pieces of the testing story are discussed, such as the start and launch application, component setup and teardown, interacting with the application and component, convergent assertions, and network. Then they talk about testing issues: the fact that cross browser and device-simulated browsers are not good enough, maintainability and when and when not to DRY (RYE), slowness and why (acceptance) testing is slow, portability and why tests are coupled to the framework, and reliability. Finally, they talk about BigTest solutions: @bigtest/cli to start / launch (Karma recommended for now) @bigtest/react, @bigtest/vue, etc for setup & teardown @bigtest/interactor for interactions @bigtest/convergence for assertions @bigtest/network in the future (Mirage recommended for now) Resources: Justin Searls – Please don't mock me This show was produced by Mandy Moore, aka @therubyrep of DevReps, LLC. Transcript: CHARLES: Hello, everybody and welcome to The Frontside Podcast, Episode 115. My name is Charles Lowell, this episode's host and a developer here at the Frontside. With me today to talk some shop is Mr Wil Wilsman. WIL: Hello. CHARLES: Hello, Wil. WIL: How's it going? CHARLES: It's going good. I'm actually pretty excited to get to jump into this topic because we're going to be talking about some of the big things that are happening at Frontside and some of the things that we've been developing in almost for the last year. WIL: Yeah. It's been about a year now. CHARLES: It's been about a year and we've talked about it in various podcast but we're going to be talking about it again because there's just been so much progress that we've made, I think in a lot of clarity in kind of what we're going for here when we talk about BigTest and testing big and how we want to roll out the BigTest framework. We just have a lot more experience using it on a number of different projects, so we get to talk about that today. Before we get started, I just wanted to talk a little bit about what BigTest is, both in terms of the framework and also the philosophy. Wil, you're the one who works the most on BigTest. When you think about philosophically, what does BigTest mean to you? WIL: It's the size of your test, not a physical size like size and storage but how much your task actually does. The test itself can be very small as our test are but it tests the whole application from the user interacting with it down to the network requests. That's the definition of the philosophy of a BigTest to me. It's to tests your application from the biggest point of view. CHARLES: Actually, achieving that can be surprisingly difficult, especially in a frontend JavaScript application and there are a lot of solutions out there for testing and we've talked about them. One of the questions that arises is when we talk about BigTest, what exactly are we talking about? Are we talking about a product that you can download and install? Are we talking about the philosophy that you just outlined? Or are we talking about the individual pieces of software that make that philosophy real? I think the answer is we're kind of talking about all three but we want to take this episode to talk about where we're going with the product. What we've identified is the subcomponent pieces of that product. In other words, in order to get started testing big, what are the things that you need to think about? What are the things that you need to do? And then what are the component pieces? Because one of the things that I think is very important to us is that you be able to arrive at wherever you are in your project, whatever framework you are using, whatever current testing solution and be able to begin using BigTest. That means, you might be using some of it or you might be using a lot of it but we want to meet you exactly where you are, so that you can then, get onboarded and start testing big. WIL: Yeah. Definitely an important distinction that we get confusion about is what is BigTests and people just assume like this whole test suite is BigTest but we used the parts of it ourselves like we use Mocha, which is not part of BigTest. We use Chai, which is not part of BigTest. We use Mirage which is kind of part of BigTest but definitely it originate in BigTest and Karma and things like that. BigTest isn't your testing suite. It's not one thing to go-to to grab, to start writing tests. It is a small pieces that you can use in conjunction with other small pieces, just to make it really easy and flexible to test your application. CHARLES: Exactly. Because it turns out that there's a lot going on in the application. Maybe we should talk about what some of those pieces are that you might want to start using BigTest with or that you might need to test big, I guess I should say. What's a good place to start? Let's start with talking about some of the issues that you want to do when your testing big. Then we can talk about what pieces of the testing story that fit in to solve those issues. One of them is you need to test that your application works, like actually works. That means you need to be able to test on a multiplicity of browsers, for example. We're limiting to the domain of web applications. There are actually a shockingly large number of browsers. It's not just Chrome. It's not just Safari. There's Mobile Chrome, Mobile Safari, which are subtly different. There's Edge and I'm sure the Mobile Edge is slightly different too, so you want to be able to test cross browser, right? WIL: Yeah, absolutely and things like Nightmare and JS DOM and things that simulated browsers, we don't necessarily think those are the best tools for writing BigTest because we want to ensure that those browser quirks are caught and tested as well. CHARLES: This is not theoretical like sometimes you'll have a syntax, like the parser is slightly different and you have something that throws a syntax error in Safari or in the Internet Explorer and your whole app is completely busted. If you just take in the time, just even trying to load the app in that browser, you would have caught that. That's what I've been on many times. WIL: Yeah and what I just saw came up yesterday, which comes up frequently is not closing your CSS Selector and Chrome doesn't really care like web to browsers don't care too much but that will fail in Edge and depending on what you're missing, the failing is part of that too but mostly, Firefox and Chrome don't care about that kind of thing. CHARLES: Right. It seems like the majority of testing solutions are kind of focused around Headless Chrome or some variation of Electron. That entire class of really dumb errors has already been caught. Like I said, to actually catch it, it takes less than a millisecond of CPU time just to load it onto the browser and see that thing doesn't work. Unfortunately, they can be catastrophic errors but the problem is how do you actually do. We want to test like cross browser. This is something that we want to do. For me, I just can't imagine shipping an application without having some form of cross browser testing, some capability of being able to say, "I want to test it," like, "We want to work on these eight browsers and so we're going to test it on these eight browsers," but how do you actually go about doing that? WIL: Right now, we are working on the BigTest CLI which will help us launch browsers but that's not complete yet. It has some bugs on. For the meantime we've been using Karma, which is great. Basically, you just have this service that's able to find the browser binary on the system and just launch them pointing to local hosts with your app loaded up and your normal development server take care of loading the test up and running the test. Karma and the BigTest CLI is just there to capture output and launch those separate browsers. CHARLES: Yeah. I remember when I was first using working with Karma and I think Testim is another tool that's in this space. There's Testim, Karma and BigTest actually is we're developing a launcher because launching is something that you're going to need but it's such a weird problem. I feel like with the browser launchers, there's three levels of inversion of control because you're starting a server that then starts another process, which then calls back to your server, which then loads the app resources, which then loads the tests and then runs the test. There's a lot of sleight of hand that has to happen and – WIL: Including injecting the adapter that you use, like the Mocha adapter, the Jasmine adapter that ends up reporting back to the CLI. That's something that Karma and Testim and BigTest will handle for you. CHARLES: Right, so you're fanning out the test suite to a suite of browsers then collecting the results but basically, you need some sort of agent living inside the browser that's going to act on behalf of the test suite, to collect the results. I remember when I first came into contact with Karma and Testim, I was like, "This is so unnecessarily complex," but then, having used it for a while and I think there are some complexity that can be removed but if you want to do cross browser testing, that kind of level of ping-ponging is there's a certain amount of it that just necessary. It's something that's actually quite complex that you need to have in your stack, in your toolbox, if you want to truly test big. WIL: Yeah and all the solutions is mechanisms for detecting when the browser has launched and restarting the browser based on its health check, etcetera and things like that that you wouldn't think of actually loading up a browser but you need to think of when you're doing automated testing. CHARLES: What is it that sets apart, for example the launcher solution? We kind of call this class of solutions launchers, so Testim, Karma, the BigTest CLI. What is it that sets BigTest CLI apart from say, Karma and Testim? WIL: We're trying to be as minimal config as possible and just really easy to get started and going. Karma has a lot of plugins that you need to make sure you have installed and loaded in the options set for those plugins. Testim has some stuff bundled but it still requires this big config bulk at the beginning that you need to passing or that's all what you were doing. We're trying to avoid that with BigTest CLI and one of the ways that we're able to avoid that is by just letting your Bundler handle bunding the test. In Karma, you need Karma webpack or something. Testim has some stuff that it needs and really, we just want like in-testing mode. When you're in the testing environment, just change your index to point your tests, instead of your application and your Bundler will do all the work and we just serve that file and collect the results. CHARLES: Right, so it doesn't matter if you're using Parcel or you're using webpack or you're using Ember CI. WIL: Yeah, Rollup even. CHARLES: Or even just like low level Broccoli or Gulp or whatever. There's a preponderance of bundling solutions and that was always something that was just a huge pain in the butt with Karma. I know it's like just getting to the point where my tests are loaded and you look with Testim, most of my experience come with Testim comes through how it's used in Ember CLI like the histrionics that undertaken just to bundle all your tests assets and your application assets and your vendor assets and just kind of bootstrap that thing. It's a lot of work. WIL: Another thing with BigTest CLI that doesn't include in Karma and Testim does is a concept of a watcher because all these Bundlers, you have HMR -- hot module reloading, Rollup and things like that. It come with plenty of plugins. Parcel is always set out of the box, so if you're using your Bundler, your existing Bundler to bundle your test, you get that watch feature for free, so it's another complexity that the BigTest CLI kind of eliminates. CHARLES: What it means is we've hidden most of that complexity. Just let the Bundler handle it, right? The Bundler is the part of your project that bundles. WIL: Yeah. CHARLES: You should have your launcher actually doing that for you but we still do need to have some way to do that set up and tear down. When we have that testing endpoint, we have some way to say, "We're starting a test, not the application. We're ending the test, tear it down," so how do you abstract that away? WIL: That's kind of something that we can't really avoid. It is just like some sort of dependency on the framework itself, your application framework. It's like you need to mount a React app. You need to mount an Ember app, etcetera and there's different ways to mount those things. This is one of the things that can't really be decoupled as much as everything else can but BigTest has BigTest React and BigTest Vue and we want to eventually gets like BigTest Ember but really, the main export of all these packages is just a simple mount helper that will mount and clean up your application for you and your testing hooks, whether you're using before each from Mocha or before from something else like Jasmine. You know, no matter what you're doing, you just have a hook that mounts your application and then, cleans it up on the next mount. CHARLES: It's worth pointing out here that this is kind of a core concern of testing and testing big is being able to mount your application and tear it down with regularity and having hooks into that process. Whether you're using BigTest or whether you're not, can you still use BigTest React and BigTest Vue, even if you weren't using anything else? WIL: Yeah, absolutely. Like I said, they just export simple mount helpers. I don't even think they have any other inner BigTest dependencies. They just have pure dependencies on their frameworks. CHARLES: Right and so, you could use it, even if you wanted to roll everything else by hand or you wanted to get started somehow and you needed to do set up and tear down, again this is something that's key to being able to test big, so you should be able to use it independently, whether you use the CLI or not, whether you're using any of the other tools or not. All of the tools can be used independently. WIL: Then another feature of the BigTest React and BigTest Vue is the tear down that happens before set up, rather than happening after your test runs, having a separate tear down. This allows it. Whether your test passes or fails, you can look at it and play with it and inspect it and debug it much easier than if you had tear down. You have to disable at tear down or throw a pause in there to keep other or something. CHARLES: Yeah, I love that. When something goes wrong, you can just let the test case run and the last test that it runs, it just leaves at set up. It does the tear down right before the set up. WIL: Exactly, yeah. At the very end of the whole test run, there's an app there waiting for you to play with. CHARLES: If you focus in on a single test, we most commonly use Mocha, so you say a '.only' to run that single focus test, then you have the state of the application at that test case set up and ready to go. You can just play with it, you can inspect it, you can actually just use it as a starting off point and interact with the app normally as you would. WIL: I want to say, Cypress does this too. They do their tear down before they're set up as well. That's how you're able to play with Cypress test. CHARLES: Yeah, I like that trick. Now, we talked about launching, setup and tear down but we haven't actually talked about much of what actually happens in the test cases themselves. We talked about how to start and launch your test suite, how to do that across a bunch of different browsers, how inside of that, you have a separate concern as applications set up and tear down and how you want to lean on how you're actual app is actually bundled because that fits in with the philosophy of testing big. You don't want to use an external Bundler for your test suite. You want to use your real Bundler, how the asset is actually going to look. But when it comes down to actually writing the tests, you need to be able to interact with at the highest level as you possibly can. When I say highest level, we want to verify that the users, when they take certain actions, we'll see certain outcomes and so, we want those outcomes and we already talked about this to be reflected in a real DOM, in a real browser. But at the same time, the real interactions, we want those to be as high fidelity as possible, so you want to be sending events to the browser. You want real mount events, real key events, real interactions. WIL: Yeah, interacting with application. That's another core philosophy that we kind of talked about earlier that defines a BigTest. It's the user interacting with your application. We're not calling methods and expecting other callbacks or arguments to be passed or clicking on a button and expecting a message to pop up that says, "Form submitted successfully." These are user-facing things were starting on and acting on. CHARLES: Yeah and then, it can be really tricky because these things don't happen synchronously. They're happening inside of your browser's event loop. I click that button and then it goes off and there's some loading state and then, I might get an error message that pops up this thing that animates out and then, goes away. The state of the browser is in constant flux. It's constantly changing and so, it can be very difficult to put your finger and say, I want to be in this state if you are limiting yourself to only reading from the DOM. Some frameworks, Ember for example, you have kind of a white box where you can actually inspect the state of the Ember run loop and use that to do some synchronization but it can be very, very hard to coordinate these interactions. WIL: Yeah. You know, to talk about getting to the solution as a BigTest interactor, which is basically modern page components or page objects. If you ever heard of page objects, it's just a way to encapsulate interacting with big pieces of your pages. It's not a new concept. It's been around for a while but BigTest interactor has kind of a new twist on it where they're immutable, composable interactions that are also convergent, which we'll get into later, which basically means if your buttons not there, it won't click the button until it is there. They're really powerful and they're making really easy and fun to write these tests. CHARLES: Yeah, they're super powerful. I remember we talked about convergences last time when we talked about BigTest but interactors, I think are definitely a new development. I think we should spend a little bit of time there talking about, not just the power but also the ergonomics of interactors because they are like page components or page objects, except they're scope to the component. Not only do they have all this wonderful stuff where it'll make sure that the component exists before it starts to interact with it and things like that but their composable. If I have a button, then there are certain operations that are valid for that button. I can click it. I can hover over it. I can do all these things. They're the operations that make it unique to the button. Now, those might actually map to real events. WIL: Similarly, their assertions about that button as well, like as primary is secondary. If this button is repeated throughout your application, you might want to make sure that your form has a primary and secondary button. CHARLES: Exactly. It really encapsulates all the knowledge of how you can interact with both in terms of taking action and reading state from that button. It almost feels like an accessibility API. It would be easy to write a screen reader if you had these interactors for every single component on the page. WIL: That's kind of what it is. It's just like you're defining an API around how your user would interact with your application and what your user would expect in the application. That's the point of page objects and interactors as you're defining this user API, essentially. CHARLES: Yeah and so, really the step that interactor take is that they take the classic page object and it make them composable, so I can have, you kind of touched on this before, a modal dialogue interactor, which is composed out of two button interactors. One for the primary action, one for the secondary action and maybe, it's aware of its own title text, so you can assert on the title text but I didn't actually have to write the individual button interactors for that modal dialog interactor. Then I might have a second modal dialog interactor or a form that's on a modal dialog just composed of the modal dialog interactors and the individual form components, which appear on that particular modal dialog. WIL: It's essentially how we've been building applications lately with components but this is for page objects in your test if you want to mirror that. You don't have to have one-to-one mappings of an interactor to a component but if you do, it's really powerful. CHARLES: Yeah. I found that when we have one-to-one interactors, that's when it just feels the best. WIL: Yeah and on top of this, if you have a component library and your component library exports the interactor that it uses for the component test, like we said, this BigTest technology, they're sprinkled also. We don't have to use interactors in big acceptance tests. We can use them for smaller component tests too, so if we ship these component interactors with the component library, your application that's consuming this component library now can test those components for free, without having to write their own interactors. It can just compose the interactors exported by the library. CHARLES: Man, I almost want you to repeat that word for word again, just so it can sink in. It's so awesome. Because when you actually go to write your tests, you're not starting from ground zero like, "How do I do this?" They're like, "I'm writing some tests for this thing and I'm using these components and so, I've already got the prepackaged interactions for those components." It's like you start writing your tests. If your tests are a 10-story building, it's like you're starting on Floor 7 and you only have to walk up to Floor 10, instead of slogging up all 10 stories. WIL: One really helpful interactor that we work within the open source stuff we've been working on is a date-picker interactor because date-pickers can be really complex. Just having that common interactor and have a date-picker on multiple forms where we can just use that one interactor, we don't have to tell every single test how to interact with that date-picker. We just say pick date and pass the date. CHARLES: Yeah, it's so awesome. That is actually a great example. It doesn't feel scary to write a test for a page that has a date-picker on it or two. If you're doing like a date range or something like that, you're like, "Oh, my God. I don't write the selectors to test this." You just import your date-picker interactor, you set the date, it actually worries about all the low level events and there you go. It feels like you're operating at a much higher level. WIL: Yeah. The interactor API essentially, you're telling me the test what the user would be doing and what the user would be seeing. CHARLES: Yeah. It's worth pointing out again. We've identified starting and launching. We've identified set up and tear down but interaction is a core concern of BigTesting, no matter what tool you're using. One of the things that we found as interactors are something that you can sprinkle on literally any test suite if you're testing an interface and it makes it better. We've used it inside big acceptance tests. We use it inside Jest, doing just little component tests. There are people in the BigTest community who have used it to basically, write component tests against a JS DOM and while theoretically, philosophically, you want to make those tests as big as you possibly can, you can use that piece in your test suite. If you are using a simulated DOM and if you're running a node in a browser, these interactors will still work and you're going to get high fidelity test cases that are resilient to this asynchrony and are composable and if they do have a full-fledged test suite, you can reuse these interactors. They are a really awesome power up that you can bring into your test suite. WIL: And they are not tied to the framework at all. We use them in React for our stuff but we've also written some in Ember. Robert's written some in the Vue and ported some test and one of the beautiful things we've seen from this is that one interactor goes everywhere. You just write the interactor once and you can use it in Ember, in React, in Vue, in those test suites. If the rest of your test suite is framework agnostic, you have this test suite that you can jump frameworks in your test suites until it works and can test your application with high fidelity. CHARLES: Yeah, it's fantastic. I remember when we first tried using interactors inside an Ember test suite because Ember comes with like a big kitchen sink in testing set up but interactors just slotted right in and there's absolutely no issue. WIL: Yeah and there is actually a speed boost even because in most of the Ember test offers a hook into the Ember run loop and interactors are not. There is actually a good speed boos just using interactors. CHARLES: Yeah. This is a good point. It's a good segue because typically, we think of acceptance tests as being really slow and one of the reasons, even the people [inaudible] acceptance tests or testing big as they think like it's going to take a long time. We found that actually we've been able to maintain a happy medium of testing big but also, having those test be really, really fast. When you say you said a speed boost from using interactors with Ember, where is that speed boost actually come from? WIL: I mentioned the Ember test offers a hook into the Ember run loop and interactors aren't and the reason of this is because interactors are converging and they wait for things in the DOM to exist before interacting with them. Instead of waiting for the framework to settle, it just waits for the thing to appear and then interacts with it immediately. If you're starting something about a button toward the top of the page, you don't really care that another button at the bottom of the page has rendered yet, unless of course you have assertion about that but if they're converging, you don't need to hook into the wrong loop to wait for the entire page to load, to interact with just one piece of it. CHARLES: Right. You're just waiting and you say, "I'm expecting something to happen and the moment I detect it, no matter what else is going on, the page could be taking 30 seconds to load but if that button appears and I can interact with it, I can take my action then or I can make my assertion then." It's about kind of removing gates -- artificial gates. WIL: Yeah. Another common thing that's helped with is animations as most test that are hooked into the run loop, you kind of have to wait for some of these animations to finish before you can even interact with the element and that means if a model has a half second animation where it flies in and you have 30 tests around this modal, those tests are extremely slow now because you have to wait for that modal to come in, whereas -- CHARLES: -- Straight up flaky. WIL: Yeah, straight up flaky. Whereas in the actual DOM, that modal is inserted pretty immediately and can be interacted with pretty immediately. With interactors, they don't need to wait for the animation to finish. They can just immediately interact with that modal but of course, if you need to wait for the animation to finish, there are options for that as well. CHARLES: Yeah. If there's some fade in that needs to happen, you can kind of assert on any state and as long as it's achieved at some point, the interactor will recognize it and recognize it at the soonest possible time that it possibly could. I remember getting bitten on one project where the modal animations in particular were so brutal. Not only were they flaky, they just were slow because there was all these manual time outs. It wasn't even a paper cut. It was kind of like a knife cut, like there's someone sitting there and kind of slashing you with a pocket knife. It just was a constant source of pain in your side. WIL: Yeah and that's how you end up with things like waits and sleeps in your test suite. When you need to wait for the animation to happen or something, you just see a sleep for four seconds with a comment because we have to wait for the components to load in. That's kind of a code now. CHARLES: Yeah, that's just asking for trouble both in terms of slowness and in terms of it's going to get flaky again. That has been kind of one the most freeing things about working with interactors and working with convergent assertions on which they're based is you just don't ever have to worry about asynchrony. Really, really truly, most of the time, you're writing your tests, like it's all synchronous and that kind of makes sense because from the user's perspective, their consciousness is synchronous and they don't care about the internal run loop. It's just they were making observations in serial and at some point, they're going to observe something, so the interactor sits at that point and really observes the application the way that your user would. WIL: Yeah. We've mentioned a few times now the convergent assertions, which interactors are based on. A little caveat there if you're using interactors and you're making non-convergent assertions, they might fail or be flaky. That's because interactors wait for the thing to be there to interact with, so as soon as the buttons there, it clicks it but it doesn't wait for after that event has fired and your application has reacted to that event, that's your application is concerned. We need something there like our convergent assertions that can converge on that state and wait for that state to be true before it considers itself passing or in times out. CHARLES: Maybe we should dig a little bit into convergent assertions. I think the last time we had a public conversation on the podcast about this, this is kind of where we were, like we hadn't built the interactors, we hadn't built these other component pieces of the testing story. We were really focused on the convergent assertion. We've talked a little bit about this but I think it's worth rehashing a little bit because it's a unique way of approaching the system but it's also kind of horrifying when you see how it works under the covers. I think when we tell people about the fact that it's basically polling underneath the covers. The timeout is configurable but it's basically polling every 10 milliseconds to observe a state. I remember the first time being confronted with this idea and I was horrified and like my programmer hackles on the back of my neck, like raised up and I was like, "Wait a minute. This is going to be slow. It's going to be computationally intensive." WIL: Yeah. That was my exact thought too because this is going to be slow. If acceptance tests are slow and we're doing an acceptance test every 10 milliseconds, it's going to be really slow and that's actually not the case completely. It's actually the opposite. They're extremely fast. CHARLES: It is shockingly fast. You've got to try it to believe how fast it is, how fast you can run acceptance tests. WIL: Yeah, talking like 100 tests in just tens of seconds. CHARLES: Right. You basically gated by how fast your framework can render. Your tests are not part of the slowness. Your test -- WIL: And also, memory leaks can be costly too. We experience that recently where we had memory leaks that were slowing down our test but we fixed those up in test and put our backup. CHARLES: Yeah, because basically, running the assertion or running the convergence is very fast. It's just a very light ping. I kind of think of it is as it is light as the brush of a photon or something that was bouncing off of a surface, so that you can observe it. It's extremely light and most of the time, it's just waiting so the test and the convergence really just gets out of the way. Just because they can run a thousand times or a hundred times in a second, it's doesn't gun it up. But the thing is it means that your tests run as fast as your application will run. You get back to the point... Was it in React where the kind of the key insight is that JavaScript is not the bottleneck? Well, your tests are not the bottleneck. WIL: Yeah. CHARLES: I guess this is what it is. I don't know if there's anything else that you want to say about convergences. WIL: No. We pretty much summed it up there and that's what interactors are based on. That's how they're able to wait for things in a DOM. It basically polls the DOM until it exists and then it moves on and actually does the interaction. CHARLES: Once again, this is actually a very low level thing on which BigTest is based but this is once again, something that you can use independently. You can write your own convergent assertions. You can write your own convergences that honestly have nothing to do with testing your assertions. It's a free standing library that you can use in your test suite or elsewhere should you choose. WIL: That doesn't need to be a DOM for BigTest convergence there. I use BigTest convergence in BigTest CLI to converge on the browser being launched. Instead of waiting for the browser to report that, I can just kind of poll and see how that process is doing and the convergence waits for that process to start before moving on. CHARLES: Right. I guess the best way I've thought about it it's a way to synchronize on observations and not on callbacks. It's a synchronization mechanism and 99% of the synchronization mechanisms that we're used to, they've involved some sort of callback, a promise, an event-listener, things like that or even a generator where control is handed back explicitly to a piece of code when something happens. Whereas, this is a fundamentally different synchronization primitive, where you are writing synchronous code that's based on observations, so what I observe this, do this. When I observe this, do this. It's extremely robust. WIL: Yeah, very. CHARLES: It is a core piece. A fundamental thing that on which interactors are based on, which the CLI is based, I don't know if it's core to writing tests but -- WIL: It definitely helps. CHARLES: It doesn't helps. We couldn't have BigTest interactor without that. WIL: No, definitely not. CHARLES: Because that's what makes it fast, that's what makes it not flaky at all and having those things, I think it makes it easy to maintain because you can work at the interactor level or the level of user interaction and you don't have to worry about synchronization, so the flow of your tests are very natural. WIL: Yeah. We don't have to explicitly wait for request to be done for making an assertion about your app. That'll just come with convergences, just waiting for test date in application to true. CHARLES: Let's talk about one more piece of the testing issue because when you're testing big, when you're testing in the browser, there's always the issue of what are you going to do about your API. You got to have your API running. It's just always an issue and this is kind of interesting because this sits at the crossroads of testing big and also, getting the most utility out of your test because in an ideal world, if you're testing really big, you're going to be using a real API. You're not going to poke holes in reality. WIL: Yeah. One of the things that we avoid in BigTest is poking holes. We're not shallow mounting the components and testing the methods and the results. We're fully mounting these things and fully interacting with them through the full DOM API. CHARLES: Yeah, exactly, using real browsers. It just occurred to me the irony of us talking about reality being things that are still running inside of a computer processor. I think we've inherited this term from that talk that Justin Searls at AssertJS in 2017. It's a really, really excellent talk. I think he gave it at RubyConf. It's the 'Don't mock me.' WIL: Yeah, it's one of my favorite talks. CHARLES: Yeah, it's a great talk. In it, he talks about the value of a test is a balance of how many holes you poke in reality and sometimes, you encounter a test where all it is like holes in reality. Whether you're mocking this, you're mocking that, you're mocking the DOM, you're mocking the browser, you're mocking your network layer, you're mocking this external API and the more holes you poke, the less useful it's going to be. Network is one of those where it can be very difficult to not poke holes in that reality because it's a huge part of your application. Your frontend application is how it's going to interact with the server but at the same time, servers are gigantic pieces of software themselves, each with their own dependencies, each with their own set up and tear down -- WIL: Have their own concerns. CHARLES: Yeah, exactly. They might be in a different language. They've got runtime, things like they might need external C libraries and crazy stuff like that. They're their own beast. To get a true big end-to-end test, you going to have to stand up your server but the problem that presents is you want your tests to be also isolatable. If you're a developer, I can go to a repo, I can do an install of my dependencies and I can run the tests without having to do any external dependencies other than the repository and the language in which I'm working. This is one where we kind of have tried to walk the line of not wanting to poke holes in reality but also, have the test be containable to the actual application. In order to do that, you need something that presents a high fidelity version of the network. You can kind of try and have your cake and eat it too. You want to have something that acts like a server and really acts like a server but it's actually not a server. WIL: And still poke as few holes as possible and the application and how that's all set up, we don't want to be intercepting methods and responding with fake data. That's not a good way to mock that network. CHARLES: Right. We want to be calling actual fetches, calling actual XMLHttpRequest. Ideally, if you've got service workers, making actual service worker requests. WIL: Basically, as far as the application is concerned, it's talking to a real server on us. CHARLES: Yeah and that's kind of the litmus test for is it a hole in reality or is it just a really great illusion? WIL: Yeah and that's a good name for Mirage, right? It's a really great illusion. CHARLES: Yeah. It is a simulation of reality, so we use Mirage, which is something from the Ember testing world but something that we have extracted and made available as BigTest Mirage. WIL: Yeah. The main difference just being is that we've taken away the Ember dependencies and the run loop stuff. It's just plain JavaScript Mirage. It works exactly the same as you use it in Ember minus the auto imports and the file... Oh, man. I can't think of that word. Aside from automatically importing your files for your server config, you have to do that manually because Ember is what provides that but other than that, it's a form of Mirage. You define models and serializers and factories and all the good stuff. CHARLES: Right and then you can use those factories and you can use those models to really give a high fidelity server. If you are building something in whatever framework, you can use BigTest Mirage to simulate that network layer. Again, we've used it in a number of different scenarios but having that in place means that you're going to be able to have those high fidelity tests where your application is actually making XMLHttpRequest but it's all isolatable, so that it can be run in repo. This isn't really related to testing but it has a fantastic capability where you can prepopulate, you can use the factories to prepopulate your server with data, so that you can use the application without the actual server being implemented. WIL: Yeah. That's extremely powerful. That's we were talking about earlier and getting at is the scenarios which are setting up specific, essentially fixtures but you're generating these fixtures. Factories are essentially high level fixtures, network of fixtures. CHARLES: Yeah, higher order of fixtures. WIL: Yeah, so the scenarios are just setting up these fixtures for a scenario of your applications, like the backend is down or the list only responds with two items as opposed to 5000 items, something like that. You want to be able to, not only test these things but be able to develop against it and Mirage makes that really easy because you can just start your app with Mirage-enabled point to that scenario and you're there. You have that exhausted scenario to develop in. CHARLES: If you've never used Mirage, it is really hard to understand just how incredibly powerful it can be. We've used it now on at least four projects, where we did develop the entire first version of the product without any backend whatsoever. It's an incredible product development tool, even apart from testing, that then informs the shape of what the API was going to be. I know we've talked about this on the podcast before but it's really an incredible technology and it is available to you no matter what framework you're using. I think it's one of the best kept secrets in JavaScript development. WIL: Yeah. That's definitely great. That said, though it does have some fallacies. It's great but it can be a little slow sometimes, so we are eventually working on a BigTest network like another piece of the BigTest pie that you'll be able to sprinkle into your application but in the meantime, praise Mirage. CHARLES: Yeah. We are going to be offering an alternative or maybe collaborating for another version of Mirage but hopefully, we can make Mirage faster, we will be able to make this thing faster, so that it can use service workers and be used in a bunch of different scenarios. Just to recap, we've talked about a lot of different components but over the past year, a couple of years, these are the things that we've identified as being really key components as big part of your acceptance testing and really your testing stack. How you're going to start and launch these things? How are you going to set them up and tear them down? How are you going to interact with the application from a user, both in terms of making assertions and how are you going to take action on behalf of the user and still have it be maintainable, have it be resistant to flakiness, have it be performant? BigTest is the answer to that for those particular areas of the testing story and so, some were using we're using existing components like we use Karma, we use Mirage to date. Those, we did not develop but where we see kind of key pieces of that puzzle missing is where we started writing the BigTest solutions so things like the interactor. Eventually, we are going to make BigTest into a product that's you're going to be able to use kind of out of the box, just like you might install Cypress, where it's a very quick set up and we make all of the decisions about the components for you. But in the meantime, we're really trying to take our time, identify those pieces of the puzzle and build the software component that fits that piece of the puzzle at the absolute best so when they're polished, use them in a more comprehensive product. Things like convergence, things like interactor, things like BigTest React, BigTest Vue and very soon, BigTest Ember. These are things that you can use today, to make your tests just that much bigger and that much better, especially interactor. It's been an incredible journey this past year as we kind of develop these individual pieces and there's just going to be more goodness to come. WIL: Absolutely. Right now, I'm working on some validation type API for interactor that I'm hoping to land soon. That'll open up the possibilities of maybe hiding away those convergent assertions a bit more in your tests and just handling this automatically. It'll be pretty good. CHARLES: It's really exciting. Writing test has got more and more easy and more and more fun over the last year for us. I think we're already starting in a pretty good place. If you have any questions about BigTest, how would folks get in touch with us? WIL: We have a BigTest Gitter channel. You can find a link to that on the BigTest website: BigTestJS.io. Just ask us questions on Gitter and we'll try to answer them. CHARLES: And as always, you can ask us directly. You can send email to Contact@Frontside.io or reach out to us on Twitter at @TheFrontside or you can actually reach out to the BigTestJS Twitter account directly and just call us on Twitter at @BigTestJS. Thank you very much, Wil. WIL: Thank you, Charles.
Guest: Dillon Kearns: @dillontkearns | GitHub | Incremental Elm In this episode, Dillon Kearns joins the show to talk about techniques for experimentation with Elm, making those experiments safe, the concept of mob programming, why you would want to experiment with Elm in the first place, and how you too can begin to experiment with Elm. Resources: Grant Maki's talk on experimenting in your team "Types Without Borders" by Dillon Kearns @ Elm Conf 2018 Dillon's Elm GraphQL library How Elm Code Tends Towards Simplicity by Dillon Kearns The CSS as ByteCode Talk by Richard Feldman This show was produced by Mandy Moore, aka @therubyrep of DevReps, LLC. Transcript: CHARLES: Hello, everybody and welcome to The Frontside Podcast, Episode 114. My name is Charles Lowell. I'm a developer here at the Frontside. With me today as co-host on the show is David. Hello, David. DAVID: Hey, guys. CHARLES: David is also a developer here at Frontside and we are going to be talking about something that we've been talking, I guess a lot about recently and we're talking about Elm. I think we first started talking about this several years ago and then it kind of simmer down a little bit but recently, it's been top of tongue. With us to talk about Elm today is Dillon Kearns. Welcome Dillon. DILLON: Thank you so much for having me. CHARLES: I understand that you are a full time Elm consultant. You have a background as a Lean and Agile coach but have recently transitioned to doing Elm consulting full time. Now, what exactly does that mean in 2018 to be an Elm consultant. DILLON: Actually a lot of my motivation for getting into Elm consulting in the first place is I kind of realize that Elm to me is just an extension of the things that I was passionate about with Agile and software craftsmanship. I'm trying to help teams have a better experience with their code, make it more maintainable, make it easier to change, make it easier to drive things based on customer feedback and I really believe that Elm helps people do that. I used a lot of the background and experiences that I've had with Agile and Lean coaching and a lot of those same skills, in order to help organizations adopt Elm. One thing I've seen a lot of teams struggling with is trying out a lot of different frameworks. I've encountered teams that have spent months, very painfully trying out different frontend frameworks and having trouble coming to consensus about that. One of the things that I think really helps address that is having an experimental and iterative approach, that you can really use the scientific method to focus on learning, rather than getting it right the first time. I think that there's really a need to help teams through that process of introducing a new frontend framework like Elm, so that that's why I've gone into full time Elm consulting. CHARLES: That's an interesting process. It sounds like you really need to be constantly sending out spikes, doing research on whether it's Elm or some other technology to help you kind of bridge the chasm to the next generation. How do you actually do that as an organization? My guess, this kind of a question independent of Elm but maybe we can talk about how you see that play out in the context of Elm. DILLON: Right and actually, for any listeners interested in that question, I would really highly recommend Grant Maki's ElmConf talk from this year. He spoke about exactly that topic and it was at ElmConf that it's relevant whether your team is considering Elm or looking at other frameworks. I think that the key is you need to get good at experimenting in a way that's low risk and in a way that you can be constantly learning and seeing how these different technologies fit in your codebase and fit for your team. There's a quote that I really like from Woody Zuill. Have you guys heard of mob programming before? CHARLES: I heard of mob programming from a paper by Richard Garfield a long, long time ago, almost 20... I don't know if it's the same concept. DILLON: Yes. It gained a lot of momentum these days. Mob programming is essentially pair programming but with more people involved. I've really enjoyed that process actually. I think it's actually a great way to experiment with different technologies because you get all of the minds together and it's a very good way to kind of transfer knowledge and explore things together but Woody Zuill talks about mob programming and he likes to ask the question, "Why did we begin doing mob programming for the team at Hunter Industries that originally started mob programming?" People would give answers like, "Because it cuts out code review from the process because you have lots of eyeballs on it in real time," or, "Because it reduces bugs," or, "Because it gives you better quality code. It gives all the best ideas into the product in real time," and all those things are valid points that are really good benefits of mob programming. But he says those things may be true but actually, they're not why we tried mob programming. The reason we tried mob programming was because the team wanted to try it. That's a really important point. The team needs to be experimenting with things that they're passionate about and they need to be exploring things on their own terms. But with that said, another lesson from that story of kind of his team at Hunter Industries discovering mob programming is that the team didn't discover mob programming in a vacuum. Really, the team discovered mob programming because the team became really excellent at experimenting and evaluating those experiments and then, they like to talk about this phrase that Kent Beck coined, 'turn up the good.' When something is working well, we often focus on the negative things and trying to eliminate those things but what happens if we take the things that are working well and 'turn that dial up to 11.' CHARLES: Yeah, I love that. I remember in the kind of the original layout of extreme programming, talking about how I really just wanted to turn up all the things that were working for 11 or to 11, so testing, refactoring, incremental releases and things like that. DILLON: Exactly. CHARLES: I actually had one question that's maybe a little bit of a diversion. This is actually the first time I've heard of mob programming. It's definitely not the same sort of mob programming I learned about in Richard Garfield's paper. I think it was more referring to massively distributed open source in the form which is really kind of commonplace now that happens on GitHub. I think it's maybe, an obsolete definition of mob programming but how many people would be in a mob like two, three, four, five, six, seven, 10? DILLON: That's a great question. Really, the answer is of course, it depends. That's a consultant's favorite answer but it really does. My rule of thumb is I find it usually three people is a very nice size for a mob. I find that mobs tend towards around three or four people but that being said, it's important to note that mob programming is all about this idea that what is the true cost of programming. I think that often we look at programming as the act of writing code, initially and that's a very limited way of looking at coding. Because of course, 90% of our effort is spent maintaining code, making decisions around code, reproducing bugs, fixing bugs, communicating with customers about bugs -- bugs are extremely expensive -- the farther out they get, until eventually they get to the point of a customer discovering them, bugs are in extremely expensive part of software. If we can minimize bugs, that's very valuable. When you look at programming on this bigger scale and look at the bigger picture of programming, then you realize that you may be able to get one person to write the code faster but then, that person needs to code review it. That person needs to go and ask somebody question down the line when they don't have context because they weren't involved in the decision making. For example, maybe there's a UX person who doesn't have context on certain choices that were made, so there's a lot of churn, so you can kind of eliminate that churn by getting all the relevant people involved right away and that's the idea. In my experience with mob programming, it works really well to keep kind of a core of around three people. Sometimes, somebody goes up to have a conversation with somebody, take a break or answer somebody's question, maybe somebody from another team has a question that type of thing and so, the team can keep coding as a pair or whatever. But ultimately, the idea is that you get faster because you're building up this shared context and you're not spending as much time down the line answering questions, doing code reviews and things like that. CHARLES: Right. I see. DAVID: That kind of matches with my experience. Mob programming on previous teams, the way we had it set up is there was a regular mob programming chat session that the whole team was invited to but it was optional. You can just show up if you wanted to and really, that sort of made it so that there was a set of people who regularly attend -- three to five people in a session -- and they were the core group, essentially. DILLON: Right. That's another great point. Invitation is a powerful technique. If you're kind of mandating the people try an experiment or work in a certain way, ultimately it's much more powerful to let the team experiment on their own and follow their passion and they'll discover great things. It's about experimenting, rather than choosing specific experiments. While we're on this topic of kind of the real cost of coding, I think this is a good point to talk about this quality in Elm because, I think that this is one of the things that really motivates me to use Elm myself and introduce it to others is that, I think that Elm really get something about programming where there's a sort of superficial ease of certain techniques that Elm kind of goes beyond and says, "Actually, let's optimize for a different set of things that we think make code more maintainable and more delightful to work with in the long run." CHARLES: I wanted to also transition between, we were on a little diversions on mob programming but do you use mob programming as explicit technique for introducing Elm when a team is considering adopting it? DILLON: That's a fantastic question. I absolutely do. Of course, I honor the ways of working in a particular organization or team. I think that's important to do but I do strongly encourage using mobbing as a technique for knowledge sharing and when I'm on-site with a client, I find it extremely powerful as a technique for knowledge sharing and also, let's say you do an experiment, somebody is off in a corner and they're trying out Vue.js or they're trying out Elm or they're trying a particular coding technique. Then they come back to their team and they say, "Hey everybody, I tried this great thing," and now they have to spend this time convincing everybody and saying like, "Wait a minute. You didn't try this, you didn't try it that way. It wouldn't actually work in our context because of this." I think that it's very powerful to have everybody kind of involved in that process so that you can evaluate it together as a team. CHARLES: Because the thing is like, when you experience win or you experience fail, it's a very visceral feeling and that's the thing that sells you or turns you away. You can argue until you're blue in the face but words have a very limited capacity to convince, especially when compared with like physical and emotional feeling. It sounds like you can get everybody to have that shared experience, whether for the good or for the bad, you're going to arrive at a decision, orders of magnitude more quickly. They have to rely in conviction of that decision spread around the team. DILLON: Exactly. I think that hits the nail on the head and you say that we have this sort of skepticism of arguments from theoretical conversations, rather than 'show me the money,' but it's actually, try solving a real problem in this and that's exactly as it should be. I think that's one of the big antidote from this problem that I've seen in a lot of environments, where there's this analysis paralysis, especially with the state of the JavaScript ecosystem these days. I think that one of the keys to improving that situation is to get good at trying things, rather than theorizing about things. We have a tendency to want to theorize and when we do that, then we say, "Can it solve this problem? Can it solve this problem? Can it solve that problem?" You can talk about that until the cows come home but it doesn't get you anywhere and it doesn't really convince anybody of anything. The key is to find very small experiments and what I really recommend and what I'm dead focused on when I'm initially working with a client is getting something into production. Now, that doesn't mean that you need to have a road map for turning your entire application into Elm. In fact that's the whole point, is that you're not trying to do that. The point is you're trying to get as realistic of an experience as possible for what problems might occur if we do this? Will the team enjoy working with this language? Will it work well with our built pipeline? Will there be any unforeseen issues? You don't know until you actually try it, so you've got to try it and you've got to try it in tiny, tiny steps and low risk experiments. CHARLES: Right but you've got to try it for real. You don't want to try it with a TodoMVC. DILLON: Exactly. It needs to be meaningful, to really have a good understanding of what it's going to be like. CHARLES: I would say that I tend to agree but I've definitely encountered the counterargument and I also think this counterargument makes sense or perhaps where the pushback lies is if I'm constantly experimenting, then what I'm doing is I'm internally fragmenting my ecosystem and there is power in similarity. Any time you introduce something different, any time you introduce one fragment, you're introducing complexity -- a mental complexity -- like maybe I have to maintain my Elm app and I also have to have my Legacy... Or not Legacy, I've got my other JavaScript tool kit that does it in one way. Maybe I've got a couple of more because I've run these other experiments. I'm not saying that there is one way but there is power in uniformity. There is power in diversity. Where do you find the balance? DILLON: Those are all excellent points. To me, I think really the key is it's about the scientific method, you could say. The thing with the scientific method is that we often forget the last part. We get really good at hypothesizing about things. Sometimes people leave it at that, which we kind of just discussed. Sometimes, people go past the hypothesizing stage and they actually run the experiment and that's great. But then, the majority of people, if they get to that point, will forget to do the last step which is to evaluate the results. I think the key here is you need to be experimenting and this is what it means for it to be a low risk experiment. It means that you're not setting yourself off in a direction where you can't turn back. You want to set it up in such a way that you can turn away from it with minimal cost. One of the things that is really helpful for that is if you build a tiny, independent, little widget in your application, try building that in Elm. Some people will do that with a little sort of login badge in the corner of their application. One of the teams where I've introduced Elm at a Fortune 10 company, actually where we introduced Elm, we started out with just a tiny little table in one page and if we wanted to back that out, it would have been trivially easy but we decided that we wanted to go in further and invest more. CHARLES: That makes a lot of sense. Effectively, you need to have a Plan B. Don't sync all of the available time that you have to invest in an experiment. Make sure that you have a Plan B and if you need to do this widget or this table in Angular or React or Ember or whatever, you are thinking about that -- how would that work. DILLON: Exactly and the thing with experiments is the purpose of an experiment is not to build something. It's to learn. I really like this kind of ethos of lean startup, which I think is really getting much more into the mainstream in the software industry, which is a wonderful thing. The idea of lean startup, the kind of core concept is this idea of validated learning. Basically, in an environment where there's uncertainty, which is pretty much most of the things you're doing in software, the main goal is you're not shipping a product like you would be if you're trying to manufacture cars as quickly as possible. The main thing that you're producing is what they call 'validated learning' and so, you want to minimize the amount of time it takes to validate or invalidate your assumptions about something and then, you want to make it as cheap as possible to move on from that. CHARLES: I like that. So if you're going to organize your development process around this principle or maybe not organize it but integrate it into development process, how do you know that you're conducting a healthy number of experiments, versus I may be conducting too many experiments? Is there a metric that you can look at? We need to have this many experiments running at all times or this is just too many or something else. DILLON: That's a really interesting question. I think I would tend to think about that more again, as looking at the way the experiments are run, rather than 'are there too many experiments?' That's just not a problem that I've seen there being too many experiments. The pain that we tend to really see in environments where experiments are hurting teams is the way the experiments are being done. It's hard to backtrack from those experiments and as you were saying before, you kind of put yourself down this path where you can't walk it back and you create this sort of rift in the way the code is being written, which makes it more difficult to work in that codebase. The thing with experiments is they can have really big payoff. Now, you want to make sure that you're not just going in and picking up every shiny object that you see. One thing that can keep you honest with that is if you're kind of coming up with a hypothesis before you start. If you're saying, "This is the value to our business and to our team if we attempt this thing and this is what will prove that it seems to have that value and this is what will tell us, 'Actually, it doesn't have that value and we should drop it and cut our losses.'" CHARLES: That's a great heuristic. As you're saying and imagining how that might have saved my bacon in the past because I've definitely made the mistake of playing with too many shiny objects and picking things because I didn't fully evaluate what I thought the value. I was explicit with myself about what is the value that is going to bring to this project or this business. I have a theory about it but I am not thinking what is my hypothesis and how am I going to validate or invalidate? I'm thinking, I've got a short term pain that I'm experiencing and I'm grasping for this thing, which I think will solve it and I'm not properly evaluating how it's going to affect me long term. DILLON: Right and that could be a great team practice to play around with is often, teams will kind of come up with action items out of retrospectives. One thing that I think can be really beneficial for teams is to kind of flip that notion of doing action items which again, it's really just doing the middle part of the experiment where you're conducting the test but you cut out the hypothesis part and the evaluation part. Try to bring that into your team's retrospective and try to have explicit hypotheses in the retrospectives and then, in the next retrospective, evaluate the results. CHARLES: All right. I will definitely keep that in mind but this feels like a fresh take on kind of how you manage software development that I haven't encountered too much, being more scientific about it. It sounds like science-oriented development. DILLON: Right. DAVID: I like that. DILLON: There are a lot of buzzwords these days in software development, in general and it's really becoming a problem, I think in the Agile community but really, what it boils down to is these basic elements and basing decisions on feedback is one of those fundamental unit. You can call the scientific method, you can call it lean startup and validated learning, you can call it agile, you can call it whatever you want but ultimately, you need to be basing things on feedback. I think of it almost like our nerves. There's actually a disorder that some people have, which can be fatal, which is that their nerves don't tell them when they're feeling pain. I think this is a great analogy for software because that can happen to companies too. They don't feel the pain of certain decisions not landing well. Because they're not getting feedback from users, they're not getting feedback from metrics and recording, they're not getting feedback from doing that final evaluation step of their experiments, so when you fall on the ground, a small cut could be extremely harmful because you don't know the damage it's doing to you. CHARLES: I think that is a good analogy. One of the things that I'm curious about is we've been discussing a lot of techniques for experimentation and how you can integrate that into your process and how you can make your experiments safe, so let's talk a little bit about -- first of all, two things -- why would I want to experiment with Elm in the first place? Because ultimately, that's why we're here and why we're having this conversation. What's compelling about it that would make me want to experiment? And then how can I begin to experiment with Elm? DILLON: I actually just published a blog post yesterday. It's called 'How Elm Code Tends Towards Simplicity.' To your question of why would a team consider Elm, I kind of talk a little bit in this blog post about a case study at a Fortune 10 company where I introduced Elm to a few of the teams there. One of the teams there, we had actually seen an Angular project that they had worked on and often, in an enterprise environment, you have projects moving from one team to another. I actually had my hands on this Angular project. It kind of moved over to another team and we were experiencing some major pain trying to make changes in this codebase. Even making the simplest change, we were finding that there were a lot of bugs that would be introduced because there's some global variable. There's some implicit state. Sometimes, it was even reaching in and tweaking the DOM and really, the topic of conversation at our team lunches was how afraid we were to touch this codebase. Fast forward a few months and this team was asking my advice on picking a new frontend framework and I introduced them to Elm. They took a run with it and it was pretty remarkable to see this same team that had really struggled with AngularJS and they didn't really have a strong sense of what were the best practices. They weren't getting any guidance from the framework itself and the tooling around it and they actually loved the experience of working with Elm because they were saying, "This is amazing. Maybe it takes a little time to figure out how to solve a particular problem on Elm but once we do, we know that we've done it in a solid way." One of the things that I think is most powerful about Elm is that it keeps you from shooting yourself in the foot. I think that's a really good headline kind of summary of what I love about Elm. For example, tweaking the DOM. Now, it might seem like a pretty obvious thing that we just won't tweak the DOM and that's fair enough. That might not be a problem for a lot of teams. People wouldn't even reach for that technique because they're disciplined about it. But at a certain point, you start taking on enough things and then go from kind of those basic things that are going to make your code more unreliable and unsafe like tweaking the DOM and you start getting into the realm of best practices. There's so much discussion these days in the JavaScript community about best practices, which is great. It's great to discuss that but my concern is that there's a new best practice each week and the team has to agree on it, you have to find techniques for enforcing it, people have to make sure that these best practices are being followed in code reviews. Then when you look at a given piece of code, you have to trust that those best practices are being followed, so it requires a lot of work to make sure, in your reducers, in redux that you are not mutating anything. With Elm, data is just immutable. That's just how it is. There are a lot of these kind of things that are baked into the language and the expressivity of the type system allows you to bake in your own constraints. One of the things that I find really compelling about Elm is its design really prevents you from shooting yourself in the foot and it gives you tools for making sure that you take it even a step further and it helps you enforce these best practices at a compiler level. CHARLES: Now what's interesting here is it's almost like the opposite tension of experimentation is a work, right? like here, we have an example of uniformity being the more powerful track but then inside the actual macroscopic process, you want a lot of experimentation and diversity. But at the microscopic level, inside your application, it sounds like you want less experimentation and you derive a lot of strength from that but -- DILLON: That's a great point. CHARLES: -- Experiments that are possible, yeah. DILLON: I think that there is a lot of pain these days in the JavaScript community. We hear people talking often about JavaScript fatigue and it's a real thing. It takes a lot of work to stay on top of the latest best practices and frameworks and that can be a lot of fun. I love learning about the latest new frameworks and tooling but ultimately as you're saying, we don't want that experimentation so much about the fundamentals. We want some dependable, solid fundamentals and then we want the experimentation to happen within there. I think that's exactly what we see in the Elm ecosystem. We have a single kind of data store or way of managing state in Elm. It's called the Elm Architecture. In fact, it's what Redux is based on and it worked extremely well and you don't have to experiment with different data stores in Elm because that's just what Elm code looks like. Now, if you want to experiment in Elm, then there is a lot of innovation happening. One of my favorite things about Elm is that the compiler and its expressiveness has sparked a lot of creativity. One of my favorite things about Elm is the library called Elm UI. Actually, a client that I'm working with right now, it's a really interesting case study. They are kind of a very small startup. They just kind of branched off of a larger startup. They're building some tooling for this ecosystem. They were engineers at a company called Procore that does cloud document management for construction companies. They wanted to get a product-ready for a big conference for their potential clients. The reason they brought me in to help them was because they wanted to reach this ambitious target of being able to do a demo of this brand new product at this conference and they wanted to iterate very quickly. One of the things that really drew them into Elm in the first place is this library Elm UI. Elm UI essentially, Richard Feldman gives a talk on it, where he uses the analogy of it being treating CSS and HTML as bytecode for your views. I think that's a really apt way to put it. If you break down this idea of CSS -- Cascading Style Sheets, it removes the cascading part of CSS and it removes the sheets part of CSS. What you're left with is a way of expressing style and it's a way of expressing style that is able to part ways with all of the baggage of the entire history of backwards compatible decisions that CSS has ever made. If you want to vertically aligned something, then you just say, "Align vertically," you know, center vertically. If you want to center something horizontally, you say center X. It creates a high level language for expressing views. My experience with Elm UI, this may not be the right choice for every team but I love it. I use it on all of the projects that I maintain personally. I love using it because it gives you that same sense of invincibility refactoring that you get with Elm, which is remarkable that you could have that feeling with managing views. CHARLES: It's definitely something that feels like a dark art and it can't be called science. It's an art. It's a science for some people but it's historically been a dark art and something fiddly to work with. In terms of being able to make the experiment with Elm, when we talked a little bit about why you might want to experiment with it in the first place, what the business case is, I guess my next question is or a question that immediately comes to mind is supposing that we have decided to experiment with this, how do you mitigate that experiment? We talked about lowering the cost, having a way to turn away from it, having a way to make it inexpensive. For example, one of the things that I think of when evaluating a new technology is how well can I use it with old technologies. I have a lot about best practices in my tool bag. We all do. We got our all favorite libraries and pathways that are just familiar to us. One of the things that I've noticed is when adopting a new technology, one of the things that makes it easy to experiment with is how well it works with the existing technologies. I know that, we talked about Elm UI, kind of rethinking style in CSS and your views and Elm itself as a completely different language within JavaScript, that can be both liberating but it can also be limiting in the sense that I can't reach back for my existing tool if no tool exists in this new space. The kind of experience that I've had where this is really worked is systems like JRuby or Clojure, where there's a very clear pathway to be able to use Java libraries from those environments, so you always have kind of an escape hatch. What's that like in Elm? DILLON: This is a really interesting conversation because it highlights, in some ways some of the most defining features of Elm. In terms of how do you kind of pull Elm into an existing application, there are a lot of different techniques for that. It's pretty straightforward to create a little Elm app. We usually don't call them components for reasons that we can get into if we want to but that's a whole can of worms. But if you've got a little Elm application that you want to use to render a widget on your page, then it's as simple as just calling Elm.yourmodule.init and rendering it onto the page there. That's quite straightforward and if you want to interface with your existing code there are several ways to do that. There's something called port in Elm, which is how you kind of communicate by sending these messages and data back and forth between your Elm app and JavaScript. Now, this is one of the decisions, I think that defines Elm as the language and the reason this is important is because Elm decided not to make the choice that a lot of other compile to JS languages do. For example, if you look at ReasonML or PureScript or a more extreme example, TypeScript. TypeScript is a superset of JavaScript, so it's trying to allow you to gradually introduce this to get some incremental improvements for your JavaScript code, so it's extremely easy to experiment with it, which we've talked about the importance of experimentation. Now, the challenge with this technique, the tradeoff here is it's great, that it then becomes very easy to transition into it and that's an excellent strategy for the goals of TypeScript. Elm has a different set of goals, so the things that elm is focused on giving you is a truly type-safe experience. When you're working with Elm, if your Elm code says that this data is a float, then it is a float. Either, it is a float or that code is not being run and so, that's very different than the experience in TypeScript where you have these escape hatches. This is an inevitable choice for any compiled to JS language. Are you going to have escape hatches or not? Elm is really the only language out there, I think that chooses not to have escape hatches and that is actually the thing that that I love about the language because that's the only way you can truly have guarantees, rather than, "Yeah, I'm pretty sure that these type guarantees hold." DAVID: Yeah, wishes and dreams. DILLON: Yeah. CHARLES: What does it mean to have no escape hatches? because you talked about ports. Does it mean like it's impossible to use an external JavaScript library? DILLON: That's an excellent question. You absolutely can use JavaScript libraries. It means that it's being explicit and upfront about the fact that there's uncertainty in these areas. That's what it comes down to. Take for example dealing with JSON. In a JavaScript application, what we get when we're dealing with JSON is you make a request up to the server, you have some callback that passes in the data you get back and then you start pulling bits and pieces off of it and you say 'response.users subzero.firstname' and you hope that none of those things are null, none of those types are different than what you expected. In a way, it's kind of letting you pretend that you have certainty there when in fact, you don't and with Elm, the approach is quite different. You have to explicitly say, "I expect my response to have this shape. I expect it to be a list of things, which have a first name and last name which are strings," and then Elm says, "Okay, great. I'm going to check your assumptions," and if you're right, then here you go and you're in a well typed-space where you know exactly what the types are and if you're wrong, then that's just another type of data, so it's just a case statement where you say, "If my assumptions were correct, then do something and if my assumptions were incorrect, then you decide what to do from there." CHARLES: Right. For me, it sounds like there is some way because ultimately, I'm going to be getting unstructured but I'm going to be getting JSON back from the server and maybe, I have some library that's going to be doing that for me and enhancing it and adding value to that JSON in some way. But then at some point, I can present it to Elm but what you just saying is I need to be complete in making sure that I handle each case. I need to do or handle the case. Explicit about saying if the assumptions that Elm wants to make, turn out to not be true, Elm is going to make me handle the case where those assumptions were not true. DILLON: Exactly. I think that TypeScript of any type is the perfect illustration of the difference. TypeScript of any type is sort of allowing you to say, "Don't type check this. Trust me here," and Elm's approach is more kind of just be explicit about what you want me to do if your assumptions are incorrect. It doesn't let you kind of come in and say, "No, I know I'm going to be right here." CHARLES: Right but there is a way to pass data structures back and forth. DILLON: There absolutely is and actually, there's a technique that's starting to gain some traction now, which I'm really excited about, which is rather than using this sort of JavaScript interrupt technique we talked about, which is again, it's very much like communicating with a server where you're kind of sending messages and getting data back -- getting these messages with data back. But there's an alternative to that which is using web components. Actually, there's quite good support for assuming that you don't need to be compatible with Internet Explorer. Basically in a nutshell, if you can wrap a sort of declarative web component around anything, it could be a Google Maps API, it could be a syntax highlighting JavaScript library, something that you don't have an Elm library for but you want to use this JavaScript library, it's actually quite a nice experience. You just render that custom element using your Elm code just as you would any other HTML in Elm. CHARLES: Yeah I like that, so the HTML becomes the canvas or composition with other JavaScript and the semantics are very well-defined and that interface is actually pretty thin. DILLON: Exactly and the key again is that you wanted to find a declarative interface, rather than an imperative one where you're kind of just doing a series of statements where you say, "Do this and then set this value and then call this and then set this call back." Instead, you're saying, "Render this Google Maps custom element," which is centered around these coordinates and has this zoom level on. You declaratively give it the bit of information that it needs to render a particular view. CHARLES: Okay. Then I guess the final question that I have around this area is about being able to integrate existing tools and functions inside of an Elm application. Because it sounds like you could theoretically develop large parts of your application, is there a way that you can actually have other areas of your application that are not currently invested in Elm still benefit from it, in the sense for kind of need of JavaScript APIs that Elm can make available. DILLON: Right, so you're kind of talking about the reverse of that Elm reaching out to JavaScript. You're asking about, can JavaScript reach out to Elm and benefit from some of its ecosystem? CHARLES: Exactly. I say that is that another potential vector for experimentation. DILLON: It's a really interesting thought. I haven't given it too much thought, to be honest but I actually have heard it come up before and my gut feeling is that it's probably more fruitful to explore the inverse, reaching out to JavaScript from Elm and the reason is kind of the main appeal of Elm is that when you're operating within Elm, you have this sense that if it compiles, it works. Because again, this central decision to not allow escape hatches is what allows you to have that sort of robustness, so you have this feeling of bullet proof refactorings and adding new features seamlessly where you change your data modeling to say, "Here's this other case that can be represented," and then suddenly, the Elm compiler says, "Tell me what to do here, tell me what to do here and tell me what to do there," and you do it and your app is working. That's the real appeal of Elm, I think and you don't really get much of that by just calling out to an Elm library from within JavaScript. That's my gut feeling on it. CHARLES: Okay, that's fair enough. On the subject of interrupt and using tools like JSON, you actually maintain a GraphQL library for Elm. You probably have a lot of experience on this. Maybe we can talk about that as a concrete case that highlights the examples. DILLON: Yeah. I think to me this is one of the things that really highlights the power of Elm, to give you a really amazing refactoring and kind of feature creating experience. A lot of Elm libraries are prefaced by the author name, so it's still DillonKearns/ElmGraphQL. I spoke about it this year at ElmConf. In a nutshell, what it does is it actually generates code based on your GraphQL schema. For anyone who doesn't know, GraphQL is just kind of a language for expressing the shape of your API and what types of data can return. What DillonKearns on GraphQL does is it looks at your GraphQL schema and it generates an API that allows you to query that API. using this library, you can actually guarantee that you're making a valid query to your server. Again, you get this bulletproof experience of refactoring in Elm where you can do something like make a change in your API and recompile your Elm code and see whether you've made a backwards incompatible change. All of this effort of doing sort of this JSON decoding I was talking about earlier where you kind of have to explicitly say, "These are my assumptions about the shape of the JSON that I'm getting back." When you're using this library, you no longer need to make any assumptions because you're able to rely on this sort of schema of your API and so you know when you're requesting this data, you don't have to run it, see if it works and then tweak it and run it again -- this sort of cycle of checking your assumptions at runtime. It moves those assumptions that you're making from runtime to compile time and it can tell you when you compile your application, it can say, "Actually, this data you're requesting, it doesn't exist," or, "It's actually called this," or, "This is actually the type of the data." CHARLES: Right. I love that. How do you do that? Because it seems like you've got a little bit of a chicken and the egg problem because the schema is defined outside of Elm, so you have to be able to parse and understand the schema in order to generate the Elm types to be able to compile Elm code against them. Maybe I'm not -- DILLON: That's exactly right. That's exactly what it does. Now, the nice thing is that GraphQL is really designed for these types of use cases. It supports them in a first class way. If you have a GraphQL API, that means you have built into it whether you know about it or not, a way to introspect the schema. All of the queries for kind of interrogating that GraphQL server and asking what types of data does this return, what are all your queries that I can run, it's built into it by the framework, so that comes for free. Getting up and running with this package I built is as simple as running a little npm CLI, pointing it to either your URL for your server or the JSON form of your schema, if you prefer and then, it generates the code for you. CHARLES: Wow, that sounds fantastic. This is the exact kind of thing that feels like it would be cool if I could just start using this library to manage the GraphQL of my application but I'm consuming that GraphQL from other JavaScript but it's the Elm code that's managing it. Do you see what I mean? DILLON: I hadn't considered that. I guess you could. You're right. Maybe I'm so smitten with Elm that it's hard to see an in-between but I guess, you could get some benefits from that approach. CHARLES: Right and as an experiment of course. DILLON: There you go. There you go. CHARLES: All right. With that, I think we'll wrap it up. Thank you so much, Dillon for coming on and talking with us on the podcast. DILLON: My pleasure. I really enjoyed the conversation. CHARLES: I actually got so many great tidbits from so many different areas of software development in Elm but also, just in kind of other things that I'm interested in trying. It was a really great conversation. DILLON: I had a lot of fun and I love discussing these things. For any listeners who are interested in this stuff, feel free to reach out to me on the Elm Slack or on Twitter. I'm at @DillonTKearns. I'm also offering a free intro Elm talk for any companies that are kind of entertaining the idea of doing an experiment with Elm. If you go to IncrementalElm.com/Intro, you can find out about some of the talks that I'm offering. CHARLES: All right. Well, thank you very much and we, as always are the Frontside. We build software that you can stake your future on and you can get in touch with us at @TheFrontside on Twitter or Contact@Frontside.io on email. Please send us any questions you might have, any topics that you'd like to hear about and we look forward to hearing from you and we will see you next week.
Panel: Aimee Knight Charles Max Wood Joe Eames AJ O’Neil Chris Ferdinandi Special Guests: Charles Lowell (New Mexico) & Taras Mankovski (Toronto) In this episode, the panel talks with two special guests Charles and Taras. Charles Lowell is a principle engineer at Frontside, and he loves to code. Taras works with Charles and joined Frontside, because of Charles’ love for coding. There are great personalities at Frontside, which are quite diverse. Check out this episode to hear about microstates, microstates with react, Redux, and much more! Show Topics: 1:20 – Chuck: Let’s talk about microstates – what is that? 1:32 – Guest: My mind is focused on the how and not the what. I will zoom my mind out and let’s talk about the purposes of microstates. It means a few things. 1.) It’s going to work no matter what framework you are using. 2.) You shouldn’t have to be constantly reinventing the wheel. React Roundup – I talked about it there at this conference. Finally, it really needs to feel JavaScript. We didn’t want you to feel like you weren’t using JavaScript. It uses computer properties off of those models. It doesn’t feel like there is anything special that you are doing. There are just a few simple rules. You can’t mutate the state in place. If you work with JavaScript you can use it very easily. Is that a high-level view? 7:13 – Panel: There are a lot of pieces. If I spoke on a few specific things I would say that it enables programming with state machines. 7:42 – Panel: We wanted it to fell like JavaScript – that’s what I heard. 7:49 – Aimee: I heard that, too. 7:59 – Guest. 8:15 – Aimee: Redux feels like JavaScript to me. 8:25 – Guest: It’s actually – a tool – that it feels natural so it’s not contrived. It’s all JavaScript. 8:49 – Panel. 9:28 – Guest: Idiomatic Ember for example. Idiomatic in the sense that it gives you object for you to work with, which are simple objects. 10:12 – Guest: You have your reducers and your...we could do those things but ultimately it’s powerful – and not action names – we use method names; the name of the method. 11:20 – Panel: I was digging through docs, and it feels like NORMAL JavaScript. It doesn’t seem like it’s tied to a certain framework or library platform? 11:45 – Guest: Yes, we felt a lot of time designing the interfaces the API and the implementation. We wanted it to feel natural but a tool that people reach for. (Guest continues to talk about WHY they created microstates.) Guest: We wanted to scale very well what you need when your needs to change. 13:39 – Chuck: I have a lot of friends who get into React and then they put in Redux then they realize they have to do a lot of work – and that makes sense to do less is more. 14:17 – Guest: To define these microstates and build them up incrementally...building smaller microstates out of larger ones. Guest continued: Will we be able to people can distribute React components a sweet array of components ready for me to use – would I be able to do the same for a small piece of state? We call them state machines, but ultimately we have some state that is driving it. Would we be able to distribute and share? 16:15 – Panel: I understand that this is tiny – but why wouldn’t I just use the native features in specific the immutability component to it? 16:42 – Guest: I’m glad you asked that question. We wanted to answer the question... Guest: With microstates you can have strict control and it gives you the benefit of doing sophisticated things very easily. 18:33 – Guest: You mentioned immutability that’s good that you did. It’s important to capture – and capturing the naturalness of JavaScript. It’s easy to build complex structures – and there is an appeal to that. We are building these graphs and these building up these trees. You brought up immutability – why through it away b/c it’s the essence of being a developer. If you have 3-4-5 levels of nesting you have to de-structure – get to the piece of data – change it – and in your state transition 80% of your code is navigating to the change and only 20% to actually make the change. You don’t have to make that tradeoff. 21:25 – Aimee: The one thing I like about the immutability b/c of the way you test it. 21:45 – Guest: There a few things you can test. 23:01 – Aimee: You did a good job of explaining it. 23:15 – Guest: It makes the things usually hard easy! With immutability you can loose control, and if that happens you can get so confused. You don’t have a way to have a way to navigate to clarity. That’s what this does is make it less confusing. It gives you order and structure. It gives you a very clear path to do things you need to do. If there is a property on your object, and if there is a way to change it... 25:29 – Guest: The only constant is change no matter what framework you are working on. 24:46 – Chuck: We are talking about the benefits and philosophy. What if I have an app – and I realize I need state management – how do I put microstates into my app? It’s using Angular or React – how do I get my data into microstates? 26:35 – Guest: I can tell you what the integration looks like for any framework. You take a type and you passed that type and some value to the create function so what you get is a microstate. (The Guest continues diving into his answer.) 28:18 – Guest: That story is very similar to Redux, basically an event emitter. The state changes on the store. Maybe this is a good time to talk about the stability benefits and the lazy benefits because microstates is both of those things. Stability – if I invoke a transition and the result is unchanged – same microstate – it doesn’t emit an event. It recognizes it internally. It will recognize that it’s the same item. Using that in Ember or Redux you’d have to be doing thousands of actions and doing all that computation, but stability at that level. Also, stability in the sense of a tree. If I change one object then that changes it won’t change an element that it doesn’t need to change. 31:33 – Advertisement: Sentry.io 32:29 – Guest: I want to go back to your question, Chuck. Did we answer it? 32:40 – Chuck: Kind of. 32:50 – Guest. 32:59 – Guest: In Angular for example you can essentially turn a microstate... 33:51 – Guest: You could implement a connect, too. Because the primitive is small – there is no limit. 34:18 – Chuck summarizes their answers into his own words. 34:42 – Guest: If you were using a vanilla React component – this dot – I will bind this. You bind all of these features and then you pass them into your template. You can take it as a property...those are those handlers. They will perform the transition, update and what needs to be updated will happen. 35:55 – Chuck: Data and transitions are 2 separate things but you melded them together to feel like 1 thing. This way it keeps clean and fast. 36:16 – Guest: Every framework helps you in each way. Microstates let’s you do a few things: the quality of your data all in one place and you can share. 38:12 – Guest: He made and integrated Microstates with Redux tools. 38:28 – Guest talks about paths, microstates to trees. 39:22 – Chuck. 39:25 – Panel: When I think about state machines I have been half listening / half going through the docs. When I think of state machines I think about discreet operations like a literal machine. Like a robot of many steps it can step through. We have been talking about frontend frameworks like React - is this applicable to the more traditional systems like mechanical control or is it geared towards Vue layered applications? 40:23 – Guest: Absolutely. We have BIG TEST and it has a Vue component. 41:15 – Guest: when you create a microstate from a type you are creating an object that you can work with. 42:11 – Guest: Joe, I know you have experience with Angular I would love to get your insight. 42:33 – Joe: I feel like I have less experience with RX.js. A lot of what we are talking about and I am a traditionalist, and I would like you to introduce you guys to this topic. From my perspective, where would someone start if they haven’t been doing Flux pattern and I hear this podcast. I think this is a great solution – where do I get started? The official documents? Or is it the right solution to that person? 43:50 – Guest: Draw out the state machine that you want to represent in your Vue. These are the states that this can be in and this is the data that is required to get from one thing to the other. It’s a rope process. The arrow corresponds to the method, and... 44:49 – Panel: It reminds me back in the day of rational rows. 44:56 – Guest: My first job we were using rational rows. 45:22 – Panelist: Think through the state transitions – interesting that you are saying that. What about that I am in the middle – do you stop and think through it or no? 46:06 – Guest: I think it’s a Trojan horse in some ways. I think what’s interesting you start to realize how you implement your state transitions. 48:00 – (Guest continues.) 48:45 – Panel: That’s interesting. Do you have that in the docs to that process of stopping and thinking through your state transitions and putting into the microstate? 49:05 – Guest: I talked about this back in 2016. I outlined that process. When this project was in the Ember community. 49:16 – Guest: The next step for us is to make this information accessible. We’ve been shedding a few topics and saying this is how to use microstates in your project. We need to write up those guides to help them benefit in their applications. 50:00 – Chuck: What’s the future look like? 50:03 – Guest: We are working on performance profiling. Essentially you can hook up microstates to a fire hose. The next thing is settling on a pattern for modeling side effects inside microstates. Microstates are STATE and it’s immutable. 52:12 – Guest: Getting documentation. We have good README but we need traditional docs, too. 52:20 – Chuck: Anything else? 52:28 – Guest: If you need help email us and gives us a shot-out. 53:03 – Chuck: Let’s do some picks! 53:05 – Advertisement for Charles Max Wood’s course! Links: Kendo UI Frontside Redux Microstates Microstates with React Taras Mankovski’s Twitter Taras Mankovski’s GitHub Taras Mankovski’s LinkedIn Taras Mankovski’s Frontside Bio Charles Lowell’s Twitter Charles Lowell’s GitHub Charles Lowell’s Frontside Bio Schedule Once Ruby on Rails Angular Get A Coder Job YouTube Talks Email: cowboyd@frontside.io Working with State Machines Twitch TV BigTest Close Brace REEF The Developer Experience YouTube Video Sponsors: Kendo UI Sentry.io – 2 months free – DEVCHAT/code Get A Coder Job Picks: Aimee ShopTalk Episode 327 Professional JavaScript for Web Developers Technical Debt Stripe Taras Twitch Channel Big Test Frontside Charles Lowell Chalkboards Sargent Art Chalk Chris Close Brace LaCroix Water Chris’s Git Hub Joe The Developer Experience Bait and Switch Good Bye Redux Recording Dungeon and Dragons AJ UtahJS Conf Start with Why The Rust Book VanillaJS w/ Chris Zero to One Charles Podwrench.com - beta getacoderjob.com
Panel: Aimee Knight Charles Max Wood Joe Eames AJ O’Neil Chris Ferdinandi Special Guests: Charles Lowell (New Mexico) & Taras Mankovski (Toronto) In this episode, the panel talks with two special guests Charles and Taras. Charles Lowell is a principle engineer at Frontside, and he loves to code. Taras works with Charles and joined Frontside, because of Charles’ love for coding. There are great personalities at Frontside, which are quite diverse. Check out this episode to hear about microstates, microstates with react, Redux, and much more! Show Topics: 1:20 – Chuck: Let’s talk about microstates – what is that? 1:32 – Guest: My mind is focused on the how and not the what. I will zoom my mind out and let’s talk about the purposes of microstates. It means a few things. 1.) It’s going to work no matter what framework you are using. 2.) You shouldn’t have to be constantly reinventing the wheel. React Roundup – I talked about it there at this conference. Finally, it really needs to feel JavaScript. We didn’t want you to feel like you weren’t using JavaScript. It uses computer properties off of those models. It doesn’t feel like there is anything special that you are doing. There are just a few simple rules. You can’t mutate the state in place. If you work with JavaScript you can use it very easily. Is that a high-level view? 7:13 – Panel: There are a lot of pieces. If I spoke on a few specific things I would say that it enables programming with state machines. 7:42 – Panel: We wanted it to fell like JavaScript – that’s what I heard. 7:49 – Aimee: I heard that, too. 7:59 – Guest. 8:15 – Aimee: Redux feels like JavaScript to me. 8:25 – Guest: It’s actually – a tool – that it feels natural so it’s not contrived. It’s all JavaScript. 8:49 – Panel. 9:28 – Guest: Idiomatic Ember for example. Idiomatic in the sense that it gives you object for you to work with, which are simple objects. 10:12 – Guest: You have your reducers and your...we could do those things but ultimately it’s powerful – and not action names – we use method names; the name of the method. 11:20 – Panel: I was digging through docs, and it feels like NORMAL JavaScript. It doesn’t seem like it’s tied to a certain framework or library platform? 11:45 – Guest: Yes, we felt a lot of time designing the interfaces the API and the implementation. We wanted it to feel natural but a tool that people reach for. (Guest continues to talk about WHY they created microstates.) Guest: We wanted to scale very well what you need when your needs to change. 13:39 – Chuck: I have a lot of friends who get into React and then they put in Redux then they realize they have to do a lot of work – and that makes sense to do less is more. 14:17 – Guest: To define these microstates and build them up incrementally...building smaller microstates out of larger ones. Guest continued: Will we be able to people can distribute React components a sweet array of components ready for me to use – would I be able to do the same for a small piece of state? We call them state machines, but ultimately we have some state that is driving it. Would we be able to distribute and share? 16:15 – Panel: I understand that this is tiny – but why wouldn’t I just use the native features in specific the immutability component to it? 16:42 – Guest: I’m glad you asked that question. We wanted to answer the question... Guest: With microstates you can have strict control and it gives you the benefit of doing sophisticated things very easily. 18:33 – Guest: You mentioned immutability that’s good that you did. It’s important to capture – and capturing the naturalness of JavaScript. It’s easy to build complex structures – and there is an appeal to that. We are building these graphs and these building up these trees. You brought up immutability – why through it away b/c it’s the essence of being a developer. If you have 3-4-5 levels of nesting you have to de-structure – get to the piece of data – change it – and in your state transition 80% of your code is navigating to the change and only 20% to actually make the change. You don’t have to make that tradeoff. 21:25 – Aimee: The one thing I like about the immutability b/c of the way you test it. 21:45 – Guest: There a few things you can test. 23:01 – Aimee: You did a good job of explaining it. 23:15 – Guest: It makes the things usually hard easy! With immutability you can loose control, and if that happens you can get so confused. You don’t have a way to have a way to navigate to clarity. That’s what this does is make it less confusing. It gives you order and structure. It gives you a very clear path to do things you need to do. If there is a property on your object, and if there is a way to change it... 25:29 – Guest: The only constant is change no matter what framework you are working on. 24:46 – Chuck: We are talking about the benefits and philosophy. What if I have an app – and I realize I need state management – how do I put microstates into my app? It’s using Angular or React – how do I get my data into microstates? 26:35 – Guest: I can tell you what the integration looks like for any framework. You take a type and you passed that type and some value to the create function so what you get is a microstate. (The Guest continues diving into his answer.) 28:18 – Guest: That story is very similar to Redux, basically an event emitter. The state changes on the store. Maybe this is a good time to talk about the stability benefits and the lazy benefits because microstates is both of those things. Stability – if I invoke a transition and the result is unchanged – same microstate – it doesn’t emit an event. It recognizes it internally. It will recognize that it’s the same item. Using that in Ember or Redux you’d have to be doing thousands of actions and doing all that computation, but stability at that level. Also, stability in the sense of a tree. If I change one object then that changes it won’t change an element that it doesn’t need to change. 31:33 – Advertisement: Sentry.io 32:29 – Guest: I want to go back to your question, Chuck. Did we answer it? 32:40 – Chuck: Kind of. 32:50 – Guest. 32:59 – Guest: In Angular for example you can essentially turn a microstate... 33:51 – Guest: You could implement a connect, too. Because the primitive is small – there is no limit. 34:18 – Chuck summarizes their answers into his own words. 34:42 – Guest: If you were using a vanilla React component – this dot – I will bind this. You bind all of these features and then you pass them into your template. You can take it as a property...those are those handlers. They will perform the transition, update and what needs to be updated will happen. 35:55 – Chuck: Data and transitions are 2 separate things but you melded them together to feel like 1 thing. This way it keeps clean and fast. 36:16 – Guest: Every framework helps you in each way. Microstates let’s you do a few things: the quality of your data all in one place and you can share. 38:12 – Guest: He made and integrated Microstates with Redux tools. 38:28 – Guest talks about paths, microstates to trees. 39:22 – Chuck. 39:25 – Panel: When I think about state machines I have been half listening / half going through the docs. When I think of state machines I think about discreet operations like a literal machine. Like a robot of many steps it can step through. We have been talking about frontend frameworks like React - is this applicable to the more traditional systems like mechanical control or is it geared towards Vue layered applications? 40:23 – Guest: Absolutely. We have BIG TEST and it has a Vue component. 41:15 – Guest: when you create a microstate from a type you are creating an object that you can work with. 42:11 – Guest: Joe, I know you have experience with Angular I would love to get your insight. 42:33 – Joe: I feel like I have less experience with RX.js. A lot of what we are talking about and I am a traditionalist, and I would like you to introduce you guys to this topic. From my perspective, where would someone start if they haven’t been doing Flux pattern and I hear this podcast. I think this is a great solution – where do I get started? The official documents? Or is it the right solution to that person? 43:50 – Guest: Draw out the state machine that you want to represent in your Vue. These are the states that this can be in and this is the data that is required to get from one thing to the other. It’s a rope process. The arrow corresponds to the method, and... 44:49 – Panel: It reminds me back in the day of rational rows. 44:56 – Guest: My first job we were using rational rows. 45:22 – Panelist: Think through the state transitions – interesting that you are saying that. What about that I am in the middle – do you stop and think through it or no? 46:06 – Guest: I think it’s a Trojan horse in some ways. I think what’s interesting you start to realize how you implement your state transitions. 48:00 – (Guest continues.) 48:45 – Panel: That’s interesting. Do you have that in the docs to that process of stopping and thinking through your state transitions and putting into the microstate? 49:05 – Guest: I talked about this back in 2016. I outlined that process. When this project was in the Ember community. 49:16 – Guest: The next step for us is to make this information accessible. We’ve been shedding a few topics and saying this is how to use microstates in your project. We need to write up those guides to help them benefit in their applications. 50:00 – Chuck: What’s the future look like? 50:03 – Guest: We are working on performance profiling. Essentially you can hook up microstates to a fire hose. The next thing is settling on a pattern for modeling side effects inside microstates. Microstates are STATE and it’s immutable. 52:12 – Guest: Getting documentation. We have good README but we need traditional docs, too. 52:20 – Chuck: Anything else? 52:28 – Guest: If you need help email us and gives us a shot-out. 53:03 – Chuck: Let’s do some picks! 53:05 – Advertisement for Charles Max Wood’s course! Links: Kendo UI Frontside Redux Microstates Microstates with React Taras Mankovski’s Twitter Taras Mankovski’s GitHub Taras Mankovski’s LinkedIn Taras Mankovski’s Frontside Bio Charles Lowell’s Twitter Charles Lowell’s GitHub Charles Lowell’s Frontside Bio Schedule Once Ruby on Rails Angular Get A Coder Job YouTube Talks Email: cowboyd@frontside.io Working with State Machines Twitch TV BigTest Close Brace REEF The Developer Experience YouTube Video Sponsors: Kendo UI Sentry.io – 2 months free – DEVCHAT/code Get A Coder Job Picks: Aimee ShopTalk Episode 327 Professional JavaScript for Web Developers Technical Debt Stripe Taras Twitch Channel Big Test Frontside Charles Lowell Chalkboards Sargent Art Chalk Chris Close Brace LaCroix Water Chris’s Git Hub Joe The Developer Experience Bait and Switch Good Bye Redux Recording Dungeon and Dragons AJ UtahJS Conf Start with Why The Rust Book VanillaJS w/ Chris Zero to One Charles Podwrench.com - beta getacoderjob.com
Panel: Aimee Knight Charles Max Wood Joe Eames AJ O’Neil Chris Ferdinandi Special Guests: Charles Lowell (New Mexico) & Taras Mankovski (Toronto) In this episode, the panel talks with two special guests Charles and Taras. Charles Lowell is a principle engineer at Frontside, and he loves to code. Taras works with Charles and joined Frontside, because of Charles’ love for coding. There are great personalities at Frontside, which are quite diverse. Check out this episode to hear about microstates, microstates with react, Redux, and much more! Show Topics: 1:20 – Chuck: Let’s talk about microstates – what is that? 1:32 – Guest: My mind is focused on the how and not the what. I will zoom my mind out and let’s talk about the purposes of microstates. It means a few things. 1.) It’s going to work no matter what framework you are using. 2.) You shouldn’t have to be constantly reinventing the wheel. React Roundup – I talked about it there at this conference. Finally, it really needs to feel JavaScript. We didn’t want you to feel like you weren’t using JavaScript. It uses computer properties off of those models. It doesn’t feel like there is anything special that you are doing. There are just a few simple rules. You can’t mutate the state in place. If you work with JavaScript you can use it very easily. Is that a high-level view? 7:13 – Panel: There are a lot of pieces. If I spoke on a few specific things I would say that it enables programming with state machines. 7:42 – Panel: We wanted it to fell like JavaScript – that’s what I heard. 7:49 – Aimee: I heard that, too. 7:59 – Guest. 8:15 – Aimee: Redux feels like JavaScript to me. 8:25 – Guest: It’s actually – a tool – that it feels natural so it’s not contrived. It’s all JavaScript. 8:49 – Panel. 9:28 – Guest: Idiomatic Ember for example. Idiomatic in the sense that it gives you object for you to work with, which are simple objects. 10:12 – Guest: You have your reducers and your...we could do those things but ultimately it’s powerful – and not action names – we use method names; the name of the method. 11:20 – Panel: I was digging through docs, and it feels like NORMAL JavaScript. It doesn’t seem like it’s tied to a certain framework or library platform? 11:45 – Guest: Yes, we felt a lot of time designing the interfaces the API and the implementation. We wanted it to feel natural but a tool that people reach for. (Guest continues to talk about WHY they created microstates.) Guest: We wanted to scale very well what you need when your needs to change. 13:39 – Chuck: I have a lot of friends who get into React and then they put in Redux then they realize they have to do a lot of work – and that makes sense to do less is more. 14:17 – Guest: To define these microstates and build them up incrementally...building smaller microstates out of larger ones. Guest continued: Will we be able to people can distribute React components a sweet array of components ready for me to use – would I be able to do the same for a small piece of state? We call them state machines, but ultimately we have some state that is driving it. Would we be able to distribute and share? 16:15 – Panel: I understand that this is tiny – but why wouldn’t I just use the native features in specific the immutability component to it? 16:42 – Guest: I’m glad you asked that question. We wanted to answer the question... Guest: With microstates you can have strict control and it gives you the benefit of doing sophisticated things very easily. 18:33 – Guest: You mentioned immutability that’s good that you did. It’s important to capture – and capturing the naturalness of JavaScript. It’s easy to build complex structures – and there is an appeal to that. We are building these graphs and these building up these trees. You brought up immutability – why through it away b/c it’s the essence of being a developer. If you have 3-4-5 levels of nesting you have to de-structure – get to the piece of data – change it – and in your state transition 80% of your code is navigating to the change and only 20% to actually make the change. You don’t have to make that tradeoff. 21:25 – Aimee: The one thing I like about the immutability b/c of the way you test it. 21:45 – Guest: There a few things you can test. 23:01 – Aimee: You did a good job of explaining it. 23:15 – Guest: It makes the things usually hard easy! With immutability you can loose control, and if that happens you can get so confused. You don’t have a way to have a way to navigate to clarity. That’s what this does is make it less confusing. It gives you order and structure. It gives you a very clear path to do things you need to do. If there is a property on your object, and if there is a way to change it... 25:29 – Guest: The only constant is change no matter what framework you are working on. 24:46 – Chuck: We are talking about the benefits and philosophy. What if I have an app – and I realize I need state management – how do I put microstates into my app? It’s using Angular or React – how do I get my data into microstates? 26:35 – Guest: I can tell you what the integration looks like for any framework. You take a type and you passed that type and some value to the create function so what you get is a microstate. (The Guest continues diving into his answer.) 28:18 – Guest: That story is very similar to Redux, basically an event emitter. The state changes on the store. Maybe this is a good time to talk about the stability benefits and the lazy benefits because microstates is both of those things. Stability – if I invoke a transition and the result is unchanged – same microstate – it doesn’t emit an event. It recognizes it internally. It will recognize that it’s the same item. Using that in Ember or Redux you’d have to be doing thousands of actions and doing all that computation, but stability at that level. Also, stability in the sense of a tree. If I change one object then that changes it won’t change an element that it doesn’t need to change. 31:33 – Advertisement: Sentry.io 32:29 – Guest: I want to go back to your question, Chuck. Did we answer it? 32:40 – Chuck: Kind of. 32:50 – Guest. 32:59 – Guest: In Angular for example you can essentially turn a microstate... 33:51 – Guest: You could implement a connect, too. Because the primitive is small – there is no limit. 34:18 – Chuck summarizes their answers into his own words. 34:42 – Guest: If you were using a vanilla React component – this dot – I will bind this. You bind all of these features and then you pass them into your template. You can take it as a property...those are those handlers. They will perform the transition, update and what needs to be updated will happen. 35:55 – Chuck: Data and transitions are 2 separate things but you melded them together to feel like 1 thing. This way it keeps clean and fast. 36:16 – Guest: Every framework helps you in each way. Microstates let’s you do a few things: the quality of your data all in one place and you can share. 38:12 – Guest: He made and integrated Microstates with Redux tools. 38:28 – Guest talks about paths, microstates to trees. 39:22 – Chuck. 39:25 – Panel: When I think about state machines I have been half listening / half going through the docs. When I think of state machines I think about discreet operations like a literal machine. Like a robot of many steps it can step through. We have been talking about frontend frameworks like React - is this applicable to the more traditional systems like mechanical control or is it geared towards Vue layered applications? 40:23 – Guest: Absolutely. We have BIG TEST and it has a Vue component. 41:15 – Guest: when you create a microstate from a type you are creating an object that you can work with. 42:11 – Guest: Joe, I know you have experience with Angular I would love to get your insight. 42:33 – Joe: I feel like I have less experience with RX.js. A lot of what we are talking about and I am a traditionalist, and I would like you to introduce you guys to this topic. From my perspective, where would someone start if they haven’t been doing Flux pattern and I hear this podcast. I think this is a great solution – where do I get started? The official documents? Or is it the right solution to that person? 43:50 – Guest: Draw out the state machine that you want to represent in your Vue. These are the states that this can be in and this is the data that is required to get from one thing to the other. It’s a rope process. The arrow corresponds to the method, and... 44:49 – Panel: It reminds me back in the day of rational rows. 44:56 – Guest: My first job we were using rational rows. 45:22 – Panelist: Think through the state transitions – interesting that you are saying that. What about that I am in the middle – do you stop and think through it or no? 46:06 – Guest: I think it’s a Trojan horse in some ways. I think what’s interesting you start to realize how you implement your state transitions. 48:00 – (Guest continues.) 48:45 – Panel: That’s interesting. Do you have that in the docs to that process of stopping and thinking through your state transitions and putting into the microstate? 49:05 – Guest: I talked about this back in 2016. I outlined that process. When this project was in the Ember community. 49:16 – Guest: The next step for us is to make this information accessible. We’ve been shedding a few topics and saying this is how to use microstates in your project. We need to write up those guides to help them benefit in their applications. 50:00 – Chuck: What’s the future look like? 50:03 – Guest: We are working on performance profiling. Essentially you can hook up microstates to a fire hose. The next thing is settling on a pattern for modeling side effects inside microstates. Microstates are STATE and it’s immutable. 52:12 – Guest: Getting documentation. We have good README but we need traditional docs, too. 52:20 – Chuck: Anything else? 52:28 – Guest: If you need help email us and gives us a shot-out. 53:03 – Chuck: Let’s do some picks! 53:05 – Advertisement for Charles Max Wood’s course! Links: Kendo UI Frontside Redux Microstates Microstates with React Taras Mankovski’s Twitter Taras Mankovski’s GitHub Taras Mankovski’s LinkedIn Taras Mankovski’s Frontside Bio Charles Lowell’s Twitter Charles Lowell’s GitHub Charles Lowell’s Frontside Bio Schedule Once Ruby on Rails Angular Get A Coder Job YouTube Talks Email: cowboyd@frontside.io Working with State Machines Twitch TV BigTest Close Brace REEF The Developer Experience YouTube Video Sponsors: Kendo UI Sentry.io – 2 months free – DEVCHAT/code Get A Coder Job Picks: Aimee ShopTalk Episode 327 Professional JavaScript for Web Developers Technical Debt Stripe Taras Twitch Channel Big Test Frontside Charles Lowell Chalkboards Sargent Art Chalk Chris Close Brace LaCroix Water Chris’s Git Hub Joe The Developer Experience Bait and Switch Good Bye Redux Recording Dungeon and Dragons AJ UtahJS Conf Start with Why The Rust Book VanillaJS w/ Chris Zero to One Charles Podwrench.com - beta getacoderjob.com
Guest: Philip Poots: GitHub | ClubCollect Previous Episode: 056: Ember vs. Elm: The Showdown with Philip Poots In this episode, Philip Poots joins the show again to talk about the beauty of simplicity, the simplicity and similarities between Elm and Ruby programming languages, whether Elixir is a distant cousin of the two, the complexity of Ember and JavaScript ecosystems (Ember helps, but is fighting a losing battle), static vs. dynamic, the ease of Rails (productivity), and the promise of Ember (productivity, convention). The panel also talks about the definition of "quality", making code long-term maintainable, and determining what is good vs. what is bad for your codebase. Resources: Michel Martens mote Learn the Elm Programming Language and Build Error-Free Apps with Richard Feldman Worse is Better: Richard P. Gabriel Gary Bernhardt's Destroy All Software Screencasts Zen and the Art of Motorcycle Maintenance: An Inquiry into Values The Calm Company It Doesn't Have to Be Crazy at Work This show was produced by Mandy Moore, aka @therubyrep of DevReps, LLC. Transcript: CHARLES:: Hello, everybody and welcome to The Frontside Podcast, Episode 113. My name is Charles Lowell. I'm a developer here at the Frontside and with me today are Taras Mankovski and David Keathley. Hello? DAVID:: Hey, guys. TARAS: Hello, hello. CHARLES:: And we're going to be talking with a serial guest on our serial podcast, Mr Philip Poots, who is the VP of engineering at ClubCollect. Welcome, Philip. PHILIP: Hey, guys. Thanks for having me on. CHARLES:: Yeah. I'm actually excited to have you on. We've had you on a couple of times before. We've been trying to get you on the podcast, I think for about a year, to talk about I think what has kind of a unique story in programming these days. The prevailing narrative is that folks start off with some language that's dynamically typed and object oriented and then at some point, they discover functional programming and then at some point, they discover static programming and they march off into a promised land of Nirvana and no bugs ever, ever happening again. It seems like it's pretty much a straight line from that point to the next point and passing through those way stations. When I talk to you, I guess... Gosh, I think you were the first person that really introduced me to Elm back at Wicked Good Ember in 2016 and it seemed like you were kind of following that arc but actually, that was a bit deceptive because then the next time I talked to you, you were saying, "No, man. I'm really into Ruby and kind of diving in and trying to get into Ruby again," and I was kind of like, "Record scratch." You're kind of jumping around the points. You're not following the preordained story arc. What is going on here? I just kind of wanted to have a conversation about that and find out what the deal was and then, what's kind have guided your journey. PHILIP: There was one event and that was ElmConf Europe, which was a fantastic conference. Really, one of the best conferences I've been to, just because I guess with the nature of early language, small conference environment. There's just a lot of things happening. There's a lot of people. Evan was there, Richard Feldman was there, the leading lights of the Elm community were there and it was fantastic. But I guess, one thing that people have always said to me is the whole way track is the best track of the conference and it's not something I really appreciated before and during the breaks, I ended up talking to a guy called Michel Martens. He is the finder of a Redis sourcing company and I guess, this was just a revelation to me. He was interested in Elm. He was friends with the guys that organized the conference and we got talking and he was like, "I do this in Ruby. I do this in Ruby. I did this in Ruby," and I was like, "What?" and he was like, "Yeah, yeah, yeah." He's a really, really humble guy but as soon as I got home, I checked him out. His GitHub is 'soveran' and it turns out he's written... I don't know, how many gems in Ruby, all with really well-chosen names, very short, very clear, very detailed. The best thing about his libraries is you can print them out on paper. What I mean by that is they were tiny. They were so small and I guess, I just never seen that before. I go into Ruby on Rails -- that was my first exposure to programming, that was my first exposure to everything -- unlike with Rails, often when you hit problems, you'd start to dive a bit deeper and ultimately, you dive so deep that you sunk essentially and you just accepted, "Okay, I'm not going to bend the framework that way this time. Let's figure out how everyone else goes with the framework and do that." Then with Ember when I moved into frontend, that was a similar thing. There were so many layers of complexity that I never felt like had a real handle on it. I kind of just thought this was the way things were. I thought it's always going to be complex. That's just the nature of the problem. That's just the problem they're trying to solve. It's a complex problem and therefore, that complexity is necessary. But it was Elm that taught me, I think that choosing the right primitives and thinking very carefully about the problem can actually give you something that's quite simple but incredibly powerful. Not only something quite simple but something so simple that it can fit inside your head, like this concept of a program fitting inside your head and Rails, I don't know how many heads I need to fit Rails in or Ember for that matter and believe me, I tried it but with Elm, there was that simplicity. When I came across this Ruby, a language I was very familiar with but this Ruby that I had never seen before, a clear example was a templating library and he calls it 'mote' and it's including comments. It's under a hundred lines of code and it does everything you would need to. Sure, there were one or two edge cases that it doesn't cover but it's like, "Let's use the trade off." It almost feels like [inaudible] because he was always a big believer in "You ain't going to need it. Let's go for that 80% win with 20% effort," and this was like that taken to the extreme. CHARLES:: I'm just curious, just to kind of put a fine point on it, it sounds like there might be more in common, like a deeper camaraderie between this style of Ruby and the style encouraged by Elm, even though that on the surface, one is a dynamically typed object oriented language and the other is a statically typed functional language and yet, there's a deeper philosophical alignment that seems like it's invisible to 99% of the discussion that happens around these languages. PHILIP: Yeah, I think so. I think the categories we and this is something Richard Feldman talks. He's a member of the Elm community. He does a lot of talks and has a course also in Frontend Masters, which I highly recommend. But he often talks about the frame of the conversation is wrong because you have good statically typed languages and you have bad statically typed languages. You have good dynamic languages and you have bad dynamic languages. For all interpretations of good and bad, right? I don't want to start any wars here. I think one of the things that Elm and Ruby have in common is the creator. Matz designed Ruby because he wanted programming to be a joy, you know? And Evan created Elm because he wanted programming to be a delight. I think if you experience both of those, like developing in both of those languages, you gain a certain appreciation for what that means. It is almost undefinable, indistinguishable, although you can see the effects of it everywhere. In Ruby, everything is an object, including nil. In Elm, it's almost he's taken everything away. Evan's taken everything away that could potentially cause you to stumble. There's a lot to learn with Elm in terms of getting your head around functional mindset and also, working with types but as far as that goes, people often call it like the Haskell Light, which I think those are a disservice to Elm because it's got different goals. CHARLES:: Yeah, you can tell that. You know, my explorations with Elm, the personality of Elm is 100% different than the personality of Haskell, if that is even a programming term that you can apply. For example, the compiler has an identity. It always talks to you in the first person, "I saw that you did this, perhaps you meant this. You should look here or I couldn't understand what you were trying to tell me." Literally that's how the Elm compiler talks to you. It actually talks to you like a person and so, it's very... Sorry, go ahead. PHILIP: No, no, I think the corollary to that is the principle of the surprise in Ruby. You know, is there going to be a method that does this? You type it out and you're like, "Oh, yes it is," which is why things like inject and reduce are both methods in enumerable. You didn't choose one over the other. It was just like, "Let's make it easy for the person who's programming to use what they know best." I think as well, maybe people don't think about this as deeply but the level of thought that Evan has put into designing Elm is crazy, like he's thought this through. I'm not sure if I said this the last time but I went to a workshop in the early days in London, which is my kind of first real exposure to Elm and Evan was giving the workshop. Someone asked him, "Why didn't you do this?" and he was like, "Well, that might be okay for now but I'm not sure that would make so much sense in 10 years," and I was kind of like, "What?" Because JavaScript and that ecosystem is something which is changing like practically hourly and this is a guy that's thinking 10 years into the future. TARAS: You might have answered it already but I'm curious of what you think is the difference, maybe it just comes down to that long term thinking but we see this in JavaScript world a lot, which is this kind of almost indifference to APIs. It almost doesn't really matter what the API is for whatever reason, there seems to be a big correlation between the API that's exposed with the popularity of the tool. I think there are some patterns, like something that's really simple, like jQuery and React have become popular because of the simplicity of their APIs. What the flip side to that? What other ways can APIs be created that we see in JavaScript world. Because we're talking about this beautiful APIs and I can relate to some of the work that Charles has been doing and I've been doing microstates but I wonder like what would be just a brief alternative to that API, so it's kind of a beautiful API. PHILIP: I don't know if anyone is familiar with the series of essays 'Worse is Better' like East Coast versus West Coast, from Richard Gabriel. The problem is, I guess and maybe this is just my understanding over my paraphrase of it, I'm not too familiar with it but I think that good APIs take time and people don't have time. If someone launches a V1 at first and it kind of does the job, people will use that over nothing and then whenever they're happy with that, they'll continue to use it and develop it and ultimately, if she's market share and then that's just the thing everyone uses and the other guy's kind of left behind like, "This is so much better." I guess this is a question, I think it was after Wicked Good Ember, I happened to be on the same trend as Tom Dale on the way back to New York and we started talking about this. I think that's his big question. I think it's also a question that still has to be answered, which is, "Will Elm ever be mainstream? Will it be the most popular thing?" aside from the question of whether it has to be or not. For me, a good APIs good design comes from understanding the problem fully -- CHARLES:: And you can understand the problem fully without time. PHILIP: Exactly and often, what happens -- at least this is what happens in my experience with the production software that I've written -- is that you don't actually understand the problem until you've developed a solution for it. Then when you've developed a solution for it, often the pressures or the commercial pressures or an open source is [inaudible] the pressures of backwards compatibility, mean that you can never refactor your way to what you think the best solution is and often, you start from scratch and the reality is people are too far away with the stuff you wrote in the past about the thing you're writing now. Those are always kind of at odds. I think there are a lot of people that are annoyed with Elm because the updates are too slow, it relies on Evan and we want to have a pool request accepted. All of the things that they don't necessarily recognize like the absence of which make Elm an Elm, if you know what I mean. The very fact that Evan does set such a high standard and does want everything to go through his personal filters because otherwise, you wouldn't gain the benefits that Elm gives you. The attention is very real in terms of I want to shift my software now and it becomes easier then. I think to go to a language like JavaScript, which has all of the escape hatches that you need, to be able to chop and change, to edit, to do what you need to do to get the job done and let's be quite honest, I think, also with Elm, that's the challenge for someone who's not an expert level like me. Once you hit a roadblock, you'll say, "Where do I go from here?" I know if I was using JavaScript, I could just like hack it and then clients are happy and everything's fine and you know there's a bit of stuff in your code that you would rather wasn't but at the end of the day, you go home and the job's done. DAVID:: Have you had to teach Elm to other people? You and I did some work like I've seen you pair with someone and guide them through the work that they needs to get done. If you had a chance to do something like that with Elm and see how that actually happens, like how do developer's mind develops as they're working through in using the tool? PHILIP: Unfortunately not. I would actually love to go through that experience. I hope none of my developers are listening to this podcast but secretly, I want to push them in the direction of Elm on the frontend. But no, but I can at least make from my own perspective. I find it very challenging at first because for me, being a Ruby developer and also, I would never say that I understood JavaScript as much as I would have liked. Coming from dynamic language, no functional experience to functional language with types, it's almost like learning a couple of different things at the same time and that was challenging. I think if I were to take someone through it, I would maybe start with a functional aspects and then move on to the type aspects or vice versa, like try and clearly breakdown and it's difficult because those two are so intertwined at some level. Gary Bernhardt of Destroy All Software Screencast, I watch quite a bit of his stuff and I had sent him an email to ask him some questions about one of the episodes that he did and he told me that he done the programming languages course, I think it's on Coursera from Daniel Grossman, so [inaudible] ML which is kind of the father of all MLs like Haskell and also Elm. I find that really helpful because he broke it down on a very basic level and his goal wasn't to teach you ML. It was to teach you functional programming. It would be a very interesting exercise, I think. I think the benefit that Elm gave you is you get to experience that delight very quickly with, "Oh, it's broken. Here's a nice message. I fix the message. It compiles. Wow, it works," and then there's a very big jump whenever you start talking about the effects. Whenever you want to actually do something like HTTP calls or dealing with the time or I guess, the impure stuff you would call in the Haskell-land and that was also kind of a bit weird. CHARLES:: Also, there's been some churn around that, right? PHILIP: That's right. When I started learning, they had signals, then they kind of pushed that all behind the scenes and made it a lot more straightforward. Then I just mastered it and I was like, "Yes, I know it," and then I was like, "All right. I don't need to know it anymore." This is the interesting thing for me because at work, most of our work now is in Elixir and Phoenix. I'm kind of picking a little bit up as I work with them. I think Elm's architecture behind the scenes is kind of based, I believe on Erlang's process model, so the idea of a mailbox and sending messages and dealing with immutable state. CHARLES:: Which is kind of ironically is very object oriented in a way, right? It's functional but also the concept of mailboxes and sending messages and essentially, if you substitute object for process, you have these thousands and thousands of processes that are sending messages back and forth to each other. PHILIP: Yeah, that's right. It's like on a grand scale, on a distributed scale. Although I wouldn't say that I'm that far with Erlang, Elixir to appreciate the reality of that yet but that's what they say absolutely. CHARLES:: Now, Phoenix and Elixir is a dynamically typed functional language. does it share the simplicity? One of the criticisms you had of Rails was that you couldn't fit it in your head. It was very difficult. Is there anything different about Elixir that kind of makes it a spirit cousin of Elm and the simple Ruby? PHILIP: I think so, yes. Absolutely. I don't think it gets to the same level but I think it's in the right direction and specifically on the framework front, it was designed specifically... I mean, in a sense it's like the anti-type to Rails because it was born out of people's frustrations with Rails. José Valim was pretty much one of Rails top core committers. Basically, every Rails application I wrote at one period, at 80% of the code written by José Valim, if you included all the gems, the device and the resourceful and all the rest of it. Elixir in many ways was born out of the kind of limitations of Ruby with Rails and Phoenix was also born out of frustrations with the complexity of Rails. While it's not as simple as say, Michel Martens' Syro which is like his web framework, which is a successor to Cuba if people have heard of that, it is a step in the right direction. I don't understand it but I certainly feel like I could. They have plug which is kind of analogous but not identical to Rack but then the whole thing is built out of plugs. I remember Yehuda Katz give a presentation like 'The Next Five Years' and essentially about Rails 3.0. This is going way back and Phoenix is in some ways the manifestation of his desire to have like the Russian doll pattern, where you could nest applications inside applications and you could have them side by side and put them inside each other and things like that. Phoenix has this concept called umbrella applications which tells that, like Ecto is a really, really nice obstruction for working with the database. CHARLES:: I see. It feels like, as opposed to being functional or static versus dynamic, the question is how do you generate your complexity? How do you cope with complexity? Because I think you touched on it at the beginning of the conversation where you thought that my problems are complex so the systems that I work with to solve those problems must necessarily also be complex. I think one of the things that I've certainly realized, kind of in the later stages of my career is that first part is true. The problems that we encounter are extremely complex but you're much better served if you actually achieve that complexity by composing radically simple systems and recombining them. To the commonality of your system is going to determine how easy it's going to work with and how well it can cope with complexity. What really drives a good system is the quality of its primitives. PHILIP: Absolutely. After ElmConf, I actually invited Michel to come to my place in the Netherlands. He live in Paris but I think he grew up Buenos Aires in Argentina. To my amazement, he said, "Yes, okay," and we spent a couple of days together and there he talked to me about Christopher Alexander and the patterns book, where patterns and design patterns actually grew out of. One of his biggest things was the code is the easiest part, like you've got to spend 80% of your time thinking deeply about the problem, like literally go outside, take long looks. I'm not sure if this is what Rich Hickey means with Hammock Driven Development. I've never actually got around to watching the talk. CHARLES:: I think it's exactly what he means. PHILIP: And he said like once you get at, the code just comes. I think Michel's work, you should really check it out. I'll send you a link to put in the show notes but everything is built out of really small libraries that do one thing and do it really well. For example, he has a library like a Redis client but the Redis client also has something called Nest, which is a way to generate the keys for nested hashes. Because that's a well-designed, the Redis client is literally just a layer on top. If you understand the primitive then, you can use the library on top really well. You can embed Syro applications within Syro applications. I guess, there you also need the luxury of time and I think this is where maybe my role as VP of engineering, which is kind of my first role of that kind, comes in here which is when you're working on the commercial pressure, try to turn around to a business guy and say, "Yes, we'll solve this problem but can we take three weeks to think about it?" It's never going to happen -- CHARLES:: No. PHILIP: Absolutely, it will never going to happen. Although the small things that I tried to do day to day now is get away from the computer, write on paper, write out the problem as you understand it, attack it from different angles, think about different viewpoints, etcetera. CHARLES:: I think if you are able to quantify the cost of not thinking about it for three weeks, then the business person that you're going to talk to is their ears are going to perk up, right? But that's so hard to do. You know, I try and make like when we're saying like, "What technologies are you going to choose? What are the long term ramifications in terms of dollars or euros or whatever currency you happen to be in for making this decision?" I wish we had more support in thinking about that but it is kind of like a one-off every time. Anyway, I'm getting a little bit off track. PHILIP: No, not at all. This is a subject I love to talk about because we kind of had a few a bit of turbulence because we thought, maybe we should get product people in, maybe we should get them a product team going and what I find was -- and this is maybe unique to the size of the company -- that actually made things a lot more difficult because you got too many heads in many ways. Sometimes, it's better to give the developer all of the context so that he can think about it and come up with the best solution because ultimately, he's the only one who can understand. I wouldn't say understands the dollars and cents but he understands the cost implications of doing it in efficient ways, which often happens when you're working in larger teams. TARAS: One thing I find really interesting about this conversation is the definition of good is really complicated here. I've observed Charles work on microstates and I work with him, like I wrote a lot of the code and we got through like five or six iterations and at every point, he got better but it is so difficult to define that. Then when you start to that conversations outside of that code context and you start to introduce business into the mix, the definition of good becomes extremely complicated. What do you think about that? How do we define it in a way? Are there cultures or engineering cultures or societal cultures that have a better definition for good that is relevant to doing quality work of this? CHARLES:: That's a deep question. PHILIP: Wow. Yeah, a really, really deep question. I think often for business, like purely commercially-driven, money-oriented good is the cheapest thing that gets the job done and often that's very short term, I think. As you alluded to Charles, that people don't think about the cost of not doing the right things, so to speak in our eyes and also, there's a huge philosophical discussion whether our definition of good as programmers and people who care about our craft is even analogous to or equal to a good in a commercial context. CHARLES:: Yes, because ultimately and this is if you have read Zen in the Art of Motorcycle Maintenance, one of the things that Pirsig talks about is what is the definition of quality. How do we define something that's good or something that's bad? One of the definitions that gets put forward is how well something is fit to purpose. Unless you understand the purpose, then you can't determine quality because the purpose defines a very rich texture, a very rich surface and so, quality is going to be the object that maps very evenly and cleanly over that surface. When it comes to what people want in a program, they're going to want very different thing. A developer might need stimulation for this is something that's very new, this is something that's going to keep my interest or it's going to be keeping my CPU max and I'm going to be learning a whole lot. A solution that actually solves for that purpose is going to be a high quality solution. Also, this is going to be fast. We're going to be able to get to market very quickly. It might be one of the purposes and so, a solution that is fast and the purpose fits so it's going to be good. Also, I think developers are just self-indulgent and looking for the next best thing in something that's going to keep their interest, although we're all guilty of that. But at the same time, we're going to be the ones maintaining software, both in our current projects and collectively when we move to a new job and we're going to be responsible for someone else's code, then we're going to be paying the cost of those decisions. We both want to minimize the pain for ourselves and minimize the pain for others who are going to be coming and working in our code to make things long term maintainable. That's one axis of purpose and therefore, an axis of quality. I think in order to measure good and bad, you really have to have a good definition of what is the purpose of that surface is so rich but the more you can map it and find out where the contours lie, the more you're going to be able to determine what's good and what's bad. TARAS: It makes me think of like what is a good hammer. A sledgehammer is a really good hammer but it's not the right hammer for every job. CHARLES:: Right. TARAS: I think what you're saying is understanding what is it that you're actually doing and then matching your solution to what you're actually trying to accomplish. PHILIP: Yeah, absolutely and in my experience, we have a Ruby team building a Rails application. That's our monolith and then, we have a couple of Elixir teams with services that have been spun out of that. This isn't proven. This is just kind of gut feel right now and it is that Elixir is sometimes slower to develop the same feature or ship it but in the long term it's more maintainable. I haven't actually gotten dived into to React and all of the amazing frameworks that it has in terms of getting things up and running quickly but in terms of the full scale application, I still think 10, 11 years on, Rails has no equal in terms of proving a business case in the shortest time possible. CHARLES:: Yeah. I feel very similarly too but the question is does your development team approach the problem as proving a business case or do they approach the problem as I want to solve the set of features? PHILIP: Yes. Where I'm working at the moment, I started out just as a software developer. I guess, we would qualify for 37 signals or sorry... base camps definition of a calm company -- CHARLES:: Of a what company? PHILIP: A calm company. Sorry. They just released a new book and called 'The Calm Company' and 'It Doesn't Have to Be Crazy at Work.' I was given in my first couple of months, a problem. It was business oriented, it had to be solved but it had to be solved well from a technical perspective because we didn't want to have to return to it every time. It was standardizing the way that we exported data from the database to Excel. You know, I was amazed because it was literally, the first time that I'd been given the space to actually dive in on a technical level to do that kind of stuff. But I think even per feature, that varies and that sometimes challenging when handing the work on because you've got to say, "This fit. Literally, we're just trying to prove, whether if we have this feature, the people will use it?" versus, "This is a feature that's going to be used every day and therefore, needs to be at good, technical quality." Those are the tradeoffs that I guess, keep you in a job. Because if it was easy, then you would need anyone to figure it out but it's always a challenge. What I like is that our tools are actually getting better and I think, with Elm for example, it's kind of major selling point is maintainability and yet, with Elm, there haven't been that many companies with Elm over a period of years that exists, that can live to tell the tale. Whereas, we certainly know with Rails applications have done well like Basecamp and GitHub. For sure, they can be super maintainable but the fact that it took GitHub to just moved Elm to Rails 5.0, I belief, the fact that it took them years and years and they were running off at fork of Rails 2.3, I think it shows the scale of the problem in that way. You know, Phoenix also went through a few issues, kind of moving architectures from the classic Rails to a more demand driven design model. I think we're getting there slowly, zig-zagging towards a place where we better understand how to write software to solve business problems. I guess, I was really interested in microstates when you shared it at Wicked Good Ember because that to me was attacking the problem from the right perspective. It's like given the fact that the ecosystem is always changing. How can we extract the business logic such that these changes don't affect the logic of our application? CHARLES:: Man, we got a lot to show you. It has changed quite a bit in the last two years. Hopefully, for the better. TARAS: It's been reduced and it's almost a quarter of its size while maintaining the same feature set and it's faster, it's lazier, it's better in every respect. It's just the ideas have actually been fairly consistent. It's just the implementation that's evolved. CHARLES:: Yeah, it's been quite a journey. It parallels kind of the story that we're talking about here in the sense that it really has been a search for primitives and a search for simplification. One of the things that we've been talking about, having these Ruby gems that do one thing and do it very, very, very well or the way that Elixir being architected has some very, very good primitives or Elm, the same kind of thing being spiritually aligned, even though on the surface, it might share more in common with Haskell. There's actually a deep alignment with a thing like Ruby and that's a very surprising result. I think one of the things that appeals to me about the type of functional programming that is ironically, I guess not present in Elm, where you have the concept of these type classes but I actually think, I love them for their simplicity. I've kind of become disenchanted with things like Lodash, even though they're nominally functional. The fact that you don't have things like monoid and functors and stuff is kind of first class participants in the ecosystems, means you have to have a bunch of throwaway functions. Those API surface area is very large, whereas if you do account for those things, these kind of ways of combining data and that's how you achieve your complexity, is not by a bunch of one-off methods that are like in Lodash, they're all provided for you so you don't have or have to write them yourself. That is one level of convenience but having access to five primitives, I think that's the power of the kind of the deeper functional programming types. PHILIP: And Charles, do you think that that gives you the ability to think at a higher level, about the problems that you're solving? Would you make that link? CHARLES:: Absolutely. PHILIP: So, if we're not doing that, then we're actually doing ourselves a disservice? CHARLES:: I would say so. PHILIP: Because we're actually creating complexity, where it shouldn't exist? CHARLES:: Yeah, I think if you have a more powerful primitive, you can think of things like async functions and generator functions, there's a common thread between async functions, generator functions, promises arrays and they're all functors. For me, that's a very profound realization and there might be a deeper spiritual link between say, an async function and an array in the same way that there's a deep spiritual link between Ruby and Elm, that if you don't see that, then you're doing yourself a disservice and you're able to think at a higher level. Also, you have a smaller tool set where each tool is more powerful. PHILIP: You did a grit, I think it was a repository with a ReadMe, where you boiled down what people would term what I would term, the scary functional language down to a very simple JavaScript. Did you ever finish that? Did you get to the monads? CHARLES:: I did get to the monads, yeah. PHILIP: Okay. I need to check that out again. I find that really, really helpful because I think one of Evan's big things with Elm is he doesn't use those terms ever and he avoids them like the plague because I think he believes they come tinged with the negative experiences of people trying Haskell and essentially getting laughed at, right? CHARLES:: Yes. I think there's something to that. TARAS: But we're doing that in microstates as well, right? In microstates documentation, even though microstates are written completely with these functional primitives, on the outside, there's almost no mention of it. It's just that when you actually go to use it, if you have an idea, one of the thing that's really powerful with microstates is that this idea that you can return another microstate from a transition and what that will do is what you kind of like what a flat map would do, which is replace that particular node with the thing that you returned it with. For a lot of people, they might not know that that's like a flat map would do but a microstate will do exactly what they wanted to do when it didn't realize that's actually should just work like that. I think, a lot of the work that we've done recently is to package all things and it make it powerful and to access the concepts that it is very familiar, something you don't need to learn. You just use it and it just works for you. CHARLES:: Right but it is something that I feel like there's unharvested value for every programmer out there in these type classes: monads and monoids and functors and co-functors or covariant functors, contravariant functors, blah-blah-blah, that entire canon. I wish there was some way to reconcile the negative connotations and baggage that that has because we feel kind of the same way and I think that Evan's absolutely right. You do want to hide that or make it so that the technology is accessible without having to know those things. But in the same way, these concepts are so powerful, both in terms of just having to think less and having to write less code but also, as a tool to say, "I've got this process. Is there any way that could it be a functor? If I can find a way that this thing is a functor, I can just save myself so much time and take so many shortcuts with it." PHILIP: And in order to be able to communicate that, or at least communicate about that, you need to have terms to call these things, right? Because you can't always just refer to the code or the pattern. It's always good to have a name. I'm with you. I see value in both, like making it approachable, so the people who don't know the terms are not frightened away. But I also see value in using the terms that have always existed to refer to those things, so that things are clear and we can communicate about them. CHARLES:: Right. definitely, there's a tradeoff there. I don't know where exactly the line is but it would be nice to be able to have our cake and eat that one too. We didn't get really to talk about the type versus dynamic in the greater context of this whole conversation. We can explore that topic a little bit. PHILIP: Well, I can finish with, I think the future is typed Erlang. Maybe, that's Elm running on BEAM. CHARLES:: Whoa. What a take? Right there, folks. I love it. I love it but what makes you say that? Typed Erlang doesn't exist right now, right? PHILIP: Exactly. CHARLES:: And Elm definitely doesn't run on BEAM. PHILIP: I don't know if I'm allowed to say this. When I was at this workshop with Evan, he mentioned that and I'm not sure whether he mentioned it just as a throwaway comment or whether this is part of his 20-year plan but I think the very fact that Elm is designed around like Erlang, the signal stuff was designed around the way Erlang does communication and processes, it means I know at least he appreciates that model. From my point of view, with my experience with Elixir and Erlang in production usage, it's not huge scale but it's scale enough to need to start doing performance work on Rails and just to see how effortless things are with Elixir and with Erlang. I think Elm in the backend would be amazing but it would have to be a slightly different language, I think because the problems are different. We began this by saying that my story was a little different to the norm because I went back to the dynamic, at the dark side but for example in Elixir, I do miss types hugely. They kind of have a little bit of a hack with Erlang because they return a lot of tuples with OK and then the object. You know, it's almost like wrapping it up in a [inaudible]. There are little things and there's Dialyzer to kind of type check and I think there are a few projects which do add types to Erlang, etcetera. But I think something that works would need to be designed from the ground up to be typed and also run in the BEAM, rather than be like a squashed version of something else to fit somewhere else, if that makes sense. CHARLES:: It makes total sense. PHILIP: I think so. I recently read a book, just to finish which was 'FSharpForFunAndProfit' is his website, Scott Wlaschin, I think. It's written up with F# but it's about designing your program in a type functional language. Using the book, you could probably then just design your programs on paper and only commit to code at the end because you're thinking right down to the level of the types and the process and the pipelines, which to me sounds amazing because I could work outside. CHARLES:: Right. All right-y. I will go ahead and wrap it up. I would just like to say thank you so much, Philip for coming on and talking about your story, as unorthodox as it might be. PHILIP: Thank you. CHARLES:: Thank you, Taras. Thank you, David. TARAS: Thank you for having us. CHARLES:: That's it for Episode 113. We are the Frontside. This is The Frontside Podcast. We build applications that you can stake your future on. If that's something that you're interested in, please get in touch with us. If you have any ideas for a future podcast, things that you'd like to hear us discuss or any feedback on the things that you did here, please just let us know. Once again, thank you Mandy for putting together this wonderful podcast and now we will see you all next time.
Guests: Amanda Hickman: @amandabee | GitHub Amberley Romo: @amberleyjohanna | GitHub | Blog In this episode, Amanda Hickman and Amberley Romo talk about how they paired up to get the safety pin, spool of thread, and the knitting yarn and needles emojis approved by the Unicode Committee so that now they are available for use worldwide. They also talk about how their two path crossed, how you can pitch and get involved in making your own emojis, and detail their quest to get a regular sewing needle approved as well. Resources: Unicode Technical Committee Draft Emoji Candidates The Unicode Consortium Members Sewing-Emoji Repo Proposal for Sewing NEEDLE AND THREAD Emoji This show was produced by Mandy Moore, aka @therubyrep of DevReps, LLC. Transcript: ROBERT: Hello, everyone. Welcome to Episode 112 of The Frontside Podcast. I'm Robert DeLuca, a software developer here at the Frontside and I'll be your episode host. With me as co-host is Charles Lowell. Hey, Charles. CHARLES: Hello, Robert. Good morning. ROBERT: Good morning. This is an exciting podcast. Today, we're going to be discussing writing a proposal to the Unicode Committee, getting it accepted and rejected. This is basically making emojis which I think is really awesome. We have two guests today who have an amazing story, Amanda Hickman and Amberley Romo. Thank you both for joining us. You two have an amazing story that I would really love to dive into and we're going to do that today. It's about basically creating your own emoji and getting that accepted so everybody can use that and I think that's super, super cool, something that I've always kind of wanted to do as a joke and it seems like that's kind of where your stories began, so you two want to jump in and start telling? I think Amanda has a great beginning to this. AMANDA: Sure. I mean, hi and thanks for having me. I don't know where to begin and really for me, this starts with learning to sew my own clothes which is an incredibly exasperating and frustrating process that involves a lot of ripping stitches back out and starting over and Instagram was a really big part of me finding patterns and finding other people who are sewing their own clothes and learning from the process. I wanted to be able to post stuff on Instagram and it started to drive me absolutely crazy, that there's emojis for wrenches and nuts and hammers and there are no textile emoji. The best I could find was scissors which is great because cutting patterns is a place where I spend a lot of time procrastinating but that was it. I knew a woman, Jennifer 8 Lee or Jenny who had led a campaign to get the dumpling emoji into the Unicode character set. I knew she'd succeeded in that but I didn't really know much more about how that had worked. I started thinking I'm going write a sewing emoji. I can do this. I can lead this campaign. I started researching it and actually reached out to Jenny and I discovered that she has created an entire organization called... What was that called? She's created an entire organization called Emojination, where she supports people who want to develop emoji proposals. CHARLES: Before you actually found the support system, you actually made the decision that you were going to do this and you found it. You know, from my perspective, I kind of see emoji is this thing that is static, it's there, it's something that we use but the idea that I, as an individual, could actually contribute to that. I probably, having come to that fork in the road would have said, "Nah, it's just it is what it is and I can't change it." What was the process in your mind to actually say, "You know what? I'm actually going to see if I can have some effect over this process?" AMANDA: It definitely started with a lot of anger and being just consistently frustrated but I knew that someone else had already done this. It was sort of on my radar that it was actually possible to change the emoji character set. I think that if I didn't know Jenny's story and it turned out I didn't know Jenny story at all but I thought I knew Jenny story but if I didn't know that basic thing that that somebody I knew who was a mere mortal like me had gone to the Emoji Subcommittee of the Unicode Consortium and petition them to add a dumpling emoji, I am sure that I wouldn't bother. But I knew from talking to her that there was basically a process and that there were a format that they want proposal in and it's possible to write them a proposal. I knew that much just because I knew Jenny. I think at that point, when I started thinking about this, the Emoji 9 -- I should be more of an expert on that actually, on emoji releases but a new release of emoji had come out. There were a bunch of things in that release and it got a little bit of traction on Twitter. I knew that the Unicode Consortium had just announced a whole new slate of emoji, so I also was generally aware that there was some kind of process by which emoji were getting released and expanded and updated. ROBERT: That's interesting. Do you know when that started? Because it seems like Apple started to add more emojis around like iOS 7 or something but it was pretty static for a while right? Or am I wrong? AMANDA: I actually am tempted to look this up but the other piece that is not irrelevant here is that at the time, I was working at a news organization called BuzzFeed that you may have heard of -- ROBERT: Maybe, I don't know. It sounds kind of familiar. AMANDA: I do feel like people kind of know who they are. I was surrounded by emoji all the time: in BuzzFeed, in internet native of the highest order and we had to use emoji all the time and I had to figure out how to get emoji into blog post which I didn't really know how to do before that. I can put them on my phone but that was it. I was immersed in emoji already. I knew that there was a project called Emojipedia, that was a whole kind of encyclopedia of emoji. One of my colleagues at BuzzFeed, a woman named Nicole Nguyen had written a really great article about the variation in the dance emoji. If you look at the dance emoji, one of the icons that some devices use is this kind of woman with her skirt flipping out behind her that looks like she's probably dancing a tango and then one of the icons that other character sets use and other devices use is a sort of round, yellow lumping figure with a rose in its mouth that you sort of want to hug but it's definitely not to impress you with its tango skill. She had written this whole article about how funny it was that you might send someone this very cute dumpling man with [inaudible] and what they would see was sexy tango woman. I think there was some discussion, it was around that time also that Apple replaced the gun emoji with a water gun. There was some discussion of the direction that the various emoji's face. One of the things that I learned around that time was that every device manufacturer produces their own character set that's native to their devices and they look very different. That means that there's a really big difference between putting a kind of like frustrated face with a gun pointing at it, which I don't really think of it as very funny but that sort of like, "I'm going to shoot myself" is very different from pointing the gun the other way which is very much like, "I'm shooting someone else," so these distinctions, what it means that the gun emoji can point two different ways when it gets used was also a conversation that was happening. None of that answers your question, though which is when did the kind of rapid expanse of emoji start to happen. ROBERT: I feel like the story is setting in the place there, though because it seems like there's a little bit of tension there that we're all kind of diverging here a little bit and it's sort of driving back towards maybe standardization. AMANDA: There's actually, as far as I know, no real move toward standardization but the Unicode Consortium has this committee that actually has representatives from definitely Apple and Microsoft and Google and I forget who else on the consortium. Jenny 8 Lee is now on the consortium and she's on the Emoji Subcommittee but they actually do get together and debate the merits of adding additional emoji, whether they're going to be representative. One of the criteria is longevity and I tend to think of this as the pager problem. There is indeed a pager emoji and I think that the Unicode Consortium wants to avoid approving a pager emoji because that was definitely a short-lived device. CHARLES: Right. I'm surprised that it actually made it. Emoji must be older than most people realize. AMANDA: My understanding is that very early Japanese computers had lots emoji. There's a lot of different Japanese holidays that are represented in emoji, a lot of Japanese food as well are represented in emoji, so if you look through the foods, there's a handful of things that haven't added recently but a lot of the original emoji definitely covered Japanese cuisine very well. ROBERT: I definitely remember when I got my first iPhone that could install iPhone OS 2, you would install an app from the App Store that then would allow you to go toggle on the emoji keyboard but you had to install an app to do it and that's kind of where the revolution started, for me at least. I remember everybody starting to sending these things around. AMANDA: But if you look at Emojipedia, which has a nice kind of rundown of historical versions of the Unicodes, back in 1999, they added what I think of as the interrobang, which is the exclamation/question mark together and a couple of different Syriac crosses. Over the years, the committee has added a whole series of wording icons and flags that all make sense but then, it is around, I would say 2014, 2015 that you start to get the zipper mouth and rolling your eyes and nerd face and all of the things that are used in conversation now -- the unicorn face. ROBERT: My regular emojis. AMANDA: Exactly. CHARLES: It certainly seems like the push to put more textile emoji ought to clear the hurdle for longevity, seeing there's kind of like, what? Several millennia of history there? And just kind of how tightly woven -- pun intended -- those things are into the human experience, right? AMANDA: Definitely. Although technically, there's still no weaving emojis. CHARLES: There's no loom? AMANDA: There's no loom and I think that a loom would be pretty hard to represent in a little 8-bit graphic but -- CHARLES: What are the constraints around? Because ultimately, we've already kind of touched on that the emoji themselves, their abstract representations and there are a couple of examples like the dancing one where the representation can vary quite widely. How do they put constraints around the representation versus the abstract concept? AMANDA: You don't have to provide a graphic but it definitely kind of smooths the path if you do and it has to be something that's representable in that little bitty square that you get. It has to be something representable in a letter-size square. If it's not something that you can clearly see at that size, it's not going to be approved. If it's not something you can clearly illustrate at that size in a way that's clearly distinct from any other emoji and also that's clearly distinct from anything else of that image could be, it's not going to be approved. Being able to actually represented in that little bitty size and I don't know... One of sort of sad fact of having ultimately worked with Emojination on the approval process is that we were assigned an illustrator and she did some illustrations for us and I never had to look at what the constraints were for the illustration because it wasn't my problem. ROBERT: Sometimes, that's really nice. AMANDA: Yes, it's very nice. I ended up doing a lot of research. What made me really sad and I don't want to jump too far ahead but one of things that made me really sad is we proposed the slate and the one thing that didn't get approved was the sewing needle and it also didn't get rejected, so it's in the sort of strange nether space. That's kind of stuck in purgatory right now. I did all this research and learned that the oldest known sewing needle is a Neanderthal needle so it predates Homo sapiens and it's 50,000 years old. CHARLES: Yeah. Not having a sewing needle just seem absurd. AMANDA: Yeah. We have been sewing with needles since before we were actually human being. ROBERT: That's a strong case. AMANDA: Yes, that's what I thought. If I sort go back to my narrative arc, I wanted to do a sewing needle and started researching it a little bit -- CHARLES: Sorry to keep you interrupting but that's literally the one that started this whole journey. AMANDA: Yes, I wanted a sewing needle and I really wanted a sewing needle. I did a little research and then I reach out to Jenny and to ask her if she had any advice. She said, "You should join my Slack," and I was like, "Oh, okay. That's the kind of advice." She and I talked about it and she said that she thought that it made more sense to propose a kind of bundle of textile emoji and I decided to do that. She and I talked it through and I think the original was probably something closer to knitting than yarn but we said knitting, a safety pin, thread and needle were the ones that kind of made the most sense. I set about writing these four proposals and one of the things that they asked for was frequently requested. One other thing that I will say about the proposal format is that they have this outline structure that is grammatically very wonky. They ask you to assert the images distinctiveness and they also ask you to demonstrate that it is frequently requested. I found a couple of really interesting resources. One, Emojipedia which is this sort of encyclopedia of emoji images and history maintains a list of the top emoji requests. I actually don't know how they generate that list or who's requesting that and where but I think it's things that they get emailed about and things people request in other contexts and sewing and knitting, I've done on that list and I started compiling it in 2016. ROBERT: To be a part of the proposal process, to show that it is requested, without that resource, you just start scouring Facebook and Twitter and history and shouting to people like, "I really want this emoji. Why it didn't exist?" That seems pretty hard. AMANDA: Actually our proposals all have Twitter screen shots of people grousing about the absence of knitting emoji and yarn emoji and sewing emoji. I know that Emojipedia, they do a bunch of research so they go out and look at based what people are grousing about on Twitter. They look at places where people are publicly saying like, "It's crazy that there's no X emoji," and that's part of their process for deciding what kinds of emojis people are asking for. Their research was one resource but we took screenshots of people saying that they needed a safety pin emoji and that was part of making the case. One of the things that I found as I was doing that research was that, I guess at this point it was almost two years ago, when the character set that included the dumpling emoji came out, there was a bunch of grousing from people saying, "Why is there not a yarn emoji?" There was a writing campaign that I think Lion Brand had adopted. Lion Brand yarn had put in this tweet saying like, "Everyone should complain. We needed a yarn emoji," but it doesn't matter how much you yell on Twitter. If you don't actually write a proposal, you're not going to get anywhere. I had been told that the Emoji Subcommittee, they're really disinclined to accept proposals that had a corporate sponsor, so they weren't going to create a yarn emoji because Lion Brand yarns wanted them to create a yarn emoji. ROBERT: Right, so it was like counter-peer proposal. AMANDA: Right. But as I was digging around the other thing I found was this woman in... I actually don't know if you're in Dallas or Austin but I found Amberley, who also put a post on Twitter and had started a petition, asking people to sign her petition for a yarn emoji proposal or a knitting emoji. I don't remember if it was a yarn emoji or a knitting emoji but I found her petition and reached out to her to ask if she was interested in co-authoring the proposal with me because she had clearly done the work. She actually had figured out how the system worked at that point. I think she knew who she was petitioning, at least. I reached out to Amberley and we worked together to refine our proposal and figure out what exactly we wanted to request. I think there were a bunch of things that were on the original list like knitting needles, yarn and needles. I think crocheting would have been on the original list. We were sort of trying to figure out what was the right set of requests that actually made sense. ROBERT: So then, this is where Amberley stories comes in and it is interesting too because she has entirely different angle for this. Maybe not entirely different but different than outright. This kind of ties back to the word software podcast mostly. It kind of ties back to the software aspect, right? AMBERLEY: Yeah. I think, really they're kind of separate stories on parallel tracks. My motivation was also two-fold like Amanda's was, where I started knitting in 2013 and I had a really good group of nerd friends with a little yarn shop up in DC, like a stitch and ditch group -- ROBERT: I love it. AMBERLEY: It was a constant sort of like, where's the insert emoji here, like where's the yarn emoji? Where's the knitting emoji? And we would sort of sarcastically use the spaghetti emoji because it was the most visually similar but that was something that was in the back of my mind but it teaches you a lot about yourself too because I was like, "Oh, this is like fiber art, not really an emoji. It's kind of technical, like on a tech space," and I didn't really connect that it was relevant or that I might have any power to change it. It just didn't occur to me at the time. ROBERT: Interesting. I feel like a lot of people are in that similar situation or maybe not situation, even though you can make change on this. AMBERLEY: Right, so my brain didn't even make it like, "Why isn't this a thing? let me look at how to make the thing." When that happened for me, Amanda mentioned using emoji and everything in the BuzzFeed space. I love how you explained BuzzFeed a while ago, it's my favorite description of BuzzFeed I ever heard. Something similar that happened for me was I was a software developer and in 2016, the Yarn package manager was released and that kind of turned something on in my head. That was like I'm seeing all these software engineers now be like, "Where's the yarn emoji?" and I'm like, "Welcome to the club." ROBERT: "Do you want to join our Slack? We can complain together." AMBERLEY: Right. It has been like a pretty decent amount of time, I'm semi-seriously ranting and complaining to my coworkers who were primarily male software engineers. I remember I went to [inaudible] in the Frost Bank Tower after work and was just like, "I'm going to figure out how this happens," and I spent a couple hours at the coffee shop. I found the Unicode site and I found their proposal process and their structure for the proposal and everything and I just started doing the research and drafting up a proposal specifically for yarn. Maybe it was a bit naive of me but to me it was like, "Okay, here's the process. I follow the process. Cool." I mean, you have to make a case and it has to be compelling and has to be well-written and it has to be supported and all that and that to me it was like, "Okay, there's a process. At the same time, I did read about the dumpling emoji but I didn't connect it to Emojination and they had started the Kickstarter. We should talk about this later but I think the sort of idea the issue of representation on the committee and who gets to define language is really interesting but I saw that they had done the Kickstarter and there was a campaign aspect to it, so I ended up just building up this simple site so that if anyone Google, they would find yarn emoji. It's still up at YarnEmoji.com and that was how Amanda found me. I got this random email, I sort of like had this burst of energy and I did all the research and I wrote the draft, sort of piecemeal, filling out the different sections of the way they have it outlined on the Unicode site and then I feel like a month or two went by and I had kind of not looked at it for a bit and then, I get this random email from this website that I almost forgot about. It was like, "Hey, I'm working on this series of proposals. If you're working on knitting or yarn or whatever, maybe we could work together," and I was like, "Well, that's sweet." Then she opened up this whole world to me. There's this whole Emojination organization, sort of 100% devoted to democratizing the process of language formation through creating emojis and so then, I got really into that. My primary motivator was yarn. CHARLES: So what's the status of the yarn spool, those emoji right now? AMBERLEY: The yarn, the spool of thread and the safety pin, they're all approved emoji for the 2018 released. Amanda and I are actually at the end. Amanda, a couple of months ago when I saw someone used the spool thread emoji for a Twitter thread -- you know how people will be like all caps thread and have a thread of tweets -- I saw someone do that just out of the blue. I was like, "Oh, my God. Is it out?" and the thing about these individual vendors, it sort of gets released piecemeal, so at the time Twitter have I think released their versions of this series of new emoji but others hadn't. CHARLES: How does that work? Because you think the Twitter would be kind of device depending on what browser you're using, like if you're on a Windows or a Mac or a Linux Box, right? ROBERT: -- Emoji set, right? I know Facebook does this too. AMBERLEY: I'm painfully aware that Facebook does it because I can't use the crossed finger emoji on Facebook because it actually gives me nightmares. ROBERT: I have to go look at this now. AMBERLEY: Because it's so creepy-looking. CHARLES: Okay. Also like Slack, for example is another. It's like a software-provided emoji set. AMANDA: Right. AMBERLEY: I'm not totally sure that Slack actually adheres to the standard Unicode set. I think it's kind of its own thing but I might be wrong about that. AMANDA: Sorry, Slack definitely supports the full Unicode set. They also have a bunch of emoji that they've added that aren't part of the set. AMBERLEY: Slack emojis? AMANDA: Yes. CHARLES: Yeah and then every Slack also has its kind of local Slack emoji. AMBERLEY: Right. CHARLES: But how does that work with --? ROBERT: Okay, this crossed-finger Facebook emoji is... yes, I agree with you, Amberley. AMBERLEY: Thank you. I had yet to find someone who disagrees with me about that. AMANDA: I have never seen it before and I'm now like, "What is going on?" CHARLES: Yeah, so how does it work if a vendor like Twitter is using a different emoji set? How does that work with cut and paste, like if I want to copy the content of one tweet into something else? Are they using an image there? AMANDA: They're using an image. I think it's doesn't happen as much anymore but for a long time, I would often get texts from people and the text message would have that little box with a little code point in it and you were like -- AMBERLEY: More like an alien thing? AMANDA: Yeah. Definitely, if you don't have the emoji character set that includes the glyph that you're looking at, you're going to get that little box that has a description of the code point and I think what's happening is that Twitter is using JavaScript or generally programming. There were air quotes but you can't see. Twitter is using their software to sub in their emoji glyph whenever someone enters that code point. Even if you don't have the most up to date Unicode on your computer, you can still see those in Twitter. If I copy and paste it into a text editor on my computer, what I'm going to see is my little box that says '01F9F5' in it but if I get it into Twitter, it shows up. I can see them on Twitter but I can't see them anywhere else. AMBERLEY: Damn, you really have the code point memorized? AMANDA: No, I -- CHARLES: Oh, man. I was really hoping -- AMBERLEY: Oh, man. ROBERT: You live and breathe it. AMANDA: No, I'm not that compulsive. AMBERLEY: We definitely have our emojis on our Twitter bios, though. AMANDA: Absolutely. ROBERT: If you see Amanda's bio, it's pretty great. AMANDA: They started showing up on Twitter and I think that somebody in Emojination probably told me they were out and that was when I first started using them. Amberley might have actually seen it. It sounds like you just saw it in the wild, which is kind of amazing. AMBERLEY: I saw it in the wild with this tweet thread and yeah, it's just [inaudible]. I was like, "Amanda, is it out?" CHARLES: Yeah, I feel like I saw that same usage too, although I obviously did not connect any dots. AMANDA: This last week, October 2nd -- I'm also looking things up. I'm just going to come to the fact that I am on a computer looking things up so I can fact check myself -- after they actually released their emoji glyph set, so by now any updated iOS device should have the full 2018 emoji, which in addition to a kind of amazing chunky yarn and safety pin, there's also a bunch of stuff. There's a broom and a laundry basket. There's a bunch of really basic, kind of household stuff that certainly belongs in the character set alongside wrenches and hammers. AMBERLEY: I think one of the big ones too for this year was the hijab? AMANDA: No, the hijab actually came out with a dumpling. Hijab has been available -- AMBERLEY: It's been up, okay. ROBERT: So did it come with iOS 12 or 12.1? I don't know for sure. I just know -- AMANDA: I'm looking at it and it's 12.1. I really feel that I should be ashamed that I have used the internet and search for this. AMBERLEY: I would say, I have no idea what their release numbers are. AMANDA: [inaudible] as it appeared for the first time in iOS for 2018 with today's release of the iOS 12.1, Beta 2 for developers. ROBERT: That is amazing. Do you get some kind of satisfaction -- like you have to, right? -- from people using the emoji and it's starting to make its way out there? AMANDA: So much. Oh, my God, yeah. AMBERLEY: I didn't really expect it, like saying that random tweet using this spool of thread for a tweet thread. I just thought and I just got so psyched. For me, I'm a knitter. I have knitter friends and it started with yarn and then really, Amanda and through Amanda, Jenny really sort of broadened my idea of what it all really meant. To think someone using it in the wild for a totally different application than I had ever thought of was like, "That's legit." AMANDA: I definitely have a sewing emoji search in my tweet deck and sometimes, when I'm feeling I need a little self-validation, I'll go look over there and find people who are saying things like, "Why is there no sewing emoji?" and I'll just reply with all the sewing emoji, like it is part of my work in this life to make sure that not only do they exist but people know about them. ROBERT: That is awesome. I would do the same thing, though to be honest. You'll be proud of that. AMANDA: Totally. ROBERT: Were there any hitches in the proposal process? I know we're kind of alluded to it but the thing that you started off one thing, Amanda didn't make it. Right? AMANDA: I know. ROBERT: So how did that process happen from you two meet each other and then going through the actual committee and the review process and then being accepted. What would that mean? AMANDA: The process is actually incredibly opaque. We wrote this whole proposal, a bunch of people edited it, which is one of the other nice things about collaborating with Emojination. There was a bunch of people who are just really excited about emoji and the kind of language making that Amberley was talking about. There's a whole bunch of people who just jumped in and gave us copy edits and feedback, which was super helpful and then, there was a deadline and we submitted it to the committee and it actually shows up in the Unicode register which is also a very official kind of document register. I was a little excited about that too but then they have their meeting. They first have a meeting and there's like a rough pass and the Emoji Subcommittee makes formal recommendations to the Unicode Consortium and then the consortium votes to accept or reject the Emoji Subcommittee's recommendations. It's a very long process but unless you're going and checking the document file and meeting minutes from the Unicode Consortium meetings, you'll never going to know that it happen. AMBERLEY: -- You know someone connected through there because one of the things in our first pass, it wasn't that it was rejected. It was that we needed to modify something. We do have art for knitting needles with yarn because at one point, I think we weren't totally sure that a ball of yarn would be visually distinct enough in this emoji size to look like yarn and so, we had put it with sort of knit piece on knitting needles. AMANDA: Oh, that's right. There was a tease of a little bit of knitted fabric. AMBERLEY: Right and I think that, probably through Jenny or the people actually in the room, the feedback I remember is that there is a crocheter in the room who was like, "Yeah, why isn't there a yarn emoji but knitting needle?" so there was a little bit of like that was how I think we ended up from knitting needles with a fiber piece to ball of yarn, maybe. AMANDA: I think that sounds right. I'm actually sure of that. It's just all coincide with my recollection. There were some things that they had questions about and that happened really fast because I feel like we had a couple of days and they have stuck to our guns and said, "No, we're only interested in knitted bit of fabric." Also, we worked with an illustrator and went back and forth with her because the initial piece that she had illustrated, I feel like the knitting needles were crossing in a way. That was not how knitting works and so, there was a little bit of back and forth around that as well. But then once they decided that the they like the thread, yarn and safety pin, we're going to move to the next stage. I actually had to go back and look at the minutes to find out that the two reasons that they didn't move the sewing needle on to the next stage is when they thought it was adequately represented by the thread, which I wholeheartedly disagree with and they thought it wasn't visually distinctive. That's so much harder because a sewing needle, which is really just a very fine piece of metal with an eye at the end, you get down to a really small size and it is maybe a little hard to know what you're looking at. But I think there's such a big difference between the static object which is the spool or the thread which represents a lot of things and is important and the needle, which is the active tool that you use to do the making, to do the mending, to do the cobbling. CHARLES: Yeah. I'm surprised that it almost isn't reversed when certainly in my mind, which I think is more culturally important in terms of the number of places which it appears, it's definitely the needle as being kind of... Yeah. AMANDA: Yeah and I think that the thread and yarn, they're important and I think that the decision to have a ball of yarn rather than a bit of knitting makes sense because there's a lot of things that you can use a ball of yarn that aren't just knitting and they think that -- AMBERLEY: And it's the first step too that doesn't exclude anyone in the fiber art community. AMANDA: But there's so many things like in sutures and closing wounds, you're not using a little spool of cotton thread for that or polyester thread and stuff like embroidery and beadwork, you might be using thread or fiber of some sort that started on a spool but you might not. Embroidery floss was not sold in a spool and there's all these places where we use needles and all kinds of different size and you don't always use thread. Sometimes, you're using yarn. Sometimes, you're using leather cord. Sometimes, you're using new bits of, I would say Yucca. You're using plant fibers to do baskets and in all of these different practices, that process of hooking it through the eye and sewing it is how it's actually made. It still sort of mystifies me why they haven't accepted it but they also didn't reject it, which is really interesting. I don't know how many other emoji are sort of sitting in this weird nether space because sometimes they just reject them outright. I think there was a proposal for a coin that they just said no. ROBERT: They were a like, "A coin?" That would be [inaudible]. AMANDA: Oh, God. ROBERT: They have to add one for every -- AMANDA: [inaudible]. CHARLES: Literally, the pager of 2017. AMANDA: Exactly. CHARLES: So what recourse is now available to you all and to us, by extension, to get the sewing needle? AMANDA: I'm actually working on a revised proposal and I've been trying to figure out what are all the arguments that I'm missing for why sewing and the needle are not adequately represented by the thread and yarn. A bunch of things that a friend of my named, Mari who's half-Japanese, half-American but lives in Guatemala and does all this kind of arts in textile work, pointed out that there's a whole holiday in Japan devoted to bringing your broken needles and thanking them for their service. I thought that was really cool. I've been trying to formulate what are all of the arguments for the necessity of both a needle and a spool. If anybody has interesting ways to phrase that, I would love for arguments. CHARLES: Yeah but it's hard to imagine the arguments is just anything being more compelling than the arguments the you just laid out that you named about seven context: shoemaking, medicine, different fibers where the needle operates completely and totally independent of the thread. It's looming so large in kind of our collective conscious like holidays, being dedicated to them, except I think the Cro-Magnon pager, which is made out of stone, I believe, the being the artifact that pre-dates... AMANDA: There's the idiom landscape as well. Things like finding a needle in a haystack, that has a very specific meaning -- ROBERT: And for puns. I've been resisting saying a pun this whole time. AMANDA: Oh, share your pun with us. AMBERLEY: Yeah, you have to say it. ROBERT: Well, you could say that trying to get this through the committee is like threading a needle. Butchered but -- AMANDA: There's a biblical quote about getting into heaven -- a camel through the eye of a needle. I forget actually how it... CHARLES: To thread a camel through the eye of a needle than for a rich man to get into heaven. AMANDA: Exactly and there's this sort of do-re-mi, saw, a needle pulling thread. There are all these places where it's about the needle and somebody had -- CHARLES: It's primarily ancient. AMANDA: I know. CHARLES: It is the prime actor. Maybe, this is a good segue into kind of talking about the makeup of the committee and the decision making process and these kind of what seem like very clear arguments might not be received as such. AMANDA: I certainly don't want to say anything bad about anybody on the committee. CHARLES: No, no, I don't think that there's anything bad. I think that being receptive to things which are familiar to us versus with things that aren't is a very natural human thing and it can be interesting to see that at work and at play. AMANDA: The Unicode Consortium is also evaluating all of these requests for whole language glyphs sets. Lots of languages and lots of character sets that are kind of obvious, like there has to be a sort of like character set like there has to be an Arabic character set but there are a lot of languages that have been left out of that because they're very small minority languages or they are historical languages, where the actual writing is no longer written the same way but there's historical reasons to be able to represent those characters. One of the reasons why the Emoji Subcommittee cares about what gets into the formal character set is that everybody has to accommodate it and there's already been, I think some grousing. People start to moan and groan about how there's too many emoji, then it's too hard to find things. CHARLES: And there's no take backs. AMANDA: There's no take backs. You can't undo it. The committee is made up of representatives from a lot of tech companies primarily, although there's a couple of other kind of odd additional folks on there. I do try to find the committee list and I can find it right now. AMBERLEY: I have it from Emojination. I don't know if it's up to date but Oracle, IBM, Microsoft, Adobe, Apple, Google, Facebook, Shopify and Netflix. The other voting members -- ROBERT: Shopify? AMBERLEY: Yeah, right? The others being the German software company, SAP and the Chinese telecom company, Huawei and the Government of Oman. AMANDA: Yeah, the Government of Oman is a fascinating one. I don't think they're the ones that are biting us on this. Especially for those tech companies, every time the emoji character set adds 10 or 12 emoji, they don't have to accommodate it on their devices. They have to put illustrators on it, they have to deal with everyone saying that the crossed fingers emoji in Facebook looks like I-don't-even-know-what. AMBERLEY: Hey, Amanda. AMANDA: It's all your fault. There's a whole process and there's non-trivial work associated with every single new emoji, so wanting to put the brakes on a little bit and be intentional about where and when they apply that work, it doesn't seem crazy to me. I just want them to approve the thing that I want. AMBERLEY: I like the way that Emojination captures it. I looked at their website earlier and actually, they take it down but their goal quote "Emojination wants to make emoji approval an inclusive representative process." There has to be a process. There's overhead involved but looking at the makeup of the decision makers are not a trivial question. CHARLES: Right. This is a great example like [inaudible] metaphor but these little artifacts, these emojis are literally being woven to the fabric of a global culture and certainly, everybody uses them and they become part of the collective subconscious. It does seem like very important to be democratic in some way. It sounds like there is a process but making sure that everyone has a stake. AMBERLEY: Yeah. ROBERT: What was the reason that they gave for not accepting the needle and thread? Was it like a soft no? You said it's like just hanging out, not really rejected but not accepted. We're going to drop a link in the show notes for the proposal and your GitHub and everything. I'm looking at the PDF that was put together and it seems like it was all a package deal like we talked about. How do they just draw or they just take like a lawyer would, just like draw or cross it out like, "Well, no but we'll take the other ones." AMANDA: Yes, basically. What they did is they need to discuss and I don't know how long they've been meeting but they need to discuss all of the proposals that have been supplied by a particular deadline and -- ROBERT: That sounds painful. AMANDA: Yeah, I mean, it's -- ROBERT: Just imagine the power of thinking about emojis. AMANDA: One of the things that they rejected, I think because there's the smiling poo face. Somebody wanted a frowning poo face and they rejected that. There's a bunch of things that actually do get rejected. I don't know if they've been really care about a smiling poo face versus a frowning poo face. ROBERT: What about an angry one? AMANDA: We got all the feelings of poo. ROBERT: We got important work to do here. AMANDA: But they go through when they're trying to figure out. I think to some degree, you want to get them when they're not tired but I think the status that it's listed right now is committee pushback, so they've set it aside until we have some concerns. We're not going to reject it outright but we're not really sure why this isn't adequately represented. Then their most recent meeting, they just kind of passed on reconsidering it, which is fine because I think I was traveling and my proposal is not done. I really want to make sure that I have consolidated every imaginable argument in one place so -- ROBERT: And make it strong as possible. AMANDA: Yeah. If people want to help the other thing that would be amazing is any and all idioms that you can think of, especially ones that are not in English or European languages, idioms in Central European languages, idioms in Asian languages that refer to needles, either translations of the kind of classic, 'finding a needle in a haystack,' but also any idioms that are kind of unusual and specific to a culture outside of what I have experience with would be amazing for making the case, so this is an international need. ROBERT: Do they need any specific or actionable feedback or do they just say, "We're going to push back on this. We're just not quite sure?" AMANDA: The two things that we're in the minutes -- there are minutes and they publish the minutes to Unicode.org -- were it was not visually distinct, which is not totally crazy. We actually worked with an illustrator to get a different image. The first image was almost at 90 degrees. It was kind of straight up and down and it is a little hard to see and the second is -- ROBERT: Especially, because it's thin. AMANDA: The second image is actually a kind of stylized needle because it's fairly a little fatter and the eye is bigger but it's much more distinctively a needle. I'm hoping that that will also convince them but you have to be able to tell at a very small size that it's a needle. The other thing that they said was that sewing was already represented by the thread, that we didn't need thread and needle but it was literally one line in the minutes that referenced that and then it sort of like, "Did you have somebody in the room or not?" and so, if there is somebody on the committee who is willing to tell you really what their concerns were, then you have some sense of what they're looking for and why they're pushing back. When you can very much see in the earliest emoji character sets that I have a hammer and I have a wrench and I use them but there's these very conventionally male tools. We have all of the kind of office supplies but all of homemaking and housekeeping and textile production, none of them were there until very, very recently. I think it does reflect the gender of the people who've been making these tools, that sewing and knitting weren't important enough as human practices to be included in this glyph set. AMBERLEY: I guess, that's non-trivial to mention because that wasn't an argument that I made in my original yarn draft and Amanda and Jenny sort of pushing to open it up to this whole slate of craft emoji. I didn't realize until they brought that up. I took a stroll through pretty much the whole slate of emoji and you can count on almost one hand the number that represented the creative endeavors or sort of more traditionally known as creative things like camera or painting palette and stuff like that. It was extremely limited. AMANDA: I think they have stuff like that. I think there's a few different variations on the camera and then there's painting palette and that's it. AMBERLEY: Oh, there's the theater mask. AMANDA: Oh, that's right. There is the theater, the happy and sad -- AMBERLEY: And I don't know it exactly and I haven't read the minutes like Amanda has but I think and I hope that that was a particularly compelling piece of that argument. AMANDA: I think they definitely heard it. AMBERLEY: Yeah. CHARLES: Opening it up then, what else is coming in the way of craft? It sounds like this is historical but these pieces are being filled in not only with the work that you all are doing but by other emoji which you're appearing. AMANDA: Yes. CHARLES: And are you in contact with other people who are kind of associated with maybe craft and textiles and other kind of what you're labeling historically creative spaces? AMANDA: I don't think there are anymore with a possible exception. Someone's working on a vinyl record proposal which I think is great. CHARLES: Yeah, that's awesome. ROBERT: Antiquated, though. AMANDA: Maybe not, I don't know. AMBERLEY: Take a stroll through the Emojination Slack and people discussed that. AMANDA: Yeah. If you click at Emojination.org, the whole Airtable database is on there. There's not a lot of other creative ones. A friend of mine got really bent out of shape about the lack of alliums and wrote a whole slate proposal for leeks and scallions and garlic and onions. ROBERT: Oh, there is a garlic one, right? AMANDA: No. I mean, there is -- AMBERLEY: Actually, I'm looking at the Unicode page for current emoji candidates. They first get listed as... I forget the exact order. They become draft candidates and then provisional candidates or vice versa but I don't see any pending further creative ones but garlic and onion are on there. AMANDA: Yes. ROBERT: That makes my Italian a little happy. AMANDA: I think there's some prosthesis, the mechanical leg and the mechanical arm, a guide dog -- AMBERLEY: Ear with hearing aid, service dog. AMANDA: Yeah, there's a good chunk of interesting things that have been left out. I guess they've been approved by the subcommittee but are still waiting on final approval by the Unicode Consortium. ROBERT: Okay. What are the next steps that we can do to help push the thread and needle proposal through it. You mentioned a couple things like coming up with idioms that are in different languages and whatnot but how can we contact you and push this effort and help? AMANDA: That's such a good question. I don't even know. I mean, I am Amanda@velociraptor.info and you're totally welcome to email me if you want to help with this and I will -- ROBERT: That's a great domain, by the way. AMANDA: Unfortunately, there's no information about velociraptors anywhere on that site. ROBERT: That's the way it should be. AMANDA: But also, if you're excited about working on emoji proposals, Emojination is an incredibly great resource and folks there, including me actually will help you identify things that are on other people's wish lists that you could work on if you just want to work on something and we'll help you refine your proposal if you know what you want and we'll help you figure out whether it's worth putting the time in or not and how to make it compelling. You can definitely check out Emojination.org. I think there's a path to get on to the Slack from there. AMBERLEY: Oh, yeah. The Slack and the Airtable. AMANDA: Yeah. ROBERT: It sounds like there's a whole community that was born out of this, where everybody is trying to help each other and collaborate and get their shared ideas across. AMANDA: Definitely and there's a woman, Melissa Thermidor who is fantastic, who actually is a social media coordinator. It's her actual title but she works for the National Health Service in the UK and was tasked with getting a whole series of health-related emoji passed. There's a bunch of things that she's -- AMBERLEY: Is she's the one doing blood. AMANDA: She's doing blood. AMBERLEY: That's a good one. AMANDA: Because there's a lot of really important health reasons why you need to be able to talk about blood and getting blood and blood borne illnesses and -- AMBERLEY: That one was listed on the emoji candidate page or blood donation medicine administration. AMANDA: Yeah. ROBERT: That's really interesting, so she works for the government, right? and that was part of her job to do that? AMANDA: Yes. ROBERT: That's awesome, actually. I love that. AMANDA: Yeah, I think the drop of blood, the bandage and the stethoscope are the three that are in the current iteration, which is interesting because the existing medical emoji were the pill and that gruesome syringe with a little drops of fluid flying off of it, which do not do a lot to encourage people to go to the doctor. ROBERT: No, not at all. AMANDA: So a few more, we're welcoming medical emoji. ROBERT: You have a GitHub. Is that where you're still doing for the follow up and the prep work for the sewing emoji? AMANDA: Yeah, that's probably the best place. I do have a Google Docs somewhere but that's probably a better place to connect even than my ridiculous Velociraptor email. The GitHub -- ROBERT: But it's still awesome. AMANDA: It is awesome. I won't lie. I'm very proud of it. I am AmandaBee -- like the Bumble Bee -- on GitHub and the sewing emoji, the original proposals are there and I will make sure that there is information about how to plug into the revised needle proposal there as well. You guys are a tech podcast, so if people want to just submit suggestions as issues on that repository, that's awesome. We'll totally take suggestions that way. ROBERT: That would be pretty rad. Well, I appreciate you two being on the podcast. I love hearing your stories and how it ended up converging in parallel tracks but it end up achieving the same goal. Still unfinished, right? Let's see if we can help push this over the finish line and get it done because I would really like to see a needle. I could definitely use that in many of my conversations already now, making all kinds of puns. Thank you, Amanda for coming on and sharing your story. AMANDA: Thanks for having me. ROBERT: And thank you, Amberley for also coming on and sharing your story. This was super awesome. AMBERLEY: Yeah and thank you for connecting us to finally have a voice conversation. AMANDA: I know. It's great to actually talk to you, Amberley. CHARLES: Oh, wow, this is the first time that you actually talked in audio? AMANDA & AMBERLEY: Yeah. ROBERT: We're making things happen here. The next thing we have to do is get this proposal through and accepted. AMANDA: Yes. CHARLES: You've converted two new faithful sewing and needle partisans here and I'm in. AMANDA: Awesome. ROBERT: I know you've already gotten, what? Three through accepted? AMANDA: Yeah. ROBERT: We talked about that, it's got to be really awesome. I think I want to try and jump in and get that same satisfaction because a lot of people use emojis. AMANDA: Exactly. CHARLES: It definitely makes me think like you look at every single emoji and there's definitely a story. Especially for the ones that have been added more recently, there's a lot of work that goes into every single pixel. That represents a lot of human time, which I'm sure you all know, so thank you. AMANDA: Thanks for having us on. AMBERLEY: Yeah, thank you guys. ROBERT: Cool. That is the podcast. We are Frontside. We build UI that you can stick your future on. I really love this podcast because it wasn't necessarily technical but had a lot of interesting conversation about how to work with a proposal and probably make a bigger impact than any of us with software, just because the sheer reach that emojis have are insane and the fact that you can influence this process is new to me and really cool, so I hope a lot of other people learn from that too. If you have any feedback that you would like to give us on the podcast, we're always open to receive feedback. We have our doors and ears open, so if you like to send an email at Contact@Frontside.io or shoot us a tweet or DM us at @TheFrontside on Twitter. We'd love to hear it. Thank you, Mandy for producing the podcast. She always does an amazing job with it. You can follow her on Twitter at @TheRubyRep. Thanks and have a good one.
In this episode, Robert, Charles, and Wil talk about the whys and hows of accessibility, as well as what makes single page applications special, why they are they harder for accessibility, and frameworks that can do this for you. Resources: #SkyQ app on #iOS from a #VoiceOver user's Perspective Rob's Routing Doc Wil's PR Single Page Apps routers are broken Greater Than Code Episode #92: A11y Ally with Rob DeLuca This show was produced by Mandy Moore, aka @therubyrep of DevReps, LLC. Transcript: ROBERT: Hello everyone. Welcome to The Frontside Podcast. This is Episode 111. I'm Robert DeLuca, a software developer here at the Frontside and I'll be your episode host. Today, we're going to be discussing accessibility in single page apps. With me as co-hosts are Charles Lowell. Hey, Charles. CHARLES: What is up Robert? ROBERT: And Wil Wilsman. Hey, Wil. WIL: Yo, yo, yo, yo. ROBERT: Sounds like we're ready to drop a disc track. We're not going to be dissing anybody here. We're going to be talking about helpful things with accessibility in single page apps. Before we get into the nitty-gritty of accessibility in single page apps because we're getting into some deep stuff, I think I want to cover a lot of 'how' because I know accessibility things are usually about why you should be doing it and then they touch on things like, "You should be using alt attributes for your images," but for single page apps, I think we need to go further. CHARLES: It always ends up like, "Then draw the rest of the accessibility owl." ROBERT: Yeah, here's your two circles and the alt-tags are your circles and then the rest of the freaking owl is focus management and everything else that comes with it. Before we get there, what is accessibility? I guess, if we trim the giant umbrella down a little bit from everything that is accessibility that can be physical space things, like wheelchair accessible ramps or things like that, what about just technology? WIL: People need assistive tech to interact with technology such as switches or keyboards and obviously screen reader is a big one but when it comes to the software itself, you could even talk about colors and people who are color blind, so red might not be red to everybody. ROBERT: Speaking from experience there? WIL: Yeah. I'm colorblind. Red is brown to me. CHARLES: Things like hearing and all of it, right? It really is like just designing it in such a way that it can be used by as many people as possible. ROBERT: Right. WIL: And that includes your mom who may not be the best with technology but she still needs to pay your bills online or something. ROBERT: Exactly, yeah and in for context to listeners that might not know, my mom is 100% blind, so it's kind of where it comes from. CHARLES: But my mom is not but she has all kinds of problems. WIL: Yeah, same with my mom. CHARLES: That also falls under the category of accessibility, right? ROBERT: Absolutely. CHARLES: Right, accounting for age and culture. ROBERT: We're blending into the why of accessibility, which is perfect. One of the things that it's such a good segue because what people are starting to realize and I think why accessibility is really starting to catch wind and get some traction is because a lot of people that grew up in the technology age were open in using technology a lot. Our parents probably did not use technology heavily. That's definitely the case for my dad. He still has a flip phone and he says, he's a low tech man living in a high tech world and just refuses to pick up technology but those are the people that just didn't use technology. But now we have a lot of people that grew up with technology and use a lot and they're aging into more disabilities and they're going to need that accessibility, which I think is really interesting to think about because that's a lot of buying power if you're just going to start moving in that needs in accessibility, right? CHARLES: That's true. I know I may need glasses pretty soon, so colors and fonts were going to be heck a lot more to me in the next five years than they have over the last 20. ROBERT: Yup, exactly and that's going to be huge for that and that's one piece of the why, so what are the other reasons that you'd pick up accessibility other than people saying that it's morally correct. I don't like starting off conversations for accessibility because it's the thing that you should be doing. WIL: I think it goes even to my user experience like power users that don't like using the mice or mouse. That's me. I really prefer to just use my keyboard for everything. When the new Firefox browser came out and I couldn't navigate through the menus the way I was used to, I went back to Chrome. ROBERT: That's interesting. CHARLES: I still have not found a good workflow for navigating tabs with the keyboard without just kind of twisting my wrist all out of shape. You have to share that with me but again, that's another thing. That's an impediment that sits between me and the application, that actually -- ROBERT: Which is really interesting, you're getting into a keyboard navigation and focus management, which is really the crux of accessibility for screen reader users and switch users. CHARLES: What I'm hearing is that in this case, including good keyboard and focus management in your application, e.g. making it accessible to screen readers, you at the same time, enabling your power users. I think a great point is that by introducing these very low friction workflows, you're actually going to be enabling other parts of your customer base and not just catering to one, that there are ripple effects throughout your system. ROBERT: It may set it for everyone. WIL: Yeah, not only people who need it but people who don't know they need it. ROBERT: Yes, exactly. Think about the WCAG spec as user experience guidelines. They're not telling you how to implement a thing specifically for a screen reader. They're telling you how to implement it in a way that works for everyone, regardless of what ailment they have. It could be a temporary ailment. It could be a permanent ailment, where they have to use a screen reader or any kind of ailment that you can think of. They're not considering just one use case. It's a broad thing that shows how you can make your application better for everyone. I think that's a better way to look at the WCAG spec than I need to read through this and make sure that this auto complete works for a screen reader. If you look at the guidelines, they're not telling you just first screen reader. They're telling you how to make it work for a switch, someone who is colorblind or who is using a dictation software. I kind of tend to look at that as UX guideline, which really helps me build a better app overall because when you nail down that user flow, everyone benefits from it because it's pretty well thought out at that point. CHARLES: I like that too and I think that thinking of it as user experience and making sure that you have a complete user experience is a good way to think about it because it kind of separates the concerns of, "I've got HTML but is my application really HTML or is it a set of workflows and the data over which those workflows operate?" It really forces you to think my application is not a set of React opponents or web components or Ember components or what have you but really, there's a deep structure to it and it makes you kind of shine a light on that deep structure and try to map its surface. Then, if you really, really know it and you capture it, then you can represent it in any medium. I think it's just a win not just for one niche group of users but also for all of our users and then also for your future users that you don't have because your application is designed better and is going to work. Who knows? Maybe there's some new interface or some new medium, some new device that comes out that hasn't even been invented yet. But if you really have a strong internal representation of what your application is, you're going to be able to be the first to move to that market. ROBERT: Right and like I said earlier, people are starting to aid in the needing accessibility thing. If you need a dollar to justify it, that's going to be a big reason coming up. As a part of the new WCAG 2.1 spec, there is a zoom. I forget what this access criteria number is but the new criteria says your app basically needs to be responsive. That kind of maps directly to what you said earlier, Charles which is like you're going to need glasses soon and fonts and being able to zoom. It's going to be really important. We see that just pop up everywhere. To give a counter example of not just for accessibility like you need it for glasses but the other side of that could be like when we give demos on low resolution projectors or screencast or anything, when we zoom the screen, the apps just still be usable and you should still be able to demo it and that's just something that you have both sides to that, where accessibility kind of works out for everybody. People on the call probably don't need glasses but to see that tiny screen that's being shared, you should be able to assume that. CHARLES: Right. I'm just kind of restating what you said but I want to make sure to call it out explicitly because it was a definitely an aha moment for me, that you basically if you build yourself an accessible website, you build it so that it works on projectors and mobile devices too, so you kind of killed two birds with one stone and you don't have to make a special effort because zooming in the screen is tantamount to viewing it on a phone or viewing it on a tablet. WIL: Yeah, we mentioned it earlier about accessibility and physical space and one of those things is being able to access something from anywhere no matter the device that person is using. ROBERT: Yes. That's a lot of the 'why' of accessibility and I think we did a pretty good job of staying away from the more argument because morally we don't know if it's the right thing to do but a lot of the times it's not in front of you, it's hard to do and I want to make sure that you don't feel bad if you're not building accessibility in your apps. It's not easy. I don't sort of like people get up and say that, "Accessibility is easy. Just do this," because it's not -- WIL: If it was easy, everybody would do it. ROBERT: Exactly. I always come back to accessibility being just like UX -- user experience -- because if it were easy, everybody would have a really great user experience too. It's a hard thing to boil down and to simplify it, right? CHARLES: Right and like every other aspect, the kind of nonfunctional requirement, I say nonfunctional requirement that's a little bit of a contradiction terms but it becomes very hard if you haven't done it from the beginning. But if you didn't start with an accessible app, you weren't thinking about that or you inherited the app or it just wasn't on your radar for whatever reason. If you got a codebase that's two years old, going back and trying to make it accessible, it was extremely hard. It's expensive, expensive, expense. WIL: Yeah, it's going to cost more in terms of money and time to added it after the fact. ROBERT: Exactly. I spent a year on helping on Visa Checkout and there were some accessibility considered in the designs but a lot of the time my feedback wasn't just like, "Yeah, sprinkle some ARIA attributes on there and you're good." It was like, "Do we really need a carousel here to represent your list of carts because it's really hard to navigate around that?" A lot of the times, it ends up boiling up to is this the best way we can represent this data? Is this the best way we can navigate and build this user flow? A lot of times -- CHARLES: And if the answer is no, it's so painful, right? ROBERT: Yeah, exactly, so all that work that was put into building that carousel and all the components that built that carousel together is thrown out because it's just not a pattern that should be used there. Doing after the fact is really hard and really expensive and usually ends up in refactoring anyways. CHARLES: I just want to point that out because a lot of people find themselves in that situation, where they're staring down at a pretty big cost. Now, the reasons why your app may not be accessible or many and good and you shouldn't feel bad, if you're finally in that situation. ROBERT: I gave a talk at Nodevember a couple of years ago called Accessibility Debt and it's just like any technical debt. Your apps going to have it. It's okay. Don't get beat up about it, especially if anybody is trying to beat you up over it, don't listen to them. It really is just like any other kind of technical form of technical debt. It's something that you'll have to deal with and it is just something you have to work through. It's not world ending. It's just another problem to work through. What are some things like everybody usually talks about like the things you can do, the basics for making your app more accessible like using all its attributes and instead of doing a div with an on-click handler, just use a button or don't overuse ARIA attributes. Are there any other things that I missed there like the basics of accessibility? WIL: The biggest is the HTML structure. Screen readers and other assistive tech were built with the standards in mind, so if you're doing nonstandard things like putting divs in H1 and adding ARIA attributes within there, you're not going to have a great time. ROBERT: Right or splattering ARIA roles all over the place, probably not a good idea. That will be harder to debug. Also fun fact, ARIA roles, while you can implement directly to the spec, you may still have bugs across the different screen reader combinations or assistive tech combinations, so that's fine. CHARLES: Keep it super simple is what I'm hearing, like use semantic markup. If you're going to introduce a custom button, still make sure that it's a button. ROBERT: Use the platform. CHARLES: Yeah, use the platform. Don't fight the platform. Probably the best example of that is people implementing their own select boxes. That's the classic example. ROBERT: Wil and I, our lives has centered around that for a little while. It's so true. Usually, it's the first thing people go to grab to reimplement because selects are just ugly. I think Firefox has the ability for you to style select options now like you can change the color and the font but you can't style that. The pop up that comes, usually that's the system dialogue, which a lot of designers don't really like. That's usually the first thing that people go to implement and that's usually actually the first thing that stop somebody from signing up. A lot of sign up forms that I see, if your date of birth is in a select format, that probably will hinder somebody that uses assistive tech from signing up. CHARLES: Yeah. You just basically bounced that entire person. The thing is people don't appreciate the cost. This gets into the whole concept of accessibility that is how much money would that person or those group of people have actually brought into your site versus the cost that you spent redoing that select box. You might be thinking, "It only took this developer actually two weeks," but when you actually look at the actual cost over the course of your application, you're not factoring that into the decision to go with custom select box. Just in our experience, it's just the truly low cost option per quality of experience, that tradeoff there is almost invariably going with platforms select, right? ROBERT: Yeah. Your secret sauce and the best UX that you're going to provide is not going to be nicely styled select box and seriously, a battle that I had to fight a lot was if you really want to implement a custom widget, decide if this is what you want to spend your time on because custom widgets aren't just quick and easy things you implement. You're not going to implement the select that's fully accessible across all the AT combos in a couple of days. CHARLES: It's a lot of work. ROBERT: Yeah. You're going to fall down on a huge rabbit hole. CHARLES: Yeah. It is just you're committing to that work over the lifetime of your application. ROBERT: Exactly, if you can maintain that now. If you implement custom check boxes and custom radios and custom input that of content edible for some reason, I think -- WIL: I think what I see is like custom date pickers -- ROBERT: Oh, I just had a rant about that. CHARLES: Date pickers are hard and there's not really a good option. You just have to open your eyes to the true cost. ROBERT: Right, exactly. That's what I always try to explain, just like you have ownership over this now and you now have to maintain this and you can't regress. CHARLES: Right and if you do regress, it's your neck that gets choked. ROBERT: Yes. A good way to put it. We've talked a lot about things that are just general accessibility but nothing specific to single page apps. I do want to say like a lot of other things that people recommend is like if you're using React, use like the JSX ES1 plugin to help you analyze if you're writing any JSX that might not be accessible or use like HTML_CodeSniffer or aXe to statically analyze the DOM that you have. Those tools are great. I'd see a lot of people [inaudible] those things as like, "Look, this automated checkers says I'm inaccessible so I'm inaccessible," and that's not the case, especially in a single page apps. You can have 100% passing automated checkers but your app also could be 100% broken and why is that? WIL: I don't know. ROBERT: I'm wobbling the router question here. WIL: Yes. I guess that would just be due to routing. In single page applications, they handle their own routing, whereas static web sites and whatnot, the routing is handled despite URLs and the browser and reloading pages. ROBERT: Right. There are probably other major differences between single page apps but the biggest thing is the routing is managed by the client, the JavaScript and on a server-rendered page, you click a link, it's going to rerender the entire page for that next page and the screen reader and the browser know exactly what to do with that, to put the focus back to the top of the page and start working through the page for you. But with single page apps, you're just replacing a piece of the DOM and the screen reader has no idea of what's going on. An automated checker cannot check for this. They cannot tell you if your routes are accessible. The reason I'm bringing this up is because this has never talked about. I don't see in accessibility talks and this is the thing that actually is most broken and makes your app pretty much unusable to anybody that's using assistive tech. There's ways that people if they're savvy can navigate around it but if they don't know what's going on, they're going to think, "All right, I pressed this button and nothing happened, so I'm just going to leave now because it's not working." CHARLES: There's a great video that you point to me, I believe a guy from the UK who has recorded a bunch of his experiences using websites and apps -- ROBERT: Yes, I want to dig it up and put it in the show notes. CHARLES: Yeah. Those are great to watch because you'll really get it. ROBERT: Yeah and that's a really unique case because he's really savvy. I believe, I could be wrong but I think he might have said that he definitely work in tech somehow -- CHARLES: Right. He knows the workarounds and he knows these things but in some of the cases it's like, "If I didn't work in tech, there's no way that I would be able to use this website." ROBERT: Yeah. This is kind of the crux of like if you ever listen to anybody that is an accessibility consultant, they'll say, "You will never ever be able to automate accessibility," and this why I tie accessibility so much to user experience, would you ever have a user experience test that can tell you in a binary fashion? Yes or no, that your app has a great user experience? No, because it's pretty subjective. CHARLES: Of how does your users feel? right? ROBERT: Yeah. CHARLES: You can rate, "My users feel great." ROBERT: Accessibility is like that because there's a lot of context that you have to carry around. It's all about context. When I transition from this page, the next thing does the user have enough context of where they're coming from to where they're going to be able to operate on that page. Is there enough information to achieve the task they want? That is pretty much the crux of why there is no binary yes or no for that because it's contextual. It varies from person to person but you do your best to make sure that that works and that you provide enough information to do something. That's kind of a single page app as a crux. This is why we have a philosophy of testing as a whole. We don't test components individually because again, you can make all of your components individually accessible there but as a whole, they might not work together because you're not providing enough context on an entire page. We did this in one of the apps that we work on, which is open source, so we can link to the PR that Wil wrote for this and I wrote an entire routing documentation around our philosophy and the different things that we tried. How did we manage the focus in that application? I'm kind of just going to lob it over to you Wil since you did the work. Do you want to give context of the holdings and how that all kind of came together? WIL: Yes. One of the features of the app is these panels. It's like this three panel system. When you click an item in a list, a third panel pops up and this goes back to the context thing where if you're using a keyboard, you can't see the screen. You click an item in a list, how do you know that third panel popped up? The solution isn't for a component. The component can't be responsible for this. The list can't focus the item. It opens or vice versa, so this is definitely an application concern, where we needed to check against the route and see whether or not, the pane is opening or was already open or wasn't open before and when this pane opens for the first time, it will just focus it and that gives a lot of the context that the user needs when they click it, like they click an item in the list, it focuses this third pane and they're on a third pane. ROBERT: Right. We didn't even just focus on the entire div. We focused on the heading of the thing that you selected. WIL: Yeah because focusing the div, it might read something off but not all the time. The main thing we're focusing on that third pane opens is the heading to let them know that the item you click, you are now on that page and reading that heading. This is the same thing that would happen if you loaded up that page statically and the screen readers would usually just focus on that first heading. ROBERT: Right. That helped a lot. For a little bit more context, the middle pane is like kind of a master detail thing going on here and in the middle pane there that we have, it's an infinite scrolling list. You have different things there and one of them is like a package. If you select the package and without the focus management and focusing on that heading, you would have to go through every single package that's in that list which could be a thousand of them before you actually get over to the pane that you just opened because source order. You have to go through each one of those. WIL: And it's the same thing is true for power users, not just screen readers. It's like if they want to use tab once that pane open, they have to tab through the entire list. ROBERT: Exactly. It wasn't keyboard accessible and it made it really hard to navigate around with a keyboard because the focus just wasn't being managed. There was a lot of work that we did there. I want to focus on the routing situation there because if you can't navigate around with a keyboard in your single page app, like you click a link and it's not selecting the next thing that should be focused and you provide the right amount of context, your app probably won't be usable without a lot of trial and error to a user, which depending on what your product is, they may not have a lot of patience for trial and error, right? WIL: Yeah. ROBERT: It's really important to try and nail down the routing situation and there are some frameworks and things out there that can do it. In the app that we're talking about, it's React app and we use React router but we don't actually really hook into the router to handle that and that's because React router doesn't provide very much information. WIL: Yeah. You can think of React router as more of an outlet system, where your routes can render anything, anywhere on the page. That's kind of dangers of accessibility and that's kind of the reason that React router can't handle accessible things very well because at any point, the route can just change the button on the page to look like something else. CHARLES: Yeah. It just pop in and pop out. There's no deeper model, right? There's not -- WIL: Yeah, there's no tree. CHARLES: Right, there's the internal state -- WIL: Like a component tree, yeah. CHARLES: The internal state of what is happening is completely opaque. You can only analyze the second and third order effects of the React tree. ROBERT: Right and especially if you have nested route components, it's really hard to determine. One of the things that I've seen people do is focus on mount, which is a very naive approach because what if you have three nested routes, they're all going to focus on mount and the last one that mounts wins and that's a focus for which nobody wins. CHARLES: You all tried that, right? ROBERT: Yes. That's part of the document that I wrote up. We tried three different approaches and we ended up landing on something because we were using React router that was more of like an application state thing, so we were checking props because we knew what the user flow was. We knew what the user, when they come to this page is trying to accomplish, so we're able to kind of figure out from where they came from or where they're going through props and focus the right things for them. WIL: One of the examples, like we talked about the third pane opening and focusing the heading inside but what you know what happens when you close that third pane, you kind of lose contexts again, so we have to focus the previous list item that was active because if you focus back at the top of the list, they've essentially lost their place and the list of results. In that case, we couldn't use on mount. That list item is already mounted. We have to listen to props and we have to look at the route through these props to determine if that third pane was open. If it just closed and if an item in the list is active, it should have focus. It's a lot of logic going on. You have to really understand your app in order to make it a very good accessible app. You can't just sprinkle in ARIA attributes and focus on mount everywhere and think it'll work fine because you're accessible. You have to really know the flow. CHARLES: Right. All those signposts point towards having a deeper application state, a deeper understanding of your application than just the render tree. At that point, it's too late and so by that, I'm definitely lobbying a Reach/Ember router. WIL: Yes. We talked a lot about the React router and we can't really be too accessible with it but to create a React router to go out and there is now Reach router. Robert, have you heard any good things about that? You're the accessibility expert here. ROBERT: I haven't played with it myself, so [audio glitch] things that were lost from the React router three to four transition, I think and also, it's actually accessible routing which is nice because it comes out of the box and you know how to implement it. I haven't played with it. I know Gatsby has implemented that for their V2, that's their default router now, which makes me really happy because a lot of static sites that were being built with Gatsby were very inaccessible and broken which made me sad but now, they're not with V2. I haven't played with Reach router but one of the things that I think it provides which was missing from React router was transition hooks. It has a concept of like where you're going from and to for your routes, which really helps figure out what you need to focus. The one thing I will say about Reach router and I'm sure it's got to be configurable somewhere but I haven't really looked and the demos that I saw, they were just focusing the div of the content that's being rendered, like it kind of just wraps with a generic div or the tabindex="-1" and then focuses it. If you have a lot of content that's inside of that div, it probably will be confusing but at least, it manages the focus somewhere. If you're now off in a no man's land, you have no idea what's going on, at least it focuses that. But if you want to really nail the experience to be better, I would recommend trying to figure out what you should focus inside of that route that you just transition to, what is the best thing. That's for React. There are other things out there for other frameworks, like I am on Ember accessibility team that's out there and we have Ember A11y, which just provide you a focusing outlet for you to be able to just drop in your app and then when the route changes, it does the same thing. It focuses the wrapping div of that outlet that was just rendered. I want to emphasize more in talking about e-holdings work that we did that we were focusing the heading because that told you exactly where you're at and what package you're on or what thing you're on. You know the name of it and now you know how you can go and navigate through that. CHARLES: But it's conceivable, so how would you do that with the Ember router? Would you just introduce some way to delegate down to a particular component? Like when I'm rendered into an outlet, it focus on this component? ROBERT: That would be interesting. I actually don't know. It's been a long time since I've messed around with the Ember router and Ember accessibility. It's definitely a great first step and that's where it kind of came from. I think there needs probably some work done to help implement on what you want to focus. That's kind of where I was going with when we were first exploring the React router stuff and e-holdings was I wanted to have this like high order component that knew of your routing tree and it knew where you're coming from or where you're going. Then from there, it would just tell you, this is the route that we're going to and there's need to be focus management done, like if this prop exists, then you can pick inside of that component that what you want to focus. It leaves it up to the implementor of what they need to focus but it could be probably just like a fallback to the general div because that's not bad. It's a good first step. There needs to be a little bit of work there to get that done probably. CHARLES: Yeah. But it is worth pointing out that by starting from a position of having an externalized application state via the route structure, in an Ember application, you're starting 95% of the way there. An Ember application, at any point, you know where you are and during your transition, you know where you're starting and where you're going to end up and you know when you leave and you know when you got there. ROBERT: Yeah. Having an Ember route pivot handler there, like where you're pivoting from and to is just so nice and it kind of made it click together a lot easier. CHARLES: Right. Reach router looks interesting but as I understand, it's still couched in React components and it feels to me like this is a problem that ought to be solved, that ought to be framework agnostic. Because if I use something like Reach or I use some other routing library for another framework, some other tool might come out that I want to use and I shouldn't be locked in -- ROBERT: Right like what if I really like Redux little router, then I have to make a choice between Redux state that I like or accessibility. CHARLES: Exactly and that feels like a false dichotomy to me. What I would want to be looking for is a platform independent or a framework independent routing library that really just helps you represent your application state and the concrete state in which your application is in at any moment and then, also be able to represent the transitions between those states fully and completely. Then if you have that, you could embed that into any framework. ROBERT: Right. That would be really nice to have. Just like pull it off the shelf and help you out there. CHARLES: If anybody is listening who wants something to do for the next 18 months, no one will thank you for 18 months but you had to get on that. We'll give you lots of thanks then. ROBERT: [inaudible] compare with you. That would be awesome. CHARLES: I would love to be part of that. ROBERT: Yeah, that would be awesome. To kind of tie back to other things, I don't know too much about Angular but I'm sure there are solutions out there to help with Angular. I know Marcy Sutton used to do a lot of work in the Angular world, so I'm sure there's something out there that helps with that. I just don't know off the top of my head right now. I wrote a Medium 'think piece'... No, I wrote a little medium blog post about how all of single page out for routers are broken. I was pleasantly surprised by Vue. Vue brings its own router and their router has a concept of before each, so before each route transition you can run some code and that really helped with implementing the live, the announcer pattern where you use ARIA live to announce something but even beyond there, if you wanted to dig in there, you could probably figure out where you're transitioning from and to and give that next route some kind of attribute that says, "This needs to be focused. Figure out what you need to focus there and inside that route, you can focus wherever you want." I thought that was a really awesome. That's kind of the crux of what I was missing from React router. I wanted the concept of knowing where I'm coming from or where I'm going and I would help with everything but unfortunately, that kind of doesn't exists because they're just components. For better or for worse, they're just components. CHARLES: The world is so much more than just components. ROBERT: Yeah, a little bit off topic, I think it's kind of funny how React kind of just shoved everything into the Vue layer, just to make it all a component. CHARLES: Yeah, it's very easy. ROBERT: Yeah, until it's not. CHARLES: Yeah, exactly. It's easy but it's not simple. ROBERT: That's a lot of talking about how routes need to be done and what you can kind of do to manage focus. It's really about managing the context and how you can provide the most contexts. For somebody to operate on that information, can they complete this thing? What can you do to make sure that you've done this properly? What steps can you take to make sure that you actually are accessible and that your routes work? WIL: Manually testing those screen reader is probably the biggest thing, you know? CHARLES: Yeah. I think the biggest thing is really watching someone who uses assistive tech on a regular basis use your application and then trying to use it yourself. WIL: Yeah. ROBERT: Right. Yeah, I'll definitely -- CHARLES: If you can get a little bit of this yourself but then it's kind of like someone who is never used a mouse before and trying to learn something new. What really helps is seeing someone who's good at it and see how their expectations are either being met or not being met. ROBERT: Yeah, it's definitely the best way. If you can find somebody that actually relies on assistive tech, there's nothing that beats that kind of feedback. If they get to your app and they're really confused, I see some people that just dismiss it because they just don't understand. That is the best feedback you can actually get. If they don't understand what's going on, you have -- CHARLES: That's on you. ROBERT: Yeah, that is going to be what happens for everybody that uses that. Well, maybe not everybody because everybody has different experiences but it's probably going to be a thing that pops up everywhere. But if you don't have access to people that are actually using assistive tech regularly and are pros at it, WebAIM provides really great tutorials for how to use a screen reader. If you're using a Mac, you have a screen reader built in and you can use that called VoiceOver. If you ever want to turn it on or turn it off, its command F5. WIL: You might have to have that shortcut enabled, though. ROBERT: Really? I'm pretty sure it's quite default. WIL: I thought I had to enable mine but I could be wrong. ROBERT: It's interesting. CHARLES: Yeah. They got great tutorial too. It's like it notices the first time you turn it on, so it tries to help you navigate bullet lists and select boxes and input fields and check boxes and all kinds of good stuff. ROBERT: Yeah. It gives you a little bit of a boot camp but WebAIM also helps with specific to website and stuff because one of the things that we ran into while working on the e-holdings project is they're transitioning from a native app to a web app and there were just things that you can do on a native app that you cannot do on a web app. Just keep that in mind also when you're testing, there's just things that will behave differently. Like you're not going to have a lot of crazy key shortcut commands like you're not going to press command F to get to the search box if your app has a prominent search box because that is going to clobber a bunch of other assistive tech key commands. WebAIM is really a good help with giving you the tutorial of how to test a web app and many different screen readers, so they have it for JAWS, they have it for NVDA, they have it for VoiceOver iOS, they have it for TalkBack, which is the Android screen reader. There's a lot of really good resources there for you to start using as screen reader and test with your app. I highly recommend using it with a screen off. You gain a lot of context by looking ahead of your cursor and -- CHARLES: Yeah. It's true. ROBERT: -- The example that I usually give to people is like, "I worked on Visa Checkout. I went through that checkout flow pretty much every day for a year and eight months in, even then turning off the screen, I was still lost." Even though I knew what each screen was and what each component was there, I would find myself confused of like, what just happened and it's because sometimes, you'll get into something like you have a dropdown that sets the focus inside the dropdown and then the dropdown disappears from the DOM and your focuses in nowhere. You're like, "What is going? what am I doing. I don't even know where I'm at," and I turn the screen back on and I'm like, "Oh, now, I know what's going on." CHARLES: Right. It really is a lot like interacting with your application as though you were interacting with Siri or Alexa but with a keyboard instead, instead of voice commands. That's an excellent point. We would understand if where would Amazon be if Alexa couldn't successfully navigate those situations. The counterpoint or even the flip side of that is if you model your web application in such a way that it can handle that type of serial interaction, instead of the highly parallel environment to being able to perceive huge amounts of information concurrently, like you can on a screen, if that were effectively serialized, that means you could represent your application through nothing but voice commands. ROBERT: Yeah. Did you provide enough context for me to build this mental map is really what I'm going on to? CHARLES: Yup. ROBERT: Which I always thought was really interesting because I always wanted to know how my mom visualizes what a website looks like because it's wildly different than any of us. She's never been able to see what a website looks like. Does it look like a bunch of nodes and graphs and webs connecting to each other and how things pieced together? It's just a different way. But it's not only about screen readers, right? You can use a keyboard to navigate and that's definitely what we did with e-holdings is like can we tab through this [audio glitch] list, hit enter and go into the detail record, edit it, close it and go back and edit another one. Is that something that we can do with just a keyboard, not even a screen reader? WIL: And with the keyboard, we're not talking about shortcuts to hit edit. We're talking about like tabbing over and hitting enter like people with accessibility issues would have to do. ROBERT: Right and that's kind of a good segue into creating use cases. If you want to know if your app actually works, if your screen reader users or if your keyword users or if your dictation users are going to be able to navigate this app, create use cases. Things like actual user flows like how would somebody actually going to use this app? What task are they going want to complete? In that case, e-holding was like an electronic holdings management system for libraries. They probably want to get in there, add a couple of books or whatever you might have to their library and get out. A use case could be like, "Can I search for this thing? I'm going to search for something specific. I'm going to go through the list, find the thing that I want. I'm going to add it, close the pane, go back and then remove one thing and can they complete that flow successfully without running into any issues or any blockers or any showstoppers." I can tell you before we did the routing management stuff, you would hit search and that was it. There was nothing else that you could do. CHARLES: Yeah. They wouldn't even announce that anything can happen. ROBERT: Exactly. WIL: Yeah, and with these librarians, it's not necessarily a matter of they can't see the screen. It's just that they don't use the mouse because they're power users. ROBERT: Right. They don't have any disabilities but that was essential to the workflow. CHARLES: Yeah, exactly. Do that change. You just lowered the activation energy of that workflow by... What? three orders of magnitude? WIL: Yeah, at least. ROBERT: Let me click here right now. I can tab, now I can tab. Right now, let me click here and now I can tab, I can tab, I can tab. It's not as nice as just being able to completely do it through a keyboard. Through us making it super keyboard accessible, that also became super screen reader accessible and the people who use dictation were able to work through and get through the app and use it now, which was really cool. It really helps when you go for those things and create use cases to really figure out how a user is actually going to work through this app. That's the best way. Just get right through it. With Visa Checkout, we're like, "Can somebody buy something?" or if they don't have an account, can they sign up and buy? those were some of the use cases that we had because it turns out, those are actually pretty important to the business. That all have been said, you also can test your components for accessibility individually because even at a smaller level, some of your components might have to manage focus. The best one I can think of is models. When you open a model, you should trap the focus inside of that model and you should be able to hit escape to close the model and when you close the model, it should go back to what triggered the model to be opened. These are all things that are individual that can be tested also. But just because your components are individually accessible, it does not mean your application as a whole was accessible either. I don't want to paint the picture like, you don't have to care about your components accessibility individually because you do. It really does help but I think a lot of people miss the whole of the application, rather than individual with components. CHARLES: Right. The components are the individual stitches but you have to follow the thread throughout the entire garment. ROBERT: Right, exactly. CHARLES: Man, what I really want to do is I want to find out how your mom visualizes website navigation and use that as a visualization technique. ROBERT: That might be a fun webcast or something. CHARLES: Yeah and then see like could we actually use it as a tool because I have to imagine, it's probably pretty simple. It's much simpler than what we think of when we think of a website because it has to be really condensed down to its essence. ROBERT: Right. Yeah and [inaudible] users, they're not dumb. They have different ways. They know the gotchas. They know things that happen. They know there are different ways of getting trolled like my mom knows about focus jumping and she gets irritated what happens but she knows generally of what to do and it depends on her patience level. Like if it's focused jumping, she's like, "I don't need to use this thing. See you," and it's just not worth the frustration but there's different ways to navigate around like if you're a real power user, you might be able to recognize like the routing is inaccessible and you can navigate by headings or by regions or different landmarks. There's many different ways for users to navigate but to use those different navigation methods, you need to have a real strong coherent document structure. Your H1s have to actually be H1s and you have to have some things that they can navigate around to kind of work around those things. It's interesting and basically, do everything you can to help those situations. If you can provide semantic markup and give it a proper structure. If you can do the focus management, it's going to help everybody. It really will. I did see when we went through the use cases and defining those things, we actually learned a little bit more about our product because we had to put ourselves in a different seat and think about it because you get real close to it. You go through that same flow nine million times and you pick up real power user things that you can do like, "I'll work on that. I got this. I'll click this button. All right, we're good." Somebody that's going through it for the first time and you put yourself in that seat, it kind of opens your eyes a little bit and makes the experience better for everyone. I think that's kind of the underlying tone there. That's the message. WIL: Yeah, accessibility makes things better for everybody. ROBERT: That was a lot of content thrown at you there. We covered what is accessibility and why you'd want to do that and then kind of like more the basics things and how automated tools are really helpful and they can help you pick out things like using improper roles and nested things but they're not going to be able to tell you if your application is truly accessible or not and never will, unless we get something like a headless screen reader, where you can write automated tests for in that fashion but you're not to get something that will just run over you app and go, "Yup, you're good." CHARLES: Even so, it's a matter of user experience and that's not something you can get a thumbs up or a thumbs down to. When you as a user, it comes with application, you know when you see it but I would say until we have androids that accurately simulate human beings, I don't think we're going to actually have automated testing. ROBERT: Yeah. There's a joke for accessibility consultants. It's like if you put four accessibility consultants in a room and tell them to give you an alt attribute for an image, you'll get four different alts. We talked about the automated checkers, right? They'll not going to get everything for you and we talked about single page apps specifically in the routing and how we handled the routing in a React app and then how you can probably do it in an Angular app and how you can do it in an Ember app and Vue and different methods of how you can kind of attack that. We're definitely giving the link to the document that I wrote and the PR so you kind of see the real 'how' of how we did it because that would probably be helpful. I think there was a lot of good information there, so I would like to thank both Charles and Wil for being awesome co-host on this and -- WIL: Thanks for being an awesome host. CHARLES: Yeah, thanks for being an awesome host. I should say, you're welcome. ROBERT: I tried, I tried. We are The Frontside. We do accessibility consulting and training. If that's anything that your team needs help with, we're more than happy to jump on a call with you to kind of figure out what your needs are and what you need to do. If you need a WCAG support statement, if you need to audit, if you just need to figure out what's the next steps for you to even do, we're more than happy to help you sort through that. To reach out for that, you can contact us at Contact@Frontside.io or Sales@Frontside.io or Info@Frontside.io. You can contact us at any different ways and we'll be more than happy to help you. As always, thank you Mandy for producing the podcast. You're awesome and the next episode, what we're going to have is also still accessibility related and I'm really excited about this. It's about writing a proposal to the Unicode Committee and getting it accepted so basically, writing a proposal to get an emoji added and that's with Amberley and Amanda. They wrote three or four Unicode specs and actually got them accepted for, I believe it was sign language and deafness. That's really cool and I'm super excited for that because they'd be the first people that I've ever talked to that have actually created an emoji and gotten it accepted. WIL: Yeah, it's pretty cool. CHARLES: Yeah, that must feel great. ROBERT: Yeah, it's going to be awesome. That's our next episode. If you have any ideas or comments or anything, you can tweet us at @TheFrontside on Twitter or you can contact us through any of the emails that I talked about earlier. We're always open to hearing feedback. Thanks for listening. Take it easy, everyone.
Panel: Charles (Chuck) Max Wood Lucas Reis Special Guests: Charles Lowell & Taras Mankovski In this episode, the panel talks with two special guests Charles and Taras. Charles Lowell is a principle engineer at Frontside, and he loves to code. Taras works with Charles and joined Frontside, because of Charles’ love for coding. There are great personalities at Frontside, which are quite diverse. Check out this episode to hear about microstates, microstates with react, OM, Redux, and much more! Show Topics: 2:32 – Chuck: Why do we need it (microstates) and why do we need another state library? 2:42 – Charles answers Chuck’s question. Charles goes to explain that if you need to increment the number, you don’t need to do it with microstates. 3:41 – Another suggestion is given on this topic. 5:13 – The application isn’t hard in-of-itself. 6:45 – Chuck makes comments, and asks: It seems to be more like object-oriented approach? 7:44 – Objects compose much more easily. When you are dealing with pure functional code you are de-structuring and restructuring. Check-out LENSES. 9:53 – Taras makes comments. What were your inspirations for microstate? 10:27 – Charles: The personal journey it started for me started back in 2015. When I was working primarily in Ember.” Charles makes a reference to OM, check it out! 15:40 – Charles: “We had a goal in mind, and we kept that goal on mind and kept ‘dipping into the candy jar.’ We had to learn about the functional mumbo-jumbo. The goal was never to use those things. Whatever tools we needed from the functional world, we borrowed from freely.” 16:50 – Chuck asks a question. 17:00 – Taras answers chuck’s question. 19:58 – Charles (guest) keeps the conversation going and goes into detail about how to handle different scenarios with different tools. 21:00 – Question: How do you think microstate enters into this situation? 21:45 – The design of microstate is that it gives you a solution that is flexible. Other options aren’t as comprehensive like where you can use it; for example Redux. 23:49 – Another way to say it is...check-out this timestamp to hear other ideas about this topic. 24:53 – Digital Ocean’s Advertisement 25:28 – Conversation is back into swing. Question: There is a very interesting design with people who are not developers. What are the benefits or do they play together? 26:41 – As a frontend shop, there is a very clean mapping between state machine and type. The type corresponds to the state transitions, among others. For every state you have a class, and you have a method for every transition. It’s a great design tool. 29:07 – We don’t talk about states very often, right now, but in the near future we will. The valuable goals for us are to give people tools that will work correctly for them. To help people be more productive that is a great goal. One thing from people, I’ve learned, is to ask yourself ‘what needs to change?’ 33:03 – Now you are touching on the subject of teaching. What about mentoring with microstates? 33:26 – Success (to one of the panelists) is defined of how confident a person is with X program or tool. If they have ease, then they are on the right path. With mentoring in microstates the design speaks its purpose, the transitions are clear, so the panelist feels that he doesn’t really have to go into a lot of detail explaining the features. 36:25 – In the React community... 39:12 – Curious: Would we really be able to distribute state like how we distribute components? What is out-of-reach now, is that we have the state machine for the autocomplete component. 40:27 – Chuck: Is there a way to test microstates? 41:28 – Shameless plug...check it out! 42:31 – Anything else? Microstates and Microstates with React. 42:48 – If anyone is interested in this, then we are interested in talking with these people and/or companies. 43:29 – Let’s go to Picks! 43:31 – Advertisement for Charles Max Wood’s course! Links: Kendo UI OM Frontside Redux Microstates Microstates with React Taras Mankovski’s Twitter Taras Mankovski’s GitHub Taras Mankovski’s LinkedIn Taras Mankovski’s Frontside Bio Charles Lowell’s Twitter Charles Lowell’s GitHub Charles Lowell’s Frontside Bio Schedule Once Ruby on Rails Angular Get A Coder Job Sponsors: Kendo UI Digital Ocean Get A Coder Job Picks: Charles (Chuck) Framework Summit – Chuck will be talking at this conference in UT. Ebook – Finding a Job. Prelaunch in August. Final version launches on Labor Day. Lucas Take care of your health! Martial Arts and Jujitsu Nutrition Charles (guest) Fantasy Land JS - Tom Harding Funcadelic.JS Taras (guest) BigTest
Panel: Charles (Chuck) Max Wood Lucas Reis Special Guests: Charles Lowell & Taras Mankovski In this episode, the panel talks with two special guests Charles and Taras. Charles Lowell is a principle engineer at Frontside, and he loves to code. Taras works with Charles and joined Frontside, because of Charles’ love for coding. There are great personalities at Frontside, which are quite diverse. Check out this episode to hear about microstates, microstates with react, OM, Redux, and much more! Show Topics: 2:32 – Chuck: Why do we need it (microstates) and why do we need another state library? 2:42 – Charles answers Chuck’s question. Charles goes to explain that if you need to increment the number, you don’t need to do it with microstates. 3:41 – Another suggestion is given on this topic. 5:13 – The application isn’t hard in-of-itself. 6:45 – Chuck makes comments, and asks: It seems to be more like object-oriented approach? 7:44 – Objects compose much more easily. When you are dealing with pure functional code you are de-structuring and restructuring. Check-out LENSES. 9:53 – Taras makes comments. What were your inspirations for microstate? 10:27 – Charles: The personal journey it started for me started back in 2015. When I was working primarily in Ember.” Charles makes a reference to OM, check it out! 15:40 – Charles: “We had a goal in mind, and we kept that goal on mind and kept ‘dipping into the candy jar.’ We had to learn about the functional mumbo-jumbo. The goal was never to use those things. Whatever tools we needed from the functional world, we borrowed from freely.” 16:50 – Chuck asks a question. 17:00 – Taras answers chuck’s question. 19:58 – Charles (guest) keeps the conversation going and goes into detail about how to handle different scenarios with different tools. 21:00 – Question: How do you think microstate enters into this situation? 21:45 – The design of microstate is that it gives you a solution that is flexible. Other options aren’t as comprehensive like where you can use it; for example Redux. 23:49 – Another way to say it is...check-out this timestamp to hear other ideas about this topic. 24:53 – Digital Ocean’s Advertisement 25:28 – Conversation is back into swing. Question: There is a very interesting design with people who are not developers. What are the benefits or do they play together? 26:41 – As a frontend shop, there is a very clean mapping between state machine and type. The type corresponds to the state transitions, among others. For every state you have a class, and you have a method for every transition. It’s a great design tool. 29:07 – We don’t talk about states very often, right now, but in the near future we will. The valuable goals for us are to give people tools that will work correctly for them. To help people be more productive that is a great goal. One thing from people, I’ve learned, is to ask yourself ‘what needs to change?’ 33:03 – Now you are touching on the subject of teaching. What about mentoring with microstates? 33:26 – Success (to one of the panelists) is defined of how confident a person is with X program or tool. If they have ease, then they are on the right path. With mentoring in microstates the design speaks its purpose, the transitions are clear, so the panelist feels that he doesn’t really have to go into a lot of detail explaining the features. 36:25 – In the React community... 39:12 – Curious: Would we really be able to distribute state like how we distribute components? What is out-of-reach now, is that we have the state machine for the autocomplete component. 40:27 – Chuck: Is there a way to test microstates? 41:28 – Shameless plug...check it out! 42:31 – Anything else? Microstates and Microstates with React. 42:48 – If anyone is interested in this, then we are interested in talking with these people and/or companies. 43:29 – Let’s go to Picks! 43:31 – Advertisement for Charles Max Wood’s course! Links: Kendo UI OM Frontside Redux Microstates Microstates with React Taras Mankovski’s Twitter Taras Mankovski’s GitHub Taras Mankovski’s LinkedIn Taras Mankovski’s Frontside Bio Charles Lowell’s Twitter Charles Lowell’s GitHub Charles Lowell’s Frontside Bio Schedule Once Ruby on Rails Angular Get A Coder Job Sponsors: Kendo UI Digital Ocean Get A Coder Job Picks: Charles (Chuck) Framework Summit – Chuck will be talking at this conference in UT. Ebook – Finding a Job. Prelaunch in August. Final version launches on Labor Day. Lucas Take care of your health! Martial Arts and Jujitsu Nutrition Charles (guest) Fantasy Land JS - Tom Harding Funcadelic.JS Taras (guest) BigTest
Guest: Saron Yitbarek: @saronyitbarek | bloggytoons | CodeNewbie | @CodeNewbies In this episode, Charles and Sam talk to Saron Yitbarek about her idea of mentorship, ideas for distributed learning for businesses to promote individual and company growth, and why it's important to take "digital sabbaths" on the regular. This show was produced by Mandy Moore, aka @therubyrep of DevReps, LLC. Transcript: SAM: Hey, everyone. Welcome to Episode 110 of The Frontside Podcast. My name is Sam Keathley. I'm a developer here at the Frontside and I will be your episode host. Today, we're here with Saron Yitbarek, discussing mentoring. She is the founder of CodeNewbie and the host of the CodeNewbie Podcast. Also with me as a co-host is Charles Lowell, who is also a developer at Frontside. Welcome Saron and welcome Charles. How are you guys doing? SARON: Thanks for having me. I'm doing pretty well. CHARLES: Hello. SAM: Today is going to be an interesting take on the mentoring talk. I mostly want to know first off, Saron, how do you feel about mentoring? What are your opinions on the mentor-mentee relationship or the value there? SARON: Yeah. I have lots of opinions on this topic. I think that the traditional structure of mentorship was usually looks like someone with less experience going to someone who has a lot more experience and say, "Will you be my mentor?" kind of like the children's book, 'Are You My Mother?' like 'Are you my mentor?' and then that person, that mother figure, that mentor looks after them and checks in on them and they have regular coffees and lunches and kind of steers them in the right, usually career-related direction. I don't think that's very realistic, to be honest. I think about why that might be. There's many different reasons. I think the fact that we're so, so, so networked and there's just so many different ways to get in contact with people and build relationships is a big reason but I think that that traditional mentorship model, that kind of one directional way of doing things is just not really needed and kind of overrated. I think that mentorship nowadays looks more like a mutually beneficial relationship, where I might reach out to someone who has more experience than me, for example in drawing, in art, not something I want to get to do but I know a crap-ton about podcasting, so I help you, you're my mentor in this specific area, in this specific topic but then, I get to be a mentor in this other thing that I'm really good at. I think it's those types of very focused topic-oriented, ideally two-way relationships that are more accurate and frankly, a more effective way of doing mentorship. SAM: Yeah, I actually agree with that. I am pretty new myself to development, only really been in this career for about a year now and I always kind of consider that mentoring relationship as a regression back to school days, where you have these -- SARON: Yeah, yeah. SAM: -- considered superior over you in some way, well, that's really not true in this community of developers and no matter where you are in development, it's all about working together and pairing. Working here has really showed me that value in paring, rather than like I have to look up to someone and regress back to feeling like a teenager in high school, like this person so good at this thing and they're the only one who can teach me. I definitely share that view of mentoring. I went to a boot camp one day when they were telling you like, "Oh, you got to find a mentor. You got to find a mentor," and I was like, "Well, but why? Why do [inaudible] together?" SARON: It's also a huge responsibility, a huge burden on the mentor too. Having to be, in a lot of ways, responsible for someone's career and trajectory and direction, that's a big responsibility. We don't have time for that, you know? On both ends, whether it's feeling like you're back in school or feeling like you have this huge responsibility, I think the traditional model isn't really the best model for either party. I think this idea of let's all learn together, let's be really focused on topics and problems that are very particular to what I'm doing, what I'm learning, what I'm trying to do. I think that works out. It feels healthier for both people. SAM: Absolutely. CHARLES: I wonder also too, it seems like it might be a little bit of a throwback to the days when people would spend 30 years in a single company or 30, 35 years in a single career where you have these people who are really these reservoirs of this intense tribal knowledge. It seems like people move around a lot more in their careers, not only in the company that they work for but also in the things that they're literally doing. I might be podcasting one day and producing a bunch of content and then, I might move into music or writing or other things like that. The careers seem to be broken apart a little bit more is one of the reasons why older models of advancement in those careers might not be as good a fit as they once were. SARON: Yes, absolutely. I'm obviously very biased because I'm in tech but when you think about the different roles that people have in tech, I feel like we're always wearing so many different hats. We have to, obviously code and be technical in that sense but we have to be really good communicators, a lot of us are speakers, we host podcasts, we organize conferences, we go to meetups, we're bloggers, we do so many other things, that this idea of, "I'm going to have a mentor who can help me in my role of being a developer," is just too big. You kind of meet people who are good at those individual pieces and those individual skills to get me to where I want to go. SAM: The whole idea of mentoring to me always reminded me of that whole Mr Miyagi relationship where you have this master of something and you're trying to learn from them. But in software development, I've learned that really no one is a master of anything because it's just changing so much and everything is so different. SARON: Yup, absolutely. CHARLES: The question is obviously, it seems like the idea that you're going to find one person who's going to represent the ideal confluence of every single skill set that you could hope to want, to be at some point in your career. That's looking more and more ludicrous. Is there a model where you can try and distribute those things, where you single out a large range of individuals? I guess the kind of what you were hinting at the beginning is that it is a lot more broken up, a lot more distributed but how do you pursue that, even if it is in a distributed manner? SARON: Yeah, that's a great question. One of the moments where I realize that the distributed model is really the only way that makes sense is, I think it was three years ago maybe. We thought about doing some kind of mentorship program in CodeNewbies, some type of way for people to link up and find people who can be there and guide in a way and we had people fill out this survey that basically said what do you want out of a mentor, what do you look for, what do you hope for to achieve but also asked how do you see yourself. Do you see yourself as a mentor, a mentee or both? It was really surprising that most people checked off both, which I thought was so interesting. It was so interesting to me that the same people who said, "I need help," with the same people who also said, "I also have help to give," and to me, that was such an amazing moment because I said, "Wow, it really isn't about this idea of I am the guide and I'm going to guide you." It's really about, "I have information expertise in one area and not in others." What I realize is there really isn't a mentor model. I think it's more of a culture of being helpful, which probably sounds really cheesy but it's true. I really think it's about saying, "I know how to do a thing. I'm going to go on Stack Overflow and answer questions. I'm going to go on Twitter and answer questions. I'm going to write blog post and share with the world." I think the real model, the real distributed mentoring model is us as individuals saying, "I just learned how to do a thing," or, "I just figured out how to do a thing well. Let me capture that. Let me capture it in a response, in an answer, in a forum, in a post and let me share that," and the more we do that as individuals, the more we have this huge amazing aggregate of knowledge that can serve as mentorship for all of us. SAM: I like the touch on that. That's kind of the idea behind Codeland Conference, where everybody thought they might be new to development and everyone is just sharing their knowledge. You might feel you really knew about it but you know a lot more than you think you do and you could still help. SARON: Yup, exactly and that's the whole idea, even the Twitter chats. When we first started, I don't have all the answers, I don't claim to, I don't really want that responsibility but I know there are a lot of people who do and I know a lot of people who have resources and opinions and who can help out. One thing that people do is they'll DM us and say, "I'm having a hard time with this." Sometimes, it's a very technical problems. Sometimes, it's a general 'I'm having a hard time getting a job,' which I do like high level question and I never answer them. I always say, "Tweet us and we'll retweet it and we'll get the whole community involved and we can have a rich conversation around it," and that's really been our motto, our philosophy. I see what they do with mentorship. If you have a mentor, you're not obligated, I guess to listen to them but the idea is kind of that you should. There is one person and you should listen to them because they know more than you. I think that's just not really fair and instead, I like to think that there are lots of different ways to do things and it's up to you to decide what's best for you and in order for you to decide, you have to have a lot of options. With Codeland, with the Twitter chats, no matter what skill level you're at, no matter how confident you feel in your coding abilities, you do have something to offer. Let's pull that out. Let's put it on the table and let's see who you can help today. SAM: That's a perfect way of introducing someone to the idea of helping or getting help from a community, rather than an individual because you never want to take one person's word as gospel on something you just know nothing about. SARON: Yeah, exactly. SAM: You can sound confident talking about something and it can be the total wrong answer but if you're seen as someone superior or someone who knows what they're doing -- SARON: And you can sell it. SAM: Yeah. You can be the best snake oil salesman in the world. SARON: Absolutely. For the CodeNewbie podcast, we do short questions at the end of each episode and one of the questions -- my favorite question -- is what's the worst advice you've ever received. I love that question because I started by saying, I think that people love giving advice. I think people love asking for advice and the assumption again is if I give you advice, then I probably know more and you should probably listen to me but that's not always true. I want people to be comfortable making their own decisions and deciding for themselves. "This may sound like it could work and this may sound like a good idea, generally speaking, but for me, it's probably not a good fit. It's probably not what's best for me," and to be comfortable rejecting advice and that was kind of the reason why I started that question, it's my favorite question and it's so interesting how much advice is good. It's kind of generic but it's a good advice. It's an advice like, "Don't quit. Keep going," which maybe a good idea and maybe you do need to quit, so being open to rejecting advice, I think is really important and that one of my favorite questions. SAM: In your experience, what is the best way to reject advice? Because when you're a new developer, when you're new at anything and you're seeking advice from a community or from a person, you don't want to come off as rude or maybe you're feeling... I don't know, not very confident but you think the advice that you were given is just bad advice. What would your advice be? I guess, what would your advice be on this advice? For your perspective, what would be a good advice to reject advice that you think is just wrong? SARON: Number one is when I ask people for advice, I try to think about and try to know upfront, do they have the same values that I do? Do they have the same worldview? Do they have the same goals? Because that's the thing too. Oh, my God, I got so much unsolicited advice. It's amazing. I've actually stopped just saying to people my ideas or what's on my mind because I know as soon as I say, "I'm thinking about this," I'll get a whole slew of unsolicited advice and I'm like, "I didn't ask for that." What I've learned is first to kind of figure out are we even on the same page because if my goal is to be a developer, if your goal is to be -- the only thing I could think of is a juggler, I don't know why -- a juggler and I ask you for a career advice, I'm going to go ahead and safely assume that what you have to say is probably not as applicable to me and so, number one is kind of identifying that. But number two is if they say something that just I know is not going to work or I've already tried, I'll just nod and say, "Thank you very much," and kind of go about my day. I don't think you have to declare whether or not you going to take it. I think just acknowledging and I think the people that give advice, I think they're trying to be helpful, they have good intentions, usually. Usually it's, "I'm trying to save you from making mistakes that I made and I'm trying to help you get to learn a little faster." I always appreciate it but knowing that I can say, "Thank you. Thank you for your time. Thank you for your perspective," but know that I don't have to go off [inaudible]. SAM: That's actually very similar to my tactic. Just not like, "Thank you. Thank you so much. Don't talk to me ever again." No. SARON: And disappear, yeah. SAM: I'm like the wind. With this pressure that either new developers or seasoned veterans are feeling about the mentor relationship, because I feel like a lot of senior developers that I've spoken with or people who've been in the business for a long time, feel like they should be mentoring or they need to take on that responsibility but there's always that hint of dread in their voice when they say about like, "Oh, I should be doing this." I always feel like it's okay to not do that. I never really understood why it was so high value to have this one-on-one relationship with an individual when you're not in school because it just feels so like... Not childish but childish. SARON: Yeah, that's one of things, frankly that I love about the tech community and also do not understand about the tech community. It's such a giving knowledge sharing community, whether you do it in a tactful way or not in a tactful way but the idea of giving back and paying it forward is just so deep. It's so, so deep that even when I've been coding for only a couple of months, I still felt this, I don't want to call pressure because pressure kind of sounds a little negative but I definitely felt this expectation that I was supposed to be blogging, supposed to be sharing and shouting and helping and doing these pay it forward type things. I don't really know where that comes from. Maybe that comes from the culture of open source, maybe that's where it kind of penetrate. I'm not sure but there's this huge need, desire, idea that we're supposed to be giving back and for that, I am very, very, very grateful. But I think that acknowledging that if you're someone who wants to be a mentor that you can do it simply by being available, literally being available, the going to be hashtag is super, super active even when we're not doing our Twitter chats and people use it to ask for help, they used it to ask questions. If you're feeling particularly giving or extra helpful that day, go on Twitter and check the hashtag and see what questions people are asking. Things like that or just super helpful and don't require a huge amount of time and effort. There's a lot of small ways to help out. That may not seem like a big deal to you but for the person who's asking for help, who has been banging their head against the wall, it's hugely valuable. SAM: Yeah, absolutely. Up until I went to my first conference this year, I didn't realize how supportive and important Twitter is. You know I always kind of considered it to be like another social media platform that I don't understand because I'm 84 years old and I just don't get it. It's been so uniquely helpful and in ways like Stack Overflow or even issues in GitHub, it just aren't. You can get so many more perspectives, so many different perspectives from people. SARON: Yeah, absolutely. Twitter has been amazing exactly for that. It's just an efficient way to crowdsource opinions and crowdsource perspectives and when you get your question answered and someone else answers it, it doesn't only benefit you the way it would if you email someone but it benefits anyone else who comes across that page, so yeah, it's hugely valuable. CHARLES: I remember the first moment I had kind of like that, a light bulb went off in my head where I was working on some really weird project that was using some strange wiki for its content storage and I was getting frustrated and I just tweeted about it and then the CTO of that company just immediately answered my question and I was like, "What?" You know, it was years ago and definitely, I was like, sound of explosion, that is where my mind exploded. I was not seeking help. I was just literally being kind of a jerk and venting frustration and lo and behold, the answer for my problem descended from the Twitter clouds. It was incredible. SARON: The Twitter clouds are the best clouds, usually CHARLES: Twitter is very... What's the word? It's very split down in the middle. SARON: Noodie? Yeah, there you go. SAM: There's this live feedback, so there's no buffer of emotion there, you know? SARON: Yeah. SAM: It's like, "Oh, this thing that you said, it made me mad. I'm going to tell you about it right now." Charles, I know that you had mentioned before that you think mentoring would be a good idea for Frontside and then, after all this discussion, have your views stayed the same? What are you feeling about that? CHARLES: As kind of the person who's like the grizzled veteran in the software world, it's definitely something that I've kind of whipped myself over the back. It's like feeling like it's something that we should do but I think it comes from the idea that people come here and we want to make sure that they're getting access to the learning that they need and the ways, in which they can level themselves up that they need, that's the kind of the prime motivator there. I always perceived mentorship as some vehicle through which to achieve that. It's something that I've heard. Obviously, we don't have a mentorship program at our company. It's something I felt that we should always be investigating. I've always felt maybe a little bit bad that we didn't have it but it's also something that I really struggled with in my career because I can't really say that I've ever had a mentor, so I don't really know what that relationship would look like but I do have a lot of people that I learned a lot of critical things from. I can look at it as kind of these seminal moments in my career, where like light bulbs went off and a lot of the time, they're associated with an individual and that individual and the thing that they taught me or multiple things that they taught me, are still with me. I have those experiences, which have been phenomenal and critical to my development as a software developer, so I guess it's just part and parcel of that impulse that you're describing to pay it forward, to realize that when you walked into the building, the lights were on and the walls were standing and the air was at a comfortable climate temperature. As you live there, you realize that there are people involved in actually, doing that maintenance and providing the building for you and the space for you to become aware of your world and then, when new people walk into the building, you want to provide them the same experience that you had. I guess that's my take on. That's the kind of thing that I would want to provide, so the question is like what does mentoring or mentoring 3.0, as we're maybe talking about in this conversation, how does that fit into that? How would you implement something like this, some sort of distributed learning in a company? I don't know. Maybe, it's not worthwhile. Maybe it is. SAM: I think it focuses more on that pair programming because when you're thinking back and you have all these people, like you have names of people that taught you something, it's multiple names. It's not just this one guy taught me all of these things. Actually, within a company, that mentoring just comes from pairing with your coworkers, seeing what different hats everybody wears and then, trying them on every now and again but not necessarily taking that individual's word as gospel, you know? CHARLES: Right and hopefully, that's not something that we've advocated for. Maybe, it's how do you introduce structure around that, to make sure that the proper ferment is happening, so that you have novel pairings and make sure the ideas that are flowing are flowing around the entire company and not just through certain set channels. SARON: Yeah. The other part of that is if you create structure around what it looks like to share outside. You know, pair programming is interesting because it's kind of one-on-one and it's hopefully, I have something important or bright to say in our pairing session. Maybe, I don't. Maybe I'm having a dull day. Maybe, I'm having a bad day but this really give you the same opportunity to take a moment and say, "What do I know? What do I want to share? What do I want to put together?" It does really give you a chance to prepare and gather yourself. It's kind of in the moment. I think having another opportunity where you can gather yourself is important and so, that might look like brown bag lunches, where everyone takes a turn and has to do a little lightning talk. That's usually the opportunity to say, "I have five minutes. I'm going to share something." Everyone has a turn, which means that the company's literally saying everyone has something to say. It's only five minutes, only a few minutes, so hopefully it won't be too terrifying if you're not big on public speaking and it's your co-worker so hopefully, it won't be terrifying because it's people you know and not total strangers. You know, a format like that, where there's structure but the company is saying, "We're going to give you time to think about what you're good at and what you know and to share that is good." I think another way to do that is by setting time aside for blogging. If your company can say, "Thirty minutes out of the week, we're going to take some time to write down five things I learned, to write down one cool thing I learned, post it publicly, post it internally about this expectation that everyone should be writing, everyone should be sharing, which also says everyone has something to share." I think those are two ideas and two ways that we can create a culture of sharing and a culture of distributed mentorship, where everyone has an opportunity to find the thing that they're excited about and specific ways to share it. SAM: Yeah, that's excellent. CHARLES: I hope you're grinning as much as I am, Sam because at least in the first half, you basically described our Lunch and Learn process. It's a little bit more than five minutes but I think another thing that clicked for me there is making sure that having a variation of expectations of quality or something like that because I feel like where we do really well is in the Lunch and Learn thing. We have a process very similar to what you describe but where we're not so good is maybe with blogging. I wonder if part of that is we just hold ourselves to such a high standard of what it is. The idea that we could throw together a blog post in 30 minutes, I love that idea but it means you really have to be willing to just say, "You know what? We're going to get it out there and we're going to make it bite-sized and the expectation is that we're not going to be writing some gigantic essay that's going to shake the industry to its core every Friday at three o'clock." There are things and if we can, I shouldn't say a reduction in quality but maybe a reduction in scope so that you can say like, "We're going to carve out 30 minutes or an hour and we're going to pick a topic that scope-appropriately for that." SARON: Absolutely and I think that knowing that you only have 30 minutes or maybe it's an hour -- an hour is probably a little bit more realistic -- but knowing that you only have an hour also forces scope. You can't write a book about JavaScript in an hour. You just can't, so the fact that by the end of that hour, you have to have something to turn in is a great way to force people to focus and do something really small and really bite-sized. The other thing is I don't think that after the hour, you need to publish it publicly. It could be, you need to turn it into someone else, to edit and look over for you and give you feedback. It's not quite ready yet but it's a solid first draft and the ideas that after that first person edits it, then it's on its way to being published. I think there's different ways to manage a scope without also making this scary thing where I have to say something for the Earth to shake. There's different things that we can do around that. CHARLES: I'm wondering what other forms of sharing that we can fit into our workday. I guess, the other baseline is just making sure that you're always... What is it? ABT -- always be tweeting. It's so easy to be write-only, not even engaging conversations but just throwing ideas out into the void. SAM: I think holding a discussion with your coworkers like if you're in an environment where you can work face to face or are constantly online, if you're more of a remote worker, I think just having a conversation with anybody, rather than just putting your idea out there, putting it out there with someone who can actually provide real time feedback in a more friendly way, then I think some people on Twitter could be. Because they're your coworkers and they're not going to call you rude names, you know? CHARLES: Right and you're also going to know, hopefully the whole trust and intent and 'are we on the same page?' that question is answered even before the text is written. SAM: Right. SARON: Yeah, absolutely and with those conversations, this actually mean [inaudible] that we do. We do like a show and tell every week and the idea is that we're both always learning random things, usually related things but sometimes, totally random but still very interesting and it will take 30 minutes to just share what we've learned. Sometimes, they'll also turn it into a blog post. Sometimes, it's just a knowledge sharing opportunity. I'm not really sure but it's a good window of opportunity to say, "We are learning and we're sharing and we have something of value to bring." The great thing about something like a show and tell is that it doesn't necessarily have to come from my brain. It doesn't have to be like, "I have a great idea that I'm going to share." It could be, "I read about this cool idea." It can be, "I heard about this cool thing that we can try and we can apply," but I still kind of credit, you know? Like I get credit for being the value bringer but the burden isn't on me every week to come up with the idea. That's a nice balance, to kind of create space to share and to promote this knowledge share and if it comes from you, it's great but if not, you're still helping other people. CHARLES: I have a question that may or may not be related in here. This has just kind of occurred to me because this is definitely something that I experienced, where I get into a mode where I become overwhelmed by the ideas that people are sharing and so, what's the balance? Because usually, we're out there searching for ideas and we're searching for novel things so that we can include them in our work, in the things that we want to do and accomplish, whether be that in tech or elsewhere. What's the balance of being heads down and being like, "You know what? I'm going to be closed to new ideas right now." Because they can be distracting, right? The [inaudible], that's actually a phenomenon and so, how do you protect yourself from sharing? What's the balance? SARON: My solution was to move to San Diego. That was my solution to that. I actually moved from New Jersey and I worked in New York City... How long has it been? Was it only been a year? Oh, my goodness, and now, I'm in San Diego and it was so interesting because that ended up being a really nice side effect. I didn't move specifically for that reason but when you are commuting in New York City every single day, there's so much going on all the time. There's just so many ideas and events and meet ups and companies and people. It's just so much. I think that it was a great place to be in my early 20s when I really just wanted to soak up everyone else's ideas and I didn't really have opinions of my own at that point and it's a great way to just kind of absorb and be this awesome sponge in the big city but after a while, I kind of realized, maybe I have my own ideas and my own thoughts, so moving to San Diego, which is a much, much, much quieter place, has been a really great way to reflect and sit with my own thoughts and feelings and opinions and just kind of focus on that. I understand that everyone can move to San Diego, although I highly recommend it but I think in that way of carving out like... What do they call it? Do they call it a tech sabbath? A digital sabbath? Am I saying that right? CHARLES: Yeah. Maybe. It sounds about right. SARON: Yeah. I came across that term recently but this idea of one day out of the week, I'm not going to be on the interweb. I'll just not going to do it. Maybe, even the whole weekend, oh, my goodness and saying, "I'm just going to stay away from things. I'm just going to create a little space for me to think and reflect." I think when I don't have the whole day and what I need is just like a moment, I find that writing things down is a great way -- a really, really good way of doing it -- even if I'm taking notes or from doing a strategy session or if I'm trying to make a decision. Usually, I'll start typing and what I've started to do recently is to say, "I'm not going to type. I'm going to plot in a notebook and I'm just going to write things down," and because writing is slower than typing, it forces you to just think. It forces you to be alone with your thoughts, for better or worse and it forces you to really just to think about what you're doing or what you're saying and reflect. It's a really, really great meditative exercise that I found. You know, finding little ways to build in an escape from the noise, ideally on a regular basis, I think is a really healthy thing to do. SAM: Yeah, I definitely agree with that. My normal method of sort of closing everybody off is I just sketch out ideas because I'm an artist. I come from that sort of perspective as I can make a drawing out of feelings more so than I can write them out. If I'm feeling really overwhelmed or if I'm trying to make sense of some information that people have given me, I just sketch it out and I think that's something that doesn't really get talked about a lot in the whole community because sometimes it's always about eat, breathe, live, code, you know? We have different skills. You have other things that you're good at that's not just code. When I was at React Rally, a lot of people were in the music and then finding a way to separate yourself from the entirety of it is definitely one of the better ways to make sense of your situation. SARON: Yup, absolutely. CHARLES: Yeah. Sleep? It's a great way to take a break. SARON: Sleep is so good. SAM: On my top three things: sleep. CHARLES: Yeah. SARON: Yeah. As a kid, I never ever thought I would look forward to my bedtime ever. You know, it's my favorite part of the day. CHARLES: Yeah but like sleep, writing, drawing, I always forget. Like when I'm caught up in a problem, I always forget that engaging in some other activity -- I like to walk in the place next to my house. I like to play my ukulele. Engaging in those activities is almost on the critical path to solving the problem and I always forget it. I think we always forget it but sometimes, you can be so frustrated and you can just shut it off, be completely alone and there is some magical process that's probably going on in your brain. I don't know exactly what it is but you come back and the answer is just sitting there, waiting practically on your desk. SARON: Absolutely. One thing, we recently moved a couple of weeks ago to a new place and it's a little bit bigger and so, I had the opportunity to unpack boxes and buy furniture that we didn't really need before or just trying to make it more of a home. I've been using it as a really great opportunity to be productive and to feel productive but not be in front of a screen. Sometimes, I even like save tasks for myself like, "I'm going to wait till the evening to put together this shelf because I'm going to need a moment to just be away from my computer." I have a plan around it so it ended up being such that, I think about every day, there's one little home activity thing that I can do, whether it's cleaning or cooking or assembly or movie or something with the home, that allows me to, because I'm not really a hobby person. I don't really do hobbies because it just feels, like no judgment on people who do. I know people get really into their hobbies but it just feels like, "Why?" You know, like, "Why?" like, "For what?" But when I put a shelf together, it's like, "I'm going to use this for books." You know what I mean? Like you're very purposeful, so home activities has been my way of carving out that space away from the screen but also feeling like I'm doing something productive, I'm getting things done. SAM: Something that I would recommend to anybody who doesn't really have that, I'm going to step away from the computer and do this thing. Something that's kind of helping me was the Pomodoro Technique, which if you're not familiar with that, it's basically a time management thing where you set your timer for work like for 25 minutes, 30 minutes or whatever and then, when the timer goes off, you take an allotted break for five minutes, 10 minutes, just so you're not doing the thing that you've been doing for the last 30 minutes. SARON: Yes, absolutely. That's a great one. SAM: My advice is to implement that if you don't have a thing, that you use for your cleaner, I guess. SARON: Oh, you want to hear some really terrible but effective advice? SAM: Yes. SARON: And this is what I found out very accidentally is if you have a really, really crappy office chair that hurts -- CHARLES: Oh, man. I'm sitting in one right now. It's literally a rocking chair from the 1800s. SARON: That sounds awesome. CHARLES: Yeah, it does sounds awesome. SARON: But if you have a crappy office chair, that can be a really great way to get away from your screen because after about an hour, your thighs will hurt and your back will hurt and you will be forced to get up and walk around. It's so funny because I have a pretty crappy office chair and my back has been hurting for so long and I just thought, "That's life." That's like my [inaudible] for myself. I'm like, "That's just how life is, my backs hurt," and then it got to a point where I was like, "I need to go and talk to someone," and as soon after that, I spoke to a conference and I was basically up and down away from any type of chair for like four or five days straight and magically, my back pain went away and I was, "Oh, my God. It's that freaking chair," and ever since I realized it, I started noticing it. I would sit down and I would go, "I'm nearing the one- Oh, there are my thighs. There they go. They're in pain." There's another great cheap life hack: get a crappy chair, after about an hour, everything will hurt. You will be forced to go do some jumping jacks for about five, 10 minutes and there's your additional sabbath. There you go. SAM: In Charles' case, probably a haunted office chair. Yeah, that's an excellent advice. I never really noticed that. Now, I'm going to. CHARLES: You know what funny is, I actually didn't even notice it but I do. I never used to get up and go on walks before or spend as much of the day standing as I do and I'm actually completely and totally oblivious to it but I think, I realize like I can attribute a lot of that to the terribly, uncomfortable chair, in which I sit on every day. SARON: I have this awesome habit of sitting cross-legged, which apparently is a very, very bad idea, my physical therapist told me, so I make it worse for myself. If you don't feel like you have enough screen time, just sit cross-legged and that will accelerate the pain process. SAM: As you said that, I am currently sitting cross-legged in my desk chair. SARON: Yeah! SAM: That's just how I sit in chairs. My legs are too short to reach the floor. SARON: Me too. I just find it so comfortable. I don't know and I'm so happy to hear that you also do this because I'm like, "Am I just weird?" because I love sitting cross-legged. It's so comfortable. It makes me really happy. SAM: No, I verified 100%, that's how I sit in all chairs. SARON: Yeah, the same. CHARLES: I can do one leg. SARON: Only one? Man, you need to work on that. SAM: -- [inaudible]. That's going to be the end of our podcast. Again, thank you Saron for coming on and having this awesome conversation about mentoring. Definitely check out the website CodeNewbie.org. We are the Frontside. We build software that you can stick a future on. Hit us up if you're looking for help building your next big thing and also, hit us up on Twitter. Hit Charles, myself, Frontside and Saron on Twitter. SARON: Thank you so much for having me. This is awesome. This is fun. SAM: Also, I want to give another thanks to Mandy, our producer for producing this lovely episode and all of our episodes. If you have any questions for us, hit us up on Twitter. Any ideas for future topics, any thoughts you have on this great conversation, let us know and we'll talk to you all later.
In the last episode, we spoke a lot about the "why?" behind microstates. This time we wanted to cover how ideas in Microstates map to different patterns used to build JavaScript applications using frameworks like React, Ember, Vue and Angular. We discussed what you need to know about Microstates if you prefer component state architecture or global store architecture like Redux, as well as how setState in React can be refactored to use Microstates. We closed off with the comments about the trade offs that component heavy frameworks make by overemphasizing the view layer at expense of other aspects of the MVC pattern. This show was produced by Mandy Moore, aka @therubyrep of DevReps, LLC. Upcoming Conference Talks: Manhattan.js - August 8th (Taras) ReactJS Austin August 6th (Charles) Transcript: CHARLES: Hello everybody and welcome to The Frontside Podcast, Episode #107. My name is Charles Lowell and we are going to be following up with our Episode 106 with the exciting conclusion of 'Microstates, the Podcast.' With me today to wrap this subject up, at least for the near term, obviously we're going to be talking about it a lot in the days and months to come, are David and Taras, also co-developers here at Frontside. Hello guys. TARAS: Hello, hello. DAVID: Hi everyone. CHARLES: Before we get into that, we'll just make a few quick announcements. First off is here at Frontside, we are going to be having some availability at the end of August. If you or anyone on your team is looking to level up in the area of testing, especially testing single page applications, acceptance testing single page applications or if you need React or general JavaScript consulting, please get in touch. We would love to work with you and love to do some of the great things together. Second, we finally released this and this is germane to the topic at hand. We released a major version of microstates this week that's based on a new and simpler architecture. That's really exciting. It doesn't really change the content of the conversation because the API hasn't changed that much. It just means that the library, which was already very small is even smaller. I think we shaved almost 40% of the size off and it's just much simpler and it's much more harmonious with the underlying JavaScript runtime. It's even more just simple JavaScript objects. Unless there's any other news that you want to cover, we'll jump right into it. TARAS: All right. Let's do it then. Let's do microstates. CHARLES: I Love to talk about microstates. This is obviously the second podcast that we're doing on microstates, just because we ended up, I think it was two weeks ago and we'd been speaking for almost an hour and really, we're just laying out the problems that microstates solve -- the problems of state management and why you often run up against complexity when you have a single state management tool that doesn't account for a bunch of different use cases. We got into microstates a little bit but we left it as kind of a cliffhanger. We talked about transactionality and laziness and immutability, ease of composition, simplicity of API, performance, memory footprint, things like this. Those were all the problems and then we're like, "Yeah, microstates. It's awesome." We're going to talk a little bit more about actual microstates proper and what is involved like the adjustments in mindset that you need to make or don't need to make when adopting a tool like microstates. TARAS: Actually, it's been really interesting for me because I just gave a talk at Toronto JS on microstates and it was pretty cool to see. There was a panel at the end. I think the people that are representing just using component state as a way of architect implication. It was managing state and application and it's representing redux. It's really interesting to see, first of all, like how curious people were. Most of questions were directed to the discussion around microstate, simply because it is a new tool but it was also just interesting to see how curious people who really loves redux, he was like, "I really love redux but I really like that fact that you guys have types," so I was like, "Oh, that's cool." Even though you really liked your solution, you actually found something in microstates interesting but at the same time, I think he was kind of missing that what aspects of Microstates overlap with redux. I think one of the things that we can do today is talk about what microstates has and what API does microstates provides and how similar they are to what people already know. I think that's one of the things that I was thinking about this conversation is that there are some things around microstates API like how to use microstates but the architectural concepts that power a microstates, they are already part of the development process and development architectures for most people that building single page applications. I think we can do is try to map these ideas not what people really know and patterns that they use to ideas in microstates, just to show how close we actually are. I think that's a good way for us to go. CHARLES: I guess we can start with, maybe one particular mode of managing state and then, how that maps to uses with microstates. TARAS: Let's start with components state because that was kind of our starting point. One of the original starting points for microstates because so much of our work was in Ember and component state is where most of Ember development happens. I think that's a good place to start. If you're someone who uses components state, if you're using Ember or Vue without something that is redux-ish which is pretty common these days, if you use components state, then one of the features of -- CHARLES: And also, if you're using React, right? Ember, Vue and React pretty much are all the same in this regard. TARAS: Yeah. Well, Ember and Vue are a little bit different in that behalf. Their particular pattern is you have data coming into the component, then you're doing some work with the data so that in Ember and Vue, you might use computed properties to derive state on that and then you might pass that state further down into other components. In React, you would do something a little bit different because there is no memorized computed properties by default, so you, a lot of times, write your computations that you pass into other computers that are written as expressions in JSX. If you're using this pattern, then one of the challenges around using this pattern is the process of lifting state becomes quite complicated. Because if you have like a bunch of state that lives in the component, it's attached to your derivation of state and the properties that you can invoke are attached to a component object. In React world, it's called lifting state. In Ember world, it'd be essential refactoring it into the parent component and passing that state in. in microstates, you're essentially moving this state into the microstate and now, what you have is this object that instead of having the state live on the component, the state now lives on the microstate object and in all frameworks, when you do that in microstates, you can use the getters to get the memorized computed properties behavior on the microstate. If you use your computed properties, computed properties are available for you on the microstate. You can do it the same way. CHARLES: It's kind of like a wrap up of that idea, not a wrap up but a summary of that idea is that the microstate essentially is a substitute for your component properties. If you're working with a component, a component has state associated with it, so you're setting properties on a component and then using the computed properties of that component. Microstates is you have all those benefits where you can set properties directly. You can define methods that like set properties as part of a transaction and you can then have simple JavaScript getters, which are just like computed properties except there's no enumeration of dependent keys. It's just, that's all they do for you. TARAS: Then the next question that arises is like, "Do I make like one huge microstate or a bunch of smaller microstates?" I think the question is really depends on the role of your component and what's nice thing about microstate is in microstates, there's two things that are kind of cool. One is when you represent your status in microstate, it makes it very easy. If you need to lift that state from the component up into the parent, you can lift that type. You can take that type and you can compose that type into the parent's microstate, if you have to and in that case, you would pass this new object that's created from that type. You'd pass the state for that component to the component that would otherwise have its own state. Essentially now, the parent has the state that would otherwise be in a child component and now, the parent is passing a state from the parent to the child. There's two kind of benefits to that. One is it gives a parent a way to control the state of the child and it also gives you a really easy way to serialize and deserialize state. You start off with having ability to serialize state for a particular component and all of his children, now you can represent that serialized state as part of the components state, so you have an easy way to restore the state of the component tree at the parent component. CHARLES: Right and that's possible because a microstate is really just encapsulating a value within a type but the value is just a simple serializable plain old JavaScript object, right? TARAS: Yeah. CHARLES: I'm just not trying to say like this is the feature that microstate provide, so when you're creating a microstate, you just pass it a POJO and it off to the races with that POJO and you can access that POJO at any single point. TARAS: Yeah. This is part of the architecture that I think are helpful for people that are using components. Is there anything else for people who are using components state that we think would be helpful? CHARLES: I think we're just realizing that you get the benefit of the laziness and immutability and the transactionality and all kind of things that we talked about last time but actually, the mental model is very similar. I think the thing to realize is that if that's the way that you used to working with state, the bridge is not too far. It's actually quite a short one. TARAS: Yeah, that's good. CHARLES: It feels very natural. It's not a huge mental shift. It's just more about a very small mental shift, a very small shift in mechanics for a very large payoff. DAVID: For instance, say you're in an Ember application using component state, what I'm used to is you're going to be passing actions around between components. Whenever you're trading that out for microstates, is it more that these actions are just bundle in with the microstate as the transition? TARAS: Yeah. One nice thing with the microstate, because the transitions that you can perform on any data type are intrinsic to the actual data type, the part of the data type, when you instantiate a microstate, you get these transitions for free. When you pass the object into a component, you can now invoke transitions on that object, that are part of that object's type. It's hard to imagine how awesome that really is because the closest pattern that you would see for that would be like, for example, if you're using Ember, you have Ember object or Ember model that you pass into a component and then, you can work a transition and because it goes through the Ember data store, your component is going to update. With microstates, there's no observation of any kind but what you're doing is when you are invoking the transition, transitions going up to the top of the root and gives you the next microstate, which updates your component tree. You get that functionality of 'data-down, actions-up'' built into the entire system of the microstates and all of that is hooked into these transitions that you can invoke on the objects that you pass into your components. CHARLES: Right, maybe David and Taras, you all could provide a concrete example of what that looks like. Because I think it is something that when I was giving talks on microstates at Ember meetups, one of the first things I like to show is you've got all these actions that you're writing either on your router or your controller but they're really actions manipulating the same type of data. It really comes down to like you're pushing and popping things off of arrays, you're toggling Booleans, you're incrementing counters, you're setting this property on this object. These are actions that we're writing and in the kind of microstates world, that's boilerplate. Because the transitions are intrinsic to the data type of the microstate that you're working with, maybe we could provide an example, like what's an action that you describe that you would pass around. TARAS: I think an easier one would be like if you have a model for example, you're composing a model into your application state or you're compose a model into your components state, so you have this model type that gets instantiated and you pass it into component that represents a model. Off that model, you might have open status like is it open or not that gets consumed by the model component, to know whether the model should be visible. Then along with that is model is open state, there is a toggle transition that you can find inside your component that is going to automatically flip the visibility of open. What will happen when you bind that transition to some kind of action handler that you invoke inside your component, when you invoke that action, it will then trigger transition at the top of that microstates and then, it will create the next state, push it through which will cause your model to change the visibility. David, that does answer the question? DAVID: Yeah, it does and that's actually the example that I have come up with whenever asked. A very simple sort of Boolean toggle. CHARLES: In a couple of my earlier demos, I should dust off that talk that I gave to the Austin Ember meetup where I was able to create an input with a dropdown menu with basically, some pretty advanced mouse behavior, all without having any component state or like storing it all in microstates and a lot of pushback was, "Aren't you putting logic in the template?" and the answer is, "Absolutely not." The logic is in the models but what I'm doing is I'm composing the actions that operate on those models inside the templates but at the template level, the action is data. If you're thinking about, we always want to have, we always want to lift state and then push that state down to the application, what this is really saying is that the actions are part of the data. It is implicit to the value that you've got. TARAS: I think that's really powerful because we do think about actions as being something that you can invoke. Like with closures, it's an action that you can invoke but considering that operation, that piece of data that invokes a transition in microstates and is derived from the type of the data, I think it's a cool concept. That's probably, one of the things that is kind of new for microstates but how you use it in your application should feel pretty much the same as you would if you were creating action handlers on your components. CHARLES: Yeah, just like kind of bundled action handlers for free. TARAS: Right. CHARLES: Maybe the way that you wrap up on this one when talking about kind of what microstates has to offer for the Ember developer, the Vue developer is really, I would say someone who's used to working with like MobX. Would be another good example is when I first started using Ember back in 2012, Ember Object solved a whole lot of issues in a really profound fundamental way and I really, really was drawn to that as the best API to be working with state. What kind of became apparent was perhaps not the best implementation. This is not like bag on Ember Object. I think it was actually amazing technology for seven years ago when most of it was written and probably, even came from before like SproutCore. It's almost 10 years old, which is incredible but it's still under heavy use in 2018 and it speaks well of how well it was constructed. I actually don't have much of a problem with Ember Object API. I just think that the way in which it's implemented that API means that there are some problems that require a different paradigm, so very much I kind of see microstates as heavily inspired by those types of APIs but with, I want to say a modern implementation but an implementation that is designed to solve the problems that have come to light over the last five years of developing single page JavaScript application. TARAS: I want to add that, when you describe with Ember Object, it exists in every framework that uses immutability. In MobX, the observation introduces the necessity to wait for a bunch of things to resolve. In Angular zones, I saw a similar purpose to ensure that if there are kind of cascading or streams or all of that stuff gets settled in into kind of restful state before you can start to assert on what's going on and with Vue, their computed properties has something similar. I think because of the complexity of having to track lots of objects and then recomputing things based on the result, that complexity in microstates is simplified by the fact that you describe you transitions. When you write them, you describe exactly what changes, so we don't need to wait for things to settle down. When you invoke a transition, that transition is explicit and based on the path of where that transition is invoked, we know exactly what needs to change, so we need to only perform one operation to compute the next state. There is no other things that we need to wait. There's no other side effects that we need to resolve before we know what the final status. I think that simplicity can be applied to all the frameworks that are using immutable APIs and derived computations from those immutable APIs. CHARLES: Uhm-mm [affirmative]. TARAS: Should we jump into talking about what microstates has to offer for people familiar with redux? CHARLES: Yeah, okay. Let's jump right into it. For folks who are familiar with immutable APIs like Ember, like Vue, like other ecosystem using MobX, a shift to microstates might feel like what you can expect. What about people who are just using often React-land and they're just using like set state. TARAS: Set state is a little bit tricky because it does get complicated over time and then you get this kind of funky things going on after a while where you start seeing things like, "I'm going to set state on this component and then once the change happens, I'm going to change some states some other state," so this kind of cascading state changes. The other part that I find particularly more challenging in set state world than it is in, I think in regular components state, like what you have with Vue and Ember. I feel like the way that the transitions, the state change handlers become part of the component, I find that part particularly kind of fragile. When you start doing refactoring, when you need to lift up state, it's like -- CHARLES: What's an example of this? TARAS: When you start off and your component owned old state and now, you need to have that state being controlled by the parent, we get into a situation where like, "What am I going to do? Am I going to do something with props as they're coming into the component or I should probably just flip that state from the child to the parent?" and now, you're refactoring the internals of a component to lift up that state so you can then combined those operation with the parent's state transitions. But then, you have this kind of added complexity of the fact that you're working with immutable data in that place. You've got these three things going on: you're refactoring your child component, you are moving these things into the parent components, you modify a parent component and now, you're also managing the complexity of forming immutable objects. I think the fact that people make it work is kind of a testament to human resilience. The people are able to solve such challenging problems and this is not a super hard one but when a component is complex enough and when stakes are high enough, these changes can become fragile. This is what I think microstates simplifies is that by taking care of the model, it makes components much simpler and makes it possible for you to just render a lot of components that are functional components without their own state. CHARLES: It actually reminds me and like I said, we're talking about what it's like to use them, not so much the implementation but that's the exact same sentiment that the author expressed in his book that was very helpful in writing microstates, which is Brian Marick. He has this book called 'Lenses for Mere Mortals' and he's talking about the practical use cases of using lenses, which microstates leans very heavily on. What he says right in the introduction is it gives you by abstracting the location and compose it the way in which your data structures are composed, it makes them very refactorable. You can say, "I want to change where the state lives. I want to change the structure of this object and I want to move it somewhere else," and it's sounds weird to say this, because the structure of the object is not coupled to the structure of the object, it means that it's very malleable, so you can move and you can say, "You know what? I don't want this address to be embedded in this person. I want this address to be inside this address book and then my person has an address book entry." Being able to make those refactors make those changes is very easy because your method of accessing the data is abstracted. It's something that applies beyond even user interface as a component state but it's something that this is a problem that's very salient when you are authoring complex UIs and compose components. It's something that you can benefit very greatly from. DAVID: Taras, you were telling us earlier about a friend of yours who is learning React for the first time over the past few months and he's mainly been using set state as you would just starting out in this world. You said that he didn't really get why you might have to go and learn some other way to manage your state and I think Charles said it, whenever your application starts to get more complex and you've got a lot of different moving parts, that's really where microstates will come out and shine for you. TARAS: Yeah. My friend kind of feel bad because he's spent the last three months learning how to work with React and how to use set state, "And now, you're telling me, I need to learn something else? Like I'm going to start from scratch?" I kind of reassured him that especially when you're a beginner, when you're learning, you learn how to address very specific challenges but you don't know how complicated these things can get when your UI gets really intricate. In those intricate scenarios is when you have to leverage your experience and you'll be able to solve these problems but if you're learning, you encounter these challenges head on and your tool does not really, like set state. Although it can be used to build really complex UIs very effectively but the complexity that over time increases as you start to manage the state, increases much more later on than does early in the beginning. Basically, with microstates, even if you have some basic proficiency with set state, when you start working with microstate, the things that you can do with microstates, you will be able to do more sophisticated things easier when you need to use them than you might realize when you start. Because microstates API is so consistent, there are a very few concepts to actually learn and they cover a broader range of use cases so when you actually starts using it, you'll encounter these challenges. What you already know it will just work for you and always just continue to work for you. That was part of kind of fundamental design on what Charles and I have spent, a lot of time putting into place from a point when we started two years ago. CHARLES: I think, and you touched on this and this is kind of what I was speaking to earlier is that when you're working with a simple system, it's simple, it's easy to work with and then the complexity starts to grow, it's never a good thing when you have to reach for a more complex tool to manage the complexity. It's a hallmark of a good system and that it can scale with you from a very simple and easy use cases to actually being able to handle very large and complex use cases but your API doesn't have to change and your usage doesn't have to change. If you have a tool that can actually scale with you from a one liner to a hundred thousand liner, that reflects very well on the design of the tool. I think you see that with things like Ruby, you see it with things like Clojure. You see it with, I would say there's even people writing like Haskell, shell scripts now but not so much with like C++. I think the analogy is very similar with microstates in the sense that when I'm looking for evaluating a language, it's not the only thing I look for but I really am looking like how is this going to work for me as a one liner? How's it going to work for me as a hundred thousand liner? If the answer is pretty consistently, then that's something that is going to get a lot of bang for your buck, so to speak. If I invest the time in learning it, I'm actually able to reap the reward of having a tool that's got my back on a whole bunch of different use cases. We talked about, in React, if I'm using set state and that's kind of a sub case of component state because I would say that in the previous systems we talked about -- Ember, Vue -- they have components state but the component state is a little bit more rich in the sense that you've got computed properties and what have you. But if we look at a system that externalize a state like redux, where you have a global state atom in your application, at least that's the way most of the redux applications that I've encountered behave, what does it look like for you? TARAS: I think it's a little bit challenging when talking about redux is redux conflate a few different things together. I think it's helpful to split those things up, so we can talk about them separately. One of the parts is how the state is delivered to components. The way that redux does is the instance is this created so every time you use connect, you essentially wired together. You can connect through the context or... I'm not sure. I think they have an observation mechanism that's internal to redux as well but they essentially connect components to the store and then they view that to deliver the state. It's kind of worth pointing out that for that like microstates bindings for React actually give you something similar out of the box through context. For those who really like the ergonomics of connect, I think it would be pretty easy to make that available for microstates. Usually, we don't even know why they wouldn't be available. CHARLES: But I do think that connect can be problematic like you can encourage you to not make components that are reusable and have isolated state. It's very easy to hitch yourself to the redux store and now, you don't have components that are shareable. TARAS: I think I would personally prefer to pass microstates instances through props because of the stability that's built into microstates and structural sharing, you can get some free optimizations and allow it to use functional components in more cases like out of the box but some people who are really like redux, they really like connect. Although it might be my personal preference, there's no reason why that wouldn't work if somebody wants me to connect function and make it available for people. CHARLES: Right and I think, there's a happy medium there too, right? TARAS: Yeah, I think -- CHARLES: -- You can connect components and then fan out that state to a set of functional components that are not connected. TARAS: There are some places where microstates and redux are very similar. When you're using in redux, you have this dispatch mechanism, where you essential saying, "I'm going to dispatch this action and your action creator is going to create an action for you with the payload," and then you're going to match that action to a reducer. One of the things that I kind of hear redux people really enjoy about redux is the one with data flow that dispatch the action and then, it reduces this next state and pushes that through and you have this kind of ingress point where everything is going through this one point. I think what's really interesting with microstates is you essentially get that. That's exactly how microstates works, in a way. The only difference is that the API is different. Any action that you invoke is going to go through a single entry point, which is going for you. Because we know the structure of the data, we know how to transition that state for you, so you don't need to reducers, so you're just defining your transitions or use the built-in transitions and then when you invoke them, we know the path. We're essentially forming for you. The path where you invoke the transition, conceptually, it kind of forms the name of the action that you are invoking. The path refers to a place where the state is going to be transitioned, then you have your transition, which is the actual reducer for that part but it's contextualize, so you don't have to think about how you need to transition that state in a great, global redux state. CHARLES: There's no matching. The matching is automatic. TARAS: Yeah, the matching is automatic, so you get that same single ingress and one directional dataflow. You get those mechanics except the APIs that you use, instead of writing the actions yourself. Instead of writing, we just use yourself. You get to use microstate types. I've heard some people who use redux who are like, "I really love the fact that microstates has types," but other people don't like types for whatever reason. Microstates comes with 'from,' which allows you to take a POJO and from that POJO, it creates a microstate and then you can invoke transitions the same way as if you had a type microstate. The only thing you don't get is you don't get to create your own transitions. You have to use the transitions that are provided for the primitive types. CHARLES: I think that there's a couple of benefits that you'll realize for free. There's laziness, reducers by default, or eager. When you dispatch an action, it will run against every reducer in the store. If the reducer matches, you're going to run the computation that's associated with that reducer. Microstates by contrast is lazy. Basically, until you try and read the property that is affected by that reducer, the reducer won't run. There's some ways that you can get around this. When you're using redux and first of all, you can actually have your reducers return objects that have getters on them. You can realize some of that laziness but again, it's work that you have to explicitly put in. I think, didn't you actually say that there is a package of plugins? There are plugins. There's basically a set of libraries that you could bundle together, which would give us an experience similar to using microstates. TARAS: Yeah. If you wanted to combine redux and reselect and immutable JS together, you can get some of the benefits, except this benefits are not integrated that well because they're still separate systems that you are essentially using together. Also, like microstates, it's four times smaller than redux and reselect and immutable JS combined together. If size matters to you and ergonomics matter to you, you actually can get a lot of ease out of using microstates while still maintaining the benefits of having redux. CHARLES: But if those things, those packages, like reselect and immutable JS, are things that are familiar and you naturally gravitate for it, then you'll probably absolutely love microstates. Because honestly, one of the ways that I think about microstates is like, what if you could have immutable JS, if there was no cost for composing the types like list and record. Immutable JS has come a long way since I've last used it or it's evolved since I've last used it but I think there's still only about four or five basic types and actually, making your own new immutable structure, your own custom type with its own custom methods that still get the benefits of structural sharing and laziness that you have on immutable JS is not something that you can do. But you could think one way to think about microstates is an immutable JS where you can make any type that you want. You're not just constrained to the record types and to their list types and set types or map types. TARAS: It's worth pointing out that at the moment, microstates doesn't map perfectly to immutable JS simply because immutable JS has certain optimizations for managing lists that kind of a great value of immutable JS and microstates doesn't have some of those pieces but -- CHARLES: They're definitely on the road map. TARAS: Yeah. Because microstates is abstracted high enough, that we can actually change internals and some people who are using microstates now, they will get benefits of ergonomics and there's already performance benefits from the stability that microstates offers but there will be a time when by upgrading to newer versions, you will not basically, need to change anything in your op but you will get the benefit of improvements to performance that we will introduce over time. Some of those improvements might come from what we will learn from immutable JS. CHARLES: Right. I have a couple of thoughts before we wrap up. What are the ramifications no matter who you are? What kind of development background, some of the benefits that you'll experience with microstates that might be a pain point or something you hadn't thought about where you are currently? A couple of things that I have jotted down is first, and this is what brought it to mind is talking about stability. Something that you see in a bunch of frameworks is having to manually track the keys that are associated with data. If I've got a list or I've got an object, being able to say, if that object changes, then I need to actually have some sort of key object, which effectively amounts to a hashing function to say, "Did it really change?" Because no matter what system you're in, you need to know how you're going to re-render. If the reference to this object changes, then the default thing needs to be, "I need to re-render it," right? You see this in Ember, in Vue. In React, there's this ability to pass a key or ask the question, "Should this component actually update?" and with microstates, that's much less of an issue because basically, it keeps the key tracking instability for you. If you are using a model with microstates, if you think of an object as a graph of nodes, no node in the graph will change unless it absolutely has to, at least that's the goal. We still actually have some work to do when you're running queries against the state of a microstate. We can cover that later but that's most largely the way it is now and definitely, the way it's going to be going forward is you don't have to do any extra work as a programmer to figure out what has actually changed. TARAS: That opens up some interesting opportunities. Imagine if you had a rendering engine that did not expect side effects to be significant and you could just say, "If I know whence they'd changes, I will then re-render that." That will be really interesting exercise, seeing like what would that look like for a performance perspective, if you have a very clear picture of what has actually changed and what part of the DOM as a result, need to be updated without having to do diffing. The [inaudible] you could actually do a diffing at much higher up. Actually, [inaudible] to diffing because you know what's changed but you can push a lot of assumptions higher up on the architecture stack. CHARLES: Right. That's actually one of my favorite thing about microstates and one of the unwritten values is like triple equals used to work everywhere and by and large, it does. It's usually simplifying but when you don't have to manually tell the computer what equivalence looks like, you can just say, "Look, are they the same object? They're not the same. If not, then they're not," and keeping that consistent is huge. TARAS: I think this is a good segue for us to kind of bring this conversation to a close and also, kind of set up potentially a third full-on conversation, which we could talk about actual architecture of microstates and design decisions because for people who just want to use microstates, they don't need to know all these details but for people who are curious, they might actually want to understand what are the considerations that remain when we were designing microstates, so maybe in a next conversation about microstates, it could be about architecture and the pieces in microstates today and then where we are going with microstates and what it could give us long term. CHARLES: Yeah, I like that idea. It is a plannable subject that we've been talking about internally for the past two years, so it makes sense that there would be plenty to speak about on the podcast. There is one other thing that I did want to bring up and that is, I think enabling to have a state solution that is composable because it allows you to think about your state first. Because really, if you do have a functional UI, where your view is a pure function of the model, that your view follows the model and so, if the view follows the model, then really, the thing that you should be thinking about first is the model that's going to be required to drive your view. I shouldn't drive. I should say derive your view because that's the primary artifact of which the view is nothing more than a function. It's a reflection onto a surface. I don't think that we have a state management solution yet, that enables that mode of thought, where I'm thinking about my prime artifact first and working forward rather than thinking about my secondary artifact and trying to kind of wishy-washy way, work backwards and reconstruct the primary artifact. I think that we've talked about all the development ergonomics and I think there's a mechanic of thought there that's enabled by this that I hope to see in more and more applications. TARAS: I think that's a really well put. I think that's something that I've been thinking about as well, as how do you convey this shift that microstates allows in terms of how we're thinking about architects and the application. For some people that value, the model, like they'll find that shift easier but regardless, I think that making that shift has a potential of simplifying your view dramatically and I'm very excited about exploring this further and having more conversations about this. CHARLES: Yeah, that's where we really kind of open up the conversation about state machines, which is also central to the conceit of microstates and using state machines as an incredible design tool but anyway, we can all get into that later. You heard it here folks, Episode 3 is coming out, although probably not for a while. We're going to be mixing up and we will be talking about microstates at least for a while. I understand that next time, we actually teased it but we based on how much material there was on microstates, we ended up packing in a second episode. We teased it last time, we're going to be talking next time about running an online conference with Twitch, so definitely look for that. Thank you, Taras. Thank you, David. TARAS: Thank you. DAVID: Yeah, it's been great. CHARLES: This is a wonderful conversation and as always, we are Frontside. As I mentioned at the top of the show, we have availability coming in August, so if working with us is something that you would like to do, we have a range of services, please get in touch. You can get in touch with us at @TheFrontside on Twitter or Contact@Frontside.io. That's it for now. I guess we should also mention that Taras that you are going to be giving a talk on microstates at ManhattanJS. When is that? TARAS: On August 8th. CHARLES: I will be giving a talk on microstates at ReactJS Austin on Monday, the 6th, so that is right around the corner. I'm excited about both of those talks, especially following so closely on the heels of the TorontoJS meetup talk, which I understand is... Is that posted online yet? TARAS: It's recorded but it should be coming out soon. We'll definitely tweet it out. CHARLES: Okay. All right. Look for that and we will see you next time.
In part I of The Frontside's microstates series, Charles Lowell, Taras Mankovski, and David Keathley talk about state management that's easy and fun and transactionality. This show was produced by Mandy Moore, aka @therubyrep of DevReps, LLC. Upcoming Conference Talks: Toronto.js - July 30th (Taras) Manhattan.js - August 8th (Taras) ReactJS Austin August 6th (Charles) Transcript: CHARLES: Hello, everybody and welcome to The Frontside Podcast, Episode 106. My name is Charles Lowell, a developer here at Frontside and I'm going to be hosting today's episode and we're going to be talking about microstates with fellow Frontside developers, David Keathley and Taras Mankovski. Welcome you all. DAVID: Hello. TARAS: Hello, hello. CHARLES: I'm really, really excited that we finally get to talk about this but before we jump into this, just wanted to let everybody know that last week, we published our roadmap for Big Test, which is a JavaScript acceptance testing framework that we've been building right here, in-house at Frontside, which can work with both Ember applications and React applications, Vue applications. It's in alpha phase and we're looking for feedback but we're really excited about it, so we're going to leave a link to that in the show notes and please go check it out. All right. Finally, the moment has come that we actually get to talk about this publicly. We've been publishing things about microstates for a while now but we feel that it's ready to share with the world and that's really, really exciting. We should probably like wind the clock back a few years because that's about how long we've been working on this and talk about, kind of the why and the what of what microstates are. It's kind of a weird word. What are we doing here? TARAS: Yes. That's interesting question because we were working for so long and after all this time, what is this specifically we're working on. I can speak personally from my personal motivations because we have conversations over the last two years to why we were doing this and I think for me personally, it's always been that I've been mentoring building complex applications for almost five years now and one of the things that I find consistently is that there are patterns for how to build complex stateful UIs that the required solutions, that are fairly reliably consistent. I can teach people certain patterns and then they can use patterns to build complex applications and those patterns scale really well, the challenge is that, there's not an easy way to express them and it's different for every framework. The way that I would teach somebody how to do it, for example in Ember versus how they would do it in React, even though the pattern itself is the same but the implementation of the pattern is different but it's different in such a way that it's very difficult to see where the consistency is, what is the same about these two patterns. It shows me that there is room for improvement. In the same way that if there is an opportunity that in the future, components will get unified under one umbrella or what component spec. The fact that we do states differently across every implementation, across every opinion, it suggests that what we might be missing is something that would unify across frameworks, how we actually do statement management. CHARLES: Yeah, that makes sense because as much as people try to buck the trend of MVC, it can't keep coming back in the rear view. Hopefully, not like a horror villain but like a caring friend, like a reliable pattern. I remember when I was in writing Java applications back in the day, the most important thing when you were writing a swing application was making sure that you had your models right. If you were modeling either a form or a dialogue or a set of pages, the most important thing was to have those models. Back in the day, we modeled those things with event listeners, very similar to where the way like a Backbone application used to work or if you're familiar with Ember Object, the way that it worked basically, adding observers to some model so that when that model change, you were able to react to that. Your representation was able to be 100% dependent on this model. That was like a Java Swing Application, which was then inherited from this pattern from Smalltalk but we keep on seeing this again and again and again. There is this piece, this state, that if you're going to be representing something, then you need to be able to get at the meat of it. That's ultimately what the model is in the MVC pattern. I remember when React first came out, everybody was like, "Oh, MVC is dead," but now in terms of state management and all of the state management solutions that we see, what that really is, is the model trying to reassert itself. Because it's such an important piece of your application, it's going to rear its head and you cannot escape it. I make it sound like a problem. I guess, it is a problem but there's a proliferation of solutions out there that are attacking the problem in order to represent something you have to know what that something is. DAVID: I think when you only work with the one framework, your perspective might be very much shaded by your experience. If you're really good at building React applications and you have subscribed to POJO is king and you don't need anything beyond that, then you're going to get good at handling POJOs and you might not realize that there is actually limitations to what you're doing. I think the same applies to people who use Ember and they're using Ember Object and they have computer properties and now, in Angular, there is observables and subscribing to streams. What was really interesting with the work that we've been doing with microstates is that and one of things that have notice with other state management solutions is that a lot of them emerge as, "Oh, I had this insight. I spend a couple of days working on it. Here's my proposal, in a way, for managing state." I think that's great because insight and understanding is really important but sometimes, taking the time to design something can be really helpful. I think that one of the things that are really interesting with microstates is that because we've been iterating on it for so long, they've been testing it for so long, we've been able to do something in microstates that's really difficult to do when you have a production application that you're like, "I want something different. Something is not enough but I don't have the time to really make it better," or, "I have an idea. I want to release this thing but I don't have any other collaborators that I can talk to and really flush out these ideas," so what you get is a solution but it doesn't always touch on a lot of the angles that you want to touch on. It's really interesting when you think about -- CHARLES: I'd say, we should talk about those angles, like what are some of those angles? DAVID: Yes. You could think about this from a few different perspectives. I would say, from Ember perspective because that's been my personal starting point. For anyone who was listening. Ember and Vue, for all intents and purposes, in this scenario are very similar because they have the same primitive, which is the computer property. The computer property in Ember, the computer property in Vue is almost identical from the way that you actually consume. In Ember, you would use computer properties and in Vue, it's just computer properties to create derived state from data that's passed into the component and then, you would use is derived state to basically, decorate information that comes with the props, so you could present it in your component. This pattern is very nice. This pattern is being used to build LinkedIn, to build applications at Apple. There are huge implications being built by Ember community using this patterns and I'm sure something similar has happening now with Vue but this pattern doesn't really exist in React in the same way. You might opt in to start using this pattern if you start using [inaudible] but it's not quite the same. Writing computer properties in Ember and in Vue is essentially free because it's so effortless, where in React, if you're using [inaudible] to create cache computer properties and memorize computer properties, you are really opting into doing a particular way of computing this properties and writing selectors that's not trivial. I think that's one thing that is available in Vue and Ember and it's a really effective pattern and you can do it in React but it's actually not possible in Angular, unless you're using, I'm guessing something like ngrx, where you can do similar things to what you would do if with redux in React. Creating derived state that you can consume in your component is one of the things that is possible but it's not possible consistently across all frameworks. That's just one of the things. CHARLES: I agree. I think that was one of the things that drew me to Ember at first coming from Backbone as I did and honestly, from the models in Java was that, in order to compute anything, you had to install a listener and then eagerly make that computation and store it somewhere, where as those frameworks, it made you feel effortless where you can just decorate some state and derive it and the information is there, the computation is there when you need to reach for it but you don't have to do expend any extra effort aside from say, "This state is derived off this other state." I think another case that I came across was immutability. Immutability is a means but the end is to have consistency inside your application. I first really started running up against this when I was working with forms and since then, I've realized that actually, a lot of the pain that I was feeling was because things were not being immutable but this is where the fact that things weren't immutable ended up causing me a lot of pain and headaches and I was having the code into being a lot more complex. Essentially, when I wanted to make something transactional if you're editing a form or something like that, then you need to essentially store off a copy of your current state. If you're editing some object, I want to say, open up a dialogue and I'm going to edit it and then I'm going to commit the changes back to that dialogue. What I'm modeling there is a transaction so I kind of need to shave off a copy or make a Xerox copy of the object and then make the changes inside to that object and then somehow, try and merge those changes upstream and then, get them back into the main lines. Really, like a very Git-like operation. It wasn't just at the object level. When you're doing things like dirty checking, you need to make what is the original value of a field versus what does the input currently have. I might be doing any number of transformations in between the actual physical representation of the field, maybe it's a date but the user interacts with it in as a string, so there has to be this kind of parsing serialize thing sitting in between that value and you need to basically, keep a copy of that field as a mini transaction within your macro transaction because you want to say like, "Is it dirty? is it really the same object?" because if you can just do an object comparison, you have to get into all that hairy like equality checking and it gets really complex really fast but it turns out that a very clean solution to this is if you just never actually make the changes in place and whenever you are making changes, you're generating new information without destroying the old information because when a form is the case where we really come up against this, where we're modeling the same object over time, so a form explicitly models the change of an object. Change is part of what it is and so, the definition of change is being able to compare something in its prior state to be able to compare it to its current state. If you're making that change by destroying the prior state, then you're going to run into a lot of trouble. It just turns out that when you're working with forms and it turns out there are a lot of use cases like this but this was the one where I first really just couldn't even... Without immutability, you want to model your change as always rolling forward and not ever destroying prior states but being able to pick and choose and can always be able to look back to where you were and compare where you are now to where you were. That's where immutability comes when you're modeling change. It's actually much easier to model change when you have explicit states that represents what was and what is. But when you start doing that, then things get a little bit messy. You have to do the compromise. There's a tradeoff, like when I say, "Set this property on this object," that is easy and that's something that I think that we can all understand. We're not idiots. That's why immutable models are very conceptually easy to grasp, to wrap your head around. If I'm modeling myself and I'm saying, "Set hand position two feet up in the air and now I'm raising my hand." That's easy to understand and it turns out that when you're changing a data structure immutably, then you have to, for example model a simple set operation as duplicate and swap. Or if say like you're swapping out one property in an array, you actually model that as a map, where you swap out the one element at that index, rather than just saying, "Just set the thing at that index," or if I'm changing a property of an object, I want to copy over all the fields except for that one which I'm going to change. If you start to do that then, you realize the benefit of always being able to look backwards but you've now introduced overhead and complexity in your code, so you've made a tradeoff. I think that's a lot of people look at saying, "I want this to be immutable," then they actually have to come to the grips with the complexity that's going to introduce. There are libraries like Immutable.js that do make that a lot better but even they have the problems. I can talk about those but one of the cases is like being able to reason about a series of states for your object, rather than just only having one copy of it ever and being stuck with having to deal with it as it is. DAVID: Yeah and I think that really important too is being able to mentally track where you've been because in my experience on other large Ember applications, I've run into so many different bugs that I could really track down to people or rather, places in the code where things are just reaching in and mutating your model, where it wouldn't make sense, so as you're writing your code, you're in a completely different state than you might have expected. CHARLES: Yes and it's easy, right? But essentially, when you have some model, you've basically got a global object. There's a lot of recomputations that needs to happen when those things change. As a result, if you look at the actual code that supports computer properties in Ember and I'm sure other frameworks as well, some of the most hairy and complex. If you look at the chain watchers and the chain nodes and all the stuff that's required to support things like computer properties, it's amazing that it works as well as it does. I've tried to actually open that up and understand that code on a number of occasions and every single time, I had to walk away in defeat. TARAS: This problem exists in React applications and Angular applications because if you think of the challenges of working with set state, I think one of the problems with working with set state in React is that your complexity increases very quickly. It starts off kind of simple when you're using immutability with set state in React. They're kind of very light and then, as you start to add features, your complexity starts to grow very quickly in a way that you really start to run into limitations. You start to confront your personal limitations of your abilities to work with immutability, so you're limited to doing immutable structures that only a few levels deep because your ability to use destructuring to create new immutable object is limited by expressiveness of JavaScript. It's gotten a lot better with the destructuring but if you do it a few levels deep, I think everybody's familiar with what happens. It starts to get really hairy. You kind of loose track of where you are. CHARLES: Yeah. The signal-to-noise ratio increases because after probably, two or three levels, the majority of your code has to do with destructuring and restructuring and very little of the actual change that you want to make. TARAS: And this is kind of interesting because people talk about declarativeness versus imperativeness but at certain point when you have a complex immutable state change, if you're doing with destructuring your code, even though it looks declarative but there is so much processing that you're doing, it's actually kind of losing the benefit of its declarativeness. It's actually starts to look more like imperative code than it does what you would expect a declarative system to look like. I think this touches on the other aspect. It kind of compromise that when you work immutably, when it compromises, you make a serialization. Your ability to represent your state as a POJO becomes restricted by the fact that you have this complex system that's wired together and you have systems like zones in Angular and in Ember Object that are able to keep track of changes in these objects but you don't have a way to restore those objects. You don't have a way to do more sophisticated things that you might want to do, especially in situations where if a service feeding you, what do UIs going to look like. In that situation, it's really helpful to be able to say, "Here's a POJO that I got from the server. I'm going to use this POJO to build this component tree that the user is going to interact with and then, as the user interacts with it, I'm going to then, capture that state, serialize it and put it back in the server." When you're using something like Ember Object or if you're doing this kind of stuff in Angular or even if you're doing this stuff with React but without using something like redux, you essentially end up doing so much wiring to accomplish that. By the time you finished writing your application, you've written a ton of code just to handle this particular use case and if you have to do this again in another application, you just rewritten that kind of code in a new application as well. CHARLES: It reminds me of the concept of a Smalltalk image, like there's no way to really get at the state of a Smalltalk thing. It's almost like you're dealing in docker containers and not actually being able to write that state down into JSON or something like that. I'm trying to casting about for an appropriate analogy. Maybe that's not a good one but what's actually happening cannot be made orthogonal. It can only exist in that one run time that you're currently running. If something wrong manifests itself, reproducing it can be extraordinarily difficult, right? TARAS: Yeah. CHARLES: Imagine if there's some render cycle that's making a bunch of mutations and there's this process that you stood up and it runs in completion, some signal comes in and those effects are like ripple through the system, there's no way at any point to have any other representation of that system than the running system itself. TARAS: There's another element to this, which I find really interesting. When you're thinking about how to architect complex UIs, it's actually helpful to get really clear about what kind of changes are happening. A lot of times when I want to see beginners are writing, especially if you task someone who is a fairly junior at building a single page applications, a lot of times what will happen is because they don't have a clear mental model of what is going on in regards to state. They end up setting a lot of properties. Every operation, every time you have an event handler because they don't have a clear model of what's going on, they end up setting like five or six or seven properties. That kind of signals that they don't have a clear picture but what that also does is a lot of times, they usually comes together with cascading state changes. Usually they're not representing a single operation as a single state change because there might actually be a bunch of things that are happening because what they're doing is they're massaging the system into submission. Not like they're not in control of the state transitions, so they use, essentially time and their dedication to kind of sort it out and make it work and eventually, would that ends up looking like that if it works for most of the cases that they are able to test for or that they able to manually see. But then they AR, not accounting for problems that they're not able to understand right now, they become discovered by users when users start to interact with the system and with the components or with that application and the application is there to get into some funky states. The tools that we have, they don't prevent that from happening. They just -- CHARLES: Right. They don't force you. I think what you're saying is that ideally, you want to model your UI as a set of transactions on your state, that you want transactionality to your state so that I basically am saying, "I'm not going around and setting seven properties in reaction to this one event." I'm saying, "This event triggers this transaction and that transaction clearly bundles up every single operation that needs to happen and the tools don't enforce that." Is that a fair --? TARAS: Yeah and then, the problem is we're working with component trees. You start off with having a set of requirements and over time, the requirements change. As the business unit understands your application better, they give you more direction of how accounting should work and then you find out that there's more interconnect at stake but then what's happening now is that the cost of refactoring those state that spread throughout the components, whether that be with set state, whether that be with the actions in Ember or even in Angular. What you end up doing is you start to change the system but change is not trivial because the actual process of changing where that state lives is not linear. It depends on the complexity of the code that you wrote and it just gets really hairy very quickly. That's where companies end up losing a lot of time. A developer could start off with a requirement, you build something and then a new requirement comes in and instead of it being a simple change, it turns into a week or two weeks refactor because you now understand the state ownership should be different. The state that you have should be in a different place. You have to manually make that change. You have no obstruction to help you express that in an easy way. CHARLES: Another thing, because we are doing a kind of a roundup of all the things that you need to account for when you actually embark on managing your state. Another is actually constraining the amount of computation that happens. If your system is based on listeners and large chain reactions of things where it's like, "This property changes so I need to notify these other 10 dependent properties that change." You can do a lot of unnecessary computation, especially if nobody is going to render. That's kind of the thing that you have to do if you're going to be immutable. You have to eagerly walk those change to see which objects are affected so that you can then invalidate those caches. A system like Ember Object, I don't know exactly how Vue works, it mitigates this somewhat by the fact that the computer properties are lazy but you still have to walk all of those chains. That can actually get out of hand. They're eager, not lazy. Then the other concern that you have, where you normally have to make a trade off around is around composability. One of the things that's really nice about immutable systems is they're very composable. If I've got some object that does one thing, I can then just set that object onto another object just by mutating one of its properties and I've effectively composed them. I can then install listeners onto that thing or I can compute properties off of that property and they can post pretty well. That's something that you get but then of course, you're losing all of the benefits of immutability, so things like Immutable.js don't really compose very well or redux doesn't really compose very well. The concept of taking a redux store and embedding it into another redux store, you just don't see that. I would never distribute and I think ultimately, the litmus test there is would I be able to share it on something like npm. Nobody shares an npm package that's just a redux store that you can dispatch actions to and observe and use it with your other redux stores. When it comes to a system like Immutable.js, that does make transitions a little bit easier over lists and arrays and maps but you still run into the exact same set of problems that you have when you have lists of maps of records and you don't really get any help there, so you have to make this tradeoff between immutability and composability, whereas a system like MobX or Ember object actually quite composable. Before we start talking about microstates, I want to say that you just throw those in there because there is just a lot of concerns out there, a lot of edge cases that actually build up but through the course of a real application, you will encounter them all. You might be making tradeoffs at the beginning that you're not realizing that you're going to need or are going to get you into trouble later on. DAVID: This actually happens in the Angular community as well because there's something really great that's happening with observables in the Angular community. I think everyone's embracing them wholeheartedly and I think that's really been pretty great to see but observable streams of composable, but objects that have on them observable stream providers of some kind, like if you have something that you can subscribe to and that is part of a property in a class, composing multiple classes together and consuming properties from those classes, there is no mechanism for composing that. That kind of composition has to be done manually. Again, you're kind of manually wiring together a bunch of objects and the big challenge is that you are manually subscribing to all those streams and unlike what you have with components. Components have lifecycle hooks. When your components is being torn down, you know you can perform some operations. If you need to remove an event listener, you have a hook where you can do that when you have a class instances like JavaScript class instances that have on them properties that have observables that you subscribe to. There are no tear down hooks for class instances, so there are no obstructions from managing unsubscribing from those streams. You essentially end up having the foundation that you can use to build complex reactive systems and you can subscribe in there really fast but wiring those things together at a bigger scale is simply not there. It's something that you have to create and enforce yourself. CHARLES: Right and I think that's probably a perfect segue into talking about microstates, which is the project that we've been alluding to for the past 30 minutes, that is I think in attempts to solve these problems and make sure that you don't actually have to compromise on those things, so you can reason about things locally but have those things be composed into a greater state. But also have them be immutable so that you can look at past states and reason over a data structure over time as opposed to just in one instance. Also, have an intuitive interface that when you're making these changes, doesn't look like half of your code is unpacking some data structure, flipping some bit in it and then repackaging it back up again. That's the context. Should we start talking about what microstates is and how it addresses those? TARAS: That's a good next step and when I start working in the ReadMe, I end up actually, I think I wrote about 40 pages. One thing that's interesting about microstates is that and this was part of the design of microstates from the work that we've done is that we intentionally wanted to make the number of ideas that you haven't microstates very little, so when you use microstates, the number of concepts that you need to remember in your mind is very few. It is a conceptually a different way of thinking about organizing your state in the same way that shifting from managing DOM elements directly to having an obstruction like component that declaratively applies changes to your DOM tree. In a same way, microstates is kind of an abstraction that allows you to declaratively describe how your state will change and it will manage the transitions for you and allows you to give the state transitions names and it allows you to give your states names as well, so you can actually name things. You don't get a POJO that has a shape but has no name. You actually get to give things names. CHARLES: So, why don't we start? I have a list in my mind. I should probably write it down of the things that we just talked about but I think the things that we talked about are ease of representation, like conceptually easy, transactional, basically serializable and immutable, lazy and composable. Those are like five or six things. But I think there are kind of aligning principle around which we gathers that the state management should feel easy. It should feel fun. One of the things that is awesome about working with components, whether you're using web components or React components or Ember components is when you get it right, you're just snapping these things to feel together and it feels great. It's like I'm just passing properties and render blocks down the tree and the framework is just doing all of the grunt work for me and I'm just operating at a very high level. That's what organizing principle with microstates as it needs to feel easy. Maybe we should start there and just say, how does that easy and fun line up with each one of those kind of unique problems around which we typically have to make tradeoffs? We could start with the interface of making a change. TARAS: I'll go back to the starting point. I remember what got me first interested in microstates is Charles, when you said that, when you have a number, there are certain operations that a number can do. We really don't need to be writing an increment operation for every... Like if you a have a number, you can increment it and decrement it. CHARLES: Honestly, every time I see state management tutorial and they tell you how to increment and decrement a number with it and you write the increment code and you track the thing and you store it back into the store, at this point I'm still annoyed with those tutorials because I'm like, "It's a number. We know we can increment it. Just show me where to plug in the code. I should not have to be writing an increment method." TARAS: Yes. And that's the kind of starting point. There are certain operations that you can perform with the primitive types. If you need to add a number or if you increment a number, we already know how to increment the number. It's part of microstates. But that in itself is nice but that's not that important. I think what's really important is that when you need to put a number into another data structure, let's say you have a nap and you're like, "I need to..." I don't know -- CHARLES: Let's say, like a click tracker that has a number of clicks. TARAS: Right. By itself, you can increment the click tracker but if you need to put a quick tracker into another app, essentially you can compose it in and you don't need to figure out how to wire the actual mechanism of how to make sure that you can update the property, like it's part of another class, for example, you don't need the way you would increment the number. When it's a part of another class versus how you would do it when you're working with it by itself is an approximately the same. The amount of work that you need to do to actually perform that operation is the same. Your complexity doesn't increase as you compose one data structure into another. CHARLES: Right. You can just say, "This is an app. It's got a click tracker and this property is a click tracker and I have to do nothing else. I can register clicks on that thing. It doesn't increase the complexity of application at all." TARAS: There's no wiring. Now, you added some new state, that state is very explicit and it's really clear that it is not impacting other parts of the state. You can operate with this thing. If you change it, it's going to work properly with all the other things that are in the type that you are adding this counter to. Those things are just going to fit well together and it's not going to break if you need to transition one more thing. All of the other transitions will work the same way. I think that kind of consistency is really meaningful, over time especially when you start to increase the amount of state that you manage in your application. CHARLES: Just the ability to work with types and just have kind of those implicit operations and have those things compose, kind of indefinitely. Moving down, we talked about easy and the other thing I would put on that is that the way in which you express those transitions, for example if microstates comes bundled with numbers and Booleans and strings and arrays and objects and kind of the stable of types that you would expect in any JavaScript application but those types are expressed using this way for expressing types, essentially. When you actually do make a transition, it feels very object-oriented, I would say, even though it's not. It feels like you're mutating but you're actually doing a transition. Does that make sense? TARAS: I think for anyone who is familiar with what it's like to write queries for GraphQL, if you're not familiar with it, it's fine. You can get a sense of that from microstates but if you're familiar with the ease of just writing a query and if your backend knows how to retrieve the data, then your queries will just give you the data that you want. That feeling is really powerful and just being able to write the query and just gives you what you want. Microstates is kind of like that. Actually, the inspiration came from experiences with GraphQL, which is that sense of ease is what we wanted to have in microstates and so you get that seems sense of like, "I can just do what I want and it just going to work and with this other thing and it just going to work," and you're just like flying through, like adding states to your application and it's just working for you and working for you and working for you and you don't have to do monkey work like gluing things together. It'll change how you are working before because you have a way of opt working with these things at a higher level. CHARLES: Right. Let's talk about transactionally or should we talk about immutability? How does this make immutability easy and fun? TARAS: I think one thing is that you don't have to write reducers and you don't have to do destructuring by hand. I think you have a way of expressing. Thinking about this, if you have a component tree and let's say you have redux and then a bunch of components like your parent component, your root component has some state using sets date and then components further down the tree also have state. You could actually express that as a microstate. What you would do is, essentially the parent component state would be the root and then the children's component states would be composed into it. The nice things about doing that is that at the root level, you have access to transition state of the children declaratively. You know where the states for those children is on the route type and you can write transitions that are going to declaratively perform multiple operations on the children state and I suppose it'll restructure to what happens with components but if you don't use this, you might have multiple sets date operations. The process of wiring data from the root down to the children is kind of complicated, where here, you have a way to represent that and perform a lot of transitions in the way that is going to be just easy at whatever level you need to operate at. CHARLES: Right. I think, for people familiar with redux, in redux you act globally and then you react locally, if that makes sense, so you dispatch an action to the entire store and every single reducer can see that action. There's ways to manage that but effectively, you have this one atom and then you have the reducers that kind of act on local state, whereas with microstates, you're basically acting locally. You're reacting locally but the effect is global. TARAS: You're participating globally. CHARLES: Yeah, participating globally but you never have to consider the context that is above your own, so you never have to be mindful or cognizant of the context in which you're enclosed because from your perspective, it just doesn't exist. DAVID: Every microstate has a set transition which is the basic transition that you can invoke, essentially in any type, so what's interesting is that it's amazing how powerful -- CHARLES: So, we should break it down really simple. Basically, when you create a microstate, with a type, you say like, "I want to create a number with the value five," and then I can just say, "That returns a microstate," and I can say, "microstate.set 10," and that will return a new microstate who's also a number but the value is 10 and that's available on all microstates. DAVID: Yes. If you have a tree of components and your state is presented by a microstate at the root level, then what you can do is you can invoke the transitions on any part of the microstate and it will just know how to properly create the next microstate for you. The example that Charles you gave of one number so that number can be inside of a class that represents state for a particular component and then that can be a part of another class or represents a state for another component but then, when you invoke a transition on one of the leaf nodes, an equal sets state on one of the leaf nodes or equal set on one leaf nodes, it will respond locally but it will actually reflect the changes globally. At the root level you're going to get a new object that causes the components to update. CHARLES: You know what? I have another concern that actually just popped into my head, which is something that I've certainly struggled with in every single application of notable complexity is stability of value. We should put that on the list. We're almost out of time to talk about this. We spend too much time... Well, not too much time, of the issues of state management, which I think you can't spend enough time talking about but I did want to pile one more on there is when you're making that transition, where you're acting locally but you're participating globally. For things that are unaffected by your action, remain unchanged. That is a super power. It's actually very hard to do with a lot of state management, especially when you're cloning a bunch of stuff, being very judicious about what you don't want to clone. Where this really comes into play is if I want to re-render something. A lot of times you have to jump through a bunch of hoops to tell, did my model really change or did only referentially change? With microstates, when you make that local change, if you're embedded in a very large graph of objects, obviously all of the objects above you are going to be changed but what about things that are off to the side of your siblings. They're outside of that scope of that change. They shouldn't be cloned. They shouldn't be copied over. They should remain the same. If you're doing a re-rendering based on the changes that are happening, that's going to be a key feature because you're not going to have to write, basically any hooks to say, should I have to re-render my component. You can always rely on triple equals. DAVID: This quality is going to describe the structural sharing and some of the other tools that are available but I think that's how it's accomplished. I think one thing that's a little bit different with microstates is that when it comes to structural sharing, it's not that difficult to do if you just do structural sharing and value. Meaning that you can do structural sharing on a value using something like lenses and not a lot of people are familiar with lenses in JavaScript but it's actually only three functions that you can use and it can give structural sharing on complex POJOs. It's pretty easy to use relative to how little people know about it but what that doesn't do is it doesn't allow you to graph of objects that have their own operation that you can invoke and that will perform structural sharing. That part, I know that is not available in any solution, I think at the level of completeness and luxury that microstate provides when you write things. CHARLES: Right because every piece of the tree kind of comes bundled with its own things that you can do with it. DAVID: I don't think you should jam everything into this podcast because there's a lot to talk about it. I think one of the things and we've talked about this a lot, which what we want to do is create an implementation for an idea of what it would look like. What would it be like if we had a composable state primitive that we could use to describe state and share state solutions in the same way that we share components like react-virtualized or whatever your particular frameworks or popular component may be. What it would look like if we had solutions to state problems that we could share and we could -- CHARLES: I think the litmus test of an awesome solution is like you look at this current crop of MVC frameworks and what's so awesome about it is they're sharing. That can happen, right? If I'm writing a React component, I can publish it on npm and other people can use it. If I have an Ember add-on, I can publish it and people can use it. They can consume those components. That's awesome and I think it's the hallmark of a great system. What would it look like if I wrote just the state piece of a file upload and I could publish it npm and then anybody, in any framework, could actually use it with their framework without paying any penalty. What would it look like if there was some transactional data store that could be built and shared and the hooks into any framework were minimal. The possibilities are really exciting around that idea, whether it is realizes microstates or not. But clearly, we feel this is something you should be able to do. DAVID: I'll add one more use kind of use case that I personally find really motivating is that there's a lot of companies that are investing into building single page applications and a lot of times, what you see happening is they're building a very similar application to what they had before because their business hasn't changed. The technology has moved on so solutions have improved. The demands for better user experience have increased but the actual business and how the things that people have to do on day to day within their company hasn't changed. What we're seeing right now is we're seeing the same, like whatever was written before in jQuery or an AngularJS is now being rewritten in React or Angular or whatever you might Choose. But whatever you like to see is it has a situation where the domain specific logic of your business is represented as a data structure that knows how to, potentially, in the future talk to the server and retrieve data from the API because that's likely not going to change. But you can use that like that's been tested and published as an npm module within your enterprise and then you can then consumed that in any framework and it's actually easier to do it this way, than to implement it in a framework-specific version of their state management. That's the part that I find the most exciting. I think one of the things, just to connect to the goal is that, we would like to keep this conversation going. If you're interested and I think this is a kind of a call to our audience that if you're interested in this, we would love to have in our podcast to talk about these things because I think there's a lot of things that microstates really is a beginning of a conversation. It's not meant to be a statement. It's meant to be a proposal that we can just talk about this. CHARLES: I agree and that's one of the reasons we're keeping it very small at this point. The core library of microstates is not setting out to accomplish too much. In the core library, there aren't even any side effects. It's actually impossible to have side effects. It means that it cannot be used for anything except for the model but that's very liberating and it let us focus on what would a system like this actually look like. DAVID: It's really exciting because this has been the biggest metric but we have microstates in the JavaScript weekly and that was great and then it got circled around when people know and it's like 800 stars now, which is not a really big deal. It's funny because somebody commented like, "How can you put something in production that only had 100 stars?" CHARLES: I think it's just important to realize that this really is the beginning of a conversation. There's a really exciting set of things to come. We haven't even talked about how we're going to model side effects, although we're going to use microstates to do it. We haven't even talked about what the various framework integrations will look like and what are the best practices for using this to organize state in your application. We've had some lively discussions internally about what that looks like. There's still a lot of questions but it's going to be a really, really exciting and edifying experience to get to answer them. DAVID: Yeah, it's pretty exciting. I'm excited too. There's been a lot of interest from people in microstates, so it's going to really great. I'm looking forward to meeting people and having conversations about how we can use microstates because I'd love to have someone create a really great solution that I could just take off the shelf and just use and not have to implement them myself. CHARLES: All right. Well, I think we could talk about microstates for at least the next three hours but we have to give everybody an opportunity to, at least like go to the bathroom or something. Microstates will return but if you're interested in learning more about microstates and you happen to be in one of the many places on which we're going to be presenting on microstates in the future, who knows? Maybe you can come in and join the conversation in person. Taras is going to be speaking at Toronto.js on July 30th. He's also going to be presenting at Manhattan.js on August 8th and then, yours truly will be presenting on microstates at React.js Austin on the 6th of August. Come out and see us. We'll drop those in the show notes and it's guaranteed to be a good time and we'll have that conversation. Until then, we are the Frontside. We lead with the why, the how and then the what, if you're interested in working with us and that helps us that we guarantee the lowest total cost of ownership for your application. We're always looking for feedback. If you have news items that you'd like to see at the head of the show or just any feedback or questions, we would be happy to answer them. Thanks today to Mandy Moore for producing our show and next time, we'll be talking with Kristian Freeman about what it's like to run an online conference with Twitch, so I'll be looking forward to that. Bye David. Bye Taras. DAVID: Yeah, thanks for having us. TARAS: Bye. CHARLES: Yup, and bye everybody. See you next time. Next Time: Running An Online-Only, Free Conference on Twitch with Kristian Freeman
A rare episode with Charles Lowell, back to reprise some DrunkAndRetired.com style rambling. We discuss, oddly, actual retirement, GI Joe vs. transformers, and talking about just one thing. Oh, and "a dry heat." Robert will be back next time. Special Guest: Charles Lowell.
A rare episode with Charles Lowell, back to reprise some DrunkAndRetired.com style rambling. We discuss, oddly, actual retirement, GI Joe vs. transformers, and talking about just one thing. Oh, and "a dry heat." Robert will be back next time. Special Guest: Charles Lowell.
Special Guests: Brian Douglas and Bex Warner of GitHub. In this episode, the panelists talk about automating GitHub with Probot. The origins of Probot are discussed, as well as making GitHub apps with the GitHub API, automating workflows with Probot, must-have Probots for every repo, and GitHub's V4 GraphQL API. References: Microstates README Probot github.com/integrations/slack github.com/marketplace/pull-reminders platform.github.community/c/integrations probot.github.io/apps/unfurl-links/ probot.github.io/docs/deployment/ probot.github.io/docs/extensions/#scheduler probot.github.io/community This show was produced by Mandy Moore, aka @therubyrep of DevReps, LLC. TRANSCRIPT: ROBERT: Hello everyone and welcome to Episode 105 of The Frontside Podcast. I'm Robert DeLuca, the director of open source here at the Frontside and I'll be your episode host. Today, we're going to be discussing automating GitHub with Probot with Brian Douglas and Bex Warner. I'm really excited about this topic. The idea of automating GitHub workflows with bots is amazing. This is something that I've been wishing the GitHub have the platform support for since I even started using GitHub for open source. Just being able to have a bot to take care of certain things like somebody doesn't leave enough of a PR description and they open up a PR, you can have a bot that just responds to it and saying, "Can you provide more information?" It's pretty awesome. With me as co-host today is Charles Lowell, who is also a developer here at the Frontside. Hey, Charles. CHARLES: Hey, Robert. ROBERT: Before we get into the discussion, I like to make a tiny little announcement. We've been building a composable and an immutable state container called Microstates. I'm sure Charles can talk about this more at length, then we will in the next podcast episode -- 106, but I would like to make a small announcement that Taras who is an awesome developer here just wrapped up a month's worth of work, creating a new ReadMe to describe the vision of Microstates and what you can do with them and everything about Microstates. If you're interested in that, I highly recommend checking out the ReadMe. I'll drop a link in the show notes for you that are interested. CHARLES: If I can add, it really is [inaudible] because it isn't like any other state management solution out there. ROBERT: No, absolutely not. I've been building something with it in React Native over the weekend of the 4th of July and it's amazing. But enough about that, you'll hear about that next episode. For this episode, I want to talk about Probot with Brian and Bex. Hi are you two doing? BRIAN: I'm well. BEX: I'm good. Thanks for having us. ROBERT: No, thank you for joining. This is really exciting. Like I said in the intro, I've been really excited about this project. I do a good amount of open source, I would say and this has been really helpful in all of our repos. We have, I think like 78 open source repos on the Frontside. We have Microstates, like we just talked about and Big Test and all of those repos use some combination of Probots that people have built and it's really nice, especially with the new Checks API that has just come out. You can integrate Probot into that, right? BEX: Yes. I, actually am currently working on shifting one of our bots from using the commits Statuses API to the Checks API. ROBERT: That's awesome. Before we go too deep into it because I want to come back to that because that sounds really cool and what the integration of that is like and what changes because I'm not even really that familiar with it. I just know it was released. I kind of want to go from the beginning here. Where did Probot come from and can we get a little bit of a history for everybody that might not know what Probot is? BEX: Sure. Probot originally started out as this simple idea to make GitHub scriptable. The original idea was you have a single file in your repository that would be like a JavaScript file and it would essentially spell out how the bot would act on your repository and the goal was to make GitHub apps accessible to people because if you ever look through our GitHub apps documentation, I think it can be a little tough to get started. There's, honestly, a lot of nonsense that you have to go through in order to get set up. For one thing, the way our GitHub app authentication works is it requires a JSON web token followed by using that JSON web token to request an installation access token and that process would be really tough for new people to get started. ROBERT: Yeah, it sounds like it. BEX: Yeah, so Probot was created to abstract all of that away and handle all of that authentication automatically and simply leave you with the payload that you get from listening on web token events and in authenticated GitHub client to make authenticated API requests while authenticating as an app. ROBERT: Cool, so that's where it started like a flat JavaScript file in the root but today, you use like EMO files and a .GitHub folder. How do that kind of progress? BEX: Originally, their use case was much simpler and it quickly became clear that a single JavaScript file in the GitHub repo was not scriptable enough and not easy enough to understand. The goal was to make like an API that could make that JavaScript file really, really easy to customize for every API of GitHub and it quickly became clear that that was not really a feasible thing to do. as time went on, it turned into this way to build Node JS applications and essentially, what the configuration files you're referring to are the way in which we make it customizable because right now, there's no way to be officially supported GitHub apps channels to pass secrets because it means you're a [inaudible] and the owners of GitHub apps, so that was just a way to kind of stop that problem. ROBERT: Gotcha, okay. BEX: The actual code for GitHub apps still lives in a Node JS module basically and the configuration file just specifies how that module runs. ROBERT: Right, so they're deployed like Heroku instances, if you want, like anywhere you can host a node app. BEX: Yup. Heroku, Now, yeah. ROBERT: Interesting. BRIAN: As a reason to that, some explorations of doing serverless deployments for Probot, I think there's a couple of issues of them. I'm not sure if anybody's shipped anything like the way they at but it's pretty much it's possible to. BEX: Just a week ago, we even released a new version in which we update our core from Node JS to TypeScript and now that things are typed, we have big plans for serverless. ROBERT: Nice. That's awesome, so then you'll be able to deploy to a Lambda and off to [inaudible]. BEX: Exactly. CHARLES: Can I actually interject here, as kind of a person who doesn't really know the relationship between GitHub apps and the GitHub marketplace and what exactly a Probot is before we hear the origin story. I would love to hear a very high level view of how this ecosystem fits together. BRIAN: I think a lot of people are pretty familiar with interacting with the GitHub API and OAuth integrations. I think I've just spent a lot of time at different companies previously to GitHub, just like making calls, either to cURL or through Node JS or more recently, [inaudible]. GitHub apps itself are a way to take all the things that you had to do to make an integration to GitHub much easier. It has a lot of cool things like OAuth, scopings, so you no longer have ask for all your repos ask access whenever someone logs in with GitHub and the connection between like, "Now have gone from OAuth to Now to GitHub apps," there was a lot of, as Bex mentioned earlier, ceremony that happens to getting set up with GitHub apps and integrations that Probot is like this tool to speed up the process of getting to the point where you just want to script some automation or some sort of workflow and it gives you all that bullet play for you. I don't know if that was a good high level for you Charles. CHARLES: Yeah. I've kind of witnessed this second hand with Robert installing a bunch of things here, so let's use an example, like you did some sort of automation on our repos, Robert, where when someone files a ticket, there's this workflow that automatically adds a triage label, so that we know that this thing hasn't even been dealt with, so we really need to address that issue. It doesn't need to be as a high priority. It doesn't need to be closed as a duplicate of something. One of the different aspects that you described there, how do they fit in terms of serving this workflow onto the end user? Or was that a good example, even? BRIAN: One of the cool thing about GitHub apps and what Probot does for you is that normally, if you want to add a label to an issue, either you Charles or Robert, would have to be admin or maintainer on the team for the Frontside and you could add labels. But somebody who opens up an issue, doesn't have that ability to have write access to your content, which is adding a label. What a GitHub app does, it actually takes a spot as if you would have another user on your platform, instead of creating a dummy account or a dummy user. Probot is basically building a bot for you to then, give you the ability to add that issue. That's sort of workflow that normally would have to happen through an actual real human could not happen through a bot without taking up a spot of like, "I guess, I probably shouldn't speak so ignorant about our platform and what we actually pay for nowadays for GitHub," but I know we used to have like a limited amount of seats for organization, like that seat no longer has now taken up and now, it could be just be used a bot can do something that normally us would take. ROBERT: Right. You no longer have to create a user to do these things. BRIAN: Correct. BEX: [inaudible] within GitHub. It's sort of built in a way that apps can take a lot of power in your repositories. CHARLES: So then, what is the relationship between Probot and an app? BEX: Probot is essentially the framework for building an app. You can definitely make the equivalent of any Probot app outside of Probot. It abstracts away all of, basically, the horrible parts and leave the easy part. CHARLES: Now, I think I'm ready to participate in this discussion. ROBERT: That was perfect, though. That's a great intro because I actually didn't have a total grasp or understanding of the relationship between GitHub apps and Probots. That's really good. BEX: Yeah. Additionally, going back a second. You mentioned the marketplace before. One thing to note that is that there actually are several Probot apps on the marketplace right now. The marketplace is essentially the home for any larger, usually third-party companies that have made apps and Probot is essentially supporting some of those. ROBERT: Interesting, so then my question would then be, do you know anybody selling their Probots. Does the marketplace charge? I'm going to assume it does. BEX: Yes. ROBERT: Okay. Is there anybody charging for their Probot? BEX: Yes. There is a quite a few, in-fact, charging for it. Recently, a pretty popular example is the GitHub Slack integration, which is if you open new issues, you can have them appear in your Slack channel. That whole application was recently rewritten by GitHub. It was previously owned by Slack and that was built on top of Probot. CHARLES: And I actually remember, we upgraded to that version. It's actually way, way, way better. BEX: I'm glad you feel that way. CHARLES: I didn't know the story behind there. I was like, "Oh, I just got a lot of... Awesome," you know? Although I don't know what's the costing. BEX: Yeah, I think that integration is actually free, so that wasn't the best example. I think it's for open source projects, at the very least. BRIAN: Brandon, one of the maintainers for the Slack integration and work at GitHub, also did a really cool talk at the SlackDev Conference a couple of weeks ago, so if you're interested what were the behind the scenes. That integration is all open source as well, so if you have request or you have features that you would like to add to the Slack integration, you can pop into the repo that hopefully will show up on the show notes because I'm not sure if it's like GitHub/Slack, but I guess we'll find that out in the show notes later on. BEX: It's Integration/Slack. BRIAN: But for an example of a paid app of a non-third party, we're not talking like Travis or Circle or another one with the big names but rather, a solo dev created. It's Pull Reminders, which is on the marketplace as of today and essentially, this gives you reminders of your pull quest, so you can actually ping inside the comments and tell Pull Reminders to say, "Tell me about the pull request like next week because it's Friday and I don't have time to look at this." ROBERT: That's awesome. I've also seen the one that's kind of related, that is like you can set your out of office at GitHub, which is actually kind of a neat concept. BEX: Was that the one where we are already changing that profile photos to have the overlay or the one where is just auto-replying to messages because I've seen a couple of -- ROBERT: I think, it's just auto-replies. BEX: Okay. CHARLES: So, it can change like your profile pictures and really, not just related to repo and history related activities but everything? BEX: Anything that you can access via the GitHub API, you can almost access via GitHub apps. There's a list of end points that I specifically enable for GitHub apps because there's something such as delete a repository that there's basically, a very few circumstances under which you want to give that permission to an app. Also, to things very specific like your profile or your personal page. About a year ago, there was an official internal audit of all of the API endpoints because there are lots of inconsistencies over what was and what wasn't enabled for GitHub apps, so they went there and kind of decided, what endpoints should be enabled and what endpoints actually get enabled. Now, that list is much longer than it was a year ago. Now, it's much more comprehensive. ROBERT: That's awesome and is this for the Rest API and the GraphQL API? BEX: Yes. Probot does support both. The Rest API is the one that specifically had all of these endpoints audited. The GraphQL, since it's a bit newer, we sort of built those and more. ROBERT: Cool. I really like working with the GraphQL API with GitHub. It makes it easier than trying to do a bunch of Rest calls. BRIAN: Yeah, there's a community form, it's like a discourse form that the API team actually manages and sort of pipes in there. Again, going back to like, if there's not something in the Slack integration that you would like to have, the form, that community is actually in there, if there's something not in the GraphQL API, that you would like to see. No promises on shipping it within an x amount of time but if enough people are requesting it obviously, there's going to be some resources [inaudible] at. ROBERT: What do you mean? We're doing open source. It has to be done yesterday. BRIAN: Yeah, exactly. And that form is at Platform.GitHub.Community, just a URL to get there. ROBERT: Awesome, that will be helpful to look through and get some recommendations in there. One of my favorite things I was going to say about the new integration for Slack and GitHub is the fact that I can highlight line numbers, paste that linked in and then it just expands it and the chat in Slack. That is so nice and I use it all the time. BEX: Yeah, I love that they built that feature. Actually, the original feature that was built on GitHub to allow those line expansions in the first place, like on GitHub itself, was actually built last summer by some folks who were also a part of my intern class at GitHub last year. ROBERT: Hey, intern power. That's awesome. BEX: Yeah. ROBERT: Everyone there is doing amazing work. I'm also following along with somebody that is also an intern and it's building a weekly digest program. BEX: Oh, yeah. That's actually a Google Summer of Code student. ROBERT: Oh, interesting. BEX: So, being sponsored through Google Summer of Code by Probot as an open source support. ROBERT: Is there anything more to unpack there? That sounds really interesting. BEX: Essentially, we submitted an application for Google Summer of Code because we thought it'd be a cool way to get more people, more students, a mentorship opportunity for the maintainers, basically and we were honestly overwhelmed. We got like almost 100 applications and it ended up being a huge of a deal but we're -- ROBERT: That's a great problem. BEX: Yeah, definitely a good problem but we were really happy. We, initially wanted to accept more students but Google limited us to only two students, so we have two Google Summer of Code students working on projects and one team of women from Rails Girls Summer of Code working on Probot. ROBERT: That would be awesome. What do they working on? BEX: I'm not sure yet. They actually just started a couple of days ago but the other Google Summer of Code student is working on a background checks API to eventually do sentiment analysis of comment history of someone new to your repository. ROBERT: That's interesting. That sounds like there will be some machine learning in there. I might just throwing out buzzwords? BEX: Most likely, I think they're just using some sentiment analysis API, like the perspective API. I don't think they're actually doing that themselves. ROBERT: Okay. CHARLES: Actually, I have a couple questions. Back on the subject of Probot. How does this square with the classic mode of integration because there was a lot out there? I think the first one that I remember that stuck in my mind was like Travis and I don't know if there had to be like a special relationship between the Travis developers and the GitHub developers, that's like, they was able to make that integration happen so many years ago. I don't know how that happened. I just remember it popped up and I was like, "Woah. This is incredible," and we see kind of the integrations gets more and more rich. For someone who's got, like you mentioned a couple of the big names, is the idea that eventually those would be able to be completely supported is GitHub apps or is it they're always going to be kind of a separate track for kind of the really deep integrations? BRIAN: I wasn't around when Travis first integrated with Lyft GitHub and I think that's a really cool integration and I know they have a very nice sized team that's able to do that. I think if we zoom back out like Probot, the way to get started with Probot is that we have the CLI command, which is to create Probot app. I believe it was intentionally copied off of create React app and the cool thing about create React app and create Probot app is that they abstract all the ceremony and boilerplate to get started really quickly. It was like, what developers or smaller teams can get started with integrating with GitHub apps. I highly doubt that Travis is going to rewrite their entire application with something like create Probot app but they're definitely going to be moving towards the new API calls, which would have been like GitHub apps. Part of the Checks API that we had launched at the end of May, Travis had blog post on how their integration with the Checks API works. They're making, though they have a lot of what Legacy endpoints and a lot of Legacy integrations in the way they integrate with GitHub, they are actively moving towards a GitHub app. I don't know if I could actually comment on their status of where they are today, to be honest but actively, we want all new apps and new integrations to follow the model of being a GitHub app, so that way, out of the box, you have access to all the newer features. You have all the access to all the newer GraphQL endpoints, if you want to use GraphQL and that way, we can serve one market, as opposed to everybody who had a GitHub integration from five or six years ago, that was all piecemeal together and sort of duct tape, like we run move away from duct tape everything together. CHARLES: I see. BEX: I definitely agree that I don't think Travis is going to switch to using Probot anytime soon and I don't think most of the large companies will be doing that but I do think, there will be shift towards GitHub apps in general. For those companies that don't already have the buildings of the GitHub app started, I think that Probot could be, in time to free some of them. BRIAN: In addition to that too, Travis and Circle and all the CI integrations, they're doing a really good job. I think the cool thing about GitHub apps is what you take away all that ceremony of getting your checks to work, now we can start opening up the door of like what's the next sort of CICD thing like? There's another term or another, I guess category of applications that can now be built to improve GitHub. CHARLES: The most amazing thing about having a great platform is the apps that you don't foresee, like it just come completely out of left field and you're like, "Woah. I can't believe that's actually a possibility now." When you have started to see some of those, some Probot or GitHub apps, you're like, "Man, I didn't see that coming. That's awesome." BEX: A hundred percent. I think it's the most exciting part of Probot because I think GitHub as a platform, we all know GitHub is the largest developer platform in the world and I think the idea that developers can build on top of this platform is the most exciting idea right now. I have honestly already seen apps that really excites me. The other day, I saw this app that was definitely not near completion but it was essentially updating and issue a comment box over and over and taking response through like checking a box and then listening on that common edit, in order to specify your coffee order. ROBERT: Woah. BEX: I was like, "Do you want an ice coffee or regular? Do you want milk or sugar and cream?" and it was going one at a time. It didn't actually order you your coffee at the end but it was super exciting to watch that. You're just editing the comment. I had never seen that before. ROBERT: That's pretty slick and that's taking the API pretty far. I'm sure there were some parsing in there and each Webhook response are like, "Was this box edited or not." That interesting. CHARLES: Yeah. Actually, now that we're having this discussion is kind of like changing my mind a little bit. Robert and I were actually talking yesterday about trying to standardize on our release management and our plan was basically to have some software that was going to run inside of our CI provider and have kind of a shared library, just a little ntm package that was shared by all of our repos but I'm thinking now, man, we should really explore doing this as a GitHub app. ROBERT: Yes, please. I've had three ideas that I really want to build out as a Probot. I'm just going to list them off and then we can build them all together and take equity and you know. I'm kidding. But the two that really excite me, that I kind of want to do is one concept that we work on this open source project for our clients and if somebody from the outside that doesn't have commit bits to be able to push to master, it would be really cool if we had a Probot that after it had an approved on the PR, from the maintainer, that the person that open the PR could then tell a Probot say, "This is approved by somebody that manages this project. Can we merge?" and then the Probot would then actually merge. I don't know if that's possible. That's something that I definitely wanted to explore. Then the other one, which is less cool, would just be like if we have a couple branches on some of our projects that we want to continue and we're not ready to put it back into master but we want to continuously run the test suite against it, so the idea there would be to have a Probot that would watch for changes on master and rebase as needed and continue to run the test suite and see where you're at. Those are the two things that I'm really excited about to do with Probot but I just want to automate everything with GitHub now. CHARLES: Right. BEX: Yeah, definitely, that first idea was actually pretty viable. I'm curious to know like how you actually get those commit links -- is that what you called it? ROBERT: Commit bits are more like commit permissions, I guess. BEX: Oh, I see. ROBERT: An outside contributor. CHARLES: Yeah, we want to push responsibility to the person who is the maintainer who can approve it but actually, the way we do it at Frontside is the person who actually is making the change is responsible for merging it. Once you get approval, you still have to hit the go button and that's just going to make sure that you're taking responsibility for saying it's done but that doesn't work for open source because people coming off the internet are going to have the right to push but we would like to give it to them, maybe via an app, if there is a maintainer who's approved it. BEX: Yeah. That's definitely something you can do. I've seen quite a few apps that, essentially add outside collaborators to the repo. Are you familiar with the... I forgot what it is called, like the all contributor section, where you cite everyone in your repo and everything and who's worked on it. There was a GitHub app that would add someone automatically after they merge their first change. CHARLES: That's awesome. ROBERT: I may have seen that on React State Museum but I'm not sure. It's a repo that we've contributed to and it has all the contributors at the bottom. It seemingly just kind of popped up there. BRIAN: There's an app that, I would like to mention too that I'm pretty excited about, that it sounds trivial too and it's almost similar... Not similar but it's sort of related to what you were talking about, Rob, with your first app, which is the WIP bot, which is the work-in-progress bot. This is a pattern of whenever I open a PR and I might not ready for a merge but I want to share my code so I can get feedback earlier on, I'll type in WIP so that append to my title of my PR. What this engineer did was every time you do WIP, it's going to go into the GitHub API and actually block the PR for merging, which is a feature available to GitHub. It's nested in your settings but the cool thing about this it actually blocks the PR for merging, so you don't have to worry about getting your, sort of like show and tell code merging the master without being ready. ROBERT: That's one of the first bots that I installed on all of our repos and then you can correct me if I'm wrong, it didn't always have the ability to block the PR from being merged but with the new Checks API, is that something that was introduced? BEX: Not exactly. The way that blocking of merging works is if you set it as the required status, so you can install any sort of CI on your account and have it not being required and ignore it whenever you feel like it, so it's really up to you to make it required. Otherwise, it just isn't checked and that's true for anyone who uses the Statuses or the Checks API. ROBERT: Okay, so that's a Statuses API. Okay, sorry. BEX: Yes. ROBERT: Also, the cool thing about that that I noticed when that was rolled out was I was now able to pick and choose and use workflows on Circle CI and each workflow is broken out as a different status check. I am now required like linting and the build and the test have to pass for these browsers before it can merge, which is really cool to be able to pick and choose. BEX: Yeah. It's awesome. I know personally on some of my repos, I have a few checks that I just don't require because I know I have to make them pass. ROBERT: Yeah. Speaking specifically about the work-in-progress bot, do you know how that works? It's open source, so I am sure I can go look. I think we want to go make a PR. We had some back and forth about this, Charles. CHARLES: I actually just [inaudible] we disagree. ROBERT: Yes. Charles opened a PR and one of his first commits in the PR had work in progress and the title had work in progress and we have this this Probot on our website and it was a blog post. You know, you make a couple more commits and you're further down, you move the work in progress in the title but the PR were still blocked because the first commit on a PR have work in progress in it. I think if it's the most recent commit or if it's in your PR title with work in progress, it should block but otherwise, it should not and Charles feels differently. CHARLES: I have about six commits and the very first one have WIP in the title or in the commit message and it blocked the whole thing but I kind of felt like it actually made me go back and I had to squash it down to two commits because I actually feel that your commit history should tell the story of the development, not like it should an absolute one-to-one journal of what happens but what you are intending. I actually felt that it could help me out because there's six commits that we're kind of all over the place and just kind of slapdash together have made me kind of go back, rethink it and tell a coherent story. I think it did me a service but it was not obvious. I definitely agree with that but I was like, "Why? Why were you still blocking?" ROBERT: Do I really [inaudible] admin privileges? BEX: I would say, I am friends with the creator of the web app. His name is Gregory Mantis and he is actually got a huge work in progress PR shifting work in progress over to using the Checks API and one of the features that he's using with the Checks API is essentially this mark as now work in progress button that will add the special line, like feel free to merge or something like that into your original PR description at the bottom. If that is there, the work in progress app will no longer be blocking. It's essentially like a hard override and honestly, that's the power at the Checks API versus the Statuses API. That's really exciting. ROBERT: Because I have seen the work in progress bot to get into a weird state, where I did remove the work in progress from the title but it didn't quite update and I'm still blocked. It's okay for me because I have admin privileges but other people on the team maybe not and they might be blocked from something that's actually work in progress. It's a lot like that hard override will be probably pretty helpful. BEX: Yeah, definitely. I think sometimes, there's some confusion with that just because of the way what perks work on GitHub and the way our pages are rendered, that you may need to refresh the page before you actually see it take effect. ROBERT: Right, yeah. Overall though, I love that bot. I go weekly, probably to the Probot apps listing and just go shopping. BEX: Wow. I'm actually the person who approves all the Probot apps to the listings so that's pretty motivating there. ROBERT: It's really nice. I am not even joking when I say shopping, I go through and I open up a bunch of tabs, I read through them, "Oh, this could be useful," that kind of thing. BEX: The first app you mentioned, which was like the one that requests more info is actually one that I built, so that was kind of funny. I guess you got that from the Probot apps too. ROBERT: Yup. That one, we definitely use on a couple of our organizations and repos. It has yelled at me a couple of times because of a blank PR. BEX: It yells at me all the time. I think I get yelled at more than people who are actually doing it wrong. ROBERT: I'm a little embarrassed like, "I should do better. I need to set an example." BEX: Definitely. ROBERT: Cool. I'm curious what both of your favorite Probot app is. This ought to be interesting. BRIAN: The app that I'm really impressed with so far, that I actually only use on a junk project at the moment, is the weekly digest one and it's mainly because I built something for this in my previous role at the company but then we shift it, which is basically go through every single repo. I worked at a company called Netlify previously and we had way too many repos to maintain... Oh, sorry, to keep track of and I was moving further and further away from the backend at the time so I was unable to keep up to date with all that was changing. I built a Lambda to watch Webhooks and then give me a digest of what was shipped like issues and PRs closed. It was way over-engineered and I never actually shipped that to actually make it work. But then the weekly digesting came out maybe a couple of weeks ago and it blew me away because I was like, "This is exactly what I needed," and I was trying to make it overly complicated through like a Lambda and like a bunch of Webhooks and this person, with only a few weeks, has the scaffolding of what I needed. That's the one thing I'm pretty excited about. It was already mentioned earlier too, as well. BEX: I guess, I would say one of my favorite ones is the unfurl a link app. I think that one it so simple but so nice. I don't know. I think having that unfurl link preview is just beautiful. Essentially what it does is it listens on issue comment creation or pull request comment creation or issues your pull request or whatever and read through the text or whatever was that issue or pull request and looks for links and then, essentially unfurls them so you can get a really nice preview of what you're going to. I think that's really beautiful and just so simple. ROBERT: Yeah. I love that one too. I have that added to all of our repos. BEX: It's so much nicer. Why would you not unfurl your links when you could unfurl your links? ROBERT: Exactly. CHARLES: I actually have a question. I think it's been touched on, probably at least twice throughout the conversation. I want to actually create a Probot, how do I actually go about deploying it? What does that look like? What does it look like to deploy and maintain it? BEX: We have a page on our docs about deployment and essentially the TL;DR is you can deploy it on any normal cloud hosting service that you wanted to deploy it. There are a few things you need to specify. For example, GitHub gives you a private key that you need to create your JWT and that private key means to be passed into your hosting service however you do that and then, there's a few bits of information that need to be pass in. We have pretty intense docs about it. Honestly, I'm not a deployment person. I usually try to let other people do that and I have never had a problem going through our docs and just getting it working immediately. BRIAN: It's also mentioned that there are examples like Heroku and Now and a couple of other ones. If you have a service that you already like, it's possible it's already in the docs, like steps to how to get that deployed. BEX: Yup and any other services are more than welcome to be added to the docs. Pull request are welcome. ROBERT: Sweet. It sounds like we need to set up a hack date to create a Probot, Charles. CHARLES: Seriously, my mind is brewing. ROBERT: I guess it's not directly related to GraphQL but there's something that I've always wanted to build. For prior history to everybody [inaudible], then the podcast, Brian and I used to work at a company called IZEA and one of the things that we built and I worked on a lot was we would create a collect metrics on people's social accounts that they're connected and do that and graph it over time. This idea came from when I was building up that feature all the way back in 2013, I want to graph the change in GitHub stars. Is there an API available for me to see like weekly GitHub stars or is that something that I still have to manually store and track? BEX: There's definitely an API endpoint to get the amount of stars and I don't see why you couldn't just do that on weekly basis and compare but I don't think there's any track that change API. ROBERT: Gotcha, like a history of it. I could do this by just stealing and looking at what the weekly digest Probot is doing because there is a change in stars section in there. I was just curious if there was now an API that was available. BRIAN: Yeah, that's more unlikely. I'm going to say no without looking at all the reference documentation. I think as far as that database, it's something you'd probably have to collect on your own but it's also a good candidate for a GitHub app, where you build a service that you can actually track stars once you've installed it and then if you want to monetize it, you can actually pay for private repo or whatever stuff like that, if you wanted to. But it sounds like a great opportunity to see this in the GitHub/Probot listings. BEX: I actually just look this app really quick in our docs because I was curious but apparently, you can receive the star creation timestamps. That could be doable through timestamp usage. ROBERT: Oh, and then I just kind of loop through back and build your graph in there. BEX: Yeah. ROBERT: Interesting. All right. Well, [inaudible] I was going to do today. BEX: Yeah. But I think it's exciting to bot the weekly digest and then what you could extract from that into stargazing is that Probot scheduler, which is essentially this all Probot extension we made that triggers a Webhook on a scheduled time period because right now, the way GitHub apps works are so centered around Webhooks. It can be difficult to find a way to trigger an action on something outside of a Webhook, like on a schedule basis. ROBERT: Yeah, that would be really helpful. I can definitely see how that would be a problem, if it's very, very central to reacting to Webhooks and events that happen on the system. BEX: Exactly. ROBERT: You're just hoping that somebody comes through and creates an event at a specific time. CHARLES: Can I ask you a question about, it's definitely on topic of extending GitHub but currently, just a question about, where the line is between what you can and cannot extend? You mentioned, for example in the rewrite of the WIP bot, being able to throw out a big button that says override this merge. Are there any plans to be able to actually extend the UI in novel ways? Everything there right now is happening with API calls, with I assume, UI elements that are related but the UI elements are static. If someone wants to put a novel piece of the UI, that button is going to require an extension of the GitHub UI by GitHub itself. Are there any plans to be able to, I know it's a dangerous waters, perhaps at a limited fashion at first but maybe more so, add different interactions and the actual application. BEX: I think this is actually the most exciting future of GitHub as a platform. In the past, GitHub APIs have only specifically supporting things that you can do through the command line or you can do through GitHub's UI itself. The Checks API introduced the very first non-integration specific UI element essentially and the merge button that I was referring to in WIP is exactly that. It's essentially this button that you can change the text of it to be whatever you want and you can listen on that action and then you can do as an integration or an app, anything that you want based on that. I think that's the most exciting direction for GitHub. Because if you look at Slack, Slack is a platform that has sort of really impressive integrations in that response. Your apps on Slack can really do all of these things, use custom UI elements, so I think the most exciting features for GitHub as a platform is all of this customization and giving the power to the apps. ROBERT: Yeah, that sounds an awesome way to be able to extend GitHub without having to try and throw the feature on to GitHub developers. BEX: Exactly. I feel that a lot of the struggle right now is that there aren't these nice ways of communicating via apps because I feel lot of the apps and bots end up just commenting on issues and pull requests and taking up a ton of screen real estate as a result and I just think that that's not the way that bot should ideally interact with the GitHub platform. They should have their own space to exist and that's the feature I'm most excited for. CHARLES: Yeah. I can think of having like progress bars for CI checks and your various appointments. It's too exciting. I'm glad. That's definitely the response I was hoping to hear. BEX: Yeah. We're excited for it too. ROBERT: Basically, you all have a massive community of a bunch of developers that would want to do this and are willing to get their hands dirty on it. Enabling that community is probably the root of all Probot is about. That's super awesome. BEX: Yup. CHARLES: That's a good place to end, because gosh, it's going to be so exciting to have the millions of developers on the planet, just like surgeon to the APIs that you're developing. BRIAN: One thing to add to that too, about the whole million developers, there's a number that's been thrown out from Stack Overflow and also, some other people who are saying like there's 50 million developers, there's 24 million developers. As far as GitHub, our public user number is 28 million, the cool thing about Probot and GitHub apps is that there's a good chance that all those people that are using GitHub today are not actually developers. They're like PMs or designers and what's really cool about this, like having interactions with that kind of platform in this way is that you can now enable all the non-developers to be able to interact with your GitHub repos and start bringing more designers and PMs onto to the GitHub platform to interact with the developers. ROBERT: That is an interesting point. That is awesome and something that I'm always looking for is a different ways to collaborate with non-developers on my team because... I don't know, developers tend to think everything is always centered around code but it's not. The shifting at work that are awesome, needs a lot of collaboration from non-devs and non-dev skills. That would be really interesting to see. I'm excited for that to play out. BRIAN: Yeah. There's a blog post that was published a month ago, I think about where the design team, design system teams rather, built the integration to Figma to update their icons effectively. I just posted that in the chat to look into but they also built this as a Probot app as well. ROBERT: That is awesome. BEX: Yeah, that one is super exciting. You would have the app comment, the diff between what the old icon versus what the new icon look like and it's just such a beautiful design change to be able to see that shift. ROBERT: Man, I'm happy that this is happening. The future seems super bright. Where can we direct people to get resources to contribute, to get involved and start really going at this? BEX: Basically, Probot.GitHub.io has all the Probot stuff, /app has all the listings for apps you can install today, /docs is where the docs are, if you want to get started and hopefully from there, we link up to the necessary things that you need to do. BRIAN: Also, what I mentioned too via Probot Slack channel, there's a Slack channel as well and they do a weekly call. I think, it's weekly or bi-weekly call to actually chat with the Probot community. If you have questions, you can actually bring your questions to the team. BEX: Yeah, we call it 'Office Hours' and it's once a week and it's under our community page, where we also have a link to our Slack. We have a link to another podcast we run and basically, how to get involved in the Probot community. ROBERT: Those are really helpful resources. I do remember seeing that Office Hours. It's on Thursdays, right? BEX: Yes. ROBERT: I was going to drop in for one and then, I actually forgot. Actually, it might be going on as we talk right now in this podcast. BEX: It starts in half an hour, I think. ROBERT: That's awesome. Cool. Well, thank you Brian and Bex for having a conversation about Probot. This is really awesome. Is there anything that you would like to plug for yourselves? How people can get in contact with you? BRIAN: Yeah, I am BdougieYO on Twitter. Everything you need to know about me is there and I am happy to say hello. I'm also helping with the GitHub developer program, which is sort of getting a soon-to-be announced rebranding. If you go to Develop.GitHub.com/Program and you want to have more conversation about the API and GitHub apps on the GitHub side, you can go there to sign up. BEX: And I am HiImBexo on Twitter. You can ping me in any Probot stuff. I'd be happy to look at any Probot code. I've been looking at it for a while now so I'm happy to do that. ROBERT: That's awesome. Thank you all for having a conversation with us. This was really fun. I'm so excited about everything you can do with Probot. This is a really fun project. I'm happy that this is happening and I will make a Probot in the future. CHARLES: I'm looking forward too. Robert has been excited for quite some time and he definitely talks a lot about it and now, I have some insight as to what -- ROBERT: It's happening, I'm telling you. Well. Thank you for being here and we are the Frontside. We build UI that you can stake your future on. We are specializing in JavaScript. We can build anything that you want throw at us. We do functional programming, React testing, Vue, anything in JavaScript, we specialize in. As always if you want to suggest anything for us to have on the podcast or talk about, you can reach out to us at Contact@Frontside.io and like I teased earlier in the podcast, next episode is going to be all about Microstates, the immutable and functional state container, composable model system that we've been building, it's controls as a brainchild for the past two years. That is next episode and I'm really excited about that. It's a really fun API and expressive to build models with. Thank you, Mandy for producing our podcast and we'll see you next episode.
In this episode, Harry Kaplanian of EBSCO joins the show to talk about the FOLIO project: a community collaboration to develop an open source Library Services Platform (LSP) designed for innovation. He discusses the why behind the the decision made to embark on this project, the benefits it brings to the market, and what makes it different from current offerings and other open source projects. Harry also shares where he sees FOLIO going in the future, challenging and unexpected outcomes of making the choice to go open source, and advice for other businesses that are considering embarking on a similar journey today. Do you have opinions on this show? Want to hear about a specific topic in the future? Reach out to us at contact@frontside.io or on Twitter at @thefrontside. This show was produced by Mandy Moore, aka @therubyrep of DevReps, LLC. TRANSCRIPT CHARLES: Hello everybody and welcome to The Frontside Podcast Episode 102. My name is Charles Lowell. I'm a founder and developer here at The Frontside and I'm going to be today's host. So today, we're going to be talking about the business of open source with Harry Kaplanian. Harry's someone that we've had the pleasure of working with over the past year and collaborating with on a very unique project, which we are going to get into later. With me is Robert DeLuca. Hey, Robert. ROBERT: Hello. CHARLES: And before we start out today, we just wanted to take care of some quick news. Just so you know, we released BigTest React. This is actually something that grew out of the work that we've been doing with Harry. It's a set of React helpers to acceptance test your React applications so that you can make sure that you know that your application is actually working on actual browsers in actual life. So, that's exciting. You can go check it out at BigTestJS.io. And with that said, let's get on with the project. So first if all, welcome, Harry. HARRY: Thank you. CHARLES: I teased this in the intro, but we'll go ahead and say it outright. The project that we've been working on with you and with EBSCO is a project called FOLIO. And it's one of the more unique projects we've ever had the pleasure of working on. So, maybe you could just give us a brief overview of what exactly FOLIO is and how it came to be. HARRY: Sure. So, FOLIO I think can best be described as an open source platform for services. Our focus has always been the library market and so, currently this platform exists in the library space. That said, there's nothing about this platform that is library-specific. It can actually be used anywhere, anytime, for anything. The platform itself, all communication occurs basically via HTTP. It's microservice-based. It's language-agnostic. And it is really the core of what we're building. However of course, a platform on its own is not very useful. And at least for right now, libraries are overall expecting to get to a point in terms of features, functionality, apps, and services that are being built. So FOLIO can actually take over the enterprise-wide day-to-day operations of what it takes to actually run a library. CHARLES: And these are libraries like university libraries, the Library of Congress, the library at Alexandria. I mean really, when you think of library, like library [3:00 inaudible]. HARRY: Yes. However, we decided to focus on a particular market, at least initially or I guess a subset of the library market. And so, that subset we've actually chosen to focus on is the academic library market. And so, really mainly colleges and universities all around the world. That said, there is no reason why others can't use this software. And it is all open source. So, if any changes are ever needed, modifications or additional applications are needed to support those additional markets, that can certainly happen at any time. CHARLES: Historically, it's a very complex problem, right? There is a lot that goes on at these libraries. But historically, the software that's driving it has not been open source. HARRY: That is correct. So, it's sort of an interesting history. And I guess you're leading into what I think was maybe your second question [Chuckles] which was: What are the circumstances or the thinking that lead to the creation of a project such as this? And really, libraries have been pretty much operating the same way, to be honest, for probably about the last 40 years. And I'm sure we either all know or have heard, most libraries are facing shrinking budgets. They're really challenged in terms of space and storage requirements. And just in general, users have changed dramatically in the last 40 years, the last 20 years, maybe even the last 10 years. Most people don't really ever expect to walk into a library and actually conduct research anymore. They expect that to happen elsewhere. And so, in many ways, they're really facing what amounts to Google and others, because that's where people expect to go. The vendors that typically created the systems and software that supported these libraries, the day-to-day operations over the years, many of them have consolidated. And there are just actually very few options left. These are really old, monolithic systems. They were built initially with the best intent. And for the first couple of years, many features and functionalities were added. But over time, these systems grew to be so large. There really was never any one person anymore that fully understood it. They find themselves in situations where if they add a little feature here or modify something there, it breaks something somewhere else within the system. And so really, they've gotten to a point where libraries have generated lists of features and functionality that they need that are frankly years long. And the vendors just can't deliver any of it. They just don't have the ability anymore. The systems are too old. In addition, really they're sort of building these walled gardens where they really try to take over all the day-to-day operations of the library but are actually starting to try and expand out in other areas of the university as well. And as they do this, they really tend to lock the system down. They're not really open to integration with other systems or other vendors. And they really see it as they are the one company you should go to that can completely manage and operate everything that goes on in your library. And when we talk about innovation in these systems, really they've been operating the same way, as I mentioned earlier, for the last 40 years. Really, the only major, major innovations is there's maybe one company now that's offering these services on the cloud. One size fits all. And the real issue there is, all of these libraries we talked to, all of them see themselves as being unique in some certain and specific way or sometimes multiple ways. All of them believe they're bringing something special to the world in terms of knowledge, research, understanding and learning. And they tend to cater or specialize often to the institution they're a part of. And so for example, if a library belongs to a medical school, of course their collections are tailored that way. A lot of their workflows and operations tailor to students that conduct research in that field. And oftentimes, even to a hospital that may be associated as well. And this will oftentimes be very different from what might exist for example in an engineering library. And so, for most of them, they look around at what's available to them. Not only has nothing changed, not only has there been no innovation, but on top of it, they're forced to choose essentially a single system. And that's not really getting them what they want or where they want to be. So I, as well as some others, spent quite a bit of time looking around at the market and trying to assess and understand all of this. And one of the things we felt was, as we look at all of this, really we're living in a world, these libraries are living in a world of closed systems, lack of transparency, lack of change, lack of innovation. How do we change that? How do we spark that innovation? And it seemed like what we really need is some sort of an ecosystem, a sustainable ecosystem or really, openness and transparency. And it seems like the only way to get that really is open source. And in fact, we actually conducted a survey and out of that survey, 100% of the libraries we spoke to really believe in open source. And when you think about it… ROBERT: Whoa. HARRY: Open source really in many ways aligns itself really nicely with the mission of the library, right? The library is there to not only promote learning but to make sure information and knowledge that normally you may not have access to is accessible to everyone. They're really leveling the playing field. And that really is a really nice analogy for what open source is doing in the marketplace as well. And so, this seemed like really, and at least, an initial ideal path to head down. And so, we did. CHARLES: Yeah. And so, you did. And I'm curious to explore more about that decisions. Because – now this is a little bit of a dated quote – but Steve Balmer, when he was the head of Microsoft, called open source communistic and un-American, then how do you reconcile that? Because obviously you represent a business that ultimately needs to be profitable and is engaging in this in order to make a profit. And so, where does the balance lie and how do you reconcile that? And obviously you all perceive this as a strong business move. And so, yeah, how do you balance those two concerns? HARRY: I think from our perspective, we looked at it as if a single vendor or maybe a group of two or three vendors end up locking up the entire market with systems that operate the entire library that are not open and they're not willing to integrate with other systems and services that exist out there, in essence that's an attack on what is the basis of our business as a company. We provide services and we provide content that libraries use and ingest and make available to their users. And essentially, it looks like they're trying to lock us out. And how does that really help in terms of creating an open and fair marketplace where people get to make choice? It doesn't. And so, what can we do to disrupt that? And building a system that allows everyone to play at least puts us on a level playing field where if an organization or a library chooses to adopt it, we're now back in a very competitive situation where really, the vendor or the organization or the individual that provides the services that best fit that library's needs wins. And that's what we really want to happen. And so, that's in essence how we win. We believe we offer better services than most others in the industry, if not everyone else in the industry. We think there's a sizable percentage of the market that truly believes that. And we just need to make sure that the mechanisms or the systems are in place that allow us to compete in that manner. And in the end, if what we end up with is a platform that allows 3, 5, 10, or 20 companies to actually compete in a reasonable manner fairly, really it's the customer in the end that wins. It's these libraries and it's the users and everyone else conducting research. ROBERT: So, I'm still kind of reeling from 100% of libraries said that they would like to have an open source platform. How many did you survey? HARRY: So to be truthful, it wasn't a large number. ROBERT: That's still insane to me. [Laughs] HARRY: Right. We focused on, however, surveying different segments of I guess really representation within the library. So, the people that do the work, the middle managers, and then really, the folks at the director level that are really responsible for running and organizing a library. ROBERT: The ones that will experience that pain of being locked in, right? HARRY: Right, right. And trying to get decision-makers at many different levels. And we tried to also cover both very small libraries to very large, enormous libraries as well. And all of them saw a benefit. One of the pieces where things sort of maybe went in the other direction as far as the statistics are concerned, if you ask them though “How many are willing to adopt open source?” the numbers aren't quite the same. In fact, they're rather dramatic. And so, at that point it's roughly about 33% said they'd be willing to adopt. CHARLES: So, why is that? HARRY: So, we asked. And interesting question. And what most of them came back and stated was, “Well, we love the idea of open source. We fully support it. We're willing to do what we can to help it and make sure this happens. However, we ourselves don't believe we've got the staffing, the resourcing, or the knowledge to host this ourselves.” And so, this leads to another really interesting thought or idea as far as open source is concerned, is so this really implies that there really needs to be sort of an ecosystem of organizations that are providing these types of services. And to be honest, the faster we can make that happen, the better off the market is. Libraries are now getting to choose vendors that they get to get their services from, which is rather interesting. Because for the first time, not only do they get to choose the vendor but if they're not happy with the vendor, they should be able to switch vendors but keep the software and the data the same. Today, in the library market, if a library chooses to switch vendors, that means essentially they're completely starting over in their library. They completely have to migrate to a fully new system which means enormous amounts of training, enormous amounts of data transfer, and all the pain that goes along with it. I had a librarian tell me once, “The next time we migrate to a fully new system, I either want to die or retire.” I just want [14:30 inaudible]. ROBERT: Man, that sounds painful. HARRY: That's how painful it was. CHARLES: Yeah, wow. Because the complexity intrinsic to these systems, it does, it boggles the mind sometimes. Just even the little slices of the ecosystem in which we're operating, there's just a lot at play there. ROBERT: Yeah. Like trying to work with other vendors and be agnostic so anybody can adopt these things that we're building. And since we had that vendor lock-in, it's hard for you to adapt the two systems, because they were built in complete silos. So, trying to merge that is an interesting challenge. HARRY: And I think everything I'm saying here, I'm talking about it from the library perspective. But if you look at other industries, I don't think it's really any different. If you look at any of the enterprise-wide management systems that are used in corporate organizations today, most of these corporate entities will stick with the system they chose for a very long time. Because it's incredibly painful, it's disruptive, and it's expensive to switch from one system to another. And so, the idea of you're not happy with a vendor and hey, I can go to somewhere else and maybe keep the system I have in place is really kind of interesting. And even better, if the vendor's not able to provide some of those features that I need but for the most part I like the system, wouldn't it be great if I can hire someone to do that work and to have those pieces built for me? And those are some of the things that open source offers. CHARLES: Right, right. So now, I understand that for example to kind of put a concrete bow on this, [Chuckles] to move away from abstraction of what it means and provide an example, so we have this open source platform, this FOLIO platform. Which if I'm a library, I could in-house, I could either use it on premises on my own servers or I could run them on AWS or I could run them on Google Cloud – there are all these different deployment options and some libraries are doing that, but that's something that they might not want to do. So, I understand that EBSCO is actually going to be for example one of the services that y'all are going to be offering around FOLIO, is actually hosting the platform so that the overhead of managing all of the servers and provisioning accounts and doing all that stuff is going to be taken care of for you. So, I can just sign up. HARRY: Yes. So, that really fed into the results of the survey. For those libraries or organizations that felt they couldn't do it themselves, where would they go? And who could provide those services? So, we're absolutely going into the business of providing these services for libraries that choose to. But at the same time, we're really strongly encouraging other vendors and organizations to provide these services as well. Because for instance, you may represent a series of libraries in let's say Hungary. You may have a vendor there you enjoy working with in the past. It's provided great services for you. There really ought to be no reason why you should not choose that vendor to provide those services for you. And again, it's an open system. There's nothing there that's excluding me and the organization I work for to provide additional services, to provide content or what have you. It's an open system. And so, we should strongly encourage that. And the reality is, if we're going to have a disruption in the market, it seems we can either have one organization or one company trying to usher this disruption along. But what if we could get 10? Or what if we could get 20? Or more companies out there ushering this disruption along simultaneously worldwide. It feels like we've got a lot more of a disruption happening in that case. And so, that's what we're strongly encouraging. CHARLES: Right. But at the same time, I can see it from if I were considering making a play like this, one of the things that would worry me is, “Okay, so we're going to disrupt the market by allowing a large tent where lots of vendors could compete.” But then, I immediately experience fear of, “Well then, how do I differentiate myself?” Because ultimately I want to be competitive. How do I make it so that I'm not a commodity in this new market? HARRY: So really, I think that's key for any vendor that chooses to compete in a market like this. Do you believe as an organization you have not only what's needed to compete but those key differentiators that at least a certain segment of the market would be interested in you and the services you offer? We believe we do. Another company that just recently announced that they're going to be supporting FOLIO and providing services around it is a company called ByWater Solutions. They've actually been in another segment of the library market for many years providing services to libraries as well. Longstanding, actually, with open source codebases. And so, this very much appeals to them. They believe… ROBERT: Oh, interesting. HARRY: They have a niche that they can provide and they are going to be doing that. And we strongly encourage it. What's again great about it is if they set up a library with FOLIO, they're not building the walled garden. They're building or providing the open platform that we can connect our services to. So in many ways, yeah there are some areas we're going to compete. But there are actually many areas that we're also going to be complementary. And I think that's what's really, really interesting to us. CHARLES: So, it sounds like you're also counting on just by having the ecosystem in place and having this sea as it were filling the ocean to enable trade, one of the bets too is that you're going to open lines of business that weren't even possible before. HARRY: Correct. So, another way to look at it – we've got our iPhones or Android devices. Apple built a platform is really what that phone is. When it was released, you could make voice calls. You could do some simple text messaging, some basic email, and I'm sure there are a couple other of things in there that I've forgotten. I do not believe that Apple had any idea that they'd actually be able to create a marketplace that had hundreds of thousands of people and vendors providing features and functionality for that platform. I am pretty sure no one at Apple had ever imagined the guitar tuner would be released. I don't know, a compass app, flashlight apps, whatever. You name it. And not only that, as a user of that platform, you're not happy with the standard email app? Well you know what? I can get a different one. In fact, I've got a choice of many and I can find the one that suits me best. And that's fine. It's an open marketplace. And so, with FOLIO today, we're in early stages. We're building out that platform. We're building out some of the basic functionality for that library that they need to operate as a library. But once this gets released, it's at that point when we believe the more interesting things are going to happen where we build a cataloging app for FOLIO. Well, we know already that there's other organizations that are starting to look at and think about, “You know what? We've got a whole other way or a whole other interest in terms of how we'd like to support those workflows on the FOLIO platform.” Or even better, librarians starting to think about, “Hey, if I've got this open platform just like Apple does, this means I now have the option to build features and functionality that take me where I want to be five years from now, 10 years from now. And I've never had that ability on the existing systems that we use today.” And so, one of the other I think advantages or benefits that something like FOLIO or a platform like this brings to the market is the ability to create that marketplace very much like Apple has done and Google has done with Android as well. ROBERT: So, we're pretty far in, but you mentioned FOLIO a couple of times. Do we want to unpack what FOLIO actually means? HARRY: Yes. FOLIO actually stands for the Future Of Libraries Is Open. ROBERT: Right there in the name. I love it. HARRY: Absolutely. CHARLES: So, we've talked about this platform. We've talked about this marketplace. We've talked about the business incentive of why you would want to do this, why it makes a lot of sense. Clearly, this takes an enormous investment. So, we think, I think, a lot of people think of open source as a bunch of wild developers running around. ROBERT: Pushing commits wherever they want. CHARLES: Pushing out code and doing stuff. And then it kind of organically grows into something that someone then picks up. ROBERT: And then you burn out. CHARLES: Yeah. [Chuckles] But that's I think a lot of people's experience with open source. But an initiative like this takes a staggering investment both in terms of capital resources but also in terms of will, in terms of management, and putting all these enormous forces in motion. There's developing the software, developing the awareness, and I think right now there are hundreds if not thousands of people working on this right now. How do you even go from zero to 6,000 like that? And I'm not talking about people. I'm talking about velocity. [Laughs] HARRY: Right. Early on, when we were conducting research, market research, one of the things we did, we spent some time looking at what we believed were successful open source projects. And I think what was interesting was in many cases it took a single individual or a company to actually create what amounts to that first piece, that first core building block that others could start to expand upon. And very often, they literally just created it themselves, made it available out there as open source, and basically told the world, “Here. Have at it. Have fun. Do with this whatever you wish.” So, we actually thought we'd like to do that. However, building a system on this scale is not that easy. Understanding the operations of a library day-to-day is not that simple. We need help. And so, what we decided early on was: could we kick off and start this project while at the same time [25:42 inaudible] that community as early as possible? To get people excited, interested, and to get this project in a place where we're not the only contributors, where there are many others contributing as well. And when I say ‘contribute' I don't necessarily mean software developers. But that is certainly one aspect of it. But one of the key pieces of building software is gaining access to subject matter experts as well. And it's absolutely key and critical. And so, our goal of course was to build this community, have that community start to provide that in-depth knowledge, those subject matter experts that we need so we can determine what it is we need to build, how we need to build it, and really go from there. And by including those people as early as possible, I think one of the things we find that happens with this project which is really sort of incredible is the excitement that starts to build. The word of mouth advertising, marketing, as librarians, libraries, vendors and individuals start to talk to others and really spread the word about this project. In many ways, it starts to compound or snowball and build on itself. But that was really challenging. And it was really actually kind of slow going in the beginning. Because we started with nothing. And one of the issues you face is, “Well, you want me to contribute my time to this project, yet I see nothing.” Right now it's just a dream. And so, one of those early, early issues was, “Can we build a team small enough, focused enough, where we can start to build some of the basic, basic, basic core pieces, to really prove to the world that this project is real? This project is actually moving forward and this project is actually delivering.” And now that we're in a state where people can actually see working software, the excitement is just starting to expand and compound rapidly. And we see it everywhere. CHARLES: So, do you find libraries or people in the target market, there's that key point where they start to feel empowered? It's real and then the thing that I think that you're going for is to have the realization dawn that not only is it real, I can put my hands on the wheel and I could control my destiny. And I can contribute now because there's this critical mass. ROBERT: You have the power to make change, which never existed in the library world before. HARRY: Or at least, not in this specific area. Because to be truthful, I think libraries as a whole believe they're making change around the world all the time. But it's really related to content. But this is actually making changes in the actual operations of the library. And they are actually empowered to get involved, to contribute, and to help us get this done. And it seems to really resonate with folks. Because it's something they haven't been able to do previously. And it seems genuinely exciting. And we announced this really, well the market learned that this was happening I'd say a little over two years ago. Maybe more like two and a half years ago. And it was sort of a gentle drip in terms of a roll out. People started to learn because we started talking to them. There was no real active marketing going on. But of course, those people started to talk to other people. And so, it happened. And when we were able to provide those first demonstrations, no matter how rudimentary it was, that's when things seemed to really kick into a much higher gear where people started talking to each other. Librarians started talking to each other and say, “Hey, this is real. This is exciting.” ROBERT: It's not vaporware. HARRY: And this is happening. Right. It's not vaporware. And we need to get involved. And so, they have. CHARLES: Yeah. And the energy and involvement just in the course of the time that we've been a part of the project is apparent. Every kind of gathering is larger and the diversity of topics that are discussed is increasing. And yeah, the conversations have burned from “What can we do with it?” to the last time the project got together as a group, it had much more of a conference-y feel. And the topics being discussed were like, “How do we actually deploy this to our real systems?” Which has been fantastic to see. And so, just to actually feel the traction is just, it's so gratifying. So, I guess one of the questions that is on my mind is – so you mentioned that you had been at this for about two and a half years. What is in your mind the most pleasant, unexpected outcome of this project that you didn't foresee two and a half years ago that you're experiencing now? HARRY: So, I think one of the one's that's surprising is I've never actually personally been involved in an open source project before. And I think one of the fears constantly working for organizations that really have been closed source – in fact, I've been that my adult working life – and you sort of walk into this with some [trepidation]. Because oh my gosh, everything I'm doing, everything I say, everything that gets documented is going to be out in the open for everyone to see? Including competition and everyone else? And I have to say, one of the most interesting things, or one of my favorite pieces here, is: it is so freeing. It's a release. It's actually amazing to not care about what your competition or anyone is doing or thinking. And it's even more amazing whenever you've got a question and someone's out there asking you for access to documentation, you can simply point them to the website, to the URL and say, “It's there. It's all there. Anything you want is there. Go take a look.” I think that's pretty amazing. The other one of course are the relationships between many, many different people who I've just never really been – or I never would have had the chance to work with before. And really hearing all these different perspectives and points of view as far as what people think are right, correct, or what we should be doing – it's great to see. And it's great to see this wildfire word of mouth message that seems to be moving everywhere. We're hearing back from people not just in North America, not just from the EU, but from the Middle East, from South and Central America. There's really just people interested everywhere. And it's amazing to me, because I don't think I spoke to anyone over there. So, it's happening. And that's just an exciting thing to see. CHARLES: Yeah, yeah. ROBERT: It's really cool that open source can connect you with the rest of the world like that. It's just so powerful. HARRY: It does. And it's an amazing thing. ROBERT: And the amount of collaboration that happens, I've never experienced it in any other way. It's really cool. [Chuckles] I don't know how else to put it. It's just, it's almost mind-blowing that you are able to across timezones, across the world, collaborate with somebody on something that you both feel passionate about and you're pushing it forward in an open manner. HARRY: Right. And it's also amazing when you have these meetings and there's people you've been collaborating with for months if not a year or more. And getting to meet them face-to-face for the first time. That's really a pretty amazing experience. ROBERT: Yeah. And it's pretty awesome. At our last gathering that we had at WolfCon, I had the absolute pleasure of chatting up a bunch of different librarians. And hearing their experiences and what they're looking for out of an experience, it's just really cool to be able to talk to those people and see how they work and help build this into the platform that they're wanting to use. HARRY: Right, right. CHARLES: Yeah. I would say on the whole, there's been a lot of those wonderful, unexpected outcomes. Were there any that presented a particular challenge or wasn't quite so much a walk along the path of roses? HARRY: So, you know I think the biggest one is – I mentioned earlier we embarked on this project a little differently versus what we saw as normally successful open source projects, right? We did not have that core, beautiful, shiny object to unleash onto the world and let them know, “Hey, have at it.” And so, it wasn't so hard in terms of getting interest. But what's been hard is the amount of interest has generated enormous numbers of organizations that are interested in contributing to the project. Which also implies then that now, not only are they coming to us asking us, “Hey, how can we help? What can we do here?” but then there's a certain aspect, because this is a first version of, “How do you manage the building and construction of that very first version of this software project when the reality of the matter is, you're not really responsible for most everyone working on the project?” And how do you get everyone aligned and organized? How do you get everyone onto that same path along with those same milestones that we've all agreed on? And even worse, how do you get agreement on those milestones so we can all walk that same path and deliver that first version? And so, I get asked sometimes by folks that are involved in other open source projects, “How do you handle this situation? Or how do you manage all the work that goes on?” And when we tell them, it's kind of surprising because it's not what they're used to seeing. And a lot of it is because of the way we started, the interest that was generated early on, and the fact that we did not have an initial version. And I don't want that to sound like, “Oh, this is a huge problem,” or, “If I were to do it again, I would not do it this way again.” I in fact would do it this way again. I think there are a lot of benefits and it's been a great and interesting way to go. It's just the management aspects of it are definitely challenging. And I think more than maybe we had initially anticipated. Going into this next time, though, I definitely know what I'm facing and I'll be prepared. CHARLES: Yeah. So with that in mind, given that you've been through this, do you have any advice to offer someone who might be considering embarking on a similar journey? And undertaking a similar Herculean task. HARRY: Well, I think you'll find it's harder than you think. And be sure to plan for that. But I guess at the same time, I think my biggest advice or my best advice would be: you need to keep an open mind. And when I mean an open mind, you need to be open to other thoughts and ideas. And I think you need to put yourself in the perspective of what if you were one of these other people that are working on the project or trying to contribute to this project? What are the things that you're doing or rather the things that you're doing now, the way you're acting, the way you're reacting, how does that look to others on the project? Because you're not the only one contributing. You're not the only organization contributing. And are the things you're doing reflecting badly in terms of others' perception or optics as to what this looks like as an open source project? And I think for me and others, there was a huge learning experience. Because your first thought is, “I'm going to tackle this like any other software project,” and it's not. It's not like that at all. CHARLES: I think that's… ROBERT: That makes sense. CHARLES: Yes, that makes sense. That's excellent. That's excellent advice. And I think that's one of the things that makes open source so powerful, is because there's that aspect just baked into it from the get go, you have to be mindful of a bunch of different perspectives. And that ultimately results in a solution that's going to be workable for a bunch of different perspectives, that's going to be flexible to accommodate for all the different use cases that all the different participants in the community might bring to the table. HARRY: And you have to be very mindful of how your actions are perceived. Because normally, your actions are perceived by the company you work for, what tends to have a particular culture. But on an open source project, you're actually dealing with many different cultures. CHARLES: Yes. HARRY: And so, that's a tight line that you have to walk. CHARLES: That was a powerful note to end on. Thank you everybody for listening. We are The Frontside and we build software that you can stake your future on. If you want to get in touch with us, continue the conversation, you can reach out to us on Twitter at @TheFrontside or drop us an email at contact@frontside.io. Thank you so much Harry, for being on the show today and talking to us about FOLIO. HARRY: And thank you very much for having me. This has been great. CHARLES: And if you want to, if there's something that you want to learn about or a topic that you want to discuss, please, please let us know. We always are open for your feedback, especially when it comes for things that you would want to hear. And if you want to get involved or learn more about the FOLIO platform, you can just head on over to FOLIO.org. There are resources for librarians, developers, and anyone who is curious about becoming a community member. Thanks as always to Mandy, our producer. And we've got a really great podcast coming up on the 14th of June. We're going to have Michael Jackson on the podcast to talk about separations of concerns in React. So again, thank you Robert. Thank you Harry. And we'll see you all next time.
In this episode of The Frontside Podcast, panelists Charles Lowell, Rob DeLuca, and Sam Keathley, discuss how much the distinction between frontend, backend, and fullstack developers matter in both personal and professional senses. The differences in mindset between these categories are also discussed, for example, how does a fullstack developer solve a problem vs how a backend or frontend developer? And also, how clear (or fuzzy) is the line that separates them? What are the skills a frontend or backend developer needs to consider themselves fully fullstack? And finally, is there any sort of tribal separation between the factions? Do you have opinions on this show? Want to hear about a specific topic in the future? Reach out to us at contact@frontside.io or on Twitter at @thefrontside. This show was produced by Mandy Moore, aka @therubyrep of DevReps, LLC. TRANSCRIPT CHARLES: Hello everybody and welcome to The Frontside Podcast Episode 101. My name is Charles Lowell. I'm a developer here at The Frontside and I'll be hosting today's episode. Today we're going to be talking about the different developer profiles that are out there. You might have heard mention of them on Stack Overflow, on job boards, on the Twitterverse. Things like frontend, backend, full stack, half stack, short stack. [Chuckles] Things like that. With me to talk about these are Rob and Sam. Rob is our director of open source here at Frontside and Sam is a software engineer here at Frontside. So, hey y'all. SAM: Hey. ROB: Hey yo. CHARLES: Alright. So, before we get into the meat of the topic, there are a few announcements that we wanted to make. First and foremost, this is some really exciting news. This week, Taras Mankovski officially joined the team at The Frontside. And we are really, really excited about that. [Cheers] ROB: That's exciting. CHARLES: Yeah. It's super exciting. Not only is Taras an exceptional engineer, he's got amazing code skills. He's extremely empathetic and a great person to work with. He's great working with clients and he's also fantastic at really working with developers who are climbing that ladder and helping them level up. So, he's been involved in mentorship literally since I've known him. I think that was the first time that we ever came into contact with him was through his mentorship work in the Ember community. And we've been collaborating with him for years on several open source projects. And so, we just are so excited and know that he's going to come on and do some amazing things by helping us be better developers, helping us be better community emissaries. And so, lots of good stuff coming out of that. ROB: Absolutely. CHARLES: Yeah, yeah. No, that's really exciting. So, definitely wanted to throw that out there. ROB: Too bad he didn't join a little bit earlier. We would have had a good time with him on this podcast. CHARLES: I know. Well he's actually, I think he's been on the podcast twice. ROB: [Laughs] That's true. CHARLES: But yeah, there's definitely a couple of topics where his input would have been absolutely critical. I think it would have been great to have him along when we were trying to run our own apprenticeship programs. Those would have worked out I think a lot better. Well, not to say that the outcomes haven't been stellar in some cases. But I think having his expertise would have made it a lot smoother. That said, before we jump in then, just wanted to let everyone know that as always, if you want to work with us you can get to us at contact@frontside.io or send a shout-out on Twitter @TheFrontside. Okay. Y'all, let's jump into it. Like what is even a stack? ROB: [Laughs] So, do you want to kind of set up where this came from, Charles? I think it came from you and I going back and forth on Twitter. And I was like, “You know what? We need to talk about this on the podcast.” CHARLES: Yeah. I think I was actually on vacation. ROB: [Laughs] That's right. You were trying to ski. CHARLES: [Laughs] I was on the ski lift. And I saw you tweeting and I was like, “No, no, no. I got to respond to that.” Probably because there was maybe a little bit of disagreement but also because I just needed something to do. Looking out the majestic scenery of the Rocky Mountains wasn't enough. So yeah, what's the backstory there? ROB: So, I can't remember what I was actually tweeting about. But I kind of feel like there is a distinction between a frontend developer, a backend developer, and at this point I don't know if a full stack developer truly exists. I mean, it does. But it's used more often than it really implies what they're doing. Usually when you hear of a full stack developer, it's like, “Man, I can sling some code on the backend and I can do a little bit of architecture stuff on the frontend. But CSS, keep that away from me.” And in my opinion, if you're a full stack developer, you need to sling CSS, too. CHARLES: Really? ROB: Yeah. CHARLES: No, no. I certainly agree. Because I was going to ask, is it fair to say that full stack developer is code for like, startup developer? ROB: Yeah. That's a good – like a utilitarian, like a jack of all trades but not a master of any? CHARLES: Right, exactly. ROB: I can take that. So, I guess the way we can start to dig into this one is like, how much distinction is there between a frontend, a backend, and a full stack developer? And does that matter in a professional sense? CHARLES: Right. And so, I think my take was that the skillsets that you need in both places are actually much more convergent than you think, than people might think. In other words, the skills that you utilize on the backend actually translate very well to the frontend. I think my stance is that what makes a frontend developer and a backend developer is more the problem sets that you gravitate towards naturally, the things that interest you. I actually was a backend developer. And now I consider myself to be primarily a frontend developer. But the skills that I had developed writing backend systems in Java translated shockingly well into frontend development with JavaScript. But it was working on the portion of the program that interfaced with the human was, that was the thing that fascinated me. And so, it's what I gravitated towards. And so, I guess in my [6:15 inaudible] or the way that I have been thinking about it is that what makes a frontend developer and a backend developer is more what gravitational pulls you respond to, rather than one particular skillset. But I know that you've got a perhaps more refined take on that, or a different take. ROB: Yeah. Before I do, I want to know what Sam's thoughts are on this. This would be a good one. SAM: Where I come from in the understanding of frontend, backend, and full stack is I – background on me: I did a bootcamp before working here. So, I did a development bootcamp where they were really excited to tell you at the end of it that, “Oh, you're a full stack developer because we've introduced you to everything. Like, touched on every sort of little topic. You're full stack. Tell everyone you're full stack.” And it's like, “Well, you're not really a specialist in any of the sense.” So, what makes a difference to me, I'm going to specifically talk about full stack, but it's like before you can say you're full stack, having knowledge of frontend and at backend, you need to have expertise in at least one of them first, or understanding of that mindset first. Like if I say that I like frontend work more then I need to understand frontend work more before I can say that I understand backend work, you know? ROB: Yeah. CHARLES: So, do you feel like the bootcamp was doing you a disservice? Or do you think they got… SAM: I don't think they did a disservice at all. So, they did a really good job. So, bootcamps are really for introducing you to everything. And then whatever you take from that is your own. So, they introduce you to all the little things like database working and then JavaScript, Ruby, whatever it is. And then it's your job to figure out in those introductions, what you resonate most with. So, I think they did a really good job in that. I think that they shouldn't tell you what you are, like tell you that because they touched on whatever it was, that you're full stack. CHARLES: Right. SAM: So, I don't think they did a disservice at all. I learned a lot. But I wouldn't say I'm full stack at all. CHARLES: So, if you feel as though you're not full stack, what do you feel that you are? SAM: Definitely frontend. CHARLES: Definitely frontend. SAM: I gravitate more towards frontend. ROB: Why would you feel that way? Is it because you're more pulled towards the UI of things? Or is that where you feel like you've gotten most of your experience now? SAM: Actually a little bit of both. I tend to like the UI of things a little bit more. It's interesting. Whenever I was going to the bootcamp, I thought I was going to be more backend because I thought I really like working with the behind the scenes sort of. You don't see this work but you know it's there. But I definitely like the UI of frontend a lot more. Just personal. ROB: I see that. SAM: And it's where my skillset is, because this is all frontend work that I'm doing here at The Frontside. So, it's what I know most at this point. CHARLES: Yeah. I'm curious too. Isn't it true that there's also a conception in the wider tech world that frontend is limited to CSS and HTML? ROB: Ooh. Yeah, that could be. CHARLES: Like when you see job posting for frontend developers, typically it's like, we want someone… ROB: Yeah. It's kind of morphed from – JavaScript developer is now a single-page app developer instead of frontend. Yeah. Yeah, I see what you're saying. It's now, it could be considered as HTML and CSS and not JavaScript anymore. [Chuckles] CHARLES: Which is so bizarre to me. SAM: [9:42 inaudible] a lot of places when a frontend developer who is also a UX designer. Someone who has the knowledge of designing things from the ground up and then only working in things that look good, rather than logic. ROB: Yeah. This can be probably an entirely other topic. But that's like the frontend of the frontend and the backend of the frontend, is like [10:03 inaudible] call it. Because there are [10:06 inaudible] developers. And single-page apps have gotten so complex these days, between your data layer and your testing and managing state on the frontend. There is a backend, an architecture, that you need to consider when you're developing a single-page app. And I've worked on teams where there are developers that are working on the frontend but they still don't like CSS. They don't do CSS things and they don't consider layout or any design. They just take the ticket and they get the data going. Get the data from the server, request it, display it on the page. And then another person would come through and style it up to the spec or the mock. But yeah, that's interesting. So for my take on this, and where this actually really came from, is we were going through some hiring. And we were trying to figure out how we can place somebody or hire someone that can be effective in the role that we immediately needed. And all of our immediate needs were in frontend-specific things. And we can actually talk about if they're frontend-frontend or frontend of the backend. But where this kind of came from or was born was: if we're hiring somebody that needs to fill a frontend role and they're primarily a backend developer, there are gotchas and things you have to know to be an effective frontend developer. Like quirky browser things that happen or like how to set up a Babel transpilation setup. And what are the gotchas here? Kind of the history of the frontend web, because there's a lot of things that are there that can cause a lot of headache if you don't know the backstory. And I know from your perspective Charles, you think – and this is isn't wrong – but you think that if somebody has a really talented backend developer, it's really just concepts that you can apply to the frontend. And I agree with that. CHARLES: Right. But you have to want to do it. [Laughs] ROB: Yes. That is also a key. CHARLES: You have to be gripped by the problem set of the frontend. You want it to inspire you to leverage those solutions. ROB: Yeah. So, would you say there's a different mindset between a frontend, backend developer, and full stack? Does a full stack developer solve a problem that's different from how a backend developer or a frontend developer would? CHARLES: That's a subtle question and probably one that might offend people, my answer to it. So, I'll go ahead and answer it. [Chuckles] ROB: Hashtag [12:23 inaudible] CHARLES: Actually, maybe not. Just in my experience historically – I want to underline the fact historically – I think that it is people who gravitate towards the outside-in mindset, in other words the very first thing they're thinking of is, “How is this going to feel to a person who's going to use this?” If that's your first question that you ask, then you probably are going to gravitate towards the frontend. Then in terms of the backend, I'd say historically – and like I said, I want to underline that, and I'll get to that later – I think it's people who are more drawn to: I want to attack the most complex problem that exists. Like distributing state over 10,000 nodes, managing huge deployment infrastructures that drive these massive websites that happen at this mind-boggling scale. And so, there you really are thinking computer to computer and, “What's the ideal interface for doing that?” And then on the frontend you're thinking more like human to computer, because that's where the interface lies. Now the reason I say… ROB: I feel so let down. Where's the controversy? CHARLES: Well, okay. Well, because I think that basically – I think the controversy is saying like people who are more naturally empathetic. I don't know. That would be the controversy thing. But I want to say historical. ROB: Yeah. I can see that. [Laughs] CHARLES: So, I want to say historical too, because I feel like one of the things that's been magical about the last two decades of software development is the dawning realization that no software you write exists in a vacuum and that it's all interconnected. And so, I think that I for example feel very strongly while I try to think about the user experience first. And the developer, I'm also thinking about developer experience first. The way that you want a system to feel is going to inform the design of the UX workflow and that's going to affect the architecture of your frontend. And if you're interfacing with it through different media, like websites, phone, I don't know, even text messages, Slack, what have you – that's going to inform then the next level of how your APIs are shaped, your public-facing APIs. Which then inform the structure of your internal deployments. So, recognizing that the decisions you make in one end of the stack and ripple through completely and that they're not what we hold as dear these concepts of separation of concerns and complete and total isolation of layers. That really is false. But what it does is it enables us to change the individual layers to more – ironically the reason you want to have separation of the layers is so that it more easily lets you change them provide a more integrated experience. So, I think that's a long way of saying that I think frontend developers are more aware of what's going on in the backend and that they might be drawn into the backend. And the same thing goes for backend developers, realizing that the decisions they make are affected by the frontend and affect the frontend. And so, I think there's been a dawning more of a collective responsibility for design, for operations, for developer experience. And I think it's great. ROB: Yeah. And to [think back off] that. So, I was kind of wondering where the controversy was, because that's kind of my exact answer, too. Where do you gravitate towards on? The difference of mindset is like, if you've got a problem and let's say you're a full stack developer, so whatever that manes, but if you're given a task where you have to implement the frontend and the backend, where you gravitate towards first I would say is how you would attack solving a problem. For me, I would immediately go to the frontend and see how a user would immediately interact with it and then work out that way. Mostly also because I like having a tight feedback loop. And the frontend provides that nicely. I can change something and immediately see the difference there. And in the backend you can spend a little bit of time, unless you're TDD which you should be – you can spend a little bit of time piecing together things and figuring out the architecture. And then you'll have something that you could show for. It's just nice to see UI get thrown on the page. And it's real and you click a button and something happens. That's a really nice feeling. And that's kind of where I gravitate towards. And if yeah, if I had to attack the problem, I'm going there first. I get the endorphins from it. [Chuckles] CHARLES: Right. So, if we want to add controversy to the other side, because you got to always be controversial, right? So, we said the really blunt horrible oversimplification is that people who are more naturally empathetic gravitate towards the frontend. You could also say that from the other perspective, people who are more internally validated will gravitate to the backend. Because if you need to get those endorphins from working on the frontend, basically what you're saying is, “I want to put it in front of people and get the feedback from those people and know whether it's good or bad.” Whereas you could say, “Actually, I don't need validation from other people. I've got this concept of this architecture that is going to be the ideal thing. I know it. I've got this vision and I'm going to chase it, regardless of what's going on.” I think you can more readily do that on the backend because you're not as open to the feedback of a whole bunch of different users. And like I said, I think those are gross oversimplifications. ROB: Yeah. I agree. And so, as a devil's advocate towards the backend developer that is less empathetic, I think actually some of the best backend developers I've ever worked with were the most empathetic people ever. Because they know that software is for humans. And humans are going to be consuming the API that they build. And they take that into consideration. CHARLES: Absolutely. And I actually think that empathy pervades good software development just wherever you find it. Because yeah, someone's going to be using your APIs whether it's software as a service or it's a library. If it's software as a service, it's got to be easy to work with. It's got to be performant. It's got to have a nice command line. And so, you have to be thinking at all times, “What is it like to use this software? What is it like to consume it? What is it like to link it into my library? What is it like to call it from a web client?” So yeah, I think you're absolutely right. I think it's actually one of the great virtues of great software developers. SAM: Yeah. So, something that I've always kind fo had a question with, especially coming from that bootcamp setting: is there any sort of tribal separation between what you would consider a frontend or a backend developer or even full stack? Like, “I'm better because I understand data layers than I understand how a button fits on a page,” that sort of thing. CHARLES: I would say absolutely yes. If you at the, just do a quick poll of the Twitterverse, it seems like people tend to hang out in little circles of similar interest. So, there's definitely a bunch of people that I follow that are always tweeting about backend stuff. ROB: Yeah. Twitter is ‘build your own echo chamber'. CHARLES: Yeah. And there are people who I follow that are tweeting nothing but mostly frontend – when I say frontend, I mean frontend of the frontend. Well basically, from CSS down to nothing deeper than React components. And then there's people who are talking primarily about the backend of the frontend. So yeah, I do think that people – there are clearly, there are different areas of the stack and I think that people do tribalize around them. So, the question is: Who are the border agents? Because always cross-border trade. Any time you have borders, there's an exchange of ideas and information and things that happen at those layers. And actually, one of the things I'm really curious about is: How does that happen? ROB: Yeah. You might have the best perspective on this because you did – you were using Java Swing back in the early 2000s building UI. And that's like backend things. And then you had a lot of experience with Rails and you've moved into the JavaScript world. And the thing that I've noticed with frontend is, we're applying a lot of concepts that existed on the backend. And we're moving all that complexity into the frontend. And that's kind of where that fuzzy line of, “Are you the frontend of the frontend? Are you the backend of the frontend?” comes from. And it's interesting that – like this is going to be another podcast. We'll end up talking about this – but we have – MVC lives on the frontend. Like in React, your C is a container component. The controller is a container component. The model is probably your Redux layer if you have it or if you're using micro-states. And the view is your components, your view layer components that you're rendering down. So, these concepts are moving from the backend to the frontend. We're just kind of renaming them and making them our own concept. But they're there. So, how does that knowledge sharing happen? Is it really just a mass exodus of backend developers interested in frontend developers now? CHARLES: I don't know. I think it goes both ways. So, I can only speak to my own experience. And that was when I first started doing frontend development was back in 2003 I think. So almost 15 years ago. We were building a touchscreen interface for an electronics vendor in the UK. And it was just so much fun, because we were getting to build these interfaces that literally looked like nothing else. And you could touch them and you had support for rudimentary gestures and there was no keyboard. And basically the only interface was your fingers and like a price scanning gun. And everything, all the entire UI had to be developed out of that. And so, it was such a novel system and it was so fun to implement that I just gravitated towards it. I think the thing that was particularly compelling was it was very interactive UI. It was high-touch, literally. This is a clerk who's sitting there using this thing as rapidly as they can to sell stuff and move people through the line. So, it was like a unique problem. But the thing that was cool about it was I realized so many – we kind of already touched on this in this conversation – so many things that I had learned on the backend applied there. And there was in order to provide that experience, there had to be a pretty complex machine behind it. And that was fascinating, that machine. And so, we were able to apply a lot of the concepts. And so, in that time on the backend I'd just come off working off a backend that had basically an event bus – we would probably call them microservices now, although we didn't call them that then – were coordinating based on all these events. And that translated over into the frontend really, really well. And I remember using a lot of the techniques for exception handling – doing it on the frontend and doing a lot of the multithreaded stuff that we were doing on the backend, trying to reconcile that with the frontend and really understanding. So, there were all these analogous concepts. And it could be – so, there was an original exodus of backend development, for me personally. And so for me, it was like I felt like I moved onto the web pretty much exclusively in 2006. And it was a good – I would say 2013 maybe is when web UI development finally caught up with Java Swing. Like it was like, that whole time I was like, “I'm using the web because it's an awesome deployment platform. And it's got all this great stuff,” but man, the developer experience is not as good as like a stateful fat GUI was back then. And now, I would say it's actually surpassed quite significantly. But now, I think there are a lot of people maybe who either – I think there's a casting back of people who are in frontend development and casting back to the web. So right now, my love affair with immutability and functional programming started by using a Clojure web framework which borrowed a lot of the ideas from Clojure. So, I guess there was an exodus there – people from Clojure wanting to develop apps, wanted to have their goodies. But then I found that and I was like, “Whoa, that's awesome.” But then that really set the hook for me. And so now, I try and go back to the well, the backend well or the shared computational well, as much as I can. So, all of that stuff of basically discovering all this functional programming stuff, this highly formalized functional programming, all came from saying, “Wow. I got a taste of that. Let me get more.” So, it's very much like trying to run through the village of the backend and ransack the stores and take as much as I can and bring it back, because I know that it's good. ROB: So earlier in the podcast, you said that you were a frontend developer. You described yourself as a frontend developer. And that's funny, because I actually consider you a full stack developer. It's because you can jump in on the backend and sling any kind of code as well as anybody could, and then you can also do the same thing on the frontend. So, the question I have here is: Do you think actually someone that truly is a full stack developer – and we can define that in a second – but do you think they're at an advantage here, because they have all that experience? You can answer yes. But why? Why do you think they have an advantage? CHARLES: I think they have an advantage in the same way that a person who's bilingual has an advantage. So, if you are living in North America, it behooves you to speak Spanish because you are now open to an entire set of markets that were previously closed to you. Well, not entirely closed but like you can trade with Mexico and you can trade with South America. You can buy and sell goods. You have access to yeah, emerging technologies that might – something special might be happening in Mexico City, or in Medellín, Colombia, and you're going to have first access to that. And so, if you don't, if you're not bilingual, then you're going to have to go through an intermediary who is. And so, there's a premium: you get to be the middle man so to speak. Or you get to be, maybe that has kind of a negative connotation, but you get to be the broker of new ideas and new technologies. If you can, if you are fluent in backend so to speak, then you can go into backend-landia and you can talk with the developers there and see what kind of cool stuff they're doing. And then you can take it across the border into frontend-land and sell it. And so, that's – I think we're very much first to the gun on – not first to the gun but we're ahead of the pack in terms of functional programming. Because we've seen that. We've seen it proved out and we've now actually started to integrate it into your day-to-day routines. And we're ahead of the pack on that, right? And so, I think that's kind of a keystone analogy for me that I think really, really captures what is the advantage in being a full stack engineer. ROB: So, well how do you define a full stack engineer or developer? What things do you have to possess to actually claim that title? In order to be a full stack developer I personally believe you have to know, you have to be comfortable with CSS. You can hate it. You can yell at it. But I think you have to be able to put together a layout if a designer gives you a comp in order to claim full stack. And in the same token, if you're a full stack developer and you mainly came from the frontend, you should be able to implement backend features. I don't actually have – so, I'm strong in the frontend, not so strong in the backend. What would be the analogous of CSS in the backend? Would it be like mocking your controller actions? [Chuckles] CHARLES: I would say you should have a familiarity with HTTP and REST, would probably be the equivalent. Kind of like a foundational technology that just every single technology is oriented around it, or most. Regardless of the protocol you're using – there are things like GraphQL which are not really RESTful, although it's a blurry line there – but they're still delivered over HTTP. And so, understanding things like CORS, understanding the things that are going to be universal to APIs. ROB: I think a lot of people try to claim full stack. I try to claim it. And I know I'm not. I can put together a pretty crummy Rails API on my own for personal projects. But I'm not going to be the one that's setting up indexes on a database to optimize a database or anything. I think that does come in time, but you have – borrowing from Brandon Haye's talks about career development – if you're a full stack developer, I think you end up having to be in the industry for a long time. Longer than what we consider a senior developer these days, I think. Because you could be a senior developer and be specialized. And we see that all the time, and that's okay. But in order to amass that knowledge and experience, I think you have to be in the industry for a long time and done those things a couple of times to really know. CHARLES: Yeah. I agree. So, can I add something there? To continue the analogy of being full stack is like being bilingual or multilingual. I go back to those analogies a lot because that's what I – linguistics is what I studied in school. But when I was studying linguistics, one of the things that was going on there was they were kind of redefining what… ROB: That makes so much more sense of your vocabulary. [Laughs] CHARLES: What it means to be bilingual. There was kind of a reorientation of that definition that was going on while I was in school. And that was being bilingual doesn't necessarily mean you're able to converse about poetry or like deep technology or give speeches in another language. It really is, it's as spectrum of… ROB: Ooh, that's quite interesting. CHARLES: The definition of bilingual is like: Do you use another language in your life? So, if you are let's say someone who is a recent immigrant to a country and let's say you've got less than 1,000 words but you're using them to buy groceries, to pay bills, to do things like that, then you are bilingual. Bilinguility is not an exclusive club. It's really, are you actually using a language? So, if… ROB: Ooh, so that actually gets my wheels turning. Does that mean that you can have a junior full stack developer? Because like, if you just merely speak the both languages, technically by that definition, I jive with that. You can speak two languages. You are bilingual. You can write in two different languages. You might be full stack. But does that mean in the software industry you are a junior full stack developer and then as you go on and get tasks that are full-stack-like, you get better on both sides? Or is it as an industry, we really have to specialize? CHARLES: To start on the first point, I think that if you – let's just restrict it to one language – if you're doing JavaScript on the frontend and using Node.js on the backend, if you're writing Node code you are I would say by definition, and certainly by the definition of bilingual that I just proffered, you would be considered full stack, a junior full stack developer like I was saying. And I think that it's just been my experience that as you go on in your career, you will cross multiple layers of the stack just because you can't keep your hands off. If you have an ownership mentality of, “Here's this thing that needs fixed,” and it's on the backend, you know what? I'm going to go ahead and learn a little bit of how to do this, because I need it to work with the thing that I'm working on. ROB: Yeah, that makes sense. CHARLES: You will just naturally be drawn over borders throughout your career just because someone's got to do it, to do the thing that… ROB: You got to solve a problem. CHARLES: Right, when you've got to solve a problem. So, I think that people become more and more full stack over the course of their careers. But that said, I do think that there are clearly areas where functional specialization is absolutely a requirement. Like if you say, “You know what? We've got this API and we need to support 600 queries per second and we've got this huge, crazy deployment…” ROB: I will not be your person to do that. CHARLES: Yeah, exactly. That's something you're very much – that is something that you're very much hiring for. And you want to be hiring for like you said, someone who has the skill and someone who, that's what they want to do. ROB: Yeah. If you need better state management and rendering performance and testability on the frontend, I'm your person. [Chuckles] If you need me to scale your API, to handle hundreds of thousands of requests a second, nope. [Laughs] So yeah, I agree with the specialization. So Sam, I want to know. Has this conversation swayed you any way? Are you more interested in being full stack or are you leaning to a side more? [Chuckles] SAM: So, I think full stack is, it's as much about skillset as it is about personality. So, like you were talking a little bit earlier on how someone who's frontend might have more empathy towards the user. And someone who's backend might have more empathy towards the computer and the developer, rather than the end user. I think someone who's full stack has to have a wider range or empathy so they can empathize with all rather than just one or the other sets. I think personality-wise, I'd be a fine full stack developer. But I think professional-wise, I do gravitate towards the frontend because that's just how I am. As a person, I like to see the visual rather than the concept. I do a lot of painting. I do a lot of drawings. So, I'm a very visual person. So, it's really helpful to me, especially for someone who's junior and who's still learning and who came from that bootcamp experience, to be able to see what I'm changing. So, I think it does kind of go a little bit into experience as well. So, I think over time when I start touching on maybe some more backend work and I see some stuff that interests me, definitely I'll gravitate towards it, just because I want to learn. But I don't know that it's like an intrinsic quality, you would want to be full stack or be backend or be frontend, you know? ROB: That's interesting. Yeah, I agree with that. CHARLES: Yeah. That is actually a really interesting point. Because I think what I heard you say there is that when you have – software is a hidden world in many ways. You talked about it in the beginning of the conversation, the areas of the site unseen. Like you know it's there but it's hidden from view. So, there's a certain amount of vision that needs to develop. So, you kind of internalize what software, like this hidden software, looks like. So what does a deployment of some load-balanced microservice clustered thing look like? Most people would not be able to answer that question. But the more time you're exposed to it, the more it becomes burned into your inner vision. ROB: Mental model? CHARLES: Yeah. You develop a mental model. Your brain literally wraps around that. It takes time. But on the frontend, especially if you're a visual person – but I would say even if you're not – I think the same would apply to someone who's using assistive technology. It's still something that you can feel with your sense and you don't have to develop a sixth sense to perceive it. So, there's literally a problem of perception there. And so, maybe frontend work is a great on-ramp for everybody, because it's so perceptual. ROB: That's exactly why I picked it. I was trying to do iOS dev before. And I could not do it for those reasons. I didn't have that nice tight feedback loop of even though it was UI, in Objective-C you had to construct your UI and buttons out of Objective-C. Unless you wanted to use Apple's Interface Builder and that was, meh. Everybody built it procedurally. And what I loved about HTML and CSS, was I could instantly throw something on the page, attach some CSS to it, and see it change immediately. And that felt so good. [Chuckles] CHARLES: Yeah, yeah, no. And it's important to get those really tight feedback loops, especially when you're starting out. ROB: So, if we had to tie this back together, did we decide that there is a distinction between frontend, backend, and full stack? And if there is, or isn't, why? SAM: I think there's like… CHARLES: I don't know if we've… go ahead. SAM: Kind of a distinction. But it's also really fuzzy, I would say. So, if I'm going to explain what the difference is between frontend and backend to someone who maybe isn't a developer, I would always go back to a video game. So, when you see your guy… ROB: Ooh, this is interesting. SAM: Yeah. [Chuckles] When you see your guy running around and you're like, “Oh, this game looks really good. I'm really into these graphics,” but do you ever think about what it takes to save the game or what is actually being saved when you go to save it? So, if you're more interested in making the guy run and making the guy and the environment look good, you might gravitate more towards frontend. But if you're really interested in saving the data, like well what is being saved when I hit ‘save to this slot' or whatever? Then you might gravitate more towards backend. ROB: Or like the network layer of the online multiplayer? SAM: Yeah, yeah. ROB: [Laughs] That's interesting. I've never used video games as an analogy. I always go like, you know, if you're a frontend developer, you know that button that you click? That's the button I create. And then the action that happens is usually what the backend developer does. I think I like the game analogy better. [Laughs] SAM: I think the game analogy is something that most people can grasp. And most people can grasp. Like I'll tell people the difference between frontend and backend if I'm talking about Facebook. Like what I do for a job is I'll show you everything that's on your Facebook page. And when somebody, or the backend is somebody who makes all of that come into play, kind of, you know? But I think video games are easier to conceptualize that abstraction of data being saved rather than trying to explain the intricacies of Facebook to somebody. ROB: [Chuckles] CHARLES: Yeah. No, I like that. I like that. So, I guess if we achieved consensus, would we say that these do exist as broad functional categories? And a full stack developer is someone who will move in between those functional categories through the course of their career. And that generally, the trend is that the longer the career goes, the more you will do that. ROB: Yeah. I can get on board with that. I'm going to use your career as like the guiding post of that. The way you just explained it, it kind of made something click for me. You describe yourself as a frontend developer. And I think in our industry with software development and the way teams work, I think you end up specializing no matter if you call yourself a full stack developer or not. Because I do think you're a full stack developer. But you've mainly been working on frontend for the past what, three years, four years? So, I think it's okay to call yourself a frontend developer. But you are a full stack developer. But as you specialize around and amass that knowledge, that's where you become that full stack developer, right? And so, at one point in your career, you were a backend developer. And then now at this point in your career, you're a frontend developer. But now you have those experiences and you can draw on both of them and apply them across the fields, right? That's super interesting to me. So, maybe it is that you have to specialize in order to achieve the full stack. And you're never truly a full stack developer in your role. But it's possible, depending on the team and the team dynamics. It's just interesting that you can be a full stack developer, also at the same time be specializing as a frontend developer at that current time, right? Yeah, that's interesting. CHARLES: Alright, so it sounds like consensus achieved. Let's all virtually hug. [Chuckles] ROB: Send the hug emoji. CHARLES: Alright everybody. That is our show for today. We are The Frontside and we build software that you can stake your future on. We would love to hear from you, especially if you have an idea for a future podcast topic. Any news that you think is something that we should discuss, even if it's not the theme of the whole episode. And we will see you next time. As always, you can give us a shout on Twitter at @TheFrontside or just drop us an email at contact@frontside.io. If you enjoyed today's podcast, it was produced by Mandy Moore, the inimitable @TheRubyRep. So thank you, Mandy. And see you all next time.
Frontside alum and original podcast host, Brandon Hays, makes a special guest appearance to talk with Charles about the evolution of The Frontside as a company: where it's been, where it's going, and more hopes, dreams, and goals for the future! Transcript CHARLES: Hello everybody. Welcome to The Frontside Podcast Episode 100. Here we are. Episode 100. My name is Charles Lowell. I'm a developer here at The Frontside and I think it's safe to say, your official podcast host. With me to celebrate the 100th episode, he was also here a few episodes ago but also was here on our first episode I believe, is the [inaudible] Hays. Hello Brandon. BRANDON: Hi. CHARLES: Welcome back to the podcast. BRANDON: Actually, are you going to light your trainee badge on fire now in a bucket, in a ceremonial pyre? CHARLES: I live in New Mexico, so I think I'm going to just after this, grab my shotgun and give myself a 21 gun salute. Just in my front yard. BRANDON: There goes old man Lowell again, with the shotgun. CHARLES: I'm just going to [gun shot sounds] in my own honor. BRANDON: I was at the Alamo this weekend, actually. And I don't know if it was just because it was fiesta in San Antonio but they had a demonstration, like a musket firing demonstration where those things are basically little cannons. They're just small cannons. It's very interesting. They're very loud. CHARLES: Yeah. They're small, handheld cannons, yeah. So wait, were you – what is fiesta? Now, as someone who grew up in Central [inaudible], I feel like I ought to know this. BRANDON: I don't know. We found out by accident because we were planning a weekend to go hang out and get drunk on the riverwalk and we took our families down with some friends and then they're like, “Oh, it's fiesta,” which is like a 10-day celebration of the history and establishment of San Antonio – which I did not know is a 300-year-old institution. So, it's like one of the oldest things in this entire western United States. So, it's pretty neat. It's different. It's weird. It's like 90 minutes from Austin. There's nothing in Austin that's older than six months. Every six months we must demolish something and then build a condo skyscraper in its place. So, it's kind of neat to be in a city where it has – walking around the Alamo, I'm realizing, “Wow. Setting aside any of the historical significance of Texas independence or whatever, this is just like a really interesting very old building. This is hundreds of years old in an area where there's nothing that's hundreds of years old.” So yeah, it was pretty cool. It was a good weekend and we got to see muskets being fired. And we saw a doctor gross my kids out by talking about the medicine of the day, in full costume and showing all of the procedures and threatening my kids with amputation. And it was a good time. We all had a good time. My nine-year-old thought it was the coolest damn thing he'd ever seen. CHARLES: Really? Did the have bloody saws and everything? BRANDON: Oh, yeah. CHARLES: Was it like a reenactment of 300-year-old surgery? BRANDON: It wasn't a full reenactment. But it was a graphic description using the tools of the time. CHARLES: Wow. BRANDON: Highly recommend, check out the Alamo. Super fun. CHARLES: That does sound really cool. BRANDON: I did not expect to have a good time and it was a good time. CHARLES: Yeah. Yeah, I know the whole reenactment with the musket firing is fun. And it is, it's actually an incredible building. Although there's been a big kerfuffle about something about how they're going to preserve the lawn. But I haven't really followed that too much. BRANDON: Yeah. Yeah, I don't care about the lawn. I care about – no offense, lawn, if lawn is listening. This is not weird, how Stanley broke our brains with the word ‘lawn'. CHARLES: That's true. BRANDON: Yeah. He broke us real good. CHARLES: Yeah. I can't see a lawn without a beard. BRANDON: So yeah. So, life has been pretty good, man. Let's see. I left Frontside September, October. CHARLES: 2016. BRANDON: 2016. CHARLES: So, it's been months. BRANDON: 18. Yeah, thereabouts, right? So, I assume that nothing happened since then and if I came back to The Frontside now, everything would be exactly as I left it. My posters are still up in my room. My Bon Jovi poster. You left my bed just as I made it, like kind of unmade. Everything is just preserved as a shrine to me. CHARLES: Pretty much. I mean, we did give away the mics to Goodwill. BRANDON: No. CHARLES: We actually did not give away those mics. BRANDON: I never even got to use them. CHARLES: I know. Well, you know part of the problem is we don't even get to use them that much either. It looks really cool and it plays really well, like our podcast studio. But you know, I'm now spending 75% of my time in Corrales, New Mexico. And at any given time, people are either working from home, or working remotely. So, a lot of times the podcast room tragically does not get used. But it looks so cool. People come in there and they're like, “Wow, you guys must be really smart and technical people.” BRANDON: I realize this is probably a rote stereotype at this point, but I am assuming the only reason that you moved is that you are dabbling in the production of meth. CHARLES: Pretty much. BRANDON: It's like, I want to learn a new trade. Programming, it's just – programming, how interesting does it stay honestly for 25 years? CHARLES: Right. Yeah, and you know, we've got some good techniques. Continuous integration, deployment, things like that. Test-first. These are things that can be applied to different verticals. And I was looking… BRANDON: [Laughs] We ship meth to production on the first day. CHARLES: Right. [Laughter] Exactly. So, I figured it was a market ripe for disruption. BRANDON: [Laughs] It's probably true. So yeah, I wanted to ask you about that. You all kind of scattered to the four winds in some ways. You have Elrich in Boston and you're in New Mexico most of the time. CHARLES: Joe is in [inaudible]. BRANDON: Oh yeah, Joe moved to New York. CHARLES: Yup. And honestly, the traffic is so bad in Austin that I'd say 50% of the time, people stay home rather than drive into our centrally-located office. So, that's actually something that we're struggling with right now because the bulk of the team is still in Austin. But the office space is underutilized. Our team size now, we have eight engineers. And five of them are in Austin. Our other staff is also in Austin. So, what do we do with the office? It's a big question. BRANDON: And that's quite a cultural change, too. Because when I was there, we would tell people, “We want to be able to do remote someday. But we just don't know how to get into that culture to change the way that we do our meetings and change the way that we do standups and coordination and communication.” I didn't feel like we had the tooling at the time. So, something – I knew that at some point there would be probably a forcing function to basically catalyze something to allow that to work. And I'm curious to know what that process was like there. CHARLES: I wish I could say that there was a process other than experiencing the force of the forcing function and then being forced into it and then just kind of dealing with it. I have not taken a poll of the other remote employees of which now I am one, at least for the time being. So, I don't want to speak for them. But it was less painful than you might imagine. And the reason is because – and it's one of those things you actually gave me this analogy back, probably three or four years ago and I love it – is sometimes you're hanging off of a precipice and you don't realize that you're toes are two inches off the ground. And then all you can perceive is the precipice and you feel the weight of your own body concentrated on your fingers gripped to the ledge. And you don't focus on the fact that you're actually, the fall is only two inches long. And that's kind of what we experience with the remote culture. Now, I don't want to say we were Pollyanna about it and didn't realize that this was the step that we were taking and making sure to check in with the remote employees. But one of the things is our communication styles were already very asynchronous both for our client work, for our internal work, using mostly Slack and GitHub pull requests and issues – certainly for the development portion, very little changed. What we didn't realize is that because of our involvement in open source, we were already acclimated to a distributed work style. We just didn't really realize it. We didn't have to change much. I think where we have a lot more work to do is kind of integrating people socially and making sure that conversations don't happen that aren't available for other people to consume asynchronously. So, if you're having some architecture problem and you're sitting next to somebody, you'll take that avenue rather than let it play out in chat or over email. And there is definitely a certain portion of that, but I think we still do a lot of pair programming. That's still our major mode. I'd say 75% of our code gets written as people collaborating. And so, while those in-office discussions do happen, the ramifications circulate rather quickly. And most of those are in the context of people pairing inside the office. Does that make sense? BRANDON: Mmhmm. CHARLES: So, I don't think the office and the physical space were as much of a bottleneck as we thought they might be. And so, because of the – a lot of people did work from home already because of the traffic. And we were involved in open source. And our communication with our clients is usually – we don't currently have any clients in Austin. So, that's all to say that the transition was actually quite natural. And I think there's some strong analogies between collaborating in open source and having a remote culture in your office. I think what we need to get better about is making sure that we get the team together at least twice a year, everybody together. Making sure that people are able to understand their priorities and get to circulate around and get introduced to a bunch of different people. And yeah, I don't know. There's definitely a lot of work to be done on the non-development front. BRANDON: It's interesting. The agile approach to things is to try something. I'm starting to think the agile and the scientific method are related where it's like, “Here's a hypothesis. Here's the experiment. Here's what we think we want to learn,” and then you learn it and you take the next step based on that information. And that failure is an option. I think that's the point of agile, is to make failure safe because it's small and you're guaranteed to learn from it. Like, the point is to learn. And so, I really, I'm starting to think that those are just basically the same thing. That agile is like the application of the scientific method to product development. And it sounds like you're being agile or experimental about your work. And the trick is, like any scientific discovery, the trick is in coming back around to it and analyzing it and deciding whether this was successful or a failure based on feedback and finding what the measurement was that you were trying to improve. So, the lesson there was, “Oh, people become disconnected from each other. We need to gather everybody for an all-hands periodically.” We didn't use to have to do that because all-hands was every week, at least. CHARLES: Right. Yeah, everybody was constantly – there was a constant chatter and you could just kind of, the context was just all sitting at that one table, in that one room on 38th Street. And all you needed to do was dip your ear into that pool of context and you're set. Whereas that's just not an option right now. So yeah. I think the danger with agile is not being concentrated in your experimentation. I think what gave us our fear about saying we're going to do remote work – because I remember we always talked about it. We danced around the issue – was are we going to lose who we are? We have a set of way that we do things. And there is power in kind of sticking to the framework of the way that you do things. Because you understand it and you know it. So, when you're pushing and you're experimenting, being able to say, “We're going to – push and we're going to focus on this one area and we're going to iterate on it and we're going to keep everything else static,” it's going to be the wall that we can walk along. But we are going to push in this area. And so, I think the dangers of you doing that in all the areas of your business or all the areas of your project, you're iterating and refining, nothing ever gets done. And so, it's kind of like once you get to some ground that's solid, when you do start iterating it, you start introducing instability. So, when you go remote you have to start thinking about remote work, whereas we didn't have to think about that before. We were essentially, the feature of saying that we were a one-office company and an on-site company is we didn't have to think about that problem. BRANDON: One thing that you were just taking about is this idea of concentrating so that your experiments are happening one or two or maybe three at a time instead of trying to run five experiments at a time. And yeah, there's another danger I think in agile of seeking local optimization where you're basically like – it's like taking a bacteria and running it through many, many, many iterations that's targeting one thing and it mutates into this weird thing that only does this one thing. Or a dog breed that the whole – did you see that, I don't know where this came from but there was some scientific findings that there was a dog that was bred in ancient prehistoric times that was bred to turn a spit to roast meat over. So, they bread a dog that the whole point of this dog was to turn a spit so that people could roast meat and go to sleep and let their dog serve it, cook for them I guess? CHARLES: Wow. BRANDON: That's pretty impressive. CHARLES: I would say like their dystopia is in the past. Or certainly canine dystopias. I guess we live in a canine dystopia. BRANDON: Not in my house. CHARLES: Not at your house. BRANDON: This place is known as a canine paradise. So yeah, I think that's a really interesting point though, that limiting the number of concurrent experiments so that you can actually respond to them in a meaningful way instead of just being like, “Wow, we learned a bunch of stuff we're doing wrong. Anyway, back to the grind.” CHARLES: [Laughs] Yeah. BRANDON: Back to sucking at everything. CHARLES: Right, right. BRANDON: That kind of feeds into a lesson that I have learned very, very, very recently in the interview process for looking for my first real job in over a decade. And that process is very humbling. And one of the humbling experiences was being rejected for a job from a very notable larger former startup here in Austin. And their interview process is really buttoned up. I got really deep into the interview process and at the end of it they're like, “Oh, you're not technical enough.” And it was really, it was like, I don't know. It was hard for me to process at the time but it's super easy now to look back and go, “Oh, I was definitely not a fit for that type of job if being able to write JavaScript on a whiteboard without the aid of Google to solve problems and refactor code is like a fundamental part of what is valued in a manager there.” That's just not going to be me. But one thing I – and it wasn't a colossal waste of time. There was a ton of time and energy I invested into that specific process, but I actually derived a ton of value out of it. Because every person I met there was focused on the same thing: their culture of making experimentation inexpensive so that everything there is framed in terms of an experiment. What's the experiment here? What's the hypothesis? What's the expected outcome? How soon can we get to a place where we can validate that outcome? So, it's kind of like everything is really lean. And yes, it does – like I asked, “What's the dark side of that?” and it can lead to optimizing for a local maximum. So, you have to pause every once in a while and reflect at a larger scale. But it changed my attitude about a lot of stuff. I tend to walk around fearing failure. That's more my speed. I'm afraid of failing because failure can be catastrophic. But that's because I take big swings at stuff. When I go give a conference talk, it better be the best conference talk of my life. When somebody's like, “Oh, that was the best conference talk I have ever seen,” I'm like, “Ah. I'm so glad you said that because if you'd said literally anything else I would have collapsed internally.” You know? The stakes are so high for everything. And making it safe for yourself to fail by treating things like an experiment and working with my teammates. And so, two or three scenarios over the phone in a week when I was managing the team at my last company, somebody would bring something to me and I'm like, I instantly went to all the reasons this probably won't work. “Here's the problem with this.” And I thought, and I immediately turn around and went, “Wait a minute. Bring me a hypothesis and the experiment and how we can experiment with this thing.” And he's like, “Well, we could try this next week and we'll know whether or not this is a good idea.” And we tried it the next week. It was like organizing an architecture team because we were waiting to hire an architect. And the results were mixed for reasons I won't get too deep into. But the fact was, it gave us the freedom to try things. And I'm trying to carry that spirit around with me now. It's been really eye-opening. So, completely like, just a 4% alteration in the way that I think about problems, but it has the ability to dramatically alter the trajectory of how I solve things in the future. CHARLES: So, do you include now inside the planning process experiments? Like, a certain number. BRANDON: Absolutely. CHARLES: So, the typical “enterprise” development is we have our features, we're going to do them in this order because they're this priority. And then agile comes along and it's like, “You need to take these things and you need to break them up into small chunks so that they can be accomplished in small time slices,” so that you don't basically bark up wrong trees. Or explore [inaudible]. BRANDON: Yeah, but that's almost like a stupider version of waterfall. CHARLES: But exactly. That's exactly my point. Whereas the problem is, there's no avenue for experimentation in there. Rather than saying the entire team is marching in this one direction that meanders around and focuses in on the local maximum, which hopefully is relative to the market landscape is the absolute maximum, saying, “We're actually going to be marching in one major direction but we're going to be sending out scouts at all points.” If you were actually – I've actually been reading a lot of ancient military history. And It's just insane that an army, or even a detachment, would go all in one clump. They're constantly sending out people. Information is really, really, really important. BRANDON: That's an extremely, extremely good point. I've actually – it's so funny, because I've used a very similar description where we are trying to chart a course to this ocean of opportunity somewhere. And we can't just send the whole team in a direction hoping that the ocean is in that direction. We have to have our Lewis and Clark. Somebody has to be the cartographer. Somebody has to be the explorer. And that means that there has to be a little bit more freedom for those explorers. I don't yet know how to translate that into software terms. I just know that that's a collaboration usually at most companies between product and development. That product is doing some of the exploring of the space and then development is doing some of the exploring of the technical capabilities and possibilities there. CHARLES: So, you see it. What's interesting is you see it in product planning, kind of in the large, with the waterfall. You see it in huge organizations. They have a research and development department. And I wonder if agile kind of saw the Balkanization of your feature set into very small component parts. Can you take the exact same principle and Balkanize your research and development and integrate it into micro-iterations? We have this R&D but we're going to integrate it into our day-to-day and week-to-week process. BRANDON: I think that is a really noble goal and I think I see some people making progress toward that. The company I interviewed with does it almost to a pathological degree where there is a point of diminishing returns where you're sort of bound to this process of experimentation. And at a certain point you can only achieve incremental results. CHARLES: Some of these problems, you just need to be able to think about them for a long, long time. I actually didn't read, I actually didn't see the talk. But everything from the title, Rich Hickey's ‘Hammock Driven Development', just that title resonated with me so much. I was like, “Yes,” because sometimes you just need to be in the hammock for six hours at a time. Or in the shower. Or hiking. Or doing whatever it is that you need to do to put yourself in a zen state where you're just, your brain is slowly turning its wheels. And it can follow every lead to its conclusion without any interruption. And sometimes that process can take hours. Sometimes it needs to take weeks. BRANDON: Right. I want to kind of pivot on that. Because that's actually one of the biggest things that I've learned in the intervening time since leaving Frontside, which is creating space instead of trying to maximize – one thing that I did when I was at Frontside and then did again at my next place and I'm realizing is really has long-term negative implications is cram as much into a work day, as much output out as possible. I'm very output-oriented. I want to jam as much into my day as possible. I want to jam as much software out the door as possible. And people describe working at Frontside while I was there as one of the most intense work experiences they'd ever had. Literally, I can project that, literally just from my own intensity of trying to cram all that stuff. And providing that space for developers to ruminate on hard problems, on some of the harder problems they encounter, providing space for managers, I've learned that a big chunk of what it is to be a manager is to be available. And so, I actually want to write a sign – I was on the fence about doing this but I think I'm actually doing to do this – I have an office and I'm going to write a sign and put it up on the door that says, “If I look busy, interrupt me and remind me I'm not doing this right.” So, creating the space to ruminate or to be available for discussion, people that protect their breathing room sometimes are made fun of, especially in American corporate culture. I walked in and they were just reading a newspaper. What the heck are you doing at work if you're just going to read a newspaper? Like no, this is actually really important time. CHARLES: I think it's, yeah, it's something that I think about a lot. And I know I've shared this analogy with you before. I don't know if I've done it on the podcast. But I saw and I can't take credit for it. I actually saw it at DevOpsDays I think in 2013. There was a woman giving a talk and she was just talking about managing developers. But one of the things that she was saying was that if you looked at a microservices architecture or you looked at just even your operating system, and if your CPU was constantly pegged, you were squeezing out 100% of every time slice, instructions were just flowing through that, you're going to have a very unhealthy, very brittle, very prone to failure software system. If our microservices were not available to actually service requests, and service excess requests, and service spikes of requests, then something is fundamentally wrong. BRANDON: I want to add to that a little bit, because the thing that I noticed in managing a team where I received a ton of pressure to peg everybody out at 100% – and it jived with my philosophy at the time of, “Hey, I'm 100% guy. Everybody I work with is 100% type people. And then, let's peg everybody at 100%. This is a startup. Let's get everything going,” and I realized very, very quickly that if you don't preserve a little buffer, 20% buffer in that level of intensity, there is no ability to share resources. Everything is now a silo. So, if you're going to peg all your CPUs out, part of that thrashing is that there's no time for people to share things with each other. And people become very protective over their little silo all of a sudden. And it causes us – it's actually like the first stage of a catastrophic cultural collapse if everybody's pegged out at 100%. And literally, just dialing down the intensity is often the only thing that's necessary to get people to feel comfortable sharing some of their time with each other. You do a really good job of that with the lunch and learns. You mentioned that y'all are doing better thoughtful lunch and learns and stuff like that. It's one of those forcing ways that you can force that and say, “Hold on. Stop the development and do some stuff where you're actually sharing things with your teammates.” CHARLES: Yeah. And we do that. My biggest concern is that that actually increases the intensity. So, one of the things we've done is we used to actually be very formal about our lunch and learns. It's like, “We've got to generate content and put it out on the web so that people can see us.” We backed away from saying – we're not going to do them as often and make sure that people can actually do them. Yeah, making sure that people don't feel overwhelmed by, “I've got a lunch and learn coming up.” The point is to share something that you're passionate about and maybe introduce some really cool ideas to ferment in people's head. Rather, that's kind of the goal. There are certain things that we do very much feel interested in generating content. But I think, we've kind of been dancing around the ideas of distributed computing and IoT and what are some of the others? BRANDON: If you say blockchain, I'm going to just virtually punch you in the face. CHARLES: [Laughs] I actually didn't. Did I say blockchain? BRANDON: No. I just was waiting for you to say it. CHARLES: Okay, no. I haven't. Well, because that's – but it is distributed computing in Web 3.0, right? These problems – and we're actually going to be podcasting about this next, so in two weeks you can tune in to listen to us talk about blockchain but in the context of distributed computing – and one of the things that we're seeing is now we're starting to pay the price of outsourcing all of our lives to these central services like Facebook and Google and Amazon. And I think now they're starting to build a credible and more mainstream movement to wrestle back that control and say, “What would it mean to have software as a service that wasn't actually dependent on some central thing?” What would it look like to have Slack where it's Slack that looked like email? Where everybody had their own email server, maybe not a bad example. But you've got an email at Gmail or Microsoft or Yahoo or your company-run that's big enough its running its own Outlook client or something like that. Email is actually a really great example. Now probably people are going to crucify me for saying this, but I think it's actually a good example of a distributed system that's worked well. I own all of my email. All the messages that you send to me, I own, and all the messages I send to you, I also own. But you also own the messages that I send to you. Information is duplicated. And it's fine. If I send you an image, yes it's on your hard drive or it's on your Google Drive. You send a message to me, it's got an attachment, I also have that attachment. But the point is that we can each own our email and we each own our email service. And we can change it up. That's not possible with Slack. That's not possible with Facebook. That's not possible with all these other sharing platforms. All of them are controlled by this one thing. And so, I think that that's something that we've been exposed to through the lunch and learns and I'm actually certainly very excited about it. It's not something that we're going to be investing in immediately. We're kind of dancing around that idea. But that's something that's come out of that. So yeah, we've kind of refocused it on, what is something that you feel good about? But back to the original point, I think that this is something that applies on all fronts. If you have a business where you can't actually take opportunities because you don't actually have people – so there's maxing out at the individual level, filling up people's workspace with client work or filling it up with what have you or having them work nights and weekends. There's individual maxing out but then there's like maxing out of your business. So, if you have – we're a consultancy – if you have 100% utilization or you're shooting for 100% utilization, that everybody is placed on a project, that is a brittle and unsustainable system. BRANDON: I wish you would have told me that 18 months before I left there. There were like two years where we were at 100% for two solid years. CHARLES: Yeah, yeah. We're still at 100%. BRANDON: Yeah. I wonder what would have happened if we'd had a little, if we had figured out how to build in space. CHARLES: Part of the problem – so, here's the thing though. Space, nice space costs nice money. BRANDON: Yeah. CHARLES: And so, that's the thing, is you have to charge more. And you have to say, “We are going to be more expensive than other people.” You have to be dedicated to be at the forefront of a cultural battle, essentially. In the same way that people were with testing, where it was very [controversial]. BRANDON: Yeah. You were with CI. CI is a given now, right? CI is… CHARLES: Yeah, like [inaudible]. BRANDON: This idea was semi-revolutionary when you and I were talking about this in 2012, 2013, that we ship to production on the first day. We don't even start building software until the CI system is set up. The first thing we do is set up Jenkins and tests and get everything, the pipeline working. And now, that's just what people do. By and large, that's how software is expected to be built. And the tooling has really come up around that. But that was an expensive way to sell software five years ago, that, “Hey, this is going to cost more than bringing in Cowboy Bob and having them come jam in your console for 40 days and ship a bunch of stuff that then will most likely collapse and you won't know about it and Cowboy Bob has ridden off into Juarez, Mexico.” CHARLES: Right, with his saddlebag stuffed with your cash. BRANDON: Yup. CHARLES: Yeah, no. So, you have to – the problem is, you know when you pick these battles, you need to be prepared to fight the war of attrition of they're not going to be able to perceive the value for six months, a year, right? You're going to have to ask your clients to bet on this strategy. And it's a bet. And you're going to have to say, “It's going to pay off in six months. It's going to pay off in a year.” And you're really going to start raking in like five years. That's when… BRANDON: Yeah. Try making that pitch to a startup founder that is borderline, that is on the verge of an anxiety attack, and you can kind of just figure out what my last year was like. And the… CHARLES: So, that's one of the reasons we don't really work with startups anymore. They have a five-year plan, but not really. BRANDON: Yeah. CHARLES: They're fighting for their survival. And they're fighting for the opportunity to have a legitimate five-year plan. And so, in that sense, it's maybe not a good fit for the way that we develop software, because you either need an extraordinarily prescient founder who has been through this before, knows the true costs of software development, and is pretty well-funded so that they can actually – because we're more expensive upfront, like a lot more expensive upfront and so sometimes they flat out don't even have the cash. And that's something that you can make a quick, “It's not a good fit,” but then there also needs to be this understanding and an acknowledgment that what you're really shooting for is your five-year dividend. BRANDON: Yeah. It is really interesting, the turn that occurs when a company finds product/market fit. By then it's too late to fix the problems. So, it's really tricky to find the balance of: how much energy do you put into the success case for a company before they have product/market fit? How much time and energy do you invest in betting that this is going to be successful versus betting that if it is successful, hopefully we'll have the time, money, and resources to redo a bunch of the things that we are going to have to apologize for later? And I think that's what makes… CHARLES: Right. Like, where do those two lines cross on that graph? BRANDON: Yeah. Because you and I have both seen startups completely sunk by somebody who was overly focused on building a scaleable architecture in a company pre-product/market fit. That is a common story where an engineer that doesn't understand the business value of what they're doing and only focused on “quality” will absolutely torpedo, they'll chew up your first million and a half of funding and leave the place in just a smoldering pile of ashes at the end. So, it is tricky. It's totally a difficult thing. But I think coming back to your point of being sort of a vanguard of cultural, the tip of the spear on somebody's cultural changes – DevOps would be one. People that were really investing in DevOps culture in 2010, 2012 saying, “Hey, this, automation, is the future of how software gets shipped, maintained, observed, supported.” And so, now it sounds like, so what is your big bet for the future? CHARLES: Boy. That's a great question. There are two bets. One you're going to like, one you're going to vomit. BRANDON: [Laughs] CHARLES: But that's okay. BRANDON: Yeah. I don't work for you. CHARLES: You need to serve, what is it? You need to serve the spiny urchin with the yellow tail. BRANDON: Is that a Sonic the Hedgehog reference? CHARLES: It's just a sushi reference. BRANDON: Oh, okay. CHARLES: Some people don't like urchin. Or maybe they don't like eggs. What it like, the roe that come with sushi. But they're on the same plate. So, I would say the first one that I've been thinking about a lot is optimizing for capacity and being able to handle spikes and not being at 100% both for people and for utilization. I think that's something that is – I don't see how you could have a healthy software development process if people are completely spiked on delivering, heads down delivering features for product. That is something that I'm betting on. Essentially, you could call it the 25% time but it's really about having excess capacity to exploit opportunities as they arise. And then being protective of that excess capacity. Because you can exploit an opportunity. Your CPU has a spike load up to 100%. But then make sure you [inaudible] down to 50% at some point, or 75%. And so, I would very much like to see Frontside have a bench where people can rotate out and they're working on different stuff that are not even client-related. They can recharge their creative tanks. They're not going to be idle. BRANDON: Yeah, I've really come around on – and I really hated this at the time – but I've actually come around to the thoughtbot style of working on a product where – because owning and managing a product and developing it as a side quest, the goal is not necessarily for that product to catch fire and become the world's next big thing and to replace your consulting revenue. The goal is to give people a sense of – think about all the stuff that you've learned in your side projects that you went back and brought to your work. And some of my biggest gains as a developer have come from having a side gig of some kind, some side project that that's how I learned Ember. That changed my life. And I would never have gotten to try it if I was waiting for somebody at work to tell me it was okay to do it. So, it's about taking that permission back for yourself and giving yourself permission to try stuff. So, it could be something like that, or it could be the content stuff that y'all do. Or it could be conference talks. It could be whatever. But the goal isn't necessarily to produce things that have a direct return. It is to create the space to allow people to flex some muscles of creativity that you may not get in your day-to-day work. And that's very difficult to offer to people in any company. Now having explored startups and larger companies, but I would say especially in a consultancy where the exchange rate is dollars for days. It's sort of like when I was freelancing. I could feel every vacation I took draining both real money and opportunity money out of my bank account. That's such a hard, difficult thing to do. And so, you actually have to create the budget ahead of time and say, “This budget is allocated to these things and it's already spent.” Anyway, that's really tough to do. CHARLES: It is hard. BRANDON: If you can exercise the discipline necessary to do that and create the environment for that, I would say you're ahead of 90% of companies in the industry. CHARLES: Yeah. Yeah, so that's something I definitely want to bet on, because I think that's where the best things come from. BRANDON: Okay. So, what's the thing I'm going to hate? CHARLES: Functional programming. BRANDON: Oh, Charles. Okay, I have to stop you. Do you know what I'm doing? Did I tell you this yet? That I am participating. When I told them this, I was like, “Charles is going to have a field day with this,” but I am participating in a Haskell study group. CHARLES: No way. BRANDON: And I'm like four exercises into this thing. I have to do four more for next week. And I'm like, “This is bizarrely easy, actually,” after as much JavaScript as you and I did in sort of a functional style and then learning Elixir. And I was like, “Wait a minute. The case statement is, Elixir just stole Haskell's case statements.” So like, so far I'm not finding functional programming to be onerous. Or anyway, but we'll see when we get to the static typing. But so far, I'm not getting any of that in the earlier lessons of the book. CHARLES: Yeah, the static typing. But the thing is, you can do – it's not 100% necessary. It isn't in Haskell, for sure. But I'm surprised. What inspired you? BRANDON: We have an architect at the office that was like, “Hey, I want to do sort of a functional programming book club.” So, we have a Slack group for FP study group. CHARLES: Are you doing ‘Haskell: From First Principles'? BRANDON: No. That one was a little actually intimidating. CHARLES: Really? BRANDON: Yeah. It gets into the lingo a little early. And we're doing one called ‘Get Programming with Haskell' that is a little more – ‘Haskell: From First Principles' is kind of math-oriented. So, for somebody with a math background but not necessarily a programming background, it's perfect. But for somebody with a programming background that is just trying to understand functional programming principles using Haskell, ‘Get Programming with Haskell' is actually a really great option. CHARLES: Okay. Actually, I have not heard of that one. BRANDON: The stuff that I'm looking at looks just like Elixir. So, it's early. But it's very comfortable so far. CHARLES: Yeah. So, this is the thing. It's all a matter of messaging and marketing. Because I really feel – so, it is like there are a lot of behaviors that you see sometimes in currently entrenched functional programming communities that I think are, well I think they're objectively repulsive. But I think they're also pragmatically repulsive and that they repulse potential community members. But I think a lot of it too is people talk about these things that are, they use abstruse terminology. And they're kind of chattering back and it's very jargon-oriented. And there's just – people operate with a different set of concrete things. So, when you and I are talking, for example we might talk about a Rails controller and that's a very concrete thing. You know exactly what I'm talking about. It's something that you have held in your hand, literally. Remember when we got that Rails codebase that came as a thumb drive? BRANDON: Yes I do. CHARLES: But the point is you knew that this had a Rails codebase on it. There were any number of controllers. And when I say controller to you, a controller is an abstraction, but not really. Once you work with an abstraction long enough, it becomes concrete. And so, part of the problem is just a mismatch in language where people are talking in their world about concrete things, things that you can touch and you can feel and you can exchange and they're very relatable. But from another person's perspective, they're talking about something that's totally abstract and totally opaque and totally what have you. And so, I feel like yeah there's a huge mismatch there. And that's been one of the big bets. The other big bet that I'm making is on this trying to make what is currently abstract to JavaScript and Ruby developers be concrete. And I think that we're going to see type classes like functor and monoid and semigroup and all these things, they're abstract to you now, become concrete over the next five years. And so, that's something that I'm betting on. BRANDON: Check out this – and I know that you have a good relationship with the people that did the other book, but it really does tend to come from more of a mathematical background. And this one actually does speak to people with JavaScript, Ruby, Python experience. Like, “Hey, here is how you will perceive these things.” And so, it's much more approachable. I'm still in the first unit of the book. But having sort of tasted it a little bit, it's like, “Wait a minute. This is actually extremely familiar and not super intimidating.” CHARLES: Exactly. And that was kind of – so, I read the other book. And I think I was also aided by the fact that I tried to learn Haskell probably for five times in the past. And so, I also had the benefit of jumping against the wall with the velcro suit and bouncing off four times. And fifth time, it stuck. So, I had just temerity on my side and a general feeling. But that's definitely – the lesson that I actually came away from reading that book was like, “Oh, there's a mismatch in concrete concepts.” It's using concrete concepts that are concrete to people with a CS background or mathematics background, or people who are brand new. Honestly, people who are brand new to programming who don't actually have JavaScript or Elixir or Ruby or any other thing to lean on, I think that the First Principles book is actually pretty decent for them, too. Because they don't have anything to compare to. BRANDON: They don't have anything to unlearn. CHARLES: Yeah, they don't have anything to unlearn whereas one of the things I took away was I was like, “Oh, man. I'm using semigroups all the time. This is something that I do constantly.” When I'm coding, I might do it eight times in a day. I just didn't have a name for it. BRANDON: Right. They're like design patterns, just at a micro level. CHARLES: Yes, micro-design patterns. Yeah, it's like a RESTful architecture for your code. In REST you only get five verbs. There's five methods, man. That's all you got. BRANDON: Okay, so those are two bets. And I want to cover one more thing because I know we're super overtime. But the last thing I want to be able to say about talking about what we've learned since I left Frontside but I want to put a bow on that. So, the two things that you're betting heavily on are functional programming as a basis for solid architectures in the future, like the work that you all are doing. And… CHARLES: I would also like to say, and this is something – let me just add one more thought. What I don't understand, and this is in no way like, I don't understand people who do the, “Saying goodbye to framework X.” That's not me with object-oriented programming. BRANDON: Often abstractions are like oversimplifications but they're really useful, sort of like Rich Hickey's Simple versus Easy. Like, “Hey, there's a lot of promise with that metaphor. It's a leaky abstraction but it's a useful abstraction.” And Gary Bernhardt's ‘Functional Core, Imperative Shell' is a leaky abstraction but it's a useful abstraction. If people haven't seen or experienced that, it's pretty good. The subtlety is that these are tools that are suited to certain situations a little better. And those same situations can exist in the same codebase, can exist in the same program. CHARLES: Yeah. I still, I love Ruby. I adore it. And in some ways, I've been researching functional programming and it's been going on for the last four years. So many times, people are like, “Oh, I just can't stand this tool anymore.” And I'm like, “Man, I still love Java.” I don't understand how learning to love something decreases your love for something else. BRANDON: That happens the first two times that you fall in love, is that you feel like you have the old thing less in order to love the new thing. And then you start realizing, “No, you are allowed to fall in love with new things without falling out of love with the old things.” I would almost use that as an interview question. Is there some way to use that as a way to gauge somebody's actual real concrete maturity as a developer? Because that is a mark of maturity. CHARLES: Yeah. I mean, you could say, “What's some tool that you no longer use that still informs your day-to-day routine?” BRANDON: Yeah. I guarantee you, people that were doing Smalltalk in the 80s think about it all the time. CHARLES: [Laughs] Yup. Yeah, exactly. Exactly. BRANDON: Alright. So, I want to cover one last thing. CHARLES: It's part of growing, right? If you're going to grow as a developer, you can't be shrinking at the same time as you're growing. Otherwise, you're like the same size, just in a different place. BRANDON: However, you don't get any Medium think piece points. Nobody does the one clap, two clap, forty, for blog posts that are like, “Why I'm still using some programming language but using one a little more than I used to use and this one a little less.” CHARLES: [Laughs] Zero claps. BRANDON: Yeah, zero claps on that think piece. I just want to cover one last thing before we wrap this up, and it is the fact that Frontside, the biggest gift that Frontside gave me was the mission for the next 20 years of my career. I think it could change, but I'm pretty confident about this, at this point. Being approximately 20 years into my career, I feel like I kind of have a feeling for what the next 20 years is about. And the Frontside really drilled that into me and helped me focus it and helped me dial it in. And it is this idea that there is an incoming generation of programmer that thinks about things differently than the previous generation in a pretty radical way. Because the previous generation all came out of the same schools. They all look the same. They all have a similar shared set of values in general. They created the Sil- – you know, I'm not actually going to be overly critical of the Silicon Valley culture that exists now. It is a result of the type of people that came out at the time that value innovation over almost anything else. People talk about ripe for disruption. The fact is, that has been an engine of economic growth and progress for society in a lot of ways that has a lot of costs that weren't factored in by a bunch of people who all thought the same way. And now, with people coming through code schools and people coming from different backgrounds and people coming from different environments, they're looking at programming and software as either an economic opportunity or something they didn't see that they could possibly do. Those doors were not open to that group of people before. There is a natural influx of people but many of them are bouncing out because they're not finding that group of people, they don't have a shared enough set of values that the people that are new are coming in and finding job opportunities, finding promotions, finding leadership positions. And so, I know now that my mission over the next 20 years of my career is to create those opportunities for people that have different backgrounds from me and different experiences. The career tracks, the promotions, endorsing and supporting and kind of sponsoring this incoming group of freshmen into our industry that come from different places, different backgrounds, different problems that they care about solving. They want to figure out how to solve the Flint Michigan water crisis instead of delivering socks to people in Silicon Valley, you know? So, I feel like we're at the beginning of a seed change in the value system potentially of our entire industry. But that's going to require training up the next generation of technical leadership. And I felt like the best thing I could do right now is learn to be a better manager, because I really like that job. And it provides the opportunity to find, hire, sponsor, promote and encourage those people to move into their own leadership positions. There are lots of other things that a person, you could be a VC and care about that stuff. You could have lots of different positions and put yourself in a position to do that. You could be a consultancy owner. You know what I mean? There are jobs that you can do that you can accomplish that goal. But it gives me such a sense of direction that when I'm looking for a job, I was looking for a home for that mission rather than just the thing that I felt like doing. Like okay, this job is important to me because I need it to house me and this mission so that I can support my family but have enough emotional overhead to participate in community stuff, but enough ability to lead within an organization, enough influence to actually push that agenda. So that the next generation of people are making better companies. So anyway, all of that came out of my time at Frontside where you and I sat around talking about: how do we build a place that is like a monastery? These were your words. You remember this? We want a monastery for code where people can just focus on becoming better developers. And underneath that though was the sense that this was a place of opportunity for people that might go somewhere else and stagnate as a developer. This will be a place to accelerate them. And so, that kind of spun me out and accelerated me into my mission. So anyway, I just wanted to point out that that was like, with a bullet, is the most important thing that came out for me in my time at Frontside, was that it clarified for me what I was trying to accomplish with the next couple of decades of my career. CHARLES: Wow. Well, that's fantastic. You definitely did a lot of that both here at Frontside and I mean you're continuing to do that. I definitely want to see more public speaking from you. Maybe some [inaudible] perfect. [Inaudible] at EmberConf was actually fantastic. But I mean, you're also able to help people find their mission, too. Like the talks you have at Keep Ruby Weird and even really, the first talk you gave at LoneStarRuby about moving Ember. It's always, how do I adapt what I'm feeling to my overall mission and then relate that back to technology? Man, I just can't wait. I can't wait. When are you going to hit the road again? BRANDON: I think this is the year. I'm going to start thinking about this stuff. I'm looking at the stuff that I wanted to talk about on this podcast and I was like, “Oh no, wait. That's like a dozen podcasts.” Like, no. Absolutely not. Not possible. I will say, I miss so much, this time that I spend with you. I don't want to let it go. I really miss working with you. I really miss having these conversations whenever I want. This has been a very, very special privilege for me to be able to do this with you today. And congratulations on Frontside continuing to thrive and grow and become more of its own entity and more of its own special flavor. And it makes me really happy to see the people coming out of there, that it's still doing its mission of making great software by making great developers. It makes me real happy. CHARLES: Yeah, yeah. Hopefully we can keep on keeping on. I do miss working with you. I miss the conversations that we would have in the kitchen which are basically an extension of this podcast. But I also, man, I really, really, really, really like working with the group of people that are here today. I've just seen them producing just some absolutely amazing things. And honestly, there's a selfish aspect to it, too. I get stimulated. My own thinking and learning is stimulated by the people that I work with. And like I said, the whole side note we had about distributed systems and IoT and just a constant ferment of things. So, I still really, really, really enjoy it. BRANDON: That makes me happy. CHARLES: And I'm really glad that we got to kick it today. BRANDON: Yeah, me too. CHARLES: I thought you were going to say that your 20-year mission was to have your perfect Emacs initialization setup. BRANDON: Oh my gosh. Some of these days, I'm going to figure out RuboCop. CHARLES: Actually, do you want to pair on that? BRANDON: Yeah, let's do that. CHARLES: Alright, everybody. I'm going to sign off. If anyone wants to continue the conversation, obviously you can get in touch with Brandon. He is misspelled @tehviking on Twitter. T-E-H-V-I-K-I-N-G. Always come at him. BRANDON: Don't @ me. CHARLES: [Laughs] BRANDON: I work for a really cool company and if you ask me about it on Twitter, I'll tell you all about it. CHARLES: Awesome. And we of course are Frontside. You can get us on Twitter at @TheFrontside or just drop us a line to contact@frontside.io. And we would love to talk to you more about this podcast and all the wonderful things that we do here, which includes building custom software that you can stake your future on, that's going to be good for the five-year outlook. So with that, goodbye Brandon. Goodbye everybody. And we will see you… BRANDON: Bye Charles. I love you. CHARLES: Me too.
Taras Mankovski: tarasm In this episode, Taras and Charles talk about a project that they work on together: Funcadelic - a Functional Programming and Category Theory for Everyday JavaScript Development. Funcadelic takes the simple idea that a single function can operate on many different data structures (typeclass oriented programming) and brings it to JavaScript. Yes, there are a lot of FP libraries out there, but this one is geared towards unlocking the magical powers of functional programming while always maintaining a tangible and accessible experience for JavaScript developers. Because if you're a JavaScript developer today, you're already using most of the structures in funcadelic! Transcript: CHARLES: Hello everybody and welcome to The Frontside Podcast Episode 99. My name is Charles Lowell, developer here at The Frontside and your podcast host-in-training. And with me today is Mr. Taras Mankovski. Welcome. TARAS: Thank you, Charles. It's a pleasure to be here. CHARLES: Yeah. So, you are ubiquitous in the JavaScript world. You do a lot of stuff with mentoring and you are involved in a bunch of different interesting projects. I think you're one of those developers who's difficult to classify, which is – that's definitely one of my favorite kind of developers. I wanted to have you on the show today because there's been a project that we've been collaborating on. And there have been some interesting ideas to come out of that and solidify through that project, at least in my head. And yeah, I thought we could maybe just talk about that a little bit. TARAS: Yeah, sounds good. It's going to be fun. : The thing that we are going to be talking about is a project called Funcadelic. It's more than really just a library, a JavaScript library on GitHub. It's kind of a different way of thinking about your code. And so, I know for me, where this really became part of my workflow was, when was it? It was about three months ago or something like that? Six months ago? TARAS: Oh, I think yeah, I think it's probably more six months ago. I think it's probably what, two months, I think probably December maybe? CHARLES: Okay. But it's hard now to imagine working without this tool on my workbench. It's been probably the biggest game-changer for me in the last, I don't know, definitely in the last several years. TARAS: Yeah, it's pretty impressive how little, how small of a library can have such a big impact in what we do day-to-day. Because it definitely makes me think differently about how I can solve problems that I solve on a daily basis when I work with React. So, it's been pretty interesting. I think for me, having worked with this library, I think what I'm getting is an understanding of how things work in a way, and a perspective on how React works, in a way that I don't think was available to [inaudible] Funcadelic. The funny thing is it's not a React library, right? It's not designed for React. It's just that… CHARLES: I don't even think that – it helps you think about React, but I don't even think it's the way that the React developers think about React, right? TARAS: Yeah, I don't think so, either. I think a lot of people are on the spectrum of understanding functional programming. And I think a lot of people use, people learn how to use React, but they don't really – I don't think a lot of people have traveled very far. I'm talking about general, majority. There's definitely people who know functional programming really well. And there's a lot of really good libraries in the JavaScript space for doing functional programming in JavaScript. But I don't think the general public, the general people that on a daily basis go into – write a render function and do ‘this.' or like ‘product.map' and then return an array of components. I don't think those people necessarily think about or get the context within which they use this tool. CHARLES: Right. And I think that's actually kind of one of the reasons I think a library like Funcadelic is so important and fills kind of a missing piece in the ecosystem, is because it really is predicated on the idea that programmers use these concepts all the time. They really are, they're foundational. But we only kind of see them out of the corner of our eye, in our peripheral vision, as being like a formal concept, like mapping. And giving a name to that. You know what I mean? Like you do the mapping, but you're not thinking about: how do I generalize over it? And I think that that for me, certainly in my journey with functional programming, I thought that it was mostly about functions. Not to say that it isn't, but that was kind of the end of the story. It's like, keep your functions pure so that the outputs are only dependent on the inputs. And away you go. And understand closures and higher-order functions, functions that return functions or take functions, and that's it. But I really feel that that's only half the story. TARAS: Part of it I think is that for people, even if you look at the kind of content that's available around functional programming, it tends to be – trying to kind of [reach] people into this idea of thinking of map, filter, reduce, kind of operations. And I think that's a place where everybody starts. But I think what happens is that you really are missing – and I think for most people. And it wasn't for me, it wasn't until you wrote the readme for Funcadelic and then I read it – up until that point I didn't really, I was also the same. I didn't know how these things were related to each other. Because there's this history and wealth of conceptual depth that connects all these things together. And these things are well-understood by people who don't – they're probably not writing JavaScript on a daily basis. They might be like Haskell programmers or Lisp programmers or ClojureScript or something like it. In other worlds, not JavaScript world. So there is all this understanding about how functional programming works but I don't think it's really leaked to the general masses of the JavaScript community. CHARLES: Yeah. TARAS: You know? And it wasn't until I started reading the readme – I'm like, “There's so many answered questions that I didn't even know these questions were asked.” You know? CHARLES: Yeah, yeah. Yeah, no. And I think you're absolutely right. It isn't accessible to the general JavaScript community. And part of that is because one person's – like when you read about these things, when I would go read about these kind of higher-order concepts, of basically classifying functions, not just of saying, “Yeah, what is the essence of a map operation? What's the essence of an apply operation?” you know, “What's the essence of concatenation?” things like that, I go read the documentation in Haskell or in Clojure. And first of all, it's hard to distinguish when you're not programming in those day-to-day, am I reading reference documentation or explanatory documentation? But even in the explanatory documentation, they're using what seems like incredibly self-referential and abstract examples. And I don't think that's necessarily a knock against those communities. I think what it is, is what's concrete to one person is abstract to another. And it's like, if you're working with those things, you're working with those sets of analogies, and then you're working with those abstractions every day, then they're concrete to you. In a sense that once it clicks in your mind and your mind kind of accepts it and rationalizes over it, then it moves from being, “Yes, it's an abstraction. But it's a concrete abstraction,” in the sense that you can have a conversation with somebody and use that abstraction as an example, as a counterpoint, and a method triangulate and reveal other abstractions. But if you're talking to somebody for whom those abstractions haven't clicked yet, then it's just, it's opaque. And it's not helpful. And so, I think that one of the things that I realized is like we are using these abstractions, we just don't have names for them. And so, I wanted to give them names and put them in the hands. And the names are weird, but they are really useful. And so yeah, maybe we could talk about some of those right now. Because I think that maybe now's a good time to actually introduce some of those abstractions. So for example, if you don't know what a functor is, it's worthless to talk about a monad, in my opinion. So, that was critical piece of information for me. Because that is like missing in every monad tutorial you ever read. At least, I must have read a thousand monad tutorials. And they kind of glossed over functor or didn't mention it. Whatever. Maybe they did but I wasn't looking. And that needs to be put front and center, that there is a natural sequence to these things. It's like, some of these abstractions are built on other abstractions and you have to – you can't skip to monad. You have to start with functor. Again, I realize that's probably gibberish gobbledy-goop to a lot of people. So really, this is what Funcadelic is about, what this conversation is about, is just saying – talking through it in real-world examples to make those abstractions concrete, so it doesn't feel strange anymore. TARAS: Yeah. It's definitely giving names to things. I think it's really helpful. One thing I really like about Funcadelic is that you're not giving names that you made up. These are names that existed for 50 years. They're historic. And so, when you talk to somebody who is familiar with functional programming and if you say ‘functor', all of a sudden, I feel much smarter. And we're actually referencing the same thing, because we are – because alternatively, you can say something like, “Something that is mappable,” right? Like a functor is essentially describing something that you can map over. CHARLES: Right. That's all it is. TARAS: Right. But you know, having a name for it, it allows you to just describe it exactly as it is. CHARLES: So yeah, there are tons of things that you can map over, right? Most of the time, we think about arrays as something that we can map over. TARAS: One thing I found really interesting in starting to use Funcadelic is that when you start thinking about things as they are like abilities – like you know with an array you can map over an array. It's something we've all been doing for a while – but then something that you end up doing a lot of. When you get familiar with an array, being able to use object map, mapping an object, becomes something you want to do at some point. Most of the time, what happens is that you're like, “Oh, I don't actually – well, I have to write this thing. How do I write this thing?” and then by the time you do this, you're like, “I'm just going to go to Lodash and I'll just get the map thing that will map an object.” At the end of the day, it doesn't quite necessarily feel right because a lot of these libraries – like, Lodash has map but it feels like there is always some kind of a compromise with how these things are implemented. It's not consistent. CHARLES: Right. TARAS: And I don't remember exactly what the problem with Lodash map was. I know for a fact there are, like there's different ways that you can map things. There are different functions available for mapping different things in Lodash. CHARLES: There's ‘map to' and ‘map in' and blah, blah, blah, blah, blah. TARAS: Yeah. All those different variations. But I think it's been really interesting. We've been using Funcadelic on a project we've been working on, on microstates. And just being able to use one function map that is going to map an array or is going to map an object and it's going to do it the same, use the same function, and it's going to do the same thing. CHARLES: Yeah. You have one function that maps over the values. And that's the thing, is you realize you can map over arrays. You can map over objects. You can map over observables. You can map over promises. You can map over trees. You can map – there's literally thousands of things that you can map over. And realizing that all of these fragmented interfaces can be coalesced into a single interface. And so, it really is, I think the biggest thing is like the power of polymorphism for a function. Because that's basically the problem that I think – it's not I want to say basically the problem. I think it is a problem that a lot of the functional programming libraries suffer from, is that the only polymorphism is object polymorphism, which is kind of the native polymorphism in JavaScript. Whereas in systems like Haskell and like Clojure, you can have a function be polymorphic. And so, one function can operate on many different kinds of data, provided it has an interface. So, when we're working in microstates, we use literally one function: map. The same, the actual same function, reference to the same function we use everywhere when we map. And we're just mapping a bunch of different things. So, I think that that's one of the reasons that I prefer Rambda, for example, over Lodash, is because it has a form of polymorphism. Most of the things like maps and lenses and applicatives and stuff, almost everything works on both objects and arrays. That's actually kind of nice. So, Rambda has a basic polymorphism. But I think one of the other things that is really empowering about Funcadelic is that it allows you to make the map function work on any data structure that you happen to come up with. Anything that you want. You can make it mappable and map will work on it. TARAS: Yeah. I think for people, it's probably quite abstract, what that actually looks like. I think one thing that's interesting is that – so for listeners, the way that would look is you have a class, you have an ES6 class, which gives name to a certain piece of data that you have in your application. And then what you can do with Funcadelic, you can then say, “This is how you would implement – if you were to map this kind of an object, if you had an instance of this object, if you wanted to map that object, you can specify: what is the function you would use?” So, even though you would use the same – you would import map from Funcadelic and you would use that map to map whatever that object is and whatever type it is, but there's a way for you to say, “For instances of this type, I would like to use this specific implementation of a map to map that object,” but use one function to do it. So, it's going to use one function that you call. But you can specify, under the hood it can specify how the actual, what is actually used to map that instance. CHARLES: Right. And then that's nice, because then you can – anything that uses a mappable object, there's a couple of reasons that that's nice. Any time you can have some sort of higher-order mechanism that just requires that something be mappable, that it be a functor. And then it's really nice because then you can have higher-order operations that they just need something that's mappable. But you don't have to use that one shot of actually having a map function as a property of the object. You can actually, you can kind of define wrapper classes or whatever, that then introduce a unique way that this object can be mapped. So, you can have the same structure and map it three different ways. Whereas you're kind of constrained by that using normal OO inheritance because you have to have a map property on your object. TARAS: Yeah. There's something else actually, when you start thinking about this. For me personally, I think the first step when we start working with objects, having mappable objects, it was the first thing that was really helpful. But then I think really quickly, right after that, I think my second thing that I started using and I think is probably my favorite feature now, is append. I think it's actually – yeah, so append is an implementation of a semigroup but I think it's simpler. A simplest metaphor would be something like object assign. So, object assign is an implementation of a semigroup, except that object assign mutates the object that you pass into it. Right? CHARLES: Right. TARAS: It doesn't create a new object for you. CHARLES: Right. So again, getting to this idea that there are some universal operations. Like there's this idea that you can take two things – I don't want to say object because that's an actual concrete type in JavaScript – two things and you can smoosh them together. And I think there was actually – wasn't there literally a big controversy about this? TARAS: Yes. CHARLES: About like, array smoosh? I think this is nice – smooshing, right? But you can smoosh things together. And with addition, I can smoosh two integers together or two numbers together and get a third number, right? Or with objects, I can take two objects and smoosh them together by merging their keys. Or I can take two strings and I can smoosh them together and I can end up with a string that's concatenated. Or I can take two arrays and smoosh them together and I've got now an array that's been concatenated. So, what's interesting is these are all very, very different types that I've just described. And yet, there is some fundamental operation about them. And I think this is actually something that's bad about JavaScript is there's five different names for all those operations that we talked about, but it's really one unifying concept. And that might seem like a small thing, but when you have five different interfaces for one fundamental construct, that leads to fragmentation and you can't treat that data uniformly. And it ends up like, paper cuts, paper cuts everywhere. Whereas if you can unify all of these into a small, one thing, which is like we can append two objects, and then we're going to have an implementation of append for array. And behind the scenes it's going to call smoosh. And we've got a universal – we've got an implementation of append for object, which is going to assign the keys in an immutable way and return a new object. Or we're going to have an implementation of append for string which is going to concatenate the strings. You might even, I don't know a hardcore FP nerd would have to probably correct me because this is just totally conjecture, although I know it's probably a solved problem – is maybe you could say we append two functions together. And that returns a function which is composed, right? That might be a way that you could say – what would it look like? Is function a semigroup? Can we append two functions together? And maybe you end up with like a pipe or a function composition or something like that. But I think that highlights, when you have these universal interfaces, because literally I feel like most of the stuff that we have been working with, there's literally five, there's like five interfaces. And everything is one of these five interfaces. And it kind of flips you on your head, because the classic programming wisdom that I have certainly have espoused for at least the last 10 years is that you don't want to race to find abstractions. That's dangerous. Because you can get locked into the wrong abstraction, right? Wait. Let the abstraction emerge. And I'm a lot less bullish on that concept now that I've discovered these things, because that wisdom is cultivated in a world where there are [billions] of abstractions, if you're giving unique names to everything and the combinations between them. But if you are coming up in a world where there's five basic abstractions, then it actually pays off to ask the question, “Is this thing a functor? Is it a semigroup? Is it a monad? What would that look like?” It's a nice thought experiment that doesn't require that much investment. You can think about it for a couple of minutes. And usually, you can come up and say, “Yes, yes. It totally is,” and I can start using it. And now I've introduced this really powerful abstraction. Or you say like, “No, no it's not.” And then that information is just as valuable. And so, it's very low-cost to experiment with abstractions. And so, I kind of think of it as – I know I'm on a little bit of a rant here but this has kind of been a major revelation for me – is that when you have very few abstractions which you compose in myriad ways versus having a whole bunch of abstractions that can't be composed very much, the cost for experimenting with abstraction and making the wrong abstraction is several orders of magnitude lower. And so, you don't have to be as cautious. And you can actually use trying on abstractions as a tool, rather than a very, very high risky undertaking. And just to kind of close that thought out, I think that – I don't know if anybody else but me remembers the world before we decided we were going to make all of our web services RESTful – like when people first started building all these web services, we were just going crazy with the endpoints. And there was no rhyme or reason. Kind of weird arbitrary levels of nesting. Sometimes, you'd throw in an ID as a query param, sometimes you'd throw it in as part of the path. And then I definitely remember, it was probably around, I don't know, 2010 for me where I listened to a podcast where James Edward Gray was talking about S3 and ‘Was it a RESTful interface?' and the O'Reilly book. And it really clicked for me. And realizing that if you constrain yourself to thinking about your API at least as these fundamental operation of manipulating resources, and you were constrained to four verbs and everything, you want to have ID-based URLs and resources and as flat as possible, those constraints actually are very enabling for consumers of your API and for actually authoring an API. And I think it's the same principle at work here. Anyway, so I'll end that rant. [Laughs] TARAS: Yeah. I think it's, I think people could probably, for those who haven't been in programming as long as Charles has been, it's probably easier – Charles I think people could probably relate to what's happening with components now, I'd imagine. Because having components essentially look the same across every framework, they all have props and they all render, return some DOM, or some variation of that. But it's kind of the same thing. You take some data and then you return something that is going to become DOM. And I think having that as a rule for what a component is, you can then make really complicated applications using these fundamental building blocks. And then you don't have to – there's not really much thinking on, “Well, how am I? What interface is this component going to have?” Okay, well you know it's going to accept props. And you know it's going to render some DOM when you actually render it, right? CHARLES: Right. TARAS: That simplicity I think is really helpful. And I think it's one of the things that – I think one of the things about Funcadelic that I really like is there's kind of a really small set of rules that are really helpful. And these rules are actually, they make it predictable. Because one thing that I find really challenging with using Lodash or using Rambda is that because there are so many functions, it's difficult to know what is actually going to happen when I do something. So, a good example would be like if you use omit from Lodash, Lodash omit will then, it will actually – one thing you can do is you can materialize your getters. So, if you have getters in your objects, those getters can become values on the new object that's created. So, that's one thing that could happen. Or if you use omit, your symbol, if you have values… CHARLES: What does omit do, by the way? I'm actually… TARAS: Omit is a pretty popular way to exclude functions. Basically like a filter equivalent for an object. It's usually used to remove some props that are coming into a component. But it can do some – it can actually change the type of the object. One thing you know for sure is if you use something like omit or if you use assign, if you have an instance of something, guaranteed, working with that object in a mutable way is going to cause some really strange things. I think with omit, if you were to have an instance, it would definitely not give you an instance of the same type, like an object of the same type. It will give you just a regular [inaudible]. And there is no real way – you could create an instance. I don't know what assign would do. I'm guessing that it would just take an instance and would put things on it. But it's really not – I think this kind of ambiguity doesn't work very well when you're trying to build something, when you really just need to know exactly how your tool behaves. I think that's one thing that with Funcadelic, because it's such a small API and because you know for a fact that the library is designed to be immutable and it's designed to preserve type information, then you know that if you use one of the operations, you will get most likely the same kind of object. And you're not – well, you will the same kind of object and it's not going to mutate that object. And so, there are some of these guarantees that are actually really helpful and [inaudible] is liberating, I think. CHARLES: Right. TARAS: Especially when you're trying to do more challenging things, not trivial, just copy some properties from one object to another. But when you're actually doing more sophisticated things, in those use cases, having these kind of rules is extremely powerful. CHARLES: Yeah. And I've definitely resisted and I think will continue to resist expanding the API surface area that much. Because it is, I think there's only five or six functions in there. But what you get for those functions is extremely powerful and extremely predictable. I think it might help to give a concrete example. Like when you were talking about object assignment. Like if you have a car class in your application and you want to do an immutable object assign, well the kind of stereotypical way to do that or the typical way to do that is you assign. You create an empty object and then you assign one car to it and then you assign the next car to it. And now you've merged these two car things, right? But then the problem is, you're going to get an object out of that, not a car. It's just going to be a vanilla object. Now, it's going to have all the properties of both, but it's not going to be a car. And that could be a problem if you've got any custom methods on the prototype, any computed properties on the prototype. It's going to be a problem. Whereas Funcadelic, you can append two cars together and you're going to know that it's going to have both of the properties. You're going to know that it's going to be of type car. And you don't have to worry about running the constructor or anything like that. It's just going to have – the properties are going to be carried over properly. If you're using a library like Lodash or Rambda or something that doesn't account for type, because in order to append two things or map something, the implementation actually lives with the type, not with the function. The function is polymorphic, but the implementation lives with the type. You can then actually, you can always return the proper type. Because it's ultimately – like if your map operation or your filter operation or your what have you operation doesn't take type into account, then there's no way to actually preserve type. But because we delegate in Funcadelic, it's core to the concept. And so, it's actually a very trivial thing to do, which is why you get that repeatability. TARAS: One of my favorite things about append and semigroup implementation for object is that you can overload getters on an existing instance. So, what you get is you get a new instance with the getter that you passed to append applied to that instance. And this is kind of trippy but it's actually really powerful because – so, let's say you have an instance of an object and that instance has some getters on it. And those getters use some properties of this object to compute their value. And so, when you want – if I need to create a copy of that object in such a way that the getters still continue to work properly but I need to override how one specific getter works for one specific instance, one specific scenario, one specific use case in my code, then I can just use append. So, the first argument is the original instance, second argument is an object that has the getters that I want that I want to overload the getters on the original object. And then append will squish those things together, smoosh those things together, and give me a new instance that has the getters that I passed in, in the second argument. And all the same things that the first object had. And that object will work. This new object will be a fully-functional object just like the original object that I still have a copy of, that I can use. But this is really interesting because one thing that I'm finding with having these kind of tools in my toolset is that I've had features that I needed to implement on a project. And the people that I work with are really technical. So, they know where problems are going to be. And so, the would write requirements for how something – for a feature that needs to be implemented. And knowing the problems, they're like, “Oh, by the way. You're going to have a problem in this area when implementing this kind of specific functionality.” And for me, I'm like, having this toolset, I'm like, “I don't see it as a problem at all.” It's so easy for me. Because I know that if I need to implement – so if I have something that has expected behavior, but I need to create something that behaves very similarly but slightly different in one particular use case, I can always just copy that object and then overload its behavior. And it's still going to be a fully-functional object. And I think that alone is just not something that you can do usually. It's not something that's available. CHARLES: Yeah. It's a technique that I think was discovered. Maybe it's not original to us. But it's just, the tools enabled it in the sense that it's like having a flashlight in a dark room. It's like, that technique was always there. It's just when it becomes so concise that it's so easy to just append one more computed property to a thing, then you just wouldn't have thought to do it otherwise. TARAS: What's interesting about this too, for me, is that – and this goes back to the context conversation that we had earlier – is that I think React brought into our lives functional programming kind of, in a big way. Because part of programming React applications is working with functional components and working with functional concepts. And a lot of the things, like Redux, a lot of these things are powered by functional programming. And they work together. They compose well together because they're all from the same paradigm. But the problem is that there are all these concepts developed together over time. And they've been tested together and they've been formulated together in languages like Haskell. But the ideas that make all of these stuff work really well together, they haven't really become available in the JavaScript community. And it feels like, it's like we're all using a language that we don't fully understand. And it's like we're all – a lot of people in the JavaScript world who works, when it comes to functional programming, it's kind of like having English as a second language. It's like, you can use the words but you don't understand the humor. And it's kind of like, you can't make a joke. It's kind of like that. You can't really express yourself correctly when you don't have full grasp of the language. And I feel like how we use functional programming in JavaScript is a little bit like that. And by starting to bring these ideas, moving the wealth of knowledge from the source into the realm where we actually need to use it now, we can actually start to take advantage – leverage all these insights that actually enables all of this. It's not like – so, the idea of how to write React and think in a reactive way or think in a functional way, those things are not just owned by the React core team or they're not owned by an elite group of developers who really understand how functional programming works. It's just available to everyone. And all you need to do is just learn some of these concepts that glue all these ideas together that are fundamental pieces of how functional programming works. CHARLES: Right. That's a great point. And it points back to kind of the original reason that I wrote Funcadelic and then started and then continued to work on it with you, is that it really was – it was actually meant as a – it started out as an educational exercise. What would these things look like if they were translated into JavaScript? And it turned out that it rapidly became core to my workflow and way of thinking. And so, it really is, there are weird names to these universal concepts. It is true. It is a foreign language. I really like that analogy. But foreign languages sound weird, and when people are talking in a foreign language, you can feel excluded. And the really, the reason that we wrote Funcadelic and the reason it's there is to make them accessible, these things accessible to you. So that those abstract foreign words can over time turn into concrete concepts that you're completely comfortable with, just like any word in any language. At some point, you approached it having no idea what that sound represented. And so, it really is trying to – the emphasis there is not to noodle about and dwell on the names of the concepts but to take the real things that you are actually doing, give them names and formalize them, to enable you to participate in this new functional world that you're describing. Because I love that sentiment. It does not belong to the React core team. It doesn't belong to an elite set of developers on this project or that project. It literally is a universal tool that is 100% achievable. And people don't even realize how close – if they've been working with JavaScript for a couple of years, how close they actually are. TARAS: Yeah. I think they're really – for most people, if they were to read the readme and then – well, I think one of the problems, it kind of works with the language metaphor, is that you need an immersion, right? I think one of the reasons why Funcadelic really stuck and functors and semigroups and [filterable] and all of these things really stuck for me, and I'm thinking about how using monads and monadic operations or applicatives and all that stuff – the reason why it all stuck for me is because I've been able to talk to you about it. And I think for people, finding a network that will allow them to practice immersively, to think about functional programming, not just occasionally – I mean, you could do it occasionally as well, it just takes much longer. But once you really, once you have a few conversations where you try to dissect a problem in a functional way and you think about what these pieces are made of, it becomes very natural, very quickly. CHARLES: Yeah. I think that's actually a really great point. And the thing is, you can immerse yourself incrementally. So you can just say, “You know what? I'm just going to start using append.” Anywhere that I would concat two strings or I concat two arrays or I would do object assign – screw that. You can even just say, “Instead of using object.assign, I'm going to use append,” and start there. Or to say, “I'm just going to start with mapping.” I think also the thing that's nice about it too, is the buy-in can be incremental. But you're right. You do need immersion. You do need practice. You need to actually use, you need to use the functions and you need to be able to use them one at a time, so that your mind can close over them. So then, you can kind of move onto the next one. TARAS: Yeah. CHARLES: So, that might be one way, is to say – because you know, I don't think I really understood. I was already using append and map ubiquitously before I really understand applicative/apply. So, you don't have to grok it all at once. You can definitely bite it in chunks. And the best way to do that is to start with one concept and really just attack it mercilessly. And then also understand that there's a natural sequence there. TARAS: I would add a little bit of a caveat to that. I think there's a thing about using – doing something for learning purposes and there's another thing about shipping things. What's interesting with Funcadelic and what's interesting about a lot of these ideas from functional programming is that I think they give you benefits that you might have not previously thought. Like for example, if you're going to concat two strings together, doing it with append is probably the most robust way of doing it, relative to just being able to use [inaudible]. CHARLES: Yeah, that's true. You wouldn't want to use a string [inaudible], wouldn't you? TARAS: Yeah. But there are areas when using – there are times when things are just not possible otherwise. Like for example, if you wanted to treat an instance of a class in an immutable way, this is simply not possible in any way. So, if you're going to say, “I'm going to work with a bunch of these instances of ES6. And I want to keep them as instances, because they have certain behaviors that I want to have. I want to have a getter, or I want to have a method on it. And I want to keep these things fully full instances, not broken, not be turned into objects. I want them to be normal instances but I want to work with them immutably. When I need to make a change, I want to get a new object and not modify that object.” So, if you set yourself that goal and you say, “This is what I'm going to do,” then you really are not left with very many options. You only really have, you have to use append from Funcadelic. And because alternatively, you're going to implement something yourself. You might as well just use append. And [I think] that's a good place. I think if you're starting to, if you need to make something lazy, if you need to delay an execution of something – so, instead of pushing that execution, instead of using object assign and then computing everything ahead of time, you can use append. And you can create a getter and you can delay the computation of that value until the point when the user actually reaches for that value. If you want to start doing that kind of stuff, you really are not left with very many options. And append is the way to do it. But that's the thing, is when you start to set these kinds of standards for yourself, you level up in a way that is very significant. I think it's like a natural progression of learning. You start off learning and anything goes, as long as you can make this website work, it's like, “I'm happy.” And then over time you get better and better at it. And then when you get good at building applications, your next step might be like, “What if I was stricter about it? What if I could actually – what would that open up for me? What would that allow me to do?” I personally think about it that way. CHARLES: Yeah. I think that's a good way to think about it. You mentioned having a network for discussing these concepts and trying to internalize them. Let me first and foremost offer myself as somebody. If this is a hill that you are interested in climbing, and I think it is a very worthwhile hill to climb because of the perspective that you will gain from its summit, please reach out to me. You can contact me at cowboyd on Twitter or cowboyd@frontside.io. I'd be happy to discuss these kinds of things, because I think that these tools are just incredibly powerful and will improve you. So, if folks want to get in touch with you, Taras, where would that be? TARAS: I'm tarasm@gmail.com and tarasm on Twitter. CHARLES: Alright. Well, thank you everybody for listening. And as always, if you want to get in contact with us at Frontside, you can just email us at contact@frontside.io or give us a shout on Twitter at @TheFrontside. Thanks everybody. We'll see you all next time for episode 100.
This Frontsider panel episode explores what virtues go into making quality software, such as having tests, making sure software is performant and accessible, and why you should try to avoid technical debt. Transcript: CHARLES: Hello everybody and welcome to The Frontside Podcast Episode 98. My name is Charles Lowell, developer here at The Frontside and your podcast host-in-training. With me today, we're going to have a round table, a Frontside round table. With me today is Elrick. ELRICK: Hey. CHARLES: Joe. JOE: How you doing? CHARLES: And of course, Will. WIL: Hello, hello. CHARLES: Welcome, y'all. We're going to be talking today about some of the things that we do around here, aside from trimming the shrubs and making coffee and snacking on Altoids. Like, way too many of them. Yeah. I was thinking we could talk a little bit about software qualities of relative things, like this software has these qualities. And I think that that kind of lofty goal of software quality is comprised of having a bunch of little qualities. The quality of having fewer bugs or the quality of having these things. And so, talking about all these things that we do and kind of what we do to make sure that we continue to do them. Or the ways that we can ensure that our software has these things. So yeah, we can just start really anywhere. WIL: Yeah. So, one core thing is obviously tests. CHARLES: That kind of falls under we want to have – really, there's two qualities there that we want, right? Is we want to have… WIL: Maintainable software. CHARLES: We want it to be maintainable. We want it to be resilient to change. And we want it to work properly, right? Yeah, so we put tests in place to make sure that that happens. JOE: Tests also inform design in a really positive way. A lot of the time, anyway. WIL: Another thing that we like to include in our apps is responsiveness. CHARLES: Yeah. And just making sure that you have – that it works on a multiplicity of devices, right? WIL: Yeah. And not just the devices, but browsers as well. CHARLES: Yeah. And it turns out it's actually really hard to do that after the fact. WIL: Right. JOE: Yeah. CHARLES: Making sure that lots of browsers, lots of devices. Because yeah, sometimes you have some weird screen width that is on some weird device, and making sure that that works. I guess there's some overlap with testing there, too, isn't it, right? Like you want to be running your tests on those devices at those resolutions to make sure that they're going to work. This is something that we aspire to but I don't think we're quite there yet. It was making sure that our applications are accessible. WIL: Yeah. JOE: I'm very excited to learn more about this as we get into this, yeah. CHARLES: Right, right. And asking the question, how is it that we actually can ensure our applications are accessible? We have very paved roads for making sure that our applications are resilient to change and that they have low bug rates and that they're well-designed via testing. But what is the analog of testing for accessibility? What's the way that you can put those guardrails in for accessibility? I have no idea. And that's an ongoing conversation here at Frontside. JOE: So, I guess I'm curious as to what technologies are actually involved in accessing a web application in – would it be reasonable to say a non-traditional way? I know there's such things as screen readers, but is that all we're talking about? Or what is the ecosystem that we have to consider supporting? CHARLES: I'm certainly not an expert on this. We'd have to get Rob in here to chew our ears off this. JOE: Yes. CHARLES: But from what I've picked up from him and from our conventions with Marcy Sutton and some other folks that we've had on the podcast, it's a big umbrella. So, it's anyone using an application in a non-traditional way. So, whether that can have to do with limited vision, hearing, movement, range of movement, cognitive ability, it's a gigantic whale of a domain. WIL: Yeah. The topic of accessibility can definitely be several podcasts on its own. CHARLES: Yeah. One thing that we've talked about is it would be great if you could drive your test suite through a screen reader or something like that. What would that even look like? There are a couple of open source ones out there, but they're Windows-only. I think it was NVDA was the big one. And then you have a screen reader that then drives the applications in your operating system, so it's going to vary per operating system. So, making sure that it's accessible on Windows, at least as I understand it, is very different from making sure that it's accessible on a Mac. JOE: Yeah, it's like a whole other layer. And it's like BrowserStack outside of the browser. CHARLES: Right. WIL: There are things that you can do from the beginning that will make it easier when you get to that point. It's just like using semantic HTML, knowing when and how to use proper aria labels. All these things, if you do it from the beginning, it's not as big of a task as bolting it on afterwards. CHARLES: Right. And I think we do have a leg up when it comes to web applications. It's within our power to change. There are cross-platform of those technologies. But as you said, it's important to put them in from the beginning. Because as we've seen, for each one of those categories, you're accumulating debt if you don't address it. So, there's technical debt. But I think that technical debt can [inaudible] into a bunch of different areas. So, there's technical debt in terms of the internal quality of your architecture, the way your software components talk to each other. And I think that that's what people mostly think of when they talk about technical debt. But I think in terms of responsiveness debt, there's a slice of the technical debt pie that has to do with making your application responsive. And so, if you don't address making your application responsive, you're accumulating debt and you might not know it. And if you're not making your application accessible, then from the beginning you're accumulating debt. So that if you have to go and try and figure out your accessibility story six months, a year, two years, you might actually uncover and say, “Whoops. I've been swiping the accessibility credit card. And holy crap, with all this. All my fines and penalties and compounded interest. Now I'm accessibility bankrupt.” And that can be scary, right? WIL: Yeah. And a lot of people don't realize with all this debt after the fact is they think they're going in and adding things like responsiveness and accessibility and tests. But really, you're also taking away previous work that's already there, things that need to be refactored. If you put these things off, you're not just adding a few hours of time. You're inflating your time exponentially. CHARLES: Right. Right, exactly. It can be intimidating but I think it's also empowering, because technical debt is like a scary subject. But if you're like, “Oh, we can actually slice our technical debt into a bunch of different categories and address them individually,” just knowing that this is an area where debt can accumulate, that's half the battle. Because the worst thing is debt you don't even see. ELRICK: Yeah. WIL: I mean, [inaudible] is big. That's a big part of accessibility, that, is most people don't think of accessibility. So, that is a huge debt that a lot of companies don't see. JOE: What about something like internationalization where I feel like I've never been in an application where that wasn't punted on to some degree. That's kind of a well-known problem, but it still takes a back burner. Do you think that if accessibility had more exposure as a concern, would it actually get the attention it deserves or is it kind of destined to, “Oh, we'll get to those yaml files later. We'll send those off for translation later,” that type of thing. ELRICK: I don't know. Sometimes I feel as though people feel as though they're trading speed away when they're building applications when they go to implement these things. Like, “Okay, well we're not really going to touch on these right now because that's going to slow us down from pushing out features.” Which is not really true. Because if you don't settle on these things early, you're not really building a solid foundation for your application in the long haul. So, I think people are like, “Oh, we'll just do it later.” CHARLES: Right. ELRICK: And, “We'll just ship features now.” CHARLES: Right. I think that's exactly right. It has this kind of secondary effect where not only do you develop the debt but you develop a culture of accumulating debt, right? Like when it comes to people getting a hold of their finances, the first thing that they have to change is they have to change their spending habits. And that can be the hardest thing. It's not just balancing the equation. It's like saying, “I need to readjust my thinking about this.” ELRICK: Yup. CHARLES: So that I'm not consistently put in this situation again. JOE: So, there's an operative word there, right, in personal finance in that usually if a company is addressing technical debt especially down the road, something that they've punted on for a while, it's far from personal. There's a board of directors or there's a special interest group involved. There's people who want features that are putting money into it. There's a lot of pressure as the company grows and more people are involved. Priorities are more likely to be lost, I guess. CHARLES: Are you saying it can be hard when your culture is spread over that many people, it can be hard to shift? JOE: Absolutely, yeah. And I guess to keep with the dash-first thing, ideally were we starting a company, we would want to start a culture for this company. A culture that recognizes the vulnerability that we all have to technical debt as applications grow. We want that upfront. But the reality is, you know, startups are eager to get things out. Companies that have been around for a long time have high-paying clients that they depend on that want certain things. And yeah, I guess I'm just saying that it has to come in from the beginning. CHARLES: Yeah. And I think that – I don't want to completely disparage technical debt entirely, because technical debt like actual debt, like financial debt, is a powerful tool that you can wield. But it's also, it's like a table saw. You can also easily slice your finger off. It doesn't mean that it's not a useful tool, right? If anyone's bought a house, it's really great that you can borrow money to buy a house. It's great that businesses can borrow money and get small business loans to get bootstrapped. And that benefits us all to have that community. I don't think that – yeah, startups definitely, they need to have technical debt as a tool that's available to them. But they just need to understand the consequences of it and be able to get a hold on it. JOE: That's a super interesting take. I never considered it that way before. CHARLES: Yeah. It's definitely not my take. I actually think the person who coined the term ‘technical debt', that was the original idea. But then people realized that technical debt can also get way out of hand. WIL: It's just like real debt. If you're not paying down a certain amount every so often, it's going to keep growing. CHARLES: Yup. You're going to have to declare bankruptcy at some point and throw out the piece of software if you don't pay a down. And that's going to be more expensive. ELRICK: Yup. That's definitely true. So, I have a question. And we see this all the time repeating itself at various companies, whether it's a startup, a large company, where they put off testing and mobile-first, user-first, accessibility-first. Like all the firsts, they just toss it to the side. Why do you all think that that happens so frequently? CHARLES: I think it comes into people not understanding that if you don't address it from the start, it won't happen naturally. There is a prime motivator that has to happen. If you don't imbue something with those qualities when it's tiny, when it's a tiny seed, a tiny crystal, you're going to have to drill through layers and layers and layers of core to put it at the crystal to begin with. I like to think of software as kind of like a tree. And we eat the fruit of the tree, and that's the features that users use. And we can tell that a fruit is delicious merely by placing it in our mouths. And we can tell what fruit is bad. But we can't really look at the fruit itself to say what caused this fruit to be good, what caused this fruit to be bad. We have to look at the tree. And I think that that's what people miss when they're developing software, is that what you really want to do is you want to build a tree that builds good fruit. You can't just take the fruit off the vine and say like, “Hey, I've got this peach but it doesn't have enough sweetness. So, I'm going to take a syringe and I'm going to inject glucose around it and make it less tart.” You say, “I want a sweet fruit,” right? ELRICK: Yeah. JOE: You could probably actually do that. CHARLES: You could. And that might be a strategy. And we see a lot of software that has those qualities of, “Oh, we're going to make this accessible,” or, “We're going to try and make this beautiful.” I happen to think that pigs are adorable animals and look great in lipstick. But that [laughs]… you could put lipstick on a pig but people can tell. And you can say, “Oh, this peach needs to have softer fruit,” and you can whack it with a mallet to actually make the meat more tender. But people are going to be able to tell. So, what you really need to do is you need to care for the peach tree rather than worry so much about the fruit. Because if you have a healthy tree, then you will have healthy fruit, right? ELRICK: Yeah. So, you want to plant good seeds. JOE: Yeah. WIL: Back to you question, Elrick, about what motivates startups and other companies to put off these things. I think the biggest thing is just time and money. They have this misconception where they're saving a little time and saving a little money now just to add it back later. But in reality, it's going to cost them tenfold time and money for adding it later, versus just spending that little bit of time and money and all that to begin with. CHARLES: That's true. JOE: It could also boil down, as far as just personal intimidation. Not so much like a business side of a thing but maybe just, think of all of the things that you listed, Elrick. It was almost a dozen dash-firsts in there. If you're sitting down at a startup that you started with three friends and just approaching these things for the first time, that's a lot to tack on right upfront. It's intimidating. CHARLES: It is intimidating. I think my message to those people is I've felt intimidated by that. I think my message to those people is like, the nice thing about it is if you attack those, if you tack all of those things from the get go, the features will take care of themselves and feel more effortless as you go on. You say like, “Oh, well actually, I don't worry about a high rate of bugs.” I want to say recidivism, but that's not the right word. A high rate of return, not on money but on – or high rate of bouncing your users. You don't want that. And if you bake that in from the beginning, parts of the software development cycle that were stressful before just aren't stressful anymore. So, if we say, “We want to have a system that is easily maintainable, well let's put that in from the very beginning.” We say that a lot. We deploy to production on day one. But what that means is, we say we have this value that we want the system to be easily maintainable. And so, we're going to do it from day one. That means that we actually – it's not something that we worry about so much on down the road. Whereas that used to be very stressful. I don't know. I remember when I started my career, there were these long release cycles where every six months, you'd release software. And the last month was just absolutely terrible as you try to stand this thing up and get it into production and then realize it's not monitored. There's no one checking the health of this thing. So, it's pissing off users at one in the morning. And… WIL: Beepers. CHARLES: What's that? WIL: Beepers. CHARLES: That's actually a great – there's a story there. The one time I got a beeper, I went canoeing in the canals of London and I tipped over my canoe and I dropped both my cellphone and the beeper that they've given me. ELRICK: What? CHARLES: I never got put on pager duty again. [Laughter] JOE: I'm going to use that next time [inaudible] with an on-call position. That's a good move. CHARLES: I remember, I definitely remember how sour my manager's face was when I turned [inaudible] the cellphone that was like, dripping with water. JOE: He was eating bad fruit, probably. CHARLES: Yeah. [Laughs] So, the other thing is we like to build beautiful applications, right? So, you have to – that match the user experience. You have to spend that time on design and beauty upfront. You will not have a beautiful application after the fact. You just need to bake it in. ELRICK: And accessible design. CHARLES: Exactly. ELRICK: Don't forget that one. CHARLES: Don't forget that, right? A responsive design. WIL: Yeah, accessibility-first in design. Yeah, responsive and all that starts in design phase, yeah. CHARLES: Yeah, all that, right? So, you want a great experience. You want an accessible experience. You want a responsive experience. You want a quality experience. You want a performant experience. That's another quality that you say. Like, “We're going to make sure that this is performant.” If you want that – and that's something that we're not always great about, right? We don't actually put in benchmarks for our software from the get go. But maybe we should. But there's perhaps a hidden cost there that we might be actually accumulating performance debt that we don't even know about. JOE: That's true. ELRICK: Interesting. JOE: So, things that pop up that are new. Like, accessibility wasn't probably always a thing in computing. Internationalization probably wasn't always a concern. Beautiful certainly wasn't a concern if you look on Wayback machine. You will see that to be true, right? [Laughter] JOE: So, all code is tech debt, I would argue. Or at least has the potential to be. And yeah, as the ecosystem as a whole evolves, being responsive to that, having plasticity in that respect, sort of like meta-first. CHARLES: Right. JOE: That could be the real challenge. WIL: Yeah, Charles is mentioning all these experience things. And so, I was thinking X-first is simply experience-first. You want you users to experience a certain quality of your app. That experience needs to start in the conception phase. CHARLES: Yeah. ELRICK: That's true. And even your developers coming in, developer experience. JOE: Yeah. CHARLES: Right. And I think the core of that X-first, that experience-first, is you need to pick which experiences. Because you can't have everything. JOE: Right, yeah. CHARLES: One, there is going to be too much. You have to say, “I'm going to sacrifice on knowing that this is a performance thing. I'm not going to include that in the core DNA of my application.” And there's just going to be things that you don't know about yet that are just unsolvable problems or that don't necessarily work. And you can say, “You know what? Hypothetically, I'm not going to make this an accessible – I'm not going to focus on accessibility.” But then you need to own that. And you need to know that you're accumulating a huge amount of debt around that. And then I think that is a particularly bad trade-off because someone's always going to come along and you're going to have to know that your application is accessible. I think once we clamp down on that, that's going to be something that we have a strategy for and we include at the beginning on every single application, right? ELRICK: Yeah. CHARLES: But I think you need to have, almost like holding the cards in your hand, say, “These are the cards. These are the X's that I'm going to have in my hand. And they are going to be core to my app.” And they're going to be part of the DNA of that tree. So that I know that the fruit is then going to have those qualities. JOE: And then you as an engineer, that goes through an iterative process as well. Just starting out, you have no idea what that DNA should look like. And short of learning from people who are wiser than you who are around you, and reading blog posts and whatnot, really the only way to know the pain of strong-arming internationalization for instance into a 15-year-old Perl application, is to go through it. And then, you know, future trees will not have this DNA. CHARLES: Right. Right. And that's the other thing. Is if you are going to include, if you are going to try and splice something into the DNA, there's a lot of work. And you just need to go for it. You acknowledge that it's going to be a lot of work. And you need to, you just need to own it and go for it. And pay that expense of actually getting it deep, deep, deep into your application's core values. So that then, you don't have to worry about it anymore. Otherwise, you're going to be paying – you're just basically signing up for a lifetime of debt. Right? WIL: Yeah. And then to make the debt analogy even more, it's like people don't understand the total debt. The end debt. People get a $30,000 loan with a 4% interest and they think they're paying back that $30,000 loan. But really, they're paying back $36,400 after all the amortization of their interest. The debt is higher than you can see, always. CHARLES: Right. WIL: And it's true in tech debt, too. React is the new hot thing now, but in 10 years we're going to be on React debt that we're migrating away from. JOE: I hope so. [Laughter] CHARLES: Maybe less, I think less than 10. WIL: Yeah. The debt is always there. And people don't realize how much they have to pay on top of what's visible. JOE: Yeah. It's an invisible vig. CHARLES: What's a vig? JOE: It's interest, in the mafia. CHARLES: Oh. JOE: Sorry. Yeah. CHARLES: I forgot you're Italian. JOE: Yeah. ELRICK: So, for people that are listening, they might be in a situation where they need to advocate to the powers that may be these X-first values. What do you all think that some of the approaches that they should take to say to whomever it is that, “We need to do this first”? Because there's times where you might say, “Hey, we need to do this first,” and people just look and say, “Oh, maybe not.” Then you need to push back on that. CHARLES: In my experience, I find that the tech debt argument is a good one. Because I think it can be, it's both limiting and empowering. Because sometimes it really is the right call to pull out your credit card and put something on it. If you need to buy water and you need to buy food and you don't have any other means, man, put it on the credit card. Right? Seriously. Even if you have no idea how you're going to pay it back. Like, whip that sucker out and stick the chip in. And it doesn't matter how much it costs. And so, sometimes that is the right call. But I think draining it of a moral or a value as a human person thing, and approaching it from a business decision and saying, really trying to attach a cost to it. Because then I think if you can drain out the emotion of it, because people really want something. They're striving to go get it and trying, give them tools to think about it rationally. That I think is a good strategy, to just say, let them know that there is a debt that's being paid here or that's being accumulated here. And it's really large. And maybe even say, “Look, if we were to put this off by six months, this might cost not twice as much. It might cost ten or even a hundred times as much.” So, by saving $5,000 now, you might actually be accumulating $50,000 worth of debt. It's [bigger] than you think. But I do like – so, I think that's one important tool. But I think then also the other important tool is to say, “If we are going to attack this, let's drive it home. Let's put it at the core. Let's make this a value that we hold so that the tree can take care of the fruit itself.” So, if we say that we're going to put in accessibility – because not all projects are greenfields. JOE: Absolutely not. CHARLES: So, what's the message to them? Sorry. You're just SOL. I think if you're a year into a project, two years into a project, and you realize, “Oh no. We need to do internationalization,” recognize that that might be something that's – that's a pillar of your architecture. Or, “We're going to make this application accessible,” don't half-ass it. WIL: Weave it in. CHARLES: Say, “We're going to transform this. We're not going to add accessibility. We're going to transform what we have into an accessible application.” Or, “We're going to transform what we have into a beautiful application.” Otherwise… WIL: Yeah [inaudible]. CHARLES: I would say leave it ugly and focus your efforts elsewhere on things where you do have your values straight. Because you're never going to have everything in line. JOE: No. WIL: Treat software like immutably. You don't add something to it. When you want to add accessibility, you're creating a whole new accessible app. ELRICK: Ooh. That's deep. CHARLES: Yeah. JOE: So, having seen – I don't know. I think it was very apt, looking at it as a business decision. I've seen it go the other way. Because at least among engineers and people on the technical side of it, this can become a very strong moral issue that people feel very strongly about. CHARLES: Because we have to live with the consequences quite honestly, right? JOE: Exactly. And that's a hard thing to translate to say an executive board that may be three levels abstracted away from you and is making those decisions. I've seen people attack or approach this I guess with that emotion built in, with the, “This is the right way to do it. Everybody else is doing it wrong.” It gets nowhere, basically. What needs to happen I think, so you talk about having this beautiful tree. But that also requires beautiful gardeners. And so, where the moral thing or the interpersonal thing comes in is there needs to be kind of an inclusive and encouraging environment that is fostered among the people tending to the tree. And that's a totally separate thing than selling the business value of it. Those things should be completely divorced. CHARLES: Yeah. It's funny. It's always hard to reconcile those two things, right? Because on one you have, “You have to take care of the raw consumption of material and the output of product.” But then also trying to – so, there's some baseline math that has to happen but making sure that that goal, it doesn't slice people. And can enable them to be happy and feel like they're doing good work. And that the things that they're doing is having meaning. It's probably an insoluble problem that we're going to be dancing around for as long as people are around. If there's one thing that we've come to recognize around here, and we've stated it many different ways from a bunch of different angles through the course of this conversation, and I would say through the course of this podcast, but that is if you want to see something in your software, make sure that you attack it from the get go. ELRICK: Intertwine it in your DNA. CHARLES: Exactly. And then you can actually experience the fruit, rather than trying to always, always trying to jam it and change it and get it into the taste you want after the fact. So, I guess that's it. Thank you so much y'all, for this conversation. I really, really enjoyed it. For those of y'all listening, if you want to continue the conversation, you can get in touch with us we are @TheFrontside on Twitter. Or you can drop us an email. We're contact@frontside.io. So, thanks Elrick. Thanks, Joe. Thanks, Will. JOE: Thank you. WIL: Thank you. ELRICK: Yup. It was great. JOE: It was fruitful. [Chuckles] ELRICK: Frontside-first. CHARLES: And well, we'll see y'all around.
Erich Gamma: @ErichGamme Dirk Baeumer: @dbktw Show Notes 01:11 - The Design Patterns Book 02:45 - The Eclipse Project 09:24 - Language Server Protocol: Overview 15:16 - What can you do with a server that implements the LSP? Incremental usage? 20:12 - Keeping the Tools in Sync and Refactoring Support 24:33 - Keeping it Performant 29:41 - What kind of proliferation of codesmart tools are there that implement the LSP? 34:51 - What are the challenges encountered trying to build abstractions that work for 40 different languages? Resources Visual Studio Code Transcript CHARLES: Hello everybody and welcome to The Frontside Podcast Episode 97. My name is Charles Lowell. I'm a developer here at The Frontside and your podcast host-in-training. And with me today, we have two very special guests. They have been working on technologies that have run very parallel to my entire career as a software developer. And we're going to talk about that. So with us today are Erich Gamma and Dirk Baeumer who are developers on the team developing VS Code, which if you're in the frontend space is taking that area of development by storm. It's just amazing, some of the things they can do. Lots of people are using it every day. Lots of people are trying it. And so, we're going to talk about the technologies that underlie that and the story of how it came to be. So, welcome Erich and welcome Dirk. ERICH: Hello from Zurich. CHARLES: Alright. Zurich to Albuquerque. Here we go. As a first start, I would have to say my first contact with this story, I at least have to mention it because – and this is for Erich – you wrote a book that was very, very instrumental in my formation as a young developer. I think I was about 22 years old when I read ‘Design Patterns'. And I don't know. I still carry a lot of those things with me to this day, even though a lot of things have changed about the way that we do development. I still carry a lot of those lessons, I think especially things like the state pattern and the strategy pattern, and stuff like that. I want to move onto other things, but I was hoping that we could talk just a little bit about, what are the things that you find still kind of relevant today? ERICH: Well, now as you said, some of the things are kind of timeless and we're lucky to have found these things. And I still love all the patterns. But I must say, things have changed, right? So, at that time, we thought objects are very cool. And as we have evolved, all of a sudden we think, “Oh, functions are actually very cool, too,” right? Closures and so on. So, I think we got more broader and of course if you use functional programming, you have many more patterns available as you program. So, I feel some of the object thinking still applies. But that's not the only thing that counts anymore. Today it's functions, stateless, immutability, and all those things within functional programming which is [straight] and which [inaudible] in our team. CHARLES: Yeah, yeah. I would love to see an update to how do these concepts transfer into functional programming. But anyway, just wanted to say thank you for that. And it was about the same time that, a few years after, I don't know the exact same timing, I want to wind back. Because we're going to talk about VS Code but before VS Code, there was a project that both of you all worked on called Eclipse, which I also used. Because at the very beginning of my career, I did a lot of Java development. And it really opened my eyes into a level of what tooling could do for you that I didn't see before. And I was wondering how did you arrive to there? Because before that, I was using Emacs and Vim and Joe's Editor and things that were editing the text files. And how did you kind of arrive at that problem? Because I feel like it's very similar to the one that VS Code solves, but this was what, 15 years ago? ERICH: I think it's older, right? CHARLES: Really? DIRK: It's 17, 18, yeah. Yeah, yeah. It was end of the millennium, right? So to be honest, Eclipse wasn't the first development tool we worked on. Then, we worked on the company ObjectTechnologyNational. They worked on Smalltalk tools. And of course, Smalltalk had a great IDE experience, right? So back then, Java became popular. One idea was, how can you preserve the great Smalltalk coding experience? [Inaudible] CHARLES: Ah, okay. DIRK: [Inaudible] and find all references, method-level history versioning, and so on. So, that was the input that got Eclipse kicked off. And one idea we had at that time, Eclipse is our opportunity to make everything right. And as we have seen now, when we did VS Code, we could even improve what we have [inaudible] at that time. So an example, in Eclipse we thought plugins are very cool and we have kind of a microkernel. And you load all of the plugins in the same process, they have a rich API, and so on, which is great. But we found over time, if you have lots of plugins and they do bad things and they run in the same process, it's not the best thing. CHARLES: Ah. Right. And so… DIRK: [Inaudible] have a different architecture. We believe now in isolation, separation. So, we now run extensions in separate process that communicates through RPC with the IDE so that we are in full control. And we can always say you can save the tool, save the document, no matter how bad a plugin behaves and decides to do an endless loop. Because in a separate process, the hope is still one CPU is open, available for you, that it can be safe from the other process. So, that's some example, right? Eclipse has done many things right, but the multi-process architecture I think is a major switch. And the other major switch is at Eclipse time you think Java is cool. Everything has to be in Java. CHARLES: Right. DIRK: No longer think like that, and that brings up this other topic of then the language servers that we can also talk at some point. CHARLES: Right, because that's the thing, is VS Code – now I've primarily been exposed to it through JavaScript and TypeScript development. But it really, it's designed to support all kinds of different languages. So, the C++ support is really good. The C support is really good. And I assume the Java support is really good. Is it safe to say? Because I only ever used Eclipse in the context of Java. Did Eclipse gain kind of a wider acceptance further beyond Java and C++? ERICH: Yeah. I think it's fair to say Eclipse has a rich ecosystem. Yeah. I think with all the tools. And it will be interesting to see that you can close the loop, because for Visual Studio Code, when you do Java development, you actually run Eclipse behind the scenes. That's how we kind of smiled at each other, Dirk and I, when he said, “Now we close the loop.” We started with all JavaScript and then we integrate Eclipse using this language server protocol and that's how we close the loop. CHARLES: Ah. DIRK: So maybe one thing I would like to add is that when you look at Eclipse and the tool and framework landscape that existed in the Java time, at that point in time when we started with Eclipse, it was very well-defined. There was Java. It was a well-defined set of libraries you were using and frameworks you were using. And if you look at the programming and tool landscape you have today, in months you see a new framework for JavaScript popping up or there's something else or another cool X, Y, Z thing. So, the tooling you build today has to be a lot more open to these new inventions, especially since they occur in a higher frequency than they did in the past. And that had influence on how we architected Visual Studio Code to give people a lower barrier of integrating their stuff into Visual Studio Code than you typically have in Eclipse. In Eclipse you needed to program in Java. With the LSP you can program in any programming language. In Eclipse, if you really want to try to do something nice with code complete and stuff like that, you had to hook up a lot of stuff. So, we raised that to another abstraction layer where we more talked about what people provide on data and we do a lot more for them in the user interface than compared for example to Eclipse, which lowers the barrier for people to integrate languages in Visual Studio Code than the barrier you had to integrate something in Eclipse. And so, [inaudible] for that one was that there are a lot more tools and programming languages out there that have importance than 10 years ago. ERICH: I'll give you an example. So, when we did C support in Eclipse, and it was also the team that seeded it. Of course, it took over and has now a great community behind it in Eclipse. But you wrote the C tooling in Java. And of course, that means you built the parser in Java and then of course, there are great C parsers around, C frameworks. But also it means you cannot dogfood what you write. You write Java but you don't program in C++. I think which is what makes VS Code so appealing is we are a very aggressive dogfooder. We want to use ourself and of course [inaudible]. That's why [inaudible] is very good. The C++ guide, they programmed C++ and they write in C++ so that's how they make it very good, that you have this feedback loop. CHARLES: And so, what's an example? We've talked about this low barrier of entry. So, if I were wanting to say, I do mostly programming in JavaScript. Let's say I wanted to add, I know all of this already exists, this infrastructure already exists, but let's say I wanted to add smart editing to JavaScript source files. What would that process look like for me as a JavaScript developer? DIRK: To be fair, whenever it comes to language services, it's never easy. But [inaudible] lower the bar. A language always means you have to do parsing, you have to do [9:59], type bindings. You have to make it fast, scale high up, and so on. So, this is never easy. But I think if you think about the different steps you can do, the first thing, let's not take JavaScript. Let's take a new language. CHARLES: Okay. DIRK: Your new cool language. CHARLES: Or maybe we take a Lisp or something where writing the parser is very easy. DIRK: Even that, you have to resolve symbols and so on. CHARLES: Okay, okay. DIRK: Even the parsing [inaudible]. But yeah, let's take a fancy language like Lisp or whatever. So, the first level I think is you want to get some nice coloring. That's the first level. CHARLES: Yes. DIRK: So, you get some coloring. And what we do there actually in VS Code is we tap into the community from TextMate. So, we use TextMate grammars to support colors in languages, which gives us access to a long [10:51 tail] of languages. So, to change the [10:54], if your language is not too exotic, you will find the grammar that describes how to color, what the tokens are in your language, and then you can get your language colored. That's step one. The next step is of course you want to get smarts like IntelliSense and so on. Ideally of course you can say, “Well, maybe there is something already around that has abstracted the parser and you can use this library.” CHARLES: Right. Because there actually are a bunch of JavaScript parsers written in JavaScript. I know I keep coming back to JavaScript, but let's assume with this language that we've got. I may not have to write a parser but I've got one. ERICH: You've got one, exactly. You've got one, right, and then technically it's not in the same language as the tool. So, that's why I don't want to go too much into JavaScript because for instance VS Code is written in TypeScript, which [transpiles] to JavaScript, which moves a little bit, makes it not as convincing as it could be. So, let's say it's a different language. Your fancy language is written, has a parser in your fancy language, which is different than the language of VS Code which is JavaScript. CHARLES: Right. ERICH: So, then the next level is to say, “Okay, well you have your code you encapsulate it in a server that you can talk to through some protocol.” And now the challenge is what protocol do you talk to? Typically in the language, the library you get, it will use some ASTs, symbols, type bindings. And what Dirk mentioned with lowering the bar is that assuming you have those ASTs, the way you talk then with our tool is through a protocol that is not at the level of the ASTs but at a higher level. CHARLES: A higher level than the ASTs. ERICH: No, yeah. A higher or simpler level. Let's give you an example. You want to find the definition of a symbol in your fancy language. The way the protocol works is you only tell it, in this document with the URI, at this position, I want to find the definition of the symbol that is this position. The request goes over the wire to the other process. Document URI, and the textual position. And what comes back of course now in the server you used AST, you find the symbol, you find the binding of the symbol which means it gives a definition for it. Of course you use your AST to analyze it. But then what gets back to send over the wire is yet another document, the reference, and the position. CHARLES: I see. So, you're really like pinpointing a point in just the raw bytes of the document. And you're saying, “Look, what is here?” And you just want to delegate that completely and totally to this other process. So, the IDE itself doesn't know anything about the document? ERICH: It knows about the document, right? CHARLES: I mean, it knows about the textual positions of the documents and the stream of characters, but not the meaning. DIRK: True. The smarts are in the server. And you talk to the smarts at the level of documents and positions. And the [good thing is] it's a protocol, is at this level it makes it easy to integrate into one editor, which is VS Code, but also into other editors. So, that's why we came up with the idea to have a common language server protocol which allows to provide a language not only for one editor but also for many editors. That was a challenge we had in VS Code. Remember when we started, we were kind of late to the game. We said, “VS Code should be in between an IDE and an editor.” But what we liked from an IDE is of course code understanding, IntelliSense. Go to definition, find all references. But how do you get that for a long tail of languages? We cannot do it all ourselves. So, we need to get a community to tap into. [Similar to] like TextMate grammars are kind of a lingua franca for coloring. So, we are looking for the lingua franca for language smarts. And that's what the language server protocol is, which means you can integrate it in different IDEs and once you've written a language server you can reuse it. CHARLES: I guess I've got two questions. What are the kind of things that I can do with a server that implements the language server protocol? And then I guess the – so we've talked about being able to find a reference. And is there a way you can incrementally implement certain parts of the protocol as you go along? ERICH: Yeah. DIRK: Yeah, basically you can. The protocol on the server and the client side talks about capabilities. The server can for example say, “I am only supporting code complete and go to definition and find all references.” And for example, something like, “Implementation hierarchy or document symbols or outline view is not supported.” And then the client adapts dynamically to the capabilities of the server. CHARLES: Okay. DIRK: That's one thing. And the set of capabilities is not fixed. So, we add them. We just added four or five new capabilities to the protocol last week. So of course, we listen to requests that come from other IDEs, what they would like to see in the protocols that we see in Visual Studio Code, we would like to extend. And that's the way we move the protocol forward. CHARLES: Okay. DIRK: It's capability-based and not so to speak version-based. So, [inaudible] versioning at the end of today. CHARLES: Right. You can incrementally say, “I'm going to have,” if I'm starting to write a server, I can say, “Well, I'm going to only start with just find definition at point.” And that's the only thing that my server can do. ERICH: Well, there are some basics, right? Keep in mind you have two processes. And once the user opens an editor, the truth is in the buffering memory on the one process. The basic thing you have to in a language, so you have to support the synchronization of [inaudible]. Once you open a file in the editor, then the truth in the buffer, and then you have to sync it over. CHARLES: Right. ERICH: [Inaudible] close the truth on the file system and you also have to tell this to the server. Because the server has to know where the truth is. DIRK: That's correct. These two open/close handshake methods and change methods, this is the minimum you have to implement. But for example, for Node itself, we provide libraries that help you with this. And the protocol is not very complicated. It's a buffer. Then it's change events. Either it's an insert, a delete, or an edit. CHARLES: So, let me try and get this straight in my head. I think I understand. The problem is that the VS Code, or your code editor, it's actually making changes to the buffer, and it needs to communicate those changes to the server. Or does the server actually make the changes itself? DIRK: The editor does make the changes. So, the protocol is spec'd in a way that as soon as an editor opens a document, the ownership travels from the server for the content to the tool. And the server is basically not allowed to read the state of that content from disk anymore, or get it [inaudible]. CHARLES: Aha. DIRK: Therefore, the client guarantees that everything the user does in that document is notified to the server, so that the server can move the document forward. CHARLES: Okay. DIRK: [Inaudible] we see the close event, that basically with the close, transfers the ownership of the document back to the language server. And it is allowed to re-read that content from disk if it wants. CHARLES: Okay. ERICH: Here, the protocol is really data-driven. Dirk mentioned that earlier, right? So, basically what flows between the server and the tool is data. So, what do we mean by data? You ask for IntelliSense or completions at the line. What follows is just the data. A list of completions that flows then from the server to the client. And then the client decides what to do with this data and decides to modify the document by inserting the completion proposed that the user selected. CHARLES: Right. And then if it decides to make any updates, it needs to send those to the server. DIRK: Exactly. CHARLES: So, if I actually insert the method that I want to call there, I'm going to be inserting nine characters, and I need to tell the server, “Hey, I just inserted nine characters to this document,” something like that? ERICH: Exactly. CHARLES: Okay. And so now how, because I remember now one of the coolest things about the class of tools of Eclipse that I hadn't really seen in the more lightweight editors – I went from Java, like so many of my generation went from Java to Ruby and then to JavaScript – once I moved out of the Java world, one of the things that I had come to expect from my tools was that they would help me make modifications to my codebase at a very high level. So, I would be able, if I had some class that was imported into say five modules in my codebase, I could say, “I want to change the name of this class,” and then it would find the references and then make the updates to those things. So, how do you manage that? So, if I have a class called ‘Person' that I want to change to ‘User', if I change it to ‘User' then it's going to break in those five different places unless I rename it to ‘User'. That's something that was very doable in the Java world. How do you keep the code editor, the tool I guess is what you were calling it, in sync? Like the server is going to make that change or does it just come back with data and says, “Here's the references if you wanted it to change”? ERICH: Yeah, yeah, yeah. So, two things you mentioned, right? Java and JavaScript or course. Java is a typed language which means you have better understanding of the code and what the reference is. In JavaScript which is typeless, you cannot know it as much, so that's actually why we developed also, we're using TypeScript. VS Code is actually [written] in TypeScript which allows you to do these kinds of things like refactorings. But if you look at the language server protocol, it has support for rename. And the way how rename is done is again it just documents positions. You say, “At this position, I want to rename the symbol with this other name.” And then you tell this to server and the server will handle the rename by giving you back a list of positions that need to be updated. CHARLES: Ah, okay. So now, I'm starting to understand what you're talking about when you say data-driven. It's literally just telling the tool – the tool proposes, “I want to do this rename.” And then the server provides all of the information that is required to actually do the rename. But it doesn't actually do the rename itself. It just provides the data. DIRK: A couple of reasons for it. The data effects, at the end of day, it's again, edit, and it's more or less the same edits the client sends to the server when the user types in the document. This is the protocol. On top of it, something that you can create a file or rename a file, this comes as a result back to the client. And then there, since it is a client/server architecture, the whole process is async. So, we have to give the client the change to revalidate if that edit structure that comes is still valid. If it is still valid, the client basically applies it. And by applying these edits to these documents, they will automatically flow back to the server until the client either closes these documents again or saves them. So, the reason being is that some of the tools may even show you a preview. You can only select some of them and apply them. So, there's always an interaction in these refactorings and to make that possible, as Erich mentioned, the whole protocol is data-driven. We don't go the server and say, “Okay, do that rename,” and he writes that back to disk. It computes a set of transformations to bring the current state of the workspace into that new state after the refactoring. CHARLES: I see. ERICH: [Inaudible] be fully transparent. Actually, no. Refactorings, Dirk [inaudible] refactorings for Eclipse so we can go deep on that. What we don't support right now in the protocol, we support edits in the buffer but when you want to rename a class in Java, you also want to rename the file. And that's something we're currently working on to support in the specification of the language server protocol. So, we don't have that yet. But we support code actions, quick fixes, that you like from Eclipse probably. And you can use then to do refactorings like extract method, extract constant or extract local variable, things like that you can do at the level of the language server protocol. CHARLES: Wow. That is… ERICH: I think [inaudible] right now. Let me go back to the Java thing. The Java language server actually has the support for refactorings. And there is now a language server protocol implementation of this Java provided by Eclipse. So, all the support you had in Eclipse for Java or most of the support is now also enabled in VS Code. CHARLES: Right. ERICH: [We don't] really have to reimplement it because you can reuse. And that's the big thought we have. You want to reuse language smarts as much as possible because they are so hard to implement. CHARLES: Right. And so, you can do that because you're providing this abstraction between the tool and the actual smarts, which is really, really cool. I do have to… how do you make it fast? Because you're describing this tool, this client and this server, and they're syncing. They're keeping this distributed state in sync and you know, how do you keep that from coming too chatty? Or is it something that you have to consider? Or is it just, maybe I'm overthinking it because I haven't dealt with it? DIRK: So, at the end of the day, it is chatty. But it is made performant in the way that it's very incremental and partly event-based. So for example, if you type in the document in the editor, you can either decide to [inaudible] sync the full content of the document, which we do not recommend but for some basic exploration, that is something people do. And we have [inaudible] the delta-encoded mechanism. So, we sync the buffer once and then after that you only get the edits the user does. These are chatty of course since the user types them, we debounce them and collapse them on the client side and only send them if we know that the server really needs to know them because we have another request we are asking the server or after a certain timeout. So, there are smarts behind it. But the protocol is kept performant by making it an incremental protocol at the end of the day, and not sending too much data back and forth. ERICH: Right. We don't serialize ASTs. We serialize positions, a list of items for completions. And actually, the transport is just JSON RPC. CHARLES: Okay. ERICH: And actually, someone, there is different usage now for language server protocol. And there is one host, Eclipse J, which brings it again back to Eclipse. They actually run language servers remote. CHARLES: Interesting. ERICH: And if you use it, you can run it on the browser, you get IntelliSense, and of course I guess it depends on how far away you are from the server. But it seems to work, according to feedback we've heard. CHARLES: Really? ERICH: The feedback we heard from them [is pleasant]. So, they use many of the language servers. CHARLES: So, is this a product that they have where the language server is running in the cloud and you send – your entire codebase essentially goes over to the language server and you can export the smarts to the cloud? ERICH: It's one step at a time. So, Eclipse J is kind of, they have what they call cloud workspace, which means the workspace is in the cloud. And [inaudible] code smarts of the workspace in the cloud, they can run the language servers in the cloud. It's a [inaudible]. One user has one workspace, has one language server. CHARLES: That sounds amazing. And if they can make it performant. ERICH: We have done cloud IDEs, right? If you look at the history from Visual Studio Code, you also had our stuff running in the cloud at some point. That's how we started. Before we pivoted to VS Code, we built – our exploration was, that's why the project is six years old. The first two years, we explored how far you can get coding done in the browser. CHARLES: Right. ERICH: And we had some [inaudible] there. CHARLES: So, I've played around with a lot of cloud IDEs and I've found them to be neat, because every few years it comes along. But yeah, it does seem that there are certain challenges that it's nice to have a client running and just be able to have the files locally. And is that a performance thing or if VS Code is written in TypeScript, theoretically it could run in a browser, right? ERICH: Of course. The [inaudible] there still runs in the browser. Then it's used by many tools that run in the browser. Like actually, if you want to edit your source code in the browser, there it's using the same editor that's running VS Code. So, that's how we started. Cloud IDEs, yeah we were at this point. We had our cloud IDE. We could edit websites in the browser, source control them, have a command line, deploy them. What we found is it's great for some scenarios like code reviews or doing small tweaks to files. But when it comes to really development, you use so many other tools. And you want to just have them. And [inaudible] a long tool chain problem. So, as a developer, you just want to use other tools as well. And that's why you can't have them all in the cloud. CHARLES: Right. ERICH: And [inaudible] we said at some point, it was a great lesson we had that you can program in the browser. But now we want to go to have a really [seven by 24] coding, you want to have a desktop experience. So, what we then did, we moved over the code we had run in the browser using a shell, the Electron shell, and can run it on the desktop. CHARLES: But there's theoretically, you could be running your language server for example in the cloud, but everything else on the desktop. ERICH: Yeah. Some people do that. DIRK: Right. CHARLES: Okay. Wow. It's crazy. It's heady stuff. We've talked about the barrier to implement the code smarts is much lower than it has been in the past. What kind of proliferation of code smart tools are there now that implement the language server protocol? Like how many different languages would you say have airtight…? DIRK: So now, [inaudible] time where we don't count anymore. You tell us a language and I can look it up, whether it's supported. Tell me a language and I can tell you whether – no, we have a website. CHARLES: Okay. DIRK: And when I look at it, we have about 40 languages. CHARLES: Wow. That's probably about, pretty much every mainstream language. DIRK: Yeah. I cannot find what isn't there. CHARLES: Yeah. It almost kind of begs the question, is this going to be the new bar for a language? Because I remember when I was starting out, really you just needed to have some interpreter or some compiler to have “a language”. And nowadays, it's not just the language. You need to have a command line tool for managing your dependencies. And you need to have a package system with a public repository where people can publish reusable units of code. And what's become expected out of a language to succeed has upped. Is having a language server implementation going to be part of the bar, the new bar, for “Hey, I'm thinking about creating a language”? I haven't really arrived until I have a package manager, I have a command line for resolving dependencies, I have documentation, and I have a language server. DIRK: I personally think that is our dream at the end of the day, to get there. We know about languages that do so. So, a lot of these language servers come for example from the people that developed the language. For example, the WASP guys, they do the compiler and they actively work on their language server as well. So, at the end of the day, the advantage of that approach since the WASP language server is written in WASP and runs in WASP, they can reuse so much code that they already have written in WASP. That's easy for them to package that up in the server and basically the people that maintain the compiler, at least the same team, maintains the language server at the end of the day. ERICH: And that's why we call [those] a win-win for the language provider. Because if you implement the language server using the language server protocol, then it can be integrated easily by the tool provider. And it's a win for the tool provider since there is a common protocol across all these languages you have to support. You can write an implementation once and again benefit and support many different languages, which makes the matrix problem one language support for each tool into more a vector, right? It reduces the matrix into a vector. You only write language servers that get integrated into different tools. CHARLES: Right. DIRK: And [inaudible] especially I think appealing for new languages that come out, because it lowers the bar for them to get into existing tools. Because if they write a language server speaking the language server protocol integrating that at the end of the day in Visual Studio Code is basically packaging up an extension for Visual Studio Code and writing 20 lines of code. CHARLES: Yeah. DIRK: And same [inaudible] for other IDEs that exist where people implemented the language protocol client side for the tool, for example. For vim or for Atom. CHARLES: Yeah. DIRK: So, new languages I think definitely, we see that trend go onto the language server protocol because that gives them an entry point into a large tool community. CHARLES: Yeah. I'm really excited about it. I'm actually an Emacs user. And that's actually how I found out about LSP, was in my Emacs newsfeed I saw that someone was starting on LSP support, and got digging into it. And I think that one of the problems that has plagued not only Emacs but all these editors is what you're describing where for example the JavaScript support was really great – is really great – in Emacs. There's refactorings. There's IntelliSense, code completion, all that stuff. But that's because someone wrote an entire JavaScript parser and code smart system in Elisp, which is just an absurd hurdle to jump over, to expected. And so, what you expect out of your editing experience, like when I went to try – if I were to go to try Python, well it's not nearly as good as what I'm expecting. And so yeah, I think it's exciting to hear what you're describing where with having some shared set of abstractions, you can offload all of that code smart onto the community that's building these new tools so that they're really easy to integrate into your environment. I think it's really exciting. Although it does make me ask – and I think we've got time for one more question – is we've been talking about all these different languages. Java, C++, JavaScript, TypeScript, Ruby, Python, et cetera, all these, the 40 languages that you talked about that have this implementation. What are the challenges that you've encountered trying to build abstractions that work for 40 different languages? All with their different syntax, all with their different conventions. It sounds like when aside from the fact that you've actually done it, I would say it's impossible. So, I'm curious. What were the unique challenges to solve there? DIRK: I think we already touched that at the beginning, the appealing stuff of the LSP is that it's not talking about the programming language itself. It's talking about things I can do with source code. For example, requesting code complete, go to definition, find all references. And the data that flows between the client and the server is not in terms of the programming language itself. It's about editor abstractions. We talk about documents and positions. We talk about edits that are applied to documents. We talk about snippets and stuff like that. And these abstractions, since they are programing-language-neutral, are a lot easier to implement for different editors. And the [inaudible] where the [inaudible] would speak AST nodes and symbols and functions and classes and methods, that at the end of the day, would not work. Because if I ask go to definition, the result is not a function or a variable definition. It's simply a position in the document with a hint which range to select. CHARLES: Okay. Yeah. ERICH: [Inaudible] places. In only a few places, we have to really abstract across languages. Like for instance, completions. When you do completions, you don't know, is it a variable? Is it a function or a method? That's where we have to abstract. But that's one of the few places. But again, it's an enumeration. DIRK: Yeah. And that's only to present an [icon]. ERICH: Yes. DIRK: It's only to give you a nice icon in front, because when you insert it, what comes back for completion item is basically a textual edit or a bunch of textual edits that when you select that completion item, we take these edits and apply them to the document buffer. And whether you edit a functional programming language or some other stuff, Prolog or whatsoever, it does not matter at the end of the day. CHARLES: Yeah. That simplicity, and treating it at that simple of a level is what unlocks all those superpowers. ERICH: It unlocks lowering the bar. But of course, if you look at some [of the demands], refactorings, whatever, they cannot easily be funneled. Not all of them can funnel to this low-level abstraction. Then of course, the criticism of the LSP protocol is that if you have already a very rich language service, you might not get it all through the LSP. DIRK: That's true. ERICH: And the [inaudible], that criticism we see of the LSP. But it's a tradeoff, like so many things in software. DIRK: Yeah. But what we learned there looking at different types of refactorings, it's more the set of input parameters that vary much between languages. The result of a refactoring can for every programming language that is at least document-based, [inaudible] in that lingo the LSP speaks. Because at the end of the day, it's textual edits to a document, right? ERICH: So, many people like LSP but there are people that don't like it. And people that have rich language services like IntelliJ, [Cool Tool], and [inaudible], even with LSP we would only get 20% of [our cool] features. Which is a little bit downgraded and not really true. But you see, it's a tradeoff. CHARLES: Right. ERICH: And if you want to [inaudible] language available broadly, I highly recommend it packaged as a language server. Your chances that it gets used, supported by different tools, is much higher than anything else. CHARLES: Right, right. So, it's kind of like, what's the UNIX thing? The universal text interface and how it seems counterintuitive but it actually just means you can literally compose anything. Because so few assumptions are made. ERICH: I would just recommend, [inaudible], go to the website that we have about the language server protocol. I'm pretty sure it will be in the introduction or whatever. It's microsoft.github.io/language-server-protocol and then you see the implementations, all the implementation of languages, who integrates language servers, and also what kind of libraries are available, if you want to implement your language server. DIRK: And a full specification. ERICH: And the specification is there as well. Yeah. CHARLES: Yeah. If you want to go ahead and do it yourself. Well, thank you so much, Erich. Thank you so much, Dirk, for coming on the show to talk about the language server protocol. It's very exciting to me and I think it's exciting for development in general because I just think by having – even if it's 20, 30, 50% code smarts for ever single language, just the billions and billions of hours that you are going to save developers over the next, over the coming years, it's a great feeling to think about. So, thank you for all your work and thank you for coming on the show. ERICH: You're welcome. DIRK: Yeah. It was fun talking to you. ERICH: Yeah. [Inaudible] CHARLES: Yeah. If people want to continue the conversation, is there a good way that they can get in touch with you? DIRK: Usually GitHub Issues. So, where the language protocol is, it's a project on GitHub. Simply find issues. We accept pull requests. I think that's the way we communicate. CHARLES: Awesome. Again, if you want to get in touch with us, you can get in touch with us at contact@frontside.io or you can reach out to us on Twitter. We're @TheFrontside. So, thank you everybody for listening. And we will see you next time.
In this Modern Web Podcast This Dot Labs CTO, Taras Mankovski @tarasm, discusses State Management with guests... Charles Lowell @cowboyd (Frontside), Michel Weststrate @mweststrate (Mobx), and Tim Dorr @timdorr (React, Redux Router) Topics covered: - MobX tracking changes in objects- Redux changes to accommodate async rendering in React- Best practices around batch operations in Redux- Immer.js To learn more visit www.thisdot.co Follow us on Twitter @moderndotweb
Katharine Beaumont: @katharinecodes Show Notes: In this episode, we hit the topic of machine learning from a 101 perspective: what it is, why it is important for us to know about it, and what it can be used for. Transcript: CHARLES: Hello everybody and welcome to The Frontside Podcast, Episode 94. My name is Charles Lowell, a developer here at The Frontside and your podcast host-in-training. Today I'm going to be flying it alone, but that's okay because we have a fantastic guest who's going to talk about a subject that I've been dying to learn about. But you know, given the number of things in the world, I haven't had a chance to get around it. But with us today is Katharine Beaumont who is a machine learning consultant. And she's going to talk to us, not surprisingly, about machine learning. So welcome, Katharine. KATHARINE: Hello. Thank you very much for having me. CHARLES: No, no, it's our pleasure. So, I guess my first question is, because I'm very much approaching this from first principles here, is what is machine learning as a discipline and how does it fit into the greater picture of technology? KATHARINE: Okay. Well, if you think about artificial intelligence which is one of those slightly undefinable fields because it encompasses so much, so it encompasses elements of robotics, linguistics, math, probability, philosophy, it has six main elements. So, a really basic definition of machine learning is getting, and this comes from Arthur Samuel in 1959, it's about getting computers to learn without being explicitly programmed. And that's hugely paraphrasing. But machine learning is an element that sits under the wider discipline of artificial intelligence. Artificial intelligence is one of those tricky to define fields because people have different opinions about what it is. And obviously philosophers can't agree what intelligence is, which makes it slightly complicated. But artificial intelligence as a broad brush is a discipline that borrows from philosophy, math, probability, statistics, linguistics, robotics, and spawned subfields like natural language processing, knowledge representation, automated reasoning, computer vision robotics, and machine learning. Machine learning is the, in a sense, the mathematical component of artificial intelligence in that from a basic point of view, even though you're looking at it from the perspective of computer science, you're utilizing algorithms that a lot of mathematicians will say, “Look, we've been doing this for years. And you've just stolen that from us,” that try and find patterns in data. And that pattern could be as basic as mapping, say, the square footage of a house to the price that it will sell at and making a prediction based on that for future examples, or it could be looking for patterns in images. CHARLES: Okay. You mentioned something that I love to do. I love stealing ideas from other disciplines. It feels great. KATHARINE: Who doesn't? CHARLES: Yeah. It's like free stuff. And the best part of ideas is the person who had it still has it after you've lifted it off of them. KATHARINE: Yeah. You just have to reference and then it's not plagiarizing. CHARLES: Yeah. So, how did you actually get into this? KATHARINE: Well, a few years ago, I was desperately bored in my job. CHARLES: So, what was that job that you were working on that was so desperately boring? You don't have to name a company. KATHARINE: Oh, I won't name the company but I will – I have to make a confession now which links back to something that we were saying off recording earlier, which was that it was doing web development. So, I'm sorry. And that's not to say that web development is boring at all. It's just that I wasn't particularly engaged, which is not a reflection on web development. CHARLES: No, no, no. I actually came – I was doing, before I got into web development, I was actually doing backend stuff for years. That was all I did. KATHARINE: Yeah, me too. I would have described myself as a server-side Java developer who then cross-trained into Ruby. And I thought I'd be doing exciting backend things in Ruby. But unfortunately, it was more, “We'd like you to move this component from this part of the page to this part of the page.” And I didn't really connect with that. And I started to wonder if I even should be a developer. CHARLES: Wow. KATHARINE: Larger forces than myself were at work to try and push me into management or analysis. And as happens, I think, after a few years. So, I started doing, in my spare time, looking at a website (and I'm sure you've heard of it) Coursera. CHARLES: Yeah. KATHARINE: So, this is the birthplace of the massive online, I can't remember what the second O is, MOOCs. Massive Online something learning. Maybe a Q in there. I'm not sure what. Do you know what the acronym is? CHARLES: I actually don't know. KATHARINE: Well, MOOCs anyway. Massive online learning courses. And there was one offered Andrew Ng from Stanford on machine learning. So, I took that and I just loved it. I really enjoyed it. And I really connected with the programming. I really enjoyed the programming. It was very fulfilling. So, it grew from that, really. And now, I've decided to go back to university. So, I'm a mature postgraduate student and I'm just currently weighing up my PhD options. So, whether to sacrifice four years for the greater good and the pursuit of knowledge or go back into an employment. So, we'll see. We'll see. And I'm quite enjoying not being employed, I have to admit. Or being employed on a freelance basis. It's wonderful. CHARLES: Right, right, right. Now, a couple of things stuck me when you were talking about – so obviously, you're studying a lot of the mathematics behind it. And you said that machine learning involves a lot of the – it's the mathematical component of artificial intelligence. But what strikes me is learning, to me, implies a lot of statefulness where you're accumulating state. Whereas my experience with mathematics is usually you're solving equations. You start from some set of facts and whether it's a dataset or some other thing, and you derive, boom, boom, boom, boom, boom, you get your answer. Whereas with learning, at least when I think about school learning, like spelling or, I don't know, paleontology or something, you're accumulating facts over a very long time. And the inferences that you make are not necessarily – they're drawn from all fo the sources that you got over all that period of time rather than some one set of facts that then you make this logical argument and poof, presto, you've got your answer. How does that square? I guess it's just a little bit off from my experience with mathematics. KATHARINE: So, I am being a bit reductionist. So probably, one way to explain it is that essentially, behind a lot of the machine learning algorithms, you're inputting numbers. And that might be the percentage of red, green, blue in a pixel for example. Or it might be the diameter of a wheel, for example, if you're looking at a component of a car. Or it might be a binary configuration if you want to input the configuration of a control panel, for example, and you're looking for anomalies. And you're running these numbers through an algorithm. And what you're getting out is either a continuous value, if you're looking at a problem with continuous data like house prices, or you're getting a probabilistic output like 60% certain these pixels together make a cat, for example. So, I am simplifying by saying it's math because what you're really doing is looking for patterns in data but a way to get a computer to understand it is to somehow input it as numbers, essentially, and to get numbers out of it. CHARLES: Oh. KATHARINE: Yeah, it's more algorithms, really. And I shouldn't have said that it was essentially math because I'm sure I'm going to get shouted at on the internet. CHARLES: Well, I certainly don't want to get you in trouble. But maybe that's a point that we should shy away a little bit, the high theoretical stuff, and bring it back. If I'm excited, not even if I'm excited, why should I be excited about it? I've heard that it's a hot topic. I've heard that a lot of people are excited about it. Is there a way that I as someone who has no specialization in this might actually be able to bring some of these techniques to bear on the problems that I'm working on? Perhaps even without understanding them first, like understanding how it works. What are some problems that I might be able to attack with these techniques? KATHARINE: Yeah, absolutely. So actually, and one of the things about machine learning that I should say is don't think, “Oh, it's not for me. I'm rubbish at math. I don't understand these concepts. I'm not willing to get my head around an algorithm,” because there are so many pre-configured APIs available from big companies like Google and Amazon and Microsoft and IBM, and I'm sure many, many more. And I'm not paid by any of them, I should say. So, you don't need to understand the inner workings of an algorithm to use it. So, one example is speech-to-text. So, if you imagine that you're working on a website and you want to make it accessible, maybe you could have a component to your navigation bar that allows users to record their voice and say, “I want to navigate to the shopping cart,” for example. And machine learning would be behind that processing. So maybe, behind it you'd have an API, I've used a few of them before just to play it, where you make a call to, say, IBM service and it returns you the text. And in your program you match on keywords like shopping cart and then change the menu bar for them. So, that's one really simple way you could do it. Another more complicated way to do it is to implement something like a recommender system. So, say you have a website where you offer customers products of some description. And the most famous example of this is Amazon and Netflix. Amazon, the shopping site, rather than now the big, big corporation. And you see what other customers like you bought, or Netflix, what you might enjoy. And that's based on taking your information, comparing your viewing habits to other people's viewing habits, and then drawing some kind of correlation between the programs you watch and trying to find programs that other people have watched that you haven't, that you might enjoy. That's more complicated, to be honest. CHARLES: But that is an example. Machine learning is what underlies all that. KATHARINE: Absolutely. And at the heart of some recommender systems, the mathematics behind it is finding a way to quantify people's preferences and measuring distances between them. But you don't need to understand that to understand the basics of how a recommender system works. CHARLES: Okay. And so, how does a recommender system work? KATHARINE: So, imagine me, yourself, and Mandy each read four books. And we rate them. But I read four books, you read three of the same books, and one different one, and Mandy reads three of the same books as me and one different one, for example. So, we've got a little gap but we're not really sure what the other person will think. And we know the genres of the books. And you can compare the genres and the ratings. So, you might rate sci-fi 6 out of 10 and romance 7 out of 10. And I might rate sci-fi and romance in equal ways. So then you might say, okay, there's a similarity between our preferences. So for this book, that Charles read, Katharine might like it. CHARLES: That makes so much sense. KATHARINE: Yeah. And maybe Mandy only likes romance, only rates it 0.3 for example. So we think, “Okay, well Mandy might not be able to recommend a book to Katharine and Charles.” CHARLES: Right, I see. Implicit in this though is there's this step of the actual learning, I guess. Or the actual teaching. How do you actually teach? Again, and this is kind of me trying to wrap my head around the concept, is I've got these set of facts and I'm inferring and I'm pattern-matching and I'm trying to draw conclusions with some certainty from this set of data. But is there this distinct actual teaching phase where you have to actually teach the computer and then it takes new facts and gives stuff? How does that work? How does it incorporate the different – I guess what I'm saying is, are there distinct phases? Or is it… KATHARINE: Yes, and it's not the same for every algorithm. So, I'm going to try and give you two examples of training. So, there's something called online learning where as a new example comes in, for example it might be – I'll just explain what a classifier is. So, this is a brief diversion. So, a classifier is a machine learning process where you're trying to put information in and you're trying to get discrete information out. So, discrete meaning like it's a cat or it's a dog or a weasel or a minion or something like that. Or, it's cancer or it's not cancer. Whereas continuous output might be the price of a car. So, in a classifier you're trying to work out what type something is. So, a really good example is there's a Google project where they've clustered artworks. So, they've taken lots of different artwork and their algorithms, which I won't explain now, have determined, “This is a ballet dancer. So, we're going to group all of these ballet dancer paintings together. This is a landscape, so we're going to group all the landscapes together.” So, online learning, you might get a bit if information in, like a picture, and you will classify it and you add it to your existing information. Whereas another type of learning is you take all of the information you have, you train the algorithm, and then you make predictions. So, you either make predictions as you go along with online learning, or you do all of the work upfront. So, one algorithm – have you heard of decision trees? CHARLES: No, I haven't. KATHARINE: So, do you ever read those rubbish teen magazines where you have a flowchart and it starts at the top like, “Do you like cats? Yes or no?” CHARLES: Oh right, yeah. KATHARINE: “Do you likes dogs? Yes or no,” and it tells you what kind of a person you are or what make up you should wear or something like that. CHARLES: Right, right, right, yeah. KATHARINE: Yeah, so decision trees are kind of like that. One example is you might get information about, it's a famous toy dataset, information about passengers on the Titanic about gender and age. And we all know the techniques on the Titanic, women and children first. I've lost my use of normal English. I'm sorry. CHARLES: Right. Maybe like a trope? KATHARINE: Yeah. So, women and children first. So, you put this data in a decision tree. And what happens is at the beginning you have all of this data. So, person A is a male. They're in their 50s. This is the type of ticket they had. And this is their income. I don't think income is one of them, but just as an example. And then you have person B, person C. So, you might have a hundred people. And the decision tree algorithm goes, “Okay, if I just looked at one of these features like gender, would that differentiate the people the most?” So, it already has the answer as to whether or not they survived or did not survive. And it's looking for the one feature that gives it the most information. And then it will split on that feature. So, you go form your thing at the top and the first question might be, “Were they male or female?” And then the decision tree will split down a level. And then your algorithm will go, “Alright, what's the next feature?” Maybe the next feature is, “Were they under the age of 30?” for example. And it works down. And you end up with this sort of flowchart. And once it's trained, you then get a new example you have, person X. and you basically just work your way through the decision tree to make the prediction of whether they survived or did not survive. CHARLES: I see. And so, do you do it with some sort of certainty? Because there's going to be variation, right? You're going to have some people who are a poor fellow in his 70s who survived. Like, it's not certain that he went down but there's some probability at the end? KATHARINE: Yes. Because you're using it to make a prediction, there is always a probabilistic element. And it depends on the decision tree algorithm. So, there are some decision tree algorithms that really don't work with contradictory data, for example. CHARLES: I see. KATHARINE: There's an element of picking your algorithm. If you're approaching a machine learning problem, you have three elements. The first one is choosing your feature and your algorithm. Then you have evaluating it, so you need a way of saying how good or bad the algorithm's doing on your data, how accurate is it for example. And then you have optimizing it, which is the dark art of machine learning. CHARLES: Right. That's the wrap across the knuckles. It's coming up with wrong numbers. KATHARINE: Yeah. That's – oh no, this took two weeks to run and I need it to take 20 seconds. Or, this is only 60% accurate and I need it to be more accurate. And it's easy, just as an example I'm just working on a course that I'm giving in a few weeks' time. And I just took a day to set up a website called Kaggle, K-A-G-G-L-E, for wine quality. So, it's got acidity, citric acid, residual sugar, chlorides. They're the features. They're the components that you're going to put in. And it has a quality score. So, what you're trying to do is you're trying to find a relationship between the features and the quality. And it's very easy to get it to work. So, I now have it working. But I have it working with a really low accuracy. So, it took maybe five minutes to get it to work and it's going to take me about half an hour to make it more accurate, and that's the optimization element. CHARLES: I see. How stable are these processes? Is it finicky and fragile so that if you get new types of features it just throws things way off? Or are there ways you could control for that? KATHARINE: Well, that's a particular type of problem and that all links in with the evaluation and optimization. And that's to do with something called overfitting. So, decision trees is an algorithm notorious for overfitting, depending on the data, that is. So, overfitting is when you get the algorithm to perform really well on the training data. And then you feed it in a new example and it might grossly misclassify it because it hasn't learned to generalize beyond the examples that you've given it. CHARLES: I see. Okay. So, it's just too concrete. It hasn't recognized deep patterns. It's only recognized something superficial. KATHARINE: Yes. So typically, if you have a finite amount of data, you only train your algorithm on a certain percentage. And then you test it on the rest. CHARLES: I see. KATHARINE: So, you hold back some data. But back to our conversation earlier about when does the training happen? Another example is something called K-nearest neighbors. You could just imagine that means three nearest neighbors, for example. So, we're trying to find who we're most similar to in a room. So, it's a room full of people. And all the people are standing next to already similar people, for example. So, you might have a room where marketing's in one corner and the software developers are in another corner and the project managers are in another corner. And you go it and you're looking for the three people who are the most similar to you and you're going to go and stand in that group, for example. So, in that type of machine learning, the training is happening as each new sample comes in, rather than upfront. And there are drawbacks to both methods and there are positives to both methods. And really, in a horrible, unsexy way, it's to do with the data. And that's normally where most people because it's the most boring part when you're talking about machine learning. CHARLES: Is the data? KATHARINE: Yeah. It's this backlash from data scientists, from the golden age of data scientists where it was the hottest job on the internet to now everyone cringing going, “Oh, I really don't want to deal with my data.” But with any machine learning problem, you can't just go, “Okay. Here you go, Charles. Here's a dataset. Learn something from it.” You need context. You need to understand it. You need to have an idea of what you're looking for. So, you're getting the machine to learn but you're using it as a tool to complement your knowledge, really. And you're feeding in your knowledge to it. And part of that are the decisions that you make on the algorithms to use. CHARLES: Okay. KATHARINE: And what you're looking for. CHARLES: So, here's something that I'm wondering is related, and again I have no idea – for some reason I always associate when people talk about neural nets as being related to teaching a computer something. Is that part of the discipline of machine learning? Or no? KATHARINE: I would say it is. But it has its own cool and trendy title of Deep Learning. But it's very much powerful machine learning. So, let's go back to this. This is a classic example of machine learning. It's probably the first example you'll come across if you do any course. House prices. So, imagine that you have a piece of paper and you're going to draw one line at one side and that is the price of a house, and you're going to draw a line at the bottom which is the size in square feet, and you're going to plot examples that you have. And you might find that there's a linear correlation between the two. So, you'll draw a line and that line of best fit is the human equivalent of doing linear regression on a computer, for example, where you're just trying to find a linear correlation between two things. And a lot of the principles in linear regression, which is a very simple learning algorithm, are found in some examples of neural networks, so some basic examples of neural networks. But instead of having one input, the size in square feet, you might have 20. And you might be repeating that process of trying to find the line of best fit with different combinations of features in different places again and again. And it scales up in complexity very quickly. But it's very similar basic principles. I'm hesitating to say ‘very similar' because they're notoriously more complicated. CHARLES: Yeah. I guess I'm not really divining what exactly, what makes it – why is it called a neural net? What makes it special? It sounds like if I'm just comparing the regressions of house prices, I'm comparing those datasets over and over again, how is that different from just a loop? What are you getting out of it? KATHARINE: Historically, neural networks come from a very simplified idea about how the brain works. So, in the early 20th century people had performed autopsies and divined the inner workings of kidneys and hearts and livers. And the brain was still a bit of a mystery. And then 2 men jointly won the Nobel Prize for Physiology, and I'm going to pronounce these names wrong, so I'm sorry. I think it's Santiago Ramón y Cajal is one and Camillo Golgi is another. And they completely disagreed about the brain but they used a staining technique from Golgi to look, using silver nitrate, at the cells in the brain. And the idea of the neuron doctrine was borne out of that, that the simplest unit to look at the brain at in order to understand it is the level of the neuron, this cell in the brain. And from there, several – well, everybody was a polyglot really, back then, so I don't want to say computer scientists. So, you had people like Frank Rosenblatt with perceptrons, Marvin Minsky and so many other people looking at a very simple idea which is that a neuron either fires or doesn't. And then you're linking boolean algebra to this cell. So, you're saying it either fires or it doesn't. And from that principle, people started drawing similarities between neurons and a basic function machine. And when I say function machine, I mean imagine when you were in elementary school and you're learning how adding up works. You might have a box with a plus on it and your teacher says, “I want you to put a three in a box and a four in a box and I want you to add them together. And what do you get out?” And obviously the answer is seven. And you can think about that little box with a plus as a function machine. So, now you could think of a little mathematical function machine where you put in some inputs and there something happens in the box. And then you'll either get a one or a zero out of it. CHARLES: And so, that's like your neuron, is the little box? KATHARINE: Yes. CHARLES: Okay. KATHARINE: Yes. So since then, the neuron doctrine is pretty much contested. There are several other elements of the brain that compose thinking and the circuitry. So, any cognitive scientist listening to this will say, “That's really not how the brain works.” You have to say it with a lot of disclaimers. But the whole idea of neural networks was borne out of this idea of thinking of a neuron as like a function machine. CHARLES: Right. And also, it doesn't actually discount the usefulness of neural networks. There are a lot of things where people didn't find what they set out to find but what they found was useful. KATHARINE: Absolutely. Yeah, and they are incredibly powerful, especially with multilayer networks which are deep networks or deep learning. CHARLES: Okay. So, I didn't want derail you from your explanation. So, you've got these little function boxes and those are the kind of neurons inside the neural network? KATHARINE: Yes. So, what happens is you might have a layer of 10 of them and you might have another layer after that of another 10. And each of them are connected in a simple neural network. And then you might have an output layer of five because you're looking to classify, I don't know, an apple into five different types of apple, for example. CHARLES: Now, when you're talking about a later, you're talking about, I've got the outputs of one layer of the network are the inputs to the next layer. KATHARINE: Yes. And in different neural network architectures they'll be connected differently. But in a simple neural network, you can assume that every neuron is connected to every neuron in the next layer. So, if you have two 10-layer, two layers each with 10 neurons in, the top neuron in one layer is going to have 10 connections going out of it into the next layer. CHARLES: Oh really? Wow, that's interesting. KATHARINE: Yes. And each connection has its own sort of configuration. CHARLES: So, you're like cross-wiring all the – okay. Wow, that's kind of… KATHARINE: And yeah, that's where the complexity comes in. CHARLES: Yeah. I was thinking it was like a simple exponential fan-out. But it's even more complicated, the number of combinations you can get. KATHARINE: Yes. But in theory, it's very simple because each neuron is like this function machine with inputs coming in and something going out. It just might go out to several different locations. But it scales up in complexity very quickly. So say we're classifying apples. So, we have five different attributes of an apple like its color, its texture, its weight, acidity. I don't know how else you would measure an apple. How shiny it is, for example. And we're putting all of that information in. And each one of those things that I've just listed is a feature. So, we might have an input per feature and we'll link them all up to the neurons in a layer and we'll move that information onto the next layer. And the really important thing is what's happening on those connections between them. Because on the connections you have weights, which is a way of changing the input from one neuron to another. So, the weight might be like 0.5 for example. So, it squashes the input. Or it might magnify it. And then you get the output. Now, once you have the output you can compare the output with what you know the right answer is. And then you have this idea of an error. So, you might be like, “You got this so wrong. It's not a Braeburn apple at all. It's actually a Granny Smith apple. And what you do is with each example that you train in, you use your information about the error to train the network. Because what you're trying to do is get the error to be as small as possible. And one of the techniques of that is called back propagation. And it's notoriously difficult to understand because it involves partial derivatives and a large element of calculus. But essentially, what you're doing is comparing the right answer with the answer that the network gave it and asking it to go back and change it, change those connection weights. CHARLES: And so, do you make them fluctuate at random or is there some – is there a method to the madness of changing the weights? KATHARINE: There is a method to the madness and it's called back propagation. And the reason I linked linear regression in early, so our really simple map of house size in square feet, and the price, is because it uses a similar technique called gradient descent which is an algorithm for looking at the error and changing the weight, those numbers on the connections, to try and get it to a minimal point. So, if we go back to our house price problem, I just want you to imagine in your mind that we've got this one axis going up which is the price, and one going across which is the size in square feet. And you've got line drawn, a diagonal line. If you just imagine now in your mind moving that line down till it's completely flat at the bottom, and then moving it up so it's vertical, so it's aligned with either axis in every point in between, what you could do is you could take the error on each of those lines. So, if we imagine we have these two axes, we have the price of a house on one side and we have the size in square feet in another. And we're going to draw a line from the top axis and we're going to sweep it down and draw a line at each point as it goes down until it's aligned with the bottom axis. And at each point we draw that line, we measure the error. So, what we'd do for that is all of the little points, all of the example data, we'd measure the difference between them and the line and we're going to, say, add them all up. So, what you'd end up with is you'd end up with a graph mapping the error against the gradient at all fo those different points. And it would look like a bowl. You'd have a lowest point. You'd have a point for some gradient where the error is the lowest. CHARLES: Right, yeah. Okay. I'm seeing it. I think I'm seeing it. So, you want to take that error function and you want to, what is it? Now this is – boy, I'm going back to high school math. You would take the derivative and find out where the tangent is and that's your min point? That's the root of the equation and that's the point where your error is lowest? KATHARINE: Yeah, exactly. Or there's an algorithm called gradient descent that does it automatically by taking little steps. So, it looks at the tangent of the gradient at a point and says, “If I move the gradient down is the error going to decrease. And if so, move in that direction.” So automatically, it tries to take steps to get to the bottom. And optimization problem with that is that you can configure the step size. So, you could take tiny, tiny steps and take forever to get to the bottom or you could take massive steps and completely miss the bottom. So, you can imagine it like walking down a hill. If you're a minion then it will take a really long time because you're tiny. And if you're a giant, you might never get to the valley. CHARLES: Right. You might just leap right across the chasm. KATHARINE: Yeah, just miss it completely. So, that's linear regression where we're looking. And that's really, even though we have a two-dimensional graph that's a one-dimensional problem because we just have the one input feature. Now, when we're looking at neural networks and we're looking at gradient descent in neural networks, each one of those connections is something that we're trying to configure to reduce the error. So suddenly, you have maybe a hundred-dimension landscape and you're trying to get to the bottom of a hill. And there might be several local optimas and one deepest valley, but you might have lots of other valleys that you could get stuck in. So, it becomes a very difficult problem. Does that make sense? CHARLES: Yes. No, that does make sense. I'm just trying to let it sink in. KATHARINE: It's completely impossible to visualize a hundred dimensions, yeah. CHARLES: I actually had to sit back and kind of close my eyes and stare up at the ceiling. KATHARINE: I think the trick is not to think about it. I heard someone say, there's another fantastic course on Coursera by a famous computer scientist studying neural networks called Geoffrey Hinton. And his advice in one of the videos on visualizing these multidimensional landscapes, say it's 15 dimensions, is to close your eyes and shout, “15!” And that hasn't worked for me, but I'm sure it's worked for some people. CHARLES: But it certainly probably makes you feel better. KATHARINE: Yeah. I think it's one of those things that's just beyond comprehension. But we can just quietly accept it. CHARLES: Right. It's just – yeah, what's nice about I guess math is you just don't have to understand it. I mean, you do. You just understand that there really is no mapping to our physical experience. And that's okay. And just let that go. It's just like, this is just some… KATHARINE: Yeah. CHARLES: We had some numbers that existed in the domain of understanding, that we can understand. And there are just some rules that we follow and if you look at the intermediate steps, well the points don't really exist in that domain of physical experience and understanding. And that's okay. We just accept and let it go. We just hope that at some point, we can translate that model back into the domain of ‘we can understand it'. KATHARINE: Oh, completely. There's a lot of faith. CHARLES: Yeah. KATHARINE: And also, I remember coming into machine learning and thinking, “This is like magic. It's amazing.” And then you study it a bit and you're like, “This is so easy. This is just basic – this is just functions. These are just glorified function machines.” And then you look into it some more and you're like, “Nope. It's definitely magic.” CHARLES: Yeah. It's a phase of, every time you come up against the wall, right? And then you realize, “Oh no, it's actually something that I can close my mind over,” until you come to the next hurdle of magic. KATHARINE: Yeah. You think you've got a grasp on things and you think, “I know the landscape,” and then you suddenly realize how much more there is to learn. And that sinking feeling that you'll never learn it all. CHARLES: Yeah, yeah. Yup. Unfortunately, it seems like in tech that's like, that's just the condition. KATHARINE: Yeah, like all of my sad, unread books. CHARLES: If I wanted to get into just really start experimenting with this stuff and start saying, “Maybe I can utilize some of these techniques for some of these problems that I'm encountering,” Where would be a good place to get started? What libraries? What online resources? What people are good to follow and ask questions of? KATHARINE: Okay. Let's start with websites. So for a start, this website called Kaggle which I mentioned earlier, and that is K-A-G-G-L-E dot com. And that has a lot of dataset resources. It also has a community of people discussing how they use the datasets. It has competitions. And it has a lot of links. I discovered recently as well a really good blog. It's on Medium and it is called ‘Machine Learning for Humans'. And that's really well-written. I really like it, actually, and it has a good section on resources called ‘The Best Machine Learning Resources'. And I should probably plug my own blog, but this one's so much better. CHARLES: But please do. KATHARINE: No, no. I have to actually write stuff for it. But there's a lot of things there about, well, if you want to learn linear algebra, what if you want to learn probability and statistics, calculus, and then just go straight to machine learning and pick up the math on the way. I would say go on Coursera, because there are courses like Andrew Ng's course on machine learning from Stanford. And Geoffrey Hinton from the University of Toronto. But there are also courses there on things like calculus and probability and statistics if you want to level up your math. If you don't want to do anything to do with the math, I would say go to the vendor websites, like AWS, Google. If you're into Java, go on deep learning for J. And a lot of them have tutorials that complement their products. Deep learning for J is one of my favorites at the moment. It's an open source Java library. It's pretty plug and play, actually. You don't need to understand a lot of it to get started with it. But it helps. And obviously then there's TensorFlow although personally, I find just other Python libraries like SciPy a lot easier than using TensorFlow. And I think naturally you'll find the resources and the people to follow on Twitter from that. But the crucial thing I'd say is don't get hung up on which language to start playing around with. So, a lot of people say, “Oh, I must need to know Python. I must need to know math.” And really, you don't. It just depends on what level you want to approach things at. So, if you want to write your own gradient descent algorithms, then Python is probably more for you, or Matlab or R or something like that. But there are libraries where you can do it in Java. I've heard rumors that there's a JavaScript library and I wouldn't be surprised. So, I would just have a look at what's out there. But try and get a grasp of the fundamentals just from an intuition point of view, because it will make your life so much easier. You might for example realize that you're using the completely wrong algorithm for the problem that you're looking at. And that's invaluable. CHARLES: Yeah. Knowing what not to do certainly is. Alright. Well, thank you so much for that, Katharine. Thank you for being on the show. Thank you for curing us of at least a small portion of our ignorance. And if people want to get in touch with you perhaps and continue the conversation, or follow you, what's a good place to get in touch? KATHARINE: Sure. Probably tweet me on Twitter. I'm @KatharineCodes but it's spelled like Katharine Hepburn. It's K-A-T-H-A-R-I-N-E codes. Because when I joined Twitter, I didn't have much of an imagination. I still don't. So, it's not particularly clever. But it's there. CHARLES: Alright. Well, fantastic. And for everybody listening at home, you can also get in touch with us. We're @TheFrontside on Twitter. Or you can just drop us a line at info@frontside.io. Thanks for listening and we will see you all next time.
Julie Moronuki: @argumatronic | argumatronic.com Show Notes: This episode is a follow-up episode to the one we did with Julie in September: Learn Haskell, Think Less. We talk a whole lot about monoids, and learning programming languages untraditionally. Transcript: CHARLES: Hello everybody and welcome to The Frontside Podcast, Episode 93. My name is Charles Lowell, a developer here at The Frontside and I am your podcast host-in-training. With me today from The Frontside is Elrick also. Hello, Elrick. ELRICK: Hey. CHARLES: How are you doing? ELRICK: I'm doing great. CHARLES: Alright. Are you ready? ELRICK: Oh yeah, I'm excited. CHARLES: You ready to do some podcasting? Alright. Because we actually have a repeat guest on today. It was a very popular episode from last year. We have with us the author of ‘Learning Haskell: From First Principles' and a book that is coming out but is not out yet but one that we're eagerly looking forward to, Julie Moronuki. Welcome. JULIE: Hi. It's great to be back. CHARLES: What was it about, was it last October? JULIE: I think it was right before I went to London to Haskell [inaudible]. CHARLES: Yeah. JULIE: Which was in early October. So yeah… CHARLES: Okay. JULIE: Late or early October, somewhere in there. CHARLES: Okay. You went to Haskell eXchange. You gave a talk on Monoids. What have you been up to since then? JULIE: Oh wow. It's been a really busy time. I moved to Atlanta and so I've had all this stuff going on. And so, I was telling a friend last night “I'm going to be on this podcast tomorrow and I don't think I have anything to talk about.” [Laughter] JULIE: Because I feel like everything has just been like, all my energy has been sucked up with the move and stuff. But I guess… CHARLES: Is it true that everybody calls it ‘Fatlanta' there? JULIE: Yeah. [Laughs] CHARLES: I've heard the term. But do people actually be like “Yes, I'm from Fatlanta.” JULIE: I've heard it a couple of times. CHARLES: Okay. JULIE: Maybe it's mostly outsiders. I'm not sure. CHARLES: [Chuckles] JULIE: But yeah, it's a real cool city and I'm real happy to be here. But yeah, I did go in October. I went to London and I spoke at Haskell eXchange which was really amazing. It was a great experience and I hope to be able to go back. I got to meet Simon Payton Jones which was incredible. Yeah, and I gave a talk on monoids, monoids and semirings. And… CHARLES: Ooh, a semiring. JULIE: Semiring. So, a semiring is a structure where there's two monoids. So, both of them have an identity element. And the identity element of one of them is an annihilator. Isn't that a great word? It's an annihilator… CHARLES: Whoa. JULIE: Of the other. So, if you think of addition and multiplication, the identity element for addition is zero, right? But if you multiply times zero, you're always going to get to zero, so it's the annihilator of multiplication. CHARLES: Whoa. I think my mind is like annihilated. [Laughter] JULIE: So, it's a structure where you're got two monoids and one of them distributes over the other, the distributive property of addition and multiplication. And the identity of one of them is the annihilator of the other. Anyway, but yeah, I gave a history of where monoids come from and that was really fun. CHARLES: Yeah. I would actually like to get a summary of that, because I think since we last talked, I've been getting a little bit deeper and deeper into these formal type classes. I'm still not doing Haskell day-to-day but I've been importing these ideas into just plain vanilla JavaScript. And it turns out, it's actually a pretty straightforward thing to do. There's definitely nothing stopping these things from existing in JavaScript. It's just, I think people find type class programming can be a tough hill to climb or something like that, or find it intimidating. JULIE: Yeah. CHARLES: But I think it's actually quite powerful. And I think one of the things that I'm coming to realize is that these are well-worn pathways for composing things. JULIE: Right. CHARLES: So, what you encounter in the wild is people generating these one-off ways of composing things. And so, for a shop like ours, we did a lot of Ruby on Rails, a lot of Ember, and both of those frameworks have very strong philosophical underpinnings that's like “You shouldn't be reinventing the wheel if you don't have to.” I think that all of these patterns even though they have crazy quixotic esoteric names, they are the wheels, the gold standard of wheel. [Laughs] They're like… JULIE: Right. CHARLES: We should not be reinventing. And so, that's what I'm coming to realize, is I'm into this. And last time you were talking, you were saying “I find monoids so fascinating.” I think it took a little bit while to seep in. But now, I feel like it's like when you look at one of those stereo vision things, like I'm seeing monoids everywhere. It's like sometimes they won't leave me alone. JULIE: In ‘Real World Haskell' there's a line I've always liked. And I'm going to misquote it slightly but paraphrasing at least. “Monoids are ubiquitous in programming. It's just in Haskell we have the ability to just talk about them as monoids.” CHARLES: Yeah, yeah. JULIE: Because we have a name and we have a framework for gathering all these similar things together. CHARLES: Right. And it helps you. I feel like it helps you because if you understand the mechanics of a monoid, you can then when you encounter a new one, you're 90% there. JULIE: Right. CHARLES: Instead of having to learn the whole thing from scratch. JULIE: Right. And as you see them over and over again, you develop a kind of intuition for when something is monoidal or something looks like a semiring. And so, you get a certain intuition where you think, “Oh, this thing is like a… this is a monad.” And so, what do I know about monads? All of a sudden, this new situation like all these things that I know about monads, I can apply to this new situation. And so, you gain some intuition for novel situations just by being able to relate them to things you already do know. CHARLES: Exactly. I want to pause here for people. The other thing that I think I've come in the last three months to embrace is just embrace the terminology. JULIE: Yeah. CHARLES: You got to just get over it. JULIE: [Chuckles] CHARLES: Think about it like learning a foreign language. The example I give is like tasku is the Finnish word for pocket. JULIE: Right. CHARLES: It sounds weird, right? Tasku. But if you say it 10 times and you think “Pocket, pocket, pocket, pocket, pocket.” JULIE: Yes, yeah. [Laughs] CHARLES: Then it's like, this is a very simple, very useful concept. JULIE: Right. CHARLES: And it's two-sided. There on the one hand, the terminology is obtuse. But at the same time, it's not. It's just, it is what it is. And it's just a symbol that's referencing a concept. JULIE: Right, right. CHARLES: It's a simple concept. So, I just want to be… I know for our listeners, I know that there's a general admonition. Don't worry about the terminology. It's… JULIE: Right, right. Like what I just said, I said the word ‘monad'. I just threw that out there at everybody, but [chuckles] it doesn't matter which one of these words we'd be talking about or whatever I call them. We could give monads a different name and it's still this concept that once you understand the concept itself, and then you can apply it in new situations, it doesn't matter then what it's called. But it does take getting used to. The words are… well, I think functor is a pretty good word for what it is. If you know the history of functor and how it came to mean what it means, I think it's a pretty good word. CHARLES: Really? So, I would love to know the history. Because functor is mystifying to me. It sounds like, I think the analogy I use is like if George Clinton and a funk parliament had an empire, the provinces, the governors of the provinces would be functors. ELRICK: [Laughs] JULIE: Yes. CHARLES: But [Laughs] that's the closest thing to an explanation I can come up with. JULIE: I might use that. I'm about to give a talk on functors. I might use that. [Laughter] ELRICK: Isn't that the name of the library? Funkadelic? CHARLES: Well, that's the name of the library that I've been… JULIE: [Could be], yeah. ELRICK: That you'd been… CHARLES: That I'd been [writing] for JavaScript. ELRICK: Yeah. CHARLES: That imports all these concepts. JULIE: [Laughs] ELRICK: Yeah. JULIE: Yeah. ELRICK: So awesome. JULIE: Yeah. Yeah, I have… CHARLES: So, what is the etymology of functor? JULIE: Well, as far as I can tell, Rudolf Carnap, the logician, invented the word. I don't know if he got it from somewhere else. But the first time I can find a reference to it is in, he wrote a book about… he was a logician but this is sort of a linguistics book. It's called ‘The Logical Syntax of Language'. And that's the first reference I know of to the word functor. And he was trying to really make language very logically systematic, which natural language is and isn't, right? [Chuckles] CHARLES: Right. JULIE: But he was only concerned with really logically systematizing everything. And so, he used the word functor to describe some kinds of function words in language that relate one part of a sentence to another part of a sentence. CHARLES: Huh. So, what's an example? JULIE: So, the example that I've used in the past is, as far as I know this is not one that Carnap himself actually uses but it's the clearest one outside of that book… well the ones inside the book I don't really think are very good examples because they're not really how people talk. So, the one that I've used to try to explain it is the word ‘not' in English where ‘not' gets applied to the whole sentence. It doesn't really change the logical structure of the sentence. It doesn't change the meaning of the sentence except for now it negates the whole thing. CHARLES: I see. JULIE: And so, it relates this sentence with this structure to a different context, which is now the whole thing has been negated. CHARLES: I see. So, the meaning changes, but the structure really doesn't. JULIE: Right. And it changes the whole meaning. CHARLES: Right. JULIE: Not just part of the sentence. So, if you imagine ‘not' applying to an entire sentence because of course we can apply it just to a single word or just to a single phrase and change the meaning just of that word or that phrase, but if you imagine a context where you've applied ‘not' to a whole sentence, to an entire proposition, because of course he's a logician. So, if you've applied ‘not' to an entire proposition, then it doesn't change the structure or the meaning of that proposition per se except for it just relates it to the category of negated propositions. CHARLES: Mmhmm. JULIE: So, that's where it comes from. And… CHARLES: But I still don't understand why he called it functor. JULIE: He's sort of making up… well, actually I think the German might be the same word. CHARLES: Ah, okay. JULIE: Because he was writing in German. Because he's looking for something that evokes the idea of ‘function word'. CHARLES: Oh. JULIE: So, if you were to take the ‘func' of ‘function' [Laughs] and the, I don't know, maybe in German there's some better explanation for making this into a particular word. But that's how I think of it. So, it's ‘function word'. And then category theorists took it from Carnap to mean a way to map a function in this category or when we're talking about Haskell, a function of this type, to a function of another type. CHARLES: Okay. JULIE: And so, it takes the entire function, preserves the structure of the function just like negation preserves the structure of the sentence, and maps the whole thing to just a different context. So, if you had a function from A to B, functor can give you a function from maybe A to maybe B. CHARLES: Right. JULIE: So, it takes the function and just maps it into a different context. CHARLES: Right. So, a JavaScript example is if I've got an array of ints and a function of ints to strings, I can take any array of ints and get an array of strings. JULIE: Right. CHARLES: Or if I have a promise that has an int in it, I can take that same function to get a promise of a string. JULIE: Yeah. CHARLES: Yeah. I had no idea that it actually came from linguistics. JULIE: Yeah. [Laughs] CHARLES: So actually, the category theorists even… it digs deeper than category theory. They were actually borrowing concepts. JULIE: They were, yes. CHARLES: We just always are borrowing concepts. ELRICK: I like the borrowing of concepts. JULIE: Yeah. ELRICK: I think where people struggle with certain things, it's tying it back to something that they're familiar with. So, that's where I get… my mind is like [makes exploding sound] “I now get it,” is when someone ties it back to something that I am… CHARLES: Right. ELRICK: Familiar with. Like Charles' work with the JavaScript, tying it with JavaScript. I'm like, “Oh, now I see what they're talking about.” JULIE: Right. CHARLES: because you realize, you're using these concepts. People are using them, just they're using them anonymously. JULIE: Right. ELRICK: True. CHARLES: They don't have names for them. JULIE: Right. ELRICK: True. CHARLES: It's literally like an anonymous function and you're just taking that lambda and assigning it to a symbol. JULIE: Yeah. CHARLES: You're like “Oh wait. I've been using this anonymous function all over the place for years. I didn't realize. Boom. This is actually a formal concept.” ELRICK: True. And I think when people say like “Don't reinvent the wheel” it's a great statement for someone that has seen a wheel already. [Laughter] ELRICK: You know what I'm saying? If you never saw a wheel, then your'e going to reinvent the wheel because you're like “Aw man. This doesn't exist.” [Chuckles] JULIE: Yeah. ELRICK: But if people are exposed to these concepts, then they wouldn't reinvent the wheel. CHARLES: Right. JULIE: Right. Yeah. CHARLES: Instead of calling in some context, calling it a roller. [Chuckles] It's a round thingy. [Laughter] JULIE: Right. Yeah, so that's a little bit what I tried to do in my monoid talk in London. I tried to give some history of monoid, where this idea comes from and why it's worth talking about these things. CHARLES: Yeah. JULIE: Why it's worth talking about the structure. CHARLES: So, why is it worth the… where did it come from and why is it worth talking about? JULIE: Oh, so back when Boole, George Boole, when he decided to start formalizing logic… CHARLES: George Boole also, he was a career-switcher too, right? He was a primary school teacher. JULIE: Right, yeah. CHARLES: If I recall. He actually, he was basically teaching. Primary school is like elementary school in England, right? JULIE: I believe so, yes. CHARLES: Yeah. I think he was like, he was basically the US equivalent of an elementary school teacher who then went on to a second and probably, thankfully a big career that left a big legacy. JULIE: Right. Although no one knew exactly how big the legacy was really, until Claude Shannon picked it up and then just changed the whole world.[Laughs] Anyway, so Boole, when he was trying to come up with a formal algebra of logic so that we could not care so much about the semantic content of arguments (we could just symbolize them and just by manipulating symbols we could determine if an argument was logically valid or not), he was… well, for disjunction and conjunction which is AND and OR – well, disjunction would be the OR and conjunction the AND – he had prior art. He had addition and multiplication to look at. So, addition is like disjunction in some important ways. And multiplication is like conjunction in some important ways. And I think it took me a while to see how addition and disjunction were like each other, but there are some important ways that they're like each other. One of them is that they share their identity values. If you think of, it's sort of like binary addition and binary multiplication because in boolean logic there's only two values: true or false. So, you have a zero and a one. So, if you think of them as being like binary addition and binary multiplication then it's easier to see the connection. Because when we think of addition of just integers in a normal base 10 or whatever, it doesn't seem that much like an OR. [Laughs] CHARLES: Mmhmm. No, it doesn't. JULIE: [Inaudible] like a logical OR. So, it took me a while to see that. But they're also related then to set intersection and union where intersect-… CHARLES: So can… Let's just stop on that for a little bit, because let me parse that. So, for OR I've got two values, like in an ‘if' statement. This OR that. If I've got a true value then I can OR that with anything and I'll get the same anything. JULIE: Right. CHARLES: So, true is the identity value of OR, right? Is that what you're saying? So, one… JULIE: Well, it's false that's the identity of OR. CHARLES: Oh, it is? JULIE: Zero is the identity of addition. CHARLES: Wait, but if I take ‘false OR one' I get… oh, I get one. JULIE: Right. CHARLES: Okay. So, if I get ‘false OR true', I get true. Okay, so false is the identity. JULIE: Yeah. CHARLES: Oh right. You're right. You're right. Because… okay, sorry. JULIE: So, just like in addition, zero is the identity. So, whatever you add to zero, that's the result, right? You're going to get [the same] CHARLES: Right. JULIE: Value back. So, with OR false is the identity and false is equivalent to zero. CHARLES: [Inaudible] ‘False OR anything' and you're getting the anything. JULIE: Right. So, the only time you'll get a false back is if it's ‘false OR false', right? CHARLES: Right. Mmhmm. JULIE: Yeah. So, false is the identity there. And then it's sort of the same for conjunction where one is the identity of multiplication and one is also the… I mean, true is then the identity of logical conjunction. CHARLES: Right. Because one AND… JULIE: ‘True AND false' will get the false back. [Inaudible] CHARLES: Right. ‘True And true' you can get the true back. JULIE: Yeah. CHARLES: Okay. JULIE: And it's also then true, getting back to what we were talking about, semirings, it's also true that false is a kind of annihilator for conjunction. That's sort of trivial, because… CHARLES: Oh, because you annihilate the value. JULIE: Right. When there's only two values it's a little bit trivial. But it is [inaudible]. So… CHARLES: But it's [inaudible]. Yeah. It demonstrates the point. JULIE: Right. CHARLES: So, if I have yeah, ‘false AND anything' is just going to be false. So, I annihilate whatever is in that position. JULIE: Right. CHARLES: And the same thing as zero is the annihilator for multiplication, right? JULIE: Right. CHARLES: Because zero times anything and you annihilate the value. JULIE: Yeah. CHARLES: And now I've got… okay, I'm seeing it. I don't know where you're going with this. [Laughter] ELRICK: Yeah. CHARLES: But I'm there with you. ELRICK: Yup. JULIE: And then it turns out there are some operations from set theory that work really similarly. So, intersection and union are similar but the ones that are closer to conjunction/disjunction are disjoint unions and cartesian products. So we don't need to talk about those a whole lot if you're not into set theory. But anyway… CHARLES: I like set theory although it's so hard to describe without pictures, without Venn diagrams. JULIE: It is. It really is, yeah. So anyway, all of these things are monoids. And they're all binary associative operations with identity elements. So, they're all monoids. And so, we've taken operations on sets, operations on logical propositions, operations on many kinds of numbers (because not all kinds of addition and multiplication I guess are associative), and we can kind of unify all of those into the same framework. And then once we have done that, then we can see that there's all these other ‘sets'. Because most of the kinds of numbers are sets and there are operations on generic sets with set theory. So, now we can say “Oh. We can do these same kinds of operations on many other kinds of sets, many other varieties of sets.” And we can see that same pattern. And then we can get a kind of intuition for “Well, if I have a disjunctive monoid where I'm adding two things or I'm OR-ing two things…” Because even though those are logically very similar, intuitively and in terms of what it means to concatenate lists versus choosing one or the other, those obviously have different practical effects. CHARLES: So, I'm going to try and come up with some concrete examples to maybe… JULIE: Okay, yeah. CHARLES: A part of them will probably be like in JavaScript, right? So, to capture the idea of a disjunctive monoid versus a conjunctive monoid. So, a disjunctive monoid is like, so in JavaScript we're got two objects. You concat them together and it's like two maps or two hashes. So, you mash them together and you get… so, for the disjunctive one you'd have all the keys from both of the hashes inside the resulting object. You take two objects. Basically we call it object assign in JavaScript where you have basically the empty object. You can take the empty object and then take any number of objects. And so, we talked about… JULIE: That would become a disjunctive monoid, right? CHARLES: That would be a disjunctive monoid because you're like basically, you're OR-ing. Yeah. JULIE: You're kind of, [inaudible] CHARLES: Hard to find the terminology. JULIE: Yeah. CHARLES: But like object assign would be a disjunctive monoid because you're like mashing these two objects. And the resulting object has all of the things from both of them. JULIE: Right. So, it's like a sum of the two, right? CHARLES: Right, right. Okay, so then another one would be like min or max where you've got this list of integers and you can basically take any two integers and you can mash them together and if you're using min, you get the one that's smaller. Basically, you're collapsing them into one value but you're actually just choosing one of them. Is that like… JULIE: Yeah. CHARLES: Would that be like a conjunctive monoid? JULIE: No, that's also disjunctive but that's more like an OR than like a sum. CHARLES: Okay. JULIE: Right. So, that's what I said. It's hard to think of disjunctive monoids I think because there's really two varieties. There's some underlying logical similarity, like the similarity in the identity values. But they're also different. Summing two things versus choosing one or the other are also very different things in a lot of ways. CHARLES: Right. Okay. JULIE: And so, I think the conjunctive monoids are all a little bit more similar, I think. [Chuckles] But the disjunctive monoids are two broad categories. And we don't really have a monoid in Haskell of lists where you're choosing one or the other. The basic list monoid is you're concatenating them. So, you're adding two lists or taking the union of them. But for maybe, the maybe type, we do have monoids in Haskell where you're just choosing either the first just value that comes up or the last just value that comes up. So, we do have a monoid of choice over the maybe type. And then we have a type class called alternative which is monoids of choice for… so, they're disjunctive monoids but instead of adding the two things together, they're choosing one or the other. CHARLES: Okay. JULIE: Though we have a type class for that. [Laughs] CHARLES: [Sighs] Oh wow. Yeah. JULIE: Mmhmm, yeah. CHARLES: I'l have to go read up on that one. JULIE: That type class comes up the most when you're parsing, because you can then parse… like if you found this thing, then parse this thing. But if you haven't found this thing, then you can keep going. And if you find this other thing later, then you can take that thing. So, you allow the possibility of choice. The first thing that you come to that matches, take that thing or parse that thing. So, that type class gets mostly used for parsing but it's not only useful for parsing. CHARLES: Okay. JULIE: So yeah. That's the most of the time when I've used it. CHARLES: Is this when you're like parsing JSON? Or is this when you're just searching some stream for some value? Like you just want to run through it until you encounter this value? Or how does that…? JULIE: Right. Say you want to run through it until you find either this value or this value. I've used it when I've been parsing command line arguments. So, let's say I have some flags that can be passed in on my command line command. There are some flags that could be passed in. So, we'll parse until we find this thing or this thing. This flag or this flag. So, if you find this flag, then we're going to go ahead and parse that and do whatever that flag says to do. If you don't find that first flag then we can keep parsing and see if you find this other flag, in which case we'll do something different. CHARLES: Okay. JULIE: It'll take the first match that it finds. Does that make sense? CHARLES: Yeah, yeah, yeah. It does. But I'm not connecting how it's a monoid. [Laughs] JULIE: How is that a monoid? Well, because it's a monoid of OR-ing CHARLES: What's the identity value or the empty value in that case? JULIE: Well, the empty value would be… let's say you have maybes. Let's say you have some kind of maybe thing, so you're parser is going to return maybe this thing, maybe whatever you're parsing. Like maybe string. CHARLES: Yeah, yeah. JULIE: So, it's going to return a maybe string. So well, nothing would be the empty. CHARLES: Okay. JULIE: But nothing is like the zero because it's a disjunction, logical OR. So, only when you have two nothings will you get back a nothing. Otherwise, it will take the first thing that it finds. CHARLES: Okay. I see. JULIE: Yeah. So, the identity then is the nothing, like false is the identity for disjunction. CHARLES: Mmhmm. Okay. JULIE: Yeah. CHARLES: [Inaudible] JULIE: Yeah. If you have nothing or this other thing, then you return this other thing. Then you return the maybe string. If you have two nothings, then you get in fact nothing. Your parsing has failed. CHARLES: Right, because you've got nothing. JULIE: Because you've got nothing. There was nothing to give you back. CHARLES: So, you concatenated all of the things together and you ended up with nothing. JULIE: Right, because there was nothing there. CHARLES: Right. [Laughs] JULIE: You found nothing. So, it's useful when you've got some possibilities that could be present and you just want to keep parsing until you find the first one that matches. And then it'll just return whatever. It'll just parse the first thing that it matches on. CHARLES: Okay, okay. JULIE: Does that make sense? CHARLES: Yeah. No, I think it makes sense. JULIE: I'm not sure. Because I feel like I kind of went down a rabbit hole there. [Laughs] CHARLES: Yeah. [Laughs] No, no. I think it makes sense. And as a quick aside, I think… so, I was, when we were talking about min and max, are min and max also like a semiring? Because negative infinity is the annihilator of min and it's the identity of max. and positive infinity is the annihilator of max but it's the identity of min. JULIE: I guess. I don't really think of min and max as having identities. Is that how [inaudible]? CHARLES: I'm just, I don't know. Well, I think if you have negative infinity and you max it with anything, you're going to get the anything, right? Negative infinity max one is one. Negative infinity/minus a billion is minus a billion. JULIE: Yeah, okay. CHARLES: I don't know. Just off the cuff. I'm just trying to… annihilators sound cool. And so… [Laughter] CHARLES: And so I'm like, I'm trying to find annihilators. JULIE: Yeah, they are cool. CHARLES: [Laughs] JULIE: One of my friends on Twitter was just talking about how he used the intuition at least of a semiring at work because he had this sort of monoid to concatenate schedules. So, he's got all these different schedules and he's got this kind of monoid to concatenate them, to merge the schedules together. But then he's got this one schedule that is special. And whenever something is in this schedule, it needs to hard override every other schedule. CHARLES: Right. JULIE: And so, that was like the annihilator. So, he was thinking of it as a semiring, because that hard override schedule is like the annihilator of all the other schedules. CHARLES: Yeah. JULIE: If anything else exists on this day or whatever, then it'd just get a hard override. So, there's a real world use. [Laughs] CHARLES: Yeah, a real world example. That's the thing that I'm finding, is that all these really very crystalline abstractions, they still play out very well I think in the real world. And they're useful as a took in terms of casting a net over a problem. Because you're like… when I'm faced with something new, I'm like “Well, let's see. Can I make it a functor?” And if I can, then I've unlocked all these goodies. I've unlocked every single composition pattern that works with functor. JULIE: Right, right. CHARLES: And it's like sometimes it fits. It almost feels like when you're working on something at home and you've got some bolt and you're trying on different diameters. So you're like, “Oh, is it 15 millimeter? Is it 8 millimeter?” JULIE: Right. [Laughs] CHARLES: “Like no, okay. Maybe it'll work with this.” But then when it clicks, then you can really ratchet with some serious torque. JULIE: Right, right. Yeah. CHARLES: So, yeah. Definitely trying to look for semirings [Laughs] is definitely beyond my [can] at this point. But I hope to get there where it can be like, if it's a fit, it's a fit. That's awesome. JULIE: Right. Yeah, it's kind of beyond my can too. Semirings are still a little bit new for me and I can't say that I find them in the wild as it were, as often as monoids or something. But I think it just takes seeing some concrete examples. So, now you know this idea exists. If you just have some concrete examples of it, then over time you develop that intuition, right? CHARLES: Right. JULIE: Like “Okay, I've seen this pattern before.” [Chuckles] CHARLES: yeah. Basically, every time now I want to fold a list, or like in JavaScript, any time you want to reduce something I'm like “There's a monoid here that I'm not seeing. Let me look for it.” JULIE: Yeah. Oh, that's cool, yeah. CHARLES: Because like, that's basically, most of the time you're doing a reduce, then like I said that's the terminology for fold in JavaScript, is you start with some reducible thing. Then you have an initial value and a function to actually concatenate two things together. JULIE: Right. CHARLES: And so, usually that initial state, that's your identity. And then that function is just your concat function from your monoid. And so, usually anytime I do a reduce, there's the three pieces. Boom. Identity value, concatenation function, it's usually right there. And so, that's the way I've found of extracting these things, is I'm very suspicious every time I'm tempted to… JULIE: [Laughs] CHARLES: A fold. I'm like “Hmm. Where's the monoid I'm missing? Is it [under the] couch?” Like, where is it? [Laughs] Because it just, it cleans it up and it makes it so much more concise. JULIE: Oh yeah, that's awesome. CHARLES: So anyhow. JULIE: Have we totally lost Elrick? ELRICK: Nope, I'm still here. JULIE: Okay. [Laughs] ELRICK: I'm sitting in and listening to you two break down these complex topics is really good. Because you guys break them down to a level where it's consumable by people that barely understand it. So, I'm just sitting here just soaking everything in like “Oh, that's awesome.” Taking notes. Yes, okay, okay. [Laughter] JULIE: Cool. ELRICK: So, I'm like riding the train in the back just hanging out, feeling the cool breeze while you guys just pull the train ahead in… [Laughter] ELRICK: In the engine department, you know? It's awesome. CHARLES: Yeah. ELRICK: I don't know if they're related. But you were talking about semirings and I heard of semigroups or semigroups. I have no idea if those two things are related. Are they related or [inaudible]? JULIE: They're kind of related. So, a semigroup is like a monoid but doesn't have an identity value. CHARLES: What is an example of a semigroup out there in the wild? Because every time I find a semigroup, I feel like it's actually a monoid. JULIE: Well, you know I feel like that a lot, too. We do have a data type in Haskell that is a non-empty list. So, there is no empty list CHARLES: Ah, right. Okay. JULIE: So then you can concatenate those lists, but there's never an identity value for it. CHARLES: I see. JULIE: Yeah. So, that's a case. There's actually a lot of comparison functions, greater than and less than. I think those are semigroups because they're binary, they're associative, but they don't have an identity value. Like if you're comparing two numbers, there's not really an identity value there. CHARLES: Right. Well, would the negative infinity work there? Let's see. Like, negative infinity greater than anything would be the anything. Well, okay wait. But greater than, that takes numbers and yields a boolean, right? JULIE: Yeah, CHARLES: Right. So, it couldn't be… could it be a semigroup? Don't semigroups have to… Doesn't the [inaudible] function have to yield the same type as the operands? JULIE: Yes. CHARLES: But a non-empty list, that's a good one. Sometimes it's basically not valid for you to have a list that doesn't have any elements, right? Because it's like the null value or the empty value and it could be like a shopping cart on Amazon. You can't have a shopping cart without at least something in it. JULIE: Right. CHARLES: Or, you can't check out without something. So, you might want to say like the shopping cart that I'm going to check out is a non-empty list. And so, you can put two non-empty lists together. But yeah, there's no value you can mash together, you can concat with anything, that isn't empty. JULIE: Right. CHARLES: So, I guess going back to your question Elrick, I don't know if it's related to semiring. But semigroup is just, it's like one-half of monoid. It's the part that concats two values together. JULIE: Right. Well, yeah. And so, it's supposed to be half a group, right? But I don't remember… CHARLES: [Laughs] JULIE: [Inaudible] all of the group stuff is, all the stuff that these types have to have to be a group. And similarly, I forget what the difference between semiring and ring is. [Chuckles] Because a ring and a group I know are not the same thing. But I forget what the difference is, too. So, I kind of got a handle on what semigroups are, and I know all my Haskell friends are going to, when they hear this podcast they're going to tweet all these examples of semigroups at me, especially my coauthor for ‘Joy of Haskell', Chris Martin. He's really into semigroups. And so, I know he's going to be very disappointed in my inability to think… [Laughter] JULIE: To think of any good examples. But it's not something that I find myself using a lot, whereas semirings are something that I have started noticing a little bit more often. So, how a monoid relates to a group is something that I can't remember off the top of my head. And I know how semirings relate to monoids, but how monoids then relate to rings and groups, I can't really remember. And so, these things are sort of all related. But the relation is not something I can spill out off the top of my head. Sorry. [Laughs] CHARLES: No, It's no worries. You know, I feel like… ELRICK: It's all good. CHARLES: What's funny is I feel like having these discussions is exactly like the discussions people have with any framework of using one that we use a lot, which is EmberJS. But if you could do with React or something, it's like, how does the model relate to the controller, relate to the router, relate to the middleware, relate to the services? You just have these things, these moving parts that fit together. And part of… I feel like exploring this space is really, absolutely no different than exploring any other software framework where you just have these things, these cooperating concepts, and they do click together. But you just have to map out the space in your head. JULIE: Yeah. This is going to sound stupid because everybody thinks that because I know Haskell I must know all these other things. But I just had to ask people to recommend me a book that could explain the relationship of HTML and CSS, because that was completely opaque to me. CHARLES: [Laughs] Yeah. JULIE: I've been involved in the making now of several websites because of the books and stuff like that. And I have a blog. It's not WordPress or anything. I did that sort of myself. So, I've done a little bit with that. But CSS is really terrifying. And… CHARLES: Right. Like query selectors, rules, properties. JULIE: Yeah. ELRICK: [Laughs] CHARLES: Again, might as well be groups and semigroups and monoids, right? JULIE: Right, right. ELRICK: Yeah. CHARLES: [Laughs] ELRICK: That is really interesting. [Chuckles] I've never heard anyone make that comparison before. But it's totally true, now that I'm thinking about it. JULIE: Yeah, yeah. CHARLES: Yeah. In the tech world we are so steeped in our own jargon that we could be… we can reject one set of jargon and be totally fine with another set. Or be like, suspicious of one set of concepts working together and be totally fine with these other designations which are somewhat arbitrary but they work. JULIE: Right. CHARLES: So, people use them. JULIE: So, it's like what you've gotten used to and what you're familiar with and that seems normal and natural to you. [Chuckles] So, the Haskell stuff, most of it seems normal and natural to me. And then I don't understand HTML and CSS. So, I bought a book. [Laughter] CHARLES: Learning HTML and CSS from first principles. JULIE: Yes, yeah. I just wanted to understand. I could tell that they do relate to each other, that there is some way that they click together. I can tell that by banging my head against them repeatedly. But I didn't really understand how, and so yeah. So, i've been reading this book to [Laughs] [learn] HTML and CSS and how they relate together. That's so important, just figuring out how things relate to each other, you know? CHARLES: Yeah. ELRICK: Yeah. That is very true. JULIE: Yeah. ELRICK: We can trade. I can teach you HTML and CSS and you can teach me Haskell. JULIE: Absolutely. ELRICK: [Laughs] CHARLES: There you go JULIE: [Laughs] ELRICK: Because I'm like, “Ooh.” I'm like, “Oh, CSS. Great. No problem.” [Laughter] ELRICK: Haskell, I'm like “Oh, I don't know.” JULIE: Yeah. CHARLES: Yeah. ELRICK: [Laughs] CHARLES: No, it's amazing [inaudible] CSS. ELRICK: Yeah. CHARLES: It is, it's a complicated system. And it's actually, it's in many ways, it's actually a pretty… it's a pretty functional system, CSS is at least. The DOM APIs are very much imperative and about mutable state. But CSS is basically yeah, completely declarative. JULIE: Right. CHARLES: Completely immutable. And yeah, the workings of the interpreter are a mystery. [Laughs] ELRICK: Yup. JULIE: YEs. And you know, for the Joy of Haskell website we use Bootstrap. And so, there was just like… there's all this magic, you know? [Laughs] ELRICK: Oh, yeah. CHARLES: Yeah. JULIE: Oh look, if I just change this little thing, suddenly it's perfectly responsive and mobile. Cool. [Laughter] JULIE: I don't know how it's doing this, but this is great. [Laughs] CHARLES: Yeah. Oh, yeah. It's an infinite space. And yeah, people forget what is so easy and intuitive is not and that there's actually a lot of learning that happened there that they're just taking for granted. JULIE: I think so many people start from HTML and CSS. That's one of their first introductions to programming, or JavaScript or some combination of all three of those. And so, to them the idea that you would be learning Haskell first and then coming around and being like “Oaky, I have to figure out HTML,” that [seems very] strange, right? [Laughter] CHARLES: Yeah. Well, definitely probably stepping into bizarro world. JULIE: And I went backwards. But [Laughs] CHARLES: Yeah. JULIE: Not that it's backwards in terms of… just backwards in terms of the normal way, progression of [inaudible] CHARLES: Yeah. It's definitely the back door. Like coming in through the catering kitchen or something. JULIE: Yes. CHARLES: Instead of the front door. Because you know the browser, you can just open up the Dev Tools and there you are. JULIE: Exactly, yeah. CHARLES: The level of accessibility is pretty astounding. And so, I think t's why it's one of the most popular avenues. JULIE: Oh, definitely. Yeah. ELRICK: It's the back door probably for web development but not the back door for programming in general. JULIE: Mm, yeah. Yeah. CHARLES: Yeah. It seems like Haskell programming has really started taking off and that the ecosystem is starting to get some of the trappings of a really less fricative developer experience in terms of the package management and a command line experience and being able to not make all of the tiny little decisions that need to be made before you're actually writing ‘hello world'. JULIE: Right. ELRICK: Interesting. Haskell has a package manager now? CHARLES: Oh, it has for a while. ELRICK: Oh, really? What is it called? I have no idea? Do you know the name off the top of your head? CHARLES: So, I actually, I'm not that familiar with the ecosystem other than every time I try it out. So I definitely will defer this question to you, Julie. JULIE: This is going to be a dumb question, I guess. What do we mean by package manager? CHARLES: So, in JavaScript, we have npm. The concept of these packages. It's code that you can download, a module that you can import, basically import symbols from. And Ruby has RubyGems. And Python has pip. JULIE: Okay, okay. CHARLES: Emacs has Emacs Packages. And usually, there's some repository and people could publish to them and you can specify dependencies. JULIE: Right, yeah. Okay, so we have a few things. Hackage is sort of the main package repository. And then we have another one called Stackage and the packages that are in Stackage are all guaranteed to work with each other. CHARLES: Mm, okay. JULIE: So, on Hackage, some of the packages that are on Hackage are not really maintained or they only work with some old versions of dependencies and stuff like that, so the people who made Stackage were like “well, if we had this set of packages that were all guaranteed to work together, the dependencies were all kept updated and they all can be made to work together, then that would be really convenient.” And then we have Cabal and we have Stack are the main… and a lot of people use Nix for the same purpose that you would use Cabal or Stack for building projects and importing dependencies and all of that. CHARLES: Right. So, Cabal and Stack would be roughly equivalent then to the way we use Yarn or JavaScript and Bundler in Ruby. You're solving the equation for, here's my root set of dependencies. Go out and solve for the set of packages that satisfy. Give me at least one solution and then download those packages and [you can] run them. JULIE: Yeah, yeah. Right, so managing your dependencies and building your project. Because Haskell's compiled, so you've got to build things. And so yeah, we have both of those. CHARLES: And now there's like web frameworks and REST frameworks. JULIE: Oh there are, yeah. We have… CHARLES: All kinds of stuff now. JULIE: We had this big proliferation of web frameworks lately. And I guess some of them are very good. I don't really do web development. But the people I know who do web development in Haskell say that some of these are very good. Yesod is supposed to be very good. Servant is sort of the new hotness. And I haven't used Servant at all though, so don't ask me questions about it. [Laughter] JULIE: But yeah, we have several big web frameworks now. There are still some probably big holes in the Haskell ecosystem in terms of what people want to see. So, that's one thing that people complain about Haskell for, is that we don't have some of the libraries they'd like to see. I'd like to see something… I would really like to see in Haskell something along the lines of like NLTK from Python. CHARLES: What is that? JULIE: Natural language toolkit. CHARLES: Oh, okay. JULIE: So yeah, Python has this… CHARLES: Yeah, Python's got all the nice science things. JULIE: They really do. And Haskell has some natural language processing libraries available but nothing along the lines of, nothing as big or easy to use and stuff as NLTK yet. So, I'd really like to see that hole get filled a little bit better. And you know… CHARLES: Well, there you go. If anyone out there is seeking fame and fortune in the Haskell community. JULIE: That's actually why I started learning Python, was just so that I could figure out NLTK well enough to start writing it in Haskell. [Laughter] JULIE: So, that's sort of my ambitious long-term project. We'll see how that goes. [Laughs] CHARLES: Nice. Before we wrap up, is there anything going on, coming up, that you want to give a shoutout to or mention or just anything exciting in general? JULIE: Yeah, so on March 30th I'm going to be giving a talk at lambda-squared which is going to be in Knoxville and is a new conference. I think it's just a single-day conference and I'm going to be giving a talk about functors. So, I'm going to try to get through all the exciting varieties of functors in a 50-minute talk. CHARLES: Ooh. JULIE: So, we'll see how that goes. Yeah. And I am still working with Chris Martin on ‘The Joy of Haskell' which should be finished this year, sometime. I'm not going to… [Laughter] JULIE: Give any more specific deadline than that. And in the process of writing Joy of Haskell, I was telling him about some things that, some things that I think are really difficult. Like in my experience, teaching Haskell some places where I find people have the biggest stumbling blocks. And I said, “What if we could do a beginner video course where instead of throwing all of these things at people at once, we separated them out?” And so, you can just worry about this set of stumbling blocks at one time and then later we can talk about this set of stumbling blocks. And so, we're doing… we're going to start a video course, a beginner Haskell video course. I think we'll be starting later this month. So, I'm pretty excited… CHARLES: Nice. JULIE: About that. Yeah. CHARLES: Yeah, I know a lot of people learn really, really well from videos. There's just some… JULIE: Yeah. [Inaudible] for me, so I'm a little nervous. But [Laughs] CHARLES: Yeah, especially if you can do… are you going to be doing live coding examples? Building out things with folks? JULIE: Yeah. CHARLES: Yeah. Well, you just needn't look no further than the popular things like RailsCasts and some of the… yeah, there's just so many good video content out there. Yeah, we'll definitely be looking for the. JULIE: Cool. CHARLIE: Alright. Well, thank you so much, Julie, for coming on. JULIE: Well, thank you for having me on. Sorry I went down some… I went kind of down some rabbit holes. Sorry about that. [Laughs] CHARLES: You know what? You go down the rabbit holes, we spend time walking around the rabbit holes. JULIE: [Laughs] CHARLES: There's something for everybody. So… [Laughter] CHARLES: And ultimately we're strolling through the meadow. So, it's all good. JULIE: [Laughs] Yeah. CHARLES: Thank you too, Elrick. JULIE: It was nice talking to you guys again. CHARLES: Yeah. ELRICK: Yeah, thank you. CHARLES: If folks want to follow up with you or reach out to you, what's the best way to get in contact with you? JULIE: I'm @argumatronic on Twitter and my blog is argumatronic.com which has an email address and some other contact information for me. So, I'd love to hear questions, comments. [Laughs] Yeah. I always [inaudible]. CHARLES: Alright, fantastic. JULIE: To talk to new people. CHARLES: Alright. And if you want to get in touch with us, we are @TheFrontside on Twitter. Or you can just drop us an email at contact@frontside.io. Thanks everybody for listening. And we will see you all later.
Sam Cates: @SamCates | GE Ventures Show Notes: 02:01 - What Corporate Investing Looks Like 03:48 - Presenting Ideas For Funding 09:01 - Democratizing Venture Capital 10:17 - ICOs and Cryptocurrency 13:53 - Evaluating Companies to Fund 21:09 - Investing in Potential Competitors 24:42 - Looking For Funding as a Company 28:04 - “Mentoring” Ideas/Companies 30:07 - Monitoring/Evaluating Company Metrics 32:47 - Putting Together a Basic Business Plan 36:05 - Making Choices: Investor and Company-wise Resources: Resin.io Series A, B, C Funding Angel Investor Seed Money Initial Coin Offering (ICO) AngelList Crunchbase Fred Wilson's Blog: (AVC.com) Transcript: CHARLES: Hello everybody and welcome to The Frontside Podcast, Episode 92. My name is Charles Lowell, a developer here at The Frontside and I am your podcast host-in-training kicking it off in 2018. [Inaudible] of our first episode. We've got Elrick also joining us. Hello, Elrick. ERICK: Hey. How you doing, Charles? CHARLES: I'm doing well. I'm doing well. You having a good new year so far? ERICK: Yeah, it's great. There's a snowstorm passing through today. So, I'm going to break in the New Year shoveling. CHARLES: Let us know if we need to parachute in some shovels for you. ERICK: [Laughs] CHARLES: And then with us today, we have Sam Cates on the show who is… a lot of times we have developers on the show. He's actually a venture… what would you describe yourself as? SAM: Yeah, I'd say I'm a venture investor with GE Ventures. So, on the corporate investing side. CHARLES: Okay. Now, I didn't even know that GE actually had a corporate investing side. Is that pretty common for a large company? SAM: You know, it's becoming increasingly common. I think in 2015 there was actually a peak of activity coming from corporate venture capital groups. And I've only seen the number of firms escalate since then. Although the dollars invested stays pretty consistent. But if you look at a lot of big companies, particularly in the common tech world like Cisco, Google, Intel, they have historically had large venture firms inside of themselves. And then GE and a lot of other industrials have since followed suit. We've been at it for about five years and we see it increasingly. CHARLES: And so, have you been with them since the beginning? SAM: Yeah, just about. I've actually been with GE for about nine years now. So, I was on the operating side in a number of the industrial businesses before I joined GE Digital and then GE Ventures. And so, it was just after GE Ventures got kicked off. CHARLES: Oh, that's exciting. So, what is it… now, we actually got connected to you through one of the companies that you actually invested in. It's something that we use and we're very interested in. Why don't you tell us a little bit about what your job looks like on a day-to-day basis and what companies you invest in? SAM: Sure. I really focus a lot of my time on Internet of Things companies. So, that's a really big trend that GE has been a part of and a leader in over the past few years. And so, we spend time investing in companies that are directly working with GE or playing in similar spaces to us. And so, Elrick and I actually met at a hackathon for one of those companies. And I always like to use that as an example because it's a good one, to demonstrate the kinds of investments we make. And that's Resin.io. I know you guys have done an episode or two talking with them. But that for example was a ‘Series A' investment that we made about two years ago. And then company essentially helps developers build connected products. And so, that's something that GE cares a lot about. We had people inside the company who found the product and loved it and that's actually how we met. CHARLES: When you say ‘Series A', can you give a brief overview of what the different stages of funding of a startup might be? SAM: Yeah, yeah, certainly. So, maybe if I take a step back and answer your original question on what I do on a day-to-day basis. A lot of my job is meeting with all kinds of new companies, whether they be early stage, usually things that would be seed funding – and we'll go into what some of those things mean – all the way through the late stage which would be companies that are maybe on the border of going public or are already profitable. And so, if we go into what kinds of investors there are, I think that's probably an interesting subject to talk more about. But they're a whole wide variety. When I said ‘Series A' I just meant a company that was at what we would call the ‘Series A' stage, and the letters act just like you'd expect. So, there's ‘Series A', ‘Series B', ‘Series C', and so on. And they all, they tend to look similar at those stages in terms of sizes and progress. But there is a range, and no two company is the same. ERICK: In today's world, it's very easy for people to create a startup. They can write some code and they can either come up, get a Raspberry Pi or some microcontrollers or whatever it is, and either do an IoT startup or a software startup. Now, when you get to the point where you have an idea and you kick it off initially, how do you go about then saying, “Let me get some funding.” How do you even get funding? SAM: Sure, yeah. And to your point, there's a huge range of technologies that are making it easier to start almost any kind of company. It's a great time to be an entrepreneur, whether it be 3D printing for hardware products, all the technologies that you were mentioning, AWS, all this stuff is contributing to reducing the cost to allow companies or people to create companies. And so, once people have gone out and experimented with some of these things and built what they think is a product the market wants, often if they require more money which may be for acquiring customers through things like Facebook Ads or simply doing further product development to make sure the product is somewhere that more customers could use it, often they can't finance it just through their own revenue. And so, there are typical stages and types of investors that people go approach looking for money. ERICK: Okay. What are those Series? I remember you mentioned something like a ‘Series A' investment. So, initially when you're looking for an investment, is that where you would… category you would be in as a startup looking for investment? They would consider you a ‘Series A' startup? SAM: Well, I want to caveat and just say every company is different. So, I see companies that… ERICK: Gotcha. SAM: Start out at a much later stage because they're able to bootstrap to that point. And bootstrap is the word that I use for a company that funds its own investment. They get paid by customers and they use that money to continue building the product. But if I talk about the range of types of financing a company may go for, I think the way that most people categorize this, first people often raise from friends and family or angels. And so, it's just money to get off the ground and maybe to pay the rent while you're doing some of that experimenting we were talking about. And then commonly after that is a seed round. And a seed round tends to be a little more institutional. So, it's maybe a more formal set of funds who exclusively invest in companies that are often pre-revenue but they have a product, or at least the beginnings of a product. And so, that's a really common category of investors. And then you get to ‘Series A' and the letters can escalate from there to the point where… ERICK: Gotcha. SAM: There can be some later rounds when they'd be ‘Series F' or even beyond, I guess. CHARLES: Right. So now, what are generally the terms on these? So, for my angel investments or my seed investments, I assume what distinguishes these is essentially how much ownership of the company you're getting for how much money. And those kind of, those change as the product solidifies. SAM: Yeah. CHARLES: And the potential becomes more visible. SAM: Yeah, it's a wide… and again, these are all… the venture is a world of ranges. There's a really wide difference between the two ends of any spectrum. So, I'll just talk in generalities though. So, I think the latest report that I've seen at least for an annual basis was PitchBook's 2016 report. And they were laying out some of the medians. So, for seed stage deals I believe it was something like one and a half million dollars raised was the median on a pre-money valuation of six and a half million. And that just means the company is worth, investors say the company is worth six and a half million dollars today. And we're going to give you a million and a half dollars invested at that price. CHARLES: So roughly, a sixth… they would take a sixth of the company then in return? SAM: Yeah. CHARLES: Okay. ERICK: Ah. CHARLES: Okay. ERICK: Okay. CHARLES: I see. That makes sense. So now, back to Elrick's original question. If I'm, I've got my product. Or I've got this idea. I've written some code. I've turned it into a prototype product. Maybe I'm moving through these various stages. What type of VC am I going to be looking for? How do I actually find the right type to be talking to? I guess what types are there even? SAM: Yeah. And one part… we mentioned a lot of the technologies that are making it easier to start companies. One part that also makes it easier is the proliferation of financing options, whether it be even more investors in these traditional structures we talked about like seed and A. And then there are other options that are emerging, things like you see a lot of people raising through what they call Initial Coin Offerings or ICOs. And then there are also things like AngelList which are attempting to democratize the investing process, make it more accessible. So traditionally, a lot of the seed A, B investors, they tend to be network-based, which can be a challenge for a lot of people that are maybe not in Silicon Valley or not a part of that network already. And so, one thing you can do is obviously go search databases that are on the web, things like Crunchbase. It's a free resource. It has a lot of deal history for investments that people have made. And it's a great resource for knowing, “Okay, this investor cares about these things.” And then in addition to that, there are also platforms that people can put their companies on. Like I mentioned, AngelList. And that's somewhere that you can list your company, you can meet investors, and they actually have some backend to actually support the investing process as well. CHARLES: So, there were two acronyms in there, or two specific technologies. [Chuckles] CHARLES: You talked about ICOs which I assumed that you said it was Initial Coin Offering. Not like insane clown offering. [Laughter] CHARLES: Which I would love to see. And then AngelList. So traditionally, these had been very network-based which brings to mind the capitalists of Old England or whatever where there's a bunch of people with cigars in a room and I realize it's not actually like that. What are each of these things? The AngelList and the ICOs? And how do they democratize that process? SAM: It's funny you should mention the old times. I think a good example of that is there are a lot of stories about the founding of General Electric. It's a 126-year-old company and back then it was largely, it was Thomas Edison working with I believe was JP Morgan to get it off the ground. And so, today there's still a bit of the network piece you're mentioning. But I think of AngelList as a place that you can essentially market to investors. If you think about the types of people that are on there, it's people that are looking to invest money in early stages in startups. And I'm not a big user of AngelList because I tend to be investing a little bit later. So, I really recommend anybody who's interested, just go check it out. It's I believe just Angel.co. CHARLES: And what about an ICO? SAM: So, an ICO is a more modern one. And it's kind of fraught with some concerns around regulations and transparency today. But I think since Thanksgiving there's been a massive wave of conversation about cryptocurrencies. And an ICO is essentially a way of creating your own cryptocurrency. The way I always explain to people, I love the analogy that people make around, think of it like I want to go build an amusement park. And in that amusement park, everything, rides, food, everything, is going to be denominated and payable in Sam-bucks. CHARLES: Ah, right. SAM: And… [Chuckles] And so, my options… CHARLES: [Laughs] That makes sense. SAM: Yeah. And my options are I can go to a bank and borrow money, I can go to investors and say, “Hey, give me the 10 million dollars it's going to take to build it,” or I can just go to the people in the place where I'm building it and say, “You want this amusement park to exist? Why don't you pre-buy these Sam-bucks?” And each one is going to cost a dollar today. And we create this universe of Sam-bucks and they're essentially valuable once you can use them in the park. And there are certainly exceptions. There are other versions of cryptocurrencies and other uses for them. But that's a conversation for another day. CHARLES: Ah, mm. SAM: I think that's just a good, easy way to understand it. CHARLES: Oh no, I like that. It's like, well not quite like carnival tickets. But yeah, that's something that everyone's familiar with. Same thing as the Xbox Marketplace. Very similar thing. So, the idea is you would buy a bunch of Sam-bucks… you would get them at pennies on the dollar, so to speak, today. SAM: Yeah, right. By the time it opens, maybe a hotdog would cost just one Sam-buck. CHARLES: Right. SAM: Whereas, when it's coming in, we'd have to spend five dollars to get that one Sam-buck. Right, the idea being those people who got in early will be rewarded. And you can see it's like a further extension of a Kickstarter or something else that you're allowing people to pre-buy into a network. CHARLES: Right. Right, okay. I can see that. ERICK: That's very interesting. [Laughs] CHARLES: And so, it's got a range of options too, because if you're really interested in the services you can go ahead and spend them on the services and get a lot of value that way or you can actually trade for someone who does want the services if you don't. SAM: I think that's exactly right. And it's just, the one that I think I would just caveat is there is a huge amount of concern at the moment, and maybe concern is too strong a word, but uncertainty around one, what are the value of these coins, these tokens? And two, how will governments react to something that looks potentially like a security or a currency? And so, that's something that still is being worked through. And even though they haven't figured that out there's still a massive amount of money being raised through these ICOs. CHARLES: [Laughs] So, it does beg the question. Why is a cryptocurrency necessary? Why not just use Xbox Marketplace points? Why not just say, “Here are Sam-bucks.” SAM: [Chuckles] CHARLES: And there's a row in my database. [Laughter] CHARLES: That's your balance of Sam-bucks. SAM: So, I think we're about to get way beyond the [inaudible] [Laughter] SAM: But I think the argument would be that some of these things are better decentralized. So in my example, you're right. That might just make more sense. But I think there are some examples around cryptocurrencies that are supporting a network of decentralized services where a centralized database historically was inconvenient or didn't provide the amount of transparency that people were looking for. CHARLES: Right, right. SAM: And so, that's a topic for a whole other podcast. CHARLES: Yeah, right. No, it makes sense. SAM: [Laughs] CHARLES: I think it's a matter of scale, right? If you're going to be just buying services but if you're going to have secondary markets where you're trading in this currency, I can see that. So, let's… [Chuckles] We'll reel that back in. SAM: [Chuckles] CHARLES: And ask a question that occurred to me. So now, we talked about your day-to-day. What exactly, when you're looking at a company to basically give money to, what are you looking for? What are the things you're like, “Oh man, I want to throw dollars at this company,” versus, “Mm. I'm going to keep them and give them some feedback and send them on their way.” SAM: There's always a set of factors that we evaluate. And I think the waiting is probably different for different types of investors. And then there's I'd say for me as a corporate VC being a part of GE, there's an extra lens which is, how is this relevant to GE? What does it mean for GE to be an investor? But if I think about just the kind of general industry lines it's: team is a really big one. So, who's building this company? Do I believe in their ability to reach this vision that they're laying out for me? Another one would be technology. What have they actually built? Is that hard to build? Do the things they want to build in the future, will those be hard to build? And do they have the skills and the people to do it? Then their technology, maybe an extension of that would be intellectual property. And besides intellectual property, just defensibility of a business in general. So then, you start thinking about, can somebody else just come along and to the same thing? Because if so, then maybe there's not a strong advantage in what the company has done so far. And then lastly, it's also just traction. How far along are they? How much have they proven the ability to execute on the plan that they're laying out? CHARLES: Right. ERICK: So, you're a corporate investor. So, there's other types of investor like an institutional VC? What are the differences between an institutional VC and a corporate VC and the other types of VC? Potentially what they'd be looking for, in terms of what they wanted best. SAM: Yeah. So, I think generally I categorize investors as institutional or corporate. And corporate [inaudible]… ERICK: Yeah. SAM: Corporate or strategic. And then there are people who exist on a spectrum there. But generally, an institutional means this is a group that is raising money from a set of limited partners who are the people who invest in the fund that are pension funds or wealthy individuals. They're large pools of institutional capital and their pure purpose is to earn return. And they may have a certain focus because they believe in this part of the market, or they like this kind of company or the stage of company. But essentially, their job is to return more money to the limited partners of that fund that were put in. That's their role in the world. And then on the corporate side, if we go the most extreme version of corporate VC, this is a group that is a part of a larger corporate. They're investing that company's money. So, in this case for me it's GE. I'm investing GE's money into these startups. And that means that I only have a single backer being GE. And I also maybe have a different lens, because my purpose is one, to earn financial return. I want to go out and I want to find good companies. I want to earn returns just like the other institutional venture capitalists. But I also have the goal of, and the strategic goal may differ by company, but for me it's about how can I help GE advance? How can I help GE understand a market? And how can GE be helpful to this company in achieving their goals? And so, for each company we use that lens as well, as a corporate. CHARLES: What I'm hearing is that you want to invest… I guess the thing is you can experience return that's not just cash. It's not just dollars. You'll experience return in raising the ocean of the business that GE is in, right? So… SAM: You said it much better than I did. [Laughter] CHARLES: Well, it's all… paraphrasing is actually easy. [Laughter] ERICK: Oh, yeah. SAM: An important skill. CHARLES: That makes a lot of sense. So, the question I have then is, you said you were looking for companies that kind of swim in a specific ocean. And each company is farther along. Are you usually finding this company I want to work with, like you are going out and finding them? Or they're coming to you looking for investment? Or is it really just, depends. SAM: So, we call that part of the process sourcing, sourcing investments. And they come from all over. So for us, there are a few different ways. One is we tend to be thesis-driven. Meaning we go out and we say, the world is changing in this way and therefore we're interested in this kind of company. And so, we'll proactively go out and research. We're also, I mentioned, a little later stage. So, I don't tend to do seed investments. I tend to do ‘Series A' and more often ‘Series B' and later. So, companies that have often already raised a seed round or raised a ‘Series A' round. So, I can actually search databases to say, “Okay, in the last two years who has raised a seed round or ‘Series A' round and these other things I'm looking for whether it be location or tied to investors or other things.” So, that's one way of being proactive is saying I want to go out and look for companies in this space that look like this. And that can be either like I mentioned, desktop research like searching the web, searching databases. Or it can be just going to conferences, right? So, on thing we spend a lot of time on in the IoT world is artificial intelligence and machine learning. It's been a big, big topic over the last year that a lot of people have invested in. So, we may go to different conferences that focus on that topic, meet lots of people that are working on it. Some companies, some individuals that are either investing in or advising their companies. And we'll talk to them. What companies are rising out of that space that we should be looking at? What technologies are changing in that space that we should be thinking about? And just trying to get smarter so that we can make the right investments and help the right companies find their way to work with GE and make our products better and help them advance their own enterprise. CHARLES: Are you investing with a mind that eventually GE might acquire this company and integrate it into GE itself? Or is it really just, “Hey, we're just going to take a part of it. We're going to have maybe a seat on the board to be able to steer a little bit. But we're pretty much going to let it be its own thing with its own autonomy and go where it was and just benefit through those secondary and tertiary effects.” SAM: Yeah, acquisitions from our portfolio by GE happened. But they're certainly not the explicit goal or our focus. I know we've had one, maybe two of our portfolio companies acquired by GE, one that I was directly working with called Bit Stew. So, we made the investment in the company. It was with the goal of using their data management platform for a lot of our applications. And at some point in working with GE and GE Digital, they decided, you know, this would make sense to be a part of GE. That wasn't why we made the investment. But it did end up being acquired by GE. And I know the team is doing really well. And it's been at GE for about a year now. So, it does happen. But when I said one or two, that's versus a portfolio of a hundred plus companies. CHARLES: Right. SAM: Since we started investing. And so, that's not what we're looking to do every time. Much more often it's about again, how does the company make GE more competitive and a better company, a better place to work. And then how do we help them advance their goals? Whether it be bringing them developers, or finding them other routes to market, or just being a customer. CHARLES: Right. SAM: So, that's really how we think about strategic value. There's a lot of different ways to create it. CHARLES: Yeah, I'm curious. Because it seems like also in a lot of these companies you're investing in potential competitors. Extensively you're operating if not in the exact same market, maybe very similar markets. There's a little bit of overlap. And so, you're kind of investing in potential competitors, right? So, where's the balance of here we're funding our competitors versus we're going to move into these markets ourselves. SAM: Yeah, and funding of “competitors” can happen. I think that we talk about that more in theory and say, “Oh sure, we'd be willing to fund a company that's out disrupting the space that we're playing in.” And we do that. It's rare that you see startups that are directly head-on competing with much more established companies like GE or other industrials or even other consumer companies. They don't take these companies head-on because that's not a way that startups have been successful in the past, right? We talk much more about disruption and saying, how is this company doing something that may indirectly compete with GE? So, you think about things like, for anybody that's not familiar with GE… actually, a lot of people associate us with our appliances which we actually don't manufacture anymore. That's [inaudible]. [Chuckles] SAM: We sold that business a few years ago. Almost everything we sell is like big, heavy industrial equipment. So, we sell aircraft engines, locomotives. We sell gas turbines, wind turbines. So, here and there a couple of things that do power generation. One trend that's affecting that industry is distributed generation of energy, energy storage. And those are parts of the market that are a less significant part of GE's business than say, heavy-duty gas turbines that sit in a power plant and generate a massive amount of power. And so, if you look at that and say, “Wow, GE Ventures is out funding storage companies. Does that mean they're funding competitors?” Well, it means that we're funding innovation that may disrupt the future of our business, but that's part of being a VC and that's part of the value that GE Ventures brings to GE. CHARLES: Right. SAM: We're out there looking at markets before they're large enough or in scope for GE. CHARLES: Mmhmm, right. And so, yes you're disrupting the space but then you're going to be a part of that disruption and have strong connections to those markets if you need to actually migrate your business completely over to them. That's kind of what I'm hearing. SAM: Yeah, absolutely. Better to disrupt yourself, right? CHARLES: [Chuckles] SAM: And be a part of the ecosystem in the future because I think the future happens with or without you. And it's really key that we get out in front of it and a part of that, a part of that discussion, a part of that process. CHARLES: And so now, you've been saying that this is, GE, this has been pretty explosive? There's a lot more happening through GE Ventures. There's a lot more happening in other companies globally, having these corporate ventures. Where do you think the balance is going to lie to say, “Hey,” I'm just going to throw out some numbers, just for theory here, it's like, “10% of our business is essentially this distributed network of semi-autonomous or mostly autonomous startups. And then we have our core business.” Does that stabilize at 50/50? Does it stabilize at 75% the other way with GE essentially becoming a capital management company? Or is it somewhere in the middle? SAM: So, GE Ventures will never be a meaningful part of GE's revenue, a meaningful part of its business as a percentage. The overall venture industry is full of funds that are on the order of like, bigger funds are on the order of, in the billions. The single-digit billions. And GE itself is a much, much larger company. Well over a hundred billion dollars in enterprise value. So, I think GE Ventures will always be a small part of the company financially. And the impact will be largely felt through how we help the rest of GE navigate the future. ERICK: You said that sometimes you go and look for companies, startups to invest in or sometimes startups come to you or come to a VC looking for funding. Now, I'm a developer or a startup founder. And I'm going to look for funding. What are some of the mistakes or pitfalls that you see that startup founders or people with an idea fall into when looking for funding that you can help them avoid? SAM: Yeah, and we do see companies that come to us. So, I mentioned a lot about how I go out looking for companies based on a thesis or a set of relevant factors or relevant things for GE. But we do have a number of inbound requests. People know some of the bigger VC brands. They know GE the big company. So, we do get inbound interest and we also get referrals from networks of VCs and some are employees and other things. But for the companies that are seeking us out, the ones that are going out looking for funding, there are some things that are really well-known in Silicon Valley and other places, or you could research online and find, but may not be obvious at first. And so, I think the first one is, who are you talking to? What investors are you seeking out? Depending on what stage you're at, what kind of business you're in, you have to understand what the landscape of potential investors are and which ones might be interested in a company like yours. So, I think there are tons of good mentors that can help people navigate that. Maybe less commonly outside of Silicon Valley, in Boston, New York, in the places where you have traditional venture ecosystems. But you see a ton of resources available online whether it be things like Fred Wilson's blog, AVC.com, or Crunchbase, TechCrunch. You can read and understand and from headlines tell what people care about. And I think that's fundamentally a really important first step. You don't want to waste an hour talking to somebody who will never… this is somebody that invests in really late-stage growth equity companies and I'm coming to them for my first investment. That's not going to work. So, I think finding the right people, step one. I think when you're going through the process of pitching and talking about your business, the pitfalls are all about understanding the strengths and weaknesses of your business and where you are today. And so, for every company, that's different. But I think just being open and honest versus glossing over a lot of the risks, these are all really risky companies. If they were easy, then you'd have a lot more competition. And so… [Laughter] SAM: I think that's one thing that I see, too. You have some company that comes in and say, “Look, here are the parts I've figured out and here are the parts I still have to figure out.” And that's a really good conversation to have. There are other companies where they say, “Look, we've figured the whole thing out. We just want you to give us some money.” And I don't think a lot of investors necessarily buy into that. And certainly, there are investors of every stripe. So, I may be speaking too broadly. But I think that's a really important part of the venture investment process, right? You're looking not just for money but also for counsel and for somebody that you're going to work with over the next, sometimes seven years or longer. CHARLES: Yeah. SAM: [Inaudible] going to be on your board and participating. So, it's a really important part. CHARLES: So, you're looking, you're actually looking not necessarily for all the answers but you're looking for the questions that they're asking, too. SAM: Yeah, absolutely. And demonstrating they understand the ins and outs of the business. And that they have the capacity to carry this onto that next stage and hopefully beyond. CHARLES: Mmhmm. So, now you said something that caught my interest there that you work with some people sometimes seven years. You enter into these long relationships. Do you generally ever do any type of, I want to say almost like… mentoring might be too strong of a word, but in the pre-investment, in other words before you actually invest in a company, do you ever work with them to prepare them for investment to say, “Hey, I think there's potential here. Work on A, B, and C and then let's talk.” And you have this image in your mind. You go, you pitch to an investor, and it's either thumbs up or it's like thumbs down and you never talk to them again. Versus, is there some ground in between where there's a conversation that evolves that eventually ends up in an investment being made? SAM: Absolutely. I think one of the parts of this industry is even when I'm not an investor in a company, I may know a company and say, “It's not a fit for me for GE Ventures but I still think that we can provide help.” It's one of the things I love about tech and about venture in general, is that people are often willing to pitch in, even when they don't have a direct financial incentive. And so, I see that a lot whether it's helping a company where we've met them and we later see an opportunity and say, “Oh, you should go and talk to this company or that company.” And then often, we may see a company that's pitching us ahead of where we would typically invest. Maybe they're looking for a ‘Series A' but given the space that they're in or what we're doing at the moment, it may not be the right time for us. And so, we'll continue to track along and keep up and get updates. Some companies do a really good job of actually providing proactive updates and sending out monthly or quarterly reports to investors they've met with before. I think there's a wide range of ways that founders do this. But it is a really good way to keep people interested in the prize. And then when you come back and say, “Hey, now I'm out raising my ‘Series B',” that's not a surprise. I knew that you were hitting these milestones, that you were doing everything you said you were going to do. And you've demonstrated a level of credibility that really adds to the pitch that you made the first time around. ERLICK: You said something, metrics. So, a venture capitalist, after they make an investment, what are some of the expectations that they may hold this startup that they just invested in… what are those expectations that they may hold them accountable for? Or those metrics that they'll be looking at? SAM: Yeah, so I think some of the really high-level ones that are common across businesses, generally growth is a really big one. So, I almost said revenue. But I wanted to caveat… [Laughter] SAM: And say growth could mean different things. It could mean number of developers. It could mean number of downloads if you're an app. It depends on what the business is. But I think growth is a huge one. Growth is a really important, that top line, that's what's going to drive a lot of the value in the business. And then below that, demonstrating that you can hit the milestones around things like margins. So, how profitable is each unit you're selling? Or how profitable is each customer? And lastly, how are you doing managing your spend? So, that's great that you're earning the right amount of money for each customer, but are you doing it by… do you have a massive number of employees and offices and all the things that are too expensive to allow you to use your money wisely as you reach the next stage? And so, those are the big milestones. It's really just growth, margins, and operating cost or burn rate as we call it. CHARLES: Mmhmm. So, that sounds like a lot of work to actually evaluate these companies. Do you do your due diligence once you've already moved in pretty solidly into the process? SAM: Yeah, these processes can move really fast. And depending on the timing, generally it's, you jump in, you learn as much as you can, as fast as you can, and you make a decision so the company can move on. I'll say there's a lot of work that goes into considering and deciding which companies to spend more time on, both for us and for them. We don't want to waste a company's time evaluating, going through more meetings, if it's not a really strong candidate for us. Because they could be spending that time better with other investors who are a better fit. And I'm not going to pretend to like the evaluation part. I have a lot of respect for the amount of not just work but of a person's energy and really, their life goes into these companies. And so, I think the hard part is building the company. And so, it's hard for me to say that evaluating is a hard part. I'm trying to understand as much as I possibly can in a month or two. I'm not going to know as much about the business as the founder does. And I'll be wrong a lot. I may miss something and not understand, whether it's because I don't see the market but it's there or because I have some underlying assumption about the way things should work that they don't meet. And I think that that's something that investors have to come to grips with. You try and get as smart as you can as fast as you can, but you're not always going to get to the right answer. ERLICK: You said that it was growth, spend, and profits were some of the metrics. That is almost all of the essential components of a business plan. I remember one time, one of our previous conversations, you emphasized how important it was for companies, or even at just a simple startup, to put together a basic business plan. Is that something that you can elaborate on a little? SAM: Yeah. So, most companies show up with a pitch deck. So, they have a set of PowerPoint slides and then they have a set of materials behind that where if you go deeper into an area they may have a white paper about their technology and they may have an Excel financial model that explains why they have these expectations about what growth and margins and all those things will look like. So, there are all of those pieces that come together into a business plan. The business plan could be written or it could be that PowerPoint. But very traditionally, it's a PowerPoint or some kind of presentation that is shared in person. There's usually a version that's sent in advance to confirm that the company and the investors should meet. And then once you clear that bar, there's a deeper presentation that often you'll give to either one or a set or a whole team of investors. And you'll go through and explain why it is you think this is a good investment opportunity for them and why you'd like to work together. And then you have a discussion about whether that's a good fit, about some of the underlying assumptions, and come to either a set of next steps for the diligence or a decision that it's not the right fit, it's not the right time to take the relationship further with more diligence and that kind of stuff. ERLICK: Yeah, because I see… well, I know a few people that have startup ideas and they kind of put the business plan on the back burner and put the actual prototype more at the forefront. They say, “Oh, we can worry about the business plan later.” [Laughs] SAM: [Chuckles] Well, I think… there's something to be said to that. There's something to be said for product and growth winning. So if you… Let's start at the early stages. If you have something that's working and that's really obvious, you may not need a… ERLICK: True. SAM: To go raise money. It all comes down to, do you have enough to get enough investors interested to raise the round that you want to raise? Because you want to have enough investors involved, enough demand, that you can be selective about who you want to work with and on what terms, right? So, what valuation and how much of the company am I giving them, and all of those things. So, if you can do all of those things with nothing but an app and one chart that shows a hockey stick of growth, that's awesome. CHARLES: You're hot. [Laughter] SAM: Often it does require much more and a much longer plan. So, even if you say, “Look, it's growing like crazy,” there's usually some set of questions behind that. So, that's great. Your free app is growing like crazy. How are you going…? [Laughter] SAM: To get paid for that? And you'll talk about that. And you'll say, “Here are the things we're planning on doing,” or here are the assumptions that we're making. And the more original, the more unique the business model is, the more discussion and explanation that may require. And that's where the business plan and a pitch deck come in handy, because it's a really good presentation aide or pre-reading to get to that answer faster. ERLICK: So, this evaluation it seems, is a two-way street. The VCs evaluating the company and also the company or the startup evaluating the VC to know whether it's going to be a good relationship. SAM: Oh, absolutely. Yeah, the best companies have choice. They have a number of investors who are interested in funding them. And certainly, that might be different at different stages or at different times, depending on what's going on in the economy and in tech and in other places. But generally, VC is a very competitive industry. I'm trying to sell my money and services as an investor versus other options that you have. And so, while it's maybe not as competitive as only one of us can buy the company like in an M&A situation. There are often more than one investor. There's still a very intense set of competition around, okay, who's going to be involved in the deal? How much money will they be able to invest? So, that's something that really can come in handy for founders. ERLICK: And what was that you just said there? M&A situation? SAM: Oh, sorry. When a company is being bought. So, when a company is being bought, it can look kind of like a fundraising process, but instead of selling a part of a company, you're selling the whole thing. And so, in that case, obviously it's a competitive situation where there's only one winner. And this is a different process. Often, the rounds that we're a part of, we're not… we're buying a minority stake just like any VC. We may be buying 5% of a company, 10% of a company. And often we're being joined by other venture investors. We really actually commonly partner with the institutional firms and they'll take a board seat. We'll invest alongside them and be an observer on the board and provide counsel. And so, it is a very competitive process. And that, while M&A is a winner-take-all, there is one buyer who is ultimately going to own this company going forward, the investing process for a venture is much more collaborative. But it is still competitive, because there can only be so many investors in one company. CHARLES: And you want to choose the right one on both… the right set. Alright. Well, I think we're running up against time. This has been a fascinating conversation into an aspect of our industry that really is providing the fuel that drives so much of this forward. So, I guess I'll close by asking you, already talked about Resin. We had them on the podcast. We love them. Are there any conferences or products that you're investing in that you feel like our audience might want to know about or anything like that? SAM: Well one, you mentioned Resin. CHARLES: Yeah. SAM: I know you guys have been a good friend to them and Elrick and I met at their hackathon. I would recommend to anybody, go try it out. It's a really cool way to play with hardware products. I am not a developer and I required a lot of help from Elrick at the hackathon. [Laughter] SAM: But at the same time, it is something that almost anybody can pull out of a box and start playing with. So, I think that's a great one. The episode you did on them were fantastic. So, I really enjoyed those ones. I'd say in general, I'm always out looking to meet new companies that are going to benefit from working with GE. I spend a lot of my time not just trying to invest but also trying to find partnerships for companies that we're looking at within GE, either selling to us or working with us. And so, if somebody thinks that there's an opportunity to do that, then I encourage them to reach out. Because I think there's a ton of opportunity. It's a really big company that really has a ton of opportunity for other partners. CHARLES: Alright. If they wanted to reach out, how would they get in touch with you? SAM: Yeah, I think maybe the best way to initially make contact, I tend to be pretty active on Twitter. So, my handle is just @SamCates. S-A-M-C-A-T-E-S. And you can also learn more through our website. If you're curious about some of the businesses I mentioned, so just GEVentures.com. And it's about to go through a whole refresh. So, go check it out. CHARLES: Alright. Well, fantastic. We will definitely look for that. And for everybody else, you can get in touch with us on Twitter at @TheFrontside or send us a line at info@frontside.io. Thank you everybody for listening. Thank so, so much Sam, for being on the podcast. SAM: Yeah, of course. It was a blast. I'm a big podcast fan and I've really enjoyed catching up on your episodes. CHARLES: Ah, and thank you Elrick, always. ERICK: It was great. SAM: Elrick, when you finish building your Raspberry Pi Battleships, I want to play. CHARLES: [Laughs] ERICK: Oh, yes. Yes. It's in the works, man. It's in progress. SAM: Alright, I'm waiting. CHARLES: Alright. Well, take it easy, everybody.
Tracy Lee: @ladyleet | ladyleet.com Ben Lesh: @benlesh | medium.com/@benlesh Show Notes: 00:50 - What is This Dot? 03:26 - The RxJS 5.5.4 Release and Characterizing RxJS 05:14 - Observable 07:06 - Operators 09:52 - Learning RxJS 11:10 - Making RxJS Functional Programming Friendly 12:52 - Lettable Operators 15:14 - Pipeline Operators 21:33 - The Concept of Mappable 23:58 - Struggles While Learning RxJS 33:09 - Documentation 36:52 - Surprising Uses of Observables 40:27 - Weird Uses of RxJS 45:25 - Announcements: WHATWG to Include Observables and RxJS 6 Resources: this.media RxJS RX Workshop Ben Lesh: Hot vs Cold Observables learnrxjs.io RxMarbles Jewelbots Transcript: CHARLES: Hello everybody and welcome to The Frontside Podcast, Episode 91. My name is Charles Lowell, a developer here at The Frontside and your podcast host-in-training. Joining me today on the podcast is Elrick Ryan. Hello, Elrick. ELRICK: Hey, what's up? CHARLES: Not much. How are you doing? ELRICK: I'm great. Very excited to have these two folks on the podcast today. I feel like I know them… CHARLES: [Laughs] ELRICK: Very well, from Twitter. CHARLES: I feel like I know them well from Twitter, too. ELRICK: [Laughs] CHARLES: But I also feel like this is a fantastic company that is doing a lot of great stuff. ELRICK: Yup. CHARLES: Also not in Twitter. It should be pointed out. We have with us Tracy Lee and Ben Lesh from This Dot company. TRACY: Hey. CHARLES: So first of all, why don't we start, for those who don't know, what exactly is This Dot? What is it that you all do and what are you hoping to accomplish? TRACY: This Dot was created about a year ago. And it was founded by myself and Taras who work on it full-time. And we have amazing people like Ben, who's also one of our co-founders, and really amazing mentors. A lot of our friends, when they refer to what we actually do, they like to call it celebrity consulting. [Laughter] TRACY: Which I think is hilarious. But it's basically core contributors of different frameworks and libraries who work with us and lend their time to mentor and consult with different companies. So, I think the beautiful part about what we're trying to do is bring together the web. And we sort of do that as well not only through consulting and trying to help people succeed, but also through This Dot Media where it's basically a big playground of JavaScripting all the things. Ben and I do Modern Web podcast together. We do RX Workshop which is RxJS training together. And Ben also has a full-time job at Google. CHARLES: What do they got you doing over there at Google? BEN: Well, I work on a project called Alkali which is an internal platform as a service built on top of Angular. That's my day job. CHARLES: So, you've been actually involved in all the major front-end frameworks, right, at some point? BEN: Yeah, yes. I got my start with Angular 1 or AngularJS now, when I was working as a web developer in Pittsburgh, Pennsylvania at a company called Aesynt which was formerly McKesson Automation. And then I was noticed by Netflix who was starting to do some Angular 1 work and they hired me to come help them. And then they decided to do Ember which is fine. And I worked on a large Ember app there. Then I worked on a couple of large React apps at Netflix. And now I'm at Google building Angular apps. CHARLES: Alright. BEN: Which is Angular 5 now, I believe. CHARLES: So, you've come the full circle. BEN: Yeah. Yeah, definitely. CHARLES: [Chuckles] I have to imagine Angular's changed a lot since you were working on it the first time. BEN: Yeah. It was completely rewritten. TRACY: I feel like Angular's the new Ember. CHARLES: Angular is the new Ember? TRACY: [Laughs] BEN: You think? TRACY: Angular is the new Ember and Vue is the new AngularJS, is basically. [Laughs] CHARLES: Okay. [Laughter] CHARLES: What's the new React then? BEN: Preact would be the React. CHARLES: Preact? Okay, or is Glimmer… BEN: [Laughs] I'm just… CHARLES: Is Glimmer the new React? BEN: Oh, sure. [Laughs] CHARLES: It's important to keep these things straight in your head. BEN: Yeah, yeah. CHARLES: Saves on confusion. TRACY: Which came first? [Chuckles] BEN: Too late. I'm already confused. CHARLES: So now, before the show you were saying that you had just, literally just released RxJS, was it 5.5.4? BEN: That's right. That's right. The patch release, yeah. CHARLES: Okay. Am I also correct in understanding that RxJS has kind of come to very front and center position in Angular? Like they've built large portions of framework around it? BEN: Yeah, it's the only dependency for Angular. It is being used in a lot of official space for Angular. For example, Angular Material's Data Table uses observables which are coming from RxJS. They've got reactive forms. The router makes use of Observable. So, the integration started kind of small which HTTPClient being written around Observable. And it's grown from there as people seem to be grabbing on and enjoying more the React programming side of things. So, it's definitely the one framework that's really embraced reactive programming outside of say, Cycle.js or something like that. CHARLES: Mmhmm. So, just to give a general background, how would you characterize RxJS? BEN: It's a library built around Observable. And Observable is a push-based primitive that gives you sets of events, really. CHARLES: Mmhmm. BEN: So, that's like Lodash for events would be a good way to put it. You can take anything that you can get pushed at you, which is pretty much value type you can imagine, and wrap it in an observable and have it pushed out of the observable. And from there, you have a set of things that you can combine. And you can concatenate them, you can filter them, you can transform them, you can combine them with other sets, and so on. So, you've got this ability to query and manipulate in a declarative way, events. CHARLES: Now, Observable is also… So, when Jay was on the podcast we were talking about Redux observable. But there was outside of the context of RxJS, it was just observables were this standalone entity. But I understand that they actually came from the RxJS project. That was the progenitor of observables even though there's talk of maybe making them part of the JavaScript spec. BEN: Yeah, that's right. That's right. So, RxJS as it stands is a reference implementation for what could land in JavaScript or what could even land in the DOM as far as an observable type. Observable itself is very primitive but RxJS has a lot of operators and optimizations and things written around Observable. That's the entire purpose of the library. CHARLES: Mmhmm. So, what kind of value-adds does it provide on top of Observable? If Observable was the primitive, what are the combinators, so to speak? BEN: Oh, right. So, similar to what Lodash would add on top of say, an iterable or arrays, you would have the same sorts of things and more inside of RxJS. So, you've got zip which you would maybe have seen in Lodash or different means of combines. Of course, map and ‘merge map' which is like a flattening sort of operation. You can concatenate them together. But you also have these time-based things. You can do debouncing or throttling of events as they're coming over in observable and you create a new observable of that. So, the value-add is the ability to compose these primitive actions. You can take on an observable and make a new observable. We call it operators. And you can use those operators to build pretty much anything you can imagine as far as an app would go. CHARLES: So, do you find that most of the time all of the operators are contained right there inside RxJS? Or if you're going to be doing reactive programming, one of your tasks is going to be defining your own operators? BEN: No, pretty much everything you'd need will be defined within RxJS. There's 60 operators or so. CHARLES: Whoa, that's a lot. BEN: It's unlikely that someone's going to come up with one. And in fact, I would say the majority of those, probably 75% of those, you can create from the other 25%. So, some of the much more primitive operators could be used… TRACY: Which is sort of what Ben did in this last release, RxJS 5…. I don't know remember when you introduced the lettable operators but you… BEN: Yeah, 5.5. TRACY: Implemented [inaudible] operators. BEN: Yeah, so a good portion of them I started implementing in terms of other operators. CHARLES: Right. So, what was that? I didn't quite catch that, Tracy. You said that, what was the operator that was introduced? TRACY: So, in one of the latest releases of RxJS, one of the more significant releases where pipeable operators were introduced, what Ben did was he went ahead and implemented a lot of operators that were currently in the library in terms of other operators, which was able to give way to reduce the size of the library from, I think it was what, 30KB bundled, gzipped, and minified, to about 30KB, which was about 60 to 70% of the operators. Right, Ben? BEN: Yeah. So, the size reduction was in part that there's a lot of factors that went into the size reduction. It would be kind of hard to pin it down to a specific operator. But I know that some of the operators like the individual operators themselves, by reimplementing reduce which is the same as doing as scan and then take last, implementing it in terms of that is going to reduce the size of it probably 90% of that one particular file. So, there's a variety of things like that that have already started and that we're going to continue to do. We didn't do it with every operator that we could have. Some operators are very, very common and consequently we want them to be as optimized as possible. For example, map. You can implement map in terms of ‘merge map' but it would be very slow to do so. It might be smaller but it would be slower. We don't want that. So, there are certain areas we're always going to try to keep fairly a hot path to optimize them as much as possible. But in other spots like reduce which is less common and isn't usually considered to be a performance bottleneck, we can cut some corners. Or ‘to array' or other things like that. CHARLES: Mmhmm. TRACY: And I think another really interesting thing is a lot of people when learning RxJS, they… it's funny because we just gave an RX Workshop course this past weekend and the people that were there just were like, “Oh, we've heard of RxJS. We think it's a cool new thing. We have no plans to implement it in real life but let's just play around with it and let me learn it.” I think as people are starting to learn RxJS, one of the things that gets them really overwhelmed is this whole idea that they're having to learn a completely new language on top of JavaScript or what operators to use. And one of our friends, Brian Troncone who is on the Learning Team, the RxJS Learning Team, he pulled up the top 15 operators that were most commonly searched on his site. And some of them were ‘switch map', ‘merge map', ‘fork join', merge, et cetera. So, you can sort of tell that even though the library has quite a few… it's funny because Ben, I think the last RX Workshop you were using pairs and you had never used it before. BEN: Yeah. TRACY: So, it's always amusing for me how many people can be on the core team but have never implemented RxJS… CHARLES: [Laughs] TRACY: A certain way. BEN: Right. Right, right, right. CHARLES: You had said one of the recent releases was about making it more friendly for functional programming. Is that a subject that we can explore? Because using observables is already pretty FP-like. BEN: What it was before is we had dot chaining. So, you would do ‘dot map' and then call a method and then you get an observable back. And then you'd say ‘dot merge' and then you'd call a method on that, and so on and so forth. Now what you have is kind of a Ramda JS style pipe function that just takes a comma-separated list of other functions that are going to act upon the observable. So, it reads pretty much the same with a little more ceremony around it I guess. But the upside is that you can develop your operators as just higher-order functions. CHARLES: Right. And you don't have to do any monkey-patching of prototypes. BEN: Exactly, exactly. CHARLES: Because actually, okay, I see. This is actually pretty exciting, I think. Because we actually ran into this problem when we were using Redux Observable where we wanted to use some operators that were used by some library but we had to basically make a pull request upstream, or fork the upstream library to include the operators so that we could use them in our application. It was really weird. BEN: Yeah. CHARLES: The reason was because it was extending the observable prototype. BEN: Yeah. And there's so many… and that's one way to add that, is you extend the observable prototype and then you override lift so you return the same type of observable everywhere. And there are so many things that lettable operators solved for us. For example… CHARLES: So, lettable operators. So, that's the word that Tracy used and you just used it. What are lettable operators? BEN: Well, I've been trying to say pipeable and get that going instead of lettable. But basically there's an operator on RxJS that's been there forever called let. And let is an operator and what you do is you give it a function. And the function gives you the source observable and you're expected to return a new observable. And the idea is that you can then write a function elsewhere that you can then compose in as though it were an operator, anywhere you want, along with your other dot-chained operators. And the realization I had a few months ago was, “Well, why don't we just make all operators like this?” And then we can use functional programming to compose them with like a reduce or whatever. And that's exactly what the lettable operators are. And that's why I started calling them lettable operators. And I kind of regret it now, because so many people are saying it and it confuses new people. Because what in the world does lettable even mean? CHARLES: Right. [Laughs] BEN: So, they are pipeable operators or functional operators. But the point is that you have a higher-order function that returns a function of a specific shape. And that function shape is, it's a function that receives an observable and returns an observable, and that's it. So, basically it's a function that transforms an observable into a new observable. That's all an operator. That's all an operator's ever been. It's just this is in a different flavor. CHARLES: Now, I'm curious. Why does it do an observable into an observable and not a stream item into an observable? Because when you're actually chaining these things together, like with a map or with a ‘flat map' or all these things, you're actually getting an individual item and then returning an observable. Well, I guess in this case of a map you're getting an item and returning an item. But like… BEN: Right, but that's not what the entire operation is. So, you've got an operation you're performing whenever you say, if you're to just even dot-chain it, you'd say ‘observable dot map'. And when you say ‘dot map', it returns a new observable. And then you say ‘dot filter' and it returns another new observable. CHARLES: Oh, gotcha, gotcha, gotcha. Okay, yeah, yeah, yeah. Yeah, yeah. BEN: So, this function just embodies that step. CHARLES: I see, I see. And isn't there some special… I feel like there's some proposal for some special JavaScript syntax to make this type of chaining? BEN: Yeah, yeah, the pipeline operator. CHARLES: Okay. BEN: I don't know. I think that's still at stage one. I don't know that it's got a lot of headway. My sources and friends that are in the TC39 seem to think that it doesn't have a lot of headway. But I really think it's important. Because if you look at… the problem is we're using a language where the most common use case is you have to build it, get the size as small as possible because you need to send it over the wire to the browser. And understandably, browsers don't want to implement every possible method they could on say, Array, right? CHARLES: Mmhmm, right. BEN: There's a proposal in for ‘flat map'. They could add zip to Array. They could add all sorts of interesting things to Array just by itself. And that's why Lodash exists, right? CHARLES: Right. BEN: Is because not everything is on Array. And then so, the onus is then put on the community to come up with these solutions and the community has to build libraries that have these constraints in size. And what stinks about that is then you have say, an older version of Lodash where you'd be like, “Okay, well it has 36 different functions in it and I'm only using 3 of them. And I have to ship them all to the browser.” CHARLES: Mmhmm. BEN: And that's not what you want. So, then we have these other solutions around tree-shaking and this and that. And the real thing is what you want is you want to be able to compose things left to right and you want to be able to have these functions that you can use on a particular type in an ad hoc way. And there's been two proposals to try to address this. One was the ‘function bind' operator, CHARLES: Mmhmm. BEN: Which is colon colon. And what that did is it said, “You can use this function as a method, as though it were a method on an object. And we'll make sure that the ‘this' inside that function comes from the instance that's on the left-hand side of colon colon.” CHARLES: Right. BEN: That had a bunch of other problems. Like there's some real debate I guess on how they would tie that down to a specific type. So, that kind of fell dead in the water even though it had made some traction. And then the pipeline operator is different. And then what it says is, “Okay, whatever is on the…” And what it looks like is a pipe and a greater than right next to each other. And whatever's on the left-hand side of that operand gets passed as the first argument to the function on the right-hand side of that operand. CHARLES: Mmhmm. BEN: And so, what that means is for the pipeable operators, instead of having to use a pipe method on observable, you can just say, “instance of observable, pipeline operator and an operator, and then pipeline operator, and then the Rx operator, and then pipeline operator and the Rx operator, and so on.” And it would just be built-in. And the reason I think that JavaScript really needs it is that means that libraries like Lodash can be written in terms of simple functions and shipped piece-meal to the browser exactly as you need them. And people would just use the pipeline operator to use them, instead of having to wrap something in a big object so you can dot-chain things together or come up with your own functional pipe thing like RxJS had to. CHARLES: Right. Because it seems it happens again and again, right? Lodash, RxJS, jQuery. You just see this pattern of chaining, which is, you know… BEN: Yeah, yeah. People want chaining. People want left to right composition. CHARLES: Mmhmm. BEN: And it's problematic in a world where you want to shake off as much unused garbage as possible. And the only way to get dot chaining is by augmenting a prototype. There's all sorts of weird problems that can come with that. And so, the functional programming approach is one method. But then people look at it and they say, “Ooh, yuck. I've got to wrap things in a function named pipe. Wouldn't it be nicer if there was just some syntax to do this?” And yeah, it would be nicer. But I have less control over that. CHARLES: Right. But the other alternative is to have right to left function composition. BEN: Right, yeah. CHARLES: But there's not any special syntax for that, either. BEN: Not very readable. CHARLES: Yeah. BEN: So, you just wrap everything. And the innermost call is the first one and then you wrap it in another function and you wrap that in another function, and so on. Yeah, that's not [inaudible]. But I will say that the pipe function itself is pretty simple. It's basically a function that takes a rest of arguments that are all functions. CHARLES: Mmhmm. BEN: And so, you have this array of functions and you just reduce over it and call them. Well, you return a function. So, it's a higher function. You return a function that takes an argument then you reduce over the functions that came in as arguments and you call each one of them with whatever result was from the previous. CHARLES: Right. Like Tracy mentioned in the pre-show, I'm an aspiring student of functional programming. So, would this be kind of like a monoid here where you're mashing all these functions together? Is your empty value? I'm just going to throw it out there. I don't know if it's true or not, but that's my conjecture. BEN: Yes. Technically, it's a monoid because it wouldn't work unless it was a monoid. Because monoids, I believe the category theory I think for monoid is that monoids can be concatenated because they definitely have an end. CHARLES: Right. BEN: So, you would not be able to reduce over all those functions and build something with that, like that, unless it was a monoid. So yeah, the fact that there's reduction involved is a cue that it's a monoid. CHARLES: Woohoo! Alright. [Laughter] CHARLES: Have you found yourself wanting to apply some of these more “rigorous” formalisms that you find out there in the development of RxJS or is that just really a secondary concern? BEN: It's a secondary concern. It's not something that I like. It's something I think about from time to time, when really, debating any kind of heavy issue, sometimes it's helpful. But when it comes to teaching anybody anything, honestly the Haskell-isms and category theory names, all they do is just confuse people. And if you tell somebody something is a functor, they're like, “What?” And if you just say it's mappable, they're like, “Oh, okay. I can map that.” CHARLES: [Laughs] Right, right. BEN: And then the purists would be like, “But they're not the same thing.” And I would be like, “But the world doesn't care. I'm sorry.” CHARLES: Yeah, yeah. I'm kind of experiencing this debate myself. I'm not quite sure which side I fall on, because on the one hand it is arbitrary. Functor is a weird name. But I wish the concept of mappable existed. It does, but I feel like it would be handy if people… because there's literally five things that are super handy, right? Like mappable, if we could have a name for monoid. But it's like, really, you just need to think in terms of these five constructs for 99% of the stuff that you do. And so, I always wonder, where does that line lie? And how… mappable, is that really more accessible than functor? Or is that only because I was exposed to the concept of mapping for 10 years before I ever heard the F word. BEN: Yes, and yes. I mean, that's… CHARLES: [Laughs] BEN: Things that are more accessible are usually more accessible because of some pre-given knowledge, right? What works in JavaScript probably isn't going to work in Haskell or Scala or something, right? CHARLES: Mmhmm. BEN: If someone's a Java developer, certain idioms might not make sense to them that come from the JavaScript world. CHARLES: Right. But if I was learning like a student, I would think mappable, I'd be thinking like, I would literally be thinking like Google Maps or something like that. I don't know. BEN: Right, right. I mean, look at C#. C#, a mapping function is always going to be called select, right, because that's C#. That's their idiom for the same thing. CHARLES: Select? BEN: Yeah. CHARLES: Really? BEN: Yeah, select. So, they'll… CHARLES: Which in Ruby is like find. BEN: Yeah. there's select and then, what's the other one, ‘select many' or something like that. [Chuckles] BEN: So, that's C#. CHARLES: Oh, like it's select from SQL. Okay. BEN: Yeah, I think that's kind of where it came from because people had link and then they had link to SQL and then they're like, well I want to do this with regular code, with just using some more… less nuanced expressions. So, I want to be able to do method calls and chain those together. And so, you end up with select functions. And I think that that exists even in Rx.NET, although I haven't used Rx.NET. CHARLES: Hmm, okay. ELRICK: So, I know you do a lot of training with Rx. What are some of the concepts that people struggle with initially? TRACY: I think when we're teaching RX Workshop, a lot of the people sort of… I'll even see senior level people struggle with explaining it, is the difference between observables and observers and then wrapping their head around the idea that, “Hey, observables are just functions in JavaScript.” So, they're always thinking observables are going to do something for you. Actually, it's not just in Angular but also in React, but whenever someone's having issues with their Rx applications, it's usually something that they're like nesting observables or they're not subscribing to something or they've sort of hot-messed themselves into a tangle. And I'm sure you've debugged a bunch of this stuff before. The first thing I always ask people is, “Have you subscribed?” Or maybe they're using an Angular… they're using pipes async but they're also calling ‘dot subscribe' on their observable. BEN: Yeah. So, like in Angular they'll do both. Yeah. There's that. I think that, yeah, that relates to the problem of people not understanding that observables are really just functions. I keep saying that over and over again and people really don't seem to take it to heart for whatever reason. [Chuckles] BEN: But you get an observable and when you're chaining all those operators together, you're making another observable or whatever, observables don't do anything until you subscribe to them. They do nothing. CHARLES: Shouldn't they be called like subscribable? BEN: Yes. [Chuckles] BEN: They probably should. But we do hand them an observer. So, you are observing something. But the point being is that they don't do anything at all until you subscribe to them. And in that regard, they're like functions, where functions don't do anything unless you call them. So, what ends up happening with an observable is you subscribe to it. You give it an observer, three callbacks which are then coerced into an observer. And it takes that observer and it hands it to the body of this observable definition and literally has an observer inside of there. And then you basically execute that function synchronously and do things, whatever those things are, to set up some sort of observation. Maybe you spin up a WebSocket and tie into some events on it and call next on the observer to get values out of your observable. The point being that if you subscribe to an observable twice, it's the same thing as calling a function twice. And for some reason, people have a hard time with that. They think, if I subscribe to the observable twice, I've only called the function once. CHARLES: I experienced this confusion. And I remember the first time that that… like, I was playing with observables and the first time I actually discovered that, that it was actually calling my… now what do you call the function that you pass to the constructor that actually does, that calls next or that gets passed the observer? TRACY: [Inaudible] BEN: I like to call it an initialization function or something. But the official name from the TC39 proposal is subscriber function. CHARLES: Subscriber function. So, like… BEN: Yeah. CHARLES: I definitely remember it was one of those [makes explosion sound] mind-blowing moments when I realized when I call my subscribe method, the entire observable got run from the very beginning. But my intuition was that this is an object. It's got some shared state, like it's this quasar that I'm now observing and I'm seeing the flashes of light coming off of it. But it's still the same object. You think of it as having yeah, not as a function. Okay. No one ever described it to me as just a function. But I think I can see it now. ELRICK: Yeah, me neither. CHARLES: But yeah, you think of it in the same way that most people think of objects, as like, “I have this object. I have a reference to it.” Let observable equal new observable. It's a single thing. It's a single identity. And so, that's the thing that I'm observing. It's not that I'm invoking this observable to observe things. And I think that's, yeah, that's a subtle nuance there. I wish I had taken y'all's course, I guess is what I'm saying. ELRICK: Yeah. BEN: Yeah. Well, I've done a few talks on it. CHARLES: [Laughs] BEN: I always try to tell people, “It's just a function. It's just a function.” I think what happens to a lot of people too is there's the fact that it's an object. But I think what it is, is people's familiarity with promises does this. Because promises are always multicast. They are always “hot”. And the reason for this is because they're eager. So, by the time you have a promise, whatever is producing value to the promise has already started. And that means that they're inherently a multicast. CHARLES: Right. BEN: So, people are used to that behavior of, I can ‘then' off of this promise and it always means one thing. And it's like, yeah, because the one thing has nothing to do with the promise. It wasn't [Chuckles] CHARLES: Right. BEN: This promise is just an interface for you to view something that happened in the past, where an observable is more low-level than that and more simple than that. It just states, “I'm a function that you call. I'm going to be able to do anything a function can do. And by the way, you're giving me an observer and I'm going to do some stuff with that too and notify you via this observer that you handed me.” Because of that you could take an observable and close over something that had already started. Say you had a WebSocket that was already running. You could create a new observable and just like any function, close over that, externally create a WebSocket. And then everyone that subscribes to that observable is tying an observer to that same WebSocket. Then you're multicast. Then you're “hot”. ELRICK: [Inaudible] CHARLES: Right. So, I was going to say that's the distinction that Jay was talking about. He was talking about we're going to just talk about… he said at the very beginning, “We're just going to talk about hot observable.” ELRICK: Yup. CHARLES: But even a hot observable is still theoretically evaluating every single time you subscribe. You're getting a new observable. You're evaluating that observable afresh each time. It just so happens that in the lexical scope of that observable subscriber function, there is this WebSocket? BEN: Yeah. So, it's the same thing. Imagine you wrote a function that when you called it created a new WebSocket and then… say, you wrote a new function that you gave an observer object to, right? An observer object has next, error, and complete. And in that function, when you called it, it created a new WebSocket and then it tied the ‘on message' and ‘on close' and whatever to your observer's next method and your observer's error message and so on. When you call that function, you would expect a new WebSocket to be created every single time. Now, let's just say alternately you create a WebSocket and then you write a new function that that function closes over that WebSocket. So, you reference the WebSocket that you externally created inside of your function. When you call that function, it's not going to create a new WebSocket every time. It's just closing over it, right? So, even though they both are basically doing the same thing, now the latter one of those two things is basically a hot observable and the former is a cold observable. Because one is multicast which is, “I'm sharing this one WebSocket with everybody,” and the other one is unicast which is, “I am going to create a new WebSocket for each person that calls me.” And that's the [inaudible] people have a hard time with. CHARLES: Right. But really, it's just a matter of scope. BEN: Yeah. The thing people have a hard time with, with observables, is not realizing that they're actually just functions. CHARLES: Yeah. I just think that maybe… see, when I hear things like multicast and unicast, that makes me think of shared state, whereas when you say it's just a matter of scope, well then I'm thinking more in terms of it being just a function. It just happens that this WebSocket was already [scoped]. BEN: Well, shared state is a matter of scope, right? CHARLES: Yes, it is. It is. Oh, sorry. Shared state associated with some object identity, right? BEN: Right. CHARLES: But again, again, it's just preconceptions, really. It's just me thinking that I've had to manage lists of listeners and have multicast observers and single-cast observers and having to manage those lists and call notify on all of them. And that's really not what's happening at all. BEN: Yeah. Well, I guess the real point is observables can have shared state or they could not have shared state. I think the most common version and the most composable version of them, they do not have any shared state. It's just one of those things where just like a function can have shared state or it could be pure, right? There's nothing wrong with either one of those two uses of a function. And there's nothing wrong with either one of those two uses of Observable. So, honest to god, that is the biggest stumbling block I think that I see people have. That and if I had to characterize it I would say fear and loathing over the number of operators. People are like… CHARLES: [Chuckles] BEN: And they really think because everyone's used to dealing with these frameworks where there's an idiomatic way to do everything, they think there's going to be an RxJS idiomatic way to do things. And that's just patently false. That's like saying there's an idiomatic way to use functions. There's not. Use it however it works. The end. It's not… CHARLES: Mmhmm, mmhmm. BEN: You don't have to use every operator in a specific way. You can use it however works for you and it's fine. ELRICK: I see that you guys are doing some fantastic work with your documentation. Was that part of RxJS 2.0 docs? TRACY: I was trying to inspire people to take on the docs initiative because I think when I was starting to learn RxJS I would get really frustrated with the docs. BEN: Yeah. TRACY: I think the docs are greatly documented but at the same time if you're not a senior developer who understands Rx already, then it's not really helpful. Because it provides more of a reference point that the guys can go back and look at, or girls. So anyways, after many attempts of trying to get somebody to lead the project I just decided to lead the project myself. [Laughter] TRACY: And try to get… the community is interesting because I think because the docs can be sometimes confusing… Brian Troncone created LearnRxJS.io. There's these other visualization projects like RxMarbles, RxViz, et cetera. And we just needed to stick everybody together. So, it's been a project that I think has been going on for the past two months or so. We have… it's just an Angular app so it's probably one of the most easiest projects to contribute to. I remember the first time I tried to contribute to the Ember docs. It literally took me an hour to sit there with a learning team, Ember Learning Team member and… actually, maybe it was two hours, just to figure out how the heck… like all the things I had to download to get my environment set up so that I could actually even contribute to the darn documentation. But with the Rx, the current RxJS docs right now is just an Angular app. You can pull it down. It's really easy. We even have people who are just working on accessibility, which is super cool, right? So, it's a very friendly place for beginners. BEN: I'm super pleased with all the people that have been working on that. Brian and everybody, especially on the accessibility front. Jen Luker [inaudible] came in and voluntarily… she's like the stopgap for all accessibility to make sure everything is accessible before we release. So, that's pretty exciting. TRACY: Yeah. ELRICK: Mmhmm. TRACY: So funny because when me and Jen started talking, she was talking about something and then I was like, “Oh my god, I'm so excited about the docs.” She's like, “I'm so excited, too! But I don't really know why I'm excited. But you're excited, so I'm excited. Why are you excited?” [Laughter] TRACY: I was like, “I don't know. But I'm excited, too!” [Chuckles] TRACY: And then all of a sudden we have accessibility. [Laughs] ELRICK: Mmhmm. Yeah, I saw some amazing screenshots. Has the new docs, have they been pushed up to the URL yet? TRACY: Nah, they are about to. We were… we want to do one more accessibility run-through before we publish it. And then we're going to document. We want to document the top 15 most viewed operators. But we should probably see that in the next two weeks or so, that the new docs will be… I mean, it'll say “Beta, beta, beta” all over everything. But actually also, some of our friends, [Dmitri] from [Valas] Software, he is working on the translation portion to make it really easy for people to translate the docs. CHARLES: Ah. TRACY: So, a lot of that came from the inspiration from the Vue.js docs. we're taking the versioning examples that Ember has done with their docs as inspiration to make sure that our versioning is really great. So, it's great that we can lend upon all the other amazing ideas in the industry. ELRICK: Oh, yeah. CHARLES: Yeah, it's fantastic. I can't wait to see them. ELRICK: Yeah, me neither. The screenshots look amazing. I was like, “Wow. These are some fabulous documentation that's going to be coming out.” I can't wait. TRACY: Yeah. Thank you. CHARLES: Setting the bar. ELRICK: Really high. [Laughter] CHARLES: Actually, I'm curious. Because observables are so low-level, is there some use of them that… what's the use of them that you found most surprising? Or, “Whoa, this was a crazy hack.” BEN: The weirdest use of observables, there's been quite a few odd ones. One of the ones that I did one time that is maybe in RxJS's wheelhouse, it was just that RxJS already existed. So, I didn't want to pull in another transducer library, was using RxJS as a transducer. Basically… in Netflix we had a situation where we had these huge, huge arrays of very large objects. And if you try to take something like that and then map it and then filter it and then map it and then filter it, we're using Array map and filter, what ends up happening is you create all sorts of intermediary arrays in-memory. And then garbage collection has to come through and clean that up. And that locks your thread. And over time, we were experiencing slowness with this app. And it would just build up until eventually it ground to a halt. And I used RxJS because it was an available tool there to wrap these arrays in an observable and then perform operations on them step-by-step, the same map, filter, and so on. But when you do that, it doesn't create intermediary arrays because it passes each value along step to step instead of producing an entire array and then doing another step and producing an entire array, and so on. So… CHARLES: So, will you just… BEN: It saved garbage collection and it increased the performance of the app. But that's just in an extreme case. I would never do that with just regular arrays. If anything, it was because it was huge, huge arrays of very large objects. CHARLES: So, you would create an observable our of the array and then just feed each element into the observable one at a time? BEN: Well, no. If you say ‘observable from' and you give it an array, that's basically what it does. CHARLES: Okay. BEN: It loops over the array and nexts those values out of the array synchronously. CHARLES: I see, I see. BEN: So, it's like having a for loop and then inside of that for loop saying, “Apply the map. Apply the filter,” whatever, to each value as they're going through. But when you look at it, if you had array map, filter, reduce, it's literally just taking the first step and saying ‘observable from' and wrapping that array and then the rest of it's still the same. CHARLES: Right. Yeah. No, that's really cool. BEN: That was a weirder use of it. I've heard tell of other things where people used observables to do audio synchronization, which is pretty interesting. Because you have to be very precise with audio synchronization. So, hooking into some of the Web Audio APIs and that sort of thing. That's pretty interesting. The WebSocket multiplexing is something I did at Netflix that's a little bit avant-garde for observable use because you essentially have an observable that is your WebSocket. And then you create another observable that closes over that observable and sends messages over the WebSocket for what you're subscribed to and not subscribed to. And it enables you to very easily retry connections and these sorts of things. I did a whole talk on that. That one's pretty weird. CHARLES: Yeah. Man, I [inaudible] to see that. BEN: But in the general use case, you click a button, you make an AJAX request, and then you get that back and maybe you make another AJAX request. Or like drag and drop and these sorts of things where you're coordinating multiple events together, is the general use case. The non-weird use case for RxJS. Tracy does weird stuff with RxJS though. [Laughter] CHARLES: Yeah, what's some weird uses of RxJS? TRACY: I think my favorite thing to do right now is to figure out how many different IoT-related things I can make work with RxJS. So, how many random things can I connect to an application using that? BEN: Tracy's projects are the best. They're so good. [Laughter] TRACY: Well, Ben and I created an application where you can take pictures of things using the Google Image API and it'll spit back a set of puns for you. So, you take a picture of a banana, it'll give you banana puns. Or you can talk to it using the speech recognition API. My latest thing is I really want to figure out how to… I haven't figured out if Bluetooth Low Energy is actually enabled on Google Home Minis. But I want to get my Google Home Mini to say ‘booty'. [Inaudible] [Laughter] CHARLES: RxJS to the rescue. [Laughter] BEN: Oh, there was, you remember Ng-Cruise. We did Ng-Cruise and on there, Alex Castillo brought… TRACY: Oh, that was so cool. BEN: All sorts of interesting… you could read your brain waves. Or there was another one that was, what is it, the Microsoft, that band put around your wrist that would sense what direction your arm was in and whether or not your hand was flexed. And people… TRACY: Yeah, so you could flip through things. BEN: Yeah. And people were using reactive programming with that to do things like grab a ball on the screen. Or you could concentrate on an image and see if it went blurry or not. ELRICK: Well, for like, Minority Report. BEN: Oh, yeah, yeah. Literally, watching a machine read your mind with observables. That was pretty cool. That's got to be the weirdest. TRACY: Yeah, or we had somebody play the piano while they were wearing one of the brainwave… it's called the OpenBCI project is what it is. And what you can do is you can actually get the instructions to 3D print out your own headset and then buy the technology that allows you to read brain waves. And so with that, it's like… I mean, it was really awesome to watch her play the piano and just see how her brain waves were going super crazy. But there's also these really cool… I don't know if you guys have heard of Jewelbots, but they're these programmable friendship bracelets that are just little Arduino devices that light up. I have two of them. I haven't even opened them. CHARLES: [Laughs] TRACY: I've been waiting to play with them with you. I don't know what we're going to do, but I just want to send you lights. Flashing lights. [Laughter] TRACY: Morse code ask you questions about RxJS while you're working. [Laughter] CHARLES: Yeah. Critical bug. Toot-toot-toot-too-too-too-too-toot-toot. [Laughter] CHARLES: RxJS Justice League. TRACY: That would actually be really fun. [Laughter] TRACY: That would be really fun. I actually really want to do that. But… CHARLES: I'm sure the next time we talk, you will have. TRACY: [Laughs] Yes. Yes, yes, yes, I know. I know. we'll do it soon. We just need to find some time while we're not going crazy with conferences and stuff like that. CHARLES: So, before we head out, is there any upcoming events, talks, releases, anything that we ought to be, we or the listeners, ought to be aware of? TRACY: Yeah, so one of the things is that Ben and I this weekend actually just recorded the latest version of RX Workshop. So, if you want to learn all about the latest, latest, newest new, you can go ahead and take that course. We go through a lot of different things like multiplex WebSockets, building an application. Everywhere from the fundamentals to the more real world implementations of RxJS. BEN: Yeah. Even in the fundamentals area, we've had friends of ours that are definitely seasoned Rx veterans come to the workshop. And most of them ask the most questions while talking about the fundamentals. Because I tend to dig into, either deep into the internals or into the why's and how's thing. Why and how things work. Even when it comes to how to subscribe to an observable. Deep detailed information about what happens if you don't provide an error handler and certain cases and how that's going to change in upcoming versions, and why that's changing in upcoming versions, and what the TC39's thoughts are on that, and so on and so forth. So, I try to get into some deeper stuff and we have a lot of fun. And we tend to be a little goofier at the workshops from time to time than we were in this podcast. Tracy and I get silly when we're together. TRACY: It's very true. [Laughter] TRACY: But I think also, soon I think there are people that are going to be championing an Observable proposal on what [inaudible]. So, aside from the TC39 Observable proposal that's currently still at stage one, I don't know Ben if you want to talk a little bit about that. BEN: Oh, yeah. So, I've been involved in conversations with folks from Netflix and Google as well, Chrome team and TC39 members, about getting the WHATWG, the ‘what wig', they're a standards body similar to W3C, to include observables as part of the DOM. The post has not been made yet. But the post is going to be made soon as long as everybody's okay with it. And what it boils down to is the idea of using observables as part of event targets. An event target is the API we're all familiar with for ‘add event listener', ‘remove event listener'. So, pretty much anywhere you'd see those methods, there might also someday be an on method that would return an observable of events. So, it's really, really interesting thing because it would bring at least the primitives of reactive programming to the browser. And at the very least it would provide maybe a nicer API for people to subscribe to events coming from different DOM elements. Because ‘add event listener' and ‘remove event listener' are a little unergonomic at times, right? CHARLES: Yeah. They're the worst. BEN: Yeah. CHARLES: That's a very polite way of putting it. BEN: [Chuckles] So, that's one thing that's coming down the pipe. Other things, RxJS 6 is in the works. We recently tied off 5.5 in a stable branch. And master is now our alpha that we're working on. So, there's going to be a lot of refactoring and changes there, trying to make the library smaller and smaller. And trying to eliminate some of the footprints that maybe people had in previous versions. So, moving things around so people aren't importing stuff that were meant to be implementation details, reducing the size of the library, trying to eliminate some bloat, that sort of thing. I'm pretty excited about that. But that's going to be in alpha ongoing for a while. And then hopefully we'll be able to move into beta mid first quarter next year. And then when that'll be out of beta, who knows? It all depends on how well people like the beta and the alpha, right? CHARLES: Alright. Well, so if folks do want to follow up with y'all either in regards to the course or to upcoming releases or any of the other great stuff that's coming along, how would they get in touch with y'all? TRACY: You can find me on Twitter @ladyleet. But Ben is @BenLesh. RX Workshop is RXWorkshop.com. I think in January we're going to be doing state of JavaScript under This Dot Media again. So, that's where all the core contributors of different frameworks and libraries come together. So, we'll definitely be giving a state of RxJS at that time. And next year also Contributor Days will be happening. So, if you go to ContributorDays.com you can see the previous RxJS Contributor Days and figure out how to get involved. So, we're always open and happy and willing to teach everybody. And again, if you want to get involved it doesn't matter whether you have little experience or lots of experience. We are always willing to show you how you can play. BEN: Yeah. You can always find us on Twitter. And don't forget that if you don't find Tracy or I on Twitter, you can always message Jay Phelps on Twitter. That's important. @_JayPhelps. Really. TRACY: Yeah. [Laughter] BEN: You'll find us. CHARLES: [Chuckles] Look for Jay in the show notes. [Laughter] CHARLES: Alright. Well, thank you so much for all the stuff that y'all do, code and otherwise. And thank you so much Ben, thank you so much Tracy, for coming on the show. BEN: Thank you. CHARLES: Bye Elrick and bye everybody. If you want to reach out to us, you can always get in touch with us at @TheFrontside or send us an email at contact@frontside.io. Alright everybody, we'll see you next week.
How do we ensure a high level of quality and maximize the refactorability of our code? Frontsiders, Wil and Charles, talk about their battle tested techniques for testing web applications, not only in React JS, but in any JavaScript framework. Links Acceptance Testing Integration Testing jest Cypress.io Assertion Ember CLI Mirage mirage-server react-trigger-change Transcript CHARLES: Hello everybody and welcome to The Frontside Podcast, Episode 90. My name is Charles Lowell, a developer here at The Frontside and your podcast host-in-training. And with me today is Mr. Wil Wilsman. Wil, who just got back from Nodevember just walked straight into the office and is ready to podcast with us on a very, very, very interesting subject, I think, today. We're going to be talking about acceptance testing in JavaScript applications, especially some of the techniques that we've developed here around testing React applications based on the lessons that we learned from the Ember community. But really, more than just React applications. Really, testing any JavaScript application from the inside out, making acceptance tests for that. So, I think we're going to talk about some of the challenges that you encounter and some of the really novel solutions that are out there that we had nothing to do with. And I guess we really didn't have, just more of cobbling together of various techniques for a powerful witch's brew for acceptance testing. Anyway, so Wil, just to round out the problem space or explore the problem space, what are some of the challenges that you encounter with an acceptance test? Actually, let me back it up even further. What is an acceptance test in a JavaScript application compared to what people normally encounter? WIL: Acceptance testing or end-to-end testing is just a problem that every JavaScript app should face. Not everyone does, but they definitely should. And basically, it's how the user interacts with your app through the browser. And every part of that we want to test, from the browser triggering browser events, interacting with the app, not calling functions or clicking buttons, and we're pretending we're a user. CHARLES: Yeah. You know, I know that when we showed up in the React space, that was not really the way that most people tested their applications. WIL: No, not at all. they're all about unit testing. Make sure every small piece of your code works, and to some degree integration testing, making sure your components work with other components. But, nothing is out there really for those big acceptance tests that you want the user to click a button and expect them be brought to a page or these fields to be filled out, et cetera. CHARLES: Mmhmm. Yeah. And there certainly was a very high level of maturity around unit testing, like you said. There are tools like Enzyme and… WIL: Jest. CHARLES: Yeah, Jest. But I was actually shocked to find out that Jest didn't even run in the browser. WIL: Yeah, it's all virtual. CHARLES: It's all virtual. It's completely and totally simulated and stubbed. And that presents some problems. WIL: Yeah. The main problem is cross-browser testing. Some people might consider that to be separate from their acceptance testing but you should be able to just run your acceptance tests in multiple browsers and be able to also test cross-browser support. CHARLES: Mmhmm. Yeah. And so, if you're using something like Jest, you're never actually running the code inside Safari. You're never actually running it inside Internet Explorer. You're actually running it in NodeJS. WIL: And you know, your user is not going to run it in Node. [Laughter] WIL: They're going to use a browser. CHARLES: I don't know about your users. WIL: [Chuckles] CHARLES: [Laughs] You know, we like to stick to the pretty advanced. It's like, go to getNodeJSBrowser.com. WIL: [Laughs] CHARLES: Enough of this Firefox BS. But not seriously, it was certainly a problem. We were looking around, because we never like to build anything ourselves if we can avoid it. But it really just seemed like there was not an off-the-shelf solution for writing these big style acceptance tests in JavaScript. There are some services out there. There's a couple now. What was the… WIL: I think the main one here is Cypress. CHARLES: Cypress, yeah. So, there's Cypress now. I've watched the instructional videos but never actually tried to integrate it into my application. WIL: Yeah. I think at its core it takes the same approach that we've been doing with how we're interacting with our tests. CHARLES: Mmhmm. Okay. The main difference is, is it that it's a service? Like you have to edit your tests through… WIL: Yeah. CHARLES: Their web browser, their web interface, and use their assertion library? WIL: Yeah. I'm not sure about the editing part. But yeah, it's their assertion library. I'm pretty sure it's their test runner and it's their testing environment. Really, the only control through that is through their UI, or through settings, basically. And you're stuck with those. You can't use other… I don't think you can use Mocha with Cypress. CHARLES: Right. WIL: Although it's very much like Mocha… CHARLES: Right. WIL: It's not. CHARLES: Right, right. And I also noticed, we'll touch on this later, the assertions, most of the side effects that were happening were happening right there inline inside your assertions. And that might be an opaque statement, but we will actually get into that later. WIL: Yeah. And I think one of the things about their side effects so to speak is everything leading up to a side effect is a promise with Cypress. CHARLES: Mmhmm. WIL: So, when you select a button and click it, Cypress is going to wait for that button to actually exist before it clicks it. CHARLES: Right, right, which is actually pretty cool. So, that's actually a perfect into into one of the primary challenges with doing acceptance testing in general in a JavaScript application. This is a problem when you're doing it in Ember. It's a problem in React. It's really a problem anywhere. And that is, how do you know when the effects of a user's interaction have been realized? Right? WIL: Yeah. And in Ember you take advantage of the run loop. Once that action happens, you wait for the run loop to complete and then your tests run. CHARLES: Right. So, the idea is that I've clicked some button or I've typed some key or I've moved the mouse. And then I listen for the run loop and when it's “settled” then I can now run my assertions because I know that the side effects that I was looking for have now been realized. WIL: Yeah, hopefully, if you're... CHARLES: Hopefully. WIL: Writing your app right. [Chuckles] CHARLES: Right, right. But that actually presents some problems in itself because it requires visibility into the internals of the framework. WIL: Yeah, so Ember is built with testing in mind. CHARLES: Mmhmm. WIL: And other libraries like React just being a view library might not be built with testing in mind. So, we don't have those hooks to wait for this loop to complete, wait for all of these things to be rendered before you continue. CHARLES: Exactly. And so, this is, I think it's actually kind of both a blessing and a curse. Because there are such strong conventions in Ember, they were able to build this wonderful acceptance testing regimen from the get go. WIL: Yeah. CHARLES: But like you said, that doesn't exist at all in the React ecosystem. And so, what do you do? There's no run loop. You're cobbling together a bunch of different components. And maybe you're using Redux, maybe you're using MobX, maybe you're using… you're certainly using React. And all of these things have their own asynchronies built-in. And there's not one unifying abstraction that's keeping track of all the asynchrony in the system. And so that presents a challenge. So, the question is then, if you're trying to not actually check and observe the state of a system until the right moment, how do you know when that right moment is? WIL: Yeah. And in early testing of a side React project I had, I would basically wait for a state to be complete before I continued my ‘before each'. And in the testing we're doing now, it's essentially what we're doing except the state is what the browser sees, or what the user would see in the browser. CHARLES: So you were actually querying the... WIL: Yeah. So, I was using Redux. So, in my app I was saying, when the Redux app is done loading... CHARLES: Mmhmm. WIL: The instance is set to true or false, then continue the test. CHARLES: Mmhmm. And so, what that means, what we're doing, is doing the same thing except observing at the DOM... WIL: Yeah, exactly. CHARLES: Level. And what it means is we actually… I would love to set this up and have a big reveal but I guess we'll just have a big reveal, is that essentially what we do is polling. WIL: Yeah. CHARLES: Right? So, when we run an assertion, let's say you click a button and you want the button to become disabled, there's an inherent asynchrony there. But what we will do is we'll actually run the assertion to see if it's disabled not one time. We'll run it a thousand times. WIL: Yeah, as many times as needed until it passes. CHARLES: Right, exactly. As many times as needed until it passes. And I think that is, at least to most programmer instincts, an odious idea. WIL: Yeah. CHARLES: [Laughs] WIL: It's like, “Oh wait, you're just looping over every single assertion how many times?” CHARLES: Yeah, exactly. And it feels, yeah, it feels weird as an idea. But when you actually see the code that it produces, it just sweeps away so much complexity. WIL: Yeah. And… CHARLES: Because you don't worry about asynchrony at all. WIL: Yeah. And it's pretty genius. If I'm a user and I click a button, it's loading when I see that it's loading. So, our tests are going to wait until that button says it's loading. And then the test passes. CHARLES: Right. And so, what we do is we essentially, we use Mocha but you could do it with QUnit or anything else, is that when you run your assertion, you declare, you have an ‘it' block or I guess, what would it be in QUnit? WIL: A test? CHARLES: A test? WIL: I think it's just a test. CHARLES: You have your test block. And so that function that actually runs the assertion and checks the state will actually run, yeah it could run three times. It could run a thousand times. It's just sitting there waiting. And it will time out. And it will only fail if that assertion has failed a thousand times or it has failed through, I think two seconds is our default. WIL: Yeah, yeah. I think we default to the runner's default timeout. CHARLES: To the runner's default timeout, yeah. WIL: Yeah. Or you can set that yourself with how we have it set up. And the other thing that comes from that is if your tests are only failing when they time out, how do you know what's actually failing? And our solution to that was we catch the error every time it fails and right before the timeout actually happens we throw the real error. CHARLES: Yeah. Exactly. But the net effect is that you're able to write your assertions completely and totally oblivious of asynchrony. You don't, we don't have to worry about asynchrony pretty much at all. I mean, we do, and we'll get into that. So, I made a global statement and then immediately contradicted it. WIL: [Chuckles] CHARLES: But hey, you got to be controversial. But for the most part, asynchrony just disappears because asynchrony is baked into the fabric. So, rather than thinking about it as a one-off concern or a onesie-twosie, it's just every single assertion is just assumed to be asynchronous. And so, that actually means you don't have to deal with promises. You don't have to deal with run loops. You don't have to deal with anything. You just write your assertion and when it passes, it passes. And there are some really unique benefits for this. And there are some challenges. So, I think one of the first benefits is that it's actually way faster. WIL: Right. CHARLES: Which is counterintuitive. WIL: It's incredibly fast. CHARLES: It's very fast. WIL: Yeah, for all the loops that's happening you might think every loop is going to slow it down slightly. But it really doesn't. Our tests, each test, even though it asserts five or six times, it takes milliseconds. CHARLES: Mmhmm. WIL: The test itself might only loop twice. CHARLES: Right. Exactly. Whereas if you're waiting for a run loop to settle, you might have some… you click a button, it disables, it also fires off an Ajax request and does all this stuff. But if all my assertion wants to know is “Is this button disabled?” then I only need to assert until that has happened. I don't need to wait until all the side effects have settled... WIL: Mmhmm. CHARLES: And then do the assertion. I just know, “Hey, my assertion, the thing that I was waiting for - that happened. Let's move on.” Yeah. And so, it's so, so fast. And that was actually, I didn't predict that. But I was definitely pleasantly surprised. WIL: Yeah, that was a very nice surprise. And all of our tests ran so much quicker than they would have in a run loop environment with Ember or something. CHARLES: Right. Yeah, yeah. That was, we actually had just come off a project where we were having that thing, that exact problem, which was that yeah, our animations were slow. Or, the animations were fine. They were perfect. [Laughs] But there were slowing down the tests. WIL: Yes. I think in that project, it was like, 30-minute tests... CHARLES: Mmhmm WIL: For the whole suite to run. CHARLES: Yeah. To which I'll add a public service announcement. I think this is a conjecture but I do believe that animations are best applied not to individual components but by the thing that uses a component. So, I shouldn't have an animation that's like, implicit to a dialog. It should be the thing that's showing the dialog that gets to decide the animation to use. Anyway, just throwing that out there. WIL: [Laughs] CHARLES: Because animations are about context. And so, the context should provide the animation, not the individual atom. Anyway, moving on. WIL: Some other podcast. CHARLES: Yeah. [Laughter] CHARLES: That's another podcast right there. But there's also, this does present some challenges or requires code to be structured in a way that facilitates this. So, there are some challenges with this approach, some things you need to be aware of if you're using this kind of system. We've kind of settled on a name for what we call these types of assertions and these types of systems. WIL: Yeah. We call them convergent assertions, because you're converging on something to happen. It's going over and over until it happens. CHARLES: Right. WIL: And yeah, a lot of these challenges that we've come across are things that you might not think of, like there are a few instances of false positives... CHARLES: Mmhmm. WIL: That happen with these convergent assertions. CHARLES: Right. So, what would be an example there? WIL: So, the most common example that I'm seeing so far is when you're asserting that something didn't happen. CHARLES: Mm, right. WIL: That would immediately pass. But if it takes your app... CHARLES: [Laughs] WIL: A few seconds for it to actually happen, then you could still have an actual failure but your test passed immediately. CHARLES: Right, right. So, what's the countermeasure then? WIL: We invert our assertions. So, we make sure they fail for a certain amount of time. CHARLES: Right. So, the normal case where you just want to say, “I want to make sure that my state converges to this particular state.” WIL: Alright. I said fail at first. I meant, pass. We have to make sure it passes for a certain period of time. CHARLES: Right, exactly. WIL: So yeah, the normal way is it fails until it passes, and then it passes. When you invert one of these convergent assertions, you're just making sure it passes repeatedly and if it fails at any point, you throw a failure. CHARLES: Right, okay. And so, that's like, if I want to check that the button is not disabled, I need to check again and again and again and again. WIL: Until you're comfortable with saying, “Alright. It's probably not going to be disabled.” CHARLES: Yeah, exactly. And so there, it's kind of weird because it is dependent on a timeout. WIL: Mmhmm. CHARLES: You could go for two seconds and then at the very end it becomes disabled. So, you just kind of have to take that on faith. But... WIL: Yeah. CHARLES: In practice, I don't think that's been much of a problem. WIL: No. CHARLES: It's more indicative of, if your button disables after... WIL: A few seconds. CHARLES: A few seconds, what's up with your... WIL: Yeah, what's up with your app? CHARLES: Yeah, exactly. WIL: If you're waiting for an Ajax request or something, an example, then you should be using something like Mirage Server. CHARLES: Right. Which is, man, we got to get into that, too. There are a couple of other things that I wanted to talk about too, with these convergent assertions. And that is, typically when you look through the READMEs for most testing frameworks, you see the simple case of the entire test, the setup, the teardown, and the actual assertions, are in the actual test. WIL: Yeah. The ‘it' block in Mocha or the test block in QUnit. You click a button, and then make sure it's disabled, and it moves onto the next test. CHARLES: Mmhmm. WIL: Then you click a different button or the same button and you assert something else in the next test. CHARLES: Mmhmm. Right. WIL: And yeah, you can't do that with convergent assertions because they're looping. So, if you click a button in a loop it's going to keep clicking that button over and over and over again. CHARLES: Yeah. [Laughs] Right, right. So, it means that you need to be very conscientious about separating the parts of your tests that actually do the things that actually act the part of the user from the part of your test that's about observation. WIL: Yeah. So, our solution to that is we move out all of our things that have side effects like clicking a button or filling in a form, all that stuff happens in ‘before each's. And all of our actual assertions happen in these convergent ‘it' blocks that loop over and over again. So, our ‘before each' runs and clicks the button and then we have 10 or so tests that will loop and wait for various states to... CHARLES: Yeah. WIL: Be true. CHARLES: Right. That means that yeah, all these assertions do is they read state. And you just have to, you do have to be conscientious. You're not allowed to have any side effects inside your tests, your actual assertion blocks. WIL: Mmhmm. CHARLES: But that's actually, it's a good use case for the whole Act-Arrange-Assert, which has been around way, way, way before these techniques. But here, we're doing Act and Arrange in our ‘before each'... WIL: Mmhmm. CHARLES: And then we're doing Assert later. And I think it actually leads for more readable... WIL: Yeah, definitely. CHARLES: Things. WIL: And it also opens the door to something that we can't really take advantage of yet but if you have 10 assertions with one ‘before each' side effect, you could run all of those assertions in parallel. CHARLES: That's right. WIL: And your tests would be 10 times more faster. CHARLES: Mmhmm. Yeah, exactly. Or you could run them in parallel or you could just run them one after the other but you wouldn't have to run that ‘before each' 10 times. WIL: Yeah. But something with that, that I found, is if we move all of our side effects to ‘before' blocks instead of ‘before each' blocks, sometimes a test three tests down that's waiting for something to happen... CHARLES: Yeah. WIL: That thing might have already happened earlier... CHARLES: Yeah. WIL: And it already went away. A loading state is the best example of that. CHARLES: Mmhmm. WIL: You show the loading state, the loading state goes away. So, if you move that button click into a ‘before' and that loading state test is three tests in, that loading state is already going to be gone. CHARLES: Yeah, so I think the long story short is we've kind of come to the conclusion that we would have to write our own runner. WIL: Yeah. CHARLES: Essentially to take advantage of this. But that said, we've done some sketching about what we would gain by writing our own runner. And the speed, we're talking about exponential speedups. WIL: Yeah. CHARLES: Maybe taking an entire acceptance test suite and having it run in five or six seconds. WIL: Yeah. We're talking about these tests that are already extremely fast. CHARLES: Mmhmm. WIL: Each test takes a few milliseconds or tens of milliseconds to complete. But then if you can run all of those at the same time, all of your tests for that entire ‘describe' block just ran tens of milliseconds. CHARLES: Right. Yeah. So, it's really exciting and pretty tantalizing. And we would love to invest the time in that. I've always wanted to write our own test runner. But never had, [chuckles] never really had a reason. WIL: Yeah. CHARLES: Certainly not just for the sheer joy of it. Although I'm sure there is joy in writing it. But that, yeah, we'll have to wait on that. But I am actually really excited about the idea of being able to maybe bring this back to the Ember community. WIL: Yeah. CHARLES: Because acceptance tests getting out of control in terms of the speed is I think a problem with Ember applications. And I think this would do a lot to address that. WIL: Yeah. CHARLES: I think, how long, if we were just using a stock Ember acceptance testing setup for this, I think we have about 250 tests in this React app... WIL: Yeah. CHARLES: How long does it take to run? WIL: Right now I think our tests take something like 20 seconds. And that's also somewhat due to they have to print the tests on the screen on Travis so that takes a little time. In an Ember setup, that could maybe take a few minutes. I mean, that's not that big of a deal, a few minutes. CHARLES: Right. WIL: But compared to 20 seconds. CHARLES: Right. you're still talking about an order of magnitude... WIL: Yeah, exactly. CHARLES: Difference. And using this, I think you could get a 30-minute test suite... WIL: Yeah. CHARLES: Down to the order of 3 minutes. WIL: Now when we're talking about those times, we're talking about the tests themselves. Of course, the CI would have to download stuff and set up the [inaudible]. CHARLES: Mmhmm, right, right. WIL: And that of course all adds to the time. CHARLES: Yeah, mmhmm, yeah. Earlier you mentioned, we talked about Ember CLI Mirage. This is actually something that is having now been using it for what, 2 years or something like that, it's just… it's impossible... WIL: To go back, yeah. CHARLES: To go back. It is. It's like [chuckles] you come outside the Ember community and you're like, “How is anybody ever dealing without this?” WIL: Yeah. CHARLES: [Laughs] WIL: A lot of the mocks are usually mocking the function that makes the request and it returns it in that function. That's what's out there currently, minus the Mirage stuff. CHARLES: Mmhmm. WIL: But once you use Mirage, you're mocking the requests themselves. CHARLES: Yeah. And you've got such great support for the whole factories. I love factories. It's something that is very prevalent in the Ruby community, and maybe not so much elsewhere. But the ability to very, very quickly crank out high-fidelity production data... WIL: Yeah. And you don't have to have files upon files of fixtures. CHARLES: Yeah, exactly. And you can change, if something about your schema changes, you can change the factory and now your test data is up and running. So, Ember has this tool called Mirage which is just, like I said, it's so fantastic. Oh yeah, it's also go support for running your application. Not just in your tests, but you can actually run your application with Mirage on and... WIL: Oh right, yeah. CHARLES: And you've got now the most incredible rapid prototyping tool. WIL: Yeah. You don't need to connect to a server to see fake data. CHARLES: Right, right. And we were even talking about this yesterday to a potential client. they're trying to, they've got to present something to investors. And how wonderful is it to just be like, “You know what? We just don't want to invest in all, we don't want to move the inertia, invest the money to generate the force to move the inertia of a backend.” Especially in this particular use case, the backend was going to be really, really heavy. WIL: Yeah. And there were some questions about the backend that we couldn't address quite yet but we wanted to start working on something that we could show, something demo-able. CHARLES: Right, exactly. And so, Mirage is just so wonderful for that. But again, Mirage is, it's an Ember-specific project. WIL: Right. CHARLES: So, the question was, “How are we going to use that?” WIL: And you actually took this on yourself. CHARLES: Mmhmm. WIL: I just saw this pop-up one day and boom, you converted Ember Mirage to vanilla JavaScript. [Chuckles] CHARLES: So, I did extract it. But the lion's share of the credit goes to the developers of Mirage themselves. Sam Selikoff and the Mirage community, they built Mirage not using much of Ember. There were some utilities that they were using, but mainly things like string helpers to convert between camel-case and dash-case, and using a Broccoli build, or using an Ember CLI build. WIL: Yeah. That was one of the challenges that we came across using Mirage outside of Ember, was how do we autoload this Mirage folder with all this Mirage config and Mirage factories and models, et cetera. CHARLES: Right. The internals were all just straight up JavaScript classes, for the most part. And so, extracting it, it was a lot of work. But 90% of the work was already done. It only took three or four days to do it. WIL: Amazing. CHARLES: Yeah. So, it was actually a really pleasant experience. I was able to swap out all of the Ember string helpers for Lodash. So now, it's good to go. It shares a Git history with Ember CLI Mirage, so it's basically a fork. WIL: Mmhmm. CHARLES: Like, a very heavily patched Ember CLI Mirage. But I keep it up-to-date so that it doesn't... WIL: Good. [Inaudible] CHARLES: Yeah, so I think the last time I merged in from master was about a month ago, something like that. Because it's got all the features that we need but it's not a big deal to rebase or just to merge it on over in. Because yeah, it's a really straightforward set of patches. WIL: Was there any talk with the creators of Ember Mirage about getting this upstream? CHARLES: So, I've talked a little bit with Sam about it. And from what I can tell, his feeling on it is like, “Hey, my goal right now is to focus on this being the best testing and data stubbing platform for Ember. Anything that happens out there, outside of that scope, that's great. And I certainly won't get in the way of it. But I'm pretty maxed out in terms of the open source credits that I have to spend.” WIL: [Chuckles] CHARLES: And there hasn't been much motion there. I'm happy where it is right now. I would like to see it merged into upstream. I think it would be great to have basically this Mirage Server and then have Ember CLI bindings for it. WIL: Yeah, yeah. I was going to say, either another Ember CLI specific package for Mirage or maybe to make it a non-breaking change or something. CHARLES: Yeah. WIL: Just like an Ember-specific entry point. CHARLES: Right, exactly. And I think that's definitely doable, if someone wants to take it on. I will say, we have been using this extracted plain vanilla JavaScript Mirage Server now for what, almost six months? WIL: Yeah. CHARLES: And it really hasn't... WIL: Yeah, I don't think I ran into one issue with that. CHARLES: Yeah. It's solid. It's really, really good. So, kudos to the Mirage team for doing that. And if anybody is interested in using Mirage in their projects, it's definitely there and we'll put it in the show notes. WIL: Yeah. We call it Mirage Server. CHARLES: Mirage Server, yeah. So, I don't know. Maybe it's time to reopen that conversation. But it has become a very integral and critical piece of the way that we test our JavaScript applications now. So, what are the foundations of it? We've got, we're using Mirage. We're using these convergent assertions. We're using Mocha, although that's really... WIL: Yeah, we have jQuery and Chai jQuery just to help us out with interacting with the browser as a user would. And I think one of the big challenges with that actually, I just remembered, was triggering changes in React. CHARLES: Yeah. WIL: I think this is pretty specific to React. You might run into problems with the view. I don't know how to mess with view. But in React, at least I think 15 or React 16, one of them, they changed the descriptor of the value property on an element so that they can appropriately interact with it, make changes, watch for changes, et cetera. So, when you set this value property using jQuery or just straight up ‘.value()', that change event isn't triggered in React. Your on-change handlers are never called. CHARLES: Wait, they actually update the JavaScript property descriptor of the DOM element. WIL: Yes. CHARLES: Boo. WIL: Yeah. So, there's a nice little helper out there called React Trigger Change. I dug through it and I've stripped some of it down to just be for more modern browsers, more modern React. But there's a lot of good code in there. And basically, it just caches that descriptor, updates the value, triggers the change, and then adds the descriptor back. And that ends up triggering that React element. CHARLES: Okay. [Laughs] WIL: [Chuckles] Yeah, so that calls your handlers and that's how we get around that. CHARLES: Right. There's a few fun little hacks there. But I think it is good to tie that into a larger point, is that the amount of touch that you have with the framework is actually very low. WIL: Yeah. CHARLES: So, the amount of affordances that we've had to make just for React, there's that, that you just mentioned. And is there… there's not much else. We had to write a test harness to mount the app. But that's like... WIL: Yeah, yeah. Our describe application helper is pretty React-specific. CHARLES: Right. WIL: So, you'd have to render it and set up a Mirage server, et cetera. CHARLES: Right. But that's application-specific setup. WIL: Yeah, that's one file. CHARLES: Right. WIL: So, your goal for acceptance tests is you want to be able to have a refactor and your acceptance tests still pass. CHARLES: Right. WIL: So, what if that refactor involves switching libraries? CHARLES: Right. WIL: If you're writing Ember acceptance tests, you're going to have to rewrite all your acceptance tests. CHARLES: Right. WIL: That's a huge downside. CHARLES: Right. WIL: So, with this method of interacting with the actual library very little, we have that one file that sets up our app and then we have that one trigger change helper, we remove those, we can use whatever framework we want underneath this. And our tests would still work. CHARLES: Yeah, exactly. And I think that we actually could theoretically, and honestly I have enough confidence in this style that we're developing the tests now, we could refactor this application to Ember and not have to rewrite our tests. WIL: Yeah, exactly. CHARLES: In fact, the tests would be an aide to do that. WIL: Yeah, and the tests would be faster than Ember testing with that run loop problem. CHARLES: Yeah, exactly. that's really something to think about or to think on, is like, “Wow. You're really at this point completely, not completely, but very loosely coupled to the actual internal library code.” Which is one of the goals of a nice, big acceptance test, is to be able to make major changes, break big bones, and be able to set them and have your acceptance test suite be the bulwark that holds it all together. WIL: Yeah. CHARLES: So, I actually don't know what a bulwark is. WIL: [Laughs] CHARLES: I just know that it's a really strong thing. [Laughter] CHARLES: Maybe we could put that in the show notes. [Laughter] WIL: A link to what that is. CHARLES: [Laughs] So, alright. Well, I'm trying to think if there is anything else that we wanted to mention. Any challenges? Any next things? WIL: So, one of our next steps is something we mentioned that Cypress does, is they wait for elements to exist before they interact with them. And we're actually not doing that in our app currently. And we don't have helpers out there for it yet. CHARLES: Right. WIL: But that's very much the next step. When we go to click an element in our ‘before each', we have these describes that are nested. Say, you have nested describes and you get down three levels into a ‘before each' when you're clicking a button. That button might not exist yet. CHARLES: Right. WIL: And especially since we're using jQuery, if you trigger a change on an empty jQuery element, it's not going to throw an error. It's just not going to tell you that it triggered anything. CHARLES: Right. WIL: So, we get those skips where that button's not getting clicked and we should really be waiting for that button to exist. CHARLES: Right. So, what we've done right now is we're converging on our assertions at the backend of a test. But at the frontend of a test we need to also be converging at some state before we can actually interact with the application. WIL: Right, yeah. CHARLES: So yeah, so that part is missing. And that actually brings up, we are very slowly but nevertheless doing, we're collecting these convergent assertions and convergent helpers in a repository on our GitHub account. We're going to be adding these things so that you can either use them out of the box or use them to make your own testing library. WIL: Yeah. And one of the other next steps that goes along with waiting for the element to exist is when you need to chain convergences. Like, wait for this element to exist and then click it and then wait for this thing to happen before actually running a test. And that presents the problem of our convergences are waiting for that timeout and those timeouts will accumulate. So if you have three chained convergences, that's now a six thousand millisecond timeout as opposed to a two thousand millisecond timeout. CHARLES: Right. WIL: So, one of the next steps is getting that tracking under control so if you chain three convergences together, they're smart about it and they still fail under the two thousand millisecond timeout. CHARLES: Right, right. So yeah, so we're going to be collecting all this stuff that we're learning into some publicly available code. We have a repository set up. I don't know if I want to announce it just yet, because it's really early days. WIL: Yeah. CHARLES: But that definitely is the plan. And that way, whether you're using Mocha or whether you're using QUnit or whether you're using Chai or jQuery, you've got these underlying primitives that help you converge on a state, whether that state is to interact with some piece of the DOM or to just assert some observation is made about that state. We'll be continuing on that. But by all means, get in touch if this is something that is of interest to you. Let's make something happen, because it's something that we're pretty excited about. And honestly, it's pretty comfortable living inside the four walls of this test suite. WIL: Yeah. CHARLES: It feels pretty good. WIL: It does, yeah. They're very fast. And some places in the test might need a little reworking, but for the most part all of our tests are very well-written, very well-readable. And you can just open up a test and know exactly what's going on. CHARLES: Yeah. Alright. Well, I think that about does it for Episode 90. Wow, Episode 90. WIL: Man, coming up on that 100. CHARLES: Yeah. We're going to have to have a birthday cake or something. WIL: Do we celebrate Episode 100 or Episode 104? CHARLES: What's 104? WIL: 104 would be 2 years. CHARLES: Oh really? WIL: Well, I mean 2 years' worth of podcasts. CHARLES: Oh, right. 2 years' worth of podcasts. Yeah. WIL: Yeah, like if you go every week. CHARLES: Maybe we should celebrate a hundred hours or something like that. WIL: Oh yeah. CHARLES: We can add up the thing or celebrate… I don't know, be like, “You've literally wasted 2 years of your life.” WIL: [Laughs] CHARLES: “2 weeks of your life listening to the podcast.” Anyway, so that's it for Episode 90. and thank you so much, Wil. WIL: Thanks for having me. CHARLES: It's always a pleasure to talk about these topics with you. And as always, if you need to get in touch with us, please reach out to us on Twitter. We are @TheFrontside. Or you can send an email to contact@frontside.io.
Kaylie Kwon: kaylieEB | kayliekwon@gmail.com Show Notes: 02:14 - Kaylie's Journey Into Software Development 09:25 - Implementing a Design System and Attacking Higher-level Workflows 15:43 - EDS Collaboration and Public Availability 19:07 - Getting Involved with The Yarn Project 20:57 - Selective Resolution 23:37 - The Warmth of the Yarn Community 27:11 - Handling Issue Communication and Tracking Resources: Eventbrite britecharts Eventbrite Spectrum Transcript: CHARLES: Hello everybody and welcome to The Frontside Podcast. My name is Charles Lowell, I'm a software developer here at the Frontside and your podcast host-in-training. With me today is Elrick Ryan. Hello Elrick. ELRICK: Hey, what's up Charles? CHARLES: Not much. Are you enjoying your morning so far? ELRICK: Yeah, my morning is going well. Everything is good. CHARLES: Lots of cups of coffee? ELRICK: No cups of coffee yet. I've been drinking a lot of green tea. CHARLES: I've actually heard that's really good for you. ELRICK: That's what I heard too. That's why I started drinking it. CHARLES: Did you continue because it tastes good or do you just live on the idea of how good it is for you? ELRICK: A little bit of both. It doesn't taste that great but it's not horrible. It's almost like an acquired taste and then when you add in, "This is good for me," then it tastes great. CHARLES: Okay. We got a nice [inaudible] there. ELRICK: Yes. CHARLES: Anyway, I guess we should, at some point get on to the main content of our podcast. We have a very interesting guest with us today who has her fingers in all kinds of pies that we were talking about just at the pre-show, just before we were recording so we're really, really happy to have Kaylie Kwon. Thank you for coming on the show and welcome. KAYLIE: Hi. Thank you for having me. CHARLES: It's going to be great. Now, you are a software engineer at Eventbrite, that's correct? KAYLIE: Yep. CHARLES: What kind of things do you do over there? KAYLIE: I used to work on part of the feature team that worked on their reserved seating product but not too long ago, I moved to our frontend platform team, which is a team that helps other frontend engineers move faster through things like working on infrastructure or Eventbrite design system and dev tools. CHARLES: So like getting into the tools that unlock the exponential productivity of the developer team? KAYLIE: Uhm-mm. CHARLES: We're going to dig into all of that because you just listed a bunch of really interesting stuff. I'm really excited to talk about the design systems, in particular but lots of different stuff. But before we do, I understand that you have a fairly unique way of entering in to the position that you're in now. Your journey didn't follow the traditional arc. Would you be willing to elaborate on that or tell us that story? KAYLIE: Yeah, totally. I graduated with a degree in art history, not related to computer science at all. Then right after, I moved to New York and worked for a small startup. It was marketing/business development role and I wasn't really happy with it. I was working on some design-y stuff on the side with HTML and CSS but I just felt like my brain just needed to be stimulated a little more. I applied to this all women's bootcamp program called Hackbright in San Francisco. It was a three-month intensive program and luckily coming out of that, I had at least some initial knowledge to get my foot in the door and Eventbrite was one of the hiring partners. They brought me in for interview. I actually had no idea that it was going to be a frontend engineering role because my bootcamp was totally in Python and it just had more with a backend focus. Ben, who spoke at React Boston Conference with me, was actually one of the interviewers and he gave me this Clojure problem and I solved it but in Python just using recursion. He was like, "You got it. Just convert it to JavaScript," and I was like, "No, it just can't be done. No JavaScript." But there's the reason he would chose to hire me and they onboarded me as an internal bootcamp within Eventbrite. The first three months I was there, I learned about React, Redux, JavaScript, ES6, all of that good stuff. Then they moved me to a feature team, where I continued, I guess working on the product and then I became involved more with open source projects. I really expressed interest in when you're a new engineer and coming onboard, there's all these assumed knowledge that isn't documented anywhere or something will be really obscure and hard to use but people will just assume that that's the status quo. This idea around developer experience and helping other developers move faster, it just kind of become a natural interest of mine. I was talking to the platform team, Ben in particular, expressed my interest in working in these areas. Just about like a month ago, we did a big re-org and I landed on the platform team. Currently, we have a lot of projects in flight. One of them being moving our dependency system from our old codebase, which first was written in RequireJS to webpack. We're building out our Eventbrite design system, which is basically a shared UI component library that other teams could leverage. Our platform team will just come in and make sure that the API is usable across different teams and maintain a consistent brand in terms of look and feel. We're also working on other tooling stuff like making sure we use Docker for our dev environment and making sure that the frontend containers don't break, making sure that everyone is on the same version of ESLint, Node and things like that. CHARLES: How do you make sure that the frontend containers don't break? KAYLIE: It's actually hard. I think one thing that we're trying to test right now is using Yarn Offline Mirror and having better caching. When you build a container, it'll look into the cache directory, which is just bunch of committed tarballs. That way, they don't have to fetch to the network each time because once the lockfile or package.JSON changes at our current state, it'll rebuild the entire container and it could take a very long time. We just have a lot of packages that we've added over time. Other things, we're experimenting along with our platform team on the backend side about having remote images. Instead of devs building their containers locally on their computer, having them remotely built by a CI system and then just pulling the container and the images down. CHARLES: Really, really you're like all over the place. That sounds so much fun. KAYLIE: Yeah, it's a lot of context switching. Sorry, if I was jumping too many topics at once. CHARLES: No, absolutely not. I think it's actually fascinating and it's actually capture the kind of scope because when you are doing development, if you yourself are not running up and down the stack, the tools that you use are. The better the tools, the better you're able to focus on the one little sliver of the stack that you're working on. If I don't have to worry about where are my Docker images are coming from or if there's a temporary network flip that I can't install my dependencies, if I don't have to worry about those things, then that makes me more effective so it's important to just kind of lay out all of the stuff that goes into making a quality developer experience. KAYLIE: Yeah, the dream is like frontend people wouldn't have to worry about any of the backend stuff and they just have this isolated environment, where they could just work on what they do best, which is JavaScript and writing features in React. Everything else just works and they could see that replicated through their dev environment as well as QA and Prod and hopefully, make every tooling that they use, like testing and linting all easily, intuitive and accessible. CHARLES: How is it that you're working up and down the stack, making sure that your CI systems, your Docker images but then also at the same time, working on a design system, which you've got collaboration, I assume with some pretty hard core devops teams but also then you're interfacing with designers. You kind of flesh out your design system. Are those the same people on the platform team? Or there are different groups within it? KAYLIE: We have designers and researchers actually, as part of our engineering unit. We work pretty closely with them to define guidelines on what the design system should look like because coming into making this designs system, one thing we really wanted to make sure that is that both sides of engineering and design have input, rather than the previous old version called Style Guide, which was more engineering-driven that not. One team would need a model so they would build a model and a different team would build a slightly different model and we would end up with five different models. They want to be maintained over time and there wasn't really a focus on accessibility or consistency of brand so the design system project was to eliminate all of those pain points. CHARLES: For people who may not be familiar with design system but it's something that certainly is cramping up more and more, what's the general strategy you take? Clearly, you talked about the kind of the pains that it solves, like I'm experiencing this pain where I've got five dialogues, I've got three ways of laying out forms, I've got these problems. What's the strategy that I go about as my organization for trying to implement this? Like now, I want to do a design system, where do I start? KAYLIE: One big thing, I think that helped us was developing Eventbrite design system as just the standalone product. You could run Eventbrite design system as a standalone with all of the documentation and components with each of their different props that could be configured and it has its own set of tests. All of the components are API-driven so there's nothing specific about business logic that it assumes. For example, our button component just assumes that I'll be receiving a type of submission and some [inaudible] body. It doesn't make any assumptions that it's going to be anything Eventbrite specific. It's high-level configurable that the end user wants it to be. Another thing that helped is we have a planned approach session right before we start working on a new component. A developer who would be building the component would meet with one of the frontend platform members and will be discussing the API of the component, the CSS that would go in it and the designer would do the final QA. The development takes longer than if you were to write just a component for your app but we're trying to build it out for a long term use. We have versioning for Eventbrite design system as a library so whenever you make updates, we added to the changelog and then it gets released and we bumped in Core, which is where all of our apps live and people have to get the new version. Basically, it's an orchestrated effort and we have a process built around scheduled releases and bumping it in Core. CHARLES: I get the wanting to have a uniform buttons, uniform labels, dialogs and things like that. How do you attack the problem of higher-level workflows like a form submission and validation process that might have a lot of different pathways? How do you guide that using a design system? KAYLIE: Those are actually really good questions because we went through issues like that. Validation form field or components that have logic around more complex inputs and validation is really hard because different teams use it in a slightly different way so we ended up having two versions of complex inputs. Now, we're in the process of deprecating the V1 and then having people move over to V2. We also use the Atomic Design Principles for Eventbrite design system. It means, things like atoms would be composed of buttons and we have molecules, which are slightly more complex. Then one level above that would be organisms, which are things like forms. Obviously, as you move up to like the higher level, it becomes more difficult for us to decide does this belong in EDS or does it belong with the app. There's lots of refactoring that sometimes ends up happening as a result of something built one way and us realizing later down the road that it wasn't actually very extensible and we just built it for this particular use case. Sometimes, it's the opposite where this was too generic. We can make it a little bit more specific and add more logic to it. It really depends on the component but that's been our strategy, just taking the components as needed. We're still in the earlier stages. We've released EDS a year ago and now, we're almost close to finishing it. We're just missing a handful of components. They'll be more [inaudible] that's available as opposed to adding a brand new component. CHARLES: Did you find guidance in existing design systems? One thing is why not just use one of the existing ones that's out there. Is it a matter of branding? Is it a matter of, "There are certain interactions which are unique to our product that we need to design system to capture them?" I guess my question is, if this is something I want to take on myself, what tools would I be able to reach for and what other design systems would I be able to look to versus what I have to contribute myself? What am I going to be looking for to share with the community and what am I going to look for that's develop that's uniquely mine? KAYLIE: I think consistency was a big issue. Sort of the genesis of idea came actually even before I joined the company but I think what they were going for was that we already had an existing component library called Style Guide and it wasn't really working out for us. To build like a common UI library was a natural assumption but making it better, making it more reusable. We looked at companies like Airbnb and Microsoft and I learned a lot from what they were doing. You definitely wanted some components that are specific to Eventbrite such as a ticket card or a media card that we use for our browse pages which would be needed for Eventbrite specific pages. I think mostly, the designers wanted more control because relying on third library means we don't always get what we want. We're actually thinking about making EDS open source at some point in the future, where it could take themes so other companies or individuals can leverage it but use their own theming scheme. If Spotify came and wanted to use EDS but using their colors and brand logo, then they could do so, just by layering a different configuration style to it. CHARLES: That's such an interesting idea. Is this something that you all have explored, maybe collaborating with another company? I'm trying to think what would be the benefit of making it publicly available, unless you would be getting lots of contributions back to improve EDS itself. Is that the idea? KAYLIE: Yeah. We're definitely set on making it public at some point in the future but making it open source is a different conversation because like you said, there's pros and cons that comes with open sourcing a package. We recently open source britecharts, which is a D3 library. It's been getting a lot of contributions. It's a good recruiting tool but it's also a lot of maintenance work outside of developers normal work hours. Also, we have to start worrying about like when we make a breaking change to a component, we're not just breaking it for our own product but for other developers. We currently have external developers. We have Eventbrite plugin system called Spectrum so developers are already building on top of that and they've been asking for something like this, where they could leverage and match the look and feel with the rest of the product. The downside is lots of maintenance hours and worrying about all the people that would be breaking the component for by just solving your use case. I feel like I didn't fully answer one of your questions, which was you have a different dynamic of people working on both really backend ops things, as well as a really frontend design system work. We definitely got very smart people on the team. Some people definitely have expertise in one area versus on others. Currently, we have five developers and some of us lean more towards the backend of the frontend work and I am one of those people working on things like Yarn and Docker and build system work but it's funny because I was thinking if I had a portfolio then, there wouldn't be any visual components to it, even though I'm a frontend developer. It would just be like terminal screens. We try to divide the work but everyone tries to, I guess develop, at least some shared knowledge around why we make the decisions that we do. CHARLES: It's interesting how they all intersect. I feel like the trend of the last 10 years has been to dev of all the things. I think the first thing was this artificial divide between developers and testing and that came together with the test driven development and test obsessed. Then there was this divide between developer and an ops and that divide went away now with the advent of the devops movement. Now, there's this divide between developers and design and I feel like that kind of wall is collapsing right now. You have developers participating a lot more in design and designers spending a lot more in development. We're seeing that but it's just funny how the devs starts integrating with all the things. KAYLIE: Yeah, that's a really good insight. CHARLES: You said you kind of naturally gravitate towards more of the backend doing the working with Docker, working with Yarn. How did you get actually involved with the Yarn Project? KAYLIE: Eventbrite converted over from NPM to Yarn maybe a year ago and the benefits that we got from converting over was awesome because we were manually editing NPM shrinkwrap, which is a nightmare and the installation speed of the container was really slow just because we were on NPM and it didn't really have any advance caching mechanisms at that point. Yarn just sped up a lot of things for us. I really like the user interface outside of just installing. You actually get a lot of freebie commands like 'yarn why,' that tells you why you have a particular dependency or you could do 'yarn check.' It was just a lot more helpful. I've been wanting to contribute to open source for a while so I did a little bit of work before then but the community was really encouraging when I first try to solve and pick up some first contribution bugs off of their backlog. At the time, they were pushing for 1.0 release so there's a lot of excitement about what Yarn will be and all the new features will be adding. I kept trying to pick places where I felt like I could be of help and then, Christoph, the manager of the Yarn Team and I believe [inaudible] team as well, reached out and asked if I wanted to build the selective resolution feature for them. I was like, "Yeah. I'll give it a go." Then I did it. CHARLES: What is selective resolution? KAYLIE: Good question -- CHARLES: Because I use Yarn every day and I've never heard this term. KAYLIE: It's something that became available with 1.0 release, much like workspaces and most of the time, you're not going to need it but sometimes, you're using a library. That library will have a nested dependency that for some reason, has a bug or you can't work with that package or maybe you just want to dedupe the package so that all of your dependencies end up using one version of that particular nested package. Selective resolution is like a way for you to override other libraries dependencies -- CHARLES: Oh, that's really cool. I like that because I've had that happens to me where I want some version of a library that has got a bug fix and yet, some other library that's depending is requiring this library and they're getting the old version and I'm like, "Nah!" ELRICK: That was happening to me last night. I wish I knew about this. CHARLES: Yes, seriously. KAYLIE: Yeah. Before this -- CHARLES: I'm glad we had you on justifying about this. KAYLIE: Awesome. Before this, you would have to file an issue with the original maintainer and maybe, it'll get fix. Maybe it won't or maybe you're stuck on the old version and you can't do anything about it. We had a really similar issue with the PhantomJS package, where we wanted to use a next patch version with a bug fix but then, something that was requiring it wasn't letting us use it. It works. I verified it. It seems like Facebook started using it as well so it was a pretty rewarding to work on that. CHARLES: That's exciting. I also plan on using it the next time I encounter this. ELRICK: To get this feature, is this a flag that you have to pass? How does it do the selective resolution? KAYLIE: As long as you're using Yarn 1.0 or above, you define resolutions field inside of your package.json, like how you would define that dependencies, you also have extra [inaudible] resolutions. On the left hand side, you put in a glob pattern that you want to match and if you want to match all packages, you just enter the name of the package. Then on the right side, you enter whatever version or path that you wanted to match. It could be a file or a GitHub link or it could be a version or a range or whatever you want. ELRICK: Nice. CHARLES: That's fantastic. Now, you mentioned that as you were coming to work on this, you were looking for features to work on but the community actually did a very good job of drawing you in and getting that contribution from you, which is actually pretty amazing when you think about it. I think a lot of open source projects, either flourished or failed by their ability to attract contributors. What was it that was particularly inviting? KAYLIE: It was a combination of things. I think one thing is they tried to point you to the actual code like if you want to submit a PR, then this is where you would get started. I actually started doing that myself on some of the issues. Everyone loves PRs more than issues so I think giving people filing the issues, some, I guess empowerment to try to fix it on their own, I think is great. They also have a Discord Channel for devs to talk about any questions that they might have, how to set up their initial dev environment, to test things on Yarn. Also, they were really nice. They were really thankful when I posted a PR or commented on the fix. They also use a lot of emojis, which helps. I think I personally found it really rewarding because it made me a better developer. Before contributing to Yarn, I didn't know that much about Node. For me, it was just fun to learn more about like, "This is how something else works," and also, the codebase uses a lot of different linting configurations, which I hadn't really used before so that was a nice learning curve there as well. ELRICK: For your initial time going to Yarn when you didn't know much about it, was the champion from the project that worked with you to get you over the hump or places that you were stuck or did you just have to kind of figured it out on your own or did you ask questions on the Discord Channel or in Slack Channel or anything? How did that process go initially? KAYLIE: I definitely pace myself. I just picked up easier bugs on a repos if possible and BYK and Mel who maintain the project would give guidance, especially through PR comments and they also answer any questions on issues. If I ask questions like what's the right approach here, because sometimes you get a bug and it's not just a bug. It actually has to do with philosophy of like, what should Yarn do in this case. There be like really minor edge cases like maybe NPM does that in a particular way, should Yarn respect that or should it try to be better? Those discussions are, I think really helpful and interesting and understanding what's going on. CHARLES: It's fantastic when the conversation just builds and you're learning stuff and then you actually feel like you came out with the best solution that was available to you at the end of the process. ELRICK: You gain a lot of context about the project during those conversations. CHARLES: Right. KAYLIE: Yeah, what was good about those selective resolution feature was it was completely community-driven, even from the RFC standpoint which submitted by someone in the community and then it was implemented by me, who doesn't work in Facebook. I think that's awesome. CHARLES: There's been a lot of thought that was put into the feature even before the implementation. That's always so critical to getting people's and giving them some specifications, some blueprint of what they're actually going to build. One of the things also that I wanted to touch on too is you mentioned before the show that they have a very unique take on the way they handle communication with the community, not just in terms of pull request but also in terms of issues. KAYLIE: Yarn issue count is around 800 -- CHARLES: Boy, that's a lot. I can feel daunting when you look at that, right? KAYLIE: Yeah, but it's actively [inaudible] project, a lot of people are passionate about it and there are bots that other projects use to just automatically close issues when they're not active. But something that BYK told me was when issues are closed, people take it somewhat personally and we just want to make sure that there's like a human touch to it and we, at least get to the issues without just automatically closing them. I really respect that. I think even when people are really frustrated, the maintainers never really lose their shit. They're always very graceful and when someone is acting outside of the standards of Code of Conduct, there's a gentle reminder. So far, all of the interactions that I've seen to the issues have been really constructive, rather than being like, "Sorry, we're not going to work on this." It feels like it is a community project and people care about how it's being used. It's actually not easy given all the different operating systems that Yarn has to accommodate. It's like a pretty low level tool so I give the guys a lot of kudos for handling that. CHARLES: Eight hundred issues means that with a human touch, that means 800 people have to actually respond to those issues and usually those 800 people are not actually 800 people. They're like 10 people or some number significantly below 800. How do you attack them to try and make sure that they're responded to in a timely fashion? KAYLIE: I think issue tracking is the hardest part of open source. I think it's in the order of issue tracking and then reviewing PRs and then submitting PRs because writing code is writing code but understanding other people's code is more difficult and understanding other people's issues and what the bug is, actually is even more difficult. You don't, obviously have their exact dev environment so sometimes, it's hard to repro. I think we do a lot of things that a lot of other projects leverage, which is we label the issues, we define priority if it's actually impacting a lot of people or if it's a critical bug like you can install a package versus maybe a warning message that could be tweet. Then we kept a board, like a GitHub board to track issues when we were heading for the 1.0 release. I'm not sure if we're still doing that but that helped to a degree. Other people from the community, not just the maintainers, jump in to try and help out an issue, which is probably one of the best things. Then when we merge pull requests, we close the issues that have been referenced. CHARLES: Yeah, it's really wonderful when that thing happens. I'm so curious like how to [inaudible] because I know that it's so disheartening when you're working on a project and you file an issue and maybe, it doesn't even go answered for one day, two days or two weeks or you submit a pull request and no one's even commenting on it. It can be really disheartening and kind of make you question the viability of the product itself, especially if you see a lot of activity elsewhere. On the one hand, I also understand that the maintainers are human and they probably have a lot of obligations. I know that I've got a lot of projects where I let the issues languish and I even have one that I'm using where I can't get any response and it's just so frustrating. KAYLIE: Yeah, even Yarn is definitely not perfect. There are definitely issues that sort of go buried. You could add us at YarnPackage/Core. That pings all of us and the core team. You could call out specific people but that's a tough one. CHARLES: This isn't with Yarn, by the way. It's a totally separate project. It's fantastic when there's that basic acknowledgement. It makes such a difference in people's perception of the entire enterprise. Looks like we're actually approaching time. If we want to give a special shout out to any talks you're going to be giving, any events that you'll be at or -- KAYLIE: Actually, Ben on my frontend platform team is speaking at Nodevember about Next React. I think that's where the shout out. And then Christoph will be speaking at AgentConf in Austria, I think in January about the future of dev tools. I think both things are [inaudible]. CHARLES: All right. Fantastic. ELRICK: Any slick future features coming out in Yarn that we should know about? KAYLIE: I'll keep you updated. CHARLES: She's sworn to secrecy. KAYLIE: Yeah. I'm not sure what I'm supposed to [inaudible] yet. CHARLES: I could hear the pause of conspiracy in your voice. Thank you so much, Kaylie for coming on the show. This has been a really wonderful conversation that's just gone all over the map and those are my favorite kind. Thank you very much. KAYLIE: Thank you so much for having me. This has been a lot of fun, guys. CHARLES: Well, if people want to get in touch with you, how would they go about doing that? KAYLIE: I'm a rarity, where I don't have Twitter. You can email me, I guess. CHARLES: File an issue on Yarn. KAYLIE: Yeah. I'm KaylieEB on GitHub and KaylieKwon@Gmail. CHARLES: As always, you can get in touch with us. We're at @TheFrontside on Twitter or just drop us a line on the email. You use that on occasion too at Contact@Frontside.io. Thank you, Kaylie. Thank you, Elrick and thank you for listening.
Dan Gebhardt: @dgeb | Cerebris Show Notes: 01:33 - The JSON API Spec and Pain Points it Solves 08:40 - Tradeoffs Between GraphQL and JSON API 19:33 - Orbit.js 26:30 - Orbit and Redux 32:22 - Using Orbit 37:24 - What's coming in Orbit? Resources: orbitjs.com (Guide Site) ember-orbit Transcript: CHARLES: Hello everybody and welcome to The Frontside Podcast, Episode 87. My name is Charles Lowell, a developer here at the Frontside and your podcast host-in-training. Joining me today in hosting the podcast is Elrick Ryan. Hello, Elrick. ELRICK: Hey, what's up, Charles? CHARLES: How are you doing today? ELRICK: I'm doing great. CHARLES: Are you pretty excited? ELRICK: I'm very excited for this podcast because this is a topic that I've heard a lot about but don't know much about and it just seems so awesome that I'm just very stoked to hear all the details today. CHARLES: Yeah, me too, especially because of who's going to be giving us those details, he's one of the kindest, smartest, most humble and wonderful people that I've had the pleasure of meeting, Mr Dan Gebhardt. Hello, Dan. DAN: Hey, Charles. Hey, Elrick. Thanks for having me on. I really enjoyed listening to this podcast. It's nice to be part of one. CHARLES: It's good to have you finally on the show. We talked over chat and we talked over email and we meet every once in a while and conferences and it's great to get to share more widely some of the great conversations that always arise in all of those contexts. For those who don't know you, you are a founder at Cerebris and that is your company, which is involved very heavily in a lot of open source projects that people are probably familiar with. One of them that we're going to be talking about today is JSON API. I bet most people didn't know that you are one of the biggest driving factors behind both the specification and several of the implementations out there. DAN: Yeah, that's been a pretty core focus of my open source work for the last few years. Actually JSON API Spec, which is perhaps a somewhat confusing name for those who aren't familiar with it. It was started by Yehuda Katz in almost three and a half years ago, I think now and it hit 1.0 a couple years ago and has stabilized since then and we've seen a lot of interesting implementations on top of it. There are some exciting stuff that's actually coming soon to this Spec that I'd like to share with you guys today. CHARLES: To give us a little bit of context, why? What pain am I experiencing that JSON API is going to solve or it's going to address or give me tools to deal with? DAN: One of its prime motivators is the elimination of bikeshedding. There's a lot of trivial decisions that are made with every implementation of an API and JSON API makes a lot of those decisions for you about how to structure your document, how to include relationships and lengths and metadata in a resource, how to represents relationships from hasOne/hasMany. Even polymorphic relationships have a type of that data. JSON API has opinions about all these things at the document structure level and it also has opinions about our protocol usage, how to use HTTP together with this media type to make requests and for servers to return responses, how to create a resource, how to add resources to relationships and things like that. CHARLES: Also, it's not just this is a serialization format. It's very much like also delving into the individual interactions and how they should be structured, more about the conversation between client and server. DAN: Yeah, in that way, it is somewhat unusual as a media type that covers both. CHARLES: Can you dig into that a little bit because I'm very curious? Something made my ears prick up was when you said, it tells you how to, for example add relationships to a resource. What would that look like? DAN: A lot of the influences behind JSON API are hypermedia-related. It's influenced by RESTful principles and includes a lot of hypermedia aspects. One aspect is how a resource represents relationships in terms of the data in document, the type and the ID that specify a linkage to that another resource in the same document but it can also include links to discover those relationships. There's a self-link for a relationship and a related link for relationship and the self-link will return the data for that relationship in the type/ID pairs. The related link will return the related resources. The Spec doesn't have strong requirements or any requirements about URL usage but instead, it describes where to find resources through these hypermedia links. If you want to say, add records to a relationship, you'd follow the self-link for that relationship when that was returned with a resource. Then you would send a post to that endpoint and you would include the relationship data in the terms of type and ID pairs. It gets down to that specifications so that removes the ambiguity of how to interact with these resources and mutate them and retrieve them. CHARLES: I see, so is there an idea then that you are going to explicitly model the relationships as individual resources? Or is that the recommendation or the requirement? DAN: The link for a relationship would point to an endpoint, which would then model the relationships that are represented that endpoint, so to say just to speak a little more concretely because certainly, it makes some simple concept sound a lot more esoteric than they really need to be. Let's just talk about an example. Let's say, we're talking about articles and comments and maybe an author. Let's say, you've fetched a collection of articles from an article's endpoint and within the article resource, you would have a relationships number, which would include comments and then comments could have links, which one of the links would be a self-link and a related link. The related link could be followed to then retrieve all the comments for that particular article. You could also, if you wanted to add a comment for that article, post to the self-link for that relationships. You post to that whatever endpoint that is specified. Maybe it's 'articles/1/comments.' It could be anything that you want. Now, the Spec does have some recommendations to make everything fit nicely in terms of the URL design patterns and such but those are not, by any means required but having those recommendations just eliminates more bikeshedding opportunities. We find it that people who are gravitate towards the Spec really appreciate having a lot of these trivial decisions made for them so even if we don't want to come down and be hard line about requiring those particular answers, we can at least provide some guidance for how things can work together nicely. There's a whole recommendation section on the site for things like URL design patterns. CHARLES: Right, so things that aren't prescribed but these are best practices that are recognized. DAN: Yeah, exactly. CHARLES: A question then that comes to mind, it sounds like JSON API solves a lot of these bikesheds or just kind of comes in and takes one side or the other for modeling both the resources and the relationships between those resources so there's the... I don't want to call it a schema but the boundaries around which resource are very clear and where they live and how they connect together. I was hoping we could maybe contrast that with some another approach, which is also become very popular and that's the GraphQL approach where you're essential assembling views at runtime for the client. It's very easy to marshal the data that you need to present to your view because you've got only one endpoint, as opposed to having to coordinate between them. I can understand the appeal of that and I was wondering if you have any insight into what the tradeoffs are between the systems and what are some of the capabilities that one can do that the other can't. CHARLES: Yeah, sure. I'm glad that you brought that up because I feel like GraphQL has become a real juggernaut, at least because of its marketing. It's very effective in being marketed for its use to developers and it's capabilities versus REST, as if a RESTful system can't possibly achieve the same outcome or the same efficiency. I'm glad to compare and contrast the two. To be honest, one of our short term goals is to better tell the story on the JSON API site, which was always a kind of a more technical spec-y site and a marketing site. That hasn't really helped its uptake as much as it could as some of the GraphQL sites are very sleek and polished. Anyway, let's get down to it. GraphQL allows you to basically define the data that you want for a particular view and that can bring together multiple related resources. It defines a way to specify exactly which fields you want in that graph of resources. We'll just stick with the articles, comments and authors example. You can specify that you want a collection of articles and perhaps the comments-related to that and the authors and you could have it all assembled in a single response. JSON API also allows you to do just that. It allows you to make requests for multiple related resources to constrain the fields that are returned for each resource and to include all of these related resources in a single document. The main difference in the representation is that JSON API requires that resources only be represented once in a single document. GraphQL may have repetition of resources throughout the document that's returned. For an instance, your articles that may nest authors and those authors like Charles Lowell, may have written three of those articles and that representation of that author is going to be repeated in that JSON API compound document, which is a term for document, which has a primary dataset combined with related resources. That single author would only be returned once as related resource and the linkage between the primary data and the related resources would be established to type/ID pairs. Instead of having the author represented three times, the same type/ID pairs would just be providing that linkage to the same author and that author resource would only be represented once. This happens to be ideal for client-side applications that number one, basically want to minimize the size of a payload that sent. Number two, don't want to after-handle repetition of data by doing extra processing of pushing the same record multiple times into a memory store that is keeping that data. I think that GraphQL is well-suited to applications that request data and display that data pretty much as returned. There is no intermediate holding onto that data in, say a memory store for later access. Basically, it lines up well with a component library link React, which wants to display that data that's returned from the server. If it wants to display that collection again, it will simply request that collection again and pretty much throw away the data once it has been rendered. CHARLES: I can see that. Dan, you and I might be some of the only folks who remember. I don't know if you ever did any Microsoft Access Program. DAN: Yes, I did, believe it or not. CHARLES: Doesn't it feel a little bit like the Access pattern all over again, where you have your components requesting data from basically, constructing a query and requesting it -- DAN: Yeah. CHARLES: And then throwing it up on the screen. DAN: You're going deep there but I do remember that. Definitely, there is that same paradigm. CHARLES: It's really powerful. DAN: It is and it's pretty accessible too because it's a direct representation of what you've requested and there's no intermediate processing. I guess the question is, whether that intermediate processing provides some value. Actually, holding onto that data provided some value because as far as I'm concerned, GraphQL is great for that rendering of DOM data, where the data has no meaning except outside of the rendering. But if you want to actually have models that have some intelligence about that data, then you want to use a store to keep those models in and you want to be able to reuse those models for other purposes. CHARLES: What might be an example? What's a concrete use case that we can ground this discussion? DAN: I would say that the big one is offline. You simply can't have just DOM data that's useful in any way in an offline application or an optimistic application, where you are doing some things client-side and only say undoing them if a request fails. But if your data is DOM and only structured for a particular view, then all you can do with that is redisplay that view. But if you understand the schema of your data and that data is available in a store, then regardless of whether you have a network connection, you can actually display that data in different ways. If that same article shows up in a collection in a list, you could also display that article on its own in a different format with more fields. If you want to, say allow editing of that data, you could allow for an editor when your app is offline. Allow changes to be made to that data and then redisplay it because you understand the fields that are in that data. CHARLES: Right and then at some point later, then spool those changes back to the server. DAN: That's right. CHARLES: It almost sounds like, ironically if a system like JSON API where you have very concrete boundaries around each of the underlying resources in your data model, it allows you to essentially do rich-querying on the client and not just the server. DAN: Yes, that's absolutely true. CHARLES: Because I feel like what you just described to me it's like, now we have some sort of store over which we can map all kinds of different queries to our own liking and there's no dependency on the server. DAN: Yeah and if you just want more web app to be pretty much a view representation of what's on the server and without additional intelligence, then GraphQL really lines up well with your needs because any extra processing you're doing is just not valuable to you. But I think a lot of the really interesting things being done in client-side applications are where your client application is pretty loaded with a lot of intelligence and you're out there autonomous and able to make sense of data. In that case, then thinking about the data only as it pertains to views is not nearly as powerful. CHARLES: Right, so you could do something like that with GraphQL but then you would have to, essentially structure your queries such that they drew the boundaries around the individual resources anyway, rather than composing them on the server. You'd have to query them discreetly into a store and then run your local operations. Then I guess at that point, it's like what are you doing? DAN: Yeah, you're still doing the extra processing of handling the repetition of any nodes that repeat and such. That's just extra processing you have to do but I agree that you certainly could structure your GraphQL queries to return data that is then loaded, say to a store that really has awareness of the data types but I don't think that is -- CHARLES: But then you're defeating the purpose, right? DAN: Yeah, it's not its selling point and it's not its strong suit. CHARLES: You've done a lot of work on the JSON API Spec. JSON API allows you to fetch discrete resources and their relationships but still, keeping one representation of each resource in the payload so it's optimized for wanting to do client-side processing and have intelligence based on these entities, which are in a store. You actually maintain a fairly mature, at this point, framework called Orbit, which helps you do some of these things. Now, what Orbit does today and I understand that you've got a lot of new features that are really exciting, that are coming down the pike. Before we get into those, what is Orbit and what do you use it for and how does it use JSON API? DAN: Orbit is a data access and synchronization library, which sounds sufficiently vague because it has a lot of low-level primitives for structuring client-side. Also, actually isomorphic can be run on the server and nodes so it's not even only used for client-side purposes but that was its original purpose. The abstraction that it includes are allowed for synchronization of data changes across multiple sources of data. Source of data might be represented by, say a JSON API server, an in-memory store, an IndexedDB database in your browser, a local storage, all of these sources of data can support an Orbit interface, which provides their access to their data and also broadcasts changes to that data. In order to coordinate the changes across multiple sources, say to back up all of your data that's in memory to IndexedDB source, you can observe the changes on one source and then sync those changes up with another. For instance, you want to structure an offline application which you have been in-memory store, which uses client-generated IDs, which then syncs up with a backend JSON API source and every change that gets made to the memory store needs to be backed up, you could configure multiple coordination strategies between the sources to make sure that the data flows so that every change that is made to the store is immediately backed up to IndexedDB. If it can't be backed up, then it fails. You can add some error handling and then when you're online, you can then also sync those changes up with a backend so you're basically pushing those changes that are local to a remote store and you're not slowing down your offline app, which you're communicating with optimistically and then only handling, say synchronization failures when there is a problem. In order to handle those problems, Orbit sources are very deterministic about their tracking of changes and they provide get-like rollback capabilities so you can look at the history of changes to a particular source and reset the history to any point there and basically handle conflicts and merges in a very get-like way. Often I use cases, the primary driver of Orbit's whole architecture, I realized that it needed to be able to give you the tools to handle any conflicts that happen when changes get sync up. Also, give you the different tools to model all the different places of data is kept in order to support the offline mode. That's a kind of a broad overview of Orbit. There's a new guide site, OrbitJS.com for those who want to dig a little deeper into it. The data is structured in the JSON API format internally to the store and the standard operations are very much influenced by the standard JSON API protocols that are allowed in the base Spec over creating records and removing records and all that crud for both records and relationships. That's where JSON API comes into Orbit. CHARLES: Right, I see. The primary use case for Orbit is offline. Is that fair to say? DAN: Yeah, that was the primary driver, although it's just not the primary -- CHARLES: It seems like you could use this in a lot of places where I might use Redux or something like that, like on the server to model... I don't know, a chat app. DAN: Yeah, definitely. CHARLES: I have a bunch of different information streams coming together and how am I going to merge them and make sense of them. DAN: Yeah, in fact, that it's primitive level. Orbit has essentially an async redux-like model for queuing up changes and applying those changes. The change sets are all immutable. There's actually a lot of immutability use here throughout the library. In order to ensure that the changes that are applied are tracked deterministically, we just can't have those changes mutating on us. There is definitely some overlap with Redux concepts in terms of the general tasks or action concepts in Redux but instead of Redux's synchronous approach, everything in Orbit is async. CHARLES: What does that mean? Redux is synchronous in the sense that there's a natural order to all actions. For those of us familiar with Redux, are you saying it would be like a store where actions can be dispatched at any time or is it more like, I've got multiple stores happening and I need to resolve them somehow so each one is synchronous? How can I make sense of that? DAN: In Redux, the actual application of an action is performed synchronously. CHARLES: Right. You can have asynchronous processes but there is a natural order to the actions that those asynchronous processes yield and then those are applied synchronously to the Redux Store. DAN: Yeah. To compare and contrast Orbit and Redux, I guess you'd first have to say there's a primary difference of -- CHARLES: I think there are a lot of people are familiar with Redux. I think it's not so much to compare and contrast it but just to use it is as an analogy of like, "Here's how it's the same here. Here's how it's different," because that's compare and contrast. DAN: There you go. CHARLES: But not in terms of evaluating them but it's like, "Maybe I should be using this instead." DAN: Right, they are sort of on different levels, although there are some primitives in Orbit to it and it's shipped across multiple libraries. There are some primitives, I think that could be useful outside of the main Orbit data application. Anyway, the way that Redux state changes are applied, the function is synchronous is all I was getting at and on Orbit, every state change that applied to a source is asynchronous so the result is never applied immediately. You'll always get a promise back and you'll never have that application happen immediately. That's one clear distinction. Another is that a redux has a big singleton global state for the entire application. Orbit very much has a model of state per source so there can be any number of sources in a particular application and the source might be an in-memory source or might represent a browser storage in XTP or might represented a socket that streaming data in. All of these have state at different, temporarily distinct state that even if they all converge to a common state, the Orbit models separately so that there's a set state per source. I'm just contrasting the global apps state that exists in Redux with the per source state in Orbit. CHARLES: It sounds like there's nothing that would be fundamentally incompatible of using Orbit really in conjunction with Redux, where Redux is kind of a materialized view of all of your different data sources presented as what you're going to render off of, right? DAN: Yeah. You could use it in a similar way to Redux Saga, where Orbit fills the role of Saga, where it's doing the asynchronous actions that results flow back into the Redux state. CHARLES: I'm just imagining having one big global atom, which is your Redux store and now I'm saying, prescribing this is an optimal architecture but I'm saying, one way it could work is it picks and chooses and assembles off of the different sources as new data becomes available. As the states change for those sources, it can be integrated into a snapshot state, which is suitable for rendering or provides one view for rendering. DAN: Yeah. You're basically talking about the in-memory source, perhaps merged with other applications state, which is not so resource-specific and that is possible to model. CHARLES: What I think I might be hear you saying is you could also just use another source which is the merge itself. DAN: Yeah. I'm not sure how much we want to continue this thought exercise because the architecture becomes almost not something I'd recommend. But I would actually like to explore how Orbit and Redux could be used together optimally. I played around a bit with Redux but I have not written a full-fledged application with it, other than a [inaudible] location. I definitely defer to you for Redux best practices and such and how people are using it in real world applications but I'd be really interested to talk that over again soon. CHARLES: Well, I just certainly don't count myself a Redux expert, although we have developed some applications with it. We'll put that on the back burner or something to explore it later. I will say this, I find Redux to be both wonderful and terrible, kind of in the same way the Java is both wonderful and terrible. We'll leave it at that. DAN: Okay. ELRICK: That was going to be my question. This is why I was very excited to hear about today was Orbit because I've heard so much about it. In terms of the implementation of Orbit into an application, what would that look like from a high-level? Has anyone used Orbit in the production app? Have you built any apps using Orbit? DAN: Yeah, definitely. There are people using Orbit with React, with Vue, with Angular and with Ember and there's an integration library called ember-orbit which makes Orbit usage really easy in Ember. In a lot of ways, working with ember-orbit feels a lot like working with Ember Data but it allows a lot more flexibility. I suppose one of its strengths and weaknesses is there's a lot of configuration that's possible because there's a lot of possibilities. The internals are exposed of how data get synchronized so you can define your strategies and sync up different sources. In terms of how it's actually used in an application, you'd start by modeling your data in terms of the resources that are in the application. You'd have a schema that defines your articles, your comments and your authors, just to keep that example going. Then that schema would be shared among all the sources in your application. You would have one source, say that might be the in-memory source and another source that is the representing a browser storage so you could, say swap out either local storage source or an IndexedDB source and use either one to provide that backup roll. You would declare those sources, you connect them to each other with strategies so that, say when memory storage changes, you would then sync that change to the browser storage source. Then you'd have back up and you'd be able to then, refresh your page and view the same data you were looking at before. Now then, if you probably want to wire up a remote source so that you're communicating with the servers so you bring in JSON API source and you would then set up a new strategy for working with that. You have to decide like, "When my memory storage changes, do I want this change to happen optimistically or pessimistically?" By that I mean, "Do I only want it to appear successful if it's been confirmed by the server." Depending upon whether you want to be optimistic or pessimistic, you setup your strategies a little differently. If you handle this change pessimistically, you wanted to block success on the successful completion of pushing of that change to your remote server. You have the set of tragedies that define the behavior of your application and then doing your crud operation is probably pretty much directly with your memory source. Then if you wanted to, say do an edit in a form, you might fork the store, now the store keeps its data in an immutable data structures. That forking that store is very cheap so you don't have a bunch of data that's copied. You're just keeping a pointer to that and getting a new pointer to that same immutable data structures. Every time they get changed, there been new references. There's an immutability under the hood but you're pretty well insulated from the annoyances of working with that immutable data structures. At that form, you make your changes, you then merge your changes back, you'd get a condensed change set of operations that then can flow through your strategies. It flow through to your backup source. It could flow through to the back to the server. I think it would feel pretty familiar for users of Ember Data because there are a lot of the API influences came from that library. But obviously, people are using just plain Orbit with other libraries, with other frameworks and finding it useful there but it definitely involves a little more configuration up front to do all that wiring that might be more implicit in library like Ember Data. CHARLES: I understand that before we go, there is some pretty exciting new things coming in Orbit. Do you feel like you're ready to mention a couple of those things or they've been kind of mixed in with the conversation? DAN: Let's see. I have the guides up, which I mentioned, which is pretty new in the last couple of months. In the last year, we did a rewrite and Orbit is now completely in TypeScript and there are no external dependencies. For a while there, I was using RxJS and observables internally and immutable JS so there's now an internal immutable library. It's lighter-weight with fewer dependencies now. I'm excited about that and finally feel like I can recommend people digging in with the guides that are up. I'm hoping to get up the API docs soon. I will say I'm excited. I just got back from a retreat in Greece. Séb Grosjean who owns the company, BookingSync does this amazing thing with the Ember community, where for the group that's working on Ember Data, he invites them every year to come to his family's place in Greece. He grew up working with his family on his rental properties, which was the inspiration for his company, BookingSync and said, "This is a fantastic opportunity that for us to get together and collaborate in a really nice place," and I had a really productive time this last week. This is the very first time I had gone. It was just fantastic and I worked with the Ember Data team. Igor Terzic and I spiked out some interesting collaboration between Orbit and Ember Data so I'm really looking for it to see where those go and hopefully, we'll see a little bit more Orbit, either directly or just through influence appearing in Ember Data. I'm looking forward in working more closely with the Ember Data team. We'll see what comes of that. CHARLES: Yeah, I, for one am very excited to see it. I'm resolved now. I'm just looking at these guides. These look fantastic and I'm resolved to give Orbit, at least a try here, either in some of our applications or maybe trying to spin up some new ones and have it the basis for some of ideas I've been playing with. DAN: That would be awesome and there's a [inaudible] channel, which I hang out into if you have any questions, if anyone out there does. CHARLES: Before we go, if anyone is interested in JSON API, is interested in Orbit, is interested in Cerebris, we mentioned a lot of things that in one way or another, map back to you. How do we get in touch to find out more about these different entities/projects? DAN: I'm at @DGeb on Twitter. My company site is Cerebris.com. Also check out, OrbitJS.com for the new guides. Reach out to me. I'm on the Ember Core Team so I'm also hanging out in the Ember community Slack, depending upon what you want to talk with me about. I'm in all these different places so I love to hear from you all. CHARLES: All right. Fantastic. We'll make sure that we put those in the show notes and I guess that's about it. Do you have anything else you want to leave folks with, any talks, papers or big news coming around soon? DAN: Something that we didn't really get a chance to talk about today, which I'm really excited about is JSON API operations, which is an extension to the base Spec, which I'll be proposing very soon. There's a future to the JSON API once it hit 1.0 a couple of years ago. It didn't just stop. We're looking at different ways to extend the base Spec and use it for different and interesting purposes. JSON API operations, I think one of the most interesting ones. The idea is basically to allow for multiple requests that are specified in the base Spec to be requested in a batch and perform transactionally on the server so the Spec will define how would each request gets wrapped. Each operation very much confirms with the base Spec concept of a request. For implementations, there's a lot of opportunity to reuse existing code for how to handle each particular operation but to provide a whole new set of capabilities by allowing you to batch them together and process and transactional it because it just unlocks a ton of different things you can do, all based on the same base concepts from JSON API. I'm really excited to have something to announce soon about that. CHARLES: That sounds like it might solve a lot of problems that are always associated with those things. It always comes up. What's our batch API look like? I don't think I've been on a project that didn't have a months-long discussion about that. I ended up like kicking it down the road and I'm just flumping something in place. DAN: Yeah, all those messy edge cases where people figure out how do we create multiple related records altogether in a single request and people do it ad hoc and do it with embedding and such and want to standardize that in the same way, that we've standardized the base operations. CHARLES: Well, that is really exciting, Dan. I wish you the best of luck and we'll be looking for it. DAN: Thanks a lot. Thanks for having me on, guys. CHARLES: It was our pleasure. Thanks. With that, we will say goodbye to everybody. Goodbye, Elrick. Goodbye, Dan. Goodbye everybody listening along at home. As always, you can get in touch with us. We're at @TheFrontside on Twitter or you can see our website at Frontside.io or just drop us a line at Contact@Frontside.io. Always love to hear from you with new podcast topics, anything that you might be interested in so look forward to hearing from you all and see you next week.
Alex Matchneer: @machty | FutureProof Retail Show Notes: Charles and Alex Matchneer have a great discussion that centers around routing in Ember.js: what they want to see in a router, what problems it solves, what's wrong with the routing solutions we currently have, and what the ideal future looks like in respect to routing. Resources: Episode 067: ember-concurrency with Alex Matchneer Cordova ember-rideshare react-router Transcript: CHARLES: Hello everybody and welcome to The Frontside Podcast, Episode #86. My name is Charles Lowell, your developer here at the Frontside and podcast host-in-training. I'm flying solo today. It's been a while but that's okay because I've got a really fantastic guest on. Actually, we debated this at the beginning of the show, whether this was the third or the fourth time he's actually been on but no times are too many so hello, Alex Matchneer. Welcome back to the podcast. ALEX: Thank you. It's great to be back. CHARLES: You're still at the same place that you were the last time. ALEX: Yeah. Still working at FutureProof Retail. I'm still working on bunch of mobile ember-cordova apps and that's definitely occupying on my time. CHARLES: Nice. Because FutureProof Retail has a large hardware component and we were doing a series on IoT, we were originally going to have you on the show to actually talk about that experience of what it's like to be a part of a startup and develop software that's going to be running on a bunch of devices and the unique set of problems that poses. But in the pre-show, we decided to scrap that because there's actually a topic that we're both very interested in and you've been heavily involved in lately and might be a really interesting preview as to what's coming in the Ember community and at large. Today we're actually going to go back to talking about the same subject that we talked about in our first podcast, which is routing: what we want to see in a router, what problems does it solve, what's wrong with the routing solutions that we have today. Talk about what that beautiful, ideal future that we want to live in looks like with respect to routing. You've been thinking about this a lot lately. What have you been thinking? ALEX: I'm an Ember core team emeritus and back when I was on it and I'm a lot more active, I did a lot of work on the router, particularly with how it handles asynchronously loading data when you click on links and go to different sections of your app. I spend a lot of time over the last three or four years figuring out the nice patterns for what you actually want to use if you're building out lots of Ember apps. Then kind of around that time, right after landing some cool stuff and some not cool such us query params, which has been a challenging aspect, I start working at this company FutureProof Retail that is like 90% of the Ember work that I do there is in mobile apps. We use Cordova so we're basically running these apps inside a web view, inside either iOS or Android so that we can stay with the technologies we are most familiar with, such as JavaScript and CSS and HTML and build apps using that. We can use Ember to do that. What I found was that I couldn't really apply a lot of the same patterns, all these nice conventions that Ember router gives you. I couldn't really find a way to map that onto what I need to build in mobile apps and there's a few different reasons. I got really busy with the startup, just trying to build these things and kind of went off the happy path where I really just couldn't find a way to make it look like an Ember app. One of the nice things about the whole points of convention over configuration as this sort of Ember and Rails philosophy is that, one of the benefits is that if you know Ember and know Rails, you can drop into someone else's apps as long they're following these basic conventions and immediately know how to be productive and know how it's structured, know how to make a change to it and have it maintain a convention and not just have everybody who's using some framework build these totally different apps from each other that have no shared conventions and whatnot. Everyone is supposed to be able to learn from each other, grow with each other as long as they stay with these conventions. I couldn't really find out how to stay within Ember conventions and build this mobile apps. For a long time, I just didn't really contribute too much to the Ember router at all. I kind of fell out of touch with how most people are using it because most people are building these desktop-centric apps and here I am working on these mobile apps after three years. CHARLES: What are some of the specific use cases that were just impossible to, or not impossible but presented a challenge? ALEX: The first one is which is I think is actually one of the easier problems to solve but still some challenging is that you want something that's called stack routing or stack navigation in a mobile app, which is if you're actually building a native iOS app or an Android app, they both have different names for how they provide you this. But you're thinking of things in terms of stacks. In Android, you might open another activity, which is a full frame of a page in your app and you can push it and then when you press the back button, which is built in in Android phones, it'll pop that off the stack and take you back to where you were. In iOS, they give you a UI navigation controller and let you push and pop view controllers and that is how they want you to think about these applications. That is contrasting to what Ember makes you think about, which is go and define your static hierarchy of all the different places that you can be in an app. But with stack-based navigation, you don't necessarily know upfront all the different orderings of which frames are going to be pushed onto what and you might have situations where you want to be able to dynamically push, say an 'Add a Credit Card' page to where you are and maybe it depends on some data that's been loaded at some lower level in the stack and you can't model that as nested routes in the way that you might think about it in classic Ember apps. It's a different structure -- CHARLES: Now, when you say lower in the stack, I'm curious, if you're entered in aren't you... Oh, you mean... I see, previously in the stack. Okay, so lower in the stack so you're thinking like your current position is at the top of the stack. ALEX: Right, yeah. CHARLES: I see. Now, let me just clarify this in my own head. Your Ember routing structure is ultimately realizes a static tree but at any moment, you are entered into one path through that tree so you do have something resembling a stack. It's just is it the pathways that the ways that you can actually get nodes onto the top of the stack is you're limited because that can't be dynamic. ALEX: Yeah, but even then, it's hard to describe what the difference is but the kind of stack that you're thinking of in terms of the classic Ember router map is more like you're in these different substates than you are different frames that you've pushed onto your -- CHARLES: There's a finite and fully enumerated set of next states. ALEX: Right. To be very concrete, if you have a post route and then a post show route and then a comments route under that and these are all nested in a row, then if you're in the comments route, you are in a kind of hierarchical stack that might have loaded the post that you're looking at and maybe the post call-to-action above that and the comments for that but you're still in one thing. You've just expressed that one thing in terms of these substates so that every other state that's in the parent state can share the same data loading. That's different from saying, "I'm on this page and now, I want to push another page on it and maybe tap some of the data that has been loaded on previous pages." That's more of a navigation stack in a hierarchical substates stack. CHARLES: Is the difference then, the data dependency? Because if you think of the Ember classic where you got the static tree, at least theoretically all of the data in the leaf nodes depends on the data that's above. It's not just being able to dynamically push stuff onto the new stack but it's also saying, you want to be able to push stuff that might have no dependency on the stuff further up and it doesn't need to be re-rendered if stuff further up the stack changes. ALEX: Correct. CHARLES: But sometimes it might. ALEX: Right so there are a lot of corner cases that come out if you try to model this new way that a lot of corner cases have been thought out of if everything matched nicely to this hierarchical substate classic Ember stack but not for navigation. If you want to do something that's stacked routing-based, I've had a few different approaches. At our company, we maintain a suite of different apps that are sort of retailer or grocery-centric and the first one we did, which is more popular flagship one is Mobile Checkout, which is an app that lets you going to stores, scan items with your phone and checkout and skip the cashier line, which is great if there's huge lines and you just want to buy a little handful of things or maybe in your shopping cart. But that is like any other mobile app is really conducive to this step navigation approach. Then we had to make a few apps after that such as like another app that is [inaudible] do a manual check then ordering app and other of handful things that you can imagine is might be used on a grocery store. I took the opportunity to like, "I don't really like how the routing turn out the main mobile checkout shopper apps so let's try different things." If you approaches, at least have their pros and cons without really feeling you're solving the problem and one is to maintain your own in-memory stack of where you were, every link to you, you might recall where you were and then use that logic in addition to what's in a URL to decide what transitions to make, which to use Liquid Fire for that. But already, there's these weird growing questions like, "Why are you even using the URL? Is it helping you at all?" That was the main issue with the main app that we did. The other approach was to try and not even use any of the 'router.map' stuff at all. I use the router.map to basically just create one wildcard route. You can use normal Ember to use it like '*half' and that basically collects the rest of the URL as a param that you can use to do whatever you want with. I was using that to basically pass to another, which is internally used by Ember to do the stack-based parsing like grab a little bit of the URL and then parse the param for that then grab another. Every time you could see your stack in the URL. That has its benefits but the worst part about it is that it's getting further away from Ember so any add on that you might want to use at Internet of Things in terms of which route you're in and has conventions like that you just can't use. I can't think of a good example at the top my head but it's like the further you get away from those norms, the less the Ember system can help you and on your own building your own framework. This is all to say that I think I have enough experience at this point to bring home some of the things to Ember and I'm excited to get back into contributing to Ember with this one particular thing that I'm focusing on now, which is... I don't even know what to call it. It's like -- CHARLES: What does it do? The route stuff? ALEX: It's route stuff. Actually, let me get into the other... That's what is tricky about stack routing and tricky to sort of, if you already have to go through a mental hurdle with thinking of the Ember router and as a stack of states or substates and you train your brain to think that way, it's really hard to take yourself out of it and realize that what you're trying to build with like a classic mobile navigation is almost looks like the same thing but it's really different. The other challenging problem, which is specific to our particular app is that you wouldn't think of it as a very heavily server-driven app but if you're writing an application that at any point can get a message from the server like, "Hey, your status has changed," and that state is heavily coupled to navigation of where you're allowed to be in your apps for the state of some certain model, then you're going to have a really hard time, I'd say in modeling an Ember. I have a really hard time convincing people of this until they've actually tried to do it themselves, which is why I'm going off and just building things showing people. CHARLES: You don't have to convince me because I think one of the biggest problems is the router is like the one non-reactive piece of Ember, which is unfortunate because it's essentially, what is the equivalent of the Redux store in a Redux application, where it's the state that drives literally the entire application and yet, any type of non-hash change driven updates, you have to manually manage. Every time that we've done it, it's been a problem and depends on what data, at that point you have to be very thoughtful because, at least from the highest level, if there's damage to a piece of the tree higher up, you need to realize those effects of that damage or that change all the way down the tree. ALEX: Exactly. That is a great way of putting it. This is maybe a good time to mention this thing called ember-rideshare. I've had a really hard time describing these problems to people so I figured what I would do is write this blog a few months back, a little article called ember-rideshare. It's just a given name to the kind of app that still really hard to write in Ember. It's a mobile app. It involves stack routing but the other part is really difficult about it is this problem of the router being in a silo. It is reactive but it's only reactive to that URL. Other things changes, they need to, like you said come in and patch up something else about the router in case you add some URL that is no longer able to present some model of whose status changed. That's an article on a blog that I can probably link to in show notes or something. When I talk about ember-rideshare, imagine using Ember to build Uber or Lyft and it's got just the slightest bit of the whole thing. The whole point of the app is to coordinate your client-side request of I want to ride with the server going off and doing a bunch of things and finding a nearby driver, displaying you bunch of driver locations and it'll show up. Then finally, find you a driver. It's a constant communication. Throughout that point, you can sort of imagine modeling all the different screens as routes but the routes that are actually allowed to see at any given time are heavily dependent on what is the current state of the user's current ride. But you shouldn't be able to go to a route that says like 'cancel ride request,' if you haven't requested a ride in a million of these other things. If you're an Ember developer and you think that's an easy problem to solve, you're probably thinking, "I would use before model hook when I'm entering that route to check the state of the model," and if it doesn't make sense for the route of entering, I want to transition elsewhere. That's fine. That's good if you're doing an app if the user is the one deciding where to navigate to. But then when you're on a route like that and then the server tells you that your ride is done, you can't still be on that route so you've got to have some kind of validations that is like, "This is no longer a valid route to be in. Is the user still in this route?" CHARLES: "Where am I going?" ALEX: Yeah. Before model doesn't really help you. It's this one-shot discrete event and you just can't capture all the different things. The ember-rideshare describes some of these problems a little bit more detail but that's the main issue with it. Like you said, what is actually missing about the router? Maybe it's reactive but it's only reactive to the URL, what about all these other things that are happening into your app? I think there's a handful of APIs in Ember that they're great but they're kind of siloed off in a way. If you want to make two different kinds of worlds meet, you've got to write a bunch of your own code yourself or you just have to do mentally going back and forth and being like, "I did this, so I can't use this kind of API." I did a lot of work on the Named Blocks RFC, which previously there is silos between if you're passing blocks to a component versus data, you've got to think about them differently and all the ways that you might forward that data to a different internal component, if you want to build these composable, reasonable internals, you got to be kind of split-brain about it. I feel the same way about how the Ember router works. It's only good at dealing with stuff that has to do with the URL and you're on your own, if you needed to react to data changing. That's what I'm trying to fix. Does that correlate with your experience of working on Ember stuff as well? CHARLES: Absolutely. I think that's a great way to put it. I think we've come to a consensus of the problem statement. I am curious to see a big separate query params. I'm going to throw that wildcard out there or maybe we should save it for later. ALEX: Yeah, I definitely going to come back to it. If I say all this cool stuff and I still don't have a solution to that, then what am I talking about? CHARLES: Right. ALEX: Which to be honest, I haven't thought of every single possible thing. I'm doing the thing where I talk about it on a podcast that everyone can guilt me into really finishing it. I actually really think that I'm going to finish it. I'm very confident in stuff I'm working on. I'm very excited to bring it to people but it is not all 100% fleshed out and I definitely appreciate anyone's help to those interested, understands the nature of the problem and wants to help me work on some of this stuff and like that, in Ember community Slack or wherever. CHARLES: Yeah, I'm really excited to hear it and see in what ways we might be able to contribute. ALEX: Basically, the goal is to find some underlying primitives that can model the current behavior without mistake because obviously, we can introduce something that's going to break into Ember apps. Basically, to recognize that the URL is something that goes through multiple passes of transformation, to eventually become the thing that displays stuff on your screen, from the very foundation of it, and this is the actual mini-course of what Ember router does internally because it involves a few different libraries and maybe this is a re-hash from the podcast that I did with you guys but -- CHARLES: Can I just say that there are some things that the Ember router really does right, that are fantastic? One of those things is it baked in to every single piece of data. It doesn't do the stack but in that tree that it models, every single node in that tree abstracts away the asynchrony of that node. I think that's absolutely huge so you get both the dependency enumerated like these are the things that I need to marshal the data to render myself and it's implicit that it might take some time. I might need to draw on a couple of different things to actually assemble this data so the asynchronous nature is modeled up front and it's implicit and it's there every single time, which turns out to be the right thing. The sampling that I've missed has been an excruciating void in all the other routing solutions that I've tried outside of the Ember community is that they just punt over asynchrony to you. You deal with it, not our problem and it's like, "Actually, that is the problem." Anyway... ALEX: That's a great point because if the router doesn't help you with any of this stuff at all, then it basically means that every one of your pages that you might want to render after the fact, probably has to have some loading logic like if data is loading, show us spinner. Otherwise, here's all the data -- CHARLES: Yeah, if something happen wrong. ALEX: Right and sometimes that is actually what you want to do. Sometimes you want to do these skeletal in UIs that looked like the page that's about to display but the date isn't there yet so everything is, regardless going to be wrapped in these 'if' statements, 'else' statements. I worked in ember-concurrency and some people are using that to basically move more of that loading into controllers, that's fine. If that's what you're actually trying to do and that's what you're opting into, that's a perfectly reasonable solution but most of times, chances are you're entering a route and you don't want to have to teach the entire template tree underneath it that has to handle all these different states. There's these nice ideas that work in some cases and I'd like to make them work in more cases than Ember helps with and a whole loading all the promises and the model hooks and absolutely going into the loading state are really cool primitives that Ember is going to do for you. The other frameworks, they don't try to be opinionated. They won't do any of that for you. Sounds like you ran into that with some of your React stuff? CHARLES: Yeah. I definitely did. There's just not much help when you actually want to model asynchrony. You can do it. It's pretty easy. You just implement the right hooks or model a series of actions, either with a Saga or Epic, if you're using redux-observable. But again, you have to assemble it by hand and you have to generate those abstractions by hand and you just want to have them at hand already and not have to worry about that. But the advantage, though is that generally those ones that you do have at hand or that you generate are fully reactive. If new information comes that's germane to that particular leaf in the tree or that particular note in the tree, there's no difference between the initial state and the update state. Whereas, in Ember, you got your first shot and then that data is now at rest. ALEX: Right. I definitely have been looking at React router, in particularly v4. I think it's all contentious for people to see it at first but being able to put things like in your render function, you can say, "If this data is present, something that's going to be past and be a prop or something," then show a loading spinner or otherwise, start matching these subroutes. That's really cool. That's expense that you can't look at essential map of all the states of your router can be in but that's also a real problem and if you can demonstrates that the state world is not in a separate silo than the routing world. CHARLES: With great power comes a lot of bugs. You do run into a lot of things where you have rogue matching. You have random things that are inside your view tree that are matching against the route and they just render and you have to be very careful because it's almost the difference between blacklisting and whitelisting. I see what you're saying. It could be confusing. ALEX: Yeah. I think it's definitely a tradeoff. I think if I had something like a match, I might have been able to maybe arrive at a stack routing solution a little earlier. I'm not sure about that. It's definitely something that could be handled by React router. I think one of things that React and React routers better at in general is that everything is, more or less a component that is more easily swappable or something else here. You're not going to have as many of those silos and I really do think, it went through a lot of churn and maybe, some people had trouble, maybe a lot of people, I don't know had trouble kind of following all the major versions. But I think React router Version 4 is pretty damn cool. I think there's a fullest realization of that kind of modular mindset. CHARLES: I think the biggest problem I have with it, though is it requires the view tree to model your routing structure. That bothers me. I feel like you could do the exact same thing. You could have a way to express your routes, not necessarily with a separate routing file. I supposed you could do it with JSX or something but actually have it be kind of orthogonal to your view tree. The way you can model this dynamically updating thing that can match against anything and maybe, even express it all in one place. Although once you get a big tree, it could be hard to control that. The part that I've come into most conflict and maybe who knows, maybe I just haven't used it enough, we've only got one application that we're using the router V4 on. But the fact that it's actually in the view tree, it bothers me. It's in the state objects. It's hard to adapt to Redux because that state is opaque. It's the routers controlling it and I would it to be not have to pass through React components but just be like, give me the firehose of the router state. ALEX: Right. I love what you're saying. If I'm going to bring this stuff to Ember, I can't suddenly make it work like matching within the view tree. That's not what I'm working at or proposing here. All the stuff is basically to empower that firehose to respond to more things that can drives views and respond to them in a live way, not like a one-shot async validation, only when you enter. CHARLES: Maybe this is what the problem that you're trying to solve and one of the things it's really nice to be able to match against anything inside the view tree is that Ember's rendering process of a route is very opaque. The process, by which an outlet gets connected, that's not something that you really have much visibility into. Is that a good statement of the problem? ALEX: That's definitely part of it. You definitely have to go to the documents. I think it's telling that -- CHARLES: I've never done it. I don't really know how that works and I've written a lot of Ember code. ALEX: How what works? CHARLES: How the route gets rendered, like the mechanics around, which I understand how the route object actually, you makes the decision to render its template and do all that stuff. I know it as a user but I don't know the mechanics and I wouldn't know how to extend it. ALEX: I'm not sure if the stuff I'd work on but it immediately make some of that stuff more clear. One of the goal or constraints is to really try and break down the silos. Whatever I'm about to propose bringing to Ember, I want it also be something that would be useful, possibly at the component or template or controller level, rather than just being this thing that lives only in the router's weird black box of logic that occasionally calls hooks that everyone knows about. CHARLES: Right. In a sense, I'd say that they both suffer from that same problem. I'm curious to hear about the firehose. ALEX: To actually get into what I think you're building here, we can dance around it all day and then we -- CHARLES: Just save it for the last 30 seconds of the podcast. That way there could be no -- ALEX: We're swapping JS for React router V4. Bye! It's basically this. What's happening today is that you have a URL, it's going to be parsed in a way that you've tied it to via the router map file, which every Ember app has the place to go to see all the different places that you can navigate to an Ember app, which is great. You basically taught Ember how to break your long URL string into these usable bits and that's going to give you an array of these things that internally who cares what they're called but they're called handler infos and they basically say, "The first element of this array is named application. Every Ember app has one. It doesn't have any params." The next one, it starts getting into what your URL actually is. Maybe it corresponds to the '/post' portion of the URL so that's going to be named 'post,' and that doesn't have any extra params either. Then there's this thing that is post show or something like that. That has a dynamic param because that's the part of the URL as like the '/123' and that corresponds to the post ID. It's basically, if you like thinking of things in terms of transformations or observables or mapping and functional transformations, that's taking a URL and turning it into an array of these useful POJOs of information. The goal is to keep transforming that into something eventually has enough data to display and templates and whatnot. In this giant black box of the Ember router, it's going through those transformations and then it's going to go through this long series of using these params and this useful array of POJO information, start hitting hooks on people's routes to load data. Hit before model after model, redirect all these things to give tasteful names to all the tons of validations and checks that you might want to do. You do cool things in your before model hooks, check if the current user is actually an admin to prevent them from going into any '/admin' subroute. That's a really cool place to go and it's also a great convention. If you're new in Ember app, you realize you can't go on this route. It should sort of click in your head and that sounds like they've got one of these redirect hooks to ensure that you're not going anywhere you're not supposed to go. All these things are really still to this day, extremely strong, well-designed, it went through many passes of review before it landed. I think they cater to a certain kinds of user-driven clicking around apps but they are extremely strong to this day. I think the only thing that's missing is the smell. That example I gave like checking if the user is an admin, it's a bit of a smell that is not reactive. It's a hook. If it passes, great. You're in the route. It's not going to keep on checking that. What I want to do is basically, either in addition to or as an alternative to specifying these one-off model hooks or these hooks that you, not only really just fire one time, have essentially what is an async computed property or an async validation that is upfront about things it depends on. Ember is going to be smart enough to constantly reevaluate these things as stuff changes. It can depend on not just URLs or URL parameters but it can also depend on data. If you're thinking about ember-rideshare, which again is the imaginary Ember app that it's essentially Lyft or Uber, if you have a current ride model loaded somewhere, maybe by a parent route or maybe it's some sort of service, you should be able to specify it like an async property or validation that says, "I depend on ride.state," and for all these subroutes, you would want to say that, either upon entry or any point in the future, if the state ever changes to something that I don't know how to handle that go to some default route. That would be already, particularly in my app, which is a subset of a different kind of ember-rideshare app, that would be a huge help because the only other alternative is to build a sibling-central coordinator to the router that isn't the router but has to sort of agree with it and then, every one of these frames that you might push onto the navigation stack, they have to do some little chunk of code and then invoke this logic and be like, "Did the state change? Go where you're supposed to go," and they have to do that logic. It would be, I think a great win for conventions as it has if it's a benefit to make people shout out their states in advance to empower them to shout out also their data constraints in advance so that you get things like automatic redirects and things change, I think that would be huge. I know that would immediately benefit off of it and I think it would fall in the same kind of problem solving that they worked on like Ember-related stuff which people don't realize how big a problem is until they see there's a better way of doing stuff. I think with that being there -- CHARLES: As an example, let's say that you're an admin and then all of a sudden, you got fired and there's an event that comes from a server that's this person is no longer an admin and it wipes out the Ember data store and then redirect you outside of the admin route or something like that. ALEX: Yeah, that's a perfect example. To be pedantic, I think a lot of people do hard refreshes between login/sign-off stuff but if you have it all in your Ember app, that would just happen automatically. You'd still want the ability to have more graceful transitions because one of the tricky things about having stuff driven by data is that you have this giant matrix of like, "If I'm in this state and this event happens, how do I handle it? How do I make it look well-designed to the user?" But you're not going to be able to hit every one of those constraints so to just have some basic logic that's just like, "Oops, something happened," you're not an admin so we move you to the sign-in page. For in those cases, we haven't fully filled in all those leaks. I think it would be a huge win and you can just progressively decorate things according to the common flows that people take through your app. CHARLES: You know, I'm just imagining this. Model promise, for example would be some computed property, then how would you enumerate your dependencies? Just do the mechanism that we have now? Or are you imagining something entirely new? ALEX: I don't have a strong opinion on it because the moment I start saying what that specific syntax is, more people will agree on what's missing and what we need to have, regardless and be like, "I don't like it." I'm leaning toward something inspired by a lot of my learnings from observables, which is actually we talked about last time. The whole thing about observables is that there's almost limitless flexibility as to if you're in observable, it can take that event. It has been another observable based on that thing. If a URL changes and you're listening to that via observable description, inside that, you could kick off another observable of Ajax request based on that URL and it doesn't make you enumerate all these things upfront. I think there is going to be a compromise between that. I think when you get into these kinds of problems, you run into stuff like Relay, which is familiar with -- CHARLES: I haven't used Relay. ALEX: Just the idea of dynamically collecting all of your dependencies upfront before hitting the server and asking for specific chunks of data that you need, it's a very promising idea. There's cases of just dynamicism where the data comes back from the server, then you realized that you need this other piece of data and there's no way you could have collected upfront, unless you statically wrote it upfront. I expect to find that with this approach that there's going to be some stuff where you just have to be more upfront about it. But I had a cool little strike the other day on auto-computed properties and I'll also link to that. It's a different way of running computer properties where you don't have to specify your depending keys upfront but your getter function gets passed a getter function itself. CHARLES: It's past the dependencies? ALEX: Not even that. Imagine writing a computer property and the first [inaudible] is a function that you can call to get a property off of this but also track that you've got that property. If it ever changes, it'll invalidate again. That means if you're implementing a [inaudible] in computer property, you don't have to write first name twice, both in your dependent keys and in the actual getter in your function, which I think is kind of cool. I'm trying to make that pattern work for this data loading thing so that you don't have to have this huge verbose thing. You just lift this stuff in one place. I've sensed that the magic will probably break down in some complicated cases but that's what I'm trying to run with because I think it's pretty cool and succinct and sort of the natural evolution of what people think of as computer properties. The other major constraint and this is also what we're talking about because it's one of the best kept secrets about the router or it's one of these things that everyone's benefiting from without realizing it, is that if a transition occurs in the router, everything in the router is going to be a possibly long asynchronous chain of operations that it collects all the data that it needs for the new routes to display. In that time, if something happens, if some hook comes along and has an exception, it can load data from the servers. If something happens then it just says 'transition.abort,' that's going to stop whatever transition is in place and you're going to stay exactly where you were and if you're not stuck in a partial transition state, that's pretty awesome. That's basically database atomic transaction semantics that people have been benefiting from if they've been using Ember for years at this point. But again, it suffers a problem being locked away in the router. That is a cool concept. You should be able to specify like I intend this change of the state this way and if I gave you something that is logically inconsistent or can't be fulfilled, don't leave me in a weird half-assed state that I need to somehow fix and know how to fix all the different places, where I might be kicking off this transaction. I'm trying desperately to preserve those semantics when data comes into it. One of the hardest things to do is and honestly, can be one of the hardest sells for people who are used to thinking about Ember is there's an issue of if you imagine whatever API we're talking about, it's probably going to live on the route. Some kind of hook that might be called resolve or something else, like what is the value of this context object that every function has? Is it a route? It's tempting to want to do that and maybe, that will end up winning but winning out is the best API to get people to use. The thing to realize is that there is no consistent value of this. This implies that there's a state of the world and you're looking at it and currently, these things have these values. But in the transaction phase, there is no stable 'this object' and you can wind up with some weird surprises. I know because, not actually these days but particularly, when a lot of the stuff landed and people started trying to do weird things and these transaction hooks, there's just like, "Why can't I grab the controller? The property isn't what I expected?" Honestly, all the stuff that is gross about query params because of this fundamental violation. You have something that pretends to be a property that is there today but is still driving this asynchronous thing that could fail. CHARLES: I kind of viewed this as playing an off-note in the jazz thing like you only want to reserve using this, unless you're the Miles Davis of JavaScript, don't use this. ALEX: And by Miles Davis, you just mean like the god of concurrency that's incorrect race-condition-y code. CHARLES: Right, so it's just like you've got the right reason and you can spot the one-in-a-million case, where it's appropriate. You can spot it in an instant. ALEX: Exactly. I'm not that person and I don't know too many people who are and that's not the API you want to land. I'm trying to, maybe wean people off on dependency on this because the way we've gotten around it in the past is to use again, is more discrete, get the value functions called 'get model' and 'get params.' These are all very in-depth stuff if you're pretty experience Ember developer but it's a way of getting a value from one of these parent routes when you're inside a transition and the rest the world can't see it but you can because you call this hook at the right time. It's super gross because it's just a method on a route that anyone can call in any given time, whether you're inside this transaction or not. The branching logic of, "Should I look up the data from the transaction object?" because once valid, I should have get the current value of a loaded route. It's really gross to me and it causes real problems that confuse people and causes them to write issues because they've given an API that makes them feel good about treating these things as stable objects. CHARLES: I'm trying to imagine now, just like a spike in my head. I know you don't want to get too into syntax but essentially, modeling the route tree as a set of observables, where essentially, instead of returning a promise from your model, you're just mapping an observable off of some combination of the URL state or what are the other streams of state you want to merge to realize that route. But what I'm not seeing, which I'm sure you also have the answer is the original problem, which was stack routing. What we've been talking about is making the router fully reactive like this fully reactive tree that's always on. But that problem seems almost orthogonal to the stock routing problem. ALEX: It is. It's been very tempting to combine them. Why it is such a hard problem? Because you've got navigation stack, which almost to this route hierarchy stack that [inaudible] about but they're separate so you can't really apply the same lessons. Then you've got stack routing, which is you want the ability for routes to while they're loading, reference data that is dynamically available to them. I don't have a solid answer but I would say, the one thing that I think is going to help is that you have a few options for what you want to stash how you want to represent a URL or where you want to stash your hierarchy. Actually just track it in-memory and if you refresh the page, it'd be like, "I depend on some data that I expected to be there but it's not. It transition elsewhere," which is not a great developer experience. You could want to be able to make changes and refresh the page and continue where you left off. Otherwise, URLs aren't actually used by mobile app users. But the other place that you could possibly put the navigation where event stack is in a query param because that can be fully dynamic and you can just sort of manage every single page. The most current page you've pop is just some top-level route but you're tracking the state on the side. I think if you solve the problem of being able to depend on things that aren't the URL or go through a more complex transition than what the router gives you by default, I think it would be possible to treat that query param or that thing you're stashing in in-memory as another source of data. The other thing that I want to try and make sure that this new API has is really treated dependency injection where you specify all the things that you need and you don't really care from a route's perspective where they come from. I think if you had that, that would solve a lot of problems with stack routing and where it gets data from. To be very specific, today if you were in that post '1, 2, 3' comments route and you needed to access the post model from within the comments route, you would probably do this model for post. Basically you're naming not just the model that you need. You're naming the route that you know provides it upfront, which I think is that. Actually, the real reason it's kind of the smell is that, if you ever need to change the nesting, maybe you need to introduce another level or you want to nest all that under an admin route. Then suddenly, you're asking for the wrong route name. You're not really sure all the different things you need to update if you ever change the nesting of your router. There's solutions like relative URLs that a lot of people thrown around but I think -- CHARLES: To go back in the observable world and specifically, the redux-observable world, it's like a simple map. You're just mapping down off of a global prop, you've got some tree of state and you're just mapping off... What was that like? A model hook and you're just mapping down off of that? Wherever that state lives, you're mapping to it and now you kind of slicing off your little garden hose off of the firehose. But still one huge -- ALEX: I've tried to apply observables to this problem. I don't think I've never seen the observable analogue of is this idea of dependency and injection. To model something as a stream that transforms over time, that's proven to be very useful but to sort of say, "I am an observable that expects these objects given to me," I'm not really sure what that API would look. CHARLES: I would say, just as a straw man perhaps, you have this dependency that it's a well-known location. It's a well-known name. With dependency [inaudible] in classic, it's like, "I depend on the off service. This thing called 'service:off' or whatever. Imagine that you have some pool of state and there's some key called ‘service.off' there and as long as I'm just basically basing my stream, the first thing I do is map off of this and maybe map off of another key and then combined those into a single stream, then I can be sure that I have those things at all times. If they change, my mapping function or my transformation function is going to get evaluated again. Does that make sense? ALEX: Yes, I think we should [inaudible] C without code or something. CHARLES: And maybe I'm thinking about it wrongheadedly but that would be a simple mechanism. ALEX: Could you run by me one more time --? CHARLES: Yeah. Let's say that we've got some authentication service that you want to depend on like you want to inject on it. You want to inject that dependency so why can't you base your stream off of that key? You have observable map, for example. The list of transformations that you would have to do to peel off multiple keys, I'm sure you could write helpers for it. But basically, probably if you're going to be wanting to inject multiple dependencies will -- ALEX: The problem is this. Basically, if you want to write your resolved observable, if this thing based on observables, remember that there is no this in a route because of the transactional reasons of what we've talked about earlier, what are you getting that from? You need to have something passed into you, to be like 'context.get observable blah.' CHARLES: I would just assume that it's implicit. I was thinking a bit basically, the simplest case would just be an observable that was basically taken off of the entire global state or whatever of the router or what have you. The way the redux-observable works is every single epic is what they call them is just a transformation on the global stream. Usually, the first thing that happens is they map down to the local context so the -- ALEX: Like a path? CHARLES: They have a helper like action of type, blah. You only see a subset of the actions that get maps to the Redux store. I think it's redux independent but at least in theory, every single epic is basically going off of the entire global state but the first in reality, what the first thing that happens is you're like, "I am only interested in this subset of the state," so you do a map off of the global state down to your local scope and then you work from there. In fact if you had the convention around that, you could even make that part implicit. It's like I return an observable that it's only seeing the stream of local states. ALEX: That makes sense if there's sort of canonical state of the world but what you're doing when you're transitioning into a route is trying to feel out another state in an asynchronous manner. Redux is the action causes state to change, now the state is this. But the action for type thing, I think that makes sense if you are subscribing to the world global action on this one store when you're constructing this new tentative, may not actually become the store, you're depending on values. What we need in our API is something that depends on values that are from a tentative store. CHARLES: It's similar so in redux-observable, you're mapping actions to actions and you're not necessarily mapping actions here. You want to get state into the equation. ALEX: Yeah and it's so almost observables. It's just this twist of transaction dependency injection. It sounds really over-engineered but the thing is it exists in Ember today and if it exists in a less siloed way, I would certainly benefit on it. I think everyone else would too. CHARLES: Okay. With that hand wave... ALEX: Oh, I didn't mean for that to come as a hand wave. CHARLES: No, no, no. I'm kidding because I think we actually have a lot more to talk about here and we're running out of time. One of the thing that I want to ask is, talking about redux-observable, talking about redux and stuff, have you given any thought as to what this might look as a library that everybody could use? ALEX: I basically have something that's using Ember CLI only because it's so easy to just use it as a sketch pad and get test passing but everything I'm building so far is just ES6 class syntax that can be transpiled in it to whatever. I'm actually realizing, there's a lot of overlap between some of the primitives that are involved and Glimmer so it may or may not have a pass that uses references for tracking when things change until no one to invalidate and refire these async hooks. But either way, I'm going to make sure it lives in the JS usable world and not just Ember's special object model end. CHARLES: Right. Those interfaces are pretty narrow. The things that implement those interfaces are huge and complex but the way, at least I understand it, isn't the reference interfaces themselves -- ALEX: They're really simple, yeah. CHARLES: -- Really simple. It could almost be copied and pasted and not have much maintenance overhead in there. Here's a question and this is probably getting too far into the weeds. Can you not model a transaction as an observable? Essentially, with a flatMap, you would merge in some observable into the chain that was basically a transaction of all the other observables from which it is composed. ALEX: You know, a transaction as it builds up all the new state over time could be part of the main tree and if there is an active transition, then that's future potential state that the world might become and it could be modeled as a leg of the Redux state. I think you could theoretically do that. Definitely worth a try. I don't think I would benefit too much from doing it now and I think this could be a premature optimization but I think there would be just quite a bit of intermediate object collection to express that. I think theoretically it works but how it's going to physically map to Ember in the near future, it would be harder [inaudible] in a way. There's actually a lot of stuff that is very redux-y that again, a lot of Ember people don't maybe know about because it's internal but the way that Ember [inaudible], I think since Edward brought some of his learnings of Liquid Fire back to core Ember, there's this concept of outlet state, which describes -- I'm not an expert on it -- what's rendered where and then each outlet gets a chunk. Like you said, a little piece of the firehose or garden hose, pulled off the main thing so it can just focus on the one piece of state. Those are simple objects that produce this part of this transformation process. That's kind of redux-y in the way that everything just gets a new POJO and stuff changes but it's not strictly redux, obviously and probably won't become that just because it's already good enough on its own. CHARLES: Yeah. I think it's actually good at this point to be hand wavy because the most important thing is to be non-committal about the syntax, like you said because that's when the bikeshedding begins and now it's not the phase. The phase is to come to some agreement about what is that we would love to see. ALEX: Basically, the thing is this. I think people need to realize that Ember won the bet that the URL is an important thing to build apps around and if you have a state that's representable in URL, that state should go in the URL so you can send links around and not break the web and have an app that works that's built on half-assed routing. The only thing I'm proposing is going to make that go away. It's just that there is already this giant world of stuff that's not expressible in Ember today because it is driven by state. If you make that as easy to express and as upfront to express, I think you can have shared conventions versus what everyone is building these apps that I have to do, which is to make a sort of separate router of state-aware stuff and not have to make those two things agree with each other because it's really hard. CHARLES: Right. At that point, you're writing your own framework. Maybe this is the next big thing because I feel like Ember usually has the best stuff way, way, way, way before. Now, we're finally getting to a point where everybody seems to realize that having a CLI is absolutely critical to the developer's experience and most frameworks aren't taken seriously until they've achieved that. It was the same thing with a router back in the day. I'm wondering what that next thing is. ALEX: I don't know. I don't think this is going to be it. I just think it's a good progression. I think a way forward that progress is still a pretty legit central structure to build apps around and just would be welcomed. CHARLES: When are you going to be done? ALEX: About two or three days. I don't know. I think I'm basically going to be continuing to get feedback like the way that a lot of that original router stuff came back or it's just like constantly hit people with real examples, Ember twiddles, things are just like, "Oh, yeah. That thing. That's a cool pattern. That sucks in my app. I didn't realize that until I saw this example." These things that really teach people why this is necessary because that's going to get people's urge to be like, "Well, you could just do..." Oh, you can't because the thing that's hard to explain. It's going to be a lot of that regardless and I hope that will kick off in the next few weeks. CHARLES: And the focus of that is going to be the ember-rideshare application. ALEX: I think that's a good one. This is one that everyone's familiar with. CHARLES: Have you already kind of implemented in it, like this kind of Frankenstein-ish, like this is the kind of histrionics that you have to go through in order to implement the style of routing or the style of application using today's Ember? Or have you started to begin experimentation with these new concepts and try to build out better ways of doing it? ALEX: I'm not strictly extracting it from one app. It's sort of combined. Like I said, the few different apps that we had were an opportunity to be like, "This sporadic stuff is hard." The main route recognizer approach was an example to try different stack routing pattern. But the thing that sort of working on is drawing from three different apps and slightly different takes on it. Basically, I have something that is close to being testable in one of my main apps that will be a great chance to validate if all the stuff is as nice as I think it is going to be. CHARLES: Okay. If the people want to get in touch with you, to help to contribute to the conversation or just publicly guilt you into moving faster towards it, how would they get in touch with you? ALEX: I'm at @Machty on Twitter and GitHub and also, the Ember community Slack. I think I'm going to try to get people to talk about this on channel called Dev Dx Router where it's focused on development stuff all around the router. This is kind of funny because I'm talking about this thing that I've only had maybe, 12 people take a look at and comment on and begins these conversations. I think maybe some people are going to hear this and be like, "What are you talking about?" but if it gets people -- CHARLES: No, no, no. You know, the best conversations seemed to be organized around you, man. I'm just trying to think of some of the best development conversations that I've had in 2017 and you were definitely, I would say the one who fomented them. It starts with 12 people but then, if enough people take interest and be like, "Wow, yeah. Oh, man. I didn't even know that was a problem. This would be a cool way of doing it." They have a tendency to balloon and some fizzle out and some end up with real results. Anyway, I'm looking forward to it. ALEX: I appreciate it and likewise, you're definitely one of the best people to talk about this stuff with. CHARLES: Well, I hope other people will love listening to our conversation. With that, we'll head on out. Thank you everybody if you've made it this far. As always, you can get in touch with us at @TheFrontside on Twitter or just send an email to Contact@Frontside.io. We will talk to you next week.
Jay Phelps: @_jayphelps | jayphelps.com Show Notes: 01:25 - RxJS 10:09 - Observers 17:49 - Back Pressure 22:11 - Async Iterators and Generators 31:30 - Mapping Resources: The Observer Pattern Hot vs Cold Observables IxJS redux-observable Transcript: CHARLES: Hello everybody and welcome to The Frontside Podcast, Episode #84. My name is Charles Lowell, a developer here at the Frontside and your podcast host-in-training. With me today is Elrick Ryan. Hello, Elrick. ELRICK: Howdy. CHARLES: You and I have actually been on a roll lately, podcasting the hell out of these podcast. ELRICK: Yeah, I know. That is very true. CHARLES: It's been you and me but it's feeling great. It's good to have you on the show again. ELRICK: Yes, wonderful, man. Let's keep it rolling. CHARLES: All right. We will keep it rolling. Today, we are going to be talking about redux-observable and to help us understand and plumb this topic, we have someone who's very qualified to talk about it. Mr Jay Phelps, who in addition to having been the co-creator of redux-observable, also is on the core team of RxJS, which is a fascinating library on which it's based for many years and is currently a senior software engineer at Netflix. Welcome to the podcast, Jay. JAY: Hey! Good morning, everyone. I guess it's not probably morning to the people listening but good morning to you all. Thanks for having me. CHARLES: I'm excited about it. I know that kind of starting with the fundamentals, RxJS is something that was on my radar for a few years and it definitely [inaudible] once we started using redux-observable but the whole concept, I often feel like the world kind of is turned upside down when I'm working with observables, when I'm working with RxJS and I'm curious, how did you come to be a part of that project and what are the things that you use it to solve? Why did the solutions that you generated shake out that way? JAY: Sure. I actually was not introduced to Rx until I started working at Netflix. Netflix does have a fairly solid reputation for their usage of Rx, not just in the JavaScript world but also in the server world. Netflix wrote the original implementation of RxJava and it's used heavily on our backend systems. CHARLES: For some reason, I had this impression and maybe I'm mistaken that Rx originally came out of Microsoft. JAY: Let me continue with the story. It's confusing and I can actually take a step back and clarify that point in particular. Rx itself was originally came from Rx.net, which was indeed created by Microsoft many, many, many, many years ago. I don't know the exact date. I think it was at least 10 years ago. They, at the same time created or about the same time, Matt Podwysocki who was working at Microsoft and still is working at Microsoft, created Rx.net and RxJS. Then many years had passed and originally, it wasn't as popular as it got in the coming years. After several years, some employees from Microsoft ended up coming to Netflix. Jafar Husain is one of those employees. He came from Microsoft to Netflix and he brought that Rx knowledge and that advocacy. Rx is very ingrained in the Netflix culture and is used a lot by various teams for various purposes. Then when I joined Netflix and I got really exposed to it. One of my coworkers at the time, Ben Lesh was asked by several people at Netflix to consider and look into rewriting RxJS. At the time, the version was RxJS 2.0 and while it was great, we had some specific requirements for our website and some of our other applications that we were hoping for a better performance, smaller bundle size and better debugability and -- CHARLES: Also, when I first evaluated it many years ago, it felt very much like a port from another language, in another culture as opposed to something that from the ground up, considered as a JavaScript library. Is that a fair statement? JAY: Yes, somewhat. Definitely, there were more considerations this time around when it was rewritten and originally, it was going to be Version 3 but the rewriting process took quite a while as these things usually do. By the time we got a version out, it was Version 5. We started when RxJS was at Version 2 but it already released Version 3 and Version 4 by the time it released for the new version like a rewrite had been able to get out. When I say a rewrite, I mean like from scratch rewrite. Matt Podwysocki who was the maintainer, almost the sole maintainer of the previous version, also is now on the core team of the new version of RxJS and has been instrumental in pushing back forward as well, he has far more experience with this than either Ben are I so he's been invaluable. Sometimes, we'll think to make those decisions. We'll be like, "Why was this decision made? Was it made because of .net?" and we'll just assume that and we'll want to change it but Matt has the history involved in that. He knows why things were changed the way they were. For example, we changed one of the operators, flatMap to mergeMap. We know somewhat we go at least, I don't want to speak for the entire team but I regret that decision. Depending on the day that I've been talking to Ben, I could convince him to regret that decision as well. But we thought that mergeMap would make more sense and that very few people in practice would have heard of the word flatMap before and had experience with that so -- CHARLES: I have to say both of the terms coming at it were pretty opaque. I think there was a bout of equivalence in opacity. JAY: Yeah, good to know. That's just an example. I don't want to stick too much on that topic. Maybe someday we'll go back to flatMap and the flatMap still exists. If you're a purist, you can use it. Ben was the primary person who was working on this. He wasn't working on it full time but pretty close to full time to get that initial version out. Even though I used it, my involvement with it was fairly low at that point and then my involvement after it was released got increased. I found more time and started to get more involved, particularly there wasn't a lot of code to write. I have some PRs and stuff like that but particularly on the planning and the issue triaging and PRs and stuff like that, which is a pain in the butt. It's just massive. Particularly, around the same time that the rewrite was getting finished that Angular decided, "You know what? We're going to bet on Rx. We're going to depend heavily on it," so you really can't write Angular without writing some Rx these days. You can get away with not knowing Rx very well. You could just call subscribe and then just do a bunch of imperative stuff but for the most part, the paved path is observables in so many fashions. Now, there's this ngrx. I don't know if you have any exposure to the Angular community. I have quite a bit. CHARLES: I haven't recently. Certainly, since that kind of heavy investment in Rx, I haven't been exposed to it. JAY: I think that was your question, right? CHARLES: Yeah. It sounds like there's a Java implementation that gave rise or a .net implementation or Java implementation gave rise to a JavaScript implementation and that's the one that you got involved with but it suggests very strongly that Rx is an idea and it's played out in a bunch of different languages but really, there is a shift or it's an idea about the way you think about your programs. It's clearly been compelling to you so what is that idea and what is that shift from the way we normally think about things? JAY: The idea was realized very early on -- CHARLES: Yeah, both 10 years ago, right. JAY: Yeah, exactly. They dubbed it their reactive extensions, which is what Rx stands for. Pretty much, name a language and it's been ported to that. There's RxSwift, which is super popular. There's even things like RxCpp and stuff which if you look at it, it's awkward. It seems like we got less language in the world for doing this sort of thing. I actually like C++ in a lot of ways but it was awkward stuffing that stuff in there. It's a really popular pattern and the idea is just basically going all in on the observer pattern, saying that like, "Most people are building things in which you want to be pushed information." You want to be pushed events and the data should stream to you. Modeling most problems in the world as a stream, once you get over the initial barrier of getting away from your normal historical way of looking at things and you look at everything as a stream, it comes very natural because you can actually model literally anything as a stream because it could be a stream of one, it could be a stream of nothing, it could be a stream of infinite number of events or it could be a stream of 10. You can model anything as a stream. Once you start thinking about that, it just becomes very natural and particularly on the UI side of things, I think there's been a lot of success in RxJava stuff at Netflix but RxJava is also used in Rx.net for client-side stuff as well, for mobile development. CHARLES: I remember when I first was introduced to it, I think there was a lot of confusion for me around an observer in the context of Rx and an observer in the context of classical MVC. One particular manifestation of the MVC architecture where you have these kind of mutable objects and you're observing their properties. Like key value observation, which factors heavily into certain UI frameworks. Backbone is one that comes to mind or if you're familiar with Java, basically the JavaBeans, like the property model listeners. I kind of had that conception of what an observer was versus Rx has a very, very different take on observable things. Do you think you could maybe show where they're different? JAY: You can get the normal classic observer pattern using Rx, using a subject basically. But there's a subject class, which you're not going to use super often but there are certain cases where it makes sense. Also, it depends on what libraries. It is used more often in the Angular world because you want to get a stream of clicks or something like that but -- CHARLES: So what would be the subject in that case? JAY: The subject in that case would be you're going to pipe, you're going to emit. Every time they click on something, you're going to 'next' something into that subject, like 'next the event' into the subject. Basically, a subject is a really great way to go from some imperative world to the observable world. Without having to write all sorts of custom glue, you can just basically say, "I've got subject. Any time I 'next' into it, just notify anyone who's listening to this." A subject is hot observable and that's the closest to the typical observer pattern because Rx, it's usually like observables and are usually lazy or cold. That's also what people call it. In the normal observer pattern, there is not necessarily any concept of laziness like you listen to something and that producer is already producing usually. CHARLES: Right. You hit on that and I think that was something that was surprising and kind of delightful when I first started using observables is to realize that they were lazy. Let me make sure I understand it. What was cool is like I was going through some of the demos and I had this observable and is part of, forgive my terminology but when you create a new observer, you pass the function that will get called every time something subscribes to it, right? JAY: When you create a new observer it passes a function -- CHARLES: When you create a new observable, you pass a function that gets called when an observer subscribes and the thing that you can pass is the thing that you can call next on, right? JAY: That's exactly right. When it's a lazy observable, that only get called... Actually, you know, continue. You had a point. CHARLES: I was going to say what was a surprising and cool for me was that every single thing that subscribe to that observable got its own history of that observable from the very beginning. It got its own function invocation so the first example I did, I wanted to iterate over an array and just send 10 items to the observer. Then when you subscribe, you're starting from one every single time: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10. It's not like I subscribe and one gets five of the elements, then I subscribe to another one and that one gets the next five or those two get the next five together. It's like they each get their own version of that observable. JAY: That's exactly right and then that's the main difference between -- CHARLES: -- The whole thing, yeah. JAY: Yeah. This is still observer pattern because the observer pattern, at least to my knowledge, I'm not an expert on this or anything but my understanding is that the observer pattern does not necessarily dictate hot versus cold, per se but traditionally I would say, people interpret it as typically the hot type of thing or also the multicast, if you want to be like these cool buzzwords. CHARLES: Right. What you're thinking of it is when you've got this mutable object, you just dive in wherever it is at that moment and you're now observing events from it. JAY: And that's traditionally because the observer pattern was mostly created for that model view controller type of pattern as well. You can think of like a stream of clicks: the users clicking on things where a stream of keyboard events. That is inherently a hot or multicast stream. You can't tell the user, "Start clicking," or, "Start typing. Don't type." You can tell them that but it's futile. The point being is that you can't control the stream. It pushes data to you and it's already pushing, whether you're subscribing or not, the data is flowing so the traditional observer pattern works really well for that. Then the observables, it's usually lazy, Now again, with subjects and with multicasting, you can get the same behavior. You can get that observable that is hot and shares its subscriptions so that when multiple people subscribe to it, they all get the exact same underlying subscription that is already producing data. CHARLES: I see. That makes sense with to call of multicast because fundamentally, there's really only one observer. There's only one stream and you just happened to be entering in at a certain point. JAY: Right. Then the other one is called unicast but not everyone calls it that but that's usually what it's called. CHARLES: There's a lot of terminology. With subjects, is that just a fancy way of saying a hot observable? JAY: Not necessarily but almost always. If you include them together, you're not wrong 99% of the time. But as a quick drive by definition, it's totally fine. If you're teaching someone really all the things and then they want to fundamentally understand it, it's good to have some distinctions. A subject is hot and multicast but just because something is hot and multicast does not mean that it necessarily has a subject backing it. Maybe from the pattern perspective, you could call it 'the subject' or 'a subject' but in the Rx world, it's not necessarily a subject. CHARLES: Then for cold observables, what are good things that they model, like the execution of an algorithm or something? there's an instance of that algorithm, I'm going to add two numbers or I'm going to reduce this tree of numbers into a product or a sum or something like that, where you start from the beginning, there's a clear beginning, there's a clear end --? JAY: Right. The observables are really good at modeling side effects, for example. Things you want to do like make an Ajax call or read a file or something like that. In fact, reading from a file is actually not a great use case for observables just because you want to be able to control the back pressure. We can talk about that next if you want but -- CHARLES: Yeah. Back pressure is a concept. It's crazy to me and when people use it, I'm like, "How do you even do that?" You can say it and sounds good in practice but it sounds really, really complex. JAY: I think I can summarize it for you. The back pressure stuff is basically you have a data producer and you want to be able to control the rate in which it is producing so that you do not overwhelm the consumer. You can consider two servers. You've got Server A and it's sending events to Server B. If Server B can only process 10 events per second but Server A sends 100 events per second, what's going to happen? There's a deficit. There's a huge 90 events per second deficit and that means that the Server B is going to get more and more behind. Eventually, it will just fall over and run out of memory, CPU or whatever happens. Back pressure is basically just being able to control that somehow. There's lots of ways, there's several policies of back pressure. There's buffering, there's dropping and then there's the pull model where it's like, "I'm going to tell you when to give me the next event," or, "I'm going to tell you to give me six more. I can handle six more, give me six more," or, "Here's the next one." Those were -- CHARLES: Then polling would have to be combined with buffering or dropping still, right? JAY: Possibly but it becomes one of those like you use it just as a rarity. Because the problem with buffering is that it becomes usually unbounded and that means you can -- CHARLES: It's blowing up the producer, right? JAY: Well, you risk money out of memory, mostly. It just becomes completely unbounded. You can bound that buffer and then it becomes dropping. Then if it's dropping, that means it's lossy. In a lot of scenarios, it's important not to drop information. CHARLES: They're useful stuff. JAY: Right. At the same time, there are a lot where you can drop it. Like at Netflix, we have this data pipeline thing for these logs and I got a good talk on this actually. This data pipeline is called Mantis job platform where all this data flows through. It uses RxJava almost exclusively and we have a lot of different back pressure mechanisms but the one of the most popular is dropping data. If the consumer is being overwhelmed with the number of logs and events that are matching to it, we just start dropping them because they're logs. We want to keep them but we make a best effort and it's perfectly fine to drop them. In the observable world, it's usually going to be either buffer or drop. You're not going to usually be able to control the producer's rate. In RxJava, there is something called the flowable, which lets you control the rate by basically saying, "Give me N number. I want N number more." It's become more important on the server-side end of things but a normal observables -- CHARLES: That makes sense. JAY: Yeah and there's even a library that's brand new that Matt Podwysocki came out with called IxJS. Instead of RxJS, it's IxJS and it's iterator. It's for async iterator and regular iterators, which is good for that other end of the spectrum for back pressure. If you want to be able to control the producer, you're saying, "Give me the next five." That's what an async iterator is incredibly perfectly suited for. CHARLES: The async iterators are kind of like the logical inverse of the observables? JAY: That's exactly right. It's the dual, if you want to use the computer science term, I assume you knew that already and it was that was just a plant. CHARLES: Actually, I didn't terminology, the dual but it's like through the looking glass world, right? JAY: Yeah, that's correct. CHARLES: One hand you got Alice in our world and she's looking in the looking glass world. I'm not saying which one is which but one is observables and one is the async generators. When you would want to use async generators is when you want to have the consumer driving the entire stream. JAY: That's right. A lot of people, when they discover async iterators, they'll ask me, "Jay, why wouldn't I just always use that? It seems like they're more flexible of the two." The reason why is because you can't always control the producer. The fundamental example that I was giving earlier like mouse events or keyboard events, you cannot control what the user does so an async iterator would be a very poor primitive to model that because what happens if you decide to give me one keyboard event at a time and one per second. You're like, "Give me the next keyboard event." If you did that one per second and someone is typing on their keyboard, what do you do with all the events while you're waiting? You could buffer them but that's a back pressure problem. You have to choose and it's a very poor primitive model to that. CHARLES: Yeah, that would be a terrible user experience. The users want the consequences of their actions to be realized at the soonest possible point. JAY: That's exactly right. The other benefit of observables is a slight increase in performance because when you subscribe to an observable, it sets up the chain and then now only data needs to flow through that chain. Whereas an async iterator, every time you call next, you're going to get a brand new promise. If you used it for something very high volume, you might be able to see how now you've got a lot of excess allocations: CPU cycles being used, garbage collecting for each one of those promises. In Netflix, we got a lot of streams, which are hundreds of thousands of events per second. If you do that, those allocations of those promises can certainly add up. It's just not the most efficient primitives for that as well. CHARLES: I thought that async iterators used essentially continuations under the covers and not promises. JAY: Under the covers, I'm not sure how you would describe what -- CHARLES: I'm not super familiar -- JAY: It's basically an -- CHARLES: You're talking about like yield, right? JAY: You're talking of generators maybe? CHARLES: Uh-uh. JAY: Iterators and generators are although related because a generator is basically a factory for creating iterators. Does that make sense? CHARLES: Yeah. JAY: Every time you call that factory, you get a brand new iterator out of it. But then when you have an async generator, it's basically the exact same thing except for the iterator returns a promise. You're basically saying, "Give me the next thing." You call next on the iterator and the next will return a promise and that promise will resolve with a value. Why is that useful to standardize? Because you can have syntactic sugar like the [inaudible]. I don't know if you've seen that yet but it's like a loop primitive, where you can basically loop over an async iterator without needing to deal with the whole thing and all that stuff. The same way normal async/await is helpful. This await is helpful at the same time. CHARLES: It's way above my pay grade but maybe you're using both. I'm trying to think with async the way that works with promises, right? JAY: That's correct. CHARLES: So you got both the continuation and a promise. That's even more overhead than the promise because you've got to stuff away that whole call stack and the promise and yield to it. But anyhow... ELRICK: This is very interesting. RxJS is a very deep subject. What I'm interested to know is you have RxJS, you have observables and you have redux. Where's the union? How did that come to play that you guys came up and said, "You know what? We need to create redux-observable library?" JAY: Yeah, great question. It came about fairly organically and over many month period of time, actually. We bring Rx people as I mentioned and we were using redux at the time and we're still using redux but my team that I was on was using redux and we were using redux thunk and the thunks were getting incredibly out of hand. It was very hard for us to do things that we were used to doing that were very simple like debouncing. We were like, "This is really hard. It's just to do something so simple that in Rx, it's so simple." We try to stuff Rx into redux thunk and try to make conventions around that. It just didn't work out well and then we initially made something we called thunk-observables -- CHARLES: We really wish you would have stuck with the name even you didn't -- JAY: Yeah, so thunk-observable is probably what you can expect. You dispatch a thunk which returns an observable and that observable will be subscribed to and that was it. Basically, that was the primitive so that let us have our debouncing but in practice, we found that the thunk stuff cause way more confusion and then also did not let us do composition as much as we wanted to. We then extended thunk-observable thing to get receive a stream of actions so that you could compose the different thunk-observable together and then receive new actions to cancel things and stuff like that. We eventually just figured out that having basically the idea of thunk-observables but having of them be like almost static factories that are like process managers, that's the pattern that we ended up on today. That we coined calling them epics. Basically, an epic is a function, which receives as an argument, a stream of all of the actions. It gets an observable of every action that's going to be dispatched and it was a hot observable so that means that if you happen to subscribe to it later, you're not going to get all the actions from before. It doesn't replay them. It's just all actions from the time you subscribe and continuing. Then what that function is expected to return is a new stream of actions and that stream that it returns of actions gets subscribed to by the middleware under the hood and basically, it just calls stored dispatch on anything you emit. You can imagine the simplest epic would be a function that gets streamed of all the actions, it applies a filter operation on all of those and looks for a particular action and then based on that action, it performs some side effect and then when that side effect comes back, it emits a different action. Basically, to notify redux like, "Here's the results. Here's the change. I'm done," or what have you. let you just make arbitrary side effects but isolate them in a way that fits very naturally with your UI layer that you're using and with redux itself with the purity of the reducers. CHARLES: Right. For example doing an Ajax request, I think that's a good example. You get some action in, you're getting every single action that's dispatched to the store, the first thing you want to do is you want to filter that stream to only the actions that are user click the save button so I want to save off this form. To some action, that's my save action so then my job is to now map that eventually. Out the end of that stream, I want to have like the save either succeeded or the save failed. There's two forks on an Ajax request. The fundamental mapping that's happening is either from the user click save, whatever you want to call that action, I want to map that to the user if the save was successful or I want to map that to the save failed or rejected. The hard part then is executing those side effects and then putting them back into the stream. How would you do that then with redux-observable? JAY: If I'm following correctly, the typical almost all epics, not all of them but almost all, they're going to have a filter at the top of the epic, they're going to filter out looking for the action they want and then they'll have some sort of merging strategy operator like mergeMap, aka flatMap or they have something like switchMap. CHARLES: I feel like that's one of the harder concepts for me and I want to actually going to be a little selfish here and try and bounce my understanding off of a bit so you can be like, "No, your mental model is incorrect," but we'll find out how it's incorrect. With the mergeMap, is that a way of saying, "I can take other observable streams and inline them inside this one stream." That's the way I've been thinking about it. I can go out to the rest of the world, I can have some other stream and I can inserted into the processing chain of this other stream. In this case, I do a mergeMap of the actual fetch or the actual post but it's not a straight chain of implication like do a filter then I do a map then I do this. I've got to take this other observable, this thing, this other asynchronous process, which really represents another stream and I want to merge it into this one stream. Then once that's done, I get the result. now, once that happens outside the mergeMap, now the Ajax results whether it's a success or failure, the result of the request, now it becomes the next item in the stream, which then I have to map down to another redux action. JAY: That's right. The mergeMap operator has an alias and it also used to just be called flatMap so you can envision that you are creating an observable of observables, a higher order of observable. With mergeMap, you want to flatten that. You're saying that every time I get an event, I'm going to call this projection function and it's going to return a new observable, an inner observable and I want to merge each one of those into each other so that it becomes one stream. That means you can have concurrency. That means you can have multiple and simultaneous concurrent Ajax calls that can finish it arbitrary times. I think it's important then to contrast that with some of the other merging strategies like switchMaps or concatMap. With switchMap, as the name implies, you switch between observables so at any point in time, you can only have one observable subscribes to at a time. If a new event comes along and you call the projection function, it returns a new observable. If the previous observable has not yet completed, it will get unsubscribed to and anything it was doing gets cancelled. In the example you're talking about, if you've got an epic that goes and fetches your user model every time they click this button, let's say that they can click that button multiple times and you don't want to make 50,000 requests. The quintessential example is actually the auto suggest stuff. As you're typing key strokes, if the request is in processing and they type another keystroke, you don't want to wait for them the previous request to come back and process it. It's not only wasteful because you'll process the JSON and all that stuff. It can introduce bugs because it may come back out of order. It may come back actually after your new one comes back and that can cause all sorts of crazy weird bugs. That's what switchMap is really great for. I call it implicit cancellation. Ben doesn't like that because he's saying that in a switchMap, you are being explicit about it but you're not calling unsubscribe on it yourself, which is why I call it implicit. It happens automatically. There will never a new event pipes through there. Then there's the third one which I don't -- CHARLES: So this switchMap ever make sense outside the context of hot observables? JAY: I would say that it makes more sense usually on hot observables but there are certainly cases like let's say that you've got a web socket observable and every time you get an event, based on that event, it make some other request and that other request may or may not take a really long time. But if you get another thing back from the web socket, you want to cancel the previous one. That's somewhat not a great example as well because sometimes people use multiplexing for the web socket so that it becomes multicast. But the point being is that there are definitely times where you will use merge, switch or concat. That concatMap being the third one where as you might imagine, you are concating the observables, you line them up. If I get a new event, I'm going to call my projection function, create that new observable but I'm not going to subscribe to it. I'm going to keep it but I'm not going to subscribe to it and tell the previous observable I was subscribing to has completed. In a way, you're buffering the observables and because you're offering them, you can get in trouble where you end up buffering infinite observables. Let's say the first one never completes for some reason because of a bug, you may infinitely buffer them. ConcatMap is not used that often. I'd say it's very rarely used. Its use mainly when you don't want lossy behavior. You want, at most wants semantics like you're only doing one per time. CHARLES: The difference between mergeMap and concatMap is what? JAY: It's you're not going to do concurrent. That's probably the best way to explain it. You're going to do that in sequence instead of concurrently. CHARLES: I see. Maybe an example would be like file uploads or something. You probably want to do your file uploads in parallel but let's say, you want to conserve bandwidth or you're working where you've got a browser that only supports 10 connections or five upload connections but when someone selects 30 files, you don't want to just drop those files. You want to be uploading 10 concurrently but as soon as one finishes off of that 10, you want to start up another one. JAY: That's absolutely right. Another example would be like you are going to hit some API and it could be used as one mechanism to kind of throttle yourself. If you want to guarantee that you're only connecting at most once to this particular end point at a time, no matter what the upstream tells you to do. CHARLES: Right. I could see that. That makes sense. JAY: I can tell that you're a little bit struggling to see the use cases, which is totally normal. ConcatMap is not used very often. It's for of those fundamental operations, which primarily why it's included is it's non-trivial to implement yourself. CHARLES: Are there any other mapping operators that we should know about? JAY: Those are the three primary ones. There's a couple of other like boutique variations but they're all variations on the exact same thing. CHARLES: Right. ELRICK: Do we have like mergeMap, switchMap and concatMap that are specific to management of observable and that's specific to redux-observable. There's people out there using something like redux-saga or they're trying to compare and contrast these two libraries. How do these things contrasts? Because I don't think you would use either of these mapping operations inside of a saga. Is that correct? JAY: I just want to make a minor clarification. The Rx stuff and these operators that we've been talking about: mergeMaps, switchMap and concatMap, they are not actually related to redux-observable really in any way. The only thing they're related is that you just might happen to use them. Redux-observable is actually a very tiny library and it defers pretty much everything to just normal idiomatic RxJS code and that's really the biggest pro in comparing it to say redux-saga. Redux-saga came significantly over a year before redux-observable and without a doubt, were influenced. The pattern is very, very, very similar but there are some just fundamental differences and one of those is that difference. The most obvious thing is if you already know RxJS very well, without a doubt you're going to want to use redux-observable. I could tell you that, it's not possible but it's very unlikely you'd choose redux-saga over redux-observable if you're already a really big Rx fan because you basically know how to do it. You just have to think that the items you're modeling, the events you're modeling are actions, which just happens to be a convention really. There are some other differences though between redux-observable and redux-saga and the biggest difference being that redux-saga takes this effects as data approach, which is like Haskell if you're familiar with that. In a lot of ways, it's identical with the IO monad but it basically just means that you're not actually performing the side effects yourself right then and there when you call their operators. There are special utility functions, instead there's like an engine or a middleware underneath that performs the side effects on your behalf. You create a generator and that generator is the pattern that is called a saga is what they call it. That generator yields these objects, these effects objects. One of the effects objects might be to call some particular API to make an Ajax request so you're going to yield that object that basically describes the side effect you want to perform but doesn't actually do it itself. It's just an object. In the middleware will form it for you so why would you put that indirection. The indirection helps when you want to do things like testing. If you want to test it, you don't actually need to perform any of the side effects. You can just call next on the iterator that you get back from the generator, you call next on it, you will get the side effect object, the effect as data that you want to perform and you can just assert based on that. You don't need to do mocking in all of that stuff. You just have to assert that the effects that were yielded were the ones that you expected. Now, I don't like that approach personally because I actually use redux-saga quite a bit, not lately. It was like a year ago. You end up implementing almost all, not all but a lot of your business logic in your actual tests themselves. Your test become less about testing the behavior or the outcome of a particular thing and more about testing how it gets to that outcome and what steps it takes to get there. Some people think that's fine. For me personally, it felt like any time I made a change in my saga, I had to make the exact same change in my test even though the behavior of the saga did not change. The actual observable outside side effects did not change in any way but I may be refactored or renamed something. It felt very redundant and to me, felt brittle because I started to wonder who tests the test then. If the tests are in a lot of ways are reimplementation of the saga, how did I really test that the behavior of what I was trying to accomplish really was accomplished. I'm not an expert on it so certainly, I'm sure there are people who have patterns around this that can mitigate some of my concerns. But for me, I'm used to testing Rx and I'm used to Rx in particular so the pattern for me of using observables just made a lot more sense to me. Also, with either of these libraries, the learning curve is really steep. If you don't know redux-observable, let's say you don't know Rx, learning Rx just to use redux-observable is a pretty huge undertaking. I would usually recommend people to not do that. A lot of times these days is spent me helping people who are frustrated because they dug themselves in a hole by using all these new technologies. They'll pull in TypeScript and [inaudible] with every all of the cool new hotness and I don't blame them because they've been told that this is all the cool stuff. CHARLES: We got some great advice on that that you can have one vanity technology on every project and live it yourself. Indulge in that coolness, in that hotness and do something exciting but make sure you have one vanity technology and one only. Everything else, be very comfortable with and one is just crazy experimental like this is the coolest new thing and we're going to just drop it in. But if the rest of your chassis is solid, then you can support that crazy experimental engine that requires you to stand up on the front of it and feed gas in with your mouth like they did in Mad Max. JAY: And there are exceptions obviously but for the most part, I subscribe to that. I answer a lot of questions on Stack Overflow and redux-observable. I actually get sad and frustrated when I see people that it's very clear that they shouldn't be using this. Their use case is not complex and they haven't learned Rx yet. It's one thing if you're like, "I'm learning this and I know that I'm going to have pain." That's totally acceptable. There's plenty of times where it's like, "It's totally okay that there is no concrete deadline and I can take my time," but a lot of times and I would say, a majority of the time, you're slipping on the deadline and that's not a good thing. The people who are coming to me are just frustrated. They're like, "Why is this so complicated?" You're right, it is really complicated but its complication helps with some of the more complicated use cases. That's the irony behind both redux-saga and redux-observable is that they're both really complex for the trivial use cases like all I want to do is make an Ajax call. That's it. I don't want to cancel it, debounce it, I won't do anything. I just want to make an Ajax call. They are the biggest hammer you could possibly think of. Don't use them for that. CHARLES: The thing is if you're not sure, whether you need redux-observable, then you actually don't need redux-observable. JAY: I would say that usually that is the case, yes. The same with redux-saga. They're in the exact same league. It's basically, "Do you need complex side effect management?" With the redux-saga, the complications, even though generators have been around for a little while, a majority of people still are not familiar with them because they're just not used all that often. They're kind of mystifying. I would say, they're not super hard to learn. They're just alien. Then to add on top of that, the effect is data and you've got multiple curveballs. Same thing with redux-observable, they could never use observables. It's pretty alien. The reactive programming idea and model is pretty alien. I would advise, if you're not using redux, don't even consider redux-saga and redux-observable personally. If you're already are using redux and you think that you might need something like this, experiment with the primitives, try and see how well your team can pick up RxJS just by itself. Just learn some of the regular RxJS tutorials, don't even look at the redux-observable docs because it's not useful in any way when it comes to this. Just try and learn a little bit Rx and see does it click. Because some people is like, "I don't understand why everyone thinks it's complicated. It's easy." But a majority of people, it takes a while before they get that like Neo in The Matrix, I-know-kung-fu moment. CHARLES: All right. We're almost at time but I hope that that moment comes to everybody. I think we've certainly enjoyed a lot of success with it already and I think once you do get your head around the basic use cases and you know how to do an Ajax request, you know how to do just simple saves and gets and what have you, doing the trivial things becomes easy because you know which pathways to travel. But before we head out, is there anything that you've got coming up in the near future, any talks, appearances, meet ups, anything whether you or otherwise that people should be aware of? JAY: I would say on this topic, there's a new beta for RxJS with this new way of using operators. It's being dubbed lettable operators, like the 'let' keyword but it's not related to that. It's basically just a way of finally being able to import operators and to use normal tree shaking that people have asked for forever. Because the problem with observables is that they're prototype-based methods and you can't reliably tree-shake methods or prototypes. We've been trying to experiment with ways to have [inaudible] and the lettable operator stuff is interesting to check out. It's a stopgap measure until JavaScript has something like the pipeline operator, which just actually moved to stage one. It's a brand new operator. If you're familiar with a lot of functional programming stuff like F# and a lot of the functional programming language is actually have the pipeline operator, it'll make it so that you can basically have syntactic sugar to apply a function, basically to pipe the result of a function into the first argument of the next function, etcetera. You can pipe the argument and return values through a series of functions. If you do that without this syntactic sugar, you've got this massive nested function invocation, which is incredibly hard to read and hard to maintain. That's why the pipeline operator is so great. I would encourage people, that's in beta right now. I think it's 5.5 but it's in beta right now. I encourage people to check it out, find bugs and get feedback. Maybe this is completely off base and it's not the right direction that the team should be going but it's based on a lot of collaboration, particularly with the Angular community. They've been, in particular asking for this because they're pretty big and they've got to use the Clojure compiler and all sorts of things for trying to make their bundles incredibly small. For me personally, I don't have any Rx talks coming up. I've been pretty obsessed with web assembly here lately. I'm an armchair compiler nerd. I don't do compilers for a living but I have done them personally for a number of years so I'm obsessive with web assembly. I have number of talks in web assembly coming up but just nothing related to Rx at this point. CHARLES: That's totally okay. I'm actually, also have been obsessed with web assembly. JAY: Have you guys done a podcast on that yet? CHARLES: I don't know. ELRICK: No, not yet. CHARLES: I actually started out to write my own list compiler in web assembly and they got totally derailed on the list compilers. Actually I ended up switching tracks on the whole web assembly thing but I was really, really excited about it. Probably, it was about three months ago or something like that but I'm still excited about it. I just haven't been working on it actively so I'm very curious to hear about those talks. Let's post them on the show notes and who knows? We do a lunch and learn every Friday here and usually, it's one of us getting up there but sometimes, we'll just watch a talk. One of us has been wanting to watch. ELRICK: And you're always welcome to come back to any time and geek out with some web assembly. CHARLES: I'd say, we haven't podcast web assembly, you know? All right, you guys, we've been at this for another hour. Let's go. Everybody listening, strap your headphones on, we're going down for another hour. Changing the subjects: web assembly. It starts right now. ELRICK: It's going down. CHARLES: No, but we will have to have you on for that. Thank you so much for coming on and talking with us about observables, Rx, redux-observable but if folks want to continue the conversation with you, they can get in contact with you how? On Twitter, email? JAY: The best way is going to be Twitter probably. I'm at @_JayPhelps. Thank you guys very much for having me on today. It was a blast. I love talking about this stuff. ELRICK: Thank you. CHARLES: Thank you and for anybody out there, we can also be reached at @TheFrontside on Twitter or just drop us a line and Contact@Frontside.io and have a great week and we'll see you next.
Julie Moronuki: @argumatronic | argumatronic.com Show Notes: 00:57 - Julie's Unique Origin Story Into Programming 03:47 - Good Resources vs Bad Resources for Learning Haskell 11:18 - Areas to Look at Before Taking on Haskell and Functional Programming 15:56 - Terminology 17:50 - The Haskell Pyramid 25:51 - Learning Haskell Vocabulary 28:20 - Monoid and Functor 42:06 - Advice for Someone Who May Not Be Interested in Programming Resources: Haskell Programming From First Principles (Haskell Book) Natural Language Processing (NLP) Learn You a Haskell for Great Good! Programming in Haskell by Graham Hutton Haskell: The Craft of Functional Programming by Simon Thompson Real World Haskell by Bryan O'Sullivan, John Goerzen, and Don Stewart Introduction to Functional Programming Course with Eric Meijer The Joy of Haskell Haskell eXchange 2017 - A Monoid For All Seasons Transcript: CHARLES: Hello everybody and welcome to The Frontside Podcast, Episode 83. My name is Charles Lowell, a developer here at the Frontside and your podcast host-in-training. With me today on the podcast is Elrick also. Hello Elrick. ELRICK: Hello. How you doing? CHARLES: I'm doing well. I'm glad to have you on this one. I'm glad to be doing this podcast in general. We have someone on the podcast today who I've been following for, I guess probably about two years because she published a book that has been very, very helpful to me. It's one that I recommend to a lot of people. It is learning Haskell from first principles. With us on the show is Julie Moronuki, who is co-author of that book. Thank you so much, Julie for coming. JULIE: Yes, hi! Happy to be here. It's nice to finally get to talk to you. CHARLES: Yeah. One of the reasons I wanted to have you on the podcast was because I feel as though you have one of the most unique origin stories because of programming and entering in the tech world. Most of us are curious, we either come from video games or maybe we just start fiddling with the web browser. You enter the maze from the entrance that is like hidden from all, I would say. You went straight to writing a book on Haskell, is that --? JULIE: That is what happened. In 2014 on Twitter, I met my co-author, Chris Allen and he has been trying to figure out better ways to teach people Haskell because the on-ramping, I guess of people to Haskell can be quite difficult. The materials that exist are not always accessible and people felt like they need the advanced math degrees before they can write Haskell. He was trying to figure out better ways to introduce people to it. Since I was this person who's never programmed before -- I have no background -- and then he thought, "This will be a very different experience, trying to teach Haskell to her." Because I have a linguistics background and stuff he thought, "That would be interesting too and maybe, she'd be interested eventually in doing NLP." I said, I'm not -- CHARLES: What's that? Acronym alert. JULIE: Oh, yeah. Sorry. Natural Language Processing. I said, "You know, I've never done any programming and I don't play video games and I never have had any desire to learn computer programming. I don't think I'm going to like this. I don't think this is going to last but sure, I will try," and so I did a little bit. I read a little bit of 'Learn You a Haskell for Great Good.' I've read some other things. CHARLES: This was before you guys had the idea of actually writing a book. JULIE: Yes. He had the idea of turning some of his thoughts about teaching Haskell into a book and as he would explain things to me, like the questions I had about 'Learn You a Haskell,' I'd be like, "We should write this down," and he would say, "It's so hard to write it though. It's easy when I'm explaining it to you and it's so hard to write it." Initially, it started that I was helping him at things that he was teaching me and then as we got further into the book and I started reading a lot of other Haskell stuff on my own and figuring stuff out, I was writing more and more of it. Then we were kind of equal co-authors after not too long. That's how it happened. I really didn't think that I would stick with Haskell or with programming. I'm still sometimes I'm not sure about programming. I'm not sure about this whole making software thing. But Haskell is so interesting to me that I'm still here. CHARLES: That is fantastic and it's a great story. I'm curious, when you were doing the proto-research to learning Haskell, coming from really truly first principles and having no experience of programming, what made a good resource versus a bad resource? What are the things that you gravitated towards and say, "This is really instructive." What was the tone there? JULIE: One of the major problems ahead of most of the Haskell resources that exist is they assume that you've done programming before because nobody learns Haskell as a first language so they all assume that you have done some programming before. They would make references to things that if you were a programmer, you would know what they meant but I didn't. That was one of the hardest things for me. Even 'Learn You a Haskell' does that to some extent. CHARLES: What's an example of that? JULIE: I had learned a little bit about recursion from linguistics because that's a thing in human language so I really understood recursion but most of the Haskell resources explain it to you primarily in terms of, "This will be like your loops in other languages." I'm going to be like, "I don't know what a loop is. This isn't helpful for me." There are a lot of things that I didn't understand so when people talk about Haskell as being a pure functional language, neither pure nor functional necessarily, I didn't have anything to contrast them with so they didn't necessarily make sense to me as things that make Haskell different from other languages. I didn't know what imperative programming was and people would say, "In contrast to imperative programming, functional programming does this," and I'd be like, "Okay, but I don't understand what the imperative programming way is so this contrast isn't making any sense to me and same thing with purity." There were a lot of things I had to learn, in fact about mutable state because I didn't know anything about it. I had some understanding of how computer memory works but still some of the ways that people talk about it were not obvious to me. CHARLES: Do you find that seeking out that contrast actually wasn't helpful? Is it noise since at least at the beginning, it's something you'll never do. It's like saying, "Over in France, they wear these kind of socks." Since I'm going out into the street in front of my house, I don't really care. JULIE: Right. In the beginning, it was a lot of noise and I understand why they do that because they are making the assumption that everybody who is learning Haskell has come from some other programming language, probably an imperative one so I understand why that happens but in the beginning, it was very much noise for me. I noticed a lot of Haskell resources, one of the first things they tell you is that in Haskell you can't do 'x = x + 1'. I was like, "If I'm reading this like it's mathematics, why would I think I could do that." If you come from a different programming language, you might well think that you can do that but in Haskell, we can't so making that contrast, when I didn't have that background was really just confusing for me. Now, because I teach people and most of them do have some background in an imperative language, understanding the contrast is more helpful to me but in the beginning it was just confusing and noise. When we wrote Haskell book, we tried not to make those kinds of references and like, "Let's assume that everybody is just like Julie, doesn't know a different programming language that we can contrast it with and let's try to write a book like that." CHARLES: Right. I think that's a key insight because some people would say there's a lot missing or that difference might stand out. Now, that you pointed out, I can see it but I don't think I noticed it while I was reading it. But one of the things that I like is because I also tried to learn Haskell through 'Learn You a Haskell,' and I didn't find it very helpful. I found it entertaining and it's not a knock against the authors. Some of the sketches were really cute but it was still more explaining... I don't know. It was explaining more of the how, than the why, if that makes any sense where I felt as though in your book, there were a lot more analogies to actual human experiences, using the visceral language saying, "A mono is something you can mash together or squeezed together." That really connected for me. Whereas, explaining it in terms of concatenation and laws and stuff like that. Those things seem cited to the secondary resources to the primary resource. JULIE: Yeah. I think that's kind of helpful for me too. There are different Haskell books that have, I think different things about them that are good. I forget the name of the book but Graham Hutton's book, the way he talks about recursion was really helpful to me. The way he explains recursion and of course, folds but folds are things that he's known for so those parts of that book are helpful for me. But really the best book other than my own of course, for me is Simon Thompson's. I think it's called ‘The Craft of Functional Programming' and I think it does better at explaining things just in terms of Haskell. Real World Haskell, I guess is really good. It was harder for me because I hadn't been a programmer before. I think it's got so many practical exercises that -- CHARLES: Was that the O'Reilly book by Irish gentleman whose name eludes me? JULIE: Yes, Brian O'Sullivan. It makes more sense to me now but there were things in it that are sort of programmer things. Because I'd never made software before, that were really confusing for me. But Simon Thompson's, because his book does have exercises and they were ones that I could understand and do. They were fairly self-contained. My first experience actually in writing a program that does IO was from his book and I was just so thrilled. I was like, "I got it. I did it." That was really helpful book for me but I don't see people recommend that one as often but that was probably the best one for me. CHARLES: Yeah, it's always a balance because the Real World Haskell didn't really worked for me, almost because the examples were too pragmatic or too complex and I picked this up when I was 10 years into my programming career and I struggled to follow the JSON parser example, which is parsing JSON is something that I've actually done several times in multiple languages and I still struggled with it. JULIE: Whereas for me, I don't even know what JSON is. This is not something I've ever dealt with. I know what it is now sort of, but it's still not something that I deal with very much. I was just like, "What is this? I don't even know what to do here." It wasn't quite as helpful for me. I've heard a lot of people have success with that one but I think they don't share quite the same richness of programming experience with Brian O'Sullivan. I think it's a little bit more difficult. ELRICK: These are a lot of amazing resources that I wish I knew about when I try to learn Haskell. I took an online course with, I think it's like Eric Meijer and that class was very intense. Looking back, what would you say are some areas that someone should, either start to look into before they step into the Haskell world, being that you didn't come from a programming background but connecting to dots backwards now? What would you say are some areas that people can slowly ramp up into to get into Haskell and functional programming? JULIE: When I teach people Haskell, the people who have the easiest time are people who have been writing Scala for a while and they've moved over to the FP in Scala side. When I first started Haskell, I heard a lot of people make jokes about how Scala is a gateway drug to Haskell. I think there's actually might be so truth in that because I certainly have a lot of students that were Java programmers, then they got interested in Scala because maybe Scala is better for some things than Java and then they start moving more and more over to the FP in Scala side. Those are probably the students that have, I think the easiest time making the transition to Haskell that I've had anyway. But you know, I think even JavaScript, trying to write in a more functional style and there are some resources for that and really, there's a very good tutorial about monads that uses all the code examples in JavaScript. I think a lot of the concepts that you can start to approach them from other languages. Haskell is still going to be weird in a lot of ways and another thing that works for a lot of people is going to Elm. Elm is similar to Haskell but different. I think that that has worked also for a lot of people getting them into understanding more functional programming concepts but with the much easier... The word easy is so -- ELRICK: It's like a relative term like, "Oh, this is easy." JULIE: It is. CHARLES: Easy to say, right. ELRICK: That's what I thought when I step into learning Haskell and functional programming. I was like, "How bad could it be?" JULIE: Right. Learning Haskell can be very bad. I'm not going to kid around about that. It's a shame because I don't think that it needs to be that bad but the way it's presented oftentimes, for various reasons, I think why Haskell gets presented the way it does but I don't think it needs to have it like that. The designer of Elm, whose name I'm not going to try to pronounce because I don't know how you say his last name, he really made an effort to for example, the error messages in Haskell can be very intimidating. The situation there has improved since I started learning Haskell but they can be quite intimidating and he really made an effort to make very friendly error messages, very helpful error messages. I think that it shows and then it makes a difference for people who are learning. If you start with Elm and then you do want to see what Haskell or PureScript, which is also frontend language, mostly. It compose of JavaScript but it's very Haskell-like, then from Elm, let's see if we can get a little more hardcore Haskell. I think the transition to Haskell or PureScript is easier from there. I think it does help to move in the functional direction from whatever language you're in, if you do FP in Scala or try moving to more functional JavaScript or even Elm. Then Haskell will make it more sense from there or be a little easier to approach. CHARLES: Yeah, and I definitely think that for, at least from my perspective, I've been able to take a lot of those concepts that I've learned from Haskell and then apply them, even inside Vanilla JavaScript. There are things that have become indispensable like mapping and folding and they exist in JavaScript. You can reduce arrays, which is a similar to a fold and then you can map arrays but understanding that map, the key insight for me that I got from learning Haskell is that there's a whole class of values that you can map, not just arrays. The standard JavaScript object is essentially a Functor and will get a little bit to that because for people listening what that word even means and the meta around the fact that they're all these weird words and how do I go about something I want to ask you about. But the trees can be mapped and the objects can be mapped and all of the sudden, it's like this one concept that I use so much for lists, it's available on all these different data structures and it's get me thinking like, "What other data structures can I use this operation? What are the things are Functors that I'm working with?" Really, it's changed my perspective to think about the type of the data structure, in terms of the operations. JULIE: I'm in favor of keeping the terminology that we have but just explaining it much better. That's the approach that I take but it can be very hard, especially it was your first learning Haskell. I don't know if you've seen the Haskell pyramid but to get sort of productive where you can write programs in Haskell is not a very high bar. It feels like it is when you first start but it's not really very high bar but Haskell just keeps growing and growing and getting deeper and deeper so you're always approaching new libraries that you've never seen before and you feel then you've been learning Haskell all over again because they're written in a very different style of Haskell or they have even more terminology, even more kinds of Functors that you've never heard of before or something like that so you're always approaching these things over again. It can be a very intimidating feeling and it makes a lot of people very uncomfortable and I'd say, if you like Haskell and that does make you feel uncomfortable, then you don't actually need to do that because a lot of people write Haskell very happily every day in their jobs even and don't do that. They don't mess with some of the newer, super cool libraries that have all this funky terminology and stuff. Some of them don't mess with them at all. CHARLES: But certainly, there is some concepts that are core. I'm thinking of like applicative and Functor and all these things that I'm learning about and I'm curious to hear about your experience as you climb that pyramid. What is the pyramid entailed? First of all, I'd love to hear more about it because this is actually the first time I heard about the Haskell pyramid. JULIE: Say you understand monads, then you can write really a lot of Haskell programs. Probably at some point, you will need to understand monads transformers but if you just get to the point where you understand monads pretty decently, you can write a lot of software so after that, then learning more is maybe going to improve your Haskell, maybe let you write some things that you couldn't write before but a lot of it above, not that these things are necessarily in an hierarchical progression. We cover monad transformers in a fair bit of detail in Haskell book but if you get anything beyond what's in Haskell book, one of those things that some of them are very interesting, some of them can make you much more productive but some of them are also people do them for fun to explore the space and some people love them and some people hate them. Haskell lets you do a lot of things for fun and exploring mathematics in ways that are interesting and exciting and may influence and in fact, have influenced other languages like [inaudible] in PureScript but not really necessary for basic Haskell programming. A nice thing happened while we're writing Haskell book. I was writing, I think it's chapter six, which is about type classes. I was writing that chapter and at the same time, my co-author had started writing the Monoid chapter. The type classes chapter comes in chapter six and we introduce a lot of the basic type classes: num and eq and some of those in that chapter because I do think it's important. Type classes are very special thing about Haskell so I think it's important to, at least start coming to groups at them early. Some people disagree with me about that and think they can ignore them for much longer. But at any rate, it is where it is and I felt that that was important. Maybe the real motivation for type classes, really until we started writing the Monoid chapter so he started writing that while I was working on type classes chapter and he sent me the beginnings of the Monoid chapter to look at. At first I thought, "We've got addition and multiplication and list concatenation and this just doesn't seem interesting. What is this generalization of a Monoid that I'm supposed to get from these three things? And why bother making it a type class," because additional and multiplication are already in the num type class and then list concatenation is just for list so why make this into a type class and what's that motivation there. With eq, we want a quality -- CHARLES: Is that how you pronounce 'eq?' JULIE: That's how I pronounce it because 'equal' or equality. CHARLES: Okay, so this is a type class for doing what? Making sure to being able to compare two values on the same value. JULIE: Yes and it's a weird one because for most data types, you can have an eq instance and you want probably, in a lot of cases to have that but we don't want because function is a data type in Haskell so you don't want to have an eq instance for functions and that's why equality is not implemented generally for everything. That's why it's a type class so there's no instance for functions because that's not decidable. You can't decide if two functions are equal, generally. Some functions you can but in the general terms, for datatype, you can't. CHARLES: That's actually a pretty profound statement. Proof of which is left as an exercise for the listener. JULIE: We got to the Monoid and I was like, "What is the [inaudible]," or something. It turns out that there are Monoids everywhere. There's all kinds of things that you want to, either concatenate or make a product of. Then having this as a type class and thinking of it in terms of like, "We've got this abstraction. We've got this category. We've got this algebraic structure. Now, we can look for in all these other places," because once you've named the thing, then you can talk about it and think about it in a little bit of the different way. It's like, "Now, we've got this group of addition, multiplication, list concatenation." Now, we've got an abstraction of that and we can think, "Where else can I see this pattern?" and it turns out it's all over the place. For me, that was one of our thought like, "Type classes are actually really cool and powerful and interesting thing." For me, that was when it seemed like, "The terminology is worth it because, now I want to think about finding these algebraic structures and in all these other places." CHARLES: Right and like a Monoid, it could essentially be called, if you're using a Java interface, like 'mashable togetherable' or 'concatenatable' or something like that. But there's a kind of one-to-one correspondence but it is a vocabulary that just needs to be learned. JULIE: I don't know much about category theory or anything but the other cool thing about Monoid for me was that there are almost always two because there's almost always one that's destructive or additive or concatenative and there's almost always one that is conjunctive or a product or multiplicative. It's often across product that would be the zipless Monoid that exist in base and it's a cross product of the two lists. There's almost always two, whereas when you think of Monoids in the very abstract looking category theory, it doesn't matter if it's addition or multiplication. The operation doesn't matter, whether it's addition or multiplication or concatenation or cross product because you generalize the actual operation to the extent where what it's going to produce. It doesn't matter anymore. For me, I still think of Monoids in terms of like set theory or Boolean Algebra, then that's one of the things that I think is difficult with Haskell where people talk about Monoids in terms of category theory but I think that's not very helpful for the actual programmer who has to actually deal with the two different instances like sum and products or concatenation and zipping are going to actually act different in a program. CHARLES: Right, they're going to yield a different set of values. JULIE: Yes. CHARLES: Is there a baseline vocabulary? I kind of think of it like learning a new language, right? JULIE: Yep. CHARLES: When you're learning Haskell, you're not just learning a new language. You're literally learning a new language. I could go and I could learn Japanese but it's going to be a struggle at some point. People say certain languages are hard and certain language are easy. I don't generally subscribe to that. I think that most of it is just going about and living in a place where they speak this language and you'll absorb it and it's the decision to go and live there -- that's kind of the primary one. But let's say, you're a foreigner and you're travelling to this country called Haskell that's got this strange language. Like other human languages, it's just got different names associated with different concepts and some of the concepts might even just be unique to that country. Just like when you're travelling and acquiring a human language, there's a certain level of vocabulary that you need to achieve before you can do things like buy groceries and be able to transact financial exchanges or have a conversation about the weather. What are the kind of the levels of vocabulary that you need to acquire to be operational in Haskell or I would say, even in functional programming because now that I've been exposed to this, I see it in Clojure. I actually see people doing this JavaScript and in Erlang, in Elixir and what have you. JULIE: Yeah, I don't really know how to answer this question. How to buy groceries in Haskell? CHARLES: Let me let scale that down because I had this horrible tendency to spend five minutes asking what I say is going to be single question but it's actually like 30. Let's take down the scope. When you were learning this vocabulary, at what point did you feel like you're really gaining traction? We're you really starting to connect the dots? JULIE: For me, I think when I got through Functor. It was when I felt like -- CHARLES: Functor and what comes before Functor? JULIE: Monoid. I think once you understand Monoid and Functor, then a lot of other concepts in Haskell will start falling into place because this is not obvious to everyone but I think once you really understand Monoid and once you really understand Functor, then applicatives are monoidal Functors and that's not obvious to everyone. Like I said, it's not obvious at first certainly, and monads have characteristics of both Monoid and Functor as well. Then you start saying, "There's all these other Functors. There's profunctors and bifunctors. I think once you really understand Monoid and Functor, a lot of the rest of Haskell starts falling into place and then type classes like alternative. Alternative is another kind of Monoid. We have all these other names that if you can see the general pattern of Monoids and Functors, I think to me anyway, a lot of it then just started falling into place. Applicatives to me seemed, I don't want to say obvious or simple but in traverse, it's same sort of thing so we have these other names for it -- traversable -- and I was like, "Why was it called traverse. I don't understand this word at all." But once I saw the type signature and what actually happens with what the function traverse does, I was like, "Okay, I see what's happening here." For me, those were the two big hills. Once I got through Monoid and Functor and really understood them well, then a lot of other stuff just come and fell into place for me. ELRICK: This is really interesting. How was a Monoid explained to you when you were first starting to learn Haskell? Then now, how do you explain what a Monoid is to someone that's learning Haskell? JULIE: When Monoid was first explained to me, it was the pattern of there's addition and multiplication and list concatenation so it generalize out that pattern and that was really hard for me to understand at first because list concatenation and addition are similar but multiplication is different. I was like, "What do these three things have in common?" What they have in common is that they take two values of a certain type and return another value of that type and that's the type signature of the main function, that's in the Monoid type class. But that doesn't really tell you very much. A lot of functions could do that, in theory at least. How you combine them is really what's interesting about Monoid and also what makes concatenation and addition different from multiplication. Fortunately in college, I had had a fair bit of exposure to Boolean Algebra so figuring out that like, "There's actually two basic genres or varieties of Monoid and they are disjunctive or additive or they are conjunctive or multiplicative," and figuring that out, to me I always think that Monoid should really be, maybe two different type classes, one for the additive Monoid like list concatenation and addition and things where you are adding two things like a set union. Then conjunctive, which would be this intersections or multiplication or cross products. I always think there's maybe should be two different type classes but there's not a good way to do that really in Haskell. Instead, we have this one type class and then we do this ugly business of wrapping them in different type names. CHARLES: Is that why you'll have a constructor for some so it's just a wrapper for an integer? JULIE: Yeah. CHARLES: I don't know if that's so bad. JULIE: I don't like it but -- CHARLES: Yeah. You know what? You do a lot more than I do so I'm going to take your word for it. JULIE: Yeah, that's exactly why. Sum and product are the wrappers for integers because integer doesn't have a Monoid. It has two Monoids over it. CHARLES: I see. There's lots of ways to combine integers. JULIE: Yeah and those are the two basic ones. Then because Monoids also have an identity so with semi-groups, then you get even more semi-groups for integers because you get max and min, because they don't have an identity so there's semi-groups. CHARLES: There's always risk getting down into the weeds with the vocabulary but I think that there's a message here because your answer to the question is really, "When I understood Monoid and I understood Functor," from that point on, the overhead that you had to expend to get other things was lower than the overhead that you had to expend to get those initial two things. For anyone listening, Monoid and Functor are probably opaque terms. You have no idea what the hell they mean. We've been talking about in things like that a little bit but then it's okay because they're a finite set of opaque terms and they're very achievable and once you can achieve those, then you've done 90% of the work and now, you're just combining them into interesting and novel ways. JULIE: Yes. I will say it that a lot of people do tell us about Haskell book that applicative is actually the hardest chapter in the book, not monad but applicative. CHARLES: Really? JULIE: Yeah. A lot of people do tell us that. Because that's the first time that you've taken the concept of Monoid and the concept of Functor and combining them into a new thing so then, once you've done that with applicative, then after that, really it's all downhill. CHARLES: Right. It seems like there's a couple of key insights. As you're climbing that hill, I like that analogy is like one, just understanding that there things like type classes so you've through attacking Monoid and through attacking Functor, you realize, "There is such a thing." By recognizing there is such a thing as a Functor, you recognize that there is the potential for other type classes like it. Then through combining it with Monoid, to get applicative, you can see, "I can actually compose these things into new instances of those things," and then that's either the crest of the hill or the Pandora's box, depending on which way you look at it. I think there's a hopeful message in there that if you can invest the time to learn these opaque terms and making them transparent to you, you can really, really, really lean heavily on that knowledge in going forward. JULIE: Yeah. I'm writing a new book now called 'The Joy of Haskell.' The idea of The Joy of Haskell is meant to be an intermediate book. For people who already know some Haskell but we want to make words like Functor more general, like in Haskell book we really focused on the type class called Functor when it's actually a concept from mathematics or actually originally from linguistics oddly enough but we really focused on the type class in there, rather than trying to explain what a Functors are generally. In the new book, in The Joy of Haskell, we're going to try to take a lot of these terms like Monoids and Functors and catamorphisms and all these other words that Haskell has used all the time and try to explain them generally. Then also give examples like interesting uses from different libraries and stuff like that. It'll service both, hopefully a guide to the vocabulary of the Haskell ecosystem and also some documentation and examples for libraries and things like that that are useful because these things do have uses. They do get used in interesting and exciting or terrifying -- maybe those are related -- ways. That's the goal of the new book is to try to make a guide to all of this vocabulary that Haskell use all the time. We're trying to do that. How do I explain Monoids, you asked. You've got two values of whatever type. It doesn't matter the type and in general, there would be two ways you can think of to combine them, either making a sum or a union of all the values in them or making some product of those values, if they contain multiple values or even if they only contain one. That's how I explain them now. I'm not certain that addition and multiplication are actually the best ways to start with that because addition and multiplication don't act quite like set union and intersection do. I'm actually thinking of them in terms of and this is how I explain monoids to the people now, I start from set theory and that sounds really heavy but it doesn't have to be because I think a lot of things about sets are -- CHARLES: They're very intuitive, especially if you have visuals. JULIE: They're very intuitive, for people to think about. Yes, exactly. I explain Monoid now more in terms of set union and intersection. I'm actually giving a talk in October. It's coming up in just a couple weeks at Haskell eXchange in October 12th and 13th in London and I'm giving a talk there called 'A Monoid For All Seasons' and I'm going to try to explain the theoretic motivation for Monoids and try to explain them in those terms. Semi-group is a little bit different because lacks the identity but I'll try to explain the alternative type class and monad plus this really the same thing as alternative. These things are also just Monoids so we have these different names because it's a different type class alternative but it's really just another kind of Monoid. I'm giving that talk about set theory in Monoids in October, in a couple of weeks. People keep asking me on Twitter, "What's your obsession with Monoids," because my name on Twitter is Monoid Mary so I try to explain why I love them so much. CHARLES: Actually, it's an awesome point, which I've just gotten to experience it is what you see like, "Oh, there are these abstract things," you start searching for them. A lot of times, you'll uncover them and it'd be a real timesaver. There's the thrill of unearthing it in the first place and then when you could say, "Now that I've identified this thing as a Monoid, there's so much less that I have to write." There's like less work that I have to do. It's the same reason that we write frameworks for ourselves in software. It's like, "We love Ruby on Rails because of all the work we don't have to do." Now, you have to expend a lot of energy to work with it, using Rails an example but there's lots of software frameworks. It's like, "If you can find a good persistence framework or you can find a good thing for making a library for handling HTTP requests and responses, why would you write it all by hand in the first place?" I think the thing that's exciting for me as a developer is being able to see, "Monoid is a thing. Functor is a thing and I can now actually use this and I can use it almost as a looking glass to explore the world around me. When I see something in the landscape that just leaps out through that lens is another great one." I've been on a big kick lately but being able to say, "This is going to save me so much time because of the thoughts that I don't have to think and the code that I don't have to write." I think connecting it back to the pragmatic, I certainly have become really obsessed, maybe not about Monoids but having a type class large in your mind. JULIE: I think it's a really powerful thing. Sometimes that jargon is really useful. It's useful in a sense that it like compresses a bunch of information into a single word to remember. It's like teaching my eight-year old multiplication and we were talking about like, "It's like addition," and for us adult, I'll just go ahead say, "It's associative and commutative," but showing him that you can do those things and that addition is like that too and we're talking about that and he was so excited to learn that there's this word 'commutative' that encapsulates this idea for both concepts so he doesn't have to think like, "Addition does this thing. Multiplication does this thing." He doesn't have to remember both of those things, like he just remembers, "Commutative and they're both like this." It kind of compresses that information and what you have to remember and think about. Then it does make it easier to see that pattern in other things, then we can find commutativity in other things because now we have this pattern that we can look for and we got a name for it. We can talk about it and really, there's a lot of stuff like that in Haskell where we find some pattern that we find useful or we want to be able to talk about or easily translate to a bunch of different types, not translate is quite the right word but you know what I mean, I think. Then we give it a name and we make type class for it and then it's, "Now, we find it even more place for us." CHARLES: Right. It's about thinking less, right? JULIE: Yeah. CHARLES: That's a big misconception is that it's not about thinking more, it's about thinking less. JULIE: It really is. I think it's because there's so much kind of upfront work, where you have to learn all this new stuff upfront, then people mistake that for how much work we're always doing but in Haskell it's like, "We did all this work upfront and now we're now we're not going to think about these things anymore." ELRICK: That sounds like a good title for a book, 'Learn Haskell and you will think less," but it's true. When I struggled through that online class, I came out of that just being able to pick up any functional programming language and just hit the ground running. It is definitely a plus and you will think less. JULIE: Yeah, in the long term, I think that you do. Haskell is not a perfect language. There are things that probably can be improved. CHARLES: Now, before we go, I wanted to ask you, having had this very unique on-ramp into programming, which apparently you're still not convinced about. I'm curious what it would take to actually convince you but the real question that I have is there any advice that you have for someone who does not have a stereotypical background in programming who may not think that they would find programming interesting, who might have any number of roadblocks in terms of their own conceptions about the path forward. What advice would you have for them? JULIE: I am a bit joking when I say that I'm still not sure about writing software. I don't feel like I'm good at it and I think this is really the key. There are a bunch of domains in programming that I don't personally care about. I don't want to make web apps and I have nothing but respect and admiration for people who do. To me, it's very, very hard. CHARLES: Mostly because our tools aren't the same. JULIE: Yeah and there's just so many things outside your own program, there are just so many things that you have to think about and deal with because there's the network and there's other people's computers and they might be doing in other people software and what it might be doing. It is insane so for me it's very hard. There's a lot of domains of programming that I don't care about and when I thought about programming, that's the kind of thing I would think about. I certainly knew a lot of people who are web developers or the common programming jobs, I guess. Some of them just weren't that appealing to me and I'm not interested in making games or graphics so those are the kinds of things that I thought about for programming. There are things though that I am interested in doing. I'm very interested in natural language processing and I guess that's related to machine learning. I've recently taken up an interest in things like the raft protocol, the consensus protocol. Those kinds of things interest me a lot and there's a lot of the theory that interests me. I'm reading a dissertation right now about implementing a non-strict lambda calculus, which is what Haskell is. It's a non-strict lambda calculus and this guy's dissertations are theoretically implementing a non-strict lambda calculus. To me, the theoretical side is really interesting but then I am also interested in certain kinds of software. For some reason, I have developed quite an interest in making Twitter bots. I think that the advice I would give -- I'm rambling a little bit -- to people who think they're not interested in programming so why should they learn or whatever, is just find the thing that you are interested in and there's probably a way you can make software for that and maybe that will be the thing that will get you interested. It might not be Haskell, maybe you are interested in making web apps, in which case I would say go for Elm or PureScript, obviously because I like functional programs but Haskell might not be the best first language for you in that case but find the thing that you're interested in and there probably is a way to write software to do that. There's probably something in programming that will interest you. It's such a vast field. CHARLES: All right. I really, really like that answer. ELRICK: Yeah, that's a beautiful advice. Find your domain. CHARLES: Yeah, it's bigger than you think. JULIE: It's much bigger than you think. CHARLES: And there is a place for you. Thank you so much for coming on the show, Julie. I really, really, really enjoyed our conversation. JULIE: Yes, so did I. This is a lot of fun. CHARLES: Thank you. Now, before we go, I understand that you are going to be in London, was it roughly very, very soon, you said you were giving a talk. JULIE: Yes, the 12th and 13th of October. It will be recorded for people who can't get in. It will be recorded, I believe. CHARLES: You will be talking on 'A Monoid For All Seasons.' JULIE: Yes. CHARLES: And then you've also got The Joy of Haskell book, which you're hacking away right now, right? JULIE: Yes. CHARLES: With that, thank you so much for both of you. Thank you all for listening. What's a good place for people to reach out for you? JULIE: If they're on Twitter, I'm very active on Twitter so I'm @argumatronic on Twitter and my blog is also Argumatronic and that has more contact information. CHARLES: Fantastic. We'll link to those in the show notes. For everybody else, thank you for listening. You can get in contact with us at @TheFrontside on Twitter and Contact@Frontside.io over email. We'd love to hear from you. This just in, we're running a special. If you go to our website and enter the promo code 'ELRICK20,' you can get that 20% discount on your next custom developed web application. Go check that out. Take it easy, everybody. Bye-bye. JULIE: Bye-bye.
Chris Chuter: @Chris_Chuter Show Notes: 00:47 - Peeple: What is it? Why? 02:59 - Iterations and User Testing 13:32 - Complexity of Installation 17:26 - Device Integration 22:15 - Setup and Installation 25:35 - Laws and Building Codes 26:39 - Getting Started in this Space 31:29 - Ensuring Quality, Integration Testing, and Deployment Pipelines 33:18 - The Manufacturing Process Resources: If This Then That (IFTTT) Transcript: CHARLES: Hello, everybody and welcome to The Frontside Podcast, Episode 82. My name is Charles Lowell, a developer here at the Frontside and your podcast host-in-training. With me is Elrick Ryan. Hello Elrick. ELRICK: Hey, hello. CHARLES: And today, we are going to be continuing our series on the Internet of Things and we have someone on the podcast today who's going to talk to us about the Internet of Things. His name is Chris Chuter and he is the CEO, inventor and founder of Peeple. Hey, Chris. CHRIS: Hey. How is it going? CHARLES: It's gone well. Thanks for coming on the program. Peeple, what is it? Why don't you give us a quick overview of the product? Obviously it pertains to IoT, what is it and how did you become involved with it? Let's delve into that. CHRIS: Yes, sure. Let me give you the elevator short version first then we can dive deeper. Peeple is caller ID for your front door. The idea is when you get a phone call and you don't answer the phone, what happens? It goes to your voicemail. You know someone called you. But today, if someone comes to your house, you have no idea that they came unless you're there. This is the central problem that we solved with Peeple. It's a little device, a hardware device, an Internet of Things device that fits over the peephole in your door in the inside of your house. When someone knocks or doors open, you get a push notification on your phone. You can open up the phone and you can see a live view of your peephole. In a nutshell, Peeple is a smart peephole. CHARLES: Is it more for the case when you're not home at all or do you find the people use it for what you would traditionally use a peephole. CHRIS: It depends on the person. Now, my personal use case is for keeping track of wandering kids and that's actually inspiration for this invention. I have two boys and when one of my boys was three years old, he managed to open the door, walk out, go on to the street and walk down to the end of the street. Now, I live in Austin and I live right off the edge of a very busy street. Now, my kid didn't die or anything like that. It's not a really sad story but a neighbor brought my kid home and it was one of those moments as a parent where you're like, "Oh my God. I'm a terrible parent." But being an inventor and an engineer, I was like, "I'm going to hook something up that just tells me when my door is opened or closed," and it morphed into this invention. We showed it to people at South by Southwest almost three or four years ago. That's when we realized we were on to something that didn't exist. It was just a little camera on the door. CHARLES: Tell me about those first versions. I'm so curious. It sounds like there's a lot of layers of functionality that you've been through, a lot of iterations so I'm curious about that. What's was that zero iteration look like? CHRIS: Version 0 was made in 24 hours. It was a hackathon for... I can't remember the name of it. There was a hackathon group that recently imploded and we won this hackathon. The hackathon thing was to make something... I'm not sure if this is for Internet of Things but we were all making that kind of stuff. I made this little Raspberry Pi demo with a little mini door and I had talked to my wife and this is how I was able to make this invention, to keep track the kid as I was busy doing other stuff but I talked her into giving me 24 hours to make this one thing. Then me and another guy, David we won this hackathon. We were like, "We've got to turn this into a real thing," because one of the awards of the hackathon was you go to Silicon Valley, you show this off and you do all this cool stuff with it. We were like, "We've got to actually turn this into something that's presentable." That was Version 0. It was just a little Raspberry Pi. CHARLES: Now, what were you doing to detect the state of the door? CHRIS: That's the crazy thing. The first version of the device had more sensors on it than the final version. The first version had everything. It had a doorbell, it had a knock sensor, it had a motion, it had a speaker that played Paul McCartney's 'Someone's Knockin' At The Door,' but it had an accelerometer. I threw everything in there the first thing and half of it worked for the hackathon demo but it was good enough to win. This is something that, I guess I could call wisdom now but the real thing I learned is you start with everything and then you narrow and get it more tuned and highly focused and more precise as a device, like the difference between the iPhone and the Samsung phones. One of them is to throw everything into it and then the iPhone is just really specialize into a few things really well. The next three years, we're pulling stuff out. CHARLES: What are some examples of that calling that you're describing where you're saying, "I'm to take this out? I'm going to take this out. I'm going to take that out." CHRIS: We got rid of things like the doorbell and some of the other sensors, mainly because it was just a wiring issue and as well as we wanted to keep track when the door was opened and closed. It didn't make sense to have the speaker on there at the time so we really focused more on the accelerometer and the knock sensor for the first version of Peeple. CHARLES: That is not the final version. Is it mostly just the accelerometer? What if someone doesn't knock? I assume there's some sort of detection that goes on with the camera. CHRIS: That's the next version. That's something that we've been working on right now, what we're going to be delivering. We have delivered our first, I would say Version 1.0 of Peeple devices to our customers. There's a thousand of these or so in the wild, all around the world and the next version we have added -- and I guess this my first real announcement of this -- a motion detection module. It's not a camera-based. It's more or less magic and it just works through the door. That's the most I'm going to say on it right now because we're probably the first hardware device that it's actually using this technology. ELRICK: That's an excellent pitch. Everyone loves magic. CHRIS: Yes, it's basically magic. It works through the door. ELRICK: As you were going to these iterations, were you doing like user testing to see what users wanted? Or did you internally say, "This doesn't make sense. Let's just take this out." CHRIS: Absolutely. That's the second part of this story. After this hackathon happened, we prepared to go on the road show to go and show it off to Silicon Valley but in the meantime, this hackathon group, I think it was called AngelHack, it imploded. One of their founders made all these disparaging comments about homeless people and what essentially happened is we lost the award. They said, "We're sorry. We can't give you the award," but we had spent about three months fine-tuning, making something pretty and putting a pitch together. I went in and I pitched at a TechCrunch Meetup in Austin and we came in second at that but during that meetup, I met one of the reporters and said, "You really need to talk to these guys in San Francisco called Highway1," so I did. We eventually ended up moving to San Francisco. Now, the reason I mentioned that to answer your question is they understand this idea of user testing, I think better than a lot of people. Even though they were focused on working on hardware and getting an IoT device that works out there, they were drilling it into our heads is, "You have to get this in people's homes now. I don't care how bad it is. I don't care if you have to hire people, to sit at a peephole and just look through it and pretend like there are hardware device. You got to do this and you have to find out what the problems are, what works. I want you to look at your biggest fears of this thing and you quash them and you do that before you put any Silicon down," so we did that as best we could. CHARLES: So you did that with the Version 0 and Version 1 devices? CHRIS: Exactly, just a Version 0, I have all these pictures. We put them in about 12 to 20 homes and we have these long extension cords powering this thing because we didn't have the batteries to figure out. We had these huge lag problems. It would take like 30 seconds to a minute before something would happen. We had all these issues but in the end, people were still like, "It had these issues. You couldn't do this," but the fact that I had a door log, a door diary as what we're calling it now, that's something I never had before. That's where your secret sauce is so we ran with that. CHARLES: Yeah. That's the kind of thing it never even occurs to you. CHRIS: Exactly. In the app, or at least the early versions of the app, is you have these versions like a calendar that are like, "Okay, I got 10 visits yesterday. I got 20 visits today. No one came to visit me today. I'm so sad," but I have a calendar of, I think it was May of last year when I got visited by three or four magazine salesman in one week so you could correlate that with, "Did we have any break ins?" or something like that. CHARLES: Yeah, it would be interesting to be able to share that data with your neighborhood or somehow coordinate that. one of things I'm curious about too is you did this user testing you were talking about, doing the wiring and the installation, it's a conversation that always comes up when you're talking about custom hardware because there's always the drive to be small, there's always the drive to be have a small form factor and then you have challenges of power like how do you power this device. How cumbersome is the installation onto someone's door? CHRIS: Yeah, we had it all. That's a big difference, I think between San Francisco or Silicon Valley and other towns is there's this acceptance and there's this readiness to participate in the tech scene. We did a call out for volunteers and we had no problems finding them. They didn't mind us coming to their house and hooking up these big, bulky things and just being real intrusive. The fact that we found these people and they were the key to this early stage of, "Do you become a product or do you not?" We were only there for four months but by the end of this time that we were there, there was this legitimate tangible feeling of we're not a prototype anymore. We're a product and we didn't have a product. It was just prettier but we could see the light at the end of the tunnel. I don't think that would have happened had we not gone through this very painful experience with all these poor people that we inflicted our device on. CHARLES: This actually is fascinating because obviously, you're back in Austin now and I never heard of programs like that, like sign up to have someone come up and test it at some alpha stage prototype in your home. That sounds crazy and yet, it sounds like they were just going out of the woodwork. CHRIS: In San Francisco, it's not a problem. If I put the call out now, I probably have to really like, "Here's an Amazon gift card." I have to start doing a little bit of bribery. ELRICK: I think I would sign up just to see the cool tech. CHRIS: Yeah and those people exist. I think we don't have the means to really find them. That infrastructure already exists. In Silicon Valley, you just go down to Starbucks. CHARLES: There ought to be some sort of meetup for people who want to experiment with very early stage IoT devices here in Austin. Maybe, we'll have to look at it. If that doesn't exist, I would love being a guinea pig. I actually think there is an untapped willingness here but there's just not -- CHRIS: I think you need a critical mass of hardware people and hardware devices that are ready to be put in doors or put in the houses. There's definitely some in there. I have a lot of friends and there are hardware meetups that we go to but this stuff takes so long and it's so hard as hardware is hard. There's that small window of, "We got this little idea of a water sprinkler. Do you think anyone want to try it out?" or something like that and then the moments gone. Then six months later, there's another one. CHARLES: Yeah. I wonder if there's a way to really decrease that iteration cycle so that you can get feedback more quickly. I guess the problem is when you need a physical device, you just needed a physical device. CHRIS: We're talking about the Maker Movement and the MakerClub. If you're part of those, these people are hard to find. People that go to Maker Faires, that's the people you're looking for. CHARLES: Right. Now, transitioning because ultimately your target customer base is not makers, not people who are willing to put up with wires and cabling and people doing protracted installation. What does the kind of 1.0 product look like? Because what I'm curious is what immediately jumps to mind is this thing sounds like it's going to probably consume a lot of power. How do you get the power to that and what are the challenges and what are the tradeoffs that you have to make to try and get that power consumption down or get the installation complexity down? How complex is it today to install? CHRIS: I guess, I'll toot my own horn a little bit but I think we have one of the easiest IoT devices on the planet to install. You can possibly not even need tools. You can use your fingers but the biggest challenge for any IoT device is getting that home network connection. If there's been a few technologies through the years in which they've tried to fix this problem, basically just like self-pairing or things like that, like how Bluetooth can sometimes be really cumbersome. Now imagine that with Wi-Fi, it's the same thing but now you've got a password you've got to throw in there. That's really the only real hiccup with the installation on our device and we tried a few things. We went through about three different Wi-Fi chips before we settled on what we were using now. The first Wi-Fi chip was a TI one, which offered this nice pairing capability but it just didn't work half the time. Then we switched to a Broadcom chip, which was really solid and stable but turned out to be the most expensive component in the whole device so we had to get rid of that. The Wi-Fi issue was something we had to solve early because it goes also toward your power consumption. We have a camera and a Wi-Fi chip and both of those take up to 140 to 200 milliamps of juice when they're on. We had to be really smart of when this thing was going to be on and that's essentially when we went in parallel with the knock accelerometer. This device stays asleep most of the time and that's how we get the many months of battery life out of it. We put a rechargeable battery inside, it only turns on when it needs to and it's just hanging around waiting for an event for the rest of the time. Those were the things we were solving to get the Version 1. CHARLES: Now, it's waiting for some event but in order to receive the event, doesn't the accelerometer need to be on? Or is there some motion detector that --? CHRIS: That's a solved problem. good news was that accelerometers are extremely low power in the nano or picoamps but that's also another reason why the motion detection was going to be a hard problem because that is not, unless you're using what's called a PIR that is not a low power solution. CHARLES: Acronym alert. What is a PIR? CHRIS: It's an infrared proximity detection. That's how almost all motion detection cameras work. They have one hole for the camera and another hole for the PIR. The problem with these are is they don't work well in sunlight, outdoor-light and things like that in one of our use cases so we were kind of stuck. That's why we've recently come up with this new motion solution that doesn't rely on that technology -- the magic solution. CHARLES: All right. When we're going to find out about the magic solution? CHRIS: As soon as I ship this next version because it is being used in a few products but it's not really stateside yet and I want to save my thunder but it's something that I think is really cool. It really is magic. It's just amazing to me that it works. CHARLES: Well, I'm eager to see it. You were talking about Wi-Fi being one of the biggest challenges. That's a perfect segue. The connection to the network for something that we're always curious is discovering a new and interesting device is always a pleasure and then the next thought that almost funnels immediately after is how can I integrate this with other strange and wonderful devices to make something even more wonderful? A question we ask everybody is have you thought about how this might be a participant in an ecosystem so if there were other devices around the home, how would they even talk to the people? How might it offer information to someone looking to, maybe do some custom integration in their home? CHRIS: That's a lot of questions in one. Essentially, there's two ways of looking at it. You can look at it from your customer's perspective, what kind of customer do I think is going to have this or is going to use this the most. Back when we came up with this, there were a lot of do-it-yourself types and If This Then That protocol was out there but we really wanted to focus on something that was incredibly easy to use and didn't require you to program anything. I was really frustrated with the whole idea of Internet of Things because it almost implied that you had to be a programmer to use it. I didn't like that at that time. I've since come around to it because there's all these great tool kits out there. We initially looked at integrating with HomeKit. We thought they'd be perfect but what a lot of consumers don't realize is early HomeKit -- I don't believe it does that anymore -- made you modify your hardware to put in this special Apple hardware. When you're making a device, it is so hard just to get the hardware down. It's so expensive. To add anything or to put anything else in there, it's a huge friction point. It's really something that small startups just can't afford to do. A big Nest or a company like that have no problem but when you're making a one device, this is a big deal so we weren't able to really leverage something like HomeKit for an API. But we do have our own cloud-based API. We're RESTful API but it's just not documented and put out in a way where we want to have people programming it. But the good news is we did leverage several APIs when we were making things like the app and doing things like the push notifications and things like that. Now, it turns out that a lot of the case we used are now integrating with things like Alexa and other device protocols so we essentially get those for free. This whole ecosystem is forming around us. Just most important is to get your device out there because you have a vision for what the device will be used for. But then your customers tell you what the device is really useful for and that's when the real work starts. CHARLES: Right. I guess, it's true you have your first line of customers and I guess the use case what I was thinking of is me being a developer. I'm thinking what products could be built then using this as a component, so to speak. Have you'd given any thought to that or have anyone had approached you to say, "This is amazing. I'd like to build this meta product that integrates that," or is it kind of early days? CHRIS: Early on, that was the approach of the Internet of Things and it merged away from that in my experience. Early on, it was all about building blocks. You got to understand, these are old Zigbee Z-Wave programmers and that was the whole concept. Then it got turned on its head by, "I really have this problem that I need to solve and I don't want to have to make a bunch of building blocks to do this." For attacking it from the other side, like you're saying, building up into pieces, I really recommend you talk to the Twine guys -- super mechanical -- they're here in Austin as well. A year or so before, we came out with Peeple. They put out this device which was exactly what you're talking about. An Internet of Things type hub where you just add in all the pieces and then you integrate with everything. They can better give you a story of how that lifeline goes. CHARLES: Yeah, because it's always something you think about because you've got all these wonderful things. CHRIS: Yeah, some would say, an Internet of Things. CHARLES: Yup, or at least a floor plan. ELRICK: When someone gets a Peeple device, what is the full installation story and set up? What is the walkthrough for that? CHRIS: We have a little video of that. What you essentially do for Peeple when you're installing it on the peephole in your door, you unscrew the peephole. Now, the way Peeple's work is they need to handle doors that are variable width, depending on where you live. There's no real standard. All of the Peeple's work by having a shaft that you screw onto another side so it's basically two pieces. Now, one of those shafts holds this bracket that we include in the package. You screw that onto your door with the peephole holding it to the door, then you turn on the Peeple device and you connect it to your home Wi-Fi and then you're ready to go. That's it. CHARLES: That's the hardware side of the onboarding and then what about the software? How do I go and look at my door diary? CHRIS: You do this during the installation. You go to My.Peeple.io and there's a little button to add your Peeple device. UI-wise, it's one user interface among all the platforms whether your Android, iPhone or on a browser. You just go to that webpage and associate your account to your Peeple devices. You will have to log in. You can log in with Gmail, Facebook or just a regular email. Then you add your device and any time you go back to that page, it will show you only the videos from your device so you have a list of all the events from your Peeple device on that page or in that app. CHARLES: That is interesting. I'm looking at the videos right now online. Although my problem actually is I've got a glass door. CHRIS: Yes, we got you covered as well. CHARLES: You do? CHRIS: Yes. The reason you have a glass door or a peephole and many people don't realize this is it because it's required by law. If you ever plan to have run out your house as a multi-family unit, you have to have a peephole or a window surface to where people can look out. Once we figured that, that's when we realized we were onto something. The first versions of Peeple came with these little adhesive pads that we called gecko skin and this is where we learned a valuable lesson. No matter how sticky you make your stickers, they're not sticky enough. We included three of these little tabs in every device to put on a glass door, if you had glass so the Peeple device would work the same way for glass door, except that you would use a sticker, instead of unscrewing the peephole. The only problem with the stickers were is they were not sticky enough. If there was condensation or a weather event or something like that, these things would fall off so we made a modification. We found better stickers and I mailed those out to all the people. But this is why hardware is hard. You're going to make these mistakes. In all our testing, we didn't find this but of course, once you have a thousand testers, you find a little more. ELRICK: That's interesting that you brought up the laws about the peephole. Were there any particular building codes or anything of that nature that you guys had to be concerned about when having Peeple installed things on their doors that you had to figure out before shipping them out? CHRIS: Not really. The Texas property code is more geared among making landlords do the right thing. In case you're wondering, I think it's Texas Property Code 94-152 that covers this. There must be an external viewable portion for all multi-family units to the front entryway. Now, this is just the Texas law. We had to look this up in a few other states and it turns out there's one in San Francisco, there's one in Virginia but they're all different. But so far, we haven't had any issues with any property codes or building code issues. CHARLES: This has been an almost four-year odyssey for you that you've been on, right? CHRIS: Right. CHARLES: You've been involved in this scene and working with hardware probably for a long time even before that, it sounds like. For people who are just getting into it, because I feel like there's this wave cresting now, where these types of startups and these types of side projects and hobby projects are just starting to enter the mainstream. Do you have any advice for anybody who would want to get into this space? CHRIS: Well, that's a great question. Of course. Now, contrary to what you just stated, I didn't have much of a hardware background. I'm a software guy. I can personally attest to the pains of becoming a hardware guy. Now, the irony of this is I do have a master's degree in electronics engineering but electronic engineering is so huge. It's such a big field that you can spend your entire career not doing much hardware. But I always had the ability to go back and build some circuits but I would say the number one thing, if you're not a hardware guy is go to some of these meetups or get involved in a community and find yourself one, someone who has experience doing hardware because coming from the software room, you're used to this flexibility of changing a few lines of code and being everything changing. Now, when you get a hardware guy onboard and our hardware guy's name is Craig, when he comes to work -- CHARLES: Or gal. CHRIS: Yeah, or gal, of course. When they look at the same problems you're looking at, they're like, "Hold on a second. Let's step back. Let's test this." There's this quantitative slowing which you need to have as hardware because once you build a PCB, a circuit board, you are now stuck with that board for the next month or so because it takes a while to make another one so get that right before you jump around and do all these changes. My first advice would be is get help. There's no shame in going out there and you might be surprised. There are so many people out there that want to join in. If you have a good idea, there's plenty of people who want to contribute. CHARLES: Would you say that there are communities out there like the software communities where you have meetups? Some of the software meetups are just fantastic, where people are so welcoming and they're just so excited to share the information that they themselves are so excited about. CHRIS: Yes and there's the same thing as on the hardware side. You would definitely go to a few hardware meetups, there are several in Austin. There's at least one every week and it's a great chance for people to tell these kinds of stories. This is a maker type community so they welcome these ideas because that's what fuels their enthusiasm. Every time someone is doing something new, they want to hear it. That's the change now. This decade has happened to where you can go out and buy a few modules and make your little device. Then there's the next big step of turning it into going from prototype to hardware but you can get all those kinks out without having to make your own printed circuit boards, without having to have a huge firmware background. Just knowing a little bit of tech and a Raspberry Pi, you can test out your inventions at this early stage without having to invest all this money and these other things. There's never been a better time to do it. I would leave your listeners with is if you got something swirling around your head, get a Pi, get a little Arduino and do it. There's nothing stopping you. CHARLES: Yeah, it's shocking how affordable they are. CHRIS: I don't even touch on China, by the way but that's the next step. CHARLES: That's the great thought that I want to leave everybody with but I actually have more questions so we won't leave everybody with that. We'll keep on going because I want to talk about China and I want to talk about something that was in there. You've touched on it a couple of times when telling your story how you go from this just do it, get it out there, get it into people's homes, just get the Version 0 out, just buy an Arduino, slap together something terrible, that is at least one millionth of the dream that you have and you've taken your first step on that odyssey. That's a very common story in software. The way that we develop software too is have these agile methodologies and these techniques to reinforce them, testing, continuous integration, continuous deployment. How does that play out? A fascinating subject to me personally is how do you do that in the context of hardware. A question that I love to ask is how do you do things like ensure quality? How do you do integration testing? How do you have a deployment pipeline if you've got these Peeple devices out there on tens of thousands of doors globally? How do you push out a bug fix or a feature update? What's the automation around that look like? CHRIS: The over-the-air updates are your friend. If you're going to make a hardware device, I recommend making a Wi-Fi enabled device because then your firmware is not locked, then you can do over-the-air updates. That has been a lifesaver. We've done maybe a dozen software updates to our device to date, sometimes little changes, sometimes big changes. But what happens is any time the Peeple device wakes up, it says, "Hello, server," and the server says, "I got an update. First, let me give you all these images." Give me the code. The devices are constantly upgradable, just like you'd expect with software. Now, with some of these Bluetooth devices, you can't do that. You've got to go out the door being ready to go with no issues. It's a friction point to tell someone, "Your headphones can't work now. You need to plug it into a computer. You need to download this firmware upgrade. You need to update the firmware doing it by hand." That just isn't going to fly in today's consumer market so I would recommend if you can, make your device a hardware Wi-Fi device, get a Wi-Fi module in there and that opens up the world to you on doing a lot of these updates, to answer the last part of your question. CHARLES: You mentioned China, since you're touching on the manufacturing process or just the market over there or --? CHRIS: Yeah, be ready to fully commit. I've been to China, maybe four times now. I have a 10-year visa. It took a while to find the right partner and you've got to be boots on the ground in the factory for a couple of weeks just getting the whole line up. It's a whole another product when you're at the manufacturing stage. You're making all these little test things, they've got to hook up the boards to certain devices, they've got to put the firmware on it, they've got to do these things. It's a whole another job. That's why when you do these Kickstarter. They say, "We're going to be out in three months," and then six months later, "We're still working on it." I have a lot of empathy for this because I've lived it. You think, "I've got everything done. My hardware works. All I have to do is team up with someone to just make it and with them, we'll ship it." There's a whole another level to just a manufacturing piece and you can't really learned. There's no real textbooks to learn this because every factories are different. Our factory is right north of Shenzhen and we talked to some US manufacturers but they just weren't competitive to be in the discussion so you pretty much have to go overseas and then you have to sit down with them and just a little bit of communication difficulties can bring down a whole manufacturing line so it's very important that you're very hands on and you see your product all the way to package. ELRICK: That's interesting. I know of it but I never really thought about it because I was really not in that position. What are some of the higher level of things that you should look out for when evaluating a manufacturing partner? CHRIS: We talked to about a half a dozen before we decided on our manufacturing partner. The big one for me was cultural fit. I talked to some of the big ones like the one that makes the Apple phones, we talked to them for a while and I just found that I would say, "We would like to do this or we need this," and then the next week, they'd be asking a question, "What about this?" and I'm like, "Oh, you didn't understand what I was really asking," so you would lose weeks just by tiny misunderstandings. I found a manufacturing partner that has a subsidiary here in the US and my main contact grew up in the United States but he also goes to China every other week. Having that kind intermediary made everything so much easier. The communication was never an issue. I was able to get things done almost twice as quick with the other manufacturers I was talking to. In the end, they also came up with a great price so it turned out to be a win-win. I would recommend talking to the bigger manufacturers but spend a lot of time on the smaller ones and really figuring out is the communication up to snuff to really make your product. It's huge. CHARLES: What a story. I'm really glad that we got to have you on the podcast, Chris because you have the story that starts from literally slapping a Raspberry Pi and an accelerometer and speaker and apparently a bunch of other things on your front door and with an extension cord and walking a continuous path to where you're flying back and forth between China and Austin to inspect and ensure your assembly line and making a real product. It demonstrates that it can be done by the fact that you have done it so I think it serves as an inspirational case for a lot of people out there who might think that this is something that they might want to do. Or think that they're capable of. Thank you so much for coming and talking about Peeple. Everybody, you can go ahead and check it out. It's Peeple.io, right? CHRIS: That's correct. CHARLES: All right. Also, is there anything else that you'd like to announce other than the magic, which you're going to keep a lid on? CHRIS: Yes, I know I'd appropriately teased everyone about that but you can go to our website. If you go to Shop.Peeple.io, we're taking preorders for this next magical version, the Peeple Version 1.1, I guess I'll call it. I would like to add just before we go is if you're going to endeavor to do something like this, make sure you have a very understanding family because they couldn't have done it without a wife and kids that understood my craziness and allowed me to have just a complete mess of our house for, I guess, for three years now. CHARLES: Thanks again and thanks everybody for listening to this episode. You can get in touch with us on Twitter. We're at @TheFrontside and you can always find us on the web at Frontside.io and there's a contact form and we'd love to hear from you, for any reason whatsoever. Thanks, everybody and we'll talk to you next week.
John Boyd: LinkedIn Show Notes: 01:27 - Knocki 03:20 - The Device 06:19 - Complexity 08:44 - Software Distribution 14:01 - Allocating Memory 18:27 - Finding Hardware Hacking Libraries 22:01 - Updating and Diffing 24:06 - Migrations 26:51 - Decentralization of IoT 35:39 - Managing the Knocki Ecosystem 40:17 - Communication Standardization Resources: Malloc Transcript: CHARLES: Hello, everybody and welcome to The Frontside Podcast, Episode #81. My name is Charles Lowell. I'm a developer and your podcast host-in-training here at the Frontside. With me today is Elrick Ryan. Hello, Elrick. ELRICK: Hey, how are you doing Charles. Welcome back. CHARLES: Yeah, thank you. It's good to be back. Today we're going to be continuing the ongoing series that we've been doing intermittently on the Internet of Things. It's a really fascinating, almost to a person fascinated with here at the Frontside. Today, we have with us to talk about this, someone who's very, very knowledgeable on the subject, John Boyd, who I got an opportunity to talk with, I guess it was about a month ago and I wish that we had the podcast recording equipment there in the room because it was just a very, very well-versed engineer, exactly the person you want to be the CTO of your company, which is very lucky for Knocki, the company that he works for, because he is in fact the CTO there. Welcome to the show, John. Thanks for coming. JOHN: Yeah, thank you very much, Charles and I'm excited to be here. I'm excited to join the conversation this week. CHARLES: Yeah, why don't you start by what it is that you do at Knocki? Most of our audience comes from software and design and product management backgrounds. You've got a very strong hardware background. How does that play in to what you do at Knock? JOHN: Yes, certainly. As you previously mentioned, I'm CTO at a startup called Knocki, which you can mount onto any surface and turn that surface into a user interface. We're recently funded on Kickstarter so we're in the process of actually trying to develop this hardware but the central concept is any surface that you mount this on will now listen for touches and vibrations so you can say, mount it on a desk and tap three times on your desk and control your smart home around you. If you have smart speakers or TV, you can tap three times out of four times and control those devices with a really natural interioractive interface made out of anything in your home already. CHARLES: Tabletops, mirrors, I assume you've tested this on a lot of different services. JOHN: Yes, I'm sure we'll talk about that more a little bit later but the goal is to be able to turn any surface into user interface. That means if you really wild and you want to use it on the window, I recommend it. But we're thinking desks, walls, doors. It has a lot of applications for disabled and handicapped individuals. Think of a child or someone in a wheelchair that can't quite reach a light switch, if they have a Knocki mounted on the wall, they can still knock on the wall to control the lights. We feel like it adds a new level of user interface to people's lives that can be helpful. CHARLES: Definitely. Seeing the product and hearing you talk about it, I definitely got that impression. Now, the device that you actually brought into the office because you did come in and talk to us, like I said it was about a month ago but it was extremely tiny. In our explorations into the Internet of Things, we do things like control our lights from within the office. At least, we're trying to control our lights within the office. For us, we're using the standard kit. We've got Raspberry Pis that we're using, that are have access to a plug and they've got a full Linux install, just a really powerful processor and by comparison to the things that you were talking about, that's energy hog by comparison. We think of it as being very lightweight but if you're talking about making some small device, it's actually really, really wasteful of resources, so to speak. What is that transition that spectrum which you moved from these one-off hobbyist things where you're using high-powered equipment to these really custom devices? How do you make that transition? And what is the difference between the two? JOHN: Our devices are about the size of a hockey puck, which is much smaller if you can think of a Raspberry Pi. Pretty difficult to fit that inside of a hockey puck, especially when you want to start adding some sensors to detect knocks and taps on a surface. I don't hate or dislike the Raspberry Pi or BeagleBone Black or any of those really quick SBCs that can get you started with IoT. But they have -- CHARLES: Acronym alert. What is an SBC? JOHN: SBC, single board computer. It's any of those credit cards size computers. CHARLES: Okay, great. So nothing against the SBCs like BeagleBone Black or Raspberry Pi. JOHN: Exactly. It's a great way to prototype ideas and get in a proof of concept out there and there are some cases where actually, they're great choices for a full-fledged product. A lot of cases in IoT, people are more concerned with things that you carry around with you so they have to be battery powered and you need to be a little bit more conscious about energy economy. You need to be very cost-effective with your components and it doesn't make sense to buy an expensive Raspberry Pi for each unit. CHARLES: Did you actually start with a Raspberry Pi, when you were developing this product or something that's like an SBC? JOHN: I actually went straight to a microcontroller dev kit. I started with Texas Instruments' CC3200 LaunchPad. It's a little bit lower level than SBC like the Raspberry Pi. It doesn't run Linux. The firmware I started off writing as a proof of concept was still embedded C bare metal software. CHARLES: How much complexity does that add? There's just a lot of nice things about having an operating system and being able to have your compilers, I guess you have a compiler tool chain, but having being able to install big programs like interpreters so that you can run Ruby and JavaScript on there. There's just nice things like scheduling. If you've got a bunch of processes running on this device, you don't have to worry about them, saying who's going to get what processor time. I assume that you're having to deal with all of that if you're writing the firmware by hand using C, right? JOHN: That's 100% true. There's definitely some great advantages to using a little more powerful system that can run a full Linux stack or full OS. As you mentioned, the design complexity is reduced a lot because you can import other people's code and you have a full operating system to handle most of the drivers in the system. You're right. There's a lot more complexity. We have to write all of that ourselves in C. But that's the fun part about it to me. I love getting down there and writing drivers that can communicate with accelerometers and set them up. As far as scheduling goes, for getting concurrent software running on your embedded system. There are RTOS's -- real-time operating system that can provide basic scheduling. For the brave, you don't even have to use that. You can use a lot of the embedded timers inside the microcontroller itself. But to answer your question, it is a lot more complex but one of the tradeoffs to get a device that small, beautiful and also has a battery life that can last many months or a year. CHARLES: Yeah, it almost sounds like the complexity but you're not going to save yourself any time prototyping it in tools that have all those things because you're essentially going to have to be rewriting your system from scratch, because those things are just a nonstarter if you want low profile devices. JOHN: Yeah, there's definitely a lot of rework would have to be done but those SBC systems are still very useful for prototyping the cloud side. Internet of Things is hardware and internet when it comes to building out your cloud interface. CHARLES: Yeah, that's definitely true. You're running a bunch of software on this device. The software that you've written, how do you actually distribute the software because we're very used to in our world, software distribution is not a problem. That's what made the web so popular. While we were willing to deal with really crappy tools on the web for a really, really, really long time, the distribution model was just so nice. You're also having to deal without that too when you're operating in the device space. But the challenges are still there. If you've got a bug on one of these things, how do you even detect it and how do you get a fix out there? JOHN: Obviously, any software is prone to bugs. Nobody writes a perfect code the first time. If you do, I'd love to hear about it. Obviously, one of the big concepts in IoT is security and to have a secure product, we need to be able to patch bugs as they arrived. A big really important feature in any good IoT product is the ability to remotely upgrade the firmware or send the patches as part of the maintainability that prevents big software bugs from turning your IoT product network into a botnet. A lot of our time is actually spent trying to make sure that our remote update capabilities are reliable, always functioning and globally distributed. You'd think this is an easy problem to solve but when you're working on a microcontroller that's not running an operating system, running bare metal code, things get a little bit more complicated when you want to make sure that any device anywhere in the world can install the next version of firmware reliably. CHARLES: Right. At any time there's a software update, it's always, it bugs me and then do I want to do this and it's always optional. There's none of that, right? It's just what a new version of the firmware goes out, boom! It goes out there. JOHN: You can design it in different ways. There are some great products out there. Apply the firmware update through the user's phone so you may open up your products application and it says, "There's an update available. Go update." That's definitely one way to do it but that's the problem if the user is not home and maybe they've set this device up in a guest house and they won't be home for six more months, then you have a device that could be vulnerable for six months, which is a long time in the world of software. CHARLES: Yeah, that's true. JOHN: To get around that, obviously our preferred solution is to have the device checked into our cloud servers to see if the device itself has updates available and then go through the download and update process that way, just to make sure even if the user is not home or never opens their mobile app, it will still get those critical security updates. CHARLES: Sounds hard. You're running a risk of bricking someone's device if that update doesn't go very well or it loses internet in the middle or power. JOHN: Very true, especially when you go towards a bare metal microcontroller with limited memory and limited processing capabilities, unreliable internet connection, a lot of work has to be done on the device side to make sure if something goes wrong during the firmware download process or installing the image correctly that it has a backup image. If you're downloading a new firmware upgrade and the download gets corrupted halfway through, make sure you have an old image that you can boot into. That's one part of it. The other part is detecting that it went bad if it gets past downloads in your image and then it reboot itself and tries to boot into it, how do you know that that image actually isn't behaving the way you want it to and then go ahead and revert back into that original stable version. CHARLES: I assume there's some key so that you can verify, not only that the image is not corrupt but it's a certified Knocki image that's coming down the wire? JOHN: Exactly. We signature verification, again something that I think anybody on the internet should be using when you download new software but make sure that the new firmware update was actually written by Knocki and you're not installing someone else's code. Another important factor is just please use HTTPS secure SSL connections to your server, then that reduces the possibility of someone taking over and giving you their own firmware image. But there are a lot of low power devices out there that are being used to make IoT products. These low power devices are important for many reasons but they have restrictions and sometimes, their security capabilities are limited. Maybe doing encryption on the device and actually are doing certificate verification. That's a costly operation. CHARLES: It sounds like there's a lot of cycles that that consumes. JOHN: Definitely. Most people try to make sure they have the resources to solve these problems but at the same time, there are a lot of developers out there that are cutting corners and that's where you get these big news stories about IoT products getting taken over. CHARLES: Along that vein, it's your reality but it constantly blows my mind that things that you're living without when you're programming for these devices like Knocki is do you have to write your own network stack? When you're doing these downloads, that's kind of like got it all. You've got the encryption piece that you've got to do to make sure that you're connecting over SSL so you've got to do the whole handshake and you've got to do the key exchange and the certificate verification and then the packets come in asynchronously so your message is arriving asynchronously in bits so the header is being assembled, now I've got the HTTP headers, now I can go ahead and get the body. There's a lot that happens for us when we're making a simple Ruby request. We're basically like resource.get. Boom! And it just comes to us fully assembled in memory. How much do you have to hand roll all of that? Are there libraries for doing it? How do you put that process together of just even downloading the image? JOHN: Fortunately, there are tons of open source freely available libraries for embedded C software that can help us solve these problems -- CHARLES: Is this like a genre of software like if I want to go look for these libraries, how I look for them? JOHN: In my example, all of our firmware is written in C or C++. Since we're working on a microcontroller with limited resources, it's important to look for libraries that don't use dynamic memory allocation. That's why it's a really big [inaudible]. Some software relies really heavily on that but -- CHARLES: When you say dynamic memory allocation, you're talking about like Malloc? JOHN: Exactly. CHARLES: You are basically are allocating memory on the heap. When you're doing for this, you basically want to do everything on the stack. Now, is that just because the instruction set of the processor doesn't support it or is it because it's just there be dragons like here there be dragons? JOHN: That particular scenario is actually just due to resource limitations. There's just not a lot of memory on our device. We do use Malloc in some cases but we have to be very careful about when we use it and make sure that it's always going to have the memory required or if it doesn't have the memory required, there's some fail safes involved. If you just use someone else's open source library and they're allocating memory left and right, they could end up causing issues on your embedded system. CHARLES: Right. Now, just a little bit of background for people who might not be fully familiar with Malloc, it's just when you're executing a program, you have this heap memory, which is where you store random stuff and then you have your stuff on like the call stack. Your variables that are on the call stack are in one place and then your just generic data structures that could be accessed from anywhere are in this thing called the heap. Our dynamic languages that we use like Ruby and JavaScript, the heap is hot stuff. Like everything gets allocated on the heap, that's why they consume these huge amounts of memory and then the things that are on the stack, really are just pointers that are referencing these big bags of data that are on the heap. But it sounds like you've got the exact opposite situation where you don't want to have big bags of memory that are just floating around in a heap and you want to do everything inside that stack. JOHN: Exactly. I couldn't have said it better. CHARLES: Anyway, you're looking for libraries that don't do that because it sounds like any time you want to allocate memory on the heap, that's going to be shared for the whole program, that space is very limited so you want to be very, very, very strict. You want to control that process. You don't want any other library that's doing it for you. Is that fair? JOHN: That's correct. That's also one specific example, dynamic memory allocation of the things that you want to make sure your other software libraries aren't going to be abusing. But in general, you need to make sure that any code that you're putting is compatible with your system. It doesn't have some special hardware requirement that your embedded system doesn't have. CHARLES: Right. For people who want to get into hardware hacking, is there some golden seal of approval like the people say like, "This library is great for embedded devices." Like I said, a lot of times when you're coming into it, you don't know what to look for so what you're really looking for is some expert or authority on the subject who can say, "This is good. This is not good." It is like, "Don't even look at this library because you're going to find something else because this is not embedded-friendly." JOHN: That's a good point. I wish there was a golden seal of approval or I wish I knew one, at least. Normally, most of our code that we uses are hosted on GitHub. Usually, we try to find software that was optimized from embedded systems and the author of that code will usually mention -- CHARLES: That [inaudible] me. JOHN: Exactly. This was designed for microcontrollers. ELRICK: I was going to ask if there's a golden standard when you're building these type of devices. Is there a checklist of things that if someone's going to build something similar that these are good things on your checklist that you should attempt to check off, if you're building this sort of device or want to build something similar. CHARLES: Now, you mean things like update and whatnot? ELRICK: Like updating or like how you were mentioning avoiding dynamic memory allocation. Anything, you can just shoot from the hip, like these are things that you should watch out for a lot of your battery power, you should look out for this or anything. JOHN: Yes. I definitely think the number one consideration that the biggest check box and [inaudible] before it goes out the door is going to be your security suite. Make sure your internet connections are encrypted: SSL, TLSL, that good stuff. Then as we hit on earlier, making sure that you always have a way of updating the device but don't use back doors. A lot of people think to update your device, you should put a back door access and you can go in and download updates that way. That's not the answer. ELRICK: That's like the back door that they were looking for in Apple like, "Do you guys have a backdoor to get into your device?" No. JOHN: No. That can be a controversial conversation. CHARLES: Yeah, or they're like, "Come on, really. It's okay. You can show us the backdoor." No, there is no backdoor. "I know you have to say that. Blink-blink." ELRICK: That's an interesting problem that you guys are solving on how to update these devices. You guys are essentially hand rolling or developing custom software to do that. JOHN: Again fortunately, we're using a Texas Instruments SSC system on a chip. They provide some core functionality, some core drivers that really help us out. For example, they provide a special bootloader that can really assist with a lot of the firmware download back up framework image checking, that sort of functionality. We don't have to write it all by scratch but we do have to write the logic to make sure that the device does check for updates and it doesn't forget to check in and talk to us. ELRICK: On the cloud side, do you guys have to write any custom software to do diffing, to make sure like -- Oh, do you diffing? Or do you just update everything all complete, like once you're updating, you're going to get a brand new update or do you diff and say, "You only need this." JOHN: Since we're working on the system that we're using, it just requires a fully-compiled image that gets installed by the bootloader. We can't really send just a patch to one part of the firmware, if that's what you're asking. CHARLES: But I assume there must be some state that's on the Knocki itself. Just even the credentials for the local Wi-Fi network, what devices it's connected to, part of the system is updating and part of it is not, I assume but how do you make sure that that state is compatible with the new firmware? JOHN: Yeah, that's another great point to keep in mind. The way we keep most of, we call it nonvolatile memory, every time the device reboot, it's going to forget about everything that was stored in RAM so we need to have somewhere in nonvolatile to store these things. We have a file system on the device that we can create files with different device configurations, algorithm, settings, Wi-Fi credentials, that sort of stuff. CHARLES: That file system, is that anything that we would even be familiar with like ZFS or is it just a custom file system that you've written or that you found on GitHub. JOHN: No, fortunately this is just a standard FAT file system. We do have some creature comforts there but that's not necessarily the norm. CHARLES: You heard it here. Is that FAT16? JOHN: No, it's FAT32. CHARLES: FAT32, described as creature comfort. JOHN: Yeah, we have a different perspective of creature comfort. CHARLES: There's a couple of things because immediately, what this brings to mind is for people who are familiar with Ruby on Rails, they have this concept of migrations, where you're migrating the schema of your database and as you have to transform the data from one format to the other, you're running these migrations. One of the things that's nice about that is if, let's say I have some system that is at Version 1, but let's say, I have one of the devices that hasn't taken an update, it starts at Version 1 and it needs to go to like Version 100. But you could have 10 format changes in between there. Is there a way to handle that case where you're basically incrementally applying a bunch of transforms? JOHN: Yes. That's another great point. We take this on a case-by-case basis. Fortunately, being a small relatively simple system, there's not a whole lot of state data to keep track of. But to handle that situation, we've written are own OTA server-side software that manages the devices sending updates -- CHARLES: Acronym alert, OTA? JOHN: I'm sorry, yeah. Another acronym, OTA -- over the air updates. That's our slang for remotely sending firmware updates. CHARLES: Sorry to interrupt. It's just we have to unpack acronyms. JOHN: No, I'm sorry. I use a lot of jargons here. CHARLES: You know what? The thing is, so do I and I just never even notice it. JOHN: To handle that scenario, the way we handle it, our cloud knows what devices are out there and what firmware updates we've sent out to it. Furthermore, when the device checks in with the cloud and ask, "Do I have an update available?" It also tells the cloud, "By the way, I'm running Version 1.0." The cloud knows, if it's on Version 1.0, there's going to be some incremental changes that need to be made before we get to that last update and we can apply those changes incrementally. CHARLES: I see. I feel like we've touched on so many of these concepts that are universal to development but only projected into the hardware space. We've talked about dynamic allocation of memory and data migrations and it sounds like what you're describing in a way with OTAs is continuous delivery, where you have some way of automatically pushing out an update and all the stuff that's involved in that. It's just really cool to hear to view through such a vastly different lens than what we're used to. ELRICK: We've been talking a lot about communication between devices and back to the cloud in things of that nature. Does that play into the conversation around decentralization of IoT infrastructure and what does decentralization of IoT even mean? JOHN: Decentralization as a new methodology or ideology that a lot of people are adopting, I shouldn't say new. It's been around forever but the idea is from a high level, looking at the internet, most of the internet is access through some central, server is hosted on you name it -- XYZ cloud hosting provider. The way you do your URL DNS resolution that goes through centralized DNS servers that say, "You want to look at Netflix?" Netflix is stored over here on this AWS server farm. Decentralization, the idea is we don't necessarily need to talk to this DNS server and talk to AWS just to get content from specific providers. If you look at IoT for example, a lot of times in our case, we want to tap three times on the table and then later on, it will do the cloud, send the message and then turn on your Philips Hue light bulb in the living room. It would be great if the message could just go directly from Knocki to the Philips Hue light bulb, rather than going to our cloud, on some centralized hosting provider, then to Philip Hue's cloud, on their provider then out to the Philips Hue light bulb. Those are some of the really popular technologies that's a lot of people are talking about that really take advantage of the concept of decentralization. But it does -- CHARLES: Let me understand because why these would be necessary. When I get why it's compelling, if I want to have my Knocki talking directly to my Philips Hue light bulb without getting your servers involved, without getting Hue's servers involved, it seems like it's going to be a lot faster and just a lot more robust. There's just less links in the chain but it presents its own problems, like on both ends of the conversation between the Knocki and the Philips Hue, how do they agree that this is sanctioned by a user? That's just leaps out. That's a hard problem to solve. ELRICK: That you use some sort of like public-private key type of encryption to say, "It's me. Am I allowed to do this?" CHARLES: How do you decentralize that? JOHN: Well, I'd like to preface this by saying I'm not an expert on that particular subject but the goal is, if you're familiar with the bit torrent protocol and how it keeps track of a lot of different peers on a network using distributed hash tables, the idea is if you know at least one other person on the network, that person can say, "There's some other people that you may be interested in talking to, that may actually want your message. I'm just a bystander on the network and I don't really need your message but this guy is interested in it." In our application, that would be our server. We have to ask our server, who's out there that wants to hear what I have to say. The server is going to say, "Knocki 123, this Phillips Hue is over here at this address, this unique resource identifier, he's going to be very interested when you have two taps or three taps on the desk so just go ahead and talk directly to him. You don't need to talk to me." There's a lot more of that goes into that about making sure that the network can heal itself if somebody goes offline. But as I said I'm not really an expert in that subject. CHARLES: Right, but it really is compelling. Would you then, maybe have some device that was just kind of your coordinator in your home or multiple devices that would act as these bit torrent trackers? JOHN: Yeah, I think -- CHARLES: Or would the devices themselves actually be able to do that, like the Knocki could actually participate in the conversation about what other devices there were in the home. JOHN: Exactly, yeah. I think in a true peer-to-peer network, any peer can talk to another peer and eventually learn where the other node that they want to talk to is. You don't have to talk to any one particular person but you can ask anybody and they can tell you how to talk to the person you're looking for. The really big advantage to decentralization in my opinion is security. A lot of times if everything is controlled through one central point, that's one central point of failure. If someone DDOS's your cloud service, then now your entire network of devices is offline, just because one location got attacked. If it's a decentralized network, there's no one central point of failure and it's very, very difficult for someone to attack your network. CHARLES: Right, that's true but the tradeoff then is complexity that your decentralize network has to agree, somehow come to some consensus. It's very easy to generate with consensus when you have one process or one point that's driving everything. JOHN: Exactly. Another big tradeoff is ownership of the data and enterprise today are really big revenue your point for a company is being able to have ownership of data and extract meaningful insights. But if your device doesn't talk to your central server every time I want to do something, how does your server know everything that your device is doing and you lose a lot of that data. There's a tradeoff there in how you're going to get the data you need to run your business but also let your device run autonomously on decentralized network. ELRICK: Do you think that this is going to be helpful or harmful to IoT? What's your views on decentralization? JOHN: I think it could be very powerful. Right now, I'm not aware of any products that are really using a decentralized architecture for IoT and the main reason for that is companies and developers are a little slow to adopt it because they want to have that ownership of every data packet that goes to the network. They own it. They can see it. But I think in the future, people will start to realize that they can still get the data that they need to run this business. They can still have visibility and control over the network the way they need to run their business without controlling every single packet. When that happens, I think it's going to be a revolution for the internet as a whole but it's really going to revolutionize IoT and devices will get lower power. They'll get faster and they'll get more secure. CHARLES: When you say being able to get the data that they need, is it just being able to asynchronously spool off the data later? I guess I'm trying to understand how they get the data if it's never talking to some central servers? Or is it just you will get the data at the time you want it or there will be some delay? I assume you can also have your server being part of... I don't know. I'm just curious how you see that playing out. JOHN: I think, every developer is going to have to tackle that on a case-by-case scenario but take for example a big brand smart thermostat company. They have a device that's going to control your AC heating and air and the house and it also collects a lot of the data from when you're home and when you're using it to be smart and adjust the temperature at certain times of day even when you're not home. Again, I don't work for any company that does that and I don't know how they're doing their devices under the hood but traditionally, they tap to a centralized server and they send a lot of this information whenever it's happening, always to the server. Every time the user adjust the temperature, it sends an update to the server and says, "The user just updated." In a decentralized network, these devices can just talk to themselves and say maybe periodically or every day and it'll just send one update and say, "The user adjusted it." You can still talk to a central server but it doesn't have to rely on the central server. CHARLES: Right. It's just what we call, an out of band process. JOHN: Exactly, not mission critical. CHARLES: Okay, I got it. Talking about the decentralization and interacting with other devices, how do you manage the ecosystem right now with Knocki? It's a general purpose interface to rise. It serves really the role of a keyboard or a mouse or some way of controlling other devices and other systems. I assume that in order to do that, you have to understand the capabilities of those systems or maybe you don't. How do you integrate these two devices? Let's go with the thermostat and the Knocki or maybe one that you're more familiar with that you've done. Do you all have to write the integration? Can a third party write the integration? Or is there some way to automatically discover and map the existing inputs of the device. I feel like we've got all these new devices are coming out day to day then and now, there's more and more permutations in which to confine these devices into a coherent system and I'm just curious to hear about that integration story from your perspective. JOHN: Certainly. If we want to configure our Knocki to tap three times and turn on our Philips Hue light bulbs -- I keep using Philips Hue just because that's what I've been actively working on lately. We currently rewrite the integrations in our own backend so the user pulls up the mobile app and says, "Knocki on my desk, every time I tap three times, listen to this Philips Hue," and then we have an integration where in the mobile app, they can essentially set a lot of the parameters that a Philips Hue light would use based on API that Philips Hue would provide us. That's the way most integrations are going to happen with third party products. They expose an API and we can write a little module and the user can configure that API. CHARLES: I see and as far as making affordances for third party people, if they want to change the behavior or add like intelligence, obviously they can configure it from the app but if I want to say add behaviors or something like that. JOHN: When you say add behaviors, you mean add new -- CHARLES: I mean like, rather than turning the lights on and off, say I want to strobe the lights or flash the lights, maybe I'm someone who's running a theater or something and during intermission, I want to knock three times to flash the overhead lights. I don't know if that's something that your integration with Hue could do but if I want to be able to add that. JOHN: Okay, I see your question. We try to enable as much of the products functionality as possible through our own integration on our mobile app but say, you're a hacker and you've come up with your own smart light that turns on any sort of party mode and flashes different colors whenever you want and your Philips Hue or any other smart light just can't quite do what you wanted to do. In the future, our goal is to have an open API that people can access and they can hopefully control their own homemade IoT devices. CHARLES: Now, what about for existing ones. You can definitely flash the lights with the Philip Hue but you're going have to have some custom software to do it, right? Do you see what I mean? You have to send a series of messages to it in sequence. JOHN: In that scenario, we currently don't support that and don't have a plan to support that. In our research, that's a really small use case of people that would be interested in that. Also, it's difficult now if we wanted to do some sort repeated command, you knock three times and then every 30 seconds, it's going to send a command in your light bulbs. We have to be careful about having processes that run away and you have a bunch of CPU power forever in the cloud. We may include features like that in the future. I think the most likely path for that sort of stuff is we'll have an open API that people can direct Knocki's inputs to their own server and then their own server can flash their Philips Hue lights as much as they want. ELRICK: Is there any standardization between the communication and what these API supposed to look, like the communication between devices? anyone can have an API, expecting one thing and someone that's writing software to communicate with that, wouldn't have to go look it up. Do you know of any standardization? JOHN: Yes. I know there have been a couple of companies out there trying to put a standard on the market and I think a standard would be a great idea. ELRICK: Yeah, I think so too. JOHN: It would be wonderful if we could just write generic control structures or information flow structures and anybody can hook their stuff up to it. As far as I know, I haven't seen any that really fit the bill. CHARLES: It feel like there's something that programming systems like software developers have been chasing for a long time is to have some distributed set of peers that they can look each other up. You can discover the capabilities of a thing without ever having to even know about in the first place. But I haven't really known that worked really well and hit that sweet spot. I'm thinking of DCOM and Java. There's like Java distributed beans or something like that. You have this idea of these objects in the cloud, which seems kind of analogous to what we're talking about now, except we're talking about actual devices, rather the software devices but who knows. Maybe it'll pan out where we'll have some standard for discovery and integration. JOHN: It's interesting that there hasn't been one already. You look at IoT and it's really ripe for standardization because a lot of the communication between devices takes the same format. You're generally just passing a small message saying, on or off or, "I read this temperature at 75 degrees. Who knows, maybe someone will solve it. CHARLES: Yeah, maybe so. Maybe the folks at Kasita. They're active integrators. They were on the podcast two episodes ago and one of their challenges was getting all these 30 things to talk together well. Maybe we can follow up with them and if they could have a standard, what they would like it to look like? JOHN: If they get on that, I would love to hear what they were working on. CHARLES: I think, maybe they mostly, have a wish list. It is like, "I wish it did this. I wish it did this. I wish it did this." ELRICK: Maybe we need to have like a 10-way podcast. It's like IoT companies and we can hash it out like the TC39 of IoT on the Frontside Podcast. CHARLES: Right, and then everybody punches each other. All right, well thank you so much, John for coming and talking with us. It's always fascinating. You can find Knocki at @Knocki on Twitter and Knocki.com. It's a great product and like I said, always a fascinating conversation so thank you so much for coming on the show. JOHN: Yeah, thank you very much for having me. It was a great conversation. CHARLES: With that, we'll say goodbye. Thank you, Elrick. ELRICK: Thank you. CHARLES: We are, as always, the Frontside at @TheFrontside on Twitter, Frontside.io on the web or just drop us a line over email, Contact@Frontside.io. Thanks everybody.
Mike North: @michaellnorth | mike.works Show Notes: 00:51 - Transitioning from CTO to Independent Trainer 03:37 - Customizing Content and Developing Curriculum 06:37 - Bringing a Developer Into the JavaScript Ecosystem 12:47 - Training Developers with Non-Traditional Backgrounds 16:56 - Keeping Up with “Fifth Gear” 19:27 - Developing Frontend Masters Courses 22:40 - “Progressive Web Apps” 34:37 - Web Security Resources: LinkedIn's REACH Program IndexedDB Transcript: CHARLES: Hello, everybody and welcome to The Frontside Podcast, Episode 79. My name is Charles Lowell, a developer at the Frontside and your podcast host-in-training. With me today is Elrick, also at the Frontside. Hello, Elrick. ELRICK: Hey, what's going on? CHARLES: Today, we are going to be talking with Mike North, who is doing all kinds of interesting stuff as per the usual so we'll jump right in. Hey, Mike. MIKE: How is it going? I'm glad to be here. CHARLES: Last time that I saw you, I think it was about a year ago at the Wicked Good Ember Conf and we were standing on the beach, drinking scotch and talking about Fastboot but you were doing something completely and totally different then than you are now so I was wondering, we were talking the conversation before we started rolling, that your role nowadays is independent consultant and personal dev trainer. I was wondering if you talk a little bit about that move from the CTO role that you're playing at your old company to kind of moving into that independent trainer, like why and how. MIKE: Yeah, I do remember talking about Fastboot at Wicked Good Ember. It feels like things have moved quite a bit since then. I have always loved teaching developers. When I've been a team lead, it's the favorite part of my job just because I get profound satisfaction out of helping people get over these hurdles that most of the time took me a much longer time with blog posts and podcasts and incomplete examples and libraries that were out of date and Stack Overflow with half answers. I've decided to dedicate myself to trying to make it easier for people in an increasingly complex web development world to wrap their head around everything. While I was a tech lead or a CTO, I always had to split my focus between helping developers grow and something else. Oftentimes, that something else was where the deadlines were and the time pressure was. It felt a little bit like I was driving a car that only had first and fifth gear where you're like on the bleeding edge of open source and what was the latest commit to master and [inaudible]. Then like, "Oh, let's be extremely patient with this person. They've never seen promises before because they came from another programming language. Let's help them digest this at their own pace." It's this slow and patient process of building up from the fundamentals and then the bleeding edge is like, "Let's use Babel Stage 0." It was very hard for those two aspects to exist at the same time in myself so I decided I'm just going for the training side. That's really all I do these days. CHARLES: It was so, but now would you qualify that as the first gear or the fifth gear? MIKE: That's the first gear. It gets you off the ground. It takes you from stop and gets you moving and then you have to develop your own expertise beyond that. But I like to think I'm developing a really, really excellent first gear. Today for example, I'm converting a bunch of Python developers at LinkedIn who are basically the ops team. I'm teaching them Ember and JavaScript at the same time through a series of about 20 exercises over three days. That process is many weeks long without assistance so this is like, "Let's get rolling much more efficiently and quickly," than via DIY approach. CHARLES: Now, do you find you have to custom-tailor for the environment or the developers moving from like someone coming from, say C# would have a different experience than someone coming from Python? MIKE: Absolutely. When I have my material, I have sections that I can drop. If you are a C# developer, I do not have to explain conceptually what 'async' and 'await' mean. You've been working with that for a while. I probably throw up a little example in C# and then the equivalent in JavaScript to sort of create a bridge from your existing expertise into the JavaScript world. Another one -- this is very true -- is teaching Ruby developers how to use Elixir. You don't have to say, "This is a router. We have controllers. There are actions and controllers." There are so many parallels that really it's more useful to help, rather than teach things from scratch to create connections back to the expertise they already have so they're not starting from zero and they can say like, "In the Ruby world, I would think of doing XYZ." Now, I have a map in between that and this new thing. CHARLES: Obviously, there's a lot, a lot, a lot of languages and environments that you could transition to, probably more than matches your own personal experience, in doing that frontline development. What kind of research do you have to do to develop a curriculum for, say someone coming from Clojure or someone coming from Scala or something like that? Maybe that never happens. MIKE: I have a pretty, pretty broad background. My entry into programming was a subset of C and then I graduated to C++ and Java and Ruby and I used to do ASP stuff. I've written iOS apps. I feel like I have enough of a foothold into various areas like I know one JVM language. That is usually enough. If you're running a lot of Clojure, I can at least speak Java to you because odds are, you're working with that and you're seeing that and you know it. Oftentimes, I have what I need. There are situations where I can borrow something in a very cursory level. Not to rip on Scala but I have not found it valuable to make connections to that particular language for clarity and [inaudible] but I have used Haskell before and I'm not a Haskell developer but it is a pure functional language. When trying to help people understand how is this different, then the JavaScript got them running where the Ruby ends up running. It's useful to use something like that. It's a very small language, very simple and you can wrap your head around the basics. ELRICK: What are some of the particular challenges that you face when bringing in a developer outside of the JavaScript ecosystem into JavaScript since JavaScript is kind of the Wild West that you can do everything in JavaScript? What are some of the challenges you face in bringing in a new developer from Python or C or whatever that may be? MIKE: You put it very well. It is definitely the Wild West. You can do anything if you have enough [inaudible] yourself and enough power to get serious stuff done. Really, it's like the explosion in number of choices and tools, the explosion of complexity. I learned JavaScript when it was something that you sprinkle on top of your Rails app for a little interactivity, a little animation on a screen or something like that. I was lucky to learn it at that point in time when that was the norm because I've been able to gradually accumulate for more than ten years now. The tooling like using Grunt, using Golf, using Brunch and then stepping up to other more sophisticated build tools. I learned those one by one in the context of real projects. Now, it's like the mountain is so high, people don't know where to start so that's a big challenge for developers. To throw them into a meaningful project like if you asked a mean JavaScript developer, not angry but the average JavaScript developer, they're like maybe -- CHARLES: I should dare to say that the average JavaScript developer is mean. MIKE: A little bit and probably maybe [inaudible] with me as well, depending on [inaudible]. But they're going to spin up some project with webpack and Babel and all of these tools. If that's your first exposure to the language and to working with the language, you're operating in an environment that you don't understand. Research shows that is the less effective option there to slowly building things up over time. I spend a lot of time going back to the basics and making sure we're not working with promises until we've explicitly focused on them, chained a couple together, managed errors and then now, we can work with Fetch. We're not going to jump into that and throw ourselves into this deep end of the pool. We want to incrementally build up skills. It takes a little bit longer but when you have that understanding as you're learning, you get a lot more out of it because anything that you can't get a grip on to as you learn it, it sort of just evaporates into thin air and don't retain that, even if you kind of fill in those holes later. CHARLES: Yeah, it could be so hard too. Actually, this has been an experience that I've been having, I would say almost for the past two years, as the tools advance, not only you are starting from a place of not understanding but the tools themselves do not teach you. I've had two moments where I got really mad. One actually was on an Ember project and one was a project using webpack but it was the same fundamental problem where in one I was actually working with someone who was very new to JavaScript and an error happens and the stack trace was some just big bundled garbage that gave no insight at all. MIKE: In vendor.js. CHARLES: Yes, in vendor.js or in bundle JS. It was like, "How is anyone supposed to learn?" The most fundamental thing about working with Ruby or working with Node or working with anything is you get a stack trace. MIKE: Debugging is really hard. I think it just takes a little time reaching out to people who are experiencing the Stockholm Syndrome like most of the time, JavaScript developer. We all are working with Ember CLI and webpack. I'm not ripping on these tools but we're used to that complexity in our lives. When we see that stack trace, we're like, "Oh, well. I probably need a source map. I'll make sure that that's there. It's natural that I'm debugging a file that the browser is not really seeing like it mapped back to my source code debugging." This is natural to us. But if you put that in front of a developer who hasn't been living under those circumstances, the number of times they raised their hand is like, "What the hell is this?" It is just amazing and it really helps. I've reset my expectations to what a normal programming experience should be and JavaScript does not provide that today. That is really challenging to keep someone in the midst of all that. CHARLES: I feel like it's hard and do you think we'll ever achieve that? Or is it just going to be a constant hamster wheel of progress versus the tooling to educate what progress has been made or to communicate what progress has been made? MIKE: I think the tooling is fine but it's just that we have a gap in terms of learning experience. We just need really -- I'm not voluntary here because I've got a ridiculous backlog -- a couple long tail horses working with vanilla JavaScript, rendering some stuff on the screen, maybe a course of React but no JSX yet, just create component. A couple of things to fill in a gap between where maybe code school leaves off and where you are expected to be by the time you start interviewing for a spot as frontend developer on a team but there's a huge chasm right now. There's the intro guides and then there's professional life and trying to bridge the gap between those is ridiculously a challenge right now due to the huge ramp up of complexity from like, "Let's do some stuff in the console," to, "Running transpile JSX code with async [inaudible]. We've got regenerator in there to polyfill generator functions." There's so much in your average JavaScript at these days. CHARLES: Your work that you're doing at LinkedIn, part of it is trying to bring and train developers who come from more nontraditional backgrounds, including a lot of things like boot camps. What is your experience of their experience coming in? Are boot camps doing the right thing? Are they teaching the right things? Are they trying to kind of parachute them on top of that mountain? Or do you find that they're just at the base camp, so to speak? Because it sounds like your approach is like you've got to really start from fundamentals so that you can understand the layers of complexity if you're going to, someday stand on them. MIKE: I think a lot of the boot camps are doing an excellent job. These days, the employees we have at LinkedIn who come from boot camps, I would bet on them against your average MIT grad every time, just because their education is so practical. It's amazing that in the world of computer science, the stuff that you're taught in school is a little bit farther removed than one would expect, compared to the stuff that we do every day in our jobs -- building real apps. I do not need to know in my day-to-day work at LinkedIn how an operating system works or how to build a device driver. This is a little bit too fundamental. It's the wrong abstraction for practical everyday work for most people. Where in these boot camps, they focus completely on the practical. In fact, I've been fortunate enough to get involved with the REACH program here at LinkedIn, where we hire explicitly people from nontraditional backgrounds like boot camps. They're not all from boot camps but many of them are. We just hired 30 of them in March. The pilot program, I think we've hired two or three in our New York office and it just went really well. It started like, "Let's double down and double down again and double that again." This time, we're doing 30 and I expect there will be a new round next year where we poll even more. The idea is we take these REACH candidates and pair them with a mentor engineer for six months. At the end of that six months, we had to make a decision as to like this person at the level we expect of an entry level software engineering hire. From what I've seen, we're doing really well at preparing these folks and they're unbelievably valuable to the teams that they've been placed in. ELRICK: That's amazing. That's very interesting. Is there a standardized curriculum thing that each mentor will follow to get this person after they entered his REACH program and then ramp them up or is it like each person just goes and looks at what the person knows and then ramps them up accordingly. MIKE: I'd say, it's a mix of both. We have a set of technical trainings for them or we'll have a testing expert from within the company and teach a little testing seminar to them. There's that standardized curriculum there. But the nature of being taught by boot camp or teaching yourself is that you're going to have holes in your knowledge and it's not often predictable where those holes will be. That's why we make sure we do this mentorship very explicitly and over a long period of time so that if it turns out that you never learned about how to work with tree-data structures. That was not part of the go-no/go decision that brought you on but we should probably, at least get you there. At least to the point where if you're traversing a down tree and you're like parent and child, what is this, what do you mean by leaf-level node. This is stuff that is actually meaningful for web developer in some cases. CHARLES: In the context of the work that you're doing with the REACH program but also touching on something that we talked about at the beginning about the first gear and the fifth gear, part of generating a curriculum is still being in contact with what's up in the fifth gear right because ultimately, what you're trying to do is you're working with people who are in first gear or looking to get a smooth transition in the first gear but at the same time, you want to set them up and you want to be in contact for what's in fifth gear now is going to be first gear in five years. How do you feed that in? MIKE: I'm fortunate to have a great team that I work with here. This group that I roll up to in LinkedIn, they're experts and you probably know of like Chris Epstein and Tom Dale and Steph Petter. A 15-minute coffee break with one of these people is enough to keep [inaudible]. Sometimes, it's a little bit like drinking from a fire hose because it's like I spend an hour with a student trying to help them understand like, "This is why a Promise is useful. Here is the callback equivalent," and then now, "Let's dive in to Glimmer. Why this track annotation is the right way to go for automatic updating." It sends me for little bit of a loop sometimes but it is definitely keeping me up to date. The other factor, of course is when you've been doing this for a while. History sort of repeats itself so a lot of the patterns that we're seeing today, I've seen somewhere else. I was working with code splitting when I was writing Dojo JavaScript code years and years ago. I was defining my module layers in a very explicit way. I had to do that. I didn't have done a webpack that would figure out, put these splits are. But I have that experience to look back to and for that reason, it is not often that an entirely new concept comes along. Oftentimes, they're like amazing refinements on things that how to smell like stuff that we've used before in the software engineering world. CHARLES: Yeah or here's something that has never been used, is very prevalent in these other context which we're going to apply here. MIKE: Exactly. CHARLES: And like, "Oh, my goodness. It's a perfect solution." In addition to the work that you're doing with LinkedIn and developing those training curricula and stuff, you're also doing some work for Frontend Masters in an area that's very exciting, I think to me. I'm sure it's exciting to you because you decided to throw a whole lot of time into developing a course for it. That's in the development of progressive web apps, which for me has been like this thing that I'm so curious about but I'm like a kitten playing with a little yarn ball. I want to dive in but I'm just going to tap it with my paws right now. MIKE: Yeah, it's a really interesting area and I think that even if you're not using progressive web technologies today, it's one of these things that sort of reinvigorates your energy for JavaScript's future and what may be possible soon. Steve and I have put together this amazing progressive web app course, which has I think like 18 short examples of iteratively building up a grocery shopping app. If you've used InstaCard or something like that, we start out with app already built and it's like a single-page app as doing everything that you would expect. After a few of the exercises, it works offline. After a few more, you can add stuff to the card and background sync, push it to the API when you come back online. We get deep, deep, deep into service workers. That's one of the areas that my work at LinkedIn and my teaching with Frontend Masters overlaps really well because I've been heavily involved in creating our service worker for LinkedIn.com. I may be able to take some of what we've learned here and disseminate it a little bit so that, hopefully fewer people have to learn the hard way. It's best to keep things simple at first and add on functionality. I'm about to cross like the [inaudible]. This is my favorite just because the example turned out to fit so well and in particular, on Frontend Masters, I think Steve and I have had contrasting teaching styles but they complement each other so well because I'm like the 'melt people's brains' instructor. I love to throw people exercises that are like 120% of what they can do and it's going to hurt, just like when you're lifting weights at the gym, like you're going to beg for mercy but we're going to make you strong. Then Steve, just listening to him, even with I am in the classroom and he is teaching me Electron. He's so energizing and he's really funny too but not in an overtly cracking jokes kind of way. He's just so fun when he teaches. I think it is a really good combination just because things lined up just by luck and through hard work and just the right way out of a couple of important areas. CHARLES: Now, just for people who might not be familiar with the term progressive web apps, what does it encompass? Do people actually call them PWAs? MIKE: No. I'm going to start, though. I like that. That carries very well over a video chat or something. Nobody knows how to spell that: P-U-A? P-W-U-A? It is a rejection of the old idea that in order to take advantage of some web technology, it has to be supported in all of the browsers that we need to support. The idea here is to hold as a core tenet of our design practices, the idea of progressive enhancement, meaning we serve up a basic experience and where we can take on these superhero features, like the ability to work offline, the ability to receive push notifications, we go ahead and do so. If your browser doesn't support this, that's unfortunate. No big deal. You still get a good experience. But if you're using a very recent version of Chrome or Safari or you have a new Android device, these browsers can take advantage of sophisticated metadata or sped up a background process that can serve up data to your app and your app doesn't even know that there's something between it and the API. That is the idea of progressive web apps -- apps that become superheroes where possible and they still work and provide a great basic experience for antiquated browsers like IE8 and Safari. CHARLES: The idea theoretically, you could work without any JavaScript or whatsoever. What's the ground floor there? MIKE: That is ideal. I think server-side rendering, which is what you're talking about there, even if JavaScript is not working, just HTML and CSS will provide a basic experience. That's great but that's not a modern browser technology thing. If you have JavaScript turned off in today's Chrome, like Chrome 60, versus IE9, both of them working with them without JavaScript. What we're really talking about here is app-like characteristics, where we are pushing web technology to the point where you will swear that this came from an App Store. It's on your home screen. It's running in the full screen. You're getting push notifications. It works offline and you can store a large amount of structured data locally on the device. All of the stuff sounds like the list of reasons to reach for native mobile technology because the mobile web is not good enough. But in fact, it has a feature set of this family of progressive web technologies. It's really like a web app that is so good and so modern that it feels and looks just like a native mobile app. CHARLES: That sounds so hard to do right. MIKE: Well, it is now, just because what we have to work with can be thought of it like a basket of ingredients, rather than a solution that we drop in. But over time, as more people start working with these ingredients, I think we're going to see a lot of consensus around the best patterns to use and boilerplate code will fall away as we can identify that the set is in fact commonly needed and not a beautiful and unique snowflake. CHARLES: Because it seems like the thing that I always struggle with is not wanting to put the critical eggs in the basket of a superhero feature or have you being able to provide an alternative if the superhero feature doesn't exist. Some features, if you just don't have it, that's fine. You can turn it on if the capabilities available but certain features are very critical to the functioning of your application. I'm casting about for an example and I'm not finding one immediately but -- MIKE: Offline is a great one. That fits pretty neatly. If you're using an older browser or if you're using Safari, which by the way, I should stop ripping on Safari. For the listeners out there, we saw a commit lend in webkit, where service for APIs are beginning to be stubbed out. No longer do we have to look at length. Service worker, enthusiasm and Safari has got it in the five-year plan. There was motion last week. We haven't seen motion in ages so thank you Safari Team. Thank you. Keep up the good work. CHARLES: Is there a discipline of Safari-ologists who monitor the movement of Safari to bring this news? MIKE: Of course, we monitor it because right now, Chrome and Firefox, they are pretty much hopeful in terms of supporting this modern stuff. Opera supports this modern stuff. Samsung's fork of Chromes support this modern stuff. Especially when we think about the mobile web, you got to worry about Android and you got to worry about iOS Safari and right now, like we've talked about these progressive web apps, you don't get that superhero experience on an iPhone or an iPad. Once we crossed that threshold, this is going to have a breakaway level of adoption because there are no more excuses. Essentially, for a mobile web experience, you can send push notifications to the user. That is huge. That is probably at the top of the list for why some people use native apps, instead of mobile web. The more we can do that, the more we can make it so that a great LinkedIn experience can be delivered to your phone without having to install a binary. I just have to update Facebook the other day and it was over 100 megabytes. Why do we need to do that? You should be able to make it work with less. I'm sure that there's some great stuff in there. Apparently, Snapchat filters are popular but I don't need this. Can we code split that away or something because I don't want to have to download that? I can't even download it on the cell network because it's over 100 megabytes. It's really exciting to see the web start to compete with this heavy mobile experience because now I think is ready. CHARLES: Now, when you talk about push notifications, you're talking about being able to send things to my lock screen. MIKE: To your lock screen while the browser is not on the foreground, while the app is not open. Essentially, you're installing a lightweight process that runs in the background. It receives events that originate from your server and the user can tap on them and then your little lightweight worker process in the background decides what to do when that tap happens, like open up the app, take them to this URL or something like that. That is a game changer. That's huge. Or background sync like the user added some items to their cart and then they lock their phone and now, their plane has landed. That's why they were offline and they get back on the internet and without them having to touch their phone, now we can push that data to the server and everything's in sync, rather than like, "Please revisit your app. We need to run some JavaScript code to flush IndexedDB or API." It still feels like a hack at that point. This is a fluid experience. ELRICK: Wow. This is exciting for me as I don't have any more space on my cellphone, thanks to all the apps that I have to install to do various things on the web. MIKE: You're not alone. CHARLES: Yeah, it's crazy and just the amount of code sharing that you can have, I guess that doesn't happen much these days on the web where you've got these popular libraries out on CDNs so that the chances are that you've got jQuery 1.2.1 on your cache, you've got 16 versions of jQuery so most of your web applications don't have to do that. I guess we kind of do the equivalent of statically linking everything. MIKE: There is a benefit near that where we have imperative code managing our cache, instead of just relying on the HTTP cache or app cache, if you have a vendor.js file that is not changing over six months, there is no reason you should be re-downloading that every time you deploy your app or letting the browser evict that, just because memory pressure is high from Google image search results or something like that. We really don't have much control over it. But with a service worker, we can say, "Hold on to this," or maybe like prefetch the next version of the app so that we're going to show you the old version now but the next time you refresh, here's the new version available instantly. It's downloaded in the background and it's like click to update your version, like it's already here waiting for you. That's huge. That's amazing. CHARLES: That is amazing. Although the complexity skeptic in me is thinking, "Oh, my goodness. Now, we've got all this state that we're storing on the server. We have to have data migrations." We need some sort of migration mechanism for our clients-side state and perhaps some transaction and rollback in case you're not able to successfully migrate your data. It sounds like a lot of fun but I'm just imagining we really are getting started here. Has there been any work on that aspect? MIKE: If you've ever worked with IndexedDB, it does have a concept of migrations. Basically, the data you store on a device has a version and when you read in what's called a file but it's a database, when you read that in, the first thing you do is you basically bring it up to date incrementally. You'll bring it in, you're looking for version nine like your code wants version nine. What you see is version two because your user hasn't been at your site for six months and you're going to take it from two to three to four to five to six. Each of those, essentially constitutes a migration. We just have to apply the same principles of forward-compatible changes. The escape hatch here is remember it's progressive enhancement so if we had to destroy everything, fall back to a basic experience and start from scratch, like discard all of our data, it's really being held there as an optimization. Some people use this immutable caching strategy or basically, like rolling out a new service worker version constitutes for the most part. Any data that wasn't created by a user you're going to discard that and you're going to fetch it new. You don't have to worry about like, "Crap. This six month-old thing is still plaguing half our users and we can't get rid of it," like you can have [inaudible]. But you should really check out this course. It is simpler than you think and what we demonstrate is not a trivial like hello service worker. It is taking in a classic single page app, making it completely offline, having it exist on the home screen and I think the service worker ends up being no more than 100 lines of code. It's not too bad. ELRICK: I'm definitely going to check that out because my progressive web app journey is still on just service workers. MIKE: That's very [inaudible], though. ELRICK: Yeah. I'm definitely checking it out. Sounds like a really fantastic course. MIKE: I've been focusing a lot on this area and another one is security. The reason I picked these two is because developers are not really going to learn about these on the critical path to [inaudible] plus they learn about them the wrong way. As the JavaScript world is becoming radically more complex with each passing year, I've tried to target some of my efforts towards areas where they are not getting as much attention as I'd like to see, just because we have to focus somewhere. Obviously, getting the app out and figuring out how to make the build tools work for us. Without that, we can't do anything at all. One of the courses that's coming in September for Frontend Masters is a one-day web security workshop or we'll do with like cross-site scripting, how to work with certificates because if you start playing with HTTP/2 -- the next generation of HTTP -- you will need to generate some certificates for development at least today you need to. I've seen some amazingly smart developers get this dangerously wrong to the point where they compromise their own machine and anything that's on that machine, just by trying to set up dev environment. Typically, I'm an optimist but when it comes to this PWA stuff and security, I am paranoid. I feel like, we as a community need to get together and have the discipline to brush up in these areas so that as we introduce all of this new stuff, we don't end up opening a bunch of holes. Nowhere near the same rigor as put into frontend compared to backend and now, the line is blurred. Right now, we're server-side rendering so our code is running on the backend somewhere so injecting something can really mess things up in a bigger way. ELRICK: Yeah, I think that's a fundamental characteristic of someone does going to be involved in security paranoia. You have to be paranoid about everything. MIKE: Yep. I don't trust anything. CHARLES: It's important to make those things easy because I'm definitely fall more into the hippie camp like, "Everything is going to be fine. Let's trust everybody," which is I know is totally unrealistic. But then you get into these secure technologies and you learned enough of it just to get the task that you're going to do and then you forget. SSL is a great example. Over the course of my career, I've learned how SSL certs have worked probably, at least 10 times. ELRICK: Right, [inaudible] you had to set it up in production. CHARLES: Yes, exactly and then I promptly forget about it, never worry about it again and then the next time I'm like, "How did that work? What's this trust chain? What?" ELRICK: Exactly. I read a study from Carnegie Mellon a couple of years ago that showed developers observe security best practices dramatically less than the general public and the general public is not good. Do you know what I'm talking about when I say a certificate warning and a browser, there's big scary red screen saying like something is wrong here? Before the Chrome team put some effort into improving that, 70% of people would click through those and proceed anyway. After their improvements, over a third of people still clicked through and that number when you just look at Canary versions of browsers, that number is actually considerably higher close to 50% of our developers. We're trained by every broken certificate system that exists on the internet like the legitimate ones or maybe some things just expired. They're training people to just click straight through these things and as a result, it is terrifyingly easy to mess with people. We have to remember as developers, our machines, those have the private deploy keys and those have the SSH keys to commit code to GitHub, we have to treat that like it's a private data. It's really, really important that we make it easy and that we make sure that that easy path is also very safe. CHARLES: Absolutely. All right. Well, thank you so much Mike for coming by and talking with us. We touched on a lot of subjects but I feel like I certainly learned a lot. MIKE: Yeah, thanks. It's been so much fun talking with you this morning. CHARLES: Anybody who wants to go and check out those courses, they're on Frontend Masters. Now correct me if I'm wrong, you've obviously got the one on progressive web apps or PWAs. If it doesn't work offline, it's faux-PWA. MIKE: Yes, I like that. That's going to become a t-shirt sometime soon. CHARLES: The fundamentals of progressive web app development, which is now released if I understand correctly. MIKE: Members have access to everything, you can watch the raw video now. The edited course will be available later this year. CHARLES: Okay, and that's with Steve Kenny. I am very much looking forward to looking at that and learning more about it. Then you've also got ones coming up in September on TypeScript web security in Visual Studio Code. MIKE: Yep and members can watch that as a live-streamed event. Frontend Masters even ask people to watch the comment stream so you'll have a proxy question asker or hand raiser in the room. It's really a great experience to be part of a live thing. CHARLES: Oh, man. That sounds awesome. Then if you are obviously doing your independent consulting and if people want to get in contact, how would they do that? MIKE: You can find me on Twitter, @MichaelLNorth or you can visit my website, Mike.Works and I have all of the courses I teach and outlines and I can just open up a little chat bubble on the lower right, ask me any questions that you have. I am really passionate about teaching people. If you like that's useful for your team, please reach out and I'd love to talk. CHARLES: Fantastic. Thanks, Mike and thanks everybody for listening to us. If you want to get in touch with us, you can always do that. We're on Twitter at @TheFrontside and email, Contact@Frontside.io. Thanks, Mike. Thanks, Elrick and I will see you all later. MIKE: Thank you so much. ELRICK: Bye.
Jason Jaynes: @jasoncjaynes Jeff Wilson: @ProfDumpster Show Notes: 00:53 - “Professor Dumpster” and Founding Kasita 05:33 - The Startup Industry 07:45 - Building the Kasita Team and Creating the Design 12:25 - Integrating Devices 16:33 - Challenges of Building These Ecosystems 24:36 - Controlling the Ecosystem: Will there be third-party developers and applications? 30:16 - Device Cohesion and User Experience 33:23 - Privacy Resources: Data for the People: How to Make Our Post-Privacy Economy Work for You by Andreas Weigend Kasita is hiring! Transcript: CHARLES: Hello, everybody and welcome to The Frontside Podcast, Episode 78. My name is Charles Lowell, a developer here at The Frontside and your podcast host-in-training. With me today are Jeff and Jason from Kasita. Now, Kasita is one of the most exciting products that I think we've gotten to work on here at Frontside in the last five years. We're going to be just talking about it because, I think it touches on a lot of the aspects of what makes software development and startups and just the emerging economy exciting. I'm really thankful that we get to have you all on the podcast. Welcome Jeff and welcome Jason. JEFF: Thanks for having us. JASON: Excited to be here. Thanks, Charles. CHARLES: Now Jeff, you are the founder of Kasita, the CEO and I believe your official title over there is 'Professor Dumpster.' Maybe you could actually unpack for us a little bit of what does that title mean? How did Kasita come about and what is it today? JEFF: A couple of years ago, I did a radical, social experiment around housing. I went and sold everything I own for a dollar an item out of a 3000-square foot house and moved into a 33-square foot used trash dumpster for a year. The idea of that project was to live in 1% the size of an average American home and try to use 1% the energy and water of the average American home. The project took a little bit of a twist, you might say and about part way through it when the dumpster started getting tricked out, I started thinking about the whole nature of housing and how we need to do something different and how that grand future probably would not be a gated community of dumpsters. CHARLES: Now, I assume you cleaned out the dumpster before you actually went to live in it. JEFF: Yeah, it was a fixer-upper. We give it a bit of a scrub and did some testing to make sure there wasn't anything nasty left in there. That went for about a year and a couple of months after that, I actually first set down with Jason because he was the only person that I knew in the entire startup scene, in the entire world. He said, "Wilson, you had some crazy ass ideas like this dumpster thing you told me about. This one might actually work, this Kasita thing." Here we are today, we're working together. CHARLES: Wow. This was something you just did on a lark. You didn't have the idea of starting this business but it was actually through the process of actually living in this dumpster for a year that the idea emerged or was there a master plan going in? JEFF: I don't know, Jason do you remember any kind of master plan when I first told you about the dumpster? JASON: No. When we first met to talk about the dumpster, it was an early morning, I believe in 2010 or 2011 and you're incubating the idea. At that point in time, there was nothing on your mind or you aren't looking towards the future of housing at all. You were just trying to figure out how you were going to move into a dumpster and people thought you would be crazy. Of course, I've validate it and I thought people would think you would be crazy. CHARLES: That is a pretty radical idea, the future of housing being 1% of what it is now. How do you see that playing out? How is that possible? How do you shift people's mindset away from that? JEFF: One of the bigger things we're trying to do with Kasita, there needs to be a massive shift in the wider way that we live in our homes. As everything else is moving towards on demand and as a service and as everything's being sort of productized, those are some of the core ideas behind Kasita. We think about Kasita a lot more like an iPhone or a Tesla than we would think about it as a single family home or an apartment block or even a micro-unit. That's why Jason and I are standing together here today is I represent a lot of ways, a kind of vision and origin story of Kasita but in a lot of ways, Jason represents the future of the software and integrated IoT that's going into these things. CHARLES: There is definitely a lot going into these things. I remember when Jason first started telling me about it because it is like an iPhone or a Tesla but, I think especially the Tesla is a great analogy because you have not just like a normal software or even really a hardware project, you've got architectural concerns. You've got manufacturing concerns. You've got, I assumed geopolitical concerns in terms of the politics around zoning and housing and real estate, all rolled up into a big startup. When I think startup, I think let's get a web application up and running and we're providing some service. This is cross-cutting at least five industries, it feels like if not more. I'm curious, what's been the experience in terms of wrangling that aspect because I think it is very unique in a startup today but it got me wondering is this going to be the normal in five years? JEFF: We've seen a movement recently in the venture community. Even a few years ago when we first started raising money was highly-regulated industries are hard, hardware is hard, "Thank you very much. We're going to go looking for our next two Stanford computer science dropouts to shove into a wee work and not have to deal with all of this kind of stuff." I think I've seen a shift to where people from the individual level up to the folks funding these things, see the massive opportunity in highly-regulated complex problems like housing and you're right. Jason and I are looking out over our shop floor here where we've got guys out there that are plumbers or traditional electricians all the way upstairs here to folks that have been mayor pro tem of large cities with PhDs. Bridging all of those individuals into a startup culture and then looking at the complexity of the landscape from a regulatory standpoint, autonomous cars are a breeze relative to the kind of complexity we're dealing with. CHARLES: Did you know this complexity walking in or was it a classic overoptimism? JEFF: No, it wasn't classic overoptimism. I'm always asked, "Are you a designer? Are you an architect? Are you a real estate developer? Are you a technology guy?" and I think if I would have been any of those besides a guy living in a dumpster, I wouldn't ever been crazy enough to try this. It's one of our core precepts as well. Jason had never worked with IoT stuff before. Our head of manufacturing used to build LEDs for Philips. Our quality guy inspected Cadillacs. Our manufacturing engineer built Boeing jets. The ideas that we're not pulling a lot of people from these traditional industries, we're pulling smart people that are passionate about our mission and to solve this, what is really a Rubik's Cube of a problem. JASON: Yeah, I think the other thing to add to that that Jeff is not getting himself enough credit is that from very early on, Jeff always looked at Kasita as a product that was going to incorporate multiple disciplines. He was very careful in how he orchestrate it and built the team to make sure that he was bringing the right expertise and the right areas together and then forcing those different disciplines to figure out how to meld and work together to build the Kasita. But the Kasita was from the beginning just about building a micro-urban home. It was about building a product of which part of that was a home, where people live obviously, but there's a whole lot more to it that we're working towards. I think even go back and Jeff, it might be relevant for you to talk a little bit about the approach that you took to just create an initial design for Kasita, which I think is revolutionary in itself. JEFF: A big part of our DNA was product from conception. When I was living in the dumpster, I recruited a couple of the top architects in the country really to help me turn that dumpster into a home. The way you're trained in architecture school, I think a lot of folks come in there with Buckminster Fuller kind of dreams and you're told pretty quick that you better bring things up to code and you better make things that sell or you're not going to eat when you get out of here. The idea was that we would start off with a product designer and not design a home. The kind of struggles in the dumpster taught me that we needed to go at a different approach so I went and recruited an industrial designer. One of the requirements for that person that he or she had never designed a home. This person had lived under a staircase and never designed a home so I said, "You're perfect." CHARLES: I like that and I'm curious, Jason from your perspective, what was it like to have gone through this? It sounds like what you're doing is asking people to bring their expertise but not their set of expectations like the industrial designer. What was it like for you coming primarily from the software development world to step into this pan-technological realm and what was that experience like and what were the things that stretched you and you found surprising? JASON: I think early on, I realized that it was going to be a bit more challenging maybe than I thought. Really, what it required was me to think outside of my discipline. Obviously, not only from the perspective of what we were doing on the IoT frontend, how we were melding software and hardware together but then going all the way over to the physical building structure and thinking about on a weekly, daily, hourly basis on how we are interacting with the other disciplines. An early example was, and this is one that I remember that's quite funny is one area that we wanted to make sure that we had covered in our research and understanding from IoT perspective was smart locks and how we were going to provide a smart locks for the data. We went out and did a lot of investigation, brought a number of leading smart lock solutions into the lab and tested them and narrow our list down. Then I recall vividly walking over to the architects to excitedly tell them we had selected our smart lock that we were going to use. They very quickly inform me that that lock wouldn't work because we needed a mortise lock and not a standard door lock. I realized that you can't work in a vacuum and just solve your problems. You have to be working together to make sure the solutions and the products you're selecting at work in accord with the overall design. That's continued to manifest itself. Every day, I'm down on the manufacturing floor, working directly with the electricians and others to make sure that our equipment is placed properly, where are we going to place our equipment, how are we routing around plumbing and pipes and other things that exist there and how are we locating things properly. It's an ongoing experience, which has definitely taken me out of my traditional software role but it's done so in a very exciting way and I've enjoyed it. It's just realizing that you have to actively be communicating across the organization with all groups and really, you can't take anything for granted. CHARLES: The number of different disciplines and technologies is really staggering, even if you limit it to just considering the set of devices that you're integrating. I was actually hoping we could talk a little bit about that. Now inside each Kasita, at least the ones that you're building right now, how many different devices do you have? How do you take all these different devices and turn them into a product or integrate them into something that itself is one product? JASON: If you were just to look at the technology bill of materials, what the products are that we're incorporating into our current Kasita design, there is around 50 different products and product parts that we're bringing together to build out the technology solution. If you narrow that down to what the end user is actually seeing and looking at, there are about seven noticeable products that the end user would see or they would recognize everything from a Sonos connecting amplifier to an Amazon Dot to a Nest Thermostat. Obviously, getting to that list of bill of materials and deciding on that 'subassembly of technology pieces,' took us quite some time in a number of iterations and a lot of outside engagement and talking to experts and trying to decide what were the best devices to bring in. But the other side of the equation was something that we kind of decided very early on in the process and kind of thinking the world of first principles was that, we wanted to make sure that Kasita was the primary interface to the user. We didn't want somebody else sitting between us and the end user. We wanted to be able to work with other products but we still felt at the end of the day that the end user, when they were living inside of a Kasita, when they were controlling the Kasita, when they were changing the state of the Kasita, they needed to go through our interface. With that as an initial first principle, you can begin to imagine that all the other parts of the system architecture and the way that we design things, the way that we select products and built things, it begin to derive themselves. Everything from that, immediately we needed an app and lo and behold. We were able, fortunately to work with you guys, the Frontside, to help us get our initial app concept up and going. It went from there and I can talk more about it. CHARLES: I think I really like that as a first principle. I really just want to inject a vigorous sense of agreement because I think it's so important, especially when this is the place where you're living. You want to imbue that inhabitant with a sense of ownership and control. I don't know if you would be able to do that if there were a bunch of different touch points and it didn't feel integrated under one product. In other words, this is my home, this is my Kasita. Is that the idea behind making sure that there was really only one interface? JEFF: We prefer to say 'Mi Kasita.' CHARLES: I love it. JASON: Absolutely, that's the idea. I think from a consumer perspective, if you've ever personally gone out and ventured through the halls of Home Depot or Best Buy and purchased some smart products off the shelf and brought them into your house and try to get them up and running, you very quickly learn that. It's not only challenging to get these devices connected in a way that you can control them but there's also this notion of there's an app for that. Every physical device you ended up putting in your how, has its own app for control and that becomes very overwhelming in a very short amount of time for the user. We did not want that to be the case with the Kasita. We wanted them to walk in the door from day one and immediately feel at home and feel like they have complete control of the Kasita, in much the same way when you go purchase an iPhone or you purchase a new Garmin watch or you purchase a new Android device, you're up and running with that ecosystem and you're interacting with that interface. We wanted people to be interacting with the Kasita interface to control their home because that's part of the product. CHARLES: I like that. It must present some unique challenges because I think you said it best. Every single device that you have comes with its own ecosystem and that ecosystem has its own APIs, its own web interfaces, its own applications and though there are walls around those ecosystems, what are some of the challenges you encounter in trying to punch holes through those walls so that you can hand information and control from one ecosystem to the other while providing a seamless experience to the user? JEFF: When you're talking about that, Jason one of the things that is often left out of this equation is at this specific point in space-time, it's very difficult to do that. But then to have any sort of semblance of planning for the future and future-proofing the system as developers usually call it, one of the reasons why you don't see a lot of Nest thermostats in multifamily development is because a developer knows that they're not going to ever have to replace a normal light switch. If it's a Lutron switch or if it is a Nest thermostat at some point, it's going to have to be replaced. Not only the physical replacement of the stuff but from a software side, making sure that we can continue to communicate with these devices in the future, I think is a big problem to solve. JASON: That's absolutely right. I think very early on, we recognize and realize that we were going to have to build software and a component that acted, if you will as a gateway for sitting between the end user and the end devices and facilitated the control of the end devices. Obviously, being able to accomplish that, one of the challenges is and I think, Charles you've seen this in your world because I know you've got experience with IoT is this whole proliferation of standards and protocols like if we're going to talk to the lightbulb or we're talking via Z-Wave or ZigBee, or do we have to go through a Philips Hue hub because that's the only way to actually communicate with it. Is there a separate way via Thread or Bluetooth you communicate with this device? In a very quick fashion, you get to this point where you can imagine that you've got a physical hardware controller that has four different radios in talking to four different device types. One for talking to Z-Wave, one for talking to ZigBee and it becomes overwhelming. We did a lot of research across the protocols that were available, mapping them across the devices. Early on, we were excited about the potential of Z-Wave but more recently, where we've shifted our attention quite honestly is looking for devices and device manufacturers who see the opportunity and Wi-Fi enabling their hardware devices and then providing either direct control of those devices in an IP-centric way over a local area network or even through the cloud. What that affords us back to Jeff's future-proofing concept is if you have Wi-Fi up and running and the device can get on the Wi-Fi network and there's a way to communicate with it, then it makes it a lot easier for us to sit between the user and that device and send commands and control that device. The other side of that, which I think continues to be a challenge and will be a challenged for the foreseeable future is a lot of the device manufacturers to the point that you brought up are still forcing you to go through the cloud to communicate with their devices. They don't allow for a local area network communication directly with the device and there's good reasons for doing that. But what that means is if you lose internet connectivity, you no longer have control of that device. CHARLES: Obviously, you've got probably pretty strict criteria about what it takes for a device to be integrated with Kasita. Is that a nonstarter right there? JASON: It's actually not. A nonstarter with be the device communicates via protocol that we can't interface with or the device works over a Wi-Fi network but has no API for controlling cloud or local. The third piece of that equation and fundamentally is the final nonstarter and really probably should be the first one and it's one that we take into consideration every time is that there should be a physical override for the user if internet connectivity is lost. What I mean by that is if we select a smart switch and the smart switch goes offline and there's no more connectivity, the user has still be able to walk to the wall and press the power button and the light should come on. There always has to be an ability for the user to fall back to the same old fashioned physical control in the absence of Internet connectivity or local area network connectivity. But the primary things are ability to fall back to physical control, ability to communicate over Wi-Fi or standard IP-based protocol, then the third one would be some form of API access, either remotely via the cloud or locally via the local area network. CHARLES: Wow, that's actually a great list. It's got me wondering, obviously you've encountered devices that have fallen on both sides of that divide. Do you feel like that's just a blip and we're going to be trending more towards devices that are happily and easily integrated or are we still seeing some moving and jostling as people maybe try and corner little parts of the market and make their device deliberately make it not easy so that you'll try and force people into that ecosystem? JASON: The latter, however we have two guerillas in the market right now that I think are helping drive the other direction in the way of Amazon and Google with Google Home and Amazon Echo. What they're doing is they're saying, "If we sit in the center and one of the interfaces for voice control for the user to control their home, then we're only going to work with devices that we can communicate with and that we can control through the cloud," and quite frankly, what that does is it puts the burden back on the device manufacturer. You could actually say three if you threw Apple in there. I don't want to leave Apple out with HomeKit. But my point is that the device manufacturer now has to find a way that the end device can either communicate via standard TCP/IP network-based connectivity that we all know and love from a developer community perspective or they have to insert a hub into the equation that can handle that form of communication and then communicate over its own proprietary wireless connection, which is in the case of Philips Hue, it's exactly what they do. JEFF: I would draw analogies here to some people get really tired of this, particularly the real estate people of me talking about the iPhone but that kind of leap into and integrated piece of hardware and software. There were certain things happening in 2007 that didn't make the iPhone or something like it, something that might happen but something that had to happen. This kind of cold death to the universe that we could see with all of these walled-off ecosystems, go in their directions and iterating into a space to a nobody owns anything and nothing talks, I think Kasita is a solution to that to where we're looking like combine all this stuff under one roof and build a single user experience, much like not having to pull your Palm Pilot out of one pocket, you're Rio MP3 player out of another and you're your Razor or whatever it was out of the other like integrating into a single experience, rather than a sort of convenience, which is what a lot of the IoT spaces right now in these walled-off ecosystems. CHARLES: That actually makes a lot of sense and clarifies it in my mind quite a bit. It clarifies one thing but then, immediately raises new questions. When the iPhone first came out, you had a set of basic integrations between your MP3 player and your web browsing and your calling and calendaring, so and so forth. Then, I don't know what was it like, a year and a half later, they actually came out with an SDK so that you could actually develop apps -- third-party developers could actually develop. Sell and distribute in apps -- to the iPhone. We're all really happy with the way that worked out. I guess my question is does this analogy carry forward then also for Kasita? Is there a future where you have third-party developers who are actually selling integrations or apps that would run on this integrated IoT product that is Kasita or am I stretching the analogy too far? JASON: I think the analogy is good with the exception that we're not looking to control the entire IoT ecosystem in a way that Apple maybe had look to control the mobile phone ecosystem with providing all of that in one box and the iPhone. We want to work with numerous hardware providers and even from that perspective, numerous folks that want to provide interfaces into our system. As we develop an architected Kasita technology system, we've taken an API-first approach and that's allowed us to build our user application layer right on top of that API but in the future, we see the opportunity to work with third-party developers to extend that, up on that and build their own interfaces to the end user. Then on the other side of the equation, if you think about what's actually controlling the devices, we're architecting that system in a way that a hardware manufacturer could take an SDK and add Kasita support for their product directly in and make it plug and play when it gets to the Kasita. We definitely see the opportunity, Charles to reach out and allow everybody to be part of this. We consider it quite frankly, a necessary thing. But we don't also want to pretend that we would look to control the whole ecosystem because we just don't have that level of scale, if you will. JEFF: And you know -- CHARLES: Not yet. JEFF: Yeah, and we try to keep our ego in the dumpster, so to speak as well. CHARLES: What would a third-party app even look like in the context of Kasita? Have you thought of like what are some things that you might be able to do? JEFF: If you don't want to call it directly an app, I think the first stage -- Jason and I haven't talked about this -- maybe more like an Alexa Skill to where you can have the Kasita do certain sets of tasks around a particular experience, which we're already building into the system the idea of moods but I don't know in terms of apps. JASON: Yeah, it's actually a really good idea. Even though we haven't talked about it, it always scares me a little bit when my boss is coming up with ideas on the fly that we have to implement but -- JEFF: But actually we will have our first -- we're going to call it a skill app, a Kasita skill app. We'll be releasing that say, October 1st. CHARLES: You heard it here first, folks. JASON: To take Jeff's idea a little further, I think that is an interesting concept when you think about the Kasita as being an end product and you provide interfaces whether it's the ability for people to write skills that tie into the Amazon Echo or an IFTTT-type capability. The Kasita, as a whole can be controlled -- all the lighting, the sound, all the different temperature, etcetera -- so now you're asking end users to write skills, to control the entire state of the building or of the home and not just doing it on a one-off basis writing skill to turn this light on and off or set the thermostat to this level. You basically box all of that together and make it much easier for people to get from Point A to Point B through our system. JEFF: Could you say that we're turning the entire Kasita into a board for people to play with, like treat the Kasita as your breadboard? JASON: I think there is some opportunity for that to the degree that will allow the user to have that much flexibility on the hardware side. I think it is still up for question but I think there's a lot of opportunity there, Charles and not only inside of the Kasita but then you can begin to see other applications as Kasita begin to multiply and people use them from many purposes. Let's take a sample of somebody owns 10 Kasitas and they use them as Airbnb properties and they allow users that live in Kasitas to come in for a short period of time into their Kasita and bring their Kasita profile with them. Immediately, they can make the Airbnb Kasita feel exactly like their Kasita feels when they're at home. Those are some interesting opportunities and ways that we see this technology potentially evolving. CHARLES: So it will have the same moods, the same behaviors. Any customizations or third-party extensions would also be in effect provided they were software-based? JASON: Yep. CHARLES: That would actually be quite amazing. I guess the other question I have in terms of hackability of Kasita is we're very interested in the IoT space and very interested in these products and we have some side projects here at Frontside also like I do a bunch of hobby stuff at home, where I try to integrate a bunch of these things. But one of the things that I really like about what you all are doing is that it's very much 'omakase' in the sense of there's an option of 10 smart locks, there's an option of this thermostat, there's an option of a million different devices but what we've done or what you've done is selected ones that we know are going to work well together. We've built the software, the control systems, both computer control systems and human control systems to get them to work together as a cohesive product. I would love to do is say, "I would just like to buy that product for my house," even though my lame tinkerings with smart switches, smart locks and audio controls and lighting, which are fun and gratifying the first few times but they don't really play nice together, give you that super sweet feeling. JEFF: This goes to the overall philosophy of Kasita. We want a turnkey, one-click housing solution. Not only for finding you a place to rent so that you're not fishing around on Craigslist for roommate or having to pay some outrageous fee in New York. You don't have to go mattress shopping. At some point, you should just have to show up with your iPhone and your toothbrush. When you start thinking about the technology inside, it's almost like folks don't really care what kind of Foxconn chip is in their iPhone or even if it was Foxconn that put it there, they just want it to work and they want it to be seamless and turnkey. It sets up a whole philosophy around, not only our smart kid in the Kasitas but it shouldn't even be a smart kid anymore. At some point, it should just be an experience so ultimately, what sort of UX inside of the Kasita are all of these things bringing you. I shouldn't have to really look at a blue glowing dot that lights up every time I walk by it to be at a comfortable temperature in my house. I shouldn't need a black tube over on my desktop that I yell commands at. I just talk or it should anticipate those actions. That's a future that I look forward to in Kasita to where we move away from having to tinker with devices and even knowing what those devices are to a true-like depth of experience. CHARLES: I like that a lot. Now, one thing that we haven't covered. We touched on it a little bit at the very beginning of the show when we talked about people feeling in control and feeling like they're truly the owner of the space is the issue of privacy. Obviously, there's a lot of a user's behavior that's going to be passing through software channels as their intentions move through the devices in the Kasita. Of course, all of these devices, they have their own ecosystems, their own vendors so how do you ensure that people's data is going to be protected, especially as it moves through potentially a bunch of different public clouds. JEFF: Yes, we gave a lot of thoughts to this. Actually, Jason put me on to this book called 'Data for the People' by Andreas Weigend. We took some inspiration on that, from that and set out on what we call it the four cornerstones of this future of the connected home. Those are agency, transparency, security and then the actual benefit that you get from this home. I gave a talk at South by called, 'The final frontier of AI is in your living room.' If that isn't black mirror, creepy enough to attract enough people, I don't know what is. In that talk, I won't take them out of order. First, we need to make sure that we're focused on transparency. Do people know what's actually being collected on them? I've been toting around my iPhone for 10 years. I'm pretty sure they know everywhere that I have been since then. I'm not really all that sure. Second, agency. Can I actually do something about it? Are we allowing people the ability to switch off, switch on, control where that data goes? Then third, security. Are we providing another level of security above what you would get out of the box? I'll let Jason talk about that in a minute. Then, the last is benefit. Am I getting ads? Am I getting a slightly better news feed focused on ads or am I getting my rent subsidized? Am I getting a better user experience, better sleep within the connected home? Those are the ways that we think about that in a bigger level. CHARLES: Is the idea that there's no benefit than it's exploitative? You want to make sure that there's benefit? JASON: Yeah, I think that onus is if you taking individual data and using it, then the onus is on you as a data collector to try to provide benefit back to the end user. If you can't do that, then I think the question should be why are you collecting the data in the first place? our goal is really looking at it from the perspective of if we know when users are turning lights on and off and what they're setting the temperature in their house to and when they're going to sleep at night, when they're waking up because we know when they turn everything off and turn it back on -- JEFF: Or where this things on the floor are from the vacuum robot. JASON: Yeah, exactly. If we have insight into that information, how are we taking that information and combining it in a valuable way that benefits the end user? I think that's the first question that we have to ask when we start looking at the data that we're collecting. But at the same time as Jeff said, that data collection really has to be based on this notion of agency, transparency and privacy or security. An agency is simply I have control over whether my data is collected. Transparency, from the perspective of I understand how my data is being used and where it's being sent and then of course, security, I know that my data is being securely transmitted and stored. When you think about security, we spend a lot of time thinking about not only the data at rest -- once it's been collected is it properly being stored and encrypted and protected -- but then how is that data being transmitted and are we putting the proper fail safes in place to make sure that somebody else can easily gain access and take control of my home and of the things that are important to me by finding back doors into the system and ways to breach them? Those are the cornerstones that we think about and we put first and foremost in our mind as we build out our architecture, build out our system and as we begin to take that data and to turn it back in useful and interesting ways for the end user. CHARLES: I think that's really important. I think it's a great comfort to hear that you all have a framework for thinking about this so that it's going to be integrated into every aspect of it. I think it's just so important, especially when it's something as critical as the space in which you're living. It's good to hear that it's not just an afterthought but that it's something that's been integrated from the start. Well, Jeff, Jason thank you guys so much for coming by and talking with us. I really think that Kasita is an exciting product and I think that it was an exciting project, certainly for us to get to work on, even though we were only seeing a very small sliver of it. We still got to perceive the whole enchilada that you guys were working on and see that just what a unique startup that really is, not just you're moving outside of software, integrating a bunch of different devices, integrating that with a unique home that's going to be designed, architected, manufactured and then thinking, then even rolling it up a degree further about how is this going to be integrated into the urban spaces in which we live. I hope that we see more startups that really engaged all those different disciplines. I think that with the technological changes that are happening, that's more and more a possibility. The price on software, the price on materials, the price on these smart devices is all coming down so it really enables people to take on scopes that might have been just completely impossible, even with someone who's overly optimistic. I hope that people look to it as an inspiration and it really was a great project for us to work on. I also understand that if someone does want to jump into this space and get involved, you all are hiring. JEFF: That's right. We are hiring for a broad range of positions. We're expecting to be doing a lot, more hiring soon. You can go to Kasita.com/Work and at the bottom of the page, you can also see that we have an open house here in Austin every Thursday morning from 9:30 to 11:30. The folks can come in and check out the crib. CHARLES: All right. Fantastic. I certainly really enjoyed getting the tour the space, what was that? Back in March? When you revealed the baby units? JEFF: Yeah, it was March at South by. CHARLES: Yeah, it's really something to see. If you are in Austin or you live here, take the time, go see it. It's really cool. With that, I guess we'll wrap it up. Thank you everybody for listening and as always, you can get in touch with us at @Frontside on Twitter or Frontside.io or send us an email at Contact@Frontside.io. Thank you all and see you next week.
We discuss what the deal is with Canadian whiskey and then talk about why we like The Economist. Your pals, @cowboyd (https://twitter.com/cowboyd) & @cote (https://twitter.com/cote).
In this episode, we talk about IoT: what's coming, why we're intrigued, and how we've already started it incorporating it in our office. In the next episodes to come, we will be having guests on the show to take a deeper dive into this technology. If you have any suggestions or know people we should reach out to, please get in touch! Transcript: CHARLES: Hello, everybody and welcome to The Frontside Podcast, Episode #77. My name is Charles Lowell, a developer here at The Frontside and your podcast host-in-training. Today, I have with me two other developers here at The Frontside. This is going to be a Frontside-only podcast and we're going to be introducing a topic that hopefully we're going to be podcasting a lot about in the coming weeks and months just because it's something that's kind of grabbed the interest of the office and seems like it's something that needs to be talked about. Hello Joe and hello Elrick. JOE: Hello, Charles. ELRICK: Hey, what's going on? CHARLES: Everything, really. Today we're going to be talking about the Internet of Things and we'll be talking a little bit about how we came to be interested in this topic and why we think this topic is important. Let's talk about why this topic is important. I think that this is a very important topic because IoT is only becoming more and more prevalent. It's emerging from the status of being this niche or boutique or very esoteric technology that's only worked on by a very small group of people to becoming very, very open and available and accessible so that anybody can buy a Raspberry Pi or an ODROID or Arduino and slap some Linux on there and connect it over the internet to a bunch of different things and the space of creative possibilities is just exploding. For me, it's very similar to where we were in the early 80s. You know, I see these IoT devices as being the hobbyist's computers, the Z80, Apple IIe, the Commodore 64 and that the people who are hacking on those things 30 years ago are going to be the people who are now leading the tech space today. I think another big and relevant analogy is web technologies. There was this inflection point where web technologies became very open, accessible, available and the people who were in it ended up being able to ride that wave for 10 years to where we are now. In both of those examples, we had the hardware and the PC revolution where the computation was distributed across a bunch of these different devices. Then over that time, we saw a migration over to the cloud and these web technologies where everything was centralized. Now, I actually think that there's a pendulum swinging back where we're actually going to see more and more computation distributed amongst physical devices, except this time, it's not going to be manifest as a PC. It's going to be manifest as these networks of devices that are just all around us. I really do think that we are on one of those watershed moments where these distributed networks of tiny devices are going to be the big next platform that when you invest in it now, this is something that's going to yield dividends for the next 20 years. I think it's an important topic but I don't think we had a well-crafted thought about it but we just kind of stumbled into the space. I was thinking we could start a little bit by talking about how we got into this and how it captured our imagination. If you rewind the clock to the stone age of 2015, I think it was the end of 2015 and it was Christmas break, that's often a time when people go and they hack on individual projects and Brandon, his project that for whatever reason, he decided to take on was he was really into Hue Bulbs at the time. We had Hue Bulbs around the office and we wired up some demos to control them from a website. He decided he wanted to take those Hue Bulbs and make them so they were accessible from our Slack. He built a server in Elixir because he also wanted to learn Elixir because if you're having fun in hacking around, it might as well pick up as many new things as you can. He built an API in Elixir that talk directly to the Hue Bulbs and the Slack integration that talk to the Elixir API and we actually are able to control all of our lights purely from Slack. We could turn them all on, we could turn them all off. That was great but then as we began to use it, we were wishing that we had control over our lights from our phones. We wish we had control over them through the website. I think, Elrick, isn't that was your first contact with the Frontside, wasn't it? ELRICK: Yes. That was my first contact with the Frontside. I was working on the lights app. I initially started working on just the user interface and bringing some different animations and working on the actual experience and the user story on that side about controlling the lights and what particular things you needed to do in trying to craft a UI around that. That's what I initially started. CHARLES: That was really fun. ELRICK: Yeah, that was really fun. That just started progressing more and more. As you said as we started to think about how could we access these lights from different places, using different devices and then that's how we stumbled into the Internet of Things. CHARLES: And it turns out, there's actually a lot of tech in the form of platforms out there that have been developed to help with this, although I would say that the water are still pretty murky as to kind of the best set of patterns to follow. ELRICK: Yes. JOE: That's hard to find information, especially with regard to design patterns. Since we've been working on this light thing, there's been so many times I've Googled and looking for prior art and found none or next to none. It's very much the Wild West. ELRICK: Yeah, because it's like going from a point where you're controlling one piece of data per se, like you have one sensor that does one thing. Now, it's starting to grow until you can have one sensor that can do multiple things and send it across different types of data and then how do you structure that data, how you capture it, how do you hold that state somewhere and it's one to one source of truth. It's just going to be the Wild West of how do you manage this, how do you structure it. It is definitely growing and changing constantly. CHARLES: I think one thing that is difficult is it feels very much like they're aligned in terms of silos. For example, the Hue has the Hue Bridge, which is capable of talking to the light bulbs and then they also have an API which is under development by which you can connect publicly to servers hosted by Philips to talk to the hues inside your office but if you want to integrate your Hue API like we did with Slack or with your iPhone or maybe some other device that you're trying to control, it becomes a little bit more difficult. You have all these vendors like Nest, MyQ and there's a whole bunch of lines like doorbells and smart this and that and everything and they're very good at talking. They have an ecosystem, this large vertical ecosystem, assigned with each one but actually getting cross cutting communication is a problem that I think is something that we've had to deal with and it's very, very difficult where we want to start having these devices talking to each other. ELRICK: Yeah, that area right there is ripe for innovation. I don't know the names off the top of my head but I know that there are people trying to make a smart hub per se. You can think of it like Jarvis from Iron Man. You buy that thing, you put it down in your house, you tell it all the devices you have and that takes care of all the communication between everything. There's definitely an area there that someone can step in and say, "You know what? I figured it out and here's your Jarvis Box." JOE: We're starting to see stuff like that with Alexa and Google has something similar. That's a little scary to me. I think that the one thing that needs to be made clear is when you're talking about these silos, it's a very good point because we think they're decentralized. We think these things are decentralized but in a way, they're not yet. We don't have peer-to-peer communication necessarily like Hue. They're going to public API but you're going through their ecosystem. You're passing through their lens, so to speak. We think Slack has distributed teams but there's a centralized server where those messages passed through so how do we break from that into full decentralization? CHARLES: Right, I know that's – ELRICK: The Jarvis Box. You could probably have a server at your house that keeps all your data there and then it spits out what it needs to spit out to the IoT server somewhere if they're doing some collection. When you leave your house, to say, "I need that information to come back to my cell phone now." Maybe in the future, you'll be able to control that, either from your house or just send out the pieces of data that you need and the centralized stuff, you can just keep at your house. CHARLES: The whole question of ownership is one that I feel is something that we have not addressed head on. Everybody is just rushing forward with how do I implement this, how do I get it done and it definitely is worth taking a step back and understanding who owns the things that I'm working with and that I'm inviting into my home. I think that smartphones provide a great example of how it can work really well for the consumer. I think certainly, in their inception I think this is mostly true if you have an iPhone. Most Android devices, you actually own that piece of hardware and the things that you install on it are very much controlled by you. I think that Apple especially, gets a big shout out for making sure and putting in those safeguards so that anyone who's participating in the ecosystem has to first acknowledge that the data is going to be owned by the user. I think that's maybe a little bit less true than it was back in 2009 or whatever but I think that there's definitely a lot of thought that went into that upfront, that I worry isn't going into with Alexa. Is Amazon protecting? Is there an understanding that if you're participating in that ecosystem that ultimately, the thing is owned by me? I feel the same way about a lot of these AI and robots where it may participate in the conversation but who is it really serving? Is it serving you or is it a proxy to serve somebody else like a Google or an Apple or an Amazon? JOE: I may just be a pessimist but I think it's safe to say that it's almost always the latter when money is involve. ELRICK: They had some situations arise where the powers that maybe we're trying to get the actual recordings and different things as Alexa is always on. Let me turn mine off because she's going to say, "Oh, did you ask me for something?" I have one sitting right here in front of me. They have been in situations where people had said, "Because that's constantly recording and that recording is going somewhere," and then if situations have arisen, they said, "We want that recording," and then Amazon is like, "No. We're not going to give you that recording because that is private information." They're trying to find a way to get around that and what laws and things are going to come out of this area that we're in right now, it's still unforeseen. But I think that companies that are in this space, know that the future of their company rests on them protecting that data and user data because if you don't, then people will sidestep and go elsewhere. CHARLES: Right. In so far, they hold that as a value. In so far, people are conscious of those concerns. If that's something that people are willing to pay money for, then you've got a market driving force pushing you in that direction. But if people don't care, they don't think and they're just like, "Whatever. It's cool," that's not going to be something that a business is going to roll into their product because ultimately, if people care, then it'll affect their bottom line. If they don't but it won't and they're going to act in their own best interest. ELRICK: True. CHARLES: I do worry that there needs to be a social awareness of what kind of powers these devices actually will end up having over our lives and hopefully, those will guide it but you're absolutely right. ELRICK: True. I view all of this IoT stuff and data is not too far off of what people do on Instagram per se like you have your pictures, you can either post crazy pictures or you can post casual pictures. How you use the power that these IoT devices are giving you is essentially falls into your hands like what am I going to send across this thing. I think that hopefully, the power falls into the user's hands and they empower people with these devices and not make them feel like a prisoner in their own home or car because this IoT things are popping up in vehicles now. If you step into your car, you start talking and your car is listening. If they go from it like the same way we approach our applications and such and say, we're going to empower the user, I think if these IoT companies take that approach and learn from the mistakes that were made in software by not empowering users, then after a couple years they're like, "Oh, my goodness. We need to empower the user." When Steve Jobs was preaching about this in the 80s and everybody thought he was crazy. Don't fall into our mistakes. Empower the users and I think that this technology in this space would just keep flourishing if they do that. CHARLES: Absolutely but it is going to take a generation of engineers to make sure they're always pushing in that direction, a generation of users who don't just wait for companies to hand power to them but demand it. ELRICK: Demand it, yes. CHARLES: Yeah, demand it and a generation of business owners who are going to listen and think about the long game and realize that that's the path to long term health and viability. ELRICK: Yep, even outside of the whole privacy thing where it's like there's too much data being sent out. People are building just cool stuff with IoT that doesn't really send that much data outside of normally that we do. Even on our phone, people use GPS all the time and that is sending data about all your locations, where you are, what restaurant you're at, what bus stop you're at, what bus you're on, what plane you're on and people are building a lot of cool things, just even using that. I saw the other day that someone had a bicycle, it has GPS and lights and gyroscopes and all kinds of stuff in that bicycle. When you're riding, the lights will go off and say, "It's time for you to take a right." It will blink in a certain sequence or take a left. It register your speed and it all comes back to your phone so it's not too outside of the norm of what we do on a regular day. There's people building things just in that sweet spot per se with these IoT devices that are building some pretty cool stuff. JOE: It's a very good point because Slack doesn't have to be centralized. It can be peer-to-peer. Hue doesn't have to be centralized outside of having a bridge on your local network. We don't really need to be phoning home for all of this stuff and if we move towards like a true decentralization, we don't need trust at that point. A company has our best interests at heart if we think about it as your trust ideal to remove the need for involving third party in the first place. CHARLES: Yeah, so what would that look like? I'm going to fast forward a little bit because we were a little bit further along on our journey and we've been experimenting with Amazon IoT services and we've been maintaining our own APIs to control our Hues directly. While they're still going through the bridge, it's not incorporating any other ecosystem but we are still routing all of this stuff through this low level Amazon infrastructure. There's a class of problems that that solves which it does help to have those primitives to be able to access your IoT devices through a firewall, to have them and be able to, at least have a known way to update themselves and distribute software to them. There's these fundamental infrastructural problems but at the same time, Amazon doesn't have any access to that data that's moving through their land, so to speak. What they're essentially doing is leasing you a railroad but they don't have new visibility into what's contained inside the cars. JOE: Do you know that? CHARLES: I actually don't know that because of course, it's through the Terms of Service. ELRICK: Who reads EULAs? They're too long. JOE: I think it's more often than not, people are going to use convenience over privacy. CHARLES: That's true so it is in keeping with what I understand of other Amazon services, which do have those guarantees. I don't know in particular for the Amazon IoT. But let's talk about that a little bit. Let's talk about a little bit about our setup and why we went to using Amazon IoT services and what it provides for us. ELRICK: We decided to use the Amazon IoT platform as a means to allow us to one control the bulbs from anywhere, to get access to them and then also to be able to distribute that change to anything we want. Coming through IoT or coming through their platform, when a change happens, you don't necessarily just have to send it to our one set of bulbs. You can send it to anything you want. You can send it to a phone, to another application somewhere, to a database. It gives you the ability and the flexibility to distribute that change or that state change anywhere. CHARLES: Which is I guess getting at the heart of it is actually managing this distributed state beast of a problem and really, the AWS IoT just helps you get your foot in the door. There are still a lot of cans of worms that are involved once you get there but for the first point that you have said, I want to unpack that a little bit because it's a problem very familiar to us but might not be to the listeners, you've got the set of devices and they come up, they connect to your Wi-Fi and that's fantastic and they can talk to other things on your Wi-Fi, on your local network and can discover services there. But what if you want to control them from outside like I want to send a message from Slack and have it affect the lights in our office. You've got to move through some public cloud to do that because Slack servers are not on our local area network. What you can do then is have essentially one thing that the IoT services provides is your device comes online and it immediately calls home to a generic location and opens up, what is in practice a web socket. You can program in whatever language you want but that's probably the analogy that's most familiar to everyone. It basically connects a web socket that then you can send messages to it in real time so any time I want to connect to that, I can do it and I don't need Hue's API. I don't need Slack's API. I can just talk to one API which is the low level Amazon -- AWS IoT API -- and I can send real time messages to my devices. That's a huge problem solved right there. But it's hard to maintain that infrastructure yourself. We could write our own AWS IoT but then we'd probably host it on AWS anyway. JOE: The real world is not a JSON Blob. That becomes a problem. In college, I took a course where we programmed robots for the majority of it and what you quickly find out is that you can't count on revolutions of a wheel or what have you. The world is imperfect. Keeping a state is one thing but keeping state reflected back and keeping state up to date is where the challenge has been for us. CHARLES: That is right because you've got this highly distributed systems. That's kind of a second class of problems that it attempts to solve for you. You got these highly distributed set of devices but even if the connections are 99.9% reliable, sometimes they're highly latent. You can't control the latency on the connection and sometimes, it fails altogether, which can affect one, how do I even read state from these things. Is the button pressed? Is the button not pressed? Is the light on? Is it off? Is the wheel spinning like you said? Or is it off? These are things that you need to know and then you need to react to those changes like, "We're spinning at 90 RPM. I want to bump it up to 10. How do I get my system to converge on that desired state based on my current state?" It's hard because you don't know all of the demons of distributed state management are in full like they have ripped off their masks and they're roaming about. ELRICK: Yep. I saw them introduced something the other day but I haven't had time to dive too deep into it. It was something called Greengrass that it will continue to gather and allow you to utilize your devices locally and it will keep all that data and then it will do the diffing, let's say when you connect back online until what your old state was and what the new state is and then go about updating everything. JOE: That could be very useful. ELRICK: Yeah. It just got implemented probably three weeks ago or something like that. It's inside of the IoT platform. I just clicked in and they said, "We have a new feature now called Greengrass," but I haven't got time to dive too deep into it but like you were saying, state management is something that's extremely difficult, especially across a distributed systems. They know it's a problem and it seem to be addressing that problem and trying to make it simpler for people and give you these tools to say, "Here are some stuff that you can leverage," and a lot of that is great. CHARLES: I think that's an excellent point and I think that it's also worth mentioning too that there's two sets of state that you have to manage. There's the runtime state, which controls the flow of data as your system operates. Then there's the static state of just what is the code that's going to run on this device. Let's say, my robot or my button that's got V1 of the software, that all it does when I push it, it rings a bell. That's V1. I want to add this awesome feature to this button that when I push it, it rings a bell and it also pops open a Topo Chico from the refrigerator or something like that. The question is how do I get that software from my laptop with that Topo Chico enhancement all the way to my button, which is what essentially amounts to being across the internet inside this private network. In the current state or when you're first starting out hacking, let's say this is based on a Raspberry Pi, I just burn a new Raspberry Pi image with my new software with V2. I walked over and I stick it into the Raspberry Pi and that doesn't really cut it. That does a great job but now, I want to turn this into a business and I want to have 20,000 of these things installed or let's think big like every home in America gets one. Every home in the planet, I want two billion of these type of devices. What happens when I come out with V3? ELRICK: Then you can either go the route of hiring -- CHARLES: Hiring a favor. ELRICK: -- Technical folks to go out, to update all your Topo Chico poppers or have your users struggle to do it or what we did, implement Resin. Let Resin update your Topo Chico poppers around the world. CHARLES: Right. There are a lot of problems in terms of static state management, runtime state management, peer-to-peer communication and problems of resiliency and robustness. I'm hoping that we can discuss these over the coming weeks and months because each one is a topic in of itself. ELRICK: And offline management too. CHARLES: And offline management too, there you go. There's another one. There's a lot to explore, a lot that's unknown and there might be people who have answers to all of these and there might be papers on them but they're buried in weird corners of the internet. I'm hoping that we can fill the podcast with a couple of guests to come in and talk about these different things. ELRICK: Yeah, that would be fantastic. CHARLES: Yeah, I'm really looking forward to it. ELRICK: I started playing around with Watson IoT. It is an IoT service that allows you to leverage the natural language processing and computing from Watson. It's pretty awesome. CHARLES: Wow, that is really cool. ELRICK: That's another space of IoT that we can explore and hopefully, we can explore over the next few podcasts. CHARLES: Yeah, awesome you all. Well, I think that's about it for this episode. Thank you, Joe. JOE: Thank you, Charles. CHARLES: Thank you, Elrick. ELRICK: Thank you, Charles. It was fantastic. CHARLES: And I look forward to hacking on the lights with you guys. That is always one of my favorite things to hack on. I don't get to do it enough but I think we're going to try and have a big throw down on state management on Friday, right? ELRICK: Oh, yeah. CHARLES: It is going to be exciting. It's going to be super nerdy and we'll let you all know what the outcome of that is. See you all next week. As always, please don't hesitate to get in touch with us. You can get us on Twitter at @Frontside or send an email to Contact@Frontside.io. We always love to hear from our listeners. Take care!
Drew Covi: @drewcovi | about.me Show Notes: 01:04 - Honeywell User Experience (HUE) 05:00 - Deliverables 06:55 - Being a “Devsigner” 17:26 - Flash and Leading to Unique Skills 30:00 - Advice for People Straddling Roles 35:27 - Leveraging Design and Development Skills Together 39:41 - Embracing the Hardware Element 42:05 - Why the “Devsigner”? Resources: AOLpress CSS Beauty CSS Zen Garden Contribute Crave Transcript: CHARLES: Hello, everybody and welcome to The Frontside Podcast, Episode #76. My name is Charles Lowell. I'm a developer here at The Frontside and your podcast host-in-training. With me is Elrick Ryan, also a developer at The Frontside. Hello. ELRICK: Hey, what's going on? CHARLES: Not much. Are you excited about today's topic? ELRICK: Very excited. CHARLES: Yeah. You got a personal stake in it because today, we have in the room, not only you but also two developers who are also designers or designers who are also developers. Our guest today is actually the first person who fit this description that I ever worked with. It was a great experience, a great collaboration and his name is Drew Covi. Drew is a senior supervisor of product design at HUE Studios in Golden Valley, Minnesota. DREW: Howdy. How are you doing? CHARLES: Good. Thanks for joining us. Now, you're going to have to explain to us two things, one, what is a super senior product designer and let's start off talking about HUE first. What exactly is HUE because I think it's a cool organization? DREW: I'm working with four people and I'm working on all sorts of brand new ideas. I think the greatest opportunity that I've had in my career at this company, Honeywell is just working with physical product and the digital space. It's a unique opportunity. Not all companies focus on both so it's really been a learning experience for me and working with a great group of creative individuals is also been a real privilege. They say that at the end of the day, the most important thing is other people that you work with and really the entire team here has been fantastic in welcoming me and letting me explore and grow as a developer and as a designer. It's been great so far. CHARLES: Fantastic. Working with that group was absolutely wonderful. What does HUE stand for? DREW: HUE is Honeywell User Experience. Our previous CEO, Dave Cote often called it 'huey' but it's just HUE, without the Jersey accent. I'm going to probably misrepresent but we have over eight to 10 studios throughout the world. Each one focuses on different businesses for the most part. The one here in Golden Valley tends to focus on homes and buildings technologies. The studio out of Seattle, actually tends to focus on, again I'm going to get the acronym wrong here but it's essentially worker safety in industrial safety. CHARLES: What is it that you all do at HUE? DREW: What we do here at the studio here in Golden Valley is we support various businesses throughout the homes and buildings technology space. About fall of last year, Honeywell went through a bit of a shift in their business and they used to do all automation control solutions. Last fall essentially, we saw that one large business that was headquartered and based out of Golden Valley, break into two areas of more direct focus. Out of Seattle, we have folks working on, I think I mentioned before but Seattle works on sensing and productivity solutions. We focus on homes and building space so we're both providing upfront research to understand what the customer needs. We're actually creating everything from very rough user flows to final UIs and we're also working with industrial designers to create final products. Those industrial designers work very closely with engineering. Honeywell has a long reputation of very strong engineering when it comes to the hardware space. We've prided ourselves on excellent instruments and excellent performance. One thing that very few people understand is that we don't just do thermostats. We're in the business of turbos. We're creating the turbos for your car. We're creating all sorts of HVAC equipment. We're also handling various safety equipment. All of these items need designing, not just for end users and consumers but they also need designing for the workers in the field. If we make a product that is more efficient, easier to use and in some cases, more attractive, not only it does lead to more sales, it leads to more efficient work forces that can work quicker essentially. You could get up on a roof and get off in record time. We're not just designing consumer products. We're actually focused on a lot of other items as well, with oftentimes very large returns on investment. CHARLES: In the work that you do and HUE does in general, it sounds like there might be a large software component. Digital design is kind of we know in the web space but then also a lot of industrial design of just how does this thing going to look, how is it going to feel, how is it going to persist, how durable is it going to be, how is it going to withstand usage. Would you get involved in that process? DREW: Usually, the entire organization gets involved with the process very early on. One of the other shifts that happen in the fall as we get involved less in the production and more on the actual marketing side, like marketing deciding what's going to be built. We're actually really at the beginning and understanding what problems need to be solved at first. As far as my practice and my skill set, we do get involved with all that discovery phase work but when it comes to actual deliverables, we oftentimes see our deliverables around the actual creation of understanding user interactions. We will take research from our user research in OVOC, which is an acronym for Observational Voice of the Customer and we'll take those learnings and translate them into whatever solution we decide to build as a team. My output is going to look like a user flow, something you build in OmniGraffle or Visio and then it can start there, which is in the physical space and then we'll actually revolve those concepts into wireframes as well. Wireframes that will then be handed off to other team members who specialize and focus on visual design. Basically, it's kind of a very hands on process from the very beginning to the very end. It's essentially just understanding everything from the physical to the digital. CHARLES: When we were working together, at least in your case, it doesn't stop there. You're actually doing a significant amount of the implementation as well. Let's explore how did you actually end up getting to that position where you were working through interactions, wireframes and workflows and then also, getting to actually build the product in the form of a complex single-page application. DREW: Sure. Absolutely. One of the components that I kind of brought here to the team was a bit of a deeper understanding of frontend web development. I'm often pulled into conversations here and there. In the case of the project that we were working on specifically, it was essentially kind of early days on that project. We had a product that was pretty old and need a lot of work and it was basically, need to be rebuilt. We hadn't seen a lot of single-page applications at that time. In my case, I actually had worked on a couple small projects in my previous job and we can get into that in a little bit, where my career path took me. But essentially, it was me trying to kind of pave the way and eventually have that work scale. It was kind of proving that it could be done, showing how it could be done and then getting other developers on board. My role here has oftentimes involved, basically becoming a liaison between our design teams and our development teams. Ultimately in this case like you mentioned, it did wind up in turning into code that ultimately got factored into production code. It was definitely a time where we were experimenting with what role we would play. I will say in full disclosure that more or less which we're trying to move towards, basically making better informed decisions but not playing as much of a role in actual production code writing. It's something that we want to help scale. I think we'll talk about that kind of role and how well it scales hopefully in a little bit here but ultimately, it kind of changed a little bit. I don't do as much code as I used to. CHARLES: Right but nevertheless, the skill is there. Don't sell yourself short. You weren't slapping together a bunch of jQuery plugins. You were standing up, basically a full stack system with a StubDeck background, then Node.JS. This is back in early days where there was a custom-build tooling. You were using CoffeeScript. There was a lot of exploration and clearly, there is a fierce curiosity which you are actually exploring and actively kind of skinning and moving into the development space, which doesn't happen until people achieve a certain level of comfort. Whether or not you're exercising those skills, I think they have served you well in terms of the things that you've been able to build but also acting in that liaison and understanding what's possible and stuff like that. Obviously, once I met you, you were already there. I'm curious in exploring that journey of coming up the design ladder but also coming up the development ladder too. Maybe we can talk about each one separately and then see how they intertwine. Let's start with the design side. How did you get into that? DREW: I can take you way, way back. I love to talk more about this in a little bit but I think we, as a generation, are kind of very unique in that. We were raised in the birth of the internet. Some of us are old enough to remember the early dial up days and I certainly was one of those. I grew up basically obsessed with drawing and art and painting. I was a designer and artist raised by an engineer, essentially. My dad didn't really have a lot of opportunities to explore his creative side to basically make a living. I want to say that although graphic design existed to a certain extent, there wasn't really the same blend of engineering skills required so he decided to take the tack of I'm going to become an engineer so I was raised in a household where he was building everything but he was also a talented artist. As a kid, I basically did a lot of advanced art classes. I'm kind of a nerd, pretty much a huge nerd. I dropped my entire tenure as a high school student. It was also kind of dawn of video games as well so we had computers coming of age. We had video games coming of age so I was raised looking at digital art effectively, 8-bit, super accessible. It's kind of so early on that it was something that I could actually fathom getting into and creating on my own. I never got to creating any games but I will say that by my late high school years, I was using a tool called AOLpress. For anybody who has ever heard of that, congratulations. You're one of the few. CHARLES: I've never heard of that. AOLpress, we're going to have to link to that in the show notes. ELRICK: I've never heard of that either. DREW: It's awesome. It's got a Wikipedia page. It's got hieroglyphs and stuff. They really went all out on this product. It's basically the precursor to the Dreamweaver. It was a very, very WYSIWYG. I'm sure you've heard of Microsoft FrontPage, maybe. It was basically a precursor to FrontPage, I would say. Same thing, those are the days of framesets and all of that. I was a kid in scouting at the time and I wanted to build a web page for the troops so I built one and put it out there. I kind of remember that moment where I was like, "I'm going to write something and put it on the internet and anybody can see it." That whole experience was just super exciting. I know that if anybody's following Kickstarter, there's one that was started called 'What Comes Next Is the Future.' It was made by Matt Braun and Matt Griffin and it really explored the birth of the web. I would recommend it on your listeners to want to really dive deep if you didn't live through it, check it out. It's a great, great film. All the regulars are there as you'd expect. Zeldman on there, talking about it amongst others. But if it were for the web, I don't know that I would be who I am or where I am today, just because it's such a unique platform. It's so open. It's so readily available. There's no barriers. I would say that I was just an arts student in high school that picked up AOLpress and then got addicted to the web. From there, it was kind of off to the races. In fact, I didn't even know that I could make a living as a graphic designer until late high school. I decided that I wanted to go to school for graphic design, went a year at the University of Minnesota-Twin Cities and at that point in time, it was pretty much all print design and then Flash. Flash took over in my second year and at that point in time, it was Flash and framesets and tables. There was no CSS for layout. It's very early days. It sounds like you might know what I'm talking about. Have you been there? ELRICK: Yeah. You know, they say everyone in the world has like a twin and I'm like, "Drew is like my technology twin." DREW: Yeah. When we were raised in that time and we had to hack it with framesets and whatever tool -- FrontPage or AOLpress -- you basically, from very early days, realized that you had to force this stuff to happen. It was not easy. There was no documentation and where there was documentation, you were grateful to have it. I remember when I was, probably just about to graduate and if I look back at my portfolio piece, it was definitely still Flash. It was Timeline-based Flash. I also think that in many ways the way the web evolved was perfect. As a designer, I was very comfortable in the Timeline tool. Before ActionScript 3.0 and before they went on object-oriented on us, it was super accessible. You could add little bits of code here and there and create animations. It kind of got you hooked. Then suddenly, I found myself needing to create full screen Flash applications and needing to actually write code. I actually having to say, "If I want this Flash experience to scale, then I need to calculate where things go. I can't just X-Y coordinate and done," so that's where I jumped off and started getting into CSS. CSS was kind of early days as well. Again, this is before iPhone. This is like people were using CSS but people didn't really think it was that important. It was actually kind of discouraged because everybody in the world was using Internet Explorer and why would you need to know CSS. It was unreliable for different browsers and Internet Explorer was the worst. I remember sitting in a Dreamweaver conference, when it was Macromedia had a conference and they showed a webpage and then they hit the print button and they said, "Does anybody here know how this happened?" because the layout had changed, everything looked better and different. It was perfect for print. I remember my hand shot up because they was like, "Nobody was really familiar yet with that print style sheets?" Incidentally, I don't think that people still are familiar with print style sheets but it was a time when finally people were starting to understand that style sheets were more than just a layout tool. You could change them for all these different form, factors and all these different platforms. It was a fun time to be coming up in this age. CHARLES: It sounds like one, CSS and two, Flash were actually kind of gateway drugs into the development world? DREW: Absolutely. CHARLES: We still have CSS, clearly but do you feel like Flash, despite what some people might think about it, it was a full virtual machine that was running. You could code on it with ActionScript. It's kind of like the JVM but only for running inside the browser. Do you feel like designers might not have that gateway available to them anymore or maybe is the web just as big of a gateway to move into that? DREW: Yeah, for sure. I certainly think, beyond a doubt that had it not been for Flash, we would see a lot less creativity in the space. I say that only because at the time, if we had just gone from tables and tried to slowly evolve things, we'd have a much different feel, I believe. Certainly, it's a gateway drug. We'll be in a different web today without it. Is it still required? Are there any equivalents? I've seen a number of drag and drop web UI on the web tools out there and many of them claim to create production quality code. It's certainly possible to get there without Flash. I think, it's certainly its time has passed but we do see tools like Sketch for instance. These are all very much screen-based design tools that seem to leverage a lot of the same web styles and the web approaches. I think we definitely have the tools there to replace Flash. But I think from my perspective, it would be very interesting to go back and imagine, would we have immersive full screen web experiences without that Flash? CHARLES: Yeah. I remember it being very much a topic of conversation, certainly at the beginning of each project or when you were going to implement a feature is, "Are we going to do this using Flash? Are we trying to do this with native HTML? Are we going to use EGADS or Java applet?" ELRICK: Oh, man. Java applets. CHARLES: That was a conversation that was had before the web eventually went out but I think when it was, everything was very, very static. I do think that Flash definitely set the expectation higher and forced the web to evolve so that it could be the natural choice in those conversations. ELRICK: The time when Flash was around, I called it the 'golden age of user interface' because you can literally build any user experience, any user interface with Flash that you could dream up. There was no limitations creatively in the world of Flash. Nowadays, we're kind of limited without box model but it's getting better year-by-year. DREW: It's interesting to me because before Flash really died out, we had these... Let's put it this way. I feel as though, for a long time the web was a very much like a poster site kind of approach. You would have tools that were pretty rough on the eyes, pretty hard to use and then like for certain films, you have these very high budget, fully immersive Flash experiences. For a blip, that did actually translate at some point into Canvas-based and then Three.JS, like 3D WebGL-based experiences in native HTML but I don't see a whole lot of that anymore. It seems as though, it kind of settled down and in many ways, I would say killing Flash kind of evolved the web from more of a presentational platform to more of a usability first platform. It was a bit of a double-edged sword. You could build anything you want like you said but there wasn't a framework to it. It wasn't really responsive and then certainly, when Steve Jobs decided he wasn't going to Flash an iPhone, that was the end of it. Essentially now, we have -- ELRICK: Steve Job dropped the hammer. CHARLES: That was the memo that was heard around the world, right? DREW: Yeah. CHARLES: I just realized that was like 10 years ago. DREW: Yeah, they're celebrating the anniversary for the last couple of months here. It's been a huge deal. CHARLES: There's probably listeners that never heard that memo but it's definitely worth a read. The memo obviously, that you guys are referring to is when Steve Jobs basically said that Flash would not be on iPhone or iPad, not now, not ever. That was the end of it. DREW: People often forget too that when it was first launched, there was no app store. He basically said point blank, "Anything you need to do on this phone, you should be able to do using the web, using native web coding," and Safari at that point in time is really paving the way to bringing those native APIs into the web. You had geolocation through web. In many ways, that too is a huge gateway drug. Suddenly, you start looking at the web, not as just like, "I could use this as a poster site or as an informational site or a new site. I can actually use this to get things done." They're actually treating this platform as a first-class citizen. That to me was super exciting. I don't know if it gets as much attention anymore in the days of Swift and the App Store but I will say that if your listeners do get a chance to check out the show I mentioned earlier, 'What Comes Next Is the Future,' they even dive deep into just how limiting the app store experience can be. At least with the web, you can create whatever you want to create and people seemed to go that you URL and install on their home screen. This is a feature that nobody uses from what I've seen but if you bookmark a web app on your home screen, you can have an icon, you can have a loading screen, you can have all this stuff and nobody really uses it for whatever reason. CHARLES: I think it's the install, it's getting the knowledge about the fact that you can do that. It's not widely disseminated. ELRICK: Yeah, I think its capabilities starting to come up now with people making progressive web apps. They're starting to utilize that being able to put icons on people screens and loading screen and splash and etcetera. CHARLES: Flash really was kind of the gateway into the development world. I'm curious what opportunities do you feel opened up as you started taking on more web technologies, more JavaScript, more CSS and mixing that with the design that you were doing? What unique skills/superpowers do you think that gave you, that made you, that helped you at that stage in your career? DREW: Yeah, for better or worse, it really was the opportunity to get a job first of all. I know that the job market has been in all sorts of flux in the last couple of decades but I would say 12 years ago, in 2005 when I was entering the workforce, graphic design was not necessarily a hot field. I can say with relative certainty that the majority of the people I graduate with, didn't necessarily make their way into graphic design as a profession. I would say probably maybe 30% to 40% actually wound up following their degrees. For the obvious reason at that time, we were starting to see digital replace print. It meant that I was able to get a job for one. It wasn't a dream job necessarily but I was basically a one-stop-shop. I was designing and developing websites as working for a company but in many ways, shapes and forms, I was kind of freelancing as things were. I had a very direct relationship with the clients that I worked with. It was basically churning out websites. If I recall correctly at the time the company wanted to essentially create a Domino's Pizza of the web where we could use CSS to essentially build the actual HTML once and then restyle it. This is actually was a time when a site called CSS Beauty was just coming of age, I think the site still exists but back then, if you want the CSS Beauty, it's big thing was you have one website and people could upload their own CSS and completely change the layout, completely change the look. CHARLES: Are you talking about CSS Zen Garden? DREW: Maybe that was it. There's two of them. CHARLES: I remember that one. DREW: CSS Zen Garden was one of them and I think CSS beauty was a clone maybe of Zen Garden for sure. Maybe you're right, Zen Garden was the one where you actually had a website and Beauty was just showcasing certain CSS sites. I think you're right. Zen Garden was the one. When they saw that, they're like, "Wow, business opportunity. We can build a whole site." We were using something called 'Cold Fusion' and... Oh, it will escape me now. I think it was called 'Contribute.' There's a product called 'Contribute' that Macromedia come up with that worked on Cold Fusion. It was basically a WordPress. You basically set up editable regions, you basically code the site once in that regard in the backend coding and then just rework CSS to create multiple sites. Actually, the opportunity to open up for me, that job was very squarely-focused around the benefits of leveraging CSS. Eventually, that grew tiring. I kind of wanted to get into the actual marketing and advertising space. From there, I started to just jump to the next job. I worked for a very, very small marketing agency. It was called 'Vetta-Zelo' at that time and we focused on lots more Flash, a little bit of CSS websites but mostly Flash Experiences and they actually used Flash in a lot of kiosks and physical spaces. I started to jump into that, understanding PHP, understanding databases because we would do things like we would install Flash Experience on little portable tablets that would then sync up survey responses to a web URL that it would then dump it into a database. About that time, I was always trying to teach myself how to get really deep into the backend of the stack. CHARLES: That was just to make sure that these Flash sites that you're developing would be scalable and more robust? Was that the natural next layer to dig down? DREW: Absolutely. At the end of the day, we wanted to have immersive Flash experiences and we wanted to have the content easy to update. I would build these really crude backend with text areas and they would update a database and then the Flash Experience would pull that in as content. In that way, we didn't have to go in and re-publish the Flash every time, essentially. It was a much more streamlined process. I think we even gave some of our clients the keys, gave them a login and password and they could change certain things. There's an outfit around here called 'Crave.' They are a restaurant in town and we built the website for them -- one of the earlier websites. When you have to do things like update times and menus and things like that, it became pretty essential to having some sort of a CMS behind it. It was all based on necessity, in other words. What you said is absolutely true. We had to evolve what we learned and I had to push what I did to lever on different needs. Throughout my career, I've been the guy who does web and design. One of the things about that is it's kind of a lonely place to be and find yourself in creative agencies, where the majority of skill sets are not in development and trying to explain what's going on or make commitments on timelines and deliver on them. Whenever a bug shows up, it's never really fully understood. It's also a challenge to manage expectations, certainly as a young professional at that point. CHARLES: Yeah, I would say, what would be some advice you would give to somebody who is straddling these roles at that early career stage where they're maybe working for creative agency and fulfilling these two roles but most of their surroundings is towards the design end. DREW: Yeah, I would say for the most part, just be upfront. If there's anything that's unknown, be upfront about it and explain. If you are early in your development career as a designer, do your homework before you committing any commitment certainly. I think it's always better to be upfront about these things than to try to over-promise and then scramble at the end. I will say that a lot of my career has been marked with the term code 'code cowboy' as a designer and teaching myself to code. It was a disparaging term, I guess. I didn't really necessarily take it that way but I think other developers are trying to use it in that way. CHARLES: [Singing to the tune of Mammas Don't Let Your Babies Grow Up to Be Cowboys] Cowboys ain't easy to love and they're harder to hold... ELRICK: It's so true. DREW: You know, I'm not even embarrassed to say it because the truth of the matter is when you're a designer, you're used to just making a mess before you kind of landed on what you're done and what's right. The entire creative process is messy. I think it's inherent. If you're one of these designers turned devs and you basically just hack it until it comes together, that's kind of a natural flow from the creative process. Certainly, as you get more experienced, you want to reduce all that uncertainty and potential for error so you do learn to hone your craft, to use version control, to embrace a framework or embrace some model-view controller approach but none of that really existed in the early days of the web. I kind of came up in a time when you had to hack it. CHARLES: Well, there's a lot of learning that can happen when you're hacking and building things that are kind of ad hoc. As you go, you get to perceive firsthand the problems with them. Without perceiving those problems first, it's hard to really understand the solutions that the internet has come up with to deal with those complexities. DREW: I would say I was like a solo designer developer throughout the early years, because at 2010, I found my people in a local agency called 'Clockwork' and for the first time, I wasn't the only developer on staff. There was a whole team of developers. In fact, the shop was started as a development shop and they were making headway into the creative space and eventually, becoming full digital partners. But had it not been for my opportunities at Clockwork, I wouldn't have picked up my skill set as a backend coder. From the very beginning at Clockwork, they expected you to get your hands dirty and code and get your hands dirty in the terminal, honestly. Command line was required even in our design work. CHARLES: And this is all designers needed to be familiar with the terminal tools --? DREW: Correct. CHARLES: -- Basic coding? DREW: Yeah. Essentially, all of our work, whether it was creative or whether it was documents, were all managed in Subversion. As a part of onboarding, you basically learned how to use Subversion. There were some GUI tools for it but for the most part, it wasn't that steep of a learning curve. It was pretty easy to follow instructions and that was the second gateway drug, I would say. My first gateway drug, again was kind of coming up in the age of the web and getting into CSS and Flash. The second gateway drug was basically being required to learn command line and learning how to navigate a computer without a display. Had not been for that, I don't think my career would have taken the turns that it did. I basically got more into the IoT space. I had set up a home NAS server with Drobo FS, is what it was called at the time and it was just a really basic machine but by jumping into that, I could start to play around with UNIX and tools there. I started using home automation, playing with that and at some point in time, I made the jump from just web into the role that I play here at Honeywell, which is Internet of Things. We do a lot of Internet of Things. In fact, our latest tagline is 'the Power of Connected' so we've embraced it all the way down to our wood mark. It's becoming the new normal for most products so it's a good time to be at the center of all these different areas of expertise, to be in development, to be in IoT and to be in design. That's my path. That's my journey. I would kind of pick it up at a bunch of fortunate circumstances, honestly. ELRICK: Having these two skill sets: your design skills and your development skills, what do you believe that that gives you in terms of an advantage? Having these two skills set and being able to leverage these two? DREW: From my perspective, having both skill sets allows me to understand. I think the biggest challenge when working with large teams, particularly in this space or in any space is to really have a common level of understanding, stepping aside from a functional role and becoming more of a liaison between design development and to be honest with you, as we look beyond that, I took a three or four or five month course in business administration, actually. It was just a night class but I wanted to be able to speak to those needs as well. I think it really is becoming a translator. Serving as a translator between those items and then also being able to understand where the actual boundaries lie, there are a lot of very talented engineers and talented designers and sometimes opportunities are missed because, either timelines are pushing engineers to cut certain functionalities or certain features and there's a lot of pressure. Where we can lend a hand, where we can point to possible alternatives, I think that's where we really build cutting edge products. When we really know each domain, we can push those boundaries. That's where I'd enjoy bringing my skill set to the table. CHARLES: Yeah. I can second that. Having actually worked with you, I think one of the greatest things was the one just with the interactions that you were coming up with, were just really spot on. It wasn't ad hoc. It wasn't some -- ELRICK: Helter-skelter? CHARLES: Yeah, it wasn't helter-skelter. It wasn't some developer coming up with like, "Hey, this is what this looks like," Or, "This is some designer putting up pie in the sky stuff." It was, "I understand what's possible and I'm going to use that to design the best thing that can be possible." It made the designs very pleasant and some of them were just really fun, I think. Thinking especially like that, the hierarchical tree selector was one -- ELRICK: Yeah, that was fun. CHARLES: -- Which the implementation of that was just a joy. But then the second thing is being able to speak with you on the development challenges and really know that you understood that language. It really is being bilingual, I guess in the sense that I'm talking to you in French and you're talking to product owners in German or whatever. But because you're bilingual, the flow of information is as frictionless as possible. DREW: I will say that it was a real pleasure from our end working with your team as well because one of the trends in many businesses throughout the world today is embracing a lean and agile approach to product design development. One of the growth opportunities, I would say in any business is fully understanding how that process works, having the courage to be upfront about what can be accomplished in the time available. I think one of the other things is fully understanding those three pegs of the stool. There's always the budget, the time and then the features of any projects. I think that working with a team that understands that really changes the dynamic. I will say that it was equally a pleasure for us to work with your team because there was just a level of courage in being very forthright and very upfront about what do we need to get the job done? What has to happen? You made my job as a translator, essentially. CHARLES: We aim to please. ELRICK: Absolutely. DREW: Absolutely. The latest evolution of kind of where my career has taken us in the company is embracing the hardware element. We've talked a bit about the web and then how that evolved and then having to get comfortable of the command line and where that took place. I've always wanted to build. I've loved designing but I always want to build it and I want to put it out there. In the last six months actually, I finally decided that I would pull the Band-Aid off and jump into soldering hardware, writing what code I could and building actual physical hardware prototypes. I think the next step for anybody who likes to follow this maker trajectory, for a creative looking to become a maker or a developer looking to get into creative is just not stopping. There's always something there and we're also fortunate to live in a time when I can go on at Adafruit, pick up a kit of parts for under $100 and build something that's completely new. Then by the way, they have a full-on tutorial that takes you through every step of the process and gives you bits of code to get started so what's your excuse at that point? If you've got $100, then you can throw and toss into a hobby, pick up a soldering iron and go to town because there are videos, there's the documentation. Documentation is just everywhere now, where it was never there before. I think the next step for us is seeing how can we very early on show real physical world products to end users and get feedback. How we're taking design now is beyond the digital and into the physical. CHARLES: That's fascinating. I feel like there's this pendulum that swings through the tech industry of things moving from hardware to software and back again. We're in the middle of the swing towards the outside or towards the hardware again, like the distributed hardware versus the dumb terminals. It's distributed across a bunch of devices rather than concentrated on one super-powered desktop computer. The pendulum is going to swing in it but it's just always fascinated to see what the actual arc that it takes is going to be. This has been a fascinating conversation and the reason I wanted to have it and we were actually talking about this before the show started officially, why this topic of 'devsigner?' I think that it's a role that is emerging. I think it's still in the early days. I think that I went from three years ago having never really met this type of person to having met and worked with you. Now, I would say having met and worked with three people here at Frontside who fulfill that role and now knowing a couple professed devsigner or people who operate clearly in the design and the developer space on Twitter. I feel like it's this emerging career track that might not be fully understood or defined right now but clearly, there's something there so we wanted to explore that. I'm curious if we might be able to open up the discussion a little bit on what is the future of this role? What tasks will it be set to accomplish? When you're assembling your team, you say, "Get me one of those because we're going to need that." How is that going to be further refined and designed so that it scales as, perhaps an official career in one, two, five, 10 or 20 years? DREW: I can only speak to my experience in this area and I can say that for the most part, it is a very unique skill set and sometimes, it's hard to come but like you said, you're working now with three people. I think it's growing in prevalence. I believe that where coding was less common in the past, it's becoming so much more common now that it's almost like an expectation just like typing. It is an expectation now. People expect you know how to type. It's not a surprise that we're going to see more and more of these individuals. I would say that any design team out there could almost invariably benefit from having somebody with this skill set, somebody who can translate design concept into a working prototype. I've seen it manifest as a prototyping role, more or less just so that we can have a tangible deliverable for developers. I think it does depend on the team, certainly. If you have small teams with talented frontend developers, then certainly you can work in a lean and agile environment and make very quick iterative change. If you have very large design teams and very large development teams, I would say that having a frontend developer with the skill set in a creative team allows that communication to happen without routine phone calls and lots of meetings, essentially. It's a crystal clear example. I've see it manifest as a prototyping role because the expectation is this code will end up in production but some of the code may. The layout code may end up in production but the functional bits may not. That's not to say that the functionality isn't a part of the experience and that, designers don't care about how well an experience performs. But typically where many designers see the disconnect is in the presentation layer. Having somebody who can carry that over is usually something that is far smaller team can handle. Does that align with your experiences? CHARLES: Yeah, that makes a lot of sense and I would say that the compliment from having this person on your development team, if you're in mainline development mode or maybe you are a small team, even if it's a production system but you don't have full time design resources, this person can slice and dice the features and understand the hierarchy of interactions and being able to put together some wireframe, some very concrete goals and set those goals for the rest of the development team. But yet also understand what goals are achievable in the iteration. I think it works from the flipside as well. Maybe what we're seeing is the agile of the [inaudible] of everything. What we've seen over the past 15 years or 20 years, what has been the arc of my career is just seeing these feedback loops in every element of product development getting smaller and smaller and smaller. On the development side, we recognize this as being able to feedback loops and verification. Having your tests, you don't actually have to deploy your system to be able to get feedback about whether it works or have it be fully assembled to get feedback about whether it works. But then that manifests in terms of continuous integration and deployment. You're bringing down the feedback loop of getting this out in front of people versus these long deployment cycles that maybe you really have a release every year. It was hard to believe but that was the norm when I started. It was yearly, maybe even once every 18 months. It was not uncommon at all to have released cycles like that. Certainly, three months was very, very short but then those tight feedback loops can also manifest itself, internally in terms of team communication and I think having people who can make those feedback loops between the product and between the implementation, every time you shorten that feedback loop, you're unlocking an exponential amount of time. DREW: Yeah, I think you kind of hit the nail on the head when you talk about setting scope and understanding things as well. Strictly speaking from agile terminology, having a product or a role that can bridge those gaps is critical. I think that the best product owners that I've worked with have understood, have had an appreciation for design but also have had some degree of a development backend as well so they know how to make those critical decisions. In any sort of iterative or agile environment, you have to dice up these features and figure out which ones are going to ship when they're going to ship. I think, yeah you hit it right out of the park with that. Whether or not you can ever have a full-on team of just prototypers, I'm not as convinced that that's necessarily scalable. It seems like there's certainly a role for teams of developer that will break down features and then there's teams of creative as well. CHARLES: I think in terms of the person who would lead that team, this role definitely seems very well fit. DREW: Exactly. CHARLES: I think it's a great opportunity for someone who's looking for a leadership position in terms of developing and seeing products to market, which is kind of similar to what you're finding yourself in today or where you're headed towards, it sounds like. DREW: Yeah, for the most part. It seems like I do find myself in a number of calls in kind of bridging those gaps. It's certainly a different dynamic in the agile environment when work with hardware. That's something that I think we're still exploring and still understanding. Certainly, there are companies that do agile with hardware but there's a whole slew of different challenges. You're not just deploying anymore. You're actually building manufacturing understanding what needs to ship with what. I think the next evolution of our company's growth into this space is how do iteratively produce hardware. ELRICK: Interesting. CHARLES: You got to keep me posted. The next time we have you on the podcast, you're going to have it all figured out, you're going to be presenting your thesis, it's a conference talk upcoming, agile hardware. ELRICK: Yeah, that would be pretty interesting. DREW: Yeah, I'll let you know. CHARLES: In the first iteration, you just throw a bunch of boiling solder on the breadboard and see what works. "Okay, now, that didn't work." DREW: I'll be honest with you. The 3D printing is making lots of possibilities open up in that space but ultimately, you got to ship. We use 3D printing and now we are using these low-cost computers to really prototype real world experiences and near-to-final industrial design. We can do that. CHARLES: Drew, this sounds like you have the coolest job. ELRICK: I know, it sounds awesome. DREW: It become even more exciting than I had initially intended. It's fun times. I think, again we're living in a time when we can 3D print stuff and have it done within a couple of hours. What better time to embrace these technologies and this creative spirit. It's kind of all around us. Honestly, it's just being fortunate. CHARLES: Yeah. Fantastic. This has been a great conversation. Thank you so much, Drew for coming on. DREW: My pleasure. Thanks for having me, guys. CHARLES: It's an amazing place. It sounds like even more fun since we got to work with you. If anybody is out there and they're in the design space and they think that, "Oh, maybe I can't do development," or it's too hard. It's not. There's a lot of people out there who are doing it and experiencing lots of good benefits. I would say that the other thing is if you're a developer, you should think about looking into the design space, something that you might be interested in. I think it's probably less common that the vectors people move from development into design and not vice versa but there's nothing that says that it can't go that way. Mostly, it's because people just aren't doing and they think that that option is not available to them but clearly, it is and clearly, it's a valuable role. I think this role is going to only get more valuable in the future. DREW: I would second that thought and that notion. I give a quick shout out to Erin O'Neal. She's a former colleague of mine who's given a number of talks about that very topic -- backend developers caring about user experience, caring about the design. She's given some talks. You could probably find her on YouTube. Anybody who wants to talk about it, I'm all over the web as DrewCovi. I think I pretty much have that user name in every platform so if you Google me, you'll find me. CHARLES: We'll look for you. Obviously, you can find us at @TheFrontside on Twitter, TheFrontside on GitHub and feel free to drop us a line at Contact@Frontside.io. Thank you for listening everybody and we'll see you next week.
Robert Jackson: @rwjblue | rwjblue.com Show Notes: 01:00 - Build Tooling in JavaScript 02:19 - Ember and Babel 07:14 - Deciding on Features 11:46 - Class 13:29 - Workflow 14:39 - Payload Size 15:24 - Config Targets 17:18 - Source Maps 25:05 - Ember Decorators, Objects and ES6 Classes 36:07 - What's next and when can we get it?! Resources: Babel.js esperanto Ember CLI Targets
In this Drunk and Retired cameo episode: What's up with Irish and Italian names, and why is the Irish brand so much bigger than the Scottish brand? Also, it seems like there's a lot to learn from 2,000 years of Europeans fighting. Charles and Coté quickly meander through all of this.
Anissa Willyard: @team_giveback | GiveBack2Schools Show Notes: 01:28 - The Mission of Mission Driven Businesses 05:05 - Defining Moments and Leaving a Legacy 11:30 - PPG (Plan, People, Go) 13:20 - Finding Your People 16:22 - Choosing a Mission 22:30 - Defining a Problem 34:19 - About GiveBack2Schools Resources: Boston: Peace of Mind Lyrics Transcript: CHARLES: Hello everybody and welcome to The Frontside Podcast, Episode 74. My name is Charles Lowell, a developer here at The Frontside and your podcast host-in-training. With me from The Frontside also is Ginger Whalen, our head of business development. Today, I'm actually excited about this episode, I'm excited about every episode but I'm excited about this one in particular because we're going to get to see something from the other side of the coin. Our listeners are on the implementation side and sometimes, when it comes to starting up a business or being a founder, the driver and the implementer are met in one person but more often, and I think in a more healthy way, they're split between multiple people serving multiple roles. Today, we're going to interview somebody who is a serial entrepreneur, who is currently a founder at a business that we came very, very close to working with. Please welcome, Anissa Willyard to the show. Hello, Anissa. ANISSA: Hello Charles. Thank you so much and thanks to The Frontside Team and Ginger. You guys are an incredible team and I really enjoy having the opportunity to be able to be here today so thanks so much. CHARLES: No problem. You're currently a founder at GiveBack2Schools, which is going to be launching soon. ANISSA: Yes, that is correct. Our goal is to launch in July 19th. We've kind of pushed it back a little just to make sure we get through the holidays and make sure that i's are dotted, t's are crossed and now, we are locked and loaded. The official date right now is the 19th and we're really excited to launch in Olathe, Kansas. CHARLES: Fantastic. We're going to actually talk a lot about that later on but first of all, there's obviously a large backstory for how you came to be doing what it is that you're doing today. I wanted to kind of delve into that a little bit. ANISSA: First of all, I think right now, we all live in such interesting times that I believe by having a business with a mission is actually is going to allow us as individuals to create such a positive change. The change that we create will actually feel us through as founders or as anybody who decides to move forward to start any kind of company. I do believe there was a quote that I live by. It's one by Zig Ziglar and it says, "It's not where you start but where you finish." To kind of take you through that journey, my hope today is that I give everybody hope, that if I can make a difference, that somebody else can too and that will give them and inspire one person to take that action to make that difference. I look back on everything in my life and kind of where I was in my journey, as far as the why. The why behind what I did. I sat down throughout my day and I broke it up into three different parts. In the first part is kind of called, 'the grownup phase.' That's where you go to school and you come through college and then your middle years where you're out, you're fighting for that career and you really are starting to climb that corporate ladder if that's the road that you decided to go down or raise families or whatever that is. Then the last part where it brings you to that moment, where you question yourself, "When this is all done, did I really make a difference? Am I leaving behind and making a better place here?" That all brings me through, like you said, "Where did it all come from and everything?" From my days with Mary Kay Cosmetics, back when I thought for sure, I was going to be an oncologist and then went out, that's with Mary Kay where I had the honor and the privilege to be trained by the woman herself back, growing up in the Pacific Northwest and having a great role model that just inspired me and be enabled at the right age of 22 years old to have earned a free car. My father thought that was just crazy, that I would sell lipstick so I might as well move on and pursue the corporate dream. Now today, to come full circle and think when it's all over, where am I heading? It kind of in a roundabout way who I was and from working in advertising, to marketing, to implementation, the jack-of-all trades and probably the master of none when it's all said and done. CHARLES: When you say you came to this moment where you had this time of reflection of saying, "Is this the legacy that I want to leave behind?" What is that when you start thinking about the things that you want to leave both to your family and to the world, maybe even long after you're gone? When did that happen and how long did that take and what was the end result of that? What path did that set you on? ANISSA: It's so interesting that you asked. The next part that probably I would share was hopefully I can make a future and take you down that path. I grew up in a very small town in the Pacific Northwest. The population was under 500. My mom actually passed away when I was in the fifth grade and my father was a logger, a timber faller, a real blue collar gentleman. My father knew how to work but he never really had an idea how to raise two small daughters from [inaudible]. He was by far no means a perfect man. He would leave us girls sometimes home alone and he only had a sixth grade education. Some of the defining moment since I look back and had time to reflect is my mom actually, when she passed away, my father was illiterate at that time. I was actually responsible for filling out the paperwork. The name Lyn spelled with a 'Y', not an 'I', my mom's, to this day, even on headstone, it's spelled wrong but it's a reminder that it's really not where you start, it's where you finish. But my mom was a great inspiration in my life. She told me every day, "Someday you're going to be somebody," and I didn't really know who that person was and she had battle with cancer. She was always encouraging and she always taught us girls to have no fear. Even I can share stories about that but then, if I looked back in my life, even though my mom wasn't around and my father was gone all the time, there was a long consistent thing I had in my life and that was school. It really allowed me to dream. School was something I looked forward to every day because they help me escape the reality that I still lived it. There was music and sports and outdoor school fun activities, the opportunity to learn and grow. My family life was so different. If I would have went to college they wouldn't have cared, nor if I would have been that home. It didn't really matter because there wasn't really a purpose. As I was decided to want to move out and went on to climb that corporate ladder, it always felt like there was something missing. I continue to rise to the top and of course, you work hard and you make your money but there was always that missing gap. I found myself in so many ways following in my father's footsteps: great worker, tons of awards, he traveled all the time and never around. For several years, I guess I question myself, "Is this really what it's all cracked up to be in." I don't know if anybody here that listens and have never heard of the music group, 'Boston.' While I enjoy classic rock and I don't know if you've ever heard the lyrics of Peace of Mind and I guess, I'll just open that up because that's not necessarily from my generation but a generation that's older than I am. But it just came on the radio one day and it made me take that pause. CHARLES: You know, I don't know the song out of hand but maybe I can look it up and sing it as the outro music for the podcast. [Laughter] CHARLES: I don't give any opportunities to sing on a podcast but when I do, I [inaudible] that. ANISSA: I would so love to hear it and sometimes, just a few lyrics have really resonated with me as I recall that moment because with my childhood, a lot of people may think, "Oh, my gosh. That poor girl..." I actually had a great childhood. I got to use my imagination and become creative and create something that most people never got to create in their mind. But the few of the lyrics -- just to share with you -- actually did define that moment. It was about five years ago and here is how just a few of them go. It says, "If you're feeling somewhat low about the dues you've been paying, you want to run but somehow, you're just not sure if you should run or stay. You just can't decide on which way to go. I understand about indecision and I don't care if I get left behind. People are always living in competition and all I want is peace of mind." I thought, "Oh, isn't that true? Well then, they kept going." They kept going to the next part and I thought, "Really, is this happening to me?" I'm driving down the road and I feel like this artist too was way before my time is talking to me and just going, "Seriously?" Then it said, "As you're climbing to the top of the company ladder, I hope it doesn't take too long because there'll come a day that it won't matter and that they you'll be gone." And I thought, "Oh, my gosh," and it just hit me like a ton of bricks and I thought, "You know what? Time to make a difference." I remember catching an airplane, flying home that evening and coming home to my husband and saying, "Guess what, baby? I'm going to do it," and that was the defining moment. GINGER: That is so great tip in such a powerful moment like that and those lyrics, isn't that the human condition? We all have encountered that indecision and questioning our purpose or our impact at one time or another and to be able to make a change and then develop this company, launch this company, hatch this idea and we'd love to hear how you then transition into this purpose-driven, mission-driven, cause-driven company. It's very powerful. ANISSA: Thank you. When you decide to do something, like I said that was five years ago, where I made that decision, and then you jump back and you go, "Well, how am I going to do this?" Okay, I have a great idea and I am going to make a difference but then you start to look at this whole picture and this is almost like an elephant sitting there in front of you, "How am I going to eat this?" Really, the next part I call and it's something that I coined to myself is called the PPG. I needed a plan, I needed people and I needed to go. That's really what I live by, was that PPG. The point on when I first started, I knew what the end game was. I thought I have a mission, I have a plan but where do you go? You start writing down all these ideas, I don't know if you guys are like me, I am a left brain, right brain person. One minute I'm analyzing something, next minute I'm trying to create it so I can go back and forth in the same sentence. I'm an oxymoron and I can really create a lot of chaos within my own world really fast where you start to doubt yourself when you think should I move forward. Even though you make that decision, it's like, "I'm going to cut off all options. I'm going to do it." When I first started, I didn't have all the answers to achieve the end game. You know, right now sitting where we are and everything that we do and all the different people we've spoken with, I don't know if you'll ever have all the answers. You just have to follow the heart. GINGER: If there's the cause that you find that mean something to you and mean something to your history. You said school was really important for you growing up as an escape in your safe place. How do you find people that will enlist or support or participate in that same cause? How did you find your people? ANISSA: The really interesting is that two of my partners and I have worked together in a different role in a different capacity for almost 15 years. What's really interesting about it is certain people gravitate together in so many ways. One of them was actually a teacher and a coach but there wasn't really any money to be made in that at that point in time so they ended up going into sales. The other one actually had the marketing background, single mom, raised three daughters, put them school as the pomp squad, head person and then my last partner is actually a former superintendent of schools. It's weird how we were all pointed in the same direction. To have that, I'm more of an outsider. I always say, they're more of a pedigree. I'm more of a mutt that kind of take care of me along the way. But they had similar backgrounds, similar heart and similar care. When she started to collaborate and share ideas, it just grew. I remember back in school where one of the founder says, "I was the child that maybe had to have that free lunch and sat alone," then you hear different ideas and then you put their friends in the room and you start talking with different coaches because they know a lot of coaches and activity director. It's just grows, Ginger like you wouldn't believe in. You get more excited and more ideas flow. I do think how we all came together and how we have all worked in different capacities made a big difference but that synergy of opening it up and being able to talk openly and freely about ideas, really is what grew this to where it is today. GINGER: The partnership is just so strong because you all have the same sense of purpose or you want to make an impact in the same area. Imagine it's really gratifying and you're really connected to your partner in ways that are unique. ANISSA: Right. It's funny because we've really have this saying, "No idea is a bad idea," and I am so glad they have those ideas for me because I'm the one who brings up the off the wall ideas and the thoughts and they encourage it every day. I think to have that synergy to pull that together has been priceless. GINGER: How would you recommend other small businesses if they're not built on a cause or built on a mission but they'd like to choose behind a certain mission. How do you say that you could go by choosing that, if they didn't all come from that background or come from that cause throughout their life? ANISSA: That's a great question. I do think that once you sit down and you get real still, which is not always easy. We have a 27-year old and we have an 11-year old daughter. To really sit still and to get close to yourself and go, "Where am I today?" If there is something that just really bugs me or something out there that is broken or if there's something that if I could fix it would make a difference, I know I'm speaking for myself. I can't solve all the world's problems but if I can make a difference in one or make a difference in one person's life, that they can leave this Earth just looking back, knowing it made a difference. What do you have to lose? A lot of times people think, "Oh, if I do a mission-driven business, I may not make money. I may not be for-profit," or, "What about this? What about that?" Put all of that aside for a minute and just start taking your old sticky notes and writing it down. This actually got even deeper with us. Our daughter play soccer at a real high level. She plays for a select team here in Austin, Texas. At one time, I even questioned myself, "Am I doing the right thing?" I thought, "You know what? I'm going to really get in the saddle to point myself right in those shoes of what our children go through and everything else again." They were doing that fundraising mission to give scholarships to children for balls and to be able to play soccer and I told our daughter and I said, "This isn't necessarily what our business is going to do but it has similar path and I asked that we go out and we try and make a difference and see what it takes and what it does." Once you make that decision, it will grow many legs no matter what it is. Just keep going with it and keep trying new things and keep experimenting. Eventually, your vision will be so clear that no matter what you woke up, this is it. GINGER: Yeah, I love it. It usually does start with one individual with the really strong passion. Also, I agree that it used to be saved for non-profit organization or some philanthropic organization or religious organization but now, corporation is for-profit, public-private, even governmental agencies and they now have a mission and they're seeing for something. I think employees these days demand it too. They really want to make a greater impact, even then just locally, not even one with global compact. They also want to stand for something collectively and going for some purpose. ANISSA: I so agree. People are changing the world by changing our ecosystem in business. It sounds so cliché. It really does. I remember the first time I heard someone actually state that, I laughed. I thought, "Ecosystem in business? What?" That has to be somebody is in a cool buzzword today. Then when you start to define how it all works together, the cliché actually means something and it starts to take a life of its own. Just like you are stating in business today, people demand more. They want to be part of something. People supporting what they helped create. It sounds like such a simple statement but in then encompasses some of the most difficult problem leaders today face. You have to always ask yourself, "Am I solving a problem?" But solving the problem will have an effect and it's just one of those, "What effects will I have?" I believe in my life, if we use such as fundraising, I always look back and think of how much cookie dough between the oldest selling and the youngest selling and all of that and I laugh and I'm thinking, "If all the cookie dough we buy, if I would have actually eaten it all, I would be a cookie today," so I am solving a problem, you know? I don't need no more cookie dough at my house. CHARLES: I think it's an important point though, any time there is a need and a great need, if you can make it exponentially easier for people to fulfill that need, then there's absolutely a business there and there's absolutely a mutually beneficial relationship that can be developed where it's okay to say, "You know what? I'm going to charge some money for this service," but the net effect is going to be everybody gets to participate and this thing is going to make things better. ANISSA: I so agree. You guys may find this funny but I believe podcast is something that make a difference in people's lives too. I really enjoy running. I run between five to seven miles a day and there's not a day that I don't turn on something that I listened to or I dive into. I do think the sharing out there today has made the difference. Just by you guys doing your podcast or simple things like that, it does bring that unity together in so many ways. GINGER: But we do hope to inspire and we are inspired by doing this, definitely. ANISSA: Oh, I can only imagine, definitely. But back to a little bit about people do help support what they help create. I think when you can define a problem, I know sometimes that does get deep and sometimes you may think it's shallow and sometimes when you start writing it down, you don't really know what the real deep problem is. You just start with a lot of different parts. You may go through days that you think, "Does this really matter at all?" Really does it? But deep down inside, that's your fuel. That's your passion that keeps you going. It also may take some time. When I first started out back years ago after my mom passed away with breast cancer, I thought, "I'm going to be an oncologist," so as I was going to college and everything, I realized that I really don't enjoy draw on your blood and dealing with all of that. I thought, I am more of that people person than I am sitting there. Even though they have a life and passion and I learned so much from that experience, my mom used to have a great, great saying and she used to always say, "Anissa, there's a lot of dead folks on this Earth. Don't be one. They're living but they're dead. Don't be one." That never came to me until many years later in life what she really meant. She meant live life with passion and go for it and never look back and I thought, "There is some to that saying, just don't walk around like a zombie. Be someone." That was when you bring it back, to define that problem like I started with, you may not even know what your problem is but you know that deep down inside. You're not excited to wake up. You're not excited to get busy at something because you're not being fulfilled and you know that somehow you did not make that difference. GINGER: Yeah, I think that's the bottom, like figure out what you want to stand for, figure out how to make an impact besides yourself and beside your community, more than global level and get behind it, get passionate about it. You keep doing that. You listen to your message. You listen to your mom. ANISSA: I did. I did. There are so many times I didn't listen to my mom and I don't know if it turned out good or bad because I've made a lot of mistakes. GINGER: We all have. ANISSA: Right. You know what, if you take that forward, you just challenge that problem from every angle. Who do I need to help solve this problem? Do you need your merchants, your consumers? Who is the [inaudible]? You may think there's a problem but they may not even know they have a problem. You know, sometimes people are in such denial. Others are not a problem and you're thinking, "Oh, you have a problem," but that's kind of where I come from and to do that. Then when you have a vision and you have people, there will be people who will not support you. They were actually say you are crazy. In that sometimes, people can take that internally and go, "Am I crazy?" Just know that that's a true compliment. If you're not always speak in everybody's language and people think you're in left-field sometimes, that's okay. Don't worry about that. The other part is when you start out on your own, you're going to hit roadblocks. Anybody who starts the business will tell you, you are always, always 100% never for sure and willing to change, no matter what as you continue towards your end game. But really, I live a lot of my life by a movie. The movie, my favorite movie of all times is The Wizard of Oz. I really do think it has several meanings to it. It does take a brain like the Scarecrow. If you're not the brains, get somebody who has those brains behind you. I always knew that when we were going, I needed somebody who has a lot smarter than I was in so many ways. It takes the heart. I've got a great heart and compassion and all of that but it does take that heart. Maybe you are more of the brains. Find somebody with that compassion at heart and it takes courage. I know that your team are there to build what you guys have and to do what you've done. It took a lot of courage. I think anybody today who is in business, I do think the mission sometimes is what get us through when we are in business because without that, there really is no end game. GINGER: Although all of us are going to start businesses based on a cause or a mission, there's a tons of ways you can get involved doing volunteering. Then you're involved with like-minded individuals chasing some cause, accomplishing something but those lessons can be an individual level, group level, company level. I completely agree. CHARLES: Yeah and I think it's important too. It's thematic on this podcast of don't go alone. I think we have some very toxic myths out there of the people, this person, kind of founder who locks himself in a room and then they sit there, eating nothing but pizza and Dr Pepper for 48 hours and they emerge with the solution to everything. That's just not how it works. Time and time again you'll see that what really makes a robust, resilient business is exactly like you said: having people who can truly fulfill each one of those roles, the brains, the heart, etcetera and really step into them completely. It's a great point that just can't be repeated enough. ANISSA: Oh, I agree. I don't know if my family would even be my family if I didn't surround myself with good people. If I lock myself behind closed doors, I may come out crazy, literally. They would not want to be around me. GINGER: Again, we can all say that at times. ANISSA: But Charles, to go back on your point, don't take this alone guys. There are people out there that have the different parts that maybe you're missing. I'm telling you, there is strength in numbers. I don't mean numbers like 10 or 20. That's great but when you're really collaborating and getting down in the nights that we worked and the early mornings and the times, we were told basically no. When you're out there fundraising and you're trying to get the money, to go where you need to go, to continue your own way, not everybody is excited about you or what you're doing. You'll have to have those partners to keep you going at every turn because they will make the difference. There are times where it gets really dark and that darkness can be just one more knock or one more call to make it happen. I encourage that spirit together. CHARLES: Yeah, I think there's also another point that I want to bring to the forth and while we're on the subject of toxic myths, I think we have also this myth and it is prevalent of the idea of startups and people embarking on businesses being very young, basically young men and I think that while young people have a lot of time on their hands, the people that you know and you can cultivate over a lifetime. They are the true asset. You have people with 10, 20, maybe even 30 years under their belts who have developed these networks which are just these -- we're from Austin. I love trees and I love the trees in Austin and one of the things that makes live oak trees, which is a very hardy native tree that survives around here, it goes through periods of sometimes it rains a lot but sometimes we go through five, six years where it doesn't rain a lot. What happens is these gigantic oak trees have these network of roots that intermesh and intermingle underneath, sometimes 10, 20, 30 feet under the ground. If you have that time in your life and you've spent that time being around extraordinary people, that will come to fruition in terms of you'll be able to grow new trees based on that shared interlocking network and the strength of that network is really a true asset. I think that another message here is that people who are advanced in their careers are almost the opposite of saying that a startup is not an option for you. It's almost like you're going to be better suited for it in many ways. ANISSA: Oh, I love that analogy. That is such a great analogy. Looking out in my backyard here in Austin, we have oak tree. I'm going to remember that every day. I agree so much. That was probably one of my biggest fears. You look at some of these great companies that are born today and come out and you're thinking, "Gosh, have I missed my prime?" Also, being a woman in business, sometimes I've been the lady in the boardroom. I've been the one that has earned that executive title. To climb that ladder isn't easy but to be a woman in business sometimes, also makes you stop and wonder, "Is anyone going to listen?" I think that the wisdom in business that we have, the so-called bloody knees from falling down and getting back up, the balance back ability, that in order to make those things happen, it was a [inaudible]. I just appreciate the fact that there are other people out there just like me who maybe want to make a difference and don't know where to go. Just realize that there is more than just me out there that has this feeling. You just need to take that chance and make it happen. GINGER: Or classic rock or classic movies. ANISSA: Right. No doubt. CHARLES: We talked about how to organize a business around a mission, I'd like to actually take a moment just a couple of minutes to talk about GiveBack straight on and let you explain because I think it's just a great concept. ANISSA: It's really funny. A little bit about GiveBack. GiveBack was actually born from an idea of being sick and tired of fundraising, if you can believe that. During the time when I was a child growing up, fundraising really hasn't changed for our educational system. Everything gets cut. I remember looking back and being able to sing songs from The Beatles, Little Submarine and all of those things. Nowadays, you don't have those things anymore. We always get these different, "Oh, got to sell cookie dough, got to sell this." In our time as a family, I was already at the end of its rope. I'm traveling. My husband's got a great career and our oldest son was actually raised by a full-time live-in nanny. Now, that's not something I'm excited to brag about but it happen. As we had our second child, we were heading down that same road. When it came to fundraising in schools and all of that, I thought how many kids out there don't even get to go to outdoor school. What we did, the concept and the idea was how can people just live and we give. That was really how it was born. From my marketing background, from working with businesses, people want to make a difference. But yet, there's really no trackability on their ROI. Yet, they also need the cost to get new clients in so there is that cost of acquisition, bringing new clients into their business. Schools need money. In order to keep going for these athletics activities and all of this and how can I tie this together that the consumer can dictate wherever they are in America that they can and maybe someday global, give back to their child soccer or maybe the chess team at the middle school or the high school. Maybe they have three kids so they're not giving up their precious time, their own resources, to go out to a school function night, when it's only Tuesday, service is poor. The restaurant wants to help but it's just not working into our time and who has the time? I remember the wagon full of cookie dough and taken it door to door and I was thinking, "If we don't get it out, it's going to spoil in our garage because our son had to hit the record and sell the most cookie dough. Yay! Great job. Now, Mommy's got to go deliver it with you." It's eight o'clock at night. We're driving around in the car, knocking on people's door to give them cookie dough. Then what percentage was really going back? That's kind of what we did. We connected the dots and we brought fundraising or funding of those activities at education and all of that into a way that it would just basically, as we always say, you get to live and let us do the giving. GINGER: That's great. Then people can really know what kind of impact they're making, how much they're giving back. I think that was the missing piece. People got frustrated in fundraising is it feels kind of good and giving to the cause, I support but what is it really doing and how much is really going back to them? Those were open questions back then. ANISSA: Right. Believe or not, they're still open today. Here we are going into the Year 2018 and they're still using envelops out there and who knows where that money's going or what percentage went back. You hear about that all the time. You see it firsthand. I agree with you and that was really something as that transparency to be able to show. You know, the school was gone, now what you did and how much went to this activity? Also, it was that full circle transparency and boy, doesn't that make a difference and to be able to let people start to dream again, if we have instruments maybe these kids could actually play them again, versus, "Oh we've got a company [inaudible]. Forget about outdoor school. Let's just go to more textbooks." It gives us a different way to bring back to our schools. GINGER: You want to talk about this going national at some point for people that are in the Austin area? ANISSA: To be honest right now, our end mission was to do what was right. As we soft launch and get out there, yes our goal is to go nationwide. We're looking more towards about the scalability but I would rather under-promise versus over-promise and not deliver. Our goal right now is just to watch for upcoming events. It was hard for me to keep it under wraps for so long. Now, we start to leak it out a little bit. I think probably the best way to put it is stay tuned and keep your eyes open. That would probably be the best verbiage to use at this point but I thank you guys for your hospitality and allowing me to do that. CHARLES: Oh, no. It was our pleasure. It was great meeting you. It was great getting to meet other members of your team. You come across folks in the line of work that we do and you all really stood out as folks who knew what they were about and we're on a mission to achieve it. I think it's important for people to hear that that you can do that, really at any point. Thank you so much. Thanks for GiveBack. ANISSA: Thank you. I just really want to say thank to all of you and I want to wish you a Happy Fourth and safe and good health out there.
Kris Van Houten: @krivaten | krivaten.com | Q2 Show Notes: 00:55 - Kris' Interest and Passion for Accessibility 06:07 - Using Ember for Accessibility: Pattern Adoption 10:13 - Context Switch Awareness and Managing Focus 12:08 - Asynchrony and Desired Interaction 14:04 - Building a Form Input Component 19:05 - Things That Are Hard to Catch 22:41 - Assistive Browsers? 28:17 - Making Things Accessible From the Start Resources: Building for Accessibility by Nathan Hammond @ Wicked Good Ember 2015 The A11y Project: Web Accessibility Checklist WCAG 2.0 checklists Why Don't Screen Readers Always Read What's on the Screen? Part 1: Punctuation and Typographic Symbols Mozilla Accessibility Kris' Blog Post Series on Accessibility: Part 1: What is accessibility and why should we care? Part 2: A Primer on Accessibility Part 3: Getting Our Apps Ready for Accessibility Part 4: Building an Accessible Icon Component in Ember Part 5: Building an Accessible Input Component in Ember Part 6: Building an Accessible Alert Component in Ember Part 7: Building an Accessible Numbers Component in Ember Part 8: Building an Accessible Currency Component in Ember Transcript: CHARLES: Hello, everybody and welcome to The Frontside Podcast, Episode 72. My name is Charles Lowell, a developer here at The Frontside and podcast host-in-training. With me today is Wil. WIL: Hello. CHARLES: Hey, Wil. Today, we're going to be talking about accessibility in single page applications with Kris Van Houten who is a developer at Q2. Hey, Kris. Thank you for coming on the show. Today, we're going to talk about something that I know a lot of people's minds here and probably elsewhere on the internet, it's a topic that's getting a lot more attention, which is a good thing and that's accessibility. We're going to explore the niche of accessibility as it applies to single page applications. Now, you're a frontend developer at Q2, what initially got you interested in and passionate about accessibility in general? KRIS: I honestly feel my path to passion in this area has been a little bit unorthodox in a number of ways. I basically started out in total apathy of this topic and over the last year, it has turned into a genuine interest of mine. About three years ago, I remember listening to an episode of ShopTalk Show with Dave Rupert and Chris Coyier and they kind of went on this large rant about accessibility and why more developers need to be concerned and compassion about it. Dave Rupert was talking about his contributions to the accessibility project and I'm sitting back and thinking to myself and this is back then, obviously, "Why would anyone who is blind want to use anything that I'm working on." I basically balked at the idea and disregarded it entirely. At that time, I was just getting my feet wet with Ember working on an application with a company here in Cincinnati and we had these conversations about, "I notice that we put this action or a clickable event on a div element, should we not be doing that? Is it that not something that we should be doing?" I remember sitting back and having this conversation and saying, "The ads been crawled by SEO and Ember isn't yelling at us for doing it. It still works fine so what the heck? Let's just go with it." Basically, every single app that come into since then has basically adopted that same mindset even before I joined the team so I know it's not just me who is thinking this. A lot of developers that have been exercising the same way of doing their code. CHARLES: Right, it's the path of least resistance. Everybody's got a job to do. Everybody's got features to deliver so that practice can be very easily self-perpetuating, right? KRIS: Exactly and I think a lot of developers just don't understand the semantic difference between a div or a label or a button or a link and how browsers can actually treat these difference HTML attributes or HTML tags differently because of how assistive technology can utilize them for per person's benefit. That's where I was a little over a year ago basically. When I first started at Q2, that first week, I got pulled into a discussion about design patterns which is another passion of mine and somehow, that turned into me joining a group that was to establish to figure out how to tackle the task of making our large app accessible. Basically, we had a company come in, audit our application and we got a big fat F for accessibility so it's something that we said, "We need to start tackling this problem." Being that, I just started at the company that week, I was going to tell them no but internally, I was panicking and saying, "I got to figure out what is this whole accessibility thing is and why it's important." I started looking for books, articles on the topic and trying to basically flood myself with information. Two things that really transformed my way of thinking was actually a talk given by Nathan Hammond at that Wicked Good Ember in 2015, where he shows an example of building an application without accessibility in mind so basically, doing what I was doing before which is we're adding actions to div tags, we're not really caring about semantic HTML, we're just making the feature or the application work. But then what he does, which I think is super powerful is he pulled up a colleague of his who is blind and had him try to use the application. He just goes through and you can see the struggle and he's actually vocalizing and talking about where he's [inaudible] with this application. Long story short, Nathan comes back up and makes a few adjustments. DHTML has [inaudible] up again and it's night and day difference just by changing the markup and by dropping in the Ember A11y add-on which helps with focusing the browser in certain areas of the content. He's able to totally transform how's individuals able to use the app. For me, that was a super powerful to come in and see that and see someone actually struggle with a website that they were trying to use. I think, [inaudible] where I always saw accessibility was it will only affects people who can't see and I think that's the other area where I've really started to have that paradigm shift was when I realized that this isn't just people who can't see. It's for people who have motor difficulties, who can't use a mouse and how to use a keyboard instead. People who have various vision issues, whether that's cataracts, colorblindness, glaucoma, dyslexia, some in these effects, not just DHTML but also affects color contrast, the fonts that we're using that impacts every area of application design and development and that's where I started to realize that that was where the paradigm shift happened in my mind where I started thinking to myself we really need to start talking about this more and getting other developers on board in general on this. CHARLES: It can be intimidating, especially when it feels like on a single page application, your divs have to do more, so to speak in the sense that it feels that your HTML is fatter. It's not just a thin layer but your HTML is actually part of the UI. KRIS: Exactly, yeah. CHARLES: When it comes to having this paradigmatic shift that you're describing, when you're looking at your single page applications, are there any insights into the general structure of the application that you feel like you've gained that are foundational, they kind of transcend accessibility? I guess, what I'm saying is, is there any way that you become like a better developer or been able to recognize foundational patterns because of having these insights surrounding accessibility? KRIS: I've been working with Ember for about three or four years now, basically since it was still in beta. Over the last several years, I have started to accumulate a lot of knowledge as to how we can utilize Ember to do a lot of the heavy lifting for us. When I started getting more passionate about the area of accessibility, first question that came into my mind are how can we use Ember to do some of the heavy lifting for us. For example, some of the things that I had done was go through and start working on developing a couple of components that basically cover a lot of things that I find ourselves doing [inaudible] a lot. Whether that might be a component to just plain icon on a page or a component to display input on a page. What we're able to do is using Ember, we can say, "Here's the icon I want to display but if I don't happen to pass in an aria-label attribute, for example. The component will add the 'aria-hidden=true' for me. Being able to really utilize the power of Ember to do some of that stuff for us on the back side of things, I guess you could say it magically. CHARLES: Let me stop you there for a second and unpack that example. What you're saying is, if I'm going to put an input on the page, if I actually don't assign an ARIA role, it's going to hide it from me? KRIS: No. I was thinking of an icon components, say if I'm using Font Awesome, for example and I want this with the trash icon so I wrote a component for our specific icon library that we're using. We pass in the icon that we want to display, again that could be the trash icon and we can also pass in an aria-label attribute to add a label to that span that will be read to the user. But if we don't pass that attribute in, the component will automatically add the 'aria-hidden=true' attribute for us so basically it skips over it. CHARLES: Yes so it won't be just garbage for a screen reader or someone navigating with a keyboard. WIL: Yeah, otherwise the screen reader tries to read the content of the icon CSS which is just the Unicode. CHARLES: Right. KRIS: Yeah. What we really is trying to figure out and what I've spent a lot of my time, especially in writing my blog series on this was while we are using React or Vue or Angular or Ember or whatever, how can we utilize the power of the single page application frameworks to do some of that heavy lifting for us in the background without us needing to explicitly define everything. I'd say, especially when you work on a large team like what I work on currently, we can't expect everybody to be extremely well-versed in the area of accessibility so if we can do some of the work for them and just encourage them to adopt these components in their daily workflow, it does some of the work for us. That's what we're working on and talking a lot about at Q2 is basically this pattern adoption. CHARLES: Right so it sounds like to kind of paraphrase, whether you're working in any framework most of them have this concept of components so really leaning hard on that idea to make components at the very granular level aware of their own accessibility. Is that fair to say? KRIS: Yeah, obviously there's more I'm sure as we go for the conversation about some of the things that I've tackled in this area but long story short, being able to utilize and recognize, you have this extremely powerful JavaScript framework at your disposal to do some of work for you so why not equip to do just that. CHARLES: Yeah. I guess that falls into my next question, which is there are component level concerns and if there are other component level concerns, I definitely want to hear about them but what immediately leaps to mind is there are also cross-cutting concerns of any single page application, what's the state of your URL and if you're using a router. Some of the content on the page is going to be changing and others isn't like how do you cope with that? What are the cross-cutting concerns of an application that span components and then how do you cope with them? KRIS: I think one thing that comes to mind as you're talking is the whole area of context switch awareness. If I click a link, if I go from the home page of an application to my profile page, how does a screen reader know that that content has now changed to present this new information to the user? I know what we were able to do was we were able to drop in the focusing out with component that's put out by the Ember accessibility team, which basically whenever we render to an outlet, that's utilizing this focusing outlet component, it will focus the browser to that main area and start presenting that information back to the users. One area that was at the top of our list as we start tackling accessibility was we need to figure out this whole context switch awareness thing because -- this is back then obviously when we first got started -- back then there was no way for a user to know when the page changed so they would basically be sitting there, waiting for any kind of feedback or whatsoever to be presented back to them and it just wasn't happening. I would say, managing focus is probably one of the top level concerns when it comes to single page applications because it's a single page application so if you click a link, the page isn't completely refreshing, prompting the screen reader to present the information back to user. That's one of the key areas that I think of. CHARLES: What about things like asynchrony because a lot of times, these context switches are not boom-boom, one-two. The content on which you want to focus isn't available yet. Usually, the analog from a visual UI would be a loading spinner or a progress bar. How do you deal with those to say, "Your content is not quite ready. If you're made to wait it's because we want your content to be of the highest quality." KRIS: Sure, yeah. We were able to drop in the focusing outlet components in our application and it took care of a good chunk of the work but it seems like in our application, we're doing something that might not be as conventional as the rest of the Ember community would like them to be so we might not use the model hook as we should. It's hard for the page to know when the contents actually ready, when it's been rendered to the DOM to present back to the user. One thing that I'm currently trying to tackle right now, to figure out how we can remedy that problem. I probably say, honestly that's the challenge I'm working on right now. I don't have a solid answer to that one at the moment. CHARLES: Irrespective of how it plugs into the tool that you're using, what would just be the desired interaction there, regardless of how you make it work? KRIS: I guess, conceptually what I'd be thinking about is how can we notify the user we're loading content right now and whether that we have an alert box that has the ARIA alerts, basically attributes set on it, that we could pass in new, basically notifications to it to let the user know, "Loading content. Please wait," and then once that content resolves, focus them on that main outlet where the content has been displayed to read that content back to the user. That's how we're trying to think about tackling this issue but we haven't have a time to implement it to see how it's going to work across all the different avenues of application. CHARLES: I did want to come back at the component level. are there any other ways that you can lean on Ember or lean on React or lean on Vue, if you're using a component or in framework, just talk a little bit more about how you use those to unlock your application and make it more accessible. KRIS: One thing I can think of is a way that we can enforce better usage of the framework that we're using is one that comes to mind is a component that I worked out in the blog series that I wrote was building a form input component. Especially, when you're trying to write an accessible app, I think about how can we enforce certain patterns when other developers come in later on and want to add a field to a form or use this component somewhere else in the application. What are some ways that we can enforce that they're doing everything, using the component correctly so that way it renders accessible mark up? What I tinkered around with and we actually just landed in our application is basically a form group component to where we pass in, obviously the value that the input is bound to. But we also pass in a label that is tied to the input and whenever you hit save and the app goes to refresh, if you don't pass the label, there's an assert statement that basically fires up an error into the console and lets you know, "You're trying to use this component, you need to pass into label attribute for the purposes of accessibility and here's the instructions on how to do it." We've been kind of toying around with this idea of enforcing patterns because again, we have several dozen developers at Q2 that are working on this stuff and they're not all wizards when it comes to accessibility but how can we gradually start getting them to the place where they're adopting these patterns and best practices. I'd say, doing things like that, we are enforcing patterns in the usage of the components as well is really a key. One thing that we implement it in our testing framework is the use of a Deque Labs' aXe engine to basically go through, we can pass it a chunk of HTML and it will give us any suggestions that it has to make that content more accessible. We're using that in our test library right now, in our test build and encouraging developers as they write new components, as they go in and modify components to throw new snippets in to make sure that the content that's being spit out here is accessible and then submit your PR again. Just trying to be more hands on in that way. CHARLES: So you actually running a GitHub agent or something that's actually in the same vein as your test suite or if you're taking like snapshotting with Percy for doing visual diff so you're actually running a third check, which is an accessibility check? KRIS: Right now, we were able to land the aXe engine into our test build a couple months back so we're just slowly incrementing that over time. We have a couple challenges in the way of getting Percy implemented but that is in our list of goals to have that running as well. But one thing that I really like about aXe engine in particular is that if your check fails, it refines improvements that you should be making. The nice thing about it is also spits out a link to a page on Deque Labs website. They give an explanation of what have found and basically educates your developers for you. To me, I think that's huge because again, we can't educate every single developer and expect them to be pros at this but we can utilize tooling like the aXe engine or the [inaudible] Chrome extensions or stuff like that to do some education for us. As we work towards automating this further and further by using the aXe engine in our development side of things or using Percy on the test build as well. See, there's all kinds of stuff you can do but that's where we are right now. CHARLES: I really like that idea because in comparison with what we talked about at the top of the show, about how there's this path of least resistance that developers will follow quite naturally and quite rationally, which can lead to not accessible applications. It sounds to me like what you're doing is a establishing the same path of least resistance but having that path guide you towards accessible applications and saying, "This path of least resistance thing, it can be an asset or a liability so we might as well make it an asset." KRIS: Yeah, for sure. We sit down once a week and we talk about whatever challenges we're trying to work through in terms of accessibility. We have a weekly meeting where we sit down and talk about it. I thought one of the key topics to those conversations is how do we get the other developers that are not in these meetings more aware, more informed and more up to speed with this that they care about it, that they're working on it and it's part of their inner dialogue as they're writing out new features that are going to be deployed out to our clients. Lots of challenges there. CHARLES: Yeah. We've talked about some of these problems that you catch, you're actually writing some assertions there on the test build so you'll actually fail if there's certain requirements that aren't met but what are the things that are more intangible? How do those come up in terms of accessibility? What are the things that you can't catch through automated testing? KRIS: Right now, some of things that we're having a hard time testing which Percy will help once we get that implemented is contrast ratios and stuff like that. That's one of the key things that comes to mind for me when I think of the things that are a little hard to catch. I think another thing that's hard to catch, especially at the aXe engine and stuff like that, won't necessarily catch is the flow of your dialogue. When I turn on a screen reader and it starts reading back this page and content to me, sure we can make it so that it doesn't read out the icons character code and a lot of stuff. It presents the information we want back to us but I think, having that information presented back to the user in a way that's legible, that makes sense to them is probably one of the bigger challenges that I've been working on a little bit. One that comes to my mind is like the reading of currencies or numbers. One thing that I found way helpful was Deque put out a very thorough article on how the different screen reader like JAWS, NVDA, Apple's VoiceOver, how they read different types of punctuation, different types of graphics symbols, how they read [inaudible], $123.50, what does a screen reader actually read back to you. That's where I've actually been spending so much of time lately is building on some components that instead of reading back what the streaming will read back by default, which should be, "Dollar sign, one-two-three-five-zero," having actually read back, "One hundred twenty-three dollars and fifty cents," so basically, writing a series of components, I would do some of that, again heavy-lifting force, in that way, our developers don't have to go in and manually add-ons aria-labels obviously. That's been a nice little challenge where something that's we are working on just testing right now and making sure it works right if there is any downsides to doing this but I want a person using a screen reader or other types of assistive technology to hear the information as I'm thinking about it. When I see $123.50, I'm thinking in my head that's, "One hundred twenty-three dollars and fifty cents," not in single digits one right after the other. Those are things that a lot of the automated software isn't catching. It's not catching like, "Your grammar is bad," or, "This isn't making any sense to me." It is catching like are you passing in or applying the attributes to HTML elements that you should be. Are you using semantic structure in your headings and stuff like that?" I think that's one of the areas where developer is need to get their hand dirty, turn on the a screen reader or use any array of different voice-over tools to actually listen to the content being present back to them to see how it's presented. CHARLES: Yeah, it's almost a difference between a syntax error versus a runtime error like we've got a lot tools that can catch the syntax errors and you can put those in and catch where you have something that's malformed but some sentences can be perfectly formed but make no sense and it takes a human set of eyes to make sure if that content is coherent. One of the things that if you're going to ship applications to people, you need to be able to try and measure as closely as possible the environments in which the people will be using your software so you can actually have an accurate measure of whether it works or not. For example, in the Ember world lately in the stuff that we've been doing with acceptance testing in React, we admit people are going to be using a multiplicity of browsers to access this application so it's very typical to use Testem or use Karma to fire up five different browsers, which if you're using BrowserStack, you can do fifty. You know, people are going to be using IE8.1 on Edge or on a Surface. They're going to be using Safari. They're going to be using Chrome and those often surface those issues but I feel like there's no access to the actual screen reader and assistive technologies to be able to make real assertions against those things. I imagine that it would be cool if there was some way that in Testem or in Karma, you could have one of your browsers be like an assistive browser that you could actually assert, I want to assert that it read it as, "One hundred fifty-three dollars and twenty-five cents," and is that on the horizon? Is that even possible? But it seems like something that we have to shoot for if we actually want to measure that these things are working if we actually want to capture data points. KRIS: Yeah, I totally agree. If you look at the documentation on W3 for how these different HTML attributes should be treated by the browser or by the assistive technology, long story short is this is not how -- in several cases -- certain screen readers are presenting the information back to you. It's not how it's treating the content. That's again, one of the areas I thought was way interesting about that. Deque article on punctuation and typographic symbols, which is like we should expect that this software is operating at this level to present this information back to user in such a way where it understands what the dollar symbol in front of a series of numbers means but it just isn't there yet. There's still work to be done. I'm hopeful for the day where our screen readers are a lot more powerful in that capacity. One that makes me a lot more hopeful about that is I don't know if it's just because I've been more interested about this over the last year but it does seem like I'm seeing a lot more people talk about accessibility. I'm seeing Apple putting out videos, talking about the efforts that they're making to make their software more accessible. It does give me hope that there's a lot more visibility on this now. There's a lot more people fighting for this cause to cause these companies to come back and say, "We're going to put more effort into this. We not just going to make a standard screen reader and ship it and just leave it there for five years and no one was going to touch it," but, "We're going to start making improvements." One thing that I did notice just over the last couple months even was that out of nowhere, we use Apple VoiceOver in Chrome, which isn't typically how people use it. They typically use it with Safari. But if you use in Chrome, it will actually read back to you as, "One hundred twenty-three dollars and fifty cents." When I came across that, I was kind of dumbfounded but then I was thinking to myself, the vast majority of people who are using screen readers aren't using this browser but that's really interesting that they're doing this now. I dream of that day where we can basically run a series of mark up through in a test or into a function and basically have to spit back, here's how screen readers going to present this back to you. I'm hopeful for that day. CHARLES: I'm wondering now like why don't major browser vendors, why is this not just a piece of a puzzle that comes when I download Firefox. Firefox has access to my speakers, why isn't there a web standard for how screen readers will treat content? Maybe there's an effort under way. KRIS: I sure hope so. Looking through documentation, we know how things are supposed to work, how we've agreed that they should work and now basically, we're just waiting for the different browser vendors and Microsoft and Apple to make the updates to their streaming technology as well as JAWS and NVDA. I'm hopeful that these changes come soon. These are improvements to the interface. CHARLES: Yeah. Any time there's a gap, you can see that's an opportunity for someone -- KRIS: For sure. CHARLES: -- To write some software that has some real impact. I know certainly, I would love to see some way to roll these things into our automated test suites. KRIS: Yeah. I searched for it but with no avail and it's a little bit beyond my knowledge of how to build something of that caliber. I hope someone else does it because I don't know how. [Laughter] CHARLES: Well, maybe in a year, maybe in two years, maybe in 10, although hopefully a lot sooner than that. KRIS: Yeah. I would judge that at the speed of things were going right now, I'm optimistic that we're going to have some much better solutions within the next year or two on this field. Especially of how much I'm seeing people talk about it now, how much it's becoming a part of the regular conversation of web development, application development. I'm really optimistic that we're going to see some strides in this area over the next couple years. CHARLES: Okay. With the time that we have left, I'm going to ask one more question. Kris, there is something that I wanted to ask you, which is let's say that I am a developer who is working on a team that is maybe it's big, maybe it's small. I've got an application or I'm starting an application and I have a desire to make it accessible. How do I establish that path of least resistance? What advice do you have for someone who's just about to take the first step on that journey to make sure that they have the outcome that they're looking for which is the most accessible single page app that they can have? KRIS: I think it's a great question. I would start out that answer by simply saying to encourage you to be somebody who cares enough to speak up and become an evangelist, become an educator and become an enforcer in your workplace for this work. You don't have to be the most knowledgeable person in the world on the topic. God knows I'm not and I still there were people come to me, asking me, "How do I make this feature, make the guidelines, make it accessible to screen readers," but I'm passionate about this topic and I'm interested in learning as much as I can about those. Step one, just being an evangelists for it. Be interested in it, care about it. I'd say, the next thing is just learn more about semantic HTML. I would say from a lot of the things that I've been trying to tackle with the application that I'm working on, just simply writing semantic markup takes care about 80% of my challenges. In just understanding what are the different elements, what are the different tags are for and how screen readers and other assistive technology see those things. To get started, I would say there's beginner, intermediate and advance stuff. I would say go to the accessibility project, which is just A11yProject.com and read through the content there. It's very entry-level. You can probably read through most of the content within an hour or two and really start to get a grasp as to what level of effort you're looking at in terms of your application. Once you get through that, if you still want to learn more, I'd say go over to Mozilla's developer network -- MDN -- and read through their documentation. On the topic, there is a little bit more exhaustive but it's still really easy to read and really easy to grasp. Based on a content they have shared there, I'd say more of an advance level is actually go through all the documentation on the W3. It's a lot more verbose, it covers a lot more of use cases, it has a lot more suggestions and just stuff ready to go over. I'm still working through that information. There's so much of it but I would say that's as a good place to get started with understanding the different attributes, what they're for and just the importance of writing semantic HTML. I would say some definitely good things to start tinkering with to find some of the low-hanging fruit in your application would be to use some of the assessment tools that are already out there. You have the [inaudible] little JavaScript snippet that you can put in your Chrome favorite's bar or you can use the aXe engine or if you even have an aXe Chrome extension that you could pop up in your application to basically give you report on some of the areas that you should be looking to make some improvements. I think it's important to view accessibility kind of like how a lot of bloggers view SEO, is that there's always more work to be done, there's always improvements you can make but the key is to take those first steps and start making those improvements. One of the nice things about the accessibility project and there's a couple other websites out there that have some of the lists, they basically have a checklist for you to go down. If you're just getting started with accessibility, they have a checklist of all the first things that you should be covering to get your app started in that realm, to start making those improvements. I know you guys do links in the show notes. I can definitely send you those things to those items to get people started. Another thing I find myself doing a lot is while we're talking about something in our chat at work in or just go off in the code pin and mock something out in HTML and then see how the screen reader reads our content back to me and then kind of tinker with it and do a little bit of self-discovery in how this all works together. There's a lot of options out there. I know just threw a lot at your listeners but I'd say, it all starts with being someone who cares about the topic and cares enough to start asking others to care as well. CHARLES: I think that's a fantastic answer and a great note to end on. But before we go, obviously we will include those things in the show notes but also the other thing that we're going to include is a link that you actually, I understand, have a series of blog posts related to all of the things that you've been talking about, which we'll also include. KRIS: Awesome. Thanks. CHARLES: Everybody, go read it. Thank you so much Kris for coming and talking with us about accessibility. I think you're right. It is a topic that's gaining a lot more traction and a lot more mind share in the mainstream that can only be a good thing.
Audrey Eschright: @ameschright | The Recompiler Show Notes: 00:50 - Background in Publishing and Open Source 06:53 - The Contributor Pool 12:37 - Open Source Bridge 15:29 - Mistakes Open Source Contributors Make 17:21 - Tools for Maintaining an Open Source Project 19:09 - Roles 23:33 - Open Source Bridge (Cont'd) 27:47 - Governance and Decision-Making 36:20 - Making Open Source Accessible, Safe, and Welcoming Resources: Free Geek Calagator PDX Activist Dreamwidth Safety First PDX Open Source Bridge: Enter the coupon code PODCAST to get $50 off a ticket! The conference will be held June 20-23, 2017 at The Eliot Center in downtown Portland, Oregon. Transcript: CHARLES: Hello, everybody and welcome to The Frontside Podcast, Episode #71. My name is Charles Lowell. I'm a developer here at The Frontside. With me also is Joe LaSala. JOE: Hello. CHARLES: Hey, Joe, another developer here at The Frontside. With us today is the publisher of The Recompiler Mag and a long-time open source contributor Audrey Eschright. Welcome Audrey. AUDREY: Hey! CHARLES: Thanks for being on the show. AUDREY: Oh, thank you. CHARLES: Today, we're going to be talking about open source and in particular, the labor that goes into open source and making that sustainable but before we get into that, I wanted to first talk about your background, both in terms of how you came to be publishing the magazine and also your background on open source, how we're arriving at the subject today. AUDREY: The magazine, in a lot of ways, I refer to it as a feminist hacker magazine. It holds together a lot of different things that I've worked on over the years so I'm going to jump all the way back to when I first encountered open source and then maybe that will fit together. When I was in high school, I first encountered the internet and the internet that was available to me at that time use things like Gopher. Gopher is a pretty web protocol and it was free software. I didn't really understand that it was free software at that point but I did understand that if I wanted to learn how to write code and the computer that I have access to were things like a bunch of really old PCs like 286's and an old Macintosh. Then there were commercial compilers for writing code and there were free compilers for writing code. There was a thing called GCC and I knew that it was on university computers and if I got access to those, then I could write code. Then I got to college and write about when open source really started to take off as this concept of how free software comes into business world. I've had that as a background of becoming a programmer and getting involved in things but after college I wasn't really sure that I want to work in technology so I took a break. When I came back, I needed a way to get myself up to date so I started volunteering with this local group called Free Geek that recycles computers. What they do is they take those computer parts and the ones that are usable, they build them into Linux boxes for people, like Linux desktop boxes. How I got back up and running was learning how to work and volunteering in an organization that was very open source based, like all of the tools that they used are just completely open source. CHARLES: Was that for budgetary reasons or they didn't want the people to burden the recipients of these computers with any licensing fees or obligations to third parties? AUDREY: It's budgetary but it's also ideological. The organization was started out of environmental interests. The original folks, they pointed to us this computer monitor that they [inaudible] as the reason that they do this, that the way computer waste is being handled was so unfriendly that you might as well just dump it in the river. They started from there but I think because those kinds of interests of creating something that was really accessible for people are really educational and accessible to lower income patrons has always been a really big part of it. I think that using Linux and using open source tools has been a big part of that. CHARLES: I think open source is so pervasive, a lot of people forget that in those days, there was a lot of radical thinking behind it, of radical accessibility like it's your basic right to be able to access every layer of your stack. It's a little bit unfortunate that you mentioned GCC that like the GNU, the Free Software Foundation isn't as much part of the conversation as they were back then. AUDREY: Yeah. I think that as more people come in to, we've shifted through these different generations basically in open source contribution and how it's formulated. The fact that I even default to open source is really interesting because a lot of the values that I referencing are those free software values. CHARLES: Fast forward to the present... AUDREY: Part of how I built my skills was by starting open source projects called Calagator. It's a community calendaring platform that makes it very easy to import things from other sources like Facebook. It's interesting, it wasn't our primary thing but it's so big now. We've been doing this for 10 years so a lot of recent change around us. We have a 10-year old [inaudible] app that is still up and running and is now in Rails engine. CHARLES: Wow. Is this an application that you can run yourself or when you say it's an engine, if I've got a Rails app, I can just drop it into any Rails app? AUDREY: Yeah, that was a direction we decided to go in a couple of years ago because my experience was that handing people in Rails app and saying, "Go fork it and then go sell it and use it in your community." That's a pretty big technical burden. At least, as an engine, it makes it a little bit more flexible for people to really come in and make some of those changes. We can bootstrap a little bit more for them. CHARLES: It's always funny to me know how some projects always run off the fork model, like there's a lot of HTML starters or editor starters where the thing is you fork it. I always hate that model because eventually, you ended up having to do this terrible dance with the upstream in order to jump around the changes that are coming through and stuff. AUDREY: Yeah and that was definitely one of the problems that we would run into. We would make changes to functionality and the frontend and the visual display of it. It was really difficult for people to pick and choose the parts that were useful for them. CHARLES: Yeah. Okay, so you've got a 10-year old Rails applications/engine, now you are actually running an instance of this engine yourself or just maintaining the open source? AUDREY: Yeah, there's actually two of them, that I'm in involved with right now. One of them is that Calagator.org. It's a Portland's techs events calendar. That was really our original site and the reason that we created this. The other one just as of a few months ago is PDXActivist.org and that is a way to get a lot of activism and political organizing off of Facebook, basically. That's really our primary target. It's just getting people an alternative to using Facebook for all of their events. CHARLES: I see. Now, having to maintain an open source project for 10 years, that's a really, really long time. AUDREY: Yeah. CHARLES: How big is the community now and how many different users have you seen as you developed this? AUDREY: Well, it's a little hard to tell. We deliberately don't do a lot of tracking, especially on PDX Activist side. I can tell you that there are a lot of events on both calendars. For the tech events, there are probably five things that you can do on any given day, maybe 10. During design week, they put all that on there too. This has been very consistent over the history of the project. I can also tell you that we've had dozens of contributors. CHARLES: Yeah, that's more what I meant when I said users. Not necessarily the consumers of the calendar but the consumers of the software that makes the calendar. AUDREY: It goes without saying that I think that those users are creating events, they are part of that because they help curate content. Like with the wiki, your user base isn't just the people who update MediaWiki. It's that people who really work on the content too. We've had dozens of people. There's a contributor's file that I didn't pull up but we can go and look at it. We made a point of crediting everybody who contributed at Code Sprint, whether or not they check in code. We have a really great documentation over the history of the project about how the different ways that people contributed and who they are. CHARLES: Yeah. I feel like that's something that often goes missing in projects, especially open source projects that you find on GitHub where there's so many people that are involved in creating software beyond just what you see in the commit history. It's kind of a poor showing of what it was all involved in the whole creative act. Sure, it's an accurate reflection if it's a one-person project who's hacking away on weekends but as your project scales, there's a lot of different stuff going on. AUDREY: Yeah, definitely. I think the other part that's really interesting for me about this is that I can point to that big contributor pool, people who have come to sprints so they've work on a project. They help define the shape of the project. Then I can tell you that we had a three-person core team for a very long time and then it was down to a two-person core team. Now, I'm not really sure which one of those is in charge. I don't look at GitHub often enough and a couple of the other computers. There isn't a lot of coaching happening anymore. We should have a wish list but there's nothing so urgent that we stop all other work and go back to making this our primary effort. CHARLES: Of the people on the core team, how many of them are developers? AUDREY: All of us. All three of us were. We come into with different cross skills. I've done a lot of documentation and mentorship. As of the others, I would say we have one person who were in design or one person who was more apps-oriented. We fill those different layers too. CHARLES: Of that group of the core contributors, outside that group of core contributors, you said you accumulate a list of all the people who contributed. What's the breakdown in the roles that those people are playing? AUDREY: You know, it has changed a lot over the course of the project. Early on, we had maybe half of the people were really doing development and the other half were helping. We took a very agile approach like index cards and users story. Maybe half of the people that show up at a given time, we just talk through the feature and do research. We were looking at a lot of integration so what needed to know what would be required to integrate it. We brainstorm a lot of things. We did in-person Code Sprints every two weeks from the year that we started, at late of January to the end of July. We had this whole set of in-person work that really shape in that. Also a lot of people who weren't necessarily contributing code that had disappeared. CHARLES: I see, so people who had a vested interest in a particular set of features could show up and voice that interest and be heard, as opposed to what you're having, it just be limited to the people who are writing the actual code. AUDREY: Yeah and we would ask people to spec it out. Just sit down with somebody and figure out how the feature could work and whether it fit with everything else to what we're doing. I do that research and investigation. Over the years, we've had this come and go in waves. Every so often, we need to go up a Rails version or make certain kinds of major updates so we get people together for that. We had some different pools of Codeschool students that have come in and really been interested in working on this to get a little bit more development experience, get some experience working with other people, have open source some resume to show off. I've been very enthusiastic about giving people that resume credit that if they need an open source of it so that they could say, "I know how to write with other people," then our projects is very happy to help them with that. CHARLES: What is the conference that you run? AUDREY: I am on the committee for Open Source Bridge. It's an annual conference for open source citizens, which is the same people who participate and benefit from open source. CHARLES: Which is pretty much the planet at this point. AUDREY: Yeah. It's funny because, I think it's just so interesting who does or doesn't identify themselves as part of that. Anybody using a computer these days is in some way benefiting from open source and could potentially contribute to it and be part of that. It's not just awareness, there are a lot of actual barriers so that, to everyone having a role in it. But the conference I co-founded it with Selena Deckelmann who's at Mozilla now. We do say over time to ask a lot of questions about how across technologies, open source comes together to build things? How projects work? What kinds of skills are involved? How we become better maintainers by being aware of our users, by communicating better, by being good moderators of online message boards and mailing lists and things like that? We've had a chance to really just look at broad swath of elements that come in. CHARLES: I think that literally every bullet point that you mentioned, I feel is something that we've come across and it has been a challenge for us, in our efforts to maintain our open source projects. Ours are mostly just libraries. There's very little by way of big, big frameworks or big, big applications. We've got it kind of easy, I would say and we still struggle with those things really understanding our users, understanding how your open source project should run and how it even fits into the bigger ecosystem. Is there a guide out there somewhere like how to how to open source? AUDREY: You know, I don't know that I've seen a single guide but there is really a lot of good writing and a lot of good conference talks on this topics. Like you said, it's just this broad set of skills and we focus so much on teaching people how to code and maybe teaching people how to code together, to be good contributors together but if you ever to maintain a project, there's leadership involved. There's communication involved. CHARLES: It seems to me that's the bulk of it, right? AUDREY: Yeah. I don't know, did you get training on that? [Laughter] AUDREY: I just decided to try things. I'm very lucky that I'm mostly made good guesses but there's some really bad ones too where later I look back at it and realized we could have done better. CHARLES: What are some of this mistakes that open source contributors often make, where they could save themselves a lot of trouble? AUDREY: I think a big one is thinking about it only in that technical framework. Even just by tools that we use, we tend to force people into contributing solely through GitHub, which means that you've got to understand somethings about the bug tracker and how tickets go and the workflow around that. CHARLES: Yeah. I've literally looking at a message in our Slack from yesterday where someone on our team who doesn't interact with GitHub said literally, "Someone is going to have to show me how because GitHub is the most confusing thing I have ever logged into." JOE: I thought about that message today too and yeah, I guess I'm wondering how do you attract those more non-technical skill sets to a project? AUDREY: It takes a lot of direct mentoring and coaching. You already has some people that are identifying themselves to you if you're having that conversation. I think I've really benefited from looking at who else is like them, who else do they know that might want to get involved and starting conversations that way. Because the biggest projects that I have worked on are these calendars, it does give us so many users that maybe are interested in having more technical involvement. If I can start looking at who's doing a lot of cleanup on there, who's paying a lot of attention to the content and the structure of the content and structuring information is also a technical skill. But people don't necessarily go from that to thinking, "I can write code," or, "I could submit a ticket and debug that thing and tell you what needs fixing now." But people can get there. We just have to be willing to talk to them about it and willing to look at it from their point of view. CHARLES: One thing that I dig out of there is that if you're running your open source project solely on GitHub, it's not going to be enough. You're going to be constrained in your growth just by the toolset and the implicit exclusivity of that toolset. What are some tools that you can bring in that are going to be more attractive? AUDREY: I think mailing list have turnout to be one of the most open-ended things that we've done. People who want to find out a little bit more, sometimes post there but also just having a good webpage, a good info pages or some sort, having your wiki actually talked about some of the less technical aspects of it. Even explaining what your project is for can be really good. You know, you start to make these assumptions like, "If they're going to go and install it, do they know?" Maybe not. I think just looking at it as a broader set of communications. CHARLES: Right. What seems self-evident to you and maybe someone who shares a lot of context to you is a mystery to someone else. It never hurts to state the obvious. It seems to me you have to be able to use tools that people are familiar with but also part of the leadership is giving people things to do, giving them a way to think about your project or giving them a way to act independently. How do you think about the different roles in an open source project so that you can then elucidate those roles so that someone coming, who is going to look at your website or who's going to be reading your e-mail list is going to be participating in your community in some way and particularly not in a code contribution way, how do you think about the different roles of your open source project so that you can kind of hand that to them? So that they can act independently like, "Here's this thing that you could do. Here's this thing that you could do. Here's this thing that you can do." What is that kind of core set of roles? CHARLES: We could think about it in terms of the actions that we take. If you go back to our lone weekend coder who put something on GitHub, you're already writing the code, making design decisions about the shape of the code, you are writing about it in some way, even if all you do is update the ReadMe to have two lines of something you're writing. You are managing any bug tickets that come in, any future request so you're doing some project management, some kind of general analysis of that. They don't necessarily have to be different roles. People implicitly take on the whole thought of that when they start a project. But they can also be split out. I hate to say like, "Give away your least favorite thing," because people sometimes do that, may dump it out there and it never gets handled well because they don't really understand what they're looking for. But it's okay to say, "I am really great at this one thing and I really struggle with this other thing." I bet there's somebody else who is just way better at organizing the stack communication and they can help me with that. If I can tell them what I need it for, maybe they can help with that. CHARLES: So you have to admit your weaknesses? AUDREY: Yeah. I think a lot of leadership is that kind of self-analysis: really seeing where you are helping the most, where you're strongest, what things absolutely have to be done with you. I don't know. I'd learned you to be really honest about that. Sometimes, the thing you enjoy doing is not the thing that you have to do because nobody else can. But often barred things that are really not fun for me, turned out to be the thing that nobody else can do. I just think that you have to spent some time thinking about that and thinking about what you can teach people too. You already have the knowledge of your project and what you're trying to do so I think what you can teach is what your mission is, what your goals are and maybe they can help you to communicate that too. CHARLES: Yeah, because it seems to me if you actually can very clearly communicate your target, then people can begin to walk towards it independently and that's almost more important than the actual taking the steps. Or the steps needed to be taken but that's something that you can provide. AUDREY: Yeah, you need that kind of definition regardless in order to make your decision and have your work actually function and the less conscious we are about, the more we tend to get a big pile of something and you go, "Now what? What do we do with that?" CHARLES: Right. I think it also flushes out if you have a clear target and you have a clear mission, by externalizing it, it makes you reflect on it more and hardens it, if that makes any sense. You have all these ideas bouncing around in your own head about the things that you might want to do or might like to do but once you actually try to express it to people and say, "You know what? We're going to do this." Then it takes on a reality of its own that is subject to more scrutiny but also subject to the constraints of the real world and that's a good thing. It means that whatever you're going to come up with is going to be more resilient. AUDREY: Yeah. I think we can be scared about putting that out there. They won't see what you see or they won't like it. Those who disagree with your goals there will go, "You really should have been building an eggplant slicer and not a tomato slicer." Yeah, I don't like tomatoes. But for more definition that we put out there, the clearer we are, the more that the people who want to [inaudible] they can find us. That's why it's so important to do it and not to dodge those kinds of questions. CHARLES: Yeah, absolutely. Now, I'm wondering so when is this conference that you're running? Is this the first one or is this the second, the third? AUDREY: Oh, no. We're on our ninth. CHARLES: You are on your ninth? Oh, my goodness. AUDREY: Yeah, it's actually just in a few weeks. It's in June, the week of the 20th, I want to say. Tickets are for sale. If you're in Portland, we had a great volunteer program where you put in eight hours over the course of the entire week. You can split out with everyone and you get a free ticket. CHARLES: Nice. This is the problem with the internet is I'm always finding out things that I wish I'd known 10 years ago. I wish I'd known about this before it actually tried to do any open source. This is the Open Source Bridge so what's a sample of what you guys are going to be talking about? AUDREY: The thing that we've added this year and it's really exciting is the activism track. We're having a lot more people to talk about what they do as code. In this other way, more of public facing way. We have Nicole Sanchez from GitHub. She's going to talk about diversity inclusion and some of the biggest [inaudible] there. We also had Emily Gorcenski doing another keynote and she talks a lot about data and ethics and has a lot of interesting things to say about how we collect and sort and process information and the impacts of that. We have a couple of workshops that are really great. One on technical interviewing and the personal skills that you need. There is a session on keyboard hacking. CHARLES: Keyboard hacking? This is in the activism track? AUDREY: No. This are across all the tracks. CHARLES: How many different tracks are there? AUDREY: There's five. CHARLES: This is a big conference. AUDREY: Yeah. It is such a great community for me to be a part of. Like I said, the different kinds of projects that people come from and bring into it and the different skills, we'll have people that are everywhere from kernel hackers to working in devops to people that kind of fit, I think what we think of it are more typical like web developer or mobile developer kind of skill set. People who run their projects, folks from Dreamwidth often come and participate and they have a lot of really great things to share because they have such an inclusive focus on how they do their project. CHARLES: Where was that? AUDREY: Dreamwidth. It's a LiveJournal spinoff. It's online community journaling website. It's in Perl, which is cool. There aren't as many outward facing things, hiring Perl programmer these days, I think. CHARLES: It's still a very active Perl project? AUDREY: Yeah. CHARLES: Wow. I did Perl a long, long time ago. AUDREY: I think it's really useful to remember that programming languages never actually die. There is always code. JOE: There's still plenty of COBOL positions out there. AUDREY: Yeah. Actually my uncle is a COBOL programmer. CHARLES: Yeah, I remember it was only some statistic where it was something like five years ago, Java, Eclipse, COBOL is the most popular programming language. The cycles are much larger than we tend to think. Surfing on the beach as we do, not realizing there's a whole ocean generating those waves. AUDREY: Yeah, I think if you're in a certain kind of technology startup plan, there's always this push to go for the nearest and shiniest on the number of JavaScript frameworks that we've gone through in the last five years. You kind of [inaudible] of all of these things that come before that are still in use. What I really loved about doing devops is that all of this pieces are still in play and there's something to learn from that. If they don't die, you don't get rid of them. You just try to build on them and keep them working usefully. CHARLES: Right. Man, that's exciting, so you have a very, very huge cross-section of the development community. It sounds like participating in here which is a quality in of itself. That must give you a pretty unique perspective being with that level of cross-discipline. Are there any insights that can only be gleaned by being able to perceive it from that high of a level? AUDREY: Well, a big one is that we all struggle with governance. We don't really talk outside of just a couple of forms for events that focus on open source maintainers. We don't talk about the governance of projects, like who was in charge and how decisions are made. But it turns out that that has just an enormous impact on what a project can actually do and how it survives. I think I might not have seen that as clearly without having people from so many different angles participating. CHARLES: I'm just trying to think of keeping it in the area of web frameworks because that's something that I'm familiar with. If we were to compare, say the governance model something like React, which is basically whatever Facebook wants, versus something in the middle like Angular, which is like an explicit governance model but also is heavily influenced by Google, versus something like... I don't know, well something like JavaScript itself, which has an open democratic model but heavily represented by major, major, major companies, versus something like Rust, which is I certainly get the feeling is a very explicit, very democratic model. All of those seem to have achieved a lot of success and this seemed like a very healthy projects but on the one hand of the spectrum like Rust, you have the super-transparent, super-democratic model and then on the React side, you've got this authoritarian model. That's opaque. How do you reconcile that those are both successful? AUDREY: I think a lot of what actually determines this stuff is who pays the developers. In both of those cases, meaning projects that present information and decision making differently but there are corporations that pay those developers and that's where the primary source of that code. Because of that, really who pays the developers determines what gets made, what code gets written. In a way, they're both doing some of the same things. They're just not giving you inside into that decision making, in some cases. CHARLES: The decision making apparatus is there, I guess the thing is this transparency to the user base matter. I would say that the user base of a thing like React dwarfs the actual corpus of decision makers. That doesn't seem to be that that decision making process is opaque. AUDREY: Well, I might be opening too much of a larger conversation by saying this but if you're familiar with the idea of algorithm transparency, decision making is encoded into things like algorithms and when we can't examine them, then we don't know how that decision was made so we don't know what biases are encoded into it. The same thing happens with code in general. You might say, "Let the outcome of this and this working really great," but there are still biases and preferences that are encoded into that that you don't have insight into. If they start to ship the project in a certain way, that include some users and excludes others. Even on just purely technical levels, you don't know what. You don't know how they got to that, you don't know if they're going to keep steering in that direction. If you're one of those people that is starting to be excluded, you don't know what you can do about it. I've seen these kinds of governance discussion even happen within Ruby in Rails. CHARLES: Yeah, it does seem like these political questions come up constantly. I remember an example that leaps to mind is a project that I was involved with was the Jenkins project, which originally was Hudson, which came out of Sun Microsystems. When Oracle bought Sun, they were basically trying to, I want to say there's always three sides to every story but from where I was sitting, they were essentially trying to subvert the project to their own needs and end up being in a fork of the project. Luckily, there was recourse there where because it was open source and because it was mostly maintained by the community and not by the company, they were able to fork it. They changed the name. They changed the logo and that was the end of the story. There was a question of which fork would survive but that was resolved within probably six months. But Jenkins lived and I think it's better off for it but I guess maybe then a question that you can one kind of stress test that you can put like, "Is it okay to put weight on this technology?" What would happen? Would my community be represented and would I be able to fork this, essentially? Maybe in that sense, React would pass that test. In the sense that it would be reasonable to fork it or something like that. I don't know. I'm just thinking of ways to try and validate if something safe to use. AUDREY: I think it's really interesting that you commented on the new change and the logo change because those kinds of trademarks are actually the most readily protected of all of the intellectual property in an open source projects. If things are going to go off and become a community project and it's being released under some open source model, often where the corporate control stays over those assets -- the name, the logo, the graphics -- maybe even some of the work [inaudible]. You have to ask if that code is still useful without that infrastructure that they provided. If you take the whole codebase and you walk off and you don't have the same developers and you don't have the same, even hosting resources or whatever, is that code still useful to you? What if you use a bug tracker? CHARLES: Right, now you own it. What's the cost now of maintaining? And are you going to get a return on that investment? AUDREY: Yeah. There's been some pretty big open source projects that have struggled with that, especially for end user facing software. Those turned out to be easy things for community to pick up. CHARLES: Can you provide any examples? AUDREY: I'm thinking of some of the stuff that happened with Open Office LibreOffice. CHARLES: Yeah, I remember that. AUDREY: There's still two different batches of people working on this and from what I understand, a whole lot of intellectual property complications. CHARLES: Yeah, it's funny how sometimes, it would be interesting to see a case study of all the major forks and the outcomes of what they were. Some I can think of, there was a fork of Ruby gems, for example I think back in 2009 that went off and was mainly, I think was a way of protest. I think some of those concerns were addressed in the main thing so that fork ended up dying, then you got the fork of io.js, which was ended up. There was a fork and then a rejoining with the Node community but I would say it was an effective tool so there was a fork but then it joined. It was a source code fork but it was a political fork. Then you have the Jenkins fork where the fork basically swallowed its ancestor and there's all these fascinating outcomes and then you've got this LibreOffice Open Office where the waters are very murky about what happened with that fork. AUDREY: I heard people say like, "If you don't like this decision, then just go fork the project." CHARLES: Because that's easy. AUDREY: And if one of your major developers does it, then maybe, like you said, they have some leverage and they can make the changes they want to see happen, [inaudible]. But in general, that's a really hard thing to pull off. You've got to be able to take your entire community with you. Part of this is have to be functional and I think people are very rarely actually make that happen. CHARLES: Right. I feel like that's a dishonest thing to say when people are like, "If you want to go fork it," because really forking the code is the easy part. It's forking the community. AUDREY: Well, if you do that, then you've got a lot of conflicts. You've got a lot of people's feelings to address. It's not a very simple thing to recover from. CHARLES: Yeah. Some people do it. We have some good examples of that happening but it doesn't always pan out for the best. How can we make open source more accessible and supportive of contributors? We've mentioned a lot of that stuff in terms of how you can support people who are contributing but there might be more to talk about that. AUDREY: Yeah, we haven't really talked about who gets to participate. We talked about what kinds of things you can do when you see that people are interested but we don't talk about how in order to be a week encoder, you've got to have those weekends free. Certainly, I am right now. CHARLES: Yeah, neither do I. AUDREY: You have to have access to a laptop if you want to go to Code Sprints or [inaudible]. Not everybody has that, even people who are programming or your own computer not owned by your employer. That can be really important. You have to have a knowledge of how open source works. I do see fairly often in conferences that focus on a lot in open source, there will be how to become an open source contributor kind of talk. That kind of cultural knowledge is really important because otherwise, you're going to GitHub and you look at it and you say, "What am I supposed to do here? What am I actually supposed to do with this?" It's just a wall of information. There's something about a project on GitHub that creates these entry points for somebody who doesn't know how open source projects work. CHARLES: Yeah and it's so hard to be able to perceive it from that person's perspective, especially if you're frog-boiled, so to speak in the community. You've been doing this for so long, these things seem self-evident that it takes a computer, it takes the time, it takes knowing where to establish a toehold. These are all non-problems for you but they're insurmountable for someone else. AUDREY: There's one other aspect of this that we haven't really talked about, which is the friendliness to the kinds of contributors that you have, the diversity of the project versus the homogeneity of the contributors, whether or not you have a code of conduct and you know how to do something with it so that people feel safe and welcome in your environment. There's a lot of people that stay away from open source projects because all they've ever seen is harassment and that behavior. You can have a counterexample but if you don't have some mechanism for showing that that won't happen in your projects, then there are folks that are never going to submit about. They're never going to make a commit. They're not going to put anything on the wiki. CHARLES: Why would voluntarily subject myself to, if the only thing on the other end of the phone is pain? AUDREY: There are plenty of people that decided just to opt out because of that. If open source projects want to see more contribution, you have to be very proactive in dealing with that. CHARLES: Yeah, I feel like it almost would be nice to have some sort of training. Even if you have a code of conduct on your open source project, I think as you grow it from something that's maybe just one or two people to where there's a larger community, the first time you have a bad actor who shows up and start slinging turds, it's shocking and you're taken aback. But just as the number of people grow in a community, that is going to happen. It's just an unfortunate fact of human nature so not having to react to it, but be prepared for it, I think is something that's extraordinarily valuable. I don't know if there's a guide for that on GitHub or guide for that on anywhere else but I think it would be very useful skill to have. AUDREY: It's just very funny that you say this because this is actually a training idea. CHARLES: Oh, really? I promise there was no payment under the table to ask that question. AUDREY: Yeah. There was some consulting around this and I started a program with a local non-profit called Safety First PDX and what we do is train user group leaders, conference organizers, open source project maintainers on exactly that: what to do with their code of conduct to enforce it and help people feel welcome in their community. I worked through a really specific examples with people about how you respond, how you have this conversations and what kinds of things you need to do to protect your contributors who are participants and be really firm about what is next in your space. CHARLES: Absolutely a critical skill for any open source project, for any open source community, for any large accumulation of people. AUDREY: And GitHub made it very easy to put a code of conduct on your project now but without these kinds of resources, I think what happens is that people get that first incident and they panic because it is scary to tell somebody that their behavior isn't okay. To tell them that they might have to step away from the project or stop doing that or even leave indefinitely, those are really hard things to get started doing. I really enjoy doing the training and getting to walkthrough that to people. CHARLES: Are you going to be offering that training anytime soon? AUDREY: We just had one here in Portland last week. We're doing it a quarterly thing but I'm also really open to bringing it elsewhere like a place to host and some sponsorship that they can throw at that and people that want to take this. CHARLES: That'll be awesome. Maybe we can have you in Austin. AUDREY: [inaudible]. CHARLES: Thank you, Joe. Thank you, Audrey for coming on the show. AUDREY: Thanks. CHARLES: It was really great to talk to you. It's great to talk about your history in open source and the things that you're doing in the community, especially the insights that you have around running sustainable open source projects. Also, thank you for talking to us about Open Source Bridge which is, I understand coming up right around the corner. If you want you can go to our podcast page and there will be a link to get $50 off if you enter in the discount code 'PODCAST.' That's $50 off of your open source bridge ticket. Be sure to go check it out. That's it for today, from The Frontside. If you're interested in hiring us, we do have availability starting in July so reach out to us. All right, everybody. Take care.
Coté doesn't like asking people to do things for him...or people in general.
Toran Billups @toranb | GitHub | Blog Show Notes: 01:44 - New Developments in ember-redux 04:23 - New Developments in the Wider Redux Community 06:26 - Using Redux in Ember 09:40 - Omit 10:45 - Reducers 25:42 - Fulfilling the Role of Middleware in Ember 28:12 - Ember Data in Redux-land 31:24 - What does Toran do with this stuff?? Links: The Frontside Podcast Episode 55: Redux and Ember with Toran Billups The Frontside Podcast Episode 18: Back-End Devs and Bridging the Stack with Toran Billups redux-offline ember-redux-yelp create-react-app "Mega Simple redux” Twiddle ember-concurrency Thomas Chen: ember-redux The Frontside Podcast Episode 067: ember-concurrency with Alex Matchneer normalizr Rich Hickey: Simple Made Easy Other Noteable Resources: ember redux: The talk Toran prepared for EmberJS DC in April 2017 github.com/foxnewsnetwork/ember-with-redux Transcript CHARLES: Hello everybody and welcome to The Frontside Podcast, Episode 69. My name is Charles Lowell. I'm a developer here at The Frontside and your podcast host-in-training. With me Wil Wilsman, also a developer here at The Frontside. Hello, Wil. WIL: Hello. CHARLES: Today, we have a special guest, an actual elite member of the three-timers club, counting this appearance. We have with us Toran Billups. Thank you for coming on to the show today. TORAN: Absolutely. I'm not sure how the third time happened but I'll take it. CHARLES: Well, this is going to be the second one, we're going to be talking about Redux and then I believe you're on the podcast back in 2014 or 2015. TORAN: That's right. CHARLES: That's one of our first episodes. Make sure to get in touch with our producer afterwards to pick up your commemorative mug and sunglasses to celebrate your third time on the show. Awesome. I'm glad to have you. We actually tend to have people back who are good podcast guests. TORAN: Thank you. CHARLES: Yeah, I'm looking forward to this one. This is actually a continuation of a podcast that we did back in January that was actually one of our more popular episodes. There was a big demand to do a second part of it. That podcast we talked about the ember-redux library, which you're a maintainer and just kind of working with Redux in Ember in general. We're going to continue where we left off with that but obviously, that was what? Almost six months ago? I was wondering maybe you can start there and there been any kind of new developments, exciting things, what's kind of the state of the state or the state of the reducer or the state of the store in ember-redux. TORAN: For ember-redux, in particular, we're working on three initiatives right now. The first is making the store creation more customizable. A lot of people that come from the React background in particular are very used to hand crafting how the stores put the other with the right middleware and enhancers and reducers and that's been fine. I wanted to drop people into the pit of success and everybody's cool with that but now we're getting to a point where there are people wanted to do different things and it's great to open the door for those people if they can, while keeping it very simple so we're working on that. We have here that's just undergoing some discussion. We're also, just as the wider Ember community -- you guys maybe involved in this as well -- and trying to get the entire stack over to Babel 6, the ember-cli Babel 6.10 plus stack. There is a breaking change between Babel 5 and 6 so we're also having some discussions about the ember-redux 3.0 version bump at some point later this year, just because we really can't adopt this without introducing basically a breaking change for older ember-cli users. CHARLES: Just in general, this is a little bit off topic, what does it mean to go from Babel 5 to Babel 6, if I'm an add-on author. TORAN: I would probably ensure that need to speak more with Robert Jackson about this. We just kind went back and forth because I thought I had a Babel compile error. He's like, "No, you're missing this dependency which is the object spread." Unfortunately, the object spread is rampant in React projects and this is totally cool. I had to actually add that and that's just a breaking difference between these two. If we adapt the new version of this in the shims underneath of it as an Ember 2.43 user, if you're on node four which is still supported, you will break without this. I'm trying to get some discussion going about what we should do here and if we even should push ahead and just say only node six is supported. There's some discussion and then back to your original question, the third piece is we've introduced the ability to replace the reducer but we need to get some examples for hot reloading the reducer. That's a separate project but it needs to be enabled by ember-redux. Those are the three main initiatives. CHARLES: Being able to you hot load your reducers, just to make changes to your reducer and you just thunk them into the application without having to lose any of the application state and that one of the reasons that's possible then is because you're reducers have no state themselves. They're just pure functions, right? TORAN: Exactly. CHARLES: Okay. Awesome. That sounds like there's a lot of cool stuff going on. Beyond ember-redux, are there any developments to look for on the horizon in the wider Redux community that might be coming to Ember soon? TORAN: Actually one of them is actually fairly new and it's already in Ember and just because I have already got a shim up for it is redux-offline, which I remember you had Alex on two episodes ago about breaking your brain around Rx. I feel like that happened for me trying to build apps offline first. This is, of course just another library that can drop in that place nicely with Redux but I feel like the community, at least it's got me thinking now about what an absolute that would really disrupt someone who's a big player today. I feel like you've built a great offline experience with true and well done data syncing. You could really step in and wreck someone who's in the space today. CHARLES: Right, so this is a sim around... what was the name of the library again? TORAN: Redux-offline. CHARLES: Okay, so it's just tooling for taking your store and making sure that you can work with your store if you're not actually connected to the network and like persisting your store across sessions? TORAN: Yeah. It uses a library that called redux-persist that takes care of and kind of hydrating the store if you have no network connectivity. But it's also beginning to apply some conventional pattern around how to retry and how to roll back. It's just an interesting look at the offline problem through the lens of an action-based immutable data flow story. It's interesting. I don't have a ton of experience with that kind of tweet and rewrote my Yelp clone with it and that was tough. That's what I mean by this. It's like I thought this is very trivial but you have to do a lot of optimistic rendering and then sort of optimistically gen primary keys that gets swapped out later and it's tricky. If you've never done offline first, which I have not, I just think offline is pretty cool and along those lines, there's been a lot of discussion around convention. There's of course, Create React app which is like a little library or CLI tool to kick off your Redux in React projects. It's kind of ember-cli, very trimmed down version of that right now and that's just getting incrementally better. Of course, you guys are in the React space so you may touch upon that story if you haven't already. CHARLES: All right. We talked to the very high level, I think the last time we had you on the show but now that the idea is gaining traction, kind of delve into more specifics about how you use Redux in Ember. I asked at the end of the last podcast, let's step through a use case like what would deleting an item look like in ember-redux land. Maybe we could pick that up right now and just understand how it all connects together. TORAN: Yeah, absolutely. Without understanding how this entire flow or this event bubbling happens is hard to get your head around it. The process we're going to walk through is exactly that use case you laid out, Charles. We're going to have a button in our component and that button, on-click the idea is to remove an item in a list that we happen to be rendering, let's say. If this is a child component like the very primitive literally button that you have and you just have your on-click equal probably a closure action in Ember, the parent component or the outer context is going to be responsible for providing the implementation details for this closure action and what it does. This is kind of the meat of what you're getting at. The high level here is there is a single method on the Redux store that you have access to and it is called dispatch. The nice thing about Redux again, the API surface area is very small. It's just a very handful of methods you need to get your head around. This one, in particular takes one parameter, this dispatch method. That parameter is a JavaScript object. Now, if you're just playing around, you just want to see the event flow up, there's only one requirement asked of you and that is type property. This JavaScript objects have a type that is often a string so it's very human friendly, you just put in there and the string remove item, let's say. Now of course, if you want to remove a particular item in this remove example, you of course want to pass some information as well beyond the type. The type is mostly just a Redux thing to help us identify it but in this case, you'll definitely know the primary key or the ID value, let's say of the item you want to remove. In addition to type, this JavaScript object, let's just say has an ID property and that can come up from the closure action somehow if you want. Once you click this, what's going to happen is you're going to fire this closure action, the closure action is going to invoke dispatch with the JavaScript object and dispatch is going to run through the reducer which is the very next step and what we do is we take the existing state. Let's say we have a list of three items, that's going to be the first argument now in the reducer. The second argument is this action which is just, as I describe it a JavaScript object with two properties: type and ID. You can imagine an ‘if' statement, conditional switch statement that says, "Is this the remove item action? If it is, okay." We had the ID of the item that want to remove and since we have a dictionary where the primary key of the item is the ID, we can just use lodash-omit and we basically use omit to filter out the ID and then use object assigned to a transform or produce a new state and then a callback occurs after this that tells your list component that somewhere in your Ember tree to re-render, now only showing two items. CHARLES: One of the things I want to point out there, you just touched on it but I think it's an interesting in the subtle point is that the lodash method that you used was omit and that's how this is kind of tangential or I'll say parallel to programming in this way is that you don't actually use methods that mutate any state. You calculate new states based on the old states. I think that's a great example of that -- that omit function -- omission is the way that you delete from something in an immutable fashion. You're actually filtering or you're returning a new copy of that dictionary that just doesn't have that entry. You're just a omitting it. You're not destroying the old one. You're not deleting anything. You're not changing it. You're just kind of Xeroxing it but without that one particular entry, which is ironic because the effect on the UI is that you have done a delete but really what you're doing is omitting there. I just think that's cool. I think it's one of the ways that using the systems teaches you to think about identity differently. Then the question I want to ask you was, this all happens in the reducer, what does that mean? Was that word mean -- reducer? I kind of like danced around that idea and I've tried to understand where that term came from and how it helps give insight into what it's doing and I come up short a lot. Maybe we can try and if we can't explain what the name of reducers, maybe there's some alternate term that we can help come up with to aid people's understanding of what is the responsibility of this thing? TORAN: Yeah, I think we can just break down reduce first then we'll talk about how it ends up looking. But I think it's reduced almost like it's defined in the array context. If I have an array of one, two and three and I invoke the reduce method on that, we actually just produced a single value, sort of flatten it out and produce a single value as the result to that, so three plus two plus one, six is the end result. What we've glossed over this entire time, probably last episode is that this reduce word, I believe is used because in Redux, we don't really just have one massive monolithic reducer for the entire state tree. We instead have many small reducers that are truly combined to do all the work across the tree. In my mind is I think reducer fits well here because we're actually going to combine all these reducers and they're all going to work on some small part of the state tree. But at the end of the day, we still have just one global add-on and that's the output. We want one global add-on with a new transform state and that is the reduced state. CHARLES: You take some set of arguments. One of which is the prior state and you reduce that into a single object. I like that. The other place where I've seen this term applied in a similar context is in Erlang. They talk about reduction so that you have an Erlang server where the way that the servers modeled is as a recursive function call where you pass in the prior state of the server plus any arguments if you're handling a request or something like that, bundled in there is going to be arguments of the request and then what that function returns is a new state, which is then used to pass in to the next state. They call this process reduction. We've got two data points. Maybe there, we can go search for the mathematical foundation of that later -- TORAN: I like it. CHARLES: -- if you want to geek out. I think that helps a lot. Essentially, to sum up the responsibility of the action is you take a set of arguments, it's going to take the existing state of the store, run it through your reducers and then it's going to set the next state of the store or yield the next state of the store. Is that a fair summary of what you would say the responsibility action is? TORAN: Yeah. I think you're right. In fact, in preparation for this talk, I just threw together really small Ember Twiddle that will link in a show notes what I call the mega trimmed down version of ember-redux. It's basically a really naive look but for conceptualizing this flow, it's about 24 lines of code that show exactly what you're saying which is I have this reducer, it's passed into this create store method in the syntax, how do this actually look? It better illustrates how the reducer is used when dispatch is invoked so a dispatch, if I was actually to walk line by line through this, which will be probably pretty terrible. But the very first line of dispatch is to just call the reducer. From that new state transformation, we just push in so the store gets a new entry into it's array of here's the next state and because we had never tampered or side effect-ted the old state, we could easily go back in time just by flipping the pointer in the array or that indexing the array back point. CHARLES: I guess my next question is we've talked a little bit about immutability and we know this reducer that we call at the very first point of the dispatch is a pure function. You were dealing with pure functions, immutable data but at least in perception for our users, our system is going to have a side effect. There's going to be calls to the network. We are, in the least in theory deleting something from that list. How do you go about modeling those side effects inside Redux? TORAN: This is a great intro because in fact a friend of mine is actually a teacher at a boot camp and he was telling me the other day that he was asked to do a brief look at Redux and his very first feeling when he was watching some of the Egghead.io videos is like, "Oh, so the reducer is pure but I have to side effects something so where do I do this?" It's not very clear, I think for the very beginner which is why we left it out of that part one podcast. Today, we're kind of hit that head on but before we get into that list, we can talk about what having actions in their simplest form look like today in your Ember app. As I mentioned earlier in this remove example, you got the button, it just takes the closure action the on-click. No big deal. The bigger work was on the parent contexts or the parent component to provide this action, which sounded very simple but imagine instead of just dispatching synchronously, which is we talked through. Imagine we only wants to dispatch that officially to change it if we have gotten a 204 back and the fetch request is deleted on the server -- normal Ajax or fetch-type flow. In this case, you start to add a little more code and imagine for the moment this is all inline in your component JS file. The component now is started to take on an additional responsibility. In addition to just providing a simple dispatch, as I say, "Go and remove this Mr Reducer later," you're now starting to put some asynchronous logic and as you imagine a real application, you grows and try catch stuff, some error handling, some loading, some modals. This gets out of hand. One of the things that I want to touch on briefly is, at least in the ember-redux case we ship this Promise-based middleware and I want to stop right there for just a second because I use that word 'middleware' and immediately we've got at least highlight what this is doing. In that pipeline -- we talked about earlier -- in the component I dispatch and it just goes right to the reducer. Well, technically there's actually a step or an extension point right before the reducer is involved and that is where middleware comes in. Technically, you could dispatch from this action and then you could handle and do the network IO type request in middleware instead, then porting on another dispatch of a different type with the final arguments to be transformed. That's really the role. CHARLES: Can middleware actually dispatch its own actions? TORAN: Correct, yep. In fact, one of the first big differences between this example as I'm kind of hacking around the component and I've got access to dispatch but there's two things I'm really actually lacking if I don't leverage middleware full on. The first is I do have some state in the component but often, it's actually just a very small slice of state that this component renders. If there's actually a little bit more information I need or actually need to tap into the full store, I don't have it and that might be considered a good constraint for most people. But there are times you imagine more complex apps, you need the store. You might even see a little bit more state, middleware provides that is where you trapped with just that slice of state and dispatch the keyword. That's about it in the component. But the other side effects are the benefit, as you break this out is you get another seam in the application where the component now is not involved with error handling and Promises and async flows or generators. It just does the basic closure action set up and fires dispatch almost synchronously like you did in our very simple example, allowing the middleware to actually step in and play the role of, "Okay, I'm going to do a fetch request and I'm going to use a generator." It's almost like the buffer for IO or asynchronous work that it was missing in our original equation. Imagine, you want to debounce something or you want to log something or you want to cancel a Promise, which you can't do, any of that stuff that's going to happen in this middleware component. That's one of the things I like about middleware as I learn more about it and the moment you get to a very complex async task where you're actually doing the typical type ahead, where you literally want to cancel and not do the JavaScript work or you like to cancel the Promise as quickly as you can, you can very quickly dive into something kind of like what you and Alex talk about with ember-currency in the Redux-land. It's called redux-saga. It's just a generator-based async work. CHARLES: Is saga kind of emerged as async library that everybody uses? I know it's very popular. TORAN: Yeah and a good reason, I mean it solves a lot of the problems that if you were to try and do the cancelation token Promise stuff that came out a while back where we're trying to figure out how to cancel Promises. There's a lot more ceremony and just a lot more state tracking on your own that generators and even when I played with this last week, which is actually redux-observable which is an Rx-based middleware. It's built by, I think Ben Lesh and Jay Phelps from Netflix or... sorry, Jay is still at Netflix but anyway... You could use Rx, you could use generators. This really is just the escape valve for async and complex side effect programming that can't or shouldn't take place in the reducer because it's pure. It shouldn't take place necessarily in a component because one of the best pieces of advice I got when I was younger was, " Toran, make sure you do or delegate," and we're talking really about levels of depth in your methods at the simplest. But it applies here as well, which is I would love it if I had just a very declarative component and I didn't have to get into the weeds as I was looking at it about, is this a Promise? Is this Redux thunk, as they call it? Is this a generator or is it Rx? I don't even care in the component for the most part. I just need to know the name of the action and the arguments. If I'm having a problem with the Rx side of the generator, I'll go into the middleware and work from that particular abstraction but you can see the benefits of the seam there. CHARLES: Are the middlewares match on the action payload in the same way that the reducers do? Is that fair to say? TORAN: That is fair and I will warn. If that seems very strange, you're probably not alone. In fact, the first time I did this with redux-saga, I was dispatching, only to turn around then dispatch again. It feels very strange the first time but again, keeping in mind that you're really trying to have a separation from the work that is side effect and the work that is pure. The first action in that scenario, we'll call it remove-saga because it's actually going to fire something to a middleware. That work is all going to be network-heavy and it's not really as easily undo-redo because it's not pure. But the second event invoked from the actual middleware itself that says, "Remove item. Here's the ID. We're good." That work could be undone-redone all day because it hits the reducer, which is sure you can in and out. CHARLES: It sounds like basically the middleware is allowing you to have a branching flow structure because they all do involve more actions getting dispatched back to the store to record any bookkeeping that needs to happen as part of that. If you want to set some spinner state, that will be an action that gets dispatched. But in terms of sync, they allow you to set up sequences of actions or if you basically have one action that will actually get resolved as ten actions or something like that. If you think about an asynchronous process, you have the action that starts it but that might end up being composed of five different actions, right? Like I want to set the application into some state that knows that I've started my delete and that means I want to show like the spinner. Then at some later point, I might want to show progress like this deletes really taking a long time and I might want to dispatch five different actions indicating each one of those little bits of progress. Then finally, I might want to say it's done or it fails so really those got decomposed into 10 actions or five actions or whatever so the middleware is really where you do that, where you decompose high level actions into smaller actions? Or it's one of the places? Is that a correct understanding? TORAN: Yeah, I think if you're an old school developer for a minute, it will cater to the audience that maybe came from early 2000s backend dev. Now, they're still pretty current in web dev. I see it talked about as business logic. I feel like this is really the bulk of the complex work, especially if you're using Rx. You're actually creating these declarative pipelines for the events to flow through. My components are much thinner by comparison. They're truly just fire off this action with the information to kick the async pipeline but in the async piece of it, there's a lot more work happening and that's I think because there's a lot of complexity in async programming. CHARLES: Right, and it's almost like with the reducers then, there's not so much business logic because you're just resolving the implications of the new state. Is that fair to say? It's like now we've got this information, what does this imply directly? TORAN: Yeah, I think there is this old [inaudible] thing where they're talking about what's should be thick. You know, thick controllers or thick models, what should it be? Of course, we never want 'thick' anything, is the right answer, I think. But the apps on building today, I feel like if any was thick-er -- a measure of degree bigger in effort or work -- it is these middleware components right now. I think the nature of what you describe, which is the reducer is not supposed to be doing anything complex. It's literally taking a piece of data in, producing a new piece of data out. Logically thinking about that takes much less effort, I think than the human brain applies to async programming in JavaScript. CHARLES: Right. I think it makes sense and some of these things are just going to be necessarily gnarly and hairy because that's where the system is coupled. I can't say anything about whether the delete succeeded or failed until I've actually fired off the request. Those are implicitly sequenced. There has to be some glue or some code declaring that those things are sequenced. That has to be specified somewhere, whereas theoretically with your reducers, you could just run them all in parallel, even. If JavaScript supported multithreading, there's absolutely no dependency between those bits of code. TORAN: I think so, yeah. I think there are still some challenges because in the reducer sometimes. We can talk about this in a few minutes but you may actually be changing several top level pieces of the tree. If you're de-normalizing, which is what we probably should touch on next, there are some cases where you want to be a little careful but like you said, generically immutable programming enables multithreading. We're not touching the same piece of state at same time. CHARLES: Right as long as that piece of state that you're touching, like you need to resolve the leaf nodes of the tree first but at any siblings, I guess is what I'm saying on there, ought to be able to be resolved in parallel. It's more an exercise in theory or just a way of thinking about it because like why you're able to do those reducers as kind of these pure functions is because there's no dependency between them. I guess I'm just trying to point out that to wrap my head around, there are places where there are just clear sequences and dependencies and those are things that would be in the middleware. TORAN: Gotcha. I came a little scared of service worker. [Laughter] CHARLES: Actually a great point is what kind is the analogous -- if there is anything analogous -- in Ember today that's fulfilling the role of middleware. What's the migration path? What's the alternative, just trying to explore like where you might be able to use these techniques that we've been describing inside your app? TORAN: I think, at least my look at it has been a service injected into a service, which sounds completely bad or sounds broken the first time you see because you're like, "We're injecting a service into an existing service." I say that because, for me at least there is a top level service that owns the data and provides read-only attributes but there should be some other piece of code -- not the component -- that is doing this asynchronous complex processing, we just talk about as middleware, that is often a different service than the service that owns the state add-on. That's what I meant by service-injected. There's some Ember service whose job is to manage the complexity and probably ended up in middleware from the Redux perspective or ember-concurrency is literally solving that in my opinion. They do a lot for you: solving the async problem generally and I haven't dug into ember-concurrency enough to know. The pipeline stuff, I think which you guys talked about, which is an RFC, that may have eventually be what I consider the Rx or redux-saga of the middleware today. CHARLES: Right. I think ember-concurrency is just absolutely fantastic but it is a hairy problem but there's some overlap in terms of what it is solving. I think that is interesting. I guess a case where you would use middleware would be anywhere that you would use ember-concurrency. I think the interesting thing to compare in contrast there is one of maybe advantages or disadvantages -- let's just call it a tradeoff -- is that with ember-concurrency, you have this middleware that is associated with an object. It's associated usually with a component or a route. When things happen to that component, you're able to affect your ember-concurrency process but it does mean, these things are sprinkles throughout your application and the rules that are governing them can be really different, depending on which part you're operating in or just because sometimes you're using them on a route. Sometimes, you're doing it on a component. Sometimes, you're doing it on a service, whereas with the putting in the middleware, it sounds like they're going to all be in one particular place. All right. Let's move on from the simple to the more complex because that's where it's at. We've talked about modeling async processes, we've talked about handling state transitions and all that, nothing typifies that more profoundly in Ember community than Ember Data as just a foundation for state and syncing it over to the network. Love it or hate it, it's very popular. What are the things that you do in ember-redux land? One, is it fundamentally incompatible with Ember Data or is it just more easily served with other alternatives? How do you handle those foundational interactions, those fun foundational async loading network interactions with Ember Data, just using an ember-redux? TORAN: Yeah, for myself, I don't have an experience actually using the two together on purpose. There is a gentleman who did a talk sometime last year and I'll dig up the YouTube clip for you guys, where he talked about his approach where you would actually produce new states so it's still Redux friendly. Ember Data itself just be a new Ember Data model, every time you transformed it. But one of the tricky points is the philosophy of both so in Ember Data, you're just invoking set on everything and not just how it works. That's how the events bubble through the system as you re-render. You never actually create a new state of the system that's a copy, minus or plus, other attributes. You just always touching a single source of truth. I felt like that was always a sticking point. Anyway, Thomas who did this talk and I'll point you guys to it, did a great job of saying like, "Look, if you're stuck in this world with a lot of Ember Data, you're having some pain points with it and you want to try Redux to see if this alleviates those by not changing the state, here is a middle ground," which he did, I think a fabulous job driving it out. Although I must admit, there's got to be some challenges in there just because of the philosophical difference between the two. CHARLES: Yeah. It definitely sounds like there's some challenges but I'm actually pretty eager to go and watch that, to see what they came up with. If you're using these snapshot states of your Ember objects, would it be possible then to take all of your save, delete records, even query and have them inside of middlewares like have a redux-saga for every single operation you want to take on the Ember Data store. TORAN: The example he showed is basically the best of both worlds. You don't want to ember-mutate so he has a special bit of code to do that. But because the rest of it is vanilla Ember, you could drop in concurrency if you want to do the saga-type generator stuff. But you could also just make your changes as you would otherwise. You use the adapters, you fetch, you save, you delete, whatever you want to do for the most part. It saves a lot on the de-normalized side, which you would have to do manually. You don't write any Ajax code, which you have to do manually on the Redux side. I think there are benefits if someone could get it to work where you're just not changing the state of Ember Data, which may actually just be the future Ember Data at some point as well. CHARLES: It sounds like there is a pathway forward if that's the way that you want to go and the road that you want to walk so we'll look for that in the show notes. But my question then is, you're here on the podcast, what do you do? TORAN: I do want to have one disclaimer here, just so that I'm not a complete poser but I am. If you guys don't know this, I'm not trying to hide this from the community but I don't work on ember-redux at work. I don't have a side gate, making money with it. I don't use it ever. I literally just build examples to try stuff out. There's both a blessing and a curse of that. The curse is that you're asking, "Hey, Toran, you're the author. How does this work, man?" I can give you my run at it which I will but there is the other side which is very clear is that I have not built and shipped Facebook -- the current company I'm with -- with X million people hitting every month. We're not using it. This isn't exactly 'Toran-stamp of approval' here but I do mess around with -- this week in particular -- Rx which I like. I think Rx is just something that it changes the way you think about the way programming, especially async programming works. I definitely cannot comment much on Rx other than I like Alex's challenge to the community on your podcasting. Go use Rx, even if you use ember-concurrency or don't use ember-concurrency, how to break your brain. It will be for the better. Actually, Jay did a mini code review with me because my first pass at Rx was just using the fetch-promise because I was like, "I want Rx for side effect modeling but I wanted to still work with Ember acceptance testing," because I still feel like Ember is leading in that way, as you guys talk about in podcast recently. What was really cool is actually there's a shim that obviously Rx has its own little Ajax thing but it is not actually Promise-based. The advantage of this that Jay called out is in the Promise-based, where I'm using ember-fetch, let's say and I'm just wrapping it with Rx, those Promises are still not really cancel-able so what Jay was warning me about is like, "If you're going to use this quick and dirty, great. but in a real app, these will still queue up in Chrome or IE and block the amount of network requests you can actually make," so don't use Promises, even though they're very familiar. Use this operator, I think it is or a helper inside Rx which is the Ajax non-Promise-based operator. Long story short though, there's a good amount of work involved that grass is greener. In Ember Data, if you've ever used 'belongs-to' or 'has-many', you have done the most magical thing right there. In all the right ways, it is amazing because once you're in Redux and you're like, "I have this very nested object wrap," but Redux isn't meant to operate on this nested object wrap. It's meant to operate on this single tree structure, at these many top level entities as I can. As a project that's pretty popular in React called normalizer, you will likely use this project eventually. Maybe, not your first 'Hello, world', but you'll use this to actually break apart or truly de-normalized the structure. What that does a lot of times is if you have a blog with comments nested all inside of it from your JSON API call or your GraphQL call, that's fine coming from the network. But since you're going to have different components listening for just the comments, maybe or different components that just listen and re-render when the blog name changes and they don't care about the comments, you want to actually de-structure that so you have a separate blog top level item and you have a separate comments top level item. They're still related so the blog can get through its comments and vice versa as belongs-to and has-many works in Ember Data but you've got to do that work now. There is, of course magical projects like redux-orm that I just can't speak to how well they work or don't work but they try and solve the more Ember Data, look at the problem which has define this and there's the RM take care of the magic for you. I actually don't mind normalizer. It's just something people should be aware of because it's more work. You've got to break that apart yourself, just as much as writing your own network requests. You're, hopefully not going to duplicate Ajax all over the place but you will have to do the work that you otherwise do not do in Ember Data for sure. CHARLES: It's very interesting. If you look at the Ember Data internals, it sounds like the Ember Data store is actually structured very similarly to the way you had structure a Redux store using something like normalizer where you have these top level collections and then some mechanism to both declare and then compute the relationships between these collections of top level objects. But I want to go back to your other point too. I just wanted to say this. Toran, you know, don't sell yourself short because you give an incredible amount of time, an incredible amount of support to the community. You're very active in the Ember Redux channel. When problems come up, you think about them, you fix them. Even if you're not actually watering the trees, you're planting the seeds. I think that's actually great. I think that is a very valuable and necessary role in any community is to have people who are essentially the Johnny Appleseed of a particular technology. I think you go around and you throw these seeds around and see where they take root, even if you're not there. You're on to the next shady lane to plant seeds, rather than stay and enjoy the shade and the fruit of the apple trees. TORAN: Yeah. I appreciate your kind words there because a couple of years ago, I got into open source because it provides good personal branding. It's like, "This project, it's Toran's. We should hire Toran." It just makes you look more from that perspective. It also gives you almost a way to skip out on tough interviews at times if people are like, "This guy have a decent program. Let's take a look at his PR. He communicates with other humans online." It gets rid of some of that. But there is a dark side. We don't talk about it because there is an upside to it, especially for consulting but the dark side can be time commitment: how much bandwidth do you have outside of your family life, your hobby, if you have one and any other open source or work-related stuff that you already have to do. For me, this is really an exercise in thought leader-y stuff. I saw the benefits of this. It made my Ember better. Even if I wasn't using Redux, it just made the Ember code I wrote at work better. It inspired me to look at different things like ember-concurrency and Rx, things that are just way out of my comfort zone two years ago. I think those are all the benefits that come with it but the easiest part is got to be some value from it. The juice is it worth the squeeze. I think the community we've built and the people using it and the problems we're solving are all definitely worth the squeeze. CHARLES: It definitely is and you can tell from the vibrancy of that community that a lot of people are experiencing that value from it. To your point, I think something that is often lost on people is that you can actually use a project without actually using it. I think that there might be many people, for example in the Ember community that have never use React but are actually in fact using it because of the wonderful patterns that have come out and it's brought to the fore. I had thought about immutability, certainly on the server side but I had thought about it really deeply on the client side until a library like React came along and people started talking about it. I would say before we actually started using React the library, we were using it in thought. You touched on that when you were saying, "It's changed the way I think, it's changed the way I code." Even it's changed the way that I do things at work, in fact you're using it in spirit, if not the actual structure but I almost feel like that's more important. It's longer-lasting and has a greater impact on you, 10 years down the road or even five years down the road when neither of the technologies that we're actually talking about today are even going to be in wide use. TORAN: Yeah, that's true. In fact the one thing I would call out that people check out, I think Alex mentioned this or at least you guys have to talked about it in passing but definitely, sometime this weekend, watch the Simple Made Easy talk by Rich Hickey. It will definitely make you think differently, regardless of the simple side or the easy side that you follow on, projects of course make tradeoffs both sides of that but it is a great talk. Especially, if someone who's been programming six months or a year or two years, they're going to get huge benefit from it, just as much as someone older like myself who has got 10 plus years in the biz. CHARLES: Yeah, I know. That is a fantastic talk. We need to link to it at every single show. TORAN: Exactly. CHARLES: Well, I think that is a fantastic note to end on so we will wrap it up. That's it from Frontside for this week. We're going to have you back, obviously Toran there's so much that we could cover. Six months down the road, we'll do part three but for now, that's it. Thank you, Wil. Thank you, Toran for podcasting with us this morning. TORAN: Thanks for having me on guys. I really appreciate it. CHARLES: Then everybody else, take it easy and we'll see you all next week.
After the cliffhanger left in Episode 62: UI for U and I, we follow up with a short discussion about how we specifically do UI Testing at The Frontside in Austin, Texas. Resources: Tweet that led to this discussion Unit Testing Acceptance Testing Ember CLI Mirage Percy Test-Driven Development Transcript: CHARLES: Hello everybody and welcome to The Frontside Podcast, Episode #68. My name is Charles Lowell. I'm a developer here at The Frontside and podcast host-in-training. I'm here today with Jeffrey and Elrick, two other developers here at The Frontside. We are going to carry on where we left off back in Episode 62. There was an implicit promise to talk about the way that we do UI testing. JEFFREY: We had a cliffhanger. CHARLES: Yeah, we did. It ended with a cliffhanger and someone actually called us on it, which I hate it because they're making more work for you. But Ian Dickinson from Twitter writes, "Hey, you guys promised to talk about how you do UI testing at The Frontside but never actually delivered on that promise." We're going to try and make good on that today. JEFFREY: We like to resolve our promises. CHARLES: Oh! [Laughter] CHARLES: You've been on vacation for a week, Jeffrey and I forgot what it was like to have you in the office. JEFFREY: Not enough code puns. CHARLES: Oh, code pun. There you go. It's like CodePen except all of the code has to make puns. I like it. Internet is awash with people bloviating about testing so we figured we'd take our turn at it. We promise to keep it short. We'll keep it focused but it seems like a value that we do have so might as well talk about it a little bit. You guys ready? JEFFREY: Let's talk about testing. CHARLES: I think one of the best things to do is to use something concrete, not just to talk about abstractions, not just to talk about things that might be but we're actually starting a project here in three days. It's going to kick off on Monday and testing is going to be a key part of that. Why don't we talk a little bit about what it's going to look like as we kind of lay the groundwork for that project? JEFFREY: As we start this project, the very minimum baseline that we want to get immediately is acceptance tests in the browser. We want to make sure that when you fire up this app, it renders, you get the fallbacks for the basic functionality of this app immediately. As we're building features on top of this app, that's when we bring in unit tests, as we say, "We're building this new component. We're building this new feature. That's part of this app. We're going to use test driven development and unit tests to drive the creation of that," but ultimately, our test of quality for that app and our assurance of quality over the long term comes from the acceptance testing. CHARLES: People often ask the question, "When is it appropriate to write those unit tests? When is it appropriate to write those acceptance tests and how much time so I spend doing each one?" Personally, when I was starting out with testing many, many, many years ago, I really, really like unit tests and I like developing my code based around unit tests. The thing that I liked about them was that they were fast. The entire test suite ran in a matter of seconds or minutes. JEFFREY: And you get coverage. CHARLES: Yes. JEFFREY: Like you get your coverage numbers up. It is like every line that I wrote here has some kind of code coverage. CHARLES: Right and it feels really good. I also think that unit tests really are great for mapping out the functionality in the sense of getting an intuitive feel of what it's like to use a unit before it's actually written. You get the experience like, "This is actually a terrible API because I can't write the test for it," so obviously it's not flexible and it's really mungy so I really, really enjoyed that. The thing that I hated most is acceptance tests. They're hard to write. They were slow to run and it seemed like when they would break, it was not always clear why. JEFFREY: Wait, so we were just singing the praises of unit tests. What's wrong with them? CHARLES: Part of it is kind of the way that you conceive of a test: what are you actually doing? I think if you think of tests not so much as something that's there for regression or something that's there to drive your design, both of which are very, very important things but more is just kind of measurement: taking a data point, what is it that I want to measure? In the case of a unit test, I want to measure that this library call does exactly what I think it's going to do so I can take that data point, I can put it on my spreadsheet and I can say, "I got that base covered." The thing is that an acceptance test just measures a completely separate thing and by acceptance test, we're talking about testing as much of the stack your entire integrated application as you can. Oh, boy. Don't get me started on terminology but when you have an acceptance test, what you're measuring is, "Does my application actually function when all the pieces are integrated?" So you're just like a scientist in the laboratory and you're making an observation and you're writing it in your notebook. Yes or no, does it work? In the same way that you're taking a perception when your eye perceived the photon of light, you can say, "That thing is there," when it bounces off the chair in the room, I know that the chair is there. If what you want to measure, what you want to perceive does in fact my application work for user, then an acceptance test is the only thing that will actually take that measurement. It will actually let you write that data point down in your notebook and move on. I think that the problem with unit tests... Well, there's nothing wrong with them, it's just they don't observe your integrated application so that means you're blind. It's only part of the story so that's why we find that acceptance test really are like the highest value tests, even though they suck to write and even though they take a long time and even though when they break. Sometimes, they break at some weird integration point that you didn't see and that's really hard to diagnose. But you know what? That same integration point, that's a potential failure in your app and it's going to be just as weird and just is esoteric to track down. That's my take on it. One of the things that you're describing, Jeffrey is we set up those acceptance tests suites, why do we set them up? What, because they're part of like a process? JEFFREY: Yeah, that process is that we start at the very beginning when our project on the very first day is when we like to set up continuous integration and make sure the repos all set up, make sure the deployment pipeline is set up. CHARLES: What do you mean by deployment pipeline? JEFFREY: The deployment pipeline looks like, "We've got our a good repo and our version control under consideration and from there, anytime there's a push to the master build, any time a pull request gets accepted, we build out an continuous integration server, whether that be Travis or Circle or any of the solutions out there. We want to run that entire suite of acceptance tests and whatever unit tests we have, every time we have a push of code to past a person's local box. CHARLES: Right, a push to master is tantamount to a push to production. JEFFREY: Yes, that is the ideal. Any time that the code has been validated that far, yes this is ready to go to production and our acceptance test suite validates that we feel good about this going out. CHARLES: Yeah so the thing is if all you have is unit tests, you have no way of perceiving whether your application will actually work in a real browser, with real DOM events talking to a real server, do you really want to push to production? [Laughter] JEFFREY: We had some client work recently. We do have a lot of acceptance tests coverage and actually a lot of unit tests too. We changed the browser running on the CI server from Chrome 54 to 58 and uncovered a lot of bugs that acceptance tests coverage found, that unit tests just would never have revealed that the end user had a problem. CHARLES: Now, why is that? Let's take it down a layer now, when we do an acceptance test, what does that actually mean? What are we describing in terms of a web application? JEFFREY: We're describing that we have an application that we're actually running in a real browser. Usually, you're going to have to have some kind of stubbed out things for authentication to make sure you can get around those real user problems but you're actually running the real application in a real browser, not in a headless browser but an actual browser with a visible DOM. CHARLES: Yeah, not a simulated DOM. JEFFREY: Exactly so you can surface the kinds of real problems that your customer will actually have. CHARLES: In terms of interaction, you are not sending JavaScript actions. JEFFREY: No, you're firing DOM events. You're saying, "Click this button," or, "Put this text into this input," and the underlying JavaScript should be pretty opaque to you. You shouldn't be calling directly at JavaScript events in any acceptance tests. You want to rely on solely what's on the DOM. CHARLES: Yeah. I would say the ideal set up with an acceptance test, you should be able to have your acceptance tests suite run and then completely and totally rewrite your application, let's say rewrite it from Ember to Angular or something like that and you don't have to change your acceptance tests suite at all. JEFFREY: Yeah, usually the only binding you have is IDEs or classes or whatever you're using to select elements and that should be it. CHARLES: Right, you're interacting with the DOM and that's it, so now in terms of the server, most modern web apps -- in fact, all of them -- certainly in the kind of swimming pools in which we splash, have a very, very heavy server component. There's a lot of requests. Just even load the page and then throughout the life of the page, it's one big chatterbox with the server, what do we do there? JEFFREY: That's when we need to pull on a tool that can mock a request for us, the one that we fall back on a lot is Ember CLI Mirage that is built on top of Pretender. It's a really nice way to run a fake server, basically. I would even take that a step further and would love for the tool to be another server completely that's running on your local box or your CI box or whatever so that you get the full debugging available in developer tools and you actually see those requests as if they were real ones that just coming from a fake server. CHARLES: Right, as Jeffrey said right now, we use a tool called Mirage, which for our listeners in the Ember community, they know exactly what we're talking about. It's the gold standard. What it does is it actually stubs out XML HTTP request, which is what most of the network traffic in your browser after the initial load is encoded in. It's got a factory API for generating high quality stub data so your code is actually making a [inaudible] request and it's all working. Unfortunately because it's stub, as you said none of the developer tools work. But there's been talk about making Mirage, you service workers so that you would have all of that running. You could still run Mirage inside your browser. It would be a different process. I think the server's workers run off thread. That actually is very exciting. It's a great tool. I think it's a great segue to talk about one of the things that we love. This is absolutely a mandatory requirement for us, when we start a project is to have acceptance testing in place. Back in the battle days, it would take us, I want to say like two weeks just to set up our acceptance tests suite for a project. This is when we were doing Backbone and this is when we were doing the early, early days of Ember. We would start a project, we'd spend all of that time upfront just so that we could have an acceptance testing framework and that was actually a really hard sell because it's like, "What are you doing? You don't have anything to show," and it's like, "Well, we're actually setting up an app-a-scope," so that we can observe your application and make sure that it's running. The very first thing that we do is we set up an app-a-scope and it's like, "Nobody sets up an app-a-scope," so they can actually see their application. But one of the great things that has happened and one of the reasons we love Ember so much is that you actually get this now pretty much for free. I think there's still some stuff that's very white box about Ember testing and a lot of times, we talked about that ideal of you should be able to swap out your entire implementation of your app but your acceptance test suite stays the same. That's not quite possible in Ember. You have to know about things like the run loop and it kind of creeps its way in. But it's 95% of the way there. It's mostly there. It's good enough. It's better than good enough. It's great. You just get that for free when you start a new Ember project. JEFFREY: We've talked a lot about the Ember ecosystem and what we like there about testing and we're going to be doing some React work soon. What's the story there? CHARLES: Well, I'm glad you asked, Jeffrey. Yes, we're going to be doing some React work soon so again, this is a new project and it's absolutely a 100% ironclad requirement that we're not going to develop applications without an app-a-scope but I think that the React community is in a place where you kind of have to build your own app-a-scope. You know, actually having kind of scoured the blogosphere, there are a lot of people in the React community who care very deeply about acceptance testing but it does not seem like yet a mainstream concern or a mainstream pathway. For example, Jest which is the tool that is very, very popular in the React community and I was actually really excited about reading the documentation. It doesn't even run in the browser. It's Node.js only, which for us is it's a nonstarter. That makes it really fast but it actually is not an app-a-scope which is what we need. It does not lead us to actually observe our application. It lets you observe the components from which your application is built but actually you're blind to what your application actually looks like in production. I was kind of bummed about that because I know that there is work and I know that the maintainers of Jest carry very deeply about making it run in the browser eventually. There are some experimental pull requests in the couple branches. Who knows? Maybe those are even merged right now but the point is, it's still very early days in there. There are a couple of people who have used like Nightmare that they've booted up and they're kind of controlling the running acceptance tests in Nightmare from Jest. Now, that sounds great but part of your app-a-scope needs to perceive different browsers. In fact, users don't use Nightmare.js. They use Chrome, they use Mobile Safari, they use Firefox and normal Safari and Edge and what have you. There's actually a great set of tool. Testem and Karma are all set up to be able to run all these browsers and have those browsers connect to your test suite and run them in parallel. Again, that's kind of a bar. That's what we're actually working towards right now. We're running some experiments to try and use Mirage and Karma with Mocha to actually get the multiplicity of browsers, actually using browsers, be able to use real DOM events and test a real API or as real as API as we can get. I'm kind of excited about that work. I hope that the acceptance testing story gets a lot better in the React community and it's early days but like I said, we care very deeply about it. I know a lot of other people care very deeply about it. Some people have some different feel like it's not necessary. Some people feel like they don't need an app-a-scope. They're like, "You know what? My application is there and I know it. I don't actually want to look at my application but I know it's there." I think that's actually the pervasive model at Facebook and obviously, they're doing something right over there. Although, who knows? I'll keep my mouth shut. No, but seriously, there's some good software that comes out of there. The Facebook application itself is pretty simple. Maybe it's enough to perceive your components and not have to perceive your application as a whole. Or there are other ways that you can go about it. If you've got lots and lots of money, you can have people do it and they do a pretty good job. I understand that's another strategy. In fact, I think that's what Facebook does. I've read a lot of the debates between why they go with unit tests, why they don't go with acceptance tests? They've got a QA Team. They probably has their own tools or who knows? Maybe they use like Capybara or WebDriver or Selenium and those tools are really, really excellent and they are truly, truly black box tools. JEFFREY: Elrick, you brought up an interesting new angle that I think is starting to come of age that I think as an awesome addition to the toolkit which is visual regression testing. How far that's come in the past couple years and how valuable that is for getting the CSS confidence. Unit test coverage is great for testing your individual components, for testing JavaScript functionality but ultimately, the confidence around CSS comes from visual regression testing. That's the only way you can get that and I think that's helping the CSS ecosystem to be a little healthier to encourage better practices there and having engineers who are more averse in JavaScript and not as averse in CSS, feel more comfortable making CSS changes because they have that type of testing to them. ELRICK: When you're doing things programmatically, you wouldn't really know what's going on visually unless you physically go and check it and then that may not be the best solution because it takes a lot of time. JEFFREY: Yeah. CHARLES: The tool that we have that have the most direct hands on experience with is Percy and I think, you guys have more experience with it than I do but it's very intriguing. When I heard about this I was like, "How does this even work?" JEFFREY: It's so great to have visual diffs. As it runs through your acceptance test suite, you specify, "Take a screenshot now," so you can compare against what came before. Sometimes you run into some finicky things that are like, "I have some data that's not completely locked in." One of the most common dis I get is, "Hey, there's deep change in the screenshot from acceptance tests." I'm like, "Oh, because I've hard code that date. Always do the same," but it's great at servicing and like, "This button move 10 pixels. Did you intend to do that?" In particular, when we upgraded a component style guide library, we noticed a lot of changes that came out of that. Actually they were okay changes but it was important to know that those has changed. CHARLES: Right and there were actually some visual regressions too. I remember there was some button turned red or something like that and it's like, "Oops, something got screwed up in the cascade," which always seems to happen. JEFFREY: I think it has caught some bug before and some changes we've made to select component and like, "This has the same functionality," but actually the classes around this particular component changed. "It doesn't look the same. What happened?" It's been a really valuable tool. CHARLES: The acceptance test from the code perspective actually completely and totally worked, right? JEFFREY: Yes. Exactly. CHARLES: But, yeah, acceptance test didn't catch it because they were not perceiving the actual visual style of the application. It's an enhancement or an add-on to your app-a-scope so that it can see more things and you can perceive more things. I love it. When you have an acceptance test suite, then it allows you to do those power add-ons because you're actually loading up your whole application in a real browser, in a real DOM and you're making it go through the paces that an actual user will do so you can do things like take visual diffs. If you're just doing unit tests in a simulated DOM, that's just not a possibility because you're not actually running the painting algorithms. Whereas, an acceptance test, you are so it allows you to perceive more things and therefore, check for more things and catch for more things. JEFFREY: I think my next area of exploration that I'm interested in is, let's say you have a web app with sound effects. Is there something to validate the sound effects? Or take a video capture of your app because that would be really cool. I'm sure there's something out there but I think I'll be my [inaudible] to go find out. CHARLES: And then we actually touched on this on the episode on accessibility, you can test your accessible APIs when you're running acceptance test. Another thing that I might add is as terms of regression testing, acceptance tests have a much longer term payoff because I feel very strongly about test driven development for your unit tests. I think unit tests are really, really great and if you're building tiny units, especially novel ones that you haven't really built before, they're not cookie cutter things like a component or something like that. It's some unique service or just a utility library or some bundle of functions. Using test to drive those out is extremely important. But I almost feel like once you've done that, you can throw those tests away. By all means, they're free to keep them around but your tests also serve as a fence, a wall, a kind of exoskeleton for code and that can be great except when you want to change and refactor the code around, you have to tear down the wall and you have to break the exoskeleton of the code. If your code is siloed into all these tiny little exoskeletons, elsewhere it's going to be very hard to move and refactor or it's going to be hard to do without rearranging those walls. I guess my point is, with an acceptance test, you're making that wall very big. It covers a lot of area so you can have relative freedom to change things internally and rapidly. While the acceptance test is slow, the speed for internal change that engenders, I think is worth it. I think that's another pay off because I think that tests do have a shelf life and I think that the shelf life for unit tests is very small. Whereas for application level tests, it's very large. Then I guess, the final thing is really, there's no such thing as acceptance tests and integration tests and unit tests and blah-blah-blah-blah-blah, all the different types of tests. It really is just a matter of scope. It's like how big are the lines that you're drawing so acceptance test is a type of test that we give a name for tests that have a very high amount of scope. They cover a lot, where in unit tests, have a very small scope but there's a whole spectrum for every piece of code that is running in between, there's a whole spectrum in between. All righty. Well, this concludes Episode 68 of The Frontside Podcast. I told you it was going to be a short one but I think it's going to be a good one. I know that this is a subject that's very, very near and dear to our hearts. We don't dedicate explicit time to it all that often but I'm sure we will return to it again in the future. Jeffrey, Elrick, thank you guys for joining us for this discussion and we're out.
We hear about the days when Charles was drilling for oil. Also, waffles: "I suffered years of floppy-waffles." We also discuss the deal with kombucha, to no effect, really. And, as always, Charles' emacs configuration tip of the week.
Alex Matchneer: @machty | FutureProof Retail Show Notes: 01:07 - The Introduction of ember-concurrency 02:15 - What is ember-concurrency? What are the problems it solves? 05:37 - Why not use observables or other alternatives? 09:49 - Could observables be used in conjunction with ember-concurrency? 12:16 - Simple Made Easy 14:23 - Coming Soon to ember-concurrency 16:04 - Communicating Changes in State; Glimmer Reference Primitives 23:09 - Using References 29:31 - Submitting RFCs; Adding Pipelines 32:10 - Pipeline Use Cases Resources: ember-concurrency The Frontside Podcast Episode 007: The Ember Router with Alex Matchneer The Frontside Podcast Episode 019: Origin Stories with Tom Dale and Alex Matchneer Introduction to ember-concurrency by Alex Matchneer from Global Ember Meetup RxJS Rich Hickey: Simple Made Easy Glimmer.js redux-saga Lauren Tan's RFC: Cancellable task pipelines Railway Oriented Programming Apache Kafka Transcript: CHARLES: Hello everybody and welcome to The Frontside Podcast, Episode 67. My name is Charles Lowell, a developer here at The Frontside and podcast host-in-training. With me today also is Elrick Ryan, a developer here at The Frontside. Hello, Elrick. ELRICK: Hey, what's going on, Charles. CHARLES: Now, we have with us today someone who we love to have on the show. Everybody probably already know him. I know the first time I actually heard about him was when we had him on the podcast the first time, I was like, "Who the hell is this guy?" But since then, he's become one of my favorite developers, just with all of the things that he's done, from Router.js to more recently ember-concurrency. We have Alex Matchneer on the program. ALEX: Hey, everybody. Thanks for having me. CHARLES: Hey, Alex and you know what? I pronounced your name right this time. First time out of the gate. Boom! ALEX: Nice. Which one did you go with? Matchnear? Matchner? [Laughter] ALEX: I really actually don't even know which ones correct anymore. CHARLES: Was it about a year ago that you first introduced ember-concurrency? ALEX: Yeah, I had a really embarrassing introduction of it at an Ember Meetup in January before it was really done and I just kind of botched it and didn't really introduce why it was even solving problems. Then a month later, I had some time to refine it, driven by the feel of that embarrassment. I guess around February of last year, it's been pretty much in its present state. CHARLES: I remember when it came out. I must've seen the non-botched version because I remember hitting the ground running with it and being able to refactor all of this code. I definitely know that I got the honed version because you provided in that initial blog post a whole host of examples like what are the symptoms, what are the cases where it solves and then before presenting the solution. I think that was great because I didn't even realize that I had a lot of pain. I didn't realize that at all. I didn't realize I had a problem but then you were very, very elegantly packaged the problem with the solution which is always great because otherwise, it's just complaining. Maybe we should talk a little bit about -- I don't think we've officially talked about -- ember-concurrency. Even though it's been out for quite a while, the way that you model these concurrent processes using the stack is just pretty incredible. Do you want to just very briefly touch on what the problem is and what have lead you to this solution? ALEX: Sure. It's a little bit difficult to sort of succinctly say what ember-concurrency is because it kind of hits them like five different separate but kind of related but not really pain points. At its core, it's just like a task primitive and it's definitely not the first library to ever introduce that the JavaScript, I think particularly when the generator function syntax was introduced into the spec, I think a few years back. Dave Herman who's also known as, I think a Little Calculist. I think he works on the TC39. I always get those groups a little confused in my head but he introduced a task.js library that let you use the generator function syntax and then lets you yield Promises to sort of pause where you were in that task and then continue when it resolved. It had some support for cancellation. It played well with Promises and I brought that to Ember in a way that fit really nicely with Ember more than it probably does or will with other frameworks like React or Vue. By bringing it to Ember, basically if you're implementing any feature that involves async, if it's a button that needs to show that it's been clicked while you're waiting for some response to come back from the server, instead of using Promises, instead of using actions, here's an ember-concurrency task. It makes it easier to express that operation you're trying to do and it makes it really easy to drive your UI with state that comes from the state of that operation -- Is the test still running? Is your form still submitting? -- Rather than having to manage a bunch of mutable flags and properties on a component or state yourself and likely get it wrong. CHARLES: Right. Essentially asynchronous processes is like a state machine and before, we were kind of managing that state machine by hand but I think what's so brilliant about this task-oriented programming, I guess is maybe a way to put it because I really think that some of these ideas are universal and not specific to ember-concurrency. But it almost like it uses the stack, just your normal programming stack to track where you are inside of a process, rather than what it felt like what we were doing before, which was managing this state machine by hand, if that makes any sense. ALEX: It does make sense a lot of sense. A lot of people ask me, if you're going to go into sort of async territory, why aren't you using something like RxJS? Rx is observables and kind of popularized by the Netflix crowd who did a bunch of presentations on them. It's super popular these days. But one of the things I really like about RxJS or at least one of the realizations I had is that I think you're still building a state machine. You're just expressing it using different primitives. In Rx, you're still building a state machine but in Rx, they make you think about it in terms of streams and events firing over time. In ember-concurrency, also you're still building a state machine but you're using the generator function syntax and the call stack like you mentioned as another way of expressing that state machine but with a lot less code. CHARLES: Right. I was actually talking to someone about ember-concurrency just a few days ago and they were saying the same thing, "Why not use observables," and at least from my perspective, maybe I didn't quite understand the question because I feel like observables are kind of only one of the concerns that ember-concurrency addresses. I'm curious when people talk about alternatives to ember-concurrency and put observables forward, maybe I don't understand it because I usually think you might be able to use observables to register the currently executing task state and every time it changes, emit a new state and is then observed by your observable subscribers. But modeling the actual process using observables does seem weird to me because with observables, they seem like very purely functional and not heavily stateful. I don't really have that much experience with it. What's meant by using observables as an alternative? Maybe we can get more into those like how you would construct a stream or something like that? ALEX: I think the canonical Rx example of something that's elegantly expressed in Rx that would be really hard to do in just normal JavaScript, if you weren't able to use observables, is that typeahead search where as you type characters into a text field, it's already beginning to hit the server and see what you might be searching for so it can drive the state of some drop down menu. That's probably the most popular example out there because one of the things it demonstrates is that if you want to debounce, you want to allow for the user to stop typing for like 200 milliseconds before it actually hits the servers so you don't overwhelm your server, then just add a debounce operator. You've basically transformed a stream of keyboard events into that text field into something that only kicks off after it hasn't gotten an event for 200 milliseconds. If you already had a working prototype in vanilla JS and you had to debounce it, you've got to move a bunch of stuff around, you've got extract something into a function, you've got to deal with cancellation. But all those things are kind of pretty elegantly built into Rx and if you can train yourself to think in terms of streams of events, that inspires you to think about where else you could apply that in your app. I think a lot of people have felt that it's like winning, most powerful abstraction that you could think about. That's why things like cycle.js are a thing or redux-observable or just anybody working with observables in the Netflix territory. I personally find [inaudible]. If you're going to express certain processes, Rx is the way to go but it has drawbacks which is it is really hard to learn. It took me a very long time and I'm pretty good at it but if you're going to adopt Rx in your code base, then a new developer comes on, it's going to be a pretty long time. In my experience, sharing some of the Rx code I've written with fellow very talented developer, it takes a really long time to explain how to invert your thinking and think of things in terms of events. If you can get to that point, more power to you but what I found with ember-concurrency stuff is you don't have to completely invert your thinking and think of everything in terms of events and streams of events. You can use this task primitive which feels really pretty close to the code you're already writing but gives you a lot of the safety guarantees and just makes it really easy to use this derived state to drive templates. Rx is a powerful paradigm and sometimes you need that sort of event-driven push based model but I think when people wonder why aren't you just using observables, they haven't really grasped how easy and familiar it is to use task and get it right on the first try and with a lot less code. CHARLES: Right. You're able to leverage the fact that I understand what a JavaScript function looks like and the sequencing is implicit by just the order in which you were numerate the steps, right? ALEX: Right. Because I think that Rich Hickey of Clojure popularized the divide between simple versus easy and Simple Made Easy is one of his popular talks that everyone should probably -- CHARLES: It's a great talk. Yeah. ELRICK: Do you see an area where observables could be used in conjunction with ember-concurrency? ALEX: It's kind of. It's been hard for me to find that use case. Probably, there's a handful of use cases where maybe it's a little awkward to think about to have something that would be elegantly handled in Rx would be model using tasks but it really hasn't struck me enough in some of the apps that I'm building, to really try and flesh that out. CHARLES: I would be curious to see a side by side comparisons. I build a lot of auto completes using ember-concurrency. I built a lot of asynchronous processes using ember-concurrency. What would that look like using nothing but Rx and just be able to have it on the left-hand side of the paper, then Rx on the right hand side of the paper are easy. ALEX: I'd be very surprised if you could find an Rx example that is less code than the task equivalent because as much as I think the autocomplete example is the best canonical example of Rx, once you actually start making something that's production-ready, you want to start driving the button state while it's running or to show a loading indicator. When you start deriving other observables off of the source observable which is the user typing into the text field, you start having to worry about, "I'm dealing with a cold observable. If I create another stream based on it, it might double subscribe and I might kick off two things. I actually want to use a published.ref version of the stuff." To actually get away from a toy example into something that's actually production-ready, requires a lot of code. From my own conversations with the people working on Rx, there's a lot of people that are working on it and they're pragmatic about it and don't think that you have to be just purist functional all the way. But when they actually ship production code, they usually resort to using like the do operator. With Rx observables, which is basically an escape hatch to let you do mutations and side effects in what is supposed to be this monadic functional thing. If the paradigm is breaking that quickly to do production code, I'd wonder if maybe there is something better out there. I just kind of keep that in mind but I'd definitely think there should be a bake-off or comparison of how you do things in both the task paradigm and observable paradigm but I think you'd find that in most cases, just do a lot more with a task, with a lot less code. CHARLES: I want to go back to the point you were about to make about Simple Made Easy. ALEX: On the divide, ember-concurrency is very easy. I still choose easy. In the case of reservable, I'm constantly choosing easy over simple and then it always helps me out because I've made that decision. I think most people inspired by Rich Hickey from the Clojure community, would look at ember-concurrency and be like, "At a task that combines derive state and does five different things seems kind of gross. Why don't you just use observables," and the result of that if you follow it through is that you end up writing a bunch of observable code that is a mess in streams and going in different directions and you've written something that's really hard to understand, even if it's seasons Rx developers looking at the code. It's just very easy to write things that are tangled. CHARLES: It's always good to have simplicity but also a system that simple without ease, I think is far less useful because like I said, it's always going to be a tradeoff between simple and easy but the problem is if your system is too simple, then it means that you're shouldering your day-to-day programming task or shouldering the complexity and you have this emergent complexity that you just can't shake because your primitives are just too simple. You could be programming in assembly language or something like that. That's really simple. You need to be able to construct simple primitives on top of simple primitives so for your immediate need, you have something that is both simple and easy, if that's ever possible. Certainly, ember-concurrency is easy and I think it just means there's maybe work to do in trying to tease apart the different concerns because like you said, there are five. But in real complex systems, there are five bajillions, maybe teasing apart those individual concerns that is composed out of simple primitives. I'm sure you've thought about that a little bit of how do I separate this and make these tasks compose a little bit better and things like that. ALEX: This is a nice segue because it might tie into some of the work that's going to be going into ember-concurrency in the next few months. A big theme of EmberConf is actually, a lot of people are joking that it should just be called GlimmerConf because a lot of it was talking about how Glimmer is going to be this composable subset of Ember, like Glimmer is going to be the rendering layer and then there might be a Glimmer router and a bunch of these Glimmer components that once you npm install them, you get Ember. Glimmers is a chance to think about Ember as a bunch of components working together under a really nice rendering layer. There's definitely some interest in bringing ember-concurrency in thinking what is so-called Glimmer-concurrency going to look like. Part of thinking about that is going to mean teasing apart some of these details as you were just saying. I don't have a lot of specifics to give right now, just that there is a lot of interest in making sure at the very early on, there is some sort of Glimmer-concurrency equivalent. Generally speaking, as part of that process is the question of how do we bring these magical ember-concurrency parameters to just Node or just vanilla JavaScript in general. Perhaps you could use these kinds of tasks on a server and in other environments. I think there's some questions of the way the ember-concurrency bundles together derived state with the actual tasks runner, are you actually going to use that derived state in the server setting? I think some of these pieces are going to have to come apart a little bit. I don't have very concrete ideas for how that's all going to look in the end. Just that I have faith that it will happen pretty easily and the result of it is going to be something that fits pretty nicely into Glimmer as well. CHARLES: Yeah, I hope so. It certainly seems like one of the core issues right because Glimmer-concurrency really should be universal. It should be some -- I don't want to prescribe your work for you -- ALEX: I don't mind. CHARLES: That wouldn't be cool. I mean, Glimmer is very stripped down. You have a very little bit on top of a raw JavaScript environment so if you're going to go there, it'll makes sense. What is this concurrent process builder look like using nothing but JavaScript? It seems like one of the hardest problems is to disentangle it from Ember object because the way that it currently computes that derive state is very intertwined with Ember object. You know the details of this more than I do but it seems like that's one of the biggest challenges is how do you communicate those changes in state without using that? That what I was thinking, it would be a good case for using observable for ember-concurrency, although not for probably the reason that people are thinking, which is for task composition and stuff but I'm very curious. ALEX: Likely the first stab at that direction would probably be using something similar, if not exactly these Glimmer reference primitives. Maybe it is worth talking about that. References are one of the core primitives that's used by Glimmer and it represents a value that might change over time and it's a value that can be lazily gotten, whereas observables, you have something that's firing events every time something changes and it makes the whole pipeline process it right then and there. With references, when something changes, you just tell the world like something's dirty. Then at a later time, maybe when in a request animation frame or some point where it actually makes sense to get the latest values, then it goes through and finds out everything that changes, does a single rerender. What it means is that you don't have the observe recode that's firing every time some value has changed. It's one of the guiding abstractions in Glimmer that makes it possible for it to be so fast and performant. It is very likely that a vanilla version of what ember-concurrency does uses references because those are already separate from the Ember object model and actually are used today in conjunction with Ember object model with the Glimmer that works with Ember today. I think that's probably, to me a first step. Clearly the reference attraction has worked wonders for Glimmer. I prefer to probably use that than observables and the push-based. CHARLES: Observables or something else. That is really, really interesting because there's nothing like vanilla JavaScript programming these days, like the equivalent of a Haskell thunk where you're just passing these things around but you're not actually using them until you actually want to pull a value. At that point, you kick off the whole chain of computation required to get that one value that you need. But it immediately brings to mind and I don't know if this is of concern to you but I was very, very enamored of Ember objects back in the day, in 2012. I was like, "Wow, this is amazing. This solves every problem that I've had." It has been a great companion and I've built some really great stuff on top of it. But now it's definitely turned into baggage. I think it's baggage for libraries that I've written and we're talking about it in the context of it being baggage here and being making it more available to the JavaScript community so I worry a little bit about Glimmer references. Would they possibly turn into something like that and could you counteract that by maybe trying to evangelize them to the wider JavaScript community like, "Here's a new reactive primitive," so that we don't end up in an eddy of the JavaScript community, do you think there's value in trying to say, "There should be some standard in the same way of observable, which is an emerging standard is for eager reactivity, have some lazy reactivity standard," or maybe it's too early for that. That might be a way of future-proofing or getting insurance for the future so you can say, "We can confidently build systems on top of this primitive." ALEX: If the worry is something based on Glimmer reference as it's going to turn into the same baggage or [inaudible] or whatever, that maybe Ember object has turned into some apps, some applications, some libraries. I'm not sure. I guess I don't really see that happening and I know that it's already gotten some validation from some of the people that have worked on Rx. In fact, a very useful primitives for certain kinds of workloads. As much as evangelism certainly helps. It's already off to a much better start than this all-powerful, god object that you can only interact with if you're using .get and .set functions. It's very lightweight. What I'm trying to say here is that there's UI workloads and then there's server-driven workloads and using Rx for both cases means that Rx suffers as a library because in the UI workload, you want something like references where you want to let a bunch of things change and then update stuff in one pass just a tick later or later in the micro task queue. But in Rx, they make you think about things in event-driven way, which might make sense for servers and stream processing but it's ugly when you want to actually build UIs with it. I think if we pay due respect to the fact these workloads are pretty different, I think the reference is way better of an abstraction for things that are UI-centric. They're simple and their performant and I think it's often much better foot than Ember object which is kind of bloated and huge and very hard to optimize. CHARLES: Right. I like that because you have to be precise with the server side things but ironically, with the references, you only care about the state at the point at which you observe it -- when the user observes it, not when the code observes it. The user observes stuff with every animation frame and there can be any number of intermediate states that you can just throw away and you don't care about. You don't need to compute them. I think what you're saying is Rx forces you to compute them. ALEX: Right and you wouldn't use a Glimmer reference for something if you're trying to batch. But in the end, keep all of the events that were fired on all the change events. You wouldn't use references because you're losing all that information until you do that poll and you get the latest value. But 99% of the time, when you're building UIs, that's what you want to use. CHARLES: Are Glimmer reference is their own standalone library or do they currently bundled with Glimmer? ALEX: I'm not sure. If they are not now, I believe the intention is for them to be at their separate repo. I was talking to Kris Selden at EmberConf and I got the impression that the intent, it might not be there now and if I want to start extracting ember-concurrency stuff into vanilla JS, I'd probably want to use a reference-ish thing, if not the official one from Glimmer. CHARLES: I know we talked about this so then, how will you able to use these lazy references to compute tasks state? How that might work or play out? ALEX: The fundamental problem right now is that everything in ember-concurrency is so glued to the Ember object model. What that means is that all ember-concurrency has to do is broadcast so the changes has happened to the state of a certain task so that you can, maybe put a loading spinner up on your template. All it has to do is use object.set and then the built in computed property observer change detection that is in Ember object model. It's going to sort of propagate these changes but that's a bunch of heavy Ember stuff that is going to exist and a lighter weight Glimmer or vanilla JS context. Instead of using .sets and expecting that the thing you're setting it on is a big, heavy Ember object model, you could just use references. Then whoever wants to get a reference to whether a task is running or not, it is running reference. Then just using the standard Glimmer abstractions. At the Glimmer-concurrency task runner, it would just basically kick those references and anyone who has one of those references can flush and get the latest value at some later point in time and then update the UI based on that. Already, as a maintainer at ember-concurrency, I see all the pieces work with that and I could probably just start working on that today. But there's just a handful of other things that I want to align with the vision of Glimmer and Glimmer-concurrency before I start working on that. ELRICK: What would be the referency equivalent in just plain JavaScript outside of Glimmer that you would use to build this on top of --? CHARLES: Like what would the API look like? If you're like, "I don't have a Glimmer. I don't have anything. I'm just --?" ELRICK: Yeah, you just have plain JavaScript. What would be the primitive that you will build this on top of? ALEX: Whether we use a standalone Glimmer references library or this separate reference thing, then I would use the term based on something Kris Selden said. In the end, the APIs is going to be pretty similar between those two but if one thing is requires, as far as I understand it, you've got to set up where in an event loop, your response is something that's changed and then you schedule at a later request animation frame, to actually do the rendering based on that. In order to use something like references, it implies that you've got to flush at a later tick or flush at a later call back. If you've got that in whatever app you're working on, it should be pretty easy to figure out where references fit in. CHARLES: I see, so you would basically say like new task, give it your task class or whatever -- I'm just making stuff up -- then you would just schedule, do a request animation frame and then just pull the task state or something like that? Or a new task reference or something like that? ALEX: You might have some function that's schedule render pass, if not yet scheduled. Then if it hasn't been scheduled, then use requests animation frame. If you call that function again, it's going to no op until that requests animation frame happened. CHARLES: Could you explain that again? ALEX: Sure. If you're actually thinking of a really low level vanilla JavaScript to your app, in the browser or something and you were just using references, then you probably have something where the thing that kicks off the reference or dirties the reference in some way, also run some function called 'schedule rerender', if one hasn't already been scheduled or something less verbose. That would just make sure that some request animation frame has been scheduled. When it flushes, then it will get all the changes but if more references are dirty at that mean time, it won't schedule additional request animation frames. I don't know, that's kind of blue-skying but that's when -- CHARLES: Right. Here's the other things, you see like being able to integrated with a third-party state management solution like Redux or something. Basically, I've got my ember-concurrency tasks and their state is then reflected inside a Redux store. How might that work, if at all? Or was that a crazy idea? ALEX: I don't know. I played around with Redux toy examples and Redux community and Ember is only stronger by the day. I'm not entirely sure how all those pieces fit together because in Redux, they really want you to propagate all of your state changes using the reducer in that single global atom. A lot of people asked me about redux-sagas, which is another generator function-driven way of firing these state mutating actions over some async operation and this is hugely popular but I don't think they have any concept of the derived state that I've been trying to do with ember-concurrency of just being able to ask a task if it's running. You can't just do that. You've got to reflect that into the global atom -- the global store --somehow and to be honest, I don't really know if that's fundamentally at odds with the Redux model, to take something like Ember or Glimmer-concurrency and make it work that way. But ideally, you wouldn't have to forward all that state into the global atom. You just be able to reference that task object. CHARLES: If the task object itself is immutable, it would have seemed like fairly trivial, like you could generate programmatically the reducer required to do that. If you had the state encapsulated, you could come and say, "Now, there's a new state here." It seems like you would be able to adapt that but you would need to be able to react to any time if that state changed to fire and action in the Redux store or fire the Redux action. ALEX: Actually, this will be an easier question to answer because in the Ember community Slack, there's a Redux channel and I know everyone there is already starting to talk about how are references, how is Glimmer is going to, how can we kind of tie these things to Redux and I think when they have some solutions lined up, a lot of the stuff that will be in so-called glimmer-concurrency will just fit in nicely. If they've got nice models for tying references to the state atom, if you will, then it's going to work with the new way. CHARLES: Okay, cool. One of the things that I wanted to talk about was a proposal that Lauren Tan, who put on to the ember-concurrency issues list, although it's an RFC. Are you accepting RFCs now for ember-concurrency? ALEX: I'm not pompous enough to have a separate RFCs repo. Issue approaches perfectly humble for me. CHARLES: But is this the first RFC or have there been a bunch of ember-concurrency RFCs? ALEX: There's been a few. It's definitely great that Ember have standardize on this boilerplate RFC model that everyone can fit their proposal into because it means that all the add-ons that people really like and really want to invest in, they get these high-quality RFCs versus like, "Hey man. It kind of nice if you can just like, have a pipeline." [Laughter] ALEX: Just because Ember invested in that process, the whole add-on community benefits from it and it's great. There's been a few RFCs that are like that. I'm not sure how many of them have made it but I've seen a few that are in that format but this one's definitely one of the nicer ones and a lot of effort was put into it and it looks really nice. ELRICK: I'm not familiar with the RFC. What was the brief overview of what was proposed? ALEX: Lauren was basically proposing that we add concept of pipelines, which is if you have a series of tasks that are stepping the pipeline of operations, then we should standardize that and then define all the steps in the pipeline so rather than having each step in the pipeline, call the next step in the pipeline. They just return some their portion of that work and then the pipeline infrastructure will automatically run the next step in that pipeline. CHARLES: It seems like also then you can derive state about the entire pipeline, rather than just the individual task. You have to manage that a little bit by hand. But the other thing, I guess I would add is it seems like if you're going to go with pipelines, rather than being a simple list, you might want to think about it as being a tree because can you have pipelines that are composed of sub-pipelines, so to speak? ALEX: Yeah. I believe the answer is yes. I'm not sure if it's spelled out in this RFC but really a pipeline just fits the task interface so if there's a task-ish thing or taskable object that you declare as a step of a pipeline or sub-pipeline, it should just work. I'm not sure if there's more work that needs to be done in spelling that out but that seems baked into it. There's a lot of due consideration for making sure these things compose really well and it's already in a really good state. CHARLES: Yeah. What are some of the use cases where you might want to use a pipeline? I'm sure, everyone who's been writing concurrent tasks has probably been maintaining their own pipeline so what is it that you're doing and how is this going to save your time and money? ALEX: Let's use the example that I've actually used in EmberConf, which is something based on my own app, which is in my app, you have to geolocate and find nearby stores that you could walk into and that process is four async steps in a row. One is getting your geolocation coordinates and then the next step is passing those the store, getting values and the third step is maybe some additional validation or just setting a timer so that your animations or any of these little async things that you have to do. But it's really a sequential operation where each time, fetch your geolocation or get it out of a cache and then step to the next thing, then the next thing. It looks okay as I have it in my production app code but it still feels a little gross that you can just look at this thing and be like, "What is the sequence of steps here?" You have to actually get the implementation of each task to see what happens next and where will it go after that. Basically, with ember-concurrency in general, there's a lot of opportunities for finding more conventions for building apps. I don't even know if we really talk about this so far but derived state is part of it. But generally speaking before ember-concurrency, there wasn't as much opportunity, I guess for some of these conventions for building these pretty standard UI flows without feeling that you're just building your own thing every single time, with chains of Promises and your own improvise cancellation scheme and all these things. I see pipelines as a next step. Well, we're pre-building lots of pipelines in our apps. We have these processes that go through these multiple steps and right now, the best we can do is set a bunch of Boolean variables and the derived state that comes with ember-concurrency helps but with pipelines, there's even more and it also structures your thinking so that if something like pipelines catches on, hopefully as an Ember developer, you'll see them in a few different places and already have that tool in how to visualize a problem, visualize a component, visualize the async flow. CHARLES: If I spent my entire morning reading the talk that Lauren referenced in RFC, which was the Railway Oriented Programming, which I think, maybe not quite but basically a visual explanation of a Maybe monad or the Either monad or whatever it's called. One of the greatest explanations of why monads are helpful and through explanation using like the Maybe, where you can have a computation that could have more than one result, either success or failure and how do I take these functions and compose them with functions that might always succeed or might not have a return value or whatever and show the tracks that move through a computation and be able to normalize every function to have the same number of tracks. I realized, I'm getting into the description of it without actually having the visuals in front of it so I'm just going to stop myself and say everybody go read it. It'll take you 35 minutes but it will eliminate so much like the chatter that you've been hearing in the background for a couple years. ALEX: I used to tell people that they should learn Rx because regardless of me liking the task primitive a little bit better, it's great. It just scrambles your brain and reorganize your thought processes and it's such an interesting library to learn. CHARLES: All right. I like it. I'm going to go learn Rx. ALEX: I've been getting, on the server side, the sort of Kafka-based architecture, Apache Kafka. Particularly, they've released some libraries in maybe the last year or so. It's a very Rx-familiar feeling library for composing new data and new aggregates and joins between different topics and streams of events. It just seems like they're at the forefront of solving these really hard problems in a very conventional way. You get into some of that stuff and you'll find that you're doing a lot of server side processing with step that just feels a lot like Rx and I find it very interesting. I haven't actually build anything with it yet but it is likely in my future and anybody that's into the event-driven model should definitely know what people are working on in this Kafka-streams world. CHARLES: That is cool. It's so interesting to see how all the problems that you encounter on working on the server side, you will encounter on the client and vice versa. You can build up a huge corpus of knowledge on one side of the API divide and you'd be surprised that if you were to go work on the other side for a time, you'll be able to leverage 99% of that knowledge. That's fantastic. I would love to get into Kafka but unfortunately, I think we're going to have to save that for another time. That's another one of those words like... I don't know. Is Kafka descended from Storm or something like that? Is it a similar concept? I remember everybody was big on Storm. ALEX: I think Storm process the events and decides what to do with them. Kafka is really just a giant storage that plugs into something, I think like Storm or [inaudible] or any of these things that actually process the events. CHARLES: Yeah, it's all Kubernetes to me. ALEX: Yeah. CHARLES: All righty. Well, with that, I think we'll wrap it up. Thank you so much, Alex for coming to talk to us. It's always enlightening. I love your approach to programming. I love how deeply you think about problems and how humble you are in approaching them because they are big. ALEX: Well, thank you. It's great to be on here. It's fun. CHARLES: All right, everybody. Take care. Bye Elrick, bye Alex.
Michael Coté: @cote | cote.io | Pivotal | Software Defined Talk Show Notes: 00:54 - Pivotal 04:39 - Being a Professional Muller aka Analyst 11:08 - Iterative Development 32:54 - Getting a Job as a Professional Muller aka Analyst Resources: Pivotal Cloud Foundry GemFire Greenplum Pivotal Labs Wardley Maps Software Defined Talk Episode #79: From a vegan, clothing optional co-op to working with banks and oil companies - Coté's professional life, part 1 Software Defined Talk Episode #85: Being an analyst without being an asshole - Coté's professional life, part 2 RedMonk Transcript: CHARLES: Hello everybody and welcome to The Frontside Podcast, Episode #66. I am a developer, Charles Lowell at The Frontside and also host-in-training for 65 episodes. This is my 66th and I'm flying alone this week but we do have on the show with us a very special guest. Actually, the person who taught me how to podcast, I think it was about 10 years ago and he was like, "Charles, we should do this podcasting thing." I started my very first podcast with him and I still haven't figured it out. But his name is Michael Coté and he's a fantastic guy and welcome to the show, Coté. MICHAEL: Thanks for having me, Charles. It's great to be here. CHARLES: Now, what are you up to these days? You're over at Pivotal. MICHAEL: That's right. I work at Pivotal and probably people who are in the developing world know them for Spring. We have most of the Spring people. Then we also have this thing Pivotal Cloud Foundry. We're not supposed to call it a platform as a service but for matters of concision, it's a platform as a service that's the runtime that you run your stuff in. Then we also have a bunch of data products like GemFire and Greenplum and things like that. Then, 'openymously', if that's a word, we have Pivotal Labs. Now -- CHARLES: I think, it's eponymously. MICHAEL: Eponymously, yes. Now, you might remember Pivotal Labs as the people who use Chef Scripts to configure their desktops. Remember that? CHARLES: Yeah, I remember that. I was into that. MICHAEL: Yeah, in coincidental kind of way, the inspiration for the project Sputnik thing, which is coincidentally because now Dell Technologies owns Pivotal so all of that stuff has come for a full circle. I guess also since I'm intro-ing myself, I work on what we call the Advocate Team because we don't call them evangelists. No one likes to be called that I guess. I guess there's 12 of us now. We just hired this person, also in Austin actually McNorma who's big in the Go community and apparently can make images of gophers really well. I'm sure she does many other extraordinary things, not just the illustrator master. Everyone else basically like codes or uses the terminal but I do slides. CHARLES: Well, that's your weapon of choice, right? It's a more elegant weapon for civilized time or something like that. I'm going to look it up on Wikia. MICHAEL: Yeah, basically what we do on our team is we just talk about all the stuff Pivotal does and problems that we solve in the way people in an organizations like would think to care about our stuff. Most of what I do is I guess you call it the management consultant type of stuff. Since I have a background as an analyst and I used to work on corporate strategy and M&A at Dell so I have a vantage point in addition to having programmed a long time ago. If you're changing your organization over to be more agile or trying to devops, we would say cloud-native with a hyphen. How do you change your organization over what works and doesn't work? Most people in large organizations, they sort of pat you on your head. I'm sure you encounter this. That sounds really nice that we would be doing all of the good, correct ways of using computers but we're basically terrible and we could never make that happen here. Thanks for talking with us, we're going to go back and stew in our own juices of awfulness. You've got to pluck them out of that self-imposed cannibal pot there in the jungle and show them that they actually can improve and do things well. CHARLES: Would you say you feel like your job is being that person who shakes them away and can be like, "Good God! Get a grip on yourself!" MICHAEL: Sure. That's a very popular second or third slide in a presentation -- the FUD slide, the Fear of Uncertainty and Doubt slide where you're basically like, "Uber!" and then everyone just like soils their pants because they're afraid that are like Airbnb and Uber and [inaudible] and Google is going to come in and, as they say, disrupt their state industry. I try not to use the slides anymore because they're obnoxious. Also, most people in large organizations nowadays, they know all of that and they've already moved to putting on a new pair of pants stage of their strategizing. CHARLES: You've got the kind of the corporate wakeup call aspect of it but then it's also seems like a huge component of your job which is when you were at RedMonk, when you were at 451 and even to a lesser extent, it was Dell who was paid well to just kind of mull it over, like just kind of sit there and asynchronously process the tech industry, kind of like organizational yeast and let it ferment, kind of trying to see where the connections lie and then once you've made that presented, do you think that's fair? That's what sprung to mind when I heard you say like, "Yeah, we just kind of sit around and think about what is Pivotal and what does it do and what's it going," but like how do you get that job of like, "I'm just kind of a professional muller." MICHAEL: That's right. First of all, I think professional muller is accurate, as long as, I guess mulling is also for -- what's that thing you drink at Christmas that you put the little -- CHARLES: Mulled wine. Like low wine. MICHAEL: I can feel like that sometimes late at night. But having a job as an analyst, I was an industry analyst at two places for a total of about eight years or so. Then as you're saying doing strategy at a company, now what I do here, essentially a lot of what you do is very difficult. I know it sounds to people. You just read a lot of the Internet. You just consume a lot of the commentary and the ideas of things that are going out there and you try to understand it and then synthesize to use that cheesy word. Synthesize it into a new form that explains what it is and then finally, the consultant part comes in where you go and meet with people or you proactively think about what people might be asking and they say something like, "What does this mean for me? And how would I apply it to solve my problems?" I guess as an example of that -- I apologize for being a little commercial but these are just the ideas I have in my head -- Ford is a customer of ours and they also have invested in us which is kind of novel. We have GE and Ford invested in Pivotal and Microsoft and Dell Technologies as an interesting mix but anyways, they have this application called the Ford Pass Application. I drive a Ford Focus -- CHARLES: Like Subaru? But you do drive a Ford. MICHAEL: Yeah, because I don't care about cars. It's a bunch of nonsense. I see this app and basically the app, if you have a more advanced one, it might tell you your mileage and even like remotely start your car. But it doesn't really do that much. You have the app and it will tell you information about your car and where to park and it even has this thing where it links to another site to book a dealership thing, which is annoying. CHARLES: Why would you want to book a dealership? To buy another car? MICHAEL: Well because the Ford Focus I have is notorious for having transmission problems so you're like, "I got to go and take it into the dealer to get all this recall stuff taken care of," so wouldn't it be nice... I don't know if you've ever worked with a car dealer but it's not desirable. CHARLES: Yeah, it would be nice if they didn't charge $6000 for everything. MICHAEL: Right. It's a classic system of having a closed market, therefore that jacks up prices and lowers customer service usually. What's the fancy word if there is a negative correlation, if you were to chart it out? Like price is negatively correlated to your satisfaction with it. Kind of like the airline industry, not to bring up a contemporary topic. You pay a lot of money to fly and you're like, "This is one of the worst experiences I've had in my life," whereas you go to the dentist and get a root canal and you're like $20 co-pay. Loving it. [Laughter] MICHAEL: Anyhow, this Ford Pass application doesn't really do very much so what does that mean for what I was explaining. If you go look up and read about it, starting back in the late-90s, your extreme programming and then your Agile Software Development and your devops nowadays, one of the major principles is what you should do is ship often. Maybe you should even ship every week or every day. Don't worry about this gigantic stack of requirements that you have and whatever you should be shipping all the time and then we've trained ourselves to no longer say failing fast. That was a fun cheeky thing back in the late-2000s. CHARLES: Did we trained ourselves not to say that anymore? MICHAEL: I don't hear it very often. CHARLES: Man, I got to go scrub my brain. MICHAEL: Yeah, well this is why you consult with me every 10 years as I tell you the new things. CHARLES: Okay, here we go. We're going to have you on the podcast again. MICHAEL: That's right. You have this idea of like, "We should be releasing weekly," but then if you go to Ford, you're like, "What does that mean?" To shave the shaggy dog here, essentially the idea that they're shipping this mobile application that doesn't really do very much is an embodiment of the idea that they should be shipping more frequently. This may be a stupid example. It's not that it's not going to do very much like permanently but as I have witnessed, very frequently they add new features so Ford is in this cadence but there's this app that instead of working on an application for two years and having everything in it, they're actually releasing it on, I don't know if it's weekly but they're releasing it on a very frequent basis, which allows them to add features. What that gets you is all the advantages of a fast iteration cycle small batch thing where they can study this actually a good feature. They can do all your Lean Startup nonsense. That's a very like weird, perhaps example of how you explain to someone like a large car manufacturer like Ford, this is what devops means for you. Therefore, why you should spend a lot of money on Pivotal? Now that's the part that lets me pay my mortgage every month, the last bit there. CHARLES: Right so Pivotal builds apps. MICHAEL: Well, the Labs people build apps for you. CHARLES: I'm kidding Coté. MICHAEL: Yeah, they actually do. The Labs people are like a boutique of another boutique like ThoughtWorks is kind of a boutique but they're kind of a boutique-y version of ThoughtWorks. That probably is terrible as someone who markets for Pivotal to do that. Do you ever notice how political candidates never really name their opposition? Like you never really want to name your competition but anyways... CHARLES: Pivotal marketing are going to come crashing through your window. Everybody, if we hear them in the next five seconds -- well, I guess you can't call 911 because this is not live. MICHAEL: Yeah, that's true. The Labs people build stuff for you and then the part that I work, in the Pivotal Cloud Foundry people, they have the actual runtime environment, the cloud platform that you would run all that stuff. Plus all the Spring nonsense for your microservices and your Spring Boot. I understand people like that. CHARLES: So good for Ford, for actually being able to experience, either in the development and the joys and the benefits that come with it. But this is actually something that I actually want to talk about independently was as I kind of advance in my career, I find myself pushing back a little bit against that incredibly tight, iterative schedule. Shipping things is fantastic and it's great but I find so much of my job these days is just trying to think out and chart a course for where those iterations will carry you and there is a huge amount of upfront design and upfront thought that it is speculatory but it's very necessary. You need to speculate about what needs to happen. Then you kind of measure against what's actually happening but I feel that kind of upfront design, upfront thought, we had this moment we're like, "We don't need that anymore. Let's throw it all in the garbage." In favor of doing things in these incredibly tight loops and finding where's the clutch point, that kind of long range thinking and long range planning comes and meets with the iterative development. I have no idea. What's the best way for those to match up those long cycles and those short cycles? Where is the clutch play? MICHAEL: I'll give you two and a half, so to speak trains of thoughts on that. One of them is I think -- CHARLES: Two and half trains of thought, I like that. Can we get straight to the half train of thought? MICHAEL: Yeah, I'm going to start with the half, which is just taking all of your questions and putting periods at the end of them before I round up to answering the question. I think a lot of the lore and the learnings you get from the Agile world is basically from consultants and teams of consultants. Necessarily, they are not domain experts in what they're doing so their notion is that we're going to learn about what it is we're doing and we don't actually know we can't predict ahead of time because we're not domain experts so they almost have this attitude of like, "We'll just figure it out on the job." Let's say The Frontside gets hired to go work on a system that allows the Forest Service to figure out which trees to go chop down or not -- CHARLES: If you're the Forest Service, we are available to do that. MICHAEL: I'm guessing you don't have a lot of arborists who have 10 or 20 years of experience working there. CHARLES: No, we don't. MICHAEL: And so you have no idea about that domain so in doing an iterative thing, you won't be able to sit down and predict like everyone knows that when you send the lumberjacks out, they're going to need these five things so we're going to have to put that that feature on there. They need to be able to call in flapjacks when they run out. That's just what's going to happen so you don't know all of these things they need to do so you just can't sit down and cogitate about it ahead of time. Also this comes in from the Lean Startup where there's a small percentage of software that's actually done globally and the notion of a Lean Startup is that when you're doing a startup, you're never going to be determined what your exit is, how you cash out, whether that's building a successful long term company while you get sold to someone or whether you IPO, you're not going to able to predict what that business model is so you just need to start churning and not think a lot ahead of time. Now, the problem becomes, I think that if you are a domain expert, as you can do the inverse of all the jokes I was just making there, you actually can sit down and start to predict things. You're like, "We know we're going to need a flapjack service," so we can predict that out and start to design around that and you can do some upfront thinking. Now similarly, developers often overlook the huge amount of governance and planning that they do for their own tools, which I know you're more cognizant of being older or more experienced, as they like to say. But basically, there's a bunch of, as we used to call it when I did real work and develop stuff, iteration zero work like we're going to need to build a build system, we're going to need a version control. You actually do know all these things you're going to need so there are all the things you can plan out and that's analogous to whatever domain you're working in. Sometimes, at least for your toolchain, it is worth sitting down and planning out what you want. Now, to hold back the people who are going to crash in my window, one of the things you should consider is using Pivotal Cloud Foundry. That's probably something you should cogitate on ahead of time. CHARLES: I think they're going to crash through your window and give you a Martini, if the marketing ninjas are going to do that and if you mention them in a positive light. MICHAEL: You know, it's 10:52 Central but if we were in London, it would probably be an appropriate time so we'll just think about that. Now, on the other hand, you don't want to go too overboard on this pre-planning. I'll give you an example from a large health insurance company that I was talking with recently. They had this mobile app -- it's always a mobile app -- that had been languishing for 15 months and it really wasn't doing anything very interesting. It was just not working well and they could never release it. This is a classic example of like, "We took a long time to release a mobile app and then we never released it again and then it blows." It's not achieving all of the business goals that we wanted. Mostly, what a health insurance company -- I've talked with a lot of the health insurance companies -- want with their mobile app is at least two things and probably many more but these would be the top of the list. One, they want their customers, their users to look up what their health insurance is, figure out doctors they can go to, the basic functioning that you expect from your health insurance company. And two, they want to encourage their customers to do healthy behaviors because if you think about it as a health insurance company, health insurance in my mind is basically like this weird gamble of like, "I'm gambling on the fact that you are going to be healthy," because then I pay out less to you and you just give me money so the healthier that your users can be, the more profit you're going to make. That's why they're always trying to encourage you to be healthy and stuff like that. The mobile app was not achieving, at least these two, if not other business goals they have. They basically were rebooting the effort. The way they started off is they had -- I don't know how many inches thick it was -- a big, old stack of requirements and the first few iterations, the product team was working on it and talking with the business analyst about this and going over it and what they sort of, as we were calling Pivotal Labs the product owner but the person who runs the team, realize is like -- to cut a long story short -- "This is kind of a waste of time. We shouldn't just prioritize these 300 features and put them in some back road and execute on them because these are the same features that we based the more abundant application on, we should probably just start releasing up the application," kind of like the FordPass app. That said, they did have a bunch of domain experience so they had a notion of basically what this app was going to do and they could start planning it out but they figured out a good balance of not paying attention to, as Martin Fowler used to call it the almighty thud, of all the requirements. What they ended up doing is they basically -- CHARLES: What's the almighty thud? MICHAEL: You know, he's got some bleaky or whatever. It's basically like we started a project and I think it's from 2004 and someone FedExed me about 600 pages of an MRD or whatever and I put it down on my table and it made a loud noise so he calls that the 'almighty thud', when you get this gigantic upfront requirement thing. What happened in this health insurance thing is they stopped listening and talking with those people and they kind of like chaff them out, not like when your rub your legs together but they kind of distracted them to that fact but eventually, they just got them out of the cycle and they started working on the app. Then lo and behold, they shipped it and things are working out better now. CHARLES: Hearing what you're saying and kind of thinking it over, I think if you're going to have an almighty thud, what you really want is you want all that upfront research and all that upfront requirements gathering or whatever, not necessarily to take the form of a set of features or some backlog of 300 things that the app 'needs' to do or 'should' do but just a catalogue of the problems, like a roadmap of the problems. MICHAEL: Exactly. CHARLES: You know, that actually is very valuable. If it's like, "These are things that are true about our users and these are the obstacles that they face. If we do choose that we want to go from Point A to Point B, where we are at Point A, then we actually have a map of what are the things that are sitting in front of that and what are the risks involved." It's like if you got -- you played, you're from my generation, you play the Oregon Trail, right? MICHAEL: Yeah. "You have dysentery." CHARLES: Right. I don't know where I'm going with this analogy but my point is developing that app is like going from Kansas City to Portland. But the thing about software is you don't necessarily have your corn meal. You don't need to say like, "We're going to need six pounds of cornmeal and we're going to need these wagons and we're going to need these mules," because this is software and you can just code a mule if you need it. But you might not need a mule, if the rivers are not in flood... I don't know. Like I said, I don't know where I'm going with this analogy. But do you see what I'm saying? The point I'm trying to make is that having the map of the Rockies and where the passes are is going to help you. MICHAEL: Yeah, this is probably where I'm supposed to expertly rattle off what Wardley maps are and how they help, which is fine. I think that's a great tool. There's this guy Simon Wardley and he's actually a great contemporary philosophizer on IT-led strategy. I think he works for CSC who no longer owns mercenaries but they used to -- Computer Science Corporations. I think they own a little bit of HP Services Division but he works for some think tank associated with CSC and he has got a couple of OSCON talks on it, where it's called a Wardley map and it's a way that you start figuring out what you're saying, which is to say your company's strategy. Using your front metaphor of the era of tall hats, if you remember that other movie, if you're on the Oregon Trail, broadly your strategy is -- and people get all up in your face about the difference between a plan and a strategy and we'll just put mute on them and edit them out of the audio because they're very annoying -- CHARLES: We'll call it an approach. MICHAEL: That's right. Your plan or your strategy -- and pardon me if I use these phrase free and loosely and everything -- is you would like to get to Oregon and you would like to live there and maybe grow apples or start a mustache wax company or some donuts, whatever it is you do out there once you get to Oregon and their strategy is -- what are the assets that I have. I have a family, I have some money and I also know some people who are going there so I'm going to buy a stagecoach and a mule, then I'm going to kind of wangle it out and we're going to go over there. Also, part of our strategy is we're going to go through the northern pass because we're used to winter versus the southern pass, which isn't the Oregon Trail because reasons. Maybe Texas isn't part of The Union yet so I don't want to deal with the transition between whatever that weird Texas thing down there -- CHARLES: The desert, there's the southwest and the desert. MICHAEL: I don't have the capabilities to survive in a desert so I need to go to the north and hopefully I won't be like that movie and have a grizzly bear rip up my backside and everything. You sort of put together this plan. Now going back to what you would do in IT world is to your point, someone does need to define what we would call the business value or the strategy, like what you want to do. Looking at the Ford thing, what Ford wants to do is they do cogitating thing ahead of time and they're like, "We manufacture cars," and you've got electric cars and Uber. That's where the scarce light comes in. In the future, who knows that people will still buy cars? It might be like that I-Robot movie where all the cars are automated and you just go into one. As a company, whose responsibility is to be as immortal as possible, we need to start making plans about how we can survive if individuals no longer buy cars. Let's do that. This is a huge upfront notion that you would have and then that does trickle down into things like my Ford thing -- I'm kind of speaking on their behalf -- if we have a direct connection with people, maybe eventually we introduce an Uber-like service. You can just check-out a Ford car. Then maybe this and maybe that. It's the strategy of how do we set ourselves up to do that. Now, I think the Agile people, what they would go for is it's really good to have that upfront strategy and you'll notice that in a lot of lean manufacturing in Agile talk, no one ever talks about this stuff, much to my extreme annoyance. They don't ever talk about who defines the strategy and who defines that you're working on this project. That's sort of left as an exercise to the reader. The Agile people would say like, "The implementation details of that are best left to the development team in an Agile model." Just like the developers are always arrogantly are like, "Hey, product manager. How about you f-off about how I should implement this? I am the expert here and let me decide how I'm going to implement the feature that you want for me." It's kind of like that rushing dolling down of things. To the development team, you worked on some, what was it? Band frame wire thing, a long time ago? It was basically like, "We don't know it. Maybe this is not the case. Let's pretend like it was." We don't know exactly how you're going to implement this stuff but our goal is that there's bands and they need sides and ways of interacting with their users so let's just figure out what that looks like but they had that upfront idea of ways that they were doing things. CHARLES: Let's start walking. MICHAEL: To add on some more. There's another edge case that you're making me think of, which is a good way of thinking through almighty thuds versus how much planning you have and that's government work. Government work that's done by contractors and especially, military contracting work. What you notice in government work is they have, seemingly way too much paperwork and process. They literally will have project managers for project managers and the project managers have to update how the project is going and they reports. If they don't do the reports correctly, their contract is penalize and you might even get fired for doing it. If anyone stops and says while the software is working, they were like, "No, no, no. don't be naive. It doesn't matter if the software is working or not, if we don't fill up the project report, we're fired." Until someone like yourself or me, it's just like your head explodes and you're like, "But working software, not a concern." In that case, it actually is part of the feature set, part of the deliverable is this nauseating amount of project reporting and upfront requirements, which has this trickle-down effect of annoyance but that's what you're getting paid for so that's what you do and if you want to make yourself feel better about it. I don't know how it is in the rest of the world but in the US, basically we think the only person worse than maybe, Lucifer is the government. I don't know why this comes about. We enjoy the fruits of the government all the time but for some reason, we just think they're awful. Whenever we give money over the government, we want to make sure that they're spending it well and if they're not corrupt and they don't hire their entire family to help them run the government and make sure that they're making extra money globally in their businesses, I wouldn't know anything about that. But essentially, you want to make sure there's no corruption so transparency is almost more important than working software. The way you achieve that transparency is with all this crazy documentation. CHARLES: Here's the thing. I agree the transparency is fantastic but nothing is more transparent than working software. Nothing is more transparent than monitored software. Nothing is more transparent than software whose, by its very nature is radiating information about itself. You can fudge a report but you can't fudge a million happy users. MICHAEL: Don't get me wrong. I'm not saying that the way that things currently operate is the ideal state. I'm saying that that desire for transparency has to be addressed and for example, using your example, let's say you were delivering working software but you were also skimming 20% off the top into some Swiss bank account -- you're basically embezzling -- and then it turns out that you need 500 developers but you only actually had 30 developers. There was corruption. The means even though the ends, even though the outcome was awesome, the means was corrupt so that's the thing in a lot of government work that you want to protect against. I just bring that up as an edge case so a principle to draw from that, when it comes to almighty thudding is like sometimes, that is part of the deliverable. We would aspire in our fail, fast, Agile world to not have a bunch of gratuitous documentation as part of the deliverable because it seems like a waste. It would be like every morning when you battle with your kids to get their shoes on, you had to write a two-page report about how you're getting ready to go to school stuff with your kids was going. As a parent you would be like, "I don't need that." However, maybe if you were like an abusive parent and it was required for you to fill out a daily status report for you to retain the parentship of your kids, maybe it would be worth of your time to fill out your daily status report. That was an awfully depressing example there. CHARLES: Let's go back to the Oregon Trail. What I'm hearing is that -- and we will take it back to the Oregon Trail -- you also need to consider, as were saying, you have some sort of strategy which is we want to go sell apples and moustache wax. But what we're going to do is we're just going to start walking, even though we don't have a map. But obviously, if you send out scouting missions, like you know where you're going, you know the West Coast is out there somewhere, you start walking but the stakes determine how much of your resources you spend on scouting and map drawing -- MICHAEL: Yeah. My way of thinking about strategy and again, people strategy is this overloaded word. But my way of thinking about strategy is you establish a goal: I would like to go to the West Coast. Now, how you figure that out could be a strategy on its own, like how did you figure out you want to go to the West Coast. But somehow, you've got to get to a prime mover. Maybe those tall hat people keep beating me up so I want to go to the West Coast. I want to go the West Coast is the prime mover. There's nothing before that. Then you've got to deal in a series of constraints. What capabilities do I have, which is another way of saying, what do I not have? And what's my current situation and context? On the Oregon Trail thing, you might be like, "I have a family of seven. I can't just get a horse and go buy a pack of cigarettes and never show up again." I guess I could do that. That's probably popular but I, as an individual have to take this family of six other people. Do I have the capabilities to do that? How could I get the cash for it? Because I need to defend against all the madness out there, I'm going to need to find some people to meet with. You're thinking and scenario planning out all of this stuff and this gets to your point of like, "If you're going to Oregon, it probably is a good idea to plan things out." You don't want to just like the next day, just figure it out. [inaudible] tell a joke. It's like, "Why do they sell luggage at the airport? Is anyone is just like, 'Screw it. Pack a clothes and we'll sort it out at the airport.'" It's an odd thing to sell at the airport. But you do some planning and you figure out ahead of time. Now, to continue the sort of pedantry of this metaphor, the other characteristic of going to the Oregon Trail, unless you're the first 10 people to do it is hundreds, if not thousands of people have done it already so you kind of know what it's going to be like. It's the equivalent, in a piece of software, if they were like, "This application is written in COBOL. I want you to now write it in --" I don't know, what are the kids do nowadays? Something.io? I-want-you-to-write-this-in-a-hot-new-language.io and basically just duplicate it. You're going to still have to discover how to do things and solve problems but if the job is just one-to-one duplicate something, then you can do a lot more upfront planning for it. CHARLES: While you're doing it, making the Uber and Airbnb. MICHAEL: Yes. CHARLES: Then you're done. MICHAEL: I think that's the truth and I want to put it another way. We used to be down here in Texas, the way we run government here is just lovely but we used to have this notion of a zero-budget, which is basically like, "Assume I'm going to give you nothing and justify every penny that I'm going to give you." I think that's a good way to think about defaults. I mean, about requirements is default is you don't need any and only get as many requirements as you need. If you're building tanks or going to the Oregon Trail, you might need a lot of requirements upfront that are actually helpful. CHARLES: But like a suit, you're just going to just strike out naked walking with. MICHAEL: That's probably a bad idea unless you -- CHARLES: Yeah, that is a bad idea but that's the bar but what happened if I were to do that? I might make it for 20 miles. MICHAEL: And build up from there and then have all the requirements that you need. I'm sure when Lewis and Clark went they were like, "We're going to need a quill and some paper and maybe a canoe and probably some guns and then let's see what happens." But that was a whole different situation than going to establish Portland. CHARLES: That was an ultimate Agile move. That was a pretty Agile project. They needed boats, they built them but they didn't leave St Louis carrying boats. MICHAEL: Right and they also didn't have a family of six that they needed to support and all this kind of stuff, right? CHARLES: Uhm-mm. MICHAEL: There was a question you asked a long time ago, not to steal the emceeing for you -- CHARLES: I would say, we need to get onto our topic -- MICHAEL: Oh, yeah. Well, maybe this is a good saying, what you're asking is, "How do you get this job?" and I don't think we ever addressed that. CHARLES: Yeah, that's a great question. You said you had to consume a lot of stuff on the internet. MICHAEL: Right. That's definitely how I do the job but I think how I get the job, there's an extended two-part interview with me on my Software Defined Talk Podcast Episode, available at SoftwareDefinedTalk.com, where I talk about my history of becoming an analyst and things like that but the way it happened is I don't have any visible hobbies, as you know Charles except reading the stuff in the Techworld. I would read about what's happening in the Techworld and would blog about it back in 2004, 2005 and I was discovered as it were by the people at RedMonk. I remember for some reason, I wrote some lengthy opinion piece about a release of Lotus Notes. I don't know why but that was a good example. This is back when all of the programming job were going to be off shored and I thought it was imminent that I was going to lose my job. I was looking for a job and I shifted over to being an analyst. That like the way that you get into this kind of business is you establish, there's two ways -- CHARLES: You established expertise, right? MICHAEL: Yeah, which is like always an unhelpful answer because it's sort of like, I was joking about this in another podcast, it's like Seth Godin's advice about doing good marketing, which is the way you do good marketing is you have an excellent product. If you have an excellent product that everyone wants to buy, then your marketing will take care of itself. I think if I'm asking how to market, I'm trying to figure out how to market a bad product. That's really what people want. CHARLES: That's also just not true. That's just like flat ass not true. That's a lie. MICHAEL: I mean, people who want to know how to diet better are not already healthy and dieting successful. You can't start with the base assumption of things are going well. CHARLES: Well, it is true. I like to think that we have an excellent product. We sell an excellent product but the thing is you can just sit on your excellent product all day and you have to tell people about it. If you want them to come sample it and try, maybe eventually buy it like the advice that you just need an excellent product. I'm amazed at anyone who can actually can say that with a straight face. MICHAEL: Well, he only writes like 150-word blogpost. I think his point is that you should aspire to have a unique situation and then marketing is easier. Similar with everyone's favorite example like an Apple or like a Pivotal or a ThoughtWorks. We eat all three of us and yourself as well, once someone gives you the benefit of the doubt of listening, you can explain why what you have is not available anywhere else. CHARLES: What it boils down to is if you want to easily differentiate, allow people to differentiate your products from others, then be different. That's fair. I'll give -- MICHAEL: To summarize it, it begets more of the tactics of how one gets a job like I do. What's the name of the short guy in Game of Thrones? 'Tyrian'? 'Tyran'? 'Tyron'? CHARLES: Tyrion. MICHAEL: At one point, Tyrion is like, "I do two things. I know things and I drink," so that's how you get into this type of business as you establish yourself as an expert and you know things. Now, the third thing which I guess Tyrion was not always required to do is you have to be able to communicate in pretty much all forms. You need to be good at written communication, at verbal communication, at PowerPoint communication, whatever all the mediums are. Just knowing something is not very useful. You also have to tell people these things. CHARLES: I think Tyrion is pretty good at that. MICHAEL: Yeah, that's true but he doesn't ever write anything. There is no Twitter or things like that. CHARLES: I feel like [inaudible] been a pretty big deal in the blogosphere. MICHAEL: Sure, no doubt. The metaphor kind of breaks down because the lattice for the continuing counterarguments do not exist in the Game of Thrones universe but whatever. CHARLES: They've got the ravens. That's like Twitter and it's bird. MICHAEL: That is true. Knowing how to deploy a raven at the right time, with the right message is valuable. CHARLES: We buffer up our ravens so that they fly right at eleven o'clock. MICHAEL: That's true. I could be convinced otherwise. CHARLES: That's why they arrived both at 6PM in the Westeros -- MICHAEL: I guess true to the metaphor of a tweet, most of the communications in Game of Thrones is either, what are they called? Little Birds? That the [inaudible] always has and then the Big Birds. You've got to tweets and the blogs. CHARLES: This is like it's nothing but Twitter. MICHAEL: Exactly. You got to really communicate across mediums. Now that the other thing that's helpful and you don't necessarily have to do this but this is what I think gets you into the larger margin. The more profitable parts of the work that I do is you have to be able to consult with people and give them advice and consulting is largely about, first figuring out the right opportunity to tell them how they can improve, which usually is it's good if they ask you first. I don't know about you but I've found that if you just pro-offer advice, especially with your spouse, you're basically told that you're a jerk. CHARLES: Well, it'd be like a personal trainer and walking around me like, "Hey man. Your muscle tone is kind of flabby. You got to really work on that." MICHAEL: The line between a good consultant and being overly-explain-y is difficult to discern but it's something that you have to master. Now, the other way you consult with people is you study them and understand what their problems are and you're sympathetic to them and I guess you can be like a British nanny and just scold them. That's a certain subset of consulting. CHARLES: Don Rickles of consulting? MICHAEL: That's right. You just help them understand how all of this knowledge that you have applies to them and hope solve their problems like the FordPass thing. When I went from being a developer to an analyst, it was a big risk to take on. I think I probably took like a $30,000 pay cut and I went from a big company health insurance to being on a $10.99 and buying your own health insurance which a whole other conversation. We talked about that every now and then but like it's a risky affair. It's not a promotion or even a lateral move. It's just an entirely different career that you go into. Then you talk with people a lot. As an analyst, you're constantly having to sort out the biases that you have with vendors who want to pay you to save things versus end-users who want to hear the truth. You can't really see a lot of Gartner and Forrester work but the work that you can see publicly from people like RedMonk, it's pretty straightforward. CHARLES: Yeah it is and whatever they did, a piece that was for one of their clients, there was always a big fat disclaimer. MICHAEL: Now, the other thing I would say is what I've noticed -- not to be all navel-gazing -- about myself and other people who are successful at whatever it is I do is there's two things. One, they constantly are putting themselves out there. I remember and this is probably still the case. This is probably all in Medium. There's probably a Medium post every quarter that's like, "If you're a developer, how do you give more talks. What your first conference talk?" Basically, the chief advice in there, other than bring business cards and rehearse is essentially like you just got to get over that idea of self-promotion. You basically have to self-promote yourself incessantly and do all those things that you find nauseous and be like, "Me, me, me," which is true. You've got to get over that thing. If you're like me and you're an introvert who actually doesn't really like that many people, except a handful of people like yourself that I'm friends or family with, you have to put on the mask of an extrovert and go out there and do all this extrovert stuff or you'll fail. I shouldn't say you'll fail, you won't increase your overall comp and margin and everything. You'll basically bottom out at about $120,000 a year or so because that's about as much as anyone will pay for someone who just write stuff but doesn't actually engage in the world and consult. You've got to do that. Then the other consequence of that is you always have to be trying out new types of content and mediums like here we are in a podcast. Long ago, you and I, in 2005 or 2004 -- CHARLES: You got me to sign up for Twitter. MICHAEL: Yeah, like we started off a podcast because I remember hearing the IT conversation stuff and John [inaudible], who is a big inspiration for me, a role model, I remember he was just trying out podcast and I was like, "All right. I'll try that out. That looks like fun," and then here we are. CHARLES: I remember you tried out the podcast and you're like, "Let's go into your backyard or my backyard. Let's talk about software for 15 minutes." I remember that very clearly and that was 12 years ago. Then I remember also like with Twitter, you're like, "Now, you should sign up for this Twitter thing," and I remember I did and that's when it was still coming through SMS on your phone and like "I'm walking around Teatown Lake. I'm going to get tea." And I was like, "Oh, my God. This is so fucking stupid." But little did I know, you were actually signed me up to a service that changed my life. MICHAEL: Yeah, it was the stage direction era of Web 2.0 where you're just supposed to give people your status updates, instead of your searing insights. But yeah, you've tried it all these different mediums because again it goes back to your job is to communicate. You need to tell people things that you know. CHARLES: Coté, what is your strategy on virtual reality? MICHAEL: My strategy in virtual reality. Well, you've caught me, Charles because I'm not into that. You remember when Time Magazine had that Chinese lady who was like a... Not Frontside. What was the name of the big virtual reality thing that was big...? CHARLES: Second Life. MICHAEL: Second Life, who is a Second Life millionaire. CHARLES: Yeah, she had armies of people. She was mining some resource in Second Life and then reselling it and she made a lot of money. MICHAEL: I don't really like visual mediums so as Marshall McLuhan would say 'hot mediums'. I guess I like the cool mediums. That's not my thing. That's where my principle fails. Maybe I'll do that one day. CHARLES: This is pretty hot. This medium is pretty like -- MICHAEL: I think maybe audio broadcast is hot. I'm just pretending like I know. This is another trick that you can deploy that my wife has picked on is most of the time, 78% of the time, I actually have no idea what I'm talking about. I just know words. I don't actually know Marshall McLuhan theory. I read that one book a long time ago and I remember that scene in Annie Hall where he gives a little diatribe to whatever the Woody Allen character is. That's the extent of my Marshall McLuhan knowledge. CHARLES: Was Marshall McLuhan actually in Annie Hall? MICHAEL: He was. CHARLES: Don't sell yourself short, Coté. MICHAEL: Sure. CHARLES: You know things and you drink so let's talk about that second aspect because I know that you like me whole tearing up as a role model. MICHAEL: I should say since we're both happily married, except for the third thing that he does which he -- CHARLES: Oh, right. MICHAEL: Another unmentionable word. He too freely hangs out with the ladies. CHARLES: Right, anyway aside from that, throughout doing all this stuff, you keep a very, very chill perspective on things. I feel like the tech world gets so wound up around itself and it gets so tight and so stressed about its own problems. There's constantly wars in JavaScript and before we were in the JavaScript world, we were warring in Ruby. I remember when Twitter went over to using Scala instead of Ruby. Oh, my goodness, it was terrible times. I feel like there's a lot of stress and yes, you want to take it seriously but I feel like you've always been able to maintain an even-keeled perspective about technology which actually allows you to commentate on it effectively and intelligently because you're able to unwind yourself from the squabbles of the day and see maybe a bigger picture or something like that. MICHAEL: That's nice of you to characterize me to use a -- is that a hanging, dangling participle there, when you're in [inaudible]? CHARLES: Yeah, I don't know. MICHAEL: I think that's also just a function of being old. CHARLES: So are you actually not stressed or is it just part of your persona of being an extrovert or something like that? MICHAEL: About the tech world? No, I'm not stressed about that. As you kind of outlined, especially I was not sent the demographics for the show, which is fine. I'll overlook that but I'm guessing that that was a joke. CHARLES: Who got some designers, developers -- MICHAEL: I'm guessing there's a lot of people who actually are on the frontlines of working on software. I think this happens also in the white collar set. But essentially, it's really easy to slip into over allegiance to something and I don't know what rhetorical fallacy this is but it's the bias of over allegiance to something, you get all wrapped up in defending a tool over something and the virtue of it, whether it's Emacs and vi. I'm sure reactive people, whatever that is, have all sorts of debates. The thing is when you're heads down on this stuff, you don't realize how petty all those discussions are. It's not so much that it's a waste of your time but it's just one battle in an overall war that you have. It's good to have opinions and figure things out but you should just relax about it because the more angry and emotional you get, you're going to make a lot of mistakes and decision and problems. I wish I had an example of this but this is one of those things that intuitively as you ages as developer, it's not like your literal age. It's just the amount of time you've been developing software. You could be a 25-year old who's been developing software for 10 years and you would probably get this notion but you just realize that stuff changes and you just learn the new things. It's kind of not a big deal like one day, you're going on and on about how vi is great and the next day you're using that Atom editor and then whatever and you just use the tool that's appropriate and it's annoying when you're younger and people are applying Hacker News with like, "You should use the tool that is appropriate," which is a stupid reply. That's just kind of how it is. Also the other thing, in the more white collar world, as an analyst, especially doing strategy for a company, you can't be biased by things because then you'll make poor decisions as an analyst. Also when you're doing strategy in M&A that result in bad business outcomes so you actually be very unbiased about things. CHARLES: I think it applies in everything. If you get too emotionally invested in one particular approach in software, literally in anything you do, it does result in bad outcomes. The problem is you may not actually realize the consequences of those bad outcomes far down the road from the poor decision that you made that caused you that outcome so you might not necessarily connect it back. MICHAEL: Yeah, and I keep bringing this up but I think another effect of being calmer in your nerd life is having something that you do outside of your programming life, which is either having a family or having hobbies or something like that but you know -- CHARLES: Or having a wild turkey. MICHAEL: Yeah but you've got to have something, a reason to stop thinking about your tech stuff or it'll consume you. I suspect when you see the older graybeards who go on and on about open source and they're very like... I don't know. What's the word? They're very over the top and fervent about tech stuff. It's probably because like me, that's their only hobby and they haven't figured out how to how to control it. It becomes part of their identity and it defines them and then they're down this twisty, turny path of annoyance to the rest of us. CHARLES: Again, don't sell yourself short, Coté. You've got plenty: you love the cooking and eating and the drinking so close this. Do you have a favorite drink that you've been mixing lately? MICHAEL: No. CHARLES: Or any kind of favorite food because every time I go over to your house, even if we're having pizza, there's always a nice hors d'oeuvre or something to drink, something to tweak that appetite for something special. I kind of wondering if there's anything that you're into. MICHAEL: I have some very basics. One, I don't know if I drink a lot or drink a little. I think the science on this is very confusing, kind of like drinking coffee. I try to drink less. I basically go back to the basics of I want cheap wine that's not terrible. That's what I'm always trying to discover. I think I've also started to rediscover just straight vodka. That's pretty good. I think that fits into the grand scheme. CHARLES: I just can't do it. I can't follow you there. I need some, what do they call them? Gin florals? I can drink gin -- MICHAEL: Oh yeah, that's good too. CHARLES: That's about as close as I can get to straight vodka. MICHAEL: And then food-wise, I just wrapped up finally figuring out how to cook fish and chicken without it tasting terrible. CHARLES: Oh! What's the secret? MICHAEL: No, I want to put a disclaimer out. There's a EULA on this. I'm not responsible for anything bad that happens but what you want to do is cook at about 10 degrees less than you're supposed to. A chicken is supposed to be 165 degrees but you take it out of the pot when it's like 150 or 155 on another part of the pan. Fish is supposed to be 145 degrees but you take it off when it's about 130 or 135. It cooks a little bit more but these guidelines to cook your meat to that thing, it ruins it. Also you can brine a chicken and things like that. Also, what you want to get is an instant meat thermometer. One of those that you can just poke in your meat so you're always checking the temperature. That's what I've been working on. CHARLES: I have a theory about that. I will laid out really quickly, maybe it's just because the juices. It's the juice that so yummy there so you want those to be locked in and boiling but not boiled away. I'm going to give that a try on my -- MICHAEL: And fish is particularly tricky. CHARLES: Because all it takes is five minutes. Sometimes, it's two minutes and 30 seconds too long and you ruin the fish. MICHAEL: Then the next theory I want to try out is that you can actually fry fish in pure butter but you've got to paper towel it off afterwards because too much butter ruins it. But I think if your paper tower it off like you do grease off of bacon, then I think that's how you achieve -- not as good as a restaurant because in a restaurant, they have those butane torches and the crisp it up on the outside or reverse sear or whatever -- CHARLES: Is that what they do? Do they just run their torch right over the fish? MICHAEL: That's all I can figure. They might also be professional cooks who know how to cook things. CHARLES: They might have done it a lot of times. They might have had someone like Gordon Ramsay yelling at them constantly. "I can't believe this fish is so terrible. Waah!" All right. I'm going to give the fish a try. I'm going to give the chicken a try and I'm going to give everything that you just spent the last hour talking about, also a try. MICHAEL: Well, thanks for having me on. It's always fun to have a show with you. I just posted yesterday our second revival of the Drunken Retired Podcast, which is over at Cote.show. It's just '.show'. URLs are crazy nowadays. I guess the only self-promotional thing I have is I'm over in Twitter @Cote. It'd be nice if everyone should just go follow me there because I'm always very sad that I don't have enough followers and they'll never verify me. I don't understand what the problem is. I'm clearly me. Then I mentioned earlier, the main podcast that I do is Software Defined Talk, which is at SoftwareDefinedTalk.com and you should come spend a lot of money on Pivotal stuff. I'm happy to tell you all about that. Just go check out Pivotal at Pivotal.io CHARLES: I guess that is about it. We will talk to everybody later. Thank you for staying tuned and listening to this supersized episode. Come check us out sometime!
When you travel, people have a lot of thoughts about Texas.
Balint Erdi: @baaz | balinterdi.com | Rock and Roll with Ember.js Show Notes: 01:58 - What is JSON API? Advantages 03:22 - Tooling and Libraries 05:49 - Relationship Loading 07:51 - Designing a Data Loading Strategy 11:23 - Pitfalls of Not Designing a Data Loading Strategy 13:53 - Ember Data 16:37 - Pagination & Sorting 23:06 - Writing a Book 25:48 - Implementing Searches with Filters 31:08 - What's next for Balint? Resources: Balint Erdi: Data Loading Patterns with JSON API @ EmberConf 2017 (Talk) Balint Erdi: Data Loading Patterns @ EmberConf 2017 (Slides) jsonapi-resources GraphQL JSON API By Example by Adolfo Builes ember-cli 101 by Adolfo Builes 33 Page Minibook + Coupon Code! Transcript: CHARLES: Hello, everybody and welcome to The Frontside Podcast, Episode 65. My name is Charles Lowell. I'm a developer here at The Frontside. With me, also from The Frontside is Elrick Ryan. Thank you for being with us, Elrick. I know this is your first podcast. ELRICK: This is my first podcast. It's great to be here. CHARLES: All right. Fantastic. Yes, we hired Elrick a little bit ago and it's been fantastic. I'm glad to get you on. With us today is a really awesome guest. His name is Balint Erdi. I actually like to tell a little bit of a story when I have an anecdote and I do have one about you that I think you might like, although you might not even remember it. But it was shortly after EmberConf. Last year you and I got on a pairing session remotely and I don't even remember what we were working on but I was struggling with this way to decorate objects without changing them, without touching them or mutating them in any way and you showed me this technique of actually decorating it by creating a new object with the old object as the prototype. Do you remember that? BALINT: Yes. I totally knew. How could I forget? CHARLES: Yeah. That one hot tip changed my life. It is one of the best techniques that I have discovered in the last five years of working with JavaScript. It really was great and I use it all the time. BALINT: Wow. Amazing. CHARLES: Thank you. I don't know if I ever said, "Thank you," but thank you, thank you, thank you. BALINT: Yeah, no problem. I also learned a lot from this pairing session actually. I didn't know that my small contribution made such an impact. I'm glad to hear that. CHARLES: Yeah, that was fantastic. We need to actually make that happen again. I don't know why we only did that once. BALINT: Yeah, we should. CHARLES: Anyway, we're here to talk about data loading, it's something that is absolutely critical to building good frontend and building UI and yet, it's something that the users never really see. Sometimes, it feels like it's 90% of the problem. BALINT: Exactly, yeah. ELRICK: Yeah, that's so true. CHARLES: We're going to talk about techniques that we use and you use and, in particular JSON API, what it is and what's so great about it. So, what is JSON API for folks who've never heard of it? BALINT: JSON API is a standard way to build APIs. I think the specification has reached 1.0, I would say two years ago or three years ago. I remember it was in June, I'm not sure which year. It basically lays down everything that's usually consider when you build an API: how do fetch relationships, how to paginate data, how to sort, all of these things that I think developers tend to invent again and again. I think probably the biggest advantage of JSON API is that it just declare a standard way to do that. It basically reduces the byte sharing going on at the start of the project. Well, not just at the start, later on too. In my talk at EmberConf, I coined a JSON API the conventional over configuration for APIs. CHARLES: I see. Pagination is something that everybody does. Why need to byte share over the syntax, like the actual data from it? BALINT: All of these things are things that everybody does. It's just that everybody does it differently. There's a lot of discussion going on which the best ways, for example when there's a team, at least every team that I was involved with had several discussions going on about what data formats to send data in and how to paginate and all these, I think details where it's more important to get to an agreement just to agree on something and move on than to get it perfectly, if at all there is a perfect way to do it. CHARLES: I guess my question is if you have the standard way of doing of everything, what kind of tooling can you build, that can you kind of inherit for free? At what level, both from the low level and then up to the top level? When I say top level, I mean what the user is seeing. BALINT: By low level, I guess you mean the actual libraries that implement JSON API in different frameworks, right? CHARLES: Exactly. Are there now a lot of libraries out there so whatever I'm using, if I'm using JSON API, is it available in a lot of different ecosystems now? BALINT: Yes. It definitely is. There is a full page on the JSONAPI.org, on the official JSON API page that just list all of the different libraries that are now implementing all these languages. I have experience with Rails, probably at Ruby too and there are three libraries and I think all of them are pretty good just for implementing JSON API. The one I use is called JSON API resources and it's very telling. Well, it's a rather simple application but I basically didn't have to write a single line of code. I've only had to write very little code in the server to implement a JSON API specific feature. Most of the relationships could be implemented with just declaring JSON API resources and then the name of the resource in the Rails application. For all the other things, I really didn't have to do that much so in every time, it was just adding one or two lines or changing a configuration value, then it was just there. CHARLES: Now, how do you choose then what relationship you want to load and which order? Is that controlled by JSON API? BALINT: It's controlled by the frontend. It is a frontend application that's going to send these requests to the backend so that's where you should consciously think about what relationships you are fetching and how. CHARLES: Right so part of the specification is a way of specifying which relationships you want to load. In my understanding, part of JSON API is an interface along with say, the user, "I want to load all of the posts that this user has made." BALINT: Yes. JSON API indeed has a keyword called 'included', which you can implement on your backend which does this. If you specify 'include' and then the name of the relationship or relationships for several ones, the backend must comply with that request and also send back those related resources. That is called compound document in JSON API parlance. ELRICK: Is that the reverse of what they doing in GraphQL because at GraphQL, I think you have to request the relationships you want on the frontend and then it kicks it off to the backend and it gives you the information back. Is it like JSON API that includes that same thing but in reverse? BALINT: I'm not sure because I'm very familiar with GraphQL but all the things it does is that you are normally fetching a resource and then if you specify 'include' then you are telling the backend, "Please also include these related resources with a primary source." CHARLES: I think it occupies a very similar concept that you want to have control on the frontend about which resources or what data gets fetched in addition. ELRICK: Yeah, it sounds very similar. CHARLES: Well, I'd love actually to do a comparison because I know there's a lot of overlap between GraphQL and this. Maybe we can get into that a little bit later. One of the things that you said back there is having this, gives you kind of fine-grain control over when and how you load your data because that always seems a pretty difficult problem to attack on your frontend because as you're rendering your application, you have to incrementally fetch little pieces of data here and there and make sure that it's already, at the time that you actually need to render something like a component and then it's got the right data at the right time. It seems like it's this constant dance of whack-a-mole and like, "I'm loading too much here. It's taking too long," versus, "I've got too many loads happening. I've got 20 requests to run the single page." How can I back those up into a single thing? How do you go about thinking and designing that data loading strategy, as you're ready to render pages? BALINT: That's a really good question. I think the short answer is how you load data has to be part of the design process. You really have to spend time thinking about how you're doing that based on the needs of the UI. I think the way you need to render the UI will suggest the way to do data loading. Especially in Ember, I think do I really need ways to, for example block the page from rendering too early so anything that you want to render first, you can fetch in this non-blocking way into model hook of Ember. Then anything else, if you're okay with rendering later, you can fetch in other ways like to set up controller hook or from the templates or from controllers or whatever way but not in the model hook. CHARLES: But if you are doing something outside of the model hook -- because this is something I feel like a pattern that comes up a lot -- and regardless of where you're operating, if you're using Ember, if you're using another framework, you have kind of this top level data loading. But then you have your nested components might need more data, how do you go about loading that data. I guess you have to think about that also. Upfront it's like, "I've got this component that might want to request more data," how do I actually design that and think about that of I've got this data that's going to come, who knows, maybe minutes after the initial route is rendered? BALINT: That would be different but I think that you can apply the same principle. You can fetch some data in the model hook in this blocking way and pass it all it down to your component if you don't want to render the component before the page renderers or you can just even fetch it from the component while the component is rendered. CHARLES: You're thinking maybe, in terms of streamlining, the rendering process so that you can begin rendering while your data is loading. Is that the use case? BALINT: Yes. If you, for example fetch the data from your component, then it's just going to fetch the data as needed. You have the whole page load as render and then when the component is render and it fetches the data, then you're going to see other requests go out to the backend. The data comes back and then the component is going to be rerendered with the new data. I think in most cases, that's totally fine but you might have a use case when you don't want to render the page before the component is all over the data that it needs. In that case, what you can do is once again, if the fetched data is needed in the model hook, it will just pass it all down to the component. ELRICK: What are some of the pitfalls that you would run into by not thinking about your data loading strategy beforehand that you can pull out and explain? BALINT: I think the classic one is what was known name in Rails and other framework is 'N + 1 problem' and it's when you fetch many related resources, then you might end up with doing end requests for a number of resources. In the case of Ember, you have for example a blog post as your model and then in your templates you write model.comments, then what's going to happen -- depending on actual library that you use on the backend but I ran into this myself -- is by default, if you are going to make those end requests. Ember solution is really, they just works. I mean, the default solution just works and you might not even run into this because you might not have the scenario. You might just have a few records. But if you have a great number of them, then it's going to be a bad experience. ELRICK: So someone that just dealing with Ember and then they go and make a request and then see all these requests come back, that would be something that they would then have to turn around and asked or fix within the backend to say like, "Only give me a certain set of this." BALINT: Yes, exactly. ELRICK: Or is there something on the Ember side with Ember data that you can say, "Only fetch X amount." BALINT: Right. I think both. I can speak about using Ember data and JSON API resources so what you can do to mitigate this case is to use links or cause relationship links instead of fetching the comments one by one, the backend can actually drive Ember data to fetch all the comments in one request from the link that it sends the frontend. You first ask for the blog post itself and then a JSON API compliant backend will send back the blog post resource but it's going to have a relationship link inside. An Ember data automatically records this so when it needs the comments for the blog post then it's going to fetch down from that provided URL. Actually, I haven't really talk about it in a lot of detail at EmberConf. CHARLES: Yeah, I see. I'm going to go off on a little bit of a tangent because I feel like this is a pattern that's coming up more and more. To give a little bit of context, I feel like the way that our data loading strategies have evolved is we're used to the page loads, then we kind of analyze the URL, we decide what data needs to be loaded to render our components and then we pull that data from the server based on a decision and then we do our render. But it seems increasingly more prevalent than we are having a combination of both pulling the data from the server and having the server push data onto us. One is what are the strategies for dealing with that? Given kind of certainly, at least in the Ember world, routes aren't reactive. Then does JSON API actually help with that at all? BALINT: Yeah, a good question. I don't think that JSON API specifies how [inaudible]. You are thinking about something like web sockets like pushing data from the server sockets. I don't think JSON API covers this summary. CHARLES: Yeah. It definitely seems like beyond the scope but I'm wondering if there are any thoughts about general strategies, about how to handle this model state that sitting at the top of your render tree. In Ember for example, in your route and how do you handle the fact that the route requests data and how do you handle data coming in after the initial render? BALINT: If you use Ember data, then you can push data coming in from a web socket, for example to the store. You probably do some massaging on the data that comes in and then you push it to the store and then depending on how you fetch the data, you might have a live collection. For example, if you do a store, find all notifications and then notification is coming in that is going to get displayed right away on your page because then your template is bound to a live collection. But not all of things in Ember data are live collections. CHARLES: Okay, it's mostly library dependent but if you're using Ember data, then you just push those directly into the store, those live collections. It kind of like a real time query. BALINT: Yeah, exactly. But what I'm saying is that not all Ember data query that you do are live collections. For example, relationships are probably not, depending on what method you use to fetch that data. You might have to do some additional footwork. CHARLES: Okay. Now, getting back into the area in which JSON API really does shine at things that can really shave off a lot of time and consequently money from your work, let's talk about those a little bit. For example, you mentioned pagination. How is JSON API going to help me if I want to have paginated data? What are the scenarios on the client where I would need paginated data? Then maybe we can walk back from here's this user interaction that I need paginated data for and how is JSON API going to help me with that? BALINT: Sure. I guess the typical scenario on the frontend is there is a long list of items and you don't want to overwhelm the user by showing them what it wants. You need to just show them page by page so JSON API recommends to use so it's agnostic about the exact paging strategy that you use. You can use the classic page-based approach or cursor-based one or start of pagination technique. It doesn't really force you to choose the way you want to do that. It also mentions that, I think the way it frames it, you might want to use the page for your parameter. I think that the libraries, at least at JSON API resources for sure, you use that page parameter to send back paginated data. You have a page and a square brackets number and then page size. The request variables are page number and page size. Then the server knows that I just have to send back the second page if the page size is 25 and they just set that [inaudible]. CHARLES: Then if you're using a library on the server side, then you don't have to do any extra work. BALINT: Yeah, exactly, depending on the library I guess. JSON API resources made this one really simple. CHARLES: I see. Then in terms of library support on the client, I assume that there are libraries, like Ember data that automatically will support this so when you're creating these live queries, you can include information about the page. BALINT: Yes. I'm very in to Ember so I'm not sure about the frameworks but I suppose that there are some ways that make this very easy for the developer in many of these frameworks like Angular and React. CHARLES: Right. Something that has just bit me on the butt so many times is when I have paginated sorted data. Imagine you've got some infinite scrolling table or not infinite scrolling but you've got some a bunch of table rows that are maybe 300 of them or 3000 of them so you don't want to load them all at the same time. But at the same time, you've got complex sorting that's happening on each column. You might have seven layers of sort. You want to sort by name, followed by ID, followed by date. One of the biggest problems I've encountered is trying to reconcile and sort on the client versus trying to sort on the server. Are there any facilities to help you deal with that? BALINT: Yes. I think the approach I usually take is if you do it on the server, then do it on the server. Do not mix the two things. In this case for example, if you are sorting and then you change just sort field, I would just send a request to the server to have the items returned according to the new sort criteria. I think that's the simplest approach because as you said, I've probably experienced things can get really messy, if you want to do that on the client. CHARLES: Then you've got the third page but when you do the sort, the contents of the third page could be anything. BALINT: Yeah, that's the other thing. I think if you change the sorting criteria, you'd probably want to go to the first page. I haven't thought through all of the scenarios but I think it's really rare that you want to stay on the fourth page while you change the sorting criteria to be created at descending. You probably want to see the first item the way it was created. CHARLES: Someone should seriously write a book about sorting and pagination and loading these data sets because seriously, I feel there's this tribal knowledge of things that people have learned from screwing it up. There's not a written down way of this is how you build the data loading layer for an infinite dataset so that you can sort, you can paginate and here are the problems that you'll encounter. BALINT: There's actually a book written by Adolfo Builes. CHARLES: He wrote a book on Ember CLI, too, right? BALINT: Exactly, Ember CLI 101. He's the same guy and he wrote JSON API By Example. I have it on my mental to-do-list to buy that book. I'm not sure if it covers these exact scenarios but he must cover several in that book. CHARLES: Well, I'll be sure to reach out from because certainly there are a couple of scenarios that have bit me too. The other one is where you have some collection that's paginated and sorted, then someone adds on the client side a thing to that collection. Say, you want to create a new row in that table, well then what do you do with that new row? chances are it's not going to be anywhere on the screen because who knows what the sort order is and the terms of the total sort, which is only the server knows and who knows what page it's going to be on? You get all these problems that compound and it would be great to have one place where people could reference them or have a little cookbook. BALINT: Sure, absolutely. I think in that scenario, the simplest thing, which I think probably works best like 99% of the cases is again, just to reset the sorting and the pagination. If I create a new record, I really want to see that new record. CHARLES: Yeah, maybe you put the new record up in a box up, at the top in a special new record place. BALINT: Sure you can go fancy and do that. That would be a good solution too. But probably you can just reset the page number and show the first page with the new record. CHARLES: Ah, yeah. I see. BALINT: It's tricky. It's going to get more complicated than this. CHARLES: You might have a problem where you don't want to lose that context to the records that you were looking at. BALINT: That's right. Somebody should write the book in this. I know somebody who wrote a book [inaudible]. CHARLES: It could be you. You haven't written a book in over a year, right? [Laughter] BALINT: It's been two years already. CHARLES: It's been two years. BALINT: Exactly. CHARLES: I'm never going to write a book again. I don't know. Do you think you might? BALINT: I think I might. I actually have this urge. CHARLES: If I recall correctly, you're one of the few people I've talked to who was like, "Yeah, you know, writing a book, it wasn't that bad. It was kind of an amazing experience." And literally, everyone else I talked to have been like, "Urgh!" ELRICK: That's really interesting too because his book keeps up with the releases of Ember so that makes it even harder. That's surprising to hear that like, "Oh, it wasn't that bad." BALINT: Exactly. Well, I kind of put off writing the second book for a while because if I just wrote a book, then I could just be done with it, I would be happy to start writing the second book. But if I have to maintain it for years, I don't have to do that but it's just so much extra work that I'd have to take this into consideration. CHARLES: Yeah. Essentially, what you've done is you've rewritten the book. In terms of absolute content, given how much everything has changed, you probably flipped over the content of the book in the same way that people flip over the atoms in their body. I think there was something like you don't have a single atom in your body three years from now that you have today. It takes about three years and you have all the matters completely and totally exchanged. It's kind of like that. BALINT: Yes, it's kind of like that. Well, I think maybe half of the book is still relatively as it was when I released it because since Ember 2 came out, Ember didn't change that much so in the 1.X series it did change a lot and I think my book originally came out when Ember was 1.10, I would say. There's a lot less work that require now to maintain that it was back then. But it had changed a lot for sure. ELRICK: Yeah, I think I bought the book on its first release. It was 1.10. I guess you automatically get assigned to the GitHub repo so you just see a constant barrage of updates, updates, updates and I'm like, "Wow, Balint is really killing it in updating this book." BALINT: Yeah. CHARLES: It is a good book and everybody should go buy it. The other thing that I want to cover to, as long as we're talking about scenarios that come up again and again, we talked about pagination, we talked about sorting, what about things like search? Is there a uniform mechanism to help you out there? BALINT: You can implement searches by using filters. It's a JSON API concept of using filters. You can pass a parameter called filter to your query and then a square bracket. You have the name of the field that you want to search and then just pass the value, the search term basically and then backend should return the items that matched according to some criteria. That's the simple case. CHARLES: It's a simple case but clearly, it's up to the backend to implement that API. BALINT: Yes. CHARLES: So I'm wondering what libraries are available if I'm doing something in Elixir or I'm doing something in Sinatra or I'm doing something using Express, how seamless is it because I feel like a lot of times you can run into problems where these leaky abstractions about the fact that one thing is a Mongo backend, then one is based on Postgres. Maybe that's a better example than a different server technology but more of sticking with a single server technology -- let's use Ruby -- but one is I'm using a Postgres backend and when I'm using, say MongoDB or some other key value store. In your experience, if you've seen this, how much does the backing store leak into the frontend? Is JSON API a good protection and the ecosystem around it from those leaks? BALINT: Yeah, that's a good question. I was about the say that the backend needs to be the abstraction that's use you from having to know what kind of persistence layer you use. The frontend shouldn't care about -- or any client of that backend -- whether you use MongoDB or Postgres. That's a responsibility of the API. You can still send, in this case for example, a filter query. However, the backend translates this to database queries. That's his job. I think the answer to your question is that JSON API does protect you from having to know the intricate details of the database. CHARLES: You might have some work to do but it's possible. BALINT: Sure. You might have a lot of work to do on the backend but it's possible. But it's not just JSON API that protects you. Any kind of API should protect you from this kind of knowledge. CHARLES: Right, unless you're using GraphQL. BALINT: Could be. I don't know exactly how GraphQL works. CHARLES: Yeah. I think there would be actually nice to read something from somebody who's got a lot of good experience with all of these, like different technologies to make a comparison. I feel kind of in the dark. Unfortunately, the problem with any technology, I feel like most of the comparisons out there, if you're going to compare, most people have a huge implicit bias for one tool and a little bit of experience with the other, maybe dangerous of like, "I don't definitely want to render opinions on my GraphQL and certainly not versus the API that I'm used to because I feel like I can't make a good comparison." BALINT: Yeah, absolutely. That's a good one. CHARLES: But it is something that I'm so intrigued because I feel like there's a lot of overlap there but who knows. BALINT: Yeah. ELRICK: They need that to do API like how they had to do MVC, they need to do API comparison. CHARLES: Yeah. One thing we could do, we could implement JSON API over GraphQL and kind of just move your backend on to the frontend. BALINT: I think you could but you just said that with GraphQL, you have to know the client on the frontend, like feels the database. CHARLES: Right. You could technically have a JSON API on top of your GraphQL backend because I think the thing that kind of freaks me out -- this is a crazy idea that no one should ever do it but I hope someone does -- about GraphQL is being as old as I am, I saw so many projects ruined by the visual basic kind of mantra of just like, "Just query your database right inside your components," and that was literally the rope that hung ten thousand projects and just made people despise visual basic development because there was no shield and then literally every button was coupled to your database. But you could theoretically have the best of both worlds where you're sitting on an abstraction that's also on your client but just moving your query language from your server over to your client or something like that. BALINT: Yes something like that could work, in theory. CHARLES: In theory. Like I said, no one should ever do that unless they really want to. BALINT: Yes. CHARLES: Fantastic. The other thing I want to ask you is kind of what do you have cookin'. You got your book, you wrote but you keep updated, you recently have been evangelizing data loading patterns most recently at Ember Conf. What's next? What's now? Or you're just kind of taking a break? BALINT: I already have published book a mini-book about these data loading scenarios. Actually, I just cover the things I told them out at EmberConf then add some more but I might make this into a full-fledged book, providing I don't have to update it. [Laughter] CHARLES: I'm going to write this book -- BALINT: But just once. CHARLES: -- But just once. BALINT: Yeah, I still have to find a way of doing that. CHARLES: All right. We will look for all of those things. One thing that just occurred to me is just how much of actually building UI and building frontend really is about thinking about the structure and flow of how you load your data and how much the user doesn't see that but how important that is to provide a good experience to that user. That's one of the things that sometimes we, as UI engineers don't like to think about but I think it is absolutely true and crucial and foundational. Thank you so much, Balint for coming by and talking with us about these important topics. We will see everybody next week with that. I will bid everybody to do. Good bye, Elrick. Good bye, Balint. BALINT: Yeah, thank you very much for having me. Goodbye.
Ginger Whalen: @gingerwhalen Show Notes: 01:28 - Everyone Does Sales 02:11 - Sales is an Exchange: Understanding and Solving Others' Problems 05:58 - “Personas”; Empathy vs Sympathy 11:47 - Empathy Requires Introspection 15:12 - Persona Example 20:33 - Making Incremental Improvements 23:15 - Challenges in the UI Business Space 29:25 - TL;DR on the Business Development Process Resources: Thinking, Fast and Slow by Daniel Kahneman Transcript: CHARLES: Hello everybody and welcome to The Frontside Podcast, Episode 64. My name is Charles Lowell. I'm a developer here at The Frontside. I'm here with Jeffrey Cherewaty. Hello, Jeffery. JEFFREY: Hey, there. CHARLES: With us today is someone also at The Frontside but not one of our developers. Her name is Ginger Whalen. Just to give you a little bit of backstory on her and why I'm so excited to have her on the podcast is we, about the middle of last year, began a search to bring someone on to help manage and grow and just have their eye on our business pipeline because that's something that's really, really critical, it turns out, to a software agency is making sure that we have clients. We put together a pretty extensive search plan and then we executed it and it was, I think about six months but in the end, we ended up hiring her. The reason we did was because she had a very unique take on what we would typically deemed the sales process so we learned a lot about it and I wanted to have her on the show so that we could just kind of share with our, I would say, mostly skewed on the technical side audience about what a healthy sales process might look like. Welcome, Ginger. Thank you for coming. GINGER: Thank you. My pleasure. It's so funny when people talk about sales and they say to me, "Oh, you're a sales person," I think that's so funny because I've never viewed people to be in sales and people not to be in sales. To me, everybody is in sales because all sales is to me is somebody just exploring somebody's problems with them and then when the time's right, you educate them and then you help them see the value of acting on some solution. Who doesn't do that? We all do that. CHARLES: We all do that. We do it in our day-to-day when we're proposing and discussing and talking about technical solutions, the same principle may be applied in a different level. GINGER: Yes, some people are just on the hook for it, that their professional goals are tied to that end game. CHARLES: Yeah. One of the things that struck me as kind of setting you apart when we were doing the interview process is the way in which, I want to say the focus of building those relationships and kind of the focus on understanding and understanding who exactly potential customers are and trying to categorize them and tailor our message so that we can actually maybe help them. Maybe you could explain a little bit about that. What's the front half of that process looks like? GINGER: That's kind of the bigger subject. I guess sales is the exchange, the educating and helping somebody solve a problem, get what they want to solve that problem. But if you're not speaking in their language and sometimes you hear the term 'knocked around personas' -- the persona you're talking to or the person you're talking to -- if you're not speaking in their language or addressing their pain, talking about their problems, it just so boring to them. It doesn't really seemed like it's going to solve their problem and you just don't hit the target right. You're probably not going make a sale in the end. That's what we talked about when we first start talking with Charles and The Frontside team. We talked about personas, who are you approaching, what are their pains -- their business pains, their pains with their engineering team? What do they come to the table with and what would they like you to help them solve? The subject today, we're going to get into this a little bit later about empathy and then is really what it's all about is that gift of really understanding somebody else's experiences, somebody else's problems, their emotions and how we use that in sales as we're going to talk about a little bit more today. CHARLES: Yeah, I really like that because the drive there is to maybe not always shoot for a sale, to really try and understand like you said, someone's problems and just go out there and talk with interview, listen to a lot of different people and realized you're not going to be a good match for most of those people out there. If that's the case, not trying to view that as a potential opportunity that you want to just kind of force through but rather say, "How can I help this person even when it's not through my services and developing that and having this sales process?" because I think we tend to have a stereotype of what it is as being like I have a goal. When I want to interact with this person, what my agenda is actually to make a sale to them. It's clearly not like that. I think it varies from business to business but that's kind of the stereotype that I had in my head before this process began. JEFFREY: It feels easier as an engineer to be able to say what I'm trying to do is solve your problem versus what I'm trying to do is a self-centered like, "I want to have your business." That's very different approach to the problem. GINGER: That's so funny because I view it as exactly the same as what you guys do. You're solving problems, you know the end game is you're going to write this code to figure this out to do this thing and this thing is your end goal. That's the same thing I'm doing as I'm trying to fit my solution with this person but the means to get there, I just have to let it unfold. Sometimes, it's really hard and sometimes it's really complicated and takes a lot of code and lot of trying and then trying something different, then that broke and then trying something different again. In sales, it's the same thing. Especially at this level, when you're with a consultancy, you have multiple buyers or multiple personas out there. You're selling up into an organization that can get pretty complex and you can have some fits and starts, just like you do when you guys are coding and trying to solve a problem. CHARLES: Now these personas, this was actually something that was really interesting to me, the kind of introduction is this as you understand the problems and pain points, you're trying to collect them into related that this person has these general challenges and problems that need to be solved. How do you go about developing those personas? What is a persona and how do you go about developing them? GINGER: First, you have to know who your decision maker is in an organization. From that, you develop your different personas. Now, some of these personas are the absolute decision maker at the end so the guys can assign. Then some of those personas are influencers throughout an organization. For example, in this field, we talk often to a lead developer. We talk to VP Engineering. We talk to CTOs. We talk to presidents and CEOs of organizations. They probably have different organizational goals, different things they're on the hook for. They have a common vision, absolutely that they're all working toward so maybe a business growth goal that they're working towards. But the VP Engineering might care more about the cohesiveness and the development of his team. He's going to be really looking for an engineering team that fits well with his team. He wants to make sure that their productivity doesn't go down. He wants to make sure that this team merges well with his so he's thinking about lowering his risk through this experience, keeping everything smooth. Where somebody at a different level, a CEO is maybe looking at something a little bit different. Are we going to hit this deadline so we can get this product out, so we can get our first customer, so we can get some revenue in? Maybe he's just looking at bottom line numbers. If they come to the table and they have all these different problems or different pains, you really have to speak to each one of them about their pain and make sure you're addressing that. That takes some empathy. Not sympathy, not saying, "Oh, my gosh. I feel so sorry for you having all these problems." That's sympathy. That's pity. But empathy is not about telling your story, "Oh, yeah. That happened to me." No. It's about understanding their story. It really does take some listening in the beginning to develop your personas and to really get, not coming to the table and saying, "I think this is what his problem should be because he's this guy, this level, in this organization," but really listening to them and letting them tell you what their concerns might be. Don't be afraid to ask for those because that is how you develop those personas. CHARLES: Right. You really just have to walk in a lot of different shoes and then write down those stories. That's what a persona is, it's writing down the narrative based on trying to discover people's actual experiences. GINGER: In the old days, they say personas are about demographics like this guy is about this age, eats this for breakfast and lives here and has this sort of family life. It's a lot more developed than that now and it really again focuses on their pain because you're the consultant in there. That's what they want. They want somebody that's going to be able to educate them on their services or something in the industry, teach them something about their business would be great. But what they're really looking for is someone to help them solve these problems, "Help me solve these problems. Can you do that?" and I would choose that person to work with if they can help me solve my problems. JEFFREY: So much of this process feels like it's a lot of listening. It's a lot of deducing what the problems are based on what you're being told. At what point in the process do we present a solution? GINGER: Yeah, a good question. Just like the timing of a joke. That's really important: the pauses and when to do what in the joke or just the flat. During the process, when you're gathering information, you're also qualifying them. As you're asking these questions and getting them to reveal their problems and pains, you have some questions. You really need to get answer to. This isn't just, "Let's go have coffee together and talk about your pains." It's really you're qualifying them. You have some strict qualifying questions that you know about an ideal client for your company. Some of those qualifiers might be how many employees do they have, what size company are they, what stage of growth are they in? Are they hypergrowth? Are they scaling? Are they a startup? Are they in a certain silo of business? Do you do especially well in education or energy? Then triggers: what's going on with their business? Did they have a lot of recent personnel changes? Is that a trigger that usually helps you segment your ideal client profile? They just have an acquisition. Is that a good trigger or signal that they're qualified prospects for you? While you're asking these questions and again, identifying your personas going through their pains, you're also qualifying them. I'd say when do you go to the part where you're selling -- Charles has been on sales calls with me and he's probably already noticed -- that you're always selling the value of your organization. You're always selling but as far as giving them a solution to their problem, you wait for that. You don't give that away, right away and you can miss your chance or miss the punchline if you give that away too soon. I'd say, once you truly understand their problem and then you get agreement from them that this is a problem, this is really important to solve and there's some urgency tied to that, when you have those things, then you can begin your presentation and show them the solution to that problem. But there's two parts: one is identifying the problem, two really find out that they agree. Play it back to them that this is a problem, how does it affect them and if there's the urgency tied to it. That's what you need to know before you propose a solution, I'd say. JEFFREY: It's interesting that you mentioned how much empathetic sales process also requires introspection and besides understanding the customer's pain points and what problems they need to solve, you also have to recognize what problems we are able to solve and what problems we enjoy solving and that I think is counterintuitive that empathy requires introspection. GINGER: I agree. You have to intellectually and emotionally identify with somebody and to the point where you can really experience what their attitude or their emotion or their feelings is. That's definitely an introspective thing. Yeah, you're right. Not the typical old school salesperson that just blabbering all over the place, being an extrovert telling their story. It really is more of an introspective. I'll even say introvert way of selling and that can be very powerful. CHARLES: As you do that, listening and do that introspection and then develop this understanding of your clientele via these personas, once these personas are developed, how do you integrate them back into the business development process? GINGER: That's a good test to say, "Can I really put myself in this person's shoes? Do I understand them? Do I have a better bond with them now because there's a mutual respect because we understand each other's experiences and thoughts and attitudes, pains and problems about something? Can I put myself in this person's shoes?" CHARLES: You spend time listening and talking and introspecting and developing these personas of your clientele so then how do you actually, with that information in hand, use that to build your business development process? Now that you have this information, how do you actually leverage it on a macroscopic scale? Like I'm kind of out there in the world now, I've got these personas in my tool box, how do I go out there and interact with people based on that? GINGER: I view that as an overlay over a sales process. That's the umbrella over the whole sales process. In a sales process, we do our first calls or connect calls or exploratory calls, again where we're learning about the company and their products, getting to know our personas. Then as you understand that persona, when you get into their goals and their plans and their challenges, when you're in the discovery part of your sales process, that's the part where you can transfer those points of pain and their challenges into your solutions. Again, that's just a part of the conversation where you're speaking about value of the company, you're letting them know you understand their pains. When you get to the presentation, this isn't really explicit. It's not explicit in saying, "I know this is a pain. This is a problem you have," you turn it on its head and you take those challenges that they've told you about and weave those into your solutions. It's kind of where the rubber hits the road there because if your solutions don't solve their problem, it's absolutely where you're going to figure that out when you're presenting or when you get ready for that presentation. Would it help if we talked about, maybe one of those personas and wove that through the sales process? CHARLES: Yeah, I think that's a good idea. GINGER: Okay, a popular one with our type of business -- the VP Engineering. The VP Engineering, when we're talking to him, we're really trying to understand what he values, what's important to him in his work, what problems need to get solved. This person is really caring about quality, the quality of the work and the value that they're getting for their money. We talked about their team developing, learning as they work with us. If that's important, we're probably going to really stress the quality of our service. How do we do that? We're going to talk about the way we code, the way we test, the way build software. We're going to give them supporting evidence and case studies of how this is played out. We're going to talk about the value in this. We're going to talk about, maybe how we've reduce some time, for example [inaudible]. We've done that recently with a client where he's getting better value for his money because we're talking about his [inaudible] in reducing some of the time to build. If we're talking about the development of his team, that something he's really concerned about. Is my team going to continue learning? Maybe we can present a story or a case study about a team that when we arrived, it looked this way and when we left, it looked this way. They had these additional skills... I don't know... Maybe you can jump in here but there's a client we worked in where their skills were totally transformed with us pair programming or working alongside them. Something like that are really important to VP Engineering. CHARLES: Yeah. One of the things that this process has taught me and made me see and try to cast for wherever I can find it is actually trying to really quantify and measure that value. I know that's really hard because what we do is development. That's a really long process. If the product you're developing is successful, that thing that you put in there is worth a lot of money. If, 'thanks,' then it's worth nothing at all and the thing that you did was negative value because it was just sunk cost. Those are two huge extremes and it feels like how do you get a little bit more fine grained in that and say, "Here's this activity that we're proposing and it's going to save this much money." You hear a lot of this people saying that you should try and do this this but how do you actually go about it? One of the things that kind of working in this way has taught me is really to try and wherever possible, it's not always possible but attach monetary value to the tradeoffs that you're proposing and the solutions like saying, "What is the impact of the work that we're going to be proposing?" "We think it will save you $30,000." When someone actually takes a look at that and says, "That is a lot of money." That is, "We think that this could save you 50 developer hours per month." They say, "That is a lot of time," which again equates into a lot of money. GINGER: That would be huge. I'm sure that would make him smile. CHARLES: Right, exactly. You always hear that but it's trying to be more deliberate about it and really you have to think a lot about it to try and get to that level. GINGER: Yeah and you just help me thinking about it all the way to the bottom line like that is terrific because in this situation, a VP Engineering, that person might have to sell it up the organization and maybe he has to sell it to the CEO and maybe the CEO is not going to spend as much time with us, may not even meet us directly. Those bottom line numbers might be really important for this persona to internally sell to the person that's actually going to give the nod and about their concern about we're going to get it right the first time, how do we help them get comfortable with that? That's one of those risk issues that a VP Engineering might have. Knowing these thing about your persona, knowing the pain that your persona has, all these things can be woven into your presentation. This is where, again the empathy comes in because you are speaking to their pains, their problems. This is an emotional sale. I don't care how technical something is, it is always in the end is going to be an emotional decision. It really is. Who's that author we're talking about that? Daniel Kahneman? His book out there: Thinking, Fast and Slow. He did this study where he analyzed super, super wicked, smart people and then the rest of us, smart people but normal people. He gave them different decision making challenges and they watched the brain patterns. In the end, the part of the brain that lit up was the emotional part lit up and not the rational side. Even though they may have weighed all the evidence and maybe had a much more complex rational decision making process with their data, in the end, the part of the brain that lit up was emotional side. No matter who it was. CHARLES: Yeah. GINGER: Good to know, right? CHARLES: It's an adaptation that our brains work so that we actually think that we're acting rationally so we fool ourselves into that. But most of the time, people are making decision with their guts. GINGER: That's how smart we are. We fool ourselves that way. CHARLES: It's the ultimate trick. I'm curious. When you have this process in place, you're developing, leaning on these personas, weaving them in to your solutions and the story that you going to be presenting to your clients, how do you iterate on that? Just like we iterate on software, how do you make those incremental improvements and take what you're learning and then feed that back into the rest of the process? GINGER: First, I'd like to answer that on a human level. I think the way we all learn this is we just practice it. As we put ourselves in another person's shoes, here's the part about it is when we're really dialed into empathy, we're dialed into that feeling of that emotion that somebody has, we're sharing in that feeling. To do that effectively and even when you're presenting a solution to your persona sale situation, talking to your husband or wife, talking to your daughter, words matter. You're really do have to choose wisely and your tone of voice matters. If you're in person with them, your body language and the volume you're using. Those are some of the things that, I think will help when you're presenting or really in any situation to express that you're with them and you're with among the emotion. Let's say that it's something you have an experienced, this VP Engineering, maybe he just had a huge failure. They did not get it right the first time. Their software broke. The money once spend on it proving that it was great, he had to spend on fixing it and starting over. Maybe that never happened to you, hopefully it hasn't but you feel the heaviness of it and you know sadness. He's sad, he's desperate, he's scared, he's going to lose his job, you've had those emotions. That's what you can join him in that emotion. Not pity, you poor thing. More like, "Oh, that sounds really tough. Wow, that doesn't sound like fun." That's empathy so you're there with them and weaving that into your presentation, again not just blowing it off, "Let's not have that happen again because it won't happen that way with us. Here's our solution." That is blowing your chance to have some empathy and sit with them in that emotion for a bit before you get onto your solution. That would be one way to weave it into the presentation. But that takes something for him to share something like that with you too because he knows you're going to counter it and say, "That's not us," but does he know you're going to share in that emotion? Like, "Hey, that didn't sound like fun. Wow. Heavy." CHARLES: Yeah. I'm curious and this is maybe a little bit of a curve ball for you but as you come into, every business is unique. I think doing UI consulting is one unique business out of many. What has been the biggest challenge of coming into this space and trying to develop a clientele inside of it? GINGER: The trick for me since I don't have a heavy technical background is to get away from the technical conversations. Since that's not my strength, I can bring in you guys, bring in a team when it's right, when that person is qualified. But the challenge with me is to find the business people that really want to talk about, "We had this great idea but we don't have enough of bandwidth to execute on it." I can get the idea sold internally and it was just too complicated for us to achieve. I can get those conversations going where really, we're talking about pains, we're talking about business problems that you can approach in any business you go into. It doesn't matter what the end game is and what we're selling. It really doesn't. Everybody has these business problems. To get people to talk about these business problems is the key to make those context to get them to open up about those business problems, business pains then you can qualify and see if it's a fit for your solution. The challenge is to initiate those conversations with people because people do want to get technical quickly and to back them up and to talk about the business first because in the end, it probably does have to get approved by somebody that really is thinking about the business pains and business problems, maybe differently than this person's thinking about it so you want to do the work in the beginning to let all the surface before getting into the technical stuff. That's been always a challenge, whether it's web design development, marketing technology or an engineering UI company like us. CHARLES: Yeah, it's always a challenge. I think you said this at the very beginning of the podcast is you need to be speaking the language of the person that you're speaking with. It seemed so obvious but the reality is though, everybody might be speaking English or one of several human languages within each one of those human languages is literally a million different actual languages, which is carrying on the context of what those people experience in their daily lives. Sounds like what I'm hearing is that there's a specific language within this business and part of it is learning to speak parts of that language, then also be like, "I need an interpreter. I need to bring in someone to speak to this person in their language." On the flipside, I think it's also great too. I'm not a native speaker of some of these higher level concepts and I think that our strength here is in speaking that strong technical language that other technical people can bond with immediately but when you start backing up and just talking about those higher level business concerns, it's really great also to have someone who speaks that natively. Also, that language of even higher than that of, "Let's understand each other's problems here," which is something that you're very fluent in. It is interesting to see that there really is many, many different languages within one language. JEFFREY: I think even within the technical conversations, there are multiple languages. There are different levels of, I don't want to say understanding but different levels of comfort and familiarity with different technologies, with different stacks. A lot of the conversations we have with companies that ours is comfortable in the stack that we're comfortable in and we're presenting that to them and that's where they're coming to us in the first place is because we have that expertise and we have to translate it into, I think that they understand and that fits their needs. GINGER: And that's something, I think you guys are really good at. I don't know if you're giving yourself enough credit there but I think you do ask really good questions about, "Tell me about the experience you want to build? What type of goals are attached to this project?" You guys asked about timeline. That's a goal when you get launched by then. You guys asked what would you say the strengths to your team are. I've heard these questions when you're talking about requirements so you can start to assess, how complex, how much we're connected, pair program with them, help them. I think you guys do ask about goals and plans and challenges. It's just kind of packaged up a different way. CHARLES: Yeah, part of that is just having borne the battle scars of not asking those questions upfront, rather than gravitating through intuition. GINGER: I think we all unfortunately or fortunately learned that way, right? "Don't step in the puddle. Don't step in the puddle. Oh! Did it again." [Laughter] CHARLES: Is there any way to learn that doesn't involve stepping in the puddle or stepping on the tack or falling out of your chair? GINGER: I guess, if you're a baby and you don't know anything yet, you're maybe 50/50. You'll get some of them right. CHARLES: Yeah, it is funny. But just a brief diversion, I just remember there was a period in her life where my daughter would just randomly fall out of chair. It was out of nowhere. It wasn't like she was sitting precariously or whatever. She would just fall out. It happened a lot and I was like, "Wow, sitting in a chair is a learned skill." Anyway, it just made me think about that is that there was actually this pain-based learning process there. GINGER: I wonder what the response was. Did the room laugh? Did the room say, "Oh, let me help you?" Maybe she was ahead of the game. [Laughter] CHARLES: A lot of times it involved tears, which made the laughing worse. I mean, you can't help but laugh. You can also hide it. GINGER: Yeah. Maybe we do need to fall a few times to learn stuff. CHARLES: Ginger, if you could summarize what your business development process looks like in a few sentences before we head out. GINGER: I would say, take your time when you're talking to new people. Get to know them. Get to know who the person is. Get to know what their pains are before you tell them how you can help them. On the empathy side of it, I think the key to empathy is really to remember that it's really not about telling your story: matching them at every step, it's really about understanding their story. CHARLES: All right. I really like that and I really like the process that you're building here. There you have it folks. With that, I'm Charles Lowell from The Frontside. I hope you enjoyed this episode. Thank you, Jeffrey. Thank you, Ginger and we will see you all next week.
Charles & Coté reboot their old podcast about regular things. Also, a rant on photo management in Apple-land and how terrible enterprise IT news it. Plus, upcoming topics.
Show Notes: 00:56 - What does UI mean? UI = User Interface 02:26 - Software Interfacing with Software vs Human Beings 03:57 - At what point does UI stop? 05:55 - Responsive and Stateful UI 10:10 - Tooling: Past, Present, and Future 16:00 - Planting Business in UI Engineering 19:26 - How The Frontside Brands Itself; JavaScript Portability Resources: Linguistic Intelligence Transcript: CHARLES: Hello everybody and welcome to The Frontside Podcast, Episode 62. My name is Charles Lowell. I'm a developer here at The Frontside and I'm here today with Jeffrey Cherewaty and Robert De Luca, also developers here at The Frontside. We're going to do a little bit of navel gazing today. We're just going to talk about how we roll and who we are and because we've there's been a lot of conversations about that, as we had a couple of different projects come in that use different tool sets so what is it about us that is constant and what is it about us that changes? I just wanted to talk a little bit about that. The core idea is about UI. The shop has always been about UI and I'm curious for you guys, what does UI mean to you all? ROBERT: The frontend to the frontend? JEFFREY: That's a loaded question. CHARLES: Okay, let's step back a little bit from that because we've been in business for a long time and right now, we're doing mostly Ember, a little React. Before that, we were doing all Ember. Before that we were doing mostly Rails. Before that we were actually doing a little bit of wxWindows stuff and before that we were actually doing stuff in Java. But all of it fell under the rubric of UI -- user interface. What is constant? I mean, there's some radically different tools. When you look back and you think, "My goodness." What could possibly connect those things that I just listed? But I think there is a common thread so I wanted to explore that a little bit. JEFFREY: The common thread, I guess at the simplest level is always that in a lot of programming, you're dealing with the interface to the end-user of your code is simply text. It's a call-in response or they're consuming a library of yours that's where the code is the end product. What we focus on instead is the the end product of what we build is the interface as the graphic representation instead of a code or a text-based interface. CHARLES: I'm curious. In terms of all the software that gets written, how much do you think- because I think that's a key distinction. There are certain software that interfaces with other software. The connections to other software, there's a very unique set of problems when dealing with that. But there is also a different class of software where it's interfacing on one side with other software but on the other, it's interfacing with human beings. What proportion of software they get? How much of it is actually interfacing with human beings? How much is actually interfacing with other software? JEFFREY: I think the most software developers underestimate how much other human beings are interfacing with other software. Even if it is a simple HTTP protocol Rust type interface, it's still being used by human beings even if it's human beings writing code. CHARLES: That's true. There's definitely like a whole metalevel and that's like -- ROBERT: Like the developer experience versus user experience? CHARLES: Yeah but it's true. There's probably very few pieces of software where the ergonomics aren't important to actual people. That's true. There's definitely like a [inaudible]. JEFFREY: What you're getting at is that there's typically much more code underneath the surface than there is that actually is visible to a human. ROBERT: That makes it all work. JEFFREY: Exactly. CHARLES: Yeah. The part that's lying just beneath the surface is an ocean in of itself, I would say. It certainly seems like because that's the ocean we swim in it, seems vast at times. JEFFREY: Where does that UI stop? At what point does it stop becoming UI engineering and becomes more like API side of your server side? Obviously, you can say like, "Right when you start talking to an API. That's the end of it." But all of the code that goes on the frontend, the people refer to this as the frontend of the backend. All that code that is just there to get data and then massage that data so then somebody can go and present it to the user is at all UI at development. It's gotten a lot murkier in the last three years. CHARLES: Yeah it's a lot murkier because the thing is the way we structure our server technologies are changing so that they can support various interactions. For example, one thing that springs to mind is treating the data that we have inside of our client and I'm thinking of a browser tab, for example. Just one browser tab, one browsing session, you load data from the server and you write data back to the server but what we're seeing is that degrades the UI experience if you treat it just like that. I feel like the general trend is to treat the data that's housed in that browser tab is merely one partial replica of some distributed data set that's being replicated across a whole bunch of nodes where your storage nodes are one of those things, the node that other people are might be using in participating as application or other nodes and then that one browser tab in itself is a node in this distributed database. But that's crazy because that's what you would consider a classical server work. But to get around the fact that application state is updating and changing rapidly and trying to keep up with it in a non-buggy fashion can be really, really difficult. I know the waters are getting a lot murkier but I think that example does touch on something that I would say is kind of classical UI problem. You have some sort of stateful something. You have some sort of long running process that's just sitting there that's kind of moving with the user because the users holding a lot of state in their head as they're interacting with your application. A lot of the context is stored in their head so the UI needs to be able to dance with them and be in lock step with them and kind of mirror the context and their understanding of where they are in the system so it's usually very stateful. At least the ones that we've been working on over the past years. JEFFREY: I think another core principle of UI engineering is responding to events in a way that just doesn't happen as much in classical server engineering. You have to respond to a message or some kind of request every once in a while but the level of responding to changes in state and changes in how the human is interacting with the interface is a whole another level in UI engineering. CHARLES: It is really is a dance, in the sense that as the human moves their virtual hand throughout your application, you have to be tracking that state at all times. JEFFREY: It's like it really shines itself there. If you compare a state machine for the server side and then a state machine for the client side because we've built both and the client side is way more complex because there's just way more that's going on, impossible things that can happen. With an API, it's like almost crud in a way with an API and then on the UI side, we have to take a lot of user interaction and boil it down in order to persist that on the API side. There's more for us to take care of, on the UI side in track and it just more complex. CHARLES: That's true. I think there might be some people who work primarily on the server. They're being like, "Wait a minute." [Laughter] JEFFREY: At least on the server side, you have one runtime that you can deal with and for frontend developer, there's just so much to contain. CHARLES: Yeah so in terms of tracking that state, certainly one of the things that I think is also unique problem for user interface is that you have to constantly be taking in those events, that you're talking about, like you're constantly having to update your state so that you keep in lockstep with where the users at. That can also be part of that information that you'd be reacting to where other users are at, besides just that user so that they can absorb that context. Yes, so reacting to where they are but then also radiating information constantly to the user so that they're both inputting at extremely high rate but they're also receiving information. Like you said, it got to be extraordinarily high bandwidth so that state that you're tracking has to be represented accurately at any given time. I think, basically the core driving principle of NBC is you have this model and that represents your state and you have these events which then update that state and then you somehow managed to instantaneously reflect that state change to the user in that tight loop. I feel like that pattern, despite people tried to bastardize it, or maybe not bastardize it but adapt it for their particular use case but it has been persistent. It's like a cockroach, man. Always live and never die. I feel people are trying to assign new acronyms and new names for it but the fundamental pattern has proven to be effective over and over again. I would say that if you're using NBC, if you're building like a system with React or you're building a system with Java Swing, like we did back in the day, you're using the same basic pattern even though the mechanics and the tools are completely different but you have this abstract representation, this model of either your state or your interactions. You can model a swipe or you can model a mouse move then you're basically reflecting that to the user and then taking updates to the model. JEFFREY: What's changed about the tooling, say back when Frontside was primarily on Java? We're like, "That was the tool." It kind of a blunt instrument for some of the UI challenges we have now. What was working well in that arena and where we come [inaudible]? CHARLES: I think there's just so many exciting things that have happened since then. Especially in the past, like three years because Swing which was the Java UI framework which was fabulous, just absolutely magnificent, was at its core an observation-based system. You had a lot of observers. You can basically have these models and your view was a separate object but then you had your model. This was definitely a watershed moment. At least for me, from inside your code, you never actually updated the view. You never said I want to paint this thing. You never said I want to change this componentry. Only thing you did in reaction to events was set properties on your model. It was a mutable model so you basically had your state which is like a chalkboard but your code, the key thing there was you could paint things in Swing, you could render things but you never actually did. You had your components that always render off of a model and you never call those imperative like draw this line here, draw this line there. You always update your model and the view reacted to it. That's fundamental and that's something that we still see today but we were using mutable models. The models themselves just got overwritten every single time. The other thing is they were observer-based so you would observe this model, you observe properties on this model and when those properties changed, you're view would rerender essentially but I guess the paradigm was the view pulled data off the models and I feel like in the last three years, really with the advent of React, I think really popularized this pattern now and everybody else is adopting this. ROBERT: The flow architecture? CHARLES: Well, yeah. It's like a push model. Instead of having the view pull from the model but via observation, you have your controller essentially push a new model onto the view. But I still think the basic components were there, where there's two things that I think have change. One is this push model -- the pull view observation versus push from an event like event generates a new model and pushes on it and then the second thing which is probably coupled to it which is the idea of having your model be immutable. The magic of that is that you essentially unbraiding time from your model so that you the programmer are managing time instead of the CPU clock, which sounds weird to say. See, you guys are like, "What the hell are you talking about?" But that's fundamentally when you have something that's mutable, you're saying this is one object. It's the same identity. ROBERT: That you've changed over time. CHARLES: That you change over time but you're writing it to it where is when you use a different model each time, you the programmer is essentially saying, "This is the same object. You are managing identity of the objects not the computer. Does that make sense? You're saying here are a bunch of states but the identity of this thing is like an abstract concept and I'm saying all the states represent the same object or same entity just over time. What you're doing is you're essentially decoupling time from your model and that's I think a key innovation because then it gives you control over time so you can actually represent time, however you want to and that allows you to do things like history and what's the time travel debugging, which I think is very different than undo/redo. I think people conflate those unnecessarily but it allows you to do undo/redo history type stuff because the timeline is totally orthogonal now. Whereas, if you have immutable state, it's like, "What this looks like if that time is absolutely coupled to right now?" I think those are key innovations but I think that the fundamental pattern is still there. You have models, you have views and you have control code that manipulates. I think the key thing that's happened over the last three years that have switch between having the views pull the model versus having the controller push on there. For example, using essentially what amounts to an observable interface. I think observables as in ES7 observables are one of the coolest innovations to come around because it gives you that convenience of having a single reference point but access to all of the states that then you can react to it. Anyway, that's a long winded thing of saying that I feel those core entities are still there, where you have this and then everybody who listens to me long enough will probably get tired of me saying this but I think that the primary one is the model, like understanding that. ROBERT: Because your UI should be eventually derived from the data that you get? CHARLES: Right. ROBERT: And it should be because it is a source of truth. CHARLES: Right and everything else can flow from that. Understanding how you're going to represent it, whether we are representing it as speech, whether you're representing it as touch, whether you're representing visually, whether you're representing it with your ears, to understanding what is it. There's still one thing that can be projected into a bunch of different media so getting at that thing is the thing. ROBERT: That was a long, winded way of saying of what is UI engineering and what is unique about it but why would you want to plant your business in UI engineering? CHARLES: Right. That's a great question. Why is it a good business to be in and why is it a good feel to study? I think I'll quote one of my favorite video games from my youth, which was Mortal Kombat 2. At the very beginning, they would say, "There is no knowledge that is not power." That was the big quote in Mortal Kombat 2 and I'm pretty sure that the knowledge of Kung Lao, I think, being able to throw his hat by hitting some key sequence of joystick and buttons, I don't think that's knowledge that's going to give me power. Sure enough that I don't think that knowing those key sequences really had much power in them but I still think it's a good saying. [Laughter] CHARLES: I was always wondering what that particular quote at the Mortal Kombat programmers really like. But anyway I think that it's good to understand it as a discipline, either here or in your organization because it gives you staying power, because you understand that deep structure. For example, if the winds change and the tools get updated -- JEFFREY: As they inevitably will. CHARLES: As they inevitably will, right. We mentioned Java, we mentioned C, we mentioned Rails, Ember, React like back down knock out, all those things, those are transitory and they're ephemeral. But that knowledge isn't and it's a rock that you can cling onto as the change of storm over the landscape of the development community. If you understand that, your transition to the next thing will be a lot easier. I think that's the business value, at least from my perspective, the ability to [inaudible] those storms and be like, "Oh, this is the same thing. It's got different clothes. It's got some nice updated patterns but I understand fundamentally what's going on. I can work with it. I can find out what the things are." They talk about their linguistic intelligence so your second language that you learn is very hard, your third one is a little bit easier but then you have these people who speak nine languages. Once they learn and go through language four or five, it's actually a lot easier for them to pick up language and you're like, "How can they do that?" And the answer is that they understand the deep structure of language that's basically baked into our DNA. Every human being is born with it -- ROBERT: There are patterns. CHARLES: Yeah, there are patterns and they know them and they understand them and they just absolutely have intimate knowledge of them so the stuff that gets put on the window dressing, they can see beneath that like X-ray vision and they can pick that up and work with it and run with it and it becomes mostly just an exercise in learning vocabulary. ROBERT: And for as long as users will be interacting with the things that we build, there will always be UI engineering. There will always be somebody out there needing work. CHARLES: Exactly. Do you want people who are fatigued by the JavaScript churn, so to speak? There are a lot of different strategies to dealing with JavaScript fatigue but a great one that I know of is to understand the core principles that are going on beneath the tools and realize that the tools really are the icing on the cake, the tip of iceberg, so to speak. JEFFREY: This kind of brings us back around full circle to what originally started this conversation on our office is how The Frontside brands itself. We have, for the past few years, been doing almost exclusively Ember work. We love Ember as a tool and a framework that's done really good things for us and we've enjoyed working with. But we want to not marry ourselves so tightly to a tool like that because we believe in the problems and the concepts more than the tools themselves. CHARLES: Exactly. We want to be married to the problems not the tools because what we're really looking for is for when someone has a special interaction that they want to see made real when they have a special product that they've maybe spent a lot of time and energy and thought and maybe money on crafting the user experience. We want those people to think of us because they're saying, "Now it's time to build the UI." Before there is even an inkling of what the toolset will be, what the implementation strategy will be, we want to be at the first and foremost at people's minds. Regardless of how it's going to pan out, we know that these folks can get it done. They are going to implement this thing that is going to be as good or not even as better than the way that we envisioned it. I think that's important so we've been trying to transition towards that type of message. ROBERT: We talked about how UI has these things that are long-lived. One of the really long-lived things and I think is going to be here for a long time is JavaScript. We do JavaScript very well. Everything we've been doing since, I would say what? Mid-2015? CHARLES: Yeah. ROBERT: -- Has been to abstract away from the UI framework and just base JavaScript libraries because that can be portable to anywhere. That can go from the server, on Node, all the way to any UI framework that you want and even native now with native script in Angular or React Native. CHARLES: It's true. It's amazing how portable JavaScript is. ROBERT: We are betting on JavaScript UI and I have gotten a couple of tweets from people because they know that we're an Ember shop really and when we talk about React, they're like, "Are you just a React shop?" I was like, "No. That's a big distinction when [inaudible]." JEFFREY: -- UI shop? ROBERT: Yeah. We want to be able to pick the framework or library that we see fit for your project and it doesn't matter if it's an Ember or Angular or Vue or insert next year's framework because it's going to be written in JavaScript or a superset of JavaScript like TypeScript. CHARLES: Exactly. We love all of those frameworks and we think that they have their individual tradeoffs but we have to think about what those tradeoffs are. Also, think about how can we share as for example, this Ember app. What can be shared between an Ember app and React Native app and you might think, it's not much. But my guess is probably a lot, like most of it could be. That's kind of a radical idea but at the same time, it's a very simple one and seeing that pan out, I feel we've actually been accomplishing that and this isn't just something that's like smoke and mirrors, this is what we've been doing and the strategy has been working and it's been paying off. ROBERT: We do UI well and it doesn't matter what way we write the code, at the end of the day we're still producing this interface that users interact with and that's our bread and butter. That's what we do really well. The framework is just the thing that we're using as the flavor of the month, year or whatever it might be, to get the job done but we are betting on UI and we do it well. CHARLES: I think, it's actually a testament to a framework as how well it can adapt to plain JavaScript. The way that we model these interactions really is with simple immutable POJOs with well-known transitions to new states. That's it. It's like secret magic sauce that is so, so simple. ROBERT: Remember earlier when I mentioned the state machines between the server side and the client side? CHARLES: Yeah. ROBERT: We modeled these as immutable state machines, CHARLES: Yeah and they're so incredibly portable. I think, Ember back in mid-2015 wasn't so great at hosting these simple objects. Now in the early part of 2017, it actually is. It's very easy to write those things. If you're using Redux or React, we're kind of more friendly to POJOs from the get-go but I would say this. I think there's clear value in not putting logic inside your reducers. Like what is a reducer for? I actually think that you are better decoupling your JavaScript from a framework like Redux, in the same way that we reap huge benefits from decoupling all of our state transition logic from any framework artifact inside of Ember. It meant that we were ready to drop in those interaction models into React. It was like boom! There was very low friction. It was extremely low friction. ROBERT: Because these interactions are in a base JavaScript library and the UI library is what's driving those interactions. You're actually talking to this underlying JavaScript library that we wrote and abstracted away from the UI framework. But we still need that UI framework that whatever you inserted into to play those actions. CHARLES: Yeah, exactly. You need Redux to manage the current state to basically manage your concept of time but maybe think twice about coupling your actual interaction model to a framework because it's going to be portable. This is a lot of pain that people had surrounding like active record. I remember for people in the Ruby, there was kind of 'aha' moment in the Rails communities like, "Wait, we literally don't have to dump everything into active records. There was this, I think mass awakening, followed by this mass sign of relief. Once kind of people realized that everything you do doesn't have actually have to fit into a one of these kind of framework archetype objects, whether it'd be a reducer or a component or a route or whatever. If you have something that's truly separate, that's portable like make it portable. Once people kind of started treating active record as just, "I want to persist on something. This is what use," and things became a lot easier. ROBERT: Because you're separating yourself from Rails and using Ruby, the language. CHARLES: Right. I think, you said it perfect, Rob. It's not to disparage any tools. The tools are absolutely critical. In fact, the tools are most of the code. The amount of code, for example an Ember application provides to you is just massive and it's all very high value. Some people would debate that but the point is either all things that you're going to need to do but they're not things that are unique to the interactions that you're trying to provide. I would actually like to see a version of Ember Data that doesn't depend on the Ember object model. I think being able to really separate that. It's pretty amazing library and if it saves an enormous amount of time, it would be fantastic. ROBERT: But only Ember can ever use it. CHARLES: But only Ember can ever use it but it is definitely a common problem and something that we miss like the 90% use cases that it covers. It's something that we miss when we're working in other frameworks. I actually think there is one more thing to touch on and this is really what separates, I think what we do from a lot of maybe agencies that are heavy on the design side but not super heavy on the UI side. I think is a major differentiator between the UX and the UI. Obviously, UI is very implementation-centric as opposed to vision-centric. I think, a good comparison is architect making blueprints versus the builder that actually has to construct the skyscraper. CHARLES: Make it real? ROBERT: Yeah, it has to make it real and that's what we do is make it real and There is an engineering component to it. While we have a very heavy focus on design and beauty and elegance and smooth interaction with the user and we study a lot of key user experience principles, there is a component to UI engineering which supersedes framework, supersedes design -- sorry, not supersedes but it's present, it's ubiquitous and that is bringing to bear all of the industry best practices around making quality, robust software. I think that's an important thing to point out. It kind of goes without saying but I think it's important to mention is if you want to have all those good things and if you want to have good models, in views, in your controller, you want to work with a bunch of different frameworks. It's all for not. If you're not using all of those things that keep quality software quality, if that makes sense -- having continuous integration, having test suites, having continuous deployment and thinking about all the operations surrounding the software. Ultimately, the UI is software just like any other system and something that we could go off and talk about for ages is how to test UI software because they are. When you've got these big stateful applications, they have their own unique testing. ROBERT: But there's a lot of engineering problems here that are to be solved. CHARLES: Right and I think it's important to be every bit as that. Anyway, I think that's about it on these subjects. It's just been kind of bouncing around the walls here and we're kind of think like, "Who are we? What is it that we do? What would you say that you do here?" JEFFREY: We have access [inaudible]. [Laughter] JEFFREY: That's what we do here. CHARLES: Right. UI engineering, man. It's taking those UX dreams, right? And it's bringing them to life. It's making sure that they're scalable, that they're maintainable, that they're testable, that all of those things can become real, regardless of the tool. ROBERT: And live long. CHARLES: And live long. ROBERT: That's the hard part. We can throw something together and match the conf. CHARLES: Right. ROBERT: But, "Will it continue to match the conf in three months?" is the question. CHARLES: Hopefully so, we think so. I think we hit that mark. ROBERT: Absolutely. It's the engineering, that's the tough part. That's the hardest part. Making it happen and making it sustainable. CHARLES: All right. Thanks everybody for listening. We are going to be back next week. We've got some pretty neat folks stopping by so be sure to check it out.
Marcy Sutton: @marcysutton | marcysutton.com | Deque Systems Show Notes: 01:07 - Deque Systems 01:54 - Accessibility Tool Integration and Testing 05:26 - Configuration and Success Criteria 07:04 - What is accessibility? WCAG 09:22 - Spurring Adoption of Accessibility 12:09 - The Accessibility Matrix 16:56 - Accessibility-First Development 18:12 - WCAG and ARIA Roles 24:57 - Test Automation vs Human Interaction 28:56 - Empathy Building 30:45 - Porting to the Web 35:57 - Accessibility in Single-page Apps and Focus Management Resources: axe-core aXe aXe Developer Tools WCAG (Web Content Accessibility Guidelines) Web Accessibility for Designers WAI-ARIA Authoring Practices 1.1 First rule of ARIA use Access Works: Usability and Accessibility Training Marcy Sutton: Notes On Client-Rendered Accessibility a11y on Slack Transcript: CHARLES: Hello everybody. Welcome to The Frontside Podcast Episode 61. My name is Charles Lowell. I'm a developer here at The Frontside. With me also is Mr Robert De Luca, a developer at The Frontside and today we have with us, Marcy Sutton who is going to be talking with us a little bit about accessibility, both in the large and the small. Welcome, Marcy. MARCY: Good morning, everyone. Happy to be here. CHARLES: I know, I understand you're actually calling us from the parking lot of a ski area. MARCY: I am at the legendary Mount Baker ski area outside of Bellingham, Washington where we have the winter that is just going on and on and on and getting after it on the last few days of my birthday vacation. ROBERT: Oh, wait. Happy birthday. CHARLES: Yeah, happy birthday. ROBERT: Happy belated or happy birthday. MARCY: Yeah, it was Sunday so still on that shiny birthday week. CHARLES: Well, thank you for getting with us on your vacation and on your birthday but doing a little bit of work, you actually work at Deque Labs. What is it that you guys do over there and what's your particular area of interest and work there? MARCY: Deque is an accessibility company. We have people who work on products and services for accessibility. We help people avoid lawsuits and make their websites and mobile apps more accessible to people with disabilities. My slice of that work is on the product team, where I work on browser extensions, APIs for developers. Basically to make it so you don't have to write every single accessibility tool or test yourself. You can pull in these APIs and get some of that experience that Deque has built up for years and years and years, which was part of the reason I went to work there was to learn from them. We make tools that make it easy for you to make use of that knowledge in your applications. ROBERT: That's awesome. It's like a base JavaScript library that can be ported anywhere, like to browser extensions. I know we use it in Ember accessibility testing. That's really cool. That's where I've gone for the way I write JavaScript. It's in a base library so everybody can use it and it's even more awesome that it's testing and like wrapping tooling around accessibility because I know a lot of developer-minded people want to see like a failed built. CHARLES: Yes, what does that experience look like? I mean, coming from someone who's never even heard of these tools, how would I integrate them into my project and what would change about my workflow? What information would it surface? MARCY: The best place and the reason I work on these products is that I saw projects go out the door broken a lot of times, when working in agencies or maybe testing isn't part of your methodology. Personally in my career, I just knew there had to be a better way. I got into software testing and the more I learned about it, the more I thought that it was sustainable, you could pull in other APIs to help you write better tests. I went to work on axe-core, which is the JavaScript library that we've talked about a second ago. That really is bottling up all of these accessibility tests that you can automate some of the accessibility checks for things like if your HTML markup is in a good state and you're using attributes correctly. Basically, saving you from having to write all of those little microtests, some of which can be sort of complicated. It's all about getting test coverage for the automated things that we can actually test for. CHARLES: You described a pretty wide-ranging coverage. How do you go about actually implementing that into your CI process? Do you just install the axe-core? Do you have to load up your browser and then pointed it out? What does that look like? MARCY: Ideally, you would already have a test suite and you could just pull in the test harness. There's all different versions of aXe. There's versions in JavaScript and in Node. The core thing that you need to test is get your app running in a browser, whether it's a headless browser or could be a mounted browser but we need those actual DOM browser APIs to check things like color contrast. We need to be sort of coupled to the DOMs so that we can run our full set of tests, which is a distinction from, say some shallow rendering that you might be doing in React testing or something like that. For accessibility tests, we need an actual DOM so you could get axe-core on npm and then pull it into your project and then you basically either require or import it, depending on what your stack looks like in JavaScript. Then you have access to all of these tests. It's pretty useful since our ecosystem has evolved to cover things like npm. I've found that it works pretty well. ROBERT: That is pretty neat. You require it into your test and then you visit a page that's fully rendered and then you do aXe check, like you call a method that runs all these checks? MARCY: Exactly. You would call axe.run and then you configure it to run, either specific tests or just one test. One of the tricks that has been helpful to know is that if you disable the color contrast rule, you don't need quite as many of the DOM APIs so it will run faster in things like JSDOM, which doesn't implement the entire browser APIs. But you could call axe.run, either in your unit tests or more likely it would be in your integration tests because you'd already have a browser instance, either through Selenium-WebDriver or karma-chrome-launcher or something like that. Then you basically call axe.run, passing a callback function and then it will return to you at set of JSON results and then you can do things with those. ROBERT: When you call run, can you pass options of what you want to check? Can you filter out things that you know might -- because I imagine like if you put this into an existing app that's been going for a while, I imagine you're going to get a bunch of fails and it might be overwhelming. Is there a way to peel a back like an onion and start working at it that way? MARCY: Yes. You can get pretty specific with our API. The GitHub for axe-core has our entire API configuration. You can get pretty specific. You can filter by tags. I imagined we're going to talk a little bit more about what WCAG is but there's a set of standards that you can break accessibility down into things that you can actually assert that they are either accessible or not. There's all different kinds of what we call success criteria. All of our rules are mapped to these actual guidelines and standards because that means that our tests are helping you solve things that are actually helpful so you could filter by the different levels. Maybe you want to configure it with custom rules. We have some additional products for that. You can get pretty specific with what you want it to run. ROBERT: It's extensible too so you can add your own stuff. MARCY: You can and we do a lot of work with some of our clients to actually help them write custom roles so that's a service that we offer. But the API is pretty configurable on the JavaScript side so you can do quite a bit of configuring on your own as well, which is cool. ROBERT: That is pretty awesome. You alluded to WCAG, I guess now we know how you can integrate a testing library into your JavaScript apps, let's take a step back a little bit and what exactly is accessibility and then you can start explaining WCAG because WCAG is a very big document that tells you how to go and be accessible. CHARLES: I assume WCAG is some acronym? MARCY: It is. Peeling that back a little bit to what is accessibility. I'm more on the digital side. There is physical accessibility as well for spaces. But when we're talking about digital accessibility, we're talking about making apps and websites that work for people with a broad range of abilities. Say, you had color blindness or a low vision or you're fully blind, you would need to be able to zoom in, you need high-contrast colors, you might use a screen reader if you're blind. But then there's other categories. People might actually fall into more than one category including motor disabilities, where maybe you can't use a mouse and you have to use a keyboard only or a keyboard with one button, which is how we think about a switch control --that's another device. You might be deaf or hard of hearing and need transcripts or close captions so any audio or video content needs an alternative of some kind. Then there's cognitive disabilities where people have learning disabilities. Maybe the language used on a website is too vague or too marketing copy speak and we need to simplify, people with traumatic brain injury like Stephen Hawking has ALS. I discovered at some point in my career that I could actually make the web a better place by supporting all different kinds of people. That's really what it's about for me is doing good craftsmanship and making sure that you're actually making things as accessible as you can. The WCAG thing that we mentioned, it stands for Web Content Accessibility Guidelines. It's just that. It's a set of guidelines, sort of a map to help you get there. You have to actually interpret those guidelines and put in the work to do it. The guideline is just a guideline but it gives us a really good roadmap of how to implement all of these different areas of accessibility. CHARLES: I actually had a question and this is a little bit harkening back to the discussion about the axe-core but also kind of straddling. How do you spur adoption, both the technology and the value inside of your development team? You know, we definitely make our web apps as accessible as we can because we have Rob on the team. But for teams that don't have Rob, how do you spurred option? How do you pitch it to your team and to your management structure? Like testing. Testing used to be controversial. I think in some pockets, it still is but it was something that you had to pitch or agile methodologies was something that you had to pitch. Now it's kind of accepted. It's a core-value of development, I think. I hope. MARCY: Definitely more so. I agree. CHARLES: Do you see a future where making applications accessible is just a tenet of development in the modern era and how do we get to that point? How do we pitch our teams to adopt that value? MARCY: Part of what I'm trying to do is meet developers where they're at and make tools that make it really easy and free to integrate things so it doesn't cost you anything to npm install a library and pull it in your project or to use a free browser extension. What we're trying to do is really help developers get there by lowering the barriers, just kind of a funny way to put it because that's what we're doing with accessibility is removing barriers for people that get access to things. I'm pretty optimistic about it. We talked a lot in the accessibility world about education is really needed because often, it's just that people don't know about it. I've made it my mission to spread the word as much as possible by doing talks and blog posts and just trying to get as many people on board as possible, instead of making them feel bad about it like, "Oh, you don't know about this? You're terrible." ROBERT: Oh man, you're speaking to me. MARCY: "-- You can do this." I try to bring people along and make them feel welcome because it's not really a fun experience to be like, "Oh you're bad because you didn't do this. You don't think about this thing." That's what I try to do. ROBERT: One of my first experiences in accessibility was like somebody giving me that moral argument like, "You're ruining people's lives. They can't do things on their computer." I just remember the response I had and it wasn't that, "Oh, you're right. I should go make this accessible. It was more of like I had a flight or fight response. I start to justify the reasons I didn't do and that wasn't a good experience so the way you put it, like meet the developers where they're at, I love that because that's how I've been operating too. I think accessibility is just another engineering problem and it can be an engineering problem that would be fun to solve. The accessibility matrix gets really hard and hairy as you get into it like -- CHARLES: Oops! Jargon alert! What is the accessibility matrix? Does the accessibility matrix has Neo? ROBERT: The different AT combos and since my experience stems from screen readers -- MARCY: Assistive technologies? ROBERT: Yeah, assistive technologies -- I'm doing a poor job here -- Basically, you have three levels that you work with here. It's the operating system, the type of assistive technology and if we're talking about the web, it's the browser. You could have like the matrix, the beaten path is MacOS, VoiceOver and Safari. That's going to be your matrix. Then on Windows, it could be Windows JAWS and Internet Explorer or Windows NVDA, which is another screen reader on Windows. JAWS is also a screen reader. The browser for NVDA would be Firefox. Then it can just fork in any of those different combinations that you could possibly imagine that makes it hard to debug for. But that's why I think this is a cool programming problem is because we can build awesome tools to help us do this and test for it like aXe. MARCY: Yeah. I would also argue that it's almost even more of a design problem. It's part of the additional challenges that we have to get our design friends and colleagues on board as well because the more that they are thinking about it before they handed off to us, the less we're going to be caught in these situations where we have to make it work in one browser and assistive technology but then it's broken somewhere else because we're trying to use really experimental APIs or we're just trying to do things differently for the mouse versus the keyboard. I can tell you that could be really difficult. The more we're thinking about making things straightforward and intuitive from the design side, not to say the easier job is going to be but the more successful, I think we can be as a team because it's more than just developments responsibility. There's good resources for designers as well, like a web accessibility for designers. If you just Google that, there's a great checklist from WebAIM. I think it's helpful to make it inclusive to people that we work with, not just in the development side because we really want them to set us up for success or else were really just fixing problems that not at their core. You know what I mean? ROBERT: Yeah, as they come down the pipe, we're kind of dealing with them instead of getting ahead of it. CHARLES: That reminds me actually of an experience that I had, a pair programming with Rob, probably about a year ago as we were making an interaction model for a select box. This was for a custom client. We actually stripped it away and we're like, "Let's just focus on what is the state machine behind this thing," so we drew it out on the board and it turned out that we were really just capturing the interaction apart from any rendering so we had a very strong model. With each state's transition, we were able to basically radiate that information with a screen reader in this case. But it was actually very trivial to do because we've actually forgotten about the DOM, forgotten about the fact that we were actually chasing a visual interaction and like I said, what is the actual user interaction? What is the information coming in and coming out? It turned out once we kind of flush that out and have developed that, hanging the interface on that skeleton was very easy and we could do it in multiple media. It feels like a similar concept where if your designers are very upfront about really exploring the information architecture of an application then being able to represent that information architecture in multiple forms becomes much easier because the joints and beams are very, very clear and they aren't bound to a particular form of representation. MARCY: Yeah, I think it a way that's definitely true. One challenge I would issue for this part of prototyping would be to consider all of the user inputs. Make sure that you're considering a keyboard user hitting an escape key to close that select or maybe they're using a screen reader on a touch device and like the single finger swipe, it's already allocated when that screen reader is running so if you have an interface that was only swipe left or right and there were no other affordances like buttons that you could actually activate, that would be an unusable interface to a mobile screen reader user. What really helps to make that information architecture stand up or hold out when you're developing it, like stay true to your vision through the process is making sure that you're considering all of those user inputs. Sometimes, developers aren't thinking about keyboard user so they're not thinking about focus styles, really trying to activate it another way. I do think that's a helpful exercise. ROBERT: Yeah, and to be fair at Frontend developers, we already have a lot to think about. It's just a lot to juggle so I can understand that's why we have tools like aXe. But what Charles is talking about, I think is actually kind of neat is we were experimenting with accessibility-first development so the people do TDD -- test driven development -- and I was trying to see if we could build something. I wanted to see if what we're writing would yield better software if we did it with an accessibility in mind from the outset. I think that's true. It was a more accessible typeahead. It was better, more well-defined user experience around the typeahead and it was because we thought about accessibility and all of the different edge cases. We really boil it down to the core problem. CHARLES: Right. We were driving it first with keys and nonstandard interaction methods. It meant that we actually got more clear interaction model lying underneath. It was decoupled from the actions that drove it completely because we had to support too from the get go. ROBERT: I thought that was neat. CHARLES: Yeah that was a fun exercise. You know, we should have blog about that because I think that actually results in better software. ROBERT: Yeah, I had a conference talk brewing in there somewhere. Just never got around to it. Talking about the web accessibility guidelines. There's different levels to it. Now, you have an A, AA, and AAA. What do those mean and where does that play into ARIA roles and stuff? MARCY: There's WCAG 2.0 and actually 2.1 is an update that they're working on right now but WCAG 2.0 is -- ROBERT: Oh, yeah. I saw that. MARCY: Yeah, there's some new stuff coming out. It's mainly for low-vision users and mobile touch things. But the WCAG 2.0 is the blessed standard that we're working with right now and the levels are different conformance levels. There's different things that you can achieve with A, AA, or AAA. Most people go for AA. AAA is pretty restrictive in what you can do and if you make it support WCAG 2.0 AA, it doesn't necessarily mean it's going to be intuitive to use. You could make it technically conformant but it won't necessarily be that beautiful or accessible. There's a bit of a dance that we have to do around that to meet these guidelines but do them in an intentional way so that we're actually making something usable. I think that goes back to that idea of craftsmanship and caring about your user to know if this actually going to work for them. There's a number of success criteria in WCAG that are broken up into different categories. There's perceivable, operable, understandable and robust. Within each of those, there's all kinds of different checkpoints that you can look at to inform how do I make this keyboard accessible. There's all kinds of really helpful documentation. That's the WCAG guidelines and within each of those, there are a number of different ways that you can code something. As I'm sure you know, there are infinite ways to code the same thing, pretty much and part of what that cover is techniques for making things accessible. They'll tell you all about Native HTML and what tools you can use within that standard. Then there's this other standard called WAI-ARIA and that's the Web Accessibility Initiative – Accessible Rich Internet Applications. That was originally created back in the day when we didn't have as many browser APIs and we didn't have great ways to expose accessibility information to screen readers. They made this API in browsers that implemented that you can actually bolt on some of the same information that you get from HTML. It's helpful if you're writing as VG or XML, where you just don't have those built in semantics so we have things ARIA role states and properties. You may have seen things like 'role="button"' or 'role="main"' or 'role="search"'. You might see that somewhere and that is just exposing programmatically bolting on a role to any element. You could put on 'div role="button"' and there's a little more that goes into that to make it an accessible button. Anytime we mentioned -- ROBERT: The tab index. MARCY: Yeah, the tab index. You have to make sure you have a keyboard event but that would be a programmatic way to create a button element. You should always start with the native button element because you get all that stuff for free but ARIA gives us an API to actually implement accessibility information. You'll see those techniques come up a lot in WCAG of how you can accomplish the same thing multiple ways. Those are some of the things that we test for in our animated tests in aXe. We check to make sure that you've only use roles that are actual roles because there is a set standard of them. We check to make sure that all of the ARIA values that you might use are actually allowed for that. Sometimes, if you're using 'role="list"' for whatever reason, you can't use a real list. It is possible to create a list with ARIA but if you had the wrong child role or something, that's a pretty easy thing that we can flag with aXe so we're sort of saving you from yourself. It helps me sometimes when I get a role wrong because we're human and we do make mistakes. There's a lot of things to remember so that's pretty key technique that aXe will help you with. That's making sure that your ARIAs used correctly because it is pretty easy -- ROBERT: That's really nice. MARCY: -- to get it wrong, to be honest. ROBERT: Yes. I've definitely done that. Being through the spec document is not the most fun. Trying to read the standards language is a little bit complicated so having a tool like aXe is really helpful for me to pick my way through it like, "aXe will tell me that this is wrong," so it narrows the problem set down for me where I can go and look at the standard and kind of tunnel vision in on that one, rather than get overwhelmed looking at that whole standard documents like there's so much here. MARCY: Yes, there is. One thing that might help with the is the initiative that people are working on called the ARIA Practices Guide, the ARIA Authoring Practices and it sort of breaks down these techniques into what is the keyboard navigation model for that component or it will break it into known patterns. This is really helpful also for designers to know what are some known patterns and how can I implement accessibly. They can really help you jumpstart to using those patterns with this more easily digestible information to tell you how to do it correctly. That has come up in the last few years that I found really useful. ROBERT: That's awesome. I think I've seen this. Is it where they tell you like, "If you're going to reimplement a checkbox, here's how you would do with ARIA?" MARCY: Exactly. I've dropped a link in the chat so we'll expose that in the show notes, I'm sure. There's more resources out there now that are really helpful. There's another one called ARIA in HTML and that one is also from the W3C and it's a note on using ARIA and HTML. That one I found to be very useful as well because they tell you this first, second, third, fourth, and fifth rules of ARIA use. The first rule of ARIA use is if you can use a Native HTML element or attribute, you should absolutely use the built-in one first. That's a big -- ROBERT: Yeah, let's stop reinventing. MARCY: Yeah, you know it's tempting because you can create these custom elements and try to bolt on ARIA but the reality is that if you're trying to make it really backwards compatible, it's just so much easier to support the native things. There is an assisted technology called Dragon NaturallySpeaking, that's a dictation method and they didn't support ARIA until 2014 so you can easily imagine some of your user base with an older assistive technology. That might be completely broken for them so that's why we really push using the native things first just because of the better support on every platform. CHARLES: I have a question about the test automation. We've been talking a lot about aXe in the way that you can do this. Did I get it right? Are my roles correct? And all these things. What's an example of something that you just can't test for in an automated fashion? It just requires human interaction just to perceive it. I mean, this would be right now, kind of in the visual sphere, the state of automation for testing like did I break the layout still requires a human. What are examples of that in terms of accessible interface where you just do the things that you have to be on the lookout for that you can't cover with automation right now? MARCY: I think context and content are some of the most difficult like writing good all text. That can be really challenging just because what makes a good alt for an image and that supposed to be a text alternative to say, "This is something useful," and Facebook has solved that by using artificial intelligence to dynamically guess what's in an image. A blind colleague of mine that works there has written about and he said he always felt left out when he would read his news feed and someone would be talking about their first love or some kind of vague status update. With this new feature, it could say, "Oh, this image that they're talking about their love is a pepperoni pizza," or something where -- [Laughter] MARCY: It's really missing the context so they've started to do automatic all text. For us doing accessibility checks, we try to keep our solution as light weight as possible and without false positives. We can check whether you have an all attribute missing like you don't even have the alt attribute at all which means that the file name would be read in the screen reader which is often terrible, depending on what your filenames are so we can check if that's missing but we can't really tell you what would make a better alt attribute, if you already have one. That's one is a bit difficult. There's another one that we're working on right now with color contrast where we can't really tell if you have a background image that's behind some text. If it has multiple pixel color values in it, even if we could read those colors, it gets really hard for us to say whether text meets color contrast when it's over an image for multiple reasons. That one's a bit tricky. I think there are some other examples throughout WCAG that we can only automate. Depending on which rule set you're using, we estimate between 30% to 40% of issues, we can actually catch with automated tests so there is quite a bit that we still need humans for. But however, I think some of these really basic ones that we can check to help you do those easy wins so that you're not getting messed up by using the attribute Aria-role when it's just role. Those kind of things. It's like we're helping you so you can save that time for those more complex task that might require a human. There's definitely no substitute for trying to use the keyboard to make sure that your app is usable from the keyboard. Test it with a screen reader, you can find people in the web accessibility Slack that might be willing to help you test it, if you're extra nice or maybe you can give them a gift card or something. There is an organization called Knowbility and they have this thing called Access Works where if you need to find a user with a disability to deduce a user testing for you because that's a great thing to do. It's very important. They can help you, as a business think up with someone who can test your app. I would definitely check out Access Works. That's really what's the missing piece. As a developer, I'm okay using a screen reader after doing accessibility for a few years but it's not my primary way of navigating so it's really helpful to have real users to test your app and that's a good way to find someone to actually test it. That sort of makes up the rest so you can get that really valuable feedback. ROBERT: I'm a firm believer in testing but also, I really do think a lot of accessibility work is just kind of empathy building and the way you do that is just sit down and actually use this assistive tech that these people will be using. In that way, you can understand as you're building it, how somebody might move their screen or cursor over the top of this and you can start to think about what the screen will read off and stuff like that. I think using a screen reader as a developer is powerful. But I agree, it will never reach the level like my mom that has been using a screen reader for seven years now. I'll never be able to use it as well as she does. It actually putting in the hands of people that do this day-to-day and live this. A far better idea and that goes beyond accessibility too. You want to user test all your apps anyway. MARCY: Yeah, exactly. I think that should be a big thing that we demand just from our organizations like how you were saying it was kind of controversial. I feel like user testing is another flavor of that where we have a bit of emotional tide of these things that we create and we want them to be perfect in the way that we have envisioned but not everyone interacts with things the same and it's really humbling to watch someone use something that you made and have it completely not get it at all. I think that's a really valuable experience. I've watched my mom or my dad or people try to use something that we assume is really intuitive and it's just not. We look at the web all day -- day-in and day-out being professionals and it's really helpful to show it to people who maybe aren't as fluent, aren't digital natives like that. CHARLES: We talked about actual user testing. We talked about the checking where you render your application and you run a set of checks. Do you have any experience with actually -- this is kind of an idea that just occurred to me, although we did a little bit of it when we were doing native applications -- using the accessible interfaces to actually drive your acceptance tests? Is that anything that you have experience with? Because it seems like on the face of it, you've got this assistive technology that surfaces the key levers of your application so is it a good idea to grab those levers from within your test case? Within your acceptance test to manipulate your application and thereby kind of front load your accessibility because in order to verify it, you must have those levers in place. MARCY: Yeah, from understanding your question correctly, you're wanting to just run your tests using accessibility features? CHARLES: Yes. For example, when we write our acceptance tests in our application, what we do is as part of setting them up, say we want to click here and I want to enter this text into this text box and I want to move this over here and that implies actually dispatching mouse events, keyboard events and then also being able to find the elements in the DOM that I want to dispatch those events on so we're kind of doing it in, I think we use CSS selectors to find them and then we use the jQuery event interface to actually create the events and send them to those elements. But it seems that part of ARIA roles or something else is like identifying the role that this element has in your application and basically saying, "For my test cases, I'm going to use these roles and I'm going to use these things and I'm going to use different access methods, keyboard mouse or whatever to manipulate my interface." Does that makes sense? MARCY: Yeah. ROBERT: I think this makes sense in the native world where in order to get the label, I think you have to use the accessibility label. CHARLES: They do that when you're functionally testing iOS apps so why not -- ROBERT: Does it port to the web, basically. CHARLES: Yeah, does that port to the web? MARCY: It does -- CHARLES: It's really long, way of saying that, I guess. Sorry you all. [Laughter] MARCY: No, and I wanted to clarify because I was wondering if you're talking about driving it with actual assistive technology, which we can't quite yet. We don't have any tools for that. But yes, you should -- ROBERT: We should explore that in Ember. MARCY: Yeah, we just don't have the hooks for that. Maybe Python and NVDAs, since it's open source, maybe AppleScripts. CHARLES: What would that look like to drive it with assistive technologies? ROBERT: We talked to some people at Apple with Ember accessibility team and if I remember correctly, we could only drive VoiceOver on MacOS with AppleScripts but there was no way to do it in any other way so you only could do it with VoiceOver on MacOS and that was still kind of murky. MARCY: Yeah, exactly. The idea would be, rather than just testing the browser, we would actually be able to run a simulator programmatically, to know is the screen reader actually exposing this information. Because a lot of it is there are things that get lost in translation, sometimes where we're following best practices and standards because we have this agreement that people who implement browsers and screen readers are going to follow those standards. It's definitely is not always smooth sailing with that. But there's sort of this disconnect between the browser testing and then actually firing it up in the screen reader and make sure it worked. We take that on faith a lot of time, which is getting back to your original question, why it's so valuable to have tests that use these interaction methods. Absolutely, either in your unit tests or even in your integration test, they can live in either place to have tests that assert and closes with the escape key or it operates with the enter key or whatever the user interaction should be, that we have tests that assert that because that way, if you leave your team or heaven forbid, you got hit by a bus or something, you have a test coverage that makes a contract of how this component should work and you have accessibility support, actually built into your test infrastructure. That is super valuable. At least we know that that part of it is there. We know we can drive it from the keyboard, which is how a lot of screen readers work. They operate on top of the keyboard so we can get really far just by having basic keyboard support. Then, if you pull in an API like axe-core, you can have it tell you if you were using ARIA wrong or something. It's sort of a combination of both where those feature tests in your actual project where you're writing something that it works with the escape key, those are custom tests for your application. I find that they're really valuable just to have in there, especially if you work on a component library or something reusable so that everybody who is contributing knows how this thing is supposed to work. I think that is really valuable. ROBERT: Absolutely. I want to talk about accessibility in single-page apps. The problem with accessibility in single-page apps is while using a screen reader, you click a link and to the screen reader user, all it says is the link was pressed. They don't actually know that the content has changed. But in Ember, we kind of solve this by focusing the outlet that has changed but in other frameworks, in your experience everywhere else, how do you combat this? What are the best ways of attacking this? CHARLES: Yeah, what are the problems that you encounter in single-page applications? MARCY: I've done quite a bit of research and blogging and conference talks on this. I'm working on the Angular team for a while. The issue with the single-page app is the page isn't being refreshed when you make a raving change or something happens dynamically. The user's focus is never refresh to the top of the page so they don't hear a title change or things like that. There's different techniques that you can employ to make that experience more accessible. The first and foremost tool to have in your toolbox is focus management so that you're programmatically sending the user's focus to this new content. Say, I have a sidebar with links in it and I click one of them, I can send focus to content wherever it loaded on the page. That way, they are both alerted to the new content because depending on where you send it. There's different techniques for this but often, we will send focus to the wrapping element so that everything will be read aloud and you can accomplish that by using tab index of -1 in your HTML. That will make this wrapper catch the focus, essentially but it won't add it to the tab order of the entire page. That's a technique that we used to shuffle focus around. I've also seen people use what's called an ARIA Live Region where you have this element somewhere on your page that's not visible. It has to be rendered so you can't use 'display: none' but you can basically pipe messages to these live regions to announce what's happening on the screen. I've just saw a React example where they put an ARIA Live attribute just on that wrapping element, instead of the focus management so anytime new content went into that element, it would just be announced. The challenge with that is that you can't always control everything on the page. That works if you control everything and you know that only this one element is getting updated at the time. But often, we work in this big ecosystem where there's a bunch of things happening. Depending on how complex your app is, you might need some sort of a focus manager, some sort of a utility that will keep track of what's focused and routed around at a correct place. That's the biggest tool for creating accessible single-page apps, that's focused management. I mean, not only for the reading content purpose but also to have their focus in the more accurate place so if they hit tab or they try to start interacting with something that they're in the right part of the page. A good example, if you think about like a modal window -- a modal window may open as a new layer over something -- that requires focus management on open so that your focus is sent into it, either to the first focusable element or to the wrapper. Then when you hit escape or close the modal, it just send your focus back. ROBERT: To the previously focused element, right? MARCY: Exactly, so that if you are using a keyboard and you can't actually use a trackpad or a mouse to get back then you're in the right place or if you're screen reader user and you can't even see the screen, then you're always in the right spot. That's actually, I think really cool. Something that's become more common place with dynamic JavaScript apps is that we can do these really cool focus management techniques. I think they're really cool, they can be challenging but that is something that we definitely need to think about as developers of single-page apps. ROBERT: Absolutely, especially since none of the single-page app frameworks out there were libraries. Actually maybe with the exception of your work on Angular, they don't come with a router focused-library built in so this is something that you have to actually think about and then pull in and do yourself. Does Angular have it, by default? MARCY: No, we never added a focus manager utility. There were some things to try and clean up that HTML, which ended up being, honestly worse than the original problem. But I've written a blog post about focus management techniques. I just dropped that in the chat. There's a smashing magazine article I wrote and it really is framework-agnostic so it sort of covers all of the things that you need to think about if you're writing a client-rendered application using Ember, React or Angular. It is something that we have to think about as developers because from the framework level, it's impossible to know what the right situation would be in your app in a given moment so we can only get so far with magic at the framework level. It's something I would like to see more of. Maybe if there is some sort of a layer manager, I think that is a tool that someone could write that would be super useful -- to make sort of an intelligent layer managing system for focus management. I've heard the Facebook team talked about how they do it internally but it's not open source so I have yet to see an open source solution for this. We have to tackle it in our own apps but once you know that that's the thing, you can really make sure that you're covering it. If you have someone with a visual disability or impairment that try and use your app, they'll probably uncover that problem pretty quickly. That's the value of user testing in case you forget. Maybe there's a few views -- ROBERT: Need to sell it. MARCY: Yeah, or maybe with your application, if you don't have visible focus styles turned on, you might not see that the focus isn't being sent. That is one trick, I will tell you in development. If you're working with focus management, turn the focus outlines on and then if you were trying to send focus before it got fully rendered or something because it has to actually be rendered to catch the focus. That is good debug flag, if you can all agree on the focus styles, for all users. I found that to be really useful in our app. You just to have those turned on so you can debug it. ROBERT: And make it really loud like this is a giant red outline. MARCY: Yeah, then you'll know, if you forgot to add tab index of -1, to make it catch the focus or like I said, maybe there's a rendering thing where you need to wait a tick by using a set time out or something. That is a good technique that I've used recently. ROBERT: Awesome. Basically, what it boils down to in single-page apps is manage your focus and enhance your focus, some might say. MARCY: Yeah, let's think about keyboard ergonomics, like if you are doing things dynamically on the screen and then you want to start typing, I think the most common example I see is autofocus. The developers, even if they aren't thinking about accessibility, they'll ask for autofocus. That in a way is focus management. The difference with autofocus is that you can only use it once and it will send your focus there automatically. But in a similar way, that's the idea of what we want is to get the user's focus point into the right spot so that they can do the right activity on the screen and they know what content they're looking at. ROBERT: Right. Sometimes, it's like navigating around a website with your keyboard, that's like power users who have Vim or Emacs or anybody that's a power user of computer that doesn't like to leave the home row, you can make your application awesome for you to use and also lay the groundwork for accessibility, if you can navigate your website with just a keyboard. MARCY: Exactly. ROBERT: Let's try to pitch it to people in that way. It's still a developer problem. CHARLES: I like that because it really highlights the fact that there is this kind of deep interaction model. The user actually is focused on one thing at a time in the application and if you track that, then it's going to be a benefit for all of your users. If you are deliberate about thinking like this is the subject of interest at this moment. You're just going to reap a lot of benefit for everybody. ROBERT: Keep coming back to it, building accessible applications yields a better application for everybody. MARCY: Absolutely. It might enable you to support some futuristic device that you haven't even thought of yet. If you have your actions decoupled from the actual input and you can do everything declaratively, that really makes it easier to try and support of use cases you haven't thought of like we need to borrow up that other keyboard combination or some touch device. It just really helps to not have everything buried in a jQuery event. ROBERT: Yes. [Laughter] MARCY: Like, "Oh, man I need to call that same functionality for multiple events. Crap." You need to decouple that real quick. ROBERT: "Let's obstruct this." CHARLES: Right. I think we're about the time. I know you've got a hard stop. You got some skiing to do. MARCY: I do. CHARLES: So we will let you get up on the mountain but thank you so much for coming by. This is been a great conversation. ROBERT: Yes, thank you for dropping all the knowledge. CHARLES: Yeah, I'm feeling lots of knowledge right on top of my head -- MARCY: Awesome. CHARLES: -- That I got to go and process. But for everybody else out there, I would say go experiment with aXe. The idea is going to be easy for developers. I know I'm going to experiment with it and then you said, there was a browser extension as well to help you out and probably call out every website that you ever use, right? MARCY: I'm dropping some links for you, just now. CHARLES: There's some links to go along with the knowledge so go check them out and you are @MarcySutton on Twitter? MARCY: That is correct. CHARLES: All right. Fantastic. Thank you so much for coming by. MARCY: Yeah, no problem. Thanks so much for having me.
Jonathan Jackson: @rondale_sc | Ember Weekend | 201 Created Show Notes: 01:01 - 201 Created 03:09 - 2017 Ember Community Survey 14:06 - Handling Changes and Churn 27:53 - FastBoot Resources: Boots and Shoeboxes [SlideShare] Typeform EmberConf JSX Isomorphic JavaScript Ember Weekend Episode #66: Bug Integrat (with Charles Lowell) Transcript: CHARLES: Hello, everybody. Welcome to The Frontside Podcast Episode 60. My name is Charles Lowell. I'm a developer here at The Frontside. With me is Robert De Luca, also a developer. Hello, Robert. ROBERT: Hello, hello. CHARLES: Today, we actually have a meeting of the podcast minds. We have with us a very special guest, Jonathan Jackson. You probably know him from the Ember Weekend Podcast. If that's your thing, it's a great podcast. I listen to it, you should definitely check it out. Hello, Jonathan. JONATHAN: Hey, how are you doing? I'm really excited to be on the podcast. I am an occasional listener. It's similar to my own podcast where if I don't edit it, I tend not to listen to it. It's when I have long trips you guys are number one, number two right behind The Adventure Zone which is a D&D podcast. CHARLES: You worked at 201 Created. Why don't you tell us a little bit about that? It's an interesting company. JONATHAN: Actually, when we book to this podcast, I was not at 201 Created. This is a very new thing for me. I think I started right around the time of Ember Conference out in San Diego and I'm just realizing that this is not exclusively an Ember podcast. This is the first podcast I've been on where I can't just assume blanket knowledge of Ember stuff. But it's an Ember conference out in San Diego. I actually gave a talk there about FastBoot which is a server side rendering technology. Right after that, the entire 201 company which I think is four. It's very small. The entire thing, the whole crew went to do a company event and basically camped out in the mountains for a few days, which was really, really fun. But I started working there and 201 is a consultancy based out of New York but I think it's more than half is remote. I think Matt's on the West Coast, two of them are in New York and I'm in Jacksonville right now. We do a lot of really cool stuff. We worked in a lot of different companies. You can actually see the website at 201-Created.com and you can see the different clientele we worked with. But we specialized in consulting, training as well. As well as a couple of other services that we offer. It's been a real great experience. It's been very fun and also I'm learning a ton which is really cool to be in a different environment. I have done consulting for a little over four years, previously at Hashrocket. I got to tell you, consulting will get your wheels turning. It's been nice to see how different consultancy takes a stab at things. It's been super fun. CHARLES: Yeah, it's a fantastic company. I've definitely known them for a while, certainly through my involvement in the Ember community and one of the things that always struck me is just how seriously they take the community aspect of it. We were talking about just a little bit ago, it was 201 that sponsors -- well, sponsor isn't really the right word for it. It does the Ember Community Survey which I think is a practice that we're now used to in the Ember community but I think it's something that I would love to see wider communities do. Maybe you could talk a little bit about that and explain what this community survey is, why it exists and what's the knowledge that's derived from it and how do we take action on that? JONATHAN: 201 also does contribute workshops and things like that. The idea is to make Ember a more inclusive space, a place where people feel comfortable being a part of our community and a big part of that is self-reflection and realizing where you have weak points and how you can actually mend areas that are being neglected or whatever. Basically, shining a light to figure out where we need to improve and a big part of that is the community survey so figuring out what technologies are being used, figuring out what demographics are represented or under-represented and trying to figure that out. It's actually been really cool. I think this is the third community survey and it's live right now. I feel like we could probably shed a little bit about some of the questions. This year, they did a really cool thing where they actually put all of the questions before they put the survey up live. They actually asked for a comment period which is very, very Ember thing to do. CHARLES: Because I was actually going to ask about that. Who is the final arbiter of the questions that get in because part of the survey is determining you're trying to get hard metrics on a set of questions but it's the questions that you don't know that you should be asking which are really the tricky ones. JONATHAN: It was really interesting to watch the document change over time. There was a committee discussion between some of the people in core team and Matt Beale and Tom Zalman who's been doing the organizational stuff as an intern at 201 and he's been doing a fantastic job of really staying on top of it. It's a surprising amount of work to get a survey together, especially when you have a comment period so there's tons of little adjustments here and there need to be made, to wording and phrasing and also like responses. Surprisingly enough, you can actually have biases in your questions based off of the responses that are allowed because multiple choice. It's been a really interesting effort and I think trying to weigh and balance that side of things, where you want things to be worded in a way to where people can answer more honestly and without a bias coming into the question. Because the questions change year over year, trying to get data that is historically relevant so we can see what versions were being used last year and versus this year, what was your experience level with Ember last year and this year. But still make those changes that are recommended. It's an interesting balancing act. I was very interested in the process. I was trying to stay as involved as possible but I think also, Isaac was working on that as well. It's been a team effort. The survey is a very interesting aspect of the Ember community and it's only been three years but it feels like longer. CHARLES: What I love is the work. Once you actually get the survey up, the work has just begun, which clearly a lot of thought went into it, not just the questions that is very beautifully presented -- ROBERT: But the survey, what's the software that you're using to do that? JONATHAN: I think Typeform is the thing that they're using. I feel like it actually works on mobile and I have a great analytics tools. If you're doing a survey, you should come talk to me. ROBERT: Does that like track people that half-fill out a survey and exit? JONATHAN: Yeah, it does. It actually does track like -- what's the word for that -- there's a word for that. Basically, the main metric that we're looking for is people who open the form then complete it and the percentage there, that's the respondent percentage. I've actually haven't seen the metrics yet. I think I might just wait until the conference. You're exactly right, this is only the first step so once everyone fills it out, then there's a bunch of data extraction because some of these questions are open-ended and allow users to directly input their own feedback and trying to sort that and make that useful information as a lot of work and a lot of effort. It's interesting to see, of course there's some obvious graphs that are going to happen like we see transitions. It's easy to graph out the number of people using things on a time axes or the X-axis or whatever. There's some kind that are obvious but I'm actually looking forward to seeing the results. I was only really involved in the actual administration of the survey in as far as I provided some feedback before the main feedback period. But it's been really a community effort which is great because it's a community survey. That's pretty neat. CHARLES: One of the questions that I have is how do you ensure, because the only people who hear about the survey are the people who are already involved. In order to get -- I don't want to say statistically valid -- a broad and more informative data set, how do you try and balance the concern of we want to make, maybe half people exposed to this who aren't inside my community or maybe sitting on the boundary somewhere or slightly over somewhere versus at some point, if someone's an attorney for example, then we don't really care if they hear about or participate in the survey. Certainly within the developer community, which is ill defined to begin with, how do you try and draw those lines to make sure that you get the best knowledgeable dataset possible? JONATHAN: You know, I don't really know. I guess, it's the meta point that I should probably make but it will prevent me from giving my opinion. Basically, I think that since this is a community survey, it makes quite a bit of sense for the community avenues for learning about Ember to be used to actually distribute the survey itself. For instance, these podcasts, like my podcast as mentioned in the survey and now, this podcast is going to mention in the survey. I want to say, "It's going to be in Ember Weekly." That's just a guess, I don't know. But there's really avenues: Reddit, Twitter, etcetera and then the Ember blog itself. Those are the means for dispersal within the Ember community. The one metric that we get from that is what can we reach or who can we reach? How many people can be reached through the normal means? That in itself a metrics. But I think it's kind of valuable to test that every once in a while. I believe over the first two, we saw 100% increase in respondents from Year 1 to Year 2 so it'll be interesting to see from two to three to see if that number continues to increase at a really rapid clip or if we're seeing some other trend there. It is an Ember survey so we are going to assume a lot of Ember-related things. We're trying to gain insight into the Ember community so it's probably not great to put it on JavaScript Weekly, for instance and get a bunch of React developers in that. They're going to be like, "I don't use this." Why are you using Redux, they don't understand. Oh, wait, Ember Redux, what's that? That sounds up my alley. ROBERT: I did feel that form out at the survey with my experience that I've had with React recently, things that I would like to see come over to Ember. I don't know... That'd be interesting to see or I wouldn't want them to fill it out because they would, obviously ruin the data set but I think another survey with more information from other communities to see like, "What's preventing you from utilizing Ember or what are the barriers to learning it?" Maybe from other communities might be interesting. That would be cool to do cross-pollinated surveys where you can be like, "We'll do it, if you do it and then React can provide us something and vice versa. I feel like the word homogenization is bad, usually but sharing ideas is good, I think. CHARLES: If you've never experienced this in your development community, the amount of work that goes into actually analyzing the survey and trying to draw and make inferences from it is just astounding. Who does that? Is that like Matt sitting up in an ivory tower? That's was just the wrong term. Basically, he can sequester himself for a month and put on his thinking cap and just come out with these mind-blowing deductions? JONATHAN: To be honest, I wasn't here last year at 201. My suspicion is that Tom will do quite a bit of the data munging, I think is the word. Then we'll go through phases where I think Isaac and Matt are working pretty closely with the survey stuff so they'll probably do feedback loops, then eventually before anything happens, I think with EmberConf, there's usually a survey blog that comes right alongside the EmberConf thing, where you share some of the results. I think that those things will go out to core and then core will start to pick it apart, toss that around exactly and then come back and basically, you're going to try to get as many people who are pretty smart, looking at it and trying to make sure the data makes sense and honest and doing the right things the survey needs to do, in relevant, I guess is the other metric. CHARLES: Now, as part of the survey, one of the things that you mentioned was the purpose is to surface weaknesses and gaps that need to be filled. When you think about your experience and the way that you filled out the survey, obviously it's anonymous, share what you're comfortable sharing but what were some of the things that you perceived as maybe holes that need to be filled and you're hoping that the survey will bring to light. JONATHAN: That's a very interesting question. I think, the thing that I'm interested in seeing is maybe different than what I've seen. One of the things I'd really like to see the survey that bring to light -- it's probably the most important metric in my mind -- is where people are at in the upgrade process because the cadence of releases an Ember is such a big facet of what makes Ember really powerful, especially for large companies and stuff like that. But I've personally seen people get stranded in certain spaces. Usually by the time they call a consultant to help them get un-stranded, they're at a point where they're going to try to work towards pushing past it. I think this is felt primarily around the 1.13 switch. People did get stranded there and some people are still working on very large apps to push past that and I would really like to see just where the community is at right now, in general. Especially as a consultant because you come into a project and I don't necessarily know what to expect. I think on certain teams, I am always shocked I see like, "Oh, you're using beta and everything. You guys are on top of this. That's really cool. Let's do some feature [inaudible]," and you're really excited. But then other times, you get called in and they're like, "We're still using 1.13 and we have bind others in our source and could you please help us?" CHARLES: Right and it's just that mountain is just too big to cross. That's something that you see in software development as the tools that you use tend to change and for lack of a better word, rot over time. In comparison to what's more newly available, it's the phenomenon of JavaScript churn, which is known in the community at large, scope down to just one framework where you've got different versions of the framework and you've got this churn. It's been somatic for Ember to try and it has been very aggressive attacking this problem and yet still, it manages to happen. How does that work, just given the amount of attention? JONATHAN: Ember, hands down handles this better than most other JavaScript projects that I've seen. I've gone to old backbone apps throughout my career and knock out in Angular one, etcetera. I've seen the rot that we're talking about here and usually, once it gets too bad, the authors of the JavaScript libraries are unable to push it forward at all. Either band in it or end of life it and you're going to have to invest your own time to get pushed past this point. In Ember, it really strenuously disagrees with that philosophy. They try super hard. All the people in core and really the community at large, the philosophy is like, "No, we're not going to break Ember. Ember is very serious here. We're not going to leave people stranded," yet it still happens. The reason I'm curious about seeing it is really about how do we make that story like a solved problem. Is it possible to do? Is it possible for us to basically make it to where the Ember community can very honestly say, "If you choose us if you choose this framework, it will be around. There will be a path forward for you for five to ten years and that's not something you can get a promise from anywhere else." I just want to see what are the ways that we can make that promise more strong. I think, the LTS was a big step in that direction. I think that was actually last EmberConf which the LTS was announced? ROBERT: Yeah, absolutely. Definitely all of our clients have moved to LTS as rather than trying to do every six weeks because they find that much easier to upgrade in between and they're more stable. JONATHAN: They're more stable and I think it's such an easier sell like if you actually start talking about going up the pipeline and you're like, "I have to talk to my boss and my boss just to clear money. We have to clear time, etcetera." We're going to put a [inaudible] aside every six weeks to upgrade seems a little untenable for a lot of companies. I think for larger companies, it's sometimes okay because they're actually utilizing some of the edge features which is cool and I think that's a big thing. I feel like I have no real insight here but I feel like that's what LinkedIn kind of does, where they're usually pushing the boundaries because they're utilizing features like engines were first brought into LinkedIn. I think it kind of pushing it at the edge. ROBERT: If you have the new LinkedIn Ember app, if you will crack open the inspector, when I last looked, I think the beginning of this week, they had two beta versions deployed. The Ember data version, that was beta and actual Ember, it was beta. CHARLES: Usually large companies are associated with big lumbering end piece that are in terrible condition. That's actually a breath of fresh air. Shout out to LinkedIn. ROBERT: Ember Data is 2.12 canary and Ember is 2.10 Beta 2 patch so it looks like they have a patch version. JONATHAN: It doesn't surprise me that Data is being pushed. I think last I spoke to [inaudible] right around December, he was doing a lot of perf work on there so I think he's really pushing that pretty hard. There's a lot of really cool stuff like that and I feel like it kind of runs the game. You see the smaller teams who choose Ember for stability, they sometimes get stranded so I want to see if some survey data can probably correlate. You could correlate the size of your company to the version of Ember you're on. Maybe, we'll see some trends around if it does it mean that smaller companies have more difficult time pushing forward. That would actually be a little counterintuitive. I would expect that smaller companies would be able to push forward at a faster clip because they usually have to support fewer browsers, etcetera. It'd be interesting to see information like that because I think that promise for ease of upgrade and there will be a path forward, that's a big part of what makes Ember really appealing to me. Especially as a consultant for four years, you see so many projects. I don't ever really want to advocate a rewrite but we're going to have to spend a significant amount of time fixing this and it's because you went with Mootools or something. Everybody guess Mootools wasn't so bad. CHARLES: But the point is that you didn't go with a holistic solution so you basically had to write your own framework. JONATHAN: Yeah. ROBERT: Yeah, in Ember, it is a feature that you will not be left behind and you can upgrade. That is something that is really nice. I have upgraded a lot of Ember apps. JONATHAN: I think Mike North calls that the patchwork app application. It's not just like React apps where you have React-Redux and Preact and all of this other stuff that you kind of piece together and make your own little quilt and that's your application. But this also happened in Backbone. It happened in jQuery before that and it was just like take this thing, take that thing, then I have this custom quilt, which is not bad. There are some advantages -- pros and cons. CHARLES: Ember is giving you a blanket. JONATHAN: And it's going to be a comforter. It's going to probably all look the same and be right. ROBERT: My experience is I love Ember and I love the convention over configuration but whenever you hit that wall of the convention is actually getting in the way now, that is a very tall wall to scale in Ember. CHARLES: Yeah, I think the flip side of it is like you say, Rob because everything does have to mesh, because that blanket has to be one solid weave, it means that you've got a hole in the blanket, the surgery required to excise that hole and then patch it -- ROBERT: I love his metaphor. His metaphor is -- [Laughter] ROBERT: It's so good. CHARLES: It takes a lot of effort. ROBERT: Today, I'm quilting daily. CHARLES: That's right. Next topic, crochet. [Laughter] CHARLES: But, yeah in order to make that surgery on the blanket to mix metaphors, which I love to do so freely, you have to make that cut and then make sure that the weave is again, seamless. I think that takes a lot of thought, it takes a lot of effort and it takes a lot of time. It means that there are shiny things out there that you might not be able to have. I think, one of the ways that the community and the technology is mitigated is with the add-on ecosystem, which is very, very strong and allows you to riff and experiment and push those boundaries. But there are core pieces, things like the rendering engine, which can't really be modified or hooked with an add-on. They can but not in deeply fundamental ways or the templating. We saw that happened. There was a big kind of shift from first, the old handlebars to -- ROBERT: HTMLbars? CHARLES: HTMLbars and then Glimmer 2, which there's been a flurry of activity around there but that was definitely one area where there was a hard wall right now. I feel like for me it's around the handlebars itself. I would like to see that environment become more powerful because certainly, with the React Native work that we've been doing around here, you get to see just how simple like the JSX model is, React aside because like Vue, you can do with JSX. I think JSX is a separate technology. It's certainly integral to React but there are a lot of other frameworks now that are using just the JSX part for the templating. Seeing that there is real power in being able to have the functional programming aspects of JavaScript right there inside your templates. From my perspective, I think that in Ember, there's a wall there that needs to be scaled. ROBERT: To be clear to the Ember developers that are listening like us kind of advocating JSX, if you are having like, "No, that's a terrible idea. I hate JSX," I had that very exact reaction about a year and a half ago. If you go look at my Twitter feed, you would see me ranting about how much JSX is a bad idea. After I actually played with it, I'm on the opposite side. I think JSX is really awesome and I think there are things to learn from it. CHARLES: I definitely love having templates. I love having the separation. I like having it in a different file but at the same time, I don't want to lose sacrifice the power that comes. I think that for people who are kind of sitting on the fence or have played with it, if you actually are strict about not having side effects and things in your templates, it really is a great experience. I think there's a lot of people who have scars from doing ERB or liquid templates, where you can have all kinds of crazy side effects -- ROBERT: That's where my scars came from -- ERB. CHARLES: Yeah, I can show. I can roll up my sleeves. I will be like, "You see this? I got that back in aught-seven with an ERB app, where they were calling out to a service from inside the template." ROBERT: Setting the variable and modifying everything. CHARLES: Yeah. There's definitely that tradeoff. One of the things that is great about the Ember community in particular is when there is, it takes a while to generate the will to recognize that this is a major problem but then the solution that you do get does, eventually match the weave of the entire blanket, which is really, really nice. But it can be frustrating when you have those core pieces of infrastructure that are presenting those walls to you. ROBERT: I'm excited for Angle Bracket components because that's actually a lot of the gripes that I had with handlebars. Whenever I got a bunch of the curlies next to each other, like a bunch of components around each other, they all kind of just mold together and seeing the brackets and just looking like HTML, it makes it so much easier to grip. CHARLES: Yeah, it's weird because you think that small things won't have big impact and you think that big changes ought to have a big impact. An example of this, I was kind of derisive of the whole Angle Brackets syntax. I was like, "Urgh! Angle Brackets, dah-dah-dah..." Then we started doing more JSX and you start seeing like, "I want to have my templating construct separate from my JavaScript and scripting constructs," and it actually makes a huge difference in clarity there. Obviously, the change to make all that happen is big but it's a small difference in the syntax. Tiny but I think it has a huge impact in the readability and the clarity of the templates and by the same token, all the performance increases. At this point, I couldn't even give a flip. It's nice. It's great but there's a barrier, there's a threshold that has been crossed, actually some time ago. Performance of rendering is -- I can't even remember the last time it was a problem. What about you? Have you run up against performance issues in your Ember apps? JONATHAN: Some performance issues but usually, they're a result of some rather inefficient rendering. Basically, a combination between user and keyboard or whatever. I wrote something really bad. It's not Ember getting in my way. I don't particularly mind the curly braces within my template but I think a big part of that is just editor choice. If your editor syntax highlights then it also knows how to indent handlebars correctly, that makes a huge difference. ROBERT: Are we about to start an Emacs versus Vim war here? [Laughter] JONATHAN: No, as a matter of fact, I suspect you would win that when the Vim -- there's no good solution for indentation in handlebar templates that I found in Vim. If anyone knows that [inaudible], "Oh, there's one plugin," please ping me on Twitter because that would be nice. CHARLES: Well, yeah. It's true. I can deal with it. I don't think it bothers me quite as much as it does Rob but I think what has been interesting is in our hypothetical code, you always like pay snippets in Slack. We started using Angle Bracket syntax just because it's so much clearer. Even though, none of us actually use it in any of our apps, when we're actually exchanging ideas, that's what we use. JONATHAN: Yeah, there's some cool things that come with Angle Brackets that aren't just aesthetic. The container element is like you don't have to deal with the tag lists stuff anymore. I feel like there's a few tradeoffs that are going to be really interesting to see when those start becoming the norm. CHARLES: Yeah, I like also the separation of what they did from JavaScript attributes to HTML attributes. It's really clear. JONATHAN: Totally. I think it's [inaudible] cool stuff. CHARLES: It's exciting. I remember being derisive of it -- not divisive, that's not the right word -- but I'm thinking like, "Why are they spending so much time on this," but I actually think it is going to have a big impact, small change. JONATHAN: Totally. CHARLES: No dis to the people who are working on it. I know it doesn't feel like a small change at all. ROBERT: Yeah, it only took a year and a lot of really hard work. [Laughter] ROBERT: Like I peek in there and I'm like, "Hmmm... Nope, not smart enough yet." CHARLES: One of the things that I want to ask you, you mentioned that at SO Ember, you gave a talk on FastBoot. You've actually got a lot of experience around the subject so I'm just curious. First of all, what were you talking about? JONATHAN: I think my talk was actually called Boots and Shoeboxes, which there's a little library function into the FastBoot suite called the Shoebox where you can communicate between node and the browser. It's not like well-known enough to where that title resonated with people. I got up on stage and I was like, "You know, we don't have any descriptions on the speaker note like website. He just talks about FastBoot. I hope that I don't disappoint you," because they had no idea what I was talking about. Actually, I feel like the problem that's the FastBoot solves is a persistent thorn in people sides. CHARLES: So what is the problem just to give full context? I think is it called like Isomorphic JavaScript for something -- JONATHAN: Yeah, I don't like using that word. CHARLES: Yeah, there's like server side, SSR -- JONATHAN: Yeah, SSR, you'll see that a lot. CHARLES: I guess the question is why would you even? JONATHAN: That's a multi-faceted question. I think the first section of it would be what's the problem? I think for a lot of people, the biggest problem is SEO. A lot of JavaScript frameworks are not search engine friendly, then that affects a lot of different things. It means that they're not archivable either so it's not like you can have this on archive. They're not very crawable. This is becoming less of an issue because Google Crawler, for instance will actually parse JavaScript now. But I feel like that's still limited. Also you have to then think about how the Crawler is going to like actually execute your JavaScript. You're like, "Wait a second, so now I have to have a compatibility table for Google Crawler? That sounds madness." I think that's a big component. There's also the idea of speed downloading as low as poor connectivity devices or locations, I guess. Having to download all the JavaScript before you see the first meaningful thing is not a very good experience. Especially for a huge swaths of different types of sites like Discourse, I think is a big Ember forum software. Forums are mostly just static text, like you just want to read the text so time in First Meaningful Paint could be like as soon as you get text onto the page, that's could be really fast. Some sites that doesn't make sense for it like if you're posting a video game or something like that, like you need interaction for that site to be meaningful. There's still tradeoffs there but there's a whole host so I guess that's the need. Then the solution for a lot of people is to start rendering JavaScript on their backend software and presenting full HTML along with a JavaScript source tag so that you get a Meaningful Paint first and then you get the JavaScript a little afterwards. The whole point of my talk, which I was basically like -- CHARLES: That's a hard problem. JONATHAN: Oh, it's a very difficult problem. CHARLES: Unlike anyone who says they have a solution, you should look at them with extreme mistrust. JONATHAN: Yeah and there's a whole bunch of different solutions that people have tried. You could actually have prerender.io, I think is the service that will actually render it for you and you put it in front of your CDN and they'll actually do that and create static files for you, which is a solution or no script tags. You basically render all of your stuff as much as you can on the server side and you put everything into no script tags and that will presents its own problems. There's a bunch of different solutions that people have tried. In FastBoot, the solution that Ember went with and I think that it's really cool because server side rendering and this is the big reveal of my talk. I think it's recorded so you can check it out. But the bigger reveal is that the server side rendering is not just about rendering. It's also about routing and data fetching and authentication and etcetera. There's a whole bevy of things that you also have to handle very well. It's not just taking a component, the view layer to component and rendering it to HTML and then serving that. It's much more than that. You want your app to basically run in node. FastBoot does that remarkably well. There are some spots where it's a little fuzzy but does it remarkably well. CHARLES: What's an example of how you would might need to handle authentication? That sounds terrible. ROBERT: One of the problems for a lot -- JONATHAN: That's exactly the problem. You actually have access to headers and stuff and FastBoot land so you can do authentication by using traditional token off, which is pretty cool. There's a lot of really cool things and routing is obviously handled quite well so the request comes in and it does the normal Ember router. The Ember app instance itself is running in node so all of the things you expect to work in the browser, work in Ember and node, with the exception of any time you need to access the DOM because the DOM is expensive like very, very, very oddly expensive. Like JSDOM is just expensive and unreliable, then you have to deal with compatibility tables for that. Anyone who has written tests for Phantom and tried to bind a function or something, they know the pain. I think it's fix now but I was always bitten by that so many times. It doesn't even give you the right error. Forget about it. CHARLES: You have all these things. It's basically authentication. It's data. It's making sure that you have in your, so to speak, headless environment as an authentic replica of your application running in the user's browser, as you can possibly retain. JONATHAN: Yeah. CHARLES: How feasible is that? Like what you're saying is that Ember takes that whole approach and says, "Okay, we're going to make sure we handle all of these cases?" JONATHAN: Yeah, I think Ember has done a phenomenal job of this. It's still alpha software, although I believe that the path to 1.0 is basically paved. It just needs some documentation. I think FastBoot hits the nail right on the head and gets a lot of the stuff really in a good place. It's also a big part of FastBoot's call to action where this stuff is possible elsewhere. You can do all of these things. You can make all of the stuff work in the React ecosystem or Vue ecosystem, etcetera. But in Ember, it's Ember install, Ember FastBoot, I think or Ember CLI FastBoot which is a really compelling sell because I've looked at some of the alternative approaches and in other ecosystems, they're very complicated. It's not possible. It's just their ad hoc -- ROBERT: And it's usually a 10,000 line medium posts that you have to follow line by line -- [Laughter] CHARLES: Right so instead of giving you actually a working code, what you get is a treasure map. JONATHAN: Yeah, exactly. It's like you just shop at Ikea. Here you go, build it. There's some really cool stuff that it unlocks and the fact that it's so low-hanging fruit, for instance Ember Weekend, which by all accounts does not need to be on FastBoot, isn't on FastBoot because it's ostensibly free and it's a good testing ground for me to learn about FastBoot. But the future -- ROBERT: It's interesting in handling audio on a FastBoot, how was that? JONATHAN: Since the user doesn't actually can't listen in node land, the user can only listen in a browser, we don't do anything with the player in FastBoot land, which is fine. There are some weird things like you have to basically have guards around like key events, for instance. Because Mousetrap relies on, I believe in jQuery to bind its events, you have to basically say, "In node land, we're not going to bind any of these Mousetrap events because they will not work," but there are some things you have to learn about the ecosystem but by and large, it's a solution that you just drop in and you just get for free. I think that's a huge sell. That's another thing with the convention over configuration argument, the model is that eventually, once the solution arrives, most of the people who are using Ember can just use it right away. It really does help with [inaudible] activity devices. There are some really interesting things about how time to first paint, I think Martin [inaudible] just released an add-on that basically says, "I'm going to take all of your JavaScript files and mark them as async and then when you download, the time to first paint becomes almost immediate," because it's just going to say, "I have HTML. Here's the HTML and serve it." Then in the background, because of script tags or whatever, it just goes and fetches the stuff in the background and you end up like time to First Meaningful Paint is really cool so it'll perform software that is super neat. If Discourse wanted to say, "Here's the stuff and we're going to make it work later," like as soon as the JavaScript has download, that's a really cool sell too. There's a lot of weird edge cases and describing the interactions is I think the hardest part about FastBoot. It's just like describing why this might be really good for you is the hardest part because a lot of people don't have these problems. If you're doing a marketing site, you're probably going to use Squarespace or something. ROBERT: Yeah, like a static site generator or something? JONATHAN: Yeah, exactly. ROBERT: Something that will give you great SEO results. JONATHAN: Exactly. ROBERT: You want to play around with that. JONATHAN: This dovetails into something Edward Faulkner was talking about eight or nine months ago when he was working on the inline content editor for Ember. It's really, really neat. I actually like to see where that's at now. I think it was Cardstack that funded a lot of the stuff for it. But if you combine things like that, then also FastBoot, you're starting to talk about something that could do what WordPress does, which is a really interesting thing like the really, really low hanging fruit. Type these few commands and you're point clicking your way to a website which is really, really cool. CHARLES: All right everybody, thank you so much for listening. Thank you, Jonathan for coming on by and talking with us today. JONATHAN: Yeah, thank you so much for having me on. This podcast is super awesome. I'm really excited to actually be able to be a part of it. I feel like you are at Ember Weekend that one time and you were in Norway? CHARLES: Finland. JONATHAN: Yeah, Finland and we weren't able to actually have a video open at the same time because of the data problem. It's been actually kind of cool to actually have a real conversation. That's been really great. CHARLES: Yeah, that has been awesome. That was a good conversation and that, your podcast obviously is EmberWeekend.com. Everybody go and check it out. Thanks for listening.
Show Notes: 01:11 - Doing Dumb Stuff aka “Throwaway Projects” 06:06 - Combatting Burnout 10:01 - Dumb Projects That Pay You Back 17:00 - Brainstorming and Abstraction 25:19 - chillestmonkey.com 20:19 - “The Iron Triangle”: Creativity, Accomplishment, and Learning Resources: React Native and Chill: A tale of stupid made fast by Charles Lowell Transcript: CHARLES: Hello, everybody and welcome to The Frontside Podcast, Episode 59. We're getting up there, 59. That's like, I don't know, it's not a milestone but it's something. ROBERT: It's like one away from 60. CHARLES: Yeah, it is. It's past middle age. It's like elderly. ROBERT: Start thinking about retirement. CHARLES: Yeah, exactly. JEFFREY: These are our golden years. [Laughter] CHARLES: Welcome to the golden years. ROBERT: All right. Possibly, we need to go and watch the Golden Girls. [Laughter] CHARLES: Actually, I think it was only five or six episodes, maybe 10 episodes, we were singing The Golden Girls theme so it all comes back around. We're here with a very special guest and that guest is nobody. It's just folks from The Frontside -- JEFFREY: I was hoping you would say it was Betty White. [Laughter] CHARLES: We're going to fly it solo or like tri-lo or like trio. ROBERT: Trello? CHARLES: Trello. I, of course, am Charles Lowell. With me is Jeffrey Cherewaty and Robert DeLuca. Hey, guys. JEFFREY & ROBERT: Hey, what's up? CHARLES: We were kicking ideas around and something that's been kind of percolating around the offices is a theme for 2017 is doing dumb stuff, just stuff that has no apparent value but that you can learn from. I think, we each have a bunch of these experiences where we've done something a very little import that ends up being really, really helpful, either both in the short-term and the near-term. JEFFREY: And who knows, maybe this episode will turn out the same way. ROBERT: Oh, how meta. This could become a black mirror episode. I'm start to questioning my values. CHARLES: I know for me, I recently did some explorations into React Native, which I found to be very edifying. I could obviously talk about that experience quite a bit I did on a blog post but I'm curious, if you guys recently had something that was a throw away, something that you did that wouldn't really matter if it had come into existence or it didn't but it's just so happens that in this thread of reality, it did. ROBERT: You know, I have. It's always been centered around the impagination library that we wrote here. I was always kind of intimidated by impagination for some reason because it was this big library that I didn't necessarily understand. I was like, "You know what? I'm just going to go for it. I'm going to go do something dumb with it," and then I just decided to implement the most useless infinite scroll. It solved absolutely nothing and as you're paginating through 500 records of robots from Faker, I sat down and spent six days and wrote some code and implemented it React Native and it was actually the most informative and fun thing I've ever did. I don't feel tied to it. CHARLES: Yeah, so what kind of inspired to do that? Because usually, it feels like there's this pressure to ship something. Ship something is like just go build something but the idea is that you're going to build something that people actually might use. ROBERT: Yeah, I always had that idea. Maybe you can think about it as like feeling getting cornered, like the pressure of shipping sort of pushing me into a corner. Then eventually, I just kind of lash it out like, "No, screw this. I'm out." I'm going to go do something that's not even useful. I don't care. I'm not going to try and support people or make it to something that other people can use. If that is what falls out of this, that's cool but I'm going to totally sidestep and this needs to be something that other people can use. Sometimes, when I go to build a project, I start thinking, "This is going to be in my GitHub public profile. What if somebody comes and finds it? What are they going to think about my code?" And I just shed all of that fear away, then what happened is I learned a ton. After that experience, I was like, "Whoa. This is massively valuable." CHARLES: Yeah, like hearing you talk about it makes me think about one time I went to a Picasso art exhibit and they had all these sketches that he'd made just in pencil from when he was younger and they were in studies. I guess, apparently artists do this a lot where it was like just a goat's leg. Or some old man's nose. You know, just in pencil or charcoal and these are little tropes that he later integrated into all of his painting -- ROBERT: That is an amazing way to put. JEFFREY: It's funny. When you see engineering tutorials are like, "This is how you learn to make software." A lot of times that is the way they're structured it is around studies. Like, "Make this little tiny thing that by itself is worthless but can be part of a greater whole." ROBERT: Yeah, take that impagination infinite scroll thing, I think three months later, it turned into a full on talk about co-chairing between React and React Native but it started with this dumb little project that I decided, "If I don't finish it, nothing bad happens. It just kind of sits there and rots." I feel no guilt. No pressure. JEFFREY: Would you say that was part of your blue period? [Laughter] CHARLES: Yeah, I like you [inaudible] point to like ship it. It's like we have a culture of ship it or don't. In any way, it's entirely up to you. ROBERT: I kind of thought to think about it as the extreme of ship it. Literally, just ship it even if it doesn't work. CHARLES: Right, ship it even if it's wrong. ROBERT: The other thing that I figured out was who is to determine what's wrong or right. I figured out that no one like I figured that out and I choose if it's wrong or right. CHARLES: Right, and it's like whether I learned something or whether I didn't or whether I get to be the arbiter of what I get out of this experience. ROBERT: And you almost always learn something. Always. CHARLES: Yeah, you definitely do. I think it can pay off, both in the short term and in the long term too. I know with my recent experience, I was feeling extreme burnout. I don't know... You feel like the code that you're working on or the things that you're doing have become a burden. I guess that's kind of to your point, Rob like when you are building something that it creates users. It creates code. It creates maintenance cost. It creates contributors. There's all this mass and inertia and momentum that are for it and that can be great if you own something massive, you can have a lot of momentum and it can be extremely energizing. But it can also be a burden, if it's something that you have to carry and you feel obligated to carry. ROBERT: Yeah and when you have those huge successful projects, things that come to mind like Ember or React or Babel or things like that, those are awesome. But I was always experiencing that responsibility with things where I had big plans for them but nobody knew that just by looking at the projects like it was still very much a work in progress. But I felt that feeling in my gut. CHARLES: Right and that mass and that inertia can come completely and totally from an internal source. It's both important for the project to be like these throwaway projects to be completely and totally free from all attachment, especially from your own dreams and your own ego and things like that. ROBERT: And when I learn to let go, that's when I learn that I'll learn. [Laughter] ROBERT: I'm going to learn-learn. CHARLES: I guess, the kind of greater point that I was making is that when you are able to approach a project like that, then it can be a really intense cure for burnout because you are just allowing yourself to create, you're allowing yourself to feel creative and actually deliver something whose success parameters you define entirely. I think, Brandon actually talked about this almost like two years ago with that robot. That project that he did where he was kind of in a similar situation and he really, really needed to do something that was not a web app. I think there was a series of talks that came out of it but that was primarily for the burnout case. ROBERT: I think I can agree with that. I might have been in the similar situation where I was starting to think about the feeling cornered. It was maybe because of burnout. CHARLES: Maybe those are one of the same. ROBERT: Because I felt like I wasn't producing anything and I was like, "I should be doing this stuff but I feel like I should be but I'm not," and the things that I'm doing aren't great enough because the things that I have done in the past and that's a bad way to think about it. JEFFREY: There's something so refreshing about being able to switch contexts of I've been working on this same app for a few months and doing kind of similar tasks over and over and that's such a nice way to create at really recharge like, "I'm just going to do something completely different and see how that feels." ROBERT: Yeah, I've gotten really good at writing computer properties but I want to write something else. JEFFREY: Just like anyone who does a lot of Ember, they're great at computer properties. CHARLES: It really is the equivalent of throwing a dart into a map. If you're feeling burnt out and you're looking for what to do, There are so many things like you actually aren't cornered. You have the universe of possibilities and the dumber the better. Choose something random that you can do in five minutes, do something random that you can do in an hour or a week or something like that so it has that payoff for being an answer to burnout. I've experienced where these types of activities come back and pay you back down the road. ROBERT: It doesn't have to solve a problem. I was always looking for the next project that I could build that would solve a problem. I always felt like I needed to solve a problem and I was just looking at it backwards. You should also be looking at things like it solve a problem. That's cool. But why not take a piece of technology and like, "How can I bend this? What can I do?" And exercise the different corners of this framework or do just something that's totally useless. CHARLES: I remember actually, that was something that Ryan, when he first started coming to the Ember meetups, he was asking questions about it and I think he was coming from Backbone and was -- ROBERT: Just Ryan from [inaudible]? CHARLES: Yeah, [Ryan Ralph?] from [inaudible] and he was asking questions and like, "Yeah, it's pretty good. The only disadvantage right now is you can't have multiple apps inside of a single tab," so literally he comes back the next month with a talk of how I embedded more than one app in a tab and here's what I had to do it. First thing was like, "Let's see how does it break." Let me prod at it and poke it in and just see what happens. I know I had an experience recently where I had done something really, really stupid with Amazon Lambda and I just very recently came back like the fact that I had actually gone through the process of just deploying a very dumb -- ROBERT: How dumb, Charles? Tell me how dumb? CHARLES: Remember when I was going on and on about badges and how we needed to have our own custom badges. I kept on trying to get everybody excited -- JEFFREY: I don't know that we need that. CHARLES: That was pretty much the response that I got from, I think in a poll of nine out of ten. It was 9.11 or something like that. ROBERT: -- That was cool but -- CHARLES: It was cool but no one wanted to put it anywhere. The badge was just a static SPG that was served on top of the Amazon Lambda but as part of that, I had to go through all of the steps through actually getting it to deploy. While that thing was completely useless and it was thrown away, that knowledge ended up becoming useful almost a year later or maybe six months later, something like that. I feel like it was sunny so it had to have been at least six months ago and so -- ROBERT: You know, that it's always sunny here. CHARLES: Uhh... Ish. I mean -- JEFFREY: It's not Philadelphia. ROBERT: I was trying to work somewhere. CHARLES: It was warm but it's a practice that often has long term payoffs. I guess that's just the learning aspect of it, the fact that you're going to learn something. ROBERT: The thing I want to stress here is you don't have to go into it expecting that. CHARLES: Yes, that actually is critical. ROBERT: Yeah, that absolutely is critical because then if you expect that, then the problem with all the things for me started with setting expectations. It was always I had expectations. You take them and throw them out. Toss those things out the window. CHARLES: Zero expectations. JEFFREY: Really, it's an adjustment of your expectations to where at any point, you can say, "No, I'm kind of done with this. I learned something but it's not going to turn into anything," versus the expectation of, "I'm going to make something amazing that thousands of people are going to use." ROBERT: Yeah, that's the way I always start out as like, "Here we go." CHARLES: "I'm going to take over the world." [Laughter] ROBERT: "I'm writing another JavaScript frameworks. Next step, world domination." CHARLES: Yeah, I know it's absolutely important to make sure that if this thing that you're doing just cease to exist, you wouldn't feel good or bad. It wouldn't make any difference. Very Zen, I think. Some people try to approach every aspect of their life that way. I'm not sure if that's healthy. ROBERT: That's another podcast. [Laughter] CHARLES: But I certainly think it is, if you at least allocate a portion of your projects to be that way. I think that applies to any creative endeavor that you try to undertake. Jeffrey, you had some actually some non-programming examples that you were talking about earlier. ROBERT: It ties in nicely to the Picasso thing. JEFFREY: Yeah, actually I was I'm moving right now to a new apartment and I have no architectural background at all but I am comfortable with vector graphics in Illustrator and doing things mathematically precisely in there. I do have a little bit of problem solving that comes out of this. It's not purely exploration but I've been like, "I'm moving to this new place. These are the pieces of furniture I have. How can I rearrange them and play and --" ROBERT: Did you actually set the scale correctly to your -- JEFFREY: Oh, absolutely. ROBERT: Oh, that's amazing. [Laughter] JEFFREY: --Otherwise it's worthless. ROBERT: This is a whole new world. This is awesome. CHARLES: -- Like there's an atomic gauge on the couch. [Laughter] JEFFREY: My couch is six feet, three inches and 24 angstroms. I have not reached that level -- [Laughter] ROBERT: Does Illustrator have that level precision? JEFFREY: It can go pretty far. But yeah, it's just an opportunity to play around and like in a situation where actually moving those pieces of furniture in 30 different ways would be a pain and just unrealistic but thinking about it more in the abstract and just being able to play with it at a scale that's playable with, turns it into something fun and creative. CHARLES: Yeah, I guess that's a luxury that we have, I guess in the modern era of being able to simulate so much and be able to apply this practice of total creativity in a consequence free zone where people might not have been able to do so before. Before the advent of computer-aided design, I don't think that would've been a possibility. ROBERT: There will be a lot of manual hand-drawing and sketches and stuff. JEFFREY: And now software it's at point where -- CHARLES: Or you just get your children to do it. [Laughter] CHARLES: "Move it over there." You guys are sweating but man, I'm feeling really creative. JEFFREY: Yeah, really making stuff happen here. But software has kind of reach the point where we have similar abstractions there to be able to do it, move pieces around like that. Maybe not as fluidly as I'm going to move all these squares around in Illustrator but we're at the point where we can kind of plug in pieces and see how it performs and plug in another piece and see how that performs in a way that feels maybe more creative and less scientific than getting lower down in the code. ROBERT: Honestly, I wonder if this is where React began because the idea of rerendering the entire dom every time was more performant like you just throw something at the wall. See if it sticks. I don't know but -- CHARLES: Yeah, you have to wonder. I'm actually become surprised at the origin story is not more widely known. ROBERT: Yeah, I also don't really know it. I kind of heard of it like -- CHARLES: I remember hearing about it the first time I was like, "That's crazy." ROBERT: Yeah, that sounds like it'd be massive performance hit. That sounds really slow. Why would you rerender everything? CHARLES: Right but then again, it's always getting into pre-[inaudible] optimization. We always fall down these paths of optimizing in our heads. Of course, we want to do incremental rendering. Why? Because it's faster but nobody's actually measured then realized that it's actually the dom that's slow. ROBERT: It's like opening up your world to this. It's just like you explore everything. JEFFREY: That's another thing we can take from design thinking and the philosophies around the design field that maybe aren't as recognized in engineering is the idea of simply brainstorming of being open to dozens of ideas, trying out dozens of ideas and seeing what feels right and being able to have so many ideas that you can throw most of them away. In software, so many times we get to the point where maybe we'll have two or three ideas but none of them are worth throwing away. We feel uncomfortable throwing out any work. CHARLES: Part of it is you were so busy. You have so much invested in your current track that it's very easy because your current track is on Rails. It goes forward, there's clearly a path forward and to hop off of those Rails, it require some sort of energy and are you going to be leaping into like a chasm? I don't know. ROBERT: Or in Jeffrey's case, if your computer dies and you're forced to think without a computer -- JEFFREY: That actually happened the other day. CHARLES: That was amazing. That was worth a story. JEFFREY: I forgot to bring my charger to the office so I have one of those new [inaudible] MacBook with the USB-C charger and we didn't have any backups in the office. I was hacking along on some configuration devops kind of stuff and just kept running experiments and eventually my battery died. Then I was forced to whiteboard out by what I was doing and I came way over to conclusion that I would have ever come to, if I had just been sitting on my computer the whole time. It was another case of where I needed that abstraction away from looking at the code to be able to rearrange the blocks in a way that made sense. ROBERT: Pull yourself out of the [inaudible]. JEFFREY: Yeah, definitely. CHARLES: I really like that idea. Again, I'm not being too familiar with the way that the design world works. Is that kind of like a modus operandis to have too many ideas that you can throw a bunch of them away at any given point? ROBERT: Where you start stealing pieces from all of them and you make one master design. In my digital design class -- shout out to Miss McDaniel -- she would always make us start off -- JEFFREY: Check for that in the show notes. [Laughter] CHARLES: -- She would always make us start off in a sketch book and I remember because I'm not an artist at all. If you know me, I draw even the worst stick figures: my arms and stick figures don't line up. That's how bad it is. She would always make us start off in a sketch book and it drove me nuts. But after doing it for two months, I finally realized the value because she would make us come up with five or six different design concepts. Then after I did the process a couple more times, I realized, "This actually has a ton of value." I sort of picking things from one of the other. It makes you think outside the box. The first couple of ones that I would turn into her she would be like, "You actually have three of almost the same design here. You need to think more outside the box. Think of something that's completely different." Seriously, it's just like you're throwing things at the wall. Whatever freestyle off top of the brain and just let it go. JEFFREY: I'm thinking of a concept side of exercise where maybe you're playing around with layouts for selling a magazine or something and you'll sketch them out and you'll sketch out maybe a few dozen of them but you don't get into the nitty-gritty details. You do it at a very-low fidelity where it's just pencil and some boxes and rearranging those boxes and maybe mark what that box is but you don't worry about the details of that box until you mix and match like, "That layout is not going to be great. This layout seems like it might be along the right track. I like this piece of this one. Let's combine them and make something completely different." Then once you have that high-level view, that make sense to you and you've gone through lots and lots of iterations, then you can start honing in on the high-fidelity details. CHARLES: This conversation makes me feel like there's definite poverty in our processes as software developers in the sense that what I'm hearing is that these things are just taken for granted, that these are the activities that you're going to be doing as part of design and what are we doing if not design. Each of us has these experiences that are kind of these one off things where we actually experience this creative space. It seemed very special and it seemed revelatory but really, it almost sounds like you need to make sure that it's something that you're revisiting again and again and you're doing it as integrated with your work. But I feel like it would be hard to pitch that -- JEFFREY: That's not understood to be part of the software design process. CHARLES: Maybe that's a little bit of what happens when we've put together our design documents -- ROBERT: Yeah, it just occurred to me like looking back through my history of how I landed to be a software developer, I actually wanted to be a designer first. That's why I was in digital design and stuff. I rejected design so much because I thought it was super... What's the word I'm looking for? Flighty? CHARLES: Wishy-washy? Non-committal? [Laughter] ROBERT: Anybody could walk in and say, "I don't like that design. It looks ugly. It doesn't look good," and I thought I was going to programming because it was more black and white like, "This is the right solution. That's not the right solution." As I've worked my way into my career, I actually realized they're actually really similar because you're designing software and software is abstract. As much as you want to try to think about as it's not, you start to develop this mental picture of the programs that you're developing. You're designing these things. It's just a different form of design and design is problem solving. CHARLES: In digital design, it's not artwork. It is measured by some quality. It's just hard to put your finger on but there is some kind of external measure of, "Does this fit the purpose for me, which it was made?" JEFFREY: "Does this solve the problem?" Usually, there are some expectations just like there are software of, "There's been best practices established. Did you stick to those best practices? And if you didn't, Why not?" Because sometimes there is a good reason to break those best practices. CHARLES: Right. I wonder if it would be interesting at integrating into our process like what we think of as the ideal pull request or issue reporting or design document have. It kind of similar to some of the RFC processes out there. It's like, "What are some crazy alternatives?" Not just any alternative -- JEFFREY: I don't just need viable alternative. CHARLES: Yeah, I don't need viable. Give me the in-viable. Give me the ones that are like, "What crack are you smoking? Oh, wait a second... Hmmm..." [Laughter] ROBERT: "All right. Here we go. I got it. It's got to be powered by nuclear --" No. [Laughter] CHARLES: Curious. I'm very, very, very curious. ROBERT: Let the curiosity fly. JEFFREY: Charles, do you had a blog post recently about a dumb project you built. Would you tell us about that? CHARLES: Sometimes, it gets stressful around here. Sometimes it gets stressful in life. Sometimes you just feel stress and there are a lot of things you can do to deal with it. Everybody has their coping mechanisms. For me, one of my secret weapon coping mechanisms -- ROBERT: Not a secret anymore. CHARLES: It's not secret anymore. [Laughter] CHARLES: This also will be in the show notes but I'll go ahead and say it right now is ChillestMonkey.com. If you need to enhance your chill, you can just go to ChillestMonkey.com and I guarantee you will not be disappointed. You will feel instantaneously better. I was in the point where I was feeling a lot of stress. I don't know exactly how it came up but I was actually with Stephanie and she was like, "Charles, you just got to do something stupid. You just got to do exactly what it is that we've been talking about on this podcast. You've got to just do it and you've got to do it fast and you've got to not look back and look forward." I was like, "Oh, my God. You're right." That planted the seed in my brain but then, I think it was the next day, I was talking about ChillestMonkey.com -- go check it out. You can pause the podcast right now and go have a look at it. It's really awesome -- And I was showing it to everybody and then I think Rob were like, "Oh, my goodness. We've got to have this on the Apple TV." ROBERT: It just lands itself perfectly. CHARLES: Yeah, I was like, "Yes. Absolutely we need it on the Apple TV. We need it on the Apple Watch. I need this on my iPhone," so right that minute, I hearken back to the conversation I had with Stephanie and I was like. "This is it. This is perfect scope." I have been looking to build something in React Native. Something that I can just completely and totally throw away, something that fits in a small time box and I also have this opportunity to build this thing that I really need but if it doesn't work out, it's fantastic. I went home. I think that was that very night and started hacking on React Native and took the Chillest Monkey, then put it first into an npm package. You can include it in any React Native application. Then once that was accomplished, I went out and checked the support for Apple TV had dropped literally ten days before, maybe even less. I was like, "It's a sign and this has got to happen." It was a little bit of a slog but we were able to get it up on our Apple TV and with a remote and feel that glassy touchpad, move it over to the Chillest Monkey and open it up and just see those wisps of hair and those tranquil eyes up there and six feet across. It was just such a great feeling. But it was something that was accomplishable within a day or two. I don't think it was more than that but it serve the purpose that I was able to learn a lot about React Native. I was able to learn about packaging, shared components. ROBERT: Actually, I remember we fought about flex box and what not for a little while. CHARLES: Yeah, that's right. I learned about laying out images and kind of the best practice around there. Also, I think this is important as I got to experience a win and it felt good to be able to experience that win. It felt like if I hadn't experienced it, it probably wouldn't have been that big of a deal. But the fact that I was able, it was a low-hanging target but it felt like a big-hanging target. It's hard because there's no such thing as a free lunch but kind of there is, when it comes to little projects like this because when you see something totally stupid come together and it works exactly you wanted it to work, then it feels like you've accomplished something major. I don't know if that's some sort of brain hackery or some sort of life hack or what but that was extremely good for my internal morale. JEFFREY: This is an example of a project where, maybe you didn't get as much creative juices flowing out of it. CHARLES: No. JEFFREY: But you've got a high accomplishment and learning value, which I guess are all, I call that the iron triangle of building dumb stuff. [Laughter] JEFFREY: You can have two of the three. CHARLES: On one side is creativity, on the other side is what? JEFFREY: Accomplishment. CHARLES: -- Is accomplishment, just shipping it and then on the third is time? It's the iron sticks -- [Laughter] JEFFREY: The third side was learning. That's creativity, accomplishment and learning. You can't have all three, sorry. It doesn't work that way. CHARLES: It doesn't work that way. Okay. [Laughter] ROBERT: Then you'll take on a big project, that's how you'll get all three. CHARLES: I like that. JEFFREY: In a particular project where you're having to throw tons of ideas at the wall, You're probably going to be learning, you're probably have to be very creative but your chance of shipping here is much lower. CHARLES: It's almost a detriment. ROBERT: Yeah. CHARLES: Whereas in this case it was take something that is easily accomplishable and accomplished it. JEFFREY: And you learned something from it. CHARLES: You learn and you accomplish but you're not particularly creative. But it's still a feather in your cap. I like that. It's a good way to categorize it. ROBERT: And the accomplishment is how you define it. In this case, you actually finished this project. CHARLES: Yeah. I finished it. We have it on Apple TV. ROBERT: Yeah, and like you said, you have to do that. For me, after I did a couple of these things, actually what I just do, a newsletter has come in and somebody was like, "Look at this new thing," and I'm just going to put it off to the side and then eventually, I'll stack two or three of those things together and decide, "We're going to take all three of these things, we're going to put them all together and see what happens," and that's how GraphQL and dove in to create React app. CHARLES: That's actually a good idea. I like the random newsletter driven development -- [Laughter] CHARLES: -- You're like, "I'm going to subscribe to this newsletter. I'm going to pick one thing from each week and after I have five things, I'm going to build something with those things. ROBERT: Wait. I have a dumb app idea. We could make this just a random app idea generator. It takes ten packages and you see like -- JEFFREY: A bunch are required to build the Hackernews clone. Isn't that a classic newsletter driven development? ROBERT: Or the Reddit clone? This feel like that's the React thing -- the Reddit clone. CHARLES: Yeah. I think it's more like Frankenstein driven development like you've got GraphQL, Vue.js and I don't know... Datomic. JEFFREY: Like a GRD stack -- CHARLES: No, like a rando stack. [Laughter] CHARLES: Rando.js, that would actually be a good -- ROBERT: You hear it first. It's our new JavaScript framework. CHARLES: A stack generator. JEFFREY: It's going to be a high on learning and not so much on accomplishment. ROBERT: You won't ship anything but damn will you learn? Every week! CHARLES: What is an example then? We've talked about the one where the accomplishment. I'm wondering if there's an affinity between those sides of the triangle. What would be an example of a project that was high on creativity, low on shipping? How do you approach projects like that and then make it okay to fail? Because I think, one of the things that was great about the Chillest Monkey Native was I did get to ship it. It wasn't much but it felt great. How do you prepare yourself for those projects which are high on creativity that you're not going to ship? What is one of those look like? JEFFREY: I think those kinds of projects are usually tend to be ones where you're more comfortable with the tools already so that you do have the space to be creative and you're not having to fight against, "I don't know how to do this." The learning is already been done, at least to a point to where you're comfortable enough to go, to feel loose and creative and be able to brainstorm without having to bump up against walls over and over. CHARLES: I guess, tender love is a master of that, where it's really has a deep knowledge of Ruby and systems programming and does some fun and creative. Do I say zany? ROBERT: The Ember example of this, I think it's Alex Matchneer. Matchneer? CHARLES: Matchneer, yeah. ROBERT: Sorry, I cannot pronounce names. The Photoshops, I mean those aren't exactly -- [Laughter] ROBERT: -- Those aren't exactly programming related but -- JEFFREY: High on creativity and accomplishment, low on learning. [Laughter] ROBERT: And then the Ember Twiddle is the one that I laughed out and I was like, "Can React router do this?" CHARLES: I don't actually see that one. ROBERT: Actually, it was broken when I went to look at it but the responses were hilarious. They're like, "Actually, no. I'm happy. I can't." [Laughter] ROBERT: But he always has the Twiddle and he's like, "What about this? It's like the programming equivalent of his Photoshops. [Laughter] JEFFREY: I'm wondering if there are particular, out of those kind of three different types of dumb projects that we've identified. If there are particular types that people gravitate toward like I know that I am higher on the, "Just go, do something creative with tools you already know side," versus, "I enjoy learning," but I'm more likely to want to accomplishment up things and be loosen in my creativity. I'm wondering what the breakdown is among different engineers of those different profiles. ROBERT: It's quite interesting and I wouldn't think of myself as an explorer but I feel like I'm driven by FOMO -- fear of missing out. That's where this comes from for me. All this technology that I hear so many good things about that I haven't even done anything with. I don't even have Hello, World or anything. That's usually where I start picking things up the shelf and I'm like, "What if I did redux, Vue.js and whatever else. Let's see if we can make all these things work together." CHARLES: Do you feel like you can, at least always be kind of touching? ROBERT: Yeah. CHARLES: Pinging different areas of the ecosystem? ROBERT: Yeah, I really like to see what they're doing because they all have different takes on things. I really like what Chris Freeman said, he's like, "I feel like programming is just gaining experience points." Like it's video games where you're just going through and you're getting experience points with different things. That's kind of the approach that I've been taken for the past five months. Since I discovered this, this is just like, "Oh, that look shiny." Maybe that's also driven because of our JavaScript-type cycle or whatever you want to call it, where something new is always coming out. Somebody is always reinventing the wheel. CHARLES: Whenever I see a cool demo or whenever I read a really provocative blog post that guides my exploration a lot, which I guess those things usually are extensions of some activity like what we're talking about that someone did. A lot of really good blog posts are just like, "I've been doing this stuff for five years and I have gained an absurd level of expertise on it. Let me take you to school." I definitely love those but a lot of it is like, "Look at this cool thing that I've done." Without talking about murdering names, what is it? Hakim...? Gosh, I can't remember his last name. The guy who does Slides. Some of his demos for CSS and JavaScript animations back in the day we're just like, "Woah, someone has just revealed a huge power source," and so I want to go do it. But most of that came from him just having to play. ROBERT: Jeffrey, you've actually got my brain ticking here. I'm thinking about where people fall in applying this like just go do something dumb and it doesn't have to mean anything. This makes me starting to think, "Maybe I need to go do something really creative in Ember." JEFFREY: It's making me think that I need to go do some learning -- [Laughter] JEFFREY: -- Just try some new stuff so I have new tools to play with. ROBERT: Because that's the whole purpose of this, right? I just recognized that I am not doing very creative things and the tool sets that I am comfortable with. CHARLES: Yeah, it is harmonious. The more learning projects you do, you acquire tools and then with familiarity with those tools, engenders creativity so you can see how the process feeds in to itself. ROBERT: Now, I'm going to build something creative. CHARLES: All right everybody. There you have it. Go out, build something useless, build something creative, build something that will help you learn and acquire new tools, new techniques and take you [inaudible] and do it quickly.
Liz Baillie @_lbaillie | GitHub | Blog | Tilde Inc. Show Notes: 01:32 - Becoming a Developer 07:54 - Website Building 12:03 - Understanding Programming 17:34 - Coming to Peace with Ignorance 22:25 - Systems Programming 26:46 - Making Goals for Yourself 28:57 - Math and Programming 38:08 - Open Source Resources: Wicked Good Ember Liz Baillie: Journey to the Center of Ember Test Helpers Fibonacci Number Freewheel: Volume One by Liz Baillie The Flatiron School Skylight Impostor Syndrome Twilio Letter to a Young Haskell Enthusiast Hello, Con! OSCON Transcript: CHARLES: Hello, everybody and welcome to The Frontside Podcast Episode 57. My name is Charles Lowell. I am a developer here at The Frontside and with me is Stephanie Riera, also a developer at The Frontside. Today, we have with us Liz Baillie, who is a developer at Tilde. I am actually really excited to have Liz on the show. I saw her at Wicked Good Ember back in June of 2016 and her talk was definitely one of the more memorable ones. You come away from a conference kind of only remembering a certain number of talks that stick in your mind and as time passes, the messages may fade but some of the message just stick with you and the one I got from her talk was a feeling of empowerment that, even though I have a lot of experience, I could approach any code base and try and grapple with it and understand it. I came away thinking, "There are a lot of code bases out there that I don't understand but if I apply a certain set of techniques and a certain level of fearlessness, I will actually get there." You know, if I want to go attack something like I don't know like Kafka or something like that, I would feel better about that. That was actually a great feeling coming away from that, a feeling of great power so thank you very much for that, Liz. LIZ: Yeah, no problem. CHARLES: Why don't we start with a conversation of how you came to be a developer? Everybody's got kind of a unique path. What's yours? LIZ: Well, I went to art school and I studied comic books. I actually have a bachelor's degree in comic books. I was a cartoonist for a number of years and at some point, maybe like 10 years ago, I had a friend who was a programmer. He's a web developer. But I didn't even what's a web developer was. But I knew he worked at home and he made his own hours and he made a lot of money. It seemed like an awesome job so I was like, "How did you get into that?" And he's like, "I don't know. I just kind of mess around and figured it out." And I was like, "Uh... I don't know what that means." Like how do you start? I have no idea. I went to the bookstore and I look at the For Dummies books and I got Programming for Dummies or something and it was like Visual Basic, I think. CHARLES: All right. What year was this? LIZ: That's 2004. I guess, it was a little more than 10 years ago. But it didn't say that on the cover. It was like 'Programming' and I was like, "Oh, cool. I'll learn programming." I don't even know what the difference of languages was or anything like that. I did a couple of exercises in that book and I had no concept of how this would become a website ever. I was making 'Hello, World' and little things that spit out Fibonacci numbers or whatever. I kind of gave up on that and I was like, "I don't care. I don't mind being poor." I'm used to it so I kept being a cartoonist, putting out books and stuff. I did a little PHP and HTML type of stuff in making websites for myself in between but I don't really consider that programming. It didn't feel like programming. CHARLES: Did you ever put any of your cartoons on the web? LIZ: Oh, yeah. Google me. They're there. [Laughter] LIZ: I might have some stuff like my web comic, I'm not sure if it's still up. But I had a web comic called Freewheel, which was about this girl who runs away from home and joins a band of magical hobos. CHARLES: That sounds like a career change to programming. It was oddly prophetic. LIZ: Yeah. It's out there. Anyway, I got to a point where, long story short, I was tired of being broken for all the time and I have to figure out some way to make money that I like doing so I thought, "I would go back to school," so I went back to school. I didn't start out with computer science but I took some math and science classes and I got really into math a lot. I really enjoyed math so I started looking into what careers can I do that are math-y. Somebody said, "If you enjoy the problem solving aspects of math, you'll love computer science," so I took a Computer Science 101 class or something like that and I got really, really into it like I just killed it. I just loved it. It was awesome. But I still didn't understand how you made that a website. In the back of my mind, I was like, "We did this thing --" We learned Python in my class so there's some program we had that like move a little turtle around and do pictures or something. I was like, "I don't understand how this makes a website." CHARLES: You got to move that turtle around a lot, especially like account for the kerning in the fonts and stuff. LIZ: Yeah. I have no idea how you make that a job, like the stuff that we were doing like spitting out Fibonacci numbers and making a little adventure game or something but how does that translate into anything else. That was in 2014 and that was around the time that web development bootcamps were starting to be more of a thing. I heard about a school called the Flatiron School in New York which is right at the time and I thought, "This sounds great. In three months, they'll actually teach me how this makes a website and finally know how does this make a website?" I applied in kind of like on a lark. I don't think I'll get in, I didn't know how can I afford it or anything and I applied and I got in. I was really lucky that my stepdad help me pay for it so I don't have to worry about it. I did that in three months and then I got a job. In November 2014, my first web job and now I know how those codes make a website so here I am today. CHARLES: What a journey. LIZ: Now, I live in Portland, Oregon and I make websites. Not really, I work on web apps, I guess is more accurate. CHARLES: So you actually went straight from the Flatiron School to working at Tilde? LIZ: No. I was in New York at the time and my first job was at an ad tech company called SimpleReach and I worked there for a little over a year before I got the job at Tilde, then I move to Portland. A year ago yesterday was my first day at Tilde. CHARLES: Fantastic. Knowing that company and knowing what they do, they must have you doing some really, really fascinating stuff. LIZ: Yeah, I do a lot of typical web stuff. I work on the Ember side of our app, Skylight. I also, more recently have been working on Rails engine that's also a gem that spits out documentation automatically, which is pretty cool. CHARLES: Now, is this documentation for the product or is it just documentation for any real site? LIZ: No, it's for our products specifically but I don't think it would be very difficult to alter for someone's personal needs, other than ours. But it's basically like if someone can write a markdown document, then we'll parse it and spit it out into HTML and all these different places so that it just updates the whole documentation site around our products. CHARLES: Basically, there's an infinite amount of stuff that has to happen to make a website because there are literally so many moving parts. What's been your favorite kind of area, I'll just say the whole website building because that really is like the tip of the iceberg. The actual iceberg goes way, way, way beneath the surface. But what's your favorite location on the iceberg so far? LIZ: I kind of like the middle, I guess. I always feel bad saying it because everybody talks badly about CSS but I just don't like it. I tried it really hard. One of my resolution this year was I'm going to try really hard and I'm going to like it more. But what I like the most is whenever I get to do pure Ruby. I learned Rust in the last year or two and anytime I get to make the stuff behind the visual aspect work or kind of like meta stuff. I'm saying this and it's totally wrong but I did my first meta programming the other day or last month. The metaprogramming that I did ended up getting cut out of [inaudible] but I got to do it before it got deleted. It was pretty cool. CHARLES: That's generally how it works. Metaprogramming is the program we do that we end up hating ourselves later for but it's really fun. LIZ: Yeah, they're like, "This is cool but this is not the most efficient to do this." It's like, "I guess, we don't have to dynamically create methods based on all our filenames. CHARLES: As far as the CSS goes, I actually see CSS like raw kale. It's actually really good for you, if you like to it eat in large quantities and it's like fantastic but it's not always the most pleasant going down. LIZ: It tastes bad. It has a terrible feel. It's like eating rubber. I am really lucky, though that I worked with a couple of people who are incredible at CSS and when I get to pair with them, it's like watching magic happen. CHARLES: Yeah, you realized, for all its quirks and strange ways that you approach it, is an outlier but it is kind of a fully-formed programming model that has a lot of depth and a lot of people have really, really generated some pretty neat abstractions and ways of dealing with CSS. But it is like, "I just want to fix this one thing," and it's basically a sea of things that I have no idea how to navigate. LIZ: It's one of those things. I always think it's funny, anyway that I come from a visual art background but the thing I like about programming is anything visual. CHARLES: That is actually really is fascinating. LIZ: Yeah, when they hired me here they're like, "You're going to be really good at design," and I'm like, "I just want to do programming." CHARLES: Like never the temptation, like this is just because you've actually kind of drank your fill of that in a past life? LIZ: I think I've talked to my coworker, Kristen about this because she actually has a design background and we paired together all the time. She's one of the people that I was talking about who are geniuses at CSS. She's a genius at it. She has a design background. We've talked about this how art and design are kind of different, like the brain stuff that I use to make a comic is really different from designing a book cover or designing an experience. It's all part of the art side of the brain but it's different compartments of the art side of the brain. I don't really have a design background as much as I have like a narrative and a drawing background. STEPHANIE: That and your interest for math that probably has a factor. LIZ: Yeah. STEPHANIE: Going back to your journey, I wanted to ask about it seems like it took you awhile to knock on different doors and finally feel like, "Now, I understand. How do I work with what I have to create a website?" We have similar backgrounds in that. We didn't start off in programming and I also went through a code boot camp. But mine was a little different where when I finish, I didn't really feel I understood what programming really was. I still felt like I understood a primitive level like just building something, just a 'Hello, World' using HTML CSS. When I finished, it took me a year and a half to actually get a full time programming job, like a legit job. Before that, I was scrambling doing three part time jobs and lots of WordPress grunt work. Even though I thought it was actual experience, it was enough experience but I feel like a lot of the programming concepts that I've had to learn and just basic functional programming, I've learned it on the job. I don't yet feel like I am a legit 'real programmer'. We were talking about the Pinocchio thing like, "I'm a real boy." But I want to be a real programmer. [Laughter] STEPHANIE: What I'm curious about is at what point did that happen? When did that click and when did you stop having -- I'm sure at some point you had -- impostor syndrome? When did that just evaporate and you're okay? LIZ: I still have impostor syndrome all the time. It's weird that it's like I have a sense of, "Oh, I can figure anything out." At this point, I know who to ask or where to look and I could figure anything out if I really wanted to. But I also feel like everyone else is better than me. I get impostor syndrome in that sense, not that I'm not a programmer but that everyone else is better than me. When did I start feeling like I was a real programmer? Definitely not at my first job. When I started my first job at SimpleReach in November 2014, I had two months in between bootcamp and the job. In that time, I made some weird little apps but nothing super serious. I made an app that I use the Twilio API to anonymously text Seal lyrics to people. It sends either lyrics from Kiss From A Rose or a fact about Kiss From A Rose. You can choose which one. I made stuff like that. CHARLES: [Singing in the tune of Kiss From A Rose] There's was so much in app can tell you so much it can touch. Okay, I'll stop. I'll stop right there. I promise. LIZ: Yeah, so I did stuff like that and I sort of wrote my own crowdfunding to go to RubyConf because I gotten an opportunity scholarship ticket that year. But I couldn't afford to go otherwise. I did a little crowdfunding thing but I did little things like that. I didn't really feel like I understood everything so I was looking on other people's code and forking stuff to make all that happen. Then I got my job and it was small-ish start up at the time and they didn't have a whole lot of on-boarding at all. It's kind of like I showed up, they gave me a computer and it took me three or four days to get their app running locally. It was just a lot of leaving me to my own devices a lot of the time in the beginning and I was kind of like, "I don't know what I'm doing. What do I do?" It took a while. As the company matured and as I matured as a programmer, they kind of develop a little more infrastructure, I guess for supporting junior engineers. As time went on, I became better and they became better at mentoring me. I don't know when I felt like a real programmer, probably sometime in the middle of that job. I gave my first technical talk, I guess or conference talk at EmberConf in 2015. I gave a lightning talk at the behest of the Leah who is now my boss. It was a five-minute talk on why testing an Ember sucked at that time. It sucked for me to learn and it was really hard. I wanted to learn it but it was really hard. Then after that, people started talking to me. They came up to me after and they are like, "Oh, my God. Blah-blah-blah." I was like, "I don't know half the stuff these people are saying. I don't understand what you're talking about." I'm going to smile and nod. But maybe a little bit after that, I kind of started feeling more that I could solve problems. I think public speaking actually helped me a lot with that like when I realized that I had something to say and that people want to hear it, then I could help other people feel empowered to learn stuff, I think that was part of it as well. CHARLES: Yeah, I really like that. Obviously, I'm going to push back a little bit on Stephanie, just in terms of the day-to-day. You definitely deliver daily as a programmer so you can look at that. You've mentioned this at the very beginning of your answer and it almost really sounds like what you came to be was more of a kind of a peace with the things that you didn't know, rather than feeling confident about the things that you did. You said something and I'm going to paraphrase it but it's like, "I got to the point where I became sure that I would be able to figure it out." Or, "I had strategies for being able to figure it out." Maybe we can unpack that a little bit because I feel that's actually very, very important and that's a skill that's important to have at any level of experience in your career, whether it's one year or whether it's 20. Certainly, that message when I saw you speak that's something that I took away as a very experienced developer. I felt actually empowered by it. What are some of those mechanisms to feel at peace with your own ignorance? LIZ: I think part of the problem for me, I started learning how to program before I went to dev bootcamp or whatever, that I was really good at stuff. I actually think that was a problem because I was used to succeeding immediately or like always doing everything right so it's hard when you start learning something and you don't realize when you first start learning programming and it's not supposed to work immediately, like you're starting with something that's broken and you're making it work. CHARLES: Right. In fact, 99% of the experience is like every time I look at a piece of software, I'm like, "Someone sat with the broken version of this for a year and then it work and that's what I got." They got to live with the working version for two seconds before it came to me and they spent the rest of the time, totally broken. LIZ: Yeah, totally. It's hard when you're used to creating something from scratch like doing comic books and like writing stories and stuff. It's never broken it's just blank and then you add to it so I'm used to that sort of workflow. Then I started in this new field where Rails is new or whatever then it's just errors as far as the eye can see until you fix it, until you configure it, you made it work. It's hard to change your mindset into that. It's easy to feel like a failure when all you see is errors and you don't know that that's normal. I helped a couple of my friends to learn to program and I think the biggest hurdle is just mentally overcoming that it's not you, you're not a failure. It's just that everything's broken until it's done. STEPHANIE: I can definitely relate to that. I was always one of those overachievers, straight A, AP class. I'm not even kidding. In my high school, they called me Hermione, which for those that don't know, that's the girl from Harry Potter. It's like you take it really personally when you feel like you're a failure. You feel like you can't deliver, you don't pull your own weight. For me, it's actually so overbearing that it can even inhibit you from doing things like public speaking or other activities. But one of the reasons why I do like to teach whenever I can is because that's when you realize, "I do know a lot of things," like how to do stuff on Git and just basic things that you don't even think twice about. I volunteered for this these high school girls and no one really gave me any instructions and I just rolled out of bed for this thing and just have them build a basic cute little web page with their picture and this and that. I had to really think hard to how do I put just a regular image tag and I had to peel back all the old layers of stuff that I don't do anymore. You don't think about those kind of things in Ember or JavaScript frameworks. I caught myself in keep on saying dom and this and that and they were like, "What is a dom?" And I'm like, "Urghh." But then I realized, I do have all this context, I guess I don't appreciate it or something. LIZ: I think talking to beginners when you're slightly above beginner-level in helping other fresh beginners is one of the best things for you as a new developer because you realized, you're like, "I actually know stuff." STEPHANIE: Yeah, that's usually the type of advice I like to give to other aspiring junior programmers. I also wanted to ask about it seems like now you're going through something similar because you tweeted or you're asking about systems programming. What's that like? LIZ: I'll start at the beginning. When I started at Tilde about a year ago, I knew that we use Rust, which is a systems programming language, a lower level language than Ruby or JavaScript. We use it for some aspects of our stacks. I thought, "That's really cool. I want to get into that nitty-gritty type of stuff so how do I learned that?” I started learning Rust but I didn't really know how to apply that knowledge. I wrote like a little adventure game in Rust and it was almost exactly the same as when I first started learning about web development, it's similar to how does this become a website, instead of like, "How does this become a computer thing?" I don't even know what systems programming is but I hear Rust is a systems programming language so I want to learn that stuff, like what is that stuff? A couple months ago, I think it was, I tweeted like, "Anybody have any probably three systems programming resources so I could learn more about systems programming?" And I got huge amount of responses. Everybody was super kind and helpful but a third of the responses were like, "Well, what kind of systems programming?" And I was like, "I..." [Laughter] CHARLES: "The kind that happens on a system?" [Laughter] LIZ: I don't know. It was kind of the same thing. I think I used this metaphor earlier but it's similar to when I first started learning programming it was like I was standing at the front of a forest and I knew that the stuff I want is in the forest but I don't even know what a tree is, you know what I mean? Eventually, I learned what a tree was then I learned what a map was and I learned how to get through that forest. But then in the middle of that forest, I was like, "Oh, there's a tunnel," like there's another stuff. "I want to get on to this tunnel," but I don't know anything about living underground, you know what I mean? Like, "What do I need? What even is there?" I have no idea so that's kind of how I feel about systems programming. At the moment, I'm trying to go into this tunnel but can I breathe down there? I don't know. Where does it lead? CHARLES: I feel like at that point when you're about to enter into the tunnel, can you intentionally apply filters for information that at that point is not useful like the difference between a stalactite and stalagmite is not useful when you haven't even gone into the cave yet and you're just like, "How do I actually just get down there with a flashlight?" How do you go about deciding which information is useful and which is not at your particular stage? Because obviously, it's all going to be useful at some point but at what point it becomes useful and what point do you just catalog it and put it for later? I feel like that's very, very hard thing to do. Do you feel like you're able to do that? LIZ: I'm not sure. I think I said this earlier but I feel like I can figure most things out at this point like if I really want to. One of the things I learned just from talking to people on Twitter about systems programming is like, "Oh, some examples of systems programming are operating system," or like a browser engine because I'm still learning Rust and I gotten to write as much lately but I know that there is servo which I believe is a browser rendering engine written in Rust, it's something like that. CHARLES: Supposedly it's going to powering Firefox at some point. LIZ: Yeah, stuff like that, I think is really interesting but now I know a little more about what to look at in terms of as far as I understand, there is probably an infinite amount of different kinds of systems: operating systems is one, maybe a browser engine is another. I can't remember the others but I'm sure people tweeted it out to me. STEPHANIE: I feel like we touched on something which is it can get overwhelming when you're starting off in something new. Trying to understand what you don't know that you don't know. LIZ: Yeah, that's the hardest thing. STEPHANIE: How can you make tangible goal marks for yourself if you don't even know what you don't know? When I first started off, when I would pair with someone that was more advanced, I remember having a realization that every time I would look for an add-on or I'm looking at someone's repo, I would take my time to read everything about it, all of the Ember documentation and I need to know everything. Then later I realized that is totally not the case. Like Charles said, people develop this filter for noise and only focusing on not the entire tool box but that one tool that they need for that one specific thing that they're doing and I realized it only when I was pairing with people and seeing that. They go to this repo, skim it, "No, this is not what we need. Let's go to the next one. Let's try to find a method that what we need," and then they would just search on the page. "Oh, this looks kind of similar. Let's plug this in," and I'm just like, "What? You can do this? You can just copy/paste someone else's stuff?" and it was amazing. But when you're starting out, you don't know all of these things and unfortunately, kind of waste a lot of time thinking that you need to know everything and you don't. CHARLES: Yeah, Cheating is totally a virtue in so many cases. [Laughter] LIZ: Totally, for sure. CHARLES: Just being like, "I don't need to understand this," but I just know that it works. You pushed at what point that happens like further and further back but that boundary of understanding is just simply always going to be there. No matter where you are, that kind of veil of ignorance, you can push it out but it's just can be further away. I am actually curious, you mentioned you got really into math, this is when you went back to school. What drew you to that and how have you applied, if you've applied? Have you found it to be an asset in your development career? LIZ: For sure. When I first went back to school, it was with the idea that this is totally different now, obviously. I thought I might become a veterinarian -- CHARLES: You need a lot of math for that, right? LIZ: Well, it's like a lot in biology and there's a lot of math and science and stuff. I had to take a bunch of science classes and take biology and chemistry so that involved taking some pre-calculus and calculus and more calculus. What I realized, though was that I hated biology and chemistry but I love the math that I was learning. I loved the process of problem solving and just figuring out puzzles. When you get into calculus, how you solve problems, they're similar to how you solve problems in programming where you have sort of a framework like I have this certain language which would be the different theorems or whatever in math and you can just pick and choose which ones will fit your problem and if you're taking a calculus test, you could be sitting next to the same person and you might come to the same answer in different ways so it's similar in programming where you have all of this documentation, you have these languages, you have use other frameworks and you can solve the same problem in a million different ways. But in terms of how people talk about needing math for programming, I don't necessarily think you need math for programming but if you already like math, it's definitely sort of a happy path, I guess because you get the same joy out of programming that you get out at solving calculus problem. But if you don't like calculus, it's okay. I don't think it's necessary. CHARLES: One of my favorite blog posts of all time is this letter to young Haskeller, I don't know if any of you guys have ever read that. It's fantastic and it's an experienced person in the Haskell community talking to someone who's just coming in and it's incredibly empathetic and wonderful. I think it's a message that needs to be heard more generally. I think it's ironic coming out of the Haskell community as it does because they definitely have a reputation for being a little bit salty and a little bit exclusive. But it's actually a very inclusive message. One of the great points they make is they say we've got the whole equation reversed. It shouldn't be, "Math is hard, therefore programming is hard." It should be, "Programming can be really fun, therefore math on which programming is based, can also be really fun." You can go both ways. If you find math fun, you can find programming fun and if you find programming fun first, you can later go and have fun with math. You can pick and choose which parts you want. I think it's a great message that needs to get out there. LIZ: I think it's also really, really important to note for anyone who might be listening that is getting in to programming, that is scared of math or has had a bad experience with math that it is not necessarily to love math. I think that scares a lot of people away and a lot of the stuff that people learn when they're first learning programming are math based. When I was in the Flatiron School, Some of the exercise we did in the beginning with just pure Ruby were Fibonacci sequence. They were sort of math-y and that turns a lot of people off and makes people scared. If someone is hearing this and has experienced that, don't be scared. You don't need to worry about it. But if you love math, then it's great but you don't have to. STEPHANIE: I'm one of those people that always had this mental block of like, "I'm not good at math." I was good at everything in school. I excelled at everything except math. I think a lot of it came from my struggle when I was a kid so you have this self-perpetuating thought that you aren't good at something. Every time you take a final or something, you blank out because you have this mental wall in your mind. What I found weird was I was doing the exact same thing. I was taking calculus for bio-sciences and physics too at the same time. In physics, I loved that class. It was so awesome and I realized that half the stuff I was doing was going backwards in all of my problems and it was fun for me. Eventually, I was taking a final for my calculus class and I didn't remember the equation that we needed for that class so I took out all the variables and I solved it as if it's a physics problem and I got the same answer and I was correct. I realized at that moment, if you just remove the negativity from your mind and you try to apply yourself in the same fashion as you would in something that you enjoy, you'll just forget for the moment that it's math, that it's something that you 'suck at'. You actually could do good in it and not get stuck. I realized I actually do like math when it's veiled as chemistry or physics. LIZ: I think a lot of people have that experience with math. They have a really bad experience when they're young and then they get stuck and they feel like they're just not good at it like somehow, on this subatomic level, you just can't change it or you're not good at it. It's not really true. STEPHANIE: Yeah. CHARLES: I actually love that example because it is, it's all integrated. We are constantly doing things like math without even realizing it. Actually, one of the things I love about the Montessori education is that's the way they actually teach it. They have all of the different great lessons, they want to convey to the children which is things like courtesy and grace, things like taking care of your things, things like music. But for all, I think they've got a bunch of different categories but they make sure that they always intersect with each other and you get that in surprising ways to make sure that if a child likes music, use the music as a way to introduce them to arithmetic. If they like arithmetic, use that as a way to introduce them to music. If they have things doing design, I don't want to say, interior designer or clothing design but practical life stuff and if that's something that a child really is drawn to, then they'll use that as an introduction to music or geography. There's all these parallels that are constantly there and you can ride whichever rail works for you to whatever area that you want to go. There is no set way to approach math. You literally can find a way that works for you. STEPHANIE: The subjects aren't mutually exclusive, "Because you're not good at this, probably you shouldn't become a programmer." CHARLES: It's not expected that every child will grow in one subject at the same rate that they'll grow in every other subject. They just let the children explore the area that they're interested in and let them go crazy. If they're really into art, they just let them explore and learn as much as they can and then slowly entice them and just show them the connections that art has to courtesy and grace to math to music to other things and let them see those connections and then follow them on their own. That's why they call it -- the kind of grown up in there -- the guide. It's really there. The way that they push is by showing them the connections but then using the kind of internal motivations of the children to move. I actually have some pretty strong feels on this. I feel like our education does leave a lot of people behind because there's this expectation that in every single subject, everybody will goose step forward at exactly the same rate and that's just a fable. It's not real. It's not how the human mind works. LIZ: Yeah. CHARLES: But yeah, I actually think, certainly for me and my connection to math has been helped by the fact of programming and now, later on after having done a lot of programming, so much more is interesting to me about math and I can see beauty in it, I think where I didn't see beauty in it before. STEPHANIE: For one of the projects that we've been working on, we have been doing an Ember upgrade. I basically needed to get some changes for one of the dependencies and I have no experience in open source, whatsoever. That happened for the past two weeks. I was making a lot of PRs to two different dependencies and that was my first experience with open source. It was less scary than I had imagined and I actually got a lot of great feedback from it. Now, I realized that it wasn't as hard as I thought it would be and most people are very receptive to your PRs or if you have questions about their open source because they need help, they need people to help them tackle all the issues that they have so I'm curious, do you have any advice for people that are interested in contributing to open source but they may find it daunting and they don't want to look dumb or do things the wrong way? LIZ: One of the things I've been interested in since I started learning programming is open source because I enjoy collaborative atmospheres and just the idea of a big group of people coming together to solve problems. It was something that I wanted to do since the beginning but it's super intimidating because when you think of people who are open source maintainers, at least to me in the beginning, they seemed way above me like Gods so I'm like, "How can I possibly be useful to these Gods?" At my last job, my manager was like, "I got a couple of goals for you and for your career." One of my goals was I want to contribute To Ember CLI Mirage. That was a goal. I just thought, "This is a great add-on. This is a great project and everyone uses it and I love it and I would love to contribute to that." I made it a goal but then in that in the middle of that time period, I got a job here at Tilde and I went to Portland. Shortly after that, I went to the repo and I was like, "I'm going to do this thing," because one of the reasons why I chose it as a project to contribute to is because I heard Sam is a really nice guy. One of the things was that I was really intimidated by the people maintaining projects is like, "Well, he's not intimidating." I feel okay about this so that's a good first step. The second step is let's find a thing to do so I look at all the issues on the repo and I find something super simple which is just adding in-line documentation. That's what I did and I was like, "Can I pick this up?" I was feeling super shy so I didn't even want to put it on the issues so I think I just pinged him on the Ember Slack and just like, "Can I help with this?" He's like, "Yeah, yeah. That's great," so I made a bunch of in-line documentation additions to the project and I made my first PR and it felt like such a way that it's not as scary at all as I thought it would be so I started contributing to other projects, things that just came up. Not so much like in your situation where it was a dependency I was using but more like I saw somebody tweet about it and like, "I just made this project and I think there's a bunch of typos. Can somebody just spell-check this for me?" I'll go in and do a couple of typo fixes. Another situation when I was reading through a repo because I want to learn and there's a project called intermezzOS which is Rust operating system, like a tiny operating system. I was just reading the code and I was like, "There's a couple of typos. I can fix this," and stuff like that and I found, through that experience, that open source maintainers are super happy to have you help in any way that you can, even if it's a little things. In the last couple of months, I started my own project which is like an app -- it's not an add-on or anything. I actually got my first couple of PRs from other people and other people are helping me build it. I don't think I've ever met but every time I get a PR, I feel like I won a prize. Every time someone contributes and I'm like, "Thank you." I cannot give you another -- [Laughter] LIZ: I love that you're helping me. You know, like I only have one hour a day to work on this thing so anything, anyone people can do to help me is so great. Now I have the experience of being on the other side and I can attest to the fact that most open source maintainers are incredibly stoked for any help they can get. Even if you're new, just find someone who's nice and ask them how you can help. STEPHANIE: Yeah, that was a realization that I had because I was communicating directly with this person in the Ember Slack as well. I had submitted a PR and later he was like, "Hey, while you're at it, do you mind adding in this one property that's missing?" And I'm just like, "All right. Sure." Later he offered if I wanted to become a collaborator because I was putting in so many PRs and like you said, he hasn't had the time to cut out a new version or to fix the things that you keep in your head, "Okay, I'm going to go back and fix this," and then someone else is like, "I want to fix this thing," go for it. That's the best. LIZ: Yeah, totally. It's a great way to learn more stuff too. CHARLES: I like the point about choosing a project that you know is not intimidating because unfortunately, there is a lot of negativity that happens out there. LIZ: Totally, I knew that and that was a big blocker for me, for a long time. CHARLES: Yeah but knowing that there are actual, I would like to say, a majority I don't know if that's true but it can feel like it's enclaves, just because negativity has a way of clouding everything and propagating but there are certainly areas where we put that way and it's very healthy, it's very collaborative and welcoming and making a definitive effort to first know that they're out there because if you have a negative experience, you make sure that you don't bounce off of that and then define them. I really like that, how you were deliberate about that. LIZ: Yeah, it seems like the most important thing, if you're a new programmer and they're like, "How do I get involve in open source," and your first advice is like, "Find someone who's really nice." It doesn't sound like the right advice but I think it is the right advice. CHARLES: That's because that's where you'll stick. LIZ: Yeah and you'll want to collaborate with that person and that project because you're not scared of being insulted or something. CHARLES: Well, that was fantastic. We can wrap it up. LIZ: I have two talks this year so far coming up. One is going to be in Toronto at the end of this month at a new conference called 'Hello, Con!' I built a type space adventure game in Rust and I built it side by side with the same game in Ruby so I can learn Rust by doing the same thing on both sides. I'm going to be talking about the similarities and differences and things I came across learning Rust as a Rubyist. I also have a similar talk in May at OSCON in Austin about learning Rust as a Rubyist but at a slightly different, longer talk. I did a version of it at RustConf last year. It's kind of in comic book form so it's all of drawings and it's sort of a story about going to a place called Rustlandia as a Ruby person and how you literally navigate that world, not just everything is sort of a metaphor. I'm getting that talk again in a longer form at OSCON in Austin in May. CHARLES: Well, fantastic. You have to stop by the office and come see us. LIZ: Yeah. CHARLES: But thank you so much -- LIZ: Thank you. CHARLES: -- Liz for taking the time to talk with us. This is a great conversation again. You know, I feel like I'm going to come away feeling that I've got more tools to deal, certainly with my daily struggles -- LIZ: Yeah, get pumped! CHARLES: -- In programming. Yeah. LIZ: Programming! Yeah! [Laughter] LIZ: -- One of the Mortal Kombat music comes in -- Tun-tun-tun-tun-tun-tun-tun-tun-tun... [Laughter] CHARLES: I remember actually seeing Mortal Kombat in a theater and I actually getting up and dancing in the theater and then the rest of the movie just sucked. It was like they spent the whole budget on the first 20 seconds of that movie. Anyhow, all right. That's it from The Frontside. Remember to get in touch with us at Frontside.io, if you're interested in UI that's engineered to make your UX dreams come true.
Philip Poots @pootsbook | GitHub Show Notes: 00:53 - What is Elm? 03:45 - The Essence of User Interface 07:59 - “Messages” 08:31 - Scalability 14:04 - Error Handling 18:47 - The Business Case 22:35 - Where is Elm on the curve of scalability? 28:36 - Learning From Elm 32:32 - “Whole Meal Solutions” Resources: Philip Poots: Elmber @ Wicked Good Ember 2016 Cycle.js Functional Reactive Programming Evan Czaplicki Test-driven Development (TDD) NoRedInk The Elm Mailing List Try Elm Elm Guide elmtutorial.org Elm For Beginners by James Moore Transcript: CHARLES: Hello, everybody. Welcome to The Frontside Podcast Episode 56. I am Charles Lowell, a developer here at The Frontside. With me is Jeffrey Cherewaty, also a developer here at The Frontside. JEFFREY: Hey-o! CHARLES: We're going to be talking today with Philip Poots, who is a fantastic individual, who I have known over the Twitters, over the e-mails, interacted with at conferences, seen him speak on at least one occasion and today we're actually going to be talking about the thing that I saw him speak at Wicked Good Ember last June. It was actually one of my favorite talks from that conference. It was on Elm for Ember developers. Thank you very much for being on the show, Philip. Why don't you tell us a little bit about what Elm is and how you came to find out about it and really kind of dive deeply into it? PHILIP: Yeah, sure. First of all, pleasure to be on the show. The Frontside is one of my favorite podcast, if not my favorite, given a cross-section of the Dadcast, the love of programming and balancing that with the business of programming. That's right, I'm an independent developer. I started off with Rails then got into the Ember quite early on. Last year, I think around January, that's when I started really investigating Elm in detail. It's actually a funny story how I came about because I was at Ember Amsterdam and it was a night where we had three members of the core team: we had Erik Bryn, he gave a talk, Alex Matchneer, gave a talk and Igor also came over because he's based in Europe. Alex always loves to investigate new things and one of the things he was getting into was Observables. I'd never heard about Observables at all so after the talk, I kind of pulled him aside and I asked him some very stupid questions. He was gracious enough to bear with me and to dive a little deeper into this stuff. Alex is kind of a quiet member of the core team, unless he's got his drum sticks but he's the guy that rewrote the [inaudible]. That was no mean feat because I got into Ember just before that moment and the way he managed to make that incredibly easy was fantastic so I kind of had an extra ear open to what he had to say. I went on this Observables talk. You know, you start off with React that was the framework that was using Observables the most. That brought me to Cycle.js -- CHARLES: Cycle.js? I haven't heard of that. Is Cycle.js a framework built on top of Observables? PHILIP: It is. There's a guy called André Staltz or André Medeiros but he uses Staltz as his name. It's largely based off the same principles as React. Cycle, basically one of his inspirations or at least one of the things which cycle is most like was the Elm architecture. He calls it Model-View-Intent. We have Model-View-Controller and Elm was model update view but essentially, the same principles. You know, I'm the kind of guy that likes to get stuck in, to go deep and where I started with Observables then I ended up at Elm. I started playing around with that, I started looking into it and I loved what I saw. The thing above all that really attracted me to it was the pure simplicity of what was going on. It was almost like they boiled down the UI paradigm to its essence and removed all the extra cruft and you just saw what you meant what you wanted and it gave you these, what I thought at the time, composable way to put things together. CHARLES: Can we unpack that just a little bit? I really love that idea of it boiled down to the essence of UI. I assume there are certain coordinating mechanisms that Elm employs. It's interesting to hear you say that Cycle.js has used Elm as an inspiration. I also understand that Redux is inspired also by the Elm architecture. I'm very curious, what are those kind of essential mechanics that drew your attention? PHILIP: I think you can look at it from two points of view. The first is, which I didn't actually learn until later but the first is essentially boiling down functional programming. You're decoupled, you're using functions and not only functional programming and there's a lot of arguments over this term but functional reactive programming. The idea of functional programming is stateless. Therefore, time is kind of the beast that you have to deal with. But FRP, then essentially boils time into the concept of values that can change over time so you have a reference to one value but in JavaScript that's an Observable. In Elm, in the beginning, when I was getting into it, that was signals. That's not all kind of hidden underneath so you don't really need to get over that conceptual hurdle anymore. Then the boiling down to the essence, I guess that's more from a code point of view with Ember. Especially at the beginning, there were a ton of different concepts that were thrown at you to begin with. It was billed as an MVC framework. It was sold as an MVC framework but you had helpers in there, you had components, you had views model controller. You had this cluster of things. You could see MVC in there but there were enough things surrounding it to kind of think, "Where does this piece of code go? Where does that piece of code go? Where should I do this?" CHARLES: There are a lot of blog posts trying to explain what exactly is the view, is the template the view? Is the controller the view? What's the difference between a view and a controller? PHILIP: The way I think about frameworks is they give you buckets to put your code. The buckets are kind of all connected together. I'm thinking at a really simplistic level. I need to write this feature, I need this bit of code, I need this bit of logic, where do I stick it? A framework says you should stick it here or you should stick it there. It's solves the need of having to think about the broader architecture and how things interact because those things have been solved for you. Now with Elm, it was just so straightforward to say, "This goes here: all the data, the model. It lives in this record, which is equivalent to a JavaScript object, we can say on a simple level. Anything you need to do with state, happens there. Then you've got the view and the view is simply a function, which takes that data structure and then you tell it how to render it to HTML and then all of the action, everything that happens in your application is also defined in one place in your update function. That's it, like no more than you need and no less than you need. CHARLES: Yeah, I love that. I feel like it is very much as 'data down, actions up' kind of boiled down to the essence. It's almost better at that paradigm than Ember is in itself, with having your view as just a function. Your state transition or your update is just a function. Then your model is just unadorned data. That's all it is. PHILIP: The type system as well in Elm, also made it really straightforward. My model is a record. That's it. My view is HTML in the beginning but then it moves to HTML, which contains messages. Essentially, I've got HTML and in the HTML, there are events or actions and those actions will send messages and really straight forward. Then the update function, I take in the current model, I take in the message and I decide via pattern matching what I want to do. There are a few extra bits and pieces around there but that's the essence of it. CHARLES: Now, when you say messages, I'm thinking this is a way of declaring what actions you will take when certain HTML elements, like events happen, like you declaratively mapping an event to the dispatch of an action. PHILIP: That's correct, with any extra data so a message is itself. Like the actions hash in Ember essentially and along with the parameters for that action, which would passing through the HTML. CHARLES: Some might argue that if something is simple to get something working, I can have a pure function that's a view, I can have a simple data structure which is my model. I can have a pure function which is my update or my state transition, how I change my data or affect changes to the model. Some might say, "That's very simple," but simple is great for simple cases. But then there's the question of scalability. As my application becomes more complex and has the interactions become more complex, does that simple paradigm actually scale? What has been your experience there? PHILIP: Straightforward answer -- it's a learning curve to scale. Why? Because it's so new and also because the things that you would have reached for in the past aren't available to you. When I think of Ember scaling, the scale is built into the framework. You need another component, you just add another component. You need another model, you add another model. There's a clear story or there's a clear way that you deliver a new feature. I think that's a fantastic aspect of Ember that you also say no. I can remember being at Wicked Good Ember and just realizing how many of the people that were speaking and how many of the people that I was talking to work for bigger companies -- Heroku Dashboard, you had LinkedIn and the fact that Ember scales across a large team size, it's a real testament to Ember, which is slightly different for the reason why I got into Ember. Also you know I had a few issues just on a side project with a friend, the pace of Elm's change meant that when you only have a limited time to devote to the project, then you don't want to be spending that time for going out where you should be in terms of the upgrade process, etcetera. That's a known path and I think that's a really clear advantage of Ember over Elm. The path in Elm is not as well-fleshed out and there's a bit of tension between Elm as a single page application and then Elm as an application that you stick on every page of your server generated app, for example. The main people who use Elm NoRedInk could employ Evan Czaplicki, the creator of Elm, there up at the minute is a Rails up. I think in Elm up for every page, I believe that certainly was the case, whether that's [inaudible], I don't know. It's not an area where it's like here's the path, go for it because the scalability of Elm, everyone came into Elm thinking, "I know react. I know Ember, the components system, 'data down, actions up'," and in React, you're really encouraged to make a component for the smallest thing on the page as a component. Then you have container components, you make your way out of the onion skin and hope you don't cry on the way. [Laughter] PHILIP: But the thing about Elm was everyone jumped in and tried to do it this way. I certainly got the impression when I was beginning that the Elm architecture was infinitely Nestable Russian dolls. It is in a way but the difficulty is then passing into component communication between parent and child and people had been figuring it on a weird ways with signals to do it but then became obsolete. The main encouragement is basically go so far with a single component that you can and then once you had problems, try not to create new components or new bits of UI but to extract the bits you need into modules. This is actually one of the things that really attracted me to Elm is that you're encouraged to lean on your programming skill set, rather than learning a whole new frameworks way of doing things so that the things that work in functional programming will work in Elm. But that's also a down side because all of a sudden, you have to exercise your programming chops. Let's be honest, a lot of the stuff we're building we're like gluing things together. We're not thinking up new architectures or ways of doing things. That's definitely a learning curve and that's definitely a struggle or something that I find difficult. CHARLES: Yeah, I feel like that touches a lot on the messages you get from the FP community. I know certainly in the interactions I've had with the Clojure community, they're very big on that and it's like, "Let's have very powerful primitives." They have that term 'decomplecting', like let's get at the core. It's like understand that essence so if we can compose and we can enable composition of these low level functions and allow you to compose the data, then you don't have to worry so much about everything else. There will be a way. I think the counterpoint to that is that you end up with a lot of different ways because there are a lot of different ways that you can compose a very small set of primitives. PHILIP: Yes, that's right. But I think one of the advantages of Elm over maybe other FP languages and this is where the similarity with Ember comes in as well. It tries very much to cement simple convention but not only by convention but actually baked into the language so people that are coming from more powerful functional programming languages often come to Elm and think, "Why can't I do that? Why do I have to write all this boilerplate code," and the reason is because then it's not going to be simple enough anymore for people to use it. Also the goal of Elm, which is this long term maintainability of large code bases, you kind of shoot yourself in the foot a bit just like you said, Charles. CHARLES: Yeah, I think that's actually a great point. It's actually one of the things that was most memorable, I think about your talk at Wicked Good was -- just a quick anecdote. I remember in 2007, when iPhone came out, I had an iPhone and my father in law came over from Finland and this is a guy who was a Nokia partisan. I mean, part of Finnish pride was everybody own -- JEFFREY: National phone of Finland. CHARLES: Yeah, exactly. Everybody owned the Nokia mobile phone. He came over, He visited me and I had an iPhone and he was like, "Can I see it?" He took it, he had the swipe to unlock and he swiped it and it unlocked and all the app icons just kind of came right into the screen and he was like, "I want one." [Laughter] CHARLES: That was all it took. I've never seen a sales process so utterly complete and so rapid in its realization. For me, I think that moment when I saw you talk was when you made a mistake like where you were trying to match against an improper attribute on the model inside the update function. The first thing that happened was that Elm caught it before you could even compile your program and the error was just beautiful. It put its finger right on and it's like you need to fix this right here so there's much tied up into that because I feel like it addresses a lot of the learning curve problems that we have in Ember. PHILIP: I don't think that's Ember specific -- CHARLES: No, it's not, it's -- PHILIP: That's the JavaScript thing, isn't it? CHARLES: Yeah. PHILIP: In many ways, I think of JavaScript as a very low level substrate. It's like sand, it's very granular and it's very hard to put together well without falling apart, whereas in Elm gives you bigger blocks, so to speak but it also defines a way through the type system where if you don't put those blocks together in the right way, it's going to tell you. That's why despite some of the ignorance of how best to do Elm apps, that's why people continue to use it because it gives them this delightful experience. CHARLES: Yeah, it was fantastic where when you fail, it picks you up, dust you off and sets you right back on the right track. I think one, that's just a freaking awesome feature and I think also two, the thing that struck me when I saw that was like, "Wow, this community has a different focus than other FP communities that I've come into contact with because I have encountered that exact same error message in Haskell and it left me puzzling and wondering what to do. It's like, "No instance for type class blah-blah-blah for class blah." Then if you're an experienced Haskeller, it does point right to the problem in the same way that like you've learn the parse Perl stack traces. You know, you see a Perl stack trace and you understand it. But they could have gone that way with Elm but the other thing that it demonstrated, it has kind of a different focus there. PHILIP: Absolutely and that really comes down to Evan Czaplicki, the creator of Elm. I was able to get over to London in October a couple of years ago or a year and a half ago now to do a workshop with him at the Code Mesh Conference. You know, just seeing him teach this stuff and saying go into this and talk about the things in a bit more detail, it was very clear. First of all that he'd had a negative experience picking up Haskell, I think it was and he just thought, it doesn't have to be this hard. The things aren't actually that hard. It's the way that we're explaining them that makes it hard. The things that are actually under the surface is really simple. He has a blanket ban on this kind of technical jargon. In the Elm community, he prefers to get things really straightforward names. I think he said to me that one of his thesis advisors or his university professor said, "Evan, that's what you get when you put a usability specialist into a programming language creator's shoes," that he does have this focus where he understands the benefits of static-type systems but he also deeply cares about the experience of not only picking up the language and learning it but also the day you solve it and that's something that just shines through. I think even if Elm never makes it into the pantheon of great programming languages like that in itself and the influence of that had already on other communities, this is fantastic. It's the tide that lifts all boats in many ways and we all benefit from that. JEFFREY: We kind of touch on this a little bit earlier. We've been talking about the ergonomics of being an engineer working with Elm or Ember. What about the business case? We've mentioned how Ember has prevent a scale fairly well in large organizations. What's Elm's path to being able to do that and where is the niche to that it fits in right now? PHILIP: I definitely think Ember comes out of Apple, I believe with sprite core. That's where it started and it's interesting to see that that's also where it's gone in terms of the focuses and making it easy to build these rich applications. I think also that Elm has a similar genesis in the sense that Evan, I believe he did an internship at Microsoft and one at Google and I think there's a conference talk as well from large JS or elm lock or something, millions of lines of code. It's definitely gunning for the CM area which is applications which are large and hairy and trying to make the maintainability a lot better by bringing the strengths of the static type system to bear and bring the simplicity that that enables. That means that the learning curve maybe is a little sharper at the beginning, in a similar way, also that Ember was and is. But then you should reach this point where the maintainability of the app outweighs the time spent in learning. I think about it a bit like test driven development. I remember back when I was doing Rails and DHH had baked TDD into the Rails itself and there was the years of the testing discussions whether to test all the time, test everything, 100% coverage or even full circle tests are a waste of time. but it's a similar philosophy in that if the tests are doing what they should be doing, which is giving you great feedback, the time it takes to get up to speed in testing, the time it takes to set up testing, the time it takes to write the tests, they pay off further down the line and that's not music to the ears of the people who want to get something into production immediately. But it's definitely music to the ears of people who will be spending a long time on maintenance in an application. That NoRedInk application is huge. They have millions of users. They've build software for teaching grammar and skills in the US and they talk all the time about the benefits of Elm. Mainly, in the sense of confidence, I have the confidence to go in and change this code. All of bits or majorities of our code bits, which are things that we've rather not like to touch. We kind of section them off. If a feature request comes in and instead of saying, "Yeah, we can do that," you try and slowly push it out the door. Elm is supposed to give you then the confidence to be able to go to any part of your code base and to change it without the fear that you're going to break something or break everything because the type system, the compiler will tell you, "You change this type, you change the signature of this function, here's where it's broken and as soon as you fix the compiler errors, functionally it works, you'd probably have a few tests to test the actual business logic of it." Probably not so with technical stuff but it's a huge time saver. That's where we want to be as developers and our relationship with our tools. We don't want to be fearing our tools. We don't want to be anxious every time we open our editor. We don't want to fear the feature requests coming in. We want to be in control of our environment and we want to be able to deliver the business value. I think that's certainly the promise of Elm. That's certainly where it wants to be. CHARLES: I love that. It sounds like that is where they want to shoot for is these big applications and they do want to scale massively. Let me ask the question then. Where is Elm do you think right now today on that curve of scalability? PHILIP: To put it in Ember terms, that's maybe the best way I can describe it. If anyone remembers the early days of Ember, I definitely feel that they're in 0.9 approaching one rather than later than that. One of the things as well, I think it's important to note is that Evan is not rushed. It just blew my mind when I heard him speak about it and it wasn't anything big or anything but just the whole kind of tenor of the conversation of the way he was teaching of how he was talking about Elm was so in contrast to the JavaScript type machine. In contrast to a new framework every month, he was like, "If Elm is going to be around in 10 years, then this is the decision that I'm going to make." In that sense, where is Elm in that journey. I'd say it's still pretty early on. I'd say also, Evan is really focused on use cases so if there's not a use case for something, there's no reason to add it. This is actually quite frustrating maybe for people who are coming out of JavaScript ecosystem and they think of a feature that they want and they submit a pull request and it just gets closed or it get set, is there use case for this? Why do you need it? Is this generalizable to everyone? Will it make sense to add this to the language? Can you do it in a library? Then figured out later, can you do it in JavaScript with port, instead of having to bring it into Elm? He's definitely building it very slowly but that may sound like a down side right now but the upside is we're going to come up with something at the end, hopefully that is battle-tested, that fits the use case and a good example of that was the URL handling at the navigation. It didn't live in Elm. Do we actually need it because he's building single page applications? Can you give me your experiences? There are a couple of libraries also built. Then the things from those libraries were then taken a little bit like Ember as well, where you use the add-on ecosystem to try new things and then things may get brought into core or make it kind of the official stamp of approval. But it's a lot slower and not as committee-based. Evan is the benevolent dictator for life. CHARLES: I really like that approach. I don't withhold my opinion from the benevolent dictator versus kind of the oligarchy that you see elsewhere. But I was thinking as you were describing it that maybe the framework really, and this is me, I'm kind of a little bit of a tree nerd, mostly in and around the trees that grow in central Texas but I love trees. Unfortunately, it sounds like oak or maybe redwood would have been like a more appropriate name because those are very slow growing, have a very hard wood, whereas in Elm, it's actually fairly short lived and has a softer wood. JEFFREY: And you also don't want cedar because that will come back and bite you -- highly allergic and toxic to humans. [Laughter] CHARLES: Right. Named Elm but slow growing hardwood that you can build a house out of. But I like that. It's so important to get things right the first time because that's where you realize exponential gains. Unfortunately, there is no substitute for exponential gains, rather than deep thought. At least that's been my experience. You can realize gigantic short term gains with taking cut corners but in terms of long term exponential, ultimately those will taper off. PHILIP: I think as well, what we've seen in Ember's experience is the decisions that you take to open up a private API or not to make something private. They limit you because all of a sudden, everyone starts using those APIs. A year down the line, you realize, "Oh, hang on. We don't want to design it in this way. We want to do it in that way," but given the commitment that Ember has to stability, you have to deal with that. Sometimes, the dealing with that actually takes more energy, more time, more effort in education, in actual code, in maintaining two branches, all of that kind of stuff then actually takes away from the time that you could have had to think up better solutions and it's a danger, you know? It's almost like as soon as you throw this into the water that it becomes firm. As soon as you throw it into the crowd, the crowd will take it and they will use it and your chance to change it or mold it will have gone. Evan talks a lot with language designers and at least I got that impression that he opens up channels of communication with people who've done this before and he thinks very deeply. He's not closed off to the idea of adding more powerful features to Elm but he would only do it when it's right to do it. It's not just is this feature right but is this the right time for this feature? That as well is just the kind of mind blowing like who thinks about this stuff, who cares about this stuff? CHARLES: Yeah, because certain features become accessible once other features are in place. I feel like it would have been as feasible to have an entire compiled language as an acceptable option before people started transpiling CoffeeScript and then with Babel, transpiling future versions of JavaScript back into JavaScript. If those intermediate steps happened, if CoffeeScript and Babel would have happened, would people be as receptive to things like Elm or things like PureScript? it's definitely there is kind of the zeitgeist of the development community really informs what features are even possible or appropriate tomorrow, even though they might have some ejectively sound qualities out to them. JEFFREY: They want to make sure they're not too far ahead of their time. CHARLES: Right, exactly. Folks got to be ready for it. Then the other thing that I wanted to ask was given the trajectory of where Elm is, some people are using it in production. You said it was around like a 0.9, 1.0. if we are actually quite happy with our current code bases, whether we're using Ember, whether using React, whether using some other framework, what are things that you can learn from Elm that you can bring home with you to your own town, wherever that is to use and make your own life better, as you develop code, to kind of increase that confidence, as you say? PHILIP: This is very relevant to my suggestion because in the past couple of months, I haven't had a chance to do any Elm but what I find is after kind of my deep dive, I've come out the other end and I feel like I've come out a better programmer. I feel like Elm is simple enough conceptually to learn quicker, faster, at least the basics more than maybe even to JavaScript framework. I think that the things that you will learn in that process are tremendously beneficial, even if you don't ever end up using Elm in a professional capacity. The thing I think that grabbed me most is because of the way the type system is, you have to deal with every error and this is something when writing Ruby or when writing Ember, you always focus on the happy path. Then once you've got the happy path in place, often the business will say, "That's done. Let's move on." Your better instincts say, "No, we haven't actually made this robust enough." But sometimes the value is also not the great in kind of shoring up all those cases. But what Elm does because of the way it's build is it forces you to deal with failure. The classic no runtime errors in production, the reason that is it may have taken you a bit longer to write your code in Elm. But you come out of it with a confidence that if something explodes, it's either something incredibly rare or it was JavaScript and not Elm. Just the forcing like, "How can I think better about how this code can fail," means a bit of extra mental effort when you do it in a language that's not Elm but it's definitely something that's worth doing, if you want to build good software. Another thing related to the failure stuff is maybe the idea of what's called an abstract data type or a union type or an enum, like the fact that you can encode different states in one object, that you deal with all of those states in a cohesive way. One of the things actually, I think it was Jack Franklin of JavaScript Playground who brought this back from Elm into Java Script was when you have a screen, you want to load some data via Ajax or Fetch, instead of just loading the page, sending off the request, getting the request back, deciding what to do with it, it's like, "Why don't we encode those states in our code." We have the waiting state, we have the find state but we have find but it was an error, we have find and it worked, we have find and it actually has no values. Instead of just dealing with those on the fly, if statements like why don't you create an object that encodes the states that your UI can be in and then you react and the UI will know what to do and you write the code to do that. But in some ways, at least to me that's a lot better than kind of hit and hope when we got an error. That's, "If do this, if do that, let's nest some callbacks or let's chance some promises." The sheer act of taking a step back and this is something Elm forces you to do because the way it is, like thinking about the domain and thinking about how best to model the domain. To be honest, it's probably just good programming practice but how often do we not do it? CHARLES: Absolutely. It seems so obvious, except it's only obvious when you're put into a system that you can't slide on it. What you're describing reminds me of this concept that I came across in the Haskell community. They talk about wholemeal solutions versus piecemeal solutions. Most of the time, they talk about wholemeal solutions as kind of finding the most general set of abstractions that solve a problem for all cases. But I hadn't applied it to this concept of that all the potential states kind of envisioning your application. As you execute your code, you'll only come across one state of the time but kind of thinking of your code as all of these sort of quantum states of your application existing at once and you need to account for them all and draw that whole picture. PHILIP: This is a really stupid analogy but I don't know if anyone's played Age of Empires or any other kind of war-based game where you have a map, you've got the fog of war and you can only see a few meters around you and you have to send out your scouts and you need to find them -- CHARLES: Yeah, yeah. PHILIP: -- Different tactics. You shore up your base and you concentrate on where you are, you send your scouts into the far ends of the map. In terms of seeking out of the unknown, I feel like Elm forces you to know the map where is when I'm programming in JavaScript or in Ruby, it's more like, "I'm here. I need to do this. I do it," and you don't necessarily force yourself to think of all the potential states or all the potential things that can happen, the events that can happen, whereas with Elm, if you don't have those, then it will tell you that those things in place. [Laughter] CHARLES: Yeah, I love that analogy. Like actually sent out your scouts to know the map. PHILIP: Yep and I think as a person, and this is maybe why it appeals to me it's like I like the feeling that I haven't left a stone unturned. I like shipping something that I'm confident in and I know it won't bite me or if it does bite me, it's purely because it was an unknown-unknown. It's like doing your due diligence. As I'm saying this, Elm is not the silver bullet. Elm is not the panacea. Elm will not solve all of your problems. But speaking in terms of contrast with JavaScript or Ruby and more dynamic languages than Elm, these are the things that happen. I think that kind of leads me on to another point which is Evan in the mailing list conversations and the decisions that he takes with Elm, I don't know if this are stages of maturity as a developer but you find your tool, you love it, it's the only thing that can do the job and then you realize other people are doing jobs pretty well, doing their other stuff, I guess each to their own. But I feel like there's another stage where it's like what are the tradeoffs involved in taking this decision or using this tool and what are the tradeoffs involved in taking this decision or using that tool. Just the way of thinking in terms of tradeoffs, instead of absolutes I find really, really valuable and that's definitely something that I've been able to apply and what are the benefits of this decision. Not only from a code point of view but from an organizational point of view, from a team point of view, from a long term point of view, from a short term point of view, having a wider arena to make those decisions, it has been really helpful for me, valuable on both side of them. CHARLES: I can see it. I believe you and I think we're coming to the end of our time here, I think I'm going to make the same recommendation that I made to everybody last week when we were talking with Toran about Redux and that is you should go out and you should try Elm. It's really easy to get started and it will guide you. The guides are excellent. The air handling is excellent and yes it is quirky and weird but it supports you. There's going to be some big functional programming boulders thrown at you but it gives you a [inaudible] to deal with them. JEFFREY: I think you need a suit of armor so we can stick with the Age of Empires metaphors. [Laughter] CHARLES: Yeah. Go out there, try it and experience the things that Philip is talking about because I feel like they are very real and they will have a good effect on you, whatever it is that you do the next day or the next week or the next month. PHILIP: They'll challenge the way you think, they'll give you new perspective and they'll give you a good counterpoint as well. You know, elm is not for everyone but only if you try it will you understand that. I think that's one of Evan and everyone in the Elm community is try it yourself. I'm not going to say Elm is right for you, Elm is right for your business. Rewrite everything in Elm, not at all, try it out, try it on a side project, try it to just having fun hacking around. If you do want to make the jump to using it in your company or whatever, don't take a decision to rewrite everything. Take one very small part of your app and do it in Elm and see how it goes. If you don't like it, you can take it out. If it works, you can build another small thing. This kind of gradual approach were far too quick to jump on a bandwagon: trash the old thing, do the new thing. Especially now, when we are building applications that are going to be maintained, when we are building applications that are going to be around for a few years, it's also just not feasible and sometimes it's not right. You just got to make that decision maturely rather than I want to use this or I want to use that, which can be tough. But ultimately, you want to deliver the value. Try Elm at Elm-lang.org/Try. You can try on the browser and see how you go. The official guide is really good. There's also Elm-Tutorial.org, which is kind of a complement to the official guide. Pragmatic Studio have released recently an updated course for learning Elm. James Moore at KnowThen.com. He has a couple of Elm courses, one is free introductory and then there's a paid course, which goes into deeper topics, whether text or whether video, whatever your thing is, there's resources available to get you to the point where you'll be able to make a decent decision. CHARLES: Well, fantastic. I'm certainly excited. I haven't dip my toes in Elm for a while and I'm actually, after this conversation pretty stoked to get it on again. I hope everybody else does. Thank you so much, Philip for coming on to the podcast. This was just a fantastic and enlightening conversation. PHILIP: Not at all. It's been an absolute pleasure. Thank you. CHARLES: All right. That's it from The Frontside. Remember to get in touch with us at Frontside.io, if you're interested in UI that is engineered to make your UX dreams come true.
Toran Billups @toranb | GitHub | Blog Show Notes: 02:23 - Ember 2.0; Data Down, Actions Up 08:28 - redux and State 16:39 - Dispatching Actions/Patterns 24:00 - Components and Routing 31:00 - ember-redux and Cloning the react-redux API 35:22 - Hot Reloading 41:22 - Audience 47:02 - Motivation 50:25 - Building Component Trees Resources: Toran Billups: Test-Driven Development By Example @ EmberConf 2015 Dan Abramov: Live React: Hot Reloading with Time Travel @ react-europe 2015 react-redux Charles Lowell: Immutability is for UI, You and I @ EmberConf 2016 redux-thunk Ryan Toronto: Safety of the herd EmberMap: Component side effects Functional Programming Browserify More Toran Billups Talks Transcript: CHARLES: Hello everybody. Welcome to The Frontside Podcast. This is Episode 55. We're broadcasting and everybody's here in Austin, although we're still remote. I am here with a really special panel today. There's me, which makes it special by default. My name is Charles Lowell. I'm a developer here at The Frontside. I'm also here with Robert De Luca, who also develops JavaScript codes at The Frontside and we have today [drum roll sound] -- I'm really, really going to work that drumroll -- Toran Billups. What's up, man? TORAN: Hey, man. Thanks for having me. I'm really excited to be here. CHARLES: Toran is with us today and he's going to be talking about a lot of things. He's going to be talking about today mostly about Redux and his efforts to meld Redux and make it useful within the Ember community. But first, if you have not heard of Toran, I think the first time we'd interacted over email, Slack briefly but then when I really, really saw you for the first time was at EmberConf, I think in 2015 and he just gave the most amazing talk on Test Driven Development and really kind of the focus on you can set up your acceptance tests from the outside into really define that behavior and set out that firm shell and then actually build your application from the outside in. You've really kind of been talking about that message. We like to have people on here who all about walking the walk. That's certainly the first thing that I've noticed that you were doing in the community but then more recently, you've been playing with Redux. Not playing with Redux, actually making a concerted effort to bring this kind of pattern into Ember. I just wanted to start out, how did you jump onto that track? TORAN: Some months after EmberConf in 2015, as you mentioned that talk was not only, probably the most well-rehearsed talk I've ever given but definitely the most well-received. I got a lot of people excited and really gave me a lot of opportunities that weren't there before that was also believe in that keynote in 2015 as when Ember 2.0 was announced. The interesting part of Ember 2.0, of course was we were using it, and it was called Ember 1.13, which actually made it really great. At the time, I was very much excited about the stability that offered. Other folks were not as much as interested in that idea or those values, in the JavaScript community so it's really exciting. Yet at the same time, there was this new mantra that was being talked about more that I knew nothing about. It was this 'data down, actions up' idea. I still remember as much as the stability was awesome, I also started to doubt not Ember core in particular but the ideas that were being espoused by other folks around the Ember core team because I didn't really understand the value. For instance, if I had the tree of components back then in early Ember 1.13 or 2.0 days and I had an Ember model or some kind of Ember object at the bottom of this tree, why would I not just do Ember.set. I remember, actually you may recall conversations you had with people at EmberConf around that time but there was actually varying definitions of what 'data down, actions up' meant to different people and actually never met the same thing to anyone. It was funny enough. Because of that, it sort of drove this fear in me that I didn't know what I was talking about. I was consulting at the time so I wanted to sound like I knew what I was talking about as you probably should. You guys are in that business so you know what I mean. Because of that doubt, eventually I sort of become apathetic to really trying to understand better what 'data down, actions up' meant or how components should be constructed and really what the wider impacts of Ember 2.0 meant. Because of that, I just found myself, not really loving learning. I felt like everything else in learning was a hyped up thing that would never happen or a hyped up thing that I didn't really understand or didn't make sense in the context of Ember at that time so I just kind of floated by. Everybody has their ebb and flows in the journey of excitement or not. For me, this was just the down moment. One of the things, like an analogy to this is when you lose your hunger in real life, you'd be very much like losing your hunger for learning. There's this interesting hormone that your body produces when you're actually physically hungry that kind of gives you the physical symptoms like your stomach cramps that tells your brain probably should eat somewhere. When those things aren't happening or that hormones not being produced, it's often because you've quit eating yourself. If you've ever gone on a fast or something weird like that on day three, your body quits secreting this hormone so you just sort of or not hungry at all, which is kind of weird. The same sort of thing was happening to me. If you ask a doctor or a physician, "What's the first thing I should do? I'm not hungry anymore." They'll tell you, "You just start eating." But I'm not hungry now so the same thing applied in my life and I think the first step really is just eat anyway. In this case, it was just learn something anyway even if you're not in love with learning right now. Eventually, your body will start producing this hormone, in the hunger example and for me, I just sort of got back in the flow and what came from this was a routine, which is really the second part of how you get your hunger back, not just eating once a day. I was eating three meals a day or more, especially if you haven't been eating. For me, I just set aside an hour a day, in addition to consulting work and things that would get me interested in loving learning again. The third component to this was trying something different. At the time, of course I was just doing Ember, I pretty much had done Ember since 2012 like some of you guys and I feel like there wasn't a lot of new. There was patterns and ideas but there was anything really challenging me. That's when I sort of looked around at the React community and I had done some React in the early days when I was super hyped up but I still feel vaguely different. Not that it's jQuery or any way but I didn't feel like this fully encompass single page out framework. The reasons I got into Ember very early on were that I wanted to build rich single page apps. If you guys remember from the early React days, that also wasn't really the messaging with React and maybe today with View. In fact, that's kind of one of the benefits or they speak to in those communities about why you use React because you don't have to use it for your whole app. You could kind of piecemeal it in, which totally cool. You got to see it out with Ember too. But in my mind, I just wanted to build a rich JavaScript lines that could compete with the iPhone or the iPhone apps that were popular in that day. Through this process, I started looking at React and really just kind of get back into it again, get going again. I wasn't really in love with it but I needed to try something outside of the realm I was used to. As part of that, I noticed there was this talk by Dan Abramov, I think he works for Facebook now, big in the React community, of course and he gave this talk at a conference in Europe that introduced Redux. What's funny, if you find out or dive deeper into that story is he actually pitched the talk, not really having built any of this and just thought, "This sounds like a great idea," and then of course the talk was accepted. Like most, he delivered on that promise and made a great talk. There are definitely courage folks to check out and I should link it to you here. We can show noted that, I'm sure. That talk make me excited mostly because there was a lot of whiz-bang. If you guys remember any great talk you've ever seen, the talk even that I gave at EmberConf that you mentioned, Charles the thing that blew people away was this very end moment that, of course I had produced to be a great moment. In the same way, Dan during this talk introduced some new ideas or new to the JavaScript community. One of those was the tooling that can change when the state doesn't change in your application. That got me sort of piqued my interest, in part because I was actually big test driven guy and I thought, "I won't use any of this stuff. It just seems cool. It's a gimmick. Tester development is how you really build app." If anything I thought to disprove it by getting involved and learning a little bit more but what I instantly found was the simplicity of data changes rerender. That sounds very high level, of course but it was almost just that simple, instead of being like how does this change to an object in my array, bubble out through notifications on the Ember side and notify the Ember change detection to rerender. Well, I'm not entirely sure so when I was start debugging that, I noticed a lot of framework code between me and the rerender. It's that's how Ember is, right? When I boiled that down in jQuery with vanilla Redux, not even using React at all, I was like, "Wow, there's just a call back. I wonder why I haven't been doing this." CHARLES: As a single callback for a global state? TORAN: Correct. CHARLES: So there's no call back for every single path in your tree. You just used that one call back? TORAN: I'll fill in for Rob here. I know he's jumping at it. You should probably define a Redux is. He's really good at asking that question. Redux in this case, for me is just a global JavaScript object to use to hydrate your templates. They'll give you some big spiel about state container, if you go check out the website. But for me and in this context of being on an Ember-centric sort of podcast, we already use this idea in Ember today. If you're just feeding your templates from some high level service, it's a very similar idea in that Redux is just a single service. In the Ember case, especially you can talk about the add-on, I maintain later, but really it's just a service with a single object that will help you populate all of your components. ROBERT: Yeah, I love Redux. I actually sort of coming into the Redux world, probably to about six to eight months ago and it was around the same thing like exploring React stuff. I share similar opinions to you as nobody really can define 'data down, actions up'. I also think that 'data down, actions up' cannot just live in the component. In a lot of the Ember apps I worked on, there's times where I'll be looking up to get a new state and it comes in from the side and something's mutating, something that I have no idea why and where it was mutated and Redux does a really good job at helping you manage what changes and why it changed. CHARLES: I have a question too. When you're actually using Redux, you said you got a single tree that you used to hydrate your templates. In the context of Ember, where do you maintain that single object? I assume you have one store, one instance of your Redux state per application? TORAN: Correct. There's just a service like you imagine in the Ember data service and that holds on to really just an identity map or a single graph object that will let you pass or pull that in by injecting the service into your components if you want to do that or your route then just asking for that state. CHARLES: Because I think that for a lot of people in the Ember community certainly, when I was kind of grappling with these ideas, the idea of having a single global object as your state seems so counterintuitive, so going to go against everything that we learned, that you have to decompose a problem into its component parts. Obviously, Redux has an answer for that so how does that work? How do you decompose that state into saying, "I'm just interested in this kind of local state." How does local state work in Redux? TORAN: I should define local state is state specific to the component. It doesn't need to bleed up and has no value at the global level. CHARLES: Usually, I got two components. Let's say, I want to store both of their states in the Redux store. Obviously, component one is not interested in seeing any state that's not related to it so it's only interested in its own state and it's not interested in any of the surrounding context. How does that work? How do you connect a single component or connect a route to the store? TORAN: There's really just a simple method on Redux -- the Redux store itself, which it says, "Give me the state." What may not sound great at first is that it say, "I will give you all the state and that is your job to pull from that or map three attributes from that whole tree into my component." Then by side effect if you're using our add-on or if you don't React-Redux, you actually subscribe then to call backs on any of those changes so if something were to be bumped, then your component is given the opportunity to rerender during that call back. CHARLES: Now, in terms of Ember-Redux, that kind plumbing is hidden from you. You don't actually have to explicitly map that state. You can say, "I want to connect this component into the Redux store," and you're just off to the races. ROBERT: Is there a mapStateToProps or... I don't know what that would be called in Ember-land. TORAN: That brings about interesting point. I literally copied this API that you guys are probably looked at from the readme from this very popular project in React called React-Redux. The word that you're using, Charles is this word connect. Actually, I like that word because that's how I think about it. I want to connect the components to the single source of truth and then respond by rerendering when something changes. The API is actually very similar on what you said, Rob. In fact, the set of mapStateToProps is just map states to computed, which is very much the same idea so instead of really defining the component like you might normally, this is where it gets a little weird for your classic Ember developer, you actually just write two functions and really only one is require. The first one is what you're hinting at Charles, which is I want to pull from the state a set of properties and as you mentioned, the plumbing is sort of hidden, magically those are actually created as CPs or Computed Properties on your component so you can go to your HPS file, your handlebars template and say, "Oh, I took number from the global state and I'm just going to map it in this function and now I can go to my handlebars template and number," and there it is. Every time you bump number up or down, you'll get a rerun in your callback and the HPS will update. The other function, as sort of glossed over is really just for your closure actions. If you would like to ask the store to do something and saying, "I would actually like to increment the number," then you can fire an action and the second block just does also additional magic, which just maps a closure action by letting you get this dispatched keyword. Dispatch in a Redux context is just, "I'm going to send an action," and you can think of it almost in vanilla JavaScript terms as, "I have an event. Someone will handle this event and I'm just going to throw it up." ROBERT: It makes its way to reducer then from there, right? TORAN: Correct. We haven't talked too much about that process. The reducer really says, "I'm going to be given a state or the initial state, if you haven't done this yet," which would be maybe in the number scenario. I'm going to start with zero as a sensible default and then I'm going to have an action, whether that's add or subtract in this simple example and in add, I'm just going to take the state coming in, even if it starts out at zero and then do something, transform it to a new state. Actually the important word here is that -- I know you guys are big in the functional world, functional programming and that's the word actually got me interested and really excited about programming again as well, in the most perfect sense -- a pure function, which just means that there are no side effects. There's no mutation or changing of the state that comes in when you do it correctly. In this case, actually instead of mutating something I'm actually returning to number two or to number one and you're like, "Now, we have both zero and one in kind of a timeline." If you think about this almost as the realistic stories, we're just kind of kicking a pointer to a new block of state. Every single time you come to reducer, we still have the old state and we can still walk backwards, which is how the time travel debugging works as we just flip the pointer back in time. As you guys have talked about and I think, Charles you mentioned last year in EmberConf, the immutability story has of course a whole slew of great properties that come with it and those we haven't even obviously talked about. But hopefully I gave people a broad overview of what the reducer does. In its most simplest form, state comes and action returns a new state. ROBERT: Yeah, in Charles's talk and his research, I got to sit next to him and watched him do that actually kind of shaped a lot of my thinking and hunger, if we want to keep that going towards doing like something that's immutable and state management in Ember. I would like to thank you, Toran for building that add-on and spearheading Redux because Redux is pretty awesome for state management. CHARLES: By the way, you did in that call out the analogy for hunger. I really, really, really like that. It's an important tidbit not to miss is that when you are feeling those kind of doldrums of development. I know I was actually ironically feeling that about the same time in 2015, feeling of in a funk because I feel like there was a lot of stuff coming down the pipe like with 'data down, actions up' but no good examples of where we've actually seen this in practice. I think Redux is an actual implementation of 'data down, actions up' so I think it's fantastic that you were able to go and seek inspiration there like, "We've got this message of the way things that ought to be doing with the applications ought to be built." But we don't actually have any concrete examples that we can look to. I think the Redux actually is almost the most pure version of that 'data down, actions up'. I guess my next question is given that you've got this global store, you've got a way to connect components. I assume there are other ways to dispatch actions from within your Ember application like what are the patterns that you're seeing emerge around this? We've talked about how you would use them in components. Suppose my tree of components gets pretty complex, how do I manage that to kind of the passing of data down? Do parent components play any role in the data that their subcomponents see? Is each component connecting directly to the store? I'm just kind of curious where that balance lies and how things are kind of playing out? TORAN: There's really two points in your bigger question. One that I was going to try out of you but then you kept going. That was really around side effects. How do you actually dispatch or make changes, requests changes and see the flow and we could talk about that really starts out briefly with a Promise based approach. With Redux, most people don't know but it's basically like asynchronous flow. Dispatch would normally be like asynchronous action where you're sort of blocking and then doing, transform and getting it back. In the simplest ways, you see there is this tool or this add-on, Redux-thunk, which you can use Promises now and async will still work as if it were synchronous essentially by firing dispatch up and letting your reducer do the work. I think that is a great introductory because especially as Ember developers, we've got a lot of experience with Promises so this is just the same thing. In most of the demos I've done and if you check out the read me, there's like a full Yelp Clone example. It's using this approach because it's a little bit more familiar to most folks. CHARLES: Just to clarify what would happen there is you're essentially getting a new state transition when every Promise resolves or rejects. If it's rejects, that's a state transition. If it resolves, that's a state transition. The next Promise that resolves is another state transition. Is that fair to say? TORAN: Assuming you want to alter and use that top level state, of course you could reject or resolve and just not even bother with the top level store. We kind of skipped over some of the benefits and we could just roll back to that briefly. Why would you use top level stores at all? You mentioned earlier and it kind of seems counterintuitive. This is basic global variable. That's what we're talking about. In the Ember example, I think it's actually sort of not weird because if you guys, your Ember data in its earlier form or even today, it really very much is that. We have this one cache of objects related or otherwise and we pass those around. They are a global object or almost like a global variable. The downside of that in my experience has been that is truly mutable and actually everything is driven by mutating those and then having callbacks or denotify property change drive your template updates. That is not the process with Redux, of course. It feels more explicit, where I can actually go look up kind of a tree or look up table of actions and see exactly what's going to happen. Then also to your second half of the question, which is like how was the components wired up? How do they map? I actually uses an interesting pattern which isn't specific to Ember-Redux or Redux, which is to create a seam in my components now where I have truly HTML CSS components. Separating those from the components and know about the data and the closure action story. Forgetting Redux for a moment and all of this actually made my regular Ember much better because I started to produce this component that would connect to a Ember data store, provide closure actions to send up in the most pure 'data down, actions up' sense and then I would connect it using the yield block, which credits to you and other folks at EmberConf that you, Charles kind of talked me into this because I was a espousing this idea but I didn't really understand that I would actually nest within this parent, the HTML component that would just be handed the properties to render. When we do this, again it still is I think a better pattern even if you're not using Redux but when we did it and I when I started with Redux, the only thing that really gets me in hot water is when people see this and they're like, "Oh, so this is the first thing that comes down from the routed controllers template. Then there's always this brief moment of like, "I'm not sure what to say. I don't want to predict the future and I'm not trying to be Mr Routable Components here." But for me, most of my controller templates are just a landing page for the component tree to begin. Again, that's not me trying to hacking the route or anything to say, "I want to use this controller as a routed to component. I think eventually when that RFC lands, this will look different, anyway so I'm not trying to have people do things really outside of the Ember ecosystem or outside of the norm." But from there, I feel like still just landing into a component, allows you this composition which is supposed is the real value of the components structure. They are too primitive to build pages and then eventually full apps. ROBERT: So if we want to drop parallel, it's container versus presentation components, right? TORAN: Yes and that of course, again stolen from, not me probably stolen from someone else in the 70s. But you know, Dan Abramov is accredited to bringing that idea about in React. Actually I like the idea because let's pretend I had done this pattern in 2013. Now, it's using Ember data or simple store or Erik Bryn's Ember model, something like that. Then eventually, the community start shifting to something else. It could be MobX, it could be Redux and whatever the case, I could just very easily swap out those connected components that have no HTML CSS. The data source changes and all the presentation components do not know. They do not change. There is actually an iterable story to refactor through, an update like that normally is kind of a [inaudible]. If you have ever done PHP in the early days or at least my PHP experience in 1999 -- no offense to PHP today -- was that everything was so stuck together or so couple that I could never refactor anything out of it. Of course, you probably do this in a consulting space as I have, where he first thing on a messy project is actually making those scenes in the application anyway to allow you to upgrade incrementally. This process is just more of an upfront thought and I don't think it's really taken hold than it needs to in the Ember community. It's just something I was experimenting with and I'm finding a lot of value because I think the connection of the data source is a different activity than HTML. ROBERT: I think it also holds a lot of value. CHARLES: I think it holds a lot of value. I think there's a dawning awareness of this. In your comment, I actually thought of two blog posts for EmberMap, which I was just reading this morning. One was talking about kind of the safety of the herd and don't worry so much about controllers versus radical components like use your controllers, use your components. Don't worry about it too much. It'll get sorted out. I definitely agree with that. Although, you definitely want to experiment when you're experiencing particular pain around something. But then, the next thing which I think came out yesterday was talking about basically components for managing side effects, which I think is an unfortunate name because I think side effects is a tainted word. But basically, the idea is having presentation components and container components and the container components are responsible for managing the state. I think that idea is valuable in of itself and I hope that it takes root. I think that's something that you're doing, something that we're doing and as people kind of realize it, it does take root, just kind of by virtue of its own value. Let me summarize if I understand it correctly. As part of these job, you've got these container components and their job is, I like the term that you used, creating a seam. Their job in the Redux world is to take a slice of that global state. You have these components whose responsibility is taking a slice of that global state and presenting that global state to HTML CSS aka presentation components that lie underneath them. Is that a fair assessment? Then if that's so, I've got a second part to that. I just want to make sure I'm understanding it correctly. For components that are further downstream on that tree, do you ever switch back to data containers like you switch between data components and presentation components and then back to data components and then to presentation components and kind of back and forth and back and forth on down the tree? Or do you mostly see it as one-kind of container component on top and then presentation components all the way down? TORAN: It's a great question. I think that still needs a fair bit of experience in the Ember community because the patterns I pulled from the source code I read a lot is mostly from the React ecosystem. Because of that, there's a very split view or a different view in that community on routing. We may share some of those views in Ember but I think for the most part, we assume routing actually and that's one of the tricky part to answer your question. This is a broad statement so I'm likely wrong in every context but I don't love to be creating these data components that don't get routed to if I can help it. I'm sure there are situations that have been really complex, places where you just have to make, no route here because I don't want change the URL for instance and I'm just going to make this thing like a routed to component with no URL to get me here. But for the most part, I treat the entry point to this route and when I land on this route at this time, it's appropriate to ask for the data likely coming from the model hook in the route. In fact, all that's still the same. That's also where it's a little weird. If you've ever seen a full component tree in a React app, they may not have a router at all. In that situation I think, Redux was in particular even better because you don't have to pass from the top app component, the same props or the same data all the way down that tree. In fact, if you read documentation about why Redux in the React ecosystem they'll say, "It gives you this place where we can create a little shim and then ask for the data down here in the [inaudible] mode. You don't have to pass it from the app to that, to that." I see those benefits but in Ember we don't really get as much from that. In fact, they still tell people who challenged the global state idea that not everything maybe should be a global state but you give up some things by doing that. The first one I would say, which I think is the most valuable for anyone doing vanilla Ember with Ember data or someone experimenting with React or Redux. Or the case I'm most interested in, the audience I'm after which is Redux in Ember, which is do you actually need to have that state in one place. The prime example of this that is the greatest use case is master detail. What I mean by that is you have a list of things and when someone drills into one of those, you can also see that at the same time. There's really two choices you can make here. One is I'm going to have two separate data sources to feed two separate components so the list will go get its data and then the detail won't even use that data at all. Just go get its own data. In that case, you may up against a problem where you need to synchronize at some point and here's the tradeoff. Either synchronize the two separate states or you have a single source of truth. That's a real benefit I think of Redux for the most part. It's like the broad, "Do I want deterministic rendering?" We've all heard the joke about the Facebook nav bar that's like, "You have one message," and you're like, "No, I just answered it down here." Well, that's a different component so the joke is like, "Oh, Redux must be working. We have one up here but I've already read the message." You know? Someone obviously is in charge of synchronizing in those sort of examples. Maybe not just doing it well or they run up against an issue synchronizing that. My experience doing back end development, colors this for me. What I rather have three databases and they kind of synchronize the state across them or I rather have the one postgres or SQL server database that is the source of truth so that when I render something to a customer, I can guarantee that it's not in a transition to be synchronized. It is the source of truth. CHARLES: Right, I really like that and I think the point that I take from that is that, and again this speaks to people who might be internally reacting to this idea of a global state is that you actually do have a global state always in your UI, whether you acknowledge it or not. It's composed of all the other distributed states that are sprinkled around your application so if you take an approach like Redux, you're kind of acknowledging that upfront that at any given time, I do in fact have a global state. I might as well deal with that explicitly. That's kind of a key innovation. I also like what you said too about kind of treating the router in Ember really leaning on the router as a good way to partition your data or drill down into a sub-piece of that global state. Inside Ember-Redux, are there explicit hooks for dealing with the Redux store inside your routes? TORAN: Yeah, that's that one that gets me the most trouble. When I see a blog post and memes that are all about the herd lately, can't help but feel like they're pointed directly at me because of some of these new ideas. CHARLES: Toran, I'm just telling you. This is a safe space. We believe in innovation here. You're okay. TORAN: Yeah. CHARLES: Let me add-on that. I didn't mean that as a knock to you. I do think they call this out of the end of the blog post. I think acting in concert with the community for the most part, actually fosters innovations and an innovative journeys like the one on which you're currently embarked because you don't have to worry about CLI tools and you don't have to worry about this. You can focus on the problem of like how does an Ember application work with a global atom as its state. TORAN: That is the idea. I mean the route is interesting. I have a little helper to your point, Charles if you've seen some of the docs or any of the examples. There is a little helper for routes and all it really does is provide dispatch as an argument. For instance, a lot of times I just want a model to be a regular function and dispatch to be an argument so I can return a Promise or do some Generator stuff as a side effect. In that way, I sort of create a shorthand which is just really simple. It allows me to say [inaudible] model and then have dispatch as an argument and run my code then just providing that to this special little helper. It's a functional type helper called route and what it does behind the scenes is it injects the Redux service for me, which is again something you can do by hand. If you really just don't like that or you want to be more in the herd, you can just have a regular route, inject the service and then get dispatched from that service and use it. ROBERT: It looks like you just dropped the version 2.0, like three hours ago. I would like to ask, we heard about your journey like you were feeling like you weren't hungry for learning. I want to know more about where you actually sat down and wanted to write this add-on on and why you chose to clone the React-Redux API and what took you on that path? TORAN: Yeah, that's a good question. Back to benefits or the reasons I got excited about, of course I mentioned during the talk that Dan Abramov did. There was some interesting dev tools. First of which was this thing Time Travel Debugging which it allows you to sort of move backwards in time and pretend as if actions and mutations or what looked like mutations that never occurred. That was very interesting. I wasn't really sure of the value, especially at the time. I told you guys around 2015, I was consulting which lucky me, I was doing Greenfield. Thankfully, I was working with a really great team and some great people, built an amazing product. I don't really understand the pain of this. For the most part Ember-set was doing its job and I didn't really have a lot of interest in learning this. But as I got more into it, also started a full time job last year, I pretty much just fix bugs for a year. Anyone who's been on one side of the fence or the other knows that the bug fixing side will sort of expose, maybe the weaknesses of the application or patterns or choices made. For me, that was really mutation or shared mutable stake aka the root of all evil. If you've ever looked at your Elm ClojureScript, Elm next is the same vein where immutability is very much there. Charles, of course gave his talk on immutability and trying to get people interested in that or more interested in the Ember community. That was really all I wanted to do to your point, Rob was provide really an outlet for people to use this and I wanted to keep the messaging away from the things I didn't like, which I think was actually something I screwed up to be fair early on. I think I was very vocal in the microcosm that I would talk to people about like, "These are the things I don't like about Ember," or I would use the word 'Ember the good parts' plus 'Ember the bad parts' and I was told not to say that anymore on the Slack channel. Once I started getting too much needed feedback -- I don't want to be negative about it -- I changed my messaging and as part of that, you mentioned Rob I basically cargo-culted or copied this API from React-Redux called connect and excluding the brief route helper that I mentioned, Charles a minute ago, the real idea here is you just call disconnect function with two other functions: mapping state and closure actions. Everything else becomes then vanilla JavaScript in this reducer function we talked about briefly where I have state coming in and I need to transform it into a new state. One interesting benefit of that -- I wasn't overly critical about until I really saw the difference is that -- I'm no longer using the Ember object. I'm not doing Ember.get and set, which immediately start to open the door some time last year for TypeScripts interest. I'm actually not a super type friendly person. I sort of left Objective C and C# and Java in my background and have like this Vietnam experience when people ask me about types. But I do understand one very critical fact that I can't dispute about types is that there are more information for the next programmer than you have without them. Again, my experience this last 12 months has been, as a maintenance programmer, I need more information. Tests are great when they're there but they also don't provide the interface or all the information about those and certainly the compiler may help as well. I don't know yet. I'm not doing any TypeScript. What I started notice is also more functional programming and maybe just not in our core yet but also things I wanted to steal from other ecosystems because I also found is very interesting. I started to study functional programming. I know like nothing about it, of course. I don't think anyone does because I can't describe a monad without getting in trouble or being wrong. For me, the real value is the separation of the data structure and the function. I'm preaching to the choir here but that was so much like an interesting idea to me and actually spurred on some of the further patterns or adoption of those in my work in Ember-Redux because this presentation and container component idea was really that I was separating the data structure from the function of the view. I think you mentioned this in your talk at EmberConf where the actual HTMLbars template is really just a function that has data in, HTML out. I started to internalize that and think about that and what were the properties I got from that, as well as I enjoyed functional programming. Some other great benefits that we've already touched on briefly are just how much more of this I felt explicit, not that Ember-set is inherently implicit but when you do a Ember-set for mutation to chase down every single place in a complex system to determine why they something render this way? It does feel a little more implicit than something like React-Redux with this connect function where I was like, "Wow," when I was doing React. Especially, I was like, "I bet I could just put a breakpoint at every connection so when that callback happens, I can know exactly what action spurred on this new callback to rerender," and that was something that was very new and interesting. Then of course, falling out of all this was another hyped tooling thing that I thought was really cool, not explicit to Redux, again but it got me interested because that's hot reloading. All hot reloading of CSS and Ember CLI, which I've never done design work which I'm not good at. But I do write some CSS or hack-on it when friends show me what to write. Then writing HTML was a separate experience. Once you wrote the CSS, you would hot reload in that course, what do you do every time you change CSS, you also change HTML, which would incur a full-page reload with a live reload tool, if you're familiar with that in Ember CLI. This tooling allowed the Redux store itself because it's stored the state, allow me to really throw the component away in the page without refreshing it and then providing a new one and just go rerender because the state was instantly mapped in and then rerendered. I actually did a demo sometime last year and like, "I'm going to build a star rating component and here it is with live reload. Here it is with hot reload. I'm not going to make a decision about which one is better. You decide," and overwhelmingly people were like, "This is a much nicer feedback loop to make HTML and CSS changes in real time." ROBERT: Agreed. Let's pedal back the hot module reloading because that is pure awesomeness. But that has a little bit of setup they have to do and changing your application. I remember we were talking about this. When you did that demo, I remember this. But there's a little bit they have to do to make your components stateless. They have to come down from the Redux store. TORAN: Correct and this actually still applies if you are, I think using Ember data as well, as you just can't pull the state to reload it anything local, which may go against what you're trying to do with your component. ROBERT: Right. That's cool but I do want to highlight a little bit something that was cool about the Redux dev tools as with all the state that you have since it's in a centrally managed place, you can take your state and then play it back over the top of something like if it did live reload and it'll just pop you right back down to where you were when you were debugging. When that page refresh happens, if you're not doing hot module reloading, you still won't lose all your state which is really cool. You just play it right back down on top and you're exactly where you were before. It's almost like you would throw a specific test that puts you into that state that you're trying to debug. TORAN: Yes, it's like git rebase. It lets me pull off my state, replay the new component function and then drop my changes on top of it and see it all viewed together. ROBERT: Yeah, I think that's massively powerful. CHARLES: Yeah, it is fantastic and that's where you get into that power. I can get on my immutability soapbox. But it turns out that as programmers, we deal in information and not throwing information away, not just flushing it down the toilet is hugely powerful. I think the thing that's so fantastic is that Redux takes this concept and then all of the tools to leverage it are there for you. I think that it is something that is missing from the Ember development story and people don't realize that it's missing, that we have all these wonderful tools, we have this conventional way of building our applications, of deploying our applications, of rendering our applications, of marshalling the data in our applications in the form of routes. But what we're lacking is this unified atomically based state management solution. I think that, Toran it's been fantastic that you have pioneered this and trying to bring what I see as a glaring gap in the developer experience to the community. I'm curious then to ask you what do you see as the future. You know, 2.0 just dropped and there's this need. I feel very strongly that Ember 3.0, 4.0 or Ember 'dot future' at some point should have a unified state management solution. How do you see the road that you're on intersecting with that future if it does in fact exist? ROBERT: Also how can I help or how can we help? TORAN: Just real brief before I dive into some of those questions. I just want to mention that 2.0, as awesome as that sounds, of course I dropped that this morning just so we could say that on podcast, really. We've had a beta in the works for Ember. The only change really, if you're like, "I just got into an Ember-Redux last week. Is it all garbage?" No, this isn't Ryan Florence 2.0 -- it was a joke for any [inaudible] router folks in here. Actually, just us removing Browserify because if you are familiar with Browserify in the Ember ecosystem, talk to Robert Jackson or Stef Penner, folks familiar with that in Broccoli, they'll tell you that one of the harder things to optimize and although, it created a great entry point to how do I use Redux? Boom! Ember Browserify, install Redux, I'm done. If you've ever seen an [inaudible] in Ember that has 'npm:', you're using Ember Browserify to pull in, either a common JS module or some kind of node module and use that in the Ember ecosystem without an additional shim. Now, what we found or learned was that bigger teams that are using this, paid a little bit of a cost and not just cold rebuilds. I'm talking hot rebuilds because Browserify just isn't friendly to get those to be optimized, I guess is the word, so we removed it completely or just use some smaller simpler shims. You actually get the performance improvements hopefully -- ROBERT: That is awesome. TORAN: -- Which is big win. Back to your question, Charles. The audience that's intended, of course is a little different than most people like me to talk about. In fact, the API itself, I think was a bit rejected. You're sort of asking like, "What does this mean in the future?" I don't really feel that the traditional Ember audience has picked up around with it because of something that's missing. You said the 'C word' earlier so convention is certainly still missing from this and even in the React ecosystem, they're just barely thinking about, "Look at all this great stuff we can do with one-way data flow and immutability and functional programming," but guess what we're giving up. No one's really come around with this perfect pattern and conventionalized it as Ember did in its early days so there's a lot of churn, I wouldn't say overly so much that you're not going to getting work done but more than the average Ember developer is aware of. My audience is actually not the average Ember developer, which may be bad for what you're asking about, Charles. Instead, it's actually the person who maybe has done React and maybe Redux or Backbone in the last two years. They love some of those patterns. They're not in love with the Ember-object because of getting set. Maybe they love TypeScript and they say, "I want to use this in Ember." They joined a new company that's a little larger than the startup they'd been on the last two years and they are using Ember. They love a lot of Ember but they would also like to use some of the predictable state patterns that you get with Redux. As well as maybe the dev tooling, things like that so they have adopted this. I feel like that really is the new audience that I aim to please or I'm falling in line with, which is a little bit outside. I feel like there's room for some fragmentation and a good beat up on me for that because when the realities of this herd conversation that we're kind of talking around a little bit is that the herd is great until something innovative needs to happen. Innovation, obviously takes some risk and I feel like that's sort of what I did last year and said, "Here's some interesting ideas. I have not shipped Facebook with it yet so let's just check this out." Of course, Ember add-ons are a great way to enable someone to try a new idea. But I think most people got into it, saw this funky connect thing and they're like, "What the heck is this?" It's a function and returns a component. All right, that's not doing that so most people bailed out. But I do hope people still and I know great folks at LinkedIn, Chris [inaudible], of course. We chat occasionally. Mostly he just tells me what I'm doing wrong. Shout out for Chris. But he knows a lot more about some of the stuff than I do and I think he is fully aware of the values that are in Redux that are great and then working hard, of course during his full time gig to apply these to Ember data and hopefully these do make their way in naturally. I just wanted to be a bit more radical. I don't want to wait around and I wasn't really involved in the Ember data project. My own fault there but I think if nothing else, the ideas will come out of it because the developers want this. Whether you're the audience I'm talking about, which is a React developer from two years ago, you're in Ember, you're eventually going to really understand and want this and then those 'data down, action up' ideas that were pretty unclear to me in 2015, will be very clear. In fact, if anyone seen or heard of this Project MobX, which is like an alternative in a way, popularity-wise to React ecosystem. It kind of looks like Ember in a way where you get sort of some more magic and what I found quickly in playing around MobX is that you can very much fall into the shared mutable state problems. The interesting part about MobX is you can opt into a strict 'data down, actions up' approach. But if you don't have the Ember battle scars like we do, you're just going to come in and say, "What's less work?" Just like in Ember when I can do a set in the [inaudible] node, why would I do 'data down, actions up' and that's the transition I want to see folks make. Hopefully they learn something from that. CHARLES: Right, I agree with you. Although, I think the time has definitely come, I think the term 'herd-mentality' is an unfortunate one. I prefer to think of it as like a pack. If you travel as a pack, you can bring down moose that are bigger than you are individually. But every once in a while, like a gigantic moose with laser horns shows up and then what are you going to do? If you're hunting as a pack, you have to introduce new things because I like that analogy a little bit better than a herd because the job of the herd is just to not get eaten, where is the pack has this idea of these entities that have to stick together. They're hunting and they're tackling different problems as they come but sharing in the benefits. But I think that there has to be room for innovation inside that herd/pack-mentality, whatever you choose. I do think this idea needs to be introduced so what I would say is that if you're listening to this podcast, you should actually go and you should try and use Toran's add-on and you should try and build something with it so that if you have opinions about how it should fit into Ember, then we can hear them. It sounds like you're taking a minimalist approach, you're emulating patterns that are proven to work in the React community so kind of enabling that seed cross-pollination right there. I would say go build something with it, experience what it's like to have your state as a single atom, experience what it's like to have incredible development tools that come along with that. I think that if you're in the Ember community today, you need to go build something with React, you need to go build something with Redux and you actually have made it one step easier to do. You don't even have to leave Ember to do that. You can build something of node with production quality code using Redux and you can experience what it's like. That's my challenge, I think to the Ember community. Go try it, go experience it because you'll come back, I think like I did. You'll come back with superpowers just from having tried that. ROBERT: Managing state becomes so easy. TORAN: Yes. I want to jump in briefly and just cover one point that we haven't talked about that's very controversial so why not drop it at the end here. I think, Rob you might have asked about it earlier and I just didn't feel brave enough to talk about it at that time. But you guys keep going back to this idea and I have to talk about a little bit too. One of the motivations is I live in Iowa. I work in Texas. Thankfully, this great company, Q2 employs me and I don't know why I'm being paid. I'm lucky to be writing JavaScript for money, probably we all are. But in the Ember local community that I'm in, a very little folks writing Ember and that was even years ago. I was like the only voice in the middle of the Midwest screaming and then folks in Minnesota would tell me that wasn't true so I went up there and did a conference as well. But for the most part, I looked around the job market too and thought, "It be really great if I understand some of the more JavaScript-centric parts of building web apps today," and when I looked at Redux functional programming, the way the reducer worked and structured, the way to React-Redux project was structured and thought, "I bet I could emulate that an Ember," such that I could actually and I believe this is to be true, that if you were in a React-Redux project or even an Angular like ‘ngRedux', which is a very similar connect binding, you could copy a whole directory of your reducer code, which is all vanilla JavaScript. If you're doing generators, which we didn't talk about but if you're doing you know any additional side effects, you copy all that vanilla JavaScript, drop it into your Ember app and it all works because it doesn't matter if it's in React or Ember or even Angular, even View if View has some connect API like this. We all share this common API that is just give me the functions that enumerate over the data and return new states of the data and call back to rerender. There's something really powerful about that but the tradeoff being there are not a lot of strong conventions, Charles that I have adopted. That's kind of what I'm cautioning here a little bit is that I'm still also just watching the other communities to see what eventually turns out not. This is going to be am Ember add-on and I don't care what everyone else is doing. This is my vision because really my vision was to make a drop in for anyone already doing Redux on any platform. CHARLES: You know, to the point, there's a pack that extends beyond the Ember community and it sounds like you're also leveraging and being a part of that. TORAN: There's an interesting idea about the hunger thing, which just tied us in and there's where the fourth thing that a doctor will tell you to get your hunger back is go experience eating with other people. There's actually a statistic that when you sit down to eat with someone else or many people, you're likely going to eat 44% more food than you did on your own. That's just, I guess a statistic that's true. I just made it up for this podcast. No, I think it's true. If that is the case, then I think that very much translates to programming as well where when I'm developing code with other folks and I'm on like the React channel and we're just talking about vanilla JavaScript, it doesn't have to be me being an Ember developer anymore, which has been a large part of what's blocked me from being, I think an asset in my local community in the broader JavaScript community. At large is every time I get a conversation it's like, "I have to do it the Ember way," and that's changing actually. The Ember has credit a lot of deprecation if you guys have seen or follow the RCs and other just Ember upgrade deprecation. We're kind of getting away from being Ember and writing just more JavaScript and even maybe sometime this year beginning ES6 classes, instead of Ember object extend. I think Ember is heading in that direction. I just went there, rather rapidly because I also was again experiencing vanilla JavaScript with other communities, View and React. ROBERT: I think we're walking on this very similar path. I'm following your footsteps right now, it sounds like. TORAN: My last point which was that third bullet about building component trees, it didn't sound like either of you guys really contest that and I'm friends with, obviously Chris Freeman, formerly The Frontside and Chris tells me, "You're trying to build full component trees once you're injected at the route level and you're not doing like a ton of HTML in your controller HPS files." Is that true? CHARLES: We treat our controller basically as a component. Sometimes, we'll be like, "This is the controller and if we ever use it in more than one place, we'll take out its component." We're not super dogmatic but we definitely see the clear separation of the route is for maintaining the data and everything else is just one tree of components just below that. ROBERT: The more I think about it though, I'm so conflicted because I really like routes in Ember and they do a lot for you. I like having the data be maintained in one spot but I don't know a single store with Redux maintaining that and using like Redux-thunk or Redux-saga. I got some exploring to do. CHARLES: I don't think those are mutually exclusive propositions. That's what you were saying at the beginning, right Toran? You still do all of your data munching in the route. There's two kind of subjects that I wanted to broach briefly, although I don't think brief is possible with them is actions, like how we talked about data down, we talked about where you draw the seams in your application, where you're loading your data, where you're mapping it to your components and having that separation into your presentation components. We didn't get to talk about reducers so much and how you map. You touched on it like the mechanics but suppose I have a to-do list and I want to delete an item and I've got some button to delete an item, that's down my component tree. How do I map that action back up to the store? I don't know if we actually have time to cover that because it is meaty-meaty subject. ROBERT: Redux part two? TORAN: Yeah, we have to follow up because really that is a little bit more of an advanced segment not that folks shouldn't hear about it. But one thing that's a radical shift, Charles that we would have to go into and talk about, which is controversial as well as most folks want to operate in one structure, one dictionary not in the array. Then immediately, everything flips to being a Lodash operation. I didn't really use Lodash at all until I got into this. You guys probably actually are smart folks to do. But for me, this store is not in array now. When I'm doing array operations like remove or filter, I'm actually operating with Lodash on an object to produce those new states and most of it is just learning the Lodash operators because I didn't actually know them so the Yelp Clone that I have out there is a very simplistic look at using Lodash with Ember. But it accomplishes some of that. Then also, the secondary piece that would also consume a ton of time that we should go into but maybe not today is switching from Thunk to Generators with Saga and then maybe even observables with RxJS, which seems like possibly the future. Those all sounds cool but I think they're going to blow the heck out of scope on this thing. CHARLES: All right. Well, thank you so much for coming by Toran. As always, our conversations are too big to fit into a single podcast. I really want to have you on again. There are so many things that we haven't even touched on. We haven't touched on the subtleties of how action dispatching works. We haven't touched on using Ember-data -- I'm just [inaudible] out there and say it. With Redux, we haven't open that can of worms and who doesn't want to just sift through a can of worms on a podcast? We are going to have you on again. I am positive of that. ROBERT: We're going to paint that bike shed. CHARLES: Yeah, we're going to paint that bike shed. It's a bike shed that needs to be painted. It's something that the community, I think needs to face head on. Thank you so much for coming by and talking with us about Ember-Redux. Everybody, go and check it out. Toran, you've got some talks coming up, if you want to mention those real quick. TORAN: Yeah, I just wanted to plug. There's possibly going to be a talk, we're still lining up the official date with the Washington DC Ember Meetup sometime in April. I planning out to fly out there actually and give this talk on Ember-Redux. I want to thank just publicly the RSA team for kind of helping sponsor me to fly out and check it out. As well as give a more in-depth talk on Ember-Redux in the Meetup setting. CHARLES: Fantastic. If you're in the area, be sure to go check that out. If not, watch it on video and then unrelated Ember-Redux, if you haven't watched Toran's EmberConf talk on Outside-In development. TORAN: That's out actually global Ember Meetup, I think. CHARLES: Okay, that one. Actually, just go watch all Toran's talks. The thing that I didn't mention at the beginning of the podcast is that you do a lot of live coding, which is just makes my bowels freeze when I think about doing it. You just pull it off so effortlessly so it's definitely, definitely worth a watch. With that we, will take it out. We'll see you guys later. That's it from The Frontside. Remember to get in touch with us at Frontside.io. If you're interested in UI, that's engineered to make UX dreams come true.
Katie Gengler @katiegengler | GitHub | Code All Day Show Notes: 01:23 - Testing 06:20 - ember-try 14:11 - Add-ons; Ember Observer 17:43 - Scoring and Rating Add-ons 25:25 - Contribution and Funding 27:41 - Code Search 30:59 - Data Visualization 32:27 - Change in the Ember Ecosystem Since Last EmberConf? 34:35 - Code All Day 35:39 - What's Next? Resources: ember-qunit liquid-fire capybara Selenium appraisal emberCLI Bower Transcript: CHARLES: Hello everybody and welcome to The Frontside Podcast Episode 54. I am your host, Charles Lowell, with me is Alex Ford. Today, we're going to be interviewing Katie Gengler. I remember very distinctly the first time that I met Katie, it was actually at the same dinner, I think that I met Godfrey at EmberConf in 2014. That was just a fantastic conversation that was had around the table and I did not realize how important the people that I was meeting were going to be in my life over the next couple of years. But Katie has gone on to do things like identify a hole in Ember add-on ecosystem so she created Ember Observer. There's a huge piece missing from being able to test this framework that spans multiple years and multiple versions and being able to make sure that your tests, especially for add-on authors, run against multiple versions so she created and maintains Ember-try. She's a part of the EmberCLI core team. She's a principal at Code All Day, which is a software consultancy and just an all-around fantastic woman. Thank you, Katie for coming on to the show and talking with us. KATIE: Thanks for having me. CHARLES: One of the things I wanted to start out the conversation with is something that's always struck me about you is there's a lot of people when it comes to testing, they talk the talk but you have always struck me as someone who walks the walk. Not just in terms of you make sure that your apps have tests in them, where your add-ons have tests in them but talking to people about testing patterns, making sure that when there are huge pieces of the ecosystem missing like Ember-try. I remember this as something that I struggled with. I was running up against this problem and all of the sudden, here comes Ember-try and you've been such a huge part of that. I want to know more about kind of your walk with testing and how that permeates so much of what you do because I think it's very important for people to hear that. KATIE: I got really lucky right out of college. My first job was at a place that where people think of mythical themes, XP-focused developers so the first thing I was told is everything is test first, everything is test-driven. I was primarily doing Ruby in Rails at the time but also JavaScript. At the beginning, we didn't have a way to test JavaScripts and there was a lot of missteps in the way of testing JavaScript until we came right around to QUnit. I was QUnit long before Ember even came along. It's kind of bit ingrained in my whole career. Michelle as well. Michelle is my partner in Code All Day. We're both very test focused. I think that's what drew us to start a company together and working together. Every project we're on, we try to write encompassing tests: test drive everything, if we're on it, projects upgrade or any project to fix. We try to write tests as a framework for everything that we're doing so we know whether we're doing something right or not. When it comes to Ember-try, that wasn't entirely my own idea. That was something that Robert Jackson and Edward Faulkner were looking for something right. I remembered that appraisals gem from Ruby. I really enjoyed being [inaudible] gems that I had written Rails so I wanted it to exists for Ember so I just kind of took a promise of to do it. It was extracted from Liquid Fire. I had some scripts that would sort of test multiple versions but it was rough. It wasn't as easy as it is today. CHARLES: Yeah, it does speak to a certain philosophy because if you're coming to a problem and it's difficult to test, you often come to a crossroads where you say, "You know what? I have a choice to make here. I can either give up and not write a test or trying and test some subset of it," Or, "I can write the thing that will let me write the test." It seems like you fall more into that second category. What would you say to people who are either, new to this idea or new in their careers and they butt up against this problem of not knowing when to give up and when to write the thing to write the test. KATIE: I almost never don't write the test so if you're suspicions are true, I will write something to be able to write the test. But there are times that I'm [inaudible] and sometimes I'm just like, "This is not going to be tested. This is not going to happen." Finding that line is pretty hard but it should be extremely rare. It's not when people come to me, I work with a client and they're telling me, "No, it's too hard to write the tests." A lot of times, it's not only how you write the test, spreading the test and learning how to write a test. It's the code you're trying to test that could be a problem. If you have a very complicated code, very side-effect driven code, it's very hard to write the easier sell, which might Ember acceptance tests. What you're really kind of on a level of integration because you do have a little bit of knowledge of what's going on and you have to be within the framework of what Ember tests wanted you to do, which is async is all completed by the time you want to have this assertions and test. That means to look different tools like going back to something like Capybara or Selenium and have some sort of test around on what you're doing in order to replace the code that makes it hard to test it at a lower level to begin with. I think a lot of people are just missing the framework for knowing what to do when their code is intractable or not, necessarily that the testing and the guides that have you tested. I think most people could go through tutorial and do tests for a little to do MVC app perfectly fine. But that's easy when you [inaudible] size of the equation so if you're already struggling with code and you're not quite sure, either in Ember, it can seem very, very hard to write tests for that. I think that's true with Rails as well. I think people that begin in Rails don't understand what they're going to testing, especially if they have an existing app that trying to add test to but unfortunately, Rails not a long ago, kind of got into everybody's heads that your tests go with what you're doing. It's just ingrained part of the Rails community. Hopefully, that will become how it is with Ember. But a lot of people are kind of slowly bring their apps Ember so they really have a lot of JavaScript and they don't necessarily know what to do or they're write in JavaScript have always are written with jQuery and a little bit [inaudible]. They don't understand how to test that. ALEX: How does Ember-try help with that? Actually I want to roll back and talk about what is Ember-try and how does it fit into testing? You mentioned the appraisal gem which I'm not familiar with. I haven't done much Rails in my life or Ruby. But can we talk about what Ember-try is? KATIE: Sure. Ember-try, at the basis, let's you run different scenarios with your test. At some point, I would've said, let's run different scenarios of dependencies for your tests so primarily changing your Ember version and that's pretty much what add-ons do but a lot of people are using it for scenarios that are completely outside of dependencies so different environment variables, different browsers. They're just having one place to have all these scenarios, where if you just put it in travis.yml like your CI configuration, you want it as easily be able to run that locally. But with Ember-try, you can do that locally. I found that it's kind of beyond my intentions, expanded beyond dependencies. Primarily, it lets you run your test in your application with different configurations. I could see running it with different feature flags, it would be what something to be interesting to do, if that's something you use. Primarily, it just lets you try the conversions and appraisal gems let you run test with different gem sets so you have a different gem file for each scenario you possibly had. That was definitely dependency-focus. CHARLES: That sounds really cool. It's almost sounds like you could even get into some sort of generative testing, where you're kind of not specifying the scenarios upfront but having some sort of mechanism to generate those scenarios so you can try and surface bugs that would only occur outside of what you're explicitly testing for, which is kind of randomly choosing different versions of environment, variables, feature flags, dependencies and stuff like that. They didn't thought of that? KATIE: Randomization [inaudible] but Ember-try really does have a kind of general route way of working on and that's we're leading to that. If you wanted to, especially for add-ons, you can specify this version compatibility keyword and your packet at JSON and give Ember and give an Ember string a version and it will generate the scenarios for you and test all those versions. These Ember strings are pretty powerful so you can say specifically versions you want. You can do a range of versions and it will take the latest patch release and a reminder, at least you don't want to be too crazy and test each of those for that add-on. But I can definitely see something random, they're really cool. Some testing thing that's like just tries to do random input into all of your inputs on a page. I've really been meaning to try that out. Sounds like [inaudible]. CHARLES: Yeah, just like to try and break it. I remember a world before Ember-try and I can't speak highly enough about it and the fact that how many bugs it has caught in the add-ons that we maintain because you're always working on the latest, hottest, greatest version of Ember and you're not thinking what about two-point releases back. They're might be not a deal breaker but some subtle bug that surfaces and break your tests and the coverage has just gotten so much better. In fact, I think that they're as brilliant as it is bundled with EmberCLI when you are building an add-on. It's like you now you get it for free. It's one of those things where it's hard to imagine what it was like, even though we lived it. KATIE: And it was less than a year ago. [Laughter] KATIE: Ember-try existed for more than 15 bundle with EmberCLI so since last EmberConf or so. CHARLES: Yeah, but it's absolutely a critical piece of the infrastructure now. KATIE: I'm glad it caught bugs for you. I don't think I've actually caught a bug with it. CHARLES: Really? KATIE: Yeah, but I don't do a lot of Ember re-add-ons. I do a lot of EmberCLI-ish add-ons. It can't change versions of EmberCLI. Not yet, we're working on that. I get some weird npm errors when I tried it but I haven't dug into it much yet. CHARLES: I don't want to dig too much into the mechanics but even when I first heard about it, I was like, "How does that even work?" Just replacing all the dependencies and having a separate node modules directory and bower and I'm like, "Man, there's so many moving parts." It was one of those things where we're so ambitious. I didn't even think it was possible. Or I didn't even think about writing it myself or whatever. It's one of those like, "Wow, okay. It can be done." ALEX: This exists now. CHARLES: Yeah. ALEX: Add-on authors are accountable now for making their add-ons work with revisions or versions a few points back like you said but it makes it so easy. The accountability is hardly accountability, turning Ember-try. It's really amazing. KATIE: What I'm laughing about is that what it actually does is not very sophisticated or crazy at all. For instance, for bower, it moves your existing bower components structure. It's a placeholder. It changes the bower.json run install. Then after the scenario, it put's everything back. CHARLES: But I don't know, it sounds so hard. It's intimidating. You got all this state and you got to make sure you put it all back. What do you do with if something hit you and abort midway. I'm sure you had to think about and deal with all that stuff at some point. ALEX: I kill my tests all the time in Ember-try and I was like, "Oops." I forgot I shouldn't do that just like for this. KATIE: Yeah, it doesn't recover so well. It's pretty hard to do things on process exit in node correctly, at least and I don't think I gotten it quite right. But there is a clean up command. Unfortunately, with the way it interacts with EmberCLIs dependency tracker so when you run an Ember command, Ember checks them to make sure all your dependencies are installed. If you still have the different bower.json and install haven't run, you have to run install before you can run the cleanup command which is kind of a drag. CHARLES: I have one final question about Ember-try. Have you given any thought to how this might be extracted and more generally applicable to the greater JavaScript ecosystem because I see this is something that Ember certainly was a trailblazer in this area. Some of these ideas came from Rails and other places. This is going to be more generally applicable and had you given thought to extracting that? KATIE: Yeah, some of [inaudible] since we first did it because we realized very early on that it doesn't depend on EmberCLI. I didn't even using it as a command line arguments parser which doesn't seem too important. But there are some assumptions we get to make. For it being an Ember app, we know how EmberCLI was structured. Some of those assumptions, I wouldn't really know with the greater node community and I gotten those assumption might not be possible at all because they don't have the standards we have on EmberCLI. It generated EmberCLI. There's generally certain things that are in place in Ember [inaudible] works for [inaudible] so there's no part of it that could be extracted. But I worry about some assumptions about no modules always being in the directory that they are in because then can be link the node modules above it. In EmberCLI, it usually doesn't support that. But other places, obviously have to. I realized that it could definitely happen but I'm not so sure that I'd want to personally support that because it's a little bit time commitment. CHARLES: Right. Maybe if someone from the outside want to step in involuntarily, you might work with it but not to personally champion that cause. KATIE: Definitely. I think it would be really cool and I do think it will end up having its own [inaudible] parser eventually, just to be able to do things like different EmberCLI versions. As long as it's not part of EmberCLI, I think that would be less confusing, though. In theory, that can be done with an EmberCLI still but I'm not clear on that. I've had people talk to me about that and I haven't fully process it yet. CHARLES: Right. Alex you mentioned something earlier I had not thought about but that was the technologies like Ember-try keep the add-on community accountable and keep them healthy by making sure that add-ons are working across a multiplicity of Ember versions and working in conjunction with other add-ons that might have version ranges. Katie, you've been a critical part of that effort. But there's also something else that you've been critical part of that you built from the ground up and that is Ember Observer. That is a different way of keeping add-ons accountable. But I think perhaps, even in a more valuable way, more of a social engineering way and that's through the creation of Ember Observer. Maybe we can talk about Ember Observer a little bit. What it is and what gave you the insight that this is something that needs to be built so I'm going to step forward and I'm going to build it? KATIE: I'm definitely going to refer again back to the Rails community. I'm a big fan of Ruby Toolbox. Whenever I needed to jam, I would go there and try to see what was available in that category kind of way. There's variables that have on there. It will have something like the popularity in the number of GitHub stars and the last time it was updated. You can see a lot of inspiration for Ember Observer in there so maybe I should step back and explain the Ember Observer. Ember Observer is a listing of all of the add-ons for the Ember community. Anything that has the Ember add-on keyword will show up there. We pull it off from npm and it all show you all that kind of information: the last updated, the number of GitHub commits, the number of stars, the number of contributors and we put all of that information and a manual view together to put a score on each add-on. You can look at it and we categorized them as well. If you look at a category, say, you're looking at a category for doing models. You would see all of a different model add-ons and be able to look between them and compare them, decide which ones to use. Or if you're thinking of building something, you can go in there and be like, "This already exists. Maybe I should just contribute to an existing thing." What gave me the idea for is I was looking at Ember add-ons, which just shows you the most recently published add-ons for that Ember add-on keyword everyday and every time I was clicking on these add-ons I go, "They did the same thing," and it just seemed like such a waste in [inaudible]. People were creating the same things and then they started clicking into and I was like, "Why they bother clicking into this. It doesn't have anything. It's just an empty add-on." We're pushing add-ons just to try that out so I thought I'd be nice if something filtered that out and I happened to have some time so I got started and dragged my husband, Phil into it. He's also an Ember and Rails developers so that's pretty convenient and my friend, Lew. Now, Michelle works on it a little bit as well. That's what drove us to build it and it's been pretty cool. I like looking all the add-ons when they come up anyway. I feel like it's not any actual work for me. It's quicker than my email each day to look at the new add-ons. ALEX: How many new add-ons are published every day on average? KATIE: On average, it's probably four to six maybe but it varies widely. If you get a holiday, you'll get like 20 add-ons because people have time off. You know, if somebody just feeling the grind at two and you'll notice that the add-on struck commeasurably too. It would be else that come on the same kind of week. ALEX: You mentioned that an add-on gets a score. Can you explain that score and how you rate at add-ons? KATIE: Sure. The score is most driven by details about the add-ons. There's Ember factors that go into it and it's out of 10 points. Five of them are from purely mechanical things, whether or not there's been more than two Git commits in the last three months, whether or not there's been a release in the last three months, whether or not they're in the top 10% of add-ons, top 10% of npm downloads for add-ons, whether they're in the top 10% of GitHub stars for add-ons. ALEX: I know that I'm a very competitive person but it also applies to software. Not just on sports or other types of competition. But I remember a moment and I'm an add-on author and my add-on had a 9 out of 10. I was just about to push some code and just about to get a release. Even before that happened, it went to a ten. The amount of satisfaction I got is kind of ridiculous. But I like it. I like the scoring system, not just for myself but also for helping me discover add-ons and picking the ones that might be right for me. I'll check out any add-on basically that might fit the description. As long as there's a readme, I'll go check it out. But it still helps along with the categorization. CHARLES: Correct me if I'm wrong but I believe you can achieve an 8 out of 10 without it being a popularity contest. By saying there's a certain concrete steps that you can take to make sure that you have test and you have a readme, that's a thing of substance. I don't remember what all the criteria are but you can get a high score without getting into the how many stars or if you're in the top 10%. I think that's awesome. But it does mean that if I see an add-on with a five or something like that, it means that they're not taking my concrete steps or it might not be as well-maintained. You know, that's definitely something to take into account. I'm curious if there are any different parameters you've thought, tweaks you thought of making to the system. Because this gets to second part of that, what things had you considered just throughout out of hand, as maybe not good ways to rate add-ons. KATIE: We haven't thought about everything. I don't particularly like the popularity aspect of it. It did feel necessary to include it in some way. The stars and theory are representative of interest, not as much as popularity but it probably gets into popularity as well then downloads are inferior for popularity. But the problem downloads and I found this happening more and more frequently is that if large companies start publishing their own add-ons, then they have a lot of developers, they are going through the roof on those downloads so they're getting that to a point that just from their own developers and I have no way of knowing if anybody else is using this. CHARLES: Probably their continuous integration with containers, right? Like if it's running on Travis or Circle, it's just sitting there spinning like pumping the download numbers. KATIE: Yeah and that frustrates me quite a bit. But I haven't found another thing that really representative popularity. Unfortunately, with npm you can get the download counts but you can't tell where they came from. There's no way to do that. I simply would like to see the things that are popular, they rate higher than the they currently do, like you said either, it's 10 points go without the popularity coming to affect it. But you do need a collaboration with at least one of the person to get eight points. If you give seven by yourself, you need to have another contributor. If you only have one contributor, you don't get that point because that's trying to be representative of sort of a bust factor but it's not truly there. You can just have one commit from somebody else to get that. CHARLES: Of all the pieces, I think that's totally fair. KATIE: Yeah, there's definitely a few other things I have in mind to bring in to the metrics but we're not quite there yet. I need to entirely refactor how the score is given so it's not exactly out of 10. The idea is to have some questions and some points that are relevant only to certain categories about us. Whether or not in add-ons testing its different versions of Ember, it might have matters for add-on but it's only adding 10 for CLI or whether or not they have a recent release. Might not matter if it's kind of a one-off with the sass plugin or something more of the build tool that doesn't change for everyone. CHARLES: Yeah, I've noticed we have that happened a couple times where we've got a component that just wraps a type of input. Until the HTML is back changes or a major API changes happens in Ember, there's no need to change it. I can definitely see that. How do you market it is like something that changes infrequently. Is it just that add-on author says it's done? You give them a bigger window or something like that? KATIE: There are probably some sort of categories that fall under that. For the input, I think if it's doing something that Ember producing components, it probably want to be upgrading every CLI, at least every three months. I think in that case, it's probably fair to require an update within that period of time. But some of the things that are more close to Broccoli like as they are in Ember add-ons, that make sense to not have that requirement. Maybe not that's the [inaudible] exact example of the kind of questions that [inaudible]. ALEX: In Ember add-on was the first time that I gave back to the open source community. It was my first open source project and Ember Observer really helped me along the way to say what is an open source best practice and I thought that was really cool. Now, it sounds like with some of the point totals, you're leaning towards Ember best practices to help Ember add-on authors along that way. I think that's really awesome and very, very useful. I would not like to see what the Ember add-on ecosystem would look like without Katie. It would be a very different place. KATIE: Thanks. I'm glad it had some help on that and I'm affecting add-on authors. I actually didn't originally think about it when I was first building it and I was really hoping to help consuming of add-ons but it really has kind of driven out people finding add-ons to not build because they contribute to existing ones. Also, driving them for the score because as you said, people get very competitive. I really didn't realize what kind of drive the score to be because to me I'm like, "Somebody else's score me. How dare you?" [Laughter] KATIE: I have had people say that to me, "How dare you score me? How dare you score my add-ons?" Well, it's mostly computerized. Even the review was manual but only thing about it has any sort of leeway is are there meaningful tests. That's really the only thing when I go through an add-on is whether or not that has any sort of leeway for the judgment of the person that's doing the reading. If there's a readme and we're kind of a rubric so that goes if you have anything in there that's other than the default Ember [inaudible] readme. Whether or not there's a build that is based entirely whether or not there's a CI tag in readme, these are for the owner to go look for them [inaudible]. We hope to automate that so I don't have to keep looking for those. A lot of add-ons turned out have to have builds when they didn't have any meaningful tests. They just had tests so that's kind of confusing. CHARLES: What do you do in that situation? You actually manually review it so that add-on would not get the point for tests. KATIE: No, they don't get the point for test and if they don't get the point for test, I put 'N/A' for whether or not, they have built so it doesn't apply. It doesn't mean anything to me if they haven't build their own test. CHARLES: Right, that makes sense. A lot of it is automated but it still sounds like consume some of your time, some of Phil's time, some of Michelle's time. I guess, my question is do you accept donations or a way that people can contribute because I see this as kind of part of the critical infrastructure of the community at this point. There might be some people out there who think, "Maybe, I could help in some way." Is there a way that people can help? If so, I'd love to hear about it. KATIE: We don't have any sort of donation or anything like that. I mean, we should. We consider it primarily just part of our open source works, part of our contributions to the community because we also make a great deal of use out of the community. Fortunately, it's not very expensive to run. It's only $20 a month VPS. Other than time, it's not really consuming very many resources. That may change over time. The number of hits is increasing and we're doing some more resource-intensive things like Code Search and we're running Ember-try scenarios from the top 100 add-ons to generate compatibility tables. It hasn't been the most reliable. Think about if you're trying to do nvm-installed times 100 add-ons, times every day, times the different Ember dependency settings so it's been very much like a game of whack-a-mole but for now, it's not bad. But we probably should think about some sort of donation by then. Maybe something that writes out the exact numerical cost of something like Ember Observer. The API is getting about 130,000 hits a month but that's the API so that's some number of quests per person. [inaudible] tells me something about 12,000 visitors each month. CHARLES: Does Ember Observer has an API? Are there any the third-party apps that you know about, that people built on top of the Ember Observer? KATIE: None that have been kind of public. I know a couple of private companies seem to be hitting the API but it's not a public API. It's really not public yet. I'm literally process of switching over to JSON API and at some point, I'll make some portion of that -- a public API -- but it's pretty hard to support that at the same time. It change Ember server pretty frequently and do any kind of migrations we need to do. Ember add-ons does all the scores from us from API end point. CHARLES: I actually wasn't aware. I remember the announcement of Code Search but how do you kind of see the usage of that? What's the primary use case when you would use Code Search on Ember add-on or Ember Observers? KATIE: I think the primary use case is if you're looking for how to use a feature. If you're creating an add-on and you want to know how to use certain hooks like [inaudible] or something like that. You can do the Code Search for that and see what other add-ons are doing. It's only searching Ember add-ons that have the repository are all set so you'll only find Ember results. That will be nicer compared to searching GitHub. Then I find another use case is more by the core team to see who is using what APIs and whether or not they can deprecate something or change something or something has become widely used since we're pretty excited about that possibility, we've never ever been searching. ALEX: That is brilliant. CHARLES: Yeah, that's fantastic. The other question that I had was this running Ember-try scenarios on the top 100 add-ons and that's something that you're doing now. Are you actually reflecting that in the Ember Observer interface? Is that an information or is that an experimental feature? Or is that reflected all the way through so if I go to Ember Observer today, I'll see that information based on those computations? KATIE: I start playing in the Ember Observer interface. It's only for the top 100 add-ons currently but hopefully expanding that to all add-ons. Especially, for few months but maybe it's not easy to notice. The only on top 100 add-ons would be on the right side bar and there will be a list of the scenarios we ran it with and whether or not it pass or not. There's add-on information there. The top 100, I'll link to right on the main page of Ember Observer so you can see in the front. CHARLES: How do you get that information back to the author of that top add-on? KATIE: We haven't actually done that. It's just on Ember Observer. It's more meant for consumers to be able to see that this add-on is compatible with all these version. We're not using their scenarios. We're using our own scenarios saying Ember from this version to this version, unless they have specified that version compatibility thing and then we'll use those auto-generated scenarios. This might get harder for add-ons that have complex scenarios so it need something else to vary along with the versions like Ember Data or maybe they're using Liquid Fire and Liquid Fire have these three different version and for each versions, it's being used. For those, we'll just have things which are unable to test this. But hopefully, this is still providing some useful information for some add-ons. A lot of add-ons, their build won't run unless they commit. In this case, this is running every night so new Ember versions released will see if that fails. On other side of it, we have a dashboard where we can see which add-ons failed and maybe see if a new commit to something broke a bunch of add-ons. It commits to something like Ember and for CLI Ember-Gate, one of the main things. CHARLES: I know that certainly that right after we get over this podcast, I'm going and running or checking up all add-ons that we maintain and making sure everything is copacetic. If you guys see me take off my headphones and dash out the door, you know where I'm going. KATIE: Got you. ALEX: I just have a further comment that I'm excited for the public API of Ember Observer just because I've been thinking a lot about data visualization lately. I think it would be really cool tool to do a deprecated API, like one of those bubble charts where like the area that's covered by this deprecated API -- I'm doing a bad job of explaining this -- just like the most use deprecated API methods and visualizing that, I think they'll be really interesting to see it. CHARLES: Right, seeing how they spread across the add-on. ALEX: Yeah, or just all add-ons in general. KATIE: I am most nervous about a public API for Code Search, though because it's a little bit resource-intensive so just freaking out a little bit about the potential of a public API for it. But an Ember Observer client is open source. If you want to add anything to the app, that I consider as public think it is. Adding to that, I really do want to figure out some way to have like a performance budget for when people add to the client because sometimes I'll get people who want to add features and I'm like, "That's just going to screw all of it. It's going to be a problem for all Ember Observer and it's going to make everything slower and it's really little slow. But JSON API fortunately, I have kind of a beta version that running and it's going to be much faster, thank God. I probably shouldn't said that either. CHARLES: Definitely we want to get that donation bin set up before the API goes public. Okay, let's turn to the internet now and we'll answer some of the questions that got twitted in. we've got a question from Jonathan Jackson and he wanted to ask you, "Where have you seen the most change in the Ember ecosystem since last EmberConf?" which was March of 2016. KATIE: There was definitely fewer add-ons being published but the add-ons that are being published are kind of, say more grown up things. We've got... I don't know if engines was before or after March. I have no idea. Time has one of those things that engines and then people doing things related to Fastfood so things are coming from more collaborative efforts, I think. This is just my gut-feeling. I have no data on this. Isn't a gut-feeling from looking at add-ons and then there's a lot of add-ons that are coming out that are specific to a particular company. I think that maybe, I hope representative of more companies getting it to Ember but hopefully, they'll make things more generic and share them back. The other problem with the popularity is like about before, where big company is getting itself into the top 100 list, probably with just its own employees only appear over the summer. I tried a few different ways to mutate the algorithm to try to get them out of there but there was no solution there. It's much fewer, novel things. Very rarely do I look at Ember add-ons and I'd be like, "Oh, that's great," but when I do, it's something very exciting. CHARLES: Right so there's a level of maturity that we're starting to see. Then I actually think that there is something in the story too, of there are now larger companies with big, big code bases that have lots of fan out on their dependency tree that just weren't there before. KATIE: Definitely. I don't think some of the large companies were there before but I think some of the largest companies are probably keeping most of their add-ons private so there's kind of mid-range of company that's big enough to donate things or willing to put things open source. A few of these companies that can have a lot of add-ons now and a lot of them are very similar to things that have already existed so you're going to be like, "I don't know why I use this," but they obviously make changes for some reason. CHARLES: The other thing that I want to talk to you about, before we wrap up, is you actually are in partnership in Code All Day. What kind of business is that? What is it you guys do? What's it like running your company, while on the same time, you're kind of managing these large pieces of the Ember ecosystem? KATIE: Code All Day is very small. It's just me and Michelle. It's a consulting company, we kind of partnered together after we left to startup and decided to do consulting together. We primarily do Ember projects, also some Rails and we try to work together and we love test-driven things. It's pretty kind of loose end. We ended up running it since it's just a partnership. We don't have any employees. But Ember Observer will take up a lot of our time and we really had an idea that it might help us get clients that way so I suppose it kind of helps our credibility but it hasn't really been great for leads so much. But fortunately, there hasn't been a big problem for us. We really enjoys spending our time. We enjoy the flexibility that consulting gives us and while that flexibility is what's going to making these things keep running. CHARLES: All right-y. Well, are there any kind of skunkworks, stealth, secret things you've got brewing in the lab, crazy ideas that you might be ready to give us a sneak preview about for inquiring minds that may want to know? KATIE: Some of them are really [inaudible] which is redoing Ember Observer with JSON API instead of currently, it's using ActiveModel serializers, which is a kind of custom API to Rails and [inaudible] fortunately, it's an API now. They're removing something called JSON API Resources so that will get the performance of Ember Observer much better and that's pretty much my primary focus at the moment. I don't really have any big skunkworks, exciting projects. I have far off ideas that hopefully will materialize into some sort of skunkworks projects. CHARLES: All right. Well, fantastic. I want to say thank you, Katie for coming on the show. I know that you are kind of a hero of mine. I think a lot of people come to our community and they see like, "Where's the value in being a member of this community, in terms of the things that I can take out of it? What does it provide for me?" And you demonstrate on a day-to-day basis, asking what you can do for your community, rather than what your community can do for you, to paraphrase JFK. I think you live that every day so I look up to you very much in that. Thank you for being such a [inaudible] of the community which I'm a part of and thank you for coming on the show. KATIE: I'm very happy that I've been here and thank you. I use a lot of your guys add-ons and it's really the community has given so much to me, which is why I ever want to participate in it. It's really great group of people. CHARLES: Yep, all right-y. Well, bye everybody. ALEX: Bye.
Steve Klabnik @steveklabnik | Blog | GitHub Show Notes: 02:56 - Getting Into Rust 05:51 - Working on Rust for Mozilla 07:01 - Writing Documentation and Preventing Burnout 13:24 - The Rust Programming Language 18:45 - Rewriting Firefox in Rust 21:20 - High-level Functions 25:23 - Typesystem and Concurrency 36:35 - Rust and Web Developers; Digging Into Rust on a Deeper Level 43:46 - The Rust Ecosystem and Using Rust on a Day-to-Day Basis 48:38 - The Rust Book Resources: Rust For Rubyists Cargo Servo Application Binary Interface (ABI) MetaLanguage (ML) Tokio Systems Programming intermezzOS Steve Klabnik: Exploring Ruby Through Rust What's new with “The Rust Programming Language”? rustbook Transcript: CHARLES: Hello everybody and welcome to The Frontside Podcast episode 51. I'm here, my name is Charles Lowell. I'll be hosting today. With me is Chris Freeman, also of The Frontside and with us is Steve Klabnik. Now, most of you probably heard of Steve before. My first encounter with Steve was actually at the LoneStarRuby Conference back in... Gosh, I don't know. It was many, many years ago and he was giving a talk on Shoes, which I also had never heard of before. It was a wonderful story of a code archaeology project where he was kind of investigating, rehabilitating, and in carrying forward a project that the 'why the lucky stiff' had done. That was a wonderful introduction but it was certainly not the last time that I encountered him in his writings and in talks and stuff, mostly within the Ruby community. But it popped up again and again, talking about Rust APIs and always making a point to take a good knowledge that he'd learned and spread it around. Personally, I've lost track of Steve or hadn't really heard much of what he was doing for a while. But then Chris came into the office and he was always talking about this language called Rust. While I've heard Rust, Chris was just all about it and wanted to have Steve come on the show because it turns out that Steve, you've been really, really, really into Rust these last few years and sounds like concentrating most of your work there. STEVE: That is totally true and accurate. Also to go back a bit, that means that you are in attendance for my very first conference talk ever. CHARLES: Really? STEVE: That was literally the first one. CHARLES: Wow, it was a great start. That was a great story. It was educational and also touching. STEVE: Thank you. It's actually interesting because what happened was is that someone else who works on Shoes have encouraged me to submit to RubyConf and I was like, "Who would want to hear me talk at a conference?" I submitted the talk and RubyConf accepted it and I was really excited. Then a bunch of other conferences noticed and two other conferences had asked me to give a talk before RubyConf happens and LoneStar was one of them and it was the first one chronologically. That moment was also very special to me as well. CHARLES: Fantastic. What year was that? STEVE: I want to say it was like 2012 or 2011. It's really hard for me to pay attention to time and date. My history is so complicated that I often forget. I've literally told people that I'm 10 years old or younger than I am because I would like mess up to date on the things. It just happens. CHARLES: Yeah, but it was a while ago and it's been quite a journey, in between now and then. STEVE: Yeah, definitely and you're also definitely right. It is now literally my day job to work on Rust so it is definitely the focus of most of my efforts. Partly, why I made that happen was because it was the focus of all my hobby efforts before I made my job. It's definitely been a couple of years that I've been a full-time on all the Rust stuff. CHARLES: How was it that you actually got into Rust? How did you hear about it before everybody else and how did it capture your attention? STEVE: I've always liked programming languages and learning different programming languages. Ruby was sort of where I became known professionally. But it wasn't the first language that I knew and I knew it was never going to be the last. As much as I always loved Ruby and I'm like literally have a tattoo on my body so I will be with Ruby forever. I always try to learn new stuff and I find it exciting. I'm from middle of nowhere, Pennsylvania in the suburbs of Pittsburgh on a cattle farm and I was visiting my parents for Christmas one year. There's not really a whole lot to do out of the very small town so I was just reading the internet, as usual and it turns out that that was the day that Rust 0.5 had been released. I saw this release announcement go by and I was like, "I vaguely heard of this programming language once or twice maybe. I don't have anything to do. Let's give it a try." I downloaded and installed it. I looked at their tutorial and the tutorial has a problem that a lot of tutorials had, which is I read it, I said, "This all makes sense," I tried this down to write a program, and I had no idea how to actually write a program in it at all. I'm just completely confused. I couldn't actually apply the sort of syntax stuff that I learned. At the same time, I was going to be working on this hypermedia book -- that was my plans for that trip -- as always, you just rewrite your tooling over and over again. You [inaudible] like, "Just don't write the thing. Write the tools that make the thing," so I wanted to try out a new way to take mark down and generate PDFs in HTML, involving pandoc. I sort of had that all set up and I said, "Well, let me give this a try run. What I'm going to do is I'm going to write down what I learned in Rust as I learned it," and sort of from a Ruby programmers perspective, I'll use that and working with my new tooling to see if it works to actually work on the real book and it will also help me understand Rust better because one of the reasons why I do all this sort of teaching and advocacy is because I think it helps me learn. Just as much as I like helping other people learn stuff, I find that the repetition and being forced to explain something to someone else really make sure that I understand what I'm talking about. That's what the thing called Rust for Rubyist became boring. I'm a sucker for alliteration and that sort of became the first to tutorial for Rust from outside of the Rust projects proper. From there, I went on to submit some pull requests because everything's open source so I wrote some documentation and funny enough, my first ever pull request to Rust was actually rejected based on procedural grounds. At the time, they didn't actually accept pull request to master, they accept this other weird branch and GitHub don't have the ability to re-target the branch of the pull request. I also, always like this story because the thing that I now on the core team of, like my first attempt at getting involved was wrong and was turned down. But I'd fixed that pull request issue and got that in but it is kind of kept working on an open source capacity for a while and then decided to ask Mozilla if I can make it my job. Luckily they said yes. CHARLES: Wow, so what? Your job at Mozilla, like you just kind of showed up and said, "I would like to have a pretty cool, awesome job, working on this brand new language," and they were like, "Sure, come on in?" STEVE: To some degree, yes. That's one way of putting it. There is always the devil in these details. The first thing is that that wouldn't have worked if I had wanted a different kind of job. But when someone comes to you and says, "I would like to write documentation for you all day," you go, "Oh, my gosh. This literally never happens." If I had wanted to like work on the compiler, I'm pretty sure they would have said no. But because they knew documentation was important and they wanted documentation and because I had already been basically doing that job in an open source way, it's like I've had a year-long interview already. Then finally, they actually didn't have headcount at the time so I actually moved on as a contractor initially and had to do some freelance work and then eventually, once we were able to hire a new person kind of got it in. They're like a cool kid story. It's like, "Oh, yeah. I totally asked Mozilla for my perfect dream job and they just gave it to me," but like that's not really the way that it works. CHARLES: Got you. That actually leads me into a question that I have wanted to ask you. You write a very good documentation as your day job and documentation is extremely hard. For me, it is extremely hard to get and stay motivated to document something that I've worked on. I think that is probably a common enough experience for programmers. We don't recognize because we use documentation that it's extremely valuable and yet, it still this thing that is just a constant uphill battle. I'm curious, how do you manage to stay motivated to write documentation for an entire programming language over the span of years? STEVE: As I'm often want to do, this has like three or four different components. I guess, there's a couple of different things involved. The first one is that I actually got accepted to go to English grad school, although I ended up not pursuing that. Like writing, it's something I have just always enjoyed. I got a Bachelor in Computer Science but then I was going to go to grad school for English and due to university shenanigans, it didn't really work out. They told me I was going to get a free ride and then accepted me and then they were like, "Oh, wait sorry. You have to pay for this." And I was like, "Wait, sorry. No, I'm not doing this anymore. That's ridiculous." That's kind of always a predilection for writing and I think that the reason why that is because I grew up basically like on Slashdot and eventually then on Dig and Reddit and all these other things. I've kind of been writing a couple paragraphs a day, basically every day in my life since I was a little kid. I think that's something that's sort of like underappreciated. Documentation is hard but it's like a skill, like any other thing. Programmers will say, "I really want to learn TDD so I'm going to make myself do some TDD, I'm going to practice it, I'm going to focus on it and that's going to be a skill that I'm going to improve," and then they see documentation, and they kind of think it's this thing that you either have the skill or you don't. But writing is just another thing like anything else that you can practice at and get better. I think maybe it's because it's a little bit farther away from the wheel house of what you do day to day, that people aren't as interested in it but it is something you're truly interested in, I think the best way to get better is just to do it and do it a lot. I say this is I'm kind of in the middle of a little bit of writer's block at the moment to be honest. Then finally, I think the other reason that I'm motivated about docs is that I actually believe that documentation is an exercise in empathy. Like good documentation, the ideal as a programmer, the ideal thing that happens in documentation is I have a question about how to use something, I go to the documentation, and it says the exact sentence that answers my exact question. As those varying degrees of vaguely gives you the right idea, versus literally tells you exactly what to do. I think that the way that you can accomplish that excellent documentation is by understanding what your users need and then preemptively figuring it and/or writing that down. I think that that requires being able to put yourself in their shoes to some degree. I'm not going to say that that's a thing that I am perfect at but I think that a valuable skill when trying to improve docs's like figure out what they actually need and then give it to them. It's doesn't always have to be in that order, like sometimes people will fail to find the thing they need, tell you what you need, and then you give it to them. That's a strategy I've used a lot and that's one reason why I hang out in the Rust IRC all the time, helping people is for a very long time, I would like sit in IRC, someone would ask a question, I would answer the question, I'd go look in the docs and see if they could have figured out themselves. If they couldn't, that would be might next doc PR. It's just like even if it's just a couple sentences like add the question from IRC into the documentation and then just do that over and over and over again and then eventually, people start learning from the docs instead of actually ask questions because they already found what they needed. CHARLES: Right. I have a question about that because once you develop those skill, I think you also still run the risk of like burning out. I know that one of the reasons I tend to always fall back to like, "I'm going to spend my time doing coding instead of documentation," Or, "I'm going to spend my time --" Even with TDD is a great example is like with TDD you get to experience those short term wins. I think that kind of prevents you from burning out, where sometimes when I'm writing documentation, it feels like I'm screaming at the void. I might be screaming really loud and really, really well but I feel like a lot of times, I'm not experiencing those wins and I'm wondering if you have any tips for like experiencing those wins. Or getting that feedback to kind of keep you motivated and keep you doing the job. Also, trying to push the level of your own documentation skill and communications skill. STEVE: Yeah, experiencing the wins is definitely a part of it. But one of the other things that is sort of part of it is that like I do the opposite. I do a lot of coding but that's my side projects. When I get fed up with writing documentation, I maintain the [inaudible] implementation that Cargo uses to resolve Rust packages, for example. If I'm feeling a little stuck on docs, I'll go write some software and then come back to the docs so that kind of help with burnout. Another thing is that I think I'm just like perpetually in a state of just barely above burnout anyway so that also sort of factors in I guess. You know, it's like Bruce Banner. The secret is that I'm always angry so -- CHARLES: So you work on open source, is that what you're saying? STEVE: Yeah, exactly. We're working on open source all the time. I've been lucky enough to make open source as my job for, basically almost my entire professional career. Although not totally. You know, at some point, you just kind of get used to it. But in terms of experience and the wins, this is also one of the reasons why I like to teach beginners specifically is that beginners allow you to remember what it's like to be a beginner, which is also part of building the empathy. By interacting with beginners a lot, you also get a lot of those wins because beginners usually ask easy questions so it's easy to figure out the answer that stuff. Then you've got that positive feedback loop kind of going. To me it's maybe not IRC literally for every project but answering questions on Stack Overflow, or whatever message board forum you have, or Twitter, like actually interacting with other people. For me at least, that's how I get that kind of sense of not screaming into the void that you have to like go into the void and find the other people there, I guess, that I'm just like come to you necessarily. CHARLES: Speaking of empathy for beginners, it just occurred to me that we didn't actually talk about what Rust is. We probably should do that. Why don't you tell us a little bit about the Rust, language, as well as, you've mentioned Cargo and [inaudible] ecosystem for us as well? Let's talk about that. STEVE: Yeah, totally. Basically, Rust is a new-ish. I should stop saying new because it's almost not really at this point. A kind of new-ish programming language, heavily sponsored by Mozilla in development. Its idea is to become a new low-level programming language. But I always hesitate when I say this because one of my old pitches for Rust used to be like, "Rust could be used anywhere. You can use C." Then people go, "I would never write, C is so cool. Rust is not for me." I'm like not do that. But the reason that people don't use C is a lot of the problems that we are also trying to fix. I guess the primary differentiator for Rust in terms of like programming languages theory is that it is safe and safety as they got specific meaning. But basically C is a very dangerous sharp tool and you can cut yourself and people who use those tools often do cut themselves, whereas Rust is like it's got a safety guard on it. It's a compiled language so its compiler actively prevents you from making some of the worst mistakes that you can make in a low-level programming language like C. It turns out that when you start building up these sort of safe abstractions on top of these really fundamentally low-level details, you actually end up with a relatively high-level programming language. I talked to a lot of people, for example from JavaScript or Ruby world or Python world who come to Rust that are modulus, some libraries, and other things. This is actually high-level enough that I feel like I could do this instead of review JavaScript all day and I would be just as comfortable. The other day, I did a little bit pair programming and we actually recreated a JavaScript library in Rust that had virtually the same interface because like you can actually build relatively high-level things so pass an enclosure to a function that does some stuff is totally normal and Rust world. That's also very familiar to people that come from the Ruby, JavaScript, Python background. Also then, as part of that is we also culturally like Rust the projects, not Rust the programming language, really, really cares about helping people understand what systems programming and like lower-level programming means. A lot of people will not program and in C or C++ because they have no idea how to get help or to learn because many people in the low-level space have this RTFM attitude or like, "If you don't know what you're doing, then get out of here," whereas in Rust world, if you ask an extremely basic question, we're like, "Welcome. We would love to have you. I would be very happy to like walk you through," like explaining how that works on these kind of low-level details. Part of the culture of Rust is to bring this sort of low-level programming to people that have rejected it before for various reasons. The reason that Mozilla cares and the reason Mozilla sponsored the project is that Firefox is written in C++, so like four million lines of C++ last I checked. Last time we did a security audit of a really pants-on-fire, terrible security bugs in Firefox, I go to this website and now they run arbitrary code on my machine kinds of terrifying bugs. Basically happened because C++ is dangerous and sharp. If you screw up, there's the kind of bad things that can happen. About 50% of those security issues in Firefox would be eliminated at compile time by the Rust compiler. That's a really huge win in general so the idea is that we are slowly rewriting Firefox and Rust over time. That's one angle of why Mozilla cares about Rust. The second part is Servo, which is a rendering engine that's built in Rust from the ground up. If you think about Firefox proper, it's got Gecko as the rendering engine inside that actually determines where things go on the page and stuff. We're also writing a new one of those from scratch called Servo in Rust. That was also to prove that the language was doing the kind of things that we need it to do. But also Servo is an impressive piece of technology in its own right so it might become its own thing and/or bits and pieces of it are already making their way into Firefox. It's kind of also a way to improve our core products. That's why Mozilla cares. CHRIS: I was curious with Servo and Servo is the layout engine. Do you know if there are any plans to write a JavaScript runtime in Rust? STEVE: That question is complicated. Sort of what it boils down to is that a Git is inherently kind of unsafe by Rust definition of unsafety. It's actually controversial like when I talk to people that work on JavaScript engines, they're pretty much 50/50 split between, "Oh, yeah. Totally Let's absolutely rewrite the whole thing in Rust because we rewrite it every two or three years anyway from scratch so why not use Rust next time," to, "Since it's massively unsafe anyway, I don't see what benefit I would actually get so why not just stick with what we know." It's like very extreme ends. It's definitely feasible but I don't know if it's going to happen and/or when exactly. CHARLES: There were two questions that I had kind of to unpack some of the things that you said in there that were just really interesting to me. You said Mozilla plans to incrementally rewrite Firefox in Rust, where it's currently four million lines of C++. Now, how does that actually work where you're talking about swapping out large parts of the runtime with something that's written in a completely separate language? How does that communication happen between those language boundaries? STEVE: There's this concept called an ABI, not API. It may sound very similar -- Application Binary Interface. What this really boils down to is assembly language does not have function calls. That's not a concept, that's in assembly. People have come up with, "If I write a function and I map it to assembly code, what's the convention about how I do things like passing an argument and return values? How those all that stuff actually work?" Because assembly is so low-level, there are multiple different ways that you can make that happen. There's a number of different specifications how to make that work so C, the programming language, has a very straightforward ABI so any programming language that knows how to call C functions, uses these convention at the assembly level to do the function call. What you can do with Rust is you can say, "Please make this Rust function follow the C calling convention," in that way, any sort of thing that knows how to call C functions can call Rust functions directly. By doing that, you can sort of say like take a chunk of code, write it in Rust, expose a C interface, and then anything that knows how to talk to C, which is virtually everything, can talk to Rust equally as well. For example, one of the earliest production uses of Rust was actually inside of a Ruby gem because Ruby can be extended to C and Ruby knows how to have C extensions. It doesn't actually need to know that it's literally written in C. It just needs to know how to generate the assembly to call the correct functions. That's actually like a thing. Basically, the process is like write a component in Rust, expose this language independent wrapper, and then call into it like you would in C code. CHARLES: So it's really, just they're sharing memory and sharing is like right there in the process and there's no overhead for the intercommunication, it sounds like? STEVE: Yeah, exactly. You could also do all the regular things with JSON-RPC over a socket or whatever if you wanted to. The most efficient way is to literally include it as your binary just like anything else. CHARLES: Which kind of leads me into my next question, which is Rubyist and Pythonista people coming from JavaScript, one of the reasons we don't like to write in C is because, as you mentioned, they're so sharp so we have safety so that you don't have to worry about memory allocation for the most part, the garbage collector kind of has your back there. You access things by reference so you never have to worry about accessing memory. That's not there but kind of the conventional wisdom is that that all comes with a pretty big cost. It's like really, really expensive. I know when I was getting into Ruby and I was explaining a lot of the pushback I got from people doing C and even Java, it was like, "It's going to be super slow because all those high-level features that you love so much, you're paying a lot. A lot for them." My understanding is that's not really true with Rust. Is that fair to say? STEVE: Well, Rust does not have a garbage collector so, yes, it does not pay that cost because it doesn't exist. Now, that also raises a bunch of other interesting questions and basically what it boils down to is a compiler and especially one that has a typesystem, basically asks you to declare certain properties of your code like this function takes one argument only and it's always a string. That's sort of what type safety means. It kind of like a fundamental level. One of the ways that Rust uses type safety is to say, "This pointer to this memory always points to valid memory," and you have to be able to demonstrate that to me at compile time. From those couple of sentences, that sounds extremely complicated but it turns out that most programming code is written in a way that actually works this way. For example, like I'd talk to Yehuda Katz a number of times because we're friends, he also works on the Rust project and he's also well-known in JavaScript and to you all, I would assume. It turns out that the style of Rust code I write is actually extremely similar to the style of JavaScript code that I write is just sometimes there are some tweaks. It is true that those features often do take up a lot of memory and/or rely on any sort of expensive, from a low-level perspective, way of doing things. But it turns out that's actually more of a function of the way that the programming language is made in semantics. You could design a programming language that feels very similar but as very different underlying characteristics. For example, Closures in Rust, the compiler is smart enough to know that if you don't actually capture an environment. Say you're going to add one to every number in a list. You want to do like .map, pass in a closure that takes one argument X and adds one to every single X and then collect that up into like the map join kind of thing, to collect into a new array. That closure that you had passed a map, while it's a closure, it's taking that one argument X and doing X + 1, so it's not really capturing an environment at all. There's actually no reason to allocate a bunch of extra memory because it turns out, it's the same thing as a regular function. The compiler is able to optimize that call away completely to the same thing as if it was a normal function and not a closure, and therefore, you're paying no overhead. Even though, like syntactically, it looks kind of like a closure. Then you're kind of think of that applied to almost everything in Rust. For example, Rust has methods but almost all of them are actually statically dispatched at compile time, as supposed to dynamically dispatched, where you need to look through some sort of object hierarchy because we don't really have inheritance. There's no way to say like this might result to a colon, this class or this class is super class, or this class is super class so I have to do this runtime look up to call functions that just doesn't actually really exist. Part of it is through the fact that these coding patterns don't strictly require this stuff. It's just the way those languages are built and part of it is because as we were building a language, we were extremely sensitive to not include features that would require this really heavy overhead. In a language, that's like a low-level of focus on details, it's extremely hard to talk about the details without code. There's a lot of details, it turns out. CHRIS: One thing that I'm very curious about and one of the things that drew me to Rust actually is the fact that its typesystem is, I guess an ML typesystem. It is like much more [inaudible] to something that you would see in a functional programming language like Haskell, than you would like a regular C++ or Java. CHARLES: Now, a Chris-acronym alert. What is an ML-style typesystem? CHRIS: I'm sure Steve can answer this better than I can but it's a typesystem that uses the Hindley-Milner algorithm for type inference. It does a lot of the heavy lifting for you, in terms of correctness. Is that correct? STEVE: Yeah, I would say more accurately, ML is a programming language. It's the name of the language so by saying like an ML-like typesystem, he means like a Java-type typesystem. It's like a similar statement but about a different language. I always forget what ML stands for specifically but like OCaml has got ML at the end so like OCaml is one of the languages that sort of the family of ML. There's like two branches of functional programming, which of course everything is wrong when you try to organize things this way. Like you could also argue Lisp as a third but there's kind of like the Haskell-style and the ML-style are these two big pillars of functional language stuff and Rust tends to be in the ML sort of family. There's lots of common features between families of programming languages and all that kind of stuff. I think the ultimate point that Chris is trying to make is when I say that Rust is a typesystem, I do not mean it's like Java. There is a wide variety of typesystems and they do all sorts of different things and actually Java has been getting increasingly better over the years as well. But it is much more canned to a functional language in the typesystem, which I think is what you were getting at and serves the actual question, right? CHRIS: Yeah. Actually, I just looked it up and ML stands for MetaLanguage. It is actually is going to serve my question really well. ML was originally designed for theorem improving in math, which is part of why it works really well in functional programming languages. But it also makes sense if you use Rust, how the compiler work from the kinds of things that it catches, like a relatively low effort on your part because it is originally designed to completely prove out a theorem so the compiler is doing that to your program. That leads to my question which is I recently heard someone else on the Rust core team talk about one of the things that Rust really seeks to improve upon is concurrency and parallelism, which is historically very hard. To do that, you could use things like mutexes or reference counting, which Rust has. But they also lean extremely heavily on the typesystem itself to sort of guarantee that your concurrent code is actually going to run safely. On one hand, I'm interested in hearing you expound on that but I'm also really curious how the C, C++, Java programmers take to that sort of thing in Rust because as I understand it, that is a pretty novel approach to that kind of problem. I wonder if there's like pushback from the existing low-level systems community on that stuff. STEVE: I'll do the second part first because it's a little simpler. One thing that I will say is we sort of didn't appreciate over time because we were creating Rust for ourselves, roughly the C++ programmers are working on Firefox, which we had to say for ourselves because I was not literally one of those people but you get the idea, is like assuming that C++ people would be the primary audience. But it turns out that a lot of people that programming C or C++ are pretty happy with it and they like doing things that way. They're a lot smaller of a population than the number of programmers who do not program of those languages, which is true for any language, basically. The sum of all other people is bigger than your specific thing. What that means, I think that in retrospect this seems obvious but at the time, it was like hard to figure out or I definitely did not understand this at that time, that most people would come to Rust from not C or C++ than they would from C and C++, just even by virtue of numbers alone. A lot of the people who are not doing it are not doing it for reasons. They've already rejected it for some sort of purpose and the people who are still doing it often are like happy with what's going on. There's definitely a little skeptical at times of the kinds of things that we can accomplish. Also, our success has been pushing C++ specifically to grow a lot of safety things so we hear a lot of people say like, "In five years, C++ is going to have this tooling that's going to make it also pretty safe, even if it's not as safe as Rust. I'll just wait for that instead." Surprise, low-level programmers are extremely conservative bunch in many instances. The first part, which is the bigger and more interesting one, the typesystem is absolutely how concurrency works in Rust. This is extremely powerful for a number of different reasons. The first one, and I think the fundamental reason why it's done this way is that typesystems don't have any runtime overhead. When you're in a performance-heavy language, that's really the key. Originally, a long ago in Rust, we actually had a garbage collector even, like a very long time ago in Rust. The primary goal was always safety and we thought the only way to accomplish that was with lots of runtime checking, heavy runtime, and all these things. Over time, as the typesystem grew, we realized we could use more and more of a typesystem to eliminate more and more of the runtime because types are checked to compile time so they have no overhead cost, which is awesome. Like Rust references, doing this validation that they're always valid is completely a compile time construct that at runtime, they're literally the same thing as C pointers. That's one reason why the typesystem is really heavily useful for concurrency because you want things to be safe. We also don't want to slow them down. The whole point of concurrency in many instances is to get a speed up. If you introduce too many safety checks to make sure that your concurrency stuff works, you lose all the gains that you were trying to get from being concurrent in the first place. Having that like as low-cost as possible is extremely important. The second one is that concurrent problems are extremely difficult to debug because you need to recreate the exact set of circumstances under which the bug happens. If you have a bug because you have two threads that have a particular access pattern on a particular variable and that's where the bug is introduced, good luck coercing your operating system scheduler into scheduling those two threads at exactly the same way as when the bug happens. To some degree of the way that you fix a lot of concurrency bugs is by introducing an extreme amount of logging and then just kind of let it run and praying that you hit into the situation that causes the bug. That really brutal and doesn't really work. By using the typesystem and verifying it upfront, you just know it will work at runtime because you've already proved the concurrency property before your code even runs. It's also just like a better debugging experience, I think in general. The way that we accomplish this task is extremely novel. I guess I should also say extremely novel to working programmers, like almost all Rust is built off of existing research that has been known in academia for a relatively long time. That's actually one of the places where it gets the name from, it's like taking ten-year old ideas that have a little bit of rust on them, that have found usefulness and bringing them to [inaudible] research. Anyway, the way we accomplish this basically is the typesystem in the standard library, the way that you spin up a new thread, it has a particular type signature and the type signature says, "Only allow the types to be sent to this new thread. There are safe to pass between threads," and/or like, "Only allow references between this thread and that thread of types that are safe to use across thread." What that means is that when you try to spin up a thread and you passes a thing that doesn't work, you get a typesystem error. It turns out this is not concurrent safe collection so it does not have the prerequisite types so therefore, you cannot pass on this thread and you're done. That's sort of like at a core level of how these things work. Then for example, mutex is a type that does have that property so by sticking with non-concurrency thing into a mutex, now you can share it safely. That means we've guaranteed that the compile time that you'd safely done this transfer between threads and that kind of thing. It's not just about mutexes but that's sort of the general approach. The last thing I want to say briefly because I just said a whole bunch of things. I'm sure, I've raised a ton of questions here is that the other powerful thing about using the typesystem for concurrency guarantees is that other people can extend it. If you write a library in Rust, your library will be exactly as concurrency safe as the standard library and as the language itself. It's not like we provide the set of concurrent collections and then we vetted our own implementations and then you're kind of your own or building your own stuff. You can use those exact same types to help guarantee properties on your stuff. Also build alternate threading situations, as well that use the same things and the ecosystem all works together so everything is just concurrency safe by default because it's like a property of typesystems that are being built into the runtime or something. CHRIS: I know that recently, there's been a lot of, I guess excitement about this library called Tokio. It's not like there's future that kind of like promises in JavaScript, then there have been abstractions just kind of consistently being built up but it seems like Tokio is the next step and it's building towards a whole stack of higher-level concurrency things. Is what you just said enables that kind of thing to happen? STEVE: Yes. Tokio is using those exact same typesystem features in order to guarantee that when you have a chain of promises, to use the JavaScript terminology instead of future things, that you make sure that they're safe. This is not literally implemented yet but Tokio, for those who are not paid hyper attention to the Rust space because this is a cutting-edge, the library is gearing up for an initial release in the next week or two. Soon after you hear this or maybe right before you hear this, it's just going to be released. It's extremely cutting edge. But in some ways it follows sort of the node model of concurrency. There's event loops, you chained together, we call them futures, you call them promises together, you put that pile a future chain and do an event loop and watch the concurrency kind of go. One example of how Rust can do cool things is you could -- this is not implemented yet but it will be in the future -- run, let's say, five event loops on five different threads. Then you just tell the framework, "Please run this future chain onto one event loop. I don't care which one," and then it will automatically load balance across the five threads and five event loops because you've guaranteed the compile time that everything is safe to pass between threads so we know that that's just trivial to do and therefore it's like not a big deal. We can add those heavy duty features without worrying about introducing very subtle bugs, which is really cool. CHRIS: That kind of leads me to my next question, which is at The Frontside, we are pretty into web development, in case you didn't know. I am someone who follow Rust a lot and I find it very interesting. But for the most part, I don't have a need to do systems programming on a regular basis. I also wouldn't even really know where to start, if I wanted to do systems programming. As I learned Rust, I tend to always gravitate towards wanting to do things that I would probably do in Ruby or Python, like write the back-end for some web app or something. That goes okay but Rust is very much still in the process of building those abstractions to the point that it's relatively digestible. So I have a couple of questions. One is do you see Rust being a thing that would be used by web developers a lot more broadly and two, how would you recommend that people like me who aren't really familiar with systems programming start to really dig into Rust on a deeper level? STEVE: I would like to think that web programmers will use Rust more often and to be honest, originally, I was extremely skeptical of this. But it's been changing rapidly as time has gone on. Part of that is because as we've gained more experience, actually in programming in Rust, the fact is Rust used to be a lot less ergonomic than it is and now it's fairly ergonomic and will only get more so in the future. That's something that web people or at least, I come from Ruby so Rubyist care a lot about ergonomics, maybe more than anything else frankly. I'm not sure it's the first tool that you'll reach for but I do believe that sometimes, it makes a lot of sense. As one example that I will use, there's not a whole lot about this but basically, npm has started using Rust on the server side for powering the registry. They have three services in production now but they were basically like JavaScript as a language we all know what is the best language for doing this. We have a service that needs a little more oomph so maybe let's rewrite that in Rust instead and use it for those kind of things. I think that there's a lot of situations for web developers where they don't realize they have the power to make things faster without just adding on more servers. I think that's kind of like a compelling sort of [inaudible]. Any sort of background job like any sort of job queue thing is like often better written in a faster language but you would not reach for that faster language first because traditionally, those faster languages have been terrible to use. I think we continue to win on the ergonomics and continue to win the libraries that web developers will reach for Rust like more often than not. In terms of the learning rest on a deeper level, I think that one of the initial things and sounds like maybe you personally are a little past that but maybe not the people who listen this podcast is that I do think that sort of building the things that you would normally build in Ruby or JavaScript or Python is the good first step. For example, right now Advent of Code has been like a really fantastic way of having these little programming projects. If you haven't seen AdventOfCode.com, it's like every day in December up until Christmas, there's a new programming project that you can build the thing in. I've been doing those in Rust and that's a lot of fun and it's a good way to practice and gain some basic literacy. But after that moving at a low-level stuff, my personal thing and I know something you've expressed interest in the past is my side project is building an operating system in Rust. More so, than just that the pitch is, "You've written JavaScript before. Let's write an operating system together. Here is this companion book and I'll show you how," and that's called intermezzOS. It's like I'm basically trying to rebuild an operating systems curriculum but in Rust instead from nothing, like we start off with assembly code and move up into Rust code. CHARLES: Now, you can't even use anything like all the things that we've been describing like threads, kernel level callbacks. You get none of that, right? You have to implement it all from scratch. You can't use POSIX or whatever. You know, 90% of your code ends up going through. STEVE: It turns out that and it's sort of like for reasons that hopefully I'll be able to fix in the future, you need about like 200 lines of assembly code before you get into Rust and then you basically don't need to use assembly again, really. It's not that big of a barrier in terms of [inaudible] things and its copy-paste stuff that I explained extremely heavily so it's like totally an accomplished real thing. Then you're in a real programming language and you can do more normal things on top of it. But one thing about that because it is my side project, the kernel is actually farther along than the tutorial is and I actually need to find some time to write more of the freaking tutorial but this is kind of my personal long-term project over the next, let's say, decade and to have a completely free and open source tutorial for you to learn about operating system developments. That's one of the things I've been doing. Another one that I think that is really extremely useful is once you gain some amount of literacy on this, you can actually start to learn more about how your regular programming language works. I've been giving this conference talk recently. It's called 'Exploring Ruby Through Rust', and I'm like, "Once you know this low-level stuff and you gain this literacy, you can look at the source code of your language as interpreter and learn stuff about it and you can contribute to it maybe even." Maybe that's not the most practical thing or whatever but now that I've spent a bunch of time with Rust, I understand Ruby on a far, deeper level than I ever did before because now I'm not afraid to go poke around in the internals and learn how it really works under the hood and I understand what those internals do far better. Maybe five years ago, I could have told you like, "Ruby is garbage collector. It's extremely basic. But I don't really know what that means." And now I can be like, "Ruby has this mark and sweep generational garbage collector. But it's not compacting or concurrent yet but maybe in a year or two. Now, that's not just a bunch of buzzwords because I have this low-level literacy." CHRIS: Yeah, that's definitely something. I forgot about but every time I go learn something in Rust and initially this happens a lot. Every time I do that and I go back to JavaScript or something else, I find that Rust inadvertently taught me something about the language that I actually work on every day. Especially, when it comes to things like references, values, and the difference between them and debugging weird prototype behavior in JavaScript became so much easier after I had spent some time working with Rust and had had to like actually deal with passing around references or dealing with life times or having the compiler yell at me for a lot of things that I thought were totally normal. Then I'm going back to JavaScript, it's like, "Wait a second --" Suddenly a lot of these pieces are starting to fit together and before what was just as weird mystery, now I can totally see what is happening and start to think about how to fix it. Even though I don't even have the same tools that I do with Rust, it still is extremely useful from that perspective. STEVE: That's awesome. I'm glad to hear it. That's how I definitely felt with Ruby for sure. CHARLES: You know, in terms of actually using it for day to day stuff, is there other plans, is the ecosystem already supporting things, say, a web framework? Like a low-level web framework like Sinatra or Express or even higher one like Rails. STEVE: I guess, like you've already qualified it as web stuff. But I would say, in a broader sense, whether or not Rust is ready today for you, it depends entirely on the ecosystem. I feel like 80% is productive in Rust as I did ever in Ruby. But that's only if there's a library that I don't have to rewrite myself because it doesn't exist yet. That number is actually growing rapidly so I just look because it's like the end of the year and our package ecosystem is actually doubles. This is a request from earlier. I didn't expect Cargo so Rust basically has bundler or yarn/npm built into the language itself. We distribute it with Rust and we have all that great package ecosystem shenanigans. Another great example of Rust over a language like C is the tooling. Basically, what happened was Yehuda and I kind of showed up in Rust world and we're like, "Why are you still using make files. We know a better way." And they're like, "Okay." Then he builds the equivalent of bundler for Rust. Then everyone's like, "Oh, yeah. This is way better. We're not using make files anymore." The tooling situation is very familiar to a dynamic programming language person because we literally had the same people write the tools. That also means you can share packages freely and briefly so operating system development thing is totally intense to be able to use your package manager to download packages to help you build an operating system. For example, X86 has custom assembly instructions that you need to use when interacting with the hardware and someone has already built a package on [inaudible] that wraps the inline assembly up in a nice to use Rust functions. I can just include that package and use it when building my operating system, which is totally mind-blowing. The npm is sort of feel into OS development is just real intense and cool. Back to the ecosystem thing, though. For web application specifically, it's good and also bad. There's actually multiple different web frameworks already at different levels of comparison. For example, you have Nickle which is kind of like Sinatra and you have Pencil, which is kind of like Flask and Python, which is also kind of like Sinatra. Then you have Iron, which is kind of like expressed in JavaScript. There's also like I know of at least two. One of is has been worked on but it's not been actually released. But the code is at least open source yet. I know a second that is being developed fully in private that has not had any public release yet. Then when the Tokio stuff comes out, People are going to be building new frameworks on top of the new async shenanigans and/or porting the async stuff into the existing frameworks. We kind of have a lot of options but there's also a lot of churn and activity and stuff going on in that space so that either terrifies you or makes you enthusiastic. They're basically is like that. We definitely don't have a Rails yet. I don't think that's because a Rails will never exist but because it's a much bigger project to build a Rails than to build a Sinatra. CHARLES: Yeah, and you just need those foundational pieces there in place before you really want to attempt that. STEVE: And I think Tokio is the real foundational piece and it's just taken us a long time to put it all together. The initial tests in Tokio, we could do a 'Hello, World' benchmark like the tech and power benchmark. Some of you are already familiar with those things, or not, they're like 'Hello, World' benchmark. We actually got faster than they are fat than all of them. It just edged out the fastest Java, which is currently the reigning benchmark on it. That's like extremely compelling. Even if after all this stuff is built on top of it but it's taken us a while to build those foundations and we're just getting that point like Tokio is going to have a release, hopefully before Christmas. I've been assured by the end of the year and then people are going to build stuff on top of it and it's just going to explode from there. Here's another little interesting pitch. I'll give you for this, is that one of the things I like about Rust on early ecosystem is it means that if you want to be that person who built the library that does X that everyone uses, there's lots of opportunity in Rust world right now. Where there's a lot of foundational libraries that you could be the person who wrote that thing when everyone knows and loves and uses. Like JavaScript is still kind of there. In Ruby, every library basically exists already so there's no more room to build a foundational thing. But if you're someone who likes working on open source and that story is compelling to you like getting involved in a younger ecosystem, it means that you can have a much larger impact. I maintained the [inaudible] library that things used. The only reason that's true is because I was around before we had one and then Yehuda wrote the initial version and now, I'm maintaining it. There's tons of space out there so if writing a web framework is the thing that's interesting to you, Rust is a great place to explore and actually doing that at the moment. CHARLES: Steve, one of the things that I know you do is you actually write the Rust Book. I heard that you're also in the process of rewriting it along with Carol Goulding, I believe. I was wondering if you could tell us a little bit about that. STEVE: As part of this Steve getting the job right in the docs on Rust thing, I kind of working on lots of stuff so up to Rust 1.0, we knew we needed to have some long form explain all the things that Rust so that became what's called the Rust programming language which I named so because the C programming language and the C++ programming language, the names of the foundational books for those languages so I wanted to continue kind of in that tradition. But there is some problems with that which is I'll say that I'm a little harder on my own work than I think other people are so I hear people tell me all the time that they love the Rust Book and that it's like one of the best programming books that have ever written. But I think it's not that great. The reason why is also because I just know that the way in which I wrote it. You have to remember that Rust 1.0 happened in May of 2015. We were working on language for six or eight years before 1.0 happened so there was lots of changes, language is changing on a daily basis. Now, it's super stable like super, super, super stable. But what that also means is in some like deeper philosophical sense, nobody had had experience programming in what really was Rust yet because we were still like finishing building it's so like how do you write a book on a language that like the precursor language is what you're using and you're trying to see like what is it going to actually end up being like at 1.0. Because it's not like we can just say, "It's done. Now, go write a book, Steve and then we'll release it at that time." The circumstances in which I wrote the original book were I had a very intense deadline of this has to be done by the 15th of May. While the language was coming together, it takes a couple months to put together a book so I had to make sure that the stuff I was starting I would need to go back and re-fix. That also means that I was like much more vague in some places where pieces were still falling into place and you're like, "This is definitely going to be the same. But this might change so I'm going to leave that part off," and then I just have to plow through because the deadline. All those things coming together means that I kind of put together this book that while good and I'm proud of the work that I did, I can do much better. At this point in time, we now have a full year and a half after Rust 1.0 has come out. I know the struggles that people have when learning Rust. I know the ways in which they succeed or fail and I've talked to a lot of people so I'm sort of rewriting the book now, bringing that knowledge and understanding in as well as the fact that the language just been around for a minute so it's much easier. As part of that, I brought on Carol. She goes by Carol Nichols or Goulding. She both has her maiden name and her married name. She's been one of my best friends for a very long time so I'm extremely happy that she's my co-author on this book. The two of us together and working on doing the rewrite, I think that it is possibly the best thing I've ever done or worked on as far as books go, like I'm extremely happy with it and you can read it online right now, if you want to and see if I'm right or wrong about that. But I think it's a far better book than the original book was. It's actually going to publish at No Starch as well. We're donating all the proceeds to charities since we're being paid to actually write the book in the first place, like [inaudible]. It's going to be a much, much easier and better way to learn the language, I think as well. CHARLES: If we want to check that out, where can we find the new version? STEVE: I'll give you a link to put in show notes or whatever as well. But it's Rust-Lang.GitHub.io/book. There's also just like a book repo in the Rust Lang organization on GitHub. All things in Rust is being developed fully in the open so you can read the drafts and see what's been done where. We're getting towards the end, slowly but surely so I'm hoping that's going to be done relatively soon. CHRIS: Well, I'm looking forward to it. CHARLES: Fantastic. Sounds like the documentation is there. It's excellent. The community is there. It's excellent and from what I'm hearing like the kind of the tower of the ecosystem is really being built up. It's not as high as a bunch of other places but it's definitely high enough to jump in and get your feet wet. If you're you know coming from almost any walk of programming. STEVE: It's a lot of work but we seem to be doing good. CHARLES: All right. Well, thanks for stopping in and talking about this with us, Steve. STEVE: Thanks so much for having me. It's been a lot of fun. CHARLES: Yeah, and now Chris, we do need to kind of figure out what is going to be our Rust project here at The Frontside. CHRIS: I'm up for that challenge. CHARLES: Yeah, that'll be some Christmas homework. All right-y. Take care everybody and thanks, as always, for listening. We'll see you next week.
Kyle Simpson: @getify | Blog | GitHub | getify@gmail.com Show Notes: 04:19 - Styles for Learning as a Newcomer 08:41 - The Structure of the Journey “If you're waiting until you're already an expert on a thing to share it with somebody else, you've waited far too long because we missed out on the most important part which was the journey.” 12:52 - Understanding a Problem vs Solving One "I want to inspire people to be uncommonly curious." 29:02 - How do you know when to stop? 35:02 - Knowing Math and Learning Programming 43:40 - The Importance of Mentorship Resources: LABjs The JavaScript Austin Meetup Pedagogy Saron Yitbarek (CodeNewbie): I don't belong in tech: Trying to find my place in the place I love, and constantly failing Dr. Seuss' The Sneetches (Star-Bellies Reference) Big O Notation You Don't Know JS: A Book Series Functional Light JavaScript Kyle's Frontend Masters Training Videos Transcript: CHARLES: Hey, everybody and welcome to the Frontside Podcast episode 50. I'm your host, Charles Lowell. With me, also from the Frontside, is Stephanie Riera. And with us today is a very special guest, Kyle Simpson, a fellow Austinite. We're going to talk about education and teaching JavaScript. You hear this name everywhere you go. I remember when I first moved to Austin, back in 2009, I moved back to Austin, I went to my very first JavaScript Meetup and there was this guy sitting there doing some live coding on this crazy thing called LABjs which was just like mind-blowing stuff that was going on that was doing crazy on-the-fly modules floating. And this was like back in 2008/2009, so awhile back. And that was my first experience. And then just moving inside tech circles, he just like popped again and again and that's just like the story. It seems like Kyle is like ubiquitous. I was at Clojure Conf last weekend and one of the speakers actually gave a shout-out to Kyle Simpson. So, I don't know Kyle if you do much Clojure, but you definitely have an impact, an outsized impact in the JavaScript community and outside the JavaScript community at large, all throughout tech. It's really a pleasure to get to have you on the show. So, welcome Kyle. KYLE: Thank you so much. It's a huge honor to be here. I'm glad to be doing this in our home city of Austin. It's fun to actually get a chance to pull again with the Austin community because a lot of my work takes me on the road. I sometimes joke that I know more about JavaScript communities in other cities than in my own since when I'm home, I'm usually with the family. And when I go to meetups, it's usually in other cities. But it's fun to be here. It's interesting you bring up that JavaScript Meetup, the original Austin JavaScript Meetup. It started in January of 2009. Actually Joe McCann, most people will know that name. He's kind of a rockstar in the Node community and one of the founders of NodeSource. I think he's up in New York City now. But he started the Austin JavaScript Meetup in January. February, I started kind of came on and we ended up kind of tag teaming running that meetup for the next couple of years. In the early days of that, you're talking about LABjs, I remember that was summer of 2009. The early days of starting up a meetup when you have 5 or 6 attendees on a really good night, mostly consists of about a week before the meetup, "Oh, we don't have a speaker. One of the two of us has to figure out something to speak about." So, I would often kind of take on that task and I would say, "Okay, what's something I'm interested in," or, "What's a library I could write that I could talk about something when I was coding?" CHARLES: So that actually born out of the need to actually present something at the Austin JavaScript Meetup? KYLE: Yes, actually LABjs, it was something at work at that time that I was trying to solve but I wasn't really making a library out of it. It was just some code that I was toodling around with. I was like, "Wow, crap! It's 3 days before the meetup. I better come up with something to talk about." So I wrote the very first version of LABjs to present at that meetup and I got a bunch of good feedback from people that were looking at it and saying, "Oh, that's cool." So I kind of ran with it. That's literally my first kind of major open source project. Most people probably originally heard of me if they've been around the tech world for a while, they originally heard of me because of that one. CHARLES: Wow! I had no idea I was there. KYLE: You were there for the origination. I was the unveiling to the world of LABjs. CHARLES: It's like the Woodstock of JavaScript. KYLE: Those were some early days of meetup stuff. You're like pick up a pizza and a couple of beers from the store on the way to the meetup and there wasn't a lot of organization but we had a lot of fun. Now, the Austin JavaScript Meetup 6 or 7 years later is, it rocks! There's 50 or 60 people every single month that attend. There's been several change-overs of leadership over that couple of years but it's still fun to think about kind of helping Joe get that off the ground way back in 2009. CHARLES: Yeah, that was fantastic. You've been in it a long time and that actually kind of brings me to one of the things that I'm curious about is you've been doing this for a long time. You've been writing JavaScript for a long time and yet you're very much connected to the Day 1 learners and the people who were coming in just kind of from at the very bottom floor. And one of the things that I find in my interactions with the people who are starting out or people who are beginning is that it can be very difficult to know what's appropriate to teach a style that's effective or teaching them because you don't have that empathy of you're so far removed from the experience, that struggle and that challenge of really clawing your way into programming. And yet that doesn't seem to be an issue for you or maybe it is. I'm just curious, one, do you perceive that as something that you have to deal with and how do you do it? KYLE: I would say that part of the way that I'm able to keep a pulse or a finger on what it's like to first start learning something comes from my own learning style which is that I, to learn a thing, I can't follow tutorials. As a matter of fact, every time somebody comes up with a new major awesome tool or framework and they put out this great tutorial and everybody raves about it, I secretly have that huge flare up of impostor syndrome because I know that I will feel really dumb if I try to go through a tutorial and learn it in that fashion. That's just not how I learn stuff. The way I learn stuff is very, very slow and methodical. It generally is taking a thing apart and trying to figure out what each of the different pieces is doing, understanding at that deeper level, and then reassembling the pieces. The reason I mentioned that is because I like to describe my process of teaching and I really think this generalizes to people beyond me that it's my process. I think that my process for teaching is just to have a narrative journey and that it's all about a process. It's not about a single event because I don't think that learning is transactional or I don't think it should be. I think learning should be a process. And so, for me, that process starts with an initial thing that I'm looking at. I get an idea of what it's about, the problem domain that it's in. I probably develop an instant sense of 'ooh, I don't really like that' or 'ooh, that's kind of interesting'. But if I'm going to learn a thing, it's not going to come through a tutorial. It's not going to come through reading a Read Me and being like, "Okay, I'm good to go," and I can sort of copy and pasting. For me, it's going to take, in some cases, reinventing parts of it as I put those pieces back together so I can really understand the choices that were made. I do a lot of reading of somebody else's source code. As an example of this ongoing journey that I'm learning, I was recently at a conference and I heard a discussion of Redux. And of course, Redux is something that tens of millions of people, I mean everybody seems to be an expert on this stuff and I don't know Redux. I don't know React. I don't spend a lot of time at that layer of the stack. But I was listening to a talk and I was thinking, "Well, that sounds interestingly similar to the stuff that I was playing around with back in 2010." It's kind of event oriented and single source of truth for the model and stuff like that. "That's cool. I should go and learn that more." So, that's on my pretty near future to-do list to kind of learn stuff. And there's a really good chance that that process, that journey that I go on to learn about Redux is going to spin off at least some sort of code that I play around with, whether that's using Redux or writing something kind of like Redux to show off how I've been thinking about the idea. And then I'll probably write about it. I know for sure that I'm going to be mentioning it in one of the chapters of my current book on functional programming. So, I'll be talking about it, at least a little blurb about how it fits into the overall scheme of things. And then probably eventually once my journey has gone a little bit further, eventually there'll be some kind of class that I spin up in a module or in one of my classes that I talk about Redux and where that comes from. And that's not because I've been on the high training. As a matter of fact, it's the opposite. I'm the latest adaptor for most of these things but that is a very long slow methodical process for me to learn something and then turn around and try to teach it to other people. CHARLES: Is the idea then that you want to kind of share that journey that you've taken or try to encourage the next learner to take a similar or different journey? How do you structure that learning once you do decide to package it up? KYLE: The structure of the journey -- I think that's an interesting way of articulating the question. What is the structure of the journey? And I would say that the structure of the journey generally, from my perspective, starts from understanding a concept, the why behind the concept, what is the problem I need to solve, and what is motivating solving the problem. And then it moves into the how are we going to do it, what are the possible ways to do that, and that really is the part of that process that you spend most of your time digging into and understanding the deeper level of the thing. So, not just using the library but understanding really how it does it and the choices that it made. And then it moves to the what. And that is inverted from the way most people's learning seems to go about because people learn most typically by, "I find the cool thing because it was in some newsletter that I read about. It seems like it would help on my job project, so I dropped the thing in, put in a couple of the snippets from the Read Me and it's working great, and maybe someday later, I'm going to go back and dig into it a little bit." I'm completely reversed. So, that's my structure for learning and I recommend that people, at least from time to time, take a similar path. It doesn't have to be exactly that way, but take a similar path to breaking down the learning. I'm sure as we go along in this discussion, I'll be able to elaborate on that more. I don't want to just churn on and on forever. But I really do think it's important for people to dig in deeper. So my process is that and the way that I teach is to take people on that journey with me. The way that I teach is to say, "This is the journey that I'm going through. This is the stuff that I understand." For the listeners that aren't teachers, there's some interesting -- I don't want to get too much into the weeds about teaching -- but there's something interesting about teaching. There's this fancy word called Pedagogy which is not the thing that you're teaching but the strategy that you want to use to teach it, the narrative that you want to take a person through. And I spend an awful lot, probably more of my time thinking about that than about the topic itself. So, my practice is to use a lot of silly metaphors. Like I'm teaching promises and I talk about future cheeseburgers and stuff for those that have read or heard from me before, they'll know the future cheeseburger. I use silly stuff like that because this stuff can be really dry and difficulty and I want it to stick in your head. I want the how to stick in your head. So, I invite other people. That doesn't mean that you have to take the entire journey that I did to learn a thing, but I really want to inspire people to learn at that level to be uncommonly curious. And I hope that when they see that I've done it and that I'm transparent about how my process works, I hope it inspires other people. CHARLES: Yeah, and I think wrapped-up in that is tied back to my usual question is because you yourself are in fact you're going through this process, it's very easy then to empathize people who are going through the same process. I really like that. KYLE: And I think everybody should look at the process of getting up to speed on the thing as an opportunity to go on that journey and to document and share that journey with others. You don't have to have the word 'teacher' in your job title to be a teacher. I think all engineers should be seeking to go outside of themselves and to explain things, if for no other reason than the purely selfish reason which is you learn it better by re-explaining it to somebody else, but all boats will rise with the tide. And so, I have taken, for years now, the perspective every time I learn a thing, I'm going to turn around and share it with somebody else. It's exactly the same thing that's interesting and almost ironic that we started our discussion today about that meetup and about LABjs. I was learning a thing and figuring out a thing and I turned around and shared it with other people and it turned out to make an impact. But the impact isn't the important part. The important part was the journey, it's not the end goals. I was always tell people, "If you're waiting until you're already an expert on a thing to share it with somebody else, you've waited far too long because we missed out on the most important part which was the journey." STEPHANIE: Kyle, you touched on something that I wanted to ask you about. You talked about the first phase being understanding a problem, and then the second one being how do we fix this. And last week, I believe, CodeNewbie has her own podcast. And she released a post on Medium and it's called 'I Don't Belong in Tech'. And she talks about how her instinct is to understand the problem versus solving it. It seemed like she feels she is an outcast in the tech community because most people are really excited about solving problems and she doesn't feel like she is solution-oriented. And I kind of wanted to ask you, is that the case? Do you think the people that excel in programming are people that are just naturally excited about solving problems? And if that's not the case, then what do you recommend for people that may not think in that fashion, they don't intuitively feel like solving problems and they feel like they approach it in a different manner? It seems like people have different learning styles. KYLE: I agree completely that people have different learning styles. And that question you just asked is so rich with so many things to mind. So, I'm excited to dig into that a little bit. I 100% agree that people have different learning styles and I think that it is why it is important for there to be people in the community who are spending their time and attention thinking about teaching and about improving educational processes. I do not think that curriculum teaches itself. And I'm somewhat suspect of the movement that we have as an industry towards almost praising or taking it as a badge of honor the label of self-taught. It's almost become its own cult that you're a self-taught person versus somebody who came through it from a more traditional path where a person was carefully thinking about how to present to you and how to teach you whether that was at a university or a tech school or mentorship or any of the others. People that are self-taught often wear that as a badge of honor. It's almost a rite of passage that you got to a level of understanding and experience and expertise because you boot strapped yourself up as opposed to the more top-down approach. And I am suspect of whether or not that's healthy for us to suggest to people that there are these two factions. It's kind of like the old Dr. Seuss book of the star bellies and the non-star bellies. I have the star belly because I got my Computer Science degree. And if you know that book and if you don't know the book, you should go look it up. But what happens in the book is at some point, there's a flip and now the people with stars are the excluded ones and the people without aren't. So I don't think it's healthy for us to promote the idea that we need two entirely different approaches that there's a person who can only be a self-taught and that's a thing, and then there's a person who gets more formal education. I really think it's important for people to spend time thinking about how stuff should be presented and not just throwing information out there for people to learn themselves. This is kind of a crazy metaphor. But if you had a one or two year old kid and you wanted them to learn to swim, the developer mindset often says, "Well, just throw them in the deep end and they'll figure it out." But that isn't how we actually would teach them to swim. We would be very, very patient, very, very cautious with them. We'll start with the shallow end, there being an adult there and they would guide them through that. And I think there's virtue in having some of your learning even if you're one of those listening that does value the whole self-taught thing or that has worked for you in the past. I think there's also value in people going through a more formalized process. Like I said, there's lots of different channels for that modality. But I think the formalized modality has value to it. And of course, that's what I spend my time thinking about. I put out a lot of information - my books and my training videos. There's a lot of that material available for people to self-teach but I really hope that what that does is not encourage people to stay disconnected but rather, to attract people to create relationships around learning because I think relationship-based learning is the most effective modality for leaning. And we've proven that for thousands of years in lots of other parts of society, but we haven't really taken that to heart in the developer world, and I think we need to more. You're right, going back to the original question that there are lots of different ways that people learn. I know CodeNewbie. What's interesting about that premise, I share a similar perspective on those that want to learn to solve a problem versus those that want to learn to figure out how to solve problems. I have a statement that I make which I will preface with I understand that sometimes it can be a little bit off but I hope that listeners will take this from the spirit that I'm trying to give it in. I think that one of the things we can do is begin to think a little bit more about what we're doing even though all of the other structure and process scaffolding around us incentivizes and encourages the opposite. I think what we see is that the developer-oriented mindset typically is more interested in first solving a problem and maybe later understanding it. Whereas I think the engineering mindset -- for lack of a better label, I'll call it the engineering mindset. That's my bias because I came up through a more traditional engineering CS degree. But I think the engineering mindset, the one that most people would say is heavily focused on problem solving and that kind of thing. I think the engineering mindset seeks first to understand the problem and maybe later solve it. If you have developer mindset that wants to solve a problem without really fully understanding it, I think one of the outcroppings of that is the churn that we see in the development community. We see people constantly every two or three years trying to go recreate a whole new stack of things because we're saying, "We need to do this thing differently." And I have seen that cycle four or five times in my career. And so, I can say with pretty strong certainty right now that somewhere in the world, there is a guy or a girl that is a 16 year old that is working on what is going to eventually replace React. I don't know what it is, I don't know who the person is, but I'm pretty sure that we're at some point going to say, "Wow!" And when that shift happens, we're going to come back and think to ourselves, "How could we have ever thought that React was the most amazing thing in the world," and we very quickly forget. So, one of the outcroppings is that continuing cycle because people are seeking to fix problems as quickly as possible and not everybody takes the time to really think about it. The authors of those tools, they think really deeply about these problems. They think so deeply about them. I am amazed the kinds of thought that goes into creating one of those frameworks or libraries. And I have a lot of deep respect to the Ember community, the React community, the Angular community. Those people spend a lot of time thinking about those problems. But the people that use them, use them as tools to get their job done honestly and earnestly so. But they very rightly focus on what can I get done with React rather than 'can I understand what React is really trying to do and use it to its best ability'. And we incentivize that. We incentivize that by saying that the people that advance at work, the people that become the senior developers at work are more likely the people that shipped more code. They're more likely the people that closed more bugs. And one path to getting there is you're just a really quick coder or you're one of those mythical unicorn 10x developers. But a lot of people aren't. I'm certainly not. I am not a fast developer. I'm one of the slower developers. I am very, very careful about what I write. So, I can't survive in that environment where I have to advance because I can write code quickly. Rather for me to survive in those environments, I advance or up my stature, if you will, because I understand the thing deeper than anybody else. And I think there's places in the workplace for both, so I'm not disparaging those that write a lot of code quickly. But one of the outcroppings of writing code quickly if jumping as quickly as you can to solving a problem, is that you often times find a local maximum. A mathematic theory about -- just visualize a curve kind of like the landscape of hills. If you're climbing a hill as quickly as possible and you get to the very top of the hill, it might not have been the tallest hill that you could have climbed. But you got to the top of the hill really quickly and you got the next promotion, you went on to the next thing. So, I think those value. Sometimes in your career and some people in the community who want to take a step back and understand a thing deeper, dig into a thing deeper, and I encourage people to have that as part of their toolset, that they would be that uncommon level of curiosity. Take a moment and by a moment I mean a couple of months, maybe a cycle at most of six months, take a step back from the just everyday churn of shipping a feature and fixing a bug and making the button blue, take a step back and say, "What are some of the things that I have been touching or I have been hearing people talk about," and dig into that thing and learn it. And learn it really deeply and really understand it, and maybe reinvent it if you need to. But really deeply get it. And then kind of resurface and look around and say, "All right, let me find the next thing." And so, I think those cycles are healthy for people. And I think the engineering mindset should be something that each of us strives to adopt at least some in the workplace as opposed to being almost drunk on the idea that the best developers among us are the ones that ship code the quickest. CHARLES: I think also too there is like if you do take the time and you then make that investment upfront, it engenders a lasting speed. In the same way that if you look at a child taking its first steps which is kind of the initial solution of a problem, it takes intense concentration. And it takes all the muscles or trembling and stuff just to coordinate the balance to take those first steps. But then once the child kind of understands the process fully -- like none of us, at this point in our lives, think about walking. It's just something that we do naturally. But because the human brain is wired at that point to take whether it's walking or any other kind of motor skills to really understand the motion and rewire the brain optimized for that motion so that it becomes second nature, so that you benefit from that again and again and again. And I feel having spent a long time in the industry seeing the speedups that can actually be gained by doing it slow. In other words, you expand that understanding to fill the room and then the door is only a few millimeters away, if that makes any sense. KYLE: I think it does. And I think for those that are listening just to address what questions may be popping up in people's heads because as a teacher, you learn to anticipate questions ahead of time. So I can see, even from our discussion so far, that somebody may be asking, "Okay, you're talking about some really nice theory here, but what about the practicality of it? What are the benefits of the thing? I don't have time to go learn React but I can get my job done by applying React to my business process." So, I think there are some practical things that we can point to that that come from being that uncommon level of curious, that deeper and slower and more methodical learning. One of those, I'd like to say -- and people always kind of smirk at me when I say this -- I call them getify's laws. They're just statements that I make. They're statements of my opinion and really the only thing that I'm an expert on is my own opinions and I'm never at a lack for those. It's just an opinion but I believe that you can't hope to solve a problem if you don't first understand it. Or to put another way the getify's law - if you don't understand why a piece of code works, you have no hope of understanding why it broke and how to fix it. So we do a lot of code that works and we don't know why. I would say probably the majority of code that gets shipped were just sort of only kind of tenuously understand it. And sometimes it's really tenuous. Sometimes it's house of cards development where you build a thing and you're not really quite sure why it works but you put a comment there that says 'don't touch this' and 'don't break it' or whatever. And then we back away and hold our breath and hope that it stays in place. That is a way to ship code quickly. The agile mindset has permeated so deep down that on every line of code we're thinking [inaudible], we're thinking, "Just get the thing working and get the test passing and then we'll worry about whether it's the appropriate way to solve the problem. We'll worry about that later." Many times we don't get a chance to worry about it later until there's a fire, until something's broken. And that fire could be that bug or that brokenness could be it's not doing what it's supposed to do or more often, that fire is it's doing it but it's impossibly slow, it's impractically slow. And now, we have to go back and think, "We thought we only had five pieces of data to ever work with, but now we have 50,000 and the algorithm completely falls apart. We can't do an [inaudible] scan anymore," or whatever. So that very narrow myopic short term mentality can be effective for people to get stuff shipped and to advance in their jobs, but it falls apart when things start breaking. If you're talking about a car and you said, "I don't need to know how a car works, so I can just drive a car." And that's entirely true. But if we took that to its logical extreme, you could say, "I don't even need to know that my vehicle runs of gasoline or electricity," or whatever. "I don't even need to know anything about the liquids that go in that make my car work. I just sit down behind the driver wheel and drive and it works." And that will work until your gas tank runs empty. And then at that point, you have no idea how to fix your problem. So you're going to have do some reading and worse, you're going to be at the mercy of somebody else. Having an instinct, having a clue about something about what's going on into the hood is pretty important when we go off the happy path and start trying to fix problems, whether that is fixing bugs or improving performance. Those are, I think, some very important practical takeaways that come from some people some of the time being the uncommonly deep learner. Just for fun, for my son in his school, I've been working on building a game that he's doing as part of his school through this Mathematics competition, I've been building a computerized version of this game. And those that have been following my Twitter have seen some interesting tweets about that. But I've been building an AI, a strategy turn based AI so that you can play against the computer for this thing. Tackling that requires you to kind of really understand or at least be able to fiddle your way through some deeper "CSC" kind of topics. But I got it working and it was too slow. You have to wait for the computer for 20 or 30 seconds for it to figure out its next move and that's slow. So then I started thinking about how I'm going to optimize this. Do I need an object pulled to pull stuff in and be able to reuse arrays or something? I was at a loss as to how to do that efficiently. I was completely lost. And so, I had to learn that thing. I had to think very carefully methodically. If I had let some sort of artificial deadline and say, "No, you just got to ship the game," it would have worked but it would have been impractical. And so, taking that time to think about it -- actually, I finally did. It took me days but I finally arrived at a solution for managing, sharing references that actually works from a performance perspective. It is practical now. I think there are practical takeaways. It's not just all the theory behind being a deeper learner. CHARLES: I've been thinking like where do you know where the stopping point is? Because the world is just packed with knowledge. When I was starting out as a programmer, there were people who just code assembly for fun and assemble their own microprocessors and stuff because that's what they did when they were coming up. But all of that stuff exists like we're standing on it. We've kind of lived on the level of the city street but beneath that is the subway and beneath that is the power grid and the gas pipes and the hog water pipes and all that stuff. Who knows how much more layers of urban development lie beneath some of these cities that are certainly way more rich than Austin. None of that stuff is present here. KYLE: I was going to say I didn't know there was a subway. I wish there was. CHARLES: Once I realized what the analogy was, I was standing in the Austin streets, I kind of transitioned to New York without telling anybody. My point is that in the computer world -- let's just keep it there -- let's say you're working with JavaScript and you're running on top of the browser. Most of modern browsers are written in C that are also using C libraries that are written on native platform tool kits whether it's OSX or Linux or Windows which then are written on top of these operating systems which then are written on top of these compiled assembly language that run on these microprocessors. It's a deep, deep ocean, so to speak. And we spend most of our time skimming around on the surface in our boats. And so, I guess the question is how do you know when to stop and how do you know when enough is enough? Ultimately, there is a tension between a deadline and the amount of time that you're going to invest in learning. And I think that there are definitely -- I feel very strongly that it's skewed way towards the deadline and that we need to kind of pull that tug of war back in the other direction. So the question is then how far is far enough? Or is it just really until you feel comfortable? KYLE: I see the world of what we do in computers and in software development as layers of a cake if you want to mix in a whole bunch of different metaphors here. There's lots of layers of abstraction that we create to utilize technology more effectively in our lives. And in a sense, the technological revolution from the 1940's onward has really been a story of adding layer upon layer upon layer upon layer just like a big tall cake of different abstraction because lots of different people need to eat at different layers. They need to get their jobs done, if you will, at different layers. There are people who need to get their job done the assembly language, the deep guts of your computer talking to the 1's and 0's kind of thing. And then there are people that need to work in the C libraries. And then there are people that need to work in the operating system tool kits. And then there are people that need to work in the browser. And then there are people that need to work in JavaScript. And then there are people that need to work on frameworks. And then there are people that need to work on transpilers that build the frameworks that then get compiled down. So we just keep adding more and more layers because there's more and more people that were opening up the world to, to participate. And not everybody is going to play on the same level. If your choice is to be a JavaScript developer, if your choice is to be a React developer, if your choice is to be somebody who builds those almost meta tools like the transpilers that go from one language to another, if you're a ClojureScript developer or whatever, my recommendation is that whatever layer of cake you eat out, whatever layer you've chosen to make your primary focus, I think you should be striving to become an absolute expert at that layer. And that is not a transactional thing when you just go watch the right video or read the right book and then you check it off. That will be, in most respects, a lifelong process. But I think you should strive to be as much of an expert at that layer as you can. But to become an expert at that layer, one of the most important things is to begin to have an understanding, a level of competency at the layer below it. So if you're a React developer, having a layer of JavaScript below it means that you want to have some pretty good core competency in the JavaScript language. In a sense, that's what my job is dedicated to doing is assisting people at playing at these higher levels at the frameworks and tools levels by understanding JavaScript more deeply. But if you're a core JavaScript developer and that's all you do, you should have some competency at the layer below that which we might say would be the browser API level or maybe the C level and understanding something about how memory is going to get managed to let you have some intuition about that. And so, the further you go down in the stack, you start to see that diminish but it never gets to zero. I don't think there should be any engineer out there that aspires to have absolutely no knowledge whatsoever of what a processor is or what 1's and 0's are or any of that. But that might not be your everyday task. If that's 4 layers removed and you have a lot less understanding of that layer, to be effective in your job, you should be an expert at your layer and be seeking to have a core competency level of understanding at the layer below that. I think that's how you start to answer, Charles, your question of how do you figure out where to stop is at this point in your career -- because many people will dance around and eat at different layers of the cake as they go. But whatever point you are at in your career now, be looking to understand how to eat all that layer of cake and then understand the one below it. I would say that would be the most effective strategy that I've come up with. CHARLES: I like that. I like that a lot. STEPHANIE: I want to become an expert in JavaScript. But sometimes I have doubts in my ability to "become a programmer" or think like a programmer. I feel like the people that excel in programming, and you could tell me if I'm wrong, I don't know if you've noticed this as well or haven't. But it seems like the people that excel at programming are the people that are naturally good at math. It's like they think in terms of math. I don't know if they're seeing algorithms in their brain or what it is. And I don't know if other people relate to this as well. But I'm a natural problem solver. I like problem solving but I don't think in terms of math. And I've taken a lot of math courses for my undergrad degree. I took Calculus for Biosciences like two and a half times, and so I beat my brain enough to understand those concepts. But I don't know if you have also seen that, if that is the case. And if it is the case, is there any advice for the hopeless programmers like myself that are not naturally good at math. But I am interested and I do want to be good at this. KYLE: There's hope, Stephanie. There's definitely hope. STEPHANIE: [Laughs] KYLE: Again, lots to unpack here and I thank you for asking that question because I actually do have a particular sensitivity to the mindset that seems to come around between people - again, the haves and have-nots. There are people that naturally gravitate to or understand math and there are people for whom math has always been an opaque topic that they keep running into over and over and they haven't been able to really fully engross themselves in or wrap themselves in. They haven't been able to get it, if you will. And as an educator, I don't teach math for a living. But as an educator, it deeply troubles me. As a parent, my son, Ethan, is about to turn six. He's in Kindergarten for the first time, so it is right on the front of my radar screen to think about how we teach these things. And I want for everybody to be able to grasp important concepts like obviously reading is important, but Math is one of those core concepts that I think is important for people to not be intimidated by. And I think we do ourselves a really big disservice and have for many, many years by approaching the teaching of that topic as there's one right way to know how to do math. In a sense, I think, we're seeing a shift to know there's lots of ways to learn and we should embrace that. Just like we said there's lots of different ways that people learn. There's lots of different ways to approach. There's lots of different pedagogy, if you will, around approaching math and I embrace that idea that we should be doing that. I think we should be starting that with our Kindergarteners and working all the way up. And I'm glad that there are educators that are thinking about that because my generation was taught that either you get it this way or you're just "not a math person". And I sat and sat alongside in my Science and English classes all the way through schooling even at University, I sat along people that were math people and on the other side of me were people that were "not math people". It's sad and it sucks that we would divide people that way or that people would choose to divide themselves that way because they struggle with math. My sphere of influence right now is my two kids - my son, Ethan, and my daughter, Emily. I really want for both of them, whether they become math nerds or nor, whether they have a minor in math like I do or it's just a thing that they learn, I want them to not be intimated by that. I want them to understand it. And I hope that I will be able to, in the sense, partner with the educational system. Our kids are in Austin IST Public Schools and they will be for their whole lives, as our plan, but I hope to, in a sense, be augmenting that by saying, "I understand that this is struggling for you, but let me help to at least not be intimidating, at least be a thing that you can grab on to and understand even if it's not interesting to you, it won't be scary to you," because I have family members and I see it personally that have self-labeled and self-[inaudible] themselves as "not math people" and they struggle their entire lives as a result. And I don't want that to happen. So, I'm glad I got at least a little chance to have some advocacy there to say I hope that people aren't being that way and aren't promoting patterns that treat people as either you know math or you don't. I was one of those people that gravitated towards math. But I can tell you for a fact that I do not use most of my math background on a regular basis as a programmer, and even if I try. As you're asking the question, I was doing some self-assessment here. What are the things that I know from math? Like I understand for example that a function is putting in an X value and getting out a Y value and when we plot that, we see a parabola on the graph. I get that concept. So, there's a tie in between the visual and the actual written out math for me. And I understand in calculus when we have the integral. I understand that's the area underneath the curve. So, I get those concepts. But does any of that stuff actually play into my day-to-day work? Not really. Maybe a little bit about functions since I'm starting to be a bit more attuned to the functional programming these days, but it really doesn't play into my job. But there is a part of math that really does play in almost daily, and that's what we call discrete math. That's more of the computer science-y take on math. And discrete math is things like understanding number systems, things like understanding the difference between representing something as a binary or representing it in base 10 or hexadecimal. There are certain concepts that we talk about like Big O Notation - that's one of those computer science-y concepts. When I was writing this game AI, I had an intuition about a loop inside of a loop is going to perform worse than just a single level of loop. But I didn't spend any time whatsoever actually calculating the Big O complexity of my algorithm. I just had some intuition about it. And the reason I have this intuition about it is because I took a discrete math class. So, if I could say anything to the people listening, for those that aren't math people or for some reason or another didn't really gravitate towards it, to be better at programming, I would encourage you to at least think about discrete math as an area to study to get some better understanding around, because I do think those things help. But I don't think that most of math, the stuff that most people struggle with, like algebra and matrices -- I was terrible at matrices. I hated them for some reason. I don't think most of that ever comes in. I learned what an icon vector was but I've never once coded it. So I don't think that you have to feel intimated coming at programming if you weren't one of those "math people" but there are some parts of math that do end up playing quite a bit into the programmers' everyday life. CHARLES: I think for me certainly whenever you have a particular problem, then that's an opportunity to actually explore the math behind it. I think a lot of times we put -- there's this kind of inverse relationship -- we put the cart before the horse. It's like you got to learn math and then you learn the programming. But it's almost like the equation ought to be reversed in the sense that it's like trying to -- let me just cast about for a random analogy here -- it's like learning the theory of fishing without actually living on the coast and going out on a boat every day. You can certainly study it and it can be interesting. Yes, we tie the line to the pole and we put it in the water but unless you actually have that problem of like, "Now, I'm trying to catch fish," you actually are connected to it in a way that's going to help you understand what solutions work and which ones don't. I often feel like we frontload way too much on the theory. I feel it has to be connected to practice, I guess is what I'm saying. Because certainly my experience, I come from a CS background, but I didn't switch to CS until my junior year of college. And that after I'd basically dropped out of school for two years to work at a startup. And I remember it made a lot more sense because I had this prior programming experience. It's like, "Oh…" Whereas, I think a lot of my classmates, they were kind of dumbfounded by, "What does this even mean? I don't understand. I don't understand how this connects to anything." But if you have that kind of experience, then the learning can come in hand with it but it has to be more of an organic process as opposed to one coming before the other. That's my experience. KYLE: I love what you're saying there and I want to build off of that to unpack some of my most recent passion and focus in the area of education and with developers because I really think what you're saying is the foundation is the same belief I have. So, I'm right there with you. If you look at humans and the way humans have passed along information and skill to the next generation, if you look at that over the course of history, we have mountains of precedent around the idea that skills are passed along through the mentorship model. Some like to call it the master apprentice model. It's the more classical way to think about it. But just the more modern way is to call it mentorship. For example, chefs. You would never suggest that the best chef at the best restaurant in town and the way that that chef got there, the way that that chef got to be the best chef in town and have the best restaurant was that they sat in a classroom and learned all the theory about how to bake a chicken. First, before going and baking the chicken, there's lots of ways that they could have gotten to be better but probably the least effective way would have been for them to learn all the theory upfront and then just go work in a kitchen for year after year after year after year and almost accidentally -- we call it the School of Hard Knocks -- accidentally learn all those experiences. That's how a lot of people define experience is experience these mistakes over time getting practice making perfect. That does work and lots of people have done that. And I certainly would say a lot of my story is that way, but I wish that I'd had a mentor. I wish that I'd had somebody like what happens with chefs to sit there and say, "I'm not just going to let you figure it out on your own and screw up a bunch of chickens. What I'm going to do is give you a little bit of information about cooking a chicken. Like for example, what the temperature of the oven needs to be, how to tell what the temperature of the chicken is to know that it's fully cooked, what materials you need and what utensils. But we're going to spend 10 minutes in the classroom talking about it and then we're going to go sit in a kitchen and you're going to watch me bake a chicken. And you're going to watch very carefully. And I want you to ask a lot of questions about what I'm doing and I'm going to talk to you about it every single step and the decisions that I'm making. And then I'm going to do that again and again and again and again. And over time, you're going to start to pick up on and glean some of my experience from observation. And then eventually, once you are starting to feel more confident about it, we're going to flip the tables and now, you're going to start baking the chicken but you're going to do it step by step methodically and I'm going to watch you and be very closely paying attention. And I'm going to ask you questions and I'm going to say 'why did you squirt that there and why did you do this and why did you turn on the oven right now'. And I'm going to ask you about that stuff and challenge you to make sure that you really got all the different pieces of what it takes to master this skill. And then I'm going to do that again and again and again. And after we feel pretty good about chickens, we'll go back to the classroom and we'll talk about cooking steak and then we're going to repeat this process over and over." That is much more the picture of how that mastery of that skill is replicated in another person is that mentorship model. And we've seen that with stone masons and carpenters and plumbers and chefs and caterers. And on down the line through most of human history, we can see that that is how skills are passed along. What I think we're missing is that there's a lot of value to be gained by applying those models to software development. As a matter of fact, I think if you ask most developers, regardless of the skill level even if they were a senior developer -- although certainly the intermediate and junior developers are more willing to admit it. But I think if you really ask in their heart of hearts, almost all the developers would say, "Oh, I'd love to have a mentor to help me get better but I don't know how or I can't afford it or there aren't any available or I don't know what to do," because we haven't prioritized as our part of the industry making education more effective. And so, when you talk about 'I need to practice a thing, not just know the theory', you're absolutely right. You're absolutely right that sitting in a classroom and learning a bunch of stuff which ironically that's what I do. I teach corporate training classes, I threw out a bunch of information then I know as I'm throwing out that information, the vast majority of it isn't going to be retained because the only stuff that will be retained is the stuff that somebody gets a chance to practice pretty quickly. And by pretty quickly, I think probably that horizon level is about three days beyond when you've learned it. If you don't get a chance to really practice, and I don't mean just the artificial stuff that we teachers come up with like a silly little toy examples and the foobar kind of stuff, I mean really practice in a context of something that matters and that you're thinking about it and you need to solve it. If you don't get a chance to practice some of that theory within about three or four days, you're going to lose it. So, I present to you a set of information in a week-long training course that you really ought to actually learn. And by learn, I mean more than just hear the theory but also practice it. You really should learn that over 12 to 24 months. But I give it to you in 40 hours because that's, again, the industry incentivizes how quickly can I check off the check box then my developers learn how to become React developers or that we learned about JavaScript. Instead of focusing on how do I really get them to know it. And the only way to really get somebody to know it is to have somebody watching over them as they practice what was taught. You teach a little bit and you have a bunch of practice. And that practice can't be just the developer has to figure it out on their own and make their own mistakes and figure it out through Stack Overflow, it has to be monitored, it has to be proactively mentored. And they have to take a little bit of learning, a lot of guided practice. Pipeline that across, that's how we make this educational process for developers become much more efficient, much more effective. And so in a sense, what I'm trying to do is disrupt my own educational model and try to [inaudible] myself as a business by saying, "I teach it in the classroom but every time I walk out of that classroom, I feel empty inside," because I feel not just tired physically because it is exhausting, but I also feel empty inside because I know that what's really missing is that I just contributed to yet another transactional learning experience. But what really is important is for people to know that learning that is most effective is learning that's relational. Learning has to be paired with people because at its heart, what we do as developers is a people game. It's not an 'instruct the computer' game. The 'instruct the computer' is a side effect of what we do in connecting with people and passing along information and skill and knowledge. We've got to do that because we are throwing all kinds of money and time and effort at figuring out how to make people learn better through books and videos and conferences and tech schools and the list just goes on and on and on. And it's good that there's a wide buffet for people to pick from, there's no question. But most of that stuff isn't as effective as it ought to be, and that is why people have to keep doing it over and over and over again. They have to try all these different things because they don't get satiated by picking the one right thing that they ought to have fixed. I happen to be obviously very bias, but I happen to believe that a lot of the problems that developers face, a lot of the overhead of code maintenance and all of that stuff, it could be solved if people were trying to implement mentorship for developer education. And that's what I'm trying to do. My recent efforts are around upending or reimagining the normal corporate developer training. Of course I still offer that. And if anybody wants me to come teach, I can still do that. But what I'm moving towards and building a startup to offer mentoring at scale because I believe that's where we move it from transactional relationship. We make it about people and then we get a chance to really practice what we're learning. That's how you take a person, and get them from being junior to being intermediate or from intermediate to senior is that they have to have practice. Would you rather them figure it out on their own and make lots and lots of mistakes or would you rather them be guided and monitored as they go through that process of practice? I think the latter would be a lot more effective. CHARLES: For people who are excited about this, who want to figure out ways in which to engage in this relational learning, this can be a very expensive proposition especially for smaller companies. But even for larger companies, one of the things you're hoping to address is a way to make this when you say 'do it at scale, make it more accessible' in terms of those time and resources. KYLE: In a sense I would be failed at the start if I didn't feel like I had a model for scaling mentorship. If I was just on here and [inaudible] saying, "Hey, everybody. You should go mentor," and I had no way of doing that, then everybody will say, "Okay, that's all well and great. Thanks." And as soon as this podcast ended, they'd just go on back to their normal day jobs and fix anything. I really do believe that there are ways to scale out and to make it effective. That doesn't necessarily mean that the price tag is cheap because time is expensive. And what we're saying is you need expert time paired with your people to get them to where you need them to go. But I want to push back on the notion that expense makes things inaccessible, because I want to point out -- this is almost like I'm doing a customer pitch. But I just want to point out that there's an awful lot of things that we spend money on as managers of software development teams. For example, I've talked to managers of software development teams and they've said, "We probably spend 70% to 80% of our time maintaining our existing code and fixing bugs, and only maybe 20% of our time implementing new stuff." Some people listening would say, "Good lord, I'd love to have 20%. We probably spend 5% of our time on that and 95% on maintenance." So those numbers differ. But there's a lot of overhead associated with running a software development team. And one of my theories which I believe we will prove out as we scale this out, one of my theories is that if you want people to avoid having to come back and return on something. One of the ways to do that is to empower them with better education before they write that line of code. I think even though yes it will be more expensive to get your developers mentored, you will actually not spend more as a company, you will spend much less because you'll be investing that money much more effectively in the things that actually make them retain and get a chance to practice that. It also means that hiring will be cheaper because you'll know much more deeply what your company wants and needs and that will make it a lot easier to identify the right candidates. It means that the total cost of ownership of yourself will be vastly less. So there's lots of dollars here that I think we can pull that we are, in a sense, wasting or misappropriating to make it more effective to invest in mentorship. I think what we need is a process and a model instead of tools to do that, and that's my mission. That's my goal. CHARLES: I am very, very excited to see that, because I know that that is something that one, holds the value. Here at the Frontside, we see as one of our core beliefs and one of our core aspects or mission is to level people up and make sure that they can improve themselves as a developer and improve the quality of the products that they develop. But it's definitely something that we've struggled with. So, I'm really curious and excited to see the outcome of this. So, we'll be on the lookout for it. Is it still under the radar? KYLE: There's no company name to announce yet. There's no website or Twitter account. But I can say that it's been under active development now for four to six months. I have a team of founders with me and we are formalizing our process in trying to get funded for that. I hope within the next few months that there'll be a much bigger announcement about where we're headed with it because I'm all in on believing that it's not just good enough for me to keep churning. I get emails and tweets weekly. I get a couple to maybe five or ten sometimes in a week of people saying, "Hey, I read your book," or, "I watched your Frontend Master's training video," or whatever, "And that's awesome. I'd love it if you could help me with something. Could you spend some time? How much would I need to pay you to get a few hours of your time to sit here and talk with me?" And it pains me to get these kinds of requests. And the reason it pains me is because I would absolutely love to spend that kind of time with people, but that kind of time is so valuable right now. Time is so valuable because the process of mentoring currently is incredibly time intensive. It's very, very expensive to do this. And that's why I think it's not done very widespread is because people haven't figured out how to make it effective, from time and money perspective. And I know that pain personally. I have to turn people down and say, "I'd love to but I'm too busy because…" and then fill in the blank with a thousand things. I'd love to have that time. What I really want is I really want to be able to do this effectively and to scale it, because people are hungry. They want to learn. I believe that what we need to do is unlock that hunger, not necessarily inspire people. I think we need to unlock and empower that hunger that people naturally tend to have to want to get better at what they do. And I think this is one of the ways that we can help people get better. STEPHANIE: I also wanted to point out that for companies that are interested in doing this, not only do junior or apprentices benefit from this model but the senior developers also benefit from this. I have personally seen someone on my team, Alex, from all the times that we've been pairing and talking about JavaScript concepts, I have seen an exponential growth in just the way that he explains things. And they do say the one that does the teaching does the learning. And I have seen that in him and I think it's something that every team could benefit from. KYLE: I 100% agree. And to build on that, I would say what we're really doing with that is we're helping people find more purpose for what they do. And who doesn't want to have more purpose and more meaning? We're giving people a way to -- as I said, all boats rise with the tide. We're giving people a way to help other people around them. And it doesn't have to be like some macro scale thing where you become an international conference speaker. You don't even have to speak at your local city meetup. You can just pair with the person next to you and help them learn the thing that you're learning, and then learn from them. And you've made the world better, you've made the workplace better, you've made yourself better. It goes back to what I said before - I think we have to get better. The biggest thing that's lacking from us developers right now, in my opinion, the emotional muscle that we need to work on the most is recognizing empathy for our fellow human beings for the people around us because I really deeply in my core believe and my DNA I believe that what we're about is people and not about the code that gets written as a result of this connections. If we can focus on building the relationships around this. You don't have to go pay for the service of the startup that I'm building. You can implement this just like what Stephanie just said, by looking for opportunities whether it's coding, whether it's code review used as a way to learn, whether it's doing a brown back lunch, you can look for ways to build the culture of learning into the workplace around you and people will benefit from that. You will inspire people to get bigger and better as a result of providing that environment. CHARLES: Fantastic! If people need to get hold of you, Kyle, what is the best way to reach out to get in touch? KYLE: The best way to get in touch with me is to find getify. That's how people know me online. That's my Twitter handle, that's my GitHub handle, it's also my Gmail address. So, getify@gmail.com. Reach out to me in any one of those channels, probably through Gmail is best. If you'd like to talk more about education, if you'd like me to come and help your team, I would love to come and do a workshop and then build a relationship with you or we can begin this mentoring process as I spin that up. If you're interested in that, if you're hoping to learn from some of my resources, you can check out my book series. I have the You Don't Know JS books that I wrote that many people know about. That's up on my GitHub. I'm almost done with my next book which is Functional Light JavaScript, you can check that out on GitHub. All that stuff you can read for free. You can also buy it if you'd like to -- and I always appreciate people if they want to support me. But check out my books. I have training videos available on FrontendMasters.com. I think I have nine courses and we've already scheduled a few more that I'm going to be teaching later in the Spring. That's a fantastic platform. There's 50 or 60 really incredibly well-versed experts that are teaching on that platform. So, check out Frontend Masters. Some of those videos are also out on Pluralsight. You can check them out as well. Check out books, check out the training videos, get in contact with me if I can help you in some way with training at your company. But I would love it that even if that isn't the channel, if you take nothing else away from the podcast, go talk about learning stuff deeper within the workplace. That will be the best takeaway that we can get from this podcast. CHARLES: Yeah, definitely. I think the things that we talked about here really apply far beyond JavaScript and even really far beyond the technical trade of programming. All right, that's a wrap. Bye everybody!
We say goodbye to our friend and co-host of The Frontside Podcast, Brandon Hays. Brandon Hays: @tehviking | blog Transcript: BRANDON: Hello, everybody and welcome to Frontside Podcast Episode 48. It is a podcast, as you can tell from the intro music, about woodworking and dads and dads working with wood. CHARLES: And power washing wood. BRANDON: Yes and power washing, sanding decks. Today, we'll be talking about the difference between mortise and tenon joints and dovetail joints and when it's worth going through the extra effort for that dovetail. Our guest today is Charles Lowell, woodworking expert. CHARLES: Hi, Brandon. BRANDON: Hi. I think I've never heard anybody acknowledge the fact that there's new music in the intro. I don't know if that has been publicly acknowledged in the podcast yet. But I was on the selection committee for that music and we had many options and we went with the one that sounded most appropriate for, I don't know, a woodworking show on PBS but I like it. CHARLES: I like PBS. I like woodworking. I like the music too and the best part is I didn't have to choose it. I just had to promise to make music myself for 50 weeks running. BRANDON: It was about 150 weeks. CHARLES: About 150 weeks. BRANDON: Yeah. Charles, we are here in the new Frontside HQ, which is sort of still under construction. It's kind of cool, though. It's a different space. I haven't been here much so it doesn't feel like home to me yet, I don't know if it does to you yet. CHARLES: It's beginning. It's still unfinished in many ways. It's not painted. We just kind of have the bare minimum for survival. The kitchen is stocked, there's coffee in the cabinets, the grinder is set up, the bar is set up, the wall sockets are set up, and the wireless router is set up. BRANDON: Home is where the vermouth is. [Laughter] CHARLES: And cavernous is really the word to describe it. BRANDON: This is a shop that's clearly set up for a version of the Frontside that does not yet exist and it's interesting to see current day Frontside in a space that is clearly marked for a future version of this. CHARLES: Yeah, you feel small walking in. There are not enough people to fill all the seats that we have in here. BRANDON: Which is kind of cool. I don't know if you recall, maybe we'll save that discussion for later but there was a prior iteration of this company where that was very much the same thing. So it's kind of cool, like a little bit of an emotional journey to see this process start over again. CHARLES: Yeah, it is definitely a new space and it's a new era. BRANDON: Yeah, for you. Not for me. [Laughter] BRANDON: You guys can go right to hell. CHARLES: Well, it is a new era for you too. BRANDON: Yeah, that's true -- a new era of fun-employment. We wanted to do one final 'dadcast' because basically, as of officially 10 days ago or so, I no longer work at the Frontside and I've had people ask me and I was like, "I don't want to talk about it." And so, I think it's probably time that we -- I don't want to belabor it. Nobody handed me a card when I quit as the co-founder of the Frontside that said, "You are officially invited now to Medium.com to write a think piece about my incredible journey." CHARLES: [Laughs] You'll get that in the mail. BRANDON: Okay. It may be sent to my old address. It's sent to our old office, which is the problem. It's lost in the mail. So yeah, instead I'm going to do my incredible journey podcast and really just regale everyone with stories of what an incredible, incredible journey it's been -- truly incredible. CHARLES: Don't leave out the travelling with the fish and the cat, and like a little pug dog. You know like all those incredible journey stories when we were kids. BRANDON: Yeah, like Homeward Bound. CHARLES: Yeah, there you go. BRANDON: I think it is like Homeward Bound. CHARLES: That time that you fell into the water with the kitten, that was adorable. BRANDON: I don't know where I'm stealing this joke from but there is an Incredible Journey sequel with a squid and a cow, truly the most unlikely of friends. [Laughter] BRANDON: Here we are giving my swan song on this podcast. I've actually actively took a month off to think about basically to try to recuperate and think about like, "Is this something that I can continue to do." I purposely took off from the podcast which was agony because I so wanted to have those conversations that you all did with Noel, and with Yehuda, and with Sarah. They turned out really good. I'm so happy with the result of stepping back and realizing, "Wait a minute, I think I'm redundant." CHARLES: It was your idea to create the podcast, in the first place and the podcast has been infused with your personality. Even on the episodes where you weren't present, like you were behind the scenes, arranging it, making sure that it all happened. I can understand like missing that part of it. It really was like your baby, so to speak. BRANDON: What's cool and agonizing is first off, I want to acknowledge that talking about a podcast that you're on while you're on the podcast actually should cause the podcast to collapse in on itself in a recursive loop. CHARLES: You're witnessing a podcast singularity. BRANDON: Then I'm commenting about the fact that we're commenting on the podcast on the podcast. So before we jump into a stack level too deep, I just want to acknowledge that I've had this experience before where I've stepped down as an organizer or a maintainer of something and I've watched the project get better as a result of me leaving. The biggest fear a person has when they step away from something like this is that the thing will collapse as a result of them leaving. The second big biggest fear is that it will not collapse as a result of you leaving. CHARLES: [Laughs] BRANDON: It's sort of like, "Oh yeah. Okay, well, you know..." And the amazing thing is watching it get better. It feels a little guilty like, "Why did I let go of the steering wheel of this thing sooner so that..." Yes, the podcast was definitely my baby and I took care of everything and then I realized so much of that work, there are people that are actually better at this than me. When we brought Mandy Moore in to help with the editing and then organizing and she's taking a bigger, mega-role in all that stuff, she does a way better job than I did. I think that's true of anybody who does a bunch of things and that means, if you are the founder of a company, that's you. You do a bunch of things if you are a founding member of a small organization. As you start letting those go, you find that there are people that like doing the stuff that you don't like to do or are better at the stuff that you do like to do. It's always an interesting, humbling process. Just that alone right there was sort of the lesson I took out of it was that letting go is something can actually improve it. CHARLES: That's actually, I think, critical to the process whether it's open source, whether it's a company, whether it's some side project that you have. There's a phase where -- and really, I guess it applies to the whole start up cycle in of itself, right? There's a phase where you do have to do everything. But then at a certain point, the critical path is you do not do those things. BRANDON: It's before you think it's going to be too, so you always feel like there's always a leap of faith you have to take to say, "I think it's time for me to hand this off and pay somebody else to do something that I've learned to survive doing." And it's before you're comfortable letting go of that thing. And so, people that are uncomfortable letting go of stuff will just hold on to everything forever because there's no clear point where you just point at that and go, "This is the time to let that go." It feels natural. It's time. It's like your kids' first day at school. You're going to cry. They're going to cry and then everything's going to be okay. That was one big thing. Actually, it's kind of linked to why I left in the first place, which is I have historically been so bad at that skill. I have not exercised the muscle of letting things go and it doesn't feel ever appropriate to let things go. So I just took on more and more physical and emotional work and tried to cover more and more bases and did so like borderline adequately in all these different areas to where I just really burned myself out to the point where I couldn't get up and go to work in the morning anymore. I think anybody can relate to this feeling of getting up and being like, "Please don't make me do this again. I don't want to get up." It's like getting up and driving to a place where you eat dirt all day. CHARLES: That's a division of the Frontside. BRANDON: And you're like -- CHARLES: Eating dirt. BRANDON: You know what? It's okay but I had that yesterday. CHARLES: Yeah, it was hard to watch especially like standing right next to you and seeing that process unfold month to month and then compound over the years. BRANDON: Yeah, 'Hard to Watch' is the title of the lifetime movie of my life. [Laughter] CHARLES: Also, I believe it's the title of a Steven Seagal movie from the early 90's. BRANDON: It was a Steven Seagal documentary about getting his hair plugs. [Laughter] BRANDON: "I'll take you to the bank, the hair plug bank." CHARLES: Oh, you beat me to the punch. That's the exact joke that I was going to make. BRANDON: I did it. "I'll take you to the bank, senator." CHARLES: What am I going to do? I guess, I have to carry the torch of early and late 90's movie references around here. You just got to let that fart go. BRANDON: Yeah, there is a dearth of the 40-year old contingent here. Yeah, in the process, I learned a lot of stuff. But the problem was that it was too late to fix it in a way that felt super repairable so it was too late to try to like, "Okay, we'll just make these small tweaks and your life will get better. Okay, let these things go." I was advised by several people like, "Hey, man. You should totally see this thing through. The Frontside is right on the cusp. It's just having its breakthrough moment. The business is doing really well. The employees here are fantastic. Everything is just kind of teed up for 2017 being a banner year." And I just went, "Yeah, but it doesn't matter like how awesome the next city is if you broke down on the side of the road, out of gas." CHARLES: Right. There's a certain point where it doesn't matter if there's more water coming down onto your bucket, like your bucket is full, the bucket is full and it's just the fact. BRANDON: Yep. I'm fortunate to be able to take a little bit of time to think about what's next. I don't think there's much more to say about that aspect of it than that. It is not complicated. People burning themselves out is a thing that happens all the time particularly among founders. There's usually a recovery period and usually they get back on their feet. It's a bummer because I love the people here but I knew I couldn't do it anymore. There may be people with Psychology degree who is listening to this and they are like, "No, no, no. there's plenty more to say about your stupid brain." That's all I've got. It's just I kind of ran out of gas. I stepped away about six weeks ago and you all are doing amazing work, you're in this awesome new space. I don't know... There are a couple things I wanted to talk about. One of them being maybe do a little bit of a retrospective about -- CHARLES: Yeah, let's wind back the clock. Do you want to have a flashback? BRANDON: Let's have a flashback. CHARLES: All right. You actually joined on, was it September of 2013? BRANDON: And we had been talking for about six months before that. Potentially longer but it was really like we started talking in earnest in early 2013. CHARLES: Yeah, it was a long process, like everything we do. It's painfully slow sometimes but that process, by which we came together to try and really do something special unfolded over a long time because there was a lot to talk about. There were so many conversations, meet-ups after meet-ups, phone calls, and just like discussing what would it look like, what are we trying to accomplish. At that time, we're really very focused on pinning down like, "What are we going to do here?" But that was what the focus was. We were talking about the details of how we're going to make it run, who we're going to hire, all that stuff. It was all about what is the purpose of this. It turns out that a lot of people give that short trip but it can be and should be a conversation that takes six months. BRANDON: I don't think I've really acknowledged this too much. But I didn't really have a purpose at the time. I just wanted a shot at trying to build something. Eventually, I knew that if I didn't take it, I would regret it. I found your vision very compelling. You had a vision from the minute we walked through your space. All that was in it was a ping pong table and in the corner, you crammed yourself an intern. At one point, two interns and you were all crammed into one corner room of this big office for some reason. CHARLES: Underneath a marionette. BRANDON: Yes. CHARLES: A marionette from Mexico. BRANDON: Yeah, from Juarez or something and I had you walk me through the space and you're like, "I have a really clear picture in my mind of what's going to happen here." You probably can describe it better than I can. It was like, "Over here..." CHARLES: I actually want to hear you say it. BRANDON: I don't remember this, maybe but -- CHARLES: I do. I remember it very vividly but I'm actually -- BRANDON: Okay, so I remember you walked me through the space and you were like, "Over here, I see two people pairing on a really hard problem. Over here, I see somebody writing open source software. Over here, I see somebody writing documentation for some open source program." You had this really compelling vision of like a beehive of activity around collaborative, open source oriented, good engineering work that grew developers into great developers. It was super clear in your mind that there was a way to do this that what you needed was an engine of growth to hook that vision up to. And I was like, "I've never done this before. But I'd sure give it a shot." CHARLES: Yeah, and the thing is part of what was missing was this understanding that to make all of those things happen, to build that great software, and give engineers the chance to really grow into that role of producing that good software, that you need so much focus on that growth pattern and to foster. You can't just say like, "We're going to adhere to these engineering practices," and boom! Voila! Out of the mix will just pop great engineers. BRANDON: There's a tremendous amount of intentionality in the process of doing that side of it and then there's a tremendous amount of infrastructure underneath that to create. That was the part I think, we both underestimated was how much infrastructure work you and I would have to do over the ensuing few years. But you cannot let go of that vision because if it's all infrastructure, then congratulations. You have a business that makes money and has no reason to exist. Nothing is a bigger turnoff to potential employees in a hard to recruit industry like software development than like, "Show up and we'll pay you." I have a lot of those opportunities. Thank you. So, it was never that. We would rather pack up and go home than that happen. It was always driven by, "If we can't make the kind of place that we want to work, then we should just shut it down." CHARLES: That's definitely was a mantra and it's hard too because in order to make that place, you have to earn that right which means ultimately, you have to have a solid business. You have to take in more money than you spend. BRANDON: I don't want to sugarcoat the initial because we needed money in the beginning anyway. So my first three or four months here was drowning in -- like I showed up on the client and they're like, "Oh, good. You're the Rails expert." And I was like, "I've been a professional programmer for OO for like 14 months." I had worked really hard and studied really hard and I was able to perform job duties. But I was like, "I didn't come here to be a Rails expert." CHARLES: Yeah, but also what was interesting is they wanted you to be a Rails expert. I think I might have sold you as a Rails expert. BRANDON: Yeah. [Laughter] CHARLES: But the thing is as you came in, with an uncanny ability to wrap your hand around what they actually needed so you are probably an expert. While there was the implementation side, I think one of the best things that we brought to that project was focusing on the code that we didn't need to write and only writing those pieces which were critical. But yeah, so you came in for the first, about four months? BRANDON: Yeah, between four and six months. It was September to like January, February. CHARLES: Right. You've got to be the Rails expert in order to have a business so that you can actually play these things out. BRANDON: Yep. And I remember thinking, before I came on I was like, "I want to run a business but here's the stuff that terrifies me -- I don't know how to set up QuickBooks. I don't know how to set up payroll. I don't know how to sell a client. I don't know how to even find clients." And you're like, "I have all of that stuff set up already." So when people are like, "Wow, how did you get the guts together to start a business?" I'm like, "I didn't. I totally [inaudible] one. CHARLES: Well, the thing is you have to start a business at multiple phases, right? I had a business running that could adequately support four people. But then, obviously, we ended up growing and we ended up growing beyond four people. Then the infrastructure of the place, alarmingly, became inadequate so we didn't have to build from the beginning. BRANDON: Another thing that is difficult to learn in business is like it's a truism that is so often repeated that has become trite, which is the things that got you to where you are, are not the things that are going to get you where you want to go. It's trite because it's also not always true, like having a vision is permanent. I was describing this to somebody. Basically, the Frontside policies for a long time were very much like a startup, where people go, "You do the laziest version of whatever you can do in a certain area," so HR practices are lazy. "Hey, we don't have HR people." And then you pat yourself on the back for being a laid back work environment. [Laughter] BRANDON: We're really laid back. We don't have any policies or procedures or payroll? Hang on, hang on. Too laid back. CHARLES: We take a relax policy for vacation. We don't have any. BRANDON: Yeah, you either do or don't go. We don't know. We don't care. That vacation policy of like not having a policy, turns out the laziest form of that winds up being harmful to the people that work there. So we had to discover a lot of the stuff the hard way and start designing and crafting the experience of working here. CHARLES: The thing is that part becomes difficult, like having a clear vision and an idea of what it is that you want to accomplish with this company, it puts you in chains. I don't want to say bad but basically, what it does is it sets very hard parameters on actions that you can take in the things and the practices that you can engage in. It is a good thing but it can feel like you're being squeezed by your own parameters at times. BRANDON: Yeah, I think you and I in the past have called that 'value death', where we had all these values and we started the company as a manifestation of these values where other people may start a company as a manifestation of a business model. CHARLES: Right. BRANDON: It sounds pretty awesome. CHARLES: It does sound pretty awesome. But a concrete example of that is our health plan. We want a place where people feel secure in their living situation so we want to pay them a fair market rate and what benefits we do offer if we want to make sure that they're second to none, that is a really, really expensive value to have. BRANDON: A lot of places will buy that value back out of profit and we want it back out of our first dollar. CHARLES: Right, so it's hard then because it really puts you and the tradeoff is you have a lot less flexibility when it comes to your finances and you see that again and again. BRANDON: I talked to other people that ran businesses in the other direction. When they started with a business plan and a business model, then layered values on top as they were able to afford them. I was like, "Man, that's such a better strategy." And they're like, "No. it's just as painful," because now every time you add a value, you watch it take away the business that you've built. So it's almost the same emotional work, if not more, to try to layer and add those things on top as you go along because you watch your business become less and less of what it was and every time you do it, it feels like you're risking your business. So by absorbing a lot of that risk upfront and defining what we wanted, we actually kind of like we didn't have to move the stake in the ground. It was already planted there and we just had to figure out how to build the business around that. I never recognized that as a tradeoff because it's always like grass is greener over there. CHARLES: It's true and I don't bring it up like a complaint because I don't have any regrets. BRANDON: I got regrets all over the place. [Laughter] CHARLES: I don't have any regret for doing it that way, in the sense that maybe it's just because I have to compare everything to the way that we write software or maybe I just think about software too much. But it really is the outside in paradigm, right? You start from your intentions, your purpose and the system that you want to have and then you infer the processes and tactics that are required to realize that and make it real. BRANDON: As long as it's not crystalline and you can adapt it, then that's a great strategy. CHARLES: Yeah, but when you set those lofty goals, it means that the implications for the infrastructure are bigger than either one of us estimated. BRANDON: Vastly so. And you're right that we didn't quite grasp the costs associated with embracing those things. One of the costs is it requires a lot of thought. It requires a lot of time to manage and it requires defining processes. It just requires a lot of like mental and time overhead that goes into, and then financial overhead that goes into managing all these things that we really cared about. That collected together, we said, "This is who we are." And we've had to back off of a couple of them because we got in over our heads on, "Hey, we're going to try to reinvent apprenticeship." Instead of doing exactly what other people are doing, I know that 8th Light takes a year to design their apprenticeship and it takes about eight people. They weren't able to even take their first apprentice until they hit eight or nine people so they had enough infrastructure of support underneath that apprentice. Instead of learning from them, why don't we just do exactly whatever we want to do, whenever we want to do it because we've met a really cool person and experienced a lot of pain at trying to reinvent things that exist elsewhere, instead of trying to look at what else is out there? Again, this is not by way of regret. This is just one of those things that people do when they have their own thing is they feel like, "Okay, I'm going to do it different," and then they realize why other people all do it the same. CHARLES: Yeah, and I certainly learned like you have to stick strong to your principles but make sure that they're within scope. You know, make sure that be aware that for every value you hold that sits at the top of the pyramid and there's a whole base that needs to support that value. So by all means, hold the value but be aware that's the top of the cone and you have to account for the volume of the rest of it. BRANDON: It comes back to like how closely does this align with your vision of where this thing is going and why. Like, "This is going to allow us to recruit. This is going to put us in a situation where 12 months from now, we've got more senior engineers to help mentor the mid and junior level engineers." And being able to play that really long game is key and it's really difficult and requires patience that I had never possessed in my life before this. CHARLES: You are an enigma in that sense and that you're both impatient yet incredibly patient. BRANDON: Anything that I have that looks like patience is 100% fake. [Laughter] BRANDON: I will put myself in a situation where I can't have the thing that I want to do right now on purpose because if it was up to me, I would have every stupid trinket and gadget the minute like -- CHARLES: Man, I got my eye on a drone right now. BRANDON: Yeah? CHARLES: Yeah. I'm taking it a little bit of Charles to Charles. But yeah, I see what you mean. BRANDON: But you learn the law of the harvest which is you have to plant the seeds. If you cannot, there's no such thing as impatiently getting the return of the values that you want to instill. CHARLES: For me, I don't view patience as a personality trait. I view patience as a behavior. BRANDON: Yeah, it's a muscle that you learn to exercise. CHARLES: Yeah, it's a muscle. It is about impulse like understanding. You said it best -- the law of the harvest that you plant the seeds and you wait and you will get corn or wheat or pumpkins or whatever it is that you harvest and like dance under the moon or whatever. BRANDON: Obviously, pumpkins. I guess you figured out my plan now is to live out to a farm. It's Decorative Gourd Season. CHARLES: This is G-rated podcast. BRANDON: Yeah, all right. Yeah. I want to kind of pivot the discussion with the few minutes that we have left. I want to talk about because it's fun kind of reminiscing about how we got together. Before we pivot to where the Frontside is going, I want to talk about why this partnership worked. There were a couple things wrong with it. Namely, that you and I are both big picture people. My strong recommendation to people that are like, "I need a partner for my business. Cool. Make sure one person is big picture and one person is more detail-oriented," because somebody is going to have to fill that gap and handle details and you'll both hate it if you're both one or the other. But barring that, I would say this is one of my favorite partnerships I've seen in our business, in our industry. I'm very lucky to have participated in it. So I want to talk about a couple of things that have worked well with that. I think the most important thing in any relationship or particularly business relationship is trust. That sounds obvious and stupid but you don't understand how hard that is to maintain because so much is at stake. You're going to have differences of opinion. You're going to have differences of priority. There's going to be money involved, sometimes large amounts of money and that makes enemies of people really quickly. Our relationship of trust has been founded on six months sussing each other out for the fact when like I recognized this the minute I met you but it took six months to confirm that you were going to be the kind of partner that put my needs above yours and I need that because I'm going to be the kind of partner that tries to put your needs above mine. So when two people are striving to do that, you're going to have a solid foundation for a partnership and everything else is secondary. You hope that they have the right skill sets and you hope they have the right other personality traits. But the ability to put the other person's needs above your own, on both sides of that exchange is like with a bullet, to me the most important piece of this. First off, thanks for continuing to be that kind of person, as I've bowed out, that has not changed one wit and even the conversations we've had. I have a few other things but I'm going to bounce it over to you to talk about stuff that makes a partnership work in your experience. CHARLES: I think having that trusting relationship absolutely is the cement in the foundation. Throughout all of this, for example, it did not come to me as a surprise that you were burnt out on this work. I was able to be there the whole time and we were having very honest and very candid conversations about this throughout the entire course. While you did mentioned that we're both very high-level, big picture thinkers and you do want to have someone who is very detail-oriented to make sure that the operations are going to happen, I think that it's amazing what you can accomplish when you have two big picture people who approach a problem from two radically different perspectives. I think your mind works very differently than mine and I really appreciate the way that your mind works. It was always so wonderful to be able to attack a problem. I'm thinking in terms of pairing sessions, in terms of when we were selling something or presenting to a client together, when we were doing trainings, being able to recognize that this person is grappling with the exact same problem than I am and they're bringing a completely different skill set and they are pushing the ball forward using a set of moves that I actually don't even possess. Yet we're focused on the same goal and both even find the process of pushing it forward like enjoyable. I know there have been so many times over the course of this time, this last three years. Even quite recently, I think the last time this happened was when we were interviewing a candidate and it was like you were talking to one of the candidates and kind of explaining who you were and what it would be like to work here. I was like, "Wow, I just really appreciate your mind." Almost like if I could take your mind and hang it on a wall. I know it sounds kind of like Hannibal Lecter. BRANDON: That's super creepy. CHARLES: Yeah, I know. [Laughter] BRANDON: You're like Sylar from the first season of Heroes. CHARLES: Yeah, I was like, "Should I go for it?" Yeah. So let's go there. No, but really, I do. I really have an admiration for the way that you approach things. I don't know, it's just I feel like it's very rare to meet somebody that you have such a strong values alignment with and you get along so personally with and their tactics are completely and totally different than your own. Oh, not totally different, right? BRANDON: Yeah, different enough. CHARLES: The angle of approach. BRANDON: Yep. I loved that I wrote down a blog post about what constitutes a senior developer and at no point did I have any evaluatable technical criteria because that is how my mind works, because I don't care about that. People were like, "What sort of evaluatable technical criteria do you use?" And I was like, "I don't know. That's Charles' territory." And then you did that podcast with it and I just sat there and I re-listened to that a couple times actually as I was writing my conference talk about because it's actually really important to develop those and it's really important to have the signifiers that you have developed those. I couldn't have done any part of this without you. So I will always be deeply, deeply grateful for the fact that you saw something in me that you kind of gave me the shot that I was waiting for somebody to give me, which is weird. Like I was waiting for somebody to ask me to come and found a company with them then you did. That doesn't happen. I don't recommend to people to sit around and wait for the perfect partner to come along and ask them to co-found a company. I think what I learned through the process of all of this is, "Go ahead and be bold and do that." The rest of that is that stuff but do find a partner that you can have that level of trust with and don't settle for one that you can't, that would be like all the horror stories I know in business. CHARLES: I cannot even imagine. BRANDON: I've seen it and it's really sad. When you said it wasn't a surprise to you that I was burned out is because we had such a deep relationship of trust that when we finally started having one-on-ones, I was able to confide everything in you because I knew that you would take that information in the right context. You could be trusted with the raw part of my feelings. The fact that I was scared, the fact that I was frustrated and you wouldn't turn that into either a weapon or you wouldn't turn it into something that made you like me less or trust me less. I just could always trust you that I could tell you unvarnished truth. CHARLES: Yeah, that's certainly when you talk about lessons learned, that's actually a very personal lesson that I took away from this experience. And really this relationship is something that you show is like you can give somebody very direct, very frank feedback and how it is received and how it affects the relationship going forward has everything to do with your emotional intent towards that person. So if you are coming from a place of love, you almost have a free reign of the things that you can say because you're just stating facts and people know that there is that love in there. BRANDON: Charles, I love you but you have cream corn on your face. [Laughter] CHARLES: But it's true that you can get the same feedback from someone who is trying to use that as a weapon and trying to make you flinch. It doesn't even have to be malevolence. It can just be -- BRANDON: Indifference? CHARLES: Indifference but it makes all the difference. So I always try and replicate that with the other relationships in my life, with varying degrees of success but I learned that here. That's a very personal but very powerful lesson that I took away in having experienced that first hand. While we're talking about different skills and different lessons learned and very personal lessons learned, another thing that I took away from here, I said I always like to bring things down to the concrete. I feel like obstructions are dangerous. BRANDON: Yeah, they could obscure your intent. CHARLES: Exactly. So when I say like, "I really appreciate your mind," one of the things that I like is when you're talking, you paint a very clear picture, like your analogies and your metaphors. They're just always on point and you're always able to relate, whether it's we're doing something with software, whether it's an interview, whether it's designing one of our business processes, always able to paint a very clear picture and tell a very compelling story around why it should be that way. So it is focused me that I feel like I've been able to grow as a communicator and really kind of perceive that communication is really the one true best practice, so to speak, and making clear so that you can build consensus around people who are trying to cooperate to do something. It's affected the way that we write pull requests, making sure that you're laying out the facts in the order in which they should be. It means that if you want to accomplish something, you're going to have to convince people that it's worth doing and that is the key thing. Sometimes you can convince them with code but not always. BRANDON: It is a lever that can give you leverage when you're trying to make a point. But you have to recognize it for what it is. It's just an additional lever. CHARLES: Right. BRANDON: All right. Well, as we continue this, I'll volley back one more thing and then I want to ask you about the future of the Frontside. I kid you not. I think I'm actually going to do this. I'm going to change my iPhone lock screen to say, "Is this something Charles would say?" I've told multiple people this so I apologize if I've said this on the podcast before but we were partners for three years, which is as intense a relationship as marriage, if not more so. In those three years, you have said nothing that could even be construed or misconstrued to me as unkind. Not even neutral. Just kind, like inquisitive kind, caring, and the reverse is not 100% true. I don't have anything that I sit around and regret. But I want to thank you for giving me a standard of kindness that I can hold no other people to but I will try to hold myself to. You have made me a better person as a result of our time working together and that you just the deep, deep example of human kindness and thoughtfulness in your approach to the way that you communicate with people. I just want to thank you for all the things that I've learned here, other than my own ability to kind of make my own way in the world which I did not have before this company. The other big thing as an example of this is the kind of kindness that a person four decades into their life can continue to have, the lack of cynicism, the lack of snideness, the lack of anything resembling superiority, so I just want to thank you for that. Thanks for being such a wonderful person. I assume that's inborn and cultivated. I love you so much. So let me shift the topic in our last few minutes here and ask because I'm curious and I'm sure people will be curious. Let me phrase it this way. We walked around a big empty building and said, "Oh, I see this over here. I see this over here. This is going to be a great place where people are encouraging each other and growing as developers." Has anything changed as you're in this new building and walk around in a big empty space that doesn't have enough people in to fill it? And is anything changed as your vision for the Frontside, like I know you're hiring a salesperson and there are a couple watershed moments. CHARLES: Absolutely, and I think this is a watershed moment. I think tying it back to where we were when you came on. It was basically you and me are the two full-time employees. We had this company. We had this idea. We had this dream and we were able to, I think, realize that dream to a large extent. There were people pairing and it was right next to me this morning and I was listening into the conversation and I was not even a part of it. My heart's desire was to jump in the middle of it and get right in there because it sounded like such a fun conversation to have. But I held back and I just listened to it and appreciate it. For all of the wonderful knowledge and value that was being generated right before my very eyes, like it very much was a realization. I think of those dreams that we had back in 2013. In order to make that happen, we had to grow this company beyond just you and me. Now, we were up to almost nine people. I think, with you leaving, we're going to back to seven because we're going to bring, like you mentioned, a salesperson. So there was definitely a couple of phases of growth. We've talked about this on a previous podcast but we kind of had another watershed moment back in March where we realized, “Wait a second. We've cowboy-coding our business. We need to actually write tests, we need to write documentation, we need to shore up these internal structures to make sure that this vision is financially viable and sustainable.” And so we did that. I think that I want to actually like call out and say like, "I don't see that I could have done that without you. That's been the story of this place for the last six months. It is a fundamental transformation of how this company operates. You know just as well as I do that those changes that we put in place, those checks and those parameters and those processes had a profound impact on the viability of this company. But in the process, it also like it did. It burned you out. I think that this last six months and this last year in particular, were incredibly hard. So the next question and part of the problem was that you and I were kind of the single points of failure in that system. We stop cowboy-coding, we put in these checks, we put in the balances, we put in these measurements but we still had a system that had these two single points of failure. I'm not going to lie with your departure that is going to put a lot of stress on the company but it begs the question now, "Is a company that has one point of single point of failure a viable company?" So for me, I don't think the vision has changed that much. I still want to be a place where great engineering happens and where there is a space where engineers or people can come in and grow to be great engineers. But the thing is the change that I want to make is scalable. It needs to be distributed so the changes that we're making right now, the air that is beginning now is going to be introducing scalability so that I can exit or I am not necessary and the people who are here, whether it's eight people, 10 people, 20 people, they can continue this process and live under the umbrella of that vision without there being any one that's critical to success of the entire network. That's what I see coming and that's where I'm focusing all of my energy. BRANDON: Well, I'm excited to see what happens with it. Not excited to show up here to work anymore. [Laughter] CHARLES: But you got to come by and podcast. BRANDON: Every once in a while. CHARLES: Every once in a while. BRANDON: All right. I really love this place and what it stands for. Not just because I got to help define it. It's sort of like everybody loves their own kids. But I think even if this was somebody else's kid, I would love and admire this company because it has stood on principles that are hard to stand on. It's done things that are harder than just getting a business up and running and it survived, which says a lot. I ran out of tenacity juice but it doesn't mean this company has and I'm really grateful to have had a great co-founder that made that possible. It made me feel like I was capable of more than I felt like I was capable before I came here. So this has been a tremendously life altering experience for me. Because of your willingness to give me a shot to risk your business on this guy that you just felt like, "I like this person. I like [inaudible] together." CHARLES: Don't sell yourself short. You shine and you burn brightly. That's apparent to anybody who stands in proximity. BRANDON: Well, thank you and I am so grateful for having had this opportunity. I hope I can use you as a reference. [Laughter] CHARLES: "To whom it may concern..." [Laughter] BRANDON: I guess my wife is like, "Are you going to get a job?" And I was like, "I don't know." CHARLES: I promise not to [inaudible] you on the internet. BRANDON: Yeah, that's the best I can hope for. Charles, this has been really great. I appreciate getting to do this with you. I will miss these 'dadcasts'. CHARLES: I will miss them too. BRANDON: I wonder if anybody listening doesn't know what those mean. Let me reiterate the story. We obviously didn't talk about being dads. But we totally are dads. Stanley, when he used to work here would call the podcast, with just me and Charles, 'dadcast' so that stuck. The problem is if we started a side podcast called 'Dadcast', people would expect us to talk about dad stuff and we're not going to do that pretty much. CHARLES: Yeah, no. It really has stuck, It's like, "So, what's the format of the podcast this week. Is it going to be a dadcast?" [Laughter] CHARLES: "Or should we try, you know, organize a panel." BRANDON: I've heard from other people are like, "When are you going to do another dadcast?" [Laughter] BRANDON: So the answer now is '???' but we'll figure it out. CHARLES: Yeah, we'll figure something out. BRANDON: Oh, yeah. You all are moving to weekly, which is a pretty big deal. Maybe, we'll be able to slip one of those in there in a few months. CHARLES: We're definitely thinking about taking our podcast game up to the next level. We had Mandy, actually came to Austin last week and we ran over a bunch of different options. So yeah, we're thinking about going weekly and I'm actually pretty excited about the content of the podcast. But it's going to be distributed. It's going to be a lot more people doing a lot more stuff. BRANDON: I love that you're removing a single point of failure thing because I think a lot of what my inability or unwillingness to do that was part of what contributed to my burnout, which is obviously not a great way to run your business, to burn yourself out of being able to go back in. I'm glad that you're doing that and I'm excited to see where the Frontside goes. So Charles, thank you for letting me come on for one last show here. CHARLES: Yeah. Well thank you, Brandon. When I think about the future and you can take this as one last contribution, I feel like this is somewhat of a daunting task but I actually take solace in the fearlessness that you've given me, in the sense that I've seen you be fearless in learning Emacs. I've seen you be fearless and having really hard conversations with clients and be fearless in having those same hard conversations with our own employees that we see every day and just watching you operate gives me knowledge, just in the sense of an incontrovertible fact that, "Yes, I can do this and I can try." And so thank you for that. Thank you for everything that you've done over the last three years. BRANDON: I love you, Charles. CHARLES: I love you too, man. BRANDON: Okay. All right. Everybody, please if you have questions for this podcast, send them in. It's the @frontside on Twitter and if you have any feedback, it'd be really awesome if you leave an iTunes review on iTunes that helps people find the Frontside podcast and share with people. Tell them that I won't wreck future ones as best as I can. I won't interject and remove the opportunity to listen to cool guest that you have coming up. Anyway, thanks everybody for listening. And Charles, thank you for being a friend. CHARLES: [Singing] You have a friend and a confidant... and you know what? BOTH: [Singing] And if you threw a party, invited everyone you knew. You would see the biggest gift would be from me and the card attached would say: THANK YOU FOR BEING A FRIEND!
In this episode, Noel Rappin, developer at Table XI, comes on the show to talk about his new book, Take My Money: Accepting Payment on the Web. Noel Rappin: @noelrap | blog | GitHub Transcript: CHARLES: Hello everybody and welcome to the Frontside Podcast Episode 47. I'm Charles Lowell and with me today is Noel Rappin. Before we get into who you are, because I know that a lot of people know you already and you need no introduction, you're actually calling in from Chicago today, right? NOEL: That's true, yes I am. CHARLES: Did anyone actually show up for work today? NOEL: Yeah, people did actually show up for work a little bit later. So, this is being recorded the day after the Cubs won the World Series last night. I will say that my commute in the parking lot at the commuter rail station in the suburbs was surprisingly empty this morning. The people were a little bit slow to come in. The game didn't end till about midnight last night. CHARLES: I'm neither a Cleveland nor a Chicago Cubs fan but it was a harrowing game. It almost gave me a heart attack. NOEL: Oh, man. CHARLES: I can't even imagine what it was like. NOEL: A Cubbie Blue since I was a kid. So yeah, it was something. CHARLES: So, you're from Chicago? NOEL: Yeah, I grew up in Chicago. I grew up in the Chicago suburbs. I went away for school and I worked in Boston for a while, but came back about 12 years ago. CHARLES: Okay. So, that kind of leads me into the perfect setup for one of our questions is how did you get into development? I've known about you since I got into the Ruby community, probably back in 2010 or something like that. How did you come to be there? NOEL: I was one of those kids who was a developer at a certain age, I guess and I had an Apple II. I started programming Applesoft BASIC in the mid-80's, I guess, and eventually decided that I enjoyed it enough and got pretty over-educated at that and that's just a very traditional background. I went and worked at what we would now call web consulting company around 2000, my first professional job was '98 - 2000. I had done a lot of different languages as a student. That job was mostly in Cold Fusion and then eventually bounced around doing a bunch of different web development solutions. At some point along the way, I picked up the Ruby Pickaxe book and then eventually came to Rails. I started Rails as a hobbyist, either pre-1.0 or very early 1.0. I used it to build some task tracker for my team and a couple of other little tools like that. And relatively shortly thereafter, I started doing it professionally. CHARLES: I think you're kind of the same generation that I am. I call this [inaudible] from that early 2000's web development background where it's kind of fueled by [inaudible] network. NOEL: Yeah, a little bit. My first professional job was at a company that is a very, very small consulting company based in Miami and Boston and they were not developers who run the company. They've been documentary film makers and there's like a whole 90's technology trajectory here because they were documentary film makers and then they were documentary CD makers and then they were asked to add interactive stuff to the CDs that they were putting out and then they became web developers as a sort of a natural extension. And suddenly, it was like 1998, every company in the universe wanted to be on the web and they were like, "Oh, this seems like a good business to be in." And they started trying to hire actual programmers who knew what they were doing even though I was just out of grad school and didn't know what I was doing but it was convincingly put in. We did all the stuff then that would be like the first application that I worked there, the production server lived under my desk and was also the staging server and the development server. We didn't use, at that point, search control although eventually we did. There were no established conventions on what was the good thing to do. CHARLES: Yeah. The production server under the desk: a more elegant weapon for a more civilized age. NOEL: [Chuckles] Certainly old. CHARLES: That was a while back but you've obviously been doing a lot more structured, a lot more formal development over the course of your career. So, most of us, myself included, were kind of content to write the software, maybe blog about it every once in a while, give the occasional conference talk, but you've really, really delved in to those activities and really kind of taken the time to really, for the topics that you're interested in, really kind of research them very thoroughly to the point where you've actually written several books. When I think about the concept of writing a book, it sounds like, "Oh, my goodness! That sounds terrible. That sounds like a hundred of my teeth pulled out," or something like that. But you've done it multiple times. I'm curious, did you experience that kind of fear and uncertainty and what was kind of the decision making process that led up to, "No, this is something that I've got to do. I'm going to sign up for this. I'm going to do this. And I'm going to make this commitment." NOEL: Fear and uncertainty is a different question. I've always written things. I was a writer in some ways even before I was a programmer. So, being able to combine the two is something that I really enjoy doing. And I've always really enjoyed teaching. The more cynical way to doing that is I enjoy explaining things in a way that makes me sound smart. But I've always enjoyed doing it in whatever context. So, when I'm not writing, I've always often been involved in training at the places that I work or other things like that. And I've always been a little bit of a theater background and a little bit of a performance background and conference talks are a way to still scratch that itch a bit. CHARLES: You mentioned that you have a graduate degree. NOEL: No, my graduate degree isn't Computer Science but I did a lot of amateur theater things in high school and college. It's not something that I have a whole lot of other outlet for other than talking about development. CHARLES: [Chuckles] You wouldn't think that it's a natural thing but I think the best talks and the best walks. What's it like to have that element of theatricality to them? NOEL: I try in the books to give them a little bit of a voice. Sometimes you get interesting responses when you try to make a technical book a little bit funny. Some people really don't like it and other people really respond to it. I do it sometimes just as a way to maintain my own interest in the 14th page of this chapter or something. As far as fear and uncertainty, that never really goes away. I have a book out that I'm writing now about payments which is a very serious topic. And as much as I've done work in that and as much as I've done research on that, there are obviously people out there who have done more work and know more certainly about individual pieces of it. And there are a lot of people who are going to take anything that I write and implement it that way. They're going to take it as the way that things are done. So, that does introduce a tremendous amount of unsettling. CHARLES: Yeah, you're on the hook. They're like, "But Noel told me that this is the way to do it and I ran out of business." NOEL: Yeah. Actually, the payment book which is called 'Take My Money' is actually only a second book in the history of Pragramatic Programmers to have a legal disclaimer at the beginning which I'm very proud of. I think that the thing that is the most worrying especially when writing something long is not so much factual errors because I can check facts. Facts can be checked. Emphasis is much, much harder to check and the cultural community practices are much, much harder to check. And so, I'm often much more worried about the idea that I might spend a lot of time on something that nobody really is interested in or that I might not spend time on a really important thing just because it's a part of the world that I don't know as well as other people in there, and therefore, I miss something. Factual errors, I worry about them too. But factual errors are relatively easy to catch often, not always. Many, many years ago, I wrote for Rock's Beginner Rails book which was purchased by approximately nobody. But it's got one of those beautiful rocks black and white cover pictures of the author on it looking terrible. At that point, I had worked in Rails professionally but only for a very, very short time and there were certainly people who had a lot more experience than I did. But for whatever reasons, I had the book contract. In that context, there's a lot of uncertainty around not being sure that I know what the newest tools are or what the best practices are. CHARLES: How do you mitigate that? How do you address that? NOEL: Research is one way. There are pluses and minuses to not being the greatest expert or something when you're writing a book like this. One of the pluses is that your experience is a little bit more immediate and it's a little bit easier for you to make it relevant to somebody coming to it for the first time. But then a lot of times, you wind up doing the kinds of things that you might do if you were actually trying to learn it. But then I am a pretty good researcher and then I can compress that down into something that took me 4 or 5 hours to Google or research, or days to figure out the best practice, or talk to some experts sometimes and then I can hopefully distill that down to something so that you don't have to go on that entire journey. You can just cut the chase a little bit. It's researching. It's finding the people who do seem to be prominent in a particular technology or a particular skill and finding out what they're doing. Sometimes, it's getting it wrong, hopefully not very often. And sometimes, it's just like this is the best I can do and I have to like, "I can't solve every problem." CHARLES: Right, just being satisfied with that. NOEL: Yeah. CHARLES: I get it. That makes a lot of sense. You mentioned that the book you just wrote, the one about taking payments on the web, I saw that. I looked at it and I was like, "Oh, that totally needs to be this book. This book totally needs to exist." And thanks that it actually got written. But in hindsight, it's obvious that there wasn't really anything that like it treated web payments as this holistic topic that needed to be addressed and yet, when I look at the last 5 years or the last 10 years, this is a body of knowledge. It's been part of 90% of the applications that I've developed and it's been something inter-rolling like they've been patterns and stuff developed around it but I've never really thought about it, "We need to draw this circle around this practice." And this needs to be researched and there needs to be at least some guide on how to do this. And so, I think from my perspective looking in hindsight, seeing why it was published, "Oh yeah, that's totally obvious that that should have existed." But being on the other end, looking into the future of, given the infinite number of things that I could write about or I could research, I need to go about kind of the selection process and say, "This is something that really ought to exist." NOEL: This came about kind of interesting to me, I guess. The purpose of the book states it out pretty clearly. I, a few years ago, started working on - not a large web project by any means, but certainly one that had a fair amount of complicated business logic around their payments. And we inherited it with the payment gateway in place and I kind of thought that, "Oh, the payment gateway is there. That's the hard part." We just basically sat and pretty quickly learned that that was not the case. As I was really looking for best practices, things people who had delved to more problems on the assumption that a lot of people had to solve some more problems. One thing in particular that I was looking at was handling the failure case where I have to do something in my database and I have to process the credit card and they can't happen simultaneously but the failure of one affects how I process the other. I'm just looking around for how other people had solved that and really being surprised how little information I found about something like that. I kind of filed that away. About a little over a year ago, I had this weird idea that I was going to start doing some self-publishing again and sooner writing about this or thought about writing this as part of that, writing about payments and data modeling and something like that. And as I started getting into it, I was like, "I had enough issues here. This is longer than like a blog post and this longer than a magazine article. There's kind of a lot here." I worked with Pragmatic, so I know the editors there and I sent them a proposal and they pushed back a little bit on the idea of, "Aside from the documentation, what else do you need?" And I was like there are all these issues here. They know, Dave, they run their own web store, they wrote their own code and I actually was - part of when I got the contract because I got to interview Dave Thomas about how the Pragmatic store works which was great. It was really neat. CHARLES: Did that material actually make it into the book itself? NOEL: Not as such because I was a little reluctant to directly quote him about the internals of the Pragmatic store, but some of the principles Dave talked about. It was actually kind of reassuring that about 85% of it agreed with stuff I already thought I knew, and about 15% of it sort of went beyond that. They have some problems that are -- because they're actually managing physical inventory which a lot, not all web stores do, which introduces a whole host of other problems. It became a combination of I actually really do think I've worked on a lot of these problems and they're interesting. And a lot of people seem to have similar problems and there's now a whole lot here, and so it seemed like a book was a way for me to go on that. Other people might have gone a different direction. I am getting kind of an interesting reaction from people. I get a lot people who are saying, "Well, I really could have used that book two years ago." And not a lot of people say, "I really need to buy this book right now," which indicates maybe the marketing needs to be tweaked a little bit. I'm pretty sure it will. It's not even fully out yet, so it will find its audience. But it's interesting how I think people still even seeing the book is there assume that they know more about what they're getting into when they start doing payments. And then they come out the other end a couple of years later and they're like, "Oh wow! I really wished I had the guide on that." CHARLES: So, what you need to do when marketing this, you need something along the lines of 'you don't need this book two years from now' or like 'don't be the person who wishes they read this two years from now, read it now'. NOEL: If I had this book when we started, I would have saved so much time and not made so many mistakes. CHARLES: Yeah. It's always interesting to me the interplay between kind of the patterns evolving and documenting the patterns and how eventually that kind of precipitates code that makes those patterns automatic. But it feels like there's always this kind of knitting edge where you have to kind of explore the problems phase and do the research and then eventually you can kind of turn and digest and write little pieces of it, but eventually it becomes like code. Have there been actual code like plug-ins or stuff that has fallen out of this, or that people can reference directly? NOEL: Yeah. Not as such. There's a reference application in the book, but I submitted a couple of patch requests to a couple of gems along the way but nothing significant. It's pretty unusual for me to co-model a book like this with a lot of new open source tools. I'm usually more about using the things that are already there. But the book definitely has an opinionated set of thoughts about how you design a Rails application and it's pretty upfront about that. We're going to put all our workflow logic in their own objects and we're going to not put a lot of stuff in the active record models and we're going to do it for these reasons. The first chapter of the book actually is not even payments. It's just shopping carts and a way to talk about the data model and the design because ultimately, money has some word aspect but a lot of it is just principles of doing good software design, encapsulated software design, and the way you will deal with any big external 3rd party dependency. But then on top of that, you have the specific pieces of money is weird, like money has real world implications and has legal implications and things like that. CHARLES: Yeah, definitely. Whenever you're dealing with money, there's an elevated level of pressure that seems like it's put on. And I know personally when I think of -- lately, I was writing my very first payment system. We actually maintained for a while the stripe-rails gem which had moderate popularity but we don't maintain it anymore. But it was that project where we did that where I kind of became an information recorder. And it kind of started me on this journey of never destroying information, kind of that immutability and like functional programming and stuff like that because I was so worried about over-writing any information about the payments coming in. I was like, let's capture it from the web hook and write it to the database and replicate it and back it up and let's never throw away a fact. NOEL: The specific advice in the book is save as little of your user's personal information as you can and as much of the transaction information as you possibly can. It definitely recommends saving the entire response in the payment gateway. It recommends making sure that you store all of the pieces of your partial price calculation so that you can recreate it even if the logic changes because that is something that has specifically bitten me in actual world. CHARLES: Can you elaborate on that when you say partial price information? NOEL: Yeah. The application that I work on, it sells tickets. And it has a processing fee. Originally, the processing fee was whatever the processing fee was and I hard-coded it as a constant in the code base because I thought I do not need to write this processing fee to the database for every individual transaction because I was wrong. And eventually of course, they changed the processing fee which would be fine if you never had to look backwards at all and try to run a report of old transactions and try to validate that the prices on old transactions still held. So, they changed the transaction fee and then they started running reports and all of the reports are off by whatever the changed amount was. CHARLES: Right, I see. So you could still maybe store that constant in the code base but you have to at least write out what it was at that point when you did the transaction. NOEL: Yes. Not all of this has managed to make its way back to the legacy code mass that I'm actually dealing with. But what I would recommend is saving all of the individual pieces. Unit price, any processing fees, any shipping fees, any tax fees, any discounts, saving all those as they were calculated as part of the transaction so that you always have them and they're not dependent on the code or the logic staying the same because all of that logic will change. CHARLES: I'm all about the information hoarding. I think it's useful stuff. [Chuckles] NOEL: I even started recommending originally a failed transaction was in a database transaction so it would go away. But I think that it's a bad idea. I think that you need to save -- whether you actually save a partially failed transaction in your database or whether you log something, the information of those failed transactions is really important and you want to make sure that you hold on to that stuff and you can go back and look at it. The idea that a bunch of transactions are always failing and you don't know about it, that's problematic. CHARLES: It sounds like you're still working on this application which [inaudible] the book. NOEL: Although I'm a lot more critical of the code now that I've gone through -- the book made me sort of think about this stuff a little more systematically that I had and made me make a lot of things concrete that weren't quite in the original application. So, the code in the book is actually better than the code in the application. CHARLES: Of course. NOEL: It's also a lot simpler. CHARLES: Exactly. [Laughs] Because it doesn't actually have to do the real work, it just has to capture the principles. NOEL: It just has to be a good enough façade to show the ideas. CHARLES: Right. So, you're still working in this application that kind of service [inaudible] for the book. But I'm curious then how you manage that because for me, writing a blog post is hugely time consuming. Writing a book must be even more so. Do you just have it dialed in that well that you can just bang it out or do you mix doing the book writing with working on the application because obviously, it sounds like both are kind of crucial to each other. You've already said, "The fact that I wrote the book is actually informing the way that I structured the code that made me write the book in the first place." And so, how do you interleave those activities to make a continuous workflow? NOEL: There's a separation there because the book is the thing I do not at work, it's my side project effectively. I'm not ever mixing them in the sense that I'm really going back and forth between one thing and the other. The application is the thing I do at work; the book is the thing I do at home. They inform each other, obviously, but separating them is not usually a problem. They're not the thing I do at the same time, so they don't relate to each other that directly. CHARLES: I see. NOEL: It's time consuming but it's what I do instead of maintaining an open source project or doing some other community thing in what would be side project kind of work. CHARLES: So you just do it on your spare time. NOEL: Usually, the book is written between 10 and midnight for people who are reading between 10 and midnight. CHARLES: [Laughs] I like that conception. Do you think it would work at all if say, with your job there were some amount of time allocated for writing the book because I think there's some interplay and hopefully, it's [inaudible] interplay between the two activities. NOEL: The way my work job is structured right now, I actually do a certain amount of time for blogging but that's company blogging. It wouldn't necessarily cover working on the book. It would certainly go faster if I had time because certainly trying to squeeze some parts of this into my allegedly spare time becomes challenging. Writing a book like this is not just writing a book. You're actually writing to some extent an application and it's not a fully featured application but it does have to work and the tests have to pass and things like that. It has to work well enough so that if somebody were to copy it, it would run. That can be frustrating when to come to writing the book and find out that, "Some gem is updated and now, there's some weird test failure in my book," is sometimes a little frustrating. CHARLES: Yeah, I know. I can imagine. You said that the book, most of the examples are in Ruby on Rails. Would you see it being valuable to someone who was not doing Rails development at all, maybe he was doing something in Python or Clojure? NOEL: Some of it is in JavaScript because the client-side Stripe library is in JavaScript. That's kind of the central part of it especially for security purposes. Some of the code would carry because the Stripe API is actually pretty similar across a lot of these languages. I think some of the principles would still hold. Some of the tools are a little bit Rails specific but some of the basic ideas of separating things in the background jobs, the way that you need to think about the administration, handling failure, like some of that I think would have some value outside of the Rails world. CHARLES: Were you explicitly thinking of how the developers with the web payment problem in general is your audience or were you really thinking this is more for Rails people or where did the kind of needle fall there? NOEL: The tricky part here is that it becomes very, very hard to write a book simultaneously using the Rails API and the Python API. It's very confusing. It's a 300-page book as it is and you have to make some decisions there. While I would like it to be valuable to anybody who's doing web payments as sort of a practical matter, Rails and the Ruby API are the ones I know best and that's where the examples are because they have to be in something and they can't be in everything. CHARLES: I remember a lot of those books that I read that used Java kind of as the one [inaudible] of the system that they were talking about. But it really was like, you're right, there's a line that we have to tread there because you can't just wave your hands and talk in one piece abstract principles that have no ground in reality. But at the same time, by giving concrete examples, you may limit the potential audience. I do think there is a path in there where your examples are accessible to people from different backgrounds. NOEL: Examples are really hard, actually. It's a very tricky thing to have an example that is complicated enough to get the point across but not so complicated to get buried in extraneous details. It's really a problem actually when you try to write about testing because almost any problem that is simple enough to be an example in a book is not complicated enough to show why test-driven development is a good idea. [Inaudible] has the same issue a lot of times like you start bringing these relatively robust methodologies to this little toy problems and it's hard to get across why they're valuable when you're working on a toy problem. Not impossible. There are people who do it. CHARLES: If I can count the number of tutorials that were like modeling cats and dogs and people, it's like I've never written a single class that even remotely resembled any of that stuff. NOEL: You're bringing in more complicated problems but then you become so sensitively dependent on the weird crooks of whatever problem you've chosen which can be difficult if you make it feel like it's less universal. If the idea is that these techniques or these tools are going to help you manage the complexity in your application, then you need to show an example that actually has enough complexity for them to sort of work. It can be really challenging. CHARLES: You mentioned also the examples [inaudible] some of them are in Ruby that you're actually opinionated about making sure that the main logic is captured in objects which I think actually furthers that goal where you're not really bound so much to the Rails APIs. But then you said obviously, the other stuff is in JavaScript just because the client side Stripe library is implemented in JavaScript. You wrote a book on JavaScript a few years back. So clearly, it factors into your development story. I like to often kind of try and tell people like we are all JavaScript developers, whether you know or not. Do you have any advice for people who were approaching JavaScript from outside the core of [inaudible]? What's the best way to make your [inaudible] approach? NOEL: I have opinions on this. The problem is that I don't do big single page JavaScript apps, for the most part. It's just not part of my nutritional background at the moment. Mostly the JavaScript I'm doing is relatively at the kind of thing they sort of derive as sprinkles of JavaScript. CHARLES: Like I said, I think it's a far larger attempt than anyone cares to acknowledge. NOEL: I feel like my decision to try very hard to stay out of the JavaScript ecosystem battles has paid off for me tremendously in most ways. At the same time, I don't have really strong opinions about JavaScript build tools because I hadn't really had a whole lot of occasion to use them. I very strongly recommend that people coming into JavaScript now that you use some of the ES6 features especially things like classes and stuff that like to give you a better structure. Structure in JavaScript gets a little bit more consistent than what you see in other programming languages. A lot of those changes were I think pretty beneficial to how I work with the language. One thing that I like to do when I am doing something that is smaller, too small to really bring in a framework because I'm still doing mostly kind of jQuery, is I have a style where I really do try to isolate the jQuery in the DOM from the logic in a way that they don't always see people do on the theory that the DOM is a big 3rd party dependency in a way that on the server side, the database is a big 3rd party dependency. And in both of those cases, we kind of pretend that they're not big 3rd party dependencies because it's easier to integrate them, but eventually that kind of hurts. And I find raw jQuery and raw JavaScript is very hard to read and hard to understand. So, I think it does a little bit better if you start burying it behind some stuff that has some more semantic [inaudible] and things like that. And then we get to the point where you need a framework that actually has data binding even if it's a small framework like Vue.js or something like that, then you have some habits that will help you as you start building them. CHARLES: I think it's been interesting. Obviously, you did JavaScript a lot here but having that separation in kind of a mental model of what is my representation of the data and how am I actually presenting that to the user. It's kind of scary how universal that concept is and how well it will serve you as you grow from the simplest little one liner JavaScript thing to a whole full-fledged single page application using whatever framework you choose. If you abide by those principles, it may not be easy but it's going to be a lot more tractable. NOEL: Your transition to using one of those tools is going to be a lot easier if you already in habit of separating out your display logic from your actual logic. CHARLES: Yeah. Believe me, I'm not looking to like draw you into any kind of controversy here with JavaScript or Rails. But in the last several Ruby conferences that I've been to, there's been a lot of talks and a lot of discussions about 'where's the Rails community headed', 'where's the Ruby community headed'. How does it fit into this kind of new ecosystems that are emerging and what's the role that it's going to play down the road? And so, I'm just kind of curious to get your take on that as someone who operates in a bunch of different environments. Is Rails something, if you're starting an application from scratch, you would pick up again? NOEL: Yeah, I still pick up Rails for new applications. I think that, especially because the ecosystem is still so far ahead of all the newer tools ecosystems, I still think it's still the best way for a small team to act like a big team. We're definitely investigating other stuff, other tools. All of our Table XI developers are trying to pick a framework that they don't use everyday ideally in a language that they use every day and build a common app in it just to try out some new muscles and get a sense of what the strengths and weaknesses of some of these other tools are. We're starting to bring them in slowly to our server and client side repertoires where they're appropriate. I think that Rails is great but it's not perfect which leads the possibility that there will be something better eventually. I think it will take a while for something like Phoenix and Elixir to build up the ecosystem and the number of 3rd party tools that the Rails community has. CHARLES: We've been playing around with a bunch of different tools here. I actually started a side project and I was using Rails and I was blown away. I was like, "Oh, I forgot how easy it actually was." And that actually can't be discounted and things like speed and things like this lightweight feeling, you can [inaudible] for the lack of an ecosystem. I certainly made that mistake with Node.js. NPM installs at the very beginning were super fast and that was the reason that you weren't installing anything at all. [Laughs] NOEL: Rails still prioritizes a certain kind of developer experience in a way that is really great for projects up to a certain size. I think there's a significant dispute of where that certain size is. I have a coworker who did a Rails project recently after also having done a bunch of JavaScript projects and had a very similar reaction, "I forgot how easy some of the stuff actually can be." Eventually, you get to a size where some of that easy search could be less [inaudible] in terms of something a little more structured. But there's still a lot of value there. I find the Node ecosystem honestly really complicated and confusing relative to the Ruby ecosystem. And some of that is just my lack of experience with it. CHARLES: We're [inaudible] everyday and it's complicated. It's just that it's huge. It's just mammoth compared to the Ruby ecosystem and I don't think it started out that way, but it certainly is like that today. NOEL: Rails is at least somewhat focused on a particular size problem. And I think that that makes the ecosystem a little bit more manageable. I think the JavaScript tools are trying to solve a lot of different problems for a lot of different people sort of under the same umbrella and it gets very hard to do that. CHARLES: Well, that's about it for our time. Thank you so much for coming and talking with us. NOEL: Thanks for having me. CHARLES: Yeah. The name of the book that you're currently writing is? NOEL: It's 'Take My Money: Accepting Payments on the Web'. It is available at PragProg.com. And then you can find anything I'm doing at NoelRappin.com. Keep an eye out there because there's going to be a new project that hopefully will start by the end of the year. I'm a little less confident that it's going to than I was, but I'm still hoping. Also you can find me on Twitter @noelrap. CHARLES: All right, fantastic. Bye everybody!
In this episode, Sarah Mei, founder of RailsBridge, Director of Ruby Central, and Chief Consultant of DevMynd Software, talks about the way we write software: What's right? What's wrong? How can we do better? The conversation examines changing code and reassessing needs. i.e.: "Does it bring me joy? Should I get rid of this thing? Do I understand this code?" She also talks about what these needs mean for others on a team. Sarah Mei: @sarahmei Links: Sarah Mei: How We Make Software: A New Theory of Teams @ Brighton Ruby 2016 The Life-Changing Magic of Tidying Up: The Japanese Art of Decluttering and Organizing by Marie Kondo Transcript: CHARLES: Welcome to the Frontside Podcast. I am Charles Lowell and with me is Robert DeLuca. We have a very special guest this week. One that I'm really excited about because the things she says and the ideas that she has - open eyes and minds all over the place, in all different types of areas that are so pertinent to the way we do our jobs. So, we'll get to it. Our guest today is Sarah Mei. SARAH: Hi. Thanks for having me. CHARLES: Like I said, we are super excited to have you here. Before we get started talking about some of the things that you've been thinking about recently, why don't you just give like a very brief introduction of how you got started with development, where you've been, and how has that brought you to where you're going right now? SARAH: You know, I actually was not one of these people that got started with it real early. I came to programming in college. I was an Engineering major. I wanted to build bridges. I wanted to be a Structural Engineer. I want to build things. I had a weird schedule the first couple of quarters of college, so I ended up taking an elective earlier than most people take it. It was a programming class in Fortran that was required for the structural engineering program. I took my class and I was like, "This is really cool." CHARLES: Wait, Fortran is what set the hook? SARAH: Yeah, and the professor of the class was like, "Well, if you think Fortran is cool. I've got some other stuff that you might like." I mean, the language and whatever doesn't really matter. What I liked about it was the fact that I could build something. I can get that same feeling of building something that you get if you build a bridge but you can do more than like one or two in your career, like you do if you're a structural engineer. I like the constant feeling of building. That's what I liked about it. So I ended up switching my major and graduating with the CS degree and coming out and doing a bunch of different things, mostly like starting in a large company and sort of doing smaller and smaller companies over time. CHARLES: Yeah, there's a lot of people in the industry who are career switchers, where they started out in something else and moved into Computer Science but I actually feel that a lot of people, like myself included, I have the degree in CS too, but that was not what I set out to do at all. It totally derailed, like the course of my life in a good way. But in that way, it's like a career switch within a career switch. ROBERT: I'm a little odd in that aspect. I came out of high school like ready to go in software. It worries me a little bit for the later half of my life. I'm like, "Oh, am I going to do software for the entire time?" CHARLES: Probably not. SARAH: That might be a good thing. You'll never know. ROBERT: Yeah. CHARLES: Yeah, seriously, what lies ahead? ROBERT: Who knows? SARAH: I feel like in a lot of places that are like, for example, in public policy and in other places where we need more people that understand tech so if we can send you out into other parts of the world knowing a whole lot about programming, that can only be good. CHARLES: Yeah. ROBERT: Yeah, this is actually kind of funny. I was telling CHARLES about this the other days, like I'm starting to view programming more as a tool to do the things that I really want to do and less as like the thing that I'm going to be doing forever. I wanted to augment and make things that I have a passion about easier. SARAH: Yeah, absolutely. CHARLES: Yeah, it's like software is eating the world so what you're doing now is just learning how to chew. ROBERT: That's a great way to put it. SARAH: You should tweet that. [Laughter] CHARLES: All right. Please continue. I'll ignore the typing sounds. SARAH: [Laughs] Switching careers is a really interesting thing because you end up with a bunch of experience that you wouldn't have had otherwise. I'm really excited actually about the next five years as we have all these folks that switched into programming from something else who are all becoming mid to senior level because they're bringing just such amazing experience from other parts of the world. CHARLES: Yeah, I know, right? It's like, "Where've you guys been my whole career?" SARAH: Right. CHARLES: It's like you understand these things, just almost like it's second nature of these things that are opaque and completely inaccessible to me. So anyhow... SARAH: That's how I got here. CHARLES: So then, after you kind of switched in college, you went out and did you just start working in programming immediately thereafter? SARAH: Yeah, I worked in a bunch of different product companies. I built products for a while. My first job actually out of college was at Microsoft up in Redmond and then I have worked at smaller and smaller and smaller companies. Then I spent about 10 years doing product stuff and then about 10 years ago, I switched into doing consulting mostly because I realized that I have a fairly short attention span for projects. And that working on a product, there wasn't anything wrong with me exactly but what would happen is when I was working with a product, I would get six months to a year into it and I'm starting to get antsy. I started to get bored and decided that I should just embrace that. And I switch to something where I am going to be on a new project every three to six months. I've been a lot happier since then. ROBERT: That's interesting. I wonder if that comes with seniority in software development and knowing your way around because consulting for me is I've gotten the experience of, "Oh, wow, I'm just finally getting a hang of this person's product or this client's product or app or whatever we're building," and it's, "All right. It's time to rotate off." It's like you just get in there and understanding everything. SARAH: There is that aspect of it for sure but even when I was much less experienced, even with my first couple of jobs, I noticed this tendency in myself to just get bored after six months on the same code base. For a long time, I thought it was because I'm not cut out for software or maybe I'm not very good at it or something. Eventually, I just realized now actually, it's just that I just need to switch projects. I'm just one of those people. That's how my brain works. I get a lot out of switching projects because the one that I switch on to, I see an entirely different way of doing things like code bases are so different. Even if you look at a hundred different Rails apps or a hundred different Ember apps, they're all so different. So switching on to somebody else's app, I learned a ton just out of that switching process. CHARLES: It sounds like the actual kind of studying the meta-level of the software is what really engages you and kind of understanding how the software came to be the way it is and not some other way. One of the factors that gave rise to that and kind of 'that's the problem' that really sunk its teeth in you, as opposed to individual business problem. Is that fair to say? SARAH: It has certainly been interesting to see different business problems and to understand different parts of industries and so on. That's definitely part of it for me but what really gets me interested is the different ways that people organize their code and by how they make the decisions that they make. ROBERT: Yeah, you get to see different problems that they've maybe put themselves into because of the way they structured something, which you wouldn't see if you wrote yourself but somebody else did and get to see, "Oh, I understand this pattern now." That's kind of been my experience out of it. I don't want to speak for you, but yeah, that's kind of how I've seen other client projects like, "Oh, this is really cool. I didn't think of a way to do this," and you get to experience many different things in many different ways. SARAH: You get to see a lot of the tradeoffs. Like a lot of times in a single code base, what would happen is I'd make a decision or we'd make a design decision of some kind. Then I'd see how it turned out. But there's no way for me to see how it would have turned out if I did it the other way. The nice thing about switching projects for me is just being able to see all of those tradeoffs, like the tradeoffs that you make tend to be pretty similar. You can see very similar situations where people do different things and how does it turn out for them. ROBERT: Right, and like one of my favorite things is where you go into a project that is totally against something, like for me it was object-oriented CSS and then you go in and you actually see it in practice, and you're like, "Oh, wow. This is turning a whole new light on it. I like this in this case." SARAH: Microservices are like that for me, where it's generally I am anti the microservice bandwagon. But then I went on one project where I was like, "Wow, they actually figured it out. This works really well. I can see why people like it," because I've seen so many work that was horribly executed. When you go on to the one where it's good and you're like, "Oh, this is why people do that. Okay." ROBERT: Yeah, it's like that light-bulb click, "Oh, yep. There's another side of this." CHARLES: Once you actually see it done right, it helps you avoid every other situation where it was done wrong and you can say, "Oh, this this was the one differentiator that made it all go right." I mean, sometimes it doesn't always boil down to that. But there's these one, two, three things that we could have done. But they were just completely and totally hidden from you because you didn't have that context. I would love to talk to you about microservices because I've certainly never seen it done right. I've heard it talked about and I've seen this beautiful world, picture-painted that looks so fantastic on the whiteboard. But I see -- SARAH: Oh, it's so beautiful, isn't it? It's like an object-oriented design diagram. I'm like, "Look at all the boxes and lines. They all line up." CHARLES: "They're beautiful." SARAH: "I can do this in Visio," and they're all like, the line, they are on the same shape. It was great. CHARLES: "And when I move this one over there, it just tells me that these two are exactly the same distance apart from that other one." Ah, so satisfying. SARAH: Yeah, and then you try and do it, is the problem. ROBERT: Then you build it and you cross your errors and everything. CHARLES: Which actually I think that brings us, recently -- we're talking on Twitter. I think that's actually very recently about kind of the difference between when we talk about software and the meta conversations we have around it. When we do talk about these abstract and perfect worlds of boxes and lines versus the actual code bases, which is the things that you've kind of been observing many, many, many since you've started consulting, and kind of the vibe between those and you know what that means. I think a lot of people aren't even aware like I certainly, before kind of reading that, wasn't really aware that that is a very, very distinct difference, like these are two very different modes for software. One that exists and one that is kind of perfect world. ROBERT: Kind of academia versus the real world, I guess. SARAH: In some ways, yeah. I remember when I was in college, we had a software design class as part of our degree program. We studied how you define objects and you write a little bit of [inaudible], like we did all this stuff. When I got out and I got into the real world and I had a job, I found it very difficult to actually apply that stuff successfully, to be able to draw a diagram and then turn it into code and have it work out the way that the diagram said it was supposed to work out. I initially thought that was because I was just not experienced enough to figure it out. But eventually, what I realized is that it doesn't work because it doesn't work. It really doesn't work to design things ahead of time and then just do them. I think there might be a certain type of person that can do that. I am not that type of person and most people aren't. I think that it takes a very unusual type of brain to be able to just draw a diagram that has already taken into account all of the things you're going to encounter once you start making it. CHARLES: Yeah, I would even go so far as to say there's probably a brain that solved that problem many, many times, that just could skip a bunch of steps. SARAH: Right, and they're not aware they're skipping them necessarily. Unless you have an entire team full of that type of brain, it's probably not a good idea in general, for the software that you're building as a group. I feel like I've been trying to talk about that concept between the difference of how we talked about software in books, in blogs, and in conference talks and then how we build the software we actually build. I feel like I've been trying to articulate that for 20 years, like since I have my [inaudible] and I was like, "This doesn't work. Why can't I make a diagram and then make it into code?" Like two days ago, I feel like I finally found a way to articulate it that captures everything that I've been trying to communicate and it was a really strange feeling. I'm like, "Wow, I finally kind of got it." One of the reasons that I came up with that, I think, is because I haven't really been thinking about it for a couple of months. I've been off and not really thinking about software stuff for a while. Oddly enough, I've been thinking about organizing my house for the last three months. All of my free time outside of my job has been thinking about like, "I've been learning how to cook, so how can I organize my kitchen so that the things I actually use every day, I don't have to dig through a drawer every single time to find them?" There's actually some interesting problems there like, "How do I make sure that all of the stuff that I need is at hand that I use all the time? All stuff that I need occasionally is still around and accessible, and then things that I don't use, I should probably just get rid of." I have this problem that I think probably a lot of people have which is that I have trouble getting rid of stuff once I have it. I live in a small apartment in San Francisco and that's not a good thing to be able to unable to get rid of things because in an apartment this size, I can let it go for a week or two maybe, but like I got to be very vigilant about it because otherwise, it just overwhelms the space. CHARLES: Yeah, there's a bunch of research that the people estimate vastly different the cost of acquisition versus the cost of loss, and they've [inaudible] way too much, like irrationally unbalanced like not wanting to lose something that they already have. SARAH: Even if I bought it for a need that I don't have any more or the need has changed or shifted. I don't buy things I don't need. There are some people that have that problem, that they buy a bunch of stuff that they don't have any particular plans for it. I don't have that problem, thankfully. I've had people in my family that have that problem which I think is why I have avoided that. But the problem I have is that once something is here, I find it very difficult to get rid of it. I look at it and I'm like, "I can think of all these reasons why I shouldn't get rid of it." Oh, that was expensive so the sunk cost fallacy of like, "Oh, I paid a lot of money for that even if it's not useful and I don't like it, I shouldn't get rid of it." Or, there'll be like a dress in my closet that I haven't worn for two years and I'm like, "Ah, maybe I should get rid of it," and I take it out and I'm like, "Oh, my God. But it looks really good on me. I like it. I should wear this. I should really wear this." So I'm going to keep it even though I haven't worn in for two years for some reason, but I should keep it anyway because it looks good. I have all these stories. I tell myself about why I can't get rid of things. A couple of weeks ago, I read a part of a book, to be totally honest with you, called The Life-Changing Magic of Tidying Up. It's written by this woman from Japan who's a professional organizer. Her name is Marie Kondo and her method is called KonMari. Basically, what it does is when you're trying to figure out whether you should get rid of something, you don't ask yourself, "Should I keep it?" What you ask yourself is, "Does this thing bring me joy? And if it brings me joy, then I keep it. If it doesn't, then I'm going to get rid of it." So that made it really easy, going back to the dress example. I'm like, "Does this dress give me joy?" And I thought about it, I was like, "No, the reason I don't wear it is because I went out to dinner and I had a bad experience at dinner so every time I look at that dress, it reminds me of that experience." And so it looks good and everything but I'm not going to wear it because it doesn't make me happy. So that was just like, "Okay, fine. I'm just going to give it away." And changing that question that I ask away from 'should I keep it' towards 'how does that make me feel' was a huge change for me because it's like, that's really easy to answer, where 'should I keep it' is a much harder question. There's these bunch of sort of ifs and maybes or what-ifs and what happens. I feel like that applying this KonMari question to stuff has changed the way that I calculate what stays and what goes in a very positive way. CHARLES: Yeah, boy, I need to get this book for several family members who will go [inaudible]. SARAH: Well, you know, I've got two kids and so there's a constant flow of stuff coming into the house. Because of the amount of space I have, there has to be a constant stuff going out. So this is something I just need to be very vigilant about and this has made it so that it takes up a lot less of my time and a lot less of my brain space, which is really awesome. It feels like it's moving my house in the right direction. I've been thinking about that sort of in various ways, on and off, for a couple of months and I haven't been thinking about software. I have this fear that like, maybe that means I'm never going to think about software again. I go through these phases where I've got like, "Oh, I'm going to come up with a bunch of new ideas," where I'm coming up with new ideas for some whatever reason. Maybe I'm making new conference talks, I'm doing stuff, and I'm thinking about software a lot. Then I go through these phases where I don't do that, like I sort of retrench and maybe... I don't know. I think about other stuff for a while. So it's been home organization for several months now. I was like this, "I'm never going to think about software again," because it's just that -- [Laughter] CHARLES: Career change. ROBERT: Oh, man. This sounds so much like my life since I moved down to Austin. SARAH: You know, I live in San Francisco and I'm not 25, I'm 40. A lot of it is like maybe I'm just too old for software now. I should just give up and live out the rest of my career doing quiet, maintenance work -- [Laughter] SARAH: Somewhere. I don't know. Then suddenly, this thing happened on Monday, where I was just like, "Oh, code, an organization." And boom! There it was. I realized, I was like, "I basically just had to give my brain some time off," like my conscious brain needs some time off from software and it wasn't that it had disappeared because what I came up with on Monday was really just how home organization applies to code because I realized that the feelings that I get when I'm trying to figure out what I should do with code are very similar to the feelings that I get what I'm trying to figure out whether I should get rid of a thing. I look at this piece of code and I'm like, "Should I change this? Should I get rid of this? Should I refactor it?" You know, why I can't get rid of that? We just spent two weeks refactoring it so I can't change it again. [Laughter] SARAH: We just put in a story for refactoring this and we spent three days and I can't go back to the [inaudible] people and tell them, "I need to change it more." Or, "I really like this code because I wrote it with someone that I really liked." CHARLES: So I don't want to get rid of it. SARAH: I don't want to get rid of it because then I would lose the memory of working with, you know. CHARLES: I actually can say that I have experienced that. SARAH: Yeah, there's a lot of reasons why you don't want to change code. What I was thinking about, like maybe I was asking the wrong question, in the same way that 'should I keep this' is the wrong question when you're talking about stuff. Maybe 'should I change this' is the wrong question when you're talking about code. Maybe it's sort of leading you in the same way with stuff that leads you down this conversation of reasons that don't really have anything to do with the essential quality of why the code is there or why the thing is there. We need something that helps us reassess our needs. So if our needs have changed, maybe you don't need that thing anymore because your needs have changed. Same way with code. If your needs have changed, maybe you don't need that code anymore, at least not in the form that it's at now. I think that question for code that, "Does it bring me joy," because joy is not something that I think is concrete enough when we're talking about code. I think the question for code is do I understand this? Do I understand what it's doing? Not just understand it like a very surface level of like, "Can I figure out what this syntax means?" But understand it more like the grok level of like, I understand this at a very deep level. I understand why it's here. I understand what problem it's solving. I understand why this abstraction is necessary. I understand how it got here. CHARLES: Yeah, how it fits into the bigger picture. SARAH: How it fits into the bigger picture, exactly, like the application. CHARLES: How it fits in with like our conventions that are just purely stylistic. SARAH: How does it fit in with the other stuff that we've been doing? How does it fit in with the product needs and the features we're trying to build and the business goals and all of that stuff, all of these different levels of understanding of why this code is here and what it does? CHARLES: Do other team members' understanding factor into that? Like, "Do I understand the way that other people understand it," so to speak? SARAH: I think that it can. But I think the important thing is whether you personally understand it. CHARLES: Okay, like it's a very personal decision. SARAH: I think it is. Hopefully, what you do is you want different people looking at the same code. You don't want just one person on a piece of code that no one else ever sees, whether it's pairing or code review or whether it's something else. It need to be really clear to someone is coming in and looking at that code what it does, what it means, and why it's there? CHARLES: Right. I guess the reason I asked the question is because a lot of times when I look at a piece of code, I try and really step outside of myself and say, "What will someone else think who has never been on this project before?" Or, "Who is on this project and they see this code, will they understand it?" SARAH: Absolutely. It's definitely a part of it when you're on a team. CHARLES: Yeah, so I'm just trying to figure out how that question factors into this framework. SARAH: I think that it depends a lot on how you distribute tasks. For example, if you work in a shop where you're pair programming most of the time, so there's always two people looking at a piece of code, 'do I understand this' is a reasonable question just for the two of you to consider, both from the fact that you can pool your knowledge but also from the fact that 'are there pieces of this that you understand that I don't understand' and vice versa. On the other hand, if you work in a shop where it's more like, "Here's the piece of code that you work on like you own this section of code." Then I think it's more important for you to be able to step outside and be like, "Okay, do I understand this? Would other people on my team understand this?" That can be a very difficult thing to assess and that's where I think it's very helpful to do things like code reviews, call people in and be like, "Hey, can I run some stuff by you. I'm trying to figure out if this is good or not," because what you want is you want a code base that is comfortable and understandable for you and for your team. Just like the thing that makes the KonMari Method powerful for stuff is that it doesn't tell you what you're going to end up with. It doesn't tell you what level of clutter versus cleanliness is good for you. It doesn't tell you. You either end up like something in one of these simple living magazines or end up something like Quarters, the TV show. There's a bunch of places in the middle, they're all fine. Everyone's going to fall somewhere differently along that line. So I've managed now that I've thought about this a lot to set up my kitchen in a way that is very comfortable for me, like I know where things are, I can find them really easily, things that I use are at hand. But other people come in, they're just like, "I have no idea where everything is," like it's very personal. The organizational system you end up with [inaudible] that you have is a very personal thing and that's why, if you look at something like staged houses, so you're selling your house, you hire someone to put in rugs and furniture and stuff and make it look like somebody lives there so that people can walk in and sort of imagine themselves in this space, they don't put any of that clutter into the stage. They don't put any books on the coffee table except the big picture books. They don't leave the remote controls on the couch. There's no plunger by the toilet. There's no like -- CHARLES: There's no Legos on the floor. ROBERT: Everything that looks good. SARAH: Everything that makes it more personal, they leave out because it looks like somebody else's mess. You go into something like that and you're like, "This is not my mess. This is somebody else's mess. It can't possibly be my house and I'm not going to buy it. ROBERT: Oh, do we do this for software in conference talks and posts? SARAH: Absolutely, we do. That's sounds very similar when you get someone new onto a project, especially if they're more senior and they'll walk in and be like this, I can't live like this. [Laughter] SARAH: This is somebody else's mess and clearly we need to make some changes. But that's the reason why they leave it out of the staged houses is because you want people to be able to imagine their own level of clutter and disorganization that superimposed on the skeleton. But real life is not that. Real life is somewhere between that and hoarders. There's a very interesting parallel there with code, which is like when we look at code, if we look at the object-oriented design textbooks, you look at conference talks, you look at blog post, sample code, it's all very staged house. It's very uncomplicated. It has no clutter in it and that's because you're supposed to be able to look at that. CHARLES: I mean, that clutter can distract the sales process so to speak. SARAH: Exactly, like they have an idea they're trying to get across and the clutter would distract people from the idea. But the problem there is the same with the staged house which it's very difficult to tell what it will be like once you move in. It's very difficult to take some of these ideas that you see demonstrated in these staged environments and take them and apply them to your code base which is probably closer to a hoarded house than to a staged house especially if it's a code base that existed for a while over time, that has been worked on by lots of different people. This is the problem that I've noticed with a lot like there's some really amazing books about software design that have come out in the last couple of years. Of course, Sandi Metz's book is at the top of my list. But the thing that people have trouble with, like they love the book. They love the book. I love the book. But then they find it very difficult to apply those principles when they sit down in front of a code base that has already been worked on for six or seven years, in some cases by maybe 50 different people, who knows, over time. How do you take those principles and start applying them in a way that moves you in the right direction? That's where people are just like, "I can't do this. I can't do this and I'm not going to do this." And it's very similar to a problem where you've got a very dirty house and you don't know where to start in order to move it towards something from the Simple Living magazines or are more like a staged house, you don't know how to start to get it in that direction and so you just kind of give up. The powerful thing about KonMari is that it doesn't give you like, "Here's what you're going to end up with it," but it gives you a way to get started on something that gives you a very easy question to answer. It moves you in the right direction. It moves your house in the right direction without being overly prescriptive about what you'll end up with. CHARLES: Yeah, what that direction even is. SARAH: What you'll end up with is personal for you, anyway. I think the question about 'do I understand this code' is similarly helpful and that it moves you and your code base in the right direction without necessarily giving you a lot of prescription about how you do it or where it goes or even where it's going to end up. It just gives you a question to ask that it tells you whether or not this code needs to change and a question is, "Do I understand it?" If I don't, it should probably change, and if I do, okay, we can just kind of leave it for now. CHARLES: So now, if you're working on a team where you have two different people, maybe different skill levels, maybe just a different temperament or different set of preferences, what do you do when the answer to that question is two different things for two different people? SARAH: Well, sort of like when you move in with someone. This is the hard part about living with somebody else, is that you have to mutually agree upon a method of keeping your house that is agreeable to both of you. Sometimes, when they say that working through a startup is like being married to someone, there's some elements of that because you basically have to figure out like, "Okay, we're going to live in this code together. If we're going to live in this code together, we better both be happy with it. How can we both be happy with it?" It involves usually, some compromise, like I really hate doing the dishes but I don't mind cooking and vice versa. You have to figure out. It really bothers me when there's socks on the floor but I don't care if you leave dirty dishes in the sink or whatever it is. You just have to have these conversations about, "What is going to make the code livable for you?" You basically want to end up with a code base that's understandable where all parts of it are understandable to everyone on the team. Now that's like an ideal. You're not going to get there. But that's kind of what you're going for. If you have two people in the code and you have disagreements about what is the right way to go, sometimes it can help to just be like, "Hey, I don't really understand this," versus, "I don't think this is the right decision and here's why I don't understand this." Sometimes, reframing the question in that way can prompt them to communicate reasons that they have for doing this that they maybe weren't able to articulate before, for example. Just like when you move in with someone, you need to have sort of this commitment to finding a level of housekeeping that you're both happy with. When you're working on the team, you do have to have sort of a mutual commitment to having a code base that everyone can live in. CHARLES: Right. I like that because having like, "I just don't understand this and here's the reason why," that being a completely totally valid answer because sometimes in a code base, where someone's brand new or maybe they're at a more junior level, they don't quite have the tools to understand it or there's a lot of steps that haven't yet taken. It's like understanding is not going to be accessible to them immediately. SARAH: And maybe that means that's the wrong decision for that code base, is that right? CHARLES: Right. SARAH: Because if something is abstracting to the point that a lot of people on the team don't get it, then it's probably not the right abstraction for that code base. That abstraction might be totally appropriate in a code base in which you've got folks that are more experienced who understand why it's there, who have the scars from previous times when they didn't do it, et cetera, et cetera, and they understand why it's there. There is sort of like intellectual understanding of like, "Yes, object-oriented design is a good thing," and then there's, I would call it almost emotional understanding of like, "Oh, yeah, there's this time that we didn't do that and that worked out badly for us." I think that folks that don't have the sort of experiential understanding, sometimes they just need to have that. They need to get that. Sometimes, what that means is you want to let them see what happens to a certain extent. Let them see what happens when you don't do that. CHARLES: Right. This reminds me actually, I've got three kids and the way our house is now versus the way it was seven years ago is wildly different -- the way that we live. You know, with our first child, I'm ashamed to admit it, like our strategy was just to kind of put safety locks all over everything: every cabinet, on the oven, not on the refrigerator, but just kind of 'childproof' the house so that we wouldn't have to change the way that we lived but it made the house really uncomfortable for our children. And kind of having observed that over the course of having the second and the third, there's not anything that we childproof really. We put the dangerous chemicals way up high, where only we can get them. It's a little bit more inconvenient if we need to access the bleach but that level of discomfort is something that we live with. We've always got cups that are set out on a cabinet that sits below the counter so we've got water cups set out so that the children can get water and stuff anytime that we want, and we try, for things that they're going to need, make sure that it's accessible if you happen to be four feet shorter. That's just a condition of who you are. So it means that the actual configuration of our house, even though it's the same house, is just radically different and it is more optimized or it's optimized as a compromise for the fact that there are people living in this house now that haven't learned how to operate everything but they just need to learn that the oven is hot and you don't go there rather than slapping a lock on it. SARAH: Your house is probably more comfortable for you as a group, right? CHARLES: Yes. SARAH: And what that means is that as the 'senior' in the house, it's slightly less comfortable for you in some ways but it's worth it. It's worth being less comfortable for you in order to increase comfort across the board for everyone in the household. CHARLES: Right, because it means that if the child needs water, I don't have to stop what I'm doing to get a cup out of the cabinet and fill it for them. SARAH: And they feel [inaudible] over the stuff in their house. They feel like they live there, like the house is for them. CHARLES: Yes. ROBERT: That builds comfort and confidence. SARAH: Yeah, I think that's a very good analogy. Anytime you have a group of people living together, everyone makes compromises in order for the house to be set up in the way that's optimized for the group. CHARLES: Yeah. "So man, how are we going to apply this to software? What's the next step? What are the concrete steps?" I guess it's just asking those questions, like asking, "Did I understand it?" SARAH: It is asking those questions and it's also, if you are one of the more experienced folks on the team, it's your job to elicit the answers to that from other people that are less experienced. They're not going to tell you. A lot of times, sometimes, they may or may not feel comfortable saying that they don't understand something. So it's your job to really try and figure out like, "Do they get this at a level that is acceptable? Do they understand why this abstraction is here at an intellectual level or at an experiential level, at an emotional level? Do they get it?" Which is not something you can really just ask. In many cases, it's your job to -- CHARLES: To just observe it. SARAH: To observe and to see how it works. If people are having a hard time understanding where things are in the code base, it could be because everything is so cluttered that you can't see anything or it could be that everything is so hidden that you can't see anything. It's sort of the staged house equivalent where everything is too abstracted, or is it the hoarded house equivalent where everything is just obscure and under piles of junk. Either way, no matter which direction you need to move towards the middle, the question is always, "Do I understand this?" ROBERT: I like this a lot. I keep on coming to the analogy of if you put a chef in a different kitchen where everything is just totally rearranged and they don't know where their knives are, where their measuring cups are and stuff, I think that plays perfectly in a software of like you put somebody into a code base that they don't know, "All right, I'll figure it out." It's not their home. It's not what they're comfortable with or used to. Yeah, I think this is making my brain work on how I can apply this. SARAH: Or if they're moving in like when you hire somebody and they 'move into your house', you need to be ready for things to change. And this is one of the reasons why I've been saying for many years in ways that I think maybe didn't quite connect as well as they could have, that really the team is the code and vice versa. Every time you add someone to the team or someone leaves the team, teams are not mutable. You get a completely new team. So, it's not like you can just sort of carry on like you did before. Every time you get someone new onto the team, everything gets reimagined, every breakdown of responsibility, every decision. You look at it in a new way when you have someone new come on to the team. If they're going to stay, like in your chef example, if this person is moving in and this is going to be their kitchen and they're sharing it with other people, then what you're going to end up with is probably something in between what it is when they get there and what they had before. They're going to bring in some ideas, you're going to keep some of your ideas and you're going to end up with something in the middle. The same thing has to happen with your code when you bring someone new onto the team. CHARLES: I really like the way that this just focuses the discussion and I know that you've talked about this a lot before, whether it's a kitchen or a house, this idea of the code not being so much the shrink-wrapped product. It's a structure, yes. It is definitely that but it's a structure that you, as people, inhabit. It protects you from certain things and it provides you certain things that you need to live. When people ask us why is a continuous delivery pipeline so important in automating all these things for deploying your software it's because the idea is this is going to be a living thing that your team will actually be living in. And every member of that team will be living in from the time they start with the company or start with the project until the time that they exit and the time that they leave. It's the actual living process that you want to make comfortable and pleasant. SARAH: And what comfortable and pleasant means will be different depending on who's on that team? It's not something that you can have like a -- CHARLES: It's not. SARAH: Right. This is why all of these things are like, "Here's how you design things." It always seemed to fall flat. I think it would be better titled like, "Here's how I did one thing once." [Laughter] SARAH: Or, "Here's what works for me." I feel like every conference talk that is about design could be, "Here's what works for me. I did this one thing once." CHARLES: You might want to try it. SARAH: You could try it. It might work for you, it might not, right? CHARLES: Right. SARAH: A lot of times where conference talks fall flat and blog post and everything else was why they're more like, "This is how you do it. This is the right way to do it." You're like, "Well, it certainly works for you." [Laughter] ROBERT: The one time I gave a conference talk, the night before I went through every slide and scrutinized it as much as I thought somebody out in the public would do it. And I think that might be where we go through in a 'stage our code'. It's like we're trying to make it perfect for somebody that might come through and scrutinize it or criticize. Because I know when I was going up to put those slides up, I wanted to make sure it was the best foot I could possibly put forward. CHARLES: Right, we don't want to be wrong but I think that's where it actually, thinking of it as 'this is what worked for me' and this is an example from my house that worked. This is a way that I organize my code and my space. That'll not take a lot of pressure off of not having like, "I am right and I'm an authority at saying that this is the right way." That's a lot of pressure. SARAH: I don't even like that. I try and frame a lot of the things that I talk about as like, "Here's the thing that works for me really well. Maybe it'll work for you too. Let me know." CHARLES: Yeah. ROBERT: Yeah, that's how I give it. CHARLES: Up until really about two years ago, I felt like that was the expectation that was put on people is to say the right thing. SARAH: That's true. And I think that there's a lot of teams where that is an unspoken requirement and that's something that we should examine. Because even within a team like 'here's a thing that works for me or here's a thing that worked on my last project' isn't very different from saying something like, "Well, industry best practice --" [Laughter] SARAH: And I think that like you get to a certain level of experience and people expect you to say things like that. In my experience, the best way to do it is 'blah'. I mean, it's not actually a super useful statement because your past experience may or may not be directly applicable to the thing you're looking at right now, no matter how experienced you are. I think that it's much more friendly to have a range of experience levels to say things like, "Well, this worked for me on this project. Let's talk about whether it could work here." CHARLES: Right, yeah. ROBERT: I really like that. CHARLES: I do. It's so hard because your human nature is to try and boil things down into a simple binary. SARAH: People would love to have a list of rules, I'll tell you that. This is a problem. This is one of the reasons why I think it's important for us to come up with these questions that you can ask that will move you in the right direction without giving you rules, that will move you in the direction of finding the rules that work for you. Because rules themselves, people really, really, really want them. But they're always misused. They're always misunderstood and what you really need are the questions that led you to those rules in the first place. That's what people really want, although maybe that's not what they are asked. ROBERT: Ah, the Steve Jobs approach. SARAH: [Inaudible] to start wearing black turtlenecks. I hate turtlenecks. ROBERT: And New Balance shoes and the jeans. [Laughter] SARAH: But yeah. I think it's one of those things where people are very hungry for guidance. But we've been giving them the wrong kind of guidance. We've been trying to give them rules. When what we really need to do is give them questions to help them develop their own judgment. ROBERT: Right. Like when I was coming up, I thought, in everything, there was a right way to do it and a wrong way to do it. I've been slowly, sadly figuring out that it's not all black and white and it's not all just logic. I've always treated programming as like, "Well, they wrote this and it's just logic so I should be able to understand this." It's been a long road to come to this conclusion that kind of like what you're talking about and this has been enlightening for me. Like you are going to solve your problems your own way, your own person, and you'll think about things differently. I really like the analogy of 'this is your house and this is how you work and live in your house'. SARAH: Right, and no one would tell you in order to be a proper human being, you have to set up your house this way. ROBERT: Exactly. SARAH: We feel comfortable telling people, in order to be a professional developer, you need to set up your code this way. I think that those are very similar statements and we should really examine a lot of these 'should' statements that are all over the place when you're talking about software. Think about whether or not they're actually serving us in our mission of doing more things with tech. Like overall, my mission here is for people to be more effective with code so that we can do more interesting things with it. I live in the TV show, Silicon Valley, essentially so I'm surrounded by these companies that are solving these little tiny problems and I'm tired of it. I want us to solve bigger ones. In order to do that, we need to get better at coding. We need to get better at managing code over time and that's what I'm trying to do. CHARLES: Because it's not going to scale, otherwise. We're out of time. We're going to have to have you on the podcast again because I don't think we've got to... what? About 15% of the things that we want to talk about? SARAH: Oh, we are overtime, aren't we? CHARLES: Yeah. But thank you so much, Sarah, for coming on and talking with everybody. You drop real quick your Twitter handle so that if people want to have follow on discussion, they can reach out to you that way, or your other preferred means of contact. SARAH: Yeah, Twitter is probably the best. My Twitter is @sarahmei, and that's mostly where I am. CHARLES: All right. Well, fantastic. As always, feel free to reach out to us too. I'm @cowboyd on Twitter. And what are you, Rob? ROBERT: @robdel12. CHARLES: All right. It's a wrap. Thank you so much, Sarah, and we'll see you in Ether and hopefully we'll have you on the podcast again sometime.
In this episode, Leah Silber, CEO of Tilde, Inc. and Ember.js core team member talks about what she's learned building communities, organizing events, and running a business. We talk about how people can move from "observer" to "participant" and grow their own healthy communities and companies. Links: Leah Silber: EmberConf 2016: The Morning-After Post-Mortem Event Driven: How to Run Memorable Tech Conferences Transcript: CHARLES: Hello everybody and welcome to the Frontside Podcast, Episode 43. I am Charles Lowell. I'm here with Brandon Hays, and a very, very special guest. Brandon, do you want to introduce her? BRANDON: Yeah, we're here with Leah Silber. She runs Tilde? 'Tild'? I always say this incorrectly. LEAH: You can't be saying it incorrectly. When we named the company, we knew we were choosing one of those names where people are going to say it and you just have to accept it. That's fate and that's how it goes. We usually say 'til-de' here. BRANDON: Okay, I'll say Tilde, and you can say 'Frontsi-de'. CHARLES: The way you say Tilde says more about you, than it does about us. BRANDON: Yeah, it's a verbal Rorschach test. We were really, really glad to have your time. We know that people actually work with you as a consultant for these kinds of things to help with communities, conferences, build their businesses. So, you have a lot of gathered expertise around these things. Would you tell us a little bit about yourself, about your background, what you do and kind of how you got involved in tech and running businesses? LEAH: Sure, not unlike an elevator pitch but I have been working in open source startups and companies, I want to say it's probably been like 10 years now or crazy something like that. But my first open source project that I was seriously involved in was jQuery, and that was a long time ago and it was pretty magical in retrospect because jQuery was, at the time, it was like coming out of nowhere. Nobody thought it was going to really make a dent in technology. John Resig was this clearly brilliant but still this nobody, sort of working on this project in his spare time, and Yehuda Katz jumped in and a bunch of other people earlier at the beginning. It was a time in the ecosystem where they were a little bit laughed at the room. In retrospect, there was a time when the ecosystem was a little more rude, like some of the competitive behaviors that happened back then. Thankfully, it just wouldn't fly right now. But it's been super cool to be involved with something and be able to witness something at the ground floor where this little idea and project that nobody takes seriously because there are these seemingly massive projects and landscape, and then just sort of watch it take over the world. It was a process obviously that took a little while. But again, in retrospect, it didn't take all that long, so that was really an amazing experience to watch. It was also my first really intense open source community learning experience. Everything from witnessing what kind of personalities got involved and how they did it, to watching John who sort of -- I want to say he's a consummate politician, but he's not a political person. I guess what I mean, he's just really good at people. CHARLES: He's like a diplomat? LEAH: He is. But like the sort of diplomat where you're in a battle and then suddenly a treaty happens and you just don't even know what happened but everybody's happy and you vaguely remember that you all hated each other a few minutes ago. He's really talented. Obviously, also having the technical chops to build something impressive helps with that. But watching how different personalities in open source interacted with each other, and even just for myself, like learning how to be a good open source citizen and learning how to contribute to a project and finding a way as a non-coder at the time to be useful in an open source project was really amazing. That was something I was involved with for a number of years. Then, slowly as time went on, I got involved in other projects and other events. And along the way, I was like, "This is really fun. Why am I working not in technology but doing this at night." Well pretty early on, I moved out from New York to California which is, I guess, the rite of passage or at least was. Got a job at my first startup, spent a couple years there, sort of again learning everything in fast forward because that's how startups work. I've done that a couple of different times over the years, thankfully not that many. I've managed to have what I consider impressive stability in a startup land where people can end up needing to change jobs, projects and positions very rapidly. Nowadays, I mostly focus on Ember work. There was a big chunk of time in the middle where I was focused on Ruby on Rails work. I do events, conferences, meet up groups, community management. A lot of the less glamorous stuff involved in once a project does become more successful, like figuring out a governance strategy, and figuring out how you protect your brand and what happens when lawyers and PR people and all these other different industry people start coming at you with all these questions that you hadn't thought about. How much infrastructure is too much infrastructure? What happens when the project starts having money? All these sorts of things. I feel like I've lived through a bunch of projects and their growing pains, and have a really solid understanding of the different routes. I'm still learning every day, and that's kind of why I love it. I started with my co-founders, I started my own company about five years ago which I'm always pleased and astonished to still be existing. Obviously, I watched companies spin up and die down overnight in the fast-paced technology sector. So, I'm a fan of stability and continuing to exist, is basically the top of my list. But that was about five years ago now, and it's been really great. I would never previously have identified myself as an entrepreneur. I had this, I want to say now, misconception that I was a support person, that I was the perfect second-in-command that I needed somebody at the top of the food chain who had these brilliant ideas and then I would be the person who would come in and say, "Great idea. Let me make it happen for you," and like operations and execution. At some point, I realized that that's not real. There's no reason that the person who has ideas has to be more in-charge than the person who makes ideas happened, like these two skill sets, if they're not in the same person are equally necessary. I think that was probably a little bit of standard sort of impostor syndrome kind of stuff. And also, there's a lot of pressure involved in thinking about yourself as in-charge of something important with high stakes. But I don't know. At some point, I think I watched enough people do the job and I served as that second-in-command or upper management kind of role for a lot of people. I realized that primarily, the difference between the people who were running the show as figureheads and the people who were actually running the show day to day, the difference primarily was just boldness. Like one of them had the audacity to say, "I can be in charge. I'm going to start a company. I'm going to do this." And that's not actually that big of delta so 'fake it until you make it'. BRANDON: I kind of want to lock in on that concept a little bit. I don't want to let that just float by on the river. That is something that has been such a profound lesson in my life over the last six months or a year that I think, a lot of us that wind up running companies kind of fall into that by accident or happenstance or something. You always have this weird left over hope from times where you work for other people, that somebody will step in and be in charge. It's a deal where everybody stands on the line and like, "Okay, whoever wants to step forward, step forward, and everybody steps back but you." And that feeling of being the last person standing when everybody else has backed, just by nature of not stepping back, you realized, "Oh, you know what --” Like there is such a thing as an operationally oriented CEO. So it really is the idea that you just said, it really is a matter of boldness and being willing to be the person, where the buck stops, is really the only difference between a person that feels like a really great second-in-command versus the person that feels like they could be running things. LEAH: There's just this magical myth of the big important idea person. Anything that's going to be successful or most things are going to be successful I guess, they're rarely just one person sitting on a mountain. One person starts with a shred of an idea, and then everybody around them sort of helps turn it into something real. So it could really be anybody who has that first instinct that it doesn't mean that that person has to be in-charge. CHARLES: I'm wondering, if there's any parallels there between, "Have you borrowed any of that boldness through community?" Or has there been any bouncing back and forth about lesson in terms of somebody has to take the lead on something. LEAH: Actually, it's harder a little bit in community stuff because taking a lead typically means or we think it typically means making a decision. I think that's where in open source, a lot of people go wrong, and a lot of open source projects end up with a top down management strategy where somebody is in-charge and that person tells them what's going to happen and then everybody, for example, freaks out about backwards compatibility. Then they're like, "Oh, yeah. I got a plan for this." But part of that is like you see a power vacuum and you think like, "Okay, I can step up and take this." But in open source, that's not really the ideal way or at least not in the philosophy of most of the projects that I've been in which is you only need to ask people what they think along the way, differently than in a professional environment. Like in Ember, we have the RFP process where we source a feedback before we make massive changes. That just makes everything, like you can even really say, "This is the exact thing I want to do," and you can lay out a really great plan and take that leadership role. But in a framework where it's not an edict, in a framework where it's like, "Okay, now everybody else, what do you think about it? How can you improve my idea?" And there's a whole bunch of things that happen. The first and most obvious is your idea gets better because people point out things that you didn't think of or don't necessarily personally have the experience or have noticed. But also, people just feel consulted in a way that is more critical. I like to think people want to feel consulted in work environments also. But I guess when you're the boss, you can get away with just saying, "This is the policy. I made the policy." In open source, people won't really let that fly. You can't just say, "This is a feature set. I've made this. That's the rule." Certainly not if you want them to use your project and contribute to your project and help you be successful. There are some similar things to think about when running a company versus running an open source project. But essentially, the project has to be a significantly more collaborative environment that makes people feel invested in the project and want to stick around and want to become other contributors so that the project can grow, succeed, and have a lot of people involved rather than just one-idea person. BRANDON: I was listening to a podcast recently that I can't remember which one it was but the people were talking about a different tech community and their definition of community really surprised me and it made me realize that people have different definitions for what a community is. LEAH: Wildly. BRANDON: Yeah, and so I'm curious about what your definition, in terms of an open source community, what it is and what it's job is for that open source project? LEAH: I don't have a dictionary definition and in a lot of cases, it's kind of a feel. Like I like to talk about sometimes how the different communities I've been involved with had a different feel. In an indirect fashion, a lot of what your community is comes from the people who are theoretically in charge, be that your core team, or your benevolent dictator, or whoever sort of the thought leader. That person or people really influence the kind of community that you create with their behavior. For example, the kind of community where the person in charge just tells you what the project is going to do next and does it, has a very different feel where the person in charge says, "Here are my thoughts. What do you guys think about it? How do you want us to do this? Do you have suggestions? Did I miss anything?" I think if there's enough premeditation and consideration that goes into the decisions that that person makes, that he or she can really shape a positive community, a collaborative community, and a supportive community. In Ember, we have managed to collect a group of amazing people who want to help each other, want to support each other, and who are enthusiastic about what's happening, and on most good days who don't freak out when they get a little worried that something isn't happening the way they want it to because they can trust that they're going to have a period of input and their needs are going to at least be considered. You don't always get the exact thing that you want. But it's a lot better if you know that your concerns were heard, evaluated, and maybe there's some other plan or way to sort of not completely screw you, basically. Ember's been really good at taking care of people and their needs even as the user base is more needy in different changing ways. I guess a community is a living breathing thing. For sure, it changes. I'm oftentimes sort of paying attention to the undercurrent of what happened, what's going to be the outcome of this? Having chats with people, especially people in theoretical leadership roles, about different ways to handle different situations that will keep as many people as possible, happy and supported. BRANDON: As you were describing that, to me, it feels like you've highlighted something interesting about communities, which is, you can use a theme and not be a part of its community. Somebody could use Ember and not choose to be a part of the Ember community. But participating in a community is kind of the desire to influence that thing in some way. Like, when you say they want to have their needs represented and they want to be a part of the RFC process, there is some part of it where I guess a lot of it has to do with just any kind of connection with other people who do the same thing or probably do the textbook one. But I also feel for a lot of people, there's a desire to be able to have their needs represented and met and feel like they are somehow a part of the direction of this thing, as well. LEAH: Yeah. It totally varies based on your personality. Some people just want to feel like they are a part of it. Even if they feel like their interests are well represented, you see people all the time looking for small ways to contribute because it's fun. It's exciting. There's progress happening, there's success, it's something that isn't like a lot of other opportunities that you have especially if you've been in some other industry, or some other kind of job. You don't always have this thing where you can sort of be part of an organism and a community and watch something evolve and maybe even have input in it, or just have 40 friends around the world who want to chat with you about it at any given time. It's fun. BRANDON: A question that I have in terms of following up on that is your role on the Ember side of things, I'm assuming and I want you to clarify if I'm not hitting it properly. My understanding of your role in the Ember ecosystem, in addition to handling a lot of the unglamorous logistical components is to help kind of grow and foster that style of community and Ember's become pretty well-known for having a unique focus on quality of community. I'm wondering if that's intentional or if there are things that you do or if it's sort of been luck. I don't know how much intentionality has gone into that or how much the community design has gone into that. LEAH: For sure, it's mixed. There's a lot of things where select events are really good example of setting the tone. And there's like an evolution of events that I like to follow in some of my newer growing communities that I'm focusing on where you start out with a little more of a campy feel, it's a little scrappy. And slowly, you iterate to get into a much more professional feel. But all along the way at those stages, having that event, the logistically top quality really sort of changes the tone of everything. If you show up to an event and it sort of haphazard and no one knows what's happening, you might all love each other and you probably will have a good time. But it's a different experience than one where it's sort of run like a well-oiled machine where you get a sense of people take this seriously. This is real. This is impressive. We're building big, amazing things together. We can accomplish together. So some of it is just on all the little things. Event is just one sort of example. But all the little things that go into the well-rounded ecosystem, I try and focus on quality so that obviously, there's a whole bunch of people focusing on the quality of the code. But I also want to focus on the quality of the events and the website and helping meet ups run quality events around the world and helping people show off that they use Ember and are proud. Any sort of these peripheral things -- the better you execute them, the more of an overall, "Wow! This is real. This is serious. I can stake my professional future on this." The more of that kind of a feeling that you're going to get. Community growth is organic in a lot of ways, obviously. But there are certainly things that you can do along the way to help foster the community growth. There's like personality things like making sure everyone's actually welcoming, and that people want to come and get to know you and work with you and get involved in your technology. There's things like the tactical processes of our RFC. Making sure there are ways for people technically to get involved. There's things like a focus on documentation, which again just makes it easy. So, it's really an overall quality thing every step along the way, and a lot of community overlook the parts of building community that aren't code. You can do that but you end up on a different trajectory than a project where you pay attention to all the peripheral things. CHARLES: So, I'm actually curious because I've witnessed the things that you've done like on a grand scale which definitely have had that air of quality that you're talking about. But I'm curious about these kind of nascent communities that you're talking about and kind of just, one, just curious about what they are because I'm curious. Then the second is, for people who might be speaking of doing something similarly, like starting something small that they want to grow into something huge, but actually that concrete, small scope. What are the things you can do with limited resources, if you have limited resources and you've got a small scope, what can you do to imbue it with that sense of quality that will carry you to higher places? LEAH: I guess the first thing to think about, and I hope this does not sound bad, but is whether or not your project actually needs a community. By that I mean I certainly think open source projects should all be actual open source projects. So you should accept pull requests, you should let people file issues, you should have collaborators, etcetera. But there have been a lot of projects along the way that I've been involved with helping with where we looked at it and said this doesn't need to be a community. This doesn't need to have a conference every year, or it'll have some sort of community right just amongst people who contribute but it doesn't need to be like Ember, like Rails. We're a whole giant ecosystem that spins up. For example, over the years, there have been projects like Handlebars, Bandler, and Thor. These are projects that tremendous numbers of people use. You don't run into anyone who says like, "I'm a member of the Thor community." And that's perfectly fine, right? There's sort of a version of a project where you have an MVP, you have a good website, you have good docs, you have a bunch of contributors, and that's all you really need. Then there's the version where you want it to be a much bigger, more involved setup. So one community that, I would say, in a nascent stage right now is the Rust community, which I've been peripherally involved with. I'm not on the team. I'm not doing significant community masterminding but I have been working with some people in the project to do agree with me on the value of this quality. And so, I've helped them. We just ran Rust Conf this last weekend which was their first conference. It was 250 people in Portland. It was really, really fantastic. I've worked with them on, for example, making quality swag over the years and trying to figure out what level of control over their brand. It's not too much but still protects the brand enough, things like that. They've also modeled some of their governance kind of stuff after the Ember community which makes sense. Yehuda is involved in both projects and he's a big proponent of a lot of that stuff. But it's been really cool to watch. Like first Rust was saying, "Oh, this how Ember does it. Let's crib some of that stuff." Now, in a lot of cases, Ember is saying, "Oh, wow! Look how Rust is doing that. Let's take that back." It's been a very good symbiotic back and forth relationship. But it really just does take the people who are leading intellectually to decide that they care about quality, to decide that they care about a collaborative community environment, to decide that they care about diversity, to decide that they care about all of these little things along the way. The earlier people recognize that these are things you need to care about, the better job that you can do. You can always sort of ride the ship most of the time. But if you start out on the right path to begin with, you're going to be able to accomplish so much more because you don't have that much course correcting to do. There is obviously, also always course correcting in Ember, in Rust, in Rails, everywhere, where somebody not speaking about code but somebody takes a misstep and the whole community sort of has to figure out like, "Okay, this is not the way we want. This is the kind of interaction to go. How are we going to fix this?" Or, "This is not the way we want. There's kind of major technical decisions to be made. How are we going to fix this?" It's an evolution. It's a collaboration. I'm absolutely a fan of the core team entity and of that core team being a medium sized group. Not tiny but a medium sized group of people who bring really, really different things to the table in terms of who they are, their backgrounds and their skills. For example, this serves me well but I am a fan of core teams that have non-coders on the core team. There's a lot of stuff that can get done for a project that doesn't involve writing a line of code. Now, obviously you want to have somebody around who understands open source and the strategy. It is in fact challenging to find people who don't have to be coders but also appreciate all this other stuff and want to be involved in it. But when you can find people like that, that's the really magical key to this more well-rounded community. I like to say, engineers are basically superheroes. They can sort of think of something and then create it and that is the power that most other kinds of people don't have. I mean, maybe contractors and welders, but if you are working at a desk job somewhere, there's not really much that you can conceive of where you have an idea, you write it down, you spend a couple months then it exists, it works, it actively changes your life. One of the downsides of that amazing superpower is that engineers can oftentimes get into a position where they think anything is possible and they don't recognize that just because they can figure out how to do something doesn't necessarily mean that they are the best people to do that thing. That comes into play a lot with these other qualitative things in an ecosystem. So, you can go to a lot of technical conferences and wonder, sort of, why is the quality not quite there? In a lot of cases, the answer is because the person in charge is not particularly skilled in this area. They are coder. They can write brilliant code but do they know how to think about where the lunch line is going to queue up and make sure that it's going to go through the phase, and that there's enough bathrooms and stuff like that. These are very vastly different skill sets. One of the past successes there is to sort of realize, "Yes, I can probably pull this off." But if I can find somebody for whom this is a natural area of expertise and I can focus on my area of expertise. Like, wow. We'll be able to accomplish so much more and everything will be better all around. BRANDON: I think you've hit on something there again that I've seen since moving into technology from -- I come from a non-technology background and there is a sense outside of this industry that people kind of have different areas of expertise. When you are an engineer, your job -- I actually think a lot of it stems from the fact that your job is to become an expert in the field of other people's jobs. So, your job is to automate something so you have to know enough about marketing to do marketing software. Then you have to know about enough about this other thing to do this other thing. So you think that you can learn anything and it's true that you can learn anything but there's a skill tree associated with each of those things. You don't realize, you're a junior level conference organizer, and maybe it might be worth talking to a senior level conference organizer. We've definitely fallen victim to that many times where we thought, "Oh, you know what? I'm pretty good at the technology side. I can give a conference talk. I must be probably pretty good at education." And it turns out that education is a multi-thousand year old skill tree that is pretty well-defined. LEAH: One of the hard parts in thinking about this for me is, I really like the MVP concept that I have taken from technology, which is you don't have to get it right. Get something out there. Figure out what the bare minimum version of whatever it is you're trying to accomplish is. Ship it, iterate. I like that and I talk about a lot of things in my life in that frame which is kind of weird when I talk to my parents about iterating on things and what not. For example, in a decision about child rearing. "Oh, my God. The first one is this way." It's a very useful way of thinking about things and I am a fan in many things of actually applying it to areas all over your life. But for something like how to run a small conference, you oftentimes don't need an MVP. You don't need to go through the stage of that level of quality because you can just work with somebody who's already learned all those lessons and already done those things. It's not like a greenfield code project where somebody has to actually start from the baseline every time it's a new project and build up all the infrastructure. You can just skip right to the front of the line. Find someone who's good at this and start out initially at a much higher level of quality. BRANDON: I want to kind of dig into that a little bit and ask about you wrote a really great blog post earlier this year about your experiences running Ember Conf. It's so obvious the amount of effort and kind of thought, not just effort, but like directed effort and thought that goes into building a really great experience for people. I'm wondering if you have like certain areas that you look for and noticing whether somebody has really put a lot of thought into designing a good conference experience for somebody. If somebody were hoping to do something like that, what are some of the areas that tell you that you're dealing with like a really good, thoughtfully designed conference? Or where does that effort go basically when you run through the process? You only have so much time and effort you can put into this thing to create that experience, not the minimum viable conference, but the conference when you look at the division of your time and doing conference stuff. I think people wind up being surprised how much goes toward one thing or another. LEAH: I'm not entirely sure how to answer that question because you don't usually get insights into what people are doing along the way. I go to fewer conferences these days because I find myself irritated. I don't even like the way it sounds but it's kind of true, where I go to a conference and I'm really trying to focus on the content and I'm really trying to focus on the goodwill of the people who are organizing it. Oftentimes, it's just so many moments where I'm like, "That should be better quality." Like it is very easy to get that right. You just need to have thought about that. You just need to plan for that. I don't think there's many opportunities to sort of figure out those things ahead of time from somebody else to make an assessment of how a show is going to go. I can say that there are sometimes things that I see on just the websites where I'm like, "Oh, that is not a good sign." There are tools around for most of the things that you would want to do like selling your tickets and collecting information, and the online pre- event functionality. It's rarely a good sign when you see someone build their own. It's sort of another engineer foible kind of thing. You don't need to reinvent the wheel. It takes substantial energy to reinvent the wheel. If you go with something that already exists, you can focus for example, on all this other quality stuff. But there's also sort of the way in which people build the content of their conference, like that's super telling. Maybe not even so much about logistics but it is pretty telling about the community or at least the people running a show as a representation of the community. For example, I think it's pretty much conventional wisdom these days that you want to have blind call for papers. It's not reasonable to me when I see someone not do it that way. I understand that small events maybe just go invitation in the first year or two. I think that's actually a pretty justifiable thing. We don't have the critical mass yet to get enough submissions in for a program, or we're not really sure how to execute on that, and our community has these specific people who we know are really talented and will do a good job. I think you end up with usually a subpar roster that way because you don't have the variety of experience that you'd be able to attract in a call for papers. But what's super weird to me is, yes, we're going to do a call for papers but we're not going to go the classic extras steps that open source and technology as a whole has learned that will really help this task go better and help eliminate latent biases and things like that. That's kind of weird. I still see it sometimes. I find myself confused by it. I suspect more than anything that people just haven't thought of it or don't know how to execute on the strategy that a lot of other conferences has figured out a successful. I don't know, part of the reason I don't like to go to conferences so much is I remember when I was making all these mistakes, and I don't want to be that person who's really critical of someone who's trying to do something good but it's hard. Once you have that experience and once you sort of know better, it's hard to watch somebody else stumble over the same things you stumbled over. You feel like, "You should know that. I know that." CHARLES: So, is there a community of conference organizers? It seems like they need to be some sort of meta conference or some sort of meta community of community organizers. BRANDON: There is Conf Conf. CHARLES: Are you serious? LEAH: I might not be impressive enough to go to something called Conf Conf. I don't know. A lot of those things actually when you get to, "Oh, it's industry, upper-tier collaborator events." They're often times invitation kind of things and I'm not very cool. I don't know. I see things like that sometimes. There's this conference about speakers. It's like speakers conf – CHARLES: Oh, speakers conf, yeah, and it's like in Aruba. LEAH: It seems cool but I'm always like, "How do you get invited to that?" CHARLES: Yeah, that's the part -- BRANDON: How do you get into that club? LEAH: I know people who should be there and don't know it exists, or people who should be there but just - I don't know, no one has invited them, and it's invitational. It's hard to say. But I guess what I was going to say is over the years, I have at times belonged to different online forums that were trying to scratch that itch. The most successful one was probably for a good five, six years. There was a very active community of conference organizers in the Ruby space. But I think most of those people have either stopped doing conferences or gotten to a place where they just don't need that much help anymore, and there hasn't been like an influx of new people doing it. So there isn't really any strong community like that that I belong to today. That's a challenge in anything you face, where once you don't need the help anymore, people don't stick around to help the other people and it's hard to sort of organize that way. BRANDON: That sounds like a big opportunity to me. But it sounds like an opening or a hole for providing something like that. Do you do any consulting for people on this? Do people hire you to help design a conference experience? Because I know that the ones you've put together -- you're pretty well-known for being one of the best in the business for this. LEAH: Thank you, I think. No pressure at all. I do consulting, though obviously, that's mostly for companies and not communities because communities don't typically have money for that sort of thing. In communities, you need to create your own experts, and in an ideal world, there are people who recognize that when they're at the beginning of the learning curve, they should reach out to people who are further along and benefit from their experience and do things like read blog post and books and what not. The blog post that you mentioned earlier, it was absurdly long. I was surprised that anybody read it and then a lot of people read it. I was surprised because of the length but I was also surprised in the way that it's difficult to come up with a conference talk where you're sort of like, "I know all this stuff," but I don't know what amongst this knowledge base is interesting to other people, and I don't know what other people don't know and I know the stuff so obviously, it's not impressive. But I sort of forced myself to write it anyway because it was just such a big endeavor and there were thoughts even for myself that I wanted to preserve for later. Then I was pleasantly surprised by how many people read it and had comments and had useful things to say, or even just like, "I appreciate how much thought went into XYZ. I wouldn't have thought that. It was value that came out of that blog post that I didn't think of at the beginning. The value of other people getting a chance to recognize what kind of thought and planning needs to go into having something like that execute flawlessly, or dealing with things when they don't execute flawlessly. BRANDON: One thing that that was an obvious expenditure of effort, because you started so early in the process, many months before the conference was the Women Helping Women Initiative. I saw that really early on. I actually don't think that was precedented in conference organizing. Can you tell us a little bit about that? I don't remember how far in advance it was, I just remember thinking it was absurdly far in advance and very well thought out and very well designed. LEAH: Well, it was early. For example, I actually have to email the Women Helping Women group today with a long list of thoughts and activities for next year. I feel like I'm super behind because it's September 15th and the conference is at the end of March. Last year, I think we were already talking basically like 2, 3 weeks after the previous year's conference had ended. So basically, I looked at the conference in 2015 and it was great. I was really happy with most areas of it but I was not happy with the representation of women. There's so many groups that in fact, the conference would benefit from that representation of, but women was obviously a target that I thought I could potentially bring something to the table on being a woman in technology and in Ember. I spent some time thinking about all the various women in tech efforts. I have been involved over the years in a lot of things that felt good and said good things but at the end of the day didn't seem to accomplish very much. Right from the beginning, I wanted it to be a tactical effort. I wanted it to be like a short term pipeline of, "We put this in one end and we get this out the other end, and we have accomplished XYZ." Because it's important to have lots of organizations that make people who are under-represented in any community feel good and feel welcome. But it's also just as important and a little bit less prominent, in most cases, to have those same people then become leaders in that community. That' sreally sort of a signal to everybody else that they're welcome and that they can accomplish things. I looked at our community and I thought there are already, actually, really awesome women here. Not as many as I'd like. But I know a lot of women who are impressive and who are like, "Why aren't they on stage?" So, I sort of approached the effort last year from the perspective of there's a lot of organizations out there working on the women in technology pipeline problem, and it's a very real problem. Hopefully, they're going to do a solid job. There's a lot of insights and I'm watching and it's cool. But I want to focus on the problem of -- let's not call it a problem -- but I want to focus on the situation of the people who are already here and helping them take those leadership roles and step up and really join the community at every level. Not just at the entry level. We've come in through the fix the pipeline problem setup. CHARLES: And then hope everything kind of magically works itself out from there. LEAH: Yeah, which shockingly, it doesn't. I don't know... We spent many, many months basically supporting people at every step along the way from, "I want to go to this conference," to, "I'm a speaker at this conference." And we did things like brainstorming about what kind of proposals people could submit. We helped each other once we had a critical mass of a bunch of talented women. We helped each other with our proposals. We helped each other with our ideas. I organized podcasts where the program committee did question and answer sessions with the women. We did some hangouts where women who had more experience than the rest of us came and talked about their first time speaking or their experience getting to know a community or their experience learning to code and we encourage each other in a lot of ways. It's just a really positively pushy support group. We encourage each other. It's funny because it was hard along the way sometimes because you don't want to be too pushy. I was always worried and sort of dancing on that line of, "I want to repeat to you, you can clearly do this. You have said things. You have accomplished things, look at your education, look at your career. You are obviously, obviously good enough and impressive enough to be on the stage and I want to keep reminding you of that so you do it. But I also don't want you to feel like you are being bullied into it." Not everybody wants to be a leader. Not everybody wants to be in a role of getting up on stage and giving a talk. I had a couple of the participants come talk to me or email me after the program saying, "That was really helpful. That was the nudge that I needed." So that made me feel good about the various interactions along the way and it's always just going to be something that you have to do in a very sensitive manner. But there are women that I knew that were here that were talented and then so many women that just came out of the woodwork where I was sort of like, "I know your name or I didn't know you at all," and like, "Why don't I know your name? You're incredible. You're better than half the people I see on stage at a conference." And so we worked on this for months and months and months. I got a bunch of women in the community who were in sort of transitional leadership roles which is I thought, "Hey, you're a community leader," but they weren't sure they were a community leader yet. I got a lot of people like that involved to help nudge along all the other people who could potentially be our next batch of community leaders. I don't know... It was really amazing and you have to pad the numbers every step along the way and at the end of the day, you actually want a conference with impressive representation of anyone. So you have to get significantly more than you need into the pipeline thinking about it, and then significantly more than you need actually submitting and then hopefully at the end of the tunnel when everything shakes out, you'll get a reasonable number of people right. Because obviously things get filtered out and at every step of the way. There's a lot of other people that you're competing with. Overall, the quality of the proposals from the women was astonishing. It was a blind call for papers process in terms of the program committee. But as the administrator, who doesn't get a vote in these things, I was managing the app that handle the voting during the process of picking our speaker and all that kind of stuff. I was just astonished by watching quietly in the background how all these ratings came in and nobody knew they were doing this but the proposals from the women were on average, like dramatically better than everybody else. I'm not like saying, "Oh, yay! We're better." I'm sure a lot of that had to do with the amount of thought that went into it because even a seasoned conference submitter who spoke in a lot of conferences might get to a point where they're sort of cranking out a proposal in a week or even a month, and a lot of these women spent six months thinking about it. But all that hard work showed in the proposals, all that hard work showed in the roster, and I really want that hard work to show around the community. Part of what I'm trying to focus on this year is how do we take that from being the EmberConf Women Helping Women program to being the general Ember Women Helping Women program. Because after EmberConf, I felt like I could really help other conferences change their way that their rosters grew. Specifically before that EmberConf, you did have a lot of well-meaning organizer saying, "I want more women represented at my conference but I can't find them." Now, I think there's certainly ways to do it and there's a lot of clever ways to meet people and encourage people. But when push comes to shove, and it's in that moment, you would just run into a lot of organizer saying, "Well, I don't know what to do. It's an open process. Anyone can submit." And now, they could look at our conference and see there's a dozen women who are clearly talented and capable. I'm going to go to all 12 of those people and ask them to submit to my conference. Maybe some of them will and maybe that will help. It's a small step but I started seeing the women who spoke at EmberConf who a lot of them hadn't spoken anywhere beforehand, pop up at conferences all over the place. So, I sort of felt like the effect was magnified and we were able to really... I don't want to say like we, all of us women together, were able to help each other not just at Ember Conf but in a larger way. We're focusing a lot on that this year or I should say, we're going to start on how can we expand the program to be a resource that helps women take leadership roles and speaking opportunities across the JavaScript ecosystem so that Ember is really truly well represented by the people who actually make up the community. BRANDON: I think it's super clear that the thought process that you put into this was well designed enough and the work that went into it was consistent enough that you both got a huge uptick in the number of women represented at the conference. But also actually overall, the quality of conference talks went up year every year as a result of all of that additional preparation. And the encouraging people from backgrounds that had something to say that maybe felt like they didn't have something to say, instead of seeing the same faces over and over again, they went all the way across the board and it ended up showing and being a better quality conference experience for attendees for that. LEAH: It turns out the diversity is actually awesome. It's not just the thing that you should talk about and put on a list because it's politically correct. But in fact, things will be better if there are more people represented from different backgrounds with different experiences, different skills, different everything. You felt it everywhere you went and felt like you were hearing new things, interesting things, and perspectives that maybe weren't always as well represented. My biggest sad point about things is I'm trying to tackle the community of women but there are so many other communities that could also be better represented in Ember and in technology at large. And I would really love for more people to step up and sort of focus. It's hard to get involved in an effort like this because it's hard to get any traction and also because honestly, there's a lot of concern about, "Am I going to do it right? Am I going to make the problem worse?" All those sorts of self-doubt issues that are legit and based on having seen other people out in the world make a good faith effort trying to fix a problem but like say something wrong and get in huge trouble. It's just challenging and it requires a lot of thought. Even me, as I talk about this right now, I'm stressed out and I'm like, "Am I using the right words? Am I going to say the wrong thing?" But it's still worth doing. And if you're persistent and you work hard, you can really, really change things and have a positive impact. BRANDON: Well, I think it sets forward a lot of good patterns for other people trying to do this thing and I think when you talk about this being scary, the point is that it's not going to be scary. It's very scary so it makes it super brave that you're attempting that stuff. I think the impact is certainly on me. I found that pretty inspiring. Certainly at the Frontside, we found it inspiring. I hope it inspires other people in other communities to follow the same patterns. LEAH: Yeah, I hope so. I mean if there are women or anybody else that feels like they have something to contribute to really improving the landscape of who our leadership is and making sure each they are represented, I guess I would encourage those people to step up. It's the same thing as being a CEO or being the second-in-command. You just need to have the boldness to decide that you're going to try and make a difference. In the Women Helping Women program specifically, I've been talking to a lot of the women that I met last year about how can you take more of a leadership role in the program? For example, I was talking a week ago to somebody who's a student in the Women Helping Women program and I was like, "I can't represent your concerns as well as you can." I was in college at some point but that was a very long time ago, and there are just perspectives that you're going to have that I'm not going to have. Even within the women's group, there's like different perspectives and I've been working with people and encouraging them to try and work with me on taking leadership roles within the program itself. So many well-meaning programs like this, by the way, like spin up, have some success and then go away because the person-in-charge got busy and that's the end of that. It's similar to a community in that if it's going to be a long term successful effort, or at least as long term as it's necessary, I'm really going to need other people in the community to step up and find ways to be involved so that I'm not actually important to the success of this anymore. Instead, it's a whole group of people and there's no one single point of failure. BRANDON: When that happens, I can't wait to read that particular blog post. When you're able to kind of demonstrate the process for other people to be able to follow and, "Hey, look at this thing. It kind of runs itself now." That sounds really awesome and you're definitely -- LEAH: Fingers crossed. BRANDON: Yeah, you're kind of off road because you're helping to find some of the things that may influence other communities, as well. So, I think that's super cool. LEAH: I hope so. BRANDON: All right. We need to get let you get back to CEO-ing. You have a company to run and you have a whole life and everything. LEAH: I'm going to put on my business jacket as soon as I hang up on you and get back to work. BRANDON: You're in Portland. It would be a like a business hoodie probably, right? LEAH: Sort of. I have this really nice blazer. It literally changes my mind some days. I'm like, "Oh, I'm not getting enough done." I'm going to put out my business blazer. Then, they shift into over drive. BRANDON: All right. We'll get your business blazer and get to business-ing. That actually sounds like a cool little business life hack. LEAH: Yeah. BRANDON: Well, thank you so much for your time on this stuff. We got about halfway through the questions I want to talk to you about so I hope we get a shot at doing this again. I really love learning from you on this stuff. I've learned a ton from your writing. I've learned a ton from watching you work and I really appreciate all you do for the Ember community, the JavaScript community, all the stuff you've done for the Ruby community. You've had a really big impact, and so it's really great to talk to somebody that kind of sets a pattern of how these things are possible. I hope that good fortune continues to follow you as you're bold in doing these things. So, thank you for all the stuff that you do. LEAH: Thank you. I hope so, as well. It's been fun. CHARLES: Thank you. All right, goodbye everybody. BRANDON: So, bye everybody. If you have any questions or anybody else that you'd like us to talk to, let us know. If you have any additional questions for Leah, hit us up on Twitter at the Frontside. CHARLES: We usually ask for people's contact information. Where can youl be found on Twitter? LEAH: I do have a Twitter account. I'm not super active because I'm busy wearing my business blazer. But my handle is @Wifelette and I try to be as responsive as I can certainly to people who reach out to me, even if I'm not broadcasting all that much. BRANDON: Awesome. Okay, well thank you again Leah, and thanks Charles. I hope everybody listening has an awesome week.
In this episode we cover how to handle apprenticeship, share with listeners how they can start participating in mentoring and apprenticeship in their companies and communities, and help people to understand the impact apprenticeship and mentoring can have on everybody involved. Links: open-source-ember-apps: A list of open source Ember apps Transcript: CHARLES: Hello everybody and welcome to the Frontside Podcast, Episode 42. I am Charles Lowell and with me is Brandon Hays, as well as some other really, really, really, really special guests, which I'm really excited to have on the show today. BRANDON: Really? CHARLES: Yeah. BRANDON: Just really? CHARLES: Really, really. [Laughter] CHARLES: I was thinking of 'really, truly' but no, I wanted to go back to 'really'. I don't know, Brandon, are you pretty stoked about the show today? BRANDON: I am. I'm really excited today. We've actually wanted to do this for a long time and we finally were able to line it up once we've figured out that we can record a podcast more than once every six weeks. So yeah, we're really -- CHARLES: Really. BRANDON: -- Really excited to have Taras Mankovski and Lennex Zinyando with us. We'll have you each introduce yourselves. But basically, the point of this podcast is we want to talk to you and you're doing some really cool stuff with apprenticeship through us, and Lennex is one of your earlier apprentices. From everything I've seen tremendously successful, Lennex, you're an awesome developer. So we wanted to talk to you, find out how you're doing that stuff but we'd love to have each of you introduce yourselves and talk a little bit about your background. So we'll start with you, Taras. TARAS: I'm really excited to be here. I also feel like I need to point out the fact that this is podcast number 42. Right? That's the meaning of life and everything. BRANDON: We would be very quite remiss to miss that. Thank you so much. We were so really excited that we forgot about this was the purpose of everything. CHARLES: That's right. We're sitting on the main nerve right now. This is it people. This is it. It's everything. TARAS: I'm really hoping that if you ask me questions, I'm going to answer them faster than this famous answer. I'm really glad to be here. I want to give a little bit of my story because I feel like a lot of things that I've doing over the last 10 years have been adding up to what I'm doing now. I have a pretty diverse background. Programming wasn't something I wanted to do because when I was 12, my dad gave me a Java book and he said, learn how to program and I'm like, "This is terrible." [Laughter] TARAS: That was a really rough introduction for me. Even though, I think I was always technically inclined and I did like [inaudible] for a long time. I did a lot of different things. Since I was a little kid, I always want to be a businessman before I think I even knew what that was. So my focus has always been on the business of things, and technology happened to be one of the only ways that I knew how to access that in a way that was actionable based on something that I could do. I've tried a lot of different things. One of the things that I did before was a company called Positive Sum, where we spent about four and a half years trying to figure out how do you build relationships where everybody who works with you wins. When that Positive Sum company and the being as 'zero sum' for me, I end up leaving and starting from scratch. I have another company called WRKTG, Inc which is like the mothership for EmberSherpa and that acronym refers to 'working together'. Everything that I've kind of have been doing has come from the perspective of how do we bring people to work together, and how do I make it possible for people to have a win-win-win scenarios. I think that's kind of what brings us to where we are today, with this conversation about apprenticeship. BRANDON: Cool, thanks for that. Then Lennex, how did you get into software and what was your background? I don't know if you come from a computer science background or you were doing something else. LENNEX: I'm Lennex Zinyando. I'm based in Harare, Zimbabwe. I've always wanted to work with computers since an early age. But then, access to computers was really hard to come by so I used to spend a lot of money going to internet cafes so that I could access the internet. After a few years of [inaudible], I went to work with a bank for a few years and I decided to leave because I really, really wanted to become a programmer developing software. I joined a local company that does mobile marketing as a technical support person. I worked with them for a few years and I met Taras on Twitter, sometime beginning of last year. Then, I paired with him so that he could mentor me. So now, I'm an apprentice at EmberSherpa. BRANDON: You said you met him on Twitter and you said, "Hey, I'd like to be your apprentice," but how did that interaction actually occur? Was there like a tweet that went out and said, "Hey, I'm looking for apprentices," or you just kind of saw that he was knowledgeable and you said, "Hey, can you help me?" How did that interaction first happen? LENNEX: Actually, I saw a conversation between him and another guy where the guy was - actually they're discussing how they could – I think they wanted to pair on it or something. Then, I just asked if I could join the process, be part of the team that was going to pair on it. Then Taras created this link for us and that's how it was started. CHARLES: So you just said, "Hey, can I join that pairing session?" And the rest is history at this point? LENNEX: Yeah, something like that. CHARLES: Now, you're a world famous software developer so it seems it works. [Laughter] BRANDON: Well, that's really cool. I'd like to ask you, Taras. Was that sort of intentional on your part? You said your arc was bending toward community and 'greater than zero sum' type work. As you were bending that arc toward that, did you have the idea of- I mean, your name was EmberSherpa on Twitter so clearly, you had some sort of training or guidance aspect to what you're doing. Was apprenticeship part of that model by then? Or is that something you kind of stumbled on? TARAS: I always imagine what things might look like. I imagine that there might be a scenario- I tried to do this with WordPress before which is create a scenario where I'm helping people learn and in the process they cultivate a community of people that follow what I do through helping them become like an engineer. It didn't work very well with WordPress but it seems to have worked with Ember. It's very much an [inaudible] process. I kind of went through a bunch of hardships trying to figure out a structure that is actually sustainable. That's the really hard part, I think, about creating like win-win scenarios is that you need to be sustainable and finding a way to make sustainable. That was really tricky. CHARLES: Yes, that's something that we're discovering as well, right? You identify these values and you say, "Hey, these are the things that I value," and then you discover how expensive they are in the process of trying to implement them and trying to do that in a way that you can keep going that doesn't basically require everything from you all the time. That's really hard. So it was something that clearly, like you cared about, you were trying to do it in WordPress. You said you ran into a lot of painful lessons there. I'd actually be curious and dig into a couple of those things that you are finding were not sustainable. TARAS: It's ironic actually, I found mentoring to not be very sustainable. From a business perspective, I think mentoring is a really great service but because it relies so much on people like myself who have to be like people that get to certain levels through their hard work, you end up wanting to get paid well for the work that you do. Then in that scenario, it becomes very difficult. Like offering mentoring services is very valuable but it's also a very premium service. So to be able to sustain myself and be able to offer a service, the businesses you want to pay for, and in a way that is going to benefit the company is going to benefit the apprentice and benefit me, those were difficult. But it seems to be shifting towards apprenticeship turns out to be the answer, for me at least. It was like one of things that was right under my nose but it took a really long time for me to see that actually apprenticeship is the answer to creating this scenario. It's really funny how something might be right in front of you but you might not see it for a year and a half. BRANDON: What's different about apprenticeship from mentorship, basically? TARAS: Mentoring for me is supporting a company that has a team of developers. Basically, the company is paying me to mentor their developers. It seems like a very fine distinction but the difference with apprenticeship is that the focus is on the individual and all the other things are kind of secondary. When I stop thinking about myself as a mentor or stop talking about mentoring and I start talking about apprenticeship, it becomes about creating structures for others to be successful like making that shift on creating opportunities for others to grow. It's one of those things where you shift your focus towards something else and it makes your path forward easier. That's kind of what I found with apprenticeship. Even though I was doing it for a while, like I started working with Lennex a year and a half before I made the shift, I really found focusing on the apprentice and making apprentice successful actually help me make my business more sustainable. BRANDON: So basically, by defining the problem scope as, "Hey, this is about an apprentice and not about how good a mentor I am," you basically discovered -- I think all businesses go through this at some point too, like anybody with a purpose ends up discovering that the thing that doesn't scale is your time, and the only way out of that is to develop a process that's focused on achieving an outcome. So apprenticeship is the definition of the process by which somebody gets better and focusing on them rather than non-scalable Taras' time. TARAS: Exactly. BRANDON: Lennex, you basically came on as an apprentice about a year ago. Can you describe the process, from your perspective of like what that apprenticeship has entailed and maybe even how that evolved over time? LENNEX: Initially, we worked on small apps. When I was trying to get the basics of Ember and to be honest, I didn't know a lot of JavaScript. So that process was kind of painful for me. It took a bit of time for Taras and myself to admit that and then I tried to focus a bit on knowing my JavaScript properly. But then after that, things have been going on smoothly. I would pair a lot if I would get a task to do with work on open source projects, and we've done [inaudible], actually contributing [inaudible], learning the whole thing, the whole process. BRANDON: So basically, you have commits on open source projects now out there in the world? LENNEX: Yes, I do. BRANDON: And how long have you've been developing software before you were basically doing that? LENNEX: Not that long. It was just a few HTML sites, a bit of PHP, then Ember is like the first programming thing that I've done. CHARLES: I think that's really interesting. Like part of the process, you seem to be describing as you kind of got thrown into the deep end of the pool. Like this is a new language for you, we're going to be doing open source projects that people actually use, and you were actually contributing working code into a real life. You know, thousands of people are using the code that you actually wrote in very early stages of your development career, which I think is a really interesting model. Taras, would you say that that is part of the model or is that just sort of a coincidence of how this went down? Like is that part of the design of the apprenticeship model? TARAS: It's certainly by design. I think there's something really beautiful about Ember, and it's the fact that it creates a possibility of an ecosystem of open source applications that solve the script problems and that is enabled by the fact that we have this collection of conventions. When I chose Ember as a technology to focus on, I kind of made a guess that this might happen. Then when I realized that Lennex needed to learn by doing real practical things, I went out looking for this applications and started off as being, first it was a HospitalRun, and I was kind of really uncertain whether the HospitalRun would even take it seriously because we're saying, "I'm going to help this person work on your application for free. Will you talk to us and tell us what we need to be able to make this happen and make it so that you can actually mark this code?" But it's proven that this seems to be something that maintainers of open source applications are willing to do, and now we have something like... Oh, I don't know what it is, but plenty applications. I have a list of open source Ember applications that we're adding to on a regular basis, and they're all opportunities for our future apprentices to contribute and improve their skills. Actually, they have something that they can show and say, "Look, I did this." BRANDON: So I'll grab that and put that in the show notes. That seems like something that just producing that list was important to your apprenticeship but actually, everything that you do when you do work on the public like this adds value elsewhere as well. Like that's a really cool and useful list for people to have of things that people can actually contribute to. TARAS: Yeah, and I hope that there are other applications that we don't know about that are going to get added to this list because if we can have more apprentices helping more open source applications, I think that's something that's really great for Ember. It's a win-win for everybody. CHARLES: So I'm curious, you've got this big list, 20, 25 open source applications that's actually a big list and I assume there's a lot of people. You talked about how making the shift towards apprenticeship allowed you to scale but I actually am curious about that process because like how does it scale? There is only one of you so how do you manage more than one apprentice -- two, three? How many is reasonable? How do you bump that number up and get more people coming along? TARAS: I think what's happening now is that my apprentices actually have their own apprentices. I think that's the key to scaling this because at a certain point what happens is, as I mentioned earlier, at one point, I paired with a hundred people and I realized that people have very common problems and depending on what stage they're in in they're learning, they all tend to have very similar problems. The way to scale [inaudible] a lot of people, I think, is to have the apprentice that just learned or apprentices that has a six months of Ember experience, they can support somebody who just started last week or started two months ago. They're actually really good persons to support that person. If I support somebody who just started Ember, it creates a situation where like I want to be able to help but I've answered this question so many times already that it becomes repetitive. It's actually takes away from the energy of me trying to explain this to the person. So having an apprentice support a new person is great for the apprentice because they get to grow. It's great for the person because they get really personal attention. BRANDON: I want to make a note. I feel like I should have noted this earlier. Part of the reason that we wanted to talk to you two about this is we've taken several swings at apprenticeship at Frontside, and we're in a sort of unique position in that. We don't see a lot of companies doing this very actively. I saw a company called Obtiva that had an apprenticeship model that I admired a lot. Then they got acquired by Groupon and everything that was good about that company died. I haven't seen this done a ton and our attempts have been kind of shots in the dark on this. It's been very challenging. Your approach to this seems to be working in a way that I have not seen happen elsewhere. What I'm hoping part of drawing out this conversation is the process of discovering what is working for you and what other people can adopt? Basically define how this is actually like it seems like a pretty successful case and how people might adopt that in their businesses, in their workplaces, and in even in their capacity. I love the concept of moving away from mentorship as a concept and toward apprenticeship where this becomes more of a defined process. I'm sure to you it felt like you're kind of fumbling around as well. But you seem to be landing on a lot of things that really seems to be working. Are you noticing any patterns in terms of somebody you get them working on real world applications, they tend to support each other, what other tools, I guess, or patterns are you using to help that process kind of stick with people and scale because this seems to last longer than just a couple months too, which is important? TARAS: I think one thing that I talk to our clients who - because we actually do sell apprenticeship. Part of the process of the apprentice evolving from someone who has never maybe done programming before or who is very new to Ember to actually being a good productive member of the team, is actually starting off doing something that is very low stress, doing something like open source contributions. Then, being in a situation where they're able to make a difference but there isn't a lot of pressure on them. So one of the ways that I set that up is I tell our clients that this person is an apprentice. The reason why you would want to hire this person is because first of all, if they have a problem, they can ask me. But also, this person, like a lot of things that we do on Ember are very repetitive. Their patterns that once you've taught, you can use that pattern over and over again. I think, at a certain point, if you implemented a few tables, implementing a fourth or a fifth table isn't that different than the first three, and you don't get that much more effective at it because you're still limited by how quickly you can type. So if an apprentice knows how to implement a table, then an apprentice is just as qualified to implement that table in Ember as I am, for example, especially if they have access to the immediate feedback that they need when they get stuck. What I've been doing is first, I've been very, very honest with people that I work with saying like, "This is what's going to happen. This is going to be the person working on your project, and this person is going to do be doing this work." Also kind of talking about the reality of Ember work which there are a lot of things that are really difficult that very smart and experienced Ember developers can solve, but there's a lot of work that we do in large Ember applications, that there's lots of paved paths for that, there's lots of just like assembly kind of the ground work and I think that's a really great opportunity for people who are learning Ember to get the repetition, the kind of muscle memory of understanding how to solve problems. I think the repetition of doing like 'data down, actions up', repetition of doing like working with computer properties, and repetition of writing tests. All these are things that apprentices need to do to become very good at writing Ember applications. BRANDON: That's actually my experience as well. I come from a nontraditional background and I knew that I got a sense for when I was trying to learn a new thing that it was a certain number of projects before I felt like somebody could give me something and I felt comfortable saying, "Hey, I can probably do that." It wasn't my first project and it wasn't like a hundred so have you kind of discovered a number in there of projects that somebody might have worked on when you talk about that repetition. Is there a number of times of repetition that you find people become comfortable with something? LENNEX: On number of projects, for being comfortable in Ember? BRANDON: Yeah, at what point do you start feeling comfortable and a little more confident that somebody could hand you something that you haven't seen before, and say, "Hey, I could probably tackle that." Is there a certain number of projects that you've handle before you felt like you could just jump in on that? LENNEX: Let's say for testing for example, writing an acceptance test. I wrote the initial test for HospitalRun. The next project that I did, I was really comfortable with acceptance testing. That was just one of those thing that I could do. I think just doing it once or twice is enough for you to feel comfortable doing something. I don't know about the other stuff, but then what I've done so far, that has been okay. CHARLES: That's interesting. What I'm hearing is you weren't doing like the whole slice of the application. You were focusing on, "Okay, the acceptance test. That is the thing that I'm going to take on. That's the thing that I'm going to own. I'm actually just not going to worry about everything that lies underneath it, or how to implement the acceptance test but I'm going to learn acceptance testing in an Ember application. Then once I've got that skill, now it's in my pocket and I can take it to the next problem." BRANDON: Lennex, at what point do you start feeling these things start to click together, where you have the tests over here and you have maybe understanding the router over there and you have maybe understanding components over here? Each of these things takes time, it's difficult, and it's a new concept. At what point do you start feeling that you have a full picture of how an application works and that you could kind of attack it from different angles? LENNEX: In my experience for HospitalRun, I did the tests and I did a lot of [inaudible] from Ember 1.X to 2.X. So I got to touch a little bit of everything. The next project in [inaudible], I implemented a feature. That feature has required me to use all the previous knowledge that I gained from HospitalRun to do something that actually [inaudible] and [inaudible] properly. In my next project, I'm now building an [inaudible] from the ground up using previous knowledge from all the other projects. So you're always building on top of what you got before. BRANDON: That actually sounds like an interesting pattern. Again, I don't know if this is sort of something you discovered or happens over time or even if I'm reading it right. But it sounds like you're really mixing up these tasks where some of them are kind of a deep dive on one small piece like, "Hey, I want you to write acceptance tests around this and you're familiar with acceptance testing." And then you have a thin slice of something like an Ember upgrade that touches all aspects of the code base but in a thinner area, as well as a ground up building of something, sounds like maybe later. Is that sort of mix of types of work intentional? Or is that just sort of something that, "Hey, if the work is there and if it seems like somebody can do then I'll grab it"? TARAS: I think some of things are very intentional like the last project that LENNEX is working on, I intentionally said that, "You now have enough experience doing all of them. You've been dealing with other people's problems for a while now. So I think it's time for you to make your own problems so you can actually understand that process a little bit better." But other things that are actually just very practical things like the reason why starting with testing. Why we start with testing is because without tests in place, I don't have guarantees that the things that Lennex is doing are not breaking functionality. Then what that means is if I don't have that guarantee, that means that the chances of me being pulled into something randomly is very high and then that's the kind of risk that I can't take. Actually what happens now is almost every conversation with new clients starts off with, "I know you guys want to do features but before we can do that, we need to build your acceptance tests suite." And so this is actually a very practical thing. I think part of apprenticeship program for us is actually teaching the things. It's a very practical thing. It's like we need to have tests to be able to do this work. BRANDON: Yeah, I mean it's one of those things. There is some debate in the community around like, "Well, how soon do you introduce that to somebody? Is it something that is an advanced concept?" I was not introduced to testing until much later in my development career. Basically, my first introduction to testing was somebody handed me the 'can't back' book called Test Driven Development which is all written in Java before I had learned to program, and I was like, "I don't get it." So that was my last attempt at testing for about two and a half years. It wasn't part of my process but when I did discover it, seemed like it did make everything easier. The fact that you're able to do that up front, I imagine probably saves a lot of time and frustration for people so they have something to like back up to during the course of their learning process. There's always like a place where they knew this thing was working. TARAS: I think there's another pattern that is underneath this which is very important to give people what they need to build to move forward. So with Lennex, when he started working on HospitalRun, I spent almost half a week, maybe almost a week setting up because they never had tests so we had to make it possible for him to write test, so we need to actually read the first acceptance test. What that means is that you need to go in and find all the problems with the app that prevent you from writing acceptance test and actually fix those problems. If I just said to Lennex, "You know, write an acceptance test for this," without giving him any paved path to do that, I think the chance of him succeeding would be very low. BRANDON: So yeah, you basically have to pre-shave all the yaks around getting the first one going. And from that point forward, they're likely to be able to carry the ball further. TARAS: Right. BRANDON: I have a question for you, Lennex, in that sort of vein is how do you know when you're stuck on something that is like well outside of your capabilities versus when you're supposed to stick with something because you know there's something you're supposed to learn? Is that a skill that you've had to develop or is that something that... I don't know, like I'm not sure how to deal with that with our apprenticeship, and I'm wondering how you've dealt with that? LENNEX: One thing that Taras has taught me is if I'm facing a challenge, I should work on it. Taras will say, "We're going for a day. If you fail, then come to me." It helps me to not give up easily. If it's beyond me, then after that day Taras is going to help me. By then, usually, I end up getting through that and finding a solution. BRANDON: Interesting. I have worried that a day was too long of a feedback loop to let somebody bang their head. But it sounds like you've landed on a day as being kind of the approximate amount of time somebody could spend and try to solve something themselves. Taras, is that basically a fair, like wind up being a good place to land? LENNEX: Well, for me and Taras, it will depend on different - a day usually means when you accept. So there isn't that much [inaudible]. CHARLES: But it makes sense when I think about it. I actually wasn't thinking about it in terms of that being an appropriate size feedback loop. But when you think about what you're actually doing when you're programming, a day is actually a very reasonable amount of time to struggle with a problem. So before you say, "Oh, man. I just need to find someone to pair with this on. I've done my Googling or whatever but I need to try and like roll back and look at for some prior [inaudible] on this or try and find connect with somebody who's sharing a similar problem." I think that actually makes a lot of sense, right? That's something that we all encounter in our daily lives. LENNEX: Yeah. TARAS: We know from people with experience how long sometimes it takes for us to understand something. CHARLES: Sometimes years. TARAS: Yeah, and I think it's really tricky. It's hard because learning takes time. That's one of the things that I'm a little... I'm not sure exactly how this is going to work but I know if clients are paying me to mentor their team, I'm going to be available to them within half an hour to an hour to answer that question because if the client can't afford to wait a day for them to get that answer. But for a person who is learning as their primary deliverable, I think a day is reasonable amount of time for a person to try to like even sleep on it. CHARLES: Right. It's true. It's a tough line. I think, Brandon, one time you said something that always stuck with me. You said that learning only happens when you bang your head against the wall so that your brain is soft enough to accept the answer. And you don't achieve learning without the banging your head against the wall first. BRANDON: Yeah, Brandon Hays, the king of violent metaphors. [Laughter] BRANDON: There is actually an important aspect to this that I don't want to drop onto the floor and that is the idea that part of this process is that you have people that have access to one another. That's part of how you are scaling it, that you're also kind of helping. It seems like you're helping build a small community of apprentices to help each other. So Lennex, my question is this, how often do you wind up using that network of people versus you feel like you'll just handle it on your own? How much do you want to have accessing that? LENNEX: I access it a lot. I usually get stuck, and then I know someone working on projects which is similar to what I'm doing. So again, I just easily ping him and then a few minutes or a few hours later, we're pairing on it. BRANDON: That seems like a way where you're able to do that without reaching out too much to Taras. It feels like you're asking for help and figuring it out. But it means that you don't have to roll up to the most experienced person for every single question, that you kind of started to build a network of people that stitching together your own mentorship in effect, I guess. LENNEX: Yeah, that's true. It's easier to understand the concept when someone is on your level or slightly above you when they explain it to you. Maybe for me, it's easier to understand. BRANDON: Actually, that jives with a lot of our experience. I can't remember what book this is from. I'm pretty sure it's stolen from another book but I read it from the book 'Pragmatic Programmer' where they talk about the zone of proximal development, where it's easier to learn from people who have just a bit more experience than you because they haven't completely forgotten how they learned the thing that you're trying to learn. After a while, expertise means that you actually internalize that to the point where you can no longer remember not knowing it. That can be really tough. I think that's actually something that a lot of mentor should probably know and recognize and understand. People that want to take apprenticeship on need to understand that you probably have lost a lot of the empathy. I know I have and it really bums me out because it doesn't feel like that long ago that I was a brand new developer. But I'll sit with somebody who's brand new and I forgot how I learned the thing that I'm sitting there trying to teach somebody, and you start getting that little burning sensation in the back of your head like, "How can you not know this?" You know that at one point you did not know this, but you don't remember not knowing it. It's a very strange sensation to have gone through all of that and then lose that empathy. So I feel like that's must be a really important concept. Another question I have for you Lennex is how much of your time do you wind up spending helping other people do that kind of stuff as well? Do you wind up doing participating as a mentorship stuff at this point? LENNEX: Yes, I do. Actually now, they are about three people that have reached out to me on Twitter, wanting my help. I've paired with them when they have their own projects. They're asking me questions. One guy that I helped with Ember stuff in our local community, and maybe because of the apprenticeship that I'm doing with Taras, they think I'm an expert in Ember. But I'm actually helping other people a lot. BRANDON: I think that's really phenomenal. That sounds like success to me. CHARLES: Yeah, I'm actually curious. Having been through this process and experienced success at it, were there any times where you kind of felt you wanted to throw up your hands and be like, "This is not working." What were those moments of frustration or mistakes that... Not mistakes, but were there any moment's frustration? What actually did you take to course correct on that? If not, that's fine too. BRANDON: Yeah, if not, I'll just know you're lying. [Laughter] TARAS: There were definitely challenges. Lennex and I both went through some difficult learning and questioned whether we're be able to do this. I don't want to speak for Lennex but there are certain things that you have to learn to be able to work with people, and they're not programming things. I think that's because obstacles that Lennex and I had have been related to communication and being reliable for each other, I think that's the part that's been hardest. But the reality, I think, is that this is what programming is, talking to people most of the time and building a network of people. So I think we need to, as a mentor to an apprentice, I need to help my apprentice understand this, and help them understand what makes me successful in my work with our clients, for example. BRANDON: You brought up one thing and I'm curious to get Lennex's perspective on this, which is being reliable for each other. I know it has been probably the single greatest challenge to our apprenticeship platform at Frontside, where we've brought apprentices on and the criticism that we had about our apprenticeship program is if I had 24/7 access to the mentorship that I needed, this process could have been done in three to four months. And instead, it's going to take seven or eight months. So it takes literally twice as long if they have to fight and scrap for every minute of mentorship that they get. I'm wondering, Lennex, if you've experienced something like that and how you've addressed it, if so? LENNEX: Maybe the thing with me is I really wanted to be part of the apprenticeship program. So I had to work really, really hard to make this work. But given that for me, this apprenticeship program has been 100% remote. There's no one standing over my shoulder or someone chasing me, asking me what I've done or whatever. I had to discipline myself so that if I get tasks that I should do, I should do then on time. Even if Taras is not available, he's not there chasing up to me whether I've done that. The other thing is I wasn't good at communicating early on. I would shy away from telling him when things are too complicated for me. I'll try to solve everything on my own until I discovered that some things you just can't learn on your own. You have to ask for help. Since then, things have been a bit better. BRANDON: That sounds like something almost universal to me. Every time we've attempted apprenticeship of any kind, maybe it's because we tend to select the people that we come into contact that say, "Hey, I like to do apprenticeship," tend to be people that are kind of self-starters. But they always tend to try to solve everything on their own. It becomes this coaching exercise of having to have that person communicate back to you. Is there a sort of a platform for that? Or is that just a style that you've developed where you're willing to reach out more frequently? Or do you have check-in points or something like that during the apprenticeship that are like standard? TARAS: We don't have anything specific. Something that I use in projects is like if you're actively working on something, if you work in a project and that's your deliverable, like half an hour or so much time you get to play around with it, and then let me know. But the emphasis is on the person building the responsibility of the deliverable and knowing what's necessary for them to build to move forward. So I'm seeing that now from Lennex which I didn't see before in the beginning but I see it now. Now, if he's stuck, he's willing to say like, "Oh, okay. I don't know what's going on here. This doesn't make sense to me." And I think that's really important. I'm not sure exactly how to cultivate that but for whatever reason, it's worked with Lennex. But I think we need to help people learn that that they have to take responsibility for their work. BRANDON: It seems to me that that's the process of learning to own their own thing, learning to kind of develop their own paths. Again, that's one of those things that winds up being a better apprenticeship pattern for the person and also is a thing that lends itself to making this thing scalable where you can benefit more people and have them help themselves in each other, rather than having to be the nexus point for all questions and all accountability systems and everything. So it seems to be working really well for you. I have one more question that I want to ask from both to Taras and Lennex. What you would say to people who are intimidated about taking on a role in mentorship knowing that, "Hey, that actually does sound harder taking on a role and trying to cultivate and participate in apprenticeship." What you would recommend to that person that is interested in it but maybe intimidated by it? Then Linux, the same question for you but from the position of somebody that's trying to break in and kind of scared and nervous that it seems difficult or scary or opaque. So can we start with you, Taras? What would you say to somebody that is like 'hey, this sounds fun and cool and everything but it sounds too hard'? TARAS: To be a mentor or to have an apprentice? BRANDON: Take on the idea or participate in apprenticeship in some way. Is there some way that you know that people could get involved that would not necessarily be super intimidating? TARAS: I'm curious what the person's intention is because if you want to help people and that's the emphasis, then you have things that are uncomfortable. Especially good things in a lot of times, they're uncomfortable until you get really good at them. If you feel uncomfortable, chances are you're not growing right now. For a lot of people that are very good technically, helping others, understand these things is the next step for them to evolve in their personal growth. If they really want to get better, having apprentice is a very confronting and a very challenging thing but it creates real opportunities to see that you're actually getting better and you're improving as a person and as a technical leader. BRANDON: Then, as far as [inaudible] said, "I don't see the opportunity for myself." My guess is there are people everywhere that could use help. Do you have any suggestions on where they can find people to help, ways that they can get involved? Do you have a place where - are you looking for additional people to jump in and get involved on the mentorship side or suggestions for ways that people might get involved? TARAS: I would love to help people who want to build apprenticeship systems in their companies. I would love to help those people if they're able to do that. Otherwise, I think that creating more systems in Ember community around apprenticeship, I think would be really helpful like making more systematic because right now, it's something that I do but I'd love for it to be something to the Ember community does. Then because from that, if we engage in that conversation, then we can create spaces where it will be easy for people to do this kind of things and find apprentices and find mentors. I'd love for this to be an Ember community thing. CHARLES: Well, it seems like this would be as good a place to start as any. So, here we are. BRANDON: Maybe after this call, we'll figure something out and put something together and tell people to look for that. If they want to participate in mentorship and they consider themselves Ember or Ember adjacent, I'd like to see something in the next little while that lets people participate in a more systematic fashion. That sounds like an opportunity, for sure. If there are people that want to participate in that, sounds like you're the person to reach out to. Then Lennex, for your sake, for people that kind of want to break into technology and are really intimidated by it, obviously you're in a remote part of the world and your access has been somewhat limited. Sounds like you even fight for internet access in your life, as well, which is a foreign concept to a lot of people. So what would you tell to those people that are intimidated by doing this, scared that they're going to waste somebody's time, scared that they'll fail at it? LENNEX: I think if someone really, really wants to learn, they should find a mentor or a good apprenticeship because learning from someone who's experienced, someone who has your best interest at heart, someone who's willing to put in the effort to teach yourself, you can beat that. If you find that that someone was willing to do that, then put in the effort and go through that program because usually, apprenticeship is tailored towards a specific individual helping the apprentice. Where else will you get that? BRANDON: And there's not a ton of access to that in the world so it sounds like that's a big problem. That's what they are fixing in the tech industry. So having you on to come here and talk about it, lend some expertise to it, and start putting some ideas, process and pushing that conversation forward, it matters a lot certainly to me personally. But also to what the point of what we're trying to do at the Frontside. If people want to get in touch with you all, how would they do that, Taras? How people get in touch with you? TARAS: I'm @EmberSherpa on Twitter, and Taras@EmberSherpa.com via email. I always try my best to talk to people as much as possible and be available to people as much possible. If anybody reaches out to me, I'd love to talk to them. CHARLES: Yeah, you definitely been accessible to us which is why we're talking today and I really appreciate that. I mean, you've been genuinely super helpful. We haven't really gotten into it in this podcast but we've worked with you quite a bit and are working to improve our apprenticeship program with your help. I can't tell you how much we appreciate it. Then, for you Lennex, how do people get in touch with you? LENNEX: I am @zinyando on Twitter, and lennex@EmberSherpa.com. BRANDON: Awesome. Well, thank you both very much. This was super enlightening. It's frustrating because I feel like we just barely scratched the surface. But I feel like there's some work that we need to do to put together some process around this. Maybe there's some blog posts or some further discussion around this to put this together. I would love to talk with you all again when we have something cool to share with people. But what you're doing already is so awesome. I'd like for more people to know about it because I literally have never seen it done as successfully, as consistently as I've seen the EmberSherpa apprenticeship set up work. Thanks very much for both of you all for putting that into the community and for sharing with us. TARAS: It means so much to hear you, Brandon, say this because I respect both of you very much. I think you guys have a really great reputation in our community, and it means the world to me to hear you guys say that. Thank you so much for having us here.
Recently, there was a flurry of activity around one of Brandon's posts about defining the term "senior developer". But he left the purely technical aspects of the role for later discussion, which left a lot of lingering questions. In this episode, Charles and Brandon dive into the technical side of identifying, hiring, and growing senior developers, and explain The Frontside's somewhat unconventional standards. Links: The Conjoined Triangles of Senior-Level Development Don't use animal names as an insult Transcript: CHARLES: Hello everybody and welcome to the Frontside Podcast, Episode 41. I am your host, Charles Lowell. With me is our other host, Brandon Hays. BRANDON: Hi, welcome back. It's been too long. It's been one week since you looked at me. CHARLES: And we were actually talking right before the start off that we don't have any witty banter prepared for this episode. BRANDON: Also, you just drop that hot Barenaked Ladies reference right on the floor, like an apple pie upside down. CHARLES: Right, you got to prep me for that. BRANDON: You're like, "Nope. Pass." [Laughter] CHARLES: Like I said, you got to prep me for that. BRANDON: "In Second 7, I'm going to drop a Barenaked Ladies reference." I'm going to send you three or four music videos. I did send you one that you didn't have time to watch about don't use animal names as an insult. When you say, you're not going to be able to riff off me on that one just yet either, but yeah, I respect you and I respect dogs so don't use dog as an insult. [Laughter] CHARLES: Is that 'They Might Be Giants'? BRANDON: It is not They Might Be Giants. It is three vegan randoes on YouTube. CHARLES: Is that the name of the band or is that just the content of the band? BRANDON: No, they're just three vegan randoes on YouTube. CHARLES: Okay, three vegan randoes is a pretty good band name. BRANDON: You'll immediately know they're from Portland so you could really pick a lot out of that from just the name of the band. We want to tight 30 today. We don't let people behind the curtain very much, Charles, where people don't see what it is that you and I do behind the scenes. But one of those things that we do is sometimes we will record a podcast and throw it in the garbage because we hate it, and this is actually one of them. You and I sat down and recorded this podcast before. Just like the hot apple pie Barenaked Ladies reference that I served you earlier, we just dumped it right in the garbage. We did not like the way that it tasted. We did not like its Barenaked Ladies references, and so this is our second attempt at this. Then the topic that we want to cover today is really important to us to not screw this up. So hopefully, we put a little more effort into preparing for this and thinking very deeply about this. The idea is that we want to understand from a technical perspective largely what it is that a senior developer is, what they do, how we can find them, how we sometimes miss that, and basically how we define a senior developer so that we can build that as a track for our people to grow toward and find the ones that want to come work with us and may self-identify a senior, how we can kind of verify that. That's the poorly defined topic in our industry, interestingly enough. CHARLES: Strange, because everybody seems to want that. BRANDON: Yeah, right. Everybody put it on their job descriptions. Anyway, I wrote a blog post about it. It did all the things that blog posts do when they sort of struck a nerve with the tech industry and they got posted around. I got called literal human feces on Reddit. CHARLES: Did they call you human feces or do they call you literal human feces? BRANDON: They said literal -- CHARLES: Did you literally get called literally human feces? BRANDON: I literally got called literal human feces. [Laughter] CHARLES: I shall wear it as a badge of honor. BRANDON: I was kind of like Ron Burgundy. I'm like, "I'm not even mad, I'm just impressed." [Laughter] CHARLES: People had some strong, strong, strong feelings about that. BRANDON: They did. So now, a skull in a cowboy hat and literal human feces are going to sit here and preach to you about what we think and how we're basically like willing to sink or basically, sink or swim for our business on this definition of senior developer because that actually is core to what it is that we're doing. I want to tee this up and then I'm going to let you just freakin' roll. I want to provide a little background. The cool thing is, I already wrote a blog post so I don't have to sit here and talk at your face about these things. I was going to tell you some additional thought I've had about this. But basically, the point of this blog post was that we generally categorize what it is to be senior developer in three broad categories that have a little bit of overlap. The three categories are technical skill, which is the one that people typically think about, evaluate for leadership and connectedness, which sounds all [inaudible] but the reality is it's actually very concrete and very important. The categories [inaudible] leadership is basically the idea that the more ownership you give someone over the broadest version of the problem you're trying to get them to solve, the better they perform. So if you give somebody a small task, they will maybe perform it. But if you give somebody ownership of the actual problem and the business problem that you're trying to solve or a user's pain point, they will find ways to solve that problem that you wouldn't have considered yourself. They will pull other people into their orbit to solve that thing. Basically, there's a sense of ownership and the experience they have in owning something all the way through to completion basically. Then, the technical skill one which we'll dive into in this podcast, that's what I want you to kind of drive the conversation around, is basically the idea that the more difficult a problem you hand somebody, the better they perform. A lot of times, that's due to experience, some of it is personality type, some of it is muscles that they like to exercise. Then, the last one is connectedness, which sounds like it's hard to define but it's actually the idea that the more people that are impacted and the more deeply people are impacted by somebody's participation in a task, the better that person performs. These people like to mentor, they like to be parts of communities, and they have a deep sense of empathy for the users that they build software for and the other developers that they're developing for alongside. CHARLES: Right, and I think that one of the reasons you got such a negative reaction, you kind of, I think, were assuming technical skill and then really kind of unpacking the leadership and the connectedness as absolutely key components to be what we consider a senior developer. So, kind of pointing out that especially from the leadership aspect, it's like you need to be a wholly-formed developer, you need to have those headlights on the car to see where it is that you're going and you can have a huge engine and like four-wheel drive and like big mud tires that can dig into any surface. But unless, you can actually see where you're going and perceive the problem holistically, those skills aren't going to be put to as good of a use. BRANDON: I think that's a good metaphor. The idea is that it's traction and direction in addition to the raw technical horsepower that you bring to the table. So pointing out and emphasizing these other areas, I think, made people - there was a lot of confusion. Some people reacted very positively to it like, "Oh my gosh, finally somebody understands that I bring more to the table than just my raw technical capability," especially people from -- CHARLES: I'm going to guess those were the people not in the literal human feces camp. BRANDON: Well, I didn't set up a Twitter poll or anything but I hope I can assume so. Then there were people that felt very strongly that I was overlooking technical skill. I was like, "No, no, no. That's a fair question. That's Charles's job here." And so that's why, at this point, I am going to drop kick this entire conversation into your side of the metaphorical foot game ball field. I think, football pitch. Pitch, right? Quidditch pitch. I'm going to drop kick the Quaffle into your side of the Quidditch pitch. CHARLES: Okay, all right. Well, let's unpack the technical skills that we look for. What we're looking for, I think, on the technical side, and this is going to seem obvious but what we're looking for is experience. We're looking for the quantity of experience and the quality of experience. We can roughly subdivide that into four key areas. First of all, what's the experience that you've developed around your curiosity? Maybe we should go and lay all these out. We look for curiosity. We look for rigor in your technical skill. We look for your ability and history for actually shipping things, and we look for you to be fearless when it comes to taking on new problems and sizing up the problems that you choose to attack. When we break down those experiences, if you are a curious person, you have been exposed to a lot of different technologies. So we're going to be looking for, you have an opinion on a bunch of different languages, a bunch of frameworks. We're going to want you to have tried a bunch of different things. We don't want you to have knocked your head against a lot of different problems, and then tried a lot of different solutions. That's going to be very indicative of your ability to bring a diversity of solutions to any given problem that you face. But it's not just the quantity of the experience that you develop. It's also like the quality. Like how much in the solutions that you develop are you looking to find the whole solution that fits your problem? How willing are you to do A-Grade work where you consider every corner case and you are willing to dedicate the CPU resources to find the best solution. Finally, what is the scope of problems that you're taking on? We've talked about the difference between quantity and quality of experience. You can make a career banging out WordPress apps. It's a great place to start but if you're doing it -- BRANDON: Or you can do a CRUD over and over again. CHARLES: Yeah, you could do CRUD over and over again. Like I said, that's a great place to start at the beginning of your career. But if 20 years on, that's what you're still doing, then you're going to have a lot of experience but is it going to be a high-quality experience? Are you actually taking on bigger and bigger problems, and is the scope of the things that you're tackling growing as you move throughout your career? BRANDON: It's interesting that's actually like fearlessness, that kind of technical fearlessness is actually a skill a person learns to practice. I was a very fearful developer a few years ago because I felt like if I charged into the bramble of a complex and thorny problem, that was going to cut me up and I die. It turns out, no, it just cuts you up real bad then you get back with thicker skin. And eventually you start going to the point where you start learning to tackle, you learn to take on things you don't understand because that's literally the job. CHARLES: I think it's important to point out that fearlessness is a skill. It is not a personality trait and it's something that you have to develop and you can actually take on problems that are too big for you at the given time and kind of have to know when it's time to step back from that. I know that's actually something that I even look for, anecdotally is asking a developer, "What are some of your greatest failures? What are the things where you took on way too much than you could chew? How did it make you fall down flat on your face?" Because that is an artifact and evidence that someone is practicing and taking on things that maybe are too big. Sometimes you're going to overshoot the mark. You know, you don't want to be taking on too little but one of the ways that you're going to do that is by accidentally taking on too much. And so I find that most people who have a senior level of experience have some big failure story. There's a skeleton in their closet of something they did wrong and they know that they can acknowledge that. I don't know. That's certainly how I feel about it. BRANDON: Yeah, I can agree with that. I guess you're saying you wrap all of those four things together like whether this person is willing to take on difficult problems, whether they actually complete them, whether they do them in a way that displays that they have enough experience to have kind of developed principles about how they approach stuff and whether they are continuously looking for new ways to approach problems. Like if you combine all of those things together is this like cool technical Voltron that exhibits a directed, focused kind of practice that over the course of 5 to 10 years yields basically a person that you can throw at any kind of technical problem and they will act like, I don't know, like nanobots just destroying that problem from the outside and until the problem doesn't exist anymore. It's thoroughly solved. CHARLES: Right. And I think that ties you to this propensity to ship. That's something that we look for. If this is a skill that you have, if you are at a senior level of experience, you will have a set of technical achievements that we can look at, that you have shipped and we can study them. Fantastically, as a happy coincidence you can see inside those things that a person has shipped. Are they rigorous? Are they curious and how fearless are they? What's the array of technologies? How unique are the solutions? How informed are they by different technologies and what is the scope of problem that you are trying to take on throughout the course of your career? So, having actual things that you have shipped whether it's products, whether it's libraries, whether it's open source, something like that, you want to be able to look at those and it is important. We don't rest everything on the GitHub. But I think at a senior level, you should have some equivalent that you can point to, public or otherwise. BRANDON: Yeah, there's some artifact that pops out of that, and if you made a career of 10 years and you come and say, "Hey, I have been doing this 10 years and I'm a very skilled senior developer but I can't show you anything." I understand that people will find themselves in situations like that but that's going to be, I think, the great exception of the rule. If you really don't have anything to show, it probably means there's some sort of practice, one of these traits that likely could use a little more exercise on that particular muscle, and that you've been leaning on compensatory muscles in other areas. So that's going to ding you in terms of like, how we evaluate somebody as whether they'll sync or not. And we're open to being surprised. Well, I should probably hopefully get to that before we end our podcast. CHARLES: I'd like to contrast it with what we look in, in a junior developer because it is very, very different. I think we've talked a lot about or there's a big conversation about, does passion matter in evaluating a candidate? I think passion is a word that's kind of fallen by the wayside but talk about maybe a less charged words like just caring deeply about a product or a solution or something like that. In a senior developer, to be quite honest, passion and caring is something that we expect to be there but it's not something that we look for because it's not something that we need to look for because if they do care about diversity with technologies, if they have this propensity to ship and they're taking on big problems, then that evidence will be there. So we're not looking for passion because it's either there or it's not. That's something that we look for more in a junior developer because we're going to be looking for that enthusiasm in the same way that you kind of track a hurricane. You don't know exactly what path it is but based on the weather patterns and the basic trajectory, you can find out essentially where it's going to end up. BRANDON: Yeah, I think there's a lot of talk in the industry about how passion, when people talk about looking for passion, what they're basically saying is we are looking to exploit people who are at the top part of their curve. When you look at how the undulations of a person's amount of passion over the course of their career, you have companies only want to clip off the top of that. As soon as you're not passionate anymore, you jettison that person. What we're looking for is somebody who's sort of stabilized that. So the passion part of it is like, "Oh, that's cool. The passion is in your past or it's in your present." But we understand that that is a sine wave and we're looking for the line in there that says, "Hey, sometimes I care more than I care about other times but the main thing is I produce things." Otherwise, instead of looking at as a sine wave of passion, you're like trying to exploit it at the top and then going, "Well, this is just going to keep going up and up and up," and you burn people out for that. Anyway, so it's not something we look for. It doesn't make sense. It's not sustainable and it's actually something that in a senior developer has stabilized enough that you're not going to see it. CHARLES: We look for, "Hey, here, we build things. So, if you're at the beginning of the career, are you following a path that will lead you to build good things and then at the end we're kind of towards the back half." We are looking at, "Hey, what are the things that you have built that are capable of building?" And that's the extent of it. I don't want to suck all the emotion out of it but the thing is that I want to suck the pressure out of it. BRANDON: Yeah, and the way that we try to assess that right now is we do this in a full day pairing interview. We do a lot of stuff that leads up to that and hopefully, the ancillary stuff we do around that helps gauge like, "Hey, this person is active in the community in some way, or they try to make contributions to people this way." We do our best to get a sense of the stuff that is not pure technical skill because delivery, it takes longer than a day. So, how well does this person deliver? How well do they work with other people? That stuff is very difficult to suss out in a day but you can generally get a sense of like technical experience capability. So I kind of want to focus on like during the course of the pairing interview, what it is that you're sussing out? How do you know a person in a pairing interview is exhibiting those technical leadership traits, technical skill traits? CHARLES: I'll beat this drum again. I'm looking for quantity and quality of experience. It comes down to little things. How comfortable is a person with their tool set and that demonstrates both the experience and you can tell if it's good experience or not. How comfortable are they with the command line? How comfortable are they with Git? Are they taking baby steps with it or they doing large motions in one kind of fell swoop? Like, how effortlessly is the muscle memory there? Whatever the tool set is, whether you're using iTerm, whether you're using C-shell or bash or whatever editor you're using, I do expect to see a familiarity that can only come through having done it again and again and again and again. So, if you're working with Git, for example, and you're kind of stumbling and stuttering at what commands to run or pausing, when you are parsing the output of a command if there's an error or something didn't go, what that says is either you don't have the experience which is most of the case or that has not been a priority in the experience that you do have. So there's like kind of a difference in the quantity and quality of experience. Some people move fast. Some people live slow but I want to say continuity that I'm looking for, in the same way that you can play a piano quickly, you can play it slowly but when people are missing notes, you can detect that. That's what I'm looking for, kind of with the tool sets. When someone is moving within a code base, I'm looking for confidence of motion. I remember an interview that we did recently where we were working in a code base. We're trying to get something running and this developer deleted like two directories at a time that caused like 10 files to be just missing from the code base. Just gone, at the stroke of a key. Then, we ran the test again, and lo and behold, like everything worked. Or the test that we were expecting to fail failed. BRANDON: Or was that the test directory? CHARLES: Sorry, what I meant was -- BRANDON: No, I know what you mean. They were comfortable enough in their understanding of the code base they were working in. CHARLES: Right. There was a high level. We were not down in the weeds. There's like you're applying very light pressure to the code base to make it move in gigantic swings so you understand the pressure points and you can zero right in on it. BRANDON: You can say you're looking for like, the aikido development strategy or whatever. CHARLES: Right, and then the other thing is being able to -- and this is important because I don't believe that every coding session or every pairing session needs to be this dance across the keyboard. Certainly there are times when you recognize that it's natural to stop, to pause, and have a discussion and say, "Wait a second. What are we doing here? Let's hash this through," and be able to converse at the higher level of where exactly are we going because normally you have fallen into this rhythm, or you've got a driver, you got a navigator but then sometimes you have to stop. You have to take your bearing and realizing when that time is and naturally transition into that, and be able to discuss the problem at its highest level where the code really and the tools that you're using really fade away into the background. They're not important. Then you can transition back and I'm looking for a pair in that conversation and I'm looking especially for someone who can teach me something. If there's a knowledge gap that I have or a perspective that I haven't seen, so when I'm looking for where it is that we're going, if someone helps me pair around a corner, that's a huge indicator that this person is senior. Because not only do they have that perception but they can also share it with me and they can effectively argue for it and make me see it, as well. BRANDON: Another thing for me in that same vein is the ability to be challenged. Like, for a person to challenge and be challenged. So they go, "Hey, I have an opinion about this," and you go, "Well, what about this?" And they're like, "Gosh. I never really thought about that." Or they defend their position or they push you on something, "Why don't you do it this way?" And you're like, "Well, I've always done it this way but let's try it your way." So the ability to challenge and be challenged particularly, that is sussed out in a pairing interview, you could probably do that asynchronously through a pull request and stuff like that. But just being able to suss out, whether somebody can challenge and be challenged as a part of the educational process means that this person sharpens the people around them and they're sharpened by the people around them. CHARLES: Absolutely. I am looking for that ability to be collaborative at that level and to educate those people around them just by virtue of their interaction. BRANDON: One thing that sort of heretical in some circles and it seems weird to me but if you look at 95% of jobs descriptions, they will tell you, "We are looking for X number of years in X technology," and I don't want to discount the years thing. The thing is gaining experience takes hours of practice, of dedicated practice and those hours add up to years. So you're typically you are talking in the scale of years when you're talking about experience so there's not really a way around the fact that you were going to be talking on the scale of years. I think that's not worthy of too much debate. It can take some people two years to learn something. It might take somebody else 10. But that you are talking about a scale of years. The thing that I want to challenge is the idea that language and framework experience is the thing that you're looking for. We have a lot of counter experience so that where people cross those boundaries and lines so I didn't really have that thought before talking to you about this so you kind of pushed my thinking in this regard, like why don't we care about that or why do we care less about that than others might? CHARLES: From my perspective, it kind of flies in the face of the way we operate. If we're requiring that someone have a certain number of years of one particular technology, we're asking them to be curious. We're asking them to be diverse. We're asking them to be able to be fluid and slot themselves into any problem space. So, why would we then demand all of those things and then, "By the way, you need to be able to be slotted into this problem space to come work here." I think that something that gets overused in the industry. I think it does make sense in certain cases. For example, not for a developer position, it's less than $250,000. Like if you want someone with five years of MySQL experience or something to manage who has to do micro-optimizations for these huge scale things and you want to pay them half a million dollars, because that's the thing that you need, you really need somebody with that much experience in that technology. Well gosh, they'd better be charging you a lot of money for that experience. Like, "You literally want to buy five years of my life? Okay. I can do that. But I'm going to think if that really is your need, that's what I'm going to be charging you for." But we're looking for people who will be able to move into any problem space that we come across and that necessarily implies language and framework. In fact, one of the things that I like best, I talked about evaluating for the ability to teach and that was inside the context of the one project that we're looking at, but I've had great experiences where someone has taught me something completely outside of certainly my level of expertise, and sometimes outside of what I've even heard of before. That's a much better like if we're working on a JavaScript project, and you can say, "Oh, this is how we do things in Scala, and maybe we can apply that here." Man, that is so valuable, and that has nothing to do with JavaScript experience and everything not to do with it. So what we're looking for is you to be able to bring to bear the arsenal of tools that you have developed throughout the course of your career and all zero them in on a problem and just blast it out of the water. So why would you limit that? Why bring one gun when you can bring 30, and one of them is a howitzer. BRANDON: That makes a ton of sense. It's the idea that is connected to our purpose which is like one of our purposes in existence is to advance the state of the art in UI engineering. That may sound like BS but it's very true. That is very much why you get up in the morning and come to work. It's like half of it is about creating a generation of leaders, and half of it is advancing the state of the art in UI engineering. Advancing the state of the art in UI engineering is not going to happen if you only bring people in who already think the things that everybody else is already thinking in UI engineering. So, we need people with orthogonal experience in other things so it's the diversity of experience that actually helps us get closer to that goal. Bringing tool sets from alternate languages and frameworks is a huge way to bring that to bear. I also think it's a way that our industry is like an adorable little baby. It's adorable. Basically, 95% of the people in our industry are generalists, and we hire, as if we're all hiring for the 5% that is a specialist because we don't know how to do this as an industry. We're just a baby. When you think about it, it is kind of adorable. We're all kind of making this up as we go along so if we can start like the goal here is to kind of push this conversation forward and understand, maybe it's counterintuitive to hire a passionate specialist when what you're looking for is a stable generalist, primarily as an industry. Start understanding, maybe these are some of the ways that we've accidentally been making our industry narrower and less diverse is we're looking for people that started programming on a TRS-80 back in 1984 or whatever. And so the problems that come along with that, we're starting to have to struggle and cope with now. So, anyway, in our own tiny little corner of the universe just for our local maximum of trying to build a better software consultancy, it makes a ton of sense for us to look for generalists. CHARLES: Yeah, that's definitely fits right with our value proposition is that we do UI but we can apply our UI skills to any number of problems. But it might not make sense. Like I said, if you're managing some gigantic database cluster and you need really, really, really specific skills, I just hope whoever that is you're charging a lot of money. BRANDON: Yeah, you can get away with generalizing and going, "Hey, I'm going to select for a lot of things." But if you're going to specialize, you basically have to go for money. I think we're getting into the place where we're pulling this into a tight 30 minutes. We've got to wrap this up. There's always a million more things to say on these topics but I feel like, I've learned a lot through the course of preparing and then doing this with you today to around like why we do what we do? What we're looking for technically? It'd be really cool, 'Hint, hint', that this turned into a blog post on your part. Because I do feel like it was the piece that was missing in people that kind of riled them up a little bit. It is an important piece like not to just -- CHARLES: It's okay people, we do care about technology. We do care about technical skill. In fact, we are a company dedicated to developing it. BRANDON: Yeah, that's true. People have to walk out with more of that than they walk in with or we fail. But that's not going to stop the people from yelling at you. CHARLES: Hey, it's fun. It's fun to yell on Twitter. BRANDON: It is funny. It's fun. All right, man. Well, this was really cool and I hope we get to do this again. We basically are in the process of kind of rebooting our podcast and getting in on track while we record it. Every two weeks at least, we have guests lined up -- CHARLES: Do we want to share like a sneak preview? BRANDON: Oh, gosh. Yeah, I mean with the people that we're talking to soon, we're going to have a conversation with Ember Sherpa about apprenticeship. We're going to have a conversation with Leah Silber about building communities. Then, there's a ton more in the hopper. This is going to be a really good rest of the year for this podcast and we're going to get consistent about it and we've hired help with it. We're really excited to have Mandy Moore. She's @DevReps on Twitter, if you are looking for any assistance with stuff like this. And we're really excited to start kind of kicking off a new -- CHARLES: A new era. BRANDON: But for those of you -- CHARLES: For those who like the intermittent, unreliable, yet sometimes pleasing dead cast, we'll interleave a few of those too. BRANDON: We'll do our best but this is yeah, for those people, I must apologize. I really liked how infrequent it was. Like, oh man. Maybe just check in every once in a while. CHARLES: Yeah, it's cool. All right. BRANDON: Charles, it's been great talking to and I can't wait for next time. CHARLES: Yep, likewise, man.
Jon is joined by Frontside founder Charles Lowell to discuss everything from EmberMap to Percy in this fun episode.
There's a huge shortage of senior developers, and one (often overlooked) way to fill those positions is by bringing up some junior developers. But how do you mentor junior devs when you have so much work to do? How can you make sure that your new hires get the support they need? This week, Charles Lowell is joined by Stephanie Riera, Lydia Guarino, and Alex Ford to talk about the challenges companies face when hiring junior devs, what steps you can take to make sure the on-boarding and training process goes smoothly, and how keep new developers productive and frustration-free. Follow the Frontside crew on Twitter: Charles Lowell Alex Ford Stephanie Riera Lydia Guarino Show Links: So You Just Finished a Bootcamp, Now What? Lydia's tweetstorm: Where do junior devs fit?
In this podcast episode, Jeffrey Biles (@jeffreybiles) of Ember Screencasts interviews Charles Lowell (@cowboyd) ride the functional programming wave and dive into what this actually means. Topics covered: - How to think about MVC - Moving logic out of controllers and components and into the template - Functional programming in ember - The rise of helpers - Addons such as ember-composabe-helpers and ember-curry-helpers - The universal UI workflow Find more podcasts, videos, trainings and online conferences at http://modern-web.org or follow us on Twitter @modernweb_.
Chase and Jonathan discuss the new getOwner API, Functional Templates focussing on a blog post by Charles Lowell, the possible ArrayProxy deprecation and more!
Wynn and Gregg Pollack did a special LIVE episode at Red Dirt Ruby Conf where they sat down with Charles Lowell to talk about embedding JavaScript engines in Ruby.
Wynn and Gregg Pollack did a special LIVE episode at Red Dirt Ruby Conf where they sat down with Charles Lowell to talk about embedding JavaScript engines in Ruby.