Podcasts about charles you

Christina returns to the podcast to revisit this episode with me – I hope you enjoy it! Divorce, Walnut Grove Style Season/Episode: Season 7, Episode 7 Air Date: November 10, 1980 Writer: Don Balluck Director: Michael Landon In Little House on the Prairie's “Divorce, Walnut Grove Style,” newlyweds Laura and Almanzo Wilder face marital strife that spirals into comedic chaos. Laura, overwhelmed as a schoolteacher, serves watery stew and neglects household chores, irking Almanzo. Their bickering escalates when Laura misinterprets a note—“My Only Love… Brenda Sue Longworth”—as proof of Almanzo's affair with an old friend, Brenda Sue, who's actually just celebrating her published song. Jealousy drives Laura to a public brawl with Brenda Sue and a dramatic exit to her parents' home, fueled by Harriet Oleson's gossip. Charles and Caroline urge reconciliation, but Laura's stubbornness persists until Almanzo clarifies the note's innocent meaning. The couple reconciles, vowing better communication, while Charles' ongoing struggle with a picture window adds humor. This episode, directed by Michael Landon, blends sitcom-style misunderstandings with heartfelt lessons about trust and love in marriage. Some Quotes chosen by me: “I'm doing the best that I can! Why can't you understand that?” (Laura) “Because all I understand is I work hard all day, I come home to a half-cooked meal and a house I can't sing in because you've turned it into a library!” (Almanzo) “There's just no excuse for a husband to call his wife stupid.” (Charles) “You know, I just can't get over that boy calling my daughter stupid…” (Charles) The opening song “Albert” is written and performed by the amazing Norwegian band, Project Brundlefly and is used with permission. Check them out at: https://www.facebook.com/ProjectBrundlefly Become a Patron! The post Divorce, Walnut Grove Style first appeared on The Little House on the Prairie Podcast: Walnut GroveCast.

Check out this Insightful Encore from October 4, 2024! Father Brian O'Brien joins Patrick to discuss How to Navigate Election Season as a Catholic (4:23) what is the purpose of politics? Have we lost sight of what politics is for? (18:04) Break 1 What does the separation of Church and state mean? (22:58) Paul - People labeling a lot of things as political. Using it as an excuse to support things that aren't right. Agrees with Fr. O'Brien. (25:46) What is the hierarchy of moral issues in the Catholic view? Charles - You have to consider all 10 commandments when picking a president. (37:53) Break 2 Vince - Is it ever moral because you're so disgusted by candidates, not to vote? Brenda - There is analysis on USCCB website and the issues. There's also a good factual one done by Million Voices.  We're not voting for personalities; we're voting for issues.

Father Brian O'Brien joins Patrick to discuss How to Navigate Election Season as a Catholic (4:23) what is the purpose of politics? Have we lost sight of what politics is for? (18:04) Break 1 What does the separation of Church and state mean? (22:58) Paul - People labeling a lot of things as political. Using it as an excuse to support things that aren't right. Agrees with Fr. O'Brien. (25:46) What is the hierarchy of moral issues in the Catholic view? Charles - You have to consider all 10 commandments when picking a president. (37:53) Break 2 Vince - Is it ever moral because you're so disgusted by candidates, not to vote? Brenda - There is analysis on USCCB website and the issues. There's also a good factual one done by Million Voices.  We're not voting for personalities; we're voting for issues.

This episode of the Live Better Sell Better Podcast features Charles Muhlbauer, Lead Revenue Enablement Manager at CB Insights. Discovery can seem like a seller ticking boxes of questions. Charles shares how to make the conversation flow by taking your prospect's answers and contextualizing them back to them. He also gives tactical advice on being accurate rather than optimistic and how this takes the pressure off of the prospect to be upfront about "no."HIGHLIGHT QUOTESBase your questions on what the prospect just said - Charles: "You can have the best discovery questions on planet Earth, but if your discovery questions are not based on what the prospect literally just told you, it will be an interrogation."Get a reply by being accurate rather than optimistic - Charles: "The main reason a prospect will ghost you is that they are afraid to tell you no. So I love always telling the prospect I'd rather be more accurate than optimistic here. It's okay to tell me no, don't worry about it. I'll say it all day." You can find out more about Charles in the links below:LinkedIn: https://www.linkedin.com/in/charlesmuhlbauer/Website: https://www.cbinsights.com/Live Better. Sell Better. is sponsored by our proud partner:Vidyard | vidyard.com

CHARLES: Hello and welcome to our "Thank You Podcast for 2022." This is Apostle Charles Wright.STEPHANIE: And this is Stephanie Wright.CHARLES: Psalm 136 says, "O give thanks unto the Lord; for he is good: for his mercy endureth forever. Oh, give thanks unto the Lord." This is Apostle Charles Wright.As we celebrate Thanksgiving, Christmas, and the New Year, we want to thank our listeners for tuning in to this podcast. Thanks to you, we have reached 52 countries and almost 900 cities worldwide!STEPHANIE: We also thank you for your financial support, letters, texts, emails, and phone calls that have inspired us to press on with passion in our purpose. That purpose is to encourage and uplift you to live an abundant life in Christ through each season in your walk with Him.CHARLES: You are a blessing to us. Most of all, we thank God for his blessings this past year and for giving us His direction on each podcast. Our theme throughout 2022 was "In the Heart of God." We started with God's Word because the Word is God's Heart because Jesus is the Word. Some of the topics we covered included: Christ - His death and resurrection; Jesus' return; Jesus as our living hope; wisdom, peace, heaven, women and men and their relationship with God, and we covered eight weeks on dreams, visions, and thoughts and how God speaks to our hearts.CHARLES: The past year, 2022, brought some financial, physical, emotional, and other hardships. Our hearts goout to you. Please remember that "Our God" sees, knows, and He cares and has not forgotten you. We continue to seize onto His Holy Word, which says, "He will never leave us nor forsake us." Hebrews 13:5. Deuteronomy 31:6 tells us to be strong and of good courage and not be afraid, for the Lord, our God, will not fail us nor forsake us. So, although we may feel forsaken, it is just a feeling, it is not a fact, and most of all, it is not the truth.STEPHANIE: Beginning December 1 and throughout the month of December, we encourage you to listen to select Christmas podcasts, which include special guest appearances, scriptures, and portions of CGM's Christmas CD, "We Love Christmas." A single song or the entire CD may be ordered on our website at cgmissions.com/podcastCHARLES: We will also be planning for the next year, 2023, so please pray for us. If you have suggestions for next year, please email us at cgmprese:nts@gmail.com And if you do not know Jesus as your personal Savior, or you have turned away from Him, invite him into your heart today. Call on him to save your life to a life that pleases God. Then, you will experience the true meaning of Christmas.STEPHANIE: In closing, I always like to remind listeners of my motto since 1998, "Don't Let Anything Steal Your Christmas." That's what happened to me. I let someone steal my Christmas, and it wasn't the Grinch. But after that experience, I refused to let anything or anyone steal my Christmas.CHARLES: So, have a safe and Happy Thanksgiving, a Blessed Christmas, and a Joyous New Year. And be sure to tune in on December 1 as we start the Christmas podcasts.CHARLES: This is Apostle Charles Wright . . .STEPHANIE: . . . and this is Stephanie. May God's peace, love, and protection be with you until we meet on December 1. And please, "Don't Let Anything Steal Your Christmas."If you want to contact us or comment on this podcast, please visit our website at cgmissions.com/podcast where all episodes are available by category in series format. We can also be reached on Instagram at @cgmissions, and on YouTube @cgmissions. CGM is a 501(c)(3) nonprofit. Donations may be made on our website: www.cgmissions.com

I have had my head down working on some big things since RapidStart CRM growth exploded, and it has been a while since you heard from me. Well, I'm getting back to it with a follow-up chat with Charles Lamanna who recently took over for James Phillips as head of Business Applications for Microsoft. This was my fourth chat with Charles, and it was interesting to back listen to them in order. It really gives you a sense of where Microsoft has come. I managed to catch him in his office having just wrapped up their year-end. Enjoy! If you want to listen to my chats with Charles in order, The first one was October of 2018, the second one was September of 2019, the third one was March of 2020. Transcript Below: Steve: Welcome to the Steve Has a Chat Podcast. Where I call someone out of the blue with a record button on, and hope to have an unscripted conversation about Microsoft business applications. Let's see how it goes. Enjoy. Charles: Hey, this is Charles Lamanna. Steve: Charles. Steve Mordue. How are you doing? Charles: Good. Great to hear from you, Steve. It's been a long time. Steve: It has been a while. Have you got some time for a chat? Charles: For you, anytime. Steve: I appreciate it. Well, I guess the big news for you obviously is putting on the big boy hat, huh? Charles: Yes. I moved up an extra floor in the Advanta building in the Microsoft Campus. Steve: Oh did you? Charles: No, I'm just kidding. But metaphorically speaking at least. Because for folks that don't know, James Phillips leaving in March of this year, I kinda stepped in across all aspects of business applications of Microsoft. And, over the last four years, I've gotten to know the place, know the people, know the business and I'm super excited about the opportunity. And I think the future has never been brighter for business at Microsoft. Steve: Well, I never got the feeling that James held you back, or any of the folks on your team back, but he certainly, we have to give him a lot of credit for really taking this thing to a whole nother level. You weren't here before, I don't think, at least with the business apps, but it was really run by morons before he took over. And he completely turned that thing around and turned it in a whole nother business. And now with you taking over, I'm expecting that to continue. I don't know if there's been some things that have been in your bag that you've wanted to do that James was keeping you from, that you're going to pull out, or if you're just going to continue the path, or what's your thinking now that you've got that gavel? Charles: So definitely not held back. I would say I was super fortunate I worked for James for, I think seven, eight years in total. So I was able to learn a bunch and he was without a doubt, the most supportive manager I've ever had in my career, in terms of both enabling and clearing paths for what we wanted to do from a vision and dreaming perspective. And if it weren't for his support, things like Power Apps would have never gotten off the ground. So, definitely. And I think as we go to the future, we have this amazing foundation. I mean, BizApps is a major and key component and pillar of the Microsoft Cloud. Charles: 10 years ago, you probably would've thought that impossible. Right. To have Dynamics and Power Platform alongside Azure and Office. Now that we're here, let's go take it to the next level. And that's the push, and it's continuing a lot of the great innovation we've already done from a data-first, AI-first approach. Kind of sprinkling in some more collaboration with teams, and really revisiting the end-user experience, the platform, to go increasingly modernize and scale it and make sure that all our components from CRM, to ERP, to Power Platform work great together. Steve: I don't think it could have achieved that status with Dynamics 365 alone. It really took the Power Platform coming into being, I think, to give it the breadth that it needed to be able to get there. With Dynamics 365, we didn't have apps for users to do small things, there was no way it was going to permeate an organization the way the Power Apps do. Charles: Yeah. Yeah, that's right. I say two things are interesting. The first is, Power Platform has allowed us to help more users and more customers with business process transformation, which is what BizApps are all about. Right? Steve: Yeah. Charles: How do you make your sales processes better, your financial processes better, and Power Platform really turbocharged that. And that earned us credibility in a lot of those departments and with a lot of those users, and we have some great data about every user who adopts Power Platform is significantly more likely to adopt Dynamics within the next year or two. So we see that symbiosis working in a way which is incredibly customer-friendly, and it helps our business. Second thing is Power Platform has even helped us reimagine parts of the Dynamics apps themselves. And I think probably two of the best examples are the connectors, which are key to the Power Platform. Charles: You see the connectors starting to show up inside all these Dynamics apps, like Customer Insights uses Power Query for data ingestion, or Viva Sales even connects to Salesforce. So there's this amazing interoperability that we have, and also enabling the end-user. Our team built Viva Sales, even though it's not in the Dynamics or Power Platform brand. But it's this idea of having an integrated experience in Office for sellers, built on connectors and built on the Office integration. So it's changed the way you think about some products, and it's also helped us go expand our user base. Steve: Yeah. I saw I was on a PGI call with that yesterday. Very, very cool stuff. At the last PAC meeting, I was supposed to be on the Viva Sales round table, but I'm like, "Yeah, that sounds boring. I think I'm going to go to this one." And I really, I went to the wrong one, I missed a good one. But you know where I am, right? I'm on the platform. Charles: Yep. Steve: And we're exploding. Our app is continuing to grow on the platform as a low-cost simpler alternative to Dynamics 365 for companies that aren't ready for that. And I'm always bugging you about, "Hey, that cool new feature you guys got in the first-party. When are we going to get that at the platform level? So ISVs, and people that are just building their own stuff from scratch, could take advantage of some of the syncs." We got the Outlook app a while ago, we've been getting some things. And when I saw Viva Sales, that was probably my only disappointment was that, at least as I understand it, it's hardwired to Dynamics or hardwired to Salesforce. And I get that trying to play those two against each other, but it's leaving guys like me out in the cold. Charles: Well, I'd say for Viva Sales, the intent is to support any CRM, and I really do mean that generally. And even customers, because there are customers out there that we talked to today who have homegrown CRMs, they coded 15 years ago. They have a whole dev team still working on it. The idea is to support interoperability with your account records, your lead records, your opportunity records, standard pipeline data. And to do that in a way which works through the connector. So today it'll earn V1, it'll only be Dynamics in Salesforce, but the intent is to make that be a general purpose adapter. And you could have a RapidStart CRM connector, which shows up and supports the contacts the way we want, and it would be connectable. That's not going to happen in the next three months, but that's the ambition. Steve: I can call you in four. Charles: I go down and said... What was that, in four Months? Steve: I can call you in four months. Charles: Yeah. Yes. Yeah. I might not pick up the phone then in four months, no I'm just kidding. Because even talking about, if people are even on Seibal. We should be able to support them with their sales. Because the idea is, you shouldn't have to transform the seller experience at the same pace that you transform your core CRM, your core system of record, and that's just the way the world's moving. Steve: Well, I love the idea that one of the challenges that CRM has always had, of course, is user adoption. It's one more place they need to go to do something. Outlook app helped with that, getting data into CRM without them having to actually go to it. It seems like yet another way for people to engage with their CRM without actually realizing they're engaging with their CRM. Charles: Exactly. Yeah. It's almost like ambient... Yeah for sure. Sorry. Yeah. I say it's almost like ambient CRM basically. How do you make it so that, instead of the user goes to your CRM, the CRM goes to the user where they are. And the outlook app was the beginnings of that. Some of the Team's integrations we've done are the beginnings of that. And that Viva Sales and that whole Viva idea is how do you elevate it? So anywhere you go, your CRM data is accessible without you having to go to a different user interface. Steve: Very cool. Very cool. So I ask you every time we get on a call about exciting features that are coming up. And in particular, maybe even some features that have launched, that didn't take off the way you thought they would and people are just missing something. We have this problem with our app sometimes, people don't understand and so they don't move forward, and it would be perfect for them. And I'm sure there's lots of features and capabilities that you guys broke a sweat building, and know in your heart, this would be awesome, but people don't seem to be getting that. What's a good example of one of those? Charles: I'd say a product which we've had a capability, where we've had a lot of customer usage from a small number of customers, but very deeply and with huge impact, and we wish were with more customers, is probably Conversation Intelligence. I'm not sure if you've seen that around the Sales app, and where that actually will sit in inside of say a phone call or a meeting and help you generate action items, and summaries, and coaching, and help you understand sentiment, and listening and talk ratio. We've used that internally at Microsoft with great success. So our digital sales reps and the folks who work our phones, they are diehard fans. We have this amazing video we released a couple months ago where we actually went out and interviewed these digital sales reps and their managers, and they just were going on and on about how great it is. Charles: And that's rare where you hear that about a piece of technology for a seller. And we have a few other external customers that have gone through that same journey, where they have a thousand digital reps, 2000 digital reps using this and just in love with it. But it's not as pervasive as we thought it would be at this point. And it's one of those things where, it's a product discovery, and easing people into the capability, because then you got to go out of your way to enable it and configure it. So we're doing work now to simplify it, and make it more accessible to more users. And we're doing that partly through Viva Sales, like conversation intelligence, the major capability of Viva Sales. Charles: And the second thing is also, there's even some culture aspects to it. Because if you use it, it's generating transcripts and recordings of a call, and not everyone's necessarily super comfortable with that. So we're even working about how do you enable more features without having to record the call, and how do you enable capabilities without having to get a transcript? Or how do you make it more natural to say, "Hey, I have a sales co-pilot thing. Are you okay if I enable it?" So there's a lot of interesting things, it's never just a technology problem. It's also a discovery and a, I'd say, change culture management problem. Steve: Yeah. I think that's been the challenge with anything AI really. A lot of people, it seem to think it might be a little too futuristic. They look at the benefit and think that's really cool, but they have no idea how to get it. And AI just in general, doesn't feel that approachable to people, even though in certain cases, it's extremely approachable. You don't have to do anything, it's approaching you. So it's a learning curve, you got to wait until my generation dies off and then you guys will see. Charles: I don't have as myopic of you, as you Steve. But I would say that, the big thing that we have to do is, there's been this evolution of AI where the AI is going to be something that automates away what humans do. And what we've realized is, AI is not even remotely close to being able to do that. But what AI can do, is it can turbocharge the people that use it. And so what we're trying to do is, how do we go expose these AI capabilities in a way where you or anyone else who uses them feels so much more productive. And just like when you first got the ability to use PC or a spreadsheet, you're like, "How did I exist before?" We're hoping we'll get to the point where, once you start using some of these AI assistive capabilities, like we've done in Conversation Intelligence, you'll be like, "How did I ever do a customer call before? And I had to take notes on paper while listening as opposed to having the AI take notes for me?" Yeah, exactly. Steve: I'm terrible about that. I'll be chicken scratching over here while I'm talking to people, and then we get off the phone I look at and I can't understand a word I wrote. Charles: Yeah. I like post-it notes next to my desk where I'm always writing stuff down. Steve: Yeah. So what else cool's coming on the horizon that we should be... That sounds like the Conversational Intelligence has been around. Sounds like Viva Sales is going to really bring that to the masses, so that one's on a path. What are some other new things that we should pay attention to that you're able to talk about? Charles: Yeah. Another one of my favorite things, which we've started to reveal some capabilities going back to last Ignite, so November of 2021. And we have some big announcements planned for the second half of 2022, is the new Contact Center related capabilities inside of Dynamics Customer Service. We have Omnichannel, we announced integrated voice, the Nuance acquisition closed, and the Nuance contact center AI team joined my group to align with customer service and contact center. So there's a lot of really exciting innovation happening there. And I'm really excited about the potential to make it super easy to get a comprehensive customer engagement story, without having to wire up eight different pieces of technology and do a ton of different complex integrations. So that's a place where there's a lot of innovation, there's new capabilities, Omnichannel, Power Virtual Agent, even the same type of conversation intelligence applied to support cases, Nuance for their Gatekeeper, which is identity and authentication verification based on voice and biometrics. Charles: There's a lot of cool stuff in that space. And that's one of the places where so many of the customers we work with are trying to improve the customer experience, and to go reduce costs. So I say that's a place where we've had a lot of exciting announcements over the last six to nine months, and we have a whole bunch more planned for the next six to nine months. So I say, stay tuned. And I won't say more than that to avoid getting in trouble by leaking information. But I just say, that's a place to really pay close attention. Steve: Who knew call centers could be cool? Charles: Yeah, exactly. Who would have thought that I'd be talking about contact centers, and how it's the next generation or next frontier of AI applications in 2022. Steve: Oh, well. Well I do have to thank you guys for the low-code advances you've continued to make in that platform. It actually allowed us to launch a, I think we're the first ones to try this, a new Service as a Subscription. Which includes awesome includes deployment, customization, training, everything except development code, which as you know today in so many of these projects, there's so little, if any of that. Charles: Yeah. Steve: Just a few years ago, if you tried to offer something like this, it really would be little more than a support agreement. But now, we're deploying, we're building, we're customizing, we're building entire things for customers all on a monthly subscription. It's an interesting concept, and hopefully I don't go broke, but... Charles: But you know what, it's fascinating. I literally was talking about this with the Power Platform team this morning. About a future where we'll have more partners who are able to sell a comprehensive service agreement, which includes the cloud hosting licenses, but also some incremental custom development and also ongoing maintenance and support. And it'll be almost this whole new industry, which will push a lot of innovation to the edges of the ecosystem, right? Steve: Yep. Charles: Not built by Microsoft, built by partners who really understand particular regions, particular industries, or particular segments. Like y'all are targeting a space where we're not trying to go take Dynamics, CRM, and go bring it down there. You can go build a world-class experience on top of our platform and provide a very much all-in-one, which exactly serves the needs of that audience and that market. And we can stay focused on building the super horizontal platform, which has great performance, great usability, incredible power, those types of things. Steve: Yeah, it sounds great. I'm glad that we had the same idea you guys did. I'll let you know, in a few months, if it was a smart one. Time will tell. Charles: Yes. Yeah. Steve: So, how are the rest of the team doing? It seems like some folks have moved around a little bit in the org, who's moved where? Charles: Yeah. So one of the big things we've been really focused on the engineering side, for the engineering organization, is bringing together strength from a product perspective that target the same type of user. And for example, we have a new customer experience platform team underneath Lori Lamkin, who leads all of our Dynamic Sales apps. So the Core Sales and Viva Sales, as well as commerce, as well as marketing, as well as customer insights. And it's very much focused on revenue generation, customer journeys, customer experiences. And what's great is by bringing those assets together, we have a great answer for B2B customers, as well as B2C. Like if you want to have self service, no touch eCommerce experience with lightweight telesales, you can do that all with those sets of applications. If you want to do a high relationship, high touch B2B sales process, you can do all of that. You're not going to use commerce, but you're probably going to use customer insights and sales, and maybe a little bit of account-based marketing. So we brought together these things, which are solving similar problems under a single leader. And that way the engineering teams can go back and forth between these different places to finish out full end-to-end customer journeys. And so that's a big area that we've spent a lot of time on, and that's a place where it's really the biggest and fastest growing category for us in the Dynamics 365 application portfolio. So that's one interesting example. Jeff Comstock, folks may know him. He's been around Dynamics 365 for a while. He continues customer service, he leads omnichannel, he's done some of this great expansion around the contact center for us. Ray Smith leads our supply chain team. So that includes things like more supply chain. Steve: So Ray moved? Charles: Yeah, yeah, yeah. He by way of acquisition to SAP then moved. He worked in Dynamic Sales for a bit, where people may have known him. And now the supply chain, and really helping us be this new data driven, AI powered, supply chain story for core supply chain execution. Then we also had some exciting announcements around process advisor and the minor acquisition to help turbocharge that. Or Georg Glantschnig who leads our finance room of the house. And basically we call the room of the house, is the collection of products which focus on serving the CFO and the finance department. And that includes the Suplari acquisition, which we had done a couple years ago, as well as the Core Dynamics, 365 finance, HR, and project operations products. Charles: So you can see how we started to build these critical paths around particular departments and particular lines of businesses with our products. And in addition to that, we also of course have Power Platform to support all of it. So it's amazing to see these things come together and converge. And we've been on this incredible run of innovation around Dynamics. I was counting it earlier this year, 29 different products in Dynamics, and really coalesced around these specific areas where we have a lot of energy, and also very well understood. I'd say synergies between the products that we have. So I'd say exciting times. Very exciting times. Steve: Customers are starting to understand it better also. Business Applications was the same thing for a long time. Then it spent the last five years reinventing itself every month, and new things exploding out of Advanta. And I think a lot of customers were having trouble just keeping up with... It's like little whackamole for them. And it takes a little time for customers to absorb what's happening, and what it's for, or what it does, and then to adopt it. And we're seeing that now. We used to have to go out and promote Power Apps to people who didn't understand what this was, or why it was. And now it's the opposite. They always come to us, looking for Power Apps, looking at those sorts of things. So that understanding seems to have finally permeated down to the customer level. But boy, it took a while. Charles: Yeah. It warms my heart. And I would say one of my favorite books is by Jim Collins, 'Good to great.' I always recommend it to folks on my team to read it. And he talks about this idea of the flywheel. It takes time to get a flywheel spinning, for the first period of time it looks like it's barely moving, but then eventually it's going super fast and it's just a blur. And you need to be consistent, and convicted, and believe in the strategy and the approach. And what's amazing about BizApps is for the last four years, we've been on the same mission, the same vision, the same ambition. And we just spend all the folks in advance at turning that flywheel, turning that flywheel. And it's started to reach that blur phase where it's spinning so fast, you can't even see it. Charles: And this, this all started years and years ago with a ton of work, but we're really at that magical moment where customers know what Power Platform is. Customers know that Microsoft gets customer experience and customer engagement. They know that Microsoft can help them optimize their supply chain. And what the good news is once that thing is going, it really builds upon itself, and I think it'll only continue that momentum further. And my favorite story is, I used to always do these executive briefings at Microsoft where we have executives come in from our customers to Redmond and we have a briefing center. It's very nice. And I would always say, let me talk about Power Apps and low-code. Charles: And everybody gives me a blank stare like, "What the heck is Power Apps? What the heck is low-code?" I go in those meetings now, and people know what Power Apps is, and they know the low-code strategy. And the only question is, "how?". Not, "should I?" Or "if?" "How do I do it with you, Microsoft?" And so different from three years ago. So anyway, so you're exactly right. A long winded answer, but I'd say it's exciting to see all of these things come together, and the benefits of just consistently repeating a message that resonates with customers. Steve: I would say at least three quarters of my customer calls today, they're bringing up right out of the gate, "We don't want any development. We want to do everything low-code, no code." So this is coming from the customer side where we used to have to explain to them what low-code, no code meant. Now they're coming demanding, "I only want low-code, no code." I think that they've come to this realization that, while low-code, no code might not be easy enough for your mom to do, it doesn't require a developer, and code does require developer. And once you've got this little blob of code in your environment, it's a black box for you. And so they don't want any of these black boxes. They want everything to be accessible. Steve: Use your knowledge to build us something complex out of low-code, but then I can still go back in there later and manipulate it, adjust it myself, or our team. So they have absolutely bought into that. And I know we originally, a lot of us partners were concerned early on that this was going to reduce the workload for partners, while our workload is more than it has ever been. Although the developers on the bench don't stay as busy as they used to. We've completely pivoted the team from developer heavy to now, we haven't even got a good title for them. A citizen developer doesn't sound right. We tell customers that, but citizen developers is what we've got so... Charles: This guy we found on the street, or gal found on the street, we just asked them to start building out. But no, it makes sense. There is almost this new role which is, it's not just pure coding expertise, it's technical development concept expertise. But even more importantly is business process and solution expertise. And that fusion of those two skill sets, that's the magic. That's what makes it special, because you understand it. Steve: Yeah. The challenge that we have with this brand new model that we just launched, because, first of all, being the first one out there is not always good because people have no idea what you're talking about. They're trying to compare it to other things. But we've got this little caveat that it's all you can eat, everything, except development code. And trying to define what that is hasn't been easy, and you get these customers coming in, "Oh, we're going to need a lot of customization. So this isn't going to work for us." And so you may need a lot of customizations, but you don't need any "development code". Charles: Yeah. Steve: And getting them to grasp that development code and customization are not synonymous, not even close. Charles: Exactly. Steve: Development code is a very small component today of customization. And once I think that they understand that, then we'll probably see more partners coming into a model like this. Because it makes a lot of sense for customers, makes a lot of sense for partners. Charles: Yeah. And if you go look at building solutions that last a decade, this is to your point, code is this little black box opaque thing, which is hard to maintain over time. If it's no code, low-code, it's easy to open it up and reconfigure as business requirements change. And it's how you build solutions that last. And I think we're getting to the phase with business software where customers are expecting to make long term technology bets. You're not going to replace your CRM every five years from now on. It's like building manufacturing plants and warehouses. These are big investments that you need to be able to amortize over a long time, to justify. And so I think to your point, no code doesn't mean no flexibility, no customization, also doesn't mean no agility. It just means you're doing it in a different way. Couldn't say it better myself. Steve: All right. Cool. Hey, listen, I'm going to let you go. I really appreciate you taking the time out of your day here when I caught you, to chat with me about this stuff, always fun talking to you Charles. I'm going to call you in four months and ask you about Viva Sales for the platform. Charles: Sounds good. Sounds good. Steve: I've got you on record there. Charles: So really appreciate you taking the time, giving me a ring, Steve. Hope you have a great rest of the summer. Steve: All right, man. Have a good one. Charles: Yep. You too.  

In this episode I give a quick recap of my interview with Dr. Charles Karuku including: - What it was like growing up in Kenya and how he came to the U.S. with only $200 in his pocket- How to discover your purpose- The 4 roles of a man and why they are important And so much more...  Dr. Charles Karuku has been in ministry since 1991. He is also a seasoned apostle to the nations. He's a mentor, trainer and speaker whose greatest passion is to equip others for ministry. He is the president of the Unity Movement the vision of which is to unite communities and offer hope that leads to city transformation. He has traveled to over 135 cities across America speaking this message of hopeClick Here to connect with Charles*************************************************************You will never maximize your potential on your own so I'm personally inviting you to come and join me in the private Extraordinary Man Facebook group so you can level up your business and your life. Just Click Here to join the Extraordinary Man private Facebook group. Iron sharpens iron and this is the #1 place for you to connect with me and other like minded men who are on a mission to maximize their potential. My goal is to help you become the man God created you to be in all areas of your life. So come and join us in the Facebook group and upgrade your business and your life.

Dr. Charles Karuku has been in ministry since 1991. He is also a seasoned apostle to the nations. He's a mentor, trainer and speaker whose greatest passion is to equip others for ministry. He is the president of the Unity Movement the vision of which is to unite communities and offer hope that leads to city transformation. He has traveled to over 135 cities across America speaking this message of hopeIn this episode, we discuss:- What it was like growing up in Kenya and how he came to the U.S. with only $200 in his pocket- How to discover your purpose- The 4 roles of a man and why they are important And so much more...  Click Here to connect with Charles**************************************************You will never maximize your potential on your own so I'm personally inviting you to come and join me in the private Extraordinary Man Facebook group so you can level up your business and your life. Just Click Here to join the Extraordinary Man private Facebook group. Iron sharpens iron and this is the #1 place for you to connect with me and other like minded men who are on a mission to maximize their potential. My goal is to help you become the man God created you to be in all areas of your life. So come and join us in the Facebook group and upgrade your business and your life.

Rich Harris talks about Svelte and Reactivity. Rich Harris: Graphics Editor on The New York Times investigations team. Resources: Svelte Please join us in these conversations! If you or someone you know would be a perfect guest, please get in touch with us at contact@frontside.io. Our goal is to get people thinking on the platform level which includes tooling, internalization, state management, routing, upgrade, and the data layer. This show was produced by Mandy Moore, aka @therubyrep of DevReps, LLC. Transcript: CHARLES: Hello and welcome to The Frontside Podcast, a place where we talk about user interfaces and everything that you need to know to build them right. TARAS: It's actually a really nice, Rich and I'm really, really happy to have a chance to actually chat with you about this because Svelte is a really fun piece technology. In many ways, it's interesting to see our technology evolve and our industry evolve through innovation, real innovation. I think Svelte 3 has really been kind of that next thought provoking technology that kind of makes you think about different ways that we can approach problems in our space. So, really excited to chat with you about this stuff. RICH: Well, thank you. Excited to be here. TARAS: I think quite a lot of people know, Rich, about your history, like how you got into what you're doing now. But I'm not sure if Charles is aware, so if you could kind of give us a little bit of a lowdown on where you kind of come from in terms of your technical background and such. RICH: Sure. I'll give you the 30-second life history. I started out as a reporter at a financial news organization. I had a Philosophy Degree and didn't know what else to do with it. So, I went into journalism. This was around the time of the great recession. And within a few weeks of me joining this company, I watched half of my colleagues get laid off and it's like, "Shit, I need to make myself more employable." And so gradually, sort of took on more and more technical responsibilities until I was writing JavaScript as part of my day job. Then from there, all these opportunities kind of opened up. And the big thing that I had in mind was building interactive pieces of journalism, data-driven, personalized, all of that sort of thing, which were being built at places like the New York Times, and The Guardian, and the BBC. That was the reason that I really wanted to get into JavaScript. And that's guided my career path ever since. CHARLES: It's interesting that this D3 and all that did come out of journalism. RICH: It's not a coincidence because when you're working under extreme time pressure and you're not building things with a view to maintain them over a long period of time, you just need to build something and get it shipped immediately. But it needs to be built in a way that is going to work across a whole range of devices. We've got native apps, we've got [inaudible], we've got our own website. And in order to do all that, you need to have tools that really guide you into the pit of success. And D3 is a perfect example of that. And a lot of people have come into JavaScript through D3. CHARLES: And so, are you still working for the same company? RICH: No. That's ancient history at this point. CHARLES: Because I'm wondering, are you actually getting to use these tools that you've been building to actually do the types of visualizations and stuff that we've been talking about? RICH: Very much so. I moved to The Guardian some years ago. And then from there, moved to Guardian US, which has an office in New York. And it was there that I started working on Svelte. I then moved to the New York Times and I'm still working on Svelte. I've used it a number of times to build things at the New York Times and the people have built things with it too. And so, yeah, it's very much informed by the demands of building high performance interactive applications on a very tight deadline. CHARLES: Okay, cool. So I've probably used, I mean, I'm an avid reader of both Guardian and the New York Times, so I've probably used a bunch of these visualizations. I had no idea what was driving them. I just assumed it was all D3. RICH: There is a lot of D3. Mike Bostock, the creator of D3, he was a linchpin at the graphics department for many years. Unfortunately we didn't overlap. He left the Times before I joined the Times, but his presence is still very much felt in the department. And a lot of people who are entering the industry, they're still becoming database practitioners by learning from D3 examples. It's been a hugely influential thing in our industry. TARAS: How long is a typical project? How long would it take to put together a visualization for an article that we typically see? RICH: It varies wildly. The graphics desk is about 50 strong and they will turn around things within a day. Like when the Notre Dame burnt down a couple of months ago, my colleagues turned around this interactive scroll driven webGL 3D reconstruction of how the fire spreads through the cathedral in less than 24 hours, which was absolutely mind blowing. But at the same time, there are projects that will take months. I work on the investigations team at the Times. And so, I'm working with people who are investigating stories for the best part of the year or sometimes more. And I'm building graphics for those. And so that, it's two very different timescales, but you need to be able to accommodate all of those different possibilities. CHARLES: So, what does the software development practice look like? I mean, because it sounds like some of this stuff, are you just throwing it together? I guess what I mean by that is, I guess the projects that we typically work on, three months is kind of a minimum that you would expect. So, you go into it, we need to make sure we've got good collaboration practices around source control and continuous integration and testing and all this stuff. But I mean, you're talking about compressing that entire process into a matter of hours. So what, do you just throw right out the window? What do you say? "We're just doing a live version of this." RICH: Our collaboration processes consist of sitting near each other. And when the time calls for it, getting in the same room as each other and just hammering stuff out on the laptop together. There's no time for messing around with continuous integration and writing tests. No one writes tests in the news graphics, it's just not a thing. CHARLES: Right. But then for those projects that stretch into like three months, I imagine there are some. Do you run into like quality concerns or things like that where you do have to take into account some of those practices? I'm just so curious because it sounds like there's actually, the difference between two hours and two months is, that's several orders of magnitude and complexity of what you're developing. RICH: It is. Although I haven't worked on a news project yet that has involved tests. And I know that's a shocking admission to a lot of people who have a development background, but it's just not part of the culture. And I guess the main difference between the codebase for a two-hour project and a two-month project is that the two-month project will strive to have some reasonable components. And that's, I think, the main thing that I've been able to get out of working on the kinds of projects that I do is instead of just throwing code at the page until it works, we actually have a bit of time to extract out common functionality and make components that can be used in subsequent interactives. So, things like scroll driven storytelling, that's much easier for me now than it was when I first built a scroll driven storytelling component like a couple of years ago. CHARLES: Yeah. That was actually literally my next question is how do you bridge that, given that you've got kind of this frothy experimentation, but you are being, sounds like, very deliberate about extracting those tools and extracting those common components? And how do you find the time to even do that? RICH: Well, this is where the component driven mindset comes in really handy, I think. I think that five or 10 years ago when people thought in terms of libraries and scripts, there wasn't like that good unit of reusability that wasn't the sort of all encompassing, like a component is just the right level of atomicity or whatever the word is. It makes sense to have things that are reusable but also very easy to tweak and manipulate and adapt to your current situation. And so, I think that the advent of component oriented development is actually quite big for those of us working in this space. And it hasn't really caught on yet to a huge degree because like I say, a lot of people are still coming with this kind of D3 script based mindset because the news industry, for some interesting and historical reasons, is slightly out of step with mainstream mode development in some ways. We don't use things like Babel a lot, for example. CHARLES: That makes sense, right? I mean, the online print is not like it's a React application or it's not like the application is all encompassing, so you really need to have a light footprint, I would imagine, because it really is a script. What you're doing is scripting in the truest sense of the word where you essentially have a whole bunch of content and then you just need to kind of -- RICH: Yeah. And the light footprint that you mentioned is key because like most new sites, we have analytics on the page and we have ads and we have comments and all of these things that involve JavaScript. And by the time our code loads, all of this other stuff is already fighting for the main thread. And so, we need to get in there as fast as we can and do our work with a minimum fuss. We don't have the capacity to be loading big frameworks and messing about on the page. So that again is one of these sort of downward pressures that kind of enforces a certain type of tool to come out of the news business. TARAS: A lot of the tooling that's available, especially on like the really fatter, bigger frameworks, the tools that you get with those frameworks, they benefit over long term. So if you have like a long running project, the weight of the abstractions, you've experienced that benefit over time and it adds up significantly. But if you're working to ship something in a day, you want something that is just like a chisel. It does exactly what you want it to do. You want to apply it in exactly the right place and you want to get it done exactly, like you want the outcome to be precise. RICH: That's true. And I think a lot of people who have built large React apps, for example, or large Ember apps, they sort of look at Svelte and think, "Well, maybe this isn't going to be applicable to my situation," because it has this bias towards being able to very quickly produce something. And I'm not convinced that that's true. I think that if you make something easier to get started with, then you're just making it easier. If you build something that is simple for beginners to use, then you're also building something simple for experts to use. And so, I don't necessarily see it as a tradeoff, I don't think we're trading long-term maintainability for short term production. But it is certainly a suspicion that I've encountered from people. TARAS: This is something that we've also encountered recently. It's been kind of a brewing discussion inside a front side about the fact that it seems to be that certain problems are actually better to rewrite than they are to maintain or refactor towards an end goal. And we found this, especially as the tools that we create have gotten more precise and more refined and simplified and lighter, it is actually easier to rewrite those things five times than it is to refactor it one time to a particular place that we want it to be. And it's interesting, like I find this to be very recent, this idea is blossoming in my mind very recently. I didn't observe this in the past. CHARLES: Do you mean in the sense that like if a tool is focused enough and a tool is simple enough, then refactoring is tantamount to a rewrite if you're talking about 200 or 300 lines of code? Is that what you mean? TARAS: Yeah. If you're sitting down to make a change or you have something in mind, it is actually easy to say, "Let's just start from scratch and then we're going to get exactly the same place in the same amount of time." But this kind of mantra of not rewriting makes me think about that, makes me question whether that's actually something that is always the right answer. RICH: I definitely question that conventional wisdom at all levels, as well. I started a bundler called Rollup as well as Svelte more recently. And Rollup was the second JavaScript bundler that I wrote, because the first one that I wrote wasn't quite capable of doing the things that I wanted. And it was easier to just start from scratch than to try and shift the existing user base of its predecessor over to this new way of doing things. Svelte 3 is a more or less complete rewrite. Svelte has had multiple, more or less, complete rewrite. Some of them weren't breaking changes. But Svelte itself was a rewrite of an earlier project that I'd started in 2013. And so in my career, I've benefited massively from learning from having built something. But then when the time comes and you realize that you can't change it in the ways that you need to change it, just rewrite it. And I think that at the other end of the spectrum, the recent debate about micro frontend has largely missed this point. People think that the benefit of the micro frontend is that people don't need to talk to each other, which is absolute nonsense. I think the benefit of this way of thinking about building applications is that it optimizes for this fact of life that we all agree is inevitable, which is that at some point, you're going to have to rewrite your code. And we spend so much energy trying to optimize for the stability of a code base over the long term. And in the process, lock ourselves into architectural and technical decisions that don't necessarily make sense three or four years down the line. And I think as an industry, would be a lot better placed if we all started thinking about how to optimize for rewrites. CHARLES: So for those of us who aren't familiar, what is the debate surrounding micro frontends? This is actually something I've heard a lot about, but I've actually never heard what micro frontends actually are. RICH: Yeah. I mean, to be clear, I don't really have a dog in this fight because I'm not building products, but the nub of it is that typically if you're building a website that maybe has like an admin page, maybe it has a a settings page, maybe it has product pages, whatever. Traditionally, these would all be parts of a single monolithic application. The micro frontend approach is to say, "Well, this team is going to own the settings page. This team is going to own the product page." And they can use whatever technologies they want to bring that about. And the detractors sort of attack a straw man version of this, "You're going to have different styles in every page. You're going to have to load Vue on one page. You're going to have to load React on the other page. It's going to be a terrible user experience," when actually its proponents aren't suggesting that at all. They're suggesting that people from these different teams coordinate a lot more that are free to deviate from some kind of grand master architectural plan when it's not suitable for a given task. And darn right. I think it means that you have a lot more agility as an engineering organization than you would if you're building this monolithic app where someone can't say, "Oh, we should use this new tool for this thing. We should use microstates when the rest of the organization is using Google docs." It's not possible. And so, you get locked into the decisions of a previous generation. CHARLES: Right. No, it makes sense. It's funny because my first reaction is like, "Oh my goodness, that's a potential for disaster." The klaxon's going to go off in your head, but then you think, really then the work is how do you actually manage it so it doesn't become a disaster. And if you can figure that out, then yeah, there is a lot of potential. RICH: Yeah. People always try and solve social problems with technology. You solve social problems with social solutions. CHARLES: Right. And you have to imagine it too, it depends on the application, right? I think Amazon, the Amazon website is developed that way where they have different teams that are responsible even down to little content boxes that are up on the toolbar. And the site doesn't really, it shows, right? Like it shows like this is kind of like slapped together, but that's not what they need. They don't need it to not look like there's slight variation with the different ways that things behave. They need to be showing for their business to work. They need to be showing the right thing at the right time. And that's the overriding concern. So having it look very beautiful and very coherent isn't necessarily a thing. Same thing in Spotify, used as another example of this. I didn't know if it was called micro frontends, but I know that they've got a similar type thing, but they are clearly the experience and having it look coherent is more important. And so, they make it work somehow. And then like you're saying, it probably involves groups of people talking to other groups of people about the priorities. So yeah, it doesn't sound to me like just like you're going to adopt micro frontends guarantees one particular set of outcomes. It really is context dependent on what you make of it. RICH: Totally. TARAS: I'm curious though, so with Svelte, essentially for your reactivity engine, you have to compile to get that reactive behavior. RICH: Yeah. TARAS: How does that play with other tools like when you actually integrate it together? I've never worked with Svelte on a large project, so I can't imagine what it looks like at scale. I was wondering if you've seen those kind of use cases and what that ends up, if there's any kind of side effects from that. RICH: As you say, the reactivity within a component is only in the local state within that component or to state that is patched in as a prop from a parent component. But we also have this concept called a store. And a store is just a project that represents a specific value and you import it from svelte/store. And there are three types of store that you get out of the box. A writable, a readable and a derived. And a writeable is just, var count = writable (0) and then you can update that and you can set it using methods on that store. Inside your marker, you can reference or in fact inside the script block in the component, you can reference the value of that store just by prefacing it with a dollar sign. And the compiler sees that and says, "Okay, we need to subscribe to this store as value and then assign it and apply the reactivity." And that is the primary way of having state that exists outside the component hierarchy. Now, I mentioned the writable, readable, and derived are the built in stores that you get, but you can actually implement your own stores. You just need to implement this very simple contract. And so,, it's entirely possible to use that API to wrap any state management solution you have. So you can wrap redux, you can wrap microstates, you can wrap state, you can wrap whatever it is, whatever your preferred state management solution is, you can adapt it to use with Svelte. And it's very sort of idiomatic and streamlined. Like it takes care of unsubscriptions when the component is unmounted. All of that stuff is just done for you. CHARLES: Digging a little bit deeper into the question of integration, how difficult would it be to take wholesale components that were implemented in Svelte and kind of integrate them with some other component framework like React? RICH: If the component is a leaf node, then it's fairly straightforward. There is a project called react-svelte which is, I say project, it's like 20 lines of code and I don't think it's [inaudible] they did for Svelte 3, which I should probably do. But that allows you to use a Svelte component in the context of React application, just using the component API the same way that you would [inaudible] or whatever. You can do that inside a React component. Or you could compile the Svelte component to a web component. And this is one of the great benefits of being a compiler is that you can target different things. You can generate a regular JavaScript class and you've got an interactive application. Or you can target a server side rendering component which will just generate some html for some given state which can then later be hydrated on the client. Or you can target a web component which you can use like any other element in the context of any framework at all. And because it's a compiler, because it's discarding all of the bits of the framework that you're not using, it's not like you're bundling an entire framework to go along with your component. And I should mention while I'm talking about being able to target different outputs, we can also, as a NativeScript project, you can target iOS and Android that same way. Where it gets a little bit more complicated is if it's not a leaf node. If you want to have a React app that contains a Svelte component that has React [inaudible], then things start to get a little bit more unwieldy, I think. It's probably technically possible, but I don't know that I would recommend it. But the point is that it is definitely possible to incrementally adopt Svelte inside an existing application, should that be what you need to do. CHARLES: You said there's a NativeScript project, but it sounds to me like you shouldn't necessarily need NativeScript, right? If you're a compiler, you can actually target Android and you could target iOS directly instead of having NativeScript as an intermediary, right? RICH: Yes. If, if we had the time to do the work, then yes. I think the big thing there would be getting styles to work because Svelte components have styles. And a regular style tag just to CSS and you can't just throw CSS in a native app. CHARLES: Right. Sometimes, I feel like it'd be a lot cooler if you could. [Laughter] RICH: NativeScript really is doing a lot of heavy lifting. Basically what it's doing is it's providing a fake dom. And so, what the NativeScript does is it targets that dom instead of the real dom and then NativeScript turns that into the native instructions. CHARLES: Okay. And you can do that because you're a compiler. TARAS: Compilers has been on our radar for some time, but I'm curious like what is your process for figuring out what it should compile to? Like how do you arrive at the final compile output? Manually, have you written that code and then, "I'm going to now change this to be dynamically generated." Or like how do you figure out what the output should be? RICH: That's pretty much it. Certainly, when the project started, it was a case of, I'm going to think like a compiler, I'm going to hand convert this declarative component code into some framework plus JavaScript. And then once that's done, sort of work backwards and figure out how a compiler would generate that code. And then the process, you do learn certain things about what the points of reusability are, which things should be abstracted out into a shared internal helper library and what things should be generated in line. The whole process is designed to produce output that is easy for a human to understand and reason about. It's not like what you would imagine compile [inaudible] to be like, it's not completely inscrutable. It's designed to be, even to that level of being well formatted, it's designed to be something that someone can look at and understand what the compiler was thinking at that moment. And there's definitely ways that we could change and improve it. There are some places where there's more duplication than we need to have. There are some places where we should be using classes instead of closures for performance and memory benefits. But these are all things that once you've got that base, having gone through that process, that you can begin to iterate on. CHARLES: It's always curious to me about when is the proper time to move to a compiler, because when you're doing everything at runtime, there's more flexibility there. But at what point do you decide, "You know what? I know that these pathways are so well worn that I'm going to lay down pavement. And I'm going to write a compiler." What was the decision process in your mind about, "Okay, now it's time." Because I think that that's maybe not a thought that occurs to most of us. It's like, "I had to write a compiler for this." Is this something that people should do more often? RICH: The [inaudible] of 'this should be a compiler' is one that is worth sort of having at the back of your head. I think there are a lot of opportunities not just in DUI framework space but in general, like is there some way that we can take this work that is currently happening at runtime and shift it into a step that only happens once. That obviously benefits users. And very often we find that benefits developers as well. I don't think there was a point at which I said, "Oh, this stuff that's happening at runtime should be happening at compile time." It was more, I mean, the actual origin has felt that it was a brain worm that someone else infected me with. Judgment is a very well known figure in the JavaScript world. He had been working on this exact idea but hadn't taken it to the point where he was ready to open source it. But he had shared like his findings and the general idea and I was just immediately smitten with this concept of getting rid of the framework runtime. At the time, the big conversation happening in the JavaScript community was about the fact that we're shipping too much JavaScript and it's affecting startup performance time. And so the initial thought was, "Well, maybe we can solve that problem by just not having the runtime." And so, that was the starting point with Svelte. Over time, I've come to realize that that is maybe not the main benefit. That is just one of the benefits that you get from this approach. You also get much faster update performance because you don't have to do this fairly expensive virtual dom different process. Lately, I've come to think that the biggest win from it is that you can write a lot less code. If you're a compiler, then you're not kind of hemmed in by the constraints of the language, so you can almost invent your own language. And if you can do that, then you can do the same things that you have been doing with an API in the language itself. And that's the basis of our system of reactivity, for example. We can build these apps that are smaller and by extension, less bug prone and more maintainable. I just wanted to quickly address the point you made about flexibility. This is a theoretical downside of being a compiler. We're throwing away the constraints about the code needing to be something that runs in the browser, but we're adding a constraint, which is that the code needs to be statically analyzable. And in theory, that results in a loss of flexibility. In practice, we haven't found that to affect the things that we can build. And I think that a lot of times when people have this conversation, they're focusing on the sort of academic concepts of flexibility. But what matters is what can you build? How easy is it to build a certain thing? And so if empirically you find that you're not restricted in the things that you can build and you can build the same things much faster, then that academic notion of flexibility doesn't, to my mind, have any real value. CHARLES: Hearing you talk reminded me of kind of a quote that I heard that always stuck with me back from early in my career. I came into programming through Perl. Perl was my first language and Perl is a very weird language. But among other things, you can actually just change the way that Perl parses code. You can write Perl that makes Perl not throw, if that makes any sense. And when asked about this feature, the guy, Larry Wall, who came up with Perl, he's like, "You program Perl, but really what you're doing is you're programming Perl with a set of semantics that you've negotiated with the compiler." And that was kind of a funny way of saying like, "You get to extend the compiler yourself." Here's like the default set of things that you can do with our compiler, but if you want to tweak it or add or modify, you can do that. And so, you can utilize the same functionality that makes it powerful in the first place. You can kind of inject that whole mode of operation into the entire workflow. Does that make sense? That's like a long way of saying, have you thought about, and is it possible to kind of extend the Svelte compiler as part of a customization or as part of the Svelte programming experience? RICH: We have a very rudimentary version of that, which is pre-processing. There's an API that comes with Svelte called preprocess. And the idea there is that you can pass in some code and it will do some very basic, like it will extract your styles, it will extract your script and it will extract your markup. And then it will give you the opportunity to replace those things with something else. So for example, you could write some futuristic JavaScript and then compile it with Babel before it gets passed to the Svelte compiler, which uses acorn and therefore needs to be able to have managed other scripts so that it can construct an abstract syntax tree. A more extreme version of that, people can use [inaudible] to write their markup instead of html. You can use Sass and Less and things like that. Generally, I don't recommend that people do because it adds these moving parts and it makes like a lot of bug reports of people just trying to figure out how to get these different moving parts to operate together. I don't know, it means that your editor plugins can't understand what's inside your style tag all of a sudden and stuff like that. So, it definitely adds some complexity, but it is possible. At the other end, at a slightly more extreme level, we have talked about making the cogeneration part plugable so that for example, the default renderer and the SSR renderer are just two examples of something that plugs into the compiler that says, "Here is the component, here's the abstract syntax tree, here's some metadata about which values are in scope," all of this stuff and then go away and generate some code from this. We haven't done that so far, partly because there hasn't been a great demand for it, but also because it's really complicated. As soon as you turn something into a plugin platform, you just magnify the number of connection points and the number of ways that things could go wrong by an order of magnitude. And so, we've been a little bit wary of doing that, but it is something that we've talked about primarily in the context of being able to do new and interesting things like target webGL directly or target the command line. There are renders for React that let you build command line apps using React components. And like we've talked about, maybe we should be able to do that. Native is another example. The NativeScript integration as you say, it could be replaced with the compiler doing that work directly, but for that to work presently, that would mean that all of that logic would need to sit in core. And it would be nice if that could be just another extension to the compiler. We're talking about a lot of engineering effort and there's higher priority items on our to do list at the moment. So, it's filed under one day. CHARLES: Right. What are those high priority items? RICH: The biggest thing I think at the moment is TypeScript integration. Surprisingly, this is probably like the number one feature request I think is that people want to be able to write Typescript inside the Svelte components and they want to be able to get TypeScript when they import the Svelte component into something else. They want to be able to get completion [inaudible] and type checking and all the rest of it. A couple of years ago, that would've been more or less than thinkable but now it's like table stakes is that you have to have first-class TypeScript support. CHARLES: Yeah, TypeScript is as popular as Babel these days, right? RICH: Yeah, I think so. I don't need to be sold on the benefits. I've been using TypeScript a lot myself. Svelte is written in TypeScript, but actually being able to write it inside your components is something that would involve as hacking around in the TypeScript compiler API in a way that, I don't know if anyone actually or any of us on the team actually knows how to do. So, we just need to spend some time and do that. But obviously when you've got an open source project, you need to deal with the bugs that arise and stuff first. So, it's difficult to find time to do a big project like that. CHARLES: So, devil's advocate here is if the compiler was open for extension, couldn't a TypeScript support be just another plugin? RICH: It could, but then you could end up with a situation where there's multiple competing TypeScript plugins and no one's sure which ones are used and they all have slightly different characteristics. I always think it's better if these things that are common feature requests that a lot of people would benefit from, if they're built into the project themselves. I go really light in the batteries included way of developing and I think this is something that we've sort of drifted away from in the frontend world over the last few years, we've drifted away from batteries included towards do it yourself. CHARLES: Assemble the entire thing. Step one, open the box and pour the thousand Lego pieces onto the floor. RICH: Yeah, but it's worse than that because at least, with a Lego set, you get the Lego pieces. It's like if you had the Lego manual showing you how to build something, but you were then responsible for going out and getting the Lego pieces, that's frontend development and I don't like it. CHARLES: Right. Yeah. I don't like that either. But still, there's a lot of people advocating directly. You really ought to be doing everything completely and totally yourself. RICH: Yes. CHARLES: And a lot of software development shops still operate that way. RICH: Yeah. I find that the people advocating for that position the most loudly, they tend to be the maintainers of the projects in question. The whole small modules philosophy, they exist for the benefit primarily of library authors and framework authors, not for the benefit of developers, much less users. And the fact that the people who are building libraries and frameworks tend to have the loudest megaphones means that that mindset, that philosophy is taken as a best practice for the industry as a whole. And I think it's a mistake to think that way. TARAS: There is also, I think, a degree of a sliding scale where you start off with like as the more experience you get, because there is more experience you get closer, you get to that kind of wanting granular control and then they kind of slides down towards granular control and then slice back up to, once you've got a lot of experience, you're like, "Okay, I don't want this control anymore." And then you kind of cast that and you get into like, "I'm now responsible for tools that my team uses," and now you're back to wanting that control because you want things to be able to click together. It's kind of like a way that your interest in that might change over time depending on your experience level and your position in the organization. So yeah, there's definitely different motivating factors. Like one of the things that we've been thinking a lot about is designing tools that are composable and granular at individual module level, but combined together into a system for consumption by regular people. So like finding those primitives that will just click together when you know how to click them together. But when you're consuming them, just feel like a holistic whole, but at the same time not being monolithic. That's a lot of things to figure out and it's a lot of things to manage over time, but that's solely the kind of things we've been thinking about a lot. RICH: I think that's what distinguishes the good projects that are going to have a long lifespan from the projects that are maybe interesting but don't have a long shelf life is whether they're designed in such a way that permits that kind of cohesion and innovation tradeoff, if you think of it as a trade off. Anyone can build the fastest thing or the smallest thing or the whatever it is thing. But building these things in a way that feels like it was designed holistically but is also flexible enough to be used with everything else that you use, that's the real design challenge. CHARLES: It's hard to know where to draw that line. Maybe one good example of this and, these are actually two projects that I'm not particularly a fan of, but I think they do a good job of operating this way. So, I guess in that sense, it means I can even be more honest about it. I don't particularly care for Redux or like observables, but we ended up using, in one of our last React projects, we had to choose between using Redux-Saga and Redux-Observable. The Redux-Observable worked very well for us. And I think one of the reasons is because they both had to kind of exist. They had to kind of co-exist is their own projects. Like Redux exists as its own entity and Observables exist as their own kind of whole ecosystem. And so, they put a lot of thought in like what is the natural way in which these two primitives compose together? As opposed to the Saga, which I don't want to disparage the project because I think it actually is a really good project. There's a lot of really good ideas there but because it's more like just bolted on to Redux and it doesn't exist outside of the ecosystem of Redux and the ideas can't flourish outside and figure out how it interfaces with other things. Like the true primitive is still unrevealed there. And so, whereas I feel like with Redux you actually have to really, really true primitives. Now, they're not necessarily my favorite primitives, but they are very refined and very like these do exactly what they are meant to do. And so when you find how they connect together, that experience is also really good. And the primitive that arises there I think ends up being better. Is that an example of what you guys are talking about? RICH: Maybe. [Laughs] TARAS: No, I think so. I mean, it's distilling to the essence, the core of what you're trying to do and then be able to combine it together. I mean, that's been kind of the thing that we've been working on at the Frontside. But also within this context, it makes me think of how does a compiler fit into that? How does that work with the compiler? It's just like when you add the compiler element, it just makes it like my mind just goes poof! [Laughter] CHARLES: Yeah, exactly. That's why I keep coming back to like, how do you, and maybe I haven't, you just have to kind of go through the experience, but it feels like maybe there's this cycle of like you build up the framework and then once it's well understood, you throw the framework away in favor of like just wiring it straight in there with the compiler and then you iterate on that process. Is that fair to say? RICH: Kind of, yeah. At the moment, I'm working on this project, so I referred a moment ago to being able to target webGL directly. At the moment, the approach that I'm taking to building webGL apps is to have webGL components inside Svelte in this project called SvelteGL. And we've used it a couple of times at the Times. It's not really production ready yet, but I think it has some promise. But it's also slightly inefficient, like it needs to have all of the shade of code available for whichever path you're going to take, whatever characteristics your materials have, you need to have all of the shade of code. And if we're smart about it, then the compiler could know ahead of time which bits of shade of code it needed to include. At the moment, it just doesn't have a way of figuring that out. And so that would be an example of paving those cow paths. Like if you do try and do everything within the compiler universe, it does restrict your freedom of movement. It's true. And to qualify my earlier statements about how the small modules philosophy is to the benefit of authors over developers, it has actually enabled this huge flourishing of innovation, particularly in the React world. We've got this plethora of different state management solutions and CSS and JS solutions. And while I, as a developer, probably don't want to deal with that, I just want there to be a single correct answer. It's definitely been to the advantage of the ecosystem as a whole to have all of this experimentation. Then in the wild, there are projects like Svelte they can then take advantage of. We can say, "Oh well, having observed all of this, this is the right way to solve this problem." And so, we can kind of bake in that and take advantage of the research that other people have done. And I think we have made contributions of our own but there is a lot of stuff in Svelte like the fact that data generally flows one way instead of having [inaudible] everywhere. Things like that are the results of having seen everyone make mistakes in the past and learning from them. So, there are tradeoffs all around. TARAS: One thing on topic of data flow here and there, one thing that I've been kind of struggling to compute is the impact of that as opposed to something where you have like one directional data flow because it seems like conceptually it's really simple. You set a property like in two way balance system, like you just propagate through stuff but we don't really have a way, you don't have any way of assessing what is the true impact of that computation. Like what is the cost of that propagation where I think it's almost easier to see the cost of that computation if you have like one directional data flow because you know that essentially everything between the moment that you invoke transition to computing the next state, that is the cost of your computation where you don't have that way of computing the result in a two way balance system. Something like Ember Run Loop or mobx or zones, Vues, reactive system. All these systems make it really difficult to understand what is the real cost of setting state. And that's something that I personally find difficult because this clarity that you have about the one directional data flow and what it takes to compute the next state, it's almost like because that cost is tangible where you're thinking about like mutation of objects and tracking their change like that cost is almost immeasurable. It just seems like a blob of changes that they have to propagate. I don't know. That's just something that I've been thinking a lot because especially with the work that we'll be doing with microstates because as you're figuring out what the next state is, you know exactly what operations are performed in a process where that might not be the case with the system that tracks changes like where you'd have with zones or with Ember Run Loop, or Vue. RICH: I would agree with that. The times that I found it to be beneficial to deviate from the top-down ideology is when you have things like form elements and you want to bind to the values of those form elements. You want to use them in some other computation. And when you do all that by having props going in and then events going out and then you intercept the event and then you set the prop, you're basically articulating what the compiler can articulate for you more effectively anyway. And so conceptually, we have two way bindings within Svelte, but mechanically everything is top down, if that makes sense. CHARLES: Is it because you can analyze the tree of top down and basically understanding when you can cheat. This might be really over-simplistic, but if you're kind of with the event, you're collecting the water and then you have to put it way up on top of the thing and it flows down. But if you can see the entire apparatus, you can say, "Actually, I've got this water and it's going to end up here, so I'm just going to cheat and put it over right there." Is that the type of thing that you're talking about where you're effectively getting a two way binding, but you're skipping the ceremony. RICH: It's kind of writing the exact same code that you would write if you were doing it using events. But if you're writing it yourself, then maybe you would do something in a slightly inefficient way perhaps. For example, with some kinds of bindings, you have to be careful to avoid an infinite loop. If you have an event that triggers a state change, the state change could trigger the event again and you get this infinite loop. A compiler can guard against that. It can say this is a binding that could have that problem, so we're going to just keep track of whether the state changes as a result of the binding. And so, the compiler can sort of solve all of these really hairy problems that you had faced as a developer while also giving you the benefit in terms of being able to write much less code and write code that expresses the relationship between these two things in a more semantic and declarative way without the danger. TARAS: This is one of the reasons why I was so excited to talk to you about this stuff, Rich, because this stuff is really interesting. I mentioned that we might, so we have a little bit more time. So I just want to mention, because I think that you might find this interesting, the [inaudible], the stuff that we were talking about that I mentioned to you before. So, I want to let Charles talk about it briefly because it's interesting, because it essentially comes down to managing asynchrony as it ties to life cycle of objects. Life cycle of objects and components are something we deal with on a regular basis. So, it's been an interesting exercise and experimenting with that. Charles, do you want to give kind of a low down? CHARLES: Sure. It's definitely something that I'm very excited about. So, Taras gets to hear like an earful pretty much every day. But the idea behind structure concurrency, I don't know if you're familiar with it. It's something that I read a fantastic -- so people have been using this for a while in the Ember community. So Alex Matchneer, who's a friend and often time guest on the podcast created a library called ember-concurrency where he brought these ideas of structure concurrency to the ember world. But it's actually very prevalent. There's C libraries and Python libraries. There's not a generic one for JavaScript yet, but the idea is just really taking the same concepts of scope that you have with variables and with components, whether they be ember components, Svelte components, React components or whatever there is, you have a tree of components or you have a of parents and children and modeling every single asynchronous process as a tree rather than what we have now, which is kind of parallel linear stacks. You call some tick happens in the event loop and you drill down and you either edit an exception or you go straight back up. The next tick of the event loop comes, you drill down to some stack and then you go back up. A promise resolves, you do that stack. And so with structure concurrency, essentially every stack can have multiple children. And so, you can fork off multiple children. But if you have an error in any of these children, it's going to propagate up the entire tree. And so, it's essentially the same idea as components except to apply to concurrent processes. And you can do some just really, really amazing things because you don't ever have to worry about some process going rogue and you don't have to worry about coordinating all these different event loops. And one of the things that I'm discovering is that I don't need like event loops. I don't really use promises anymore. Like actually, I was watching, I think it was why I was watching your talk when you're talking about Svelte 3, when you're like -- or maybe did you write a blog post about we've got to stop saying that virtual doms are fast? RICH: Yes, I did. CHARLES: So I think it was that one. I was reading that one and it jived with me because it's just like, why can't we just go and do the work? We've got the event, we can just do the work. And one of the things that I'm discovering is with using the construction concurrency with generators, I'm experiencing a very similar phenomenon where these stack traces, like if there's an error, the stack traces like three lines long because you're basically doing the work and you're executing all these stacks and you're pausing them with a generator. And then when an event happens, you just resume right where you left off. There's no like, we've got this event, let's push it into this event queue that's waiting behind these three event loops. And then we're draining these queues one at a time. It's like, nope, the event happens. You can just resume right where you were. You're in the middle of a function call, in the middle of like [inaudible] block. You just go without any ceremony, without any fuss. You just go straight to where you were, and the stack and the context and all the variables and everything is there preserved exactly where you left it. So, it's really like you're just taking the book right off the shelf and going right to your bookmark and continuing along. Rather than when you've got things like the run loop in ember or the zones in angular where you have all these mechanics to reconstruct the context of where you were to make sure that you don't have some event listener. An event listeners created inside of a context and making sure that that context is either reconstructed or the event listener doesn't fire. All these problems just cease to exist when you take this approach. And so, if it's pertinent to this conversation, that was a surprising result for me was that if you're using essentially code routines to manage your concurrency, you don't need event loops, you don't need buffers, you don't need any of this other stuff. You just use the JavaScript call stack. And that's enough. RICH: I'm not going to pretend to have fully understood everything you just said but it does sound interesting. It does have something not that dissimilar to ember's run loop because if you have two state changes right next to each other, X+=1, Y+=1, you want to have a single update resulting from those. So instead of instruments in the code such that your components are updated immediately after X+=1, it waits until the end of the event loop and then it will flush all of the pending changes simultaneously. So, what you're describing sounds quite wonderful and I hope to understand that better. You have also reminded me that Alex Matchneer implemented this idea in Svelte, it's called svelte-concurrency. And when he sent it to me, I was out in the woods somewhere and I couldn't take a look at it and it went on my mental to do list and you just brought it to the top of that to do list. So yeah, we have some common ground here, I think. CHARLES: All right. TARAS: This is a really, really fascinating conversation. Thank you, Rich, so much for joining us. CHARLES: Thank you for listening. If you or someone you know has something to say about building user interfaces that simply must be heard, please get in touch with us. We can be found on Twitter at @thefrontside or over just plain old email at contact@frontside.io. Thanks and see you next time.

In this internal Frontside Podcast episode, Charles, Taras, and Jeffrey analyze the NativeScript Mobile Platform. Please join us in these conversations! If you or someone you know would be a perfect guest, please get in touch with us at contact@frontside.io. Our goal is to get people thinking on the platform level which includes tooling, internalization, state management, routing, upgrade, and the data layer. This show was produced by Mandy Moore, aka @therubyrep of DevReps, LLC. Transcript: CHARLES: Hello and welcome to The Frontside Podcast, a place where we talk about user interfaces and everything that you need to know to build them right. My name is Charles, a developer here at Frontside. With me today are Taras and Jeffrey. TARAS: Hello everyone. CHARLES: Today, we're going to be talking about NativeScript, in particular, and evaluating technologies and frameworks, kind of at the meta level. So, I'm kind of excited about it because we've been pretty heavily involved with NativeScript for the past three months or so. And so, we've gotten to look at it both from beginners' eyes being kind of totally fresh to the platform, but then actually having to start to pump up against some of the edge cases which is what always ends up happening when you actually use a framework for real. Let's get started. TARAS: All right. I think there's a lot of things that we could talk about because when we would start looking at NativeScript, the length that we were looking at NativeScript through this is that this platform that our client is going to be using for doing development of large applications. So, what does NativeScript need to have to be able to support potentially hundreds of developers building apps? We started looking at it and one things that made us consider NativeScript early on was it kind of provides a platform that allows you to encode in JavaScript and run it on mobile. And we saw this kind of emergence of Angular and Vue.js running on top of NativeScript. So, those things together is kind of exciting. CHARLES: There was also an implementation in progress of React and there were a couple of spikes of Ember also running on top of NativeScript. So, my first impression was initially very favorable. The onboarding experience is actually pretty nice because it was JavaScript and the application was interpreted, there's the ability to completely and totally dynamically change the application at runtime. So, they have essentially an application called the NativeScript Playground which lets you flash a QR code at it and then it will go in to the URL associated with that QR code and it will download all of the assets for a NativeScript application running at that URL. So, all the JavaScript, all the templates, all the whatever, it'll pull it down, it will actually start running like within that app. So, the Playground app then becomes your actual app that you want to use. There's no App Store, no TestFlight, no Google Play. There's no gatekeeping to delivering your application into a running app. And I thought that was really, really cool and really, really compelling. TARAS: We should clarify that this is specifically for preview purposes because if you're going to be shipping the application to production, you still need to go through all those things before... CHARLES: Yes. TARAS: But the onboarding process, you could just install the preview app and then you can point a QR code and it will open that app, whether it's in Angular or in Vue, that app will open up in the preview app and you have a native app that you could play around with. CHARLES: Right. JEFFREY: And that's key both for the engineers who are playing around with this and building this and also really key for the non-engineers who are part of the team to be able to really easily spin up and see what the engineers on the team are working on. CHARLES: That's exactly why we thought, "Hey, we want to be able to use this mechanism for preview apps." In the same way on the server side, you have preview apps associated with a pull request. When we saw this, what we immediately wanted to do was have a bot post a comment onto a pull request with a QR code, so that anybody could just, boom, test out this app on their phone. TARAS: We ultimately ended up setting that up but not quite that way because the original idea of being able to have something like danger bot post the QR code to the comments, you can kind of point out with your phone and open the preview app, that didn't actually pan out. Charles tried to implement that. What happened there? CHARLES: What it actually turned out was that the preview functionality was dependent on a central server, a central NativeScript server. So rather than kind of statically bundling the assets and just saying 'these assets are this URL and just pull them in and bootstrap your NativeScript application that way', it required a lot of extra stuff. So, it required you to be running a Webpack Dev Server that was building your assets and then basically registering and doing some port forwarding with that dev server to a central NativeScript service that was provided by the company that underpins NativeScript. And that connection needed to be hot and live the whole time for that to work. So, while it was really cool that you could get the QR codes up and running, unfortunately that functionality could not be decoupled from the hot update and the central service. Those central services were kind of hard coded into the tools. TARAS: Yeah. So we eventually ended up implementing the preview apps that we wanted but we ended up using Appetize.io to essentially -- the process there is you build the app, you upload the app to Appetize and then danger bot embeds a link to a URL where you can open that app and it will essentially stream like it's running somewhere in a simulator for iOS, an emulator for Android and it will stream a video of that and you can interact with it, kind of like a VNC setup. CHARLES: Yeah. TARAS: And that actually accomplished the goal. It's just we weren't able to do the way that we thought we were hoping to do it straight off with the preview app mechanism. CHARLES: It accomplished the goal. And Appetize is an incredible service that lets you preview the apps on pretty much any type of Android device, any type of iOS device, right there inside of a pull request. But what it didn't allow us to do was pop up your actual device, your actual phone and scan a QR code off of the pull request and pull down the assets. That would have been amazing. But it doesn't always work out that way. And I don't know if that would work long term anyhow because you can't pull down native libraries over the wire and funk them in. That's a big, big no-no. So, the process does have limitations. But nevertheless, that part was really cool. TARAS: Yeah. That was kind of the entry point, the onboarding. And then I think one of the things that was kind of, I remember at the time when we were talking about the NativeScript architecture because we were starting to understand more about how it works. The idea itself is really kind of amazing actually because you have this V8 where you can run your JavaScript code and then they're kind of wired together on iOS and Android. They're wired to the native implementation. So when you're interacting with it, I think the thing that's really great about NativeScript is that the runtime environment for JavaScript essentially gives you API access. In JavaScript, you could say, "I want to create a Java view," and there will be a Java view that's rendered in the actual native device. You're using the same -- the APIs that you find on the Android docs or iOS docs, all of those APIs are available to you as JavaScript. So, you [crosstalk] as JavaScript. And it's seamless, right? CHARLES: Yeah, and it makes it very, very handy. The language is different but the APIs are exactly the same. There is an attempt to make cross-platform components and cross-platform classes that serve the needs on both platforms and then delegate to the platform on which you happen to be running. But those are not mandatory, and the low level APIs are always available to you. An example of this is in iOS, kind of the core foundational object is NSObject. All the controllers, the views, the things, all of them are descended from this object. I can go from object and I can go in from JavaScript and I can just say {let object = new NSObject} and boom! I've got a reference to the actual object and I can pass it around to any other iOS API. That is really, really powerful that there's nothing off limits. There's nothing at an arm's distance. There's really not much you can't do because all of those things are available to you. There's nothing that's off limits. That means that they can build cross-platform components on top of those APIs. Whereas a sort of system like React Native which does have cross-platform components, that's kind of where the base layer is but you can't crack open the hatch and go down the next level and start mucking around, unless you want to actually start meddling with the React Native source code or recompiling Swift in Java code. TARAS: For me, I think this architecture is probably my favorite part of NativeScript. JEFFREY: Mine too. CHARLES: Yeah, me too. TARAS: I really like this part. I kind of hope that everything else is as clever as that was. CHARLES: Because among other things, it allowed us to write a Bluetooth. We were able to implement Bluetooth using nothing but JavaScript. We didn't actually have to go down and do any Swift and do any asynchronous message passing between the iOS libraries and the JavaScript libraries. It's like, "No." We've just got a very simple cross-platform interface that instantiates an implementation for Android and an implementation for iOS, but both of them are like JavaScript. And so, it really is you're doing native development but it's JavaScript all the way down. TARAS: Yeah. And when you're writing plugins, your plugin is actually JavaScript plugin that is assuming iOS APIs and Android APIs. CHARLES: Yeah. And if you have to have a native plugin like a CocoaPod or an Android Package, you just install it and you can instantiate it from JavaScript. There's no fuss, no muss, no ceremony. It's just like, "Hey, I want to use the..." what was the one we like to use? The Material-UI floating button which is a CocoaPod. You download it, you link it into your application, and then you just instantiate it from JavaScript. TARAS: That was really cool. The challenging part was that a lot of that kind of awesomeness, like everything around it wasn't quite as polished. And so, one of the big things is that like around tooling, because one of the things about having grown up in a way like in the Ember community, in a sense, we have a certain expectation of what the level of polish from tooling that we would expect. And it's kind of supported in the way like when you look at how React or React Native tooling is, even Angular tooling, it's very polished. You kind of expect to see what you need to see when you're looking at a CLI input and you don't see anything else. That level of polish. I think part of the changes that they're going through, maybe that's part of the reason but that same level of polish isn't available around the tooling. CHARLES: There are these fantastic qualities about the platform and it is amazing. We were using Angular and a lot of people are using Vue and things like that and that actually is pretty incredible. And there is nice tooling, there is command line stuff, but we started to run into issues where, for example, it was very clear that we were pretty much, as far as I could tell, one of the very, very few people running a NativeScript project on CircleCI or in a CI environment at all. It had capability for testing, both for acceptance testing and for unit testing, but it required changes to the core framework and the core tools in order to get those tests to work in a CI environment. JEFFREY: Before we kind of get into the testing story there, some of the issues were around determinism of reliably reproducing your whole NativeScript environment and stack every time because that's such a key feature of doing it. And on a CI server, it's like, "Hey, we need this to load in the same exact packages every time." And so, we ran into challenges there. TARAS: I think we spent almost two days. There's example projects in different combinations. One thing that was off was that there's a pattern that is applied in a lot of the plugins in NativeScript ecosystem is installing things. So, you run npm install and npm install will generate some files. And so, when we're trying to move it over to a CI, there were files, like there's hooks, like TypeScript hooks that were excluded that you can ignore, but they were necessary to compile the TypeScript. And so, what was happening is when we're running these at CI, the application, we would build the app but the app would crash the moment that you start it. And the reason for that was that the JavaScript files that were transpiled from TypeScript to JavaScript, those JavaScript files were actually never included because they were never transpiled in CI because the hooks directory, like we weren't preserving it between our tasks and so... CHARLES: Right. We weren't caching. This was an artifact of the install. And so, we were caching the install, so essentially the yarn.lock was not changing. But the directory was not getting generated unless the cache key changed. TARAS: And we spent spent quite a lot of time... CHARLES: Two or three days out. TARAS: Yeah. CHARLES: What that said is, "Oh, nobody's really running this in CI." Nobody's actually building an app from scratch every time. TARAS: There are people in NativeScript team that actually does a great job of documenting. They did have example projects that exist but sometimes that example project doesn't fit like a perfect combination of what you're looking for. There was an example project that was showing how to run on CI but it didn't use TypeScript. And so, that's where we lost a lot of time. CHARLES: Right. JEFFREY: So, let's talk about testing since that's kind of the core, the most important part of why you even want continuous integration capabilities to begin with. What did we run into there? What did it look like? TARAS: Well, I think it's safe to say that we were really on a bleeding edge of testing capabilities in NativeScript ecosystem with Angular, at least. But I think it was still an interesting project. We were using the latest builds. And I have to say I think this is one of those things that's going to be kind of consistent through this, is like the people in NativeScript team are amazing. They're so easy to work with. They're so accommodating. When we ask for stuff, they're on it. But it was a lot of things we're trying to figure out like how do we run unit tests, what can we do. Ideally, we wanted to run, first and foremost, we started with how do we run functional testing. So we spent quite a lot of time trying to get Appium set up. I spent a good two to three weeks on that and it was not productively spent time. CHARLES: I think ultimately, we had to pull back from it. And there were a number of reasons. Part of that is there are multiple paradigms for how you can build your NativeScript application. So as we speak, there's a move towards using Webpack to build all of your JavaScript in your style sheet assets because it's very much like a React Native application. You've got style sheets, you've got JavaScript assets, that some of them might be in TypeScript, some of them you might be using Babel, and you need to actually transpile them down to include them in a way that your underlying JavaScript runtime is going to be able to understand. But that wasn't always so. They have their own build system and packaging system, they kind of used the TypeScript compiler ad-hoc, if you were using TypeScript, which we were. And so, this was kind of this orthogonal complexity, I guess, where you have your unit testing and it has to play nice with this one package or Webpack. There were multiple ways to package your app. And so, we ran into problems where, like TypeScript kept coming up as a problem and the way in which we were bundling our assets. So, in order to get TypeScript to work, we kind of had to get Webpack running. But the problem is it felt like three quarters of the tooling wasn't Webpack compatible yet. And so, it meant that other pieces of the build were breaking because of this. And so, we had to be on the bleeding edge of several different aspects of the runtime. And the problem is when you're on the bleeding edge, that can break other stuff. TARAS: But there's complexity in running on native platforms that I think a lot of this complexity is kind of leaking to development experience because one of the challenges is your tests need to run on the native device in the application. So, you have to build the app. You have to push the app into the actual device. So, there's like all the setup of installing the at the app on the device. CHARLES: You have to launch the simulator. TARAS: Yeah, right. CHARLES: To make sure the device is connected. TARAS: And you run your tests in there. So, that created kind of this situation where we say let's just kind of set Appium aside and just use unit testing which is a very small fraction of the kind of testing that we actually want to do. It will test very little. But let's just do that because getting functional testing to work was really kind of not going anywhere. So once we start doing unit testing, one of the challenges is that it takes like 30 seconds to start your tests. And then, if you for whatever reason, made a mistake, the moment you cancel the build, it leaves, like it doesn't clean up of itself well. So, it leaves processes running in the background. And so now, you spend another like 10 to 15 minutes Googling around for a cookie, "How do you find these processes and stop them?" So, we eventually settled on having a script that does that, but this is the kind of things you have to end up doing because there's a bunch of things that are wired together, but they're not wired together in a way that is seamless. And so, you end up kind of just debugging a lot of stuff where you just want to run some tests but you end up doing all these other stuff. CHARLES: Right. TARAS: And you spend a couple of minutes just doing something that you'd expect to happen in like 20 seconds. CHARLES: Right. There is a feeling that every aspect of the system is coupled to every other aspect of the system in kind of varying ways of interconnectedness. And that's not what you want for a very, very complex system. You want it to be extremely modular. So, I think we should keep the command line tool. There's probably a separate discussion, I think, about that. But you have to close the book on the Appium and the unit testing. I think the other problem was that you have to run these things on simulators. On macOS, that's not a problem because the simulators ship with X code. And so, you don't actually require an external service. Whereas in CI on Android, it's very unlikely that you're going to have Android emulators on hand because they require a separate virtual machine. Android emulation is actually quite heavy. If you're running through Android Studio or something locally, you essentially need VirtualBox or some equivalent to run your Android simulator because you actually need that simulated hardware. If I understand correctly, that was actually not something that had been really accounted for. It was that you might want to be running simulators not on the same machine as what you were developing on or what the actual that you were building on. TARAS: Yeah, a lot of the tooling seems to be designed around this idea that you're going to be building and running everything on your machine. And so, you can spin up a virtual machine easily. But in CircleCI, for example, they don't support running a virtual machine inside of a Docker container because for that, you need a feature of a virtualization that is not supported in many CI platforms. You have to run a parallel server if you want to have like Appium running, for example. You need to have a separate server running like an Azure or a Google Cloud somewhere that is able to run virtualized servers that have a host machine that's being guest systems that are running the actual Android emulators of different versions. And so, when I started doing research in this, there are companies that are doing this really well but it's not unusual to be using hardware from Amazon that costs thousands and thousands of dollars per month. I think for anyone who's getting into mobile development, I would say the hidden gem of Android world is Genymotion. Those that do a lot of Android development, they know about it. But Genymotion has both like a desktop environment and it has SaaS offering that they're in the process of releasing. And so, what it allows you to do is when you run it locally or on your local machine, it allows you to create a virtual machine that is running in VirtualBox and then it allows you to run kind of optimized environment for running Android. And when you do that, it's really fast. It's very smooth. It makes running Android devices locally as easy as it is to run iOS devices on macOS. CHARLES: I remember starting out and trying to actually just get any Android emulator running on my Mac and I couldn't even do it. JEFFREY: It was such a huge time saver. CHARLES: Yeah. TARAS: And to have this Saas offering is really great because you could basically create your virtual machines on demand and then you install into a virtual machine from your CI server and then you run your tests there. That's kind of the key that I found to be able to run tests and automate it against emulated devices for Android. Genymotion is really great. CHARLES: Yeah. Again that's the kind of thing that you need when you're in CI. And so, one of the things, I think, one of our discoveries is that there just isn't -- when we started working on this and we haven't seen a culture of running these tools in the cloud and accounting for the fact that you might have not all of the tools running on the same machine. From, I would say, the beginning, I remember the kind of the diagnostics command didn't work but we were running it on a CI server. So, there's a diagnostics command that you run to see do you have this, do you have that, do you have that. It would work and give meaningful results when I wanted to debug my CI server because when we were initially getting set up, something wasn't building right, there was some dependency missing. And I just wanted a diagnosis but it was trying to install all those tools for me. And I was like, "No, no, no. I don't want you to do anything. I don't want to install them. I'm going to be doing all of that as part of the setup of the CI environment. It's going to be installed, it's going to be cached. I don't want you to just try and like massage my system into a suitable state for NativeScript development. I just want you to diagnose what is wrong. Tell me, am I missing this compiler? Maybe I've got the wrong version of Android SDK. Tell me what's going on." And I couldn't get that to work. That was very frustrating. I think it was because the kind of bulk of the assumptions was that it was going to be individual developers working on their own laptops or their own desktop computers to build, to test, to distribute these applications. I think that's becoming less and less the case. I mean, at this point, that's not a way that we're willing to operate. TARAS: And we eventually figured out how to do all this stuff, right? CHARLES: Yeah, we have. JEFFREY: We have. TARAS: We have the entire process working but it took a lot longer than one would imagine. It took all the time that we had allocated to it which we thought was very generous amount of time but it took like almost a month to get everything set up. The great part of this is that we do have now everything working. And so, there's a repo where people could take a look if they want to get all stuff working on CI, but it took quite a bit of work in figuring out. CHARLES: Yeah. Actually, I think worth probably a Screencast to show some of those capabilities because it is really exciting. I mean, when you actually think about the pipeline in its entirety. But we never were able to get functional testing working. TARAS: And then the challenge here is that because we were essentially looking at NativeScript, going back to this question like, "What do we need to be able to have like hundreds of developers potentially running on this platform?" And so there's a lot of considerations and this tool is just one of them. I think the other one that is a big one is like what are the capabilities of the view layer because that's where most of developers were spending most of their time. We got stuck a little bit about that because I spent a lot of time working in the view layer. The thing that was really great and the thing that I really liked about it is the fact that you have a collection of components that you can use in Angular. You render it as component and then that component is going to look correctly on iOS and is going to look correctly on Android. From a single code base, it's building appropriate components for iOS and Android. What I think is really confusing in that case, though, is because the Android and iOS components don't have parity in a sense. They don't behave exactly the same. And there is also a kind of a reputation in the NativeScript documentation that Android tends to be slower, much slower than iOS. And so, when you start to run into performance problems and you start to run into those pretty fast because it is not really clear what is necessary to not optimize NativeScript, when you start to run into performance problems, it's not really clear like where is it coming from. Right now, the profiling that they have for the UI is very limited. They're kind of in the process of migrating over to chrome.debugger, but profiling in chrome.debugger is not implemented. You can do performance optimization using Android tooling but that's only going to tell you performance of the Java side, or the iOS side is not going to tell you the performance of the code that's running inside of JavaScript. It's not really clear what is causing the problem. If you don't know what's happening, you kind of write it off as like, "I think it's just Android being slow." In reality, when you actually start to dig deeper, you realize there's things about the Android implementation of the components that are different or the views that are different than iOS. And it's the differences that add up to weird performance problems. That's probably the thing that gave me the most hesitation because one of the things that made me think like if we want to be able to give this to a team of like 50 people, we need to have our own view layer because we cannot rely on components. An example of this would be, they have a list ticker on iOS, it doesn't omit change events when you scroll. If the list is moving, it change events and not omit it. But on Android, every time that a different item shows up on a screen, it changes the selection. And so now, you've got this view that's a meeting on Android as a meeting change events. I made an issue around this and the response was that while there's a workaround that you can have for this, but that's hard. Work around is not a solution. CHARLES: Right. When you have a leaky abstraction like that. TARAS: Part of the problem is because people use leak abstraction. And so, what's happened in Native -- we actually got on the call with NativeScript core team and they're excellent in really being very helpful, understanding what the problems are, and providing pass on making things better. But what's happened as a result of having this leaky abstraction is that people are relying on the leak. And so now, the leak is the API. And so, we can't change that. JEFFREY: Right. CHARLES: And the answer that you really need there is, "We can't change that without breaking stuff. Here's our migration path for deprecating this and introducing a new API." And that gets more into the process stuff and it seems like the process for making changes to the underlying API, I think, could use a little love in the sense that it's kind of opaque as to where the platform is going. There's not a concept of like an [RSC], there's no roadmap about what to expect. What is this API going to look like in the future? Is this stable? If I were writing a software and someone said, "Hey, there's this leaky abstraction," I think my reaction would be, "We've got to fix this." And we also have to acknowledge that there are users who may depend on this. And so, we have to be very deliberate about it. TARAS: The challenge with this too is that NativeScript kind of outgrew its hands because I think originally, it wasn't meant to be hosting Angular and hosting Vue. Vue didn't exist. Angular didn't exist when NativeScript started. So I think what's happened is that these views that were available, I wouldn't call them components because they don't act like components, but they're exposed in Angular like components but the API feel like Vue objects. So these Vue objects that you consume, that you render in Angular, for example, or in Vue.js, they are the same APIs that NativeScript had before Angular and Vue.js. CHARLES: Right. You know what? It feels like there's a MVC framework, like a Circa 2010, 2012 MVC framework that has now become the foundational layer for Vue frameworks that have had significant advances in the way we conceive of model in Vue and how data is generated and passed around and how views are rendered off of the data and how reactivity is changed. But there's still, the underlying platform has not evolved. And in fact, this was originally user-facing APIs and now these APIs have become foundational for other user-facing APIs but haven't had the iteration and evolution to make them robust. TARAS: And flexible enough. As a result, you have the situation where not only is it really super easy to deoptimize the views simply because the requirements of keeping performance expectations are not obvious. One of the things that I found is that the list which is, lists are like 50% of most applications. Before I go into the problem with list, the nice thing about lists in NativeScript is that because they're interacting directly with native APIs, you have really fast list when they're optimized. They're really easy to work with. But they easily get deoptimized by the fact that the expectation to keep the list fast, you have to use this API in NativeScript called array observable and observable. And this is not to be confused with like... CHARLES: [Inaudible] observables? TARAS: Yeah. CHARLES: It's not to be confused, but in fact, every conversation involves a lot of confusion. Because we were using observables, right? TARAS: And we were actually using observables. So, we're using observable [inaudible] and we're using this array observables and object observables. And so, it's necessary for NativeScript to, essentially what it expects for list to be fast, is it expects that it's going to receive an array observable which is an object that wraps an array because it needs to know when an order or length of data rate changes. So what happens when you pass an array observable, a NativeScript array observable into a list? It will listen for change events on that object. But if you want to change the value of each of the items, like if you want to change a property on the object and have your view remain optimized, the array observable has to have an observable object which allows NativeScript ListView to listen for changes, property changes on the object. You pass this array observable which contains observables that ListView listens for changes on to make sure that it knows how to correctly apply this change to the list. If you don't have this magic, like if you haven't figured out this recipe for ListView performance success, you're going to have a really hard time because it's really not clear at what point and how this thing got deoptimized, why has it just gotten slower. CHARLES: There's a lot of iteration that needs to happen there and it's not clear what the plan, what the priority, or even how you will even begin to go about this. Because I think that the internal working is that it seems basically to be controlled by one company. I don't recall seeing any contribution from anybody except for Progress which is Progress Incorporated is the company that's kind of the controlling interest, the original company that developed it. TARAS: The way this showed itself very practically is that to make changes too -- so they have a ListView which comes with NativeScript public and there's RadListView which is the component that has a lot of stuff on it. Like if you want to pull to refresh or if you want to do like laser loading a data or if you want to do a filtering, you want to do -- so most people use RadListView. But RadListView, you can install, so there's no limitation when you build to install it, and your node modules has the source code for that. But the source code, the original TypeScript code, untranspiled code is not publicly available. They have a process for doing this and it's very nice that everybody's very kind and very accommodating. You send an email, they'll give you access to this repo and then you'll have the ability to contribute. NativeScript core team is very helpful and they're open to contributions. There are changes that need to be done to the Angular implementation to make it faster without having to put the requirements of the observable thing, and so they can give you a path to make that stuff happen but it's not open source in the sense that it's not a traditional open source that we would kind of expect. So, there's all kinds of hoops that you need to jump through and the source code is very difficult to read because it's transpiled from TypeScript to JavaScript. CHARLES: And there was a certain level of opacity in terms of process. For example, I filed an issue which was actually a blocker. For us, it was actually causing our Android build not to work. I didn't hear anything about it. And then, all of a sudden like four days later, a fix came through referencing another repository on which this thing depended with. There was not a lot of context service. So it was obviously referencing a bunch of context that probably happened between two people in a face-to-face conversation. But I couldn't really tell what was going on, why it was an issue, because there was no comment. It was just a pull request that was referencing this issue. I never got a notification. I actually had to go and be like, "Hey, I really would like for this issue to be solved. I wonder if I..." I was actually going to post a, "Hey, is there any progress on this?" Or, "Is there any way that I can help? What can I do to get this looked at?" And I saw that there was another pull request that had referenced my issue. And it was merged and I looked down, but then there was no indication of when this would be available for public release, how I might be able to work around it. And so, the strange loop that didn't get connected was, "Hey, you've got a user who files an issue. You actually use this as the impetus to fix the issue and make a release." But then that whole process was completely invisible to me. TARAS: You know what? It sounds like you wanted for it to work [inaudible] but you got a pulling mechanism. CHARLES: Yeah, exactly. Well, I wanted someone to say like, "Hey, here's what's going on, and we're looking right into it." Or, "We're going to look into it in like two months," or, "We can't address this now. But here's a workaround for it." Or, "I don't have a workaround." That's just kind of the expectation that you have when you're playing with open source. In many ways, it does not feel like an open source project. TARAS: Let's just do a quick note about Saas. Jeffrey, what did you find about the styling of NativeScript views? JEFFREY: All the components that come kind of shipped as part of the NativeScript core set of components all have styles attached to them. They have CSS attached to them. And as part of the standard data script workflow, with your build toy, you have SaaS available which is very nice. But actually on a recent project, we're not using Saas at all. We're simply using post-CSS and we were able to kick out some CSS variables that turned out to be really nice for theming. So as kind of a future friendly experiment, we were trying to have a light theme and a dark theme since that is very recently now a core part of Android and very likely will be part of iOS this year, where there's kind of a light theme and a dark theme for everything. We were trying to do that. The simplest way to do that with standard web tools is with CSS variables. You can have the flexibility, you have the theming with those. It's so nice. You just, "Hey, my primary color is this color in one scenario and it's this color in another." And we just didn't really have the flexibility to do that with SaaS by itself. And so, that's kind of a limitation of the tooling right now that I hope in the future, we'll have some more sophisticated CSS tools. And really, NativeScript's move toward Webpack and having that as a primary part of the workflow really opens up that possibility that I hope somebody runs with in the near future. TARAS: Yes, let's bring it all back together. CHARLES: Can we pause for a moment? Because I actually do think it's important that we at least touch on the command line. I can give a little bit of a kind rant in here but I think that's actually something really important that we have to talk specifically about that. The other thing that I wanted to touch on very briefly as we kind of draw to the close is the command line tooling, in particular in NativeScript. I think that this is probably one of the weakest points of the platform. And again, I don't want to disparage anybody working on NativeScript. It's an extraordinarily complex problem. This is a command line tool that needs to manage launching simulators, installing things into simulators, pushing code to those simulators. It needs to handle hot updates to things that it's running on, devices and simulators. So, it needs to be building JavaScript assets either with Babel or with TypeScript. It needs to be building those SaaS assets that you were just talking about, image assets. But it needs to be doing all of this for two platforms, so it needs to be managing everything that I just described. It needs to be managing on iOS. Everything that I've just described needs to be managed on Android, as well. It needs to work for a single developer's desktop. It also needs to work with all of those components that I just described distributed out in the Cloud. So, we're talking about an extraordinarily complex piece of software. And I think that unfortunately, the NativeScript CLI does not inspire confidence because it can do all of those tasks. But Taras, you also mentioned often if you stop the process midway, it will leave a thousand things open and they're just spewing output to your console. The console output, unfortunately, means there's a big noise to signal ratio because it puts out all of the content for Webpack. Every little thing that it's doing with any of the devices, it's logging to the console. So, it doesn't give you a sense of control. So, what you really are looking for in terms of a command line is, "Hey, I've got this incredible sprawl of complexity and I want to feel like I'm on top of it." And unfortunately, by leaving these things open and having so much console output and having the console output not be formatted well, there's all kinds of colors. Every single tool that you're using whether it's Webpack or whether it's Karma or whether it's just console outputs that you are happing inside of your NativeScript application, the brand of those tools comes through. Webpack is a great example. Its console output feels very Webpack. So when you've got Webpack content randomly interleaved with your console content from your Mocha content, from Karma, all of these competing brands, it doesn't feel like a cohesive developer experience. And so, I really, really hope that -- so, to the point being where I felt like I could not live with that command line tool without rewriting it myself. If we want to use this platform long term, we'd have to either have an alternative command line tool or really, really, really help the NativeScript team completely and totally rewrite the command line experience. TARAS: I would love to work on fixing a lot of these parts about NativeScript if there was a way to actually do it in terms of like, if they wanted to pay us to help them kind of bring some of these things to a state that would match. For example, what's available in Ember or available in React CLI, I would love to do that. CHARLES: React Native, yeah. TARAS: Yeah, let's do that work. But who knows what's in store? A lot of awesome platform like the idea around NativeScript architecture is fascinating and it's really, really powerful and really wonderful people doing some, trying to tackle really challenging problems, but it's all glued together in a way that doesn't instill confidence. And it just makes everything feel wobbly, just makes it feel like you never know, is it a problem? Where's the problem from? What is causing this? CHARLES: Yeah. And if I fix this thing, is it going to break something else? TARAS: Yeah, we've seen it happen actually with one of the solutions that was introduced to a bug that you were referring to earlier. CHARLES: Yeah. So that was our three months experience working with NativeScript. TARAS: We are considering other things now, very seriously looking at Flutter as an alternative for the same client, same scenario. Flutter is looking pretty exciting. There's a lot of things that are really good there. So in three months, we'll do another report and talk about Flutter and what we found. So, that's it. CHARLES: And I will say I'm actually not like super excited about dart but I'm in dart spot. JEFFREY: That's a whole other conversation for yet another episode. CHARLES: I think that, to continue the conversation maybe next week, next time we have kind of an internal podcast, is I would like to really talk about platform evaluation because really you need three months, at least, to get a good idea of this. Is this going to work for the next five years? And most of the time, we give it a week or give it a two week. Or someone comes on who's really excited about this one particular technology and you go off on that tangent. I think there's an interesting meta discussion about how do you select technologies. And we don't have time for that now, obviously. But it's definitely something that I want to have in the future. TARAS: Sounds good. I think that will be a good conversation for sure. CHARLES: I guess that is kind of the executive summary on NativeScript from our perspective. With us being three months in, I think, like you said, there's a lot there. Thank you for listening. If you or someone you know has something to say about building user interfaces that simply must be heard, please get in touch with us. We can be found on Twitter at @TheFrontside or over just plain old email at contact@frontside.io. Thanks and see you next time.

In this internal episode, Charles and Wil talk about testing issues and BigTest solutions. Pieces of the testing story are discussed, such as the start and launch application, component setup and teardown, interacting with the application and component, convergent assertions, and network. Then they talk about testing issues: the fact that cross browser and device-simulated browsers are not good enough, maintainability and when and when not to DRY (RYE), slowness and why (acceptance) testing is slow, portability and why tests are coupled to the framework, and reliability. Finally, they talk about BigTest solutions: @bigtest/cli to start / launch (Karma recommended for now) @bigtest/react, @bigtest/vue, etc for setup & teardown @bigtest/interactor for interactions @bigtest/convergence for assertions @bigtest/network in the future (Mirage recommended for now) Resources: Justin Searls – Please don't mock me This show was produced by Mandy Moore, aka @therubyrep of DevReps, LLC. Transcript: CHARLES: Hello, everybody and welcome to The Frontside Podcast, Episode 115. My name is Charles Lowell, this episode's host and a developer here at the Frontside. With me today to talk some shop is Mr Wil Wilsman. WIL: Hello. CHARLES: Hello, Wil. WIL: How's it going? CHARLES: It's going good. I'm actually pretty excited to get to jump into this topic because we're going to be talking about some of the big things that are happening at Frontside and some of the things that we've been developing in almost for the last year. WIL: Yeah. It's been about a year now. CHARLES: It's been about a year and we've talked about it in various podcast but we're going to be talking about it again because there's just been so much progress that we've made, I think in a lot of clarity in kind of what we're going for here when we talk about BigTest and testing big and how we want to roll out the BigTest framework. We just have a lot more experience using it on a number of different projects, so we get to talk about that today. Before we get started, I just wanted to talk a little bit about what BigTest is, both in terms of the framework and also the philosophy. Wil, you're the one who works the most on BigTest. When you think about philosophically, what does BigTest mean to you? WIL: It's the size of your test, not a physical size like size and storage but how much your task actually does. The test itself can be very small as our test are but it tests the whole application from the user interacting with it down to the network requests. That's the definition of the philosophy of a BigTest to me. It's to tests your application from the biggest point of view. CHARLES: Actually, achieving that can be surprisingly difficult, especially in a frontend JavaScript application and there are a lot of solutions out there for testing and we've talked about them. One of the questions that arises is when we talk about BigTest, what exactly are we talking about? Are we talking about a product that you can download and install? Are we talking about the philosophy that you just outlined? Or are we talking about the individual pieces of software that make that philosophy real? I think the answer is we're kind of talking about all three but we want to take this episode to talk about where we're going with the product. What we've identified is the subcomponent pieces of that product. In other words, in order to get started testing big, what are the things that you need to think about? What are the things that you need to do? And then what are the component pieces? Because one of the things that I think is very important to us is that you be able to arrive at wherever you are in your project, whatever framework you are using, whatever current testing solution and be able to begin using BigTest. That means, you might be using some of it or you might be using a lot of it but we want to meet you exactly where you are, so that you can then, get onboarded and start testing big. WIL: Yeah. Definitely an important distinction that we get confusion about is what is BigTests and people just assume like this whole test suite is BigTest but we used the parts of it ourselves like we use Mocha, which is not part of BigTest. We use Chai, which is not part of BigTest. We use Mirage which is kind of part of BigTest but definitely it originate in BigTest and Karma and things like that. BigTest isn't your testing suite. It's not one thing to go-to to grab, to start writing tests. It is a small pieces that you can use in conjunction with other small pieces, just to make it really easy and flexible to test your application. CHARLES: Exactly. Because it turns out that there's a lot going on in the application. Maybe we should talk about what some of those pieces are that you might want to start using BigTest with or that you might need to test big, I guess I should say. What's a good place to start? Let's start with talking about some of the issues that you want to do when your testing big. Then we can talk about what pieces of the testing story that fit in to solve those issues. One of them is you need to test that your application works, like actually works. That means you need to be able to test on a multiplicity of browsers, for example. We're limiting to the domain of web applications. There are actually a shockingly large number of browsers. It's not just Chrome. It's not just Safari. There's Mobile Chrome, Mobile Safari, which are subtly different. There's Edge and I'm sure the Mobile Edge is slightly different too, so you want to be able to test cross browser, right? WIL: Yeah, absolutely and things like Nightmare and JS DOM and things that simulated browsers, we don't necessarily think those are the best tools for writing BigTest because we want to ensure that those browser quirks are caught and tested as well. CHARLES: This is not theoretical like sometimes you'll have a syntax, like the parser is slightly different and you have something that throws a syntax error in Safari or in the Internet Explorer and your whole app is completely busted. If you just take in the time, just even trying to load the app in that browser, you would have caught that. That's what I've been on many times. WIL: Yeah and what I just saw came up yesterday, which comes up frequently is not closing your CSS Selector and Chrome doesn't really care like web to browsers don't care too much but that will fail in Edge and depending on what you're missing, the failing is part of that too but mostly, Firefox and Chrome don't care about that kind of thing. CHARLES: Right. It seems like the majority of testing solutions are kind of focused around Headless Chrome or some variation of Electron. That entire class of really dumb errors has already been caught. Like I said, to actually catch it, it takes less than a millisecond of CPU time just to load it onto the browser and see that thing doesn't work. Unfortunately, they can be catastrophic errors but the problem is how do you actually do. We want to test like cross browser. This is something that we want to do. For me, I just can't imagine shipping an application without having some form of cross browser testing, some capability of being able to say, "I want to test it," like, "We want to work on these eight browsers and so we're going to test it on these eight browsers," but how do you actually go about doing that? WIL: Right now, we are working on the BigTest CLI which will help us launch browsers but that's not complete yet. It has some bugs on. For the meantime we've been using Karma, which is great. Basically, you just have this service that's able to find the browser binary on the system and just launch them pointing to local hosts with your app loaded up and your normal development server take care of loading the test up and running the test. Karma and the BigTest CLI is just there to capture output and launch those separate browsers. CHARLES: Yeah. I remember when I was first using working with Karma and I think Testim is another tool that's in this space. There's Testim, Karma and BigTest actually is we're developing a launcher because launching is something that you're going to need but it's such a weird problem. I feel like with the browser launchers, there's three levels of inversion of control because you're starting a server that then starts another process, which then calls back to your server, which then loads the app resources, which then loads the tests and then runs the test. There's a lot of sleight of hand that has to happen and – WIL: Including injecting the adapter that you use, like the Mocha adapter, the Jasmine adapter that ends up reporting back to the CLI. That's something that Karma and Testim and BigTest will handle for you. CHARLES: Right, so you're fanning out the test suite to a suite of browsers then collecting the results but basically, you need some sort of agent living inside the browser that's going to act on behalf of the test suite, to collect the results. I remember when I first came into contact with Karma and Testim, I was like, "This is so unnecessarily complex," but then, having used it for a while and I think there are some complexity that can be removed but if you want to do cross browser testing, that kind of level of ping-ponging is there's a certain amount of it that just necessary. It's something that's actually quite complex that you need to have in your stack, in your toolbox, if you want to truly test big. WIL: Yeah and all the solutions is mechanisms for detecting when the browser has launched and restarting the browser based on its health check, etcetera and things like that that you wouldn't think of actually loading up a browser but you need to think of when you're doing automated testing. CHARLES: What is it that sets apart, for example the launcher solution? We kind of call this class of solutions launchers, so Testim, Karma, the BigTest CLI. What is it that sets BigTest CLI apart from say, Karma and Testim? WIL: We're trying to be as minimal config as possible and just really easy to get started and going. Karma has a lot of plugins that you need to make sure you have installed and loaded in the options set for those plugins. Testim has some stuff bundled but it still requires this big config bulk at the beginning that you need to passing or that's all what you were doing. We're trying to avoid that with BigTest CLI and one of the ways that we're able to avoid that is by just letting your Bundler handle bunding the test. In Karma, you need Karma webpack or something. Testim has some stuff that it needs and really, we just want like in-testing mode. When you're in the testing environment, just change your index to point your tests, instead of your application and your Bundler will do all the work and we just serve that file and collect the results. CHARLES: Right, so it doesn't matter if you're using Parcel or you're using webpack or you're using Ember CI. WIL: Yeah, Rollup even. CHARLES: Or even just like low level Broccoli or Gulp or whatever. There's a preponderance of bundling solutions and that was always something that was just a huge pain in the butt with Karma. I know it's like just getting to the point where my tests are loaded and you look with Testim, most of my experience come with Testim comes through how it's used in Ember CLI like the histrionics that undertaken just to bundle all your tests assets and your application assets and your vendor assets and just kind of bootstrap that thing. It's a lot of work. WIL: Another thing with BigTest CLI that doesn't include in Karma and Testim does is a concept of a watcher because all these Bundlers, you have HMR -- hot module reloading, Rollup and things like that. It come with plenty of plugins. Parcel is always set out of the box, so if you're using your Bundler, your existing Bundler to bundle your test, you get that watch feature for free, so it's another complexity that the BigTest CLI kind of eliminates. CHARLES: What it means is we've hidden most of that complexity. Just let the Bundler handle it, right? The Bundler is the part of your project that bundles. WIL: Yeah. CHARLES: You should have your launcher actually doing that for you but we still do need to have some way to do that set up and tear down. When we have that testing endpoint, we have some way to say, "We're starting a test, not the application. We're ending the test, tear it down," so how do you abstract that away? WIL: That's kind of something that we can't really avoid. It is just like some sort of dependency on the framework itself, your application framework. It's like you need to mount a React app. You need to mount an Ember app, etcetera and there's different ways to mount those things. This is one of the things that can't really be decoupled as much as everything else can but BigTest has BigTest React and BigTest Vue and we want to eventually gets like BigTest Ember but really, the main export of all these packages is just a simple mount helper that will mount and clean up your application for you and your testing hooks, whether you're using before each from Mocha or before from something else like Jasmine. You know, no matter what you're doing, you just have a hook that mounts your application and then, cleans it up on the next mount. CHARLES: It's worth pointing out here that this is kind of a core concern of testing and testing big is being able to mount your application and tear it down with regularity and having hooks into that process. Whether you're using BigTest or whether you're not, can you still use BigTest React and BigTest Vue, even if you weren't using anything else? WIL: Yeah, absolutely. Like I said, they just export simple mount helpers. I don't even think they have any other inner BigTest dependencies. They just have pure dependencies on their frameworks. CHARLES: Right and so, you could use it, even if you wanted to roll everything else by hand or you wanted to get started somehow and you needed to do set up and tear down, again this is something that's key to being able to test big, so you should be able to use it independently, whether you use the CLI or not, whether you're using any of the other tools or not. All of the tools can be used independently. WIL: Then another feature of the BigTest React and BigTest Vue is the tear down that happens before set up, rather than happening after your test runs, having a separate tear down. This allows it. Whether your test passes or fails, you can look at it and play with it and inspect it and debug it much easier than if you had tear down. You have to disable at tear down or throw a pause in there to keep other or something. CHARLES: Yeah, I love that. When something goes wrong, you can just let the test case run and the last test that it runs, it just leaves at set up. It does the tear down right before the set up. WIL: Exactly, yeah. At the very end of the whole test run, there's an app there waiting for you to play with. CHARLES: If you focus in on a single test, we most commonly use Mocha, so you say a '.only' to run that single focus test, then you have the state of the application at that test case set up and ready to go. You can just play with it, you can inspect it, you can actually just use it as a starting off point and interact with the app normally as you would. WIL: I want to say, Cypress does this too. They do their tear down before they're set up as well. That's how you're able to play with Cypress test. CHARLES: Yeah, I like that trick. Now, we talked about launching, setup and tear down but we haven't actually talked about much of what actually happens in the test cases themselves. We talked about how to start and launch your test suite, how to do that across a bunch of different browsers, how inside of that, you have a separate concern as applications set up and tear down and how you want to lean on how you're actual app is actually bundled because that fits in with the philosophy of testing big. You don't want to use an external Bundler for your test suite. You want to use your real Bundler, how the asset is actually going to look. But when it comes down to actually writing the tests, you need to be able to interact with at the highest level as you possibly can. When I say highest level, we want to verify that the users, when they take certain actions, we'll see certain outcomes and so, we want those outcomes and we already talked about this to be reflected in a real DOM, in a real browser. But at the same time, the real interactions, we want those to be as high fidelity as possible, so you want to be sending events to the browser. You want real mount events, real key events, real interactions. WIL: Yeah, interacting with application. That's another core philosophy that we kind of talked about earlier that defines a BigTest. It's the user interacting with your application. We're not calling methods and expecting other callbacks or arguments to be passed or clicking on a button and expecting a message to pop up that says, "Form submitted successfully." These are user-facing things were starting on and acting on. CHARLES: Yeah and then, it can be really tricky because these things don't happen synchronously. They're happening inside of your browser's event loop. I click that button and then it goes off and there's some loading state and then, I might get an error message that pops up this thing that animates out and then, goes away. The state of the browser is in constant flux. It's constantly changing and so, it can be very difficult to put your finger and say, I want to be in this state if you are limiting yourself to only reading from the DOM. Some frameworks, Ember for example, you have kind of a white box where you can actually inspect the state of the Ember run loop and use that to do some synchronization but it can be very, very hard to coordinate these interactions. WIL: Yeah. You know, to talk about getting to the solution as a BigTest interactor, which is basically modern page components or page objects. If you ever heard of page objects, it's just a way to encapsulate interacting with big pieces of your pages. It's not a new concept. It's been around for a while but BigTest interactor has kind of a new twist on it where they're immutable, composable interactions that are also convergent, which we'll get into later, which basically means if your buttons not there, it won't click the button until it is there. They're really powerful and they're making really easy and fun to write these tests. CHARLES: Yeah, they're super powerful. I remember we talked about convergences last time when we talked about BigTest but interactors, I think are definitely a new development. I think we should spend a little bit of time there talking about, not just the power but also the ergonomics of interactors because they are like page components or page objects, except they're scope to the component. Not only do they have all this wonderful stuff where it'll make sure that the component exists before it starts to interact with it and things like that but their composable. If I have a button, then there are certain operations that are valid for that button. I can click it. I can hover over it. I can do all these things. They're the operations that make it unique to the button. Now, those might actually map to real events. WIL: Similarly, their assertions about that button as well, like as primary is secondary. If this button is repeated throughout your application, you might want to make sure that your form has a primary and secondary button. CHARLES: Exactly. It really encapsulates all the knowledge of how you can interact with both in terms of taking action and reading state from that button. It almost feels like an accessibility API. It would be easy to write a screen reader if you had these interactors for every single component on the page. WIL: That's kind of what it is. It's just like you're defining an API around how your user would interact with your application and what your user would expect in the application. That's the point of page objects and interactors as you're defining this user API, essentially. CHARLES: Yeah and so, really the step that interactor take is that they take the classic page object and it make them composable, so I can have, you kind of touched on this before, a modal dialogue interactor, which is composed out of two button interactors. One for the primary action, one for the secondary action and maybe, it's aware of its own title text, so you can assert on the title text but I didn't actually have to write the individual button interactors for that modal dialog interactor. Then I might have a second modal dialog interactor or a form that's on a modal dialog just composed of the modal dialog interactors and the individual form components, which appear on that particular modal dialog. WIL: It's essentially how we've been building applications lately with components but this is for page objects in your test if you want to mirror that. You don't have to have one-to-one mappings of an interactor to a component but if you do, it's really powerful. CHARLES: Yeah. I found that when we have one-to-one interactors, that's when it just feels the best. WIL: Yeah and on top of this, if you have a component library and your component library exports the interactor that it uses for the component test, like we said, this BigTest technology, they're sprinkled also. We don't have to use interactors in big acceptance tests. We can use them for smaller component tests too, so if we ship these component interactors with the component library, your application that's consuming this component library now can test those components for free, without having to write their own interactors. It can just compose the interactors exported by the library. CHARLES: Man, I almost want you to repeat that word for word again, just so it can sink in. It's so awesome. Because when you actually go to write your tests, you're not starting from ground zero like, "How do I do this?" They're like, "I'm writing some tests for this thing and I'm using these components and so, I've already got the prepackaged interactions for those components." It's like you start writing your tests. If your tests are a 10-story building, it's like you're starting on Floor 7 and you only have to walk up to Floor 10, instead of slogging up all 10 stories. WIL: One really helpful interactor that we work within the open source stuff we've been working on is a date-picker interactor because date-pickers can be really complex. Just having that common interactor and have a date-picker on multiple forms where we can just use that one interactor, we don't have to tell every single test how to interact with that date-picker. We just say pick date and pass the date. CHARLES: Yeah, it's so awesome. That is actually a great example. It doesn't feel scary to write a test for a page that has a date-picker on it or two. If you're doing like a date range or something like that, you're like, "Oh, my God. I don't write the selectors to test this." You just import your date-picker interactor, you set the date, it actually worries about all the low level events and there you go. It feels like you're operating at a much higher level. WIL: Yeah. The interactor API essentially, you're telling me the test what the user would be doing and what the user would be seeing. CHARLES: Yeah. It's worth pointing out again. We've identified starting and launching. We've identified set up and tear down but interaction is a core concern of BigTesting, no matter what tool you're using. One of the things that we found as interactors are something that you can sprinkle on literally any test suite if you're testing an interface and it makes it better. We've used it inside big acceptance tests. We use it inside Jest, doing just little component tests. There are people in the BigTest community who have used it to basically, write component tests against a JS DOM and while theoretically, philosophically, you want to make those tests as big as you possibly can, you can use that piece in your test suite. If you are using a simulated DOM and if you're running a node in a browser, these interactors will still work and you're going to get high fidelity test cases that are resilient to this asynchrony and are composable and if they do have a full-fledged test suite, you can reuse these interactors. They are a really awesome power up that you can bring into your test suite. WIL: And they are not tied to the framework at all. We use them in React for our stuff but we've also written some in Ember. Robert's written some in the Vue and ported some test and one of the beautiful things we've seen from this is that one interactor goes everywhere. You just write the interactor once and you can use it in Ember, in React, in Vue, in those test suites. If the rest of your test suite is framework agnostic, you have this test suite that you can jump frameworks in your test suites until it works and can test your application with high fidelity. CHARLES: Yeah, it's fantastic. I remember when we first tried using interactors inside an Ember test suite because Ember comes with like a big kitchen sink in testing set up but interactors just slotted right in and there's absolutely no issue. WIL: Yeah and there is actually a speed boost even because in most of the Ember test offers a hook into the Ember run loop and interactors are not. There is actually a good speed boos just using interactors. CHARLES: Yeah. This is a good point. It's a good segue because typically, we think of acceptance tests as being really slow and one of the reasons, even the people [inaudible] acceptance tests or testing big as they think like it's going to take a long time. We found that actually we've been able to maintain a happy medium of testing big but also, having those test be really, really fast. When you say you said a speed boost from using interactors with Ember, where is that speed boost actually come from? WIL: I mentioned the Ember test offers a hook into the Ember run loop and interactors aren't and the reason of this is because interactors are converging and they wait for things in the DOM to exist before interacting with them. Instead of waiting for the framework to settle, it just waits for the thing to appear and then interacts with it immediately. If you're starting something about a button toward the top of the page, you don't really care that another button at the bottom of the page has rendered yet, unless of course you have assertion about that but if they're converging, you don't need to hook into the wrong loop to wait for the entire page to load, to interact with just one piece of it. CHARLES: Right. You're just waiting and you say, "I'm expecting something to happen and the moment I detect it, no matter what else is going on, the page could be taking 30 seconds to load but if that button appears and I can interact with it, I can take my action then or I can make my assertion then." It's about kind of removing gates -- artificial gates. WIL: Yeah. Another common thing that's helped with is animations as most test that are hooked into the run loop, you kind of have to wait for some of these animations to finish before you can even interact with the element and that means if a model has a half second animation where it flies in and you have 30 tests around this modal, those tests are extremely slow now because you have to wait for that modal to come in, whereas -- CHARLES: -- Straight up flaky. WIL: Yeah, straight up flaky. Whereas in the actual DOM, that modal is inserted pretty immediately and can be interacted with pretty immediately. With interactors, they don't need to wait for the animation to finish. They can just immediately interact with that modal but of course, if you need to wait for the animation to finish, there are options for that as well. CHARLES: Yeah. If there's some fade in that needs to happen, you can kind of assert on any state and as long as it's achieved at some point, the interactor will recognize it and recognize it at the soonest possible time that it possibly could. I remember getting bitten on one project where the modal animations in particular were so brutal. Not only were they flaky, they just were slow because there was all these manual time outs. It wasn't even a paper cut. It was kind of like a knife cut, like there's someone sitting there and kind of slashing you with a pocket knife. It just was a constant source of pain in your side. WIL: Yeah and that's how you end up with things like waits and sleeps in your test suite. When you need to wait for the animation to happen or something, you just see a sleep for four seconds with a comment because we have to wait for the components to load in. That's kind of a code now. CHARLES: Yeah, that's just asking for trouble both in terms of slowness and in terms of it's going to get flaky again. That has been kind of one the most freeing things about working with interactors and working with convergent assertions on which they're based is you just don't ever have to worry about asynchrony. Really, really truly, most of the time, you're writing your tests, like it's all synchronous and that kind of makes sense because from the user's perspective, their consciousness is synchronous and they don't care about the internal run loop. It's just they were making observations in serial and at some point, they're going to observe something, so the interactor sits at that point and really observes the application the way that your user would. WIL: Yeah. We've mentioned a few times now the convergent assertions, which interactors are based on. A little caveat there if you're using interactors and you're making non-convergent assertions, they might fail or be flaky. That's because interactors wait for the thing to be there to interact with, so as soon as the buttons there, it clicks it but it doesn't wait for after that event has fired and your application has reacted to that event, that's your application is concerned. We need something there like our convergent assertions that can converge on that state and wait for that state to be true before it considers itself passing or in times out. CHARLES: Maybe we should dig a little bit into convergent assertions. I think the last time we had a public conversation on the podcast about this, this is kind of where we were, like we hadn't built the interactors, we hadn't built these other component pieces of the testing story. We were really focused on the convergent assertion. We've talked a little bit about this but I think it's worth rehashing a little bit because it's a unique way of approaching the system but it's also kind of horrifying when you see how it works under the covers. I think when we tell people about the fact that it's basically polling underneath the covers. The timeout is configurable but it's basically polling every 10 milliseconds to observe a state. I remember the first time being confronted with this idea and I was horrified and like my programmer hackles on the back of my neck, like raised up and I was like, "Wait a minute. This is going to be slow. It's going to be computationally intensive." WIL: Yeah. That was my exact thought too because this is going to be slow. If acceptance tests are slow and we're doing an acceptance test every 10 milliseconds, it's going to be really slow and that's actually not the case completely. It's actually the opposite. They're extremely fast. CHARLES: It is shockingly fast. You've got to try it to believe how fast it is, how fast you can run acceptance tests. WIL: Yeah, talking like 100 tests in just tens of seconds. CHARLES: Right. You basically gated by how fast your framework can render. Your tests are not part of the slowness. Your test -- WIL: And also, memory leaks can be costly too. We experience that recently where we had memory leaks that were slowing down our test but we fixed those up in test and put our backup. CHARLES: Yeah, because basically, running the assertion or running the convergence is very fast. It's just a very light ping. I kind of think of it is as it is light as the brush of a photon or something that was bouncing off of a surface, so that you can observe it. It's extremely light and most of the time, it's just waiting so the test and the convergence really just gets out of the way. Just because they can run a thousand times or a hundred times in a second, it's doesn't gun it up. But the thing is it means that your tests run as fast as your application will run. You get back to the point... Was it in React where the kind of the key insight is that JavaScript is not the bottleneck? Well, your tests are not the bottleneck. WIL: Yeah. CHARLES: I guess this is what it is. I don't know if there's anything else that you want to say about convergences. WIL: No. We pretty much summed it up there and that's what interactors are based on. That's how they're able to wait for things in a DOM. It basically polls the DOM until it exists and then it moves on and actually does the interaction. CHARLES: Once again, this is actually a very low level thing on which BigTest is based but this is once again, something that you can use independently. You can write your own convergent assertions. You can write your own convergences that honestly have nothing to do with testing your assertions. It's a free standing library that you can use in your test suite or elsewhere should you choose. WIL: That doesn't need to be a DOM for BigTest convergence there. I use BigTest convergence in BigTest CLI to converge on the browser being launched. Instead of waiting for the browser to report that, I can just kind of poll and see how that process is doing and the convergence waits for that process to start before moving on. CHARLES: Right. I guess the best way I've thought about it it's a way to synchronize on observations and not on callbacks. It's a synchronization mechanism and 99% of the synchronization mechanisms that we're used to, they've involved some sort of callback, a promise, an event-listener, things like that or even a generator where control is handed back explicitly to a piece of code when something happens. Whereas, this is a fundamentally different synchronization primitive, where you are writing synchronous code that's based on observations, so what I observe this, do this. When I observe this, do this. It's extremely robust. WIL: Yeah, very. CHARLES: It is a core piece. A fundamental thing that on which interactors are based on, which the CLI is based, I don't know if it's core to writing tests but -- WIL: It definitely helps. CHARLES: It doesn't helps. We couldn't have BigTest interactor without that. WIL: No, definitely not. CHARLES: Because that's what makes it fast, that's what makes it not flaky at all and having those things, I think it makes it easy to maintain because you can work at the interactor level or the level of user interaction and you don't have to worry about synchronization, so the flow of your tests are very natural. WIL: Yeah. We don't have to explicitly wait for request to be done for making an assertion about your app. That'll just come with convergences, just waiting for test date in application to true. CHARLES: Let's talk about one more piece of the testing issue because when you're testing big, when you're testing in the browser, there's always the issue of what are you going to do about your API. You got to have your API running. It's just always an issue and this is kind of interesting because this sits at the crossroads of testing big and also, getting the most utility out of your test because in an ideal world, if you're testing really big, you're going to be using a real API. You're not going to poke holes in reality. WIL: Yeah. One of the things that we avoid in BigTest is poking holes. We're not shallow mounting the components and testing the methods and the results. We're fully mounting these things and fully interacting with them through the full DOM API. CHARLES: Yeah, exactly, using real browsers. It just occurred to me the irony of us talking about reality being things that are still running inside of a computer processor. I think we've inherited this term from that talk that Justin Searls at AssertJS in 2017. It's a really, really excellent talk. I think he gave it at RubyConf. It's the 'Don't mock me.' WIL: Yeah, it's one of my favorite talks. CHARLES: Yeah, it's a great talk. In it, he talks about the value of a test is a balance of how many holes you poke in reality and sometimes, you encounter a test where all it is like holes in reality. Whether you're mocking this, you're mocking that, you're mocking the DOM, you're mocking the browser, you're mocking your network layer, you're mocking this external API and the more holes you poke, the less useful it's going to be. Network is one of those where it can be very difficult to not poke holes in that reality because it's a huge part of your application. Your frontend application is how it's going to interact with the server but at the same time, servers are gigantic pieces of software themselves, each with their own dependencies, each with their own set up and tear down -- WIL: Have their own concerns. CHARLES: Yeah, exactly. They might be in a different language. They've got runtime, things like they might need external C libraries and crazy stuff like that. They're their own beast. To get a true big end-to-end test, you going to have to stand up your server but the problem that presents is you want your tests to be also isolatable. If you're a developer, I can go to a repo, I can do an install of my dependencies and I can run the tests without having to do any external dependencies other than the repository and the language in which I'm working. This is one where we kind of have tried to walk the line of not wanting to poke holes in reality but also, have the test be containable to the actual application. In order to do that, you need something that presents a high fidelity version of the network. You can kind of try and have your cake and eat it too. You want to have something that acts like a server and really acts like a server but it's actually not a server. WIL: And still poke as few holes as possible and the application and how that's all set up, we don't want to be intercepting methods and responding with fake data. That's not a good way to mock that network. CHARLES: Right. We want to be calling actual fetches, calling actual XMLHttpRequest. Ideally, if you've got service workers, making actual service worker requests. WIL: Basically, as far as the application is concerned, it's talking to a real server on us. CHARLES: Yeah and that's kind of the litmus test for is it a hole in reality or is it just a really great illusion? WIL: Yeah and that's a good name for Mirage, right? It's a really great illusion. CHARLES: Yeah. It is a simulation of reality, so we use Mirage, which is something from the Ember testing world but something that we have extracted and made available as BigTest Mirage. WIL: Yeah. The main difference just being is that we've taken away the Ember dependencies and the run loop stuff. It's just plain JavaScript Mirage. It works exactly the same as you use it in Ember minus the auto imports and the file... Oh, man. I can't think of that word. Aside from automatically importing your files for your server config, you have to do that manually because Ember is what provides that but other than that, it's a form of Mirage. You define models and serializers and factories and all the good stuff. CHARLES: Right and then you can use those factories and you can use those models to really give a high fidelity server. If you are building something in whatever framework, you can use BigTest Mirage to simulate that network layer. Again, we've used it in a number of different scenarios but having that in place means that you're going to be able to have those high fidelity tests where your application is actually making XMLHttpRequest but it's all isolatable, so that it can be run in repo. This isn't really related to testing but it has a fantastic capability where you can prepopulate, you can use the factories to prepopulate your server with data, so that you can use the application without the actual server being implemented. WIL: Yeah. That's extremely powerful. That's we were talking about earlier and getting at is the scenarios which are setting up specific, essentially fixtures but you're generating these fixtures. Factories are essentially high level fixtures, network of fixtures. CHARLES: Yeah, higher order of fixtures. WIL: Yeah, so the scenarios are just setting up these fixtures for a scenario of your applications, like the backend is down or the list only responds with two items as opposed to 5000 items, something like that. You want to be able to, not only test these things but be able to develop against it and Mirage makes that really easy because you can just start your app with Mirage-enabled point to that scenario and you're there. You have that exhausted scenario to develop in. CHARLES: If you've never used Mirage, it is really hard to understand just how incredibly powerful it can be. We've used it now on at least four projects, where we did develop the entire first version of the product without any backend whatsoever. It's an incredible product development tool, even apart from testing, that then informs the shape of what the API was going to be. I know we've talked about this on the podcast before but it's really an incredible technology and it is available to you no matter what framework you're using. I think it's one of the best kept secrets in JavaScript development. WIL: Yeah. That's definitely great. That said, though it does have some fallacies. It's great but it can be a little slow sometimes, so we are eventually working on a BigTest network like another piece of the BigTest pie that you'll be able to sprinkle into your application but in the meantime, praise Mirage. CHARLES: Yeah. We are going to be offering an alternative or maybe collaborating for another version of Mirage but hopefully, we can make Mirage faster, we will be able to make this thing faster, so that it can use service workers and be used in a bunch of different scenarios. Just to recap, we've talked about a lot of different components but over the past year, a couple of years, these are the things that we've identified as being really key components as big part of your acceptance testing and really your testing stack. How you're going to start and launch these things? How are you going to set them up and tear them down? How are you going to interact with the application from a user, both in terms of making assertions and how are you going to take action on behalf of the user and still have it be maintainable, have it be resistant to flakiness, have it be performant? BigTest is the answer to that for those particular areas of the testing story and so, some were using we're using existing components like we use Karma, we use Mirage to date. Those, we did not develop but where we see kind of key pieces of that puzzle missing is where we started writing the BigTest solutions so things like the interactor. Eventually, we are going to make BigTest into a product that's you're going to be able to use kind of out of the box, just like you might install Cypress, where it's a very quick set up and we make all of the decisions about the components for you. But in the meantime, we're really trying to take our time, identify those pieces of the puzzle and build the software component that fits that piece of the puzzle at the absolute best so when they're polished, use them in a more comprehensive product. Things like convergence, things like interactor, things like BigTest React, BigTest Vue and very soon, BigTest Ember. These are things that you can use today, to make your tests just that much bigger and that much better, especially interactor. It's been an incredible journey this past year as we kind of develop these individual pieces and there's just going to be more goodness to come. WIL: Absolutely. Right now, I'm working on some validation type API for interactor that I'm hoping to land soon. That'll open up the possibilities of maybe hiding away those convergent assertions a bit more in your tests and just handling this automatically. It'll be pretty good. CHARLES: It's really exciting. Writing test has got more and more easy and more and more fun over the last year for us. I think we're already starting in a pretty good place. If you have any questions about BigTest, how would folks get in touch with us? WIL: We have a BigTest Gitter channel. You can find a link to that on the BigTest website: BigTestJS.io. Just ask us questions on Gitter and we'll try to answer them. CHARLES: And as always, you can ask us directly. You can send email to Contact@Frontside.io or reach out to us on Twitter at @TheFrontside or you can actually reach out to the BigTestJS Twitter account directly and just call us on Twitter at @BigTestJS. Thank you very much, Wil. WIL: Thank you, Charles.

Panel: Charles Max Wood Chris Fritz Erik Hanchett Divya Sasidharan In this episode, the panel talks amongst themselves on the topic: how does one contribute to opensource work? They discuss the various ways that they contribute, such as speaking at conferences, recording videos for YouTube, podcasting, among others. Check-out today’s episode to get some insight and inspiration of how YOU can contribute to YOUR community!  Show Topics: 1:11 – We have decided we haven’t completed this topic 1:23 – Last time we went around the panel and see how we contribute? One of the ways I contribute to opensource is organizing events and conferences. Divya, you write some code – a little bit? 2:05 – Divya. 2:11 – Panelist: Divya, you speak at conferences, write blog posts, and code. Super top-secret project? 2:33 – Divya: I am trying to grow. Maybe I can talk about the secret project later? 2:56 – Panelist: Yes, I contribute through videos and education. I’ve tried in the past seeing issues in opensource, but I find that I am better at teaching. Charles you run a Vue Podcast? 3:29 – Chuck: Yeah, that’s what they say. I work on the podcasts, online conferences, eBooks, and online summits. Lastly, Code Badges that is on Kickstarter. 4:06 – Panelist: How we can contribute to opensource and still make a living. What is free and what we charge for? Finding a balance is important – we covered that last time. How to get into opensource in a variety of ways: How do you start speaking at conferences? How to you write code for opensource? Divya, how do they start? Do you need a public speaking degree? 5:29 – Divya: It might help. To get started with public speaking – it’s deceptively easy but then it’s not at the same time. You submit a proposal to a conference and it’s either accepted or declined. You have to learn how to CRAFT your ideas in a CFP to show the panel that this topic is RELEVANT to the conference and that you are an expert. It’s not the speaking that’s the hard part it’s the writing of the proposal. 7:00 – Panelist: You have talked about CFP – what is that? 7:09 – Divya: It’s a Call For Papers (CFP). It’s just a process of being accepted at a conference. Sometimes conferences have an open call – where they might have a Google form or some software to fill out some details. They will ask for your personal details, a short draft, the title of your talk, and a longer description (why you should be the speaker, etc.). It’s a multi-step process. Even though YOU are the right person to talk about X topic – you don’t have to be – you just have to SOUND like you know what you are talking about. Show that you’ve done your researched, and that you have some understanding. Also, that you are capable of presenting the information at the conference. That’s what I mean by being “THE BEST” person. 9:33- Charles: They aren’t looking always for the expert-level of explaining X topic. Even if it’s at the basic level that’s great. If you can deliver it well then they might pick your proposal. I have spoken at a number of conferences, and I started talking at Meetups. Most organizers are desperate for people to give talks. If you talk at these informal settings – then you get feedback from 10:47 – Divya: Yes, lightning talks are great for that, too. This way you are flushing out what you do and don’t want to talk about. 11:07 – Charles: A lot of people don’t realize that they are good speakers. The way to get better is to do it. I am a member of Toast Masters. You gain experience by talking at many different events. 12:23 – Panelist: I don’t know much about Toast Masters – what is it? 12:29 – Charles: Toast Masters, yes, they collect dues. As you sit in the meeting you have time to give feedback and get feedback. They have a “MM” master, and a grammatical master, and another specialist that they give you feedback. It’s a really constructive and friendly environment. 13:42 – I’ve been to Toast Masters and the meetings are early in the morning. 7:00 or 7:30 AM start time. Everything Chuck just said. I went to a couple and they don’t force you to talk. You can go just to see what it’s about. 14:21 – Charles makes more comments. 14:48 – Meetups is a great way to get into the community, too. What if Toast Masters sounds intimidating, and you don’t think you can speak at a Meetup just, yet. Are there more 15:18 – You can be the town crier. Stand on the soapbox and... 15:32 – There is someone sitting on a soapbox and screaming to a crowd. 15:43 – Chuck: You can do a YouTube video or a podcast, but I think getting the live feedback is super important. Toastmasters are so friendly and I’ve never been in front of a hostile crowd. You get up and they are rooting for you. It’s not as scary as you make it out to be. You aren’t going to ruin your reputation. 16:48 – Local Theater! That helps a lot, to me, because you have lines to read off of the script. You are a character and you get to do whatever you want. Also, teaching really helps. You don’t have to be a professional teacher but there are volunteer areas at a local library or your community centers and libraries. Find opportunities! 18:18 – Divya: Improvisation is good for that, too, back to Chris’ point. Improvisation you don’t have the lines, but it forces you to think on the spot. It helps you practice to think on the spot. 19:04 – Teaching is good for that, too. It makes you think on the spot. You have to respond on the fly. Life teaching is Improvisation. 19:31 – Charles: You learn the patterns that work. 19:57 – Panelist: There are some websites that can track your CFP due dates. You can apply to talk to 5-6 different conferences. You pitch the same idea to 5-6 conferences and you are bound to get picked for at least 1 of those conferences. 20:51 – Divya: There is an account that tweets the CFP due dates that are closing in 1-2 weeks. Check Twitter. 21:25 – Chuck: Take your CFP and have someone else look at it. I know a bunch of conference organizers and ask them for their feedback. 21:48 – Title and description need to be there. 22:48 – Divya: Look at past events to see what was already done in past conferences. This is to see what they are kind of looking for. Divya talks about certain conferences and their past schedules. 23:52 – Eric was saying earlier that you could send in more than 1 proposal. Another one suggests sending in 3 proposals. Someone would love to accept you, but say there is someone else you beats you by a hair. 24:31 – Divya: The CFP process is usually blind and they don’t “see” you until later. Most conferences try to do this so there is no bias. They will ask for no name, but only focusing on content. 25:28 – Sarah May has some great suggestions. Look at the show notes under LINKS. 25:57 – Advertisement – Get A Coder Job! 26:34 – We have talked about how you submit your proposals. Maybe let’s transition into another topic, like education. Eric – do you have any tips into writing blog posts and such? 27:36 – Eric: Find a topic that you want to learn and/or you are expert on. Going out there and putting out content for something you are learning. If you get something wrong then someone will probably call you out. Like Reddit you might get more criticism then vs. your own blog. I look for topics that interest me. 28:30 – Panelist: How do you get people to see it? 28:40 – Eric: Consistency – sharing on your social media channels. Reddit, Frontend, and/or other sites. I’m doing this for myself (first), and secondary I am teaching other people. 29:23 – Getting feedback from people is great. 29:40 – Eric: It’s a process to build that audience, build quality content, and keep up with it. Facebook groups – hey I put this content out there. Another way you can do it is work with a publisher and try going to a site called PluralSite. 30:47 – Do you have to be famous, like Joe, to get onto their site? 31:09 – Chuck: The audition process I got screwed on. They ask you to record a video, fix anything in the video, and then they will tell you if they will accept your courses or not. 31:37 – People who will distribute your content, there is a screening process. Guest blog, too, will get your name out there. 32:23 – Chuck: You just have to be a level above the reader. 32:37 – Odds are that you can explain it better than someone who learned it 5 years ago. Even if it’s a basic JavaScript thing that you JUST learned, who cares put it out there. If you made X mistake then I’m sure thousands of other developers have made the same mistake. 33:17 – Twitter is a great platform, too. A short and sweet Tweet – show them your main idea and it can get 34:01 – Comments. 34:04 – I use Ghost for my blogging platform. You can start off on Wordpress and others write on Medium. 34:25 – Divya: I like to own my own content so I don’t write on Medium anymore. 34:40 – I like my content on my OWN site. That’s why I haven’t been using Medium anymore. There are more pop-ups and such, too, so that’s why I don’t like it. 35:06 – Divya: If you don’t want to start up your own site, Medium is nice. Other users pick it up, which is an easy way to spread content right away. 37:13 – Chuck: Some of them will pay you for that. 37:23 – Sarah Drasner on the Vue team is an editor of CSS tricks. Good way to get your content out there. 37:48 – Divya: Sarah will work with you. Not only do you get access to put content out there, but then you get feedback from Sarah, too! 38:19 – Remember if you are doing a guest post – make sure to put out solid examples and good content. You want to put time and effort into it, so put more 39:02 – Any more advice on educational content? 39:11 – Chuck: I am always looking for guests for the podcasts and topics. You reach out and say I would like to be a guest on such and such a show. 39:39 – I thought back in the day – oh those podcast hosts are for THOSE famous people. They must have some journalism degree, and here I AM! It apparently is not that bad. 40:19 – Chuck: When I was coding semi-professionally for 1 year and my friend Eric Berry (Teach Me To Code – website) he was looking for someone to record videos for him. I submitted a video and I just walked through how to do basic routing. Basic for Ruby on Rails users, and I said that this is my first video. I tweeted that information. Screen Flow reached out to me because I mentioned their name, and I got a license and a microphone to help me record my videos! That gave me the confidence to start podcasting. It’s scary and I’m thinking I will screw this up, I don’t have professional equipment, and look at me now! 42:46 – To be a podcast host it isn’t much. 42:55 – Chuck: I am trying to make podcasting easier. The hard part is preparing the content, get it edited, getting it posted. It’s all the other stuff. Recording and talking isn’t that bad. 43:28 – What are my steps if I want to start a new podcast? 43:39 – What microphone should I get? 43:48 - $100-$130 is the Yeti microphone. Do I need a professional microphone? People can’t tell when guests talk on their iPhone microphone or not. Especially if you already have those then you won’t be out if you don’t want to continue with podcasting. Record for free with Audacity. Have something to talk about and somewhere to post it. 45:01 – Panelist asks Chuck more questions. 45:13 – Divya. 45:29 – It’s easier if everyone is in the same room. If the sound quality is good enough then people will stay, but if the quality is poor then people will go away. I recommend Wordpress - it’s super easy. You can host on Amazon, but if you will host long-term then use Libsyn or Blubrry. Great platforms will cost you less then some others. 46:58 – iTunes? 47:04 – Podcast through iTunes you just give them a RSS feed. All you do is fill out some forms. Submit that and it will run – same for Google Play. You might want to get some artwork. In the beginning for me I got a stock image – edited it – and that was it. One I got one of my headshots and put the title on there. 48:06 – Then when people will hear this... 48:23 – Summary: microphone, content, set up WordPress, submit it to iTunes, and record frequently. Keep improving. 48:46 – Anything you are doing anything online – make sure your mantra is “this is good enough.” If you spend tons of hours trying to perfect it – you might drive yourself crazy. 49:18 – Not everyone will enjoy podcasting or YouTubing – so make sure you don’t invest a lot of money at first to see where you are. 50:06 – Educational content topic continued. Contributing to coder depositories. What’s the best way to get into that? 50:28 – Chuck: Some will say: This one is good for a newbie to tackle. You just reach out – don’t just pick it up and tackle it – I would reach out to the person first. Understand what they need and then work on it, because they might have 2 other people working on it. 51:11 – Divya: Hacktoberfest – Digital Ocean – they publish opensource projects. 52:22 – Yeah check it out because you can get a free t-shirt! 53:50 – Chuck: Doing the work that the hotshots don’t want to do. It helps everyone out, but it might not be the most glamorous job.  55:11 – Spelling mistakes – scan the code base. 55:43 – Divya: If you do small contributions that people DON’T want to do – then these contributors will see you and you will be on their radar. You start building a relationship. Eventually people will start giving you more responsibilities, etc. 56:59 – Chuck: I have seen people been contributors through Ruby on Rails. They got the gig because the core team sees your previous work is reliable and good work. 57:26 – Is there a core contributor guideline? 57:37 – Good question. If Divya likes you then you are in. 57:47 – It’s Evan who makes those decisions, but we are working on a formal guideline. 58:52 – Will they kick you out? 59:00 – Unless they were doing bad stuff that means pain for other people you won’t get kicked out. 59:33 – Representing Vue to some degree, too. The people who are representing Vue are apart of it. We are trying to get a better answer for it, so it’s complicated, but working on it. 1:00:02 – How did you get on the team? Well, I was contributing code, I was discussing ways to better x, y, and z. Evan invited me to come into the core team. Basically he did it so he wouldn’t have to keep babysitting us. 1:01:06 – Chuck. 1:01:20 – Panelist. 1:01:48 – Panelist: One of our core team members got his job because he was answering questions from the community. He is not a software developer by training, but his background is a business analyst. You don’t have to contribute a ton of code. He was a guest so check out the past episode. See show notes for links. 1:03:05 – Chuck: We need to go to picks and I think that topic would be great for Joe! 1:03:24 – Advertisement – Fresh Books! Links: Vue React Angular JavaScript DevChat TV GitHub Meetup Ghost.Org Miriam Suzanne’s Twitter Sarah Mei’s Article: What Your Conference Proposal is Missing WordPress Sarah Drasner’s Twitter CSS Tricks Netlify Sponsors: Get A Coder Job! Cache Fly Kendo UI Picks: Eric Headless CMS Dyvia Blogspot - Building a 3D iDesigner with Vue.js The Twitch Streamers Who Spend Years Broadcasting to No One Chris Cat Content Twitter Account https://www.patreon.com/akryum The Great British Baking Show Charles Embrace the Struggle SoftCover.io getacoderjob.com swag.devchat.tv

Panel: Charles Max Wood Chris Fritz Erik Hanchett Divya Sasidharan In this episode, the panel talks amongst themselves on the topic: how does one contribute to opensource work? They discuss the various ways that they contribute, such as speaking at conferences, recording videos for YouTube, podcasting, among others. Check-out today’s episode to get some insight and inspiration of how YOU can contribute to YOUR community!  Show Topics: 1:11 – We have decided we haven’t completed this topic 1:23 – Last time we went around the panel and see how we contribute? One of the ways I contribute to opensource is organizing events and conferences. Divya, you write some code – a little bit? 2:05 – Divya. 2:11 – Panelist: Divya, you speak at conferences, write blog posts, and code. Super top-secret project? 2:33 – Divya: I am trying to grow. Maybe I can talk about the secret project later? 2:56 – Panelist: Yes, I contribute through videos and education. I’ve tried in the past seeing issues in opensource, but I find that I am better at teaching. Charles you run a Vue Podcast? 3:29 – Chuck: Yeah, that’s what they say. I work on the podcasts, online conferences, eBooks, and online summits. Lastly, Code Badges that is on Kickstarter. 4:06 – Panelist: How we can contribute to opensource and still make a living. What is free and what we charge for? Finding a balance is important – we covered that last time. How to get into opensource in a variety of ways: How do you start speaking at conferences? How to you write code for opensource? Divya, how do they start? Do you need a public speaking degree? 5:29 – Divya: It might help. To get started with public speaking – it’s deceptively easy but then it’s not at the same time. You submit a proposal to a conference and it’s either accepted or declined. You have to learn how to CRAFT your ideas in a CFP to show the panel that this topic is RELEVANT to the conference and that you are an expert. It’s not the speaking that’s the hard part it’s the writing of the proposal. 7:00 – Panelist: You have talked about CFP – what is that? 7:09 – Divya: It’s a Call For Papers (CFP). It’s just a process of being accepted at a conference. Sometimes conferences have an open call – where they might have a Google form or some software to fill out some details. They will ask for your personal details, a short draft, the title of your talk, and a longer description (why you should be the speaker, etc.). It’s a multi-step process. Even though YOU are the right person to talk about X topic – you don’t have to be – you just have to SOUND like you know what you are talking about. Show that you’ve done your researched, and that you have some understanding. Also, that you are capable of presenting the information at the conference. That’s what I mean by being “THE BEST” person. 9:33- Charles: They aren’t looking always for the expert-level of explaining X topic. Even if it’s at the basic level that’s great. If you can deliver it well then they might pick your proposal. I have spoken at a number of conferences, and I started talking at Meetups. Most organizers are desperate for people to give talks. If you talk at these informal settings – then you get feedback from 10:47 – Divya: Yes, lightning talks are great for that, too. This way you are flushing out what you do and don’t want to talk about. 11:07 – Charles: A lot of people don’t realize that they are good speakers. The way to get better is to do it. I am a member of Toast Masters. You gain experience by talking at many different events. 12:23 – Panelist: I don’t know much about Toast Masters – what is it? 12:29 – Charles: Toast Masters, yes, they collect dues. As you sit in the meeting you have time to give feedback and get feedback. They have a “MM” master, and a grammatical master, and another specialist that they give you feedback. It’s a really constructive and friendly environment. 13:42 – I’ve been to Toast Masters and the meetings are early in the morning. 7:00 or 7:30 AM start time. Everything Chuck just said. I went to a couple and they don’t force you to talk. You can go just to see what it’s about. 14:21 – Charles makes more comments. 14:48 – Meetups is a great way to get into the community, too. What if Toast Masters sounds intimidating, and you don’t think you can speak at a Meetup just, yet. Are there more 15:18 – You can be the town crier. Stand on the soapbox and... 15:32 – There is someone sitting on a soapbox and screaming to a crowd. 15:43 – Chuck: You can do a YouTube video or a podcast, but I think getting the live feedback is super important. Toastmasters are so friendly and I’ve never been in front of a hostile crowd. You get up and they are rooting for you. It’s not as scary as you make it out to be. You aren’t going to ruin your reputation. 16:48 – Local Theater! That helps a lot, to me, because you have lines to read off of the script. You are a character and you get to do whatever you want. Also, teaching really helps. You don’t have to be a professional teacher but there are volunteer areas at a local library or your community centers and libraries. Find opportunities! 18:18 – Divya: Improvisation is good for that, too, back to Chris’ point. Improvisation you don’t have the lines, but it forces you to think on the spot. It helps you practice to think on the spot. 19:04 – Teaching is good for that, too. It makes you think on the spot. You have to respond on the fly. Life teaching is Improvisation. 19:31 – Charles: You learn the patterns that work. 19:57 – Panelist: There are some websites that can track your CFP due dates. You can apply to talk to 5-6 different conferences. You pitch the same idea to 5-6 conferences and you are bound to get picked for at least 1 of those conferences. 20:51 – Divya: There is an account that tweets the CFP due dates that are closing in 1-2 weeks. Check Twitter. 21:25 – Chuck: Take your CFP and have someone else look at it. I know a bunch of conference organizers and ask them for their feedback. 21:48 – Title and description need to be there. 22:48 – Divya: Look at past events to see what was already done in past conferences. This is to see what they are kind of looking for. Divya talks about certain conferences and their past schedules. 23:52 – Eric was saying earlier that you could send in more than 1 proposal. Another one suggests sending in 3 proposals. Someone would love to accept you, but say there is someone else you beats you by a hair. 24:31 – Divya: The CFP process is usually blind and they don’t “see” you until later. Most conferences try to do this so there is no bias. They will ask for no name, but only focusing on content. 25:28 – Sarah May has some great suggestions. Look at the show notes under LINKS. 25:57 – Advertisement – Get A Coder Job! 26:34 – We have talked about how you submit your proposals. Maybe let’s transition into another topic, like education. Eric – do you have any tips into writing blog posts and such? 27:36 – Eric: Find a topic that you want to learn and/or you are expert on. Going out there and putting out content for something you are learning. If you get something wrong then someone will probably call you out. Like Reddit you might get more criticism then vs. your own blog. I look for topics that interest me. 28:30 – Panelist: How do you get people to see it? 28:40 – Eric: Consistency – sharing on your social media channels. Reddit, Frontend, and/or other sites. I’m doing this for myself (first), and secondary I am teaching other people. 29:23 – Getting feedback from people is great. 29:40 – Eric: It’s a process to build that audience, build quality content, and keep up with it. Facebook groups – hey I put this content out there. Another way you can do it is work with a publisher and try going to a site called PluralSite. 30:47 – Do you have to be famous, like Joe, to get onto their site? 31:09 – Chuck: The audition process I got screwed on. They ask you to record a video, fix anything in the video, and then they will tell you if they will accept your courses or not. 31:37 – People who will distribute your content, there is a screening process. Guest blog, too, will get your name out there. 32:23 – Chuck: You just have to be a level above the reader. 32:37 – Odds are that you can explain it better than someone who learned it 5 years ago. Even if it’s a basic JavaScript thing that you JUST learned, who cares put it out there. If you made X mistake then I’m sure thousands of other developers have made the same mistake. 33:17 – Twitter is a great platform, too. A short and sweet Tweet – show them your main idea and it can get 34:01 – Comments. 34:04 – I use Ghost for my blogging platform. You can start off on Wordpress and others write on Medium. 34:25 – Divya: I like to own my own content so I don’t write on Medium anymore. 34:40 – I like my content on my OWN site. That’s why I haven’t been using Medium anymore. There are more pop-ups and such, too, so that’s why I don’t like it. 35:06 – Divya: If you don’t want to start up your own site, Medium is nice. Other users pick it up, which is an easy way to spread content right away. 37:13 – Chuck: Some of them will pay you for that. 37:23 – Sarah Drasner on the Vue team is an editor of CSS tricks. Good way to get your content out there. 37:48 – Divya: Sarah will work with you. Not only do you get access to put content out there, but then you get feedback from Sarah, too! 38:19 – Remember if you are doing a guest post – make sure to put out solid examples and good content. You want to put time and effort into it, so put more 39:02 – Any more advice on educational content? 39:11 – Chuck: I am always looking for guests for the podcasts and topics. You reach out and say I would like to be a guest on such and such a show. 39:39 – I thought back in the day – oh those podcast hosts are for THOSE famous people. They must have some journalism degree, and here I AM! It apparently is not that bad. 40:19 – Chuck: When I was coding semi-professionally for 1 year and my friend Eric Berry (Teach Me To Code – website) he was looking for someone to record videos for him. I submitted a video and I just walked through how to do basic routing. Basic for Ruby on Rails users, and I said that this is my first video. I tweeted that information. Screen Flow reached out to me because I mentioned their name, and I got a license and a microphone to help me record my videos! That gave me the confidence to start podcasting. It’s scary and I’m thinking I will screw this up, I don’t have professional equipment, and look at me now! 42:46 – To be a podcast host it isn’t much. 42:55 – Chuck: I am trying to make podcasting easier. The hard part is preparing the content, get it edited, getting it posted. It’s all the other stuff. Recording and talking isn’t that bad. 43:28 – What are my steps if I want to start a new podcast? 43:39 – What microphone should I get? 43:48 - $100-$130 is the Yeti microphone. Do I need a professional microphone? People can’t tell when guests talk on their iPhone microphone or not. Especially if you already have those then you won’t be out if you don’t want to continue with podcasting. Record for free with Audacity. Have something to talk about and somewhere to post it. 45:01 – Panelist asks Chuck more questions. 45:13 – Divya. 45:29 – It’s easier if everyone is in the same room. If the sound quality is good enough then people will stay, but if the quality is poor then people will go away. I recommend Wordpress - it’s super easy. You can host on Amazon, but if you will host long-term then use Libsyn or Blubrry. Great platforms will cost you less then some others. 46:58 – iTunes? 47:04 – Podcast through iTunes you just give them a RSS feed. All you do is fill out some forms. Submit that and it will run – same for Google Play. You might want to get some artwork. In the beginning for me I got a stock image – edited it – and that was it. One I got one of my headshots and put the title on there. 48:06 – Then when people will hear this... 48:23 – Summary: microphone, content, set up WordPress, submit it to iTunes, and record frequently. Keep improving. 48:46 – Anything you are doing anything online – make sure your mantra is “this is good enough.” If you spend tons of hours trying to perfect it – you might drive yourself crazy. 49:18 – Not everyone will enjoy podcasting or YouTubing – so make sure you don’t invest a lot of money at first to see where you are. 50:06 – Educational content topic continued. Contributing to coder depositories. What’s the best way to get into that? 50:28 – Chuck: Some will say: This one is good for a newbie to tackle. You just reach out – don’t just pick it up and tackle it – I would reach out to the person first. Understand what they need and then work on it, because they might have 2 other people working on it. 51:11 – Divya: Hacktoberfest – Digital Ocean – they publish opensource projects. 52:22 – Yeah check it out because you can get a free t-shirt! 53:50 – Chuck: Doing the work that the hotshots don’t want to do. It helps everyone out, but it might not be the most glamorous job.  55:11 – Spelling mistakes – scan the code base. 55:43 – Divya: If you do small contributions that people DON’T want to do – then these contributors will see you and you will be on their radar. You start building a relationship. Eventually people will start giving you more responsibilities, etc. 56:59 – Chuck: I have seen people been contributors through Ruby on Rails. They got the gig because the core team sees your previous work is reliable and good work. 57:26 – Is there a core contributor guideline? 57:37 – Good question. If Divya likes you then you are in. 57:47 – It’s Evan who makes those decisions, but we are working on a formal guideline. 58:52 – Will they kick you out? 59:00 – Unless they were doing bad stuff that means pain for other people you won’t get kicked out. 59:33 – Representing Vue to some degree, too. The people who are representing Vue are apart of it. We are trying to get a better answer for it, so it’s complicated, but working on it. 1:00:02 – How did you get on the team? Well, I was contributing code, I was discussing ways to better x, y, and z. Evan invited me to come into the core team. Basically he did it so he wouldn’t have to keep babysitting us. 1:01:06 – Chuck. 1:01:20 – Panelist. 1:01:48 – Panelist: One of our core team members got his job because he was answering questions from the community. He is not a software developer by training, but his background is a business analyst. You don’t have to contribute a ton of code. He was a guest so check out the past episode. See show notes for links. 1:03:05 – Chuck: We need to go to picks and I think that topic would be great for Joe! 1:03:24 – Advertisement – Fresh Books! Links: Vue React Angular JavaScript DevChat TV GitHub Meetup Ghost.Org Miriam Suzanne’s Twitter Sarah Mei’s Article: What Your Conference Proposal is Missing WordPress Sarah Drasner’s Twitter CSS Tricks Netlify Sponsors: Get A Coder Job! Cache Fly Kendo UI Picks: Eric Headless CMS Dyvia Blogspot - Building a 3D iDesigner with Vue.js The Twitch Streamers Who Spend Years Broadcasting to No One Chris Cat Content Twitter Account https://www.patreon.com/akryum The Great British Baking Show Charles Embrace the Struggle SoftCover.io getacoderjob.com swag.devchat.tv

Guests: Amanda Hickman: @amandabee | GitHub Amberley Romo: @amberleyjohanna | GitHub | Blog In this episode, Amanda Hickman and Amberley Romo talk about how they paired up to get the safety pin, spool of thread, and the knitting yarn and needles emojis approved by the Unicode Committee so that now they are available for use worldwide. They also talk about how their two path crossed, how you can pitch and get involved in making your own emojis, and detail their quest to get a regular sewing needle approved as well. Resources: Unicode Technical Committee Draft Emoji Candidates The Unicode Consortium Members Sewing-Emoji Repo Proposal for Sewing NEEDLE AND THREAD Emoji This show was produced by Mandy Moore, aka @therubyrep of DevReps, LLC. Transcript: ROBERT: Hello, everyone. Welcome to Episode 112 of The Frontside Podcast. I'm Robert DeLuca, a software developer here at the Frontside and I'll be your episode host. With me as co-host is Charles Lowell. Hey, Charles. CHARLES: Hello, Robert. Good morning. ROBERT: Good morning. This is an exciting podcast. Today, we're going to be discussing writing a proposal to the Unicode Committee, getting it accepted and rejected. This is basically making emojis which I think is really awesome. We have two guests today who have an amazing story, Amanda Hickman and Amberley Romo. Thank you both for joining us. You two have an amazing story that I would really love to dive into and we're going to do that today. It's about basically creating your own emoji and getting that accepted so everybody can use that and I think that's super, super cool, something that I've always kind of wanted to do as a joke and it seems like that's kind of where your stories began, so you two want to jump in and start telling? I think Amanda has a great beginning to this. AMANDA: Sure. I mean, hi and thanks for having me. I don't know where to begin and really for me, this starts with learning to sew my own clothes which is an incredibly exasperating and frustrating process that involves a lot of ripping stitches back out and starting over and Instagram was a really big part of me finding patterns and finding other people who are sewing their own clothes and learning from the process. I wanted to be able to post stuff on Instagram and it started to drive me absolutely crazy, that there's emojis for wrenches and nuts and hammers and there are no textile emoji. The best I could find was scissors which is great because cutting patterns is a place where I spend a lot of time procrastinating but that was it. I knew a woman, Jennifer 8 Lee or Jenny who had led a campaign to get the dumpling emoji into the Unicode character set. I knew she'd succeeded in that but I didn't really know much more about how that had worked. I started thinking I'm going write a sewing emoji. I can do this. I can lead this campaign. I started researching it and actually reached out to Jenny and I discovered that she has created an entire organization called... What was that called? She's created an entire organization called Emojination, where she supports people who want to develop emoji proposals. CHARLES: Before you actually found the support system, you actually made the decision that you were going to do this and you found it. You know, from my perspective, I kind of see emoji is this thing that is static, it's there, it's something that we use but the idea that I, as an individual, could actually contribute to that. I probably, having come to that fork in the road would have said, "Nah, it's just it is what it is and I can't change it." What was the process in your mind to actually say, "You know what? I'm actually going to see if I can have some effect over this process?" AMANDA: It definitely started with a lot of anger and being just consistently frustrated but I knew that someone else had already done this. It was sort of on my radar that it was actually possible to change the emoji character set. I think that if I didn't know Jenny's story and it turned out I didn't know Jenny story at all but I thought I knew Jenny story but if I didn't know that basic thing that that somebody I knew who was a mere mortal like me had gone to the Emoji Subcommittee of the Unicode Consortium and petition them to add a dumpling emoji, I am sure that I wouldn't bother. But I knew from talking to her that there was basically a process and that there were a format that they want proposal in and it's possible to write them a proposal. I knew that much just because I knew Jenny. I think at that point, when I started thinking about this, the Emoji 9 -- I should be more of an expert on that actually, on emoji releases but a new release of emoji had come out. There were a bunch of things in that release and it got a little bit of traction on Twitter. I knew that the Unicode Consortium had just announced a whole new slate of emoji, so I also was generally aware that there was some kind of process by which emoji were getting released and expanded and updated. ROBERT: That's interesting. Do you know when that started? Because it seems like Apple started to add more emojis around like iOS 7 or something but it was pretty static for a while right? Or am I wrong? AMANDA: I actually am tempted to look this up but the other piece that is not irrelevant here is that at the time, I was working at a news organization called BuzzFeed that you may have heard of -- ROBERT: Maybe, I don't know. It sounds kind of familiar. AMANDA: I do feel like people kind of know who they are. I was surrounded by emoji all the time: in BuzzFeed, in internet native of the highest order and we had to use emoji all the time and I had to figure out how to get emoji into blog post which I didn't really know how to do before that. I can put them on my phone but that was it. I was immersed in emoji already. I knew that there was a project called Emojipedia, that was a whole kind of encyclopedia of emoji. One of my colleagues at BuzzFeed, a woman named Nicole Nguyen had written a really great article about the variation in the dance emoji. If you look at the dance emoji, one of the icons that some devices use is this kind of woman with her skirt flipping out behind her that looks like she's probably dancing a tango and then one of the icons that other character sets use and other devices use is a sort of round, yellow lumping figure with a rose in its mouth that you sort of want to hug but it's definitely not to impress you with its tango skill. She had written this whole article about how funny it was that you might send someone this very cute dumpling man with [inaudible] and what they would see was sexy tango woman. I think there was some discussion, it was around that time also that Apple replaced the gun emoji with a water gun. There was some discussion of the direction that the various emoji's face. One of the things that I learned around that time was that every device manufacturer produces their own character set that's native to their devices and they look very different. That means that there's a really big difference between putting a kind of like frustrated face with a gun pointing at it, which I don't really think of it as very funny but that sort of like, "I'm going to shoot myself" is very different from pointing the gun the other way which is very much like, "I'm shooting someone else," so these distinctions, what it means that the gun emoji can point two different ways when it gets used was also a conversation that was happening. None of that answers your question, though which is when did the kind of rapid expanse of emoji start to happen. ROBERT: I feel like the story is setting in the place there, though because it seems like there's a little bit of tension there that we're all kind of diverging here a little bit and it's sort of driving back towards maybe standardization. AMANDA: There's actually, as far as I know, no real move toward standardization but the Unicode Consortium has this committee that actually has representatives from definitely Apple and Microsoft and Google and I forget who else on the consortium. Jenny 8 Lee is now on the consortium and she's on the Emoji Subcommittee but they actually do get together and debate the merits of adding additional emoji, whether they're going to be representative. One of the criteria is longevity and I tend to think of this as the pager problem. There is indeed a pager emoji and I think that the Unicode Consortium wants to avoid approving a pager emoji because that was definitely a short-lived device. CHARLES: Right. I'm surprised that it actually made it. Emoji must be older than most people realize. AMANDA: My understanding is that very early Japanese computers had lots emoji. There's a lot of different Japanese holidays that are represented in emoji, a lot of Japanese food as well are represented in emoji, so if you look through the foods, there's a handful of things that haven't added recently but a lot of the original emoji definitely covered Japanese cuisine very well. ROBERT: I definitely remember when I got my first iPhone that could install iPhone OS 2, you would install an app from the App Store that then would allow you to go toggle on the emoji keyboard but you had to install an app to do it and that's kind of where the revolution started, for me at least. I remember everybody starting to sending these things around. AMANDA: But if you look at Emojipedia, which has a nice kind of rundown of historical versions of the Unicodes, back in 1999, they added what I think of as the interrobang, which is the exclamation/question mark together and a couple of different Syriac crosses. Over the years, the committee has added a whole series of wording icons and flags that all make sense but then, it is around, I would say 2014, 2015 that you start to get the zipper mouth and rolling your eyes and nerd face and all of the things that are used in conversation now -- the unicorn face. ROBERT: My regular emojis. AMANDA: Exactly. CHARLES: It certainly seems like the push to put more textile emoji ought to clear the hurdle for longevity, seeing there's kind of like, what? Several millennia of history there? And just kind of how tightly woven -- pun intended -- those things are into the human experience, right? AMANDA: Definitely. Although technically, there's still no weaving emojis. CHARLES: There's no loom? AMANDA: There's no loom and I think that a loom would be pretty hard to represent in a little 8-bit graphic but -- CHARLES: What are the constraints around? Because ultimately, we've already kind of touched on that the emoji themselves, their abstract representations and there are a couple of examples like the dancing one where the representation can vary quite widely. How do they put constraints around the representation versus the abstract concept? AMANDA: You don't have to provide a graphic but it definitely kind of smooths the path if you do and it has to be something that's representable in that little bitty square that you get. It has to be something representable in a letter-size square. If it's not something that you can clearly see at that size, it's not going to be approved. If it's not something you can clearly illustrate at that size in a way that's clearly distinct from any other emoji and also that's clearly distinct from anything else of that image could be, it's not going to be approved. Being able to actually represented in that little bitty size and I don't know... One of sort of sad fact of having ultimately worked with Emojination on the approval process is that we were assigned an illustrator and she did some illustrations for us and I never had to look at what the constraints were for the illustration because it wasn't my problem. ROBERT: Sometimes, that's really nice. AMANDA: Yes, it's very nice. I ended up doing a lot of research. What made me really sad and I don't want to jump too far ahead but one of things that made me really sad is we proposed the slate and the one thing that didn't get approved was the sewing needle and it also didn't get rejected, so it's in the sort of strange nether space. That's kind of stuck in purgatory right now. I did all this research and learned that the oldest known sewing needle is a Neanderthal needle so it predates Homo sapiens and it's 50,000 years old. CHARLES: Yeah. Not having a sewing needle just seem absurd. AMANDA: Yeah. We have been sewing with needles since before we were actually human being. ROBERT: That's a strong case. AMANDA: Yes, that's what I thought. If I sort go back to my narrative arc, I wanted to do a sewing needle and started researching it a little bit -- CHARLES: Sorry to keep you interrupting but that's literally the one that started this whole journey. AMANDA: Yes, I wanted a sewing needle and I really wanted a sewing needle. I did a little research and then I reach out to Jenny and to ask her if she had any advice. She said, "You should join my Slack," and I was like, "Oh, okay. That's the kind of advice." She and I talked about it and she said that she thought that it made more sense to propose a kind of bundle of textile emoji and I decided to do that. She and I talked it through and I think the original was probably something closer to knitting than yarn but we said knitting, a safety pin, thread and needle were the ones that kind of made the most sense. I set about writing these four proposals and one of the things that they asked for was frequently requested. One other thing that I will say about the proposal format is that they have this outline structure that is grammatically very wonky. They ask you to assert the images distinctiveness and they also ask you to demonstrate that it is frequently requested. I found a couple of really interesting resources. One, Emojipedia which is this sort of encyclopedia of emoji images and history maintains a list of the top emoji requests. I actually don't know how they generate that list or who's requesting that and where but I think it's things that they get emailed about and things people request in other contexts and sewing and knitting, I've done on that list and I started compiling it in 2016. ROBERT: To be a part of the proposal process, to show that it is requested, without that resource, you just start scouring Facebook and Twitter and history and shouting to people like, "I really want this emoji. Why it didn't exist?" That seems pretty hard. AMANDA: Actually our proposals all have Twitter screen shots of people grousing about the absence of knitting emoji and yarn emoji and sewing emoji. I know that Emojipedia, they do a bunch of research so they go out and look at based what people are grousing about on Twitter. They look at places where people are publicly saying like, "It's crazy that there's no X emoji," and that's part of their process for deciding what kinds of emojis people are asking for. Their research was one resource but we took screenshots of people saying that they needed a safety pin emoji and that was part of making the case. One of the things that I found as I was doing that research was that, I guess at this point it was almost two years ago, when the character set that included the dumpling emoji came out, there was a bunch of grousing from people saying, "Why is there not a yarn emoji?" There was a writing campaign that I think Lion Brand had adopted. Lion Brand yarn had put in this tweet saying like, "Everyone should complain. We needed a yarn emoji," but it doesn't matter how much you yell on Twitter. If you don't actually write a proposal, you're not going to get anywhere. I had been told that the Emoji Subcommittee, they're really disinclined to accept proposals that had a corporate sponsor, so they weren't going to create a yarn emoji because Lion Brand yarns wanted them to create a yarn emoji. ROBERT: Right, so it was like counter-peer proposal. AMANDA: Right. But as I was digging around the other thing I found was this woman in... I actually don't know if you're in Dallas or Austin but I found Amberley, who also put a post on Twitter and had started a petition, asking people to sign her petition for a yarn emoji proposal or a knitting emoji. I don't remember if it was a yarn emoji or a knitting emoji but I found her petition and reached out to her to ask if she was interested in co-authoring the proposal with me because she had clearly done the work. She actually had figured out how the system worked at that point. I think she knew who she was petitioning, at least. I reached out to Amberley and we worked together to refine our proposal and figure out what exactly we wanted to request. I think there were a bunch of things that were on the original list like knitting needles, yarn and needles. I think crocheting would have been on the original list. We were sort of trying to figure out what was the right set of requests that actually made sense. ROBERT: So then, this is where Amberley stories comes in and it is interesting too because she has entirely different angle for this. Maybe not entirely different but different than outright. This kind of ties back to the word software podcast mostly. It kind of ties back to the software aspect, right? AMBERLEY: Yeah. I think, really they're kind of separate stories on parallel tracks. My motivation was also two-fold like Amanda's was, where I started knitting in 2013 and I had a really good group of nerd friends with a little yarn shop up in DC, like a stitch and ditch group -- ROBERT: I love it. AMBERLEY: It was a constant sort of like, where's the insert emoji here, like where's the yarn emoji? Where's the knitting emoji? And we would sort of sarcastically use the spaghetti emoji because it was the most visually similar but that was something that was in the back of my mind but it teaches you a lot about yourself too because I was like, "Oh, this is like fiber art, not really an emoji. It's kind of technical, like on a tech space," and I didn't really connect that it was relevant or that I might have any power to change it. It just didn't occur to me at the time. ROBERT: Interesting. I feel like a lot of people are in that similar situation or maybe not situation, even though you can make change on this. AMBERLEY: Right, so my brain didn't even make it like, "Why isn't this a thing? let me look at how to make the thing." When that happened for me, Amanda mentioned using emoji and everything in the BuzzFeed space. I love how you explained BuzzFeed a while ago, it's my favorite description of BuzzFeed I ever heard. Something similar that happened for me was I was a software developer and in 2016, the Yarn package manager was released and that kind of turned something on in my head. That was like I'm seeing all these software engineers now be like, "Where's the yarn emoji?" and I'm like, "Welcome to the club." ROBERT: "Do you want to join our Slack? We can complain together." AMBERLEY: Right. It has been like a pretty decent amount of time, I'm semi-seriously ranting and complaining to my coworkers who were primarily male software engineers. I remember I went to [inaudible] in the Frost Bank Tower after work and was just like, "I'm going to figure out how this happens," and I spent a couple hours at the coffee shop. I found the Unicode site and I found their proposal process and their structure for the proposal and everything and I just started doing the research and drafting up a proposal specifically for yarn. Maybe it was a bit naive of me but to me it was like, "Okay, here's the process. I follow the process. Cool." I mean, you have to make a case and it has to be compelling and has to be well-written and it has to be supported and all that and that to me it was like, "Okay, there's a process. At the same time, I did read about the dumpling emoji but I didn't connect it to Emojination and they had started the Kickstarter. We should talk about this later but I think the sort of idea the issue of representation on the committee and who gets to define language is really interesting but I saw that they had done the Kickstarter and there was a campaign aspect to it, so I ended up just building up this simple site so that if anyone Google, they would find yarn emoji. It's still up at YarnEmoji.com and that was how Amanda found me. I got this random email, I sort of like had this burst of energy and I did all the research and I wrote the draft, sort of piecemeal, filling out the different sections of the way they have it outlined on the Unicode site and then I feel like a month or two went by and I had kind of not looked at it for a bit and then, I get this random email from this website that I almost forgot about. It was like, "Hey, I'm working on this series of proposals. If you're working on knitting or yarn or whatever, maybe we could work together," and I was like, "Well, that's sweet." Then she opened up this whole world to me. There's this whole Emojination organization, sort of 100% devoted to democratizing the process of language formation through creating emojis and so then, I got really into that. My primary motivator was yarn. CHARLES: So what's the status of the yarn spool, those emoji right now? AMBERLEY: The yarn, the spool of thread and the safety pin, they're all approved emoji for the 2018 released. Amanda and I are actually at the end. Amanda, a couple of months ago when I saw someone used the spool thread emoji for a Twitter thread -- you know how people will be like all caps thread and have a thread of tweets -- I saw someone do that just out of the blue. I was like, "Oh, my God. Is it out?" and the thing about these individual vendors, it sort of gets released piecemeal, so at the time Twitter have I think released their versions of this series of new emoji but others hadn't. CHARLES: How does that work? Because you think the Twitter would be kind of device depending on what browser you're using, like if you're on a Windows or a Mac or a Linux Box, right? ROBERT: -- Emoji set, right? I know Facebook does this too. AMBERLEY: I'm painfully aware that Facebook does it because I can't use the crossed finger emoji on Facebook because it actually gives me nightmares. ROBERT: I have to go look at this now. AMBERLEY: Because it's so creepy-looking. CHARLES: Okay. Also like Slack, for example is another. It's like a software-provided emoji set. AMANDA: Right. AMBERLEY: I'm not totally sure that Slack actually adheres to the standard Unicode set. I think it's kind of its own thing but I might be wrong about that. AMANDA: Sorry, Slack definitely supports the full Unicode set. They also have a bunch of emoji that they've added that aren't part of the set. AMBERLEY: Slack emojis? AMANDA: Yes. CHARLES: Yeah and then every Slack also has its kind of local Slack emoji. AMBERLEY: Right. CHARLES: But how does that work with --? ROBERT: Okay, this crossed-finger Facebook emoji is... yes, I agree with you, Amberley. AMBERLEY: Thank you. I had yet to find someone who disagrees with me about that. AMANDA: I have never seen it before and I'm now like, "What is going on?" CHARLES: Yeah, so how does it work if a vendor like Twitter is using a different emoji set? How does that work with cut and paste, like if I want to copy the content of one tweet into something else? Are they using an image there? AMANDA: They're using an image. I think it's doesn't happen as much anymore but for a long time, I would often get texts from people and the text message would have that little box with a little code point in it and you were like -- AMBERLEY: More like an alien thing? AMANDA: Yeah. Definitely, if you don't have the emoji character set that includes the glyph that you're looking at, you're going to get that little box that has a description of the code point and I think what's happening is that Twitter is using JavaScript or generally programming. There were air quotes but you can't see. Twitter is using their software to sub in their emoji glyph whenever someone enters that code point. Even if you don't have the most up to date Unicode on your computer, you can still see those in Twitter. If I copy and paste it into a text editor on my computer, what I'm going to see is my little box that says '01F9F5' in it but if I get it into Twitter, it shows up. I can see them on Twitter but I can't see them anywhere else. AMBERLEY: Damn, you really have the code point memorized? AMANDA: No, I -- CHARLES: Oh, man. I was really hoping -- AMBERLEY: Oh, man. ROBERT: You live and breathe it. AMANDA: No, I'm not that compulsive. AMBERLEY: We definitely have our emojis on our Twitter bios, though. AMANDA: Absolutely. ROBERT: If you see Amanda's bio, it's pretty great. AMANDA: They started showing up on Twitter and I think that somebody in Emojination probably told me they were out and that was when I first started using them. Amberley might have actually seen it. It sounds like you just saw it in the wild, which is kind of amazing. AMBERLEY: I saw it in the wild with this tweet thread and yeah, it's just [inaudible]. I was like, "Amanda, is it out?" CHARLES: Yeah, I feel like I saw that same usage too, although I obviously did not connect any dots. AMANDA: This last week, October 2nd -- I'm also looking things up. I'm just going to come to the fact that I am on a computer looking things up so I can fact check myself -- after they actually released their emoji glyph set, so by now any updated iOS device should have the full 2018 emoji, which in addition to a kind of amazing chunky yarn and safety pin, there's also a bunch of stuff. There's a broom and a laundry basket. There's a bunch of really basic, kind of household stuff that certainly belongs in the character set alongside wrenches and hammers. AMBERLEY: I think one of the big ones too for this year was the hijab? AMANDA: No, the hijab actually came out with a dumpling. Hijab has been available -- AMBERLEY: It's been up, okay. ROBERT: So did it come with iOS 12 or 12.1? I don't know for sure. I just know -- AMANDA: I'm looking at it and it's 12.1. I really feel that I should be ashamed that I have used the internet and search for this. AMBERLEY: I would say, I have no idea what their release numbers are. AMANDA: [inaudible] as it appeared for the first time in iOS for 2018 with today's release of the iOS 12.1, Beta 2 for developers. ROBERT: That is amazing. Do you get some kind of satisfaction -- like you have to, right? -- from people using the emoji and it's starting to make its way out there? AMANDA: So much. Oh, my God, yeah. AMBERLEY: I didn't really expect it, like saying that random tweet using this spool of thread for a tweet thread. I just thought and I just got so psyched. For me, I'm a knitter. I have knitter friends and it started with yarn and then really, Amanda and through Amanda, Jenny really sort of broadened my idea of what it all really meant. To think someone using it in the wild for a totally different application than I had ever thought of was like, "That's legit." AMANDA: I definitely have a sewing emoji search in my tweet deck and sometimes, when I'm feeling I need a little self-validation, I'll go look over there and find people who are saying things like, "Why is there no sewing emoji?" and I'll just reply with all the sewing emoji, like it is part of my work in this life to make sure that not only do they exist but people know about them. ROBERT: That is awesome. I would do the same thing, though to be honest. You'll be proud of that. AMANDA: Totally. ROBERT: Were there any hitches in the proposal process? I know we're kind of alluded to it but the thing that you started off one thing, Amanda didn't make it. Right? AMANDA: I know. ROBERT: So how did that process happen from you two meet each other and then going through the actual committee and the review process and then being accepted. What would that mean? AMANDA: The process is actually incredibly opaque. We wrote this whole proposal, a bunch of people edited it, which is one of the other nice things about collaborating with Emojination. There was a bunch of people who are just really excited about emoji and the kind of language making that Amberley was talking about. There's a whole bunch of people who just jumped in and gave us copy edits and feedback, which was super helpful and then, there was a deadline and we submitted it to the committee and it actually shows up in the Unicode register which is also a very official kind of document register. I was a little excited about that too but then they have their meeting. They first have a meeting and there's like a rough pass and the Emoji Subcommittee makes formal recommendations to the Unicode Consortium and then the consortium votes to accept or reject the Emoji Subcommittee's recommendations. It's a very long process but unless you're going and checking the document file and meeting minutes from the Unicode Consortium meetings, you'll never going to know that it happen. AMBERLEY: -- You know someone connected through there because one of the things in our first pass, it wasn't that it was rejected. It was that we needed to modify something. We do have art for knitting needles with yarn because at one point, I think we weren't totally sure that a ball of yarn would be visually distinct enough in this emoji size to look like yarn and so, we had put it with sort of knit piece on knitting needles. AMANDA: Oh, that's right. There was a tease of a little bit of knitted fabric. AMBERLEY: Right and I think that, probably through Jenny or the people actually in the room, the feedback I remember is that there is a crocheter in the room who was like, "Yeah, why isn't there a yarn emoji but knitting needle?" so there was a little bit of like that was how I think we ended up from knitting needles with a fiber piece to ball of yarn, maybe. AMANDA: I think that sounds right. I'm actually sure of that. It's just all coincide with my recollection. There were some things that they had questions about and that happened really fast because I feel like we had a couple of days and they have stuck to our guns and said, "No, we're only interested in knitted bit of fabric." Also, we worked with an illustrator and went back and forth with her because the initial piece that she had illustrated, I feel like the knitting needles were crossing in a way. That was not how knitting works and so, there was a little bit of back and forth around that as well. But then once they decided that the they like the thread, yarn and safety pin, we're going to move to the next stage. I actually had to go back and look at the minutes to find out that the two reasons that they didn't move the sewing needle on to the next stage is when they thought it was adequately represented by the thread, which I wholeheartedly disagree with and they thought it wasn't visually distinctive. That's so much harder because a sewing needle, which is really just a very fine piece of metal with an eye at the end, you get down to a really small size and it is maybe a little hard to know what you're looking at. But I think there's such a big difference between the static object which is the spool or the thread which represents a lot of things and is important and the needle, which is the active tool that you use to do the making, to do the mending, to do the cobbling. CHARLES: Yeah. I'm surprised that it almost isn't reversed when certainly in my mind, which I think is more culturally important in terms of the number of places which it appears, it's definitely the needle as being kind of... Yeah. AMANDA: Yeah and I think that the thread and yarn, they're important and I think that the decision to have a ball of yarn rather than a bit of knitting makes sense because there's a lot of things that you can use a ball of yarn that aren't just knitting and they think that -- AMBERLEY: And it's the first step too that doesn't exclude anyone in the fiber art community. AMANDA: But there's so many things like in sutures and closing wounds, you're not using a little spool of cotton thread for that or polyester thread and stuff like embroidery and beadwork, you might be using thread or fiber of some sort that started on a spool but you might not. Embroidery floss was not sold in a spool and there's all these places where we use needles and all kinds of different size and you don't always use thread. Sometimes, you're using yarn. Sometimes, you're using leather cord. Sometimes, you're using new bits of, I would say Yucca. You're using plant fibers to do baskets and in all of these different practices, that process of hooking it through the eye and sewing it is how it's actually made. It still sort of mystifies me why they haven't accepted it but they also didn't reject it, which is really interesting. I don't know how many other emoji are sort of sitting in this weird nether space because sometimes they just reject them outright. I think there was a proposal for a coin that they just said no. ROBERT: They were a like, "A coin?" That would be [inaudible]. AMANDA: Oh, God. ROBERT: They have to add one for every -- AMANDA: [inaudible]. CHARLES: Literally, the pager of 2017. AMANDA: Exactly. CHARLES: So what recourse is now available to you all and to us, by extension, to get the sewing needle? AMANDA: I'm actually working on a revised proposal and I've been trying to figure out what are all the arguments that I'm missing for why sewing and the needle are not adequately represented by the thread and yarn. A bunch of things that a friend of my named, Mari who's half-Japanese, half-American but lives in Guatemala and does all this kind of arts in textile work, pointed out that there's a whole holiday in Japan devoted to bringing your broken needles and thanking them for their service. I thought that was really cool. I've been trying to formulate what are all of the arguments for the necessity of both a needle and a spool. If anybody has interesting ways to phrase that, I would love for arguments. CHARLES: Yeah but it's hard to imagine the arguments is just anything being more compelling than the arguments the you just laid out that you named about seven context: shoemaking, medicine, different fibers where the needle operates completely and totally independent of the thread. It's looming so large in kind of our collective conscious like holidays, being dedicated to them, except I think the Cro-Magnon pager, which is made out of stone, I believe, the being the artifact that pre-dates... AMANDA: There's the idiom landscape as well. Things like finding a needle in a haystack, that has a very specific meaning -- ROBERT: And for puns. I've been resisting saying a pun this whole time. AMANDA: Oh, share your pun with us. AMBERLEY: Yeah, you have to say it. ROBERT: Well, you could say that trying to get this through the committee is like threading a needle. Butchered but -- AMANDA: There's a biblical quote about getting into heaven -- a camel through the eye of a needle. I forget actually how it... CHARLES: To thread a camel through the eye of a needle than for a rich man to get into heaven. AMANDA: Exactly and there's this sort of do-re-mi, saw, a needle pulling thread. There are all these places where it's about the needle and somebody had -- CHARLES: It's primarily ancient. AMANDA: I know. CHARLES: It is the prime actor. Maybe, this is a good segue into kind of talking about the makeup of the committee and the decision making process and these kind of what seem like very clear arguments might not be received as such. AMANDA: I certainly don't want to say anything bad about anybody on the committee. CHARLES: No, no, I don't think that there's anything bad. I think that being receptive to things which are familiar to us versus with things that aren't is a very natural human thing and it can be interesting to see that at work and at play. AMANDA: The Unicode Consortium is also evaluating all of these requests for whole language glyphs sets. Lots of languages and lots of character sets that are kind of obvious, like there has to be a sort of like character set like there has to be an Arabic character set but there are a lot of languages that have been left out of that because they're very small minority languages or they are historical languages, where the actual writing is no longer written the same way but there's historical reasons to be able to represent those characters. One of the reasons why the Emoji Subcommittee cares about what gets into the formal character set is that everybody has to accommodate it and there's already been, I think some grousing. People start to moan and groan about how there's too many emoji, then it's too hard to find things. CHARLES: And there's no take backs. AMANDA: There's no take backs. You can't undo it. The committee is made up of representatives from a lot of tech companies primarily, although there's a couple of other kind of odd additional folks on there. I do try to find the committee list and I can find it right now. AMBERLEY: I have it from Emojination. I don't know if it's up to date but Oracle, IBM, Microsoft, Adobe, Apple, Google, Facebook, Shopify and Netflix. The other voting members -- ROBERT: Shopify? AMBERLEY: Yeah, right? The others being the German software company, SAP and the Chinese telecom company, Huawei and the Government of Oman. AMANDA: Yeah, the Government of Oman is a fascinating one. I don't think they're the ones that are biting us on this. Especially for those tech companies, every time the emoji character set adds 10 or 12 emoji, they don't have to accommodate it on their devices. They have to put illustrators on it, they have to deal with everyone saying that the crossed fingers emoji in Facebook looks like I-don't-even-know-what. AMBERLEY: Hey, Amanda. AMANDA: It's all your fault. There's a whole process and there's non-trivial work associated with every single new emoji, so wanting to put the brakes on a little bit and be intentional about where and when they apply that work, it doesn't seem crazy to me. I just want them to approve the thing that I want. AMBERLEY: I like the way that Emojination captures it. I looked at their website earlier and actually, they take it down but their goal quote "Emojination wants to make emoji approval an inclusive representative process." There has to be a process. There's overhead involved but looking at the makeup of the decision makers are not a trivial question. CHARLES: Right. This is a great example like [inaudible] metaphor but these little artifacts, these emojis are literally being woven to the fabric of a global culture and certainly, everybody uses them and they become part of the collective subconscious. It does seem like very important to be democratic in some way. It sounds like there is a process but making sure that everyone has a stake. AMBERLEY: Yeah. ROBERT: What was the reason that they gave for not accepting the needle and thread? Was it like a soft no? You said it's like just hanging out, not really rejected but not accepted. We're going to drop a link in the show notes for the proposal and your GitHub and everything. I'm looking at the PDF that was put together and it seems like it was all a package deal like we talked about. How do they just draw or they just take like a lawyer would, just like draw or cross it out like, "Well, no but we'll take the other ones." AMANDA: Yes, basically. What they did is they need to discuss and I don't know how long they've been meeting but they need to discuss all of the proposals that have been supplied by a particular deadline and -- ROBERT: That sounds painful. AMANDA: Yeah, I mean, it's -- ROBERT: Just imagine the power of thinking about emojis. AMANDA: One of the things that they rejected, I think because there's the smiling poo face. Somebody wanted a frowning poo face and they rejected that. There's a bunch of things that actually do get rejected. I don't know if they've been really care about a smiling poo face versus a frowning poo face. ROBERT: What about an angry one? AMANDA: We got all the feelings of poo. ROBERT: We got important work to do here. AMANDA: But they go through when they're trying to figure out. I think to some degree, you want to get them when they're not tired but I think the status that it's listed right now is committee pushback, so they've set it aside until we have some concerns. We're not going to reject it outright but we're not really sure why this isn't adequately represented. Then their most recent meeting, they just kind of passed on reconsidering it, which is fine because I think I was traveling and my proposal is not done. I really want to make sure that I have consolidated every imaginable argument in one place so -- ROBERT: And make it strong as possible. AMANDA: Yeah. If people want to help the other thing that would be amazing is any and all idioms that you can think of, especially ones that are not in English or European languages, idioms in Central European languages, idioms in Asian languages that refer to needles, either translations of the kind of classic, 'finding a needle in a haystack,' but also any idioms that are kind of unusual and specific to a culture outside of what I have experience with would be amazing for making the case, so this is an international need. ROBERT: Do they need any specific or actionable feedback or do they just say, "We're going to push back on this. We're just not quite sure?" AMANDA: The two things that we're in the minutes -- there are minutes and they publish the minutes to Unicode.org -- were it was not visually distinct, which is not totally crazy. We actually worked with an illustrator to get a different image. The first image was almost at 90 degrees. It was kind of straight up and down and it is a little hard to see and the second is -- ROBERT: Especially, because it's thin. AMANDA: The second image is actually a kind of stylized needle because it's fairly a little fatter and the eye is bigger but it's much more distinctively a needle. I'm hoping that that will also convince them but you have to be able to tell at a very small size that it's a needle. The other thing that they said was that sewing was already represented by the thread, that we didn't need thread and needle but it was literally one line in the minutes that referenced that and then it sort of like, "Did you have somebody in the room or not?" and so, if there is somebody on the committee who is willing to tell you really what their concerns were, then you have some sense of what they're looking for and why they're pushing back. When you can very much see in the earliest emoji character sets that I have a hammer and I have a wrench and I use them but there's these very conventionally male tools. We have all of the kind of office supplies but all of homemaking and housekeeping and textile production, none of them were there until very, very recently. I think it does reflect the gender of the people who've been making these tools, that sewing and knitting weren't important enough as human practices to be included in this glyph set. AMBERLEY: I guess, that's non-trivial to mention because that wasn't an argument that I made in my original yarn draft and Amanda and Jenny sort of pushing to open it up to this whole slate of craft emoji. I didn't realize until they brought that up. I took a stroll through pretty much the whole slate of emoji and you can count on almost one hand the number that represented the creative endeavors or sort of more traditionally known as creative things like camera or painting palette and stuff like that. It was extremely limited. AMANDA: I think they have stuff like that. I think there's a few different variations on the camera and then there's painting palette and that's it. AMBERLEY: Oh, there's the theater mask. AMANDA: Oh, that's right. There is the theater, the happy and sad -- AMBERLEY: And I don't know it exactly and I haven't read the minutes like Amanda has but I think and I hope that that was a particularly compelling piece of that argument. AMANDA: I think they definitely heard it. AMBERLEY: Yeah. CHARLES: Opening it up then, what else is coming in the way of craft? It sounds like this is historical but these pieces are being filled in not only with the work that you all are doing but by other emoji which you're appearing. AMANDA: Yes. CHARLES: And are you in contact with other people who are kind of associated with maybe craft and textiles and other kind of what you're labeling historically creative spaces? AMANDA: I don't think there are anymore with a possible exception. Someone's working on a vinyl record proposal which I think is great. CHARLES: Yeah, that's awesome. ROBERT: Antiquated, though. AMANDA: Maybe not, I don't know. AMBERLEY: Take a stroll through the Emojination Slack and people discussed that. AMANDA: Yeah. If you click at Emojination.org, the whole Airtable database is on there. There's not a lot of other creative ones. A friend of mine got really bent out of shape about the lack of alliums and wrote a whole slate proposal for leeks and scallions and garlic and onions. ROBERT: Oh, there is a garlic one, right? AMANDA: No. I mean, there is -- AMBERLEY: Actually, I'm looking at the Unicode page for current emoji candidates. They first get listed as... I forget the exact order. They become draft candidates and then provisional candidates or vice versa but I don't see any pending further creative ones but garlic and onion are on there. AMANDA: Yes. ROBERT: That makes my Italian a little happy. AMANDA: I think there's some prosthesis, the mechanical leg and the mechanical arm, a guide dog -- AMBERLEY: Ear with hearing aid, service dog. AMANDA: Yeah, there's a good chunk of interesting things that have been left out. I guess they've been approved by the subcommittee but are still waiting on final approval by the Unicode Consortium. ROBERT: Okay. What are the next steps that we can do to help push the thread and needle proposal through it. You mentioned a couple things like coming up with idioms that are in different languages and whatnot but how can we contact you and push this effort and help? AMANDA: That's such a good question. I don't even know. I mean, I am Amanda@velociraptor.info and you're totally welcome to email me if you want to help with this and I will -- ROBERT: That's a great domain, by the way. AMANDA: Unfortunately, there's no information about velociraptors anywhere on that site. ROBERT: That's the way it should be. AMANDA: But also, if you're excited about working on emoji proposals, Emojination is an incredibly great resource and folks there, including me actually will help you identify things that are on other people's wish lists that you could work on if you just want to work on something and we'll help you refine your proposal if you know what you want and we'll help you figure out whether it's worth putting the time in or not and how to make it compelling. You can definitely check out Emojination.org. I think there's a path to get on to the Slack from there. AMBERLEY: Oh, yeah. The Slack and the Airtable. AMANDA: Yeah. ROBERT: It sounds like there's a whole community that was born out of this, where everybody is trying to help each other and collaborate and get their shared ideas across. AMANDA: Definitely and there's a woman, Melissa Thermidor who is fantastic, who actually is a social media coordinator. It's her actual title but she works for the National Health Service in the UK and was tasked with getting a whole series of health-related emoji passed. There's a bunch of things that she's -- AMBERLEY: Is she's the one doing blood. AMANDA: She's doing blood. AMBERLEY: That's a good one. AMANDA: Because there's a lot of really important health reasons why you need to be able to talk about blood and getting blood and blood borne illnesses and -- AMBERLEY: That one was listed on the emoji candidate page or blood donation medicine administration. AMANDA: Yeah. ROBERT: That's really interesting, so she works for the government, right? and that was part of her job to do that? AMANDA: Yes. ROBERT: That's awesome, actually. I love that. AMANDA: Yeah, I think the drop of blood, the bandage and the stethoscope are the three that are in the current iteration, which is interesting because the existing medical emoji were the pill and that gruesome syringe with a little drops of fluid flying off of it, which do not do a lot to encourage people to go to the doctor. ROBERT: No, not at all. AMANDA: So a few more, we're welcoming medical emoji. ROBERT: You have a GitHub. Is that where you're still doing for the follow up and the prep work for the sewing emoji? AMANDA: Yeah, that's probably the best place. I do have a Google Docs somewhere but that's probably a better place to connect even than my ridiculous Velociraptor email. The GitHub -- ROBERT: But it's still awesome. AMANDA: It is awesome. I won't lie. I'm very proud of it. I am AmandaBee -- like the Bumble Bee -- on GitHub and the sewing emoji, the original proposals are there and I will make sure that there is information about how to plug into the revised needle proposal there as well. You guys are a tech podcast, so if people want to just submit suggestions as issues on that repository, that's awesome. We'll totally take suggestions that way. ROBERT: That would be pretty rad. Well, I appreciate you two being on the podcast. I love hearing your stories and how it ended up converging in parallel tracks but it end up achieving the same goal. Still unfinished, right? Let's see if we can help push this over the finish line and get it done because I would really like to see a needle. I could definitely use that in many of my conversations already now, making all kinds of puns. Thank you, Amanda for coming on and sharing your story. AMANDA: Thanks for having me. ROBERT: And thank you, Amberley for also coming on and sharing your story. This was super awesome. AMBERLEY: Yeah and thank you for connecting us to finally have a voice conversation. AMANDA: I know. It's great to actually talk to you, Amberley. CHARLES: Oh, wow, this is the first time that you actually talked in audio? AMANDA & AMBERLEY: Yeah. ROBERT: We're making things happen here. The next thing we have to do is get this proposal through and accepted. AMANDA: Yes. CHARLES: You've converted two new faithful sewing and needle partisans here and I'm in. AMANDA: Awesome. ROBERT: I know you've already gotten, what? Three through accepted? AMANDA: Yeah. ROBERT: We talked about that, it's got to be really awesome. I think I want to try and jump in and get that same satisfaction because a lot of people use emojis. AMANDA: Exactly. CHARLES: It definitely makes me think like you look at every single emoji and there's definitely a story. Especially for the ones that have been added more recently, there's a lot of work that goes into every single pixel. That represents a lot of human time, which I'm sure you all know, so thank you. AMANDA: Thanks for having us on. AMBERLEY: Yeah, thank you guys. ROBERT: Cool. That is the podcast. We are Frontside. We build UI that you can stick your future on. I really love this podcast because it wasn't necessarily technical but had a lot of interesting conversation about how to work with a proposal and probably make a bigger impact than any of us with software, just because the sheer reach that emojis have are insane and the fact that you can influence this process is new to me and really cool, so I hope a lot of other people learn from that too. If you have any feedback that you would like to give us on the podcast, we're always open to receive feedback. We have our doors and ears open, so if you like to send an email at Contact@Frontside.io or shoot us a tweet or DM us at @TheFrontside on Twitter. We'd love to hear it. Thank you, Mandy for producing the podcast. She always does an amazing job with it. You can follow her on Twitter at @TheRubyRep. Thanks and have a good one.

Panel: Aimee Knight Charles Max Wood Christopher Ferdinandi (Boston) Special Guests: Dan Shappir (Tel Aviv) In this episode, the panel talks with Dan Shappir who is a computer software developer and performance specialist at Wix.com. As Dan states, his job is to make 100 million websites (hosted on the Wix platform) load and execute faster! Past employment includes working for companies, such as: Ericom, Ericom Software, and BackWeb. He studied at Technion Institute of Management and currently lives in Tel Aviv, Israel. The panel talks about web performance API among other things. Check it out! Show Topics: 1:29 – Charles: Let us know who you are and why you’re famous! 1:39 – “Hello!” from Dan Shappir. 2:25 – Charles: You should say that you go to EACH site EVERY day out of the millions of sites out there. 2:53 – Charles: My mom mentioned Wix to me at first. My mom teaches High School Math. 3:16 – Dan: Yes that is our mission statement. That everyone can get a website without the knowledge of how to build a website. 3:52 – Aimee makes her comments. 3:59 – Dan: On our platform we try to offer people flexibility. There are bounds and limits, but people can do their very own thing, though. To make Wix faster because as we add more features and functionality that is our goal. 4:40 – Chuck: Okay, I know how to make X perform a little bit better. You are looking at a platform that controls TONS of sites, how do you even go about that? 4:58 – Dan: It is more difficult then that. We have millions of users leveraging the platform but there are a lot of developers in Wix who are developing the platform. I don’t think anyone at Wix has a total grasp of the complexity of the platform that we built. We have hundreds of frontend people working on our platform. All of them have pieces to the kingdom. We have processes in place with code reviews and whatnot, but there is so much going on. There is a change every 2 minutes, 24/7. We need to make sure progressing instead of regressing.  6:54 – Aimee: I think it was interesting in one of the links you sent over. Because you know when something is getting worse you consider that a bug. 7:15 – Dan: It is more than a bug because if we see regression in performance then that is a problem. I can literally see any part of the organization and say, “stop” if it will 7:57 – Chuck: We are talking about performance, but what does that mean? What measures are there? 8:15: Dan: We are looking at performance can mean different things in different contents. User sites, for example, most important aspect is load time. How quickly the page loads and gets open to the viewer to that specific site. When they click something they want it instantly and no drag time. It does change in different contexts. 9:58 – Chuck: People do talk about load time. People have different definitions of it. 10:12: Dan: Excellent question. When you look at the different sites through Wix. Different people who build sites – load time can mean something else to everybody. It can mean when you see the MAIN text or the MAIN image. If it’s on an ECON site then how soon can they purchase or on a booking site, how long can the person book X product. I heard someone at a conference say that load time is when: HERO TEXT And HERO IMAGE are displayed. 12:14 – Chuck: What is faster React or Vue? 12:21 – NEW HOST: Not sure. It all depends. 12:34 – Dan: We are big into React. We are one of the big React users outside of Facebook. I joined Wix four years ago, and even back then we were rebuilding our framework using React. One of our main modifications is because we wanted to do server-side rendered. 13:27 – Christopher asks Dan a question. 14:16 – Dan: We are in transition in this regard. Before we were totally client-site rendered, and that was the case until middle of last year. Then we deployed... Dan: We are 100% server-side rendered now. Some things we are still using JavaScript. We have another project going on now and it’s fully CSS, and little JavaScript as possible. What you might want to do with that site is... You might get in a few months every Wix site will be visible even if JavaScript is disabled. 16:26 – Aimee adds in her comments and observations to this topic. 16:55 – Dan: We don’t want things displayed incorrectly before it lays out. We hide the content while it’s downloading then make it visible. They lay-outing are done faster, because... 17:44 – Christopher asks Dan a question. 18:04 – Dan: I got into API... Either you are moving forward or are you moving back. AKA – You are either progressing or regressing. Different stages: 1.) Development stage 2.) Pre-Production (automated tools that check the performance with specific use cases) 3.) Check it out! It’s beneficial to use these APIs. 21:11 – Christopher: What is performance APIs? 21:38 – Dan: There is a working group – Todd from Microsoft and others who are exposing the information (that is available in the browser) out into the browser. When the browser downloads a certain source (image, font, etc.) it can measure the various stages of downloading that feature.  You have these different sages of downloading this resource. The browser can measure each of these stages and then expose them to you. Basically it’s for the browser to expose this information to you and in a way that is coherent and uniform. It essentially maintains this buffer that puts performance entries sequentially. Dan continues explaining this topic in detail. 25:55 – Dan: You have this internal buffer... 28:45 – Advertisement – Sentry – They support opensource. 29:39 – Christopher: everything you are saying seems that I can use this or that tab right now... Why would I prefer the API to something visual, hypothetically? 30:03 – Dan: Three Different Stages. (See above.) This information is very, very helpful during the developmental stage. Say you got a link from someone... Dan mentions: Performance.mark 34:04 – Aimee: When you were talking about resource-ends. Many people don’t know what this is. Can you spend 2-3 minutes about how you guys are using these? Are there people can add for big bang for their buck? 34:41 – Dan: This might want to be a topic for its own podcast show. Dan gives a definition of what a resource-end means. Go back to fonts as an example. Pre-connect for example, too. 39:03 – Dan: Like I said, it’s a huge topic. You have to exercise some care. Bandwidth is limited. Make sure you aren’t blocking other resources that you do need right now. 40:02 – Aimee: Sounds like a lot of great things to tap into. Another question I have is about bundling. 40:27 – Dan: One of the things that we try to do (given that we are depending on the JavaScript we are downloading) we need to download JavaScript content to the client side. It has been shown often that JS is the most impactful resources that you need to download. You really want to be as smart as possible with that. What is even more challenging is the network protocols are changing. Dan continues to go in-depth about this topic. Dan: What we have found is that you want to strive to bundle resources together. 44:10 – Aimee: Makes sense. 44:15 – Dan continues talking about this topic. 45:23 – Chuck asks two questions. (First question is now and second question is at 51:32.) 2 Questions: 1. You gather information from web performance AI - What system is that? 45:42 – Dan: I am not the expert in that. I will try not to give misleading information. Actually let me phrase it different. There are 3rd party tools that you can use leverage in your website. IF you are building for commercial reasons I highly recommend that you use performance-monitoring solution. I am not going to advertise one because there are tons out there. We ended up rolling out our own infrastructure because our use case is different than most. At a conference I talked with a vendor and we talked about... 51:32 – 2nd Question from Charles to Dan: Now you’ve gathered this information now what to you do? What patterns? What do you look for? And how do you decide to optimize things? 54:23 – Chuck: Back to that question, Dan. How should they react to it and what are they looking for 54:41 – Dan: Three main ways: 1.) Generate alerts 2.) See trends over long period of time 3.) Looking at real-time graphs. Frontend developer pro is that likely being woken up in the middle of the night is lower. We might be looking at the real time graph after we deployed... 57:31 – Advertisement – Get a Coder Job! 58:10 – Picks! Links: JavaScript jQuery React Elixir Elm Vue Wix Window Performance Web Performance Terra Genesis Terra Genesis: Space Colony The One Thing DevChat TV – YouTube GitHub: Off Side HBO: Insecure Wix: Engineering JavaScript Riddle JavaScript Riddles for Fun and for Profit Dan Shappir’s Twitter Dan Shappir’s LinkedIn Dan Shappir’s Crunch Base Dan Shappir’s GitHub Dan Shappir’s Talk through Fluent Dan Shappir’s Medium Dan Shappir’s YouTube Talk: JavaScript riddles for fun and profit Sponsors: Code Badges Kendo UI Sentry Digital Ocean Cache Fly   Picks: Aimee: Waking up early! How to Deal with Dirty Side Effects in Your Pure Functional JavaScript Chris: Offside - Toomuchdesign Insecure TV Show Charles: Terraform - Game “The One Thing" Code Badge DevChat on YouTube Dan Wix Engineering JavaScript Riddle

Panel: Aimee Knight Charles Max Wood Christopher Ferdinandi (Boston) Special Guests: Dan Shappir (Tel Aviv) In this episode, the panel talks with Dan Shappir who is a computer software developer and performance specialist at Wix.com. As Dan states, his job is to make 100 million websites (hosted on the Wix platform) load and execute faster! Past employment includes working for companies, such as: Ericom, Ericom Software, and BackWeb. He studied at Technion Institute of Management and currently lives in Tel Aviv, Israel. The panel talks about web performance API among other things. Check it out! Show Topics: 1:29 – Charles: Let us know who you are and why you’re famous! 1:39 – “Hello!” from Dan Shappir. 2:25 – Charles: You should say that you go to EACH site EVERY day out of the millions of sites out there. 2:53 – Charles: My mom mentioned Wix to me at first. My mom teaches High School Math. 3:16 – Dan: Yes that is our mission statement. That everyone can get a website without the knowledge of how to build a website. 3:52 – Aimee makes her comments. 3:59 – Dan: On our platform we try to offer people flexibility. There are bounds and limits, but people can do their very own thing, though. To make Wix faster because as we add more features and functionality that is our goal. 4:40 – Chuck: Okay, I know how to make X perform a little bit better. You are looking at a platform that controls TONS of sites, how do you even go about that? 4:58 – Dan: It is more difficult then that. We have millions of users leveraging the platform but there are a lot of developers in Wix who are developing the platform. I don’t think anyone at Wix has a total grasp of the complexity of the platform that we built. We have hundreds of frontend people working on our platform. All of them have pieces to the kingdom. We have processes in place with code reviews and whatnot, but there is so much going on. There is a change every 2 minutes, 24/7. We need to make sure progressing instead of regressing.  6:54 – Aimee: I think it was interesting in one of the links you sent over. Because you know when something is getting worse you consider that a bug. 7:15 – Dan: It is more than a bug because if we see regression in performance then that is a problem. I can literally see any part of the organization and say, “stop” if it will 7:57 – Chuck: We are talking about performance, but what does that mean? What measures are there? 8:15: Dan: We are looking at performance can mean different things in different contents. User sites, for example, most important aspect is load time. How quickly the page loads and gets open to the viewer to that specific site. When they click something they want it instantly and no drag time. It does change in different contexts. 9:58 – Chuck: People do talk about load time. People have different definitions of it. 10:12: Dan: Excellent question. When you look at the different sites through Wix. Different people who build sites – load time can mean something else to everybody. It can mean when you see the MAIN text or the MAIN image. If it’s on an ECON site then how soon can they purchase or on a booking site, how long can the person book X product. I heard someone at a conference say that load time is when: HERO TEXT And HERO IMAGE are displayed. 12:14 – Chuck: What is faster React or Vue? 12:21 – NEW HOST: Not sure. It all depends. 12:34 – Dan: We are big into React. We are one of the big React users outside of Facebook. I joined Wix four years ago, and even back then we were rebuilding our framework using React. One of our main modifications is because we wanted to do server-side rendered. 13:27 – Christopher asks Dan a question. 14:16 – Dan: We are in transition in this regard. Before we were totally client-site rendered, and that was the case until middle of last year. Then we deployed... Dan: We are 100% server-side rendered now. Some things we are still using JavaScript. We have another project going on now and it’s fully CSS, and little JavaScript as possible. What you might want to do with that site is... You might get in a few months every Wix site will be visible even if JavaScript is disabled. 16:26 – Aimee adds in her comments and observations to this topic. 16:55 – Dan: We don’t want things displayed incorrectly before it lays out. We hide the content while it’s downloading then make it visible. They lay-outing are done faster, because... 17:44 – Christopher asks Dan a question. 18:04 – Dan: I got into API... Either you are moving forward or are you moving back. AKA – You are either progressing or regressing. Different stages: 1.) Development stage 2.) Pre-Production (automated tools that check the performance with specific use cases) 3.) Check it out! It’s beneficial to use these APIs. 21:11 – Christopher: What is performance APIs? 21:38 – Dan: There is a working group – Todd from Microsoft and others who are exposing the information (that is available in the browser) out into the browser. When the browser downloads a certain source (image, font, etc.) it can measure the various stages of downloading that feature.  You have these different sages of downloading this resource. The browser can measure each of these stages and then expose them to you. Basically it’s for the browser to expose this information to you and in a way that is coherent and uniform. It essentially maintains this buffer that puts performance entries sequentially. Dan continues explaining this topic in detail. 25:55 – Dan: You have this internal buffer... 28:45 – Advertisement – Sentry – They support opensource. 29:39 – Christopher: everything you are saying seems that I can use this or that tab right now... Why would I prefer the API to something visual, hypothetically? 30:03 – Dan: Three Different Stages. (See above.) This information is very, very helpful during the developmental stage. Say you got a link from someone... Dan mentions: Performance.mark 34:04 – Aimee: When you were talking about resource-ends. Many people don’t know what this is. Can you spend 2-3 minutes about how you guys are using these? Are there people can add for big bang for their buck? 34:41 – Dan: This might want to be a topic for its own podcast show. Dan gives a definition of what a resource-end means. Go back to fonts as an example. Pre-connect for example, too. 39:03 – Dan: Like I said, it’s a huge topic. You have to exercise some care. Bandwidth is limited. Make sure you aren’t blocking other resources that you do need right now. 40:02 – Aimee: Sounds like a lot of great things to tap into. Another question I have is about bundling. 40:27 – Dan: One of the things that we try to do (given that we are depending on the JavaScript we are downloading) we need to download JavaScript content to the client side. It has been shown often that JS is the most impactful resources that you need to download. You really want to be as smart as possible with that. What is even more challenging is the network protocols are changing. Dan continues to go in-depth about this topic. Dan: What we have found is that you want to strive to bundle resources together. 44:10 – Aimee: Makes sense. 44:15 – Dan continues talking about this topic. 45:23 – Chuck asks two questions. (First question is now and second question is at 51:32.) 2 Questions: 1. You gather information from web performance AI - What system is that? 45:42 – Dan: I am not the expert in that. I will try not to give misleading information. Actually let me phrase it different. There are 3rd party tools that you can use leverage in your website. IF you are building for commercial reasons I highly recommend that you use performance-monitoring solution. I am not going to advertise one because there are tons out there. We ended up rolling out our own infrastructure because our use case is different than most. At a conference I talked with a vendor and we talked about... 51:32 – 2nd Question from Charles to Dan: Now you’ve gathered this information now what to you do? What patterns? What do you look for? And how do you decide to optimize things? 54:23 – Chuck: Back to that question, Dan. How should they react to it and what are they looking for 54:41 – Dan: Three main ways: 1.) Generate alerts 2.) See trends over long period of time 3.) Looking at real-time graphs. Frontend developer pro is that likely being woken up in the middle of the night is lower. We might be looking at the real time graph after we deployed... 57:31 – Advertisement – Get a Coder Job! 58:10 – Picks! Links: JavaScript jQuery React Elixir Elm Vue Wix Window Performance Web Performance Terra Genesis Terra Genesis: Space Colony The One Thing DevChat TV – YouTube GitHub: Off Side HBO: Insecure Wix: Engineering JavaScript Riddle JavaScript Riddles for Fun and for Profit Dan Shappir’s Twitter Dan Shappir’s LinkedIn Dan Shappir’s Crunch Base Dan Shappir’s GitHub Dan Shappir’s Talk through Fluent Dan Shappir’s Medium Dan Shappir’s YouTube Talk: JavaScript riddles for fun and profit Sponsors: Code Badges Kendo UI Sentry Digital Ocean Cache Fly   Picks: Aimee: Waking up early! How to Deal with Dirty Side Effects in Your Pure Functional JavaScript Chris: Offside - Toomuchdesign Insecure TV Show Charles: Terraform - Game “The One Thing" Code Badge DevChat on YouTube Dan Wix Engineering JavaScript Riddle

Panel: Aimee Knight Charles Max Wood Christopher Ferdinandi (Boston) Special Guests: Dan Shappir (Tel Aviv) In this episode, the panel talks with Dan Shappir who is a computer software developer and performance specialist at Wix.com. As Dan states, his job is to make 100 million websites (hosted on the Wix platform) load and execute faster! Past employment includes working for companies, such as: Ericom, Ericom Software, and BackWeb. He studied at Technion Institute of Management and currently lives in Tel Aviv, Israel. The panel talks about web performance API among other things. Check it out! Show Topics: 1:29 – Charles: Let us know who you are and why you’re famous! 1:39 – “Hello!” from Dan Shappir. 2:25 – Charles: You should say that you go to EACH site EVERY day out of the millions of sites out there. 2:53 – Charles: My mom mentioned Wix to me at first. My mom teaches High School Math. 3:16 – Dan: Yes that is our mission statement. That everyone can get a website without the knowledge of how to build a website. 3:52 – Aimee makes her comments. 3:59 – Dan: On our platform we try to offer people flexibility. There are bounds and limits, but people can do their very own thing, though. To make Wix faster because as we add more features and functionality that is our goal. 4:40 – Chuck: Okay, I know how to make X perform a little bit better. You are looking at a platform that controls TONS of sites, how do you even go about that? 4:58 – Dan: It is more difficult then that. We have millions of users leveraging the platform but there are a lot of developers in Wix who are developing the platform. I don’t think anyone at Wix has a total grasp of the complexity of the platform that we built. We have hundreds of frontend people working on our platform. All of them have pieces to the kingdom. We have processes in place with code reviews and whatnot, but there is so much going on. There is a change every 2 minutes, 24/7. We need to make sure progressing instead of regressing.  6:54 – Aimee: I think it was interesting in one of the links you sent over. Because you know when something is getting worse you consider that a bug. 7:15 – Dan: It is more than a bug because if we see regression in performance then that is a problem. I can literally see any part of the organization and say, “stop” if it will 7:57 – Chuck: We are talking about performance, but what does that mean? What measures are there? 8:15: Dan: We are looking at performance can mean different things in different contents. User sites, for example, most important aspect is load time. How quickly the page loads and gets open to the viewer to that specific site. When they click something they want it instantly and no drag time. It does change in different contexts. 9:58 – Chuck: People do talk about load time. People have different definitions of it. 10:12: Dan: Excellent question. When you look at the different sites through Wix. Different people who build sites – load time can mean something else to everybody. It can mean when you see the MAIN text or the MAIN image. If it’s on an ECON site then how soon can they purchase or on a booking site, how long can the person book X product. I heard someone at a conference say that load time is when: HERO TEXT And HERO IMAGE are displayed. 12:14 – Chuck: What is faster React or Vue? 12:21 – NEW HOST: Not sure. It all depends. 12:34 – Dan: We are big into React. We are one of the big React users outside of Facebook. I joined Wix four years ago, and even back then we were rebuilding our framework using React. One of our main modifications is because we wanted to do server-side rendered. 13:27 – Christopher asks Dan a question. 14:16 – Dan: We are in transition in this regard. Before we were totally client-site rendered, and that was the case until middle of last year. Then we deployed... Dan: We are 100% server-side rendered now. Some things we are still using JavaScript. We have another project going on now and it’s fully CSS, and little JavaScript as possible. What you might want to do with that site is... You might get in a few months every Wix site will be visible even if JavaScript is disabled. 16:26 – Aimee adds in her comments and observations to this topic. 16:55 – Dan: We don’t want things displayed incorrectly before it lays out. We hide the content while it’s downloading then make it visible. They lay-outing are done faster, because... 17:44 – Christopher asks Dan a question. 18:04 – Dan: I got into API... Either you are moving forward or are you moving back. AKA – You are either progressing or regressing. Different stages: 1.) Development stage 2.) Pre-Production (automated tools that check the performance with specific use cases) 3.) Check it out! It’s beneficial to use these APIs. 21:11 – Christopher: What is performance APIs? 21:38 – Dan: There is a working group – Todd from Microsoft and others who are exposing the information (that is available in the browser) out into the browser. When the browser downloads a certain source (image, font, etc.) it can measure the various stages of downloading that feature.  You have these different sages of downloading this resource. The browser can measure each of these stages and then expose them to you. Basically it’s for the browser to expose this information to you and in a way that is coherent and uniform. It essentially maintains this buffer that puts performance entries sequentially. Dan continues explaining this topic in detail. 25:55 – Dan: You have this internal buffer... 28:45 – Advertisement – Sentry – They support opensource. 29:39 – Christopher: everything you are saying seems that I can use this or that tab right now... Why would I prefer the API to something visual, hypothetically? 30:03 – Dan: Three Different Stages. (See above.) This information is very, very helpful during the developmental stage. Say you got a link from someone... Dan mentions: Performance.mark 34:04 – Aimee: When you were talking about resource-ends. Many people don’t know what this is. Can you spend 2-3 minutes about how you guys are using these? Are there people can add for big bang for their buck? 34:41 – Dan: This might want to be a topic for its own podcast show. Dan gives a definition of what a resource-end means. Go back to fonts as an example. Pre-connect for example, too. 39:03 – Dan: Like I said, it’s a huge topic. You have to exercise some care. Bandwidth is limited. Make sure you aren’t blocking other resources that you do need right now. 40:02 – Aimee: Sounds like a lot of great things to tap into. Another question I have is about bundling. 40:27 – Dan: One of the things that we try to do (given that we are depending on the JavaScript we are downloading) we need to download JavaScript content to the client side. It has been shown often that JS is the most impactful resources that you need to download. You really want to be as smart as possible with that. What is even more challenging is the network protocols are changing. Dan continues to go in-depth about this topic. Dan: What we have found is that you want to strive to bundle resources together. 44:10 – Aimee: Makes sense. 44:15 – Dan continues talking about this topic. 45:23 – Chuck asks two questions. (First question is now and second question is at 51:32.) 2 Questions: 1. You gather information from web performance AI - What system is that? 45:42 – Dan: I am not the expert in that. I will try not to give misleading information. Actually let me phrase it different. There are 3rd party tools that you can use leverage in your website. IF you are building for commercial reasons I highly recommend that you use performance-monitoring solution. I am not going to advertise one because there are tons out there. We ended up rolling out our own infrastructure because our use case is different than most. At a conference I talked with a vendor and we talked about... 51:32 – 2nd Question from Charles to Dan: Now you’ve gathered this information now what to you do? What patterns? What do you look for? And how do you decide to optimize things? 54:23 – Chuck: Back to that question, Dan. How should they react to it and what are they looking for 54:41 – Dan: Three main ways: 1.) Generate alerts 2.) See trends over long period of time 3.) Looking at real-time graphs. Frontend developer pro is that likely being woken up in the middle of the night is lower. We might be looking at the real time graph after we deployed... 57:31 – Advertisement – Get a Coder Job! 58:10 – Picks! Links: JavaScript jQuery React Elixir Elm Vue Wix Window Performance Web Performance Terra Genesis Terra Genesis: Space Colony The One Thing DevChat TV – YouTube GitHub: Off Side HBO: Insecure Wix: Engineering JavaScript Riddle JavaScript Riddles for Fun and for Profit Dan Shappir’s Twitter Dan Shappir’s LinkedIn Dan Shappir’s Crunch Base Dan Shappir’s GitHub Dan Shappir’s Talk through Fluent Dan Shappir’s Medium Dan Shappir’s YouTube Talk: JavaScript riddles for fun and profit Sponsors: Code Badges Kendo UI Sentry Digital Ocean Cache Fly   Picks: Aimee: Waking up early! How to Deal with Dirty Side Effects in Your Pure Functional JavaScript Chris: Offside - Toomuchdesign Insecure TV Show Charles: Terraform - Game “The One Thing" Code Badge DevChat on YouTube Dan Wix Engineering JavaScript Riddle

In this episode, Robert, Charles, and Wil talk about the whys and hows of accessibility, as well as what makes single page applications special, why they are they harder for accessibility, and frameworks that can do this for you. Resources: #SkyQ app on #iOS from a #VoiceOver user's Perspective Rob's Routing Doc Wil's PR Single Page Apps routers are broken Greater Than Code Episode #92: A11y Ally with Rob DeLuca This show was produced by Mandy Moore, aka @therubyrep of DevReps, LLC. Transcript: ROBERT: Hello everyone. Welcome to The Frontside Podcast. This is Episode 111. I'm Robert DeLuca, a software developer here at the Frontside and I'll be your episode host. Today, we're going to be discussing accessibility in single page apps. With me as co-hosts are Charles Lowell. Hey, Charles. CHARLES: What is up Robert? ROBERT: And Wil Wilsman. Hey, Wil. WIL: Yo, yo, yo, yo. ROBERT: Sounds like we're ready to drop a disc track. We're not going to be dissing anybody here. We're going to be talking about helpful things with accessibility in single page apps. Before we get into the nitty-gritty of accessibility in single page apps because we're getting into some deep stuff, I think I want to cover a lot of 'how' because I know accessibility things are usually about why you should be doing it and then they touch on things like, "You should be using alt attributes for your images," but for single page apps, I think we need to go further. CHARLES: It always ends up like, "Then draw the rest of the accessibility owl." ROBERT: Yeah, here's your two circles and the alt-tags are your circles and then the rest of the freaking owl is focus management and everything else that comes with it. Before we get there, what is accessibility? I guess, if we trim the giant umbrella down a little bit from everything that is accessibility that can be physical space things, like wheelchair accessible ramps or things like that, what about just technology? WIL: People need assistive tech to interact with technology such as switches or keyboards and obviously screen reader is a big one but when it comes to the software itself, you could even talk about colors and people who are color blind, so red might not be red to everybody. ROBERT: Speaking from experience there? WIL: Yeah. I'm colorblind. Red is brown to me. CHARLES: Things like hearing and all of it, right? It really is like just designing it in such a way that it can be used by as many people as possible. ROBERT: Right. WIL: And that includes your mom who may not be the best with technology but she still needs to pay your bills online or something. ROBERT: Exactly, yeah and in for context to listeners that might not know, my mom is 100% blind, so it's kind of where it comes from. CHARLES: But my mom is not but she has all kinds of problems. WIL: Yeah, same with my mom. CHARLES: That also falls under the category of accessibility, right? ROBERT: Absolutely. CHARLES: Right, accounting for age and culture. ROBERT: We're blending into the why of accessibility, which is perfect. One of the things that it's such a good segue because what people are starting to realize and I think why accessibility is really starting to catch wind and get some traction is because a lot of people that grew up in the technology age were open in using technology a lot. Our parents probably did not use technology heavily. That's definitely the case for my dad. He still has a flip phone and he says, he's a low tech man living in a high tech world and just refuses to pick up technology but those are the people that just didn't use technology. But now we have a lot of people that grew up with technology and use a lot and they're aging into more disabilities and they're going to need that accessibility, which I think is really interesting to think about because that's a lot of buying power if you're just going to start moving in that needs in accessibility, right? CHARLES: That's true. I know I may need glasses pretty soon, so colors and fonts were going to be heck a lot more to me in the next five years than they have over the last 20. ROBERT: Yup, exactly and that's going to be huge for that and that's one piece of the why, so what are the other reasons that you'd pick up accessibility other than people saying that it's morally correct. I don't like starting off conversations for accessibility because it's the thing that you should be doing. WIL: I think it goes even to my user experience like power users that don't like using the mice or mouse. That's me. I really prefer to just use my keyboard for everything. When the new Firefox browser came out and I couldn't navigate through the menus the way I was used to, I went back to Chrome. ROBERT: That's interesting. CHARLES: I still have not found a good workflow for navigating tabs with the keyboard without just kind of twisting my wrist all out of shape. You have to share that with me but again, that's another thing. That's an impediment that sits between me and the application, that actually -- ROBERT: Which is really interesting, you're getting into a keyboard navigation and focus management, which is really the crux of accessibility for screen reader users and switch users. CHARLES: What I'm hearing is that in this case, including good keyboard and focus management in your application, e.g. making it accessible to screen readers, you at the same time, enabling your power users. I think a great point is that by introducing these very low friction workflows, you're actually going to be enabling other parts of your customer base and not just catering to one, that there are ripple effects throughout your system. ROBERT: It may set it for everyone. WIL: Yeah, not only people who need it but people who don't know they need it. ROBERT: Yes, exactly. Think about the WCAG spec as user experience guidelines. They're not telling you how to implement a thing specifically for a screen reader. They're telling you how to implement it in a way that works for everyone, regardless of what ailment they have. It could be a temporary ailment. It could be a permanent ailment, where they have to use a screen reader or any kind of ailment that you can think of. They're not considering just one use case. It's a broad thing that shows how you can make your application better for everyone. I think that's a better way to look at the WCAG spec than I need to read through this and make sure that this auto complete works for a screen reader. If you look at the guidelines, they're not telling you just first screen reader. They're telling you how to make it work for a switch, someone who is colorblind or who is using a dictation software. I kind of tend to look at that as UX guideline, which really helps me build a better app overall because when you nail down that user flow, everyone benefits from it because it's pretty well thought out at that point. CHARLES: I like that too and I think that thinking of it as user experience and making sure that you have a complete user experience is a good way to think about it because it kind of separates the concerns of, "I've got HTML but is my application really HTML or is it a set of workflows and the data over which those workflows operate?" It really forces you to think my application is not a set of React opponents or web components or Ember components or what have you but really, there's a deep structure to it and it makes you kind of shine a light on that deep structure and try to map its surface. Then, if you really, really know it and you capture it, then you can represent it in any medium. I think it's just a win not just for one niche group of users but also for all of our users and then also for your future users that you don't have because your application is designed better and is going to work. Who knows? Maybe there's some new interface or some new medium, some new device that comes out that hasn't even been invented yet. But if you really have a strong internal representation of what your application is, you're going to be able to be the first to move to that market. ROBERT: Right and like I said earlier, people are starting to aid in the needing accessibility thing. If you need a dollar to justify it, that's going to be a big reason coming up. As a part of the new WCAG 2.1 spec, there is a zoom. I forget what this access criteria number is but the new criteria says your app basically needs to be responsive. That kind of maps directly to what you said earlier, Charles which is like you're going to need glasses soon and fonts and being able to zoom. It's going to be really important. We see that just pop up everywhere. To give a counter example of not just for accessibility like you need it for glasses but the other side of that could be like when we give demos on low resolution projectors or screencast or anything, when we zoom the screen, the apps just still be usable and you should still be able to demo it and that's just something that you have both sides to that, where accessibility kind of works out for everybody. People on the call probably don't need glasses but to see that tiny screen that's being shared, you should be able to assume that. CHARLES: Right. I'm just kind of restating what you said but I want to make sure to call it out explicitly because it was a definitely an aha moment for me, that you basically if you build yourself an accessible website, you build it so that it works on projectors and mobile devices too, so you kind of killed two birds with one stone and you don't have to make a special effort because zooming in the screen is tantamount to viewing it on a phone or viewing it on a tablet. WIL: Yeah, we mentioned it earlier about accessibility and physical space and one of those things is being able to access something from anywhere no matter the device that person is using. ROBERT: Yes. That's a lot of the 'why' of accessibility and I think we did a pretty good job of staying away from the more argument because morally we don't know if it's the right thing to do but a lot of the times it's not in front of you, it's hard to do and I want to make sure that you don't feel bad if you're not building accessibility in your apps. It's not easy. I don't sort of like people get up and say that, "Accessibility is easy. Just do this," because it's not -- WIL: If it was easy, everybody would do it. ROBERT: Exactly. I always come back to accessibility being just like UX -- user experience -- because if it were easy, everybody would have a really great user experience too. It's a hard thing to boil down and to simplify it, right? CHARLES: Right and like every other aspect, the kind of nonfunctional requirement, I say nonfunctional requirement that's a little bit of a contradiction terms but it becomes very hard if you haven't done it from the beginning. But if you didn't start with an accessible app, you weren't thinking about that or you inherited the app or it just wasn't on your radar for whatever reason. If you got a codebase that's two years old, going back and trying to make it accessible, it was extremely hard. It's expensive, expensive, expense. WIL: Yeah, it's going to cost more in terms of money and time to added it after the fact. ROBERT: Exactly. I spent a year on helping on Visa Checkout and there were some accessibility considered in the designs but a lot of the time my feedback wasn't just like, "Yeah, sprinkle some ARIA attributes on there and you're good." It was like, "Do we really need a carousel here to represent your list of carts because it's really hard to navigate around that?" A lot of the times, it ends up boiling up to is this the best way we can represent this data? Is this the best way we can navigate and build this user flow? A lot of times -- CHARLES: And if the answer is no, it's so painful, right? ROBERT: Yeah, exactly, so all that work that was put into building that carousel and all the components that built that carousel together is thrown out because it's just not a pattern that should be used there. Doing after the fact is really hard and really expensive and usually ends up in refactoring anyways. CHARLES: I just want to point that out because a lot of people find themselves in that situation, where they're staring down at a pretty big cost. Now, the reasons why your app may not be accessible or many and good and you shouldn't feel bad, if you're finally in that situation. ROBERT: I gave a talk at Nodevember a couple of years ago called Accessibility Debt and it's just like any technical debt. Your apps going to have it. It's okay. Don't get beat up about it, especially if anybody is trying to beat you up over it, don't listen to them. It really is just like any other kind of technical form of technical debt. It's something that you'll have to deal with and it is just something you have to work through. It's not world ending. It's just another problem to work through. What are some things like everybody usually talks about like the things you can do, the basics for making your app more accessible like using all its attributes and instead of doing a div with an on-click handler, just use a button or don't overuse ARIA attributes. Are there any other things that I missed there like the basics of accessibility? WIL: The biggest is the HTML structure. Screen readers and other assistive tech were built with the standards in mind, so if you're doing nonstandard things like putting divs in H1 and adding ARIA attributes within there, you're not going to have a great time. ROBERT: Right or splattering ARIA roles all over the place, probably not a good idea. That will be harder to debug. Also fun fact, ARIA roles, while you can implement directly to the spec, you may still have bugs across the different screen reader combinations or assistive tech combinations, so that's fine. CHARLES: Keep it super simple is what I'm hearing, like use semantic markup. If you're going to introduce a custom button, still make sure that it's a button. ROBERT: Use the platform. CHARLES: Yeah, use the platform. Don't fight the platform. Probably the best example of that is people implementing their own select boxes. That's the classic example. ROBERT: Wil and I, our lives has centered around that for a little while. It's so true. Usually, it's the first thing people go to grab to reimplement because selects are just ugly. I think Firefox has the ability for you to style select options now like you can change the color and the font but you can't style that. The pop up that comes, usually that's the system dialogue, which a lot of designers don't really like. That's usually the first thing that people go to implement and that's usually actually the first thing that stop somebody from signing up. A lot of sign up forms that I see, if your date of birth is in a select format, that probably will hinder somebody that uses assistive tech from signing up. CHARLES: Yeah. You just basically bounced that entire person. The thing is people don't appreciate the cost. This gets into the whole concept of accessibility that is how much money would that person or those group of people have actually brought into your site versus the cost that you spent redoing that select box. You might be thinking, "It only took this developer actually two weeks," but when you actually look at the actual cost over the course of your application, you're not factoring that into the decision to go with custom select box. Just in our experience, it's just the truly low cost option per quality of experience, that tradeoff there is almost invariably going with platforms select, right? ROBERT: Yeah. Your secret sauce and the best UX that you're going to provide is not going to be nicely styled select box and seriously, a battle that I had to fight a lot was if you really want to implement a custom widget, decide if this is what you want to spend your time on because custom widgets aren't just quick and easy things you implement. You're not going to implement the select that's fully accessible across all the AT combos in a couple of days. CHARLES: It's a lot of work. ROBERT: Yeah. You're going to fall down on a huge rabbit hole. CHARLES: Yeah. It is just you're committing to that work over the lifetime of your application. ROBERT: Exactly, if you can maintain that now. If you implement custom check boxes and custom radios and custom input that of content edible for some reason, I think -- WIL: I think what I see is like custom date pickers -- ROBERT: Oh, I just had a rant about that. CHARLES: Date pickers are hard and there's not really a good option. You just have to open your eyes to the true cost. ROBERT: Right, exactly. That's what I always try to explain, just like you have ownership over this now and you now have to maintain this and you can't regress. CHARLES: Right and if you do regress, it's your neck that gets choked. ROBERT: Yes. A good way to put it. We've talked a lot about things that are just general accessibility but nothing specific to single page apps. I do want to say like a lot of other things that people recommend is like if you're using React, use like the JSX ES1 plugin to help you analyze if you're writing any JSX that might not be accessible or use like HTML_CodeSniffer or aXe to statically analyze the DOM that you have. Those tools are great. I'd see a lot of people [inaudible] those things as like, "Look, this automated checkers says I'm inaccessible so I'm inaccessible," and that's not the case, especially in a single page apps. You can have 100% passing automated checkers but your app also could be 100% broken and why is that? WIL: I don't know. ROBERT: I'm wobbling the router question here. WIL: Yes. I guess that would just be due to routing. In single page applications, they handle their own routing, whereas static web sites and whatnot, the routing is handled despite URLs and the browser and reloading pages. ROBERT: Right. There are probably other major differences between single page apps but the biggest thing is the routing is managed by the client, the JavaScript and on a server-rendered page, you click a link, it's going to rerender the entire page for that next page and the screen reader and the browser know exactly what to do with that, to put the focus back to the top of the page and start working through the page for you. But with single page apps, you're just replacing a piece of the DOM and the screen reader has no idea of what's going on. An automated checker cannot check for this. They cannot tell you if your routes are accessible. The reason I'm bringing this up is because this has never talked about. I don't see in accessibility talks and this is the thing that actually is most broken and makes your app pretty much unusable to anybody that's using assistive tech. There's ways that people if they're savvy can navigate around it but if they don't know what's going on, they're going to think, "All right, I pressed this button and nothing happened, so I'm just going to leave now because it's not working." CHARLES: There's a great video that you point to me, I believe a guy from the UK who has recorded a bunch of his experiences using websites and apps -- ROBERT: Yes, I want to dig it up and put it in the show notes. CHARLES: Yeah. Those are great to watch because you'll really get it. ROBERT: Yeah and that's a really unique case because he's really savvy. I believe, I could be wrong but I think he might have said that he definitely work in tech somehow -- CHARLES: Right. He knows the workarounds and he knows these things but in some of the cases it's like, "If I didn't work in tech, there's no way that I would be able to use this website." ROBERT: Yeah. This is kind of the crux of like if you ever listen to anybody that is an accessibility consultant, they'll say, "You will never ever be able to automate accessibility," and this why I tie accessibility so much to user experience, would you ever have a user experience test that can tell you in a binary fashion? Yes or no, that your app has a great user experience? No, because it's pretty subjective. CHARLES: Of how does your users feel? right? ROBERT: Yeah. CHARLES: You can rate, "My users feel great." ROBERT: Accessibility is like that because there's a lot of context that you have to carry around. It's all about context. When I transition from this page, the next thing does the user have enough context of where they're coming from to where they're going to be able to operate on that page. Is there enough information to achieve the task they want? That is pretty much the crux of why there is no binary yes or no for that because it's contextual. It varies from person to person but you do your best to make sure that that works and that you provide enough information to do something. That's kind of a single page app as a crux. This is why we have a philosophy of testing as a whole. We don't test components individually because again, you can make all of your components individually accessible there but as a whole, they might not work together because you're not providing enough context on an entire page. We did this in one of the apps that we work on, which is open source, so we can link to the PR that Wil wrote for this and I wrote an entire routing documentation around our philosophy and the different things that we tried. How did we manage the focus in that application? I'm kind of just going to lob it over to you Wil since you did the work. Do you want to give context of the holdings and how that all kind of came together? WIL: Yes. One of the features of the app is these panels. It's like this three panel system. When you click an item in a list, a third panel pops up and this goes back to the context thing where if you're using a keyboard, you can't see the screen. You click an item in a list, how do you know that third panel popped up? The solution isn't for a component. The component can't be responsible for this. The list can't focus the item. It opens or vice versa, so this is definitely an application concern, where we needed to check against the route and see whether or not, the pane is opening or was already open or wasn't open before and when this pane opens for the first time, it will just focus it and that gives a lot of the context that the user needs when they click it, like they click an item in the list, it focuses this third pane and they're on a third pane. ROBERT: Right. We didn't even just focus on the entire div. We focused on the heading of the thing that you selected. WIL: Yeah because focusing the div, it might read something off but not all the time. The main thing we're focusing on that third pane opens is the heading to let them know that the item you click, you are now on that page and reading that heading. This is the same thing that would happen if you loaded up that page statically and the screen readers would usually just focus on that first heading. ROBERT: Right. That helped a lot. For a little bit more context, the middle pane is like kind of a master detail thing going on here and in the middle pane there that we have, it's an infinite scrolling list. You have different things there and one of them is like a package. If you select the package and without the focus management and focusing on that heading, you would have to go through every single package that's in that list which could be a thousand of them before you actually get over to the pane that you just opened because source order. You have to go through each one of those. WIL: And it's the same thing is true for power users, not just screen readers. It's like if they want to use tab once that pane open, they have to tab through the entire list. ROBERT: Exactly. It wasn't keyboard accessible and it made it really hard to navigate around with a keyboard because the focus just wasn't being managed. There was a lot of work that we did there. I want to focus on the routing situation there because if you can't navigate around with a keyboard in your single page app, like you click a link and it's not selecting the next thing that should be focused and you provide the right amount of context, your app probably won't be usable without a lot of trial and error to a user, which depending on what your product is, they may not have a lot of patience for trial and error, right? WIL: Yeah. ROBERT: It's really important to try and nail down the routing situation and there are some frameworks and things out there that can do it. In the app that we're talking about, it's React app and we use React router but we don't actually really hook into the router to handle that and that's because React router doesn't provide very much information. WIL: Yeah. You can think of React router as more of an outlet system, where your routes can render anything, anywhere on the page. That's kind of dangers of accessibility and that's kind of the reason that React router can't handle accessible things very well because at any point, the route can just change the button on the page to look like something else. CHARLES: Yeah. It just pop in and pop out. There's no deeper model, right? There's not -- WIL: Yeah, there's no tree. CHARLES: Right, there's the internal state -- WIL: Like a component tree, yeah. CHARLES: The internal state of what is happening is completely opaque. You can only analyze the second and third order effects of the React tree. ROBERT: Right and especially if you have nested route components, it's really hard to determine. One of the things that I've seen people do is focus on mount, which is a very naive approach because what if you have three nested routes, they're all going to focus on mount and the last one that mounts wins and that's a focus for which nobody wins. CHARLES: You all tried that, right? ROBERT: Yes. That's part of the document that I wrote up. We tried three different approaches and we ended up landing on something because we were using React router that was more of like an application state thing, so we were checking props because we knew what the user flow was. We knew what the user, when they come to this page is trying to accomplish, so we're able to kind of figure out from where they came from or where they're going through props and focus the right things for them. WIL: One of the examples, like we talked about the third pane opening and focusing the heading inside but what you know what happens when you close that third pane, you kind of lose contexts again, so we have to focus the previous list item that was active because if you focus back at the top of the list, they've essentially lost their place and the list of results. In that case, we couldn't use on mount. That list item is already mounted. We have to listen to props and we have to look at the route through these props to determine if that third pane was open. If it just closed and if an item in the list is active, it should have focus. It's a lot of logic going on. You have to really understand your app in order to make it a very good accessible app. You can't just sprinkle in ARIA attributes and focus on mount everywhere and think it'll work fine because you're accessible. You have to really know the flow. CHARLES: Right. All those signposts point towards having a deeper application state, a deeper understanding of your application than just the render tree. At that point, it's too late and so by that, I'm definitely lobbying a Reach/Ember router. WIL: Yes. We talked a lot about the React router and we can't really be too accessible with it but to create a React router to go out and there is now Reach router. Robert, have you heard any good things about that? You're the accessibility expert here. ROBERT: I haven't played with it myself, so [audio glitch] things that were lost from the React router three to four transition, I think and also, it's actually accessible routing which is nice because it comes out of the box and you know how to implement it. I haven't played with it. I know Gatsby has implemented that for their V2, that's their default router now, which makes me really happy because a lot of static sites that were being built with Gatsby were very inaccessible and broken which made me sad but now, they're not with V2. I haven't played with Reach router but one of the things that I think it provides which was missing from React router was transition hooks. It has a concept of like where you're going from and to for your routes, which really helps figure out what you need to focus. The one thing I will say about Reach router and I'm sure it's got to be configurable somewhere but I haven't really looked and the demos that I saw, they were just focusing the div of the content that's being rendered, like it kind of just wraps with a generic div or the tabindex="-1" and then focuses it. If you have a lot of content that's inside of that div, it probably will be confusing but at least, it manages the focus somewhere. If you're now off in a no man's land, you have no idea what's going on, at least it focuses that. But if you want to really nail the experience to be better, I would recommend trying to figure out what you should focus inside of that route that you just transition to, what is the best thing. That's for React. There are other things out there for other frameworks, like I am on Ember accessibility team that's out there and we have Ember A11y, which just provide you a focusing outlet for you to be able to just drop in your app and then when the route changes, it does the same thing. It focuses the wrapping div of that outlet that was just rendered. I want to emphasize more in talking about e-holdings work that we did that we were focusing the heading because that told you exactly where you're at and what package you're on or what thing you're on. You know the name of it and now you know how you can go and navigate through that. CHARLES: But it's conceivable, so how would you do that with the Ember router? Would you just introduce some way to delegate down to a particular component? Like when I'm rendered into an outlet, it focus on this component? ROBERT: That would be interesting. I actually don't know. It's been a long time since I've messed around with the Ember router and Ember accessibility. It's definitely a great first step and that's where it kind of came from. I think there needs probably some work done to help implement on what you want to focus. That's kind of where I was going with when we were first exploring the React router stuff and e-holdings was I wanted to have this like high order component that knew of your routing tree and it knew where you're coming from or where you're going. Then from there, it would just tell you, this is the route that we're going to and there's need to be focus management done, like if this prop exists, then you can pick inside of that component that what you want to focus. It leaves it up to the implementor of what they need to focus but it could be probably just like a fallback to the general div because that's not bad. It's a good first step. There needs to be a little bit of work there to get that done probably. CHARLES: Yeah. But it is worth pointing out that by starting from a position of having an externalized application state via the route structure, in an Ember application, you're starting 95% of the way there. An Ember application, at any point, you know where you are and during your transition, you know where you're starting and where you're going to end up and you know when you leave and you know when you got there. ROBERT: Yeah. Having an Ember route pivot handler there, like where you're pivoting from and to is just so nice and it kind of made it click together a lot easier. CHARLES: Right. Reach router looks interesting but as I understand, it's still couched in React components and it feels to me like this is a problem that ought to be solved, that ought to be framework agnostic. Because if I use something like Reach or I use some other routing library for another framework, some other tool might come out that I want to use and I shouldn't be locked in -- ROBERT: Right like what if I really like Redux little router, then I have to make a choice between Redux state that I like or accessibility. CHARLES: Exactly and that feels like a false dichotomy to me. What I would want to be looking for is a platform independent or a framework independent routing library that really just helps you represent your application state and the concrete state in which your application is in at any moment and then, also be able to represent the transitions between those states fully and completely. Then if you have that, you could embed that into any framework. ROBERT: Right. That would be really nice to have. Just like pull it off the shelf and help you out there. CHARLES: If anybody is listening who wants something to do for the next 18 months, no one will thank you for 18 months but you had to get on that. We'll give you lots of thanks then. ROBERT: [inaudible] compare with you. That would be awesome. CHARLES: I would love to be part of that. ROBERT: Yeah, that would be awesome. To kind of tie back to other things, I don't know too much about Angular but I'm sure there are solutions out there to help with Angular. I know Marcy Sutton used to do a lot of work in the Angular world, so I'm sure there's something out there that helps with that. I just don't know off the top of my head right now. I wrote a Medium 'think piece'... No, I wrote a little medium blog post about how all of single page out for routers are broken. I was pleasantly surprised by Vue. Vue brings its own router and their router has a concept of before each, so before each route transition you can run some code and that really helped with implementing the live, the announcer pattern where you use ARIA live to announce something but even beyond there, if you wanted to dig in there, you could probably figure out where you're transitioning from and to and give that next route some kind of attribute that says, "This needs to be focused. Figure out what you need to focus there and inside that route, you can focus wherever you want." I thought that was a really awesome. That's kind of the crux of what I was missing from React router. I wanted the concept of knowing where I'm coming from or where I'm going and I would help with everything but unfortunately, that kind of doesn't exists because they're just components. For better or for worse, they're just components. CHARLES: The world is so much more than just components. ROBERT: Yeah, a little bit off topic, I think it's kind of funny how React kind of just shoved everything into the Vue layer, just to make it all a component. CHARLES: Yeah, it's very easy. ROBERT: Yeah, until it's not. CHARLES: Yeah, exactly. It's easy but it's not simple. ROBERT: That's a lot of talking about how routes need to be done and what you can kind of do to manage focus. It's really about managing the context and how you can provide the most contexts. For somebody to operate on that information, can they complete this thing? What can you do to make sure that you've done this properly? What steps can you take to make sure that you actually are accessible and that your routes work? WIL: Manually testing those screen reader is probably the biggest thing, you know? CHARLES: Yeah. I think the biggest thing is really watching someone who uses assistive tech on a regular basis use your application and then trying to use it yourself. WIL: Yeah. ROBERT: Right. Yeah, I'll definitely -- CHARLES: If you can get a little bit of this yourself but then it's kind of like someone who is never used a mouse before and trying to learn something new. What really helps is seeing someone who's good at it and see how their expectations are either being met or not being met. ROBERT: Yeah, it's definitely the best way. If you can find somebody that actually relies on assistive tech, there's nothing that beats that kind of feedback. If they get to your app and they're really confused, I see some people that just dismiss it because they just don't understand. That is the best feedback you can actually get. If they don't understand what's going on, you have -- CHARLES: That's on you. ROBERT: Yeah, that is going to be what happens for everybody that uses that. Well, maybe not everybody because everybody has different experiences but it's probably going to be a thing that pops up everywhere. But if you don't have access to people that are actually using assistive tech regularly and are pros at it, WebAIM provides really great tutorials for how to use a screen reader. If you're using a Mac, you have a screen reader built in and you can use that called VoiceOver. If you ever want to turn it on or turn it off, its command F5. WIL: You might have to have that shortcut enabled, though. ROBERT: Really? I'm pretty sure it's quite default. WIL: I thought I had to enable mine but I could be wrong. ROBERT: It's interesting. CHARLES: Yeah. They got great tutorial too. It's like it notices the first time you turn it on, so it tries to help you navigate bullet lists and select boxes and input fields and check boxes and all kinds of good stuff. ROBERT: Yeah. It gives you a little bit of a boot camp but WebAIM also helps with specific to website and stuff because one of the things that we ran into while working on the e-holdings project is they're transitioning from a native app to a web app and there were just things that you can do on a native app that you cannot do on a web app. Just keep that in mind also when you're testing, there's just things that will behave differently. Like you're not going to have a lot of crazy key shortcut commands like you're not going to press command F to get to the search box if your app has a prominent search box because that is going to clobber a bunch of other assistive tech key commands. WebAIM is really a good help with giving you the tutorial of how to test a web app and many different screen readers, so they have it for JAWS, they have it for NVDA, they have it for VoiceOver iOS, they have it for TalkBack, which is the Android screen reader. There's a lot of really good resources there for you to start using as screen reader and test with your app. I highly recommend using it with a screen off. You gain a lot of context by looking ahead of your cursor and -- CHARLES: Yeah. It's true. ROBERT: -- The example that I usually give to people is like, "I worked on Visa Checkout. I went through that checkout flow pretty much every day for a year and eight months in, even then turning off the screen, I was still lost." Even though I knew what each screen was and what each component was there, I would find myself confused of like, what just happened and it's because sometimes, you'll get into something like you have a dropdown that sets the focus inside the dropdown and then the dropdown disappears from the DOM and your focuses in nowhere. You're like, "What is going? what am I doing. I don't even know where I'm at," and I turn the screen back on and I'm like, "Oh, now, I know what's going on." CHARLES: Right. It really is a lot like interacting with your application as though you were interacting with Siri or Alexa but with a keyboard instead, instead of voice commands. That's an excellent point. We would understand if where would Amazon be if Alexa couldn't successfully navigate those situations. The counterpoint or even the flip side of that is if you model your web application in such a way that it can handle that type of serial interaction, instead of the highly parallel environment to being able to perceive huge amounts of information concurrently, like you can on a screen, if that were effectively serialized, that means you could represent your application through nothing but voice commands. ROBERT: Yeah. Did you provide enough context for me to build this mental map is really what I'm going on to? CHARLES: Yup. ROBERT: Which I always thought was really interesting because I always wanted to know how my mom visualizes what a website looks like because it's wildly different than any of us. She's never been able to see what a website looks like. Does it look like a bunch of nodes and graphs and webs connecting to each other and how things pieced together? It's just a different way. But it's not only about screen readers, right? You can use a keyboard to navigate and that's definitely what we did with e-holdings is like can we tab through this [audio glitch] list, hit enter and go into the detail record, edit it, close it and go back and edit another one. Is that something that we can do with just a keyboard, not even a screen reader? WIL: And with the keyboard, we're not talking about shortcuts to hit edit. We're talking about like tabbing over and hitting enter like people with accessibility issues would have to do. ROBERT: Right and that's kind of a good segue into creating use cases. If you want to know if your app actually works, if your screen reader users or if your keyword users or if your dictation users are going to be able to navigate this app, create use cases. Things like actual user flows like how would somebody actually going to use this app? What task are they going want to complete? In that case, e-holding was like an electronic holdings management system for libraries. They probably want to get in there, add a couple of books or whatever you might have to their library and get out. A use case could be like, "Can I search for this thing? I'm going to search for something specific. I'm going to go through the list, find the thing that I want. I'm going to add it, close the pane, go back and then remove one thing and can they complete that flow successfully without running into any issues or any blockers or any showstoppers." I can tell you before we did the routing management stuff, you would hit search and that was it. There was nothing else that you could do. CHARLES: Yeah. They wouldn't even announce that anything can happen. ROBERT: Exactly. WIL: Yeah, and with these librarians, it's not necessarily a matter of they can't see the screen. It's just that they don't use the mouse because they're power users. ROBERT: Right. They don't have any disabilities but that was essential to the workflow. CHARLES: Yeah, exactly. Do that change. You just lowered the activation energy of that workflow by... What? three orders of magnitude? WIL: Yeah, at least. ROBERT: Let me click here right now. I can tab, now I can tab. Right now, let me click here and now I can tab, I can tab, I can tab. It's not as nice as just being able to completely do it through a keyboard. Through us making it super keyboard accessible, that also became super screen reader accessible and the people who use dictation were able to work through and get through the app and use it now, which was really cool. It really helps when you go for those things and create use cases to really figure out how a user is actually going to work through this app. That's the best way. Just get right through it. With Visa Checkout, we're like, "Can somebody buy something?" or if they don't have an account, can they sign up and buy? those were some of the use cases that we had because it turns out, those are actually pretty important to the business. That all have been said, you also can test your components for accessibility individually because even at a smaller level, some of your components might have to manage focus. The best one I can think of is models. When you open a model, you should trap the focus inside of that model and you should be able to hit escape to close the model and when you close the model, it should go back to what triggered the model to be opened. These are all things that are individual that can be tested also. But just because your components are individually accessible, it does not mean your application as a whole was accessible either. I don't want to paint the picture like, you don't have to care about your components accessibility individually because you do. It really does help but I think a lot of people miss the whole of the application, rather than individual with components. CHARLES: Right. The components are the individual stitches but you have to follow the thread throughout the entire garment. ROBERT: Right, exactly. CHARLES: Man, what I really want to do is I want to find out how your mom visualizes website navigation and use that as a visualization technique. ROBERT: That might be a fun webcast or something. CHARLES: Yeah and then see like could we actually use it as a tool because I have to imagine, it's probably pretty simple. It's much simpler than what we think of when we think of a website because it has to be really condensed down to its essence. ROBERT: Right. Yeah and [inaudible] users, they're not dumb. They have different ways. They know the gotchas. They know things that happen. They know there are different ways of getting trolled like my mom knows about focus jumping and she gets irritated what happens but she knows generally of what to do and it depends on her patience level. Like if it's focused jumping, she's like, "I don't need to use this thing. See you," and it's just not worth the frustration but there's different ways to navigate around like if you're a real power user, you might be able to recognize like the routing is inaccessible and you can navigate by headings or by regions or different landmarks. There's many different ways for users to navigate but to use those different navigation methods, you need to have a real strong coherent document structure. Your H1s have to actually be H1s and you have to have some things that they can navigate around to kind of work around those things. It's interesting and basically, do everything you can to help those situations. If you can provide semantic markup and give it a proper structure. If you can do the focus management, it's going to help everybody. It really will. I did see when we went through the use cases and defining those things, we actually learned a little bit more about our product because we had to put ourselves in a different seat and think about it because you get real close to it. You go through that same flow nine million times and you pick up real power user things that you can do like, "I'll work on that. I got this. I'll click this button. All right, we're good." Somebody that's going through it for the first time and you put yourself in that seat, it kind of opens your eyes a little bit and makes the experience better for everyone. I think that's kind of the underlying tone there. That's the message. WIL: Yeah, accessibility makes things better for everybody. ROBERT: That was a lot of content thrown at you there. We covered what is accessibility and why you'd want to do that and then kind of like more the basics things and how automated tools are really helpful and they can help you pick out things like using improper roles and nested things but they're not going to be able to tell you if your application is truly accessible or not and never will, unless we get something like a headless screen reader, where you can write automated tests for in that fashion but you're not to get something that will just run over you app and go, "Yup, you're good." CHARLES: Even so, it's a matter of user experience and that's not something you can get a thumbs up or a thumbs down to. When you as a user, it comes with application, you know when you see it but I would say until we have androids that accurately simulate human beings, I don't think we're going to actually have automated testing. ROBERT: Yeah. There's a joke for accessibility consultants. It's like if you put four accessibility consultants in a room and tell them to give you an alt attribute for an image, you'll get four different alts. We talked about the automated checkers, right? They'll not going to get everything for you and we talked about single page apps specifically in the routing and how we handled the routing in a React app and then how you can probably do it in an Angular app and how you can do it in an Ember app and Vue and different methods of how you can kind of attack that. We're definitely giving the link to the document that I wrote and the PR so you kind of see the real 'how' of how we did it because that would probably be helpful. I think there was a lot of good information there, so I would like to thank both Charles and Wil for being awesome co-host on this and -- WIL: Thanks for being an awesome host. CHARLES: Yeah, thanks for being an awesome host. I should say, you're welcome. ROBERT: I tried, I tried. We are The Frontside. We do accessibility consulting and training. If that's anything that your team needs help with, we're more than happy to jump on a call with you to kind of figure out what your needs are and what you need to do. If you need a WCAG support statement, if you need to audit, if you just need to figure out what's the next steps for you to even do, we're more than happy to help you sort through that. To reach out for that, you can contact us at Contact@Frontside.io or Sales@Frontside.io or Info@Frontside.io. You can contact us at any different ways and we'll be more than happy to help you. As always, thank you Mandy for producing the podcast. You're awesome and the next episode, what we're going to have is also still accessibility related and I'm really excited about this. It's about writing a proposal to the Unicode Committee and getting it accepted so basically, writing a proposal to get an emoji added and that's with Amberley and Amanda. They wrote three or four Unicode specs and actually got them accepted for, I believe it was sign language and deafness. That's really cool and I'm super excited for that because they'd be the first people that I've ever talked to that have actually created an emoji and gotten it accepted. WIL: Yeah, it's pretty cool. CHARLES: Yeah, that must feel great. ROBERT: Yeah, it's going to be awesome. That's our next episode. If you have any ideas or comments or anything, you can tweet us at @TheFrontside on Twitter or you can contact us through any of the emails that I talked about earlier. We're always open to hearing feedback. Thanks for listening. Take it easy, everyone.

Guest: Saron Yitbarek: @saronyitbarek | bloggytoons | CodeNewbie | @CodeNewbies In this episode, Charles and Sam talk to Saron Yitbarek about her idea of mentorship, ideas for distributed learning for businesses to promote individual and company growth, and why it's important to take "digital sabbaths" on the regular. This show was produced by Mandy Moore, aka @therubyrep of DevReps, LLC. Transcript: SAM: Hey, everyone. Welcome to Episode 110 of The Frontside Podcast. My name is Sam Keathley. I'm a developer here at the Frontside and I will be your episode host. Today, we're here with Saron Yitbarek, discussing mentoring. She is the founder of CodeNewbie and the host of the CodeNewbie Podcast. Also with me as a co-host is Charles Lowell, who is also a developer at Frontside. Welcome Saron and welcome Charles. How are you guys doing? SARON: Thanks for having me. I'm doing pretty well. CHARLES: Hello. SAM: Today is going to be an interesting take on the mentoring talk. I mostly want to know first off, Saron, how do you feel about mentoring? What are your opinions on the mentor-mentee relationship or the value there? SARON: Yeah. I have lots of opinions on this topic. I think that the traditional structure of mentorship was usually looks like someone with less experience going to someone who has a lot more experience and say, "Will you be my mentor?" kind of like the children's book, 'Are You My Mother?' like 'Are you my mentor?' and then that person, that mother figure, that mentor looks after them and checks in on them and they have regular coffees and lunches and kind of steers them in the right, usually career-related direction. I don't think that's very realistic, to be honest. I think about why that might be. There's many different reasons. I think the fact that we're so, so, so networked and there's just so many different ways to get in contact with people and build relationships is a big reason but I think that that traditional mentorship model, that kind of one directional way of doing things is just not really needed and kind of overrated. I think that mentorship nowadays looks more like a mutually beneficial relationship, where I might reach out to someone who has more experience than me, for example in drawing, in art, not something I want to get to do but I know a crap-ton about podcasting, so I help you, you're my mentor in this specific area, in this specific topic but then, I get to be a mentor in this other thing that I'm really good at. I think it's those types of very focused topic-oriented, ideally two-way relationships that are more accurate and frankly, a more effective way of doing mentorship. SAM: Yeah, I actually agree with that. I am pretty new myself to development, only really been in this career for about a year now and I always kind of consider that mentoring relationship as a regression back to school days, where you have these -- SARON: Yeah, yeah. SAM: -- considered superior over you in some way, well, that's really not true in this community of developers and no matter where you are in development, it's all about working together and pairing. Working here has really showed me that value in paring, rather than like I have to look up to someone and regress back to feeling like a teenager in high school, like this person so good at this thing and they're the only one who can teach me. I definitely share that view of mentoring. I went to a boot camp one day when they were telling you like, "Oh, you got to find a mentor. You got to find a mentor," and I was like, "Well, but why? Why do [inaudible] together?" SARON: It's also a huge responsibility, a huge burden on the mentor too. Having to be, in a lot of ways, responsible for someone's career and trajectory and direction, that's a big responsibility. We don't have time for that, you know? On both ends, whether it's feeling like you're back in school or feeling like you have this huge responsibility, I think the traditional model isn't really the best model for either party. I think this idea of let's all learn together, let's be really focused on topics and problems that are very particular to what I'm doing, what I'm learning, what I'm trying to do. I think that works out. It feels healthier for both people. SAM: Absolutely. CHARLES: I wonder also too, it seems like it might be a little bit of a throwback to the days when people would spend 30 years in a single company or 30, 35 years in a single career where you have these people who are really these reservoirs of this intense tribal knowledge. It seems like people move around a lot more in their careers, not only in the company that they work for but also in the things that they're literally doing. I might be podcasting one day and producing a bunch of content and then, I might move into music or writing or other things like that. The careers seem to be broken apart a little bit more is one of the reasons why older models of advancement in those careers might not be as good a fit as they once were. SARON: Yes, absolutely. I'm obviously very biased because I'm in tech but when you think about the different roles that people have in tech, I feel like we're always wearing so many different hats. We have to, obviously code and be technical in that sense but we have to be really good communicators, a lot of us are speakers, we host podcasts, we organize conferences, we go to meetups, we're bloggers, we do so many other things, that this idea of, "I'm going to have a mentor who can help me in my role of being a developer," is just too big. You kind of meet people who are good at those individual pieces and those individual skills to get me to where I want to go. SAM: The whole idea of mentoring to me always reminded me of that whole Mr Miyagi relationship where you have this master of something and you're trying to learn from them. But in software development, I've learned that really no one is a master of anything because it's just changing so much and everything is so different. SARON: Yup, absolutely. CHARLES: The question is obviously, it seems like the idea that you're going to find one person who's going to represent the ideal confluence of every single skill set that you could hope to want, to be at some point in your career. That's looking more and more ludicrous. Is there a model where you can try and distribute those things, where you single out a large range of individuals? I guess the kind of what you were hinting at the beginning is that it is a lot more broken up, a lot more distributed but how do you pursue that, even if it is in a distributed manner? SARON: Yeah, that's a great question. One of the moments where I realize that the distributed model is really the only way that makes sense is, I think it was three years ago maybe. We thought about doing some kind of mentorship program in CodeNewbies, some type of way for people to link up and find people who can be there and guide in a way and we had people fill out this survey that basically said what do you want out of a mentor, what do you look for, what do you hope for to achieve but also asked how do you see yourself. Do you see yourself as a mentor, a mentee or both? It was really surprising that most people checked off both, which I thought was so interesting. It was so interesting to me that the same people who said, "I need help," with the same people who also said, "I also have help to give," and to me, that was such an amazing moment because I said, "Wow, it really isn't about this idea of I am the guide and I'm going to guide you." It's really about, "I have information expertise in one area and not in others." What I realize is there really isn't a mentor model. I think it's more of a culture of being helpful, which probably sounds really cheesy but it's true. I really think it's about saying, "I know how to do a thing. I'm going to go on Stack Overflow and answer questions. I'm going to go on Twitter and answer questions. I'm going to write blog post and share with the world." I think the real model, the real distributed mentoring model is us as individuals saying, "I just learned how to do a thing," or, "I just figured out how to do a thing well. Let me capture that. Let me capture it in a response, in an answer, in a forum, in a post and let me share that," and the more we do that as individuals, the more we have this huge amazing aggregate of knowledge that can serve as mentorship for all of us. SAM: I like the touch on that. That's kind of the idea behind Codeland Conference, where everybody thought they might be new to development and everyone is just sharing their knowledge. You might feel you really knew about it but you know a lot more than you think you do and you could still help. SARON: Yup, exactly and that's the whole idea, even the Twitter chats. When we first started, I don't have all the answers, I don't claim to, I don't really want that responsibility but I know there are a lot of people who do and I know a lot of people who have resources and opinions and who can help out. One thing that people do is they'll DM us and say, "I'm having a hard time with this." Sometimes, it's a very technical problems. Sometimes, it's a general 'I'm having a hard time getting a job,' which I do like high level question and I never answer them. I always say, "Tweet us and we'll retweet it and we'll get the whole community involved and we can have a rich conversation around it," and that's really been our motto, our philosophy. I see what they do with mentorship. If you have a mentor, you're not obligated, I guess to listen to them but the idea is kind of that you should. There is one person and you should listen to them because they know more than you. I think that's just not really fair and instead, I like to think that there are lots of different ways to do things and it's up to you to decide what's best for you and in order for you to decide, you have to have a lot of options. With Codeland, with the Twitter chats, no matter what skill level you're at, no matter how confident you feel in your coding abilities, you do have something to offer. Let's pull that out. Let's put it on the table and let's see who you can help today. SAM: That's a perfect way of introducing someone to the idea of helping or getting help from a community, rather than an individual because you never want to take one person's word as gospel on something you just know nothing about. SARON: Yeah, exactly. SAM: You can sound confident talking about something and it can be the total wrong answer but if you're seen as someone superior or someone who knows what they're doing -- SARON: And you can sell it. SAM: Yeah. You can be the best snake oil salesman in the world. SARON: Absolutely. For the CodeNewbie podcast, we do short questions at the end of each episode and one of the questions -- my favorite question -- is what's the worst advice you've ever received. I love that question because I started by saying, I think that people love giving advice. I think people love asking for advice and the assumption again is if I give you advice, then I probably know more and you should probably listen to me but that's not always true. I want people to be comfortable making their own decisions and deciding for themselves. "This may sound like it could work and this may sound like a good idea, generally speaking, but for me, it's probably not a good fit. It's probably not what's best for me," and to be comfortable rejecting advice and that was kind of the reason why I started that question, it's my favorite question and it's so interesting how much advice is good. It's kind of generic but it's a good advice. It's an advice like, "Don't quit. Keep going," which maybe a good idea and maybe you do need to quit, so being open to rejecting advice, I think is really important and that one of my favorite questions. SAM: In your experience, what is the best way to reject advice? Because when you're a new developer, when you're new at anything and you're seeking advice from a community or from a person, you don't want to come off as rude or maybe you're feeling... I don't know, not very confident but you think the advice that you were given is just bad advice. What would your advice be? I guess, what would your advice be on this advice? For your perspective, what would be a good advice to reject advice that you think is just wrong? SARON: Number one is when I ask people for advice, I try to think about and try to know upfront, do they have the same values that I do? Do they have the same worldview? Do they have the same goals? Because that's the thing too. Oh, my God, I got so much unsolicited advice. It's amazing. I've actually stopped just saying to people my ideas or what's on my mind because I know as soon as I say, "I'm thinking about this," I'll get a whole slew of unsolicited advice and I'm like, "I didn't ask for that." What I've learned is first to kind of figure out are we even on the same page because if my goal is to be a developer, if your goal is to be -- the only thing I could think of is a juggler, I don't know why -- a juggler and I ask you for a career advice, I'm going to go ahead and safely assume that what you have to say is probably not as applicable to me and so, number one is kind of identifying that. But number two is if they say something that just I know is not going to work or I've already tried, I'll just nod and say, "Thank you very much," and kind of go about my day. I don't think you have to declare whether or not you going to take it. I think just acknowledging and I think the people that give advice, I think they're trying to be helpful, they have good intentions, usually. Usually it's, "I'm trying to save you from making mistakes that I made and I'm trying to help you get to learn a little faster." I always appreciate it but knowing that I can say, "Thank you. Thank you for your time. Thank you for your perspective," but know that I don't have to go off [inaudible]. SAM: That's actually very similar to my tactic. Just not like, "Thank you. Thank you so much. Don't talk to me ever again." No. SARON: And disappear, yeah. SAM: I'm like the wind. With this pressure that either new developers or seasoned veterans are feeling about the mentor relationship, because I feel like a lot of senior developers that I've spoken with or people who've been in the business for a long time, feel like they should be mentoring or they need to take on that responsibility but there's always that hint of dread in their voice when they say about like, "Oh, I should be doing this." I always feel like it's okay to not do that. I never really understood why it was so high value to have this one-on-one relationship with an individual when you're not in school because it just feels so like... Not childish but childish. SARON: Yeah, that's one of things, frankly that I love about the tech community and also do not understand about the tech community. It's such a giving knowledge sharing community, whether you do it in a tactful way or not in a tactful way but the idea of giving back and paying it forward is just so deep. It's so, so deep that even when I've been coding for only a couple of months, I still felt this, I don't want to call pressure because pressure kind of sounds a little negative but I definitely felt this expectation that I was supposed to be blogging, supposed to be sharing and shouting and helping and doing these pay it forward type things. I don't really know where that comes from. Maybe that comes from the culture of open source, maybe that's where it kind of penetrate. I'm not sure but there's this huge need, desire, idea that we're supposed to be giving back and for that, I am very, very, very grateful. But I think that acknowledging that if you're someone who wants to be a mentor that you can do it simply by being available, literally being available, the going to be hashtag is super, super active even when we're not doing our Twitter chats and people use it to ask for help, they used it to ask questions. If you're feeling particularly giving or extra helpful that day, go on Twitter and check the hashtag and see what questions people are asking. Things like that or just super helpful and don't require a huge amount of time and effort. There's a lot of small ways to help out. That may not seem like a big deal to you but for the person who's asking for help, who has been banging their head against the wall, it's hugely valuable. SAM: Yeah, absolutely. Up until I went to my first conference this year, I didn't realize how supportive and important Twitter is. You know I always kind of considered it to be like another social media platform that I don't understand because I'm 84 years old and I just don't get it. It's been so uniquely helpful and in ways like Stack Overflow or even issues in GitHub, it just aren't. You can get so many more perspectives, so many different perspectives from people. SARON: Yeah, absolutely. Twitter has been amazing exactly for that. It's just an efficient way to crowdsource opinions and crowdsource perspectives and when you get your question answered and someone else answers it, it doesn't only benefit you the way it would if you email someone but it benefits anyone else who comes across that page, so yeah, it's hugely valuable. CHARLES: I remember the first moment I had kind of like that, a light bulb went off in my head where I was working on some really weird project that was using some strange wiki for its content storage and I was getting frustrated and I just tweeted about it and then the CTO of that company just immediately answered my question and I was like, "What?" You know, it was years ago and definitely, I was like, sound of explosion, that is where my mind exploded. I was not seeking help. I was just literally being kind of a jerk and venting frustration and lo and behold, the answer for my problem descended from the Twitter clouds. It was incredible. SARON: The Twitter clouds are the best clouds, usually CHARLES: Twitter is very... What's the word? It's very split down in the middle. SARON: Noodie? Yeah, there you go. SAM: There's this live feedback, so there's no buffer of emotion there, you know? SARON: Yeah. SAM: It's like, "Oh, this thing that you said, it made me mad. I'm going to tell you about it right now." Charles, I know that you had mentioned before that you think mentoring would be a good idea for Frontside and then, after all this discussion, have your views stayed the same? What are you feeling about that? CHARLES: As kind of the person who's like the grizzled veteran in the software world, it's definitely something that I've kind of whipped myself over the back. It's like feeling like it's something that we should do but I think it comes from the idea that people come here and we want to make sure that they're getting access to the learning that they need and the ways, in which they can level themselves up that they need, that's the kind of the prime motivator there. I always perceived mentorship as some vehicle through which to achieve that. It's something that I've heard. Obviously, we don't have a mentorship program at our company. It's something I felt that we should always be investigating. I've always felt maybe a little bit bad that we didn't have it but it's also something that I really struggled with in my career because I can't really say that I've ever had a mentor, so I don't really know what that relationship would look like but I do have a lot of people that I learned a lot of critical things from. I can look at it as kind of these seminal moments in my career, where like light bulbs went off and a lot of the time, they're associated with an individual and that individual and the thing that they taught me or multiple things that they taught me, are still with me. I have those experiences, which have been phenomenal and critical to my development as a software developer, so I guess it's just part and parcel of that impulse that you're describing to pay it forward, to realize that when you walked into the building, the lights were on and the walls were standing and the air was at a comfortable climate temperature. As you live there, you realize that there are people involved in actually, doing that maintenance and providing the building for you and the space for you to become aware of your world and then, when new people walk into the building, you want to provide them the same experience that you had. I guess that's my take on. That's the kind of thing that I would want to provide, so the question is like what does mentoring or mentoring 3.0, as we're maybe talking about in this conversation, how does that fit into that? How would you implement something like this, some sort of distributed learning in a company? I don't know. Maybe, it's not worthwhile. Maybe it is. SAM: I think it focuses more on that pair programming because when you're thinking back and you have all these people, like you have names of people that taught you something, it's multiple names. It's not just this one guy taught me all of these things. Actually, within a company, that mentoring just comes from pairing with your coworkers, seeing what different hats everybody wears and then, trying them on every now and again but not necessarily taking that individual's word as gospel, you know? CHARLES: Right and hopefully, that's not something that we've advocated for. Maybe, it's how do you introduce structure around that, to make sure that the proper ferment is happening, so that you have novel pairings and make sure the ideas that are flowing are flowing around the entire company and not just through certain set channels. SARON: Yeah. The other part of that is if you create structure around what it looks like to share outside. You know, pair programming is interesting because it's kind of one-on-one and it's hopefully, I have something important or bright to say in our pairing session. Maybe, I don't. Maybe I'm having a dull day. Maybe, I'm having a bad day but this really give you the same opportunity to take a moment and say, "What do I know? What do I want to share? What do I want to put together?" It does really give you a chance to prepare and gather yourself. It's kind of in the moment. I think having another opportunity where you can gather yourself is important and so, that might look like brown bag lunches, where everyone takes a turn and has to do a little lightning talk. That's usually the opportunity to say, "I have five minutes. I'm going to share something." Everyone has a turn, which means that the company's literally saying everyone has something to say. It's only five minutes, only a few minutes, so hopefully it won't be too terrifying if you're not big on public speaking and it's your co-worker so hopefully, it won't be terrifying because it's people you know and not total strangers. You know, a format like that, where there's structure but the company is saying, "We're going to give you time to think about what you're good at and what you know and to share that is good." I think another way to do that is by setting time aside for blogging. If your company can say, "Thirty minutes out of the week, we're going to take some time to write down five things I learned, to write down one cool thing I learned, post it publicly, post it internally about this expectation that everyone should be writing, everyone should be sharing, which also says everyone has something to share." I think those are two ideas and two ways that we can create a culture of sharing and a culture of distributed mentorship, where everyone has an opportunity to find the thing that they're excited about and specific ways to share it. SAM: Yeah, that's excellent. CHARLES: I hope you're grinning as much as I am, Sam because at least in the first half, you basically described our Lunch and Learn process. It's a little bit more than five minutes but I think another thing that clicked for me there is making sure that having a variation of expectations of quality or something like that because I feel like where we do really well is in the Lunch and Learn thing. We have a process very similar to what you describe but where we're not so good is maybe with blogging. I wonder if part of that is we just hold ourselves to such a high standard of what it is. The idea that we could throw together a blog post in 30 minutes, I love that idea but it means you really have to be willing to just say, "You know what? We're going to get it out there and we're going to make it bite-sized and the expectation is that we're not going to be writing some gigantic essay that's going to shake the industry to its core every Friday at three o'clock." There are things and if we can, I shouldn't say a reduction in quality but maybe a reduction in scope so that you can say like, "We're going to carve out 30 minutes or an hour and we're going to pick a topic that scope-appropriately for that." SARON: Absolutely and I think that knowing that you only have 30 minutes or maybe it's an hour -- an hour is probably a little bit more realistic -- but knowing that you only have an hour also forces scope. You can't write a book about JavaScript in an hour. You just can't, so the fact that by the end of that hour, you have to have something to turn in is a great way to force people to focus and do something really small and really bite-sized. The other thing is I don't think that after the hour, you need to publish it publicly. It could be, you need to turn it into someone else, to edit and look over for you and give you feedback. It's not quite ready yet but it's a solid first draft and the ideas that after that first person edits it, then it's on its way to being published. I think there's different ways to manage a scope without also making this scary thing where I have to say something for the Earth to shake. There's different things that we can do around that. CHARLES: I'm wondering what other forms of sharing that we can fit into our workday. I guess, the other baseline is just making sure that you're always... What is it? ABT -- always be tweeting. It's so easy to be write-only, not even engaging conversations but just throwing ideas out into the void. SAM: I think holding a discussion with your coworkers like if you're in an environment where you can work face to face or are constantly online, if you're more of a remote worker, I think just having a conversation with anybody, rather than just putting your idea out there, putting it out there with someone who can actually provide real time feedback in a more friendly way, then I think some people on Twitter could be. Because they're your coworkers and they're not going to call you rude names, you know? CHARLES: Right and you're also going to know, hopefully the whole trust and intent and 'are we on the same page?' that question is answered even before the text is written. SAM: Right. SARON: Yeah, absolutely and with those conversations, this actually mean [inaudible] that we do. We do like a show and tell every week and the idea is that we're both always learning random things, usually related things but sometimes, totally random but still very interesting and it will take 30 minutes to just share what we've learned. Sometimes, they'll also turn it into a blog post. Sometimes, it's just a knowledge sharing opportunity. I'm not really sure but it's a good window of opportunity to say, "We are learning and we're sharing and we have something of value to bring." The great thing about something like a show and tell is that it doesn't necessarily have to come from my brain. It doesn't have to be like, "I have a great idea that I'm going to share." It could be, "I read about this cool idea." It can be, "I heard about this cool thing that we can try and we can apply," but I still kind of credit, you know? Like I get credit for being the value bringer but the burden isn't on me every week to come up with the idea. That's a nice balance, to kind of create space to share and to promote this knowledge share and if it comes from you, it's great but if not, you're still helping other people. CHARLES: I have a question that may or may not be related in here. This has just kind of occurred to me because this is definitely something that I experienced, where I get into a mode where I become overwhelmed by the ideas that people are sharing and so, what's the balance? Because usually, we're out there searching for ideas and we're searching for novel things so that we can include them in our work, in the things that we want to do and accomplish, whether be that in tech or elsewhere. What's the balance of being heads down and being like, "You know what? I'm going to be closed to new ideas right now." Because they can be distracting, right? The [inaudible], that's actually a phenomenon and so, how do you protect yourself from sharing? What's the balance? SARON: My solution was to move to San Diego. That was my solution to that. I actually moved from New Jersey and I worked in New York City... How long has it been? Was it only been a year? Oh, my goodness, and now, I'm in San Diego and it was so interesting because that ended up being a really nice side effect. I didn't move specifically for that reason but when you are commuting in New York City every single day, there's so much going on all the time. There's just so many ideas and events and meet ups and companies and people. It's just so much. I think that it was a great place to be in my early 20s when I really just wanted to soak up everyone else's ideas and I didn't really have opinions of my own at that point and it's a great way to just kind of absorb and be this awesome sponge in the big city but after a while, I kind of realized, maybe I have my own ideas and my own thoughts, so moving to San Diego, which is a much, much, much quieter place, has been a really great way to reflect and sit with my own thoughts and feelings and opinions and just kind of focus on that. I understand that everyone can move to San Diego, although I highly recommend it but I think in that way of carving out like... What do they call it? Do they call it a tech sabbath? A digital sabbath? Am I saying that right? CHARLES: Yeah. Maybe. It sounds about right. SARON: Yeah. I came across that term recently but this idea of one day out of the week, I'm not going to be on the interweb. I'll just not going to do it. Maybe, even the whole weekend, oh, my goodness and saying, "I'm just going to stay away from things. I'm just going to create a little space for me to think and reflect." I think when I don't have the whole day and what I need is just like a moment, I find that writing things down is a great way -- a really, really good way of doing it -- even if I'm taking notes or from doing a strategy session or if I'm trying to make a decision. Usually, I'll start typing and what I've started to do recently is to say, "I'm not going to type. I'm going to plot in a notebook and I'm just going to write things down," and because writing is slower than typing, it forces you to just think. It forces you to be alone with your thoughts, for better or worse and it forces you to really just to think about what you're doing or what you're saying and reflect. It's a really, really great meditative exercise that I found. You know, finding little ways to build in an escape from the noise, ideally on a regular basis, I think is a really healthy thing to do. SAM: Yeah, I definitely agree with that. My normal method of sort of closing everybody off is I just sketch out ideas because I'm an artist. I come from that sort of perspective as I can make a drawing out of feelings more so than I can write them out. If I'm feeling really overwhelmed or if I'm trying to make sense of some information that people have given me, I just sketch it out and I think that's something that doesn't really get talked about a lot in the whole community because sometimes it's always about eat, breathe, live, code, you know? We have different skills. You have other things that you're good at that's not just code. When I was at React Rally, a lot of people were in the music and then finding a way to separate yourself from the entirety of it is definitely one of the better ways to make sense of your situation. SARON: Yup, absolutely. CHARLES: Yeah. Sleep? It's a great way to take a break. SARON: Sleep is so good. SAM: On my top three things: sleep. CHARLES: Yeah. SARON: Yeah. As a kid, I never ever thought I would look forward to my bedtime ever. You know, it's my favorite part of the day. CHARLES: Yeah but like sleep, writing, drawing, I always forget. Like when I'm caught up in a problem, I always forget that engaging in some other activity -- I like to walk in the place next to my house. I like to play my ukulele. Engaging in those activities is almost on the critical path to solving the problem and I always forget it. I think we always forget it but sometimes, you can be so frustrated and you can just shut it off, be completely alone and there is some magical process that's probably going on in your brain. I don't know exactly what it is but you come back and the answer is just sitting there, waiting practically on your desk. SARON: Absolutely. One thing, we recently moved a couple of weeks ago to a new place and it's a little bit bigger and so, I had the opportunity to unpack boxes and buy furniture that we didn't really need before or just trying to make it more of a home. I've been using it as a really great opportunity to be productive and to feel productive but not be in front of a screen. Sometimes, I even like save tasks for myself like, "I'm going to wait till the evening to put together this shelf because I'm going to need a moment to just be away from my computer." I have a plan around it so it ended up being such that, I think about every day, there's one little home activity thing that I can do, whether it's cleaning or cooking or assembly or movie or something with the home, that allows me to, because I'm not really a hobby person. I don't really do hobbies because it just feels, like no judgment on people who do. I know people get really into their hobbies but it just feels like, "Why?" You know, like, "Why?" like, "For what?" But when I put a shelf together, it's like, "I'm going to use this for books." You know what I mean? Like you're very purposeful, so home activities has been my way of carving out that space away from the screen but also feeling like I'm doing something productive, I'm getting things done. SAM: Something that I would recommend to anybody who doesn't really have that, I'm going to step away from the computer and do this thing. Something that's kind of helping me was the Pomodoro Technique, which if you're not familiar with that, it's basically a time management thing where you set your timer for work like for 25 minutes, 30 minutes or whatever and then, when the timer goes off, you take an allotted break for five minutes, 10 minutes, just so you're not doing the thing that you've been doing for the last 30 minutes. SARON: Yes, absolutely. That's a great one. SAM: My advice is to implement that if you don't have a thing, that you use for your cleaner, I guess. SARON: Oh, you want to hear some really terrible but effective advice? SAM: Yes. SARON: And this is what I found out very accidentally is if you have a really, really crappy office chair that hurts -- CHARLES: Oh, man. I'm sitting in one right now. It's literally a rocking chair from the 1800s. SARON: That sounds awesome. CHARLES: Yeah, it does sounds awesome. SARON: But if you have a crappy office chair, that can be a really great way to get away from your screen because after about an hour, your thighs will hurt and your back will hurt and you will be forced to get up and walk around. It's so funny because I have a pretty crappy office chair and my back has been hurting for so long and I just thought, "That's life." That's like my [inaudible] for myself. I'm like, "That's just how life is, my backs hurt," and then it got to a point where I was like, "I need to go and talk to someone," and as soon after that, I spoke to a conference and I was basically up and down away from any type of chair for like four or five days straight and magically, my back pain went away and I was, "Oh, my God. It's that freaking chair," and ever since I realized it, I started noticing it. I would sit down and I would go, "I'm nearing the one- Oh, there are my thighs. There they go. They're in pain." There's another great cheap life hack: get a crappy chair, after about an hour, everything will hurt. You will be forced to go do some jumping jacks for about five, 10 minutes and there's your additional sabbath. There you go. SAM: In Charles' case, probably a haunted office chair. Yeah, that's an excellent advice. I never really noticed that. Now, I'm going to. CHARLES: You know what funny is, I actually didn't even notice it but I do. I never used to get up and go on walks before or spend as much of the day standing as I do and I'm actually completely and totally oblivious to it but I think, I realize like I can attribute a lot of that to the terribly, uncomfortable chair, in which I sit on every day. SARON: I have this awesome habit of sitting cross-legged, which apparently is a very, very bad idea, my physical therapist told me, so I make it worse for myself. If you don't feel like you have enough screen time, just sit cross-legged and that will accelerate the pain process. SAM: As you said that, I am currently sitting cross-legged in my desk chair. SARON: Yeah! SAM: That's just how I sit in chairs. My legs are too short to reach the floor. SARON: Me too. I just find it so comfortable. I don't know and I'm so happy to hear that you also do this because I'm like, "Am I just weird?" because I love sitting cross-legged. It's so comfortable. It makes me really happy. SAM: No, I verified 100%, that's how I sit in all chairs. SARON: Yeah, the same. CHARLES: I can do one leg. SARON: Only one? Man, you need to work on that. SAM: -- [inaudible]. That's going to be the end of our podcast. Again, thank you Saron for coming on and having this awesome conversation about mentoring. Definitely check out the website CodeNewbie.org. We are the Frontside. We build software that you can stick a future on. Hit us up if you're looking for help building your next big thing and also, hit us up on Twitter. Hit Charles, myself, Frontside and Saron on Twitter. SARON: Thank you so much for having me. This is awesome. This is fun. SAM: Also, I want to give another thanks to Mandy, our producer for producing this lovely episode and all of our episodes. If you have any questions for us, hit us up on Twitter. Any ideas for future topics, any thoughts you have on this great conversation, let us know and we'll talk to you all later.

In the last episode, we spoke a lot about the "why?" behind microstates. This time we wanted to cover how ideas in Microstates map to different patterns used to build JavaScript applications using frameworks like React, Ember, Vue and Angular. We discussed what you need to know about Microstates if you prefer component state architecture or global store architecture like Redux, as well as how setState in React can be refactored to use Microstates. We closed off with the comments about the trade offs that component heavy frameworks make by overemphasizing the view layer at expense of other aspects of the MVC pattern. This show was produced by Mandy Moore, aka @therubyrep of DevReps, LLC. Upcoming Conference Talks: Manhattan.js - August 8th (Taras) ReactJS Austin August 6th (Charles) Transcript: CHARLES: Hello everybody and welcome to The Frontside Podcast, Episode #107. My name is Charles Lowell and we are going to be following up with our Episode 106 with the exciting conclusion of 'Microstates, the Podcast.' With me today to wrap this subject up, at least for the near term, obviously we're going to be talking about it a lot in the days and months to come, are David and Taras, also co-developers here at Frontside. Hello guys. TARAS: Hello, hello. DAVID: Hi everyone. CHARLES: Before we get into that, we'll just make a few quick announcements. First off is here at Frontside, we are going to be having some availability at the end of August. If you or anyone on your team is looking to level up in the area of testing, especially testing single page applications, acceptance testing single page applications or if you need React or general JavaScript consulting, please get in touch. We would love to work with you and love to do some of the great things together. Second, we finally released this and this is germane to the topic at hand. We released a major version of microstates this week that's based on a new and simpler architecture. That's really exciting. It doesn't really change the content of the conversation because the API hasn't changed that much. It just means that the library, which was already very small is even smaller. I think we shaved almost 40% of the size off and it's just much simpler and it's much more harmonious with the underlying JavaScript runtime. It's even more just simple JavaScript objects. Unless there's any other news that you want to cover, we'll jump right into it. TARAS: All right. Let's do it then. Let's do microstates. CHARLES: I Love to talk about microstates. This is obviously the second podcast that we're doing on microstates, just because we ended up, I think it was two weeks ago and we'd been speaking for almost an hour and really, we're just laying out the problems that microstates solve -- the problems of state management and why you often run up against complexity when you have a single state management tool that doesn't account for a bunch of different use cases. We got into microstates a little bit but we left it as kind of a cliffhanger. We talked about transactionality and laziness and immutability, ease of composition, simplicity of API, performance, memory footprint, things like this. Those were all the problems and then we're like, "Yeah, microstates. It's awesome." We're going to talk a little bit more about actual microstates proper and what is involved like the adjustments in mindset that you need to make or don't need to make when adopting a tool like microstates. TARAS: Actually, it's been really interesting for me because I just gave a talk at Toronto JS on microstates and it was pretty cool to see. There was a panel at the end. I think the people that are representing just using component state as a way of architect implication. It was managing state and application and it's representing redux. It's really interesting to see, first of all, like how curious people were. Most of questions were directed to the discussion around microstate, simply because it is a new tool but it was also just interesting to see how curious people who really loves redux, he was like, "I really love redux but I really like that fact that you guys have types," so I was like, "Oh, that's cool." Even though you really liked your solution, you actually found something in microstates interesting but at the same time, I think he was kind of missing that what aspects of Microstates overlap with redux. I think one of the things that we can do today is talk about what microstates has and what API does microstates provides and how similar they are to what people already know. I think that's one of the things that I was thinking about this conversation is that there are some things around microstates API like how to use microstates but the architectural concepts that power a microstates, they are already part of the development process and development architectures for most people that building single page applications. I think we can do is try to map these ideas not what people really know and patterns that they use to ideas in microstates, just to show how close we actually are. I think that's a good way for us to go. CHARLES: I guess we can start with, maybe one particular mode of managing state and then, how that maps to uses with microstates. TARAS: Let's start with components state because that was kind of our starting point. One of the original starting points for microstates because so much of our work was in Ember and component state is where most of Ember development happens. I think that's a good place to start. If you're someone who uses components state, if you're using Ember or Vue without something that is redux-ish which is pretty common these days, if you use components state, then one of the features of -- CHARLES: And also, if you're using React, right? Ember, Vue and React pretty much are all the same in this regard. TARAS: Yeah. Well, Ember and Vue are a little bit different in that behalf. Their particular pattern is you have data coming into the component, then you're doing some work with the data so that in Ember and Vue, you might use computed properties to derive state on that and then you might pass that state further down into other components. In React, you would do something a little bit different because there is no memorized computed properties by default, so you, a lot of times, write your computations that you pass into other computers that are written as expressions in JSX. If you're using this pattern, then one of the challenges around using this pattern is the process of lifting state becomes quite complicated. Because if you have like a bunch of state that lives in the component, it's attached to your derivation of state and the properties that you can invoke are attached to a component object. In React world, it's called lifting state. In Ember world, it'd be essential refactoring it into the parent component and passing that state in. in microstates, you're essentially moving this state into the microstate and now, what you have is this object that instead of having the state live on the component, the state now lives on the microstate object and in all frameworks, when you do that in microstates, you can use the getters to get the memorized computed properties behavior on the microstate. If you use your computed properties, computed properties are available for you on the microstate. You can do it the same way. CHARLES: It's kind of like a wrap up of that idea, not a wrap up but a summary of that idea is that the microstate essentially is a substitute for your component properties. If you're working with a component, a component has state associated with it, so you're setting properties on a component and then using the computed properties of that component. Microstates is you have all those benefits where you can set properties directly. You can define methods that like set properties as part of a transaction and you can then have simple JavaScript getters, which are just like computed properties except there's no enumeration of dependent keys. It's just, that's all they do for you. TARAS: Then the next question that arises is like, "Do I make like one huge microstate or a bunch of smaller microstates?" I think the question is really depends on the role of your component and what's nice thing about microstate is in microstates, there's two things that are kind of cool. One is when you represent your status in microstate, it makes it very easy. If you need to lift that state from the component up into the parent, you can lift that type. You can take that type and you can compose that type into the parent's microstate, if you have to and in that case, you would pass this new object that's created from that type. You'd pass the state for that component to the component that would otherwise have its own state. Essentially now, the parent has the state that would otherwise be in a child component and now, the parent is passing a state from the parent to the child. There's two kind of benefits to that. One is it gives a parent a way to control the state of the child and it also gives you a really easy way to serialize and deserialize state. You start off with having ability to serialize state for a particular component and all of his children, now you can represent that serialized state as part of the components state, so you have an easy way to restore the state of the component tree at the parent component. CHARLES: Right and that's possible because a microstate is really just encapsulating a value within a type but the value is just a simple serializable plain old JavaScript object, right? TARAS: Yeah. CHARLES: I'm just not trying to say like this is the feature that microstate provide, so when you're creating a microstate, you just pass it a POJO and it off to the races with that POJO and you can access that POJO at any single point. TARAS: Yeah. This is part of the architecture that I think are helpful for people that are using components. Is there anything else for people who are using components state that we think would be helpful? CHARLES: I think we're just realizing that you get the benefit of the laziness and immutability and the transactionality and all kind of things that we talked about last time but actually, the mental model is very similar. I think the thing to realize is that if that's the way that you used to working with state, the bridge is not too far. It's actually quite a short one. TARAS: Yeah, that's good. CHARLES: It feels very natural. It's not a huge mental shift. It's just more about a very small mental shift, a very small shift in mechanics for a very large payoff. DAVID: For instance, say you're in an Ember application using component state, what I'm used to is you're going to be passing actions around between components. Whenever you're trading that out for microstates, is it more that these actions are just bundle in with the microstate as the transition? TARAS: Yeah. One nice thing with the microstate, because the transitions that you can perform on any data type are intrinsic to the actual data type, the part of the data type, when you instantiate a microstate, you get these transitions for free. When you pass the object into a component, you can now invoke transitions on that object, that are part of that object's type. It's hard to imagine how awesome that really is because the closest pattern that you would see for that would be like, for example, if you're using Ember, you have Ember object or Ember model that you pass into a component and then, you can work a transition and because it goes through the Ember data store, your component is going to update. With microstates, there's no observation of any kind but what you're doing is when you are invoking the transition, transitions going up to the top of the root and gives you the next microstate, which updates your component tree. You get that functionality of 'data-down, actions-up'' built into the entire system of the microstates and all of that is hooked into these transitions that you can invoke on the objects that you pass into your components. CHARLES: Right, maybe David and Taras, you all could provide a concrete example of what that looks like. Because I think it is something that when I was giving talks on microstates at Ember meetups, one of the first things I like to show is you've got all these actions that you're writing either on your router or your controller but they're really actions manipulating the same type of data. It really comes down to like you're pushing and popping things off of arrays, you're toggling Booleans, you're incrementing counters, you're setting this property on this object. These are actions that we're writing and in the kind of microstates world, that's boilerplate. Because the transitions are intrinsic to the data type of the microstate that you're working with, maybe we could provide an example, like what's an action that you describe that you would pass around. TARAS: I think an easier one would be like if you have a model for example, you're composing a model into your application state or you're compose a model into your components state, so you have this model type that gets instantiated and you pass it into component that represents a model. Off that model, you might have open status like is it open or not that gets consumed by the model component, to know whether the model should be visible. Then along with that is model is open state, there is a toggle transition that you can find inside your component that is going to automatically flip the visibility of open. What will happen when you bind that transition to some kind of action handler that you invoke inside your component, when you invoke that action, it will then trigger transition at the top of that microstates and then, it will create the next state, push it through which will cause your model to change the visibility. David, that does answer the question? DAVID: Yeah, it does and that's actually the example that I have come up with whenever asked. A very simple sort of Boolean toggle. CHARLES: In a couple of my earlier demos, I should dust off that talk that I gave to the Austin Ember meetup where I was able to create an input with a dropdown menu with basically, some pretty advanced mouse behavior, all without having any component state or like storing it all in microstates and a lot of pushback was, "Aren't you putting logic in the template?" and the answer is, "Absolutely not." The logic is in the models but what I'm doing is I'm composing the actions that operate on those models inside the templates but at the template level, the action is data. If you're thinking about, we always want to have, we always want to lift state and then push that state down to the application, what this is really saying is that the actions are part of the data. It is implicit to the value that you've got. TARAS: I think that's really powerful because we do think about actions as being something that you can invoke. Like with closures, it's an action that you can invoke but considering that operation, that piece of data that invokes a transition in microstates and is derived from the type of the data, I think it's a cool concept. That's probably, one of the things that is kind of new for microstates but how you use it in your application should feel pretty much the same as you would if you were creating action handlers on your components. CHARLES: Yeah, just like kind of bundled action handlers for free. TARAS: Right. CHARLES: Maybe the way that you wrap up on this one when talking about kind of what microstates has to offer for the Ember developer, the Vue developer is really, I would say someone who's used to working with like MobX. Would be another good example is when I first started using Ember back in 2012, Ember Object solved a whole lot of issues in a really profound fundamental way and I really, really was drawn to that as the best API to be working with state. What kind of became apparent was perhaps not the best implementation. This is not like bag on Ember Object. I think it was actually amazing technology for seven years ago when most of it was written and probably, even came from before like SproutCore. It's almost 10 years old, which is incredible but it's still under heavy use in 2018 and it speaks well of how well it was constructed. I actually don't have much of a problem with Ember Object API. I just think that the way in which it's implemented that API means that there are some problems that require a different paradigm, so very much I kind of see microstates as heavily inspired by those types of APIs but with, I want to say a modern implementation but an implementation that is designed to solve the problems that have come to light over the last five years of developing single page JavaScript application. TARAS: I want to add that, when you describe with Ember Object, it exists in every framework that uses immutability. In MobX, the observation introduces the necessity to wait for a bunch of things to resolve. In Angular zones, I saw a similar purpose to ensure that if there are kind of cascading or streams or all of that stuff gets settled in into kind of restful state before you can start to assert on what's going on and with Vue, their computed properties has something similar. I think because of the complexity of having to track lots of objects and then recomputing things based on the result, that complexity in microstates is simplified by the fact that you describe you transitions. When you write them, you describe exactly what changes, so we don't need to wait for things to settle down. When you invoke a transition, that transition is explicit and based on the path of where that transition is invoked, we know exactly what needs to change, so we need to only perform one operation to compute the next state. There is no other things that we need to wait. There's no other side effects that we need to resolve before we know what the final status. I think that simplicity can be applied to all the frameworks that are using immutable APIs and derived computations from those immutable APIs. CHARLES: Uhm-mm [affirmative]. TARAS: Should we jump into talking about what microstates has to offer for people familiar with redux? CHARLES: Yeah, okay. Let's jump right into it. For folks who are familiar with immutable APIs like Ember, like Vue, like other ecosystem using MobX, a shift to microstates might feel like what you can expect. What about people who are just using often React-land and they're just using like set state. TARAS: Set state is a little bit tricky because it does get complicated over time and then you get this kind of funky things going on after a while where you start seeing things like, "I'm going to set state on this component and then once the change happens, I'm going to change some states some other state," so this kind of cascading state changes. The other part that I find particularly more challenging in set state world than it is in, I think in regular components state, like what you have with Vue and Ember. I feel like the way that the transitions, the state change handlers become part of the component, I find that part particularly kind of fragile. When you start doing refactoring, when you need to lift up state, it's like -- CHARLES: What's an example of this? TARAS: When you start off and your component owned old state and now, you need to have that state being controlled by the parent, we get into a situation where like, "What am I going to do? Am I going to do something with props as they're coming into the component or I should probably just flip that state from the child to the parent?" and now, you're refactoring the internals of a component to lift up that state so you can then combined those operation with the parent's state transitions. But then, you have this kind of added complexity of the fact that you're working with immutable data in that place. You've got these three things going on: you're refactoring your child component, you are moving these things into the parent components, you modify a parent component and now, you're also managing the complexity of forming immutable objects. I think the fact that people make it work is kind of a testament to human resilience. The people are able to solve such challenging problems and this is not a super hard one but when a component is complex enough and when stakes are high enough, these changes can become fragile. This is what I think microstates simplifies is that by taking care of the model, it makes components much simpler and makes it possible for you to just render a lot of components that are functional components without their own state. CHARLES: It actually reminds me and like I said, we're talking about what it's like to use them, not so much the implementation but that's the exact same sentiment that the author expressed in his book that was very helpful in writing microstates, which is Brian Marick. He has this book called 'Lenses for Mere Mortals' and he's talking about the practical use cases of using lenses, which microstates leans very heavily on. What he says right in the introduction is it gives you by abstracting the location and compose it the way in which your data structures are composed, it makes them very refactorable. You can say, "I want to change where the state lives. I want to change the structure of this object and I want to move it somewhere else," and it's sounds weird to say this, because the structure of the object is not coupled to the structure of the object, it means that it's very malleable, so you can move and you can say, "You know what? I don't want this address to be embedded in this person. I want this address to be inside this address book and then my person has an address book entry." Being able to make those refactors make those changes is very easy because your method of accessing the data is abstracted. It's something that applies beyond even user interface as a component state but it's something that this is a problem that's very salient when you are authoring complex UIs and compose components. It's something that you can benefit very greatly from. DAVID: Taras, you were telling us earlier about a friend of yours who is learning React for the first time over the past few months and he's mainly been using set state as you would just starting out in this world. You said that he didn't really get why you might have to go and learn some other way to manage your state and I think Charles said it, whenever your application starts to get more complex and you've got a lot of different moving parts, that's really where microstates will come out and shine for you. TARAS: Yeah. My friend kind of feel bad because he's spent the last three months learning how to work with React and how to use set state, "And now, you're telling me, I need to learn something else? Like I'm going to start from scratch?" I kind of reassured him that especially when you're a beginner, when you're learning, you learn how to address very specific challenges but you don't know how complicated these things can get when your UI gets really intricate. In those intricate scenarios is when you have to leverage your experience and you'll be able to solve these problems but if you're learning, you encounter these challenges head on and your tool does not really, like set state. Although it can be used to build really complex UIs very effectively but the complexity that over time increases as you start to manage the state, increases much more later on than does early in the beginning. Basically, with microstates, even if you have some basic proficiency with set state, when you start working with microstate, the things that you can do with microstates, you will be able to do more sophisticated things easier when you need to use them than you might realize when you start. Because microstates API is so consistent, there are a very few concepts to actually learn and they cover a broader range of use cases so when you actually starts using it, you'll encounter these challenges. What you already know it will just work for you and always just continue to work for you. That was part of kind of fundamental design on what Charles and I have spent, a lot of time putting into place from a point when we started two years ago. CHARLES: I think, and you touched on this and this is kind of what I was speaking to earlier is that when you're working with a simple system, it's simple, it's easy to work with and then the complexity starts to grow, it's never a good thing when you have to reach for a more complex tool to manage the complexity. It's a hallmark of a good system and that it can scale with you from a very simple and easy use cases to actually being able to handle very large and complex use cases but your API doesn't have to change and your usage doesn't have to change. If you have a tool that can actually scale with you from a one liner to a hundred thousand liner, that reflects very well on the design of the tool. I think you see that with things like Ruby, you see it with things like Clojure. You see it with, I would say there's even people writing like Haskell, shell scripts now but not so much with like C++. I think the analogy is very similar with microstates in the sense that when I'm looking for evaluating a language, it's not the only thing I look for but I really am looking like how is this going to work for me as a one liner? How's it going to work for me as a hundred thousand liner? If the answer is pretty consistently, then that's something that is going to get a lot of bang for your buck, so to speak. If I invest the time in learning it, I'm actually able to reap the reward of having a tool that's got my back on a whole bunch of different use cases. We talked about, in React, if I'm using set state and that's kind of a sub case of component state because I would say that in the previous systems we talked about -- Ember, Vue -- they have components state but the component state is a little bit more rich in the sense that you've got computed properties and what have you. But if we look at a system that externalize a state like redux, where you have a global state atom in your application, at least that's the way most of the redux applications that I've encountered behave, what does it look like for you? TARAS: I think it's a little bit challenging when talking about redux is redux conflate a few different things together. I think it's helpful to split those things up, so we can talk about them separately. One of the parts is how the state is delivered to components. The way that redux does is the instance is this created so every time you use connect, you essentially wired together. You can connect through the context or... I'm not sure. I think they have an observation mechanism that's internal to redux as well but they essentially connect components to the store and then they view that to deliver the state. It's kind of worth pointing out that for that like microstates bindings for React actually give you something similar out of the box through context. For those who really like the ergonomics of connect, I think it would be pretty easy to make that available for microstates. Usually, we don't even know why they wouldn't be available. CHARLES: But I do think that connect can be problematic like you can encourage you to not make components that are reusable and have isolated state. It's very easy to hitch yourself to the redux store and now, you don't have components that are shareable. TARAS: I think I would personally prefer to pass microstates instances through props because of the stability that's built into microstates and structural sharing, you can get some free optimizations and allow it to use functional components in more cases like out of the box but some people who are really like redux, they really like connect. Although it might be my personal preference, there's no reason why that wouldn't work if somebody wants me to connect function and make it available for people. CHARLES: Right and I think, there's a happy medium there too, right? TARAS: Yeah, I think -- CHARLES: -- You can connect components and then fan out that state to a set of functional components that are not connected. TARAS: There are some places where microstates and redux are very similar. When you're using in redux, you have this dispatch mechanism, where you essential saying, "I'm going to dispatch this action and your action creator is going to create an action for you with the payload," and then you're going to match that action to a reducer. One of the things that I kind of hear redux people really enjoy about redux is the one with data flow that dispatch the action and then, it reduces this next state and pushes that through and you have this kind of ingress point where everything is going through this one point. I think what's really interesting with microstates is you essentially get that. That's exactly how microstates works, in a way. The only difference is that the API is different. Any action that you invoke is going to go through a single entry point, which is going for you. Because we know the structure of the data, we know how to transition that state for you, so you don't need to reducers, so you're just defining your transitions or use the built-in transitions and then when you invoke them, we know the path. We're essentially forming for you. The path where you invoke the transition, conceptually, it kind of forms the name of the action that you are invoking. The path refers to a place where the state is going to be transitioned, then you have your transition, which is the actual reducer for that part but it's contextualize, so you don't have to think about how you need to transition that state in a great, global redux state. CHARLES: There's no matching. The matching is automatic. TARAS: Yeah, the matching is automatic, so you get that same single ingress and one directional dataflow. You get those mechanics except the APIs that you use, instead of writing the actions yourself. Instead of writing, we just use yourself. You get to use microstate types. I've heard some people who use redux who are like, "I really love the fact that microstates has types," but other people don't like types for whatever reason. Microstates comes with 'from,' which allows you to take a POJO and from that POJO, it creates a microstate and then you can invoke transitions the same way as if you had a type microstate. The only thing you don't get is you don't get to create your own transitions. You have to use the transitions that are provided for the primitive types. CHARLES: I think that there's a couple of benefits that you'll realize for free. There's laziness, reducers by default, or eager. When you dispatch an action, it will run against every reducer in the store. If the reducer matches, you're going to run the computation that's associated with that reducer. Microstates by contrast is lazy. Basically, until you try and read the property that is affected by that reducer, the reducer won't run. There's some ways that you can get around this. When you're using redux and first of all, you can actually have your reducers return objects that have getters on them. You can realize some of that laziness but again, it's work that you have to explicitly put in. I think, didn't you actually say that there is a package of plugins? There are plugins. There's basically a set of libraries that you could bundle together, which would give us an experience similar to using microstates. TARAS: Yeah. If you wanted to combine redux and reselect and immutable JS together, you can get some of the benefits, except this benefits are not integrated that well because they're still separate systems that you are essentially using together. Also, like microstates, it's four times smaller than redux and reselect and immutable JS combined together. If size matters to you and ergonomics matter to you, you actually can get a lot of ease out of using microstates while still maintaining the benefits of having redux. CHARLES: But if those things, those packages, like reselect and immutable JS, are things that are familiar and you naturally gravitate for it, then you'll probably absolutely love microstates. Because honestly, one of the ways that I think about microstates is like, what if you could have immutable JS, if there was no cost for composing the types like list and record. Immutable JS has come a long way since I've last used it or it's evolved since I've last used it but I think there's still only about four or five basic types and actually, making your own new immutable structure, your own custom type with its own custom methods that still get the benefits of structural sharing and laziness that you have on immutable JS is not something that you can do. But you could think one way to think about microstates is an immutable JS where you can make any type that you want. You're not just constrained to the record types and to their list types and set types or map types. TARAS: It's worth pointing out that at the moment, microstates doesn't map perfectly to immutable JS simply because immutable JS has certain optimizations for managing lists that kind of a great value of immutable JS and microstates doesn't have some of those pieces but -- CHARLES: They're definitely on the road map. TARAS: Yeah. Because microstates is abstracted high enough, that we can actually change internals and some people who are using microstates now, they will get benefits of ergonomics and there's already performance benefits from the stability that microstates offers but there will be a time when by upgrading to newer versions, you will not basically, need to change anything in your op but you will get the benefit of improvements to performance that we will introduce over time. Some of those improvements might come from what we will learn from immutable JS. CHARLES: Right. I have a couple of thoughts before we wrap up. What are the ramifications no matter who you are? What kind of development background, some of the benefits that you'll experience with microstates that might be a pain point or something you hadn't thought about where you are currently? A couple of things that I have jotted down is first, and this is what brought it to mind is talking about stability. Something that you see in a bunch of frameworks is having to manually track the keys that are associated with data. If I've got a list or I've got an object, being able to say, if that object changes, then I need to actually have some sort of key object, which effectively amounts to a hashing function to say, "Did it really change?" Because no matter what system you're in, you need to know how you're going to re-render. If the reference to this object changes, then the default thing needs to be, "I need to re-render it," right? You see this in Ember, in Vue. In React, there's this ability to pass a key or ask the question, "Should this component actually update?" and with microstates, that's much less of an issue because basically, it keeps the key tracking instability for you. If you are using a model with microstates, if you think of an object as a graph of nodes, no node in the graph will change unless it absolutely has to, at least that's the goal. We still actually have some work to do when you're running queries against the state of a microstate. We can cover that later but that's most largely the way it is now and definitely, the way it's going to be going forward is you don't have to do any extra work as a programmer to figure out what has actually changed. TARAS: That opens up some interesting opportunities. Imagine if you had a rendering engine that did not expect side effects to be significant and you could just say, "If I know whence they'd changes, I will then re-render that." That will be really interesting exercise, seeing like what would that look like for a performance perspective, if you have a very clear picture of what has actually changed and what part of the DOM as a result, need to be updated without having to do diffing. The [inaudible] you could actually do a diffing at much higher up. Actually, [inaudible] to diffing because you know what's changed but you can push a lot of assumptions higher up on the architecture stack. CHARLES: Right. That's actually one of my favorite thing about microstates and one of the unwritten values is like triple equals used to work everywhere and by and large, it does. It's usually simplifying but when you don't have to manually tell the computer what equivalence looks like, you can just say, "Look, are they the same object? They're not the same. If not, then they're not," and keeping that consistent is huge. TARAS: I think this is a good segue for us to kind of bring this conversation to a close and also, kind of set up potentially a third full-on conversation, which we could talk about actual architecture of microstates and design decisions because for people who just want to use microstates, they don't need to know all these details but for people who are curious, they might actually want to understand what are the considerations that remain when we were designing microstates, so maybe in a next conversation about microstates, it could be about architecture and the pieces in microstates today and then where we are going with microstates and what it could give us long term. CHARLES: Yeah, I like that idea. It is a plannable subject that we've been talking about internally for the past two years, so it makes sense that there would be plenty to speak about on the podcast. There is one other thing that I did want to bring up and that is, I think enabling to have a state solution that is composable because it allows you to think about your state first. Because really, if you do have a functional UI, where your view is a pure function of the model, that your view follows the model and so, if the view follows the model, then really, the thing that you should be thinking about first is the model that's going to be required to drive your view. I shouldn't drive. I should say derive your view because that's the primary artifact of which the view is nothing more than a function. It's a reflection onto a surface. I don't think that we have a state management solution yet, that enables that mode of thought, where I'm thinking about my prime artifact first and working forward rather than thinking about my secondary artifact and trying to kind of wishy-washy way, work backwards and reconstruct the primary artifact. I think that we've talked about all the development ergonomics and I think there's a mechanic of thought there that's enabled by this that I hope to see in more and more applications. TARAS: I think that's a really well put. I think that's something that I've been thinking about as well, as how do you convey this shift that microstates allows in terms of how we're thinking about architects and the application. For some people that value, the model, like they'll find that shift easier but regardless, I think that making that shift has a potential of simplifying your view dramatically and I'm very excited about exploring this further and having more conversations about this. CHARLES: Yeah, that's where we really kind of open up the conversation about state machines, which is also central to the conceit of microstates and using state machines as an incredible design tool but anyway, we can all get into that later. You heard it here folks, Episode 3 is coming out, although probably not for a while. We're going to be mixing up and we will be talking about microstates at least for a while. I understand that next time, we actually teased it but we based on how much material there was on microstates, we ended up packing in a second episode. We teased it last time, we're going to be talking next time about running an online conference with Twitch, so definitely look for that. Thank you, Taras. Thank you, David. TARAS: Thank you. DAVID: Yeah, it's been great. CHARLES: This is a wonderful conversation and as always, we are Frontside. As I mentioned at the top of the show, we have availability coming in August, so if working with us is something that you would like to do, we have a range of services, please get in touch. You can get in touch with us at @TheFrontside on Twitter or Contact@Frontside.io. That's it for now. I guess we should also mention that Taras that you are going to be giving a talk on microstates at ManhattanJS. When is that? TARAS: On August 8th. CHARLES: I will be giving a talk on microstates at ReactJS Austin on Monday, the 6th, so that is right around the corner. I'm excited about both of those talks, especially following so closely on the heels of the TorontoJS meetup talk, which I understand is... Is that posted online yet? TARAS: It's recorded but it should be coming out soon. We'll definitely tweet it out. CHARLES: Okay. All right. Look for that and we will see you next time.

In part I of The Frontside's microstates series, Charles Lowell, Taras Mankovski, and David Keathley talk about state management that's easy and fun and transactionality. This show was produced by Mandy Moore, aka @therubyrep of DevReps, LLC. Upcoming Conference Talks: Toronto.js - July 30th (Taras) Manhattan.js - August 8th (Taras) ReactJS Austin August 6th (Charles) Transcript: CHARLES: Hello, everybody and welcome to The Frontside Podcast, Episode 106. My name is Charles Lowell, a developer here at Frontside and I'm going to be hosting today's episode and we're going to be talking about microstates with fellow Frontside developers, David Keathley and Taras Mankovski. Welcome you all. DAVID: Hello. TARAS: Hello, hello. CHARLES: I'm really, really excited that we finally get to talk about this but before we jump into this, just wanted to let everybody know that last week, we published our roadmap for Big Test, which is a JavaScript acceptance testing framework that we've been building right here, in-house at Frontside, which can work with both Ember applications and React applications, Vue applications. It's in alpha phase and we're looking for feedback but we're really excited about it, so we're going to leave a link to that in the show notes and please go check it out. All right. Finally, the moment has come that we actually get to talk about this publicly. We've been publishing things about microstates for a while now but we feel that it's ready to share with the world and that's really, really exciting. We should probably like wind the clock back a few years because that's about how long we've been working on this and talk about, kind of the why and the what of what microstates are. It's kind of a weird word. What are we doing here? TARAS: Yes. That's interesting question because we were working for so long and after all this time, what is this specifically we're working on. I can speak personally from my personal motivations because we have conversations over the last two years to why we were doing this and I think for me personally, it's always been that I've been mentoring building complex applications for almost five years now and one of the things that I find consistently is that there are patterns for how to build complex stateful UIs that the required solutions, that are fairly reliably consistent. I can teach people certain patterns and then they can use patterns to build complex applications and those patterns scale really well, the challenge is that, there's not an easy way to express them and it's different for every framework. The way that I would teach somebody how to do it, for example in Ember versus how they would do it in React, even though the pattern itself is the same but the implementation of the pattern is different but it's different in such a way that it's very difficult to see where the consistency is, what is the same about these two patterns. It shows me that there is room for improvement. In the same way that if there is an opportunity that in the future, components will get unified under one umbrella or what component spec. The fact that we do states differently across every implementation, across every opinion, it suggests that what we might be missing is something that would unify across frameworks, how we actually do statement management. CHARLES: Yeah, that makes sense because as much as people try to buck the trend of MVC, it can't keep coming back in the rear view. Hopefully, not like a horror villain but like a caring friend, like a reliable pattern. I remember when I was in writing Java applications back in the day, the most important thing when you were writing a swing application was making sure that you had your models right. If you were modeling either a form or a dialogue or a set of pages, the most important thing was to have those models. Back in the day, we modeled those things with event listeners, very similar to where the way like a Backbone application used to work or if you're familiar with Ember Object, the way that it worked basically, adding observers to some model so that when that model change, you were able to react to that. Your representation was able to be 100% dependent on this model. That was like a Java Swing Application, which was then inherited from this pattern from Smalltalk but we keep on seeing this again and again and again. There is this piece, this state, that if you're going to be representing something, then you need to be able to get at the meat of it. That's ultimately what the model is in the MVC pattern. I remember when React first came out, everybody was like, "Oh, MVC is dead," but now in terms of state management and all of the state management solutions that we see, what that really is, is the model trying to reassert itself. Because it's such an important piece of your application, it's going to rear its head and you cannot escape it. I make it sound like a problem. I guess, it is a problem but there's a proliferation of solutions out there that are attacking the problem in order to represent something you have to know what that something is. DAVID: I think when you only work with the one framework, your perspective might be very much shaded by your experience. If you're really good at building React applications and you have subscribed to POJO is king and you don't need anything beyond that, then you're going to get good at handling POJOs and you might not realize that there is actually limitations to what you're doing. I think the same applies to people who use Ember and they're using Ember Object and they have computer properties and now, in Angular, there is observables and subscribing to streams. What was really interesting with the work that we've been doing with microstates is that and one of things that have notice with other state management solutions is that a lot of them emerge as, "Oh, I had this insight. I spend a couple of days working on it. Here's my proposal, in a way, for managing state." I think that's great because insight and understanding is really important but sometimes, taking the time to design something can be really helpful. I think that one of the things that are really interesting with microstates is that because we've been iterating on it for so long, they've been testing it for so long, we've been able to do something in microstates that's really difficult to do when you have a production application that you're like, "I want something different. Something is not enough but I don't have the time to really make it better," or, "I have an idea. I want to release this thing but I don't have any other collaborators that I can talk to and really flush out these ideas," so what you get is a solution but it doesn't always touch on a lot of the angles that you want to touch on. It's really interesting when you think about -- CHARLES: I'd say, we should talk about those angles, like what are some of those angles? DAVID: Yes. You could think about this from a few different perspectives. I would say, from Ember perspective because that's been my personal starting point. For anyone who was listening. Ember and Vue, for all intents and purposes, in this scenario are very similar because they have the same primitive, which is the computer property. The computer property in Ember, the computer property in Vue is almost identical from the way that you actually consume. In Ember, you would use computer properties and in Vue, it's just computer properties to create derived state from data that's passed into the component and then, you would use is derived state to basically, decorate information that comes with the props, so you could present it in your component. This pattern is very nice. This pattern is being used to build LinkedIn, to build applications at Apple. There are huge implications being built by Ember community using this patterns and I'm sure something similar has happening now with Vue but this pattern doesn't really exist in React in the same way. You might opt in to start using this pattern if you start using [inaudible] but it's not quite the same. Writing computer properties in Ember and in Vue is essentially free because it's so effortless, where in React, if you're using [inaudible] to create cache computer properties and memorize computer properties, you are really opting into doing a particular way of computing this properties and writing selectors that's not trivial. I think that's one thing that is available in Vue and Ember and it's a really effective pattern and you can do it in React but it's actually not possible in Angular, unless you're using, I'm guessing something like ngrx, where you can do similar things to what you would do if with redux in React. Creating derived state that you can consume in your component is one of the things that is possible but it's not possible consistently across all frameworks. That's just one of the things. CHARLES: I agree. I think that was one of the things that drew me to Ember at first coming from Backbone as I did and honestly, from the models in Java was that, in order to compute anything, you had to install a listener and then eagerly make that computation and store it somewhere, where as those frameworks, it made you feel effortless where you can just decorate some state and derive it and the information is there, the computation is there when you need to reach for it but you don't have to do expend any extra effort aside from say, "This state is derived off this other state." I think another case that I came across was immutability. Immutability is a means but the end is to have consistency inside your application. I first really started running up against this when I was working with forms and since then, I've realized that actually, a lot of the pain that I was feeling was because things were not being immutable but this is where the fact that things weren't immutable ended up causing me a lot of pain and headaches and I was having the code into being a lot more complex. Essentially, when I wanted to make something transactional if you're editing a form or something like that, then you need to essentially store off a copy of your current state. If you're editing some object, I want to say, open up a dialogue and I'm going to edit it and then I'm going to commit the changes back to that dialogue. What I'm modeling there is a transaction so I kind of need to shave off a copy or make a Xerox copy of the object and then make the changes inside to that object and then somehow, try and merge those changes upstream and then, get them back into the main lines. Really, like a very Git-like operation. It wasn't just at the object level. When you're doing things like dirty checking, you need to make what is the original value of a field versus what does the input currently have. I might be doing any number of transformations in between the actual physical representation of the field, maybe it's a date but the user interacts with it in as a string, so there has to be this kind of parsing serialize thing sitting in between that value and you need to basically, keep a copy of that field as a mini transaction within your macro transaction because you want to say like, "Is it dirty? is it really the same object?" because if you can just do an object comparison, you have to get into all that hairy like equality checking and it gets really complex really fast but it turns out that a very clean solution to this is if you just never actually make the changes in place and whenever you are making changes, you're generating new information without destroying the old information because when a form is the case where we really come up against this, where we're modeling the same object over time, so a form explicitly models the change of an object. Change is part of what it is and so, the definition of change is being able to compare something in its prior state to be able to compare it to its current state. If you're making that change by destroying the prior state, then you're going to run into a lot of trouble. It just turns out that when you're working with forms and it turns out there are a lot of use cases like this but this was the one where I first really just couldn't even... Without immutability, you want to model your change as always rolling forward and not ever destroying prior states but being able to pick and choose and can always be able to look back to where you were and compare where you are now to where you were. That's where immutability comes when you're modeling change. It's actually much easier to model change when you have explicit states that represents what was and what is. But when you start doing that, then things get a little bit messy. You have to do the compromise. There's a tradeoff, like when I say, "Set this property on this object," that is easy and that's something that I think that we can all understand. We're not idiots. That's why immutable models are very conceptually easy to grasp, to wrap your head around. If I'm modeling myself and I'm saying, "Set hand position two feet up in the air and now I'm raising my hand." That's easy to understand and it turns out that when you're changing a data structure immutably, then you have to, for example model a simple set operation as duplicate and swap. Or if say like you're swapping out one property in an array, you actually model that as a map, where you swap out the one element at that index, rather than just saying, "Just set the thing at that index," or if I'm changing a property of an object, I want to copy over all the fields except for that one which I'm going to change. If you start to do that then, you realize the benefit of always being able to look backwards but you've now introduced overhead and complexity in your code, so you've made a tradeoff. I think that's a lot of people look at saying, "I want this to be immutable," then they actually have to come to the grips with the complexity that's going to introduce. There are libraries like Immutable.js that do make that a lot better but even they have the problems. I can talk about those but one of the cases is like being able to reason about a series of states for your object, rather than just only having one copy of it ever and being stuck with having to deal with it as it is. DAVID: Yeah and I think that really important too is being able to mentally track where you've been because in my experience on other large Ember applications, I've run into so many different bugs that I could really track down to people or rather, places in the code where things are just reaching in and mutating your model, where it wouldn't make sense, so as you're writing your code, you're in a completely different state than you might have expected. CHARLES: Yes and it's easy, right? But essentially, when you have some model, you've basically got a global object. There's a lot of recomputations that needs to happen when those things change. As a result, if you look at the actual code that supports computer properties in Ember and I'm sure other frameworks as well, some of the most hairy and complex. If you look at the chain watchers and the chain nodes and all the stuff that's required to support things like computer properties, it's amazing that it works as well as it does. I've tried to actually open that up and understand that code on a number of occasions and every single time, I had to walk away in defeat. TARAS: This problem exists in React applications and Angular applications because if you think of the challenges of working with set state, I think one of the problems with working with set state in React is that your complexity increases very quickly. It starts off kind of simple when you're using immutability with set state in React. They're kind of very light and then, as you start to add features, your complexity starts to grow very quickly in a way that you really start to run into limitations. You start to confront your personal limitations of your abilities to work with immutability, so you're limited to doing immutable structures that only a few levels deep because your ability to use destructuring to create new immutable object is limited by expressiveness of JavaScript. It's gotten a lot better with the destructuring but if you do it a few levels deep, I think everybody's familiar with what happens. It starts to get really hairy. You kind of loose track of where you are. CHARLES: Yeah. The signal-to-noise ratio increases because after probably, two or three levels, the majority of your code has to do with destructuring and restructuring and very little of the actual change that you want to make. TARAS: And this is kind of interesting because people talk about declarativeness versus imperativeness but at certain point when you have a complex immutable state change, if you're doing with destructuring your code, even though it looks declarative but there is so much processing that you're doing, it's actually kind of losing the benefit of its declarativeness. It's actually starts to look more like imperative code than it does what you would expect a declarative system to look like. I think this touches on the other aspect. It kind of compromise that when you work immutably, when it compromises, you make a serialization. Your ability to represent your state as a POJO becomes restricted by the fact that you have this complex system that's wired together and you have systems like zones in Angular and in Ember Object that are able to keep track of changes in these objects but you don't have a way to restore those objects. You don't have a way to do more sophisticated things that you might want to do, especially in situations where if a service feeding you, what do UIs going to look like. In that situation, it's really helpful to be able to say, "Here's a POJO that I got from the server. I'm going to use this POJO to build this component tree that the user is going to interact with and then, as the user interacts with it, I'm going to then, capture that state, serialize it and put it back in the server." When you're using something like Ember Object or if you're doing this kind of stuff in Angular or even if you're doing this stuff with React but without using something like redux, you essentially end up doing so much wiring to accomplish that. By the time you finished writing your application, you've written a ton of code just to handle this particular use case and if you have to do this again in another application, you just rewritten that kind of code in a new application as well. CHARLES: It reminds me of the concept of a Smalltalk image, like there's no way to really get at the state of a Smalltalk thing. It's almost like you're dealing in docker containers and not actually being able to write that state down into JSON or something like that. I'm trying to casting about for an appropriate analogy. Maybe that's not a good one but what's actually happening cannot be made orthogonal. It can only exist in that one run time that you're currently running. If something wrong manifests itself, reproducing it can be extraordinarily difficult, right? TARAS: Yeah. CHARLES: Imagine if there's some render cycle that's making a bunch of mutations and there's this process that you stood up and it runs in completion, some signal comes in and those effects are like ripple through the system, there's no way at any point to have any other representation of that system than the running system itself. TARAS: There's another element to this, which I find really interesting. When you're thinking about how to architect complex UIs, it's actually helpful to get really clear about what kind of changes are happening. A lot of times when I want to see beginners are writing, especially if you task someone who is a fairly junior at building a single page applications, a lot of times what will happen is because they don't have a clear mental model of what is going on in regards to state. They end up setting a lot of properties. Every operation, every time you have an event handler because they don't have a clear model of what's going on, they end up setting like five or six or seven properties. That kind of signals that they don't have a clear picture but what that also does is a lot of times, they usually comes together with cascading state changes. Usually they're not representing a single operation as a single state change because there might actually be a bunch of things that are happening because what they're doing is they're massaging the system into submission. Not like they're not in control of the state transitions, so they use, essentially time and their dedication to kind of sort it out and make it work and eventually, would that ends up looking like that if it works for most of the cases that they are able to test for or that they able to manually see. But then they AR, not accounting for problems that they're not able to understand right now, they become discovered by users when users start to interact with the system and with the components or with that application and the application is there to get into some funky states. The tools that we have, they don't prevent that from happening. They just -- CHARLES: Right. They don't force you. I think what you're saying is that ideally, you want to model your UI as a set of transactions on your state, that you want transactionality to your state so that I basically am saying, "I'm not going around and setting seven properties in reaction to this one event." I'm saying, "This event triggers this transaction and that transaction clearly bundles up every single operation that needs to happen and the tools don't enforce that." Is that a fair --? TARAS: Yeah and then, the problem is we're working with component trees. You start off with having a set of requirements and over time, the requirements change. As the business unit understands your application better, they give you more direction of how accounting should work and then you find out that there's more interconnect at stake but then what's happening now is that the cost of refactoring those state that spread throughout the components, whether that be with set state, whether that be with the actions in Ember or even in Angular. What you end up doing is you start to change the system but change is not trivial because the actual process of changing where that state lives is not linear. It depends on the complexity of the code that you wrote and it just gets really hairy very quickly. That's where companies end up losing a lot of time. A developer could start off with a requirement, you build something and then a new requirement comes in and instead of it being a simple change, it turns into a week or two weeks refactor because you now understand the state ownership should be different. The state that you have should be in a different place. You have to manually make that change. You have no obstruction to help you express that in an easy way. CHARLES: Another thing, because we are doing a kind of a roundup of all the things that you need to account for when you actually embark on managing your state. Another is actually constraining the amount of computation that happens. If your system is based on listeners and large chain reactions of things where it's like, "This property changes so I need to notify these other 10 dependent properties that change." You can do a lot of unnecessary computation, especially if nobody is going to render. That's kind of the thing that you have to do if you're going to be immutable. You have to eagerly walk those change to see which objects are affected so that you can then invalidate those caches. A system like Ember Object, I don't know exactly how Vue works, it mitigates this somewhat by the fact that the computer properties are lazy but you still have to walk all of those chains. That can actually get out of hand. They're eager, not lazy. Then the other concern that you have, where you normally have to make a trade off around is around composability. One of the things that's really nice about immutable systems is they're very composable. If I've got some object that does one thing, I can then just set that object onto another object just by mutating one of its properties and I've effectively composed them. I can then install listeners onto that thing or I can compute properties off of that property and they can post pretty well. That's something that you get but then of course, you're losing all of the benefits of immutability, so things like Immutable.js don't really compose very well or redux doesn't really compose very well. The concept of taking a redux store and embedding it into another redux store, you just don't see that. I would never distribute and I think ultimately, the litmus test there is would I be able to share it on something like npm. Nobody shares an npm package that's just a redux store that you can dispatch actions to and observe and use it with your other redux stores. When it comes to a system like Immutable.js, that does make transitions a little bit easier over lists and arrays and maps but you still run into the exact same set of problems that you have when you have lists of maps of records and you don't really get any help there, so you have to make this tradeoff between immutability and composability, whereas a system like MobX or Ember object actually quite composable. Before we start talking about microstates, I want to say that you just throw those in there because there is just a lot of concerns out there, a lot of edge cases that actually build up but through the course of a real application, you will encounter them all. You might be making tradeoffs at the beginning that you're not realizing that you're going to need or are going to get you into trouble later on. DAVID: This actually happens in the Angular community as well because there's something really great that's happening with observables in the Angular community. I think everyone's embracing them wholeheartedly and I think that's really been pretty great to see but observable streams of composable, but objects that have on them observable stream providers of some kind, like if you have something that you can subscribe to and that is part of a property in a class, composing multiple classes together and consuming properties from those classes, there is no mechanism for composing that. That kind of composition has to be done manually. Again, you're kind of manually wiring together a bunch of objects and the big challenge is that you are manually subscribing to all those streams and unlike what you have with components. Components have lifecycle hooks. When your components is being torn down, you know you can perform some operations. If you need to remove an event listener, you have a hook where you can do that when you have a class instances like JavaScript class instances that have on them properties that have observables that you subscribe to. There are no tear down hooks for class instances, so there are no obstructions from managing unsubscribing from those streams. You essentially end up having the foundation that you can use to build complex reactive systems and you can subscribe in there really fast but wiring those things together at a bigger scale is simply not there. It's something that you have to create and enforce yourself. CHARLES: Right and I think that's probably a perfect segue into talking about microstates, which is the project that we've been alluding to for the past 30 minutes, that is I think in attempts to solve these problems and make sure that you don't actually have to compromise on those things, so you can reason about things locally but have those things be composed into a greater state. But also have them be immutable so that you can look at past states and reason over a data structure over time as opposed to just in one instance. Also, have an intuitive interface that when you're making these changes, doesn't look like half of your code is unpacking some data structure, flipping some bit in it and then repackaging it back up again. That's the context. Should we start talking about what microstates is and how it addresses those? TARAS: That's a good next step and when I start working in the ReadMe, I end up actually, I think I wrote about 40 pages. One thing that's interesting about microstates is that and this was part of the design of microstates from the work that we've done is that we intentionally wanted to make the number of ideas that you haven't microstates very little, so when you use microstates, the number of concepts that you need to remember in your mind is very few. It is a conceptually a different way of thinking about organizing your state in the same way that shifting from managing DOM elements directly to having an obstruction like component that declaratively applies changes to your DOM tree. In a same way, microstates is kind of an abstraction that allows you to declaratively describe how your state will change and it will manage the transitions for you and allows you to give the state transitions names and it allows you to give your states names as well, so you can actually name things. You don't get a POJO that has a shape but has no name. You actually get to give things names. CHARLES: So, why don't we start? I have a list in my mind. I should probably write it down of the things that we just talked about but I think the things that we talked about are ease of representation, like conceptually easy, transactional, basically serializable and immutable, lazy and composable. Those are like five or six things. But I think there are kind of aligning principle around which we gathers that the state management should feel easy. It should feel fun. One of the things that is awesome about working with components, whether you're using web components or React components or Ember components is when you get it right, you're just snapping these things to feel together and it feels great. It's like I'm just passing properties and render blocks down the tree and the framework is just doing all of the grunt work for me and I'm just operating at a very high level. That's what organizing principle with microstates as it needs to feel easy. Maybe we should start there and just say, how does that easy and fun line up with each one of those kind of unique problems around which we typically have to make tradeoffs? We could start with the interface of making a change. TARAS: I'll go back to the starting point. I remember what got me first interested in microstates is Charles, when you said that, when you have a number, there are certain operations that a number can do. We really don't need to be writing an increment operation for every... Like if you a have a number, you can increment it and decrement it. CHARLES: Honestly, every time I see state management tutorial and they tell you how to increment and decrement a number with it and you write the increment code and you track the thing and you store it back into the store, at this point I'm still annoyed with those tutorials because I'm like, "It's a number. We know we can increment it. Just show me where to plug in the code. I should not have to be writing an increment method." TARAS: Yes. And that's the kind of starting point. There are certain operations that you can perform with the primitive types. If you need to add a number or if you increment a number, we already know how to increment the number. It's part of microstates. But that in itself is nice but that's not that important. I think what's really important is that when you need to put a number into another data structure, let's say you have a nap and you're like, "I need to..." I don't know -- CHARLES: Let's say, like a click tracker that has a number of clicks. TARAS: Right. By itself, you can increment the click tracker but if you need to put a quick tracker into another app, essentially you can compose it in and you don't need to figure out how to wire the actual mechanism of how to make sure that you can update the property, like it's part of another class, for example, you don't need the way you would increment the number. When it's a part of another class versus how you would do it when you're working with it by itself is an approximately the same. The amount of work that you need to do to actually perform that operation is the same. Your complexity doesn't increase as you compose one data structure into another. CHARLES: Right. You can just say, "This is an app. It's got a click tracker and this property is a click tracker and I have to do nothing else. I can register clicks on that thing. It doesn't increase the complexity of application at all." TARAS: There's no wiring. Now, you added some new state, that state is very explicit and it's really clear that it is not impacting other parts of the state. You can operate with this thing. If you change it, it's going to work properly with all the other things that are in the type that you are adding this counter to. Those things are just going to fit well together and it's not going to break if you need to transition one more thing. All of the other transitions will work the same way. I think that kind of consistency is really meaningful, over time especially when you start to increase the amount of state that you manage in your application. CHARLES: Just the ability to work with types and just have kind of those implicit operations and have those things compose, kind of indefinitely. Moving down, we talked about easy and the other thing I would put on that is that the way in which you express those transitions, for example if microstates comes bundled with numbers and Booleans and strings and arrays and objects and kind of the stable of types that you would expect in any JavaScript application but those types are expressed using this way for expressing types, essentially. When you actually do make a transition, it feels very object-oriented, I would say, even though it's not. It feels like you're mutating but you're actually doing a transition. Does that make sense? TARAS: I think for anyone who is familiar with what it's like to write queries for GraphQL, if you're not familiar with it, it's fine. You can get a sense of that from microstates but if you're familiar with the ease of just writing a query and if your backend knows how to retrieve the data, then your queries will just give you the data that you want. That feeling is really powerful and just being able to write the query and just gives you what you want. Microstates is kind of like that. Actually, the inspiration came from experiences with GraphQL, which is that sense of ease is what we wanted to have in microstates and so you get that seems sense of like, "I can just do what I want and it just going to work and with this other thing and it just going to work," and you're just like flying through, like adding states to your application and it's just working for you and working for you and working for you and you don't have to do monkey work like gluing things together. It'll change how you are working before because you have a way of opt working with these things at a higher level. CHARLES: Right. Let's talk about transactionally or should we talk about immutability? How does this make immutability easy and fun? TARAS: I think one thing is that you don't have to write reducers and you don't have to do destructuring by hand. I think you have a way of expressing. Thinking about this, if you have a component tree and let's say you have redux and then a bunch of components like your parent component, your root component has some state using sets date and then components further down the tree also have state. You could actually express that as a microstate. What you would do is, essentially the parent component state would be the root and then the children's component states would be composed into it. The nice things about doing that is that at the root level, you have access to transition state of the children declaratively. You know where the states for those children is on the route type and you can write transitions that are going to declaratively perform multiple operations on the children state and I suppose it'll restructure to what happens with components but if you don't use this, you might have multiple sets date operations. The process of wiring data from the root down to the children is kind of complicated, where here, you have a way to represent that and perform a lot of transitions in the way that is going to be just easy at whatever level you need to operate at. CHARLES: Right. I think, for people familiar with redux, in redux you act globally and then you react locally, if that makes sense, so you dispatch an action to the entire store and every single reducer can see that action. There's ways to manage that but effectively, you have this one atom and then you have the reducers that kind of act on local state, whereas with microstates, you're basically acting locally. You're reacting locally but the effect is global. TARAS: You're participating globally. CHARLES: Yeah, participating globally but you never have to consider the context that is above your own, so you never have to be mindful or cognizant of the context in which you're enclosed because from your perspective, it just doesn't exist. DAVID: Every microstate has a set transition which is the basic transition that you can invoke, essentially in any type, so what's interesting is that it's amazing how powerful -- CHARLES: So, we should break it down really simple. Basically, when you create a microstate, with a type, you say like, "I want to create a number with the value five," and then I can just say, "That returns a microstate," and I can say, "microstate.set 10," and that will return a new microstate who's also a number but the value is 10 and that's available on all microstates. DAVID: Yes. If you have a tree of components and your state is presented by a microstate at the root level, then what you can do is you can invoke the transitions on any part of the microstate and it will just know how to properly create the next microstate for you. The example that Charles you gave of one number so that number can be inside of a class that represents state for a particular component and then that can be a part of another class or represents a state for another component but then, when you invoke a transition on one of the leaf nodes, an equal sets state on one of the leaf nodes or equal set on one leaf nodes, it will respond locally but it will actually reflect the changes globally. At the root level you're going to get a new object that causes the components to update. CHARLES: You know what? I have another concern that actually just popped into my head, which is something that I've certainly struggled with in every single application of notable complexity is stability of value. We should put that on the list. We're almost out of time to talk about this. We spend too much time... Well, not too much time, of the issues of state management, which I think you can't spend enough time talking about but I did want to pile one more on there is when you're making that transition, where you're acting locally but you're participating globally. For things that are unaffected by your action, remain unchanged. That is a super power. It's actually very hard to do with a lot of state management, especially when you're cloning a bunch of stuff, being very judicious about what you don't want to clone. Where this really comes into play is if I want to re-render something. A lot of times you have to jump through a bunch of hoops to tell, did my model really change or did only referentially change? With microstates, when you make that local change, if you're embedded in a very large graph of objects, obviously all of the objects above you are going to be changed but what about things that are off to the side of your siblings. They're outside of that scope of that change. They shouldn't be cloned. They shouldn't be copied over. They should remain the same. If you're doing a re-rendering based on the changes that are happening, that's going to be a key feature because you're not going to have to write, basically any hooks to say, should I have to re-render my component. You can always rely on triple equals. DAVID: This quality is going to describe the structural sharing and some of the other tools that are available but I think that's how it's accomplished. I think one thing that's a little bit different with microstates is that when it comes to structural sharing, it's not that difficult to do if you just do structural sharing and value. Meaning that you can do structural sharing on a value using something like lenses and not a lot of people are familiar with lenses in JavaScript but it's actually only three functions that you can use and it can give structural sharing on complex POJOs. It's pretty easy to use relative to how little people know about it but what that doesn't do is it doesn't allow you to graph of objects that have their own operation that you can invoke and that will perform structural sharing. That part, I know that is not available in any solution, I think at the level of completeness and luxury that microstate provides when you write things. CHARLES: Right because every piece of the tree kind of comes bundled with its own things that you can do with it. DAVID: I don't think you should jam everything into this podcast because there's a lot to talk about it. I think one of the things and we've talked about this a lot, which what we want to do is create an implementation for an idea of what it would look like. What would it be like if we had a composable state primitive that we could use to describe state and share state solutions in the same way that we share components like react-virtualized or whatever your particular frameworks or popular component may be. What it would look like if we had solutions to state problems that we could share and we could -- CHARLES: I think the litmus test of an awesome solution is like you look at this current crop of MVC frameworks and what's so awesome about it is they're sharing. That can happen, right? If I'm writing a React component, I can publish it on npm and other people can use it. If I have an Ember add-on, I can publish it and people can use it. They can consume those components. That's awesome and I think it's the hallmark of a great system. What would it look like if I wrote just the state piece of a file upload and I could publish it npm and then anybody, in any framework, could actually use it with their framework without paying any penalty. What would it look like if there was some transactional data store that could be built and shared and the hooks into any framework were minimal. The possibilities are really exciting around that idea, whether it is realizes microstates or not. But clearly, we feel this is something you should be able to do. DAVID: I'll add one more use kind of use case that I personally find really motivating is that there's a lot of companies that are investing into building single page applications and a lot of times, what you see happening is they're building a very similar application to what they had before because their business hasn't changed. The technology has moved on so solutions have improved. The demands for better user experience have increased but the actual business and how the things that people have to do on day to day within their company hasn't changed. What we're seeing right now is we're seeing the same, like whatever was written before in jQuery or an AngularJS is now being rewritten in React or Angular or whatever you might Choose. But whatever you like to see is it has a situation where the domain specific logic of your business is represented as a data structure that knows how to, potentially, in the future talk to the server and retrieve data from the API because that's likely not going to change. But you can use that like that's been tested and published as an npm module within your enterprise and then you can then consumed that in any framework and it's actually easier to do it this way, than to implement it in a framework-specific version of their state management. That's the part that I find the most exciting. I think one of the things, just to connect to the goal is that, we would like to keep this conversation going. If you're interested and I think this is a kind of a call to our audience that if you're interested in this, we would love to have in our podcast to talk about these things because I think there's a lot of things that microstates really is a beginning of a conversation. It's not meant to be a statement. It's meant to be a proposal that we can just talk about this. CHARLES: I agree and that's one of the reasons we're keeping it very small at this point. The core library of microstates is not setting out to accomplish too much. In the core library, there aren't even any side effects. It's actually impossible to have side effects. It means that it cannot be used for anything except for the model but that's very liberating and it let us focus on what would a system like this actually look like. DAVID: It's really exciting because this has been the biggest metric but we have microstates in the JavaScript weekly and that was great and then it got circled around when people know and it's like 800 stars now, which is not a really big deal. It's funny because somebody commented like, "How can you put something in production that only had 100 stars?" CHARLES: I think it's just important to realize that this really is the beginning of a conversation. There's a really exciting set of things to come. We haven't even talked about how we're going to model side effects, although we're going to use microstates to do it. We haven't even talked about what the various framework integrations will look like and what are the best practices for using this to organize state in your application. We've had some lively discussions internally about what that looks like. There's still a lot of questions but it's going to be a really, really exciting and edifying experience to get to answer them. DAVID: Yeah, it's pretty exciting. I'm excited too. There's been a lot of interest from people in microstates, so it's going to really great. I'm looking forward to meeting people and having conversations about how we can use microstates because I'd love to have someone create a really great solution that I could just take off the shelf and just use and not have to implement them myself. CHARLES: All right. Well, I think we could talk about microstates for at least the next three hours but we have to give everybody an opportunity to, at least like go to the bathroom or something. Microstates will return but if you're interested in learning more about microstates and you happen to be in one of the many places on which we're going to be presenting on microstates in the future, who knows? Maybe you can come in and join the conversation in person. Taras is going to be speaking at Toronto.js on July 30th. He's also going to be presenting at Manhattan.js on August 8th and then, yours truly will be presenting on microstates at React.js Austin on the 6th of August. Come out and see us. We'll drop those in the show notes and it's guaranteed to be a good time and we'll have that conversation. Until then, we are the Frontside. We lead with the why, the how and then the what, if you're interested in working with us and that helps us that we guarantee the lowest total cost of ownership for your application. We're always looking for feedback. If you have news items that you'd like to see at the head of the show or just any feedback or questions, we would be happy to answer them. Thanks today to Mandy Moore for producing our show and next time, we'll be talking with Kristian Freeman about what it's like to run an online conference with Twitch, so I'll be looking forward to that. Bye David. Bye Taras. DAVID: Yeah, thanks for having us. TARAS: Bye. CHARLES: Yup, and bye everybody. See you next time. Next Time: Running An Online-Only, Free Conference on Twitch with Kristian Freeman

In this episode of The Frontside Podcast, panelists Charles Lowell, Rob DeLuca, and Sam Keathley, discuss how much the distinction between frontend, backend, and fullstack developers matter in both personal and professional senses. The differences in mindset between these categories are also discussed, for example, how does a fullstack developer solve a problem vs how a backend or frontend developer? And also, how clear (or fuzzy) is the line that separates them? What are the skills a frontend or backend developer needs to consider themselves fully fullstack? And finally, is there any sort of tribal separation between the factions? Do you have opinions on this show? Want to hear about a specific topic in the future? Reach out to us at contact@frontside.io or on Twitter at @thefrontside. This show was produced by Mandy Moore, aka @therubyrep of DevReps, LLC. TRANSCRIPT CHARLES: Hello everybody and welcome to The Frontside Podcast Episode 101. My name is Charles Lowell. I'm a developer here at The Frontside and I'll be hosting today's episode. Today we're going to be talking about the different developer profiles that are out there. You might have heard mention of them on Stack Overflow, on job boards, on the Twitterverse. Things like frontend, backend, full stack, half stack, short stack. [Chuckles] Things like that. With me to talk about these are Rob and Sam. Rob is our director of open source here at Frontside and Sam is a software engineer here at Frontside. So, hey y'all. SAM: Hey. ROB: Hey yo. CHARLES: Alright. So, before we get into the meat of the topic, there are a few announcements that we wanted to make. First and foremost, this is some really exciting news. This week, Taras Mankovski officially joined the team at The Frontside. And we are really, really excited about that. [Cheers] ROB: That's exciting. CHARLES: Yeah. It's super exciting. Not only is Taras an exceptional engineer, he's got amazing code skills. He's extremely empathetic and a great person to work with. He's great working with clients and he's also fantastic at really working with developers who are climbing that ladder and helping them level up. So, he's been involved in mentorship literally since I've known him. I think that was the first time that we ever came into contact with him was through his mentorship work in the Ember community. And we've been collaborating with him for years on several open source projects. And so, we just are so excited and know that he's going to come on and do some amazing things by helping us be better developers, helping us be better community emissaries. And so, lots of good stuff coming out of that. ROB: Absolutely. CHARLES: Yeah, yeah. No, that's really exciting. So, definitely wanted to throw that out there. ROB: Too bad he didn't join a little bit earlier. We would have had a good time with him on this podcast. CHARLES: I know. Well he's actually, I think he's been on the podcast twice. ROB: [Laughs] That's true. CHARLES: But yeah, there's definitely a couple of topics where his input would have been absolutely critical. I think it would have been great to have him along when we were trying to run our own apprenticeship programs. Those would have worked out I think a lot better. Well, not to say that the outcomes haven't been stellar in some cases. But I think having his expertise would have made it a lot smoother. That said, before we jump in then, just wanted to let everyone know that as always, if you want to work with us you can get to us at contact@frontside.io or send a shout-out on Twitter @TheFrontside. Okay. Y'all, let's jump into it. Like what is even a stack? ROB: [Laughs] So, do you want to kind of set up where this came from, Charles? I think it came from you and I going back and forth on Twitter. And I was like, “You know what? We need to talk about this on the podcast.” CHARLES: Yeah. I think I was actually on vacation. ROB: [Laughs] That's right. You were trying to ski. CHARLES: [Laughs] I was on the ski lift. And I saw you tweeting and I was like, “No, no, no. I got to respond to that.” Probably because there was maybe a little bit of disagreement but also because I just needed something to do. Looking out the majestic scenery of the Rocky Mountains wasn't enough. So yeah, what's the backstory there? ROB: So, I can't remember what I was actually tweeting about. But I kind of feel like there is a distinction between a frontend developer, a backend developer, and at this point I don't know if a full stack developer truly exists. I mean, it does. But it's used more often than it really implies what they're doing. Usually when you hear of a full stack developer, it's like, “Man, I can sling some code on the backend and I can do a little bit of architecture stuff on the frontend. But CSS, keep that away from me.” And in my opinion, if you're a full stack developer, you need to sling CSS, too. CHARLES: Really? ROB: Yeah. CHARLES: No, no. I certainly agree. Because I was going to ask, is it fair to say that full stack developer is code for like, startup developer? ROB: Yeah. That's a good – like a utilitarian, like a jack of all trades but not a master of any? CHARLES: Right, exactly. ROB: I can take that. So, I guess the way we can start to dig into this one is like, how much distinction is there between a frontend, a backend, and a full stack developer? And does that matter in a professional sense? CHARLES: Right. And so, I think my take was that the skillsets that you need in both places are actually much more convergent than you think, than people might think. In other words, the skills that you utilize on the backend actually translate very well to the frontend. I think my stance is that what makes a frontend developer and a backend developer is more the problem sets that you gravitate towards naturally, the things that interest you. I actually was a backend developer. And now I consider myself to be primarily a frontend developer. But the skills that I had developed writing backend systems in Java translated shockingly well into frontend development with JavaScript. But it was working on the portion of the program that interfaced with the human was, that was the thing that fascinated me. And so, it's what I gravitated towards. And so, I guess in my [6:15 inaudible] or the way that I have been thinking about it is that what makes a frontend developer and a backend developer is more what gravitational pulls you respond to, rather than one particular skillset. But I know that you've got a perhaps more refined take on that, or a different take. ROB: Yeah. Before I do, I want to know what Sam's thoughts are on this. This would be a good one. SAM: Where I come from in the understanding of frontend, backend, and full stack is I – background on me: I did a bootcamp before working here. So, I did a development bootcamp where they were really excited to tell you at the end of it that, “Oh, you're a full stack developer because we've introduced you to everything. Like, touched on every sort of little topic. You're full stack. Tell everyone you're full stack.” And it's like, “Well, you're not really a specialist in any of the sense.” So, what makes a difference to me, I'm going to specifically talk about full stack, but it's like before you can say you're full stack, having knowledge of frontend and at backend, you need to have expertise in at least one of them first, or understanding of that mindset first. Like if I say that I like frontend work more then I need to understand frontend work more before I can say that I understand backend work, you know? ROB: Yeah. CHARLES: So, do you feel like the bootcamp was doing you a disservice? Or do you think they got… SAM: I don't think they did a disservice at all. So, they did a really good job. So, bootcamps are really for introducing you to everything. And then whatever you take from that is your own. So, they introduce you to all the little things like database working and then JavaScript, Ruby, whatever it is. And then it's your job to figure out in those introductions, what you resonate most with. So, I think they did a really good job in that. I think that they shouldn't tell you what you are, like tell you that because they touched on whatever it was, that you're full stack. CHARLES: Right. SAM: So, I don't think they did a disservice at all. I learned a lot. But I wouldn't say I'm full stack at all. CHARLES: So, if you feel as though you're not full stack, what do you feel that you are? SAM: Definitely frontend. CHARLES: Definitely frontend. SAM: I gravitate more towards frontend. ROB: Why would you feel that way? Is it because you're more pulled towards the UI of things? Or is that where you feel like you've gotten most of your experience now? SAM: Actually a little bit of both. I tend to like the UI of things a little bit more. It's interesting. Whenever I was going to the bootcamp, I thought I was going to be more backend because I thought I really like working with the behind the scenes sort of. You don't see this work but you know it's there. But I definitely like the UI of frontend a lot more. Just personal. ROB: I see that. SAM: And it's where my skillset is, because this is all frontend work that I'm doing here at The Frontside. So, it's what I know most at this point. CHARLES: Yeah. I'm curious too. Isn't it true that there's also a conception in the wider tech world that frontend is limited to CSS and HTML? ROB: Ooh. Yeah, that could be. CHARLES: Like when you see job posting for frontend developers, typically it's like, we want someone… ROB: Yeah. It's kind of morphed from – JavaScript developer is now a single-page app developer instead of frontend. Yeah. Yeah, I see what you're saying. It's now, it could be considered as HTML and CSS and not JavaScript anymore. [Chuckles] CHARLES: Which is so bizarre to me. SAM: [9:42 inaudible] a lot of places when a frontend developer who is also a UX designer. Someone who has the knowledge of designing things from the ground up and then only working in things that look good, rather than logic. ROB: Yeah. This can be probably an entirely other topic. But that's like the frontend of the frontend and the backend of the frontend, is like [10:03 inaudible] call it. Because there are [10:06 inaudible] developers. And single-page apps have gotten so complex these days, between your data layer and your testing and managing state on the frontend. There is a backend, an architecture, that you need to consider when you're developing a single-page app. And I've worked on teams where there are developers that are working on the frontend but they still don't like CSS. They don't do CSS things and they don't consider layout or any design. They just take the ticket and they get the data going. Get the data from the server, request it, display it on the page. And then another person would come through and style it up to the spec or the mock. But yeah, that's interesting. So for my take on this, and where this actually really came from, is we were going through some hiring. And we were trying to figure out how we can place somebody or hire someone that can be effective in the role that we immediately needed. And all of our immediate needs were in frontend-specific things. And we can actually talk about if they're frontend-frontend or frontend of the backend. But where this kind of came from or was born was: if we're hiring somebody that needs to fill a frontend role and they're primarily a backend developer, there are gotchas and things you have to know to be an effective frontend developer. Like quirky browser things that happen or like how to set up a Babel transpilation setup. And what are the gotchas here? Kind of the history of the frontend web, because there's a lot of things that are there that can cause a lot of headache if you don't know the backstory. And I know from your perspective Charles, you think – and this is isn't wrong – but you think that if somebody has a really talented backend developer, it's really just concepts that you can apply to the frontend. And I agree with that. CHARLES: Right. But you have to want to do it. [Laughs] ROB: Yes. That is also a key. CHARLES: You have to be gripped by the problem set of the frontend. You want it to inspire you to leverage those solutions. ROB: Yeah. So, would you say there's a different mindset between a frontend, backend developer, and full stack? Does a full stack developer solve a problem that's different from how a backend developer or a frontend developer would? CHARLES: That's a subtle question and probably one that might offend people, my answer to it. So, I'll go ahead and answer it. [Chuckles] ROB: Hashtag [12:23 inaudible] CHARLES: Actually, maybe not. Just in my experience historically – I want to underline the fact historically – I think that it is people who gravitate towards the outside-in mindset, in other words the very first thing they're thinking of is, “How is this going to feel to a person who's going to use this?” If that's your first question that you ask, then you probably are going to gravitate towards the frontend. Then in terms of the backend, I'd say historically – and like I said, I want to underline that, and I'll get to that later – I think it's people who are more drawn to: I want to attack the most complex problem that exists. Like distributing state over 10,000 nodes, managing huge deployment infrastructures that drive these massive websites that happen at this mind-boggling scale. And so, there you really are thinking computer to computer and, “What's the ideal interface for doing that?” And then on the frontend you're thinking more like human to computer, because that's where the interface lies. Now the reason I say… ROB: I feel so let down. Where's the controversy? CHARLES: Well, okay. Well, because I think that basically – I think the controversy is saying like people who are more naturally empathetic. I don't know. That would be the controversy thing. But I want to say historical. ROB: Yeah. I can see that. [Laughs] CHARLES: So, I want to say historical too, because I feel like one of the things that's been magical about the last two decades of software development is the dawning realization that no software you write exists in a vacuum and that it's all interconnected. And so, I think that I for example feel very strongly while I try to think about the user experience first. And the developer, I'm also thinking about developer experience first. The way that you want a system to feel is going to inform the design of the UX workflow and that's going to affect the architecture of your frontend. And if you're interfacing with it through different media, like websites, phone, I don't know, even text messages, Slack, what have you – that's going to inform then the next level of how your APIs are shaped, your public-facing APIs. Which then inform the structure of your internal deployments. So, recognizing that the decisions you make in one end of the stack and ripple through completely and that they're not what we hold as dear these concepts of separation of concerns and complete and total isolation of layers. That really is false. But what it does is it enables us to change the individual layers to more – ironically the reason you want to have separation of the layers is so that it more easily lets you change them provide a more integrated experience. So, I think that's a long way of saying that I think frontend developers are more aware of what's going on in the backend and that they might be drawn into the backend. And the same thing goes for backend developers, realizing that the decisions they make are affected by the frontend and affect the frontend. And so, I think there's been a dawning more of a collective responsibility for design, for operations, for developer experience. And I think it's great. ROB: Yeah. And to [think back off] that. So, I was kind of wondering where the controversy was, because that's kind of my exact answer, too. Where do you gravitate towards on? The difference of mindset is like, if you've got a problem and let's say you're a full stack developer, so whatever that manes, but if you're given a task where you have to implement the frontend and the backend, where you gravitate towards first I would say is how you would attack solving a problem. For me, I would immediately go to the frontend and see how a user would immediately interact with it and then work out that way. Mostly also because I like having a tight feedback loop. And the frontend provides that nicely. I can change something and immediately see the difference there. And in the backend you can spend a little bit of time, unless you're TDD which you should be – you can spend a little bit of time piecing together things and figuring out the architecture. And then you'll have something that you could show for. It's just nice to see UI get thrown on the page. And it's real and you click a button and something happens. That's a really nice feeling. And that's kind of where I gravitate towards. And if yeah, if I had to attack the problem, I'm going there first. I get the endorphins from it. [Chuckles] CHARLES: Right. So, if we want to add controversy to the other side, because you got to always be controversial, right? So, we said the really blunt horrible oversimplification is that people who are more naturally empathetic gravitate towards the frontend. You could also say that from the other perspective, people who are more internally validated will gravitate to the backend. Because if you need to get those endorphins from working on the frontend, basically what you're saying is, “I want to put it in front of people and get the feedback from those people and know whether it's good or bad.” Whereas you could say, “Actually, I don't need validation from other people. I've got this concept of this architecture that is going to be the ideal thing. I know it. I've got this vision and I'm going to chase it, regardless of what's going on.” I think you can more readily do that on the backend because you're not as open to the feedback of a whole bunch of different users. And like I said, I think those are gross oversimplifications. ROB: Yeah. I agree. And so, as a devil's advocate towards the backend developer that is less empathetic, I think actually some of the best backend developers I've ever worked with were the most empathetic people ever. Because they know that software is for humans. And humans are going to be consuming the API that they build. And they take that into consideration. CHARLES: Absolutely. And I actually think that empathy pervades good software development just wherever you find it. Because yeah, someone's going to be using your APIs whether it's software as a service or it's a library. If it's software as a service, it's got to be easy to work with. It's got to be performant. It's got to have a nice command line. And so, you have to be thinking at all times, “What is it like to use this software? What is it like to consume it? What is it like to link it into my library? What is it like to call it from a web client?” So yeah, I think you're absolutely right. I think it's actually one of the great virtues of great software developers. SAM: Yeah. So, something that I've always kind fo had a question with, especially coming from that bootcamp setting: is there any sort of tribal separation between what you would consider a frontend or a backend developer or even full stack? Like, “I'm better because I understand data layers than I understand how a button fits on a page,” that sort of thing. CHARLES: I would say absolutely yes. If you at the, just do a quick poll of the Twitterverse, it seems like people tend to hang out in little circles of similar interest. So, there's definitely a bunch of people that I follow that are always tweeting about backend stuff. ROB: Yeah. Twitter is ‘build your own echo chamber'. CHARLES: Yeah. And there are people who I follow that are tweeting nothing but mostly frontend – when I say frontend, I mean frontend of the frontend. Well basically, from CSS down to nothing deeper than React components. And then there's people who are talking primarily about the backend of the frontend. So yeah, I do think that people – there are clearly, there are different areas of the stack and I think that people do tribalize around them. So, the question is: Who are the border agents? Because always cross-border trade. Any time you have borders, there's an exchange of ideas and information and things that happen at those layers. And actually, one of the things I'm really curious about is: How does that happen? ROB: Yeah. You might have the best perspective on this because you did – you were using Java Swing back in the early 2000s building UI. And that's like backend things. And then you had a lot of experience with Rails and you've moved into the JavaScript world. And the thing that I've noticed with frontend is, we're applying a lot of concepts that existed on the backend. And we're moving all that complexity into the frontend. And that's kind of where that fuzzy line of, “Are you the frontend of the frontend? Are you the backend of the frontend?” comes from. And it's interesting that – like this is going to be another podcast. We'll end up talking about this – but we have – MVC lives on the frontend. Like in React, your C is a container component. The controller is a container component. The model is probably your Redux layer if you have it or if you're using micro-states. And the view is your components, your view layer components that you're rendering down. So, these concepts are moving from the backend to the frontend. We're just kind of renaming them and making them our own concept. But they're there. So, how does that knowledge sharing happen? Is it really just a mass exodus of backend developers interested in frontend developers now? CHARLES: I don't know. I think it goes both ways. So, I can only speak to my own experience. And that was when I first started doing frontend development was back in 2003 I think. So almost 15 years ago. We were building a touchscreen interface for an electronics vendor in the UK. And it was just so much fun, because we were getting to build these interfaces that literally looked like nothing else. And you could touch them and you had support for rudimentary gestures and there was no keyboard. And basically the only interface was your fingers and like a price scanning gun. And everything, all the entire UI had to be developed out of that. And so, it was such a novel system and it was so fun to implement that I just gravitated towards it. I think the thing that was particularly compelling was it was very interactive UI. It was high-touch, literally. This is a clerk who's sitting there using this thing as rapidly as they can to sell stuff and move people through the line. So, it was like a unique problem. But the thing that was cool about it was I realized so many – we kind of already touched on this in this conversation – so many things that I had learned on the backend applied there. And there was in order to provide that experience, there had to be a pretty complex machine behind it. And that was fascinating, that machine. And so, we were able to apply a lot of the concepts. And so, in that time on the backend I'd just come off working off a backend that had basically an event bus – we would probably call them microservices now, although we didn't call them that then – were coordinating based on all these events. And that translated over into the frontend really, really well. And I remember using a lot of the techniques for exception handling – doing it on the frontend and doing a lot of the multithreaded stuff that we were doing on the backend, trying to reconcile that with the frontend and really understanding. So, there were all these analogous concepts. And it could be – so, there was an original exodus of backend development, for me personally. And so for me, it was like I felt like I moved onto the web pretty much exclusively in 2006. And it was a good – I would say 2013 maybe is when web UI development finally caught up with Java Swing. Like it was like, that whole time I was like, “I'm using the web because it's an awesome deployment platform. And it's got all this great stuff,” but man, the developer experience is not as good as like a stateful fat GUI was back then. And now, I would say it's actually surpassed quite significantly. But now, I think there are a lot of people maybe who either – I think there's a casting back of people who are in frontend development and casting back to the web. So right now, my love affair with immutability and functional programming started by using a Clojure web framework which borrowed a lot of the ideas from Clojure. So, I guess there was an exodus there – people from Clojure wanting to develop apps, wanted to have their goodies. But then I found that and I was like, “Whoa, that's awesome.” But then that really set the hook for me. And so now, I try and go back to the well, the backend well or the shared computational well, as much as I can. So, all of that stuff of basically discovering all this functional programming stuff, this highly formalized functional programming, all came from saying, “Wow. I got a taste of that. Let me get more.” So, it's very much like trying to run through the village of the backend and ransack the stores and take as much as I can and bring it back, because I know that it's good. ROB: So earlier in the podcast, you said that you were a frontend developer. You described yourself as a frontend developer. And that's funny, because I actually consider you a full stack developer. It's because you can jump in on the backend and sling any kind of code as well as anybody could, and then you can also do the same thing on the frontend. So, the question I have here is: Do you think actually someone that truly is a full stack developer – and we can define that in a second – but do you think they're at an advantage here, because they have all that experience? You can answer yes. But why? Why do you think they have an advantage? CHARLES: I think they have an advantage in the same way that a person who's bilingual has an advantage. So, if you are living in North America, it behooves you to speak Spanish because you are now open to an entire set of markets that were previously closed to you. Well, not entirely closed but like you can trade with Mexico and you can trade with South America. You can buy and sell goods. You have access to yeah, emerging technologies that might – something special might be happening in Mexico City, or in Medellín, Colombia, and you're going to have first access to that. And so, if you don't, if you're not bilingual, then you're going to have to go through an intermediary who is. And so, there's a premium: you get to be the middle man so to speak. Or you get to be, maybe that has kind of a negative connotation, but you get to be the broker of new ideas and new technologies. If you can, if you are fluent in backend so to speak, then you can go into backend-landia and you can talk with the developers there and see what kind of cool stuff they're doing. And then you can take it across the border into frontend-land and sell it. And so, that's – I think we're very much first to the gun on – not first to the gun but we're ahead of the pack in terms of functional programming. Because we've seen that. We've seen it proved out and we've now actually started to integrate it into your day-to-day routines. And we're ahead of the pack on that, right? And so, I think that's kind of a keystone analogy for me that I think really, really captures what is the advantage in being a full stack engineer. ROB: So, well how do you define a full stack engineer or developer? What things do you have to possess to actually claim that title? In order to be a full stack developer I personally believe you have to know, you have to be comfortable with CSS. You can hate it. You can yell at it. But I think you have to be able to put together a layout if a designer gives you a comp in order to claim full stack. And in the same token, if you're a full stack developer and you mainly came from the frontend, you should be able to implement backend features. I don't actually have – so, I'm strong in the frontend, not so strong in the backend. What would be the analogous of CSS in the backend? Would it be like mocking your controller actions? [Chuckles] CHARLES: I would say you should have a familiarity with HTTP and REST, would probably be the equivalent. Kind of like a foundational technology that just every single technology is oriented around it, or most. Regardless of the protocol you're using – there are things like GraphQL which are not really RESTful, although it's a blurry line there – but they're still delivered over HTTP. And so, understanding things like CORS, understanding the things that are going to be universal to APIs. ROB: I think a lot of people try to claim full stack. I try to claim it. And I know I'm not. I can put together a pretty crummy Rails API on my own for personal projects. But I'm not going to be the one that's setting up indexes on a database to optimize a database or anything. I think that does come in time, but you have – borrowing from Brandon Haye's talks about career development – if you're a full stack developer, I think you end up having to be in the industry for a long time. Longer than what we consider a senior developer these days, I think. Because you could be a senior developer and be specialized. And we see that all the time, and that's okay. But in order to amass that knowledge and experience, I think you have to be in the industry for a long time and done those things a couple of times to really know. CHARLES: Yeah. I agree. So, can I add something there? To continue the analogy of being full stack is like being bilingual or multilingual. I go back to those analogies a lot because that's what I – linguistics is what I studied in school. But when I was studying linguistics, one of the things that was going on there was they were kind of redefining what… ROB: That makes so much more sense of your vocabulary. [Laughs] CHARLES: What it means to be bilingual. There was kind of a reorientation of that definition that was going on while I was in school. And that was being bilingual doesn't necessarily mean you're able to converse about poetry or like deep technology or give speeches in another language. It really is, it's as spectrum of… ROB: Ooh, that's quite interesting. CHARLES: The definition of bilingual is like: Do you use another language in your life? So, if you are let's say someone who is a recent immigrant to a country and let's say you've got less than 1,000 words but you're using them to buy groceries, to pay bills, to do things like that, then you are bilingual. Bilinguility is not an exclusive club. It's really, are you actually using a language? So, if… ROB: Ooh, so that actually gets my wheels turning. Does that mean that you can have a junior full stack developer? Because like, if you just merely speak the both languages, technically by that definition, I jive with that. You can speak two languages. You are bilingual. You can write in two different languages. You might be full stack. But does that mean in the software industry you are a junior full stack developer and then as you go on and get tasks that are full-stack-like, you get better on both sides? Or is it as an industry, we really have to specialize? CHARLES: To start on the first point, I think that if you – let's just restrict it to one language – if you're doing JavaScript on the frontend and using Node.js on the backend, if you're writing Node code you are I would say by definition, and certainly by the definition of bilingual that I just proffered, you would be considered full stack, a junior full stack developer like I was saying. And I think that it's just been my experience that as you go on in your career, you will cross multiple layers of the stack just because you can't keep your hands off. If you have an ownership mentality of, “Here's this thing that needs fixed,” and it's on the backend, you know what? I'm going to go ahead and learn a little bit of how to do this, because I need it to work with the thing that I'm working on. ROB: Yeah, that makes sense. CHARLES: You will just naturally be drawn over borders throughout your career just because someone's got to do it, to do the thing that… ROB: You got to solve a problem. CHARLES: Right, when you've got to solve a problem. So, I think that people become more and more full stack over the course of their careers. But that said, I do think that there are clearly areas where functional specialization is absolutely a requirement. Like if you say, “You know what? We've got this API and we need to support 600 queries per second and we've got this huge, crazy deployment…” ROB: I will not be your person to do that. CHARLES: Yeah, exactly. That's something you're very much – that is something that you're very much hiring for. And you want to be hiring for like you said, someone who has the skill and someone who, that's what they want to do. ROB: Yeah. If you need better state management and rendering performance and testability on the frontend, I'm your person. [Chuckles] If you need me to scale your API, to handle hundreds of thousands of requests a second, nope. [Laughs] So yeah, I agree with the specialization. So Sam, I want to know. Has this conversation swayed you any way? Are you more interested in being full stack or are you leaning to a side more? [Chuckles] SAM: So, I think full stack is, it's as much about skillset as it is about personality. So, like you were talking a little bit earlier on how someone who's frontend might have more empathy towards the user. And someone who's backend might have more empathy towards the computer and the developer, rather than the end user. I think someone who's full stack has to have a wider range or empathy so they can empathize with all rather than just one or the other sets. I think personality-wise, I'd be a fine full stack developer. But I think professional-wise, I do gravitate towards the frontend because that's just how I am. As a person, I like to see the visual rather than the concept. I do a lot of painting. I do a lot of drawings. So, I'm a very visual person. So, it's really helpful to me, especially for someone who's junior and who's still learning and who came from that bootcamp experience, to be able to see what I'm changing. So, I think it does kind of go a little bit into experience as well. So, I think over time when I start touching on maybe some more backend work and I see some stuff that interests me, definitely I'll gravitate towards it, just because I want to learn. But I don't know that it's like an intrinsic quality, you would want to be full stack or be backend or be frontend, you know? ROB: That's interesting. Yeah, I agree with that. CHARLES: Yeah. That is actually a really interesting point. Because I think what I heard you say there is that when you have – software is a hidden world in many ways. You talked about it in the beginning of the conversation, the areas of the site unseen. Like you know it's there but it's hidden from view. So, there's a certain amount of vision that needs to develop. So, you kind of internalize what software, like this hidden software, looks like. So what does a deployment of some load-balanced microservice clustered thing look like? Most people would not be able to answer that question. But the more time you're exposed to it, the more it becomes burned into your inner vision. ROB: Mental model? CHARLES: Yeah. You develop a mental model. Your brain literally wraps around that. It takes time. But on the frontend, especially if you're a visual person – but I would say even if you're not – I think the same would apply to someone who's using assistive technology. It's still something that you can feel with your sense and you don't have to develop a sixth sense to perceive it. So, there's literally a problem of perception there. And so, maybe frontend work is a great on-ramp for everybody, because it's so perceptual. ROB: That's exactly why I picked it. I was trying to do iOS dev before. And I could not do it for those reasons. I didn't have that nice tight feedback loop of even though it was UI, in Objective-C you had to construct your UI and buttons out of Objective-C. Unless you wanted to use Apple's Interface Builder and that was, meh. Everybody built it procedurally. And what I loved about HTML and CSS, was I could instantly throw something on the page, attach some CSS to it, and see it change immediately. And that felt so good. [Chuckles] CHARLES: Yeah, yeah, no. And it's important to get those really tight feedback loops, especially when you're starting out. ROB: So, if we had to tie this back together, did we decide that there is a distinction between frontend, backend, and full stack? And if there is, or isn't, why? SAM: I think there's like… CHARLES: I don't know if we've… go ahead. SAM: Kind of a distinction. But it's also really fuzzy, I would say. So, if I'm going to explain what the difference is between frontend and backend to someone who maybe isn't a developer, I would always go back to a video game. So, when you see your guy… ROB: Ooh, this is interesting. SAM: Yeah. [Chuckles] When you see your guy running around and you're like, “Oh, this game looks really good. I'm really into these graphics,” but do you ever think about what it takes to save the game or what is actually being saved when you go to save it? So, if you're more interested in making the guy run and making the guy and the environment look good, you might gravitate more towards frontend. But if you're really interested in saving the data, like well what is being saved when I hit ‘save to this slot' or whatever? Then you might gravitate more towards backend. ROB: Or like the network layer of the online multiplayer? SAM: Yeah, yeah. ROB: [Laughs] That's interesting. I've never used video games as an analogy. I always go like, you know, if you're a frontend developer, you know that button that you click? That's the button I create. And then the action that happens is usually what the backend developer does. I think I like the game analogy better. [Laughs] SAM: I think the game analogy is something that most people can grasp. And most people can grasp. Like I'll tell people the difference between frontend and backend if I'm talking about Facebook. Like what I do for a job is I'll show you everything that's on your Facebook page. And when somebody, or the backend is somebody who makes all of that come into play, kind of, you know? But I think video games are easier to conceptualize that abstraction of data being saved rather than trying to explain the intricacies of Facebook to somebody. ROB: [Chuckles] CHARLES: Yeah. No, I like that. I like that. So, I guess if we achieved consensus, would we say that these do exist as broad functional categories? And a full stack developer is someone who will move in between those functional categories through the course of their career. And that generally, the trend is that the longer the career goes, the more you will do that. ROB: Yeah. I can get on board with that. I'm going to use your career as like the guiding post of that. The way you just explained it, it kind of made something click for me. You describe yourself as a frontend developer. And I think in our industry with software development and the way teams work, I think you end up specializing no matter if you call yourself a full stack developer or not. Because I do think you're a full stack developer. But you've mainly been working on frontend for the past what, three years, four years? So, I think it's okay to call yourself a frontend developer. But you are a full stack developer. But as you specialize around and amass that knowledge, that's where you become that full stack developer, right? And so, at one point in your career, you were a backend developer. And then now at this point in your career, you're a frontend developer. But now you have those experiences and you can draw on both of them and apply them across the fields, right? That's super interesting to me. So, maybe it is that you have to specialize in order to achieve the full stack. And you're never truly a full stack developer in your role. But it's possible, depending on the team and the team dynamics. It's just interesting that you can be a full stack developer, also at the same time be specializing as a frontend developer at that current time, right? Yeah, that's interesting. CHARLES: Alright, so it sounds like consensus achieved. Let's all virtually hug. [Chuckles] ROB: Send the hug emoji. CHARLES: Alright everybody. That is our show for today. We are The Frontside and we build software that you can stake your future on. We would love to hear from you, especially if you have an idea for a future podcast topic. Any news that you think is something that we should discuss, even if it's not the theme of the whole episode. And we will see you next time. As always, you can give us a shout on Twitter at @TheFrontside or just drop us an email at contact@frontside.io. If you enjoyed today's podcast, it was produced by Mandy Moore, the inimitable @TheRubyRep. So thank you, Mandy. And see you all next time.

Frontside alum and original podcast host, Brandon Hays, makes a special guest appearance to talk with Charles about the evolution of The Frontside as a company: where it's been, where it's going, and more hopes, dreams, and goals for the future! Transcript CHARLES: Hello everybody. Welcome to The Frontside Podcast Episode 100. Here we are. Episode 100. My name is Charles Lowell. I'm a developer here at The Frontside and I think it's safe to say, your official podcast host. With me to celebrate the 100th episode, he was also here a few episodes ago but also was here on our first episode I believe, is the [inaudible] Hays. Hello Brandon. BRANDON: Hi. CHARLES: Welcome back to the podcast. BRANDON: Actually, are you going to light your trainee badge on fire now in a bucket, in a ceremonial pyre? CHARLES: I live in New Mexico, so I think I'm going to just after this, grab my shotgun and give myself a 21 gun salute. Just in my front yard. BRANDON: There goes old man Lowell again, with the shotgun. CHARLES: I'm just going to [gun shot sounds] in my own honor. BRANDON: I was at the Alamo this weekend, actually. And I don't know if it was just because it was fiesta in San Antonio but they had a demonstration, like a musket firing demonstration where those things are basically little cannons. They're just small cannons. It's very interesting. They're very loud. CHARLES: Yeah. They're small, handheld cannons, yeah. So wait, were you – what is fiesta? Now, as someone who grew up in Central [inaudible], I feel like I ought to know this. BRANDON: I don't know. We found out by accident because we were planning a weekend to go hang out and get drunk on the riverwalk and we took our families down with some friends and then they're like, “Oh, it's fiesta,” which is like a 10-day celebration of the history and establishment of San Antonio – which I did not know is a 300-year-old institution. So, it's like one of the oldest things in this entire western United States. So, it's pretty neat. It's different. It's weird. It's like 90 minutes from Austin. There's nothing in Austin that's older than six months. Every six months we must demolish something and then build a condo skyscraper in its place. So, it's kind of neat to be in a city where it has – walking around the Alamo, I'm realizing, “Wow. Setting aside any of the historical significance of Texas independence or whatever, this is just like a really interesting very old building. This is hundreds of years old in an area where there's nothing that's hundreds of years old.” So yeah, it was pretty cool. It was a good weekend and we got to see muskets being fired. And we saw a doctor gross my kids out by talking about the medicine of the day, in full costume and showing all of the procedures and threatening my kids with amputation. And it was a good time. We all had a good time. My nine-year-old thought it was the coolest damn thing he'd ever seen. CHARLES: Really? Did the have bloody saws and everything? BRANDON: Oh, yeah. CHARLES: Was it like a reenactment of 300-year-old surgery? BRANDON: It wasn't a full reenactment. But it was a graphic description using the tools of the time. CHARLES: Wow. BRANDON: Highly recommend, check out the Alamo. Super fun. CHARLES: That does sound really cool. BRANDON: I did not expect to have a good time and it was a good time. CHARLES: Yeah. Yeah, I know the whole reenactment with the musket firing is fun. And it is, it's actually an incredible building. Although there's been a big kerfuffle about something about how they're going to preserve the lawn. But I haven't really followed that too much. BRANDON: Yeah. Yeah, I don't care about the lawn. I care about – no offense, lawn, if lawn is listening. This is not weird, how Stanley broke our brains with the word ‘lawn'. CHARLES: That's true. BRANDON: Yeah. He broke us real good. CHARLES: Yeah. I can't see a lawn without a beard. BRANDON: So yeah. So, life has been pretty good, man. Let's see. I left Frontside September, October. CHARLES: 2016. BRANDON: 2016. CHARLES: So, it's been months. BRANDON: 18. Yeah, thereabouts, right? So, I assume that nothing happened since then and if I came back to The Frontside now, everything would be exactly as I left it. My posters are still up in my room. My Bon Jovi poster. You left my bed just as I made it, like kind of unmade. Everything is just preserved as a shrine to me. CHARLES: Pretty much. I mean, we did give away the mics to Goodwill. BRANDON: No. CHARLES: We actually did not give away those mics. BRANDON: I never even got to use them. CHARLES: I know. Well, you know part of the problem is we don't even get to use them that much either. It looks really cool and it plays really well, like our podcast studio. But you know, I'm now spending 75% of my time in Corrales, New Mexico. And at any given time, people are either working from home, or working remotely. So, a lot of times the podcast room tragically does not get used. But it looks so cool. People come in there and they're like, “Wow, you guys must be really smart and technical people.” BRANDON: I realize this is probably a rote stereotype at this point, but I am assuming the only reason that you moved is that you are dabbling in the production of meth. CHARLES: Pretty much. BRANDON: It's like, I want to learn a new trade. Programming, it's just – programming, how interesting does it stay honestly for 25 years? CHARLES: Right. Yeah, and you know, we've got some good techniques. Continuous integration, deployment, things like that. Test-first. These are things that can be applied to different verticals. And I was looking… BRANDON: [Laughs] We ship meth to production on the first day. CHARLES: Right. [Laughter] Exactly. So, I figured it was a market ripe for disruption. BRANDON: [Laughs] It's probably true. So yeah, I wanted to ask you about that. You all kind of scattered to the four winds in some ways. You have Elrich in Boston and you're in New Mexico most of the time. CHARLES: Joe is in [inaudible]. BRANDON: Oh yeah, Joe moved to New York. CHARLES: Yup. And honestly, the traffic is so bad in Austin that I'd say 50% of the time, people stay home rather than drive into our centrally-located office. So, that's actually something that we're struggling with right now because the bulk of the team is still in Austin. But the office space is underutilized. Our team size now, we have eight engineers. And five of them are in Austin. Our other staff is also in Austin. So, what do we do with the office? It's a big question. BRANDON: And that's quite a cultural change, too. Because when I was there, we would tell people, “We want to be able to do remote someday. But we just don't know how to get into that culture to change the way that we do our meetings and change the way that we do standups and coordination and communication.” I didn't feel like we had the tooling at the time. So, something – I knew that at some point there would be probably a forcing function to basically catalyze something to allow that to work. And I'm curious to know what that process was like there. CHARLES: I wish I could say that there was a process other than experiencing the force of the forcing function and then being forced into it and then just kind of dealing with it. I have not taken a poll of the other remote employees of which now I am one, at least for the time being. So, I don't want to speak for them. But it was less painful than you might imagine. And the reason is because – and it's one of those things you actually gave me this analogy back, probably three or four years ago and I love it – is sometimes you're hanging off of a precipice and you don't realize that you're toes are two inches off the ground. And then all you can perceive is the precipice and you feel the weight of your own body concentrated on your fingers gripped to the ledge. And you don't focus on the fact that you're actually, the fall is only two inches long. And that's kind of what we experience with the remote culture. Now, I don't want to say we were Pollyanna about it and didn't realize that this was the step that we were taking and making sure to check in with the remote employees. But one of the things is our communication styles were already very asynchronous both for our client work, for our internal work, using mostly Slack and GitHub pull requests and issues – certainly for the development portion, very little changed. What we didn't realize is that because of our involvement in open source, we were already acclimated to a distributed work style. We just didn't really realize it. We didn't have to change much. I think where we have a lot more work to do is kind of integrating people socially and making sure that conversations don't happen that aren't available for other people to consume asynchronously. So, if you're having some architecture problem and you're sitting next to somebody, you'll take that avenue rather than let it play out in chat or over email. And there is definitely a certain portion of that, but I think we still do a lot of pair programming. That's still our major mode. I'd say 75% of our code gets written as people collaborating. And so, while those in-office discussions do happen, the ramifications circulate rather quickly. And most of those are in the context of people pairing inside the office. Does that make sense? BRANDON: Mmhmm. CHARLES: So, I don't think the office and the physical space were as much of a bottleneck as we thought they might be. And so, because of the – a lot of people did work from home already because of the traffic. And we were involved in open source. And our communication with our clients is usually – we don't currently have any clients in Austin. So, that's all to say that the transition was actually quite natural. And I think there's some strong analogies between collaborating in open source and having a remote culture in your office. I think what we need to get better about is making sure that we get the team together at least twice a year, everybody together. Making sure that people are able to understand their priorities and get to circulate around and get introduced to a bunch of different people. And yeah, I don't know. There's definitely a lot of work to be done on the non-development front. BRANDON: It's interesting. The agile approach to things is to try something. I'm starting to think the agile and the scientific method are related where it's like, “Here's a hypothesis. Here's the experiment. Here's what we think we want to learn,” and then you learn it and you take the next step based on that information. And that failure is an option. I think that's the point of agile, is to make failure safe because it's small and you're guaranteed to learn from it. Like, the point is to learn. And so, I really, I'm starting to think that those are just basically the same thing. That agile is like the application of the scientific method to product development. And it sounds like you're being agile or experimental about your work. And the trick is, like any scientific discovery, the trick is in coming back around to it and analyzing it and deciding whether this was successful or a failure based on feedback and finding what the measurement was that you were trying to improve. So, the lesson there was, “Oh, people become disconnected from each other. We need to gather everybody for an all-hands periodically.” We didn't use to have to do that because all-hands was every week, at least. CHARLES: Right. Yeah, everybody was constantly – there was a constant chatter and you could just kind of, the context was just all sitting at that one table, in that one room on 38th Street. And all you needed to do was dip your ear into that pool of context and you're set. Whereas that's just not an option right now. So yeah. I think the danger with agile is not being concentrated in your experimentation. I think what gave us our fear about saying we're going to do remote work – because I remember we always talked about it. We danced around the issue – was are we going to lose who we are? We have a set of way that we do things. And there is power in kind of sticking to the framework of the way that you do things. Because you understand it and you know it. So, when you're pushing and you're experimenting, being able to say, “We're going to – push and we're going to focus on this one area and we're going to iterate on it and we're going to keep everything else static,” it's going to be the wall that we can walk along. But we are going to push in this area. And so, I think the dangers of you doing that in all the areas of your business or all the areas of your project, you're iterating and refining, nothing ever gets done. And so, it's kind of like once you get to some ground that's solid, when you do start iterating it, you start introducing instability. So, when you go remote you have to start thinking about remote work, whereas we didn't have to think about that before. We were essentially, the feature of saying that we were a one-office company and an on-site company is we didn't have to think about that problem. BRANDON: One thing that you were just taking about is this idea of concentrating so that your experiments are happening one or two or maybe three at a time instead of trying to run five experiments at a time. And yeah, there's another danger I think in agile of seeking local optimization where you're basically like – it's like taking a bacteria and running it through many, many, many iterations that's targeting one thing and it mutates into this weird thing that only does this one thing. Or a dog breed that the whole – did you see that, I don't know where this came from but there was some scientific findings that there was a dog that was bred in ancient prehistoric times that was bred to turn a spit to roast meat over. So, they bread a dog that the whole point of this dog was to turn a spit so that people could roast meat and go to sleep and let their dog serve it, cook for them I guess? CHARLES: Wow. BRANDON: That's pretty impressive. CHARLES: I would say like their dystopia is in the past. Or certainly canine dystopias. I guess we live in a canine dystopia. BRANDON: Not in my house. CHARLES: Not at your house. BRANDON: This place is known as a canine paradise. So yeah, I think that's a really interesting point though, that limiting the number of concurrent experiments so that you can actually respond to them in a meaningful way instead of just being like, “Wow, we learned a bunch of stuff we're doing wrong. Anyway, back to the grind.” CHARLES: [Laughs] Yeah. BRANDON: Back to sucking at everything. CHARLES: Right, right. BRANDON: That kind of feeds into a lesson that I have learned very, very, very recently in the interview process for looking for my first real job in over a decade. And that process is very humbling. And one of the humbling experiences was being rejected for a job from a very notable larger former startup here in Austin. And their interview process is really buttoned up. I got really deep into the interview process and at the end of it they're like, “Oh, you're not technical enough.” And it was really, it was like, I don't know. It was hard for me to process at the time but it's super easy now to look back and go, “Oh, I was definitely not a fit for that type of job if being able to write JavaScript on a whiteboard without the aid of Google to solve problems and refactor code is like a fundamental part of what is valued in a manager there.” That's just not going to be me. But one thing I – and it wasn't a colossal waste of time. There was a ton of time and energy I invested into that specific process, but I actually derived a ton of value out of it. Because every person I met there was focused on the same thing: their culture of making experimentation inexpensive so that everything there is framed in terms of an experiment. What's the experiment here? What's the hypothesis? What's the expected outcome? How soon can we get to a place where we can validate that outcome? So, it's kind of like everything is really lean. And yes, it does – like I asked, “What's the dark side of that?” and it can lead to optimizing for a local maximum. So, you have to pause every once in a while and reflect at a larger scale. But it changed my attitude about a lot of stuff. I tend to walk around fearing failure. That's more my speed. I'm afraid of failing because failure can be catastrophic. But that's because I take big swings at stuff. When I go give a conference talk, it better be the best conference talk of my life. When somebody's like, “Oh, that was the best conference talk I have ever seen,” I'm like, “Ah. I'm so glad you said that because if you'd said literally anything else I would have collapsed internally.” You know? The stakes are so high for everything. And making it safe for yourself to fail by treating things like an experiment and working with my teammates. And so, two or three scenarios over the phone in a week when I was managing the team at my last company, somebody would bring something to me and I'm like, I instantly went to all the reasons this probably won't work. “Here's the problem with this.” And I thought, and I immediately turn around and went, “Wait a minute. Bring me a hypothesis and the experiment and how we can experiment with this thing.” And he's like, “Well, we could try this next week and we'll know whether or not this is a good idea.” And we tried it the next week. It was like organizing an architecture team because we were waiting to hire an architect. And the results were mixed for reasons I won't get too deep into. But the fact was, it gave us the freedom to try things. And I'm trying to carry that spirit around with me now. It's been really eye-opening. So, completely like, just a 4% alteration in the way that I think about problems, but it has the ability to dramatically alter the trajectory of how I solve things in the future. CHARLES: So, do you include now inside the planning process experiments? Like, a certain number. BRANDON: Absolutely. CHARLES: So, the typical “enterprise” development is we have our features, we're going to do them in this order because they're this priority. And then agile comes along and it's like, “You need to take these things and you need to break them up into small chunks so that they can be accomplished in small time slices,” so that you don't basically bark up wrong trees. Or explore [inaudible]. BRANDON: Yeah, but that's almost like a stupider version of waterfall. CHARLES: But exactly. That's exactly my point. Whereas the problem is, there's no avenue for experimentation in there. Rather than saying the entire team is marching in this one direction that meanders around and focuses in on the local maximum, which hopefully is relative to the market landscape is the absolute maximum, saying, “We're actually going to be marching in one major direction but we're going to be sending out scouts at all points.” If you were actually – I've actually been reading a lot of ancient military history. And It's just insane that an army, or even a detachment, would go all in one clump. They're constantly sending out people. Information is really, really, really important. BRANDON: That's an extremely, extremely good point. I've actually – it's so funny, because I've used a very similar description where we are trying to chart a course to this ocean of opportunity somewhere. And we can't just send the whole team in a direction hoping that the ocean is in that direction. We have to have our Lewis and Clark. Somebody has to be the cartographer. Somebody has to be the explorer. And that means that there has to be a little bit more freedom for those explorers. I don't yet know how to translate that into software terms. I just know that that's a collaboration usually at most companies between product and development. That product is doing some of the exploring of the space and then development is doing some of the exploring of the technical capabilities and possibilities there. CHARLES: So, you see it. What's interesting is you see it in product planning, kind of in the large, with the waterfall. You see it in huge organizations. They have a research and development department. And I wonder if agile kind of saw the Balkanization of your feature set into very small component parts. Can you take the exact same principle and Balkanize your research and development and integrate it into micro-iterations? We have this R&D but we're going to integrate it into our day-to-day and week-to-week process. BRANDON: I think that is a really noble goal and I think I see some people making progress toward that. The company I interviewed with does it almost to a pathological degree where there is a point of diminishing returns where you're sort of bound to this process of experimentation. And at a certain point you can only achieve incremental results. CHARLES: Some of these problems, you just need to be able to think about them for a long, long time. I actually didn't read, I actually didn't see the talk. But everything from the title, Rich Hickey's ‘Hammock Driven Development', just that title resonated with me so much. I was like, “Yes,” because sometimes you just need to be in the hammock for six hours at a time. Or in the shower. Or hiking. Or doing whatever it is that you need to do to put yourself in a zen state where you're just, your brain is slowly turning its wheels. And it can follow every lead to its conclusion without any interruption. And sometimes that process can take hours. Sometimes it needs to take weeks. BRANDON: Right. I want to kind of pivot on that. Because that's actually one of the biggest things that I've learned in the intervening time since leaving Frontside, which is creating space instead of trying to maximize – one thing that I did when I was at Frontside and then did again at my next place and I'm realizing is really has long-term negative implications is cram as much into a work day, as much output out as possible. I'm very output-oriented. I want to jam as much into my day as possible. I want to jam as much software out the door as possible. And people describe working at Frontside while I was there as one of the most intense work experiences they'd ever had. Literally, I can project that, literally just from my own intensity of trying to cram all that stuff. And providing that space for developers to ruminate on hard problems, on some of the harder problems they encounter, providing space for managers, I've learned that a big chunk of what it is to be a manager is to be available. And so, I actually want to write a sign – I was on the fence about doing this but I think I'm actually doing to do this – I have an office and I'm going to write a sign and put it up on the door that says, “If I look busy, interrupt me and remind me I'm not doing this right.” So, creating the space to ruminate or to be available for discussion, people that protect their breathing room sometimes are made fun of, especially in American corporate culture. I walked in and they were just reading a newspaper. What the heck are you doing at work if you're just going to read a newspaper? Like no, this is actually really important time. CHARLES: I think it's, yeah, it's something that I think about a lot. And I know I've shared this analogy with you before. I don't know if I've done it on the podcast. But I saw and I can't take credit for it. I actually saw it at DevOpsDays I think in 2013. There was a woman giving a talk and she was just talking about managing developers. But one of the things that she was saying was that if you looked at a microservices architecture or you looked at just even your operating system, and if your CPU was constantly pegged, you were squeezing out 100% of every time slice, instructions were just flowing through that, you're going to have a very unhealthy, very brittle, very prone to failure software system. If our microservices were not available to actually service requests, and service excess requests, and service spikes of requests, then something is fundamentally wrong. BRANDON: I want to add to that a little bit, because the thing that I noticed in managing a team where I received a ton of pressure to peg everybody out at 100% – and it jived with my philosophy at the time of, “Hey, I'm 100% guy. Everybody I work with is 100% type people. And then, let's peg everybody at 100%. This is a startup. Let's get everything going,” and I realized very, very quickly that if you don't preserve a little buffer, 20% buffer in that level of intensity, there is no ability to share resources. Everything is now a silo. So, if you're going to peg all your CPUs out, part of that thrashing is that there's no time for people to share things with each other. And people become very protective over their little silo all of a sudden. And it causes us – it's actually like the first stage of a catastrophic cultural collapse if everybody's pegged out at 100%. And literally, just dialing down the intensity is often the only thing that's necessary to get people to feel comfortable sharing some of their time with each other. You do a really good job of that with the lunch and learns. You mentioned that y'all are doing better thoughtful lunch and learns and stuff like that. It's one of those forcing ways that you can force that and say, “Hold on. Stop the development and do some stuff where you're actually sharing things with your teammates.” CHARLES: Yeah. And we do that. My biggest concern is that that actually increases the intensity. So, one of the things we've done is we used to actually be very formal about our lunch and learns. It's like, “We've got to generate content and put it out on the web so that people can see us.” We backed away from saying – we're not going to do them as often and make sure that people can actually do them. Yeah, making sure that people don't feel overwhelmed by, “I've got a lunch and learn coming up.” The point is to share something that you're passionate about and maybe introduce some really cool ideas to ferment in people's head. Rather, that's kind of the goal. There are certain things that we do very much feel interested in generating content. But I think, we've kind of been dancing around the ideas of distributed computing and IoT and what are some of the others? BRANDON: If you say blockchain, I'm going to just virtually punch you in the face. CHARLES: [Laughs] I actually didn't. Did I say blockchain? BRANDON: No. I just was waiting for you to say it. CHARLES: Okay, no. I haven't. Well, because that's – but it is distributed computing in Web 3.0, right? These problems – and we're actually going to be podcasting about this next, so in two weeks you can tune in to listen to us talk about blockchain but in the context of distributed computing – and one of the things that we're seeing is now we're starting to pay the price of outsourcing all of our lives to these central services like Facebook and Google and Amazon. And I think now they're starting to build a credible and more mainstream movement to wrestle back that control and say, “What would it mean to have software as a service that wasn't actually dependent on some central thing?” What would it look like to have Slack where it's Slack that looked like email? Where everybody had their own email server, maybe not a bad example. But you've got an email at Gmail or Microsoft or Yahoo or your company-run that's big enough its running its own Outlook client or something like that. Email is actually a really great example. Now probably people are going to crucify me for saying this, but I think it's actually a good example of a distributed system that's worked well. I own all of my email. All the messages that you send to me, I own, and all the messages I send to you, I also own. But you also own the messages that I send to you. Information is duplicated. And it's fine. If I send you an image, yes it's on your hard drive or it's on your Google Drive. You send a message to me, it's got an attachment, I also have that attachment. But the point is that we can each own our email and we each own our email service. And we can change it up. That's not possible with Slack. That's not possible with Facebook. That's not possible with all these other sharing platforms. All of them are controlled by this one thing. And so, I think that that's something that we've been exposed to through the lunch and learns and I'm actually certainly very excited about it. It's not something that we're going to be investing in immediately. We're kind of dancing around that idea. But that's something that's come out of that. So yeah, we've kind of refocused it on, what is something that you feel good about? But back to the original point, I think that this is something that applies on all fronts. If you have a business where you can't actually take opportunities because you don't actually have people – so there's maxing out at the individual level, filling up people's workspace with client work or filling it up with what have you or having them work nights and weekends. There's individual maxing out but then there's like maxing out of your business. So, if you have – we're a consultancy – if you have 100% utilization or you're shooting for 100% utilization, that everybody is placed on a project, that is a brittle and unsustainable system. BRANDON: I wish you would have told me that 18 months before I left there. There were like two years where we were at 100% for two solid years. CHARLES: Yeah, yeah. We're still at 100%. BRANDON: Yeah. I wonder what would have happened if we'd had a little, if we had figured out how to build in space. CHARLES: Part of the problem – so, here's the thing though. Space, nice space costs nice money. BRANDON: Yeah. CHARLES: And so, that's the thing, is you have to charge more. And you have to say, “We are going to be more expensive than other people.” You have to be dedicated to be at the forefront of a cultural battle, essentially. In the same way that people were with testing, where it was very [controversial]. BRANDON: Yeah. You were with CI. CI is a given now, right? CI is… CHARLES: Yeah, like [inaudible]. BRANDON: This idea was semi-revolutionary when you and I were talking about this in 2012, 2013, that we ship to production on the first day. We don't even start building software until the CI system is set up. The first thing we do is set up Jenkins and tests and get everything, the pipeline working. And now, that's just what people do. By and large, that's how software is expected to be built. And the tooling has really come up around that. But that was an expensive way to sell software five years ago, that, “Hey, this is going to cost more than bringing in Cowboy Bob and having them come jam in your console for 40 days and ship a bunch of stuff that then will most likely collapse and you won't know about it and Cowboy Bob has ridden off into Juarez, Mexico.” CHARLES: Right, with his saddlebag stuffed with your cash. BRANDON: Yup. CHARLES: Yeah, no. So, you have to – the problem is, you know when you pick these battles, you need to be prepared to fight the war of attrition of they're not going to be able to perceive the value for six months, a year, right? You're going to have to ask your clients to bet on this strategy. And it's a bet. And you're going to have to say, “It's going to pay off in six months. It's going to pay off in a year.” And you're really going to start raking in like five years. That's when… BRANDON: Yeah. Try making that pitch to a startup founder that is borderline, that is on the verge of an anxiety attack, and you can kind of just figure out what my last year was like. And the… CHARLES: So, that's one of the reasons we don't really work with startups anymore. They have a five-year plan, but not really. BRANDON: Yeah. CHARLES: They're fighting for their survival. And they're fighting for the opportunity to have a legitimate five-year plan. And so, in that sense, it's maybe not a good fit for the way that we develop software, because you either need an extraordinarily prescient founder who has been through this before, knows the true costs of software development, and is pretty well-funded so that they can actually – because we're more expensive upfront, like a lot more expensive upfront and so sometimes they flat out don't even have the cash. And that's something that you can make a quick, “It's not a good fit,” but then there also needs to be this understanding and an acknowledgment that what you're really shooting for is your five-year dividend. BRANDON: Yeah. It is really interesting, the turn that occurs when a company finds product/market fit. By then it's too late to fix the problems. So, it's really tricky to find the balance of: how much energy do you put into the success case for a company before they have product/market fit? How much time and energy do you invest in betting that this is going to be successful versus betting that if it is successful, hopefully we'll have the time, money, and resources to redo a bunch of the things that we are going to have to apologize for later? And I think that's what makes… CHARLES: Right. Like, where do those two lines cross on that graph? BRANDON: Yeah. Because you and I have both seen startups completely sunk by somebody who was overly focused on building a scaleable architecture in a company pre-product/market fit. That is a common story where an engineer that doesn't understand the business value of what they're doing and only focused on “quality” will absolutely torpedo, they'll chew up your first million and a half of funding and leave the place in just a smoldering pile of ashes at the end. So, it is tricky. It's totally a difficult thing. But I think coming back to your point of being sort of a vanguard of cultural, the tip of the spear on somebody's cultural changes – DevOps would be one. People that were really investing in DevOps culture in 2010, 2012 saying, “Hey, this, automation, is the future of how software gets shipped, maintained, observed, supported.” And so, now it sounds like, so what is your big bet for the future? CHARLES: Boy. That's a great question. There are two bets. One you're going to like, one you're going to vomit. BRANDON: [Laughs] CHARLES: But that's okay. BRANDON: Yeah. I don't work for you. CHARLES: You need to serve, what is it? You need to serve the spiny urchin with the yellow tail. BRANDON: Is that a Sonic the Hedgehog reference? CHARLES: It's just a sushi reference. BRANDON: Oh, okay. CHARLES: Some people don't like urchin. Or maybe they don't like eggs. What it like, the roe that come with sushi. But they're on the same plate. So, I would say the first one that I've been thinking about a lot is optimizing for capacity and being able to handle spikes and not being at 100% both for people and for utilization. I think that's something that is – I don't see how you could have a healthy software development process if people are completely spiked on delivering, heads down delivering features for product. That is something that I'm betting on. Essentially, you could call it the 25% time but it's really about having excess capacity to exploit opportunities as they arise. And then being protective of that excess capacity. Because you can exploit an opportunity. Your CPU has a spike load up to 100%. But then make sure you [inaudible] down to 50% at some point, or 75%. And so, I would very much like to see Frontside have a bench where people can rotate out and they're working on different stuff that are not even client-related. They can recharge their creative tanks. They're not going to be idle. BRANDON: Yeah, I've really come around on – and I really hated this at the time – but I've actually come around to the thoughtbot style of working on a product where – because owning and managing a product and developing it as a side quest, the goal is not necessarily for that product to catch fire and become the world's next big thing and to replace your consulting revenue. The goal is to give people a sense of – think about all the stuff that you've learned in your side projects that you went back and brought to your work. And some of my biggest gains as a developer have come from having a side gig of some kind, some side project that that's how I learned Ember. That changed my life. And I would never have gotten to try it if I was waiting for somebody at work to tell me it was okay to do it. So, it's about taking that permission back for yourself and giving yourself permission to try stuff. So, it could be something like that, or it could be the content stuff that y'all do. Or it could be conference talks. It could be whatever. But the goal isn't necessarily to produce things that have a direct return. It is to create the space to allow people to flex some muscles of creativity that you may not get in your day-to-day work. And that's very difficult to offer to people in any company. Now having explored startups and larger companies, but I would say especially in a consultancy where the exchange rate is dollars for days. It's sort of like when I was freelancing. I could feel every vacation I took draining both real money and opportunity money out of my bank account. That's such a hard, difficult thing to do. And so, you actually have to create the budget ahead of time and say, “This budget is allocated to these things and it's already spent.” Anyway, that's really tough to do. CHARLES: It is hard. BRANDON: If you can exercise the discipline necessary to do that and create the environment for that, I would say you're ahead of 90% of companies in the industry. CHARLES: Yeah. Yeah, so that's something I definitely want to bet on, because I think that's where the best things come from. BRANDON: Okay. So, what's the thing I'm going to hate? CHARLES: Functional programming. BRANDON: Oh, Charles. Okay, I have to stop you. Do you know what I'm doing? Did I tell you this yet? That I am participating. When I told them this, I was like, “Charles is going to have a field day with this,” but I am participating in a Haskell study group. CHARLES: No way. BRANDON: And I'm like four exercises into this thing. I have to do four more for next week. And I'm like, “This is bizarrely easy, actually,” after as much JavaScript as you and I did in sort of a functional style and then learning Elixir. And I was like, “Wait a minute. The case statement is, Elixir just stole Haskell's case statements.” So like, so far I'm not finding functional programming to be onerous. Or anyway, but we'll see when we get to the static typing. But so far, I'm not getting any of that in the earlier lessons of the book. CHARLES: Yeah, the static typing. But the thing is, you can do – it's not 100% necessary. It isn't in Haskell, for sure. But I'm surprised. What inspired you? BRANDON: We have an architect at the office that was like, “Hey, I want to do sort of a functional programming book club.” So, we have a Slack group for FP study group. CHARLES: Are you doing ‘Haskell: From First Principles'? BRANDON: No. That one was a little actually intimidating. CHARLES: Really? BRANDON: Yeah. It gets into the lingo a little early. And we're doing one called ‘Get Programming with Haskell' that is a little more – ‘Haskell: From First Principles' is kind of math-oriented. So, for somebody with a math background but not necessarily a programming background, it's perfect. But for somebody with a programming background that is just trying to understand functional programming principles using Haskell, ‘Get Programming with Haskell' is actually a really great option. CHARLES: Okay. Actually, I have not heard of that one. BRANDON: The stuff that I'm looking at looks just like Elixir. So, it's early. But it's very comfortable so far. CHARLES: Yeah. So, this is the thing. It's all a matter of messaging and marketing. Because I really feel – so, it is like there are a lot of behaviors that you see sometimes in currently entrenched functional programming communities that I think are, well I think they're objectively repulsive. But I think they're also pragmatically repulsive and that they repulse potential community members. But I think a lot of it too is people talk about these things that are, they use abstruse terminology. And they're kind of chattering back and it's very jargon-oriented. And there's just – people operate with a different set of concrete things. So, when you and I are talking, for example we might talk about a Rails controller and that's a very concrete thing. You know exactly what I'm talking about. It's something that you have held in your hand, literally. Remember when we got that Rails codebase that came as a thumb drive? BRANDON: Yes I do. CHARLES: But the point is you knew that this had a Rails codebase on it. There were any number of controllers. And when I say controller to you, a controller is an abstraction, but not really. Once you work with an abstraction long enough, it becomes concrete. And so, part of the problem is just a mismatch in language where people are talking in their world about concrete things, things that you can touch and you can feel and you can exchange and they're very relatable. But from another person's perspective, they're talking about something that's totally abstract and totally opaque and totally what have you. And so, I feel like yeah there's a huge mismatch there. And that's been one of the big bets. The other big bet that I'm making is on this trying to make what is currently abstract to JavaScript and Ruby developers be concrete. And I think that we're going to see type classes like functor and monoid and semigroup and all these things, they're abstract to you now, become concrete over the next five years. And so, that's something that I'm betting on. BRANDON: Check out this – and I know that you have a good relationship with the people that did the other book, but it really does tend to come from more of a mathematical background. And this one actually does speak to people with JavaScript, Ruby, Python experience. Like, “Hey, here is how you will perceive these things.” And so, it's much more approachable. I'm still in the first unit of the book. But having sort of tasted it a little bit, it's like, “Wait a minute. This is actually extremely familiar and not super intimidating.” CHARLES: Exactly. And that was kind of – so, I read the other book. And I think I was also aided by the fact that I tried to learn Haskell probably for five times in the past. And so, I also had the benefit of jumping against the wall with the velcro suit and bouncing off four times. And fifth time, it stuck. So, I had just temerity on my side and a general feeling. But that's definitely – the lesson that I actually came away from reading that book was like, “Oh, there's a mismatch in concrete concepts.” It's using concrete concepts that are concrete to people with a CS background or mathematics background, or people who are brand new. Honestly, people who are brand new to programming who don't actually have JavaScript or Elixir or Ruby or any other thing to lean on, I think that the First Principles book is actually pretty decent for them, too. Because they don't have anything to compare to. BRANDON: They don't have anything to unlearn. CHARLES: Yeah, they don't have anything to unlearn whereas one of the things I took away was I was like, “Oh, man. I'm using semigroups all the time. This is something that I do constantly.” When I'm coding, I might do it eight times in a day. I just didn't have a name for it. BRANDON: Right. They're like design patterns, just at a micro level. CHARLES: Yes, micro-design patterns. Yeah, it's like a RESTful architecture for your code. In REST you only get five verbs. There's five methods, man. That's all you got. BRANDON: Okay, so those are two bets. And I want to cover one more thing because I know we're super overtime. But the last thing I want to be able to say about talking about what we've learned since I left Frontside but I want to put a bow on that. So, the two things that you're betting heavily on are functional programming as a basis for solid architectures in the future, like the work that you all are doing. And… CHARLES: I would also like to say, and this is something – let me just add one more thought. What I don't understand, and this is in no way like, I don't understand people who do the, “Saying goodbye to framework X.” That's not me with object-oriented programming. BRANDON: Often abstractions are like oversimplifications but they're really useful, sort of like Rich Hickey's Simple versus Easy. Like, “Hey, there's a lot of promise with that metaphor. It's a leaky abstraction but it's a useful abstraction.” And Gary Bernhardt's ‘Functional Core, Imperative Shell' is a leaky abstraction but it's a useful abstraction. If people haven't seen or experienced that, it's pretty good. The subtlety is that these are tools that are suited to certain situations a little better. And those same situations can exist in the same codebase, can exist in the same program. CHARLES: Yeah. I still, I love Ruby. I adore it. And in some ways, I've been researching functional programming and it's been going on for the last four years. So many times, people are like, “Oh, I just can't stand this tool anymore.” And I'm like, “Man, I still love Java.” I don't understand how learning to love something decreases your love for something else. BRANDON: That happens the first two times that you fall in love, is that you feel like you have the old thing less in order to love the new thing. And then you start realizing, “No, you are allowed to fall in love with new things without falling out of love with the old things.” I would almost use that as an interview question. Is there some way to use that as a way to gauge somebody's actual real concrete maturity as a developer? Because that is a mark of maturity. CHARLES: Yeah. I mean, you could say, “What's some tool that you no longer use that still informs your day-to-day routine?” BRANDON: Yeah. I guarantee you, people that were doing Smalltalk in the 80s think about it all the time. CHARLES: [Laughs] Yup. Yeah, exactly. Exactly. BRANDON: Alright. So, I want to cover one last thing. CHARLES: It's part of growing, right? If you're going to grow as a developer, you can't be shrinking at the same time as you're growing. Otherwise, you're like the same size, just in a different place. BRANDON: However, you don't get any Medium think piece points. Nobody does the one clap, two clap, forty, for blog posts that are like, “Why I'm still using some programming language but using one a little more than I used to use and this one a little less.” CHARLES: [Laughs] Zero claps. BRANDON: Yeah, zero claps on that think piece. I just want to cover one last thing before we wrap this up, and it is the fact that Frontside, the biggest gift that Frontside gave me was the mission for the next 20 years of my career. I think it could change, but I'm pretty confident about this, at this point. Being approximately 20 years into my career, I feel like I kind of have a feeling for what the next 20 years is about. And the Frontside really drilled that into me and helped me focus it and helped me dial it in. And it is this idea that there is an incoming generation of programmer that thinks about things differently than the previous generation in a pretty radical way. Because the previous generation all came out of the same schools. They all look the same. They all have a similar shared set of values in general. They created the Sil- – you know, I'm not actually going to be overly critical of the Silicon Valley culture that exists now. It is a result of the type of people that came out at the time that value innovation over almost anything else. People talk about ripe for disruption. The fact is, that has been an engine of economic growth and progress for society in a lot of ways that has a lot of costs that weren't factored in by a bunch of people who all thought the same way. And now, with people coming through code schools and people coming from different backgrounds and people coming from different environments, they're looking at programming and software as either an economic opportunity or something they didn't see that they could possibly do. Those doors were not open to that group of people before. There is a natural influx of people but many of them are bouncing out because they're not finding that group of people, they don't have a shared enough set of values that the people that are new are coming in and finding job opportunities, finding promotions, finding leadership positions. And so, I know now that my mission over the next 20 years of my career is to create those opportunities for people that have different backgrounds from me and different experiences. The career tracks, the promotions, endorsing and supporting and kind of sponsoring this incoming group of freshmen into our industry that come from different places, different backgrounds, different problems that they care about solving. They want to figure out how to solve the Flint Michigan water crisis instead of delivering socks to people in Silicon Valley, you know? So, I feel like we're at the beginning of a seed change in the value system potentially of our entire industry. But that's going to require training up the next generation of technical leadership. And I felt like the best thing I could do right now is learn to be a better manager, because I really like that job. And it provides the opportunity to find, hire, sponsor, promote and encourage those people to move into their own leadership positions. There are lots of other things that a person, you could be a VC and care about that stuff. You could have lots of different positions and put yourself in a position to do that. You could be a consultancy owner. You know what I mean? There are jobs that you can do that you can accomplish that goal. But it gives me such a sense of direction that when I'm looking for a job, I was looking for a home for that mission rather than just the thing that I felt like doing. Like okay, this job is important to me because I need it to house me and this mission so that I can support my family but have enough emotional overhead to participate in community stuff, but enough ability to lead within an organization, enough influence to actually push that agenda. So that the next generation of people are making better companies. So anyway, all of that came out of my time at Frontside where you and I sat around talking about: how do we build a place that is like a monastery? These were your words. You remember this? We want a monastery for code where people can just focus on becoming better developers. And underneath that though was the sense that this was a place of opportunity for people that might go somewhere else and stagnate as a developer. This will be a place to accelerate them. And so, that kind of spun me out and accelerated me into my mission. So anyway, I just wanted to point out that that was like, with a bullet, is the most important thing that came out for me in my time at Frontside, was that it clarified for me what I was trying to accomplish with the next couple of decades of my career. CHARLES: Wow. Well, that's fantastic. You definitely did a lot of that both here at Frontside and I mean you're continuing to do that. I definitely want to see more public speaking from you. Maybe some [inaudible] perfect. [Inaudible] at EmberConf was actually fantastic. But I mean, you're also able to help people find their mission, too. Like the talks you have at Keep Ruby Weird and even really, the first talk you gave at LoneStarRuby about moving Ember. It's always, how do I adapt what I'm feeling to my overall mission and then relate that back to technology? Man, I just can't wait. I can't wait. When are you going to hit the road again? BRANDON: I think this is the year. I'm going to start thinking about this stuff. I'm looking at the stuff that I wanted to talk about on this podcast and I was like, “Oh no, wait. That's like a dozen podcasts.” Like, no. Absolutely not. Not possible. I will say, I miss so much, this time that I spend with you. I don't want to let it go. I really miss working with you. I really miss having these conversations whenever I want. This has been a very, very special privilege for me to be able to do this with you today. And congratulations on Frontside continuing to thrive and grow and become more of its own entity and more of its own special flavor. And it makes me really happy to see the people coming out of there, that it's still doing its mission of making great software by making great developers. It makes me real happy. CHARLES: Yeah, yeah. Hopefully we can keep on keeping on. I do miss working with you. I miss the conversations that we would have in the kitchen which are basically an extension of this podcast. But I also, man, I really, really, really, really like working with the group of people that are here today. I've just seen them producing just some absolutely amazing things. And honestly, there's a selfish aspect to it, too. I get stimulated. My own thinking and learning is stimulated by the people that I work with. And like I said, the whole side note we had about distributed systems and IoT and just a constant ferment of things. So, I still really, really, really enjoy it. BRANDON: That makes me happy. CHARLES: And I'm really glad that we got to kick it today. BRANDON: Yeah, me too. CHARLES: I thought you were going to say that your 20-year mission was to have your perfect Emacs initialization setup. BRANDON: Oh my gosh. Some of these days, I'm going to figure out RuboCop. CHARLES: Actually, do you want to pair on that? BRANDON: Yeah, let's do that. CHARLES: Alright, everybody. I'm going to sign off. If anyone wants to continue the conversation, obviously you can get in touch with Brandon. He is misspelled @tehviking on Twitter. T-E-H-V-I-K-I-N-G. Always come at him. BRANDON: Don't @ me. CHARLES: [Laughs] BRANDON: I work for a really cool company and if you ask me about it on Twitter, I'll tell you all about it. CHARLES: Awesome. And we of course are Frontside. You can get us on Twitter at @TheFrontside or just drop us a line to contact@frontside.io. And we would love to talk to you more about this podcast and all the wonderful things that we do here, which includes building custom software that you can stake your future on, that's going to be good for the five-year outlook. So with that, goodbye Brandon. Goodbye everybody. And we will see you… BRANDON: Bye Charles. I love you. CHARLES: Me too.

This Frontsider panel episode explores what virtues go into making quality software, such as having tests, making sure software is performant and accessible, and why you should try to avoid technical debt. Transcript: CHARLES: Hello everybody and welcome to The Frontside Podcast Episode 98. My name is Charles Lowell, developer here at The Frontside and your podcast host-in-training. With me today, we're going to have a round table, a Frontside round table. With me today is Elrick. ELRICK: Hey. CHARLES: Joe. JOE: How you doing? CHARLES: And of course, Will. WIL: Hello, hello. CHARLES: Welcome, y'all. We're going to be talking today about some of the things that we do around here, aside from trimming the shrubs and making coffee and snacking on Altoids. Like, way too many of them. Yeah. I was thinking we could talk a little bit about software qualities of relative things, like this software has these qualities. And I think that that kind of lofty goal of software quality is comprised of having a bunch of little qualities. The quality of having fewer bugs or the quality of having these things. And so, talking about all these things that we do and kind of what we do to make sure that we continue to do them. Or the ways that we can ensure that our software has these things. So yeah, we can just start really anywhere. WIL: Yeah. So, one core thing is obviously tests. CHARLES: That kind of falls under we want to have – really, there's two qualities there that we want, right? Is we want to have… WIL: Maintainable software. CHARLES: We want it to be maintainable. We want it to be resilient to change. And we want it to work properly, right? Yeah, so we put tests in place to make sure that that happens. JOE: Tests also inform design in a really positive way. A lot of the time, anyway. WIL: Another thing that we like to include in our apps is responsiveness. CHARLES: Yeah. And just making sure that you have – that it works on a multiplicity of devices, right? WIL: Yeah. And not just the devices, but browsers as well. CHARLES: Yeah. And it turns out it's actually really hard to do that after the fact. WIL: Right. JOE: Yeah. CHARLES: Making sure that lots of browsers, lots of devices. Because yeah, sometimes you have some weird screen width that is on some weird device, and making sure that that works. I guess there's some overlap with testing there, too, isn't it, right? Like you want to be running your tests on those devices at those resolutions to make sure that they're going to work. This is something that we aspire to but I don't think we're quite there yet. It was making sure that our applications are accessible. WIL: Yeah. JOE: I'm very excited to learn more about this as we get into this, yeah. CHARLES: Right, right. And asking the question, how is it that we actually can ensure our applications are accessible? We have very paved roads for making sure that our applications are resilient to change and that they have low bug rates and that they're well-designed via testing. But what is the analog of testing for accessibility? What's the way that you can put those guardrails in for accessibility? I have no idea. And that's an ongoing conversation here at Frontside. JOE: So, I guess I'm curious as to what technologies are actually involved in accessing a web application in – would it be reasonable to say a non-traditional way? I know there's such things as screen readers, but is that all we're talking about? Or what is the ecosystem that we have to consider supporting? CHARLES: I'm certainly not an expert on this. We'd have to get Rob in here to chew our ears off this. JOE: Yes. CHARLES: But from what I've picked up from him and from our conventions with Marcy Sutton and some other folks that we've had on the podcast, it's a big umbrella. So, it's anyone using an application in a non-traditional way. So, whether that can have to do with limited vision, hearing, movement, range of movement, cognitive ability, it's a gigantic whale of a domain. WIL: Yeah. The topic of accessibility can definitely be several podcasts on its own. CHARLES: Yeah. One thing that we've talked about is it would be great if you could drive your test suite through a screen reader or something like that. What would that even look like? There are a couple of open source ones out there, but they're Windows-only. I think it was NVDA was the big one. And then you have a screen reader that then drives the applications in your operating system, so it's going to vary per operating system. So, making sure that it's accessible on Windows, at least as I understand it, is very different from making sure that it's accessible on a Mac. JOE: Yeah, it's like a whole other layer. And it's like BrowserStack outside of the browser. CHARLES: Right. WIL: There are things that you can do from the beginning that will make it easier when you get to that point. It's just like using semantic HTML, knowing when and how to use proper aria labels. All these things, if you do it from the beginning, it's not as big of a task as bolting it on afterwards. CHARLES: Right. And I think we do have a leg up when it comes to web applications. It's within our power to change. There are cross-platform of those technologies. But as you said, it's important to put them in from the beginning. Because as we've seen, for each one of those categories, you're accumulating debt if you don't address it. So, there's technical debt. But I think that technical debt can [inaudible] into a bunch of different areas. So, there's technical debt in terms of the internal quality of your architecture, the way your software components talk to each other. And I think that that's what people mostly think of when they talk about technical debt. But I think in terms of responsiveness debt, there's a slice of the technical debt pie that has to do with making your application responsive. And so, if you don't address making your application responsive, you're accumulating debt and you might not know it. And if you're not making your application accessible, then from the beginning you're accumulating debt. So that if you have to go and try and figure out your accessibility story six months, a year, two years, you might actually uncover and say, “Whoops. I've been swiping the accessibility credit card. And holy crap, with all this. All my fines and penalties and compounded interest. Now I'm accessibility bankrupt.” And that can be scary, right? WIL: Yeah. And a lot of people don't realize with all this debt after the fact is they think they're going in and adding things like responsiveness and accessibility and tests. But really, you're also taking away previous work that's already there, things that need to be refactored. If you put these things off, you're not just adding a few hours of time. You're inflating your time exponentially. CHARLES: Right. Right, exactly. It can be intimidating but I think it's also empowering, because technical debt is like a scary subject. But if you're like, “Oh, we can actually slice our technical debt into a bunch of different categories and address them individually,” just knowing that this is an area where debt can accumulate, that's half the battle. Because the worst thing is debt you don't even see. ELRICK: Yeah. WIL: I mean, [inaudible] is big. That's a big part of accessibility, that, is most people don't think of accessibility. So, that is a huge debt that a lot of companies don't see. JOE: What about something like internationalization where I feel like I've never been in an application where that wasn't punted on to some degree. That's kind of a well-known problem, but it still takes a back burner. Do you think that if accessibility had more exposure as a concern, would it actually get the attention it deserves or is it kind of destined to, “Oh, we'll get to those yaml files later. We'll send those off for translation later,” that type of thing. ELRICK: I don't know. Sometimes I feel as though people feel as though they're trading speed away when they're building applications when they go to implement these things. Like, “Okay, well we're not really going to touch on these right now because that's going to slow us down from pushing out features.” Which is not really true. Because if you don't settle on these things early, you're not really building a solid foundation for your application in the long haul. So, I think people are like, “Oh, we'll just do it later.” CHARLES: Right. ELRICK: And, “We'll just ship features now.” CHARLES: Right. I think that's exactly right. It has this kind of secondary effect where not only do you develop the debt but you develop a culture of accumulating debt, right? Like when it comes to people getting a hold of their finances, the first thing that they have to change is they have to change their spending habits. And that can be the hardest thing. It's not just balancing the equation. It's like saying, “I need to readjust my thinking about this.” ELRICK: Yup. CHARLES: So that I'm not consistently put in this situation again. JOE: So, there's an operative word there, right, in personal finance in that usually if a company is addressing technical debt especially down the road, something that they've punted on for a while, it's far from personal. There's a board of directors or there's a special interest group involved. There's people who want features that are putting money into it. There's a lot of pressure as the company grows and more people are involved. Priorities are more likely to be lost, I guess. CHARLES: Are you saying it can be hard when your culture is spread over that many people, it can be hard to shift? JOE: Absolutely, yeah. And I guess to keep with the dash-first thing, ideally were we starting a company, we would want to start a culture for this company. A culture that recognizes the vulnerability that we all have to technical debt as applications grow. We want that upfront. But the reality is, you know, startups are eager to get things out. Companies that have been around for a long time have high-paying clients that they depend on that want certain things. And yeah, I guess I'm just saying that it has to come in from the beginning. CHARLES: Yeah. And I think that – I don't want to completely disparage technical debt entirely, because technical debt like actual debt, like financial debt, is a powerful tool that you can wield. But it's also, it's like a table saw. You can also easily slice your finger off. It doesn't mean that it's not a useful tool, right? If anyone's bought a house, it's really great that you can borrow money to buy a house. It's great that businesses can borrow money and get small business loans to get bootstrapped. And that benefits us all to have that community. I don't think that – yeah, startups definitely, they need to have technical debt as a tool that's available to them. But they just need to understand the consequences of it and be able to get a hold on it. JOE: That's a super interesting take. I never considered it that way before. CHARLES: Yeah. It's definitely not my take. I actually think the person who coined the term ‘technical debt', that was the original idea. But then people realized that technical debt can also get way out of hand. WIL: It's just like real debt. If you're not paying down a certain amount every so often, it's going to keep growing. CHARLES: Yup. You're going to have to declare bankruptcy at some point and throw out the piece of software if you don't pay a down. And that's going to be more expensive. ELRICK: Yup. That's definitely true. So, I have a question. And we see this all the time repeating itself at various companies, whether it's a startup, a large company, where they put off testing and mobile-first, user-first, accessibility-first. Like all the firsts, they just toss it to the side. Why do you all think that that happens so frequently? CHARLES: I think it comes into people not understanding that if you don't address it from the start, it won't happen naturally. There is a prime motivator that has to happen. If you don't imbue something with those qualities when it's tiny, when it's a tiny seed, a tiny crystal, you're going to have to drill through layers and layers and layers of core to put it at the crystal to begin with. I like to think of software as kind of like a tree. And we eat the fruit of the tree, and that's the features that users use. And we can tell that a fruit is delicious merely by placing it in our mouths. And we can tell what fruit is bad. But we can't really look at the fruit itself to say what caused this fruit to be good, what caused this fruit to be bad. We have to look at the tree. And I think that that's what people miss when they're developing software, is that what you really want to do is you want to build a tree that builds good fruit. You can't just take the fruit off the vine and say like, “Hey, I've got this peach but it doesn't have enough sweetness. So, I'm going to take a syringe and I'm going to inject glucose around it and make it less tart.” You say, “I want a sweet fruit,” right? ELRICK: Yeah. JOE: You could probably actually do that. CHARLES: You could. And that might be a strategy. And we see a lot of software that has those qualities of, “Oh, we're going to make this accessible,” or, “We're going to try and make this beautiful.” I happen to think that pigs are adorable animals and look great in lipstick. But that [laughs]… you could put lipstick on a pig but people can tell. And you can say, “Oh, this peach needs to have softer fruit,” and you can whack it with a mallet to actually make the meat more tender. But people are going to be able to tell. So, what you really need to do is you need to care for the peach tree rather than worry so much about the fruit. Because if you have a healthy tree, then you will have healthy fruit, right? ELRICK: Yeah. So, you want to plant good seeds. JOE: Yeah. WIL: Back to you question, Elrick, about what motivates startups and other companies to put off these things. I think the biggest thing is just time and money. They have this misconception where they're saving a little time and saving a little money now just to add it back later. But in reality, it's going to cost them tenfold time and money for adding it later, versus just spending that little bit of time and money and all that to begin with. CHARLES: That's true. JOE: It could also boil down, as far as just personal intimidation. Not so much like a business side of a thing but maybe just, think of all of the things that you listed, Elrick. It was almost a dozen dash-firsts in there. If you're sitting down at a startup that you started with three friends and just approaching these things for the first time, that's a lot to tack on right upfront. It's intimidating. CHARLES: It is intimidating. I think my message to those people is I've felt intimidated by that. I think my message to those people is like, the nice thing about it is if you attack those, if you tack all of those things from the get go, the features will take care of themselves and feel more effortless as you go on. You say like, “Oh, well actually, I don't worry about a high rate of bugs.” I want to say recidivism, but that's not the right word. A high rate of return, not on money but on – or high rate of bouncing your users. You don't want that. And if you bake that in from the beginning, parts of the software development cycle that were stressful before just aren't stressful anymore. So, if we say, “We want to have a system that is easily maintainable, well let's put that in from the very beginning.” We say that a lot. We deploy to production on day one. But what that means is, we say we have this value that we want the system to be easily maintainable. And so, we're going to do it from day one. That means that we actually – it's not something that we worry about so much on down the road. Whereas that used to be very stressful. I don't know. I remember when I started my career, there were these long release cycles where every six months, you'd release software. And the last month was just absolutely terrible as you try to stand this thing up and get it into production and then realize it's not monitored. There's no one checking the health of this thing. So, it's pissing off users at one in the morning. And… WIL: Beepers. CHARLES: What's that? WIL: Beepers. CHARLES: That's actually a great – there's a story there. The one time I got a beeper, I went canoeing in the canals of London and I tipped over my canoe and I dropped both my cellphone and the beeper that they've given me. ELRICK: What? CHARLES: I never got put on pager duty again. [Laughter] JOE: I'm going to use that next time [inaudible] with an on-call position. That's a good move. CHARLES: I remember, I definitely remember how sour my manager's face was when I turned [inaudible] the cellphone that was like, dripping with water. JOE: He was eating bad fruit, probably. CHARLES: Yeah. [Laughs] So, the other thing is we like to build beautiful applications, right? So, you have to – that match the user experience. You have to spend that time on design and beauty upfront. You will not have a beautiful application after the fact. You just need to bake it in. ELRICK: And accessible design. CHARLES: Exactly. ELRICK: Don't forget that one. CHARLES: Don't forget that, right? A responsive design. WIL: Yeah, accessibility-first in design. Yeah, responsive and all that starts in design phase, yeah. CHARLES: Yeah, all that, right? So, you want a great experience. You want an accessible experience. You want a responsive experience. You want a quality experience. You want a performant experience. That's another quality that you say. Like, “We're going to make sure that this is performant.” If you want that – and that's something that we're not always great about, right? We don't actually put in benchmarks for our software from the get go. But maybe we should. But there's perhaps a hidden cost there that we might be actually accumulating performance debt that we don't even know about. JOE: That's true. ELRICK: Interesting. JOE: So, things that pop up that are new. Like, accessibility wasn't probably always a thing in computing. Internationalization probably wasn't always a concern. Beautiful certainly wasn't a concern if you look on Wayback machine. You will see that to be true, right? [Laughter] JOE: So, all code is tech debt, I would argue. Or at least has the potential to be. And yeah, as the ecosystem as a whole evolves, being responsive to that, having plasticity in that respect, sort of like meta-first. CHARLES: Right. JOE: That could be the real challenge. WIL: Yeah, Charles is mentioning all these experience things. And so, I was thinking X-first is simply experience-first. You want you users to experience a certain quality of your app. That experience needs to start in the conception phase. CHARLES: Yeah. ELRICK: That's true. And even your developers coming in, developer experience. JOE: Yeah. CHARLES: Right. And I think the core of that X-first, that experience-first, is you need to pick which experiences. Because you can't have everything. JOE: Right, yeah. CHARLES: One, there is going to be too much. You have to say, “I'm going to sacrifice on knowing that this is a performance thing. I'm not going to include that in the core DNA of my application.” And there's just going to be things that you don't know about yet that are just unsolvable problems or that don't necessarily work. And you can say, “You know what? Hypothetically, I'm not going to make this an accessible – I'm not going to focus on accessibility.” But then you need to own that. And you need to know that you're accumulating a huge amount of debt around that. And then I think that is a particularly bad trade-off because someone's always going to come along and you're going to have to know that your application is accessible. I think once we clamp down on that, that's going to be something that we have a strategy for and we include at the beginning on every single application, right? ELRICK: Yeah. CHARLES: But I think you need to have, almost like holding the cards in your hand, say, “These are the cards. These are the X's that I'm going to have in my hand. And they are going to be core to my app.” And they're going to be part of the DNA of that tree. So that I know that the fruit is then going to have those qualities. JOE: And then you as an engineer, that goes through an iterative process as well. Just starting out, you have no idea what that DNA should look like. And short of learning from people who are wiser than you who are around you, and reading blog posts and whatnot, really the only way to know the pain of strong-arming internationalization for instance into a 15-year-old Perl application, is to go through it. And then, you know, future trees will not have this DNA. CHARLES: Right. Right. And that's the other thing. Is if you are going to include, if you are going to try and splice something into the DNA, there's a lot of work. And you just need to go for it. You acknowledge that it's going to be a lot of work. And you need to, you just need to own it and go for it. And pay that expense of actually getting it deep, deep, deep into your application's core values. So that then, you don't have to worry about it anymore. Otherwise, you're going to be paying – you're just basically signing up for a lifetime of debt. Right? WIL: Yeah. And then to make the debt analogy even more, it's like people don't understand the total debt. The end debt. People get a $30,000 loan with a 4% interest and they think they're paying back that $30,000 loan. But really, they're paying back $36,400 after all the amortization of their interest. The debt is higher than you can see, always. CHARLES: Right. WIL: And it's true in tech debt, too. React is the new hot thing now, but in 10 years we're going to be on React debt that we're migrating away from. JOE: I hope so. [Laughter] CHARLES: Maybe less, I think less than 10. WIL: Yeah. The debt is always there. And people don't realize how much they have to pay on top of what's visible. JOE: Yeah. It's an invisible vig. CHARLES: What's a vig? JOE: It's interest, in the mafia. CHARLES: Oh. JOE: Sorry. Yeah. CHARLES: I forgot you're Italian. JOE: Yeah. ELRICK: So, for people that are listening, they might be in a situation where they need to advocate to the powers that may be these X-first values. What do you all think that some of the approaches that they should take to say to whomever it is that, “We need to do this first”? Because there's times where you might say, “Hey, we need to do this first,” and people just look and say, “Oh, maybe not.” Then you need to push back on that. CHARLES: In my experience, I find that the tech debt argument is a good one. Because I think it can be, it's both limiting and empowering. Because sometimes it really is the right call to pull out your credit card and put something on it. If you need to buy water and you need to buy food and you don't have any other means, man, put it on the credit card. Right? Seriously. Even if you have no idea how you're going to pay it back. Like, whip that sucker out and stick the chip in. And it doesn't matter how much it costs. And so, sometimes that is the right call. But I think draining it of a moral or a value as a human person thing, and approaching it from a business decision and saying, really trying to attach a cost to it. Because then I think if you can drain out the emotion of it, because people really want something. They're striving to go get it and trying, give them tools to think about it rationally. That I think is a good strategy, to just say, let them know that there is a debt that's being paid here or that's being accumulated here. And it's really large. And maybe even say, “Look, if we were to put this off by six months, this might cost not twice as much. It might cost ten or even a hundred times as much.” So, by saving $5,000 now, you might actually be accumulating $50,000 worth of debt. It's [bigger] than you think. But I do like – so, I think that's one important tool. But I think then also the other important tool is to say, “If we are going to attack this, let's drive it home. Let's put it at the core. Let's make this a value that we hold so that the tree can take care of the fruit itself.” So, if we say that we're going to put in accessibility – because not all projects are greenfields. JOE: Absolutely not. CHARLES: So, what's the message to them? Sorry. You're just SOL. I think if you're a year into a project, two years into a project, and you realize, “Oh no. We need to do internationalization,” recognize that that might be something that's – that's a pillar of your architecture. Or, “We're going to make this application accessible,” don't half-ass it. WIL: Weave it in. CHARLES: Say, “We're going to transform this. We're not going to add accessibility. We're going to transform what we have into an accessible application.” Or, “We're going to transform what we have into a beautiful application.” Otherwise… WIL: Yeah [inaudible]. CHARLES: I would say leave it ugly and focus your efforts elsewhere on things where you do have your values straight. Because you're never going to have everything in line. JOE: No. WIL: Treat software like immutably. You don't add something to it. When you want to add accessibility, you're creating a whole new accessible app. ELRICK: Ooh. That's deep. CHARLES: Yeah. JOE: So, having seen – I don't know. I think it was very apt, looking at it as a business decision. I've seen it go the other way. Because at least among engineers and people on the technical side of it, this can become a very strong moral issue that people feel very strongly about. CHARLES: Because we have to live with the consequences quite honestly, right? JOE: Exactly. And that's a hard thing to translate to say an executive board that may be three levels abstracted away from you and is making those decisions. I've seen people attack or approach this I guess with that emotion built in, with the, “This is the right way to do it. Everybody else is doing it wrong.” It gets nowhere, basically. What needs to happen I think, so you talk about having this beautiful tree. But that also requires beautiful gardeners. And so, where the moral thing or the interpersonal thing comes in is there needs to be kind of an inclusive and encouraging environment that is fostered among the people tending to the tree. And that's a totally separate thing than selling the business value of it. Those things should be completely divorced. CHARLES: Yeah. It's funny. It's always hard to reconcile those two things, right? Because on one you have, “You have to take care of the raw consumption of material and the output of product.” But then also trying to – so, there's some baseline math that has to happen but making sure that that goal, it doesn't slice people. And can enable them to be happy and feel like they're doing good work. And that the things that they're doing is having meaning. It's probably an insoluble problem that we're going to be dancing around for as long as people are around. If there's one thing that we've come to recognize around here, and we've stated it many different ways from a bunch of different angles through the course of this conversation, and I would say through the course of this podcast, but that is if you want to see something in your software, make sure that you attack it from the get go. ELRICK: Intertwine it in your DNA. CHARLES: Exactly. And then you can actually experience the fruit, rather than trying to always, always trying to jam it and change it and get it into the taste you want after the fact. So, I guess that's it. Thank you so much y'all, for this conversation. I really, really enjoyed it. For those of y'all listening, if you want to continue the conversation, you can get in touch with us we are @TheFrontside on Twitter. Or you can drop us an email. We're contact@frontside.io. So, thanks Elrick. Thanks, Joe. Thanks, Will. JOE: Thank you. WIL: Thank you. ELRICK: Yup. It was great. JOE: It was fruitful. [Chuckles] ELRICK: Frontside-first. CHARLES: And well, we'll see y'all around.

The best inventions are born of necessity. When Charles Palleschi purchased a site that sold popcorn, popcorn kernels, and popcorn related accessories ecommerce business, he quickly realised that he needed to scale and parts of his business. That need inspired Charles to create Spark Shipping, the leading dropship software which automates millions upon millions of orders for eCommerce and retail entrepreneurs every month. 00:02 Salena: Hey there, and welcome to this week's episode of the bringing business to retail podcast. The best inventions are born of necessity. When Charles Palleschi purchased a store that sells popcorn, of all things, popcorn kernels and popcorn related accessories, the e-commerce business was a stepping stone for him to realize that if he wanted to grow he was going to have to scale and automate parts of his business. And that need inspired Charles to create spark shipping, the leading drop ship software which automates millions upon millions of orders for a-commerce and bricks and mortar retailers every single month. I am really excited to be talking about drop shipping because it is something that a lot of retailers may not know that they are even doing. So welcome to the show Charles. 00:53 Charles: Hey, thanks a lot for having me on. Salena: This is a New York accent am I right? Charles: Close, from Australia, it may sound like that. But Boston you are very close to that. Salena: Still getting use to my accents. Like Americans love Australian accent. We love all the individual accents.  Charles: You will hear it pretty quickly. It's very Boston. Salena: 01:19 Great, okay. So tell us what exactly is drop shipping because I have a feeling that some of the people who are listening today will know what it is, but others will actually be doing it already in some sort of informal arrangement in their store. And they don't even know that they are doing it. So what is drop shipping? For more click here

Julie Moronuki: @argumatronic | argumatronic.com Show Notes: This episode is a follow-up episode to the one we did with Julie in September: Learn Haskell, Think Less. We talk a whole lot about monoids, and learning programming languages untraditionally. Transcript: CHARLES: Hello everybody and welcome to The Frontside Podcast, Episode 93. My name is Charles Lowell, a developer here at The Frontside and I am your podcast host-in-training. With me today from The Frontside is Elrick also. Hello, Elrick. ELRICK: Hey. CHARLES: How are you doing? ELRICK: I'm doing great. CHARLES: Alright. Are you ready? ELRICK: Oh yeah, I'm excited. CHARLES: You ready to do some podcasting? Alright. Because we actually have a repeat guest on today. It was a very popular episode from last year. We have with us the author of ‘Learning Haskell: From First Principles' and a book that is coming out but is not out yet but one that we're eagerly looking forward to, Julie Moronuki. Welcome. JULIE: Hi. It's great to be back. CHARLES: What was it about, was it last October? JULIE: I think it was right before I went to London to Haskell [inaudible]. CHARLES: Yeah. JULIE: Which was in early October. So yeah… CHARLES: Okay. JULIE: Late or early October, somewhere in there. CHARLES: Okay. You went to Haskell eXchange. You gave a talk on Monoids. What have you been up to since then? JULIE: Oh wow. It's been a really busy time. I moved to Atlanta and so I've had all this stuff going on. And so, I was telling a friend last night “I'm going to be on this podcast tomorrow and I don't think I have anything to talk about.” [Laughter] JULIE: Because I feel like everything has just been like, all my energy has been sucked up with the move and stuff. But I guess… CHARLES: Is it true that everybody calls it ‘Fatlanta' there? JULIE: Yeah. [Laughs] CHARLES: I've heard the term. But do people actually be like “Yes, I'm from Fatlanta.” JULIE: I've heard it a couple of times. CHARLES: Okay. JULIE: Maybe it's mostly outsiders. I'm not sure. CHARLES: [Chuckles] JULIE: But yeah, it's a real cool city and I'm real happy to be here. But yeah, I did go in October. I went to London and I spoke at Haskell eXchange which was really amazing. It was a great experience and I hope to be able to go back. I got to meet Simon Payton Jones which was incredible. Yeah, and I gave a talk on monoids, monoids and semirings. And… CHARLES: Ooh, a semiring. JULIE: Semiring. So, a semiring is a structure where there's two monoids. So, both of them have an identity element. And the identity element of one of them is an annihilator. Isn't that a great word? It's an annihilator… CHARLES: Whoa. JULIE: Of the other. So, if you think of addition and multiplication, the identity element for addition is zero, right? But if you multiply times zero, you're always going to get to zero, so it's the annihilator of multiplication. CHARLES: Whoa. I think my mind is like annihilated. [Laughter] JULIE: So, it's a structure where you're got two monoids and one of them distributes over the other, the distributive property of addition and multiplication. And the identity of one of them is the annihilator of the other. Anyway, but yeah, I gave a history of where monoids come from and that was really fun. CHARLES: Yeah. I would actually like to get a summary of that, because I think since we last talked, I've been getting a little bit deeper and deeper into these formal type classes. I'm still not doing Haskell day-to-day but I've been importing these ideas into just plain vanilla JavaScript. And it turns out, it's actually a pretty straightforward thing to do. There's definitely nothing stopping these things from existing in JavaScript. It's just, I think people find type class programming can be a tough hill to climb or something like that, or find it intimidating. JULIE: Yeah. CHARLES: But I think it's actually quite powerful. And I think one of the things that I'm coming to realize is that these are well-worn pathways for composing things. JULIE: Right. CHARLES: So, what you encounter in the wild is people generating these one-off ways of composing things. And so, for a shop like ours, we did a lot of Ruby on Rails, a lot of Ember, and both of those frameworks have very strong philosophical underpinnings that's like “You shouldn't be reinventing the wheel if you don't have to.” I think that all of these patterns even though they have crazy quixotic esoteric names, they are the wheels, the gold standard of wheel. [Laughs] They're like… JULIE: Right. CHARLES: We should not be reinventing. And so, that's what I'm coming to realize, is I'm into this. And last time you were talking, you were saying “I find monoids so fascinating.” I think it took a little bit while to seep in. But now, I feel like it's like when you look at one of those stereo vision things, like I'm seeing monoids everywhere. It's like sometimes they won't leave me alone. JULIE: In ‘Real World Haskell' there's a line I've always liked. And I'm going to misquote it slightly but paraphrasing at least. “Monoids are ubiquitous in programming. It's just in Haskell we have the ability to just talk about them as monoids.” CHARLES: Yeah, yeah. JULIE: Because we have a name and we have a framework for gathering all these similar things together. CHARLES: Right. And it helps you. I feel like it helps you because if you understand the mechanics of a monoid, you can then when you encounter a new one, you're 90% there. JULIE: Right. CHARLES: Instead of having to learn the whole thing from scratch. JULIE: Right. And as you see them over and over again, you develop a kind of intuition for when something is monoidal or something looks like a semiring. And so, you get a certain intuition where you think, “Oh, this thing is like a… this is a monad.” And so, what do I know about monads? All of a sudden, this new situation like all these things that I know about monads, I can apply to this new situation. And so, you gain some intuition for novel situations just by being able to relate them to things you already do know. CHARLES: Exactly. I want to pause here for people. The other thing that I think I've come in the last three months to embrace is just embrace the terminology. JULIE: Yeah. CHARLES: You got to just get over it. JULIE: [Chuckles] CHARLES: Think about it like learning a foreign language. The example I give is like tasku is the Finnish word for pocket. JULIE: Right. CHARLES: It sounds weird, right? Tasku. But if you say it 10 times and you think “Pocket, pocket, pocket, pocket, pocket.” JULIE: Yes, yeah. [Laughs] CHARLES: Then it's like, this is a very simple, very useful concept. JULIE: Right. CHARLES: And it's two-sided. There on the one hand, the terminology is obtuse. But at the same time, it's not. It's just, it is what it is. And it's just a symbol that's referencing a concept. JULIE: Right, right. CHARLES: It's a simple concept. So, I just want to be… I know for our listeners, I know that there's a general admonition. Don't worry about the terminology. It's… JULIE: Right, right. Like what I just said, I said the word ‘monad'. I just threw that out there at everybody, but [chuckles] it doesn't matter which one of these words we'd be talking about or whatever I call them. We could give monads a different name and it's still this concept that once you understand the concept itself, and then you can apply it in new situations, it doesn't matter then what it's called. But it does take getting used to. The words are… well, I think functor is a pretty good word for what it is. If you know the history of functor and how it came to mean what it means, I think it's a pretty good word. CHARLES: Really? So, I would love to know the history. Because functor is mystifying to me. It sounds like, I think the analogy I use is like if George Clinton and a funk parliament had an empire, the provinces, the governors of the provinces would be functors. ELRICK: [Laughs] JULIE: Yes. CHARLES: But [Laughs] that's the closest thing to an explanation I can come up with. JULIE: I might use that. I'm about to give a talk on functors. I might use that. [Laughter] ELRICK: Isn't that the name of the library? Funkadelic? CHARLES: Well, that's the name of the library that I've been… JULIE: [Could be], yeah. ELRICK: That you'd been… CHARLES: That I'd been [writing] for JavaScript. ELRICK: Yeah. CHARLES: That imports all these concepts. JULIE: [Laughs] ELRICK: Yeah. JULIE: Yeah. ELRICK: So awesome. JULIE: Yeah. Yeah, I have… CHARLES: So, what is the etymology of functor? JULIE: Well, as far as I can tell, Rudolf Carnap, the logician, invented the word. I don't know if he got it from somewhere else. But the first time I can find a reference to it is in, he wrote a book about… he was a logician but this is sort of a linguistics book. It's called ‘The Logical Syntax of Language'. And that's the first reference I know of to the word functor. And he was trying to really make language very logically systematic, which natural language is and isn't, right? [Chuckles] CHARLES: Right. JULIE: But he was only concerned with really logically systematizing everything. And so, he used the word functor to describe some kinds of function words in language that relate one part of a sentence to another part of a sentence. CHARLES: Huh. So, what's an example? JULIE: So, the example that I've used in the past is, as far as I know this is not one that Carnap himself actually uses but it's the clearest one outside of that book… well the ones inside the book I don't really think are very good examples because they're not really how people talk. So, the one that I've used to try to explain it is the word ‘not' in English where ‘not' gets applied to the whole sentence. It doesn't really change the logical structure of the sentence. It doesn't change the meaning of the sentence except for now it negates the whole thing. CHARLES: I see. JULIE: And so, it relates this sentence with this structure to a different context, which is now the whole thing has been negated. CHARLES: I see. So, the meaning changes, but the structure really doesn't. JULIE: Right. And it changes the whole meaning. CHARLES: Right. JULIE: Not just part of the sentence. So, if you imagine ‘not' applying to an entire sentence because of course we can apply it just to a single word or just to a single phrase and change the meaning just of that word or that phrase, but if you imagine a context where you've applied ‘not' to a whole sentence, to an entire proposition, because of course he's a logician. So, if you've applied ‘not' to an entire proposition, then it doesn't change the structure or the meaning of that proposition per se except for it just relates it to the category of negated propositions. CHARLES: Mmhmm. JULIE: So, that's where it comes from. And… CHARLES: But I still don't understand why he called it functor. JULIE: He's sort of making up… well, actually I think the German might be the same word. CHARLES: Ah, okay. JULIE: Because he was writing in German. Because he's looking for something that evokes the idea of ‘function word'. CHARLES: Oh. JULIE: So, if you were to take the ‘func' of ‘function' [Laughs] and the, I don't know, maybe in German there's some better explanation for making this into a particular word. But that's how I think of it. So, it's ‘function word'. And then category theorists took it from Carnap to mean a way to map a function in this category or when we're talking about Haskell, a function of this type, to a function of another type. CHARLES: Okay. JULIE: And so, it takes the entire function, preserves the structure of the function just like negation preserves the structure of the sentence, and maps the whole thing to just a different context. So, if you had a function from A to B, functor can give you a function from maybe A to maybe B. CHARLES: Right. JULIE: So, it takes the function and just maps it into a different context. CHARLES: Right. So, a JavaScript example is if I've got an array of ints and a function of ints to strings, I can take any array of ints and get an array of strings. JULIE: Right. CHARLES: Or if I have a promise that has an int in it, I can take that same function to get a promise of a string. JULIE: Yeah. CHARLES: Yeah. I had no idea that it actually came from linguistics. JULIE: Yeah. [Laughs] CHARLES: So actually, the category theorists even… it digs deeper than category theory. They were actually borrowing concepts. JULIE: They were, yes. CHARLES: We just always are borrowing concepts. ELRICK: I like the borrowing of concepts. JULIE: Yeah. ELRICK: I think where people struggle with certain things, it's tying it back to something that they're familiar with. So, that's where I get… my mind is like [makes exploding sound] “I now get it,” is when someone ties it back to something that I am… CHARLES: Right. ELRICK: Familiar with. Like Charles' work with the JavaScript, tying it with JavaScript. I'm like, “Oh, now I see what they're talking about.” JULIE: Right. CHARLES: because you realize, you're using these concepts. People are using them, just they're using them anonymously. JULIE: Right. ELRICK: True. CHARLES: They don't have names for them. JULIE: Right. ELRICK: True. CHARLES: It's literally like an anonymous function and you're just taking that lambda and assigning it to a symbol. JULIE: Yeah. CHARLES: You're like “Oh wait. I've been using this anonymous function all over the place for years. I didn't realize. Boom. This is actually a formal concept.” ELRICK: True. And I think when people say like “Don't reinvent the wheel” it's a great statement for someone that has seen a wheel already. [Laughter] ELRICK: You know what I'm saying? If you never saw a wheel, then your'e going to reinvent the wheel because you're like “Aw man. This doesn't exist.” [Chuckles] JULIE: Yeah. ELRICK: But if people are exposed to these concepts, then they wouldn't reinvent the wheel. CHARLES: Right. JULIE: Right. Yeah. CHARLES: Instead of calling in some context, calling it a roller. [Chuckles] It's a round thingy. [Laughter] JULIE: Right. Yeah, so that's a little bit what I tried to do in my monoid talk in London. I tried to give some history of monoid, where this idea comes from and why it's worth talking about these things. CHARLES: Yeah. JULIE: Why it's worth talking about the structure. CHARLES: So, why is it worth the… where did it come from and why is it worth talking about? JULIE: Oh, so back when Boole, George Boole, when he decided to start formalizing logic… CHARLES: George Boole also, he was a career-switcher too, right? He was a primary school teacher. JULIE: Right, yeah. CHARLES: If I recall. He actually, he was basically teaching. Primary school is like elementary school in England, right? JULIE: I believe so, yes. CHARLES: Yeah. I think he was like, he was basically the US equivalent of an elementary school teacher who then went on to a second and probably, thankfully a big career that left a big legacy. JULIE: Right. Although no one knew exactly how big the legacy was really, until Claude Shannon picked it up and then just changed the whole world.[Laughs] Anyway, so Boole, when he was trying to come up with a formal algebra of logic so that we could not care so much about the semantic content of arguments (we could just symbolize them and just by manipulating symbols we could determine if an argument was logically valid or not), he was… well, for disjunction and conjunction which is AND and OR – well, disjunction would be the OR and conjunction the AND – he had prior art. He had addition and multiplication to look at. So, addition is like disjunction in some important ways. And multiplication is like conjunction in some important ways. And I think it took me a while to see how addition and disjunction were like each other, but there are some important ways that they're like each other. One of them is that they share their identity values. If you think of, it's sort of like binary addition and binary multiplication because in boolean logic there's only two values: true or false. So, you have a zero and a one. So, if you think of them as being like binary addition and binary multiplication then it's easier to see the connection. Because when we think of addition of just integers in a normal base 10 or whatever, it doesn't seem that much like an OR. [Laughs] CHARLES: Mmhmm. No, it doesn't. JULIE: [Inaudible] like a logical OR. So, it took me a while to see that. But they're also related then to set intersection and union where intersect-… CHARLES: So can… Let's just stop on that for a little bit, because let me parse that. So, for OR I've got two values, like in an ‘if' statement. This OR that. If I've got a true value then I can OR that with anything and I'll get the same anything. JULIE: Right. CHARLES: So, true is the identity value of OR, right? Is that what you're saying? So, one… JULIE: Well, it's false that's the identity of OR. CHARLES: Oh, it is? JULIE: Zero is the identity of addition. CHARLES: Wait, but if I take ‘false OR one' I get… oh, I get one. JULIE: Right. CHARLES: Okay. So, if I get ‘false OR true', I get true. Okay, so false is the identity. JULIE: Yeah. CHARLES: Oh right. You're right. You're right. Because… okay, sorry. JULIE: So, just like in addition, zero is the identity. So, whatever you add to zero, that's the result, right? You're going to get [the same] CHARLES: Right. JULIE: Value back. So, with OR false is the identity and false is equivalent to zero. CHARLES: [Inaudible] ‘False OR anything' and you're getting the anything. JULIE: Right. So, the only time you'll get a false back is if it's ‘false OR false', right? CHARLES: Right. Mmhmm. JULIE: Yeah. So, false is the identity there. And then it's sort of the same for conjunction where one is the identity of multiplication and one is also the… I mean, true is then the identity of logical conjunction. CHARLES: Right. Because one AND… JULIE: ‘True AND false' will get the false back. [Inaudible] CHARLES: Right. ‘True And true' you can get the true back. JULIE: Yeah. CHARLES: Okay. JULIE: And it's also then true, getting back to what we were talking about, semirings, it's also true that false is a kind of annihilator for conjunction. That's sort of trivial, because… CHARLES: Oh, because you annihilate the value. JULIE: Right. When there's only two values it's a little bit trivial. But it is [inaudible]. So… CHARLES: But it's [inaudible]. Yeah. It demonstrates the point. JULIE: Right. CHARLES: So, if I have yeah, ‘false AND anything' is just going to be false. So, I annihilate whatever is in that position. JULIE: Right. CHARLES: And the same thing as zero is the annihilator for multiplication, right? JULIE: Right. CHARLES: Because zero times anything and you annihilate the value. JULIE: Yeah. CHARLES: And now I've got… okay, I'm seeing it. I don't know where you're going with this. [Laughter] ELRICK: Yeah. CHARLES: But I'm there with you. ELRICK: Yup. JULIE: And then it turns out there are some operations from set theory that work really similarly. So, intersection and union are similar but the ones that are closer to conjunction/disjunction are disjoint unions and cartesian products. So we don't need to talk about those a whole lot if you're not into set theory. But anyway… CHARLES: I like set theory although it's so hard to describe without pictures, without Venn diagrams. JULIE: It is. It really is, yeah. So anyway, all of these things are monoids. And they're all binary associative operations with identity elements. So, they're all monoids. And so, we've taken operations on sets, operations on logical propositions, operations on many kinds of numbers (because not all kinds of addition and multiplication I guess are associative), and we can kind of unify all of those into the same framework. And then once we have done that, then we can see that there's all these other ‘sets'. Because most of the kinds of numbers are sets and there are operations on generic sets with set theory. So, now we can say “Oh. We can do these same kinds of operations on many other kinds of sets, many other varieties of sets.” And we can see that same pattern. And then we can get a kind of intuition for “Well, if I have a disjunctive monoid where I'm adding two things or I'm OR-ing two things…” Because even though those are logically very similar, intuitively and in terms of what it means to concatenate lists versus choosing one or the other, those obviously have different practical effects. CHARLES: So, I'm going to try and come up with some concrete examples to maybe… JULIE: Okay, yeah. CHARLES: A part of them will probably be like in JavaScript, right? So, to capture the idea of a disjunctive monoid versus a conjunctive monoid. So, a disjunctive monoid is like, so in JavaScript we're got two objects. You concat them together and it's like two maps or two hashes. So, you mash them together and you get… so, for the disjunctive one you'd have all the keys from both of the hashes inside the resulting object. You take two objects. Basically we call it object assign in JavaScript where you have basically the empty object. You can take the empty object and then take any number of objects. And so, we talked about… JULIE: That would become a disjunctive monoid, right? CHARLES: That would be a disjunctive monoid because you're like basically, you're OR-ing. Yeah. JULIE: You're kind of, [inaudible] CHARLES: Hard to find the terminology. JULIE: Yeah. CHARLES: But like object assign would be a disjunctive monoid because you're like mashing these two objects. And the resulting object has all of the things from both of them. JULIE: Right. So, it's like a sum of the two, right? CHARLES: Right, right. Okay, so then another one would be like min or max where you've got this list of integers and you can basically take any two integers and you can mash them together and if you're using min, you get the one that's smaller. Basically, you're collapsing them into one value but you're actually just choosing one of them. Is that like… JULIE: Yeah. CHARLES: Would that be like a conjunctive monoid? JULIE: No, that's also disjunctive but that's more like an OR than like a sum. CHARLES: Okay. JULIE: Right. So, that's what I said. It's hard to think of disjunctive monoids I think because there's really two varieties. There's some underlying logical similarity, like the similarity in the identity values. But they're also different. Summing two things versus choosing one or the other are also very different things in a lot of ways. CHARLES: Right. Okay. JULIE: And so, I think the conjunctive monoids are all a little bit more similar, I think. [Chuckles] But the disjunctive monoids are two broad categories. And we don't really have a monoid in Haskell of lists where you're choosing one or the other. The basic list monoid is you're concatenating them. So, you're adding two lists or taking the union of them. But for maybe, the maybe type, we do have monoids in Haskell where you're just choosing either the first just value that comes up or the last just value that comes up. So, we do have a monoid of choice over the maybe type. And then we have a type class called alternative which is monoids of choice for… so, they're disjunctive monoids but instead of adding the two things together, they're choosing one or the other. CHARLES: Okay. JULIE: Though we have a type class for that. [Laughs] CHARLES: [Sighs] Oh wow. Yeah. JULIE: Mmhmm, yeah. CHARLES: I'l have to go read up on that one. JULIE: That type class comes up the most when you're parsing, because you can then parse… like if you found this thing, then parse this thing. But if you haven't found this thing, then you can keep going. And if you find this other thing later, then you can take that thing. So, you allow the possibility of choice. The first thing that you come to that matches, take that thing or parse that thing. So, that type class gets mostly used for parsing but it's not only useful for parsing. CHARLES: Okay. JULIE: So yeah. That's the most of the time when I've used it. CHARLES: Is this when you're like parsing JSON? Or is this when you're just searching some stream for some value? Like you just want to run through it until you encounter this value? Or how does that…? JULIE: Right. Say you want to run through it until you find either this value or this value. I've used it when I've been parsing command line arguments. So, let's say I have some flags that can be passed in on my command line command. There are some flags that could be passed in. So, we'll parse until we find this thing or this thing. This flag or this flag. So, if you find this flag, then we're going to go ahead and parse that and do whatever that flag says to do. If you don't find that first flag then we can keep parsing and see if you find this other flag, in which case we'll do something different. CHARLES: Okay. JULIE: It'll take the first match that it finds. Does that make sense? CHARLES: Yeah, yeah, yeah. It does. But I'm not connecting how it's a monoid. [Laughs] JULIE: How is that a monoid? Well, because it's a monoid of OR-ing CHARLES: What's the identity value or the empty value in that case? JULIE: Well, the empty value would be… let's say you have maybes. Let's say you have some kind of maybe thing, so you're parser is going to return maybe this thing, maybe whatever you're parsing. Like maybe string. CHARLES: Yeah, yeah. JULIE: So, it's going to return a maybe string. So well, nothing would be the empty. CHARLES: Okay. JULIE: But nothing is like the zero because it's a disjunction, logical OR. So, only when you have two nothings will you get back a nothing. Otherwise, it will take the first thing that it finds. CHARLES: Okay. I see. JULIE: Yeah. So, the identity then is the nothing, like false is the identity for disjunction. CHARLES: Mmhmm. Okay. JULIE: Yeah. CHARLES: [Inaudible] JULIE: Yeah. If you have nothing or this other thing, then you return this other thing. Then you return the maybe string. If you have two nothings, then you get in fact nothing. Your parsing has failed. CHARLES: Right, because you've got nothing. JULIE: Because you've got nothing. There was nothing to give you back. CHARLES: So, you concatenated all of the things together and you ended up with nothing. JULIE: Right, because there was nothing there. CHARLES: Right. [Laughs] JULIE: You found nothing. So, it's useful when you've got some possibilities that could be present and you just want to keep parsing until you find the first one that matches. And then it'll just return whatever. It'll just parse the first thing that it matches on. CHARLES: Okay, okay. JULIE: Does that make sense? CHARLES: Yeah. No, I think it makes sense. JULIE: I'm not sure. Because I feel like I kind of went down a rabbit hole there. [Laughs] CHARLES: Yeah. [Laughs] No, no. I think it makes sense. And as a quick aside, I think… so, I was, when we were talking about min and max, are min and max also like a semiring? Because negative infinity is the annihilator of min and it's the identity of max. and positive infinity is the annihilator of max but it's the identity of min. JULIE: I guess. I don't really think of min and max as having identities. Is that how [inaudible]? CHARLES: I'm just, I don't know. Well, I think if you have negative infinity and you max it with anything, you're going to get the anything, right? Negative infinity max one is one. Negative infinity/minus a billion is minus a billion. JULIE: Yeah, okay. CHARLES: I don't know. Just off the cuff. I'm just trying to… annihilators sound cool. And so… [Laughter] CHARLES: And so I'm like, I'm trying to find annihilators. JULIE: Yeah, they are cool. CHARLES: [Laughs] JULIE: One of my friends on Twitter was just talking about how he used the intuition at least of a semiring at work because he had this sort of monoid to concatenate schedules. So, he's got all these different schedules and he's got this kind of monoid to concatenate them, to merge the schedules together. But then he's got this one schedule that is special. And whenever something is in this schedule, it needs to hard override every other schedule. CHARLES: Right. JULIE: And so, that was like the annihilator. So, he was thinking of it as a semiring, because that hard override schedule is like the annihilator of all the other schedules. CHARLES: Yeah. JULIE: If anything else exists on this day or whatever, then it'd just get a hard override. So, there's a real world use. [Laughs] CHARLES: Yeah, a real world example. That's the thing that I'm finding, is that all these really very crystalline abstractions, they still play out very well I think in the real world. And they're useful as a took in terms of casting a net over a problem. Because you're like… when I'm faced with something new, I'm like “Well, let's see. Can I make it a functor?” And if I can, then I've unlocked all these goodies. I've unlocked every single composition pattern that works with functor. JULIE: Right, right. CHARLES: And it's like sometimes it fits. It almost feels like when you're working on something at home and you've got some bolt and you're trying on different diameters. So you're like, “Oh, is it 15 millimeter? Is it 8 millimeter?” JULIE: Right. [Laughs] CHARLES: “Like no, okay. Maybe it'll work with this.” But then when it clicks, then you can really ratchet with some serious torque. JULIE: Right, right. Yeah. CHARLES: So, yeah. Definitely trying to look for semirings [Laughs] is definitely beyond my [can] at this point. But I hope to get there where it can be like, if it's a fit, it's a fit. That's awesome. JULIE: Right. Yeah, it's kind of beyond my can too. Semirings are still a little bit new for me and I can't say that I find them in the wild as it were, as often as monoids or something. But I think it just takes seeing some concrete examples. So, now you know this idea exists. If you just have some concrete examples of it, then over time you develop that intuition, right? CHARLES: Right. JULIE: Like “Okay, I've seen this pattern before.” [Chuckles] CHARLES: yeah. Basically, every time now I want to fold a list, or like in JavaScript, any time you want to reduce something I'm like “There's a monoid here that I'm not seeing. Let me look for it.” JULIE: Yeah. Oh, that's cool, yeah. CHARLES: Because like, that's basically, most of the time you're doing a reduce, then like I said that's the terminology for fold in JavaScript, is you start with some reducible thing. Then you have an initial value and a function to actually concatenate two things together. JULIE: Right. CHARLES: And so, usually that initial state, that's your identity. And then that function is just your concat function from your monoid. And so, usually anytime I do a reduce, there's the three pieces. Boom. Identity value, concatenation function, it's usually right there. And so, that's the way I've found of extracting these things, is I'm very suspicious every time I'm tempted to… JULIE: [Laughs] CHARLES: A fold. I'm like “Hmm. Where's the monoid I'm missing? Is it [under the] couch?” Like, where is it? [Laughs] Because it just, it cleans it up and it makes it so much more concise. JULIE: Oh yeah, that's awesome. CHARLES: So anyhow. JULIE: Have we totally lost Elrick? ELRICK: Nope, I'm still here. JULIE: Okay. [Laughs] ELRICK: I'm sitting in and listening to you two break down these complex topics is really good. Because you guys break them down to a level where it's consumable by people that barely understand it. So, I'm just sitting here just soaking everything in like “Oh, that's awesome.” Taking notes. Yes, okay, okay. [Laughter] JULIE: Cool. ELRICK: So, I'm like riding the train in the back just hanging out, feeling the cool breeze while you guys just pull the train ahead in… [Laughter] ELRICK: In the engine department, you know? It's awesome. CHARLES: Yeah. ELRICK: I don't know if they're related. But you were talking about semirings and I heard of semigroups or semigroups. I have no idea if those two things are related. Are they related or [inaudible]? JULIE: They're kind of related. So, a semigroup is like a monoid but doesn't have an identity value. CHARLES: What is an example of a semigroup out there in the wild? Because every time I find a semigroup, I feel like it's actually a monoid. JULIE: Well, you know I feel like that a lot, too. We do have a data type in Haskell that is a non-empty list. So, there is no empty list CHARLES: Ah, right. Okay. JULIE: So then you can concatenate those lists, but there's never an identity value for it. CHARLES: I see. JULIE: Yeah. So, that's a case. There's actually a lot of comparison functions, greater than and less than. I think those are semigroups because they're binary, they're associative, but they don't have an identity value. Like if you're comparing two numbers, there's not really an identity value there. CHARLES: Right. Well, would the negative infinity work there? Let's see. Like, negative infinity greater than anything would be the anything. Well, okay wait. But greater than, that takes numbers and yields a boolean, right? JULIE: Yeah, CHARLES: Right. So, it couldn't be… could it be a semigroup? Don't semigroups have to… Doesn't the [inaudible] function have to yield the same type as the operands? JULIE: Yes. CHARLES: But a non-empty list, that's a good one. Sometimes it's basically not valid for you to have a list that doesn't have any elements, right? Because it's like the null value or the empty value and it could be like a shopping cart on Amazon. You can't have a shopping cart without at least something in it. JULIE: Right. CHARLES: Or, you can't check out without something. So, you might want to say like the shopping cart that I'm going to check out is a non-empty list. And so, you can put two non-empty lists together. But yeah, there's no value you can mash together, you can concat with anything, that isn't empty. JULIE: Right. CHARLES: So, I guess going back to your question Elrick, I don't know if it's related to semiring. But semigroup is just, it's like one-half of monoid. It's the part that concats two values together. JULIE: Right. Well, yeah. And so, it's supposed to be half a group, right? But I don't remember… CHARLES: [Laughs] JULIE: [Inaudible] all of the group stuff is, all the stuff that these types have to have to be a group. And similarly, I forget what the difference between semiring and ring is. [Chuckles] Because a ring and a group I know are not the same thing. But I forget what the difference is, too. So, I kind of got a handle on what semigroups are, and I know all my Haskell friends are going to, when they hear this podcast they're going to tweet all these examples of semigroups at me, especially my coauthor for ‘Joy of Haskell', Chris Martin. He's really into semigroups. And so, I know he's going to be very disappointed in my inability to think… [Laughter] JULIE: To think of any good examples. But it's not something that I find myself using a lot, whereas semirings are something that I have started noticing a little bit more often. So, how a monoid relates to a group is something that I can't remember off the top of my head. And I know how semirings relate to monoids, but how monoids then relate to rings and groups, I can't really remember. And so, these things are sort of all related. But the relation is not something I can spill out off the top of my head. Sorry. [Laughs] CHARLES: No, It's no worries. You know, I feel like… ELRICK: It's all good. CHARLES: What's funny is I feel like having these discussions is exactly like the discussions people have with any framework of using one that we use a lot, which is EmberJS. But if you could do with React or something, it's like, how does the model relate to the controller, relate to the router, relate to the middleware, relate to the services? You just have these things, these moving parts that fit together. And part of… I feel like exploring this space is really, absolutely no different than exploring any other software framework where you just have these things, these cooperating concepts, and they do click together. But you just have to map out the space in your head. JULIE: Yeah. This is going to sound stupid because everybody thinks that because I know Haskell I must know all these other things. But I just had to ask people to recommend me a book that could explain the relationship of HTML and CSS, because that was completely opaque to me. CHARLES: [Laughs] Yeah. JULIE: I've been involved in the making now of several websites because of the books and stuff like that. And I have a blog. It's not WordPress or anything. I did that sort of myself. So, I've done a little bit with that. But CSS is really terrifying. And… CHARLES: Right. Like query selectors, rules, properties. JULIE: Yeah. ELRICK: [Laughs] CHARLES: Again, might as well be groups and semigroups and monoids, right? JULIE: Right, right. ELRICK: Yeah. CHARLES: [Laughs] ELRICK: That is really interesting. [Chuckles] I've never heard anyone make that comparison before. But it's totally true, now that I'm thinking about it. JULIE: Yeah, yeah. CHARLES: Yeah. In the tech world we are so steeped in our own jargon that we could be… we can reject one set of jargon and be totally fine with another set. Or be like, suspicious of one set of concepts working together and be totally fine with these other designations which are somewhat arbitrary but they work. JULIE: Right. CHARLES: So, people use them. JULIE: So, it's like what you've gotten used to and what you're familiar with and that seems normal and natural to you. [Chuckles] So, the Haskell stuff, most of it seems normal and natural to me. And then I don't understand HTML and CSS. So, I bought a book. [Laughter] CHARLES: Learning HTML and CSS from first principles. JULIE: Yes, yeah. I just wanted to understand. I could tell that they do relate to each other, that there is some way that they click together. I can tell that by banging my head against them repeatedly. But I didn't really understand how, and so yeah. So, i've been reading this book to [Laughs] [learn] HTML and CSS and how they relate together. That's so important, just figuring out how things relate to each other, you know? CHARLES: Yeah. ELRICK: Yeah. That is very true. JULIE: Yeah. ELRICK: We can trade. I can teach you HTML and CSS and you can teach me Haskell. JULIE: Absolutely. ELRICK: [Laughs] CHARLES: There you go JULIE: [Laughs] ELRICK: Because I'm like, “Ooh.” I'm like, “Oh, CSS. Great. No problem.” [Laughter] ELRICK: Haskell, I'm like “Oh, I don't know.” JULIE: Yeah. CHARLES: Yeah. ELRICK: [Laughs] CHARLES: No, it's amazing [inaudible] CSS. ELRICK: Yeah. CHARLES: It is, it's a complicated system. And it's actually, it's in many ways, it's actually a pretty… it's a pretty functional system, CSS is at least. The DOM APIs are very much imperative and about mutable state. But CSS is basically yeah, completely declarative. JULIE: Right. CHARLES: Completely immutable. And yeah, the workings of the interpreter are a mystery. [Laughs] ELRICK: Yup. JULIE: YEs. And you know, for the Joy of Haskell website we use Bootstrap. And so, there was just like… there's all this magic, you know? [Laughs] ELRICK: Oh, yeah. CHARLES: Yeah. JULIE: Oh look, if I just change this little thing, suddenly it's perfectly responsive and mobile. Cool. [Laughter] JULIE: I don't know how it's doing this, but this is great. [Laughs] CHARLES: Yeah. Oh, yeah. It's an infinite space. And yeah, people forget what is so easy and intuitive is not and that there's actually a lot of learning that happened there that they're just taking for granted. JULIE: I think so many people start from HTML and CSS. That's one of their first introductions to programming, or JavaScript or some combination of all three of those. And so, to them the idea that you would be learning Haskell first and then coming around and being like “Oaky, I have to figure out HTML,” that [seems very] strange, right? [Laughter] CHARLES: Yeah. Well, definitely probably stepping into bizarro world. JULIE: And I went backwards. But [Laughs] CHARLES: Yeah. JULIE: Not that it's backwards in terms of… just backwards in terms of the normal way, progression of [inaudible] CHARLES: Yeah. It's definitely the back door. Like coming in through the catering kitchen or something. JULIE: Yes. CHARLES: Instead of the front door. Because you know the browser, you can just open up the Dev Tools and there you are. JULIE: Exactly, yeah. CHARLES: The level of accessibility is pretty astounding. And so, I think t's why it's one of the most popular avenues. JULIE: Oh, definitely. Yeah. ELRICK: It's the back door probably for web development but not the back door for programming in general. JULIE: Mm, yeah. Yeah. CHARLES: Yeah. It seems like Haskell programming has really started taking off and that the ecosystem is starting to get some of the trappings of a really less fricative developer experience in terms of the package management and a command line experience and being able to not make all of the tiny little decisions that need to be made before you're actually writing ‘hello world'. JULIE: Right. ELRICK: Interesting. Haskell has a package manager now? CHARLES: Oh, it has for a while. ELRICK: Oh, really? What is it called? I have no idea? Do you know the name off the top of your head? CHARLES: So, I actually, I'm not that familiar with the ecosystem other than every time I try it out. So I definitely will defer this question to you, Julie. JULIE: This is going to be a dumb question, I guess. What do we mean by package manager? CHARLES: So, in JavaScript, we have npm. The concept of these packages. It's code that you can download, a module that you can import, basically import symbols from. And Ruby has RubyGems. And Python has pip. JULIE: Okay, okay. CHARLES: Emacs has Emacs Packages. And usually, there's some repository and people could publish to them and you can specify dependencies. JULIE: Right, yeah. Okay, so we have a few things. Hackage is sort of the main package repository. And then we have another one called Stackage and the packages that are in Stackage are all guaranteed to work with each other. CHARLES: Mm, okay. JULIE: So, on Hackage, some of the packages that are on Hackage are not really maintained or they only work with some old versions of dependencies and stuff like that, so the people who made Stackage were like “well, if we had this set of packages that were all guaranteed to work together, the dependencies were all kept updated and they all can be made to work together, then that would be really convenient.” And then we have Cabal and we have Stack are the main… and a lot of people use Nix for the same purpose that you would use Cabal or Stack for building projects and importing dependencies and all of that. CHARLES: Right. So, Cabal and Stack would be roughly equivalent then to the way we use Yarn or JavaScript and Bundler in Ruby. You're solving the equation for, here's my root set of dependencies. Go out and solve for the set of packages that satisfy. Give me at least one solution and then download those packages and [you can] run them. JULIE: Yeah, yeah. Right, so managing your dependencies and building your project. Because Haskell's compiled, so you've got to build things. And so yeah, we have both of those. CHARLES: And now there's like web frameworks and REST frameworks. JULIE: Oh there are, yeah. We have… CHARLES: All kinds of stuff now. JULIE: We had this big proliferation of web frameworks lately. And I guess some of them are very good. I don't really do web development. But the people I know who do web development in Haskell say that some of these are very good. Yesod is supposed to be very good. Servant is sort of the new hotness. And I haven't used Servant at all though, so don't ask me questions about it. [Laughter] JULIE: But yeah, we have several big web frameworks now. There are still some probably big holes in the Haskell ecosystem in terms of what people want to see. So, that's one thing that people complain about Haskell for, is that we don't have some of the libraries they'd like to see. I'd like to see something… I would really like to see in Haskell something along the lines of like NLTK from Python. CHARLES: What is that? JULIE: Natural language toolkit. CHARLES: Oh, okay. JULIE: So yeah, Python has this… CHARLES: Yeah, Python's got all the nice science things. JULIE: They really do. And Haskell has some natural language processing libraries available but nothing along the lines of, nothing as big or easy to use and stuff as NLTK yet. So, I'd really like to see that hole get filled a little bit better. And you know… CHARLES: Well, there you go. If anyone out there is seeking fame and fortune in the Haskell community. JULIE: That's actually why I started learning Python, was just so that I could figure out NLTK well enough to start writing it in Haskell. [Laughter] JULIE: So, that's sort of my ambitious long-term project. We'll see how that goes. [Laughs] CHARLES: Nice. Before we wrap up, is there anything going on, coming up, that you want to give a shoutout to or mention or just anything exciting in general? JULIE: Yeah, so on March 30th I'm going to be giving a talk at lambda-squared which is going to be in Knoxville and is a new conference. I think it's just a single-day conference and I'm going to be giving a talk about functors. So, I'm going to try to get through all the exciting varieties of functors in a 50-minute talk. CHARLES: Ooh. JULIE: So, we'll see how that goes. Yeah. And I am still working with Chris Martin on ‘The Joy of Haskell' which should be finished this year, sometime. I'm not going to… [Laughter] JULIE: Give any more specific deadline than that. And in the process of writing Joy of Haskell, I was telling him about some things that, some things that I think are really difficult. Like in my experience, teaching Haskell some places where I find people have the biggest stumbling blocks. And I said, “What if we could do a beginner video course where instead of throwing all of these things at people at once, we separated them out?” And so, you can just worry about this set of stumbling blocks at one time and then later we can talk about this set of stumbling blocks. And so, we're doing… we're going to start a video course, a beginner Haskell video course. I think we'll be starting later this month. So, I'm pretty excited… CHARLES: Nice. JULIE: About that. Yeah. CHARLES: Yeah, I know a lot of people learn really, really well from videos. There's just some… JULIE: Yeah. [Inaudible] for me, so I'm a little nervous. But [Laughs] CHARLES: Yeah, especially if you can do… are you going to be doing live coding examples? Building out things with folks? JULIE: Yeah. CHARLES: Yeah. Well, you just needn't look no further than the popular things like RailsCasts and some of the… yeah, there's just so many good video content out there. Yeah, we'll definitely be looking for the. JULIE: Cool. CHARLIE: Alright. Well, thank you so much, Julie, for coming on. JULIE: Well, thank you for having me on. Sorry I went down some… I went kind of down some rabbit holes. Sorry about that. [Laughs] CHARLES: You know what? You go down the rabbit holes, we spend time walking around the rabbit holes. JULIE: [Laughs] CHARLES: There's something for everybody. So… [Laughter] CHARLES: And ultimately we're strolling through the meadow. So, it's all good. JULIE: [Laughs] Yeah. CHARLES: Thank you too, Elrick. JULIE: It was nice talking to you guys again. CHARLES: Yeah. ELRICK: Yeah, thank you. CHARLES: If folks want to follow up with you or reach out to you, what's the best way to get in contact with you? JULIE: I'm @argumatronic on Twitter and my blog is argumatronic.com which has an email address and some other contact information for me. So, I'd love to hear questions, comments. [Laughs] Yeah. I always [inaudible]. CHARLES: Alright, fantastic. JULIE: To talk to new people. CHARLES: Alright. And if you want to get in touch with us, we are @TheFrontside on Twitter. Or you can just drop us an email at contact@frontside.io. Thanks everybody for listening. And we will see you all later.

Sam Cates: @SamCates | GE Ventures Show Notes: 02:01 - What Corporate Investing Looks Like 03:48 - Presenting Ideas For Funding 09:01 - Democratizing Venture Capital 10:17 - ICOs and Cryptocurrency 13:53 - Evaluating Companies to Fund 21:09 - Investing in Potential Competitors 24:42 - Looking For Funding as a Company 28:04 - “Mentoring” Ideas/Companies 30:07 - Monitoring/Evaluating Company Metrics 32:47 - Putting Together a Basic Business Plan 36:05 - Making Choices: Investor and Company-wise Resources: Resin.io Series A, B, C Funding Angel Investor Seed Money Initial Coin Offering (ICO) AngelList Crunchbase Fred Wilson's Blog: (AVC.com) Transcript: CHARLES: Hello everybody and welcome to The Frontside Podcast, Episode 92. My name is Charles Lowell, a developer here at The Frontside and I am your podcast host-in-training kicking it off in 2018. [Inaudible] of our first episode. We've got Elrick also joining us. Hello, Elrick. ERICK: Hey. How you doing, Charles? CHARLES: I'm doing well. I'm doing well. You having a good new year so far? ERICK: Yeah, it's great. There's a snowstorm passing through today. So, I'm going to break in the New Year shoveling. CHARLES: Let us know if we need to parachute in some shovels for you. ERICK: [Laughs] CHARLES: And then with us today, we have Sam Cates on the show who is… a lot of times we have developers on the show. He's actually a venture… what would you describe yourself as? SAM: Yeah, I'd say I'm a venture investor with GE Ventures. So, on the corporate investing side. CHARLES: Okay. Now, I didn't even know that GE actually had a corporate investing side. Is that pretty common for a large company? SAM: You know, it's becoming increasingly common. I think in 2015 there was actually a peak of activity coming from corporate venture capital groups. And I've only seen the number of firms escalate since then. Although the dollars invested stays pretty consistent. But if you look at a lot of big companies, particularly in the common tech world like Cisco, Google, Intel, they have historically had large venture firms inside of themselves. And then GE and a lot of other industrials have since followed suit. We've been at it for about five years and we see it increasingly. CHARLES: And so, have you been with them since the beginning? SAM: Yeah, just about. I've actually been with GE for about nine years now. So, I was on the operating side in a number of the industrial businesses before I joined GE Digital and then GE Ventures. And so, it was just after GE Ventures got kicked off. CHARLES: Oh, that's exciting. So, what is it… now, we actually got connected to you through one of the companies that you actually invested in. It's something that we use and we're very interested in. Why don't you tell us a little bit about what your job looks like on a day-to-day basis and what companies you invest in? SAM: Sure. I really focus a lot of my time on Internet of Things companies. So, that's a really big trend that GE has been a part of and a leader in over the past few years. And so, we spend time investing in companies that are directly working with GE or playing in similar spaces to us. And so, Elrick and I actually met at a hackathon for one of those companies. And I always like to use that as an example because it's a good one, to demonstrate the kinds of investments we make. And that's Resin.io. I know you guys have done an episode or two talking with them. But that for example was a ‘Series A' investment that we made about two years ago. And then company essentially helps developers build connected products. And so, that's something that GE cares a lot about. We had people inside the company who found the product and loved it and that's actually how we met. CHARLES: When you say ‘Series A', can you give a brief overview of what the different stages of funding of a startup might be? SAM: Yeah, yeah, certainly. So, maybe if I take a step back and answer your original question on what I do on a day-to-day basis. A lot of my job is meeting with all kinds of new companies, whether they be early stage, usually things that would be seed funding – and we'll go into what some of those things mean – all the way through the late stage which would be companies that are maybe on the border of going public or are already profitable. And so, if we go into what kinds of investors there are, I think that's probably an interesting subject to talk more about. But they're a whole wide variety. When I said ‘Series A' I just meant a company that was at what we would call the ‘Series A' stage, and the letters act just like you'd expect. So, there's ‘Series A', ‘Series B', ‘Series C', and so on. And they all, they tend to look similar at those stages in terms of sizes and progress. But there is a range, and no two company is the same. ERICK: In today's world, it's very easy for people to create a startup. They can write some code and they can either come up, get a Raspberry Pi or some microcontrollers or whatever it is, and either do an IoT startup or a software startup. Now, when you get to the point where you have an idea and you kick it off initially, how do you go about then saying, “Let me get some funding.” How do you even get funding? SAM: Sure, yeah. And to your point, there's a huge range of technologies that are making it easier to start almost any kind of company. It's a great time to be an entrepreneur, whether it be 3D printing for hardware products, all the technologies that you were mentioning, AWS, all this stuff is contributing to reducing the cost to allow companies or people to create companies. And so, once people have gone out and experimented with some of these things and built what they think is a product the market wants, often if they require more money which may be for acquiring customers through things like Facebook Ads or simply doing further product development to make sure the product is somewhere that more customers could use it, often they can't finance it just through their own revenue. And so, there are typical stages and types of investors that people go approach looking for money. ERICK: Okay. What are those Series? I remember you mentioned something like a ‘Series A' investment. So, initially when you're looking for an investment, is that where you would… category you would be in as a startup looking for investment? They would consider you a ‘Series A' startup? SAM: Well, I want to caveat and just say every company is different. So, I see companies that… ERICK: Gotcha. SAM: Start out at a much later stage because they're able to bootstrap to that point. And bootstrap is the word that I use for a company that funds its own investment. They get paid by customers and they use that money to continue building the product. But if I talk about the range of types of financing a company may go for, I think the way that most people categorize this, first people often raise from friends and family or angels. And so, it's just money to get off the ground and maybe to pay the rent while you're doing some of that experimenting we were talking about. And then commonly after that is a seed round. And a seed round tends to be a little more institutional. So, it's maybe a more formal set of funds who exclusively invest in companies that are often pre-revenue but they have a product, or at least the beginnings of a product. And so, that's a really common category of investors. And then you get to ‘Series A' and the letters can escalate from there to the point where… ERICK: Gotcha. SAM: There can be some later rounds when they'd be ‘Series F' or even beyond, I guess. CHARLES: Right. So now, what are generally the terms on these? So, for my angel investments or my seed investments, I assume what distinguishes these is essentially how much ownership of the company you're getting for how much money. And those kind of, those change as the product solidifies. SAM: Yeah. CHARLES: And the potential becomes more visible. SAM: Yeah, it's a wide… and again, these are all… the venture is a world of ranges. There's a really wide difference between the two ends of any spectrum. So, I'll just talk in generalities though. So, I think the latest report that I've seen at least for an annual basis was PitchBook's 2016 report. And they were laying out some of the medians. So, for seed stage deals I believe it was something like one and a half million dollars raised was the median on a pre-money valuation of six and a half million. And that just means the company is worth, investors say the company is worth six and a half million dollars today. And we're going to give you a million and a half dollars invested at that price. CHARLES: So roughly, a sixth… they would take a sixth of the company then in return? SAM: Yeah. CHARLES: Okay. ERICK: Ah. CHARLES: Okay. ERICK: Okay. CHARLES: I see. That makes sense. So now, back to Elrick's original question. If I'm, I've got my product. Or I've got this idea. I've written some code. I've turned it into a prototype product. Maybe I'm moving through these various stages. What type of VC am I going to be looking for? How do I actually find the right type to be talking to? I guess what types are there even? SAM: Yeah. And one part… we mentioned a lot of the technologies that are making it easier to start companies. One part that also makes it easier is the proliferation of financing options, whether it be even more investors in these traditional structures we talked about like seed and A. And then there are other options that are emerging, things like you see a lot of people raising through what they call Initial Coin Offerings or ICOs. And then there are also things like AngelList which are attempting to democratize the investing process, make it more accessible. So traditionally, a lot of the seed A, B investors, they tend to be network-based, which can be a challenge for a lot of people that are maybe not in Silicon Valley or not a part of that network already. And so, one thing you can do is obviously go search databases that are on the web, things like Crunchbase. It's a free resource. It has a lot of deal history for investments that people have made. And it's a great resource for knowing, “Okay, this investor cares about these things.” And then in addition to that, there are also platforms that people can put their companies on. Like I mentioned, AngelList. And that's somewhere that you can list your company, you can meet investors, and they actually have some backend to actually support the investing process as well. CHARLES: So, there were two acronyms in there, or two specific technologies. [Chuckles] CHARLES: You talked about ICOs which I assumed that you said it was Initial Coin Offering. Not like insane clown offering. [Laughter] CHARLES: Which I would love to see. And then AngelList. So traditionally, these had been very network-based which brings to mind the capitalists of Old England or whatever where there's a bunch of people with cigars in a room and I realize it's not actually like that. What are each of these things? The AngelList and the ICOs? And how do they democratize that process? SAM: It's funny you should mention the old times. I think a good example of that is there are a lot of stories about the founding of General Electric. It's a 126-year-old company and back then it was largely, it was Thomas Edison working with I believe was JP Morgan to get it off the ground. And so, today there's still a bit of the network piece you're mentioning. But I think of AngelList as a place that you can essentially market to investors. If you think about the types of people that are on there, it's people that are looking to invest money in early stages in startups. And I'm not a big user of AngelList because I tend to be investing a little bit later. So, I really recommend anybody who's interested, just go check it out. It's I believe just Angel.co. CHARLES: And what about an ICO? SAM: So, an ICO is a more modern one. And it's kind of fraught with some concerns around regulations and transparency today. But I think since Thanksgiving there's been a massive wave of conversation about cryptocurrencies. And an ICO is essentially a way of creating your own cryptocurrency. The way I always explain to people, I love the analogy that people make around, think of it like I want to go build an amusement park. And in that amusement park, everything, rides, food, everything, is going to be denominated and payable in Sam-bucks. CHARLES: Ah, right. SAM: And… [Chuckles] And so, my options… CHARLES: [Laughs] That makes sense. SAM: Yeah. And my options are I can go to a bank and borrow money, I can go to investors and say, “Hey, give me the 10 million dollars it's going to take to build it,” or I can just go to the people in the place where I'm building it and say, “You want this amusement park to exist? Why don't you pre-buy these Sam-bucks?” And each one is going to cost a dollar today. And we create this universe of Sam-bucks and they're essentially valuable once you can use them in the park. And there are certainly exceptions. There are other versions of cryptocurrencies and other uses for them. But that's a conversation for another day. CHARLES: Ah, mm. SAM: I think that's just a good, easy way to understand it. CHARLES: Oh no, I like that. It's like, well not quite like carnival tickets. But yeah, that's something that everyone's familiar with. Same thing as the Xbox Marketplace. Very similar thing. So, the idea is you would buy a bunch of Sam-bucks… you would get them at pennies on the dollar, so to speak, today. SAM: Yeah, right. By the time it opens, maybe a hotdog would cost just one Sam-buck. CHARLES: Right. SAM: Whereas, when it's coming in, we'd have to spend five dollars to get that one Sam-buck. Right, the idea being those people who got in early will be rewarded. And you can see it's like a further extension of a Kickstarter or something else that you're allowing people to pre-buy into a network. CHARLES: Right. Right, okay. I can see that. ERICK: That's very interesting. [Laughs] CHARLES: And so, it's got a range of options too, because if you're really interested in the services you can go ahead and spend them on the services and get a lot of value that way or you can actually trade for someone who does want the services if you don't. SAM: I think that's exactly right. And it's just, the one that I think I would just caveat is there is a huge amount of concern at the moment, and maybe concern is too strong a word, but uncertainty around one, what are the value of these coins, these tokens? And two, how will governments react to something that looks potentially like a security or a currency? And so, that's something that still is being worked through. And even though they haven't figured that out there's still a massive amount of money being raised through these ICOs. CHARLES: [Laughs] So, it does beg the question. Why is a cryptocurrency necessary? Why not just use Xbox Marketplace points? Why not just say, “Here are Sam-bucks.” SAM: [Chuckles] CHARLES: And there's a row in my database. [Laughter] CHARLES: That's your balance of Sam-bucks. SAM: So, I think we're about to get way beyond the [inaudible] [Laughter] SAM: But I think the argument would be that some of these things are better decentralized. So in my example, you're right. That might just make more sense. But I think there are some examples around cryptocurrencies that are supporting a network of decentralized services where a centralized database historically was inconvenient or didn't provide the amount of transparency that people were looking for. CHARLES: Right, right. SAM: And so, that's a topic for a whole other podcast. CHARLES: Yeah, right. No, it makes sense. SAM: [Laughs] CHARLES: I think it's a matter of scale, right? If you're going to be just buying services but if you're going to have secondary markets where you're trading in this currency, I can see that. So, let's… [Chuckles] We'll reel that back in. SAM: [Chuckles] CHARLES: And ask a question that occurred to me. So now, we talked about your day-to-day. What exactly, when you're looking at a company to basically give money to, what are you looking for? What are the things you're like, “Oh man, I want to throw dollars at this company,” versus, “Mm. I'm going to keep them and give them some feedback and send them on their way.” SAM: There's always a set of factors that we evaluate. And I think the waiting is probably different for different types of investors. And then there's I'd say for me as a corporate VC being a part of GE, there's an extra lens which is, how is this relevant to GE? What does it mean for GE to be an investor? But if I think about just the kind of general industry lines it's: team is a really big one. So, who's building this company? Do I believe in their ability to reach this vision that they're laying out for me? Another one would be technology. What have they actually built? Is that hard to build? Do the things they want to build in the future, will those be hard to build? And do they have the skills and the people to do it? Then their technology, maybe an extension of that would be intellectual property. And besides intellectual property, just defensibility of a business in general. So then, you start thinking about, can somebody else just come along and to the same thing? Because if so, then maybe there's not a strong advantage in what the company has done so far. And then lastly, it's also just traction. How far along are they? How much have they proven the ability to execute on the plan that they're laying out? CHARLES: Right. ERICK: So, you're a corporate investor. So, there's other types of investor like an institutional VC? What are the differences between an institutional VC and a corporate VC and the other types of VC? Potentially what they'd be looking for, in terms of what they wanted best. SAM: Yeah. So, I think generally I categorize investors as institutional or corporate. And corporate [inaudible]… ERICK: Yeah. SAM: Corporate or strategic. And then there are people who exist on a spectrum there. But generally, an institutional means this is a group that is raising money from a set of limited partners who are the people who invest in the fund that are pension funds or wealthy individuals. They're large pools of institutional capital and their pure purpose is to earn return. And they may have a certain focus because they believe in this part of the market, or they like this kind of company or the stage of company. But essentially, their job is to return more money to the limited partners of that fund that were put in. That's their role in the world. And then on the corporate side, if we go the most extreme version of corporate VC, this is a group that is a part of a larger corporate. They're investing that company's money. So, in this case for me it's GE. I'm investing GE's money into these startups. And that means that I only have a single backer being GE. And I also maybe have a different lens, because my purpose is one, to earn financial return. I want to go out and I want to find good companies. I want to earn returns just like the other institutional venture capitalists. But I also have the goal of, and the strategic goal may differ by company, but for me it's about how can I help GE advance? How can I help GE understand a market? And how can GE be helpful to this company in achieving their goals? And so, for each company we use that lens as well, as a corporate. CHARLES: What I'm hearing is that you want to invest… I guess the thing is you can experience return that's not just cash. It's not just dollars. You'll experience return in raising the ocean of the business that GE is in, right? So… SAM: You said it much better than I did. [Laughter] CHARLES: Well, it's all… paraphrasing is actually easy. [Laughter] ERICK: Oh, yeah. SAM: An important skill. CHARLES: That makes a lot of sense. So, the question I have then is, you said you were looking for companies that kind of swim in a specific ocean. And each company is farther along. Are you usually finding this company I want to work with, like you are going out and finding them? Or they're coming to you looking for investment? Or is it really just, depends. SAM: So, we call that part of the process sourcing, sourcing investments. And they come from all over. So for us, there are a few different ways. One is we tend to be thesis-driven. Meaning we go out and we say, the world is changing in this way and therefore we're interested in this kind of company. And so, we'll proactively go out and research. We're also, I mentioned, a little later stage. So, I don't tend to do seed investments. I tend to do ‘Series A' and more often ‘Series B' and later. So, companies that have often already raised a seed round or raised a ‘Series A' round. So, I can actually search databases to say, “Okay, in the last two years who has raised a seed round or ‘Series A' round and these other things I'm looking for whether it be location or tied to investors or other things.” So, that's one way of being proactive is saying I want to go out and look for companies in this space that look like this. And that can be either like I mentioned, desktop research like searching the web, searching databases. Or it can be just going to conferences, right? So, on thing we spend a lot of time on in the IoT world is artificial intelligence and machine learning. It's been a big, big topic over the last year that a lot of people have invested in. So, we may go to different conferences that focus on that topic, meet lots of people that are working on it. Some companies, some individuals that are either investing in or advising their companies. And we'll talk to them. What companies are rising out of that space that we should be looking at? What technologies are changing in that space that we should be thinking about? And just trying to get smarter so that we can make the right investments and help the right companies find their way to work with GE and make our products better and help them advance their own enterprise. CHARLES: Are you investing with a mind that eventually GE might acquire this company and integrate it into GE itself? Or is it really just, “Hey, we're just going to take a part of it. We're going to have maybe a seat on the board to be able to steer a little bit. But we're pretty much going to let it be its own thing with its own autonomy and go where it was and just benefit through those secondary and tertiary effects.” SAM: Yeah, acquisitions from our portfolio by GE happened. But they're certainly not the explicit goal or our focus. I know we've had one, maybe two of our portfolio companies acquired by GE, one that I was directly working with called Bit Stew. So, we made the investment in the company. It was with the goal of using their data management platform for a lot of our applications. And at some point in working with GE and GE Digital, they decided, you know, this would make sense to be a part of GE. That wasn't why we made the investment. But it did end up being acquired by GE. And I know the team is doing really well. And it's been at GE for about a year now. So, it does happen. But when I said one or two, that's versus a portfolio of a hundred plus companies. CHARLES: Right. SAM: Since we started investing. And so, that's not what we're looking to do every time. Much more often it's about again, how does the company make GE more competitive and a better company, a better place to work. And then how do we help them advance their goals? Whether it be bringing them developers, or finding them other routes to market, or just being a customer. CHARLES: Right. SAM: So, that's really how we think about strategic value. There's a lot of different ways to create it. CHARLES: Yeah, I'm curious. Because it seems like also in a lot of these companies you're investing in potential competitors. Extensively you're operating if not in the exact same market, maybe very similar markets. There's a little bit of overlap. And so, you're kind of investing in potential competitors, right? So, where's the balance of here we're funding our competitors versus we're going to move into these markets ourselves. SAM: Yeah, and funding of “competitors” can happen. I think that we talk about that more in theory and say, “Oh sure, we'd be willing to fund a company that's out disrupting the space that we're playing in.” And we do that. It's rare that you see startups that are directly head-on competing with much more established companies like GE or other industrials or even other consumer companies. They don't take these companies head-on because that's not a way that startups have been successful in the past, right? We talk much more about disruption and saying, how is this company doing something that may indirectly compete with GE? So, you think about things like, for anybody that's not familiar with GE… actually, a lot of people associate us with our appliances which we actually don't manufacture anymore. That's [inaudible]. [Chuckles] SAM: We sold that business a few years ago. Almost everything we sell is like big, heavy industrial equipment. So, we sell aircraft engines, locomotives. We sell gas turbines, wind turbines. So, here and there a couple of things that do power generation. One trend that's affecting that industry is distributed generation of energy, energy storage. And those are parts of the market that are a less significant part of GE's business than say, heavy-duty gas turbines that sit in a power plant and generate a massive amount of power. And so, if you look at that and say, “Wow, GE Ventures is out funding storage companies. Does that mean they're funding competitors?” Well, it means that we're funding innovation that may disrupt the future of our business, but that's part of being a VC and that's part of the value that GE Ventures brings to GE. CHARLES: Right. SAM: We're out there looking at markets before they're large enough or in scope for GE. CHARLES: Mmhmm, right. And so, yes you're disrupting the space but then you're going to be a part of that disruption and have strong connections to those markets if you need to actually migrate your business completely over to them. That's kind of what I'm hearing. SAM: Yeah, absolutely. Better to disrupt yourself, right? CHARLES: [Chuckles] SAM: And be a part of the ecosystem in the future because I think the future happens with or without you. And it's really key that we get out in front of it and a part of that, a part of that discussion, a part of that process. CHARLES: And so now, you've been saying that this is, GE, this has been pretty explosive? There's a lot more happening through GE Ventures. There's a lot more happening in other companies globally, having these corporate ventures. Where do you think the balance is going to lie to say, “Hey,” I'm just going to throw out some numbers, just for theory here, it's like, “10% of our business is essentially this distributed network of semi-autonomous or mostly autonomous startups. And then we have our core business.” Does that stabilize at 50/50? Does it stabilize at 75% the other way with GE essentially becoming a capital management company? Or is it somewhere in the middle? SAM: So, GE Ventures will never be a meaningful part of GE's revenue, a meaningful part of its business as a percentage. The overall venture industry is full of funds that are on the order of like, bigger funds are on the order of, in the billions. The single-digit billions. And GE itself is a much, much larger company. Well over a hundred billion dollars in enterprise value. So, I think GE Ventures will always be a small part of the company financially. And the impact will be largely felt through how we help the rest of GE navigate the future. ERICK: You said that sometimes you go and look for companies, startups to invest in or sometimes startups come to you or come to a VC looking for funding. Now, I'm a developer or a startup founder. And I'm going to look for funding. What are some of the mistakes or pitfalls that you see that startup founders or people with an idea fall into when looking for funding that you can help them avoid? SAM: Yeah, and we do see companies that come to us. So, I mentioned a lot about how I go out looking for companies based on a thesis or a set of relevant factors or relevant things for GE. But we do have a number of inbound requests. People know some of the bigger VC brands. They know GE the big company. So, we do get inbound interest and we also get referrals from networks of VCs and some are employees and other things. But for the companies that are seeking us out, the ones that are going out looking for funding, there are some things that are really well-known in Silicon Valley and other places, or you could research online and find, but may not be obvious at first. And so, I think the first one is, who are you talking to? What investors are you seeking out? Depending on what stage you're at, what kind of business you're in, you have to understand what the landscape of potential investors are and which ones might be interested in a company like yours. So, I think there are tons of good mentors that can help people navigate that. Maybe less commonly outside of Silicon Valley, in Boston, New York, in the places where you have traditional venture ecosystems. But you see a ton of resources available online whether it be things like Fred Wilson's blog, AVC.com, or Crunchbase, TechCrunch. You can read and understand and from headlines tell what people care about. And I think that's fundamentally a really important first step. You don't want to waste an hour talking to somebody who will never… this is somebody that invests in really late-stage growth equity companies and I'm coming to them for my first investment. That's not going to work. So, I think finding the right people, step one. I think when you're going through the process of pitching and talking about your business, the pitfalls are all about understanding the strengths and weaknesses of your business and where you are today. And so, for every company, that's different. But I think just being open and honest versus glossing over a lot of the risks, these are all really risky companies. If they were easy, then you'd have a lot more competition. And so… [Laughter] SAM: I think that's one thing that I see, too. You have some company that comes in and say, “Look, here are the parts I've figured out and here are the parts I still have to figure out.” And that's a really good conversation to have. There are other companies where they say, “Look, we've figured the whole thing out. We just want you to give us some money.” And I don't think a lot of investors necessarily buy into that. And certainly, there are investors of every stripe. So, I may be speaking too broadly. But I think that's a really important part of the venture investment process, right? You're looking not just for money but also for counsel and for somebody that you're going to work with over the next, sometimes seven years or longer. CHARLES: Yeah. SAM: [Inaudible] going to be on your board and participating. So, it's a really important part. CHARLES: So, you're looking, you're actually looking not necessarily for all the answers but you're looking for the questions that they're asking, too. SAM: Yeah, absolutely. And demonstrating they understand the ins and outs of the business. And that they have the capacity to carry this onto that next stage and hopefully beyond. CHARLES: Mmhmm. So, now you said something that caught my interest there that you work with some people sometimes seven years. You enter into these long relationships. Do you generally ever do any type of, I want to say almost like… mentoring might be too strong of a word, but in the pre-investment, in other words before you actually invest in a company, do you ever work with them to prepare them for investment to say, “Hey, I think there's potential here. Work on A, B, and C and then let's talk.” And you have this image in your mind. You go, you pitch to an investor, and it's either thumbs up or it's like thumbs down and you never talk to them again. Versus, is there some ground in between where there's a conversation that evolves that eventually ends up in an investment being made? SAM: Absolutely. I think one of the parts of this industry is even when I'm not an investor in a company, I may know a company and say, “It's not a fit for me for GE Ventures but I still think that we can provide help.” It's one of the things I love about tech and about venture in general, is that people are often willing to pitch in, even when they don't have a direct financial incentive. And so, I see that a lot whether it's helping a company where we've met them and we later see an opportunity and say, “Oh, you should go and talk to this company or that company.” And then often, we may see a company that's pitching us ahead of where we would typically invest. Maybe they're looking for a ‘Series A' but given the space that they're in or what we're doing at the moment, it may not be the right time for us. And so, we'll continue to track along and keep up and get updates. Some companies do a really good job of actually providing proactive updates and sending out monthly or quarterly reports to investors they've met with before. I think there's a wide range of ways that founders do this. But it is a really good way to keep people interested in the prize. And then when you come back and say, “Hey, now I'm out raising my ‘Series B',” that's not a surprise. I knew that you were hitting these milestones, that you were doing everything you said you were going to do. And you've demonstrated a level of credibility that really adds to the pitch that you made the first time around. ERLICK: You said something, metrics. So, a venture capitalist, after they make an investment, what are some of the expectations that they may hold this startup that they just invested in… what are those expectations that they may hold them accountable for? Or those metrics that they'll be looking at? SAM: Yeah, so I think some of the really high-level ones that are common across businesses, generally growth is a really big one. So, I almost said revenue. But I wanted to caveat… [Laughter] SAM: And say growth could mean different things. It could mean number of developers. It could mean number of downloads if you're an app. It depends on what the business is. But I think growth is a huge one. Growth is a really important, that top line, that's what's going to drive a lot of the value in the business. And then below that, demonstrating that you can hit the milestones around things like margins. So, how profitable is each unit you're selling? Or how profitable is each customer? And lastly, how are you doing managing your spend? So, that's great that you're earning the right amount of money for each customer, but are you doing it by… do you have a massive number of employees and offices and all the things that are too expensive to allow you to use your money wisely as you reach the next stage? And so, those are the big milestones. It's really just growth, margins, and operating cost or burn rate as we call it. CHARLES: Mmhmm. So, that sounds like a lot of work to actually evaluate these companies. Do you do your due diligence once you've already moved in pretty solidly into the process? SAM: Yeah, these processes can move really fast. And depending on the timing, generally it's, you jump in, you learn as much as you can, as fast as you can, and you make a decision so the company can move on. I'll say there's a lot of work that goes into considering and deciding which companies to spend more time on, both for us and for them. We don't want to waste a company's time evaluating, going through more meetings, if it's not a really strong candidate for us. Because they could be spending that time better with other investors who are a better fit. And I'm not going to pretend to like the evaluation part. I have a lot of respect for the amount of not just work but of a person's energy and really, their life goes into these companies. And so, I think the hard part is building the company. And so, it's hard for me to say that evaluating is a hard part. I'm trying to understand as much as I possibly can in a month or two. I'm not going to know as much about the business as the founder does. And I'll be wrong a lot. I may miss something and not understand, whether it's because I don't see the market but it's there or because I have some underlying assumption about the way things should work that they don't meet. And I think that that's something that investors have to come to grips with. You try and get as smart as you can as fast as you can, but you're not always going to get to the right answer. ERLICK: You said that it was growth, spend, and profits were some of the metrics. That is almost all of the essential components of a business plan. I remember one time, one of our previous conversations, you emphasized how important it was for companies, or even at just a simple startup, to put together a basic business plan. Is that something that you can elaborate on a little? SAM: Yeah. So, most companies show up with a pitch deck. So, they have a set of PowerPoint slides and then they have a set of materials behind that where if you go deeper into an area they may have a white paper about their technology and they may have an Excel financial model that explains why they have these expectations about what growth and margins and all those things will look like. So, there are all of those pieces that come together into a business plan. The business plan could be written or it could be that PowerPoint. But very traditionally, it's a PowerPoint or some kind of presentation that is shared in person. There's usually a version that's sent in advance to confirm that the company and the investors should meet. And then once you clear that bar, there's a deeper presentation that often you'll give to either one or a set or a whole team of investors. And you'll go through and explain why it is you think this is a good investment opportunity for them and why you'd like to work together. And then you have a discussion about whether that's a good fit, about some of the underlying assumptions, and come to either a set of next steps for the diligence or a decision that it's not the right fit, it's not the right time to take the relationship further with more diligence and that kind of stuff. ERLICK: Yeah, because I see… well, I know a few people that have startup ideas and they kind of put the business plan on the back burner and put the actual prototype more at the forefront. They say, “Oh, we can worry about the business plan later.” [Laughs] SAM: [Chuckles] Well, I think… there's something to be said to that. There's something to be said for product and growth winning. So if you… Let's start at the early stages. If you have something that's working and that's really obvious, you may not need a… ERLICK: True. SAM: To go raise money. It all comes down to, do you have enough to get enough investors interested to raise the round that you want to raise? Because you want to have enough investors involved, enough demand, that you can be selective about who you want to work with and on what terms, right? So, what valuation and how much of the company am I giving them, and all of those things. So, if you can do all of those things with nothing but an app and one chart that shows a hockey stick of growth, that's awesome. CHARLES: You're hot. [Laughter] SAM: Often it does require much more and a much longer plan. So, even if you say, “Look, it's growing like crazy,” there's usually some set of questions behind that. So, that's great. Your free app is growing like crazy. How are you going…? [Laughter] SAM: To get paid for that? And you'll talk about that. And you'll say, “Here are the things we're planning on doing,” or here are the assumptions that we're making. And the more original, the more unique the business model is, the more discussion and explanation that may require. And that's where the business plan and a pitch deck come in handy, because it's a really good presentation aide or pre-reading to get to that answer faster. ERLICK: So, this evaluation it seems, is a two-way street. The VCs evaluating the company and also the company or the startup evaluating the VC to know whether it's going to be a good relationship. SAM: Oh, absolutely. Yeah, the best companies have choice. They have a number of investors who are interested in funding them. And certainly, that might be different at different stages or at different times, depending on what's going on in the economy and in tech and in other places. But generally, VC is a very competitive industry. I'm trying to sell my money and services as an investor versus other options that you have. And so, while it's maybe not as competitive as only one of us can buy the company like in an M&A situation. There are often more than one investor. There's still a very intense set of competition around, okay, who's going to be involved in the deal? How much money will they be able to invest? So, that's something that really can come in handy for founders. ERLICK: And what was that you just said there? M&A situation? SAM: Oh, sorry. When a company is being bought. So, when a company is being bought, it can look kind of like a fundraising process, but instead of selling a part of a company, you're selling the whole thing. And so, in that case, obviously it's a competitive situation where there's only one winner. And this is a different process. Often, the rounds that we're a part of, we're not… we're buying a minority stake just like any VC. We may be buying 5% of a company, 10% of a company. And often we're being joined by other venture investors. We really actually commonly partner with the institutional firms and they'll take a board seat. We'll invest alongside them and be an observer on the board and provide counsel. And so, it is a very competitive process. And that, while M&A is a winner-take-all, there is one buyer who is ultimately going to own this company going forward, the investing process for a venture is much more collaborative. But it is still competitive, because there can only be so many investors in one company. CHARLES: And you want to choose the right one on both… the right set. Alright. Well, I think we're running up against time. This has been a fascinating conversation into an aspect of our industry that really is providing the fuel that drives so much of this forward. So, I guess I'll close by asking you, already talked about Resin. We had them on the podcast. We love them. Are there any conferences or products that you're investing in that you feel like our audience might want to know about or anything like that? SAM: Well one, you mentioned Resin. CHARLES: Yeah. SAM: I know you guys have been a good friend to them and Elrick and I met at their hackathon. I would recommend to anybody, go try it out. It's a really cool way to play with hardware products. I am not a developer and I required a lot of help from Elrick at the hackathon. [Laughter] SAM: But at the same time, it is something that almost anybody can pull out of a box and start playing with. So, I think that's a great one. The episode you did on them were fantastic. So, I really enjoyed those ones. I'd say in general, I'm always out looking to meet new companies that are going to benefit from working with GE. I spend a lot of my time not just trying to invest but also trying to find partnerships for companies that we're looking at within GE, either selling to us or working with us. And so, if somebody thinks that there's an opportunity to do that, then I encourage them to reach out. Because I think there's a ton of opportunity. It's a really big company that really has a ton of opportunity for other partners. CHARLES: Alright. If they wanted to reach out, how would they get in touch with you? SAM: Yeah, I think maybe the best way to initially make contact, I tend to be pretty active on Twitter. So, my handle is just @SamCates. S-A-M-C-A-T-E-S. And you can also learn more through our website. If you're curious about some of the businesses I mentioned, so just GEVentures.com. And it's about to go through a whole refresh. So, go check it out. CHARLES: Alright. Well, fantastic. We will definitely look for that. And for everybody else, you can get in touch with us on Twitter at @TheFrontside or send us a line at info@frontside.io. Thank you everybody for listening. Thank so, so much Sam, for being on the podcast. SAM: Yeah, of course. It was a blast. I'm a big podcast fan and I've really enjoyed catching up on your episodes. CHARLES: Ah, and thank you Elrick, always. ERICK: It was great. SAM: Elrick, when you finish building your Raspberry Pi Battleships, I want to play. CHARLES: [Laughs] ERICK: Oh, yes. Yes. It's in the works, man. It's in progress. SAM: Alright, I'm waiting. CHARLES: Alright. Well, take it easy, everybody.

Tracy Lee: @ladyleet | ladyleet.com Ben Lesh: @benlesh | medium.com/@benlesh Show Notes: 00:50 - What is This Dot? 03:26 - The RxJS 5.5.4 Release and Characterizing RxJS 05:14 - Observable 07:06 - Operators 09:52 - Learning RxJS 11:10 - Making RxJS Functional Programming Friendly 12:52 - Lettable Operators 15:14 - Pipeline Operators 21:33 - The Concept of Mappable 23:58 - Struggles While Learning RxJS 33:09 - Documentation 36:52 - Surprising Uses of Observables 40:27 - Weird Uses of RxJS 45:25 - Announcements: WHATWG to Include Observables and RxJS 6 Resources: this.media RxJS RX Workshop Ben Lesh: Hot vs Cold Observables learnrxjs.io RxMarbles Jewelbots Transcript: CHARLES: Hello everybody and welcome to The Frontside Podcast, Episode 91. My name is Charles Lowell, a developer here at The Frontside and your podcast host-in-training. Joining me today on the podcast is Elrick Ryan. Hello, Elrick. ELRICK: Hey, what's up? CHARLES: Not much. How are you doing? ELRICK: I'm great. Very excited to have these two folks on the podcast today. I feel like I know them… CHARLES: [Laughs] ELRICK: Very well, from Twitter. CHARLES: I feel like I know them well from Twitter, too. ELRICK: [Laughs] CHARLES: But I also feel like this is a fantastic company that is doing a lot of great stuff. ELRICK: Yup. CHARLES: Also not in Twitter. It should be pointed out. We have with us Tracy Lee and Ben Lesh from This Dot company. TRACY: Hey. CHARLES: So first of all, why don't we start, for those who don't know, what exactly is This Dot? What is it that you all do and what are you hoping to accomplish? TRACY: This Dot was created about a year ago. And it was founded by myself and Taras who work on it full-time. And we have amazing people like Ben, who's also one of our co-founders, and really amazing mentors. A lot of our friends, when they refer to what we actually do, they like to call it celebrity consulting. [Laughter] TRACY: Which I think is hilarious. But it's basically core contributors of different frameworks and libraries who work with us and lend their time to mentor and consult with different companies. So, I think the beautiful part about what we're trying to do is bring together the web. And we sort of do that as well not only through consulting and trying to help people succeed, but also through This Dot Media where it's basically a big playground of JavaScripting all the things. Ben and I do Modern Web podcast together. We do RX Workshop which is RxJS training together. And Ben also has a full-time job at Google. CHARLES: What do they got you doing over there at Google? BEN: Well, I work on a project called Alkali which is an internal platform as a service built on top of Angular. That's my day job. CHARLES: So, you've been actually involved in all the major front-end frameworks, right, at some point? BEN: Yeah, yes. I got my start with Angular 1 or AngularJS now, when I was working as a web developer in Pittsburgh, Pennsylvania at a company called Aesynt which was formerly McKesson Automation. And then I was noticed by Netflix who was starting to do some Angular 1 work and they hired me to come help them. And then they decided to do Ember which is fine. And I worked on a large Ember app there. Then I worked on a couple of large React apps at Netflix. And now I'm at Google building Angular apps. CHARLES: Alright. BEN: Which is Angular 5 now, I believe. CHARLES: So, you've come the full circle. BEN: Yeah. Yeah, definitely. CHARLES: [Chuckles] I have to imagine Angular's changed a lot since you were working on it the first time. BEN: Yeah. It was completely rewritten. TRACY: I feel like Angular's the new Ember. CHARLES: Angular is the new Ember? TRACY: [Laughs] BEN: You think? TRACY: Angular is the new Ember and Vue is the new AngularJS, is basically. [Laughs] CHARLES: Okay. [Laughter] CHARLES: What's the new React then? BEN: Preact would be the React. CHARLES: Preact? Okay, or is Glimmer… BEN: [Laughs] I'm just… CHARLES: Is Glimmer the new React? BEN: Oh, sure. [Laughs] CHARLES: It's important to keep these things straight in your head. BEN: Yeah, yeah. CHARLES: Saves on confusion. TRACY: Which came first? [Chuckles] BEN: Too late. I'm already confused. CHARLES: So now, before the show you were saying that you had just, literally just released RxJS, was it 5.5.4? BEN: That's right. That's right. The patch release, yeah. CHARLES: Okay. Am I also correct in understanding that RxJS has kind of come to very front and center position in Angular? Like they've built large portions of framework around it? BEN: Yeah, it's the only dependency for Angular. It is being used in a lot of official space for Angular. For example, Angular Material's Data Table uses observables which are coming from RxJS. They've got reactive forms. The router makes use of Observable. So, the integration started kind of small which HTTPClient being written around Observable. And it's grown from there as people seem to be grabbing on and enjoying more the React programming side of things. So, it's definitely the one framework that's really embraced reactive programming outside of say, Cycle.js or something like that. CHARLES: Mmhmm. So, just to give a general background, how would you characterize RxJS? BEN: It's a library built around Observable. And Observable is a push-based primitive that gives you sets of events, really. CHARLES: Mmhmm. BEN: So, that's like Lodash for events would be a good way to put it. You can take anything that you can get pushed at you, which is pretty much value type you can imagine, and wrap it in an observable and have it pushed out of the observable. And from there, you have a set of things that you can combine. And you can concatenate them, you can filter them, you can transform them, you can combine them with other sets, and so on. So, you've got this ability to query and manipulate in a declarative way, events. CHARLES: Now, Observable is also… So, when Jay was on the podcast we were talking about Redux observable. But there was outside of the context of RxJS, it was just observables were this standalone entity. But I understand that they actually came from the RxJS project. That was the progenitor of observables even though there's talk of maybe making them part of the JavaScript spec. BEN: Yeah, that's right. That's right. So, RxJS as it stands is a reference implementation for what could land in JavaScript or what could even land in the DOM as far as an observable type. Observable itself is very primitive but RxJS has a lot of operators and optimizations and things written around Observable. That's the entire purpose of the library. CHARLES: Mmhmm. So, what kind of value-adds does it provide on top of Observable? If Observable was the primitive, what are the combinators, so to speak? BEN: Oh, right. So, similar to what Lodash would add on top of say, an iterable or arrays, you would have the same sorts of things and more inside of RxJS. So, you've got zip which you would maybe have seen in Lodash or different means of combines. Of course, map and ‘merge map' which is like a flattening sort of operation. You can concatenate them together. But you also have these time-based things. You can do debouncing or throttling of events as they're coming over in observable and you create a new observable of that. So, the value-add is the ability to compose these primitive actions. You can take on an observable and make a new observable. We call it operators. And you can use those operators to build pretty much anything you can imagine as far as an app would go. CHARLES: So, do you find that most of the time all of the operators are contained right there inside RxJS? Or if you're going to be doing reactive programming, one of your tasks is going to be defining your own operators? BEN: No, pretty much everything you'd need will be defined within RxJS. There's 60 operators or so. CHARLES: Whoa, that's a lot. BEN: It's unlikely that someone's going to come up with one. And in fact, I would say the majority of those, probably 75% of those, you can create from the other 25%. So, some of the much more primitive operators could be used… TRACY: Which is sort of what Ben did in this last release, RxJS 5…. I don't know remember when you introduced the lettable operators but you… BEN: Yeah, 5.5. TRACY: Implemented [inaudible] operators. BEN: Yeah, so a good portion of them I started implementing in terms of other operators. CHARLES: Right. So, what was that? I didn't quite catch that, Tracy. You said that, what was the operator that was introduced? TRACY: So, in one of the latest releases of RxJS, one of the more significant releases where pipeable operators were introduced, what Ben did was he went ahead and implemented a lot of operators that were currently in the library in terms of other operators, which was able to give way to reduce the size of the library from, I think it was what, 30KB bundled, gzipped, and minified, to about 30KB, which was about 60 to 70% of the operators. Right, Ben? BEN: Yeah. So, the size reduction was in part that there's a lot of factors that went into the size reduction. It would be kind of hard to pin it down to a specific operator. But I know that some of the operators like the individual operators themselves, by reimplementing reduce which is the same as doing as scan and then take last, implementing it in terms of that is going to reduce the size of it probably 90% of that one particular file. So, there's a variety of things like that that have already started and that we're going to continue to do. We didn't do it with every operator that we could have. Some operators are very, very common and consequently we want them to be as optimized as possible. For example, map. You can implement map in terms of ‘merge map' but it would be very slow to do so. It might be smaller but it would be slower. We don't want that. So, there are certain areas we're always going to try to keep fairly a hot path to optimize them as much as possible. But in other spots like reduce which is less common and isn't usually considered to be a performance bottleneck, we can cut some corners. Or ‘to array' or other things like that. CHARLES: Mmhmm. TRACY: And I think another really interesting thing is a lot of people when learning RxJS, they… it's funny because we just gave an RX Workshop course this past weekend and the people that were there just were like, “Oh, we've heard of RxJS. We think it's a cool new thing. We have no plans to implement it in real life but let's just play around with it and let me learn it.” I think as people are starting to learn RxJS, one of the things that gets them really overwhelmed is this whole idea that they're having to learn a completely new language on top of JavaScript or what operators to use. And one of our friends, Brian Troncone who is on the Learning Team, the RxJS Learning Team, he pulled up the top 15 operators that were most commonly searched on his site. And some of them were ‘switch map', ‘merge map', ‘fork join', merge, et cetera. So, you can sort of tell that even though the library has quite a few… it's funny because Ben, I think the last RX Workshop you were using pairs and you had never used it before. BEN: Yeah. TRACY: So, it's always amusing for me how many people can be on the core team but have never implemented RxJS… CHARLES: [Laughs] TRACY: A certain way. BEN: Right. Right, right, right. CHARLES: You had said one of the recent releases was about making it more friendly for functional programming. Is that a subject that we can explore? Because using observables is already pretty FP-like. BEN: What it was before is we had dot chaining. So, you would do ‘dot map' and then call a method and then you get an observable back. And then you'd say ‘dot merge' and then you'd call a method on that, and so on and so forth. Now what you have is kind of a Ramda JS style pipe function that just takes a comma-separated list of other functions that are going to act upon the observable. So, it reads pretty much the same with a little more ceremony around it I guess. But the upside is that you can develop your operators as just higher-order functions. CHARLES: Right. And you don't have to do any monkey-patching of prototypes. BEN: Exactly, exactly. CHARLES: Because actually, okay, I see. This is actually pretty exciting, I think. Because we actually ran into this problem when we were using Redux Observable where we wanted to use some operators that were used by some library but we had to basically make a pull request upstream, or fork the upstream library to include the operators so that we could use them in our application. It was really weird. BEN: Yeah. CHARLES: The reason was because it was extending the observable prototype. BEN: Yeah. And there's so many… and that's one way to add that, is you extend the observable prototype and then you override lift so you return the same type of observable everywhere. And there are so many things that lettable operators solved for us. For example… CHARLES: So, lettable operators. So, that's the word that Tracy used and you just used it. What are lettable operators? BEN: Well, I've been trying to say pipeable and get that going instead of lettable. But basically there's an operator on RxJS that's been there forever called let. And let is an operator and what you do is you give it a function. And the function gives you the source observable and you're expected to return a new observable. And the idea is that you can then write a function elsewhere that you can then compose in as though it were an operator, anywhere you want, along with your other dot-chained operators. And the realization I had a few months ago was, “Well, why don't we just make all operators like this?” And then we can use functional programming to compose them with like a reduce or whatever. And that's exactly what the lettable operators are. And that's why I started calling them lettable operators. And I kind of regret it now, because so many people are saying it and it confuses new people. Because what in the world does lettable even mean? CHARLES: Right. [Laughs] BEN: So, they are pipeable operators or functional operators. But the point is that you have a higher-order function that returns a function of a specific shape. And that function shape is, it's a function that receives an observable and returns an observable, and that's it. So, basically it's a function that transforms an observable into a new observable. That's all an operator. That's all an operator's ever been. It's just this is in a different flavor. CHARLES: Now, I'm curious. Why does it do an observable into an observable and not a stream item into an observable? Because when you're actually chaining these things together, like with a map or with a ‘flat map' or all these things, you're actually getting an individual item and then returning an observable. Well, I guess in this case of a map you're getting an item and returning an item. But like… BEN: Right, but that's not what the entire operation is. So, you've got an operation you're performing whenever you say, if you're to just even dot-chain it, you'd say ‘observable dot map'. And when you say ‘dot map', it returns a new observable. And then you say ‘dot filter' and it returns another new observable. CHARLES: Oh, gotcha, gotcha, gotcha. Okay, yeah, yeah, yeah. Yeah, yeah. BEN: So, this function just embodies that step. CHARLES: I see, I see. And isn't there some special… I feel like there's some proposal for some special JavaScript syntax to make this type of chaining? BEN: Yeah, yeah, the pipeline operator. CHARLES: Okay. BEN: I don't know. I think that's still at stage one. I don't know that it's got a lot of headway. My sources and friends that are in the TC39 seem to think that it doesn't have a lot of headway. But I really think it's important. Because if you look at… the problem is we're using a language where the most common use case is you have to build it, get the size as small as possible because you need to send it over the wire to the browser. And understandably, browsers don't want to implement every possible method they could on say, Array, right? CHARLES: Mmhmm, right. BEN: There's a proposal in for ‘flat map'. They could add zip to Array. They could add all sorts of interesting things to Array just by itself. And that's why Lodash exists, right? CHARLES: Right. BEN: Is because not everything is on Array. And then so, the onus is then put on the community to come up with these solutions and the community has to build libraries that have these constraints in size. And what stinks about that is then you have say, an older version of Lodash where you'd be like, “Okay, well it has 36 different functions in it and I'm only using 3 of them. And I have to ship them all to the browser.” CHARLES: Mmhmm. BEN: And that's not what you want. So, then we have these other solutions around tree-shaking and this and that. And the real thing is what you want is you want to be able to compose things left to right and you want to be able to have these functions that you can use on a particular type in an ad hoc way. And there's been two proposals to try to address this. One was the ‘function bind' operator, CHARLES: Mmhmm. BEN: Which is colon colon. And what that did is it said, “You can use this function as a method, as though it were a method on an object. And we'll make sure that the ‘this' inside that function comes from the instance that's on the left-hand side of colon colon.” CHARLES: Right. BEN: That had a bunch of other problems. Like there's some real debate I guess on how they would tie that down to a specific type. So, that kind of fell dead in the water even though it had made some traction. And then the pipeline operator is different. And then what it says is, “Okay, whatever is on the…” And what it looks like is a pipe and a greater than right next to each other. And whatever's on the left-hand side of that operand gets passed as the first argument to the function on the right-hand side of that operand. CHARLES: Mmhmm. BEN: And so, what that means is for the pipeable operators, instead of having to use a pipe method on observable, you can just say, “instance of observable, pipeline operator and an operator, and then pipeline operator, and then the Rx operator, and then pipeline operator and the Rx operator, and so on.” And it would just be built-in. And the reason I think that JavaScript really needs it is that means that libraries like Lodash can be written in terms of simple functions and shipped piece-meal to the browser exactly as you need them. And people would just use the pipeline operator to use them, instead of having to wrap something in a big object so you can dot-chain things together or come up with your own functional pipe thing like RxJS had to. CHARLES: Right. Because it seems it happens again and again, right? Lodash, RxJS, jQuery. You just see this pattern of chaining, which is, you know… BEN: Yeah, yeah. People want chaining. People want left to right composition. CHARLES: Mmhmm. BEN: And it's problematic in a world where you want to shake off as much unused garbage as possible. And the only way to get dot chaining is by augmenting a prototype. There's all sorts of weird problems that can come with that. And so, the functional programming approach is one method. But then people look at it and they say, “Ooh, yuck. I've got to wrap things in a function named pipe. Wouldn't it be nicer if there was just some syntax to do this?” And yeah, it would be nicer. But I have less control over that. CHARLES: Right. But the other alternative is to have right to left function composition. BEN: Right, yeah. CHARLES: But there's not any special syntax for that, either. BEN: Not very readable. CHARLES: Yeah. BEN: So, you just wrap everything. And the innermost call is the first one and then you wrap it in another function and you wrap that in another function, and so on. Yeah, that's not [inaudible]. But I will say that the pipe function itself is pretty simple. It's basically a function that takes a rest of arguments that are all functions. CHARLES: Mmhmm. BEN: And so, you have this array of functions and you just reduce over it and call them. Well, you return a function. So, it's a higher function. You return a function that takes an argument then you reduce over the functions that came in as arguments and you call each one of them with whatever result was from the previous. CHARLES: Right. Like Tracy mentioned in the pre-show, I'm an aspiring student of functional programming. So, would this be kind of like a monoid here where you're mashing all these functions together? Is your empty value? I'm just going to throw it out there. I don't know if it's true or not, but that's my conjecture. BEN: Yes. Technically, it's a monoid because it wouldn't work unless it was a monoid. Because monoids, I believe the category theory I think for monoid is that monoids can be concatenated because they definitely have an end. CHARLES: Right. BEN: So, you would not be able to reduce over all those functions and build something with that, like that, unless it was a monoid. So yeah, the fact that there's reduction involved is a cue that it's a monoid. CHARLES: Woohoo! Alright. [Laughter] CHARLES: Have you found yourself wanting to apply some of these more “rigorous” formalisms that you find out there in the development of RxJS or is that just really a secondary concern? BEN: It's a secondary concern. It's not something that I like. It's something I think about from time to time, when really, debating any kind of heavy issue, sometimes it's helpful. But when it comes to teaching anybody anything, honestly the Haskell-isms and category theory names, all they do is just confuse people. And if you tell somebody something is a functor, they're like, “What?” And if you just say it's mappable, they're like, “Oh, okay. I can map that.” CHARLES: [Laughs] Right, right. BEN: And then the purists would be like, “But they're not the same thing.” And I would be like, “But the world doesn't care. I'm sorry.” CHARLES: Yeah, yeah. I'm kind of experiencing this debate myself. I'm not quite sure which side I fall on, because on the one hand it is arbitrary. Functor is a weird name. But I wish the concept of mappable existed. It does, but I feel like it would be handy if people… because there's literally five things that are super handy, right? Like mappable, if we could have a name for monoid. But it's like, really, you just need to think in terms of these five constructs for 99% of the stuff that you do. And so, I always wonder, where does that line lie? And how… mappable, is that really more accessible than functor? Or is that only because I was exposed to the concept of mapping for 10 years before I ever heard the F word. BEN: Yes, and yes. I mean, that's… CHARLES: [Laughs] BEN: Things that are more accessible are usually more accessible because of some pre-given knowledge, right? What works in JavaScript probably isn't going to work in Haskell or Scala or something, right? CHARLES: Mmhmm. BEN: If someone's a Java developer, certain idioms might not make sense to them that come from the JavaScript world. CHARLES: Right. But if I was learning like a student, I would think mappable, I'd be thinking like, I would literally be thinking like Google Maps or something like that. I don't know. BEN: Right, right. I mean, look at C#. C#, a mapping function is always going to be called select, right, because that's C#. That's their idiom for the same thing. CHARLES: Select? BEN: Yeah. CHARLES: Really? BEN: Yeah, select. So, they'll… CHARLES: Which in Ruby is like find. BEN: Yeah. there's select and then, what's the other one, ‘select many' or something like that. [Chuckles] BEN: So, that's C#. CHARLES: Oh, like it's select from SQL. Okay. BEN: Yeah, I think that's kind of where it came from because people had link and then they had link to SQL and then they're like, well I want to do this with regular code, with just using some more… less nuanced expressions. So, I want to be able to do method calls and chain those together. And so, you end up with select functions. And I think that that exists even in Rx.NET, although I haven't used Rx.NET. CHARLES: Hmm, okay. ELRICK: So, I know you do a lot of training with Rx. What are some of the concepts that people struggle with initially? TRACY: I think when we're teaching RX Workshop, a lot of the people sort of… I'll even see senior level people struggle with explaining it, is the difference between observables and observers and then wrapping their head around the idea that, “Hey, observables are just functions in JavaScript.” So, they're always thinking observables are going to do something for you. Actually, it's not just in Angular but also in React, but whenever someone's having issues with their Rx applications, it's usually something that they're like nesting observables or they're not subscribing to something or they've sort of hot-messed themselves into a tangle. And I'm sure you've debugged a bunch of this stuff before. The first thing I always ask people is, “Have you subscribed?” Or maybe they're using an Angular… they're using pipes async but they're also calling ‘dot subscribe' on their observable. BEN: Yeah. So, like in Angular they'll do both. Yeah. There's that. I think that, yeah, that relates to the problem of people not understanding that observables are really just functions. I keep saying that over and over again and people really don't seem to take it to heart for whatever reason. [Chuckles] BEN: But you get an observable and when you're chaining all those operators together, you're making another observable or whatever, observables don't do anything until you subscribe to them. They do nothing. CHARLES: Shouldn't they be called like subscribable? BEN: Yes. [Chuckles] BEN: They probably should. But we do hand them an observer. So, you are observing something. But the point being is that they don't do anything at all until you subscribe to them. And in that regard, they're like functions, where functions don't do anything unless you call them. So, what ends up happening with an observable is you subscribe to it. You give it an observer, three callbacks which are then coerced into an observer. And it takes that observer and it hands it to the body of this observable definition and literally has an observer inside of there. And then you basically execute that function synchronously and do things, whatever those things are, to set up some sort of observation. Maybe you spin up a WebSocket and tie into some events on it and call next on the observer to get values out of your observable. The point being that if you subscribe to an observable twice, it's the same thing as calling a function twice. And for some reason, people have a hard time with that. They think, if I subscribe to the observable twice, I've only called the function once. CHARLES: I experienced this confusion. And I remember the first time that that… like, I was playing with observables and the first time I actually discovered that, that it was actually calling my… now what do you call the function that you pass to the constructor that actually does, that calls next or that gets passed the observer? TRACY: [Inaudible] BEN: I like to call it an initialization function or something. But the official name from the TC39 proposal is subscriber function. CHARLES: Subscriber function. So, like… BEN: Yeah. CHARLES: I definitely remember it was one of those [makes explosion sound] mind-blowing moments when I realized when I call my subscribe method, the entire observable got run from the very beginning. But my intuition was that this is an object. It's got some shared state, like it's this quasar that I'm now observing and I'm seeing the flashes of light coming off of it. But it's still the same object. You think of it as having yeah, not as a function. Okay. No one ever described it to me as just a function. But I think I can see it now. ELRICK: Yeah, me neither. CHARLES: But yeah, you think of it in the same way that most people think of objects, as like, “I have this object. I have a reference to it.” Let observable equal new observable. It's a single thing. It's a single identity. And so, that's the thing that I'm observing. It's not that I'm invoking this observable to observe things. And I think that's, yeah, that's a subtle nuance there. I wish I had taken y'all's course, I guess is what I'm saying. ELRICK: Yeah. BEN: Yeah. Well, I've done a few talks on it. CHARLES: [Laughs] BEN: I always try to tell people, “It's just a function. It's just a function.” I think what happens to a lot of people too is there's the fact that it's an object. But I think what it is, is people's familiarity with promises does this. Because promises are always multicast. They are always “hot”. And the reason for this is because they're eager. So, by the time you have a promise, whatever is producing value to the promise has already started. And that means that they're inherently a multicast. CHARLES: Right. BEN: So, people are used to that behavior of, I can ‘then' off of this promise and it always means one thing. And it's like, yeah, because the one thing has nothing to do with the promise. It wasn't [Chuckles] CHARLES: Right. BEN: This promise is just an interface for you to view something that happened in the past, where an observable is more low-level than that and more simple than that. It just states, “I'm a function that you call. I'm going to be able to do anything a function can do. And by the way, you're giving me an observer and I'm going to do some stuff with that too and notify you via this observer that you handed me.” Because of that you could take an observable and close over something that had already started. Say you had a WebSocket that was already running. You could create a new observable and just like any function, close over that, externally create a WebSocket. And then everyone that subscribes to that observable is tying an observer to that same WebSocket. Then you're multicast. Then you're “hot”. ELRICK: [Inaudible] CHARLES: Right. So, I was going to say that's the distinction that Jay was talking about. He was talking about we're going to just talk about… he said at the very beginning, “We're just going to talk about hot observable.” ELRICK: Yup. CHARLES: But even a hot observable is still theoretically evaluating every single time you subscribe. You're getting a new observable. You're evaluating that observable afresh each time. It just so happens that in the lexical scope of that observable subscriber function, there is this WebSocket? BEN: Yeah. So, it's the same thing. Imagine you wrote a function that when you called it created a new WebSocket and then… say, you wrote a new function that you gave an observer object to, right? An observer object has next, error, and complete. And in that function, when you called it, it created a new WebSocket and then it tied the ‘on message' and ‘on close' and whatever to your observer's next method and your observer's error message and so on. When you call that function, you would expect a new WebSocket to be created every single time. Now, let's just say alternately you create a WebSocket and then you write a new function that that function closes over that WebSocket. So, you reference the WebSocket that you externally created inside of your function. When you call that function, it's not going to create a new WebSocket every time. It's just closing over it, right? So, even though they both are basically doing the same thing, now the latter one of those two things is basically a hot observable and the former is a cold observable. Because one is multicast which is, “I'm sharing this one WebSocket with everybody,” and the other one is unicast which is, “I am going to create a new WebSocket for each person that calls me.” And that's the [inaudible] people have a hard time with. CHARLES: Right. But really, it's just a matter of scope. BEN: Yeah. The thing people have a hard time with, with observables, is not realizing that they're actually just functions. CHARLES: Yeah. I just think that maybe… see, when I hear things like multicast and unicast, that makes me think of shared state, whereas when you say it's just a matter of scope, well then I'm thinking more in terms of it being just a function. It just happens that this WebSocket was already [scoped]. BEN: Well, shared state is a matter of scope, right? CHARLES: Yes, it is. It is. Oh, sorry. Shared state associated with some object identity, right? BEN: Right. CHARLES: But again, again, it's just preconceptions, really. It's just me thinking that I've had to manage lists of listeners and have multicast observers and single-cast observers and having to manage those lists and call notify on all of them. And that's really not what's happening at all. BEN: Yeah. Well, I guess the real point is observables can have shared state or they could not have shared state. I think the most common version and the most composable version of them, they do not have any shared state. It's just one of those things where just like a function can have shared state or it could be pure, right? There's nothing wrong with either one of those two uses of a function. And there's nothing wrong with either one of those two uses of Observable. So, honest to god, that is the biggest stumbling block I think that I see people have. That and if I had to characterize it I would say fear and loathing over the number of operators. People are like… CHARLES: [Chuckles] BEN: And they really think because everyone's used to dealing with these frameworks where there's an idiomatic way to do everything, they think there's going to be an RxJS idiomatic way to do things. And that's just patently false. That's like saying there's an idiomatic way to use functions. There's not. Use it however it works. The end. It's not… CHARLES: Mmhmm, mmhmm. BEN: You don't have to use every operator in a specific way. You can use it however works for you and it's fine. ELRICK: I see that you guys are doing some fantastic work with your documentation. Was that part of RxJS 2.0 docs? TRACY: I was trying to inspire people to take on the docs initiative because I think when I was starting to learn RxJS I would get really frustrated with the docs. BEN: Yeah. TRACY: I think the docs are greatly documented but at the same time if you're not a senior developer who understands Rx already, then it's not really helpful. Because it provides more of a reference point that the guys can go back and look at, or girls. So anyways, after many attempts of trying to get somebody to lead the project I just decided to lead the project myself. [Laughter] TRACY: And try to get… the community is interesting because I think because the docs can be sometimes confusing… Brian Troncone created LearnRxJS.io. There's these other visualization projects like RxMarbles, RxViz, et cetera. And we just needed to stick everybody together. So, it's been a project that I think has been going on for the past two months or so. We have… it's just an Angular app so it's probably one of the most easiest projects to contribute to. I remember the first time I tried to contribute to the Ember docs. It literally took me an hour to sit there with a learning team, Ember Learning Team member and… actually, maybe it was two hours, just to figure out how the heck… like all the things I had to download to get my environment set up so that I could actually even contribute to the darn documentation. But with the Rx, the current RxJS docs right now is just an Angular app. You can pull it down. It's really easy. We even have people who are just working on accessibility, which is super cool, right? So, it's a very friendly place for beginners. BEN: I'm super pleased with all the people that have been working on that. Brian and everybody, especially on the accessibility front. Jen Luker [inaudible] came in and voluntarily… she's like the stopgap for all accessibility to make sure everything is accessible before we release. So, that's pretty exciting. TRACY: Yeah. ELRICK: Mmhmm. TRACY: So funny because when me and Jen started talking, she was talking about something and then I was like, “Oh my god, I'm so excited about the docs.” She's like, “I'm so excited, too! But I don't really know why I'm excited. But you're excited, so I'm excited. Why are you excited?” [Laughter] TRACY: I was like, “I don't know. But I'm excited, too!” [Chuckles] TRACY: And then all of a sudden we have accessibility. [Laughs] ELRICK: Mmhmm. Yeah, I saw some amazing screenshots. Has the new docs, have they been pushed up to the URL yet? TRACY: Nah, they are about to. We were… we want to do one more accessibility run-through before we publish it. And then we're going to document. We want to document the top 15 most viewed operators. But we should probably see that in the next two weeks or so, that the new docs will be… I mean, it'll say “Beta, beta, beta” all over everything. But actually also, some of our friends, [Dmitri] from [Valas] Software, he is working on the translation portion to make it really easy for people to translate the docs. CHARLES: Ah. TRACY: So, a lot of that came from the inspiration from the Vue.js docs. we're taking the versioning examples that Ember has done with their docs as inspiration to make sure that our versioning is really great. So, it's great that we can lend upon all the other amazing ideas in the industry. ELRICK: Oh, yeah. CHARLES: Yeah, it's fantastic. I can't wait to see them. ELRICK: Yeah, me neither. The screenshots look amazing. I was like, “Wow. These are some fabulous documentation that's going to be coming out.” I can't wait. TRACY: Yeah. Thank you. CHARLES: Setting the bar. ELRICK: Really high. [Laughter] CHARLES: Actually, I'm curious. Because observables are so low-level, is there some use of them that… what's the use of them that you found most surprising? Or, “Whoa, this was a crazy hack.” BEN: The weirdest use of observables, there's been quite a few odd ones. One of the ones that I did one time that is maybe in RxJS's wheelhouse, it was just that RxJS already existed. So, I didn't want to pull in another transducer library, was using RxJS as a transducer. Basically… in Netflix we had a situation where we had these huge, huge arrays of very large objects. And if you try to take something like that and then map it and then filter it and then map it and then filter it, we're using Array map and filter, what ends up happening is you create all sorts of intermediary arrays in-memory. And then garbage collection has to come through and clean that up. And that locks your thread. And over time, we were experiencing slowness with this app. And it would just build up until eventually it ground to a halt. And I used RxJS because it was an available tool there to wrap these arrays in an observable and then perform operations on them step-by-step, the same map, filter, and so on. But when you do that, it doesn't create intermediary arrays because it passes each value along step to step instead of producing an entire array and then doing another step and producing an entire array, and so on. So… CHARLES: So, will you just… BEN: It saved garbage collection and it increased the performance of the app. But that's just in an extreme case. I would never do that with just regular arrays. If anything, it was because it was huge, huge arrays of very large objects. CHARLES: So, you would create an observable our of the array and then just feed each element into the observable one at a time? BEN: Well, no. If you say ‘observable from' and you give it an array, that's basically what it does. CHARLES: Okay. BEN: It loops over the array and nexts those values out of the array synchronously. CHARLES: I see, I see. BEN: So, it's like having a for loop and then inside of that for loop saying, “Apply the map. Apply the filter,” whatever, to each value as they're going through. But when you look at it, if you had array map, filter, reduce, it's literally just taking the first step and saying ‘observable from' and wrapping that array and then the rest of it's still the same. CHARLES: Right. Yeah. No, that's really cool. BEN: That was a weirder use of it. I've heard tell of other things where people used observables to do audio synchronization, which is pretty interesting. Because you have to be very precise with audio synchronization. So, hooking into some of the Web Audio APIs and that sort of thing. That's pretty interesting. The WebSocket multiplexing is something I did at Netflix that's a little bit avant-garde for observable use because you essentially have an observable that is your WebSocket. And then you create another observable that closes over that observable and sends messages over the WebSocket for what you're subscribed to and not subscribed to. And it enables you to very easily retry connections and these sorts of things. I did a whole talk on that. That one's pretty weird. CHARLES: Yeah. Man, I [inaudible] to see that. BEN: But in the general use case, you click a button, you make an AJAX request, and then you get that back and maybe you make another AJAX request. Or like drag and drop and these sorts of things where you're coordinating multiple events together, is the general use case. The non-weird use case for RxJS. Tracy does weird stuff with RxJS though. [Laughter] CHARLES: Yeah, what's some weird uses of RxJS? TRACY: I think my favorite thing to do right now is to figure out how many different IoT-related things I can make work with RxJS. So, how many random things can I connect to an application using that? BEN: Tracy's projects are the best. They're so good. [Laughter] TRACY: Well, Ben and I created an application where you can take pictures of things using the Google Image API and it'll spit back a set of puns for you. So, you take a picture of a banana, it'll give you banana puns. Or you can talk to it using the speech recognition API. My latest thing is I really want to figure out how to… I haven't figured out if Bluetooth Low Energy is actually enabled on Google Home Minis. But I want to get my Google Home Mini to say ‘booty'. [Inaudible] [Laughter] CHARLES: RxJS to the rescue. [Laughter] BEN: Oh, there was, you remember Ng-Cruise. We did Ng-Cruise and on there, Alex Castillo brought… TRACY: Oh, that was so cool. BEN: All sorts of interesting… you could read your brain waves. Or there was another one that was, what is it, the Microsoft, that band put around your wrist that would sense what direction your arm was in and whether or not your hand was flexed. And people… TRACY: Yeah, so you could flip through things. BEN: Yeah. And people were using reactive programming with that to do things like grab a ball on the screen. Or you could concentrate on an image and see if it went blurry or not. ELRICK: Well, for like, Minority Report. BEN: Oh, yeah, yeah. Literally, watching a machine read your mind with observables. That was pretty cool. That's got to be the weirdest. TRACY: Yeah, or we had somebody play the piano while they were wearing one of the brainwave… it's called the OpenBCI project is what it is. And what you can do is you can actually get the instructions to 3D print out your own headset and then buy the technology that allows you to read brain waves. And so with that, it's like… I mean, it was really awesome to watch her play the piano and just see how her brain waves were going super crazy. But there's also these really cool… I don't know if you guys have heard of Jewelbots, but they're these programmable friendship bracelets that are just little Arduino devices that light up. I have two of them. I haven't even opened them. CHARLES: [Laughs] TRACY: I've been waiting to play with them with you. I don't know what we're going to do, but I just want to send you lights. Flashing lights. [Laughter] TRACY: Morse code ask you questions about RxJS while you're working. [Laughter] CHARLES: Yeah. Critical bug. Toot-toot-toot-too-too-too-too-toot-toot. [Laughter] CHARLES: RxJS Justice League. TRACY: That would actually be really fun. [Laughter] TRACY: That would be really fun. I actually really want to do that. But… CHARLES: I'm sure the next time we talk, you will have. TRACY: [Laughs] Yes. Yes, yes, yes, I know. I know. we'll do it soon. We just need to find some time while we're not going crazy with conferences and stuff like that. CHARLES: So, before we head out, is there any upcoming events, talks, releases, anything that we ought to be, we or the listeners, ought to be aware of? TRACY: Yeah, so one of the things is that Ben and I this weekend actually just recorded the latest version of RX Workshop. So, if you want to learn all about the latest, latest, newest new, you can go ahead and take that course. We go through a lot of different things like multiplex WebSockets, building an application. Everywhere from the fundamentals to the more real world implementations of RxJS. BEN: Yeah. Even in the fundamentals area, we've had friends of ours that are definitely seasoned Rx veterans come to the workshop. And most of them ask the most questions while talking about the fundamentals. Because I tend to dig into, either deep into the internals or into the why's and how's thing. Why and how things work. Even when it comes to how to subscribe to an observable. Deep detailed information about what happens if you don't provide an error handler and certain cases and how that's going to change in upcoming versions, and why that's changing in upcoming versions, and what the TC39's thoughts are on that, and so on and so forth. So, I try to get into some deeper stuff and we have a lot of fun. And we tend to be a little goofier at the workshops from time to time than we were in this podcast. Tracy and I get silly when we're together. TRACY: It's very true. [Laughter] TRACY: But I think also, soon I think there are people that are going to be championing an Observable proposal on what [inaudible]. So, aside from the TC39 Observable proposal that's currently still at stage one, I don't know Ben if you want to talk a little bit about that. BEN: Oh, yeah. So, I've been involved in conversations with folks from Netflix and Google as well, Chrome team and TC39 members, about getting the WHATWG, the ‘what wig', they're a standards body similar to W3C, to include observables as part of the DOM. The post has not been made yet. But the post is going to be made soon as long as everybody's okay with it. And what it boils down to is the idea of using observables as part of event targets. An event target is the API we're all familiar with for ‘add event listener', ‘remove event listener'. So, pretty much anywhere you'd see those methods, there might also someday be an on method that would return an observable of events. So, it's really, really interesting thing because it would bring at least the primitives of reactive programming to the browser. And at the very least it would provide maybe a nicer API for people to subscribe to events coming from different DOM elements. Because ‘add event listener' and ‘remove event listener' are a little unergonomic at times, right? CHARLES: Yeah. They're the worst. BEN: Yeah. CHARLES: That's a very polite way of putting it. BEN: [Chuckles] So, that's one thing that's coming down the pipe. Other things, RxJS 6 is in the works. We recently tied off 5.5 in a stable branch. And master is now our alpha that we're working on. So, there's going to be a lot of refactoring and changes there, trying to make the library smaller and smaller. And trying to eliminate some of the footprints that maybe people had in previous versions. So, moving things around so people aren't importing stuff that were meant to be implementation details, reducing the size of the library, trying to eliminate some bloat, that sort of thing. I'm pretty excited about that. But that's going to be in alpha ongoing for a while. And then hopefully we'll be able to move into beta mid first quarter next year. And then when that'll be out of beta, who knows? It all depends on how well people like the beta and the alpha, right? CHARLES: Alright. Well, so if folks do want to follow up with y'all either in regards to the course or to upcoming releases or any of the other great stuff that's coming along, how would they get in touch with y'all? TRACY: You can find me on Twitter @ladyleet. But Ben is @BenLesh. RX Workshop is RXWorkshop.com. I think in January we're going to be doing state of JavaScript under This Dot Media again. So, that's where all the core contributors of different frameworks and libraries come together. So, we'll definitely be giving a state of RxJS at that time. And next year also Contributor Days will be happening. So, if you go to ContributorDays.com you can see the previous RxJS Contributor Days and figure out how to get involved. So, we're always open and happy and willing to teach everybody. And again, if you want to get involved it doesn't matter whether you have little experience or lots of experience. We are always willing to show you how you can play. BEN: Yeah. You can always find us on Twitter. And don't forget that if you don't find Tracy or I on Twitter, you can always message Jay Phelps on Twitter. That's important. @_JayPhelps. Really. TRACY: Yeah. [Laughter] BEN: You'll find us. CHARLES: [Chuckles] Look for Jay in the show notes. [Laughter] CHARLES: Alright. Well, thank you so much for all the stuff that y'all do, code and otherwise. And thank you so much Ben, thank you so much Tracy, for coming on the show. BEN: Thank you. CHARLES: Bye Elrick and bye everybody. If you want to reach out to us, you can always get in touch with us at @TheFrontside or send us an email at contact@frontside.io. Alright everybody, we'll see you next week.

How do we ensure a high level of quality and maximize the refactorability of our code? Frontsiders, Wil and Charles, talk about their battle tested techniques for testing web applications, not only in React JS, but in any JavaScript framework. Links Acceptance Testing Integration Testing jest Cypress.io Assertion Ember CLI Mirage mirage-server react-trigger-change Transcript CHARLES: Hello everybody and welcome to The Frontside Podcast, Episode 90. My name is Charles Lowell, a developer here at The Frontside and your podcast host-in-training. And with me today is Mr. Wil Wilsman. Wil, who just got back from Nodevember just walked straight into the office and is ready to podcast with us on a very, very, very interesting subject, I think, today. We're going to be talking about acceptance testing in JavaScript applications, especially some of the techniques that we've developed here around testing React applications based on the lessons that we learned from the Ember community. But really, more than just React applications. Really, testing any JavaScript application from the inside out, making acceptance tests for that. So, I think we're going to talk about some of the challenges that you encounter and some of the really novel solutions that are out there that we had nothing to do with. And I guess we really didn't have, just more of cobbling together of various techniques for a powerful witch's brew for acceptance testing. Anyway, so Wil, just to round out the problem space or explore the problem space, what are some of the challenges that you encounter with an acceptance test? Actually, let me back it up even further. What is an acceptance test in a JavaScript application compared to what people normally encounter? WIL: Acceptance testing or end-to-end testing is just a problem that every JavaScript app should face. Not everyone does, but they definitely should. And basically, it's how the user interacts with your app through the browser. And every part of that we want to test, from the browser triggering browser events, interacting with the app, not calling functions or clicking buttons, and we're pretending we're a user. CHARLES: Yeah. You know, I know that when we showed up in the React space, that was not really the way that most people tested their applications. WIL: No, not at all. they're all about unit testing. Make sure every small piece of your code works, and to some degree integration testing, making sure your components work with other components. But, nothing is out there really for those big acceptance tests that you want the user to click a button and expect them be brought to a page or these fields to be filled out, et cetera. CHARLES: Mmhmm. Yeah. And there certainly was a very high level of maturity around unit testing, like you said. There are tools like Enzyme and… WIL: Jest. CHARLES: Yeah, Jest. But I was actually shocked to find out that Jest didn't even run in the browser. WIL: Yeah, it's all virtual. CHARLES: It's all virtual. It's completely and totally simulated and stubbed. And that presents some problems. WIL: Yeah. The main problem is cross-browser testing. Some people might consider that to be separate from their acceptance testing but you should be able to just run your acceptance tests in multiple browsers and be able to also test cross-browser support. CHARLES: Mmhmm. Yeah. And so, if you're using something like Jest, you're never actually running the code inside Safari. You're never actually running it inside Internet Explorer. You're actually running it in NodeJS. WIL: And you know, your user is not going to run it in Node. [Laughter] WIL: They're going to use a browser. CHARLES: I don't know about your users. WIL: [Chuckles] CHARLES: [Laughs] You know, we like to stick to the pretty advanced. It's like, go to getNodeJSBrowser.com. WIL: [Laughs] CHARLES: Enough of this Firefox BS. But not seriously, it was certainly a problem. We were looking around, because we never like to build anything ourselves if we can avoid it. But it really just seemed like there was not an off-the-shelf solution for writing these big style acceptance tests in JavaScript. There are some services out there. There's a couple now. What was the… WIL: I think the main one here is Cypress. CHARLES: Cypress, yeah. So, there's Cypress now. I've watched the instructional videos but never actually tried to integrate it into my application. WIL: Yeah. I think at its core it takes the same approach that we've been doing with how we're interacting with our tests. CHARLES: Mmhmm. Okay. The main difference is, is it that it's a service? Like you have to edit your tests through… WIL: Yeah. CHARLES: Their web browser, their web interface, and use their assertion library? WIL: Yeah. I'm not sure about the editing part. But yeah, it's their assertion library. I'm pretty sure it's their test runner and it's their testing environment. Really, the only control through that is through their UI, or through settings, basically. And you're stuck with those. You can't use other… I don't think you can use Mocha with Cypress. CHARLES: Right. WIL: Although it's very much like Mocha… CHARLES: Right. WIL: It's not. CHARLES: Right, right. And I also noticed, we'll touch on this later, the assertions, most of the side effects that were happening were happening right there inline inside your assertions. And that might be an opaque statement, but we will actually get into that later. WIL: Yeah. And I think one of the things about their side effects so to speak is everything leading up to a side effect is a promise with Cypress. CHARLES: Mmhmm. WIL: So, when you select a button and click it, Cypress is going to wait for that button to actually exist before it clicks it. CHARLES: Right, right, which is actually pretty cool. So, that's actually a perfect into into one of the primary challenges with doing acceptance testing in general in a JavaScript application. This is a problem when you're doing it in Ember. It's a problem in React. It's really a problem anywhere. And that is, how do you know when the effects of a user's interaction have been realized? Right? WIL: Yeah. And in Ember you take advantage of the run loop. Once that action happens, you wait for the run loop to complete and then your tests run. CHARLES: Right. So, the idea is that I've clicked some button or I've typed some key or I've moved the mouse. And then I listen for the run loop and when it's “settled” then I can now run my assertions because I know that the side effects that I was looking for have now been realized. WIL: Yeah, hopefully, if you're... CHARLES: Hopefully. WIL: Writing your app right. [Chuckles] CHARLES: Right, right. But that actually presents some problems in itself because it requires visibility into the internals of the framework. WIL: Yeah, so Ember is built with testing in mind. CHARLES: Mmhmm. WIL: And other libraries like React just being a view library might not be built with testing in mind. So, we don't have those hooks to wait for this loop to complete, wait for all of these things to be rendered before you continue. CHARLES: Exactly. And so, this is, I think it's actually kind of both a blessing and a curse. Because there are such strong conventions in Ember, they were able to build this wonderful acceptance testing regimen from the get go. WIL: Yeah. CHARLES: But like you said, that doesn't exist at all in the React ecosystem. And so, what do you do? There's no run loop. You're cobbling together a bunch of different components. And maybe you're using Redux, maybe you're using MobX, maybe you're using… you're certainly using React. And all of these things have their own asynchronies built-in. And there's not one unifying abstraction that's keeping track of all the asynchrony in the system. And so that presents a challenge. So, the question is then, if you're trying to not actually check and observe the state of a system until the right moment, how do you know when that right moment is? WIL: Yeah. And in early testing of a side React project I had, I would basically wait for a state to be complete before I continued my ‘before each'. And in the testing we're doing now, it's essentially what we're doing except the state is what the browser sees, or what the user would see in the browser. CHARLES: So you were actually querying the... WIL: Yeah. So, I was using Redux. So, in my app I was saying, when the Redux app is done loading... CHARLES: Mmhmm. WIL: The instance is set to true or false, then continue the test. CHARLES: Mmhmm. And so, what that means, what we're doing, is doing the same thing except observing at the DOM... WIL: Yeah, exactly. CHARLES: Level. And what it means is we actually… I would love to set this up and have a big reveal but I guess we'll just have a big reveal, is that essentially what we do is polling. WIL: Yeah. CHARLES: Right? So, when we run an assertion, let's say you click a button and you want the button to become disabled, there's an inherent asynchrony there. But what we will do is we'll actually run the assertion to see if it's disabled not one time. We'll run it a thousand times. WIL: Yeah, as many times as needed until it passes. CHARLES: Right, exactly. As many times as needed until it passes. And I think that is, at least to most programmer instincts, an odious idea. WIL: Yeah. CHARLES: [Laughs] WIL: It's like, “Oh wait, you're just looping over every single assertion how many times?” CHARLES: Yeah, exactly. And it feels, yeah, it feels weird as an idea. But when you actually see the code that it produces, it just sweeps away so much complexity. WIL: Yeah. And… CHARLES: Because you don't worry about asynchrony at all. WIL: Yeah. And it's pretty genius. If I'm a user and I click a button, it's loading when I see that it's loading. So, our tests are going to wait until that button says it's loading. And then the test passes. CHARLES: Right. And so, what we do is we essentially, we use Mocha but you could do it with QUnit or anything else, is that when you run your assertion, you declare, you have an ‘it' block or I guess, what would it be in QUnit? WIL: A test? CHARLES: A test? WIL: I think it's just a test. CHARLES: You have your test block. And so that function that actually runs the assertion and checks the state will actually run, yeah it could run three times. It could run a thousand times. It's just sitting there waiting. And it will time out. And it will only fail if that assertion has failed a thousand times or it has failed through, I think two seconds is our default. WIL: Yeah, yeah. I think we default to the runner's default timeout. CHARLES: To the runner's default timeout, yeah. WIL: Yeah. Or you can set that yourself with how we have it set up. And the other thing that comes from that is if your tests are only failing when they time out, how do you know what's actually failing? And our solution to that was we catch the error every time it fails and right before the timeout actually happens we throw the real error. CHARLES: Yeah. Exactly. But the net effect is that you're able to write your assertions completely and totally oblivious of asynchrony. You don't, we don't have to worry about asynchrony pretty much at all. I mean, we do, and we'll get into that. So, I made a global statement and then immediately contradicted it. WIL: [Chuckles] CHARLES: But hey, you got to be controversial. But for the most part, asynchrony just disappears because asynchrony is baked into the fabric. So, rather than thinking about it as a one-off concern or a onesie-twosie, it's just every single assertion is just assumed to be asynchronous. And so, that actually means you don't have to deal with promises. You don't have to deal with run loops. You don't have to deal with anything. You just write your assertion and when it passes, it passes. And there are some really unique benefits for this. And there are some challenges. So, I think one of the first benefits is that it's actually way faster. WIL: Right. CHARLES: Which is counterintuitive. WIL: It's incredibly fast. CHARLES: It's very fast. WIL: Yeah, for all the loops that's happening you might think every loop is going to slow it down slightly. But it really doesn't. Our tests, each test, even though it asserts five or six times, it takes milliseconds. CHARLES: Mmhmm. WIL: The test itself might only loop twice. CHARLES: Right. Exactly. Whereas if you're waiting for a run loop to settle, you might have some… you click a button, it disables, it also fires off an Ajax request and does all this stuff. But if all my assertion wants to know is “Is this button disabled?” then I only need to assert until that has happened. I don't need to wait until all the side effects have settled... WIL: Mmhmm. CHARLES: And then do the assertion. I just know, “Hey, my assertion, the thing that I was waiting for - that happened. Let's move on.” Yeah. And so, it's so, so fast. And that was actually, I didn't predict that. But I was definitely pleasantly surprised. WIL: Yeah, that was a very nice surprise. And all of our tests ran so much quicker than they would have in a run loop environment with Ember or something. CHARLES: Right. Yeah, yeah. That was, we actually had just come off a project where we were having that thing, that exact problem, which was that yeah, our animations were slow. Or, the animations were fine. They were perfect. [Laughs] But there were slowing down the tests. WIL: Yes. I think in that project, it was like, 30-minute tests... CHARLES: Mmhmm WIL: For the whole suite to run. CHARLES: Yeah. To which I'll add a public service announcement. I think this is a conjecture but I do believe that animations are best applied not to individual components but by the thing that uses a component. So, I shouldn't have an animation that's like, implicit to a dialog. It should be the thing that's showing the dialog that gets to decide the animation to use. Anyway, just throwing that out there. WIL: [Laughs] CHARLES: Because animations are about context. And so, the context should provide the animation, not the individual atom. Anyway, moving on. WIL: Some other podcast. CHARLES: Yeah. [Laughter] CHARLES: That's another podcast right there. But there's also, this does present some challenges or requires code to be structured in a way that facilitates this. So, there are some challenges with this approach, some things you need to be aware of if you're using this kind of system. We've kind of settled on a name for what we call these types of assertions and these types of systems. WIL: Yeah. We call them convergent assertions, because you're converging on something to happen. It's going over and over until it happens. CHARLES: Right. WIL: And yeah, a lot of these challenges that we've come across are things that you might not think of, like there are a few instances of false positives... CHARLES: Mmhmm. WIL: That happen with these convergent assertions. CHARLES: Right. So, what would be an example there? WIL: So, the most common example that I'm seeing so far is when you're asserting that something didn't happen. CHARLES: Mm, right. WIL: That would immediately pass. But if it takes your app... CHARLES: [Laughs] WIL: A few seconds for it to actually happen, then you could still have an actual failure but your test passed immediately. CHARLES: Right, right. So, what's the countermeasure then? WIL: We invert our assertions. So, we make sure they fail for a certain amount of time. CHARLES: Right. So, the normal case where you just want to say, “I want to make sure that my state converges to this particular state.” WIL: Alright. I said fail at first. I meant, pass. We have to make sure it passes for a certain period of time. CHARLES: Right, exactly. WIL: So yeah, the normal way is it fails until it passes, and then it passes. When you invert one of these convergent assertions, you're just making sure it passes repeatedly and if it fails at any point, you throw a failure. CHARLES: Right, okay. And so, that's like, if I want to check that the button is not disabled, I need to check again and again and again and again. WIL: Until you're comfortable with saying, “Alright. It's probably not going to be disabled.” CHARLES: Yeah, exactly. And so there, it's kind of weird because it is dependent on a timeout. WIL: Mmhmm. CHARLES: You could go for two seconds and then at the very end it becomes disabled. So, you just kind of have to take that on faith. But... WIL: Yeah. CHARLES: In practice, I don't think that's been much of a problem. WIL: No. CHARLES: It's more indicative of, if your button disables after... WIL: A few seconds. CHARLES: A few seconds, what's up with your... WIL: Yeah, what's up with your app? CHARLES: Yeah, exactly. WIL: If you're waiting for an Ajax request or something, an example, then you should be using something like Mirage Server. CHARLES: Right. Which is, man, we got to get into that, too. There are a couple of other things that I wanted to talk about too, with these convergent assertions. And that is, typically when you look through the READMEs for most testing frameworks, you see the simple case of the entire test, the setup, the teardown, and the actual assertions, are in the actual test. WIL: Yeah. The ‘it' block in Mocha or the test block in QUnit. You click a button, and then make sure it's disabled, and it moves onto the next test. CHARLES: Mmhmm. WIL: Then you click a different button or the same button and you assert something else in the next test. CHARLES: Mmhmm. Right. WIL: And yeah, you can't do that with convergent assertions because they're looping. So, if you click a button in a loop it's going to keep clicking that button over and over and over again. CHARLES: Yeah. [Laughs] Right, right. So, it means that you need to be very conscientious about separating the parts of your tests that actually do the things that actually act the part of the user from the part of your test that's about observation. WIL: Yeah. So, our solution to that is we move out all of our things that have side effects like clicking a button or filling in a form, all that stuff happens in ‘before each's. And all of our actual assertions happen in these convergent ‘it' blocks that loop over and over again. So, our ‘before each' runs and clicks the button and then we have 10 or so tests that will loop and wait for various states to... CHARLES: Yeah. WIL: Be true. CHARLES: Right. That means that yeah, all these assertions do is they read state. And you just have to, you do have to be conscientious. You're not allowed to have any side effects inside your tests, your actual assertion blocks. WIL: Mmhmm. CHARLES: But that's actually, it's a good use case for the whole Act-Arrange-Assert, which has been around way, way, way before these techniques. But here, we're doing Act and Arrange in our ‘before each'... WIL: Mmhmm. CHARLES: And then we're doing Assert later. And I think it actually leads for more readable... WIL: Yeah, definitely. CHARLES: Things. WIL: And it also opens the door to something that we can't really take advantage of yet but if you have 10 assertions with one ‘before each' side effect, you could run all of those assertions in parallel. CHARLES: That's right. WIL: And your tests would be 10 times more faster. CHARLES: Mmhmm. Yeah, exactly. Or you could run them in parallel or you could just run them one after the other but you wouldn't have to run that ‘before each' 10 times. WIL: Yeah. But something with that, that I found, is if we move all of our side effects to ‘before' blocks instead of ‘before each' blocks, sometimes a test three tests down that's waiting for something to happen... CHARLES: Yeah. WIL: That thing might have already happened earlier... CHARLES: Yeah. WIL: And it already went away. A loading state is the best example of that. CHARLES: Mmhmm. WIL: You show the loading state, the loading state goes away. So, if you move that button click into a ‘before' and that loading state test is three tests in, that loading state is already going to be gone. CHARLES: Yeah, so I think the long story short is we've kind of come to the conclusion that we would have to write our own runner. WIL: Yeah. CHARLES: Essentially to take advantage of this. But that said, we've done some sketching about what we would gain by writing our own runner. And the speed, we're talking about exponential speedups. WIL: Yeah. CHARLES: Maybe taking an entire acceptance test suite and having it run in five or six seconds. WIL: Yeah. We're talking about these tests that are already extremely fast. CHARLES: Mmhmm. WIL: Each test takes a few milliseconds or tens of milliseconds to complete. But then if you can run all of those at the same time, all of your tests for that entire ‘describe' block just ran tens of milliseconds. CHARLES: Right. Yeah. So, it's really exciting and pretty tantalizing. And we would love to invest the time in that. I've always wanted to write our own test runner. But never had, [chuckles] never really had a reason. WIL: Yeah. CHARLES: Certainly not just for the sheer joy of it. Although I'm sure there is joy in writing it. But that, yeah, we'll have to wait on that. But I am actually really excited about the idea of being able to maybe bring this back to the Ember community. WIL: Yeah. CHARLES: Because acceptance tests getting out of control in terms of the speed is I think a problem with Ember applications. And I think this would do a lot to address that. WIL: Yeah. CHARLES: I think, how long, if we were just using a stock Ember acceptance testing setup for this, I think we have about 250 tests in this React app... WIL: Yeah. CHARLES: How long does it take to run? WIL: Right now I think our tests take something like 20 seconds. And that's also somewhat due to they have to print the tests on the screen on Travis so that takes a little time. In an Ember setup, that could maybe take a few minutes. I mean, that's not that big of a deal, a few minutes. CHARLES: Right. WIL: But compared to 20 seconds. CHARLES: Right. you're still talking about an order of magnitude... WIL: Yeah, exactly. CHARLES: Difference. And using this, I think you could get a 30-minute test suite... WIL: Yeah. CHARLES: Down to the order of 3 minutes. WIL: Now when we're talking about those times, we're talking about the tests themselves. Of course, the CI would have to download stuff and set up the [inaudible]. CHARLES: Mmhmm, right, right. WIL: And that of course all adds to the time. CHARLES: Yeah, mmhmm, yeah. Earlier you mentioned, we talked about Ember CLI Mirage. This is actually something that is having now been using it for what, 2 years or something like that, it's just… it's impossible... WIL: To go back, yeah. CHARLES: To go back. It is. It's like [chuckles] you come outside the Ember community and you're like, “How is anybody ever dealing without this?” WIL: Yeah. CHARLES: [Laughs] WIL: A lot of the mocks are usually mocking the function that makes the request and it returns it in that function. That's what's out there currently, minus the Mirage stuff. CHARLES: Mmhmm. WIL: But once you use Mirage, you're mocking the requests themselves. CHARLES: Yeah. And you've got such great support for the whole factories. I love factories. It's something that is very prevalent in the Ruby community, and maybe not so much elsewhere. But the ability to very, very quickly crank out high-fidelity production data... WIL: Yeah. And you don't have to have files upon files of fixtures. CHARLES: Yeah, exactly. And you can change, if something about your schema changes, you can change the factory and now your test data is up and running. So, Ember has this tool called Mirage which is just, like I said, it's so fantastic. Oh yeah, it's also go support for running your application. Not just in your tests, but you can actually run your application with Mirage on and... WIL: Oh right, yeah. CHARLES: And you've got now the most incredible rapid prototyping tool. WIL: Yeah. You don't need to connect to a server to see fake data. CHARLES: Right, right. And we were even talking about this yesterday to a potential client. they're trying to, they've got to present something to investors. And how wonderful is it to just be like, “You know what? We just don't want to invest in all, we don't want to move the inertia, invest the money to generate the force to move the inertia of a backend.” Especially in this particular use case, the backend was going to be really, really heavy. WIL: Yeah. And there were some questions about the backend that we couldn't address quite yet but we wanted to start working on something that we could show, something demo-able. CHARLES: Right, exactly. And so, Mirage is just so wonderful for that. But again, Mirage is, it's an Ember-specific project. WIL: Right. CHARLES: So, the question was, “How are we going to use that?” WIL: And you actually took this on yourself. CHARLES: Mmhmm. WIL: I just saw this pop-up one day and boom, you converted Ember Mirage to vanilla JavaScript. [Chuckles] CHARLES: So, I did extract it. But the lion's share of the credit goes to the developers of Mirage themselves. Sam Selikoff and the Mirage community, they built Mirage not using much of Ember. There were some utilities that they were using, but mainly things like string helpers to convert between camel-case and dash-case, and using a Broccoli build, or using an Ember CLI build. WIL: Yeah. That was one of the challenges that we came across using Mirage outside of Ember, was how do we autoload this Mirage folder with all this Mirage config and Mirage factories and models, et cetera. CHARLES: Right. The internals were all just straight up JavaScript classes, for the most part. And so, extracting it, it was a lot of work. But 90% of the work was already done. It only took three or four days to do it. WIL: Amazing. CHARLES: Yeah. So, it was actually a really pleasant experience. I was able to swap out all of the Ember string helpers for Lodash. So now, it's good to go. It shares a Git history with Ember CLI Mirage, so it's basically a fork. WIL: Mmhmm. CHARLES: Like, a very heavily patched Ember CLI Mirage. But I keep it up-to-date so that it doesn't... WIL: Good. [Inaudible] CHARLES: Yeah, so I think the last time I merged in from master was about a month ago, something like that. Because it's got all the features that we need but it's not a big deal to rebase or just to merge it on over in. Because yeah, it's a really straightforward set of patches. WIL: Was there any talk with the creators of Ember Mirage about getting this upstream? CHARLES: So, I've talked a little bit with Sam about it. And from what I can tell, his feeling on it is like, “Hey, my goal right now is to focus on this being the best testing and data stubbing platform for Ember. Anything that happens out there, outside of that scope, that's great. And I certainly won't get in the way of it. But I'm pretty maxed out in terms of the open source credits that I have to spend.” WIL: [Chuckles] CHARLES: And there hasn't been much motion there. I'm happy where it is right now. I would like to see it merged into upstream. I think it would be great to have basically this Mirage Server and then have Ember CLI bindings for it. WIL: Yeah, yeah. I was going to say, either another Ember CLI specific package for Mirage or maybe to make it a non-breaking change or something. CHARLES: Yeah. WIL: Just like an Ember-specific entry point. CHARLES: Right, exactly. And I think that's definitely doable, if someone wants to take it on. I will say, we have been using this extracted plain vanilla JavaScript Mirage Server now for what, almost six months? WIL: Yeah. CHARLES: And it really hasn't... WIL: Yeah, I don't think I ran into one issue with that. CHARLES: Yeah. It's solid. It's really, really good. So, kudos to the Mirage team for doing that. And if anybody is interested in using Mirage in their projects, it's definitely there and we'll put it in the show notes. WIL: Yeah. We call it Mirage Server. CHARLES: Mirage Server, yeah. So, I don't know. Maybe it's time to reopen that conversation. But it has become a very integral and critical piece of the way that we test our JavaScript applications now. So, what are the foundations of it? We've got, we're using Mirage. We're using these convergent assertions. We're using Mocha, although that's really... WIL: Yeah, we have jQuery and Chai jQuery just to help us out with interacting with the browser as a user would. And I think one of the big challenges with that actually, I just remembered, was triggering changes in React. CHARLES: Yeah. WIL: I think this is pretty specific to React. You might run into problems with the view. I don't know how to mess with view. But in React, at least I think 15 or React 16, one of them, they changed the descriptor of the value property on an element so that they can appropriately interact with it, make changes, watch for changes, et cetera. So, when you set this value property using jQuery or just straight up ‘.value()', that change event isn't triggered in React. Your on-change handlers are never called. CHARLES: Wait, they actually update the JavaScript property descriptor of the DOM element. WIL: Yes. CHARLES: Boo. WIL: Yeah. So, there's a nice little helper out there called React Trigger Change. I dug through it and I've stripped some of it down to just be for more modern browsers, more modern React. But there's a lot of good code in there. And basically, it just caches that descriptor, updates the value, triggers the change, and then adds the descriptor back. And that ends up triggering that React element. CHARLES: Okay. [Laughs] WIL: [Chuckles] Yeah, so that calls your handlers and that's how we get around that. CHARLES: Right. There's a few fun little hacks there. But I think it is good to tie that into a larger point, is that the amount of touch that you have with the framework is actually very low. WIL: Yeah. CHARLES: So, the amount of affordances that we've had to make just for React, there's that, that you just mentioned. And is there… there's not much else. We had to write a test harness to mount the app. But that's like... WIL: Yeah, yeah. Our describe application helper is pretty React-specific. CHARLES: Right. WIL: So, you'd have to render it and set up a Mirage server, et cetera. CHARLES: Right. But that's application-specific setup. WIL: Yeah, that's one file. CHARLES: Right. WIL: So, your goal for acceptance tests is you want to be able to have a refactor and your acceptance tests still pass. CHARLES: Right. WIL: So, what if that refactor involves switching libraries? CHARLES: Right. WIL: If you're writing Ember acceptance tests, you're going to have to rewrite all your acceptance tests. CHARLES: Right. WIL: That's a huge downside. CHARLES: Right. WIL: So, with this method of interacting with the actual library very little, we have that one file that sets up our app and then we have that one trigger change helper, we remove those, we can use whatever framework we want underneath this. And our tests would still work. CHARLES: Yeah, exactly. And I think that we actually could theoretically, and honestly I have enough confidence in this style that we're developing the tests now, we could refactor this application to Ember and not have to rewrite our tests. WIL: Yeah, exactly. CHARLES: In fact, the tests would be an aide to do that. WIL: Yeah, and the tests would be faster than Ember testing with that run loop problem. CHARLES: Yeah, exactly. that's really something to think about or to think on, is like, “Wow. You're really at this point completely, not completely, but very loosely coupled to the actual internal library code.” Which is one of the goals of a nice, big acceptance test, is to be able to make major changes, break big bones, and be able to set them and have your acceptance test suite be the bulwark that holds it all together. WIL: Yeah. CHARLES: So, I actually don't know what a bulwark is. WIL: [Laughs] CHARLES: I just know that it's a really strong thing. [Laughter] CHARLES: Maybe we could put that in the show notes. [Laughter] WIL: A link to what that is. CHARLES: [Laughs] So, alright. Well, I'm trying to think if there is anything else that we wanted to mention. Any challenges? Any next things? WIL: So, one of our next steps is something we mentioned that Cypress does, is they wait for elements to exist before they interact with them. And we're actually not doing that in our app currently. And we don't have helpers out there for it yet. CHARLES: Right. WIL: But that's very much the next step. When we go to click an element in our ‘before each', we have these describes that are nested. Say, you have nested describes and you get down three levels into a ‘before each' when you're clicking a button. That button might not exist yet. CHARLES: Right. WIL: And especially since we're using jQuery, if you trigger a change on an empty jQuery element, it's not going to throw an error. It's just not going to tell you that it triggered anything. CHARLES: Right. WIL: So, we get those skips where that button's not getting clicked and we should really be waiting for that button to exist. CHARLES: Right. So, what we've done right now is we're converging on our assertions at the backend of a test. But at the frontend of a test we need to also be converging at some state before we can actually interact with the application. WIL: Right, yeah. CHARLES: So yeah, so that part is missing. And that actually brings up, we are very slowly but nevertheless doing, we're collecting these convergent assertions and convergent helpers in a repository on our GitHub account. We're going to be adding these things so that you can either use them out of the box or use them to make your own testing library. WIL: Yeah. And one of the other next steps that goes along with waiting for the element to exist is when you need to chain convergences. Like, wait for this element to exist and then click it and then wait for this thing to happen before actually running a test. And that presents the problem of our convergences are waiting for that timeout and those timeouts will accumulate. So if you have three chained convergences, that's now a six thousand millisecond timeout as opposed to a two thousand millisecond timeout. CHARLES: Right. WIL: So, one of the next steps is getting that tracking under control so if you chain three convergences together, they're smart about it and they still fail under the two thousand millisecond timeout. CHARLES: Right, right. So yeah, so we're going to be collecting all this stuff that we're learning into some publicly available code. We have a repository set up. I don't know if I want to announce it just yet, because it's really early days. WIL: Yeah. CHARLES: But that definitely is the plan. And that way, whether you're using Mocha or whether you're using QUnit or whether you're using Chai or jQuery, you've got these underlying primitives that help you converge on a state, whether that state is to interact with some piece of the DOM or to just assert some observation is made about that state. We'll be continuing on that. But by all means, get in touch if this is something that is of interest to you. Let's make something happen, because it's something that we're pretty excited about. And honestly, it's pretty comfortable living inside the four walls of this test suite. WIL: Yeah. CHARLES: It feels pretty good. WIL: It does, yeah. They're very fast. And some places in the test might need a little reworking, but for the most part all of our tests are very well-written, very well-readable. And you can just open up a test and know exactly what's going on. CHARLES: Yeah. Alright. Well, I think that about does it for Episode 90. Wow, Episode 90. WIL: Man, coming up on that 100. CHARLES: Yeah. We're going to have to have a birthday cake or something. WIL: Do we celebrate Episode 100 or Episode 104? CHARLES: What's 104? WIL: 104 would be 2 years. CHARLES: Oh really? WIL: Well, I mean 2 years' worth of podcasts. CHARLES: Oh, right. 2 years' worth of podcasts. Yeah. WIL: Yeah, like if you go every week. CHARLES: Maybe we should celebrate a hundred hours or something like that. WIL: Oh yeah. CHARLES: We can add up the thing or celebrate… I don't know, be like, “You've literally wasted 2 years of your life.” WIL: [Laughs] CHARLES: “2 weeks of your life listening to the podcast.” Anyway, so that's it for Episode 90. and thank you so much, Wil. WIL: Thanks for having me. CHARLES: It's always a pleasure to talk about these topics with you. And as always, if you need to get in touch with us, please reach out to us on Twitter. We are @TheFrontside. Or you can send an email to contact@frontside.io.

Kaylie Kwon: kaylieEB | kayliekwon@gmail.com Show Notes: 02:14 - Kaylie's Journey Into Software Development 09:25 - Implementing a Design System and Attacking Higher-level Workflows 15:43 - EDS Collaboration and Public Availability 19:07 - Getting Involved with The Yarn Project 20:57 - Selective Resolution 23:37 - The Warmth of the Yarn Community 27:11 - Handling Issue Communication and Tracking Resources: Eventbrite britecharts Eventbrite Spectrum Transcript: CHARLES: Hello everybody and welcome to The Frontside Podcast. My name is Charles Lowell, I'm a software developer here at the Frontside and your podcast host-in-training. With me today is Elrick Ryan. Hello Elrick. ELRICK: Hey, what's up Charles? CHARLES: Not much. Are you enjoying your morning so far? ELRICK: Yeah, my morning is going well. Everything is good. CHARLES: Lots of cups of coffee? ELRICK: No cups of coffee yet. I've been drinking a lot of green tea. CHARLES: I've actually heard that's really good for you. ELRICK: That's what I heard too. That's why I started drinking it. CHARLES: Did you continue because it tastes good or do you just live on the idea of how good it is for you? ELRICK: A little bit of both. It doesn't taste that great but it's not horrible. It's almost like an acquired taste and then when you add in, "This is good for me," then it tastes great. CHARLES: Okay. We got a nice [inaudible] there. ELRICK: Yes. CHARLES: Anyway, I guess we should, at some point get on to the main content of our podcast. We have a very interesting guest with us today who has her fingers in all kinds of pies that we were talking about just at the pre-show, just before we were recording so we're really, really happy to have Kaylie Kwon. Thank you for coming on the show and welcome. KAYLIE: Hi. Thank you for having me. CHARLES: It's going to be great. Now, you are a software engineer at Eventbrite, that's correct? KAYLIE: Yep. CHARLES: What kind of things do you do over there? KAYLIE: I used to work on part of the feature team that worked on their reserved seating product but not too long ago, I moved to our frontend platform team, which is a team that helps other frontend engineers move faster through things like working on infrastructure or Eventbrite design system and dev tools. CHARLES: So like getting into the tools that unlock the exponential productivity of the developer team? KAYLIE: Uhm-mm. CHARLES: We're going to dig into all of that because you just listed a bunch of really interesting stuff. I'm really excited to talk about the design systems, in particular but lots of different stuff. But before we do, I understand that you have a fairly unique way of entering in to the position that you're in now. Your journey didn't follow the traditional arc. Would you be willing to elaborate on that or tell us that story? KAYLIE: Yeah, totally. I graduated with a degree in art history, not related to computer science at all. Then right after, I moved to New York and worked for a small startup. It was marketing/business development role and I wasn't really happy with it. I was working on some design-y stuff on the side with HTML and CSS but I just felt like my brain just needed to be stimulated a little more. I applied to this all women's bootcamp program called Hackbright in San Francisco. It was a three-month intensive program and luckily coming out of that, I had at least some initial knowledge to get my foot in the door and Eventbrite was one of the hiring partners. They brought me in for interview. I actually had no idea that it was going to be a frontend engineering role because my bootcamp was totally in Python and it just had more with a backend focus. Ben, who spoke at React Boston Conference with me, was actually one of the interviewers and he gave me this Clojure problem and I solved it but in Python just using recursion. He was like, "You got it. Just convert it to JavaScript," and I was like, "No, it just can't be done. No JavaScript." But there's the reason he would chose to hire me and they onboarded me as an internal bootcamp within Eventbrite. The first three months I was there, I learned about React, Redux, JavaScript, ES6, all of that good stuff. Then they moved me to a feature team, where I continued, I guess working on the product and then I became involved more with open source projects. I really expressed interest in when you're a new engineer and coming onboard, there's all these assumed knowledge that isn't documented anywhere or something will be really obscure and hard to use but people will just assume that that's the status quo. This idea around developer experience and helping other developers move faster, it just kind of become a natural interest of mine. I was talking to the platform team, Ben in particular, expressed my interest in working in these areas. Just about like a month ago, we did a big re-org and I landed on the platform team. Currently, we have a lot of projects in flight. One of them being moving our dependency system from our old codebase, which first was written in RequireJS to webpack. We're building out our Eventbrite design system, which is basically a shared UI component library that other teams could leverage. Our platform team will just come in and make sure that the API is usable across different teams and maintain a consistent brand in terms of look and feel. We're also working on other tooling stuff like making sure we use Docker for our dev environment and making sure that the frontend containers don't break, making sure that everyone is on the same version of ESLint, Node and things like that. CHARLES: How do you make sure that the frontend containers don't break? KAYLIE: It's actually hard. I think one thing that we're trying to test right now is using Yarn Offline Mirror and having better caching. When you build a container, it'll look into the cache directory, which is just bunch of committed tarballs. That way, they don't have to fetch to the network each time because once the lockfile or package.JSON changes at our current state, it'll rebuild the entire container and it could take a very long time. We just have a lot of packages that we've added over time. Other things, we're experimenting along with our platform team on the backend side about having remote images. Instead of devs building their containers locally on their computer, having them remotely built by a CI system and then just pulling the container and the images down. CHARLES: Really, really you're like all over the place. That sounds so much fun. KAYLIE: Yeah, it's a lot of context switching. Sorry, if I was jumping too many topics at once. CHARLES: No, absolutely not. I think it's actually fascinating and it's actually capture the kind of scope because when you are doing development, if you yourself are not running up and down the stack, the tools that you use are. The better the tools, the better you're able to focus on the one little sliver of the stack that you're working on. If I don't have to worry about where are my Docker images are coming from or if there's a temporary network flip that I can't install my dependencies, if I don't have to worry about those things, then that makes me more effective so it's important to just kind of lay out all of the stuff that goes into making a quality developer experience. KAYLIE: Yeah, the dream is like frontend people wouldn't have to worry about any of the backend stuff and they just have this isolated environment, where they could just work on what they do best, which is JavaScript and writing features in React. Everything else just works and they could see that replicated through their dev environment as well as QA and Prod and hopefully, make every tooling that they use, like testing and linting all easily, intuitive and accessible. CHARLES: How is it that you're working up and down the stack, making sure that your CI systems, your Docker images but then also at the same time, working on a design system, which you've got collaboration, I assume with some pretty hard core devops teams but also then you're interfacing with designers. You kind of flesh out your design system. Are those the same people on the platform team? Or there are different groups within it? KAYLIE: We have designers and researchers actually, as part of our engineering unit. We work pretty closely with them to define guidelines on what the design system should look like because coming into making this designs system, one thing we really wanted to make sure that is that both sides of engineering and design have input, rather than the previous old version called Style Guide, which was more engineering-driven that not. One team would need a model so they would build a model and a different team would build a slightly different model and we would end up with five different models. They want to be maintained over time and there wasn't really a focus on accessibility or consistency of brand so the design system project was to eliminate all of those pain points. CHARLES: For people who may not be familiar with design system but it's something that certainly is cramping up more and more, what's the general strategy you take? Clearly, you talked about the kind of the pains that it solves, like I'm experiencing this pain where I've got five dialogues, I've got three ways of laying out forms, I've got these problems. What's the strategy that I go about as my organization for trying to implement this? Like now, I want to do a design system, where do I start? KAYLIE: One big thing, I think that helped us was developing Eventbrite design system as just the standalone product. You could run Eventbrite design system as a standalone with all of the documentation and components with each of their different props that could be configured and it has its own set of tests. All of the components are API-driven so there's nothing specific about business logic that it assumes. For example, our button component just assumes that I'll be receiving a type of submission and some [inaudible] body. It doesn't make any assumptions that it's going to be anything Eventbrite specific. It's high-level configurable that the end user wants it to be. Another thing that helped is we have a planned approach session right before we start working on a new component. A developer who would be building the component would meet with one of the frontend platform members and will be discussing the API of the component, the CSS that would go in it and the designer would do the final QA. The development takes longer than if you were to write just a component for your app but we're trying to build it out for a long term use. We have versioning for Eventbrite design system as a library so whenever you make updates, we added to the changelog and then it gets released and we bumped in Core, which is where all of our apps live and people have to get the new version. Basically, it's an orchestrated effort and we have a process built around scheduled releases and bumping it in Core. CHARLES: I get the wanting to have a uniform buttons, uniform labels, dialogs and things like that. How do you attack the problem of higher-level workflows like a form submission and validation process that might have a lot of different pathways? How do you guide that using a design system? KAYLIE: Those are actually really good questions because we went through issues like that. Validation form field or components that have logic around more complex inputs and validation is really hard because different teams use it in a slightly different way so we ended up having two versions of complex inputs. Now, we're in the process of deprecating the V1 and then having people move over to V2. We also use the Atomic Design Principles for Eventbrite design system. It means, things like atoms would be composed of buttons and we have molecules, which are slightly more complex. Then one level above that would be organisms, which are things like forms. Obviously, as you move up to like the higher level, it becomes more difficult for us to decide does this belong in EDS or does it belong with the app. There's lots of refactoring that sometimes ends up happening as a result of something built one way and us realizing later down the road that it wasn't actually very extensible and we just built it for this particular use case. Sometimes, it's the opposite where this was too generic. We can make it a little bit more specific and add more logic to it. It really depends on the component but that's been our strategy, just taking the components as needed. We're still in the earlier stages. We've released EDS a year ago and now, we're almost close to finishing it. We're just missing a handful of components. They'll be more [inaudible] that's available as opposed to adding a brand new component. CHARLES: Did you find guidance in existing design systems? One thing is why not just use one of the existing ones that's out there. Is it a matter of branding? Is it a matter of, "There are certain interactions which are unique to our product that we need to design system to capture them?" I guess my question is, if this is something I want to take on myself, what tools would I be able to reach for and what other design systems would I be able to look to versus what I have to contribute myself? What am I going to be looking for to share with the community and what am I going to look for that's develop that's uniquely mine? KAYLIE: I think consistency was a big issue. Sort of the genesis of idea came actually even before I joined the company but I think what they were going for was that we already had an existing component library called Style Guide and it wasn't really working out for us. To build like a common UI library was a natural assumption but making it better, making it more reusable. We looked at companies like Airbnb and Microsoft and I learned a lot from what they were doing. You definitely wanted some components that are specific to Eventbrite such as a ticket card or a media card that we use for our browse pages which would be needed for Eventbrite specific pages. I think mostly, the designers wanted more control because relying on third library means we don't always get what we want. We're actually thinking about making EDS open source at some point in the future, where it could take themes so other companies or individuals can leverage it but use their own theming scheme. If Spotify came and wanted to use EDS but using their colors and brand logo, then they could do so, just by layering a different configuration style to it. CHARLES: That's such an interesting idea. Is this something that you all have explored, maybe collaborating with another company? I'm trying to think what would be the benefit of making it publicly available, unless you would be getting lots of contributions back to improve EDS itself. Is that the idea? KAYLIE: Yeah. We're definitely set on making it public at some point in the future but making it open source is a different conversation because like you said, there's pros and cons that comes with open sourcing a package. We recently open source britecharts, which is a D3 library. It's been getting a lot of contributions. It's a good recruiting tool but it's also a lot of maintenance work outside of developers normal work hours. Also, we have to start worrying about like when we make a breaking change to a component, we're not just breaking it for our own product but for other developers. We currently have external developers. We have Eventbrite plugin system called Spectrum so developers are already building on top of that and they've been asking for something like this, where they could leverage and match the look and feel with the rest of the product. The downside is lots of maintenance hours and worrying about all the people that would be breaking the component for by just solving your use case. I feel like I didn't fully answer one of your questions, which was you have a different dynamic of people working on both really backend ops things, as well as a really frontend design system work. We definitely got very smart people on the team. Some people definitely have expertise in one area versus on others. Currently, we have five developers and some of us lean more towards the backend of the frontend work and I am one of those people working on things like Yarn and Docker and build system work but it's funny because I was thinking if I had a portfolio then, there wouldn't be any visual components to it, even though I'm a frontend developer. It would just be like terminal screens. We try to divide the work but everyone tries to, I guess develop, at least some shared knowledge around why we make the decisions that we do. CHARLES: It's interesting how they all intersect. I feel like the trend of the last 10 years has been to dev of all the things. I think the first thing was this artificial divide between developers and testing and that came together with the test driven development and test obsessed. Then there was this divide between developer and an ops and that divide went away now with the advent of the devops movement. Now, there's this divide between developers and design and I feel like that kind of wall is collapsing right now. You have developers participating a lot more in design and designers spending a lot more in development. We're seeing that but it's just funny how the devs starts integrating with all the things. KAYLIE: Yeah, that's a really good insight. CHARLES: You said you kind of naturally gravitate towards more of the backend doing the working with Docker, working with Yarn. How did you get actually involved with the Yarn Project? KAYLIE: Eventbrite converted over from NPM to Yarn maybe a year ago and the benefits that we got from converting over was awesome because we were manually editing NPM shrinkwrap, which is a nightmare and the installation speed of the container was really slow just because we were on NPM and it didn't really have any advance caching mechanisms at that point. Yarn just sped up a lot of things for us. I really like the user interface outside of just installing. You actually get a lot of freebie commands like 'yarn why,' that tells you why you have a particular dependency or you could do 'yarn check.' It was just a lot more helpful. I've been wanting to contribute to open source for a while so I did a little bit of work before then but the community was really encouraging when I first try to solve and pick up some first contribution bugs off of their backlog. At the time, they were pushing for 1.0 release so there's a lot of excitement about what Yarn will be and all the new features will be adding. I kept trying to pick places where I felt like I could be of help and then, Christoph, the manager of the Yarn Team and I believe [inaudible] team as well, reached out and asked if I wanted to build the selective resolution feature for them. I was like, "Yeah. I'll give it a go." Then I did it. CHARLES: What is selective resolution? KAYLIE: Good question -- CHARLES: Because I use Yarn every day and I've never heard this term. KAYLIE: It's something that became available with 1.0 release, much like workspaces and most of the time, you're not going to need it but sometimes, you're using a library. That library will have a nested dependency that for some reason, has a bug or you can't work with that package or maybe you just want to dedupe the package so that all of your dependencies end up using one version of that particular nested package. Selective resolution is like a way for you to override other libraries dependencies -- CHARLES: Oh, that's really cool. I like that because I've had that happens to me where I want some version of a library that has got a bug fix and yet, some other library that's depending is requiring this library and they're getting the old version and I'm like, "Nah!" ELRICK: That was happening to me last night. I wish I knew about this. CHARLES: Yes, seriously. KAYLIE: Yeah. Before this -- CHARLES: I'm glad we had you on justifying about this. KAYLIE: Awesome. Before this, you would have to file an issue with the original maintainer and maybe, it'll get fix. Maybe it won't or maybe you're stuck on the old version and you can't do anything about it. We had a really similar issue with the PhantomJS package, where we wanted to use a next patch version with a bug fix but then, something that was requiring it wasn't letting us use it. It works. I verified it. It seems like Facebook started using it as well so it was a pretty rewarding to work on that. CHARLES: That's exciting. I also plan on using it the next time I encounter this. ELRICK: To get this feature, is this a flag that you have to pass? How does it do the selective resolution? KAYLIE: As long as you're using Yarn 1.0 or above, you define resolutions field inside of your package.json, like how you would define that dependencies, you also have extra [inaudible] resolutions. On the left hand side, you put in a glob pattern that you want to match and if you want to match all packages, you just enter the name of the package. Then on the right side, you enter whatever version or path that you wanted to match. It could be a file or a GitHub link or it could be a version or a range or whatever you want. ELRICK: Nice. CHARLES: That's fantastic. Now, you mentioned that as you were coming to work on this, you were looking for features to work on but the community actually did a very good job of drawing you in and getting that contribution from you, which is actually pretty amazing when you think about it. I think a lot of open source projects, either flourished or failed by their ability to attract contributors. What was it that was particularly inviting? KAYLIE: It was a combination of things. I think one thing is they tried to point you to the actual code like if you want to submit a PR, then this is where you would get started. I actually started doing that myself on some of the issues. Everyone loves PRs more than issues so I think giving people filing the issues, some, I guess empowerment to try to fix it on their own, I think is great. They also have a Discord Channel for devs to talk about any questions that they might have, how to set up their initial dev environment, to test things on Yarn. Also, they were really nice. They were really thankful when I posted a PR or commented on the fix. They also use a lot of emojis, which helps. I think I personally found it really rewarding because it made me a better developer. Before contributing to Yarn, I didn't know that much about Node. For me, it was just fun to learn more about like, "This is how something else works," and also, the codebase uses a lot of different linting configurations, which I hadn't really used before so that was a nice learning curve there as well. ELRICK: For your initial time going to Yarn when you didn't know much about it, was the champion from the project that worked with you to get you over the hump or places that you were stuck or did you just have to kind of figured it out on your own or did you ask questions on the Discord Channel or in Slack Channel or anything? How did that process go initially? KAYLIE: I definitely pace myself. I just picked up easier bugs on a repos if possible and BYK and Mel who maintain the project would give guidance, especially through PR comments and they also answer any questions on issues. If I ask questions like what's the right approach here, because sometimes you get a bug and it's not just a bug. It actually has to do with philosophy of like, what should Yarn do in this case. There be like really minor edge cases like maybe NPM does that in a particular way, should Yarn respect that or should it try to be better? Those discussions are, I think really helpful and interesting and understanding what's going on. CHARLES: It's fantastic when the conversation just builds and you're learning stuff and then you actually feel like you came out with the best solution that was available to you at the end of the process. ELRICK: You gain a lot of context about the project during those conversations. CHARLES: Right. KAYLIE: Yeah, what was good about those selective resolution feature was it was completely community-driven, even from the RFC standpoint which submitted by someone in the community and then it was implemented by me, who doesn't work in Facebook. I think that's awesome. CHARLES: There's been a lot of thought that was put into the feature even before the implementation. That's always so critical to getting people's and giving them some specifications, some blueprint of what they're actually going to build. One of the things also that I wanted to touch on too is you mentioned before the show that they have a very unique take on the way they handle communication with the community, not just in terms of pull request but also in terms of issues. KAYLIE: Yarn issue count is around 800 -- CHARLES: Boy, that's a lot. I can feel daunting when you look at that, right? KAYLIE: Yeah, but it's actively [inaudible] project, a lot of people are passionate about it and there are bots that other projects use to just automatically close issues when they're not active. But something that BYK told me was when issues are closed, people take it somewhat personally and we just want to make sure that there's like a human touch to it and we, at least get to the issues without just automatically closing them. I really respect that. I think even when people are really frustrated, the maintainers never really lose their shit. They're always very graceful and when someone is acting outside of the standards of Code of Conduct, there's a gentle reminder. So far, all of the interactions that I've seen to the issues have been really constructive, rather than being like, "Sorry, we're not going to work on this." It feels like it is a community project and people care about how it's being used. It's actually not easy given all the different operating systems that Yarn has to accommodate. It's like a pretty low level tool so I give the guys a lot of kudos for handling that. CHARLES: Eight hundred issues means that with a human touch, that means 800 people have to actually respond to those issues and usually those 800 people are not actually 800 people. They're like 10 people or some number significantly below 800. How do you attack them to try and make sure that they're responded to in a timely fashion? KAYLIE: I think issue tracking is the hardest part of open source. I think it's in the order of issue tracking and then reviewing PRs and then submitting PRs because writing code is writing code but understanding other people's code is more difficult and understanding other people's issues and what the bug is, actually is even more difficult. You don't, obviously have their exact dev environment so sometimes, it's hard to repro. I think we do a lot of things that a lot of other projects leverage, which is we label the issues, we define priority if it's actually impacting a lot of people or if it's a critical bug like you can install a package versus maybe a warning message that could be tweet. Then we kept a board, like a GitHub board to track issues when we were heading for the 1.0 release. I'm not sure if we're still doing that but that helped to a degree. Other people from the community, not just the maintainers, jump in to try and help out an issue, which is probably one of the best things. Then when we merge pull requests, we close the issues that have been referenced. CHARLES: Yeah, it's really wonderful when that thing happens. I'm so curious like how to [inaudible] because I know that it's so disheartening when you're working on a project and you file an issue and maybe, it doesn't even go answered for one day, two days or two weeks or you submit a pull request and no one's even commenting on it. It can be really disheartening and kind of make you question the viability of the product itself, especially if you see a lot of activity elsewhere. On the one hand, I also understand that the maintainers are human and they probably have a lot of obligations. I know that I've got a lot of projects where I let the issues languish and I even have one that I'm using where I can't get any response and it's just so frustrating. KAYLIE: Yeah, even Yarn is definitely not perfect. There are definitely issues that sort of go buried. You could add us at YarnPackage/Core. That pings all of us and the core team. You could call out specific people but that's a tough one. CHARLES: This isn't with Yarn, by the way. It's a totally separate project. It's fantastic when there's that basic acknowledgement. It makes such a difference in people's perception of the entire enterprise. Looks like we're actually approaching time. If we want to give a special shout out to any talks you're going to be giving, any events that you'll be at or -- KAYLIE: Actually, Ben on my frontend platform team is speaking at Nodevember about Next React. I think that's where the shout out. And then Christoph will be speaking at AgentConf in Austria, I think in January about the future of dev tools. I think both things are [inaudible]. CHARLES: All right. Fantastic. ELRICK: Any slick future features coming out in Yarn that we should know about? KAYLIE: I'll keep you updated. CHARLES: She's sworn to secrecy. KAYLIE: Yeah. I'm not sure what I'm supposed to [inaudible] yet. CHARLES: I could hear the pause of conspiracy in your voice. Thank you so much, Kaylie for coming on the show. This has been a really wonderful conversation that's just gone all over the map and those are my favorite kind. Thank you very much. KAYLIE: Thank you so much for having me. This has been a lot of fun, guys. CHARLES: Well, if people want to get in touch with you, how would they go about doing that? KAYLIE: I'm a rarity, where I don't have Twitter. You can email me, I guess. CHARLES: File an issue on Yarn. KAYLIE: Yeah. I'm KaylieEB on GitHub and KaylieKwon@Gmail. CHARLES: As always, you can get in touch with us. We're at @TheFrontside on Twitter or just drop us a line on the email. You use that on occasion too at Contact@Frontside.io. Thank you, Kaylie. Thank you, Elrick and thank you for listening.

Dan Gebhardt: @dgeb | Cerebris Show Notes: 01:33 - The JSON API Spec and Pain Points it Solves 08:40 - Tradeoffs Between GraphQL and JSON API 19:33 - Orbit.js 26:30 - Orbit and Redux 32:22 - Using Orbit 37:24 - What's coming in Orbit? Resources: orbitjs.com (Guide Site) ember-orbit Transcript: CHARLES: Hello everybody and welcome to The Frontside Podcast, Episode 87. My name is Charles Lowell, a developer here at the Frontside and your podcast host-in-training. Joining me today in hosting the podcast is Elrick Ryan. Hello, Elrick. ELRICK: Hey, what's up, Charles? CHARLES: How are you doing today? ELRICK: I'm doing great. CHARLES: Are you pretty excited? ELRICK: I'm very excited for this podcast because this is a topic that I've heard a lot about but don't know much about and it just seems so awesome that I'm just very stoked to hear all the details today. CHARLES: Yeah, me too, especially because of who's going to be giving us those details, he's one of the kindest, smartest, most humble and wonderful people that I've had the pleasure of meeting, Mr Dan Gebhardt. Hello, Dan. DAN: Hey, Charles. Hey, Elrick. Thanks for having me on. I really enjoyed listening to this podcast. It's nice to be part of one. CHARLES: It's good to have you finally on the show. We talked over chat and we talked over email and we meet every once in a while and conferences and it's great to get to share more widely some of the great conversations that always arise in all of those contexts. For those who don't know you, you are a founder at Cerebris and that is your company, which is involved very heavily in a lot of open source projects that people are probably familiar with. One of them that we're going to be talking about today is JSON API. I bet most people didn't know that you are one of the biggest driving factors behind both the specification and several of the implementations out there. DAN: Yeah, that's been a pretty core focus of my open source work for the last few years. Actually JSON API Spec, which is perhaps a somewhat confusing name for those who aren't familiar with it. It was started by Yehuda Katz in almost three and a half years ago, I think now and it hit 1.0 a couple years ago and has stabilized since then and we've seen a lot of interesting implementations on top of it. There are some exciting stuff that's actually coming soon to this Spec that I'd like to share with you guys today. CHARLES: To give us a little bit of context, why? What pain am I experiencing that JSON API is going to solve or it's going to address or give me tools to deal with? DAN: One of its prime motivators is the elimination of bikeshedding. There's a lot of trivial decisions that are made with every implementation of an API and JSON API makes a lot of those decisions for you about how to structure your document, how to include relationships and lengths and metadata in a resource, how to represents relationships from hasOne/hasMany. Even polymorphic relationships have a type of that data. JSON API has opinions about all these things at the document structure level and it also has opinions about our protocol usage, how to use HTTP together with this media type to make requests and for servers to return responses, how to create a resource, how to add resources to relationships and things like that. CHARLES: Also, it's not just this is a serialization format. It's very much like also delving into the individual interactions and how they should be structured, more about the conversation between client and server. DAN: Yeah, in that way, it is somewhat unusual as a media type that covers both. CHARLES: Can you dig into that a little bit because I'm very curious? Something made my ears prick up was when you said, it tells you how to, for example add relationships to a resource. What would that look like? DAN: A lot of the influences behind JSON API are hypermedia-related. It's influenced by RESTful principles and includes a lot of hypermedia aspects. One aspect is how a resource represents relationships in terms of the data in document, the type and the ID that specify a linkage to that another resource in the same document but it can also include links to discover those relationships. There's a self-link for a relationship and a related link for relationship and the self-link will return the data for that relationship in the type/ID pairs. The related link will return the related resources. The Spec doesn't have strong requirements or any requirements about URL usage but instead, it describes where to find resources through these hypermedia links. If you want to say, add records to a relationship, you'd follow the self-link for that relationship when that was returned with a resource. Then you would send a post to that endpoint and you would include the relationship data in the terms of type and ID pairs. It gets down to that specifications so that removes the ambiguity of how to interact with these resources and mutate them and retrieve them. CHARLES: I see, so is there an idea then that you are going to explicitly model the relationships as individual resources? Or is that the recommendation or the requirement? DAN: The link for a relationship would point to an endpoint, which would then model the relationships that are represented that endpoint, so to say just to speak a little more concretely because certainly, it makes some simple concept sound a lot more esoteric than they really need to be. Let's just talk about an example. Let's say, we're talking about articles and comments and maybe an author. Let's say, you've fetched a collection of articles from an article's endpoint and within the article resource, you would have a relationships number, which would include comments and then comments could have links, which one of the links would be a self-link and a related link. The related link could be followed to then retrieve all the comments for that particular article. You could also, if you wanted to add a comment for that article, post to the self-link for that relationships. You post to that whatever endpoint that is specified. Maybe it's 'articles/1/comments.' It could be anything that you want. Now, the Spec does have some recommendations to make everything fit nicely in terms of the URL design patterns and such but those are not, by any means required but having those recommendations just eliminates more bikeshedding opportunities. We find it that people who are gravitate towards the Spec really appreciate having a lot of these trivial decisions made for them so even if we don't want to come down and be hard line about requiring those particular answers, we can at least provide some guidance for how things can work together nicely. There's a whole recommendation section on the site for things like URL design patterns. CHARLES: Right, so things that aren't prescribed but these are best practices that are recognized. DAN: Yeah, exactly. CHARLES: A question then that comes to mind, it sounds like JSON API solves a lot of these bikesheds or just kind of comes in and takes one side or the other for modeling both the resources and the relationships between those resources so there's the... I don't want to call it a schema but the boundaries around which resource are very clear and where they live and how they connect together. I was hoping we could maybe contrast that with some another approach, which is also become very popular and that's the GraphQL approach where you're essential assembling views at runtime for the client. It's very easy to marshal the data that you need to present to your view because you've got only one endpoint, as opposed to having to coordinate between them. I can understand the appeal of that and I was wondering if you have any insight into what the tradeoffs are between the systems and what are some of the capabilities that one can do that the other can't. CHARLES: Yeah, sure. I'm glad that you brought that up because I feel like GraphQL has become a real juggernaut, at least because of its marketing. It's very effective in being marketed for its use to developers and it's capabilities versus REST, as if a RESTful system can't possibly achieve the same outcome or the same efficiency. I'm glad to compare and contrast the two. To be honest, one of our short term goals is to better tell the story on the JSON API site, which was always a kind of a more technical spec-y site and a marketing site. That hasn't really helped its uptake as much as it could as some of the GraphQL sites are very sleek and polished. Anyway, let's get down to it. GraphQL allows you to basically define the data that you want for a particular view and that can bring together multiple related resources. It defines a way to specify exactly which fields you want in that graph of resources. We'll just stick with the articles, comments and authors example. You can specify that you want a collection of articles and perhaps the comments-related to that and the authors and you could have it all assembled in a single response. JSON API also allows you to do just that. It allows you to make requests for multiple related resources to constrain the fields that are returned for each resource and to include all of these related resources in a single document. The main difference in the representation is that JSON API requires that resources only be represented once in a single document. GraphQL may have repetition of resources throughout the document that's returned. For an instance, your articles that may nest authors and those authors like Charles Lowell, may have written three of those articles and that representation of that author is going to be repeated in that JSON API compound document, which is a term for document, which has a primary dataset combined with related resources. That single author would only be returned once as related resource and the linkage between the primary data and the related resources would be established to type/ID pairs. Instead of having the author represented three times, the same type/ID pairs would just be providing that linkage to the same author and that author resource would only be represented once. This happens to be ideal for client-side applications that number one, basically want to minimize the size of a payload that sent. Number two, don't want to after-handle repetition of data by doing extra processing of pushing the same record multiple times into a memory store that is keeping that data. I think that GraphQL is well-suited to applications that request data and display that data pretty much as returned. There is no intermediate holding onto that data in, say a memory store for later access. Basically, it lines up well with a component library link React, which wants to display that data that's returned from the server. If it wants to display that collection again, it will simply request that collection again and pretty much throw away the data once it has been rendered. CHARLES: I can see that. Dan, you and I might be some of the only folks who remember. I don't know if you ever did any Microsoft Access Program. DAN: Yes, I did, believe it or not. CHARLES: Doesn't it feel a little bit like the Access pattern all over again, where you have your components requesting data from basically, constructing a query and requesting it -- DAN: Yeah. CHARLES: And then throwing it up on the screen. DAN: You're going deep there but I do remember that. Definitely, there is that same paradigm. CHARLES: It's really powerful. DAN: It is and it's pretty accessible too because it's a direct representation of what you've requested and there's no intermediate processing. I guess the question is, whether that intermediate processing provides some value. Actually, holding onto that data provided some value because as far as I'm concerned, GraphQL is great for that rendering of DOM data, where the data has no meaning except outside of the rendering. But if you want to actually have models that have some intelligence about that data, then you want to use a store to keep those models in and you want to be able to reuse those models for other purposes. CHARLES: What might be an example? What's a concrete use case that we can ground this discussion? DAN: I would say that the big one is offline. You simply can't have just DOM data that's useful in any way in an offline application or an optimistic application, where you are doing some things client-side and only say undoing them if a request fails. But if your data is DOM and only structured for a particular view, then all you can do with that is redisplay that view. But if you understand the schema of your data and that data is available in a store, then regardless of whether you have a network connection, you can actually display that data in different ways. If that same article shows up in a collection in a list, you could also display that article on its own in a different format with more fields. If you want to, say allow editing of that data, you could allow for an editor when your app is offline. Allow changes to be made to that data and then redisplay it because you understand the fields that are in that data. CHARLES: Right and then at some point later, then spool those changes back to the server. DAN: That's right. CHARLES: It almost sounds like, ironically if a system like JSON API where you have very concrete boundaries around each of the underlying resources in your data model, it allows you to essentially do rich-querying on the client and not just the server. DAN: Yes, that's absolutely true. CHARLES: Because I feel like what you just described to me it's like, now we have some sort of store over which we can map all kinds of different queries to our own liking and there's no dependency on the server. DAN: Yeah and if you just want more web app to be pretty much a view representation of what's on the server and without additional intelligence, then GraphQL really lines up well with your needs because any extra processing you're doing is just not valuable to you. But I think a lot of the really interesting things being done in client-side applications are where your client application is pretty loaded with a lot of intelligence and you're out there autonomous and able to make sense of data. In that case, then thinking about the data only as it pertains to views is not nearly as powerful. CHARLES: Right, so you could do something like that with GraphQL but then you would have to, essentially structure your queries such that they drew the boundaries around the individual resources anyway, rather than composing them on the server. You'd have to query them discreetly into a store and then run your local operations. Then I guess at that point, it's like what are you doing? DAN: Yeah, you're still doing the extra processing of handling the repetition of any nodes that repeat and such. That's just extra processing you have to do but I agree that you certainly could structure your GraphQL queries to return data that is then loaded, say to a store that really has awareness of the data types but I don't think that is -- CHARLES: But then you're defeating the purpose, right? DAN: Yeah, it's not its selling point and it's not its strong suit. CHARLES: You've done a lot of work on the JSON API Spec. JSON API allows you to fetch discrete resources and their relationships but still, keeping one representation of each resource in the payload so it's optimized for wanting to do client-side processing and have intelligence based on these entities, which are in a store. You actually maintain a fairly mature, at this point, framework called Orbit, which helps you do some of these things. Now, what Orbit does today and I understand that you've got a lot of new features that are really exciting, that are coming down the pike. Before we get into those, what is Orbit and what do you use it for and how does it use JSON API? DAN: Orbit is a data access and synchronization library, which sounds sufficiently vague because it has a lot of low-level primitives for structuring client-side. Also, actually isomorphic can be run on the server and nodes so it's not even only used for client-side purposes but that was its original purpose. The abstraction that it includes are allowed for synchronization of data changes across multiple sources of data. Source of data might be represented by, say a JSON API server, an in-memory store, an IndexedDB database in your browser, a local storage, all of these sources of data can support an Orbit interface, which provides their access to their data and also broadcasts changes to that data. In order to coordinate the changes across multiple sources, say to back up all of your data that's in memory to IndexedDB source, you can observe the changes on one source and then sync those changes up with another. For instance, you want to structure an offline application which you have been in-memory store, which uses client-generated IDs, which then syncs up with a backend JSON API source and every change that gets made to the memory store needs to be backed up, you could configure multiple coordination strategies between the sources to make sure that the data flows so that every change that is made to the store is immediately backed up to IndexedDB. If it can't be backed up, then it fails. You can add some error handling and then when you're online, you can then also sync those changes up with a backend so you're basically pushing those changes that are local to a remote store and you're not slowing down your offline app, which you're communicating with optimistically and then only handling, say synchronization failures when there is a problem. In order to handle those problems, Orbit sources are very deterministic about their tracking of changes and they provide get-like rollback capabilities so you can look at the history of changes to a particular source and reset the history to any point there and basically handle conflicts and merges in a very get-like way. Often I use cases, the primary driver of Orbit's whole architecture, I realized that it needed to be able to give you the tools to handle any conflicts that happen when changes get sync up. Also, give you the different tools to model all the different places of data is kept in order to support the offline mode. That's a kind of a broad overview of Orbit. There's a new guide site, OrbitJS.com for those who want to dig a little deeper into it. The data is structured in the JSON API format internally to the store and the standard operations are very much influenced by the standard JSON API protocols that are allowed in the base Spec over creating records and removing records and all that crud for both records and relationships. That's where JSON API comes into Orbit. CHARLES: Right, I see. The primary use case for Orbit is offline. Is that fair to say? DAN: Yeah, that was the primary driver, although it's just not the primary -- CHARLES: It seems like you could use this in a lot of places where I might use Redux or something like that, like on the server to model... I don't know, a chat app. DAN: Yeah, definitely. CHARLES: I have a bunch of different information streams coming together and how am I going to merge them and make sense of them. DAN: Yeah, in fact, that it's primitive level. Orbit has essentially an async redux-like model for queuing up changes and applying those changes. The change sets are all immutable. There's actually a lot of immutability use here throughout the library. In order to ensure that the changes that are applied are tracked deterministically, we just can't have those changes mutating on us. There is definitely some overlap with Redux concepts in terms of the general tasks or action concepts in Redux but instead of Redux's synchronous approach, everything in Orbit is async. CHARLES: What does that mean? Redux is synchronous in the sense that there's a natural order to all actions. For those of us familiar with Redux, are you saying it would be like a store where actions can be dispatched at any time or is it more like, I've got multiple stores happening and I need to resolve them somehow so each one is synchronous? How can I make sense of that? DAN: In Redux, the actual application of an action is performed synchronously. CHARLES: Right. You can have asynchronous processes but there is a natural order to the actions that those asynchronous processes yield and then those are applied synchronously to the Redux Store. DAN: Yeah. To compare and contrast Orbit and Redux, I guess you'd first have to say there's a primary difference of -- CHARLES: I think there are a lot of people are familiar with Redux. I think it's not so much to compare and contrast it but just to use it is as an analogy of like, "Here's how it's the same here. Here's how it's different," because that's compare and contrast. DAN: There you go. CHARLES: But not in terms of evaluating them but it's like, "Maybe I should be using this instead." DAN: Right, they are sort of on different levels, although there are some primitives in Orbit to it and it's shipped across multiple libraries. There are some primitives, I think that could be useful outside of the main Orbit data application. Anyway, the way that Redux state changes are applied, the function is synchronous is all I was getting at and on Orbit, every state change that applied to a source is asynchronous so the result is never applied immediately. You'll always get a promise back and you'll never have that application happen immediately. That's one clear distinction. Another is that a redux has a big singleton global state for the entire application. Orbit very much has a model of state per source so there can be any number of sources in a particular application and the source might be an in-memory source or might represent a browser storage in XTP or might represented a socket that streaming data in. All of these have state at different, temporarily distinct state that even if they all converge to a common state, the Orbit models separately so that there's a set state per source. I'm just contrasting the global apps state that exists in Redux with the per source state in Orbit. CHARLES: It sounds like there's nothing that would be fundamentally incompatible of using Orbit really in conjunction with Redux, where Redux is kind of a materialized view of all of your different data sources presented as what you're going to render off of, right? DAN: Yeah. You could use it in a similar way to Redux Saga, where Orbit fills the role of Saga, where it's doing the asynchronous actions that results flow back into the Redux state. CHARLES: I'm just imagining having one big global atom, which is your Redux store and now I'm saying, prescribing this is an optimal architecture but I'm saying, one way it could work is it picks and chooses and assembles off of the different sources as new data becomes available. As the states change for those sources, it can be integrated into a snapshot state, which is suitable for rendering or provides one view for rendering. DAN: Yeah. You're basically talking about the in-memory source, perhaps merged with other applications state, which is not so resource-specific and that is possible to model. CHARLES: What I think I might be hear you saying is you could also just use another source which is the merge itself. DAN: Yeah. I'm not sure how much we want to continue this thought exercise because the architecture becomes almost not something I'd recommend. But I would actually like to explore how Orbit and Redux could be used together optimally. I played around a bit with Redux but I have not written a full-fledged application with it, other than a [inaudible] location. I definitely defer to you for Redux best practices and such and how people are using it in real world applications but I'd be really interested to talk that over again soon. CHARLES: Well, I just certainly don't count myself a Redux expert, although we have developed some applications with it. We'll put that on the back burner or something to explore it later. I will say this, I find Redux to be both wonderful and terrible, kind of in the same way the Java is both wonderful and terrible. We'll leave it at that. DAN: Okay. ELRICK: That was going to be my question. This is why I was very excited to hear about today was Orbit because I've heard so much about it. In terms of the implementation of Orbit into an application, what would that look like from a high-level? Has anyone used Orbit in the production app? Have you built any apps using Orbit? DAN: Yeah, definitely. There are people using Orbit with React, with Vue, with Angular and with Ember and there's an integration library called ember-orbit which makes Orbit usage really easy in Ember. In a lot of ways, working with ember-orbit feels a lot like working with Ember Data but it allows a lot more flexibility. I suppose one of its strengths and weaknesses is there's a lot of configuration that's possible because there's a lot of possibilities. The internals are exposed of how data get synchronized so you can define your strategies and sync up different sources. In terms of how it's actually used in an application, you'd start by modeling your data in terms of the resources that are in the application. You'd have a schema that defines your articles, your comments and your authors, just to keep that example going. Then that schema would be shared among all the sources in your application. You would have one source, say that might be the in-memory source and another source that is the representing a browser storage so you could, say swap out either local storage source or an IndexedDB source and use either one to provide that backup roll. You would declare those sources, you connect them to each other with strategies so that, say when memory storage changes, you would then sync that change to the browser storage source. Then you'd have back up and you'd be able to then, refresh your page and view the same data you were looking at before. Now then, if you probably want to wire up a remote source so that you're communicating with the servers so you bring in JSON API source and you would then set up a new strategy for working with that. You have to decide like, "When my memory storage changes, do I want this change to happen optimistically or pessimistically?" By that I mean, "Do I only want it to appear successful if it's been confirmed by the server." Depending upon whether you want to be optimistic or pessimistic, you setup your strategies a little differently. If you handle this change pessimistically, you wanted to block success on the successful completion of pushing of that change to your remote server. You have the set of tragedies that define the behavior of your application and then doing your crud operation is probably pretty much directly with your memory source. Then if you wanted to, say do an edit in a form, you might fork the store, now the store keeps its data in an immutable data structures. That forking that store is very cheap so you don't have a bunch of data that's copied. You're just keeping a pointer to that and getting a new pointer to that same immutable data structures. Every time they get changed, there been new references. There's an immutability under the hood but you're pretty well insulated from the annoyances of working with that immutable data structures. At that form, you make your changes, you then merge your changes back, you'd get a condensed change set of operations that then can flow through your strategies. It flow through to your backup source. It could flow through to the back to the server. I think it would feel pretty familiar for users of Ember Data because there are a lot of the API influences came from that library. But obviously, people are using just plain Orbit with other libraries, with other frameworks and finding it useful there but it definitely involves a little more configuration up front to do all that wiring that might be more implicit in library like Ember Data. CHARLES: I understand that before we go, there is some pretty exciting new things coming in Orbit. Do you feel like you're ready to mention a couple of those things or they've been kind of mixed in with the conversation? DAN: Let's see. I have the guides up, which I mentioned, which is pretty new in the last couple of months. In the last year, we did a rewrite and Orbit is now completely in TypeScript and there are no external dependencies. For a while there, I was using RxJS and observables internally and immutable JS so there's now an internal immutable library. It's lighter-weight with fewer dependencies now. I'm excited about that and finally feel like I can recommend people digging in with the guides that are up. I'm hoping to get up the API docs soon. I will say I'm excited. I just got back from a retreat in Greece. Séb Grosjean who owns the company, BookingSync does this amazing thing with the Ember community, where for the group that's working on Ember Data, he invites them every year to come to his family's place in Greece. He grew up working with his family on his rental properties, which was the inspiration for his company, BookingSync and said, "This is a fantastic opportunity that for us to get together and collaborate in a really nice place," and I had a really productive time this last week. This is the very first time I had gone. It was just fantastic and I worked with the Ember Data team. Igor Terzic and I spiked out some interesting collaboration between Orbit and Ember Data so I'm really looking for it to see where those go and hopefully, we'll see a little bit more Orbit, either directly or just through influence appearing in Ember Data. I'm looking forward in working more closely with the Ember Data team. We'll see what comes of that. CHARLES: Yeah, I, for one am very excited to see it. I'm resolved now. I'm just looking at these guides. These look fantastic and I'm resolved to give Orbit, at least a try here, either in some of our applications or maybe trying to spin up some new ones and have it the basis for some of ideas I've been playing with. DAN: That would be awesome and there's a [inaudible] channel, which I hang out into if you have any questions, if anyone out there does. CHARLES: Before we go, if anyone is interested in JSON API, is interested in Orbit, is interested in Cerebris, we mentioned a lot of things that in one way or another, map back to you. How do we get in touch to find out more about these different entities/projects? DAN: I'm at @DGeb on Twitter. My company site is Cerebris.com. Also check out, OrbitJS.com for the new guides. Reach out to me. I'm on the Ember Core Team so I'm also hanging out in the Ember community Slack, depending upon what you want to talk with me about. I'm in all these different places so I love to hear from you all. CHARLES: All right. Fantastic. We'll make sure that we put those in the show notes and I guess that's about it. Do you have anything else you want to leave folks with, any talks, papers or big news coming around soon? DAN: Something that we didn't really get a chance to talk about today, which I'm really excited about is JSON API operations, which is an extension to the base Spec, which I'll be proposing very soon. There's a future to the JSON API once it hit 1.0 a couple of years ago. It didn't just stop. We're looking at different ways to extend the base Spec and use it for different and interesting purposes. JSON API operations, I think one of the most interesting ones. The idea is basically to allow for multiple requests that are specified in the base Spec to be requested in a batch and perform transactionally on the server so the Spec will define how would each request gets wrapped. Each operation very much confirms with the base Spec concept of a request. For implementations, there's a lot of opportunity to reuse existing code for how to handle each particular operation but to provide a whole new set of capabilities by allowing you to batch them together and process and transactional it because it just unlocks a ton of different things you can do, all based on the same base concepts from JSON API. I'm really excited to have something to announce soon about that. CHARLES: That sounds like it might solve a lot of problems that are always associated with those things. It always comes up. What's our batch API look like? I don't think I've been on a project that didn't have a months-long discussion about that. I ended up like kicking it down the road and I'm just flumping something in place. DAN: Yeah, all those messy edge cases where people figure out how do we create multiple related records altogether in a single request and people do it ad hoc and do it with embedding and such and want to standardize that in the same way, that we've standardized the base operations. CHARLES: Well, that is really exciting, Dan. I wish you the best of luck and we'll be looking for it. DAN: Thanks a lot. Thanks for having me on, guys. CHARLES: It was our pleasure. Thanks. With that, we will say goodbye to everybody. Goodbye, Elrick. Goodbye, Dan. Goodbye everybody listening along at home. As always, you can get in touch with us. We're at @TheFrontside on Twitter or you can see our website at Frontside.io or just drop us a line at Contact@Frontside.io. Always love to hear from you with new podcast topics, anything that you might be interested in so look forward to hearing from you all and see you next week.

Alex Matchneer: @machty | FutureProof Retail Show Notes: Charles and Alex Matchneer have a great discussion that centers around routing in Ember.js: what they want to see in a router, what problems it solves, what's wrong with the routing solutions we currently have, and what the ideal future looks like in respect to routing. Resources: Episode 067: ember-concurrency with Alex Matchneer Cordova ember-rideshare react-router Transcript: CHARLES: Hello everybody and welcome to The Frontside Podcast, Episode #86. My name is Charles Lowell, your developer here at the Frontside and podcast host-in-training. I'm flying solo today. It's been a while but that's okay because I've got a really fantastic guest on. Actually, we debated this at the beginning of the show, whether this was the third or the fourth time he's actually been on but no times are too many so hello, Alex Matchneer. Welcome back to the podcast. ALEX: Thank you. It's great to be back. CHARLES: You're still at the same place that you were the last time. ALEX: Yeah. Still working at FutureProof Retail. I'm still working on bunch of mobile ember-cordova apps and that's definitely occupying on my time. CHARLES: Nice. Because FutureProof Retail has a large hardware component and we were doing a series on IoT, we were originally going to have you on the show to actually talk about that experience of what it's like to be a part of a startup and develop software that's going to be running on a bunch of devices and the unique set of problems that poses. But in the pre-show, we decided to scrap that because there's actually a topic that we're both very interested in and you've been heavily involved in lately and might be a really interesting preview as to what's coming in the Ember community and at large. Today we're actually going to go back to talking about the same subject that we talked about in our first podcast, which is routing: what we want to see in a router, what problems does it solve, what's wrong with the routing solutions that we have today. Talk about what that beautiful, ideal future that we want to live in looks like with respect to routing. You've been thinking about this a lot lately. What have you been thinking? ALEX: I'm an Ember core team emeritus and back when I was on it and I'm a lot more active, I did a lot of work on the router, particularly with how it handles asynchronously loading data when you click on links and go to different sections of your app. I spend a lot of time over the last three or four years figuring out the nice patterns for what you actually want to use if you're building out lots of Ember apps. Then kind of around that time, right after landing some cool stuff and some not cool such us query params, which has been a challenging aspect, I start working at this company FutureProof Retail that is like 90% of the Ember work that I do there is in mobile apps. We use Cordova so we're basically running these apps inside a web view, inside either iOS or Android so that we can stay with the technologies we are most familiar with, such as JavaScript and CSS and HTML and build apps using that. We can use Ember to do that. What I found was that I couldn't really apply a lot of the same patterns, all these nice conventions that Ember router gives you. I couldn't really find a way to map that onto what I need to build in mobile apps and there's a few different reasons. I got really busy with the startup, just trying to build these things and kind of went off the happy path where I really just couldn't find a way to make it look like an Ember app. One of the nice things about the whole points of convention over configuration as this sort of Ember and Rails philosophy is that, one of the benefits is that if you know Ember and know Rails, you can drop into someone else's apps as long they're following these basic conventions and immediately know how to be productive and know how it's structured, know how to make a change to it and have it maintain a convention and not just have everybody who's using some framework build these totally different apps from each other that have no shared conventions and whatnot. Everyone is supposed to be able to learn from each other, grow with each other as long as they stay with these conventions. I couldn't really find out how to stay within Ember conventions and build this mobile apps. For a long time, I just didn't really contribute too much to the Ember router at all. I kind of fell out of touch with how most people are using it because most people are building these desktop-centric apps and here I am working on these mobile apps after three years. CHARLES: What are some of the specific use cases that were just impossible to, or not impossible but presented a challenge? ALEX: The first one is which is I think is actually one of the easier problems to solve but still some challenging is that you want something that's called stack routing or stack navigation in a mobile app, which is if you're actually building a native iOS app or an Android app, they both have different names for how they provide you this. But you're thinking of things in terms of stacks. In Android, you might open another activity, which is a full frame of a page in your app and you can push it and then when you press the back button, which is built in in Android phones, it'll pop that off the stack and take you back to where you were. In iOS, they give you a UI navigation controller and let you push and pop view controllers and that is how they want you to think about these applications. That is contrasting to what Ember makes you think about, which is go and define your static hierarchy of all the different places that you can be in an app. But with stack-based navigation, you don't necessarily know upfront all the different orderings of which frames are going to be pushed onto what and you might have situations where you want to be able to dynamically push, say an 'Add a Credit Card' page to where you are and maybe it depends on some data that's been loaded at some lower level in the stack and you can't model that as nested routes in the way that you might think about it in classic Ember apps. It's a different structure -- CHARLES: Now, when you say lower in the stack, I'm curious, if you're entered in aren't you... Oh, you mean... I see, previously in the stack. Okay, so lower in the stack so you're thinking like your current position is at the top of the stack. ALEX: Right, yeah. CHARLES: I see. Now, let me just clarify this in my own head. Your Ember routing structure is ultimately realizes a static tree but at any moment, you are entered into one path through that tree so you do have something resembling a stack. It's just is it the pathways that the ways that you can actually get nodes onto the top of the stack is you're limited because that can't be dynamic. ALEX: Yeah, but even then, it's hard to describe what the difference is but the kind of stack that you're thinking of in terms of the classic Ember router map is more like you're in these different substates than you are different frames that you've pushed onto your -- CHARLES: There's a finite and fully enumerated set of next states. ALEX: Right. To be very concrete, if you have a post route and then a post show route and then a comments route under that and these are all nested in a row, then if you're in the comments route, you are in a kind of hierarchical stack that might have loaded the post that you're looking at and maybe the post call-to-action above that and the comments for that but you're still in one thing. You've just expressed that one thing in terms of these substates so that every other state that's in the parent state can share the same data loading. That's different from saying, "I'm on this page and now, I want to push another page on it and maybe tap some of the data that has been loaded on previous pages." That's more of a navigation stack in a hierarchical substates stack. CHARLES: Is the difference then, the data dependency? Because if you think of the Ember classic where you got the static tree, at least theoretically all of the data in the leaf nodes depends on the data that's above. It's not just being able to dynamically push stuff onto the new stack but it's also saying, you want to be able to push stuff that might have no dependency on the stuff further up and it doesn't need to be re-rendered if stuff further up the stack changes. ALEX: Correct. CHARLES: But sometimes it might. ALEX: Right so there are a lot of corner cases that come out if you try to model this new way that a lot of corner cases have been thought out of if everything matched nicely to this hierarchical substate classic Ember stack but not for navigation. If you want to do something that's stacked routing-based, I've had a few different approaches. At our company, we maintain a suite of different apps that are sort of retailer or grocery-centric and the first one we did, which is more popular flagship one is Mobile Checkout, which is an app that lets you going to stores, scan items with your phone and checkout and skip the cashier line, which is great if there's huge lines and you just want to buy a little handful of things or maybe in your shopping cart. But that is like any other mobile app is really conducive to this step navigation approach. Then we had to make a few apps after that such as like another app that is [inaudible] do a manual check then ordering app and other of handful things that you can imagine is might be used on a grocery store. I took the opportunity to like, "I don't really like how the routing turn out the main mobile checkout shopper apps so let's try different things." If you approaches, at least have their pros and cons without really feeling you're solving the problem and one is to maintain your own in-memory stack of where you were, every link to you, you might recall where you were and then use that logic in addition to what's in a URL to decide what transitions to make, which to use Liquid Fire for that. But already, there's these weird growing questions like, "Why are you even using the URL? Is it helping you at all?" That was the main issue with the main app that we did. The other approach was to try and not even use any of the 'router.map' stuff at all. I use the router.map to basically just create one wildcard route. You can use normal Ember to use it like '*half' and that basically collects the rest of the URL as a param that you can use to do whatever you want with. I was using that to basically pass to another, which is internally used by Ember to do the stack-based parsing like grab a little bit of the URL and then parse the param for that then grab another. Every time you could see your stack in the URL. That has its benefits but the worst part about it is that it's getting further away from Ember so any add on that you might want to use at Internet of Things in terms of which route you're in and has conventions like that you just can't use. I can't think of a good example at the top my head but it's like the further you get away from those norms, the less the Ember system can help you and on your own building your own framework. This is all to say that I think I have enough experience at this point to bring home some of the things to Ember and I'm excited to get back into contributing to Ember with this one particular thing that I'm focusing on now, which is... I don't even know what to call it. It's like -- CHARLES: What does it do? The route stuff? ALEX: It's route stuff. Actually, let me get into the other... That's what is tricky about stack routing and tricky to sort of, if you already have to go through a mental hurdle with thinking of the Ember router and as a stack of states or substates and you train your brain to think that way, it's really hard to take yourself out of it and realize that what you're trying to build with like a classic mobile navigation is almost looks like the same thing but it's really different. The other challenging problem, which is specific to our particular app is that you wouldn't think of it as a very heavily server-driven app but if you're writing an application that at any point can get a message from the server like, "Hey, your status has changed," and that state is heavily coupled to navigation of where you're allowed to be in your apps for the state of some certain model, then you're going to have a really hard time, I'd say in modeling an Ember. I have a really hard time convincing people of this until they've actually tried to do it themselves, which is why I'm going off and just building things showing people. CHARLES: You don't have to convince me because I think one of the biggest problems is the router is like the one non-reactive piece of Ember, which is unfortunate because it's essentially, what is the equivalent of the Redux store in a Redux application, where it's the state that drives literally the entire application and yet, any type of non-hash change driven updates, you have to manually manage. Every time that we've done it, it's been a problem and depends on what data, at that point you have to be very thoughtful because, at least from the highest level, if there's damage to a piece of the tree higher up, you need to realize those effects of that damage or that change all the way down the tree. ALEX: Exactly. That is a great way of putting it. This is maybe a good time to mention this thing called ember-rideshare. I've had a really hard time describing these problems to people so I figured what I would do is write this blog a few months back, a little article called ember-rideshare. It's just a given name to the kind of app that still really hard to write in Ember. It's a mobile app. It involves stack routing but the other part is really difficult about it is this problem of the router being in a silo. It is reactive but it's only reactive to that URL. Other things changes, they need to, like you said come in and patch up something else about the router in case you add some URL that is no longer able to present some model of whose status changed. That's an article on a blog that I can probably link to in show notes or something. When I talk about ember-rideshare, imagine using Ember to build Uber or Lyft and it's got just the slightest bit of the whole thing. The whole point of the app is to coordinate your client-side request of I want to ride with the server going off and doing a bunch of things and finding a nearby driver, displaying you bunch of driver locations and it'll show up. Then finally, find you a driver. It's a constant communication. Throughout that point, you can sort of imagine modeling all the different screens as routes but the routes that are actually allowed to see at any given time are heavily dependent on what is the current state of the user's current ride. But you shouldn't be able to go to a route that says like 'cancel ride request,' if you haven't requested a ride in a million of these other things. If you're an Ember developer and you think that's an easy problem to solve, you're probably thinking, "I would use before model hook when I'm entering that route to check the state of the model," and if it doesn't make sense for the route of entering, I want to transition elsewhere. That's fine. That's good if you're doing an app if the user is the one deciding where to navigate to. But then when you're on a route like that and then the server tells you that your ride is done, you can't still be on that route so you've got to have some kind of validations that is like, "This is no longer a valid route to be in. Is the user still in this route?" CHARLES: "Where am I going?" ALEX: Yeah. Before model doesn't really help you. It's this one-shot discrete event and you just can't capture all the different things. The ember-rideshare describes some of these problems a little bit more detail but that's the main issue with it. Like you said, what is actually missing about the router? Maybe it's reactive but it's only reactive to the URL, what about all these other things that are happening into your app? I think there's a handful of APIs in Ember that they're great but they're kind of siloed off in a way. If you want to make two different kinds of worlds meet, you've got to write a bunch of your own code yourself or you just have to do mentally going back and forth and being like, "I did this, so I can't use this kind of API." I did a lot of work on the Named Blocks RFC, which previously there is silos between if you're passing blocks to a component versus data, you've got to think about them differently and all the ways that you might forward that data to a different internal component, if you want to build these composable, reasonable internals, you got to be kind of split-brain about it. I feel the same way about how the Ember router works. It's only good at dealing with stuff that has to do with the URL and you're on your own, if you needed to react to data changing. That's what I'm trying to fix. Does that correlate with your experience of working on Ember stuff as well? CHARLES: Absolutely. I think that's a great way to put it. I think we've come to a consensus of the problem statement. I am curious to see a big separate query params. I'm going to throw that wildcard out there or maybe we should save it for later. ALEX: Yeah, I definitely going to come back to it. If I say all this cool stuff and I still don't have a solution to that, then what am I talking about? CHARLES: Right. ALEX: Which to be honest, I haven't thought of every single possible thing. I'm doing the thing where I talk about it on a podcast that everyone can guilt me into really finishing it. I actually really think that I'm going to finish it. I'm very confident in stuff I'm working on. I'm very excited to bring it to people but it is not all 100% fleshed out and I definitely appreciate anyone's help to those interested, understands the nature of the problem and wants to help me work on some of this stuff and like that, in Ember community Slack or wherever. CHARLES: Yeah, I'm really excited to hear it and see in what ways we might be able to contribute. ALEX: Basically, the goal is to find some underlying primitives that can model the current behavior without mistake because obviously, we can introduce something that's going to break into Ember apps. Basically, to recognize that the URL is something that goes through multiple passes of transformation, to eventually become the thing that displays stuff on your screen, from the very foundation of it, and this is the actual mini-course of what Ember router does internally because it involves a few different libraries and maybe this is a re-hash from the podcast that I did with you guys but -- CHARLES: Can I just say that there are some things that the Ember router really does right, that are fantastic? One of those things is it baked in to every single piece of data. It doesn't do the stack but in that tree that it models, every single node in that tree abstracts away the asynchrony of that node. I think that's absolutely huge so you get both the dependency enumerated like these are the things that I need to marshal the data to render myself and it's implicit that it might take some time. I might need to draw on a couple of different things to actually assemble this data so the asynchronous nature is modeled up front and it's implicit and it's there every single time, which turns out to be the right thing. The sampling that I've missed has been an excruciating void in all the other routing solutions that I've tried outside of the Ember community is that they just punt over asynchrony to you. You deal with it, not our problem and it's like, "Actually, that is the problem." Anyway... ALEX: That's a great point because if the router doesn't help you with any of this stuff at all, then it basically means that every one of your pages that you might want to render after the fact, probably has to have some loading logic like if data is loading, show us spinner. Otherwise, here's all the data -- CHARLES: Yeah, if something happen wrong. ALEX: Right and sometimes that is actually what you want to do. Sometimes you want to do these skeletal in UIs that looked like the page that's about to display but the date isn't there yet so everything is, regardless going to be wrapped in these 'if' statements, 'else' statements. I worked in ember-concurrency and some people are using that to basically move more of that loading into controllers, that's fine. If that's what you're actually trying to do and that's what you're opting into, that's a perfectly reasonable solution but most of times, chances are you're entering a route and you don't want to have to teach the entire template tree underneath it that has to handle all these different states. There's these nice ideas that work in some cases and I'd like to make them work in more cases than Ember helps with and a whole loading all the promises and the model hooks and absolutely going into the loading state are really cool primitives that Ember is going to do for you. The other frameworks, they don't try to be opinionated. They won't do any of that for you. Sounds like you ran into that with some of your React stuff? CHARLES: Yeah. I definitely did. There's just not much help when you actually want to model asynchrony. You can do it. It's pretty easy. You just implement the right hooks or model a series of actions, either with a Saga or Epic, if you're using redux-observable. But again, you have to assemble it by hand and you have to generate those abstractions by hand and you just want to have them at hand already and not have to worry about that. But the advantage, though is that generally those ones that you do have at hand or that you generate are fully reactive. If new information comes that's germane to that particular leaf in the tree or that particular note in the tree, there's no difference between the initial state and the update state. Whereas, in Ember, you got your first shot and then that data is now at rest. ALEX: Right. I definitely have been looking at React router, in particularly v4. I think it's all contentious for people to see it at first but being able to put things like in your render function, you can say, "If this data is present, something that's going to be past and be a prop or something," then show a loading spinner or otherwise, start matching these subroutes. That's really cool. That's expense that you can't look at essential map of all the states of your router can be in but that's also a real problem and if you can demonstrates that the state world is not in a separate silo than the routing world. CHARLES: With great power comes a lot of bugs. You do run into a lot of things where you have rogue matching. You have random things that are inside your view tree that are matching against the route and they just render and you have to be very careful because it's almost the difference between blacklisting and whitelisting. I see what you're saying. It could be confusing. ALEX: Yeah. I think it's definitely a tradeoff. I think if I had something like a match, I might have been able to maybe arrive at a stack routing solution a little earlier. I'm not sure about that. It's definitely something that could be handled by React router. I think one of things that React and React routers better at in general is that everything is, more or less a component that is more easily swappable or something else here. You're not going to have as many of those silos and I really do think, it went through a lot of churn and maybe, some people had trouble, maybe a lot of people, I don't know had trouble kind of following all the major versions. But I think React router Version 4 is pretty damn cool. I think there's a fullest realization of that kind of modular mindset. CHARLES: I think the biggest problem I have with it, though is it requires the view tree to model your routing structure. That bothers me. I feel like you could do the exact same thing. You could have a way to express your routes, not necessarily with a separate routing file. I supposed you could do it with JSX or something but actually have it be kind of orthogonal to your view tree. The way you can model this dynamically updating thing that can match against anything and maybe, even express it all in one place. Although once you get a big tree, it could be hard to control that. The part that I've come into most conflict and maybe who knows, maybe I just haven't used it enough, we've only got one application that we're using the router V4 on. But the fact that it's actually in the view tree, it bothers me. It's in the state objects. It's hard to adapt to Redux because that state is opaque. It's the routers controlling it and I would it to be not have to pass through React components but just be like, give me the firehose of the router state. ALEX: Right. I love what you're saying. If I'm going to bring this stuff to Ember, I can't suddenly make it work like matching within the view tree. That's not what I'm working at or proposing here. All the stuff is basically to empower that firehose to respond to more things that can drives views and respond to them in a live way, not like a one-shot async validation, only when you enter. CHARLES: Maybe this is what the problem that you're trying to solve and one of the things it's really nice to be able to match against anything inside the view tree is that Ember's rendering process of a route is very opaque. The process, by which an outlet gets connected, that's not something that you really have much visibility into. Is that a good statement of the problem? ALEX: That's definitely part of it. You definitely have to go to the documents. I think it's telling that -- CHARLES: I've never done it. I don't really know how that works and I've written a lot of Ember code. ALEX: How what works? CHARLES: How the route gets rendered, like the mechanics around, which I understand how the route object actually, you makes the decision to render its template and do all that stuff. I know it as a user but I don't know the mechanics and I wouldn't know how to extend it. ALEX: I'm not sure if the stuff I'd work on but it immediately make some of that stuff more clear. One of the goal or constraints is to really try and break down the silos. Whatever I'm about to propose bringing to Ember, I want it also be something that would be useful, possibly at the component or template or controller level, rather than just being this thing that lives only in the router's weird black box of logic that occasionally calls hooks that everyone knows about. CHARLES: Right. In a sense, I'd say that they both suffer from that same problem. I'm curious to hear about the firehose. ALEX: To actually get into what I think you're building here, we can dance around it all day and then we -- CHARLES: Just save it for the last 30 seconds of the podcast. That way there could be no -- ALEX: We're swapping JS for React router V4. Bye! It's basically this. What's happening today is that you have a URL, it's going to be parsed in a way that you've tied it to via the router map file, which every Ember app has the place to go to see all the different places that you can navigate to an Ember app, which is great. You basically taught Ember how to break your long URL string into these usable bits and that's going to give you an array of these things that internally who cares what they're called but they're called handler infos and they basically say, "The first element of this array is named application. Every Ember app has one. It doesn't have any params." The next one, it starts getting into what your URL actually is. Maybe it corresponds to the '/post' portion of the URL so that's going to be named 'post,' and that doesn't have any extra params either. Then there's this thing that is post show or something like that. That has a dynamic param because that's the part of the URL as like the '/123' and that corresponds to the post ID. It's basically, if you like thinking of things in terms of transformations or observables or mapping and functional transformations, that's taking a URL and turning it into an array of these useful POJOs of information. The goal is to keep transforming that into something eventually has enough data to display and templates and whatnot. In this giant black box of the Ember router, it's going through those transformations and then it's going to go through this long series of using these params and this useful array of POJO information, start hitting hooks on people's routes to load data. Hit before model after model, redirect all these things to give tasteful names to all the tons of validations and checks that you might want to do. You do cool things in your before model hooks, check if the current user is actually an admin to prevent them from going into any '/admin' subroute. That's a really cool place to go and it's also a great convention. If you're new in Ember app, you realize you can't go on this route. It should sort of click in your head and that sounds like they've got one of these redirect hooks to ensure that you're not going anywhere you're not supposed to go. All these things are really still to this day, extremely strong, well-designed, it went through many passes of review before it landed. I think they cater to a certain kinds of user-driven clicking around apps but they are extremely strong to this day. I think the only thing that's missing is the smell. That example I gave like checking if the user is an admin, it's a bit of a smell that is not reactive. It's a hook. If it passes, great. You're in the route. It's not going to keep on checking that. What I want to do is basically, either in addition to or as an alternative to specifying these one-off model hooks or these hooks that you, not only really just fire one time, have essentially what is an async computed property or an async validation that is upfront about things it depends on. Ember is going to be smart enough to constantly reevaluate these things as stuff changes. It can depend on not just URLs or URL parameters but it can also depend on data. If you're thinking about ember-rideshare, which again is the imaginary Ember app that it's essentially Lyft or Uber, if you have a current ride model loaded somewhere, maybe by a parent route or maybe it's some sort of service, you should be able to specify it like an async property or validation that says, "I depend on ride.state," and for all these subroutes, you would want to say that, either upon entry or any point in the future, if the state ever changes to something that I don't know how to handle that go to some default route. That would be already, particularly in my app, which is a subset of a different kind of ember-rideshare app, that would be a huge help because the only other alternative is to build a sibling-central coordinator to the router that isn't the router but has to sort of agree with it and then, every one of these frames that you might push onto the navigation stack, they have to do some little chunk of code and then invoke this logic and be like, "Did the state change? Go where you're supposed to go," and they have to do that logic. It would be, I think a great win for conventions as it has if it's a benefit to make people shout out their states in advance to empower them to shout out also their data constraints in advance so that you get things like automatic redirects and things change, I think that would be huge. I know that would immediately benefit off of it and I think it would fall in the same kind of problem solving that they worked on like Ember-related stuff which people don't realize how big a problem is until they see there's a better way of doing stuff. I think with that being there -- CHARLES: As an example, let's say that you're an admin and then all of a sudden, you got fired and there's an event that comes from a server that's this person is no longer an admin and it wipes out the Ember data store and then redirect you outside of the admin route or something like that. ALEX: Yeah, that's a perfect example. To be pedantic, I think a lot of people do hard refreshes between login/sign-off stuff but if you have it all in your Ember app, that would just happen automatically. You'd still want the ability to have more graceful transitions because one of the tricky things about having stuff driven by data is that you have this giant matrix of like, "If I'm in this state and this event happens, how do I handle it? How do I make it look well-designed to the user?" But you're not going to be able to hit every one of those constraints so to just have some basic logic that's just like, "Oops, something happened," you're not an admin so we move you to the sign-in page. For in those cases, we haven't fully filled in all those leaks. I think it would be a huge win and you can just progressively decorate things according to the common flows that people take through your app. CHARLES: You know, I'm just imagining this. Model promise, for example would be some computed property, then how would you enumerate your dependencies? Just do the mechanism that we have now? Or are you imagining something entirely new? ALEX: I don't have a strong opinion on it because the moment I start saying what that specific syntax is, more people will agree on what's missing and what we need to have, regardless and be like, "I don't like it." I'm leaning toward something inspired by a lot of my learnings from observables, which is actually we talked about last time. The whole thing about observables is that there's almost limitless flexibility as to if you're in observable, it can take that event. It has been another observable based on that thing. If a URL changes and you're listening to that via observable description, inside that, you could kick off another observable of Ajax request based on that URL and it doesn't make you enumerate all these things upfront. I think there is going to be a compromise between that. I think when you get into these kinds of problems, you run into stuff like Relay, which is familiar with -- CHARLES: I haven't used Relay. ALEX: Just the idea of dynamically collecting all of your dependencies upfront before hitting the server and asking for specific chunks of data that you need, it's a very promising idea. There's cases of just dynamicism where the data comes back from the server, then you realized that you need this other piece of data and there's no way you could have collected upfront, unless you statically wrote it upfront. I expect to find that with this approach that there's going to be some stuff where you just have to be more upfront about it. But I had a cool little strike the other day on auto-computed properties and I'll also link to that. It's a different way of running computer properties where you don't have to specify your depending keys upfront but your getter function gets passed a getter function itself. CHARLES: It's past the dependencies? ALEX: Not even that. Imagine writing a computer property and the first [inaudible] is a function that you can call to get a property off of this but also track that you've got that property. If it ever changes, it'll invalidate again. That means if you're implementing a [inaudible] in computer property, you don't have to write first name twice, both in your dependent keys and in the actual getter in your function, which I think is kind of cool. I'm trying to make that pattern work for this data loading thing so that you don't have to have this huge verbose thing. You just lift this stuff in one place. I've sensed that the magic will probably break down in some complicated cases but that's what I'm trying to run with because I think it's pretty cool and succinct and sort of the natural evolution of what people think of as computer properties. The other major constraint and this is also what we're talking about because it's one of the best kept secrets about the router or it's one of these things that everyone's benefiting from without realizing it, is that if a transition occurs in the router, everything in the router is going to be a possibly long asynchronous chain of operations that it collects all the data that it needs for the new routes to display. In that time, if something happens, if some hook comes along and has an exception, it can load data from the servers. If something happens then it just says 'transition.abort,' that's going to stop whatever transition is in place and you're going to stay exactly where you were and if you're not stuck in a partial transition state, that's pretty awesome. That's basically database atomic transaction semantics that people have been benefiting from if they've been using Ember for years at this point. But again, it suffers a problem being locked away in the router. That is a cool concept. You should be able to specify like I intend this change of the state this way and if I gave you something that is logically inconsistent or can't be fulfilled, don't leave me in a weird half-assed state that I need to somehow fix and know how to fix all the different places, where I might be kicking off this transaction. I'm trying desperately to preserve those semantics when data comes into it. One of the hardest things to do is and honestly, can be one of the hardest sells for people who are used to thinking about Ember is there's an issue of if you imagine whatever API we're talking about, it's probably going to live on the route. Some kind of hook that might be called resolve or something else, like what is the value of this context object that every function has? Is it a route? It's tempting to want to do that and maybe, that will end up winning but winning out is the best API to get people to use. The thing to realize is that there is no consistent value of this. This implies that there's a state of the world and you're looking at it and currently, these things have these values. But in the transaction phase, there is no stable 'this object' and you can wind up with some weird surprises. I know because, not actually these days but particularly, when a lot of the stuff landed and people started trying to do weird things and these transaction hooks, there's just like, "Why can't I grab the controller? The property isn't what I expected?" Honestly, all the stuff that is gross about query params because of this fundamental violation. You have something that pretends to be a property that is there today but is still driving this asynchronous thing that could fail. CHARLES: I kind of viewed this as playing an off-note in the jazz thing like you only want to reserve using this, unless you're the Miles Davis of JavaScript, don't use this. ALEX: And by Miles Davis, you just mean like the god of concurrency that's incorrect race-condition-y code. CHARLES: Right, so it's just like you've got the right reason and you can spot the one-in-a-million case, where it's appropriate. You can spot it in an instant. ALEX: Exactly. I'm not that person and I don't know too many people who are and that's not the API you want to land. I'm trying to, maybe wean people off on dependency on this because the way we've gotten around it in the past is to use again, is more discrete, get the value functions called 'get model' and 'get params.' These are all very in-depth stuff if you're pretty experience Ember developer but it's a way of getting a value from one of these parent routes when you're inside a transition and the rest the world can't see it but you can because you call this hook at the right time. It's super gross because it's just a method on a route that anyone can call in any given time, whether you're inside this transaction or not. The branching logic of, "Should I look up the data from the transaction object?" because once valid, I should have get the current value of a loaded route. It's really gross to me and it causes real problems that confuse people and causes them to write issues because they've given an API that makes them feel good about treating these things as stable objects. CHARLES: I'm trying to imagine now, just like a spike in my head. I know you don't want to get too into syntax but essentially, modeling the route tree as a set of observables, where essentially, instead of returning a promise from your model, you're just mapping an observable off of some combination of the URL state or what are the other streams of state you want to merge to realize that route. But what I'm not seeing, which I'm sure you also have the answer is the original problem, which was stack routing. What we've been talking about is making the router fully reactive like this fully reactive tree that's always on. But that problem seems almost orthogonal to the stock routing problem. ALEX: It is. It's been very tempting to combine them. Why it is such a hard problem? Because you've got navigation stack, which almost to this route hierarchy stack that [inaudible] about but they're separate so you can't really apply the same lessons. Then you've got stack routing, which is you want the ability for routes to while they're loading, reference data that is dynamically available to them. I don't have a solid answer but I would say, the one thing that I think is going to help is that you have a few options for what you want to stash how you want to represent a URL or where you want to stash your hierarchy. Actually just track it in-memory and if you refresh the page, it'd be like, "I depend on some data that I expected to be there but it's not. It transition elsewhere," which is not a great developer experience. You could want to be able to make changes and refresh the page and continue where you left off. Otherwise, URLs aren't actually used by mobile app users. But the other place that you could possibly put the navigation where event stack is in a query param because that can be fully dynamic and you can just sort of manage every single page. The most current page you've pop is just some top-level route but you're tracking the state on the side. I think if you solve the problem of being able to depend on things that aren't the URL or go through a more complex transition than what the router gives you by default, I think it would be possible to treat that query param or that thing you're stashing in in-memory as another source of data. The other thing that I want to try and make sure that this new API has is really treated dependency injection where you specify all the things that you need and you don't really care from a route's perspective where they come from. I think if you had that, that would solve a lot of problems with stack routing and where it gets data from. To be very specific, today if you were in that post '1, 2, 3' comments route and you needed to access the post model from within the comments route, you would probably do this model for post. Basically you're naming not just the model that you need. You're naming the route that you know provides it upfront, which I think is that. Actually, the real reason it's kind of the smell is that, if you ever need to change the nesting, maybe you need to introduce another level or you want to nest all that under an admin route. Then suddenly, you're asking for the wrong route name. You're not really sure all the different things you need to update if you ever change the nesting of your router. There's solutions like relative URLs that a lot of people thrown around but I think -- CHARLES: To go back in the observable world and specifically, the redux-observable world, it's like a simple map. You're just mapping down off of a global prop, you've got some tree of state and you're just mapping off... What was that like? A model hook and you're just mapping down off of that? Wherever that state lives, you're mapping to it and now you kind of slicing off your little garden hose off of the firehose. But still one huge -- ALEX: I've tried to apply observables to this problem. I don't think I've never seen the observable analogue of is this idea of dependency and injection. To model something as a stream that transforms over time, that's proven to be very useful but to sort of say, "I am an observable that expects these objects given to me," I'm not really sure what that API would look. CHARLES: I would say, just as a straw man perhaps, you have this dependency that it's a well-known location. It's a well-known name. With dependency [inaudible] in classic, it's like, "I depend on the off service. This thing called 'service:off' or whatever. Imagine that you have some pool of state and there's some key called ‘service.off' there and as long as I'm just basically basing my stream, the first thing I do is map off of this and maybe map off of another key and then combined those into a single stream, then I can be sure that I have those things at all times. If they change, my mapping function or my transformation function is going to get evaluated again. Does that make sense? ALEX: Yes, I think we should [inaudible] C without code or something. CHARLES: And maybe I'm thinking about it wrongheadedly but that would be a simple mechanism. ALEX: Could you run by me one more time --? CHARLES: Yeah. Let's say that we've got some authentication service that you want to depend on like you want to inject on it. You want to inject that dependency so why can't you base your stream off of that key? You have observable map, for example. The list of transformations that you would have to do to peel off multiple keys, I'm sure you could write helpers for it. But basically, probably if you're going to be wanting to inject multiple dependencies will -- ALEX: The problem is this. Basically, if you want to write your resolved observable, if this thing based on observables, remember that there is no this in a route because of the transactional reasons of what we've talked about earlier, what are you getting that from? You need to have something passed into you, to be like 'context.get observable blah.' CHARLES: I would just assume that it's implicit. I was thinking a bit basically, the simplest case would just be an observable that was basically taken off of the entire global state or whatever of the router or what have you. The way the redux-observable works is every single epic is what they call them is just a transformation on the global stream. Usually, the first thing that happens is they map down to the local context so the -- ALEX: Like a path? CHARLES: They have a helper like action of type, blah. You only see a subset of the actions that get maps to the Redux store. I think it's redux independent but at least in theory, every single epic is basically going off of the entire global state but the first in reality, what the first thing that happens is you're like, "I am only interested in this subset of the state," so you do a map off of the global state down to your local scope and then you work from there. In fact if you had the convention around that, you could even make that part implicit. It's like I return an observable that it's only seeing the stream of local states. ALEX: That makes sense if there's sort of canonical state of the world but what you're doing when you're transitioning into a route is trying to feel out another state in an asynchronous manner. Redux is the action causes state to change, now the state is this. But the action for type thing, I think that makes sense if you are subscribing to the world global action on this one store when you're constructing this new tentative, may not actually become the store, you're depending on values. What we need in our API is something that depends on values that are from a tentative store. CHARLES: It's similar so in redux-observable, you're mapping actions to actions and you're not necessarily mapping actions here. You want to get state into the equation. ALEX: Yeah and it's so almost observables. It's just this twist of transaction dependency injection. It sounds really over-engineered but the thing is it exists in Ember today and if it exists in a less siloed way, I would certainly benefit on it. I think everyone else would too. CHARLES: Okay. With that hand wave... ALEX: Oh, I didn't mean for that to come as a hand wave. CHARLES: No, no, no. I'm kidding because I think we actually have a lot more to talk about here and we're running out of time. One of the thing that I want to ask is, talking about redux-observable, talking about redux and stuff, have you given any thought as to what this might look as a library that everybody could use? ALEX: I basically have something that's using Ember CLI only because it's so easy to just use it as a sketch pad and get test passing but everything I'm building so far is just ES6 class syntax that can be transpiled in it to whatever. I'm actually realizing, there's a lot of overlap between some of the primitives that are involved and Glimmer so it may or may not have a pass that uses references for tracking when things change until no one to invalidate and refire these async hooks. But either way, I'm going to make sure it lives in the JS usable world and not just Ember's special object model end. CHARLES: Right. Those interfaces are pretty narrow. The things that implement those interfaces are huge and complex but the way, at least I understand it, isn't the reference interfaces themselves -- ALEX: They're really simple, yeah. CHARLES: -- Really simple. It could almost be copied and pasted and not have much maintenance overhead in there. Here's a question and this is probably getting too far into the weeds. Can you not model a transaction as an observable? Essentially, with a flatMap, you would merge in some observable into the chain that was basically a transaction of all the other observables from which it is composed. ALEX: You know, a transaction as it builds up all the new state over time could be part of the main tree and if there is an active transition, then that's future potential state that the world might become and it could be modeled as a leg of the Redux state. I think you could theoretically do that. Definitely worth a try. I don't think I would benefit too much from doing it now and I think this could be a premature optimization but I think there would be just quite a bit of intermediate object collection to express that. I think theoretically it works but how it's going to physically map to Ember in the near future, it would be harder [inaudible] in a way. There's actually a lot of stuff that is very redux-y that again, a lot of Ember people don't maybe know about because it's internal but the way that Ember [inaudible], I think since Edward brought some of his learnings of Liquid Fire back to core Ember, there's this concept of outlet state, which describes -- I'm not an expert on it -- what's rendered where and then each outlet gets a chunk. Like you said, a little piece of the firehose or garden hose, pulled off the main thing so it can just focus on the one piece of state. Those are simple objects that produce this part of this transformation process. That's kind of redux-y in the way that everything just gets a new POJO and stuff changes but it's not strictly redux, obviously and probably won't become that just because it's already good enough on its own. CHARLES: Yeah. I think it's actually good at this point to be hand wavy because the most important thing is to be non-committal about the syntax, like you said because that's when the bikeshedding begins and now it's not the phase. The phase is to come to some agreement about what is that we would love to see. ALEX: Basically, the thing is this. I think people need to realize that Ember won the bet that the URL is an important thing to build apps around and if you have a state that's representable in URL, that state should go in the URL so you can send links around and not break the web and have an app that works that's built on half-assed routing. The only thing I'm proposing is going to make that go away. It's just that there is already this giant world of stuff that's not expressible in Ember today because it is driven by state. If you make that as easy to express and as upfront to express, I think you can have shared conventions versus what everyone is building these apps that I have to do, which is to make a sort of separate router of state-aware stuff and not have to make those two things agree with each other because it's really hard. CHARLES: Right. At that point, you're writing your own framework. Maybe this is the next big thing because I feel like Ember usually has the best stuff way, way, way, way before. Now, we're finally getting to a point where everybody seems to realize that having a CLI is absolutely critical to the developer's experience and most frameworks aren't taken seriously until they've achieved that. It was the same thing with a router back in the day. I'm wondering what that next thing is. ALEX: I don't know. I don't think this is going to be it. I just think it's a good progression. I think a way forward that progress is still a pretty legit central structure to build apps around and just would be welcomed. CHARLES: When are you going to be done? ALEX: About two or three days. I don't know. I think I'm basically going to be continuing to get feedback like the way that a lot of that original router stuff came back or it's just like constantly hit people with real examples, Ember twiddles, things are just like, "Oh, yeah. That thing. That's a cool pattern. That sucks in my app. I didn't realize that until I saw this example." These things that really teach people why this is necessary because that's going to get people's urge to be like, "Well, you could just do..." Oh, you can't because the thing that's hard to explain. It's going to be a lot of that regardless and I hope that will kick off in the next few weeks. CHARLES: And the focus of that is going to be the ember-rideshare application. ALEX: I think that's a good one. This is one that everyone's familiar with. CHARLES: Have you already kind of implemented in it, like this kind of Frankenstein-ish, like this is the kind of histrionics that you have to go through in order to implement the style of routing or the style of application using today's Ember? Or have you started to begin experimentation with these new concepts and try to build out better ways of doing it? ALEX: I'm not strictly extracting it from one app. It's sort of combined. Like I said, the few different apps that we had were an opportunity to be like, "This sporadic stuff is hard." The main route recognizer approach was an example to try different stack routing pattern. But the thing that sort of working on is drawing from three different apps and slightly different takes on it. Basically, I have something that is close to being testable in one of my main apps that will be a great chance to validate if all the stuff is as nice as I think it is going to be. CHARLES: Okay. If the people want to get in touch with you, to help to contribute to the conversation or just publicly guilt you into moving faster towards it, how would they get in touch with you? ALEX: I'm at @Machty on Twitter and GitHub and also, the Ember community Slack. I think I'm going to try to get people to talk about this on channel called Dev Dx Router where it's focused on development stuff all around the router. This is kind of funny because I'm talking about this thing that I've only had maybe, 12 people take a look at and comment on and begins these conversations. I think maybe some people are going to hear this and be like, "What are you talking about?" but if it gets people -- CHARLES: No, no, no. You know, the best conversations seemed to be organized around you, man. I'm just trying to think of some of the best development conversations that I've had in 2017 and you were definitely, I would say the one who fomented them. It starts with 12 people but then, if enough people take interest and be like, "Wow, yeah. Oh, man. I didn't even know that was a problem. This would be a cool way of doing it." They have a tendency to balloon and some fizzle out and some end up with real results. Anyway, I'm looking forward to it. ALEX: I appreciate it and likewise, you're definitely one of the best people to talk about this stuff with. CHARLES: Well, I hope other people will love listening to our conversation. With that, we'll head on out. Thank you everybody if you've made it this far. As always, you can get in touch with us at @TheFrontside on Twitter or just send an email to Contact@Frontside.io. We will talk to you next week.

Jay Phelps: @_jayphelps | jayphelps.com Show Notes: 01:25 - RxJS 10:09 - Observers 17:49 - Back Pressure 22:11 - Async Iterators and Generators 31:30 - Mapping Resources: The Observer Pattern Hot vs Cold Observables IxJS redux-observable Transcript: CHARLES: Hello everybody and welcome to The Frontside Podcast, Episode #84. My name is Charles Lowell, a developer here at the Frontside and your podcast host-in-training. With me today is Elrick Ryan. Hello, Elrick. ELRICK: Howdy. CHARLES: You and I have actually been on a roll lately, podcasting the hell out of these podcast. ELRICK: Yeah, I know. That is very true. CHARLES: It's been you and me but it's feeling great. It's good to have you on the show again. ELRICK: Yes, wonderful, man. Let's keep it rolling. CHARLES: All right. We will keep it rolling. Today, we are going to be talking about redux-observable and to help us understand and plumb this topic, we have someone who's very qualified to talk about it. Mr Jay Phelps, who in addition to having been the co-creator of redux-observable, also is on the core team of RxJS, which is a fascinating library on which it's based for many years and is currently a senior software engineer at Netflix. Welcome to the podcast, Jay. JAY: Hey! Good morning, everyone. I guess it's not probably morning to the people listening but good morning to you all. Thanks for having me. CHARLES: I'm excited about it. I know that kind of starting with the fundamentals, RxJS is something that was on my radar for a few years and it definitely [inaudible] once we started using redux-observable but the whole concept, I often feel like the world kind of is turned upside down when I'm working with observables, when I'm working with RxJS and I'm curious, how did you come to be a part of that project and what are the things that you use it to solve? Why did the solutions that you generated shake out that way? JAY: Sure. I actually was not introduced to Rx until I started working at Netflix. Netflix does have a fairly solid reputation for their usage of Rx, not just in the JavaScript world but also in the server world. Netflix wrote the original implementation of RxJava and it's used heavily on our backend systems. CHARLES: For some reason, I had this impression and maybe I'm mistaken that Rx originally came out of Microsoft. JAY: Let me continue with the story. It's confusing and I can actually take a step back and clarify that point in particular. Rx itself was originally came from Rx.net, which was indeed created by Microsoft many, many, many, many years ago. I don't know the exact date. I think it was at least 10 years ago. They, at the same time created or about the same time, Matt Podwysocki who was working at Microsoft and still is working at Microsoft, created Rx.net and RxJS. Then many years had passed and originally, it wasn't as popular as it got in the coming years. After several years, some employees from Microsoft ended up coming to Netflix. Jafar Husain is one of those employees. He came from Microsoft to Netflix and he brought that Rx knowledge and that advocacy. Rx is very ingrained in the Netflix culture and is used a lot by various teams for various purposes. Then when I joined Netflix and I got really exposed to it. One of my coworkers at the time, Ben Lesh was asked by several people at Netflix to consider and look into rewriting RxJS. At the time, the version was RxJS 2.0 and while it was great, we had some specific requirements for our website and some of our other applications that we were hoping for a better performance, smaller bundle size and better debugability and -- CHARLES: Also, when I first evaluated it many years ago, it felt very much like a port from another language, in another culture as opposed to something that from the ground up, considered as a JavaScript library. Is that a fair statement? JAY: Yes, somewhat. Definitely, there were more considerations this time around when it was rewritten and originally, it was going to be Version 3 but the rewriting process took quite a while as these things usually do. By the time we got a version out, it was Version 5. We started when RxJS was at Version 2 but it already released Version 3 and Version 4 by the time it released for the new version like a rewrite had been able to get out. When I say a rewrite, I mean like from scratch rewrite. Matt Podwysocki who was the maintainer, almost the sole maintainer of the previous version, also is now on the core team of the new version of RxJS and has been instrumental in pushing back forward as well, he has far more experience with this than either Ben are I so he's been invaluable. Sometimes, we'll think to make those decisions. We'll be like, "Why was this decision made? Was it made because of .net?" and we'll just assume that and we'll want to change it but Matt has the history involved in that. He knows why things were changed the way they were. For example, we changed one of the operators, flatMap to mergeMap. We know somewhat we go at least, I don't want to speak for the entire team but I regret that decision. Depending on the day that I've been talking to Ben, I could convince him to regret that decision as well. But we thought that mergeMap would make more sense and that very few people in practice would have heard of the word flatMap before and had experience with that so -- CHARLES: I have to say both of the terms coming at it were pretty opaque. I think there was a bout of equivalence in opacity. JAY: Yeah, good to know. That's just an example. I don't want to stick too much on that topic. Maybe someday we'll go back to flatMap and the flatMap still exists. If you're a purist, you can use it. Ben was the primary person who was working on this. He wasn't working on it full time but pretty close to full time to get that initial version out. Even though I used it, my involvement with it was fairly low at that point and then my involvement after it was released got increased. I found more time and started to get more involved, particularly there wasn't a lot of code to write. I have some PRs and stuff like that but particularly on the planning and the issue triaging and PRs and stuff like that, which is a pain in the butt. It's just massive. Particularly, around the same time that the rewrite was getting finished that Angular decided, "You know what? We're going to bet on Rx. We're going to depend heavily on it," so you really can't write Angular without writing some Rx these days. You can get away with not knowing Rx very well. You could just call subscribe and then just do a bunch of imperative stuff but for the most part, the paved path is observables in so many fashions. Now, there's this ngrx. I don't know if you have any exposure to the Angular community. I have quite a bit. CHARLES: I haven't recently. Certainly, since that kind of heavy investment in Rx, I haven't been exposed to it. JAY: I think that was your question, right? CHARLES: Yeah. It sounds like there's a Java implementation that gave rise or a .net implementation or Java implementation gave rise to a JavaScript implementation and that's the one that you got involved with but it suggests very strongly that Rx is an idea and it's played out in a bunch of different languages but really, there is a shift or it's an idea about the way you think about your programs. It's clearly been compelling to you so what is that idea and what is that shift from the way we normally think about things? JAY: The idea was realized very early on -- CHARLES: Yeah, both 10 years ago, right. JAY: Yeah, exactly. They dubbed it their reactive extensions, which is what Rx stands for. Pretty much, name a language and it's been ported to that. There's RxSwift, which is super popular. There's even things like RxCpp and stuff which if you look at it, it's awkward. It seems like we got less language in the world for doing this sort of thing. I actually like C++ in a lot of ways but it was awkward stuffing that stuff in there. It's a really popular pattern and the idea is just basically going all in on the observer pattern, saying that like, "Most people are building things in which you want to be pushed information." You want to be pushed events and the data should stream to you. Modeling most problems in the world as a stream, once you get over the initial barrier of getting away from your normal historical way of looking at things and you look at everything as a stream, it comes very natural because you can actually model literally anything as a stream because it could be a stream of one, it could be a stream of nothing, it could be a stream of infinite number of events or it could be a stream of 10. You can model anything as a stream. Once you start thinking about that, it just becomes very natural and particularly on the UI side of things, I think there's been a lot of success in RxJava stuff at Netflix but RxJava is also used in Rx.net for client-side stuff as well, for mobile development. CHARLES: I remember when I first was introduced to it, I think there was a lot of confusion for me around an observer in the context of Rx and an observer in the context of classical MVC. One particular manifestation of the MVC architecture where you have these kind of mutable objects and you're observing their properties. Like key value observation, which factors heavily into certain UI frameworks. Backbone is one that comes to mind or if you're familiar with Java, basically the JavaBeans, like the property model listeners. I kind of had that conception of what an observer was versus Rx has a very, very different take on observable things. Do you think you could maybe show where they're different? JAY: You can get the normal classic observer pattern using Rx, using a subject basically. But there's a subject class, which you're not going to use super often but there are certain cases where it makes sense. Also, it depends on what libraries. It is used more often in the Angular world because you want to get a stream of clicks or something like that but -- CHARLES: So what would be the subject in that case? JAY: The subject in that case would be you're going to pipe, you're going to emit. Every time they click on something, you're going to 'next' something into that subject, like 'next the event' into the subject. Basically, a subject is a really great way to go from some imperative world to the observable world. Without having to write all sorts of custom glue, you can just basically say, "I've got subject. Any time I 'next' into it, just notify anyone who's listening to this." A subject is hot observable and that's the closest to the typical observer pattern because Rx, it's usually like observables and are usually lazy or cold. That's also what people call it. In the normal observer pattern, there is not necessarily any concept of laziness like you listen to something and that producer is already producing usually. CHARLES: Right. You hit on that and I think that was something that was surprising and kind of delightful when I first started using observables is to realize that they were lazy. Let me make sure I understand it. What was cool is like I was going through some of the demos and I had this observable and is part of, forgive my terminology but when you create a new observer, you pass the function that will get called every time something subscribes to it, right? JAY: When you create a new observer it passes a function -- CHARLES: When you create a new observable, you pass a function that gets called when an observer subscribes and the thing that you can pass is the thing that you can call next on, right? JAY: That's exactly right. When it's a lazy observable, that only get called... Actually, you know, continue. You had a point. CHARLES: I was going to say what was a surprising and cool for me was that every single thing that subscribe to that observable got its own history of that observable from the very beginning. It got its own function invocation so the first example I did, I wanted to iterate over an array and just send 10 items to the observer. Then when you subscribe, you're starting from one every single time: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10. It's not like I subscribe and one gets five of the elements, then I subscribe to another one and that one gets the next five or those two get the next five together. It's like they each get their own version of that observable. JAY: That's exactly right and then that's the main difference between -- CHARLES: -- The whole thing, yeah. JAY: Yeah. This is still observer pattern because the observer pattern, at least to my knowledge, I'm not an expert on this or anything but my understanding is that the observer pattern does not necessarily dictate hot versus cold, per se but traditionally I would say, people interpret it as typically the hot type of thing or also the multicast, if you want to be like these cool buzzwords. CHARLES: Right. What you're thinking of it is when you've got this mutable object, you just dive in wherever it is at that moment and you're now observing events from it. JAY: And that's traditionally because the observer pattern was mostly created for that model view controller type of pattern as well. You can think of like a stream of clicks: the users clicking on things where a stream of keyboard events. That is inherently a hot or multicast stream. You can't tell the user, "Start clicking," or, "Start typing. Don't type." You can tell them that but it's futile. The point being is that you can't control the stream. It pushes data to you and it's already pushing, whether you're subscribing or not, the data is flowing so the traditional observer pattern works really well for that. Then the observables, it's usually lazy, Now again, with subjects and with multicasting, you can get the same behavior. You can get that observable that is hot and shares its subscriptions so that when multiple people subscribe to it, they all get the exact same underlying subscription that is already producing data. CHARLES: I see. That makes sense with to call of multicast because fundamentally, there's really only one observer. There's only one stream and you just happened to be entering in at a certain point. JAY: Right. Then the other one is called unicast but not everyone calls it that but that's usually what it's called. CHARLES: There's a lot of terminology. With subjects, is that just a fancy way of saying a hot observable? JAY: Not necessarily but almost always. If you include them together, you're not wrong 99% of the time. But as a quick drive by definition, it's totally fine. If you're teaching someone really all the things and then they want to fundamentally understand it, it's good to have some distinctions. A subject is hot and multicast but just because something is hot and multicast does not mean that it necessarily has a subject backing it. Maybe from the pattern perspective, you could call it 'the subject' or 'a subject' but in the Rx world, it's not necessarily a subject. CHARLES: Then for cold observables, what are good things that they model, like the execution of an algorithm or something? there's an instance of that algorithm, I'm going to add two numbers or I'm going to reduce this tree of numbers into a product or a sum or something like that, where you start from the beginning, there's a clear beginning, there's a clear end --? JAY: Right. The observables are really good at modeling side effects, for example. Things you want to do like make an Ajax call or read a file or something like that. In fact, reading from a file is actually not a great use case for observables just because you want to be able to control the back pressure. We can talk about that next if you want but -- CHARLES: Yeah. Back pressure is a concept. It's crazy to me and when people use it, I'm like, "How do you even do that?" You can say it and sounds good in practice but it sounds really, really complex. JAY: I think I can summarize it for you. The back pressure stuff is basically you have a data producer and you want to be able to control the rate in which it is producing so that you do not overwhelm the consumer. You can consider two servers. You've got Server A and it's sending events to Server B. If Server B can only process 10 events per second but Server A sends 100 events per second, what's going to happen? There's a deficit. There's a huge 90 events per second deficit and that means that the Server B is going to get more and more behind. Eventually, it will just fall over and run out of memory, CPU or whatever happens. Back pressure is basically just being able to control that somehow. There's lots of ways, there's several policies of back pressure. There's buffering, there's dropping and then there's the pull model where it's like, "I'm going to tell you when to give me the next event," or, "I'm going to tell you to give me six more. I can handle six more, give me six more," or, "Here's the next one." Those were -- CHARLES: Then polling would have to be combined with buffering or dropping still, right? JAY: Possibly but it becomes one of those like you use it just as a rarity. Because the problem with buffering is that it becomes usually unbounded and that means you can -- CHARLES: It's blowing up the producer, right? JAY: Well, you risk money out of memory, mostly. It just becomes completely unbounded. You can bound that buffer and then it becomes dropping. Then if it's dropping, that means it's lossy. In a lot of scenarios, it's important not to drop information. CHARLES: They're useful stuff. JAY: Right. At the same time, there are a lot where you can drop it. Like at Netflix, we have this data pipeline thing for these logs and I got a good talk on this actually. This data pipeline is called Mantis job platform where all this data flows through. It uses RxJava almost exclusively and we have a lot of different back pressure mechanisms but the one of the most popular is dropping data. If the consumer is being overwhelmed with the number of logs and events that are matching to it, we just start dropping them because they're logs. We want to keep them but we make a best effort and it's perfectly fine to drop them. In the observable world, it's usually going to be either buffer or drop. You're not going to usually be able to control the producer's rate. In RxJava, there is something called the flowable, which lets you control the rate by basically saying, "Give me N number. I want N number more." It's become more important on the server-side end of things but a normal observables -- CHARLES: That makes sense. JAY: Yeah and there's even a library that's brand new that Matt Podwysocki came out with called IxJS. Instead of RxJS, it's IxJS and it's iterator. It's for async iterator and regular iterators, which is good for that other end of the spectrum for back pressure. If you want to be able to control the producer, you're saying, "Give me the next five." That's what an async iterator is incredibly perfectly suited for. CHARLES: The async iterators are kind of like the logical inverse of the observables? JAY: That's exactly right. It's the dual, if you want to use the computer science term, I assume you knew that already and it was that was just a plant. CHARLES: Actually, I didn't terminology, the dual but it's like through the looking glass world, right? JAY: Yeah, that's correct. CHARLES: One hand you got Alice in our world and she's looking in the looking glass world. I'm not saying which one is which but one is observables and one is the async generators. When you would want to use async generators is when you want to have the consumer driving the entire stream. JAY: That's right. A lot of people, when they discover async iterators, they'll ask me, "Jay, why wouldn't I just always use that? It seems like they're more flexible of the two." The reason why is because you can't always control the producer. The fundamental example that I was giving earlier like mouse events or keyboard events, you cannot control what the user does so an async iterator would be a very poor primitive to model that because what happens if you decide to give me one keyboard event at a time and one per second. You're like, "Give me the next keyboard event." If you did that one per second and someone is typing on their keyboard, what do you do with all the events while you're waiting? You could buffer them but that's a back pressure problem. You have to choose and it's a very poor primitive model to that. CHARLES: Yeah, that would be a terrible user experience. The users want the consequences of their actions to be realized at the soonest possible point. JAY: That's exactly right. The other benefit of observables is a slight increase in performance because when you subscribe to an observable, it sets up the chain and then now only data needs to flow through that chain. Whereas an async iterator, every time you call next, you're going to get a brand new promise. If you used it for something very high volume, you might be able to see how now you've got a lot of excess allocations: CPU cycles being used, garbage collecting for each one of those promises. In Netflix, we got a lot of streams, which are hundreds of thousands of events per second. If you do that, those allocations of those promises can certainly add up. It's just not the most efficient primitives for that as well. CHARLES: I thought that async iterators used essentially continuations under the covers and not promises. JAY: Under the covers, I'm not sure how you would describe what -- CHARLES: I'm not super familiar -- JAY: It's basically an -- CHARLES: You're talking about like yield, right? JAY: You're talking of generators maybe? CHARLES: Uh-uh. JAY: Iterators and generators are although related because a generator is basically a factory for creating iterators. Does that make sense? CHARLES: Yeah. JAY: Every time you call that factory, you get a brand new iterator out of it. But then when you have an async generator, it's basically the exact same thing except for the iterator returns a promise. You're basically saying, "Give me the next thing." You call next on the iterator and the next will return a promise and that promise will resolve with a value. Why is that useful to standardize? Because you can have syntactic sugar like the [inaudible]. I don't know if you've seen that yet but it's like a loop primitive, where you can basically loop over an async iterator without needing to deal with the whole thing and all that stuff. The same way normal async/await is helpful. This await is helpful at the same time. CHARLES: It's way above my pay grade but maybe you're using both. I'm trying to think with async the way that works with promises, right? JAY: That's correct. CHARLES: So you got both the continuation and a promise. That's even more overhead than the promise because you've got to stuff away that whole call stack and the promise and yield to it. But anyhow... ELRICK: This is very interesting. RxJS is a very deep subject. What I'm interested to know is you have RxJS, you have observables and you have redux. Where's the union? How did that come to play that you guys came up and said, "You know what? We need to create redux-observable library?" JAY: Yeah, great question. It came about fairly organically and over many month period of time, actually. We bring Rx people as I mentioned and we were using redux at the time and we're still using redux but my team that I was on was using redux and we were using redux thunk and the thunks were getting incredibly out of hand. It was very hard for us to do things that we were used to doing that were very simple like debouncing. We were like, "This is really hard. It's just to do something so simple that in Rx, it's so simple." We try to stuff Rx into redux thunk and try to make conventions around that. It just didn't work out well and then we initially made something we called thunk-observables -- CHARLES: We really wish you would have stuck with the name even you didn't -- JAY: Yeah, so thunk-observable is probably what you can expect. You dispatch a thunk which returns an observable and that observable will be subscribed to and that was it. Basically, that was the primitive so that let us have our debouncing but in practice, we found that the thunk stuff cause way more confusion and then also did not let us do composition as much as we wanted to. We then extended thunk-observable thing to get receive a stream of actions so that you could compose the different thunk-observable together and then receive new actions to cancel things and stuff like that. We eventually just figured out that having basically the idea of thunk-observables but having of them be like almost static factories that are like process managers, that's the pattern that we ended up on today. That we coined calling them epics. Basically, an epic is a function, which receives as an argument, a stream of all of the actions. It gets an observable of every action that's going to be dispatched and it was a hot observable so that means that if you happen to subscribe to it later, you're not going to get all the actions from before. It doesn't replay them. It's just all actions from the time you subscribe and continuing. Then what that function is expected to return is a new stream of actions and that stream that it returns of actions gets subscribed to by the middleware under the hood and basically, it just calls stored dispatch on anything you emit. You can imagine the simplest epic would be a function that gets streamed of all the actions, it applies a filter operation on all of those and looks for a particular action and then based on that action, it performs some side effect and then when that side effect comes back, it emits a different action. Basically, to notify redux like, "Here's the results. Here's the change. I'm done," or what have you. let you just make arbitrary side effects but isolate them in a way that fits very naturally with your UI layer that you're using and with redux itself with the purity of the reducers. CHARLES: Right. For example doing an Ajax request, I think that's a good example. You get some action in, you're getting every single action that's dispatched to the store, the first thing you want to do is you want to filter that stream to only the actions that are user click the save button so I want to save off this form. To some action, that's my save action so then my job is to now map that eventually. Out the end of that stream, I want to have like the save either succeeded or the save failed. There's two forks on an Ajax request. The fundamental mapping that's happening is either from the user click save, whatever you want to call that action, I want to map that to the user if the save was successful or I want to map that to the save failed or rejected. The hard part then is executing those side effects and then putting them back into the stream. How would you do that then with redux-observable? JAY: If I'm following correctly, the typical almost all epics, not all of them but almost all, they're going to have a filter at the top of the epic, they're going to filter out looking for the action they want and then they'll have some sort of merging strategy operator like mergeMap, aka flatMap or they have something like switchMap. CHARLES: I feel like that's one of the harder concepts for me and I want to actually going to be a little selfish here and try and bounce my understanding off of a bit so you can be like, "No, your mental model is incorrect," but we'll find out how it's incorrect. With the mergeMap, is that a way of saying, "I can take other observable streams and inline them inside this one stream." That's the way I've been thinking about it. I can go out to the rest of the world, I can have some other stream and I can inserted into the processing chain of this other stream. In this case, I do a mergeMap of the actual fetch or the actual post but it's not a straight chain of implication like do a filter then I do a map then I do this. I've got to take this other observable, this thing, this other asynchronous process, which really represents another stream and I want to merge it into this one stream. Then once that's done, I get the result. now, once that happens outside the mergeMap, now the Ajax results whether it's a success or failure, the result of the request, now it becomes the next item in the stream, which then I have to map down to another redux action. JAY: That's right. The mergeMap operator has an alias and it also used to just be called flatMap so you can envision that you are creating an observable of observables, a higher order of observable. With mergeMap, you want to flatten that. You're saying that every time I get an event, I'm going to call this projection function and it's going to return a new observable, an inner observable and I want to merge each one of those into each other so that it becomes one stream. That means you can have concurrency. That means you can have multiple and simultaneous concurrent Ajax calls that can finish it arbitrary times. I think it's important then to contrast that with some of the other merging strategies like switchMaps or concatMap. With switchMap, as the name implies, you switch between observables so at any point in time, you can only have one observable subscribes to at a time. If a new event comes along and you call the projection function, it returns a new observable. If the previous observable has not yet completed, it will get unsubscribed to and anything it was doing gets cancelled. In the example you're talking about, if you've got an epic that goes and fetches your user model every time they click this button, let's say that they can click that button multiple times and you don't want to make 50,000 requests. The quintessential example is actually the auto suggest stuff. As you're typing key strokes, if the request is in processing and they type another keystroke, you don't want to wait for them the previous request to come back and process it. It's not only wasteful because you'll process the JSON and all that stuff. It can introduce bugs because it may come back out of order. It may come back actually after your new one comes back and that can cause all sorts of crazy weird bugs. That's what switchMap is really great for. I call it implicit cancellation. Ben doesn't like that because he's saying that in a switchMap, you are being explicit about it but you're not calling unsubscribe on it yourself, which is why I call it implicit. It happens automatically. There will never a new event pipes through there. Then there's the third one which I don't -- CHARLES: So this switchMap ever make sense outside the context of hot observables? JAY: I would say that it makes more sense usually on hot observables but there are certainly cases like let's say that you've got a web socket observable and every time you get an event, based on that event, it make some other request and that other request may or may not take a really long time. But if you get another thing back from the web socket, you want to cancel the previous one. That's somewhat not a great example as well because sometimes people use multiplexing for the web socket so that it becomes multicast. But the point being is that there are definitely times where you will use merge, switch or concat. That concatMap being the third one where as you might imagine, you are concating the observables, you line them up. If I get a new event, I'm going to call my projection function, create that new observable but I'm not going to subscribe to it. I'm going to keep it but I'm not going to subscribe to it and tell the previous observable I was subscribing to has completed. In a way, you're buffering the observables and because you're offering them, you can get in trouble where you end up buffering infinite observables. Let's say the first one never completes for some reason because of a bug, you may infinitely buffer them. ConcatMap is not used that often. I'd say it's very rarely used. Its use mainly when you don't want lossy behavior. You want, at most wants semantics like you're only doing one per time. CHARLES: The difference between mergeMap and concatMap is what? JAY: It's you're not going to do concurrent. That's probably the best way to explain it. You're going to do that in sequence instead of concurrently. CHARLES: I see. Maybe an example would be like file uploads or something. You probably want to do your file uploads in parallel but let's say, you want to conserve bandwidth or you're working where you've got a browser that only supports 10 connections or five upload connections but when someone selects 30 files, you don't want to just drop those files. You want to be uploading 10 concurrently but as soon as one finishes off of that 10, you want to start up another one. JAY: That's absolutely right. Another example would be like you are going to hit some API and it could be used as one mechanism to kind of throttle yourself. If you want to guarantee that you're only connecting at most once to this particular end point at a time, no matter what the upstream tells you to do. CHARLES: Right. I could see that. That makes sense. JAY: I can tell that you're a little bit struggling to see the use cases, which is totally normal. ConcatMap is not used very often. It's for of those fundamental operations, which primarily why it's included is it's non-trivial to implement yourself. CHARLES: Are there any other mapping operators that we should know about? JAY: Those are the three primary ones. There's a couple of other like boutique variations but they're all variations on the exact same thing. CHARLES: Right. ELRICK: Do we have like mergeMap, switchMap and concatMap that are specific to management of observable and that's specific to redux-observable. There's people out there using something like redux-saga or they're trying to compare and contrast these two libraries. How do these things contrasts? Because I don't think you would use either of these mapping operations inside of a saga. Is that correct? JAY: I just want to make a minor clarification. The Rx stuff and these operators that we've been talking about: mergeMaps, switchMap and concatMap, they are not actually related to redux-observable really in any way. The only thing they're related is that you just might happen to use them. Redux-observable is actually a very tiny library and it defers pretty much everything to just normal idiomatic RxJS code and that's really the biggest pro in comparing it to say redux-saga. Redux-saga came significantly over a year before redux-observable and without a doubt, were influenced. The pattern is very, very, very similar but there are some just fundamental differences and one of those is that difference. The most obvious thing is if you already know RxJS very well, without a doubt you're going to want to use redux-observable. I could tell you that, it's not possible but it's very unlikely you'd choose redux-saga over redux-observable if you're already a really big Rx fan because you basically know how to do it. You just have to think that the items you're modeling, the events you're modeling are actions, which just happens to be a convention really. There are some other differences though between redux-observable and redux-saga and the biggest difference being that redux-saga takes this effects as data approach, which is like Haskell if you're familiar with that. In a lot of ways, it's identical with the IO monad but it basically just means that you're not actually performing the side effects yourself right then and there when you call their operators. There are special utility functions, instead there's like an engine or a middleware underneath that performs the side effects on your behalf. You create a generator and that generator is the pattern that is called a saga is what they call it. That generator yields these objects, these effects objects. One of the effects objects might be to call some particular API to make an Ajax request so you're going to yield that object that basically describes the side effect you want to perform but doesn't actually do it itself. It's just an object. In the middleware will form it for you so why would you put that indirection. The indirection helps when you want to do things like testing. If you want to test it, you don't actually need to perform any of the side effects. You can just call next on the iterator that you get back from the generator, you call next on it, you will get the side effect object, the effect as data that you want to perform and you can just assert based on that. You don't need to do mocking in all of that stuff. You just have to assert that the effects that were yielded were the ones that you expected. Now, I don't like that approach personally because I actually use redux-saga quite a bit, not lately. It was like a year ago. You end up implementing almost all, not all but a lot of your business logic in your actual tests themselves. Your test become less about testing the behavior or the outcome of a particular thing and more about testing how it gets to that outcome and what steps it takes to get there. Some people think that's fine. For me personally, it felt like any time I made a change in my saga, I had to make the exact same change in my test even though the behavior of the saga did not change. The actual observable outside side effects did not change in any way but I may be refactored or renamed something. It felt very redundant and to me, felt brittle because I started to wonder who tests the test then. If the tests are in a lot of ways are reimplementation of the saga, how did I really test that the behavior of what I was trying to accomplish really was accomplished. I'm not an expert on it so certainly, I'm sure there are people who have patterns around this that can mitigate some of my concerns. But for me, I'm used to testing Rx and I'm used to Rx in particular so the pattern for me of using observables just made a lot more sense to me. Also, with either of these libraries, the learning curve is really steep. If you don't know redux-observable, let's say you don't know Rx, learning Rx just to use redux-observable is a pretty huge undertaking. I would usually recommend people to not do that. A lot of times these days is spent me helping people who are frustrated because they dug themselves in a hole by using all these new technologies. They'll pull in TypeScript and [inaudible] with every all of the cool new hotness and I don't blame them because they've been told that this is all the cool stuff. CHARLES: We got some great advice on that that you can have one vanity technology on every project and live it yourself. Indulge in that coolness, in that hotness and do something exciting but make sure you have one vanity technology and one only. Everything else, be very comfortable with and one is just crazy experimental like this is the coolest new thing and we're going to just drop it in. But if the rest of your chassis is solid, then you can support that crazy experimental engine that requires you to stand up on the front of it and feed gas in with your mouth like they did in Mad Max. JAY: And there are exceptions obviously but for the most part, I subscribe to that. I answer a lot of questions on Stack Overflow and redux-observable. I actually get sad and frustrated when I see people that it's very clear that they shouldn't be using this. Their use case is not complex and they haven't learned Rx yet. It's one thing if you're like, "I'm learning this and I know that I'm going to have pain." That's totally acceptable. There's plenty of times where it's like, "It's totally okay that there is no concrete deadline and I can take my time," but a lot of times and I would say, a majority of the time, you're slipping on the deadline and that's not a good thing. The people who are coming to me are just frustrated. They're like, "Why is this so complicated?" You're right, it is really complicated but its complication helps with some of the more complicated use cases. That's the irony behind both redux-saga and redux-observable is that they're both really complex for the trivial use cases like all I want to do is make an Ajax call. That's it. I don't want to cancel it, debounce it, I won't do anything. I just want to make an Ajax call. They are the biggest hammer you could possibly think of. Don't use them for that. CHARLES: The thing is if you're not sure, whether you need redux-observable, then you actually don't need redux-observable. JAY: I would say that usually that is the case, yes. The same with redux-saga. They're in the exact same league. It's basically, "Do you need complex side effect management?" With the redux-saga, the complications, even though generators have been around for a little while, a majority of people still are not familiar with them because they're just not used all that often. They're kind of mystifying. I would say, they're not super hard to learn. They're just alien. Then to add on top of that, the effect is data and you've got multiple curveballs. Same thing with redux-observable, they could never use observables. It's pretty alien. The reactive programming idea and model is pretty alien. I would advise, if you're not using redux, don't even consider redux-saga and redux-observable personally. If you're already are using redux and you think that you might need something like this, experiment with the primitives, try and see how well your team can pick up RxJS just by itself. Just learn some of the regular RxJS tutorials, don't even look at the redux-observable docs because it's not useful in any way when it comes to this. Just try and learn a little bit Rx and see does it click. Because some people is like, "I don't understand why everyone thinks it's complicated. It's easy." But a majority of people, it takes a while before they get that like Neo in The Matrix, I-know-kung-fu moment. CHARLES: All right. We're almost at time but I hope that that moment comes to everybody. I think we've certainly enjoyed a lot of success with it already and I think once you do get your head around the basic use cases and you know how to do an Ajax request, you know how to do just simple saves and gets and what have you, doing the trivial things becomes easy because you know which pathways to travel. But before we head out, is there anything that you've got coming up in the near future, any talks, appearances, meet ups, anything whether you or otherwise that people should be aware of? JAY: I would say on this topic, there's a new beta for RxJS with this new way of using operators. It's being dubbed lettable operators, like the 'let' keyword but it's not related to that. It's basically just a way of finally being able to import operators and to use normal tree shaking that people have asked for forever. Because the problem with observables is that they're prototype-based methods and you can't reliably tree-shake methods or prototypes. We've been trying to experiment with ways to have [inaudible] and the lettable operator stuff is interesting to check out. It's a stopgap measure until JavaScript has something like the pipeline operator, which just actually moved to stage one. It's a brand new operator. If you're familiar with a lot of functional programming stuff like F# and a lot of the functional programming language is actually have the pipeline operator, it'll make it so that you can basically have syntactic sugar to apply a function, basically to pipe the result of a function into the first argument of the next function, etcetera. You can pipe the argument and return values through a series of functions. If you do that without this syntactic sugar, you've got this massive nested function invocation, which is incredibly hard to read and hard to maintain. That's why the pipeline operator is so great. I would encourage people, that's in beta right now. I think it's 5.5 but it's in beta right now. I encourage people to check it out, find bugs and get feedback. Maybe this is completely off base and it's not the right direction that the team should be going but it's based on a lot of collaboration, particularly with the Angular community. They've been, in particular asking for this because they're pretty big and they've got to use the Clojure compiler and all sorts of things for trying to make their bundles incredibly small. For me personally, I don't have any Rx talks coming up. I've been pretty obsessed with web assembly here lately. I'm an armchair compiler nerd. I don't do compilers for a living but I have done them personally for a number of years so I'm obsessive with web assembly. I have number of talks in web assembly coming up but just nothing related to Rx at this point. CHARLES: That's totally okay. I'm actually, also have been obsessed with web assembly. JAY: Have you guys done a podcast on that yet? CHARLES: I don't know. ELRICK: No, not yet. CHARLES: I actually started out to write my own list compiler in web assembly and they got totally derailed on the list compilers. Actually I ended up switching tracks on the whole web assembly thing but I was really, really excited about it. Probably, it was about three months ago or something like that but I'm still excited about it. I just haven't been working on it actively so I'm very curious to hear about those talks. Let's post them on the show notes and who knows? We do a lunch and learn every Friday here and usually, it's one of us getting up there but sometimes, we'll just watch a talk. One of us has been wanting to watch. ELRICK: And you're always welcome to come back to any time and geek out with some web assembly. CHARLES: I'd say, we haven't podcast web assembly, you know? All right, you guys, we've been at this for another hour. Let's go. Everybody listening, strap your headphones on, we're going down for another hour. Changing the subjects: web assembly. It starts right now. ELRICK: It's going down. CHARLES: No, but we will have to have you on for that. Thank you so much for coming on and talking with us about observables, Rx, redux-observable but if folks want to continue the conversation with you, they can get in contact with you how? On Twitter, email? JAY: The best way is going to be Twitter probably. I'm at @_JayPhelps. Thank you guys very much for having me on today. It was a blast. I love talking about this stuff. ELRICK: Thank you. CHARLES: Thank you and for anybody out there, we can also be reached at @TheFrontside on Twitter or just drop us a line and Contact@Frontside.io and have a great week and we'll see you next.

Julie Moronuki: @argumatronic | argumatronic.com Show Notes: 00:57 - Julie's Unique Origin Story Into Programming 03:47 - Good Resources vs Bad Resources for Learning Haskell 11:18 - Areas to Look at Before Taking on Haskell and Functional Programming 15:56 - Terminology 17:50 - The Haskell Pyramid 25:51 - Learning Haskell Vocabulary 28:20 - Monoid and Functor 42:06 - Advice for Someone Who May Not Be Interested in Programming Resources: Haskell Programming From First Principles (Haskell Book) Natural Language Processing (NLP) Learn You a Haskell for Great Good! Programming in Haskell by Graham Hutton Haskell: The Craft of Functional Programming by Simon Thompson Real World Haskell by Bryan O'Sullivan, John Goerzen, and Don Stewart Introduction to Functional Programming Course with Eric Meijer The Joy of Haskell Haskell eXchange 2017 - A Monoid For All Seasons Transcript: CHARLES: Hello everybody and welcome to The Frontside Podcast, Episode 83. My name is Charles Lowell, a developer here at the Frontside and your podcast host-in-training. With me today on the podcast is Elrick also. Hello Elrick. ELRICK: Hello. How you doing? CHARLES: I'm doing well. I'm glad to have you on this one. I'm glad to be doing this podcast in general. We have someone on the podcast today who I've been following for, I guess probably about two years because she published a book that has been very, very helpful to me. It's one that I recommend to a lot of people. It is learning Haskell from first principles. With us on the show is Julie Moronuki, who is co-author of that book. Thank you so much, Julie for coming. JULIE: Yes, hi! Happy to be here. It's nice to finally get to talk to you. CHARLES: Yeah. One of the reasons I wanted to have you on the podcast was because I feel as though you have one of the most unique origin stories because of programming and entering in the tech world. Most of us are curious, we either come from video games or maybe we just start fiddling with the web browser. You enter the maze from the entrance that is like hidden from all, I would say. You went straight to writing a book on Haskell, is that --? JULIE: That is what happened. In 2014 on Twitter, I met my co-author, Chris Allen and he has been trying to figure out better ways to teach people Haskell because the on-ramping, I guess of people to Haskell can be quite difficult. The materials that exist are not always accessible and people felt like they need the advanced math degrees before they can write Haskell. He was trying to figure out better ways to introduce people to it. Since I was this person who's never programmed before -- I have no background -- and then he thought, "This will be a very different experience, trying to teach Haskell to her." Because I have a linguistics background and stuff he thought, "That would be interesting too and maybe, she'd be interested eventually in doing NLP." I said, I'm not -- CHARLES: What's that? Acronym alert. JULIE: Oh, yeah. Sorry. Natural Language Processing. I said, "You know, I've never done any programming and I don't play video games and I never have had any desire to learn computer programming. I don't think I'm going to like this. I don't think this is going to last but sure, I will try," and so I did a little bit. I read a little bit of 'Learn You a Haskell for Great Good.' I've read some other things. CHARLES: This was before you guys had the idea of actually writing a book. JULIE: Yes. He had the idea of turning some of his thoughts about teaching Haskell into a book and as he would explain things to me, like the questions I had about 'Learn You a Haskell,' I'd be like, "We should write this down," and he would say, "It's so hard to write it though. It's easy when I'm explaining it to you and it's so hard to write it." Initially, it started that I was helping him at things that he was teaching me and then as we got further into the book and I started reading a lot of other Haskell stuff on my own and figuring stuff out, I was writing more and more of it. Then we were kind of equal co-authors after not too long. That's how it happened. I really didn't think that I would stick with Haskell or with programming. I'm still sometimes I'm not sure about programming. I'm not sure about this whole making software thing. But Haskell is so interesting to me that I'm still here. CHARLES: That is fantastic and it's a great story. I'm curious, when you were doing the proto-research to learning Haskell, coming from really truly first principles and having no experience of programming, what made a good resource versus a bad resource? What are the things that you gravitated towards and say, "This is really instructive." What was the tone there? JULIE: One of the major problems ahead of most of the Haskell resources that exist is they assume that you've done programming before because nobody learns Haskell as a first language so they all assume that you have done some programming before. They would make references to things that if you were a programmer, you would know what they meant but I didn't. That was one of the hardest things for me. Even 'Learn You a Haskell' does that to some extent. CHARLES: What's an example of that? JULIE: I had learned a little bit about recursion from linguistics because that's a thing in human language so I really understood recursion but most of the Haskell resources explain it to you primarily in terms of, "This will be like your loops in other languages." I'm going to be like, "I don't know what a loop is. This isn't helpful for me." There are a lot of things that I didn't understand so when people talk about Haskell as being a pure functional language, neither pure nor functional necessarily, I didn't have anything to contrast them with so they didn't necessarily make sense to me as things that make Haskell different from other languages. I didn't know what imperative programming was and people would say, "In contrast to imperative programming, functional programming does this," and I'd be like, "Okay, but I don't understand what the imperative programming way is so this contrast isn't making any sense to me and same thing with purity." There were a lot of things I had to learn, in fact about mutable state because I didn't know anything about it. I had some understanding of how computer memory works but still some of the ways that people talk about it were not obvious to me. CHARLES: Do you find that seeking out that contrast actually wasn't helpful? Is it noise since at least at the beginning, it's something you'll never do. It's like saying, "Over in France, they wear these kind of socks." Since I'm going out into the street in front of my house, I don't really care. JULIE: Right. In the beginning, it was a lot of noise and I understand why they do that because they are making the assumption that everybody who is learning Haskell has come from some other programming language, probably an imperative one so I understand why that happens but in the beginning, it was very much noise for me. I noticed a lot of Haskell resources, one of the first things they tell you is that in Haskell you can't do 'x = x + 1'. I was like, "If I'm reading this like it's mathematics, why would I think I could do that." If you come from a different programming language, you might well think that you can do that but in Haskell, we can't so making that contrast, when I didn't have that background was really just confusing for me. Now, because I teach people and most of them do have some background in an imperative language, understanding the contrast is more helpful to me but in the beginning it was just confusing and noise. When we wrote Haskell book, we tried not to make those kinds of references and like, "Let's assume that everybody is just like Julie, doesn't know a different programming language that we can contrast it with and let's try to write a book like that." CHARLES: Right. I think that's a key insight because some people would say there's a lot missing or that difference might stand out. Now, that you pointed out, I can see it but I don't think I noticed it while I was reading it. But one of the things that I like is because I also tried to learn Haskell through 'Learn You a Haskell,' and I didn't find it very helpful. I found it entertaining and it's not a knock against the authors. Some of the sketches were really cute but it was still more explaining... I don't know. It was explaining more of the how, than the why, if that makes any sense where I felt as though in your book, there were a lot more analogies to actual human experiences, using the visceral language saying, "A mono is something you can mash together or squeezed together." That really connected for me. Whereas, explaining it in terms of concatenation and laws and stuff like that. Those things seem cited to the secondary resources to the primary resource. JULIE: Yeah. I think that's kind of helpful for me too. There are different Haskell books that have, I think different things about them that are good. I forget the name of the book but Graham Hutton's book, the way he talks about recursion was really helpful to me. The way he explains recursion and of course, folds but folds are things that he's known for so those parts of that book are helpful for me. But really the best book other than my own of course, for me is Simon Thompson's. I think it's called ‘The Craft of Functional Programming' and I think it does better at explaining things just in terms of Haskell. Real World Haskell, I guess is really good. It was harder for me because I hadn't been a programmer before. I think it's got so many practical exercises that -- CHARLES: Was that the O'Reilly book by Irish gentleman whose name eludes me? JULIE: Yes, Brian O'Sullivan. It makes more sense to me now but there were things in it that are sort of programmer things. Because I'd never made software before, that were really confusing for me. But Simon Thompson's, because his book does have exercises and they were ones that I could understand and do. They were fairly self-contained. My first experience actually in writing a program that does IO was from his book and I was just so thrilled. I was like, "I got it. I did it." That was really helpful book for me but I don't see people recommend that one as often but that was probably the best one for me. CHARLES: Yeah, it's always a balance because the Real World Haskell didn't really worked for me, almost because the examples were too pragmatic or too complex and I picked this up when I was 10 years into my programming career and I struggled to follow the JSON parser example, which is parsing JSON is something that I've actually done several times in multiple languages and I still struggled with it. JULIE: Whereas for me, I don't even know what JSON is. This is not something I've ever dealt with. I know what it is now sort of, but it's still not something that I deal with very much. I was just like, "What is this? I don't even know what to do here." It wasn't quite as helpful for me. I've heard a lot of people have success with that one but I think they don't share quite the same richness of programming experience with Brian O'Sullivan. I think it's a little bit more difficult. ELRICK: These are a lot of amazing resources that I wish I knew about when I try to learn Haskell. I took an online course with, I think it's like Eric Meijer and that class was very intense. Looking back, what would you say are some areas that someone should, either start to look into before they step into the Haskell world, being that you didn't come from a programming background but connecting to dots backwards now? What would you say are some areas that people can slowly ramp up into to get into Haskell and functional programming? JULIE: When I teach people Haskell, the people who have the easiest time are people who have been writing Scala for a while and they've moved over to the FP in Scala side. When I first started Haskell, I heard a lot of people make jokes about how Scala is a gateway drug to Haskell. I think there's actually might be so truth in that because I certainly have a lot of students that were Java programmers, then they got interested in Scala because maybe Scala is better for some things than Java and then they start moving more and more over to the FP in Scala side. Those are probably the students that have, I think the easiest time making the transition to Haskell that I've had anyway. But you know, I think even JavaScript, trying to write in a more functional style and there are some resources for that and really, there's a very good tutorial about monads that uses all the code examples in JavaScript. I think a lot of the concepts that you can start to approach them from other languages. Haskell is still going to be weird in a lot of ways and another thing that works for a lot of people is going to Elm. Elm is similar to Haskell but different. I think that that has worked also for a lot of people getting them into understanding more functional programming concepts but with the much easier... The word easy is so -- ELRICK: It's like a relative term like, "Oh, this is easy." JULIE: It is. CHARLES: Easy to say, right. ELRICK: That's what I thought when I step into learning Haskell and functional programming. I was like, "How bad could it be?" JULIE: Right. Learning Haskell can be very bad. I'm not going to kid around about that. It's a shame because I don't think that it needs to be that bad but the way it's presented oftentimes, for various reasons, I think why Haskell gets presented the way it does but I don't think it needs to have it like that. The designer of Elm, whose name I'm not going to try to pronounce because I don't know how you say his last name, he really made an effort to for example, the error messages in Haskell can be very intimidating. The situation there has improved since I started learning Haskell but they can be quite intimidating and he really made an effort to make very friendly error messages, very helpful error messages. I think that it shows and then it makes a difference for people who are learning. If you start with Elm and then you do want to see what Haskell or PureScript, which is also frontend language, mostly. It compose of JavaScript but it's very Haskell-like, then from Elm, let's see if we can get a little more hardcore Haskell. I think the transition to Haskell or PureScript is easier from there. I think it does help to move in the functional direction from whatever language you're in, if you do FP in Scala or try moving to more functional JavaScript or even Elm. Then Haskell will make it more sense from there or be a little easier to approach. CHARLES: Yeah, and I definitely think that for, at least from my perspective, I've been able to take a lot of those concepts that I've learned from Haskell and then apply them, even inside Vanilla JavaScript. There are things that have become indispensable like mapping and folding and they exist in JavaScript. You can reduce arrays, which is a similar to a fold and then you can map arrays but understanding that map, the key insight for me that I got from learning Haskell is that there's a whole class of values that you can map, not just arrays. The standard JavaScript object is essentially a Functor and will get a little bit to that because for people listening what that word even means and the meta around the fact that they're all these weird words and how do I go about something I want to ask you about. But the trees can be mapped and the objects can be mapped and all of the sudden, it's like this one concept that I use so much for lists, it's available on all these different data structures and it's get me thinking like, "What other data structures can I use this operation? What are the things are Functors that I'm working with?" Really, it's changed my perspective to think about the type of the data structure, in terms of the operations. JULIE: I'm in favor of keeping the terminology that we have but just explaining it much better. That's the approach that I take but it can be very hard, especially it was your first learning Haskell. I don't know if you've seen the Haskell pyramid but to get sort of productive where you can write programs in Haskell is not a very high bar. It feels like it is when you first start but it's not really very high bar but Haskell just keeps growing and growing and getting deeper and deeper so you're always approaching new libraries that you've never seen before and you feel then you've been learning Haskell all over again because they're written in a very different style of Haskell or they have even more terminology, even more kinds of Functors that you've never heard of before or something like that so you're always approaching these things over again. It can be a very intimidating feeling and it makes a lot of people very uncomfortable and I'd say, if you like Haskell and that does make you feel uncomfortable, then you don't actually need to do that because a lot of people write Haskell very happily every day in their jobs even and don't do that. They don't mess with some of the newer, super cool libraries that have all this funky terminology and stuff. Some of them don't mess with them at all. CHARLES: But certainly, there is some concepts that are core. I'm thinking of like applicative and Functor and all these things that I'm learning about and I'm curious to hear about your experience as you climb that pyramid. What is the pyramid entailed? First of all, I'd love to hear more about it because this is actually the first time I heard about the Haskell pyramid. JULIE: Say you understand monads, then you can write really a lot of Haskell programs. Probably at some point, you will need to understand monads transformers but if you just get to the point where you understand monads pretty decently, you can write a lot of software so after that, then learning more is maybe going to improve your Haskell, maybe let you write some things that you couldn't write before but a lot of it above, not that these things are necessarily in an hierarchical progression. We cover monad transformers in a fair bit of detail in Haskell book but if you get anything beyond what's in Haskell book, one of those things that some of them are very interesting, some of them can make you much more productive but some of them are also people do them for fun to explore the space and some people love them and some people hate them. Haskell lets you do a lot of things for fun and exploring mathematics in ways that are interesting and exciting and may influence and in fact, have influenced other languages like [inaudible] in PureScript but not really necessary for basic Haskell programming. A nice thing happened while we're writing Haskell book. I was writing, I think it's chapter six, which is about type classes. I was writing that chapter and at the same time, my co-author had started writing the Monoid chapter. The type classes chapter comes in chapter six and we introduce a lot of the basic type classes: num and eq and some of those in that chapter because I do think it's important. Type classes are very special thing about Haskell so I think it's important to, at least start coming to groups at them early. Some people disagree with me about that and think they can ignore them for much longer. But at any rate, it is where it is and I felt that that was important. Maybe the real motivation for type classes, really until we started writing the Monoid chapter so he started writing that while I was working on type classes chapter and he sent me the beginnings of the Monoid chapter to look at. At first I thought, "We've got addition and multiplication and list concatenation and this just doesn't seem interesting. What is this generalization of a Monoid that I'm supposed to get from these three things? And why bother making it a type class," because additional and multiplication are already in the num type class and then list concatenation is just for list so why make this into a type class and what's that motivation there. With eq, we want a quality -- CHARLES: Is that how you pronounce 'eq?' JULIE: That's how I pronounce it because 'equal' or equality. CHARLES: Okay, so this is a type class for doing what? Making sure to being able to compare two values on the same value. JULIE: Yes and it's a weird one because for most data types, you can have an eq instance and you want probably, in a lot of cases to have that but we don't want because function is a data type in Haskell so you don't want to have an eq instance for functions and that's why equality is not implemented generally for everything. That's why it's a type class so there's no instance for functions because that's not decidable. You can't decide if two functions are equal, generally. Some functions you can but in the general terms, for datatype, you can't. CHARLES: That's actually a pretty profound statement. Proof of which is left as an exercise for the listener. JULIE: We got to the Monoid and I was like, "What is the [inaudible]," or something. It turns out that there are Monoids everywhere. There's all kinds of things that you want to, either concatenate or make a product of. Then having this as a type class and thinking of it in terms of like, "We've got this abstraction. We've got this category. We've got this algebraic structure. Now, we can look for in all these other places," because once you've named the thing, then you can talk about it and think about it in a little bit of the different way. It's like, "Now, we've got this group of addition, multiplication, list concatenation." Now, we've got an abstraction of that and we can think, "Where else can I see this pattern?" and it turns out it's all over the place. For me, that was one of our thought like, "Type classes are actually really cool and powerful and interesting thing." For me, that was when it seemed like, "The terminology is worth it because, now I want to think about finding these algebraic structures and in all these other places." CHARLES: Right and like a Monoid, it could essentially be called, if you're using a Java interface, like 'mashable togetherable' or 'concatenatable' or something like that. But there's a kind of one-to-one correspondence but it is a vocabulary that just needs to be learned. JULIE: I don't know much about category theory or anything but the other cool thing about Monoid for me was that there are almost always two because there's almost always one that's destructive or additive or concatenative and there's almost always one that is conjunctive or a product or multiplicative. It's often across product that would be the zipless Monoid that exist in base and it's a cross product of the two lists. There's almost always two, whereas when you think of Monoids in the very abstract looking category theory, it doesn't matter if it's addition or multiplication. The operation doesn't matter, whether it's addition or multiplication or concatenation or cross product because you generalize the actual operation to the extent where what it's going to produce. It doesn't matter anymore. For me, I still think of Monoids in terms of like set theory or Boolean Algebra, then that's one of the things that I think is difficult with Haskell where people talk about Monoids in terms of category theory but I think that's not very helpful for the actual programmer who has to actually deal with the two different instances like sum and products or concatenation and zipping are going to actually act different in a program. CHARLES: Right, they're going to yield a different set of values. JULIE: Yes. CHARLES: Is there a baseline vocabulary? I kind of think of it like learning a new language, right? JULIE: Yep. CHARLES: When you're learning Haskell, you're not just learning a new language. You're literally learning a new language. I could go and I could learn Japanese but it's going to be a struggle at some point. People say certain languages are hard and certain language are easy. I don't generally subscribe to that. I think that most of it is just going about and living in a place where they speak this language and you'll absorb it and it's the decision to go and live there -- that's kind of the primary one. But let's say, you're a foreigner and you're travelling to this country called Haskell that's got this strange language. Like other human languages, it's just got different names associated with different concepts and some of the concepts might even just be unique to that country. Just like when you're travelling and acquiring a human language, there's a certain level of vocabulary that you need to achieve before you can do things like buy groceries and be able to transact financial exchanges or have a conversation about the weather. What are the kind of the levels of vocabulary that you need to acquire to be operational in Haskell or I would say, even in functional programming because now that I've been exposed to this, I see it in Clojure. I actually see people doing this JavaScript and in Erlang, in Elixir and what have you. JULIE: Yeah, I don't really know how to answer this question. How to buy groceries in Haskell? CHARLES: Let me let scale that down because I had this horrible tendency to spend five minutes asking what I say is going to be single question but it's actually like 30. Let's take down the scope. When you were learning this vocabulary, at what point did you feel like you're really gaining traction? We're you really starting to connect the dots? JULIE: For me, I think when I got through Functor. It was when I felt like -- CHARLES: Functor and what comes before Functor? JULIE: Monoid. I think once you understand Monoid and Functor, then a lot of other concepts in Haskell will start falling into place because this is not obvious to everyone but I think once you really understand Monoid and once you really understand Functor, then applicatives are monoidal Functors and that's not obvious to everyone. Like I said, it's not obvious at first certainly, and monads have characteristics of both Monoid and Functor as well. Then you start saying, "There's all these other Functors. There's profunctors and bifunctors. I think once you really understand Monoid and Functor, a lot of the rest of Haskell starts falling into place and then type classes like alternative. Alternative is another kind of Monoid. We have all these other names that if you can see the general pattern of Monoids and Functors, I think to me anyway, a lot of it then just started falling into place. Applicatives to me seemed, I don't want to say obvious or simple but in traverse, it's same sort of thing so we have these other names for it -- traversable -- and I was like, "Why was it called traverse. I don't understand this word at all." But once I saw the type signature and what actually happens with what the function traverse does, I was like, "Okay, I see what's happening here." For me, those were the two big hills. Once I got through Monoid and Functor and really understood them well, then a lot of other stuff just come and fell into place for me. ELRICK: This is really interesting. How was a Monoid explained to you when you were first starting to learn Haskell? Then now, how do you explain what a Monoid is to someone that's learning Haskell? JULIE: When Monoid was first explained to me, it was the pattern of there's addition and multiplication and list concatenation so it generalize out that pattern and that was really hard for me to understand at first because list concatenation and addition are similar but multiplication is different. I was like, "What do these three things have in common?" What they have in common is that they take two values of a certain type and return another value of that type and that's the type signature of the main function, that's in the Monoid type class. But that doesn't really tell you very much. A lot of functions could do that, in theory at least. How you combine them is really what's interesting about Monoid and also what makes concatenation and addition different from multiplication. Fortunately in college, I had had a fair bit of exposure to Boolean Algebra so figuring out that like, "There's actually two basic genres or varieties of Monoid and they are disjunctive or additive or they are conjunctive or multiplicative," and figuring that out, to me I always think that Monoid should really be, maybe two different type classes, one for the additive Monoid like list concatenation and addition and things where you are adding two things like a set union. Then conjunctive, which would be this intersections or multiplication or cross products. I always think there's maybe should be two different type classes but there's not a good way to do that really in Haskell. Instead, we have this one type class and then we do this ugly business of wrapping them in different type names. CHARLES: Is that why you'll have a constructor for some so it's just a wrapper for an integer? JULIE: Yeah. CHARLES: I don't know if that's so bad. JULIE: I don't like it but -- CHARLES: Yeah. You know what? You do a lot more than I do so I'm going to take your word for it. JULIE: Yeah, that's exactly why. Sum and product are the wrappers for integers because integer doesn't have a Monoid. It has two Monoids over it. CHARLES: I see. There's lots of ways to combine integers. JULIE: Yeah and those are the two basic ones. Then because Monoids also have an identity so with semi-groups, then you get even more semi-groups for integers because you get max and min, because they don't have an identity so there's semi-groups. CHARLES: There's always risk getting down into the weeds with the vocabulary but I think that there's a message here because your answer to the question is really, "When I understood Monoid and I understood Functor," from that point on, the overhead that you had to expend to get other things was lower than the overhead that you had to expend to get those initial two things. For anyone listening, Monoid and Functor are probably opaque terms. You have no idea what the hell they mean. We've been talking about in things like that a little bit but then it's okay because they're a finite set of opaque terms and they're very achievable and once you can achieve those, then you've done 90% of the work and now, you're just combining them into interesting and novel ways. JULIE: Yes. I will say it that a lot of people do tell us about Haskell book that applicative is actually the hardest chapter in the book, not monad but applicative. CHARLES: Really? JULIE: Yeah. A lot of people do tell us that. Because that's the first time that you've taken the concept of Monoid and the concept of Functor and combining them into a new thing so then, once you've done that with applicative, then after that, really it's all downhill. CHARLES: Right. It seems like there's a couple of key insights. As you're climbing that hill, I like that analogy is like one, just understanding that there things like type classes so you've through attacking Monoid and through attacking Functor, you realize, "There is such a thing." By recognizing there is such a thing as a Functor, you recognize that there is the potential for other type classes like it. Then through combining it with Monoid, to get applicative, you can see, "I can actually compose these things into new instances of those things," and then that's either the crest of the hill or the Pandora's box, depending on which way you look at it. I think there's a hopeful message in there that if you can invest the time to learn these opaque terms and making them transparent to you, you can really, really, really lean heavily on that knowledge in going forward. JULIE: Yeah. I'm writing a new book now called 'The Joy of Haskell.' The idea of The Joy of Haskell is meant to be an intermediate book. For people who already know some Haskell but we want to make words like Functor more general, like in Haskell book we really focused on the type class called Functor when it's actually a concept from mathematics or actually originally from linguistics oddly enough but we really focused on the type class in there, rather than trying to explain what a Functors are generally. In the new book, in The Joy of Haskell, we're going to try to take a lot of these terms like Monoids and Functors and catamorphisms and all these other words that Haskell has used all the time and try to explain them generally. Then also give examples like interesting uses from different libraries and stuff like that. It'll service both, hopefully a guide to the vocabulary of the Haskell ecosystem and also some documentation and examples for libraries and things like that that are useful because these things do have uses. They do get used in interesting and exciting or terrifying -- maybe those are related -- ways. That's the goal of the new book is to try to make a guide to all of this vocabulary that Haskell use all the time. We're trying to do that. How do I explain Monoids, you asked. You've got two values of whatever type. It doesn't matter the type and in general, there would be two ways you can think of to combine them, either making a sum or a union of all the values in them or making some product of those values, if they contain multiple values or even if they only contain one. That's how I explain them now. I'm not certain that addition and multiplication are actually the best ways to start with that because addition and multiplication don't act quite like set union and intersection do. I'm actually thinking of them in terms of and this is how I explain monoids to the people now, I start from set theory and that sounds really heavy but it doesn't have to be because I think a lot of things about sets are -- CHARLES: They're very intuitive, especially if you have visuals. JULIE: They're very intuitive, for people to think about. Yes, exactly. I explain Monoid now more in terms of set union and intersection. I'm actually giving a talk in October. It's coming up in just a couple weeks at Haskell eXchange in October 12th and 13th in London and I'm giving a talk there called 'A Monoid For All Seasons' and I'm going to try to explain the theoretic motivation for Monoids and try to explain them in those terms. Semi-group is a little bit different because lacks the identity but I'll try to explain the alternative type class and monad plus this really the same thing as alternative. These things are also just Monoids so we have these different names because it's a different type class alternative but it's really just another kind of Monoid. I'm giving that talk about set theory in Monoids in October, in a couple of weeks. People keep asking me on Twitter, "What's your obsession with Monoids," because my name on Twitter is Monoid Mary so I try to explain why I love them so much. CHARLES: Actually, it's an awesome point, which I've just gotten to experience it is what you see like, "Oh, there are these abstract things," you start searching for them. A lot of times, you'll uncover them and it'd be a real timesaver. There's the thrill of unearthing it in the first place and then when you could say, "Now that I've identified this thing as a Monoid, there's so much less that I have to write." There's like less work that I have to do. It's the same reason that we write frameworks for ourselves in software. It's like, "We love Ruby on Rails because of all the work we don't have to do." Now, you have to expend a lot of energy to work with it, using Rails an example but there's lots of software frameworks. It's like, "If you can find a good persistence framework or you can find a good thing for making a library for handling HTTP requests and responses, why would you write it all by hand in the first place?" I think the thing that's exciting for me as a developer is being able to see, "Monoid is a thing. Functor is a thing and I can now actually use this and I can use it almost as a looking glass to explore the world around me. When I see something in the landscape that just leaps out through that lens is another great one." I've been on a big kick lately but being able to say, "This is going to save me so much time because of the thoughts that I don't have to think and the code that I don't have to write." I think connecting it back to the pragmatic, I certainly have become really obsessed, maybe not about Monoids but having a type class large in your mind. JULIE: I think it's a really powerful thing. Sometimes that jargon is really useful. It's useful in a sense that it like compresses a bunch of information into a single word to remember. It's like teaching my eight-year old multiplication and we were talking about like, "It's like addition," and for us adult, I'll just go ahead say, "It's associative and commutative," but showing him that you can do those things and that addition is like that too and we're talking about that and he was so excited to learn that there's this word 'commutative' that encapsulates this idea for both concepts so he doesn't have to think like, "Addition does this thing. Multiplication does this thing." He doesn't have to remember both of those things, like he just remembers, "Commutative and they're both like this." It kind of compresses that information and what you have to remember and think about. Then it does make it easier to see that pattern in other things, then we can find commutativity in other things because now we have this pattern that we can look for and we got a name for it. We can talk about it and really, there's a lot of stuff like that in Haskell where we find some pattern that we find useful or we want to be able to talk about or easily translate to a bunch of different types, not translate is quite the right word but you know what I mean, I think. Then we give it a name and we make type class for it and then it's, "Now, we find it even more place for us." CHARLES: Right. It's about thinking less, right? JULIE: Yeah. CHARLES: That's a big misconception is that it's not about thinking more, it's about thinking less. JULIE: It really is. I think it's because there's so much kind of upfront work, where you have to learn all this new stuff upfront, then people mistake that for how much work we're always doing but in Haskell it's like, "We did all this work upfront and now we're now we're not going to think about these things anymore." ELRICK: That sounds like a good title for a book, 'Learn Haskell and you will think less," but it's true. When I struggled through that online class, I came out of that just being able to pick up any functional programming language and just hit the ground running. It is definitely a plus and you will think less. JULIE: Yeah, in the long term, I think that you do. Haskell is not a perfect language. There are things that probably can be improved. CHARLES: Now, before we go, I wanted to ask you, having had this very unique on-ramp into programming, which apparently you're still not convinced about. I'm curious what it would take to actually convince you but the real question that I have is there any advice that you have for someone who does not have a stereotypical background in programming who may not think that they would find programming interesting, who might have any number of roadblocks in terms of their own conceptions about the path forward. What advice would you have for them? JULIE: I am a bit joking when I say that I'm still not sure about writing software. I don't feel like I'm good at it and I think this is really the key. There are a bunch of domains in programming that I don't personally care about. I don't want to make web apps and I have nothing but respect and admiration for people who do. To me, it's very, very hard. CHARLES: Mostly because our tools aren't the same. JULIE: Yeah and there's just so many things outside your own program, there are just so many things that you have to think about and deal with because there's the network and there's other people's computers and they might be doing in other people software and what it might be doing. It is insane so for me it's very hard. There's a lot of domains of programming that I don't care about and when I thought about programming, that's the kind of thing I would think about. I certainly knew a lot of people who are web developers or the common programming jobs, I guess. Some of them just weren't that appealing to me and I'm not interested in making games or graphics so those are the kinds of things that I thought about for programming. There are things though that I am interested in doing. I'm very interested in natural language processing and I guess that's related to machine learning. I've recently taken up an interest in things like the raft protocol, the consensus protocol. Those kinds of things interest me a lot and there's a lot of the theory that interests me. I'm reading a dissertation right now about implementing a non-strict lambda calculus, which is what Haskell is. It's a non-strict lambda calculus and this guy's dissertations are theoretically implementing a non-strict lambda calculus. To me, the theoretical side is really interesting but then I am also interested in certain kinds of software. For some reason, I have developed quite an interest in making Twitter bots. I think that the advice I would give -- I'm rambling a little bit -- to people who think they're not interested in programming so why should they learn or whatever, is just find the thing that you are interested in and there's probably a way you can make software for that and maybe that will be the thing that will get you interested. It might not be Haskell, maybe you are interested in making web apps, in which case I would say go for Elm or PureScript, obviously because I like functional programs but Haskell might not be the best first language for you in that case but find the thing that you're interested in and there probably is a way to write software to do that. There's probably something in programming that will interest you. It's such a vast field. CHARLES: All right. I really, really like that answer. ELRICK: Yeah, that's a beautiful advice. Find your domain. CHARLES: Yeah, it's bigger than you think. JULIE: It's much bigger than you think. CHARLES: And there is a place for you. Thank you so much for coming on the show, Julie. I really, really, really enjoyed our conversation. JULIE: Yes, so did I. This is a lot of fun. CHARLES: Thank you. Now, before we go, I understand that you are going to be in London, was it roughly very, very soon, you said you were giving a talk. JULIE: Yes, the 12th and 13th of October. It will be recorded for people who can't get in. It will be recorded, I believe. CHARLES: You will be talking on 'A Monoid For All Seasons.' JULIE: Yes. CHARLES: And then you've also got The Joy of Haskell book, which you're hacking away right now, right? JULIE: Yes. CHARLES: With that, thank you so much for both of you. Thank you all for listening. What's a good place for people to reach out for you? JULIE: If they're on Twitter, I'm very active on Twitter so I'm @argumatronic on Twitter and my blog is also Argumatronic and that has more contact information. CHARLES: Fantastic. We'll link to those in the show notes. For everybody else, thank you for listening. You can get in contact with us at @TheFrontside on Twitter and Contact@Frontside.io over email. We'd love to hear from you. This just in, we're running a special. If you go to our website and enter the promo code 'ELRICK20,' you can get that 20% discount on your next custom developed web application. Go check that out. Take it easy, everybody. Bye-bye. JULIE: Bye-bye.

Chris Chuter: @Chris_Chuter Show Notes: 00:47 - Peeple: What is it? Why? 02:59 - Iterations and User Testing 13:32 - Complexity of Installation 17:26 - Device Integration 22:15 - Setup and Installation 25:35 - Laws and Building Codes 26:39 - Getting Started in this Space 31:29 - Ensuring Quality, Integration Testing, and Deployment Pipelines 33:18 - The Manufacturing Process Resources: If This Then That (IFTTT) Transcript: CHARLES: Hello, everybody and welcome to The Frontside Podcast, Episode 82. My name is Charles Lowell, a developer here at the Frontside and your podcast host-in-training. With me is Elrick Ryan. Hello Elrick. ELRICK: Hey, hello. CHARLES: And today, we are going to be continuing our series on the Internet of Things and we have someone on the podcast today who's going to talk to us about the Internet of Things. His name is Chris Chuter and he is the CEO, inventor and founder of Peeple. Hey, Chris. CHRIS: Hey. How is it going? CHARLES: It's gone well. Thanks for coming on the program. Peeple, what is it? Why don't you give us a quick overview of the product? Obviously it pertains to IoT, what is it and how did you become involved with it? Let's delve into that. CHRIS: Yes, sure. Let me give you the elevator short version first then we can dive deeper. Peeple is caller ID for your front door. The idea is when you get a phone call and you don't answer the phone, what happens? It goes to your voicemail. You know someone called you. But today, if someone comes to your house, you have no idea that they came unless you're there. This is the central problem that we solved with Peeple. It's a little device, a hardware device, an Internet of Things device that fits over the peephole in your door in the inside of your house. When someone knocks or doors open, you get a push notification on your phone. You can open up the phone and you can see a live view of your peephole. In a nutshell, Peeple is a smart peephole. CHARLES: Is it more for the case when you're not home at all or do you find the people use it for what you would traditionally use a peephole. CHRIS: It depends on the person. Now, my personal use case is for keeping track of wandering kids and that's actually inspiration for this invention. I have two boys and when one of my boys was three years old, he managed to open the door, walk out, go on to the street and walk down to the end of the street. Now, I live in Austin and I live right off the edge of a very busy street. Now, my kid didn't die or anything like that. It's not a really sad story but a neighbor brought my kid home and it was one of those moments as a parent where you're like, "Oh my God. I'm a terrible parent." But being an inventor and an engineer, I was like, "I'm going to hook something up that just tells me when my door is opened or closed," and it morphed into this invention. We showed it to people at South by Southwest almost three or four years ago. That's when we realized we were on to something that didn't exist. It was just a little camera on the door. CHARLES: Tell me about those first versions. I'm so curious. It sounds like there's a lot of layers of functionality that you've been through, a lot of iterations so I'm curious about that. What's was that zero iteration look like? CHRIS: Version 0 was made in 24 hours. It was a hackathon for... I can't remember the name of it. There was a hackathon group that recently imploded and we won this hackathon. The hackathon thing was to make something... I'm not sure if this is for Internet of Things but we were all making that kind of stuff. I made this little Raspberry Pi demo with a little mini door and I had talked to my wife and this is how I was able to make this invention, to keep track the kid as I was busy doing other stuff but I talked her into giving me 24 hours to make this one thing. Then me and another guy, David we won this hackathon. We were like, "We've got to turn this into a real thing," because one of the awards of the hackathon was you go to Silicon Valley, you show this off and you do all this cool stuff with it. We were like, "We've got to actually turn this into something that's presentable." That was Version 0. It was just a little Raspberry Pi. CHARLES: Now, what were you doing to detect the state of the door? CHRIS: That's the crazy thing. The first version of the device had more sensors on it than the final version. The first version had everything. It had a doorbell, it had a knock sensor, it had a motion, it had a speaker that played Paul McCartney's 'Someone's Knockin' At The Door,' but it had an accelerometer. I threw everything in there the first thing and half of it worked for the hackathon demo but it was good enough to win. This is something that, I guess I could call wisdom now but the real thing I learned is you start with everything and then you narrow and get it more tuned and highly focused and more precise as a device, like the difference between the iPhone and the Samsung phones. One of them is to throw everything into it and then the iPhone is just really specialize into a few things really well. The next three years, we're pulling stuff out. CHARLES: What are some examples of that calling that you're describing where you're saying, "I'm to take this out? I'm going to take this out. I'm going to take that out." CHRIS: We got rid of things like the doorbell and some of the other sensors, mainly because it was just a wiring issue and as well as we wanted to keep track when the door was opened and closed. It didn't make sense to have the speaker on there at the time so we really focused more on the accelerometer and the knock sensor for the first version of Peeple. CHARLES: That is not the final version. Is it mostly just the accelerometer? What if someone doesn't knock? I assume there's some sort of detection that goes on with the camera. CHRIS: That's the next version. That's something that we've been working on right now, what we're going to be delivering. We have delivered our first, I would say Version 1.0 of Peeple devices to our customers. There's a thousand of these or so in the wild, all around the world and the next version we have added -- and I guess this my first real announcement of this -- a motion detection module. It's not a camera-based. It's more or less magic and it just works through the door. That's the most I'm going to say on it right now because we're probably the first hardware device that it's actually using this technology. ELRICK: That's an excellent pitch. Everyone loves magic. CHRIS: Yes, it's basically magic. It works through the door. ELRICK: As you were going to these iterations, were you doing like user testing to see what users wanted? Or did you internally say, "This doesn't make sense. Let's just take this out." CHRIS: Absolutely. That's the second part of this story. After this hackathon happened, we prepared to go on the road show to go and show it off to Silicon Valley but in the meantime, this hackathon group, I think it was called AngelHack, it imploded. One of their founders made all these disparaging comments about homeless people and what essentially happened is we lost the award. They said, "We're sorry. We can't give you the award," but we had spent about three months fine-tuning, making something pretty and putting a pitch together. I went in and I pitched at a TechCrunch Meetup in Austin and we came in second at that but during that meetup, I met one of the reporters and said, "You really need to talk to these guys in San Francisco called Highway1," so I did. We eventually ended up moving to San Francisco. Now, the reason I mentioned that to answer your question is they understand this idea of user testing, I think better than a lot of people. Even though they were focused on working on hardware and getting an IoT device that works out there, they were drilling it into our heads is, "You have to get this in people's homes now. I don't care how bad it is. I don't care if you have to hire people, to sit at a peephole and just look through it and pretend like there are hardware device. You got to do this and you have to find out what the problems are, what works. I want you to look at your biggest fears of this thing and you quash them and you do that before you put any Silicon down," so we did that as best we could. CHARLES: So you did that with the Version 0 and Version 1 devices? CHRIS: Exactly, just a Version 0, I have all these pictures. We put them in about 12 to 20 homes and we have these long extension cords powering this thing because we didn't have the batteries to figure out. We had these huge lag problems. It would take like 30 seconds to a minute before something would happen. We had all these issues but in the end, people were still like, "It had these issues. You couldn't do this," but the fact that I had a door log, a door diary as what we're calling it now, that's something I never had before. That's where your secret sauce is so we ran with that. CHARLES: Yeah. That's the kind of thing it never even occurs to you. CHRIS: Exactly. In the app, or at least the early versions of the app, is you have these versions like a calendar that are like, "Okay, I got 10 visits yesterday. I got 20 visits today. No one came to visit me today. I'm so sad," but I have a calendar of, I think it was May of last year when I got visited by three or four magazine salesman in one week so you could correlate that with, "Did we have any break ins?" or something like that. CHARLES: Yeah, it would be interesting to be able to share that data with your neighborhood or somehow coordinate that. one of things I'm curious about too is you did this user testing you were talking about, doing the wiring and the installation, it's a conversation that always comes up when you're talking about custom hardware because there's always the drive to be small, there's always the drive to be have a small form factor and then you have challenges of power like how do you power this device. How cumbersome is the installation onto someone's door? CHRIS: Yeah, we had it all. That's a big difference, I think between San Francisco or Silicon Valley and other towns is there's this acceptance and there's this readiness to participate in the tech scene. We did a call out for volunteers and we had no problems finding them. They didn't mind us coming to their house and hooking up these big, bulky things and just being real intrusive. The fact that we found these people and they were the key to this early stage of, "Do you become a product or do you not?" We were only there for four months but by the end of this time that we were there, there was this legitimate tangible feeling of we're not a prototype anymore. We're a product and we didn't have a product. It was just prettier but we could see the light at the end of the tunnel. I don't think that would have happened had we not gone through this very painful experience with all these poor people that we inflicted our device on. CHARLES: This actually is fascinating because obviously, you're back in Austin now and I never heard of programs like that, like sign up to have someone come up and test it at some alpha stage prototype in your home. That sounds crazy and yet, it sounds like they were just going out of the woodwork. CHRIS: In San Francisco, it's not a problem. If I put the call out now, I probably have to really like, "Here's an Amazon gift card." I have to start doing a little bit of bribery. ELRICK: I think I would sign up just to see the cool tech. CHRIS: Yeah and those people exist. I think we don't have the means to really find them. That infrastructure already exists. In Silicon Valley, you just go down to Starbucks. CHARLES: There ought to be some sort of meetup for people who want to experiment with very early stage IoT devices here in Austin. Maybe, we'll have to look at it. If that doesn't exist, I would love being a guinea pig. I actually think there is an untapped willingness here but there's just not -- CHRIS: I think you need a critical mass of hardware people and hardware devices that are ready to be put in doors or put in the houses. There's definitely some in there. I have a lot of friends and there are hardware meetups that we go to but this stuff takes so long and it's so hard as hardware is hard. There's that small window of, "We got this little idea of a water sprinkler. Do you think anyone want to try it out?" or something like that and then the moments gone. Then six months later, there's another one. CHARLES: Yeah. I wonder if there's a way to really decrease that iteration cycle so that you can get feedback more quickly. I guess the problem is when you need a physical device, you just needed a physical device. CHRIS: We're talking about the Maker Movement and the MakerClub. If you're part of those, these people are hard to find. People that go to Maker Faires, that's the people you're looking for. CHARLES: Right. Now, transitioning because ultimately your target customer base is not makers, not people who are willing to put up with wires and cabling and people doing protracted installation. What does the kind of 1.0 product look like? Because what I'm curious is what immediately jumps to mind is this thing sounds like it's going to probably consume a lot of power. How do you get the power to that and what are the challenges and what are the tradeoffs that you have to make to try and get that power consumption down or get the installation complexity down? How complex is it today to install? CHRIS: I guess, I'll toot my own horn a little bit but I think we have one of the easiest IoT devices on the planet to install. You can possibly not even need tools. You can use your fingers but the biggest challenge for any IoT device is getting that home network connection. If there's been a few technologies through the years in which they've tried to fix this problem, basically just like self-pairing or things like that, like how Bluetooth can sometimes be really cumbersome. Now imagine that with Wi-Fi, it's the same thing but now you've got a password you've got to throw in there. That's really the only real hiccup with the installation on our device and we tried a few things. We went through about three different Wi-Fi chips before we settled on what we were using now. The first Wi-Fi chip was a TI one, which offered this nice pairing capability but it just didn't work half the time. Then we switched to a Broadcom chip, which was really solid and stable but turned out to be the most expensive component in the whole device so we had to get rid of that. The Wi-Fi issue was something we had to solve early because it goes also toward your power consumption. We have a camera and a Wi-Fi chip and both of those take up to 140 to 200 milliamps of juice when they're on. We had to be really smart of when this thing was going to be on and that's essentially when we went in parallel with the knock accelerometer. This device stays asleep most of the time and that's how we get the many months of battery life out of it. We put a rechargeable battery inside, it only turns on when it needs to and it's just hanging around waiting for an event for the rest of the time. Those were the things we were solving to get the Version 1. CHARLES: Now, it's waiting for some event but in order to receive the event, doesn't the accelerometer need to be on? Or is there some motion detector that --? CHRIS: That's a solved problem. good news was that accelerometers are extremely low power in the nano or picoamps but that's also another reason why the motion detection was going to be a hard problem because that is not, unless you're using what's called a PIR that is not a low power solution. CHARLES: Acronym alert. What is a PIR? CHRIS: It's an infrared proximity detection. That's how almost all motion detection cameras work. They have one hole for the camera and another hole for the PIR. The problem with these are is they don't work well in sunlight, outdoor-light and things like that in one of our use cases so we were kind of stuck. That's why we've recently come up with this new motion solution that doesn't rely on that technology -- the magic solution. CHARLES: All right. When we're going to find out about the magic solution? CHRIS: As soon as I ship this next version because it is being used in a few products but it's not really stateside yet and I want to save my thunder but it's something that I think is really cool. It really is magic. It's just amazing to me that it works. CHARLES: Well, I'm eager to see it. You were talking about Wi-Fi being one of the biggest challenges. That's a perfect segue. The connection to the network for something that we're always curious is discovering a new and interesting device is always a pleasure and then the next thought that almost funnels immediately after is how can I integrate this with other strange and wonderful devices to make something even more wonderful? A question we ask everybody is have you thought about how this might be a participant in an ecosystem so if there were other devices around the home, how would they even talk to the people? How might it offer information to someone looking to, maybe do some custom integration in their home? CHRIS: That's a lot of questions in one. Essentially, there's two ways of looking at it. You can look at it from your customer's perspective, what kind of customer do I think is going to have this or is going to use this the most. Back when we came up with this, there were a lot of do-it-yourself types and If This Then That protocol was out there but we really wanted to focus on something that was incredibly easy to use and didn't require you to program anything. I was really frustrated with the whole idea of Internet of Things because it almost implied that you had to be a programmer to use it. I didn't like that at that time. I've since come around to it because there's all these great tool kits out there. We initially looked at integrating with HomeKit. We thought they'd be perfect but what a lot of consumers don't realize is early HomeKit -- I don't believe it does that anymore -- made you modify your hardware to put in this special Apple hardware. When you're making a device, it is so hard just to get the hardware down. It's so expensive. To add anything or to put anything else in there, it's a huge friction point. It's really something that small startups just can't afford to do. A big Nest or a company like that have no problem but when you're making a one device, this is a big deal so we weren't able to really leverage something like HomeKit for an API. But we do have our own cloud-based API. We're RESTful API but it's just not documented and put out in a way where we want to have people programming it. But the good news is we did leverage several APIs when we were making things like the app and doing things like the push notifications and things like that. Now, it turns out that a lot of the case we used are now integrating with things like Alexa and other device protocols so we essentially get those for free. This whole ecosystem is forming around us. Just most important is to get your device out there because you have a vision for what the device will be used for. But then your customers tell you what the device is really useful for and that's when the real work starts. CHARLES: Right. I guess, it's true you have your first line of customers and I guess the use case what I was thinking of is me being a developer. I'm thinking what products could be built then using this as a component, so to speak. Have you'd given any thought to that or have anyone had approached you to say, "This is amazing. I'd like to build this meta product that integrates that," or is it kind of early days? CHRIS: Early on, that was the approach of the Internet of Things and it merged away from that in my experience. Early on, it was all about building blocks. You got to understand, these are old Zigbee Z-Wave programmers and that was the whole concept. Then it got turned on its head by, "I really have this problem that I need to solve and I don't want to have to make a bunch of building blocks to do this." For attacking it from the other side, like you're saying, building up into pieces, I really recommend you talk to the Twine guys -- super mechanical -- they're here in Austin as well. A year or so before, we came out with Peeple. They put out this device which was exactly what you're talking about. An Internet of Things type hub where you just add in all the pieces and then you integrate with everything. They can better give you a story of how that lifeline goes. CHARLES: Yeah, because it's always something you think about because you've got all these wonderful things. CHRIS: Yeah, some would say, an Internet of Things. CHARLES: Yup, or at least a floor plan. ELRICK: When someone gets a Peeple device, what is the full installation story and set up? What is the walkthrough for that? CHRIS: We have a little video of that. What you essentially do for Peeple when you're installing it on the peephole in your door, you unscrew the peephole. Now, the way Peeple's work is they need to handle doors that are variable width, depending on where you live. There's no real standard. All of the Peeple's work by having a shaft that you screw onto another side so it's basically two pieces. Now, one of those shafts holds this bracket that we include in the package. You screw that onto your door with the peephole holding it to the door, then you turn on the Peeple device and you connect it to your home Wi-Fi and then you're ready to go. That's it. CHARLES: That's the hardware side of the onboarding and then what about the software? How do I go and look at my door diary? CHRIS: You do this during the installation. You go to My.Peeple.io and there's a little button to add your Peeple device. UI-wise, it's one user interface among all the platforms whether your Android, iPhone or on a browser. You just go to that webpage and associate your account to your Peeple devices. You will have to log in. You can log in with Gmail, Facebook or just a regular email. Then you add your device and any time you go back to that page, it will show you only the videos from your device so you have a list of all the events from your Peeple device on that page or in that app. CHARLES: That is interesting. I'm looking at the videos right now online. Although my problem actually is I've got a glass door. CHRIS: Yes, we got you covered as well. CHARLES: You do? CHRIS: Yes. The reason you have a glass door or a peephole and many people don't realize this is it because it's required by law. If you ever plan to have run out your house as a multi-family unit, you have to have a peephole or a window surface to where people can look out. Once we figured that, that's when we realized we were onto something. The first versions of Peeple came with these little adhesive pads that we called gecko skin and this is where we learned a valuable lesson. No matter how sticky you make your stickers, they're not sticky enough. We included three of these little tabs in every device to put on a glass door, if you had glass so the Peeple device would work the same way for glass door, except that you would use a sticker, instead of unscrewing the peephole. The only problem with the stickers were is they were not sticky enough. If there was condensation or a weather event or something like that, these things would fall off so we made a modification. We found better stickers and I mailed those out to all the people. But this is why hardware is hard. You're going to make these mistakes. In all our testing, we didn't find this but of course, once you have a thousand testers, you find a little more. ELRICK: That's interesting that you brought up the laws about the peephole. Were there any particular building codes or anything of that nature that you guys had to be concerned about when having Peeple installed things on their doors that you had to figure out before shipping them out? CHRIS: Not really. The Texas property code is more geared among making landlords do the right thing. In case you're wondering, I think it's Texas Property Code 94-152 that covers this. There must be an external viewable portion for all multi-family units to the front entryway. Now, this is just the Texas law. We had to look this up in a few other states and it turns out there's one in San Francisco, there's one in Virginia but they're all different. But so far, we haven't had any issues with any property codes or building code issues. CHARLES: This has been an almost four-year odyssey for you that you've been on, right? CHRIS: Right. CHARLES: You've been involved in this scene and working with hardware probably for a long time even before that, it sounds like. For people who are just getting into it, because I feel like there's this wave cresting now, where these types of startups and these types of side projects and hobby projects are just starting to enter the mainstream. Do you have any advice for anybody who would want to get into this space? CHRIS: Well, that's a great question. Of course. Now, contrary to what you just stated, I didn't have much of a hardware background. I'm a software guy. I can personally attest to the pains of becoming a hardware guy. Now, the irony of this is I do have a master's degree in electronics engineering but electronic engineering is so huge. It's such a big field that you can spend your entire career not doing much hardware. But I always had the ability to go back and build some circuits but I would say the number one thing, if you're not a hardware guy is go to some of these meetups or get involved in a community and find yourself one, someone who has experience doing hardware because coming from the software room, you're used to this flexibility of changing a few lines of code and being everything changing. Now, when you get a hardware guy onboard and our hardware guy's name is Craig, when he comes to work -- CHARLES: Or gal. CHRIS: Yeah, or gal, of course. When they look at the same problems you're looking at, they're like, "Hold on a second. Let's step back. Let's test this." There's this quantitative slowing which you need to have as hardware because once you build a PCB, a circuit board, you are now stuck with that board for the next month or so because it takes a while to make another one so get that right before you jump around and do all these changes. My first advice would be is get help. There's no shame in going out there and you might be surprised. There are so many people out there that want to join in. If you have a good idea, there's plenty of people who want to contribute. CHARLES: Would you say that there are communities out there like the software communities where you have meetups? Some of the software meetups are just fantastic, where people are so welcoming and they're just so excited to share the information that they themselves are so excited about. CHRIS: Yes and there's the same thing as on the hardware side. You would definitely go to a few hardware meetups, there are several in Austin. There's at least one every week and it's a great chance for people to tell these kinds of stories. This is a maker type community so they welcome these ideas because that's what fuels their enthusiasm. Every time someone is doing something new, they want to hear it. That's the change now. This decade has happened to where you can go out and buy a few modules and make your little device. Then there's the next big step of turning it into going from prototype to hardware but you can get all those kinks out without having to make your own printed circuit boards, without having to have a huge firmware background. Just knowing a little bit of tech and a Raspberry Pi, you can test out your inventions at this early stage without having to invest all this money and these other things. There's never been a better time to do it. I would leave your listeners with is if you got something swirling around your head, get a Pi, get a little Arduino and do it. There's nothing stopping you. CHARLES: Yeah, it's shocking how affordable they are. CHRIS: I don't even touch on China, by the way but that's the next step. CHARLES: That's the great thought that I want to leave everybody with but I actually have more questions so we won't leave everybody with that. We'll keep on going because I want to talk about China and I want to talk about something that was in there. You've touched on it a couple of times when telling your story how you go from this just do it, get it out there, get it into people's homes, just get the Version 0 out, just buy an Arduino, slap together something terrible, that is at least one millionth of the dream that you have and you've taken your first step on that odyssey. That's a very common story in software. The way that we develop software too is have these agile methodologies and these techniques to reinforce them, testing, continuous integration, continuous deployment. How does that play out? A fascinating subject to me personally is how do you do that in the context of hardware. A question that I love to ask is how do you do things like ensure quality? How do you do integration testing? How do you have a deployment pipeline if you've got these Peeple devices out there on tens of thousands of doors globally? How do you push out a bug fix or a feature update? What's the automation around that look like? CHRIS: The over-the-air updates are your friend. If you're going to make a hardware device, I recommend making a Wi-Fi enabled device because then your firmware is not locked, then you can do over-the-air updates. That has been a lifesaver. We've done maybe a dozen software updates to our device to date, sometimes little changes, sometimes big changes. But what happens is any time the Peeple device wakes up, it says, "Hello, server," and the server says, "I got an update. First, let me give you all these images." Give me the code. The devices are constantly upgradable, just like you'd expect with software. Now, with some of these Bluetooth devices, you can't do that. You've got to go out the door being ready to go with no issues. It's a friction point to tell someone, "Your headphones can't work now. You need to plug it into a computer. You need to download this firmware upgrade. You need to update the firmware doing it by hand." That just isn't going to fly in today's consumer market so I would recommend if you can, make your device a hardware Wi-Fi device, get a Wi-Fi module in there and that opens up the world to you on doing a lot of these updates, to answer the last part of your question. CHARLES: You mentioned China, since you're touching on the manufacturing process or just the market over there or --? CHRIS: Yeah, be ready to fully commit. I've been to China, maybe four times now. I have a 10-year visa. It took a while to find the right partner and you've got to be boots on the ground in the factory for a couple of weeks just getting the whole line up. It's a whole another product when you're at the manufacturing stage. You're making all these little test things, they've got to hook up the boards to certain devices, they've got to put the firmware on it, they've got to do these things. It's a whole another job. That's why when you do these Kickstarter. They say, "We're going to be out in three months," and then six months later, "We're still working on it." I have a lot of empathy for this because I've lived it. You think, "I've got everything done. My hardware works. All I have to do is team up with someone to just make it and with them, we'll ship it." There's a whole another level to just a manufacturing piece and you can't really learned. There's no real textbooks to learn this because every factories are different. Our factory is right north of Shenzhen and we talked to some US manufacturers but they just weren't competitive to be in the discussion so you pretty much have to go overseas and then you have to sit down with them and just a little bit of communication difficulties can bring down a whole manufacturing line so it's very important that you're very hands on and you see your product all the way to package. ELRICK: That's interesting. I know of it but I never really thought about it because I was really not in that position. What are some of the higher level of things that you should look out for when evaluating a manufacturing partner? CHRIS: We talked to about a half a dozen before we decided on our manufacturing partner. The big one for me was cultural fit. I talked to some of the big ones like the one that makes the Apple phones, we talked to them for a while and I just found that I would say, "We would like to do this or we need this," and then the next week, they'd be asking a question, "What about this?" and I'm like, "Oh, you didn't understand what I was really asking," so you would lose weeks just by tiny misunderstandings. I found a manufacturing partner that has a subsidiary here in the US and my main contact grew up in the United States but he also goes to China every other week. Having that kind intermediary made everything so much easier. The communication was never an issue. I was able to get things done almost twice as quick with the other manufacturers I was talking to. In the end, they also came up with a great price so it turned out to be a win-win. I would recommend talking to the bigger manufacturers but spend a lot of time on the smaller ones and really figuring out is the communication up to snuff to really make your product. It's huge. CHARLES: What a story. I'm really glad that we got to have you on the podcast, Chris because you have the story that starts from literally slapping a Raspberry Pi and an accelerometer and speaker and apparently a bunch of other things on your front door and with an extension cord and walking a continuous path to where you're flying back and forth between China and Austin to inspect and ensure your assembly line and making a real product. It demonstrates that it can be done by the fact that you have done it so I think it serves as an inspirational case for a lot of people out there who might think that this is something that they might want to do. Or think that they're capable of. Thank you so much for coming and talking about Peeple. Everybody, you can go ahead and check it out. It's Peeple.io, right? CHRIS: That's correct. CHARLES: All right. Also, is there anything else that you'd like to announce other than the magic, which you're going to keep a lid on? CHRIS: Yes, I know I'd appropriately teased everyone about that but you can go to our website. If you go to Shop.Peeple.io, we're taking preorders for this next magical version, the Peeple Version 1.1, I guess I'll call it. I would like to add just before we go is if you're going to endeavor to do something like this, make sure you have a very understanding family because they couldn't have done it without a wife and kids that understood my craziness and allowed me to have just a complete mess of our house for, I guess, for three years now. CHARLES: Thanks again and thanks everybody for listening to this episode. You can get in touch with us on Twitter. We're at @TheFrontside and you can always find us on the web at Frontside.io and there's a contact form and we'd love to hear from you, for any reason whatsoever. Thanks, everybody and we'll talk to you next week.

John Boyd: LinkedIn Show Notes: 01:27 - Knocki 03:20 - The Device 06:19 - Complexity 08:44 - Software Distribution 14:01 - Allocating Memory 18:27 - Finding Hardware Hacking Libraries 22:01 - Updating and Diffing 24:06 - Migrations 26:51 - Decentralization of IoT 35:39 - Managing the Knocki Ecosystem 40:17 - Communication Standardization Resources: Malloc Transcript: CHARLES: Hello, everybody and welcome to The Frontside Podcast, Episode #81. My name is Charles Lowell. I'm a developer and your podcast host-in-training here at the Frontside. With me today is Elrick Ryan. Hello, Elrick. ELRICK: Hey, how are you doing Charles. Welcome back. CHARLES: Yeah, thank you. It's good to be back. Today we're going to be continuing the ongoing series that we've been doing intermittently on the Internet of Things. It's a really fascinating, almost to a person fascinated with here at the Frontside. Today, we have with us to talk about this, someone who's very, very knowledgeable on the subject, John Boyd, who I got an opportunity to talk with, I guess it was about a month ago and I wish that we had the podcast recording equipment there in the room because it was just a very, very well-versed engineer, exactly the person you want to be the CTO of your company, which is very lucky for Knocki, the company that he works for, because he is in fact the CTO there. Welcome to the show, John. Thanks for coming. JOHN: Yeah, thank you very much, Charles and I'm excited to be here. I'm excited to join the conversation this week. CHARLES: Yeah, why don't you start by what it is that you do at Knocki? Most of our audience comes from software and design and product management backgrounds. You've got a very strong hardware background. How does that play in to what you do at Knock? JOHN: Yes, certainly. As you previously mentioned, I'm CTO at a startup called Knocki, which you can mount onto any surface and turn that surface into a user interface. We're recently funded on Kickstarter so we're in the process of actually trying to develop this hardware but the central concept is any surface that you mount this on will now listen for touches and vibrations so you can say, mount it on a desk and tap three times on your desk and control your smart home around you. If you have smart speakers or TV, you can tap three times out of four times and control those devices with a really natural interioractive interface made out of anything in your home already. CHARLES: Tabletops, mirrors, I assume you've tested this on a lot of different services. JOHN: Yes, I'm sure we'll talk about that more a little bit later but the goal is to be able to turn any surface into user interface. That means if you really wild and you want to use it on the window, I recommend it. But we're thinking desks, walls, doors. It has a lot of applications for disabled and handicapped individuals. Think of a child or someone in a wheelchair that can't quite reach a light switch, if they have a Knocki mounted on the wall, they can still knock on the wall to control the lights. We feel like it adds a new level of user interface to people's lives that can be helpful. CHARLES: Definitely. Seeing the product and hearing you talk about it, I definitely got that impression. Now, the device that you actually brought into the office because you did come in and talk to us, like I said it was about a month ago but it was extremely tiny. In our explorations into the Internet of Things, we do things like control our lights from within the office. At least, we're trying to control our lights within the office. For us, we're using the standard kit. We've got Raspberry Pis that we're using, that are have access to a plug and they've got a full Linux install, just a really powerful processor and by comparison to the things that you were talking about, that's energy hog by comparison. We think of it as being very lightweight but if you're talking about making some small device, it's actually really, really wasteful of resources, so to speak. What is that transition that spectrum which you moved from these one-off hobbyist things where you're using high-powered equipment to these really custom devices? How do you make that transition? And what is the difference between the two? JOHN: Our devices are about the size of a hockey puck, which is much smaller if you can think of a Raspberry Pi. Pretty difficult to fit that inside of a hockey puck, especially when you want to start adding some sensors to detect knocks and taps on a surface. I don't hate or dislike the Raspberry Pi or BeagleBone Black or any of those really quick SBCs that can get you started with IoT. But they have -- CHARLES: Acronym alert. What is an SBC? JOHN: SBC, single board computer. It's any of those credit cards size computers. CHARLES: Okay, great. So nothing against the SBCs like BeagleBone Black or Raspberry Pi. JOHN: Exactly. It's a great way to prototype ideas and get in a proof of concept out there and there are some cases where actually, they're great choices for a full-fledged product. A lot of cases in IoT, people are more concerned with things that you carry around with you so they have to be battery powered and you need to be a little bit more conscious about energy economy. You need to be very cost-effective with your components and it doesn't make sense to buy an expensive Raspberry Pi for each unit. CHARLES: Did you actually start with a Raspberry Pi, when you were developing this product or something that's like an SBC? JOHN: I actually went straight to a microcontroller dev kit. I started with Texas Instruments' CC3200 LaunchPad. It's a little bit lower level than SBC like the Raspberry Pi. It doesn't run Linux. The firmware I started off writing as a proof of concept was still embedded C bare metal software. CHARLES: How much complexity does that add? There's just a lot of nice things about having an operating system and being able to have your compilers, I guess you have a compiler tool chain, but having being able to install big programs like interpreters so that you can run Ruby and JavaScript on there. There's just nice things like scheduling. If you've got a bunch of processes running on this device, you don't have to worry about them, saying who's going to get what processor time. I assume that you're having to deal with all of that if you're writing the firmware by hand using C, right? JOHN: That's 100% true. There's definitely some great advantages to using a little more powerful system that can run a full Linux stack or full OS. As you mentioned, the design complexity is reduced a lot because you can import other people's code and you have a full operating system to handle most of the drivers in the system. You're right. There's a lot more complexity. We have to write all of that ourselves in C. But that's the fun part about it to me. I love getting down there and writing drivers that can communicate with accelerometers and set them up. As far as scheduling goes, for getting concurrent software running on your embedded system. There are RTOS's -- real-time operating system that can provide basic scheduling. For the brave, you don't even have to use that. You can use a lot of the embedded timers inside the microcontroller itself. But to answer your question, it is a lot more complex but one of the tradeoffs to get a device that small, beautiful and also has a battery life that can last many months or a year. CHARLES: Yeah, it almost sounds like the complexity but you're not going to save yourself any time prototyping it in tools that have all those things because you're essentially going to have to be rewriting your system from scratch, because those things are just a nonstarter if you want low profile devices. JOHN: Yeah, there's definitely a lot of rework would have to be done but those SBC systems are still very useful for prototyping the cloud side. Internet of Things is hardware and internet when it comes to building out your cloud interface. CHARLES: Yeah, that's definitely true. You're running a bunch of software on this device. The software that you've written, how do you actually distribute the software because we're very used to in our world, software distribution is not a problem. That's what made the web so popular. While we were willing to deal with really crappy tools on the web for a really, really, really long time, the distribution model was just so nice. You're also having to deal without that too when you're operating in the device space. But the challenges are still there. If you've got a bug on one of these things, how do you even detect it and how do you get a fix out there? JOHN: Obviously, any software is prone to bugs. Nobody writes a perfect code the first time. If you do, I'd love to hear about it. Obviously, one of the big concepts in IoT is security and to have a secure product, we need to be able to patch bugs as they arrived. A big really important feature in any good IoT product is the ability to remotely upgrade the firmware or send the patches as part of the maintainability that prevents big software bugs from turning your IoT product network into a botnet. A lot of our time is actually spent trying to make sure that our remote update capabilities are reliable, always functioning and globally distributed. You'd think this is an easy problem to solve but when you're working on a microcontroller that's not running an operating system, running bare metal code, things get a little bit more complicated when you want to make sure that any device anywhere in the world can install the next version of firmware reliably. CHARLES: Right. At any time there's a software update, it's always, it bugs me and then do I want to do this and it's always optional. There's none of that, right? It's just what a new version of the firmware goes out, boom! It goes out there. JOHN: You can design it in different ways. There are some great products out there. Apply the firmware update through the user's phone so you may open up your products application and it says, "There's an update available. Go update." That's definitely one way to do it but that's the problem if the user is not home and maybe they've set this device up in a guest house and they won't be home for six more months, then you have a device that could be vulnerable for six months, which is a long time in the world of software. CHARLES: Yeah, that's true. JOHN: To get around that, obviously our preferred solution is to have the device checked into our cloud servers to see if the device itself has updates available and then go through the download and update process that way, just to make sure even if the user is not home or never opens their mobile app, it will still get those critical security updates. CHARLES: Sounds hard. You're running a risk of bricking someone's device if that update doesn't go very well or it loses internet in the middle or power. JOHN: Very true, especially when you go towards a bare metal microcontroller with limited memory and limited processing capabilities, unreliable internet connection, a lot of work has to be done on the device side to make sure if something goes wrong during the firmware download process or installing the image correctly that it has a backup image. If you're downloading a new firmware upgrade and the download gets corrupted halfway through, make sure you have an old image that you can boot into. That's one part of it. The other part is detecting that it went bad if it gets past downloads in your image and then it reboot itself and tries to boot into it, how do you know that that image actually isn't behaving the way you want it to and then go ahead and revert back into that original stable version. CHARLES: I assume there's some key so that you can verify, not only that the image is not corrupt but it's a certified Knocki image that's coming down the wire? JOHN: Exactly. We signature verification, again something that I think anybody on the internet should be using when you download new software but make sure that the new firmware update was actually written by Knocki and you're not installing someone else's code. Another important factor is just please use HTTPS secure SSL connections to your server, then that reduces the possibility of someone taking over and giving you their own firmware image. But there are a lot of low power devices out there that are being used to make IoT products. These low power devices are important for many reasons but they have restrictions and sometimes, their security capabilities are limited. Maybe doing encryption on the device and actually are doing certificate verification. That's a costly operation. CHARLES: It sounds like there's a lot of cycles that that consumes. JOHN: Definitely. Most people try to make sure they have the resources to solve these problems but at the same time, there are a lot of developers out there that are cutting corners and that's where you get these big news stories about IoT products getting taken over. CHARLES: Along that vein, it's your reality but it constantly blows my mind that things that you're living without when you're programming for these devices like Knocki is do you have to write your own network stack? When you're doing these downloads, that's kind of like got it all. You've got the encryption piece that you've got to do to make sure that you're connecting over SSL so you've got to do the whole handshake and you've got to do the key exchange and the certificate verification and then the packets come in asynchronously so your message is arriving asynchronously in bits so the header is being assembled, now I've got the HTTP headers, now I can go ahead and get the body. There's a lot that happens for us when we're making a simple Ruby request. We're basically like resource.get. Boom! And it just comes to us fully assembled in memory. How much do you have to hand roll all of that? Are there libraries for doing it? How do you put that process together of just even downloading the image? JOHN: Fortunately, there are tons of open source freely available libraries for embedded C software that can help us solve these problems -- CHARLES: Is this like a genre of software like if I want to go look for these libraries, how I look for them? JOHN: In my example, all of our firmware is written in C or C++. Since we're working on a microcontroller with limited resources, it's important to look for libraries that don't use dynamic memory allocation. That's why it's a really big [inaudible]. Some software relies really heavily on that but -- CHARLES: When you say dynamic memory allocation, you're talking about like Malloc? JOHN: Exactly. CHARLES: You are basically are allocating memory on the heap. When you're doing for this, you basically want to do everything on the stack. Now, is that just because the instruction set of the processor doesn't support it or is it because it's just there be dragons like here there be dragons? JOHN: That particular scenario is actually just due to resource limitations. There's just not a lot of memory on our device. We do use Malloc in some cases but we have to be very careful about when we use it and make sure that it's always going to have the memory required or if it doesn't have the memory required, there's some fail safes involved. If you just use someone else's open source library and they're allocating memory left and right, they could end up causing issues on your embedded system. CHARLES: Right. Now, just a little bit of background for people who might not be fully familiar with Malloc, it's just when you're executing a program, you have this heap memory, which is where you store random stuff and then you have your stuff on like the call stack. Your variables that are on the call stack are in one place and then your just generic data structures that could be accessed from anywhere are in this thing called the heap. Our dynamic languages that we use like Ruby and JavaScript, the heap is hot stuff. Like everything gets allocated on the heap, that's why they consume these huge amounts of memory and then the things that are on the stack, really are just pointers that are referencing these big bags of data that are on the heap. But it sounds like you've got the exact opposite situation where you don't want to have big bags of memory that are just floating around in a heap and you want to do everything inside that stack. JOHN: Exactly. I couldn't have said it better. CHARLES: Anyway, you're looking for libraries that don't do that because it sounds like any time you want to allocate memory on the heap, that's going to be shared for the whole program, that space is very limited so you want to be very, very, very strict. You want to control that process. You don't want any other library that's doing it for you. Is that fair? JOHN: That's correct. That's also one specific example, dynamic memory allocation of the things that you want to make sure your other software libraries aren't going to be abusing. But in general, you need to make sure that any code that you're putting is compatible with your system. It doesn't have some special hardware requirement that your embedded system doesn't have. CHARLES: Right. For people who want to get into hardware hacking, is there some golden seal of approval like the people say like, "This library is great for embedded devices." Like I said, a lot of times when you're coming into it, you don't know what to look for so what you're really looking for is some expert or authority on the subject who can say, "This is good. This is not good." It is like, "Don't even look at this library because you're going to find something else because this is not embedded-friendly." JOHN: That's a good point. I wish there was a golden seal of approval or I wish I knew one, at least. Normally, most of our code that we uses are hosted on GitHub. Usually, we try to find software that was optimized from embedded systems and the author of that code will usually mention -- CHARLES: That [inaudible] me. JOHN: Exactly. This was designed for microcontrollers. ELRICK: I was going to ask if there's a golden standard when you're building these type of devices. Is there a checklist of things that if someone's going to build something similar that these are good things on your checklist that you should attempt to check off, if you're building this sort of device or want to build something similar. CHARLES: Now, you mean things like update and whatnot? ELRICK: Like updating or like how you were mentioning avoiding dynamic memory allocation. Anything, you can just shoot from the hip, like these are things that you should watch out for a lot of your battery power, you should look out for this or anything. JOHN: Yes. I definitely think the number one consideration that the biggest check box and [inaudible] before it goes out the door is going to be your security suite. Make sure your internet connections are encrypted: SSL, TLSL, that good stuff. Then as we hit on earlier, making sure that you always have a way of updating the device but don't use back doors. A lot of people think to update your device, you should put a back door access and you can go in and download updates that way. That's not the answer. ELRICK: That's like the back door that they were looking for in Apple like, "Do you guys have a backdoor to get into your device?" No. JOHN: No. That can be a controversial conversation. CHARLES: Yeah, or they're like, "Come on, really. It's okay. You can show us the backdoor." No, there is no backdoor. "I know you have to say that. Blink-blink." ELRICK: That's an interesting problem that you guys are solving on how to update these devices. You guys are essentially hand rolling or developing custom software to do that. JOHN: Again fortunately, we're using a Texas Instruments SSC system on a chip. They provide some core functionality, some core drivers that really help us out. For example, they provide a special bootloader that can really assist with a lot of the firmware download back up framework image checking, that sort of functionality. We don't have to write it all by scratch but we do have to write the logic to make sure that the device does check for updates and it doesn't forget to check in and talk to us. ELRICK: On the cloud side, do you guys have to write any custom software to do diffing, to make sure like -- Oh, do you diffing? Or do you just update everything all complete, like once you're updating, you're going to get a brand new update or do you diff and say, "You only need this." JOHN: Since we're working on the system that we're using, it just requires a fully-compiled image that gets installed by the bootloader. We can't really send just a patch to one part of the firmware, if that's what you're asking. CHARLES: But I assume there must be some state that's on the Knocki itself. Just even the credentials for the local Wi-Fi network, what devices it's connected to, part of the system is updating and part of it is not, I assume but how do you make sure that that state is compatible with the new firmware? JOHN: Yeah, that's another great point to keep in mind. The way we keep most of, we call it nonvolatile memory, every time the device reboot, it's going to forget about everything that was stored in RAM so we need to have somewhere in nonvolatile to store these things. We have a file system on the device that we can create files with different device configurations, algorithm, settings, Wi-Fi credentials, that sort of stuff. CHARLES: That file system, is that anything that we would even be familiar with like ZFS or is it just a custom file system that you've written or that you found on GitHub. JOHN: No, fortunately this is just a standard FAT file system. We do have some creature comforts there but that's not necessarily the norm. CHARLES: You heard it here. Is that FAT16? JOHN: No, it's FAT32. CHARLES: FAT32, described as creature comfort. JOHN: Yeah, we have a different perspective of creature comfort. CHARLES: There's a couple of things because immediately, what this brings to mind is for people who are familiar with Ruby on Rails, they have this concept of migrations, where you're migrating the schema of your database and as you have to transform the data from one format to the other, you're running these migrations. One of the things that's nice about that is if, let's say I have some system that is at Version 1, but let's say, I have one of the devices that hasn't taken an update, it starts at Version 1 and it needs to go to like Version 100. But you could have 10 format changes in between there. Is there a way to handle that case where you're basically incrementally applying a bunch of transforms? JOHN: Yes. That's another great point. We take this on a case-by-case basis. Fortunately, being a small relatively simple system, there's not a whole lot of state data to keep track of. But to handle that situation, we've written are own OTA server-side software that manages the devices sending updates -- CHARLES: Acronym alert, OTA? JOHN: I'm sorry, yeah. Another acronym, OTA -- over the air updates. That's our slang for remotely sending firmware updates. CHARLES: Sorry to interrupt. It's just we have to unpack acronyms. JOHN: No, I'm sorry. I use a lot of jargons here. CHARLES: You know what? The thing is, so do I and I just never even notice it. JOHN: To handle that scenario, the way we handle it, our cloud knows what devices are out there and what firmware updates we've sent out to it. Furthermore, when the device checks in with the cloud and ask, "Do I have an update available?" It also tells the cloud, "By the way, I'm running Version 1.0." The cloud knows, if it's on Version 1.0, there's going to be some incremental changes that need to be made before we get to that last update and we can apply those changes incrementally. CHARLES: I see. I feel like we've touched on so many of these concepts that are universal to development but only projected into the hardware space. We've talked about dynamic allocation of memory and data migrations and it sounds like what you're describing in a way with OTAs is continuous delivery, where you have some way of automatically pushing out an update and all the stuff that's involved in that. It's just really cool to hear to view through such a vastly different lens than what we're used to. ELRICK: We've been talking a lot about communication between devices and back to the cloud in things of that nature. Does that play into the conversation around decentralization of IoT infrastructure and what does decentralization of IoT even mean? JOHN: Decentralization as a new methodology or ideology that a lot of people are adopting, I shouldn't say new. It's been around forever but the idea is from a high level, looking at the internet, most of the internet is access through some central, server is hosted on you name it -- XYZ cloud hosting provider. The way you do your URL DNS resolution that goes through centralized DNS servers that say, "You want to look at Netflix?" Netflix is stored over here on this AWS server farm. Decentralization, the idea is we don't necessarily need to talk to this DNS server and talk to AWS just to get content from specific providers. If you look at IoT for example, a lot of times in our case, we want to tap three times on the table and then later on, it will do the cloud, send the message and then turn on your Philips Hue light bulb in the living room. It would be great if the message could just go directly from Knocki to the Philips Hue light bulb, rather than going to our cloud, on some centralized hosting provider, then to Philip Hue's cloud, on their provider then out to the Philips Hue light bulb. Those are some of the really popular technologies that's a lot of people are talking about that really take advantage of the concept of decentralization. But it does -- CHARLES: Let me understand because why these would be necessary. When I get why it's compelling, if I want to have my Knocki talking directly to my Philips Hue light bulb without getting your servers involved, without getting Hue's servers involved, it seems like it's going to be a lot faster and just a lot more robust. There's just less links in the chain but it presents its own problems, like on both ends of the conversation between the Knocki and the Philips Hue, how do they agree that this is sanctioned by a user? That's just leaps out. That's a hard problem to solve. ELRICK: That you use some sort of like public-private key type of encryption to say, "It's me. Am I allowed to do this?" CHARLES: How do you decentralize that? JOHN: Well, I'd like to preface this by saying I'm not an expert on that particular subject but the goal is, if you're familiar with the bit torrent protocol and how it keeps track of a lot of different peers on a network using distributed hash tables, the idea is if you know at least one other person on the network, that person can say, "There's some other people that you may be interested in talking to, that may actually want your message. I'm just a bystander on the network and I don't really need your message but this guy is interested in it." In our application, that would be our server. We have to ask our server, who's out there that wants to hear what I have to say. The server is going to say, "Knocki 123, this Phillips Hue is over here at this address, this unique resource identifier, he's going to be very interested when you have two taps or three taps on the desk so just go ahead and talk directly to him. You don't need to talk to me." There's a lot more of that goes into that about making sure that the network can heal itself if somebody goes offline. But as I said I'm not really an expert in that subject. CHARLES: Right, but it really is compelling. Would you then, maybe have some device that was just kind of your coordinator in your home or multiple devices that would act as these bit torrent trackers? JOHN: Yeah, I think -- CHARLES: Or would the devices themselves actually be able to do that, like the Knocki could actually participate in the conversation about what other devices there were in the home. JOHN: Exactly, yeah. I think in a true peer-to-peer network, any peer can talk to another peer and eventually learn where the other node that they want to talk to is. You don't have to talk to any one particular person but you can ask anybody and they can tell you how to talk to the person you're looking for. The really big advantage to decentralization in my opinion is security. A lot of times if everything is controlled through one central point, that's one central point of failure. If someone DDOS's your cloud service, then now your entire network of devices is offline, just because one location got attacked. If it's a decentralized network, there's no one central point of failure and it's very, very difficult for someone to attack your network. CHARLES: Right, that's true but the tradeoff then is complexity that your decentralize network has to agree, somehow come to some consensus. It's very easy to generate with consensus when you have one process or one point that's driving everything. JOHN: Exactly. Another big tradeoff is ownership of the data and enterprise today are really big revenue your point for a company is being able to have ownership of data and extract meaningful insights. But if your device doesn't talk to your central server every time I want to do something, how does your server know everything that your device is doing and you lose a lot of that data. There's a tradeoff there in how you're going to get the data you need to run your business but also let your device run autonomously on decentralized network. ELRICK: Do you think that this is going to be helpful or harmful to IoT? What's your views on decentralization? JOHN: I think it could be very powerful. Right now, I'm not aware of any products that are really using a decentralized architecture for IoT and the main reason for that is companies and developers are a little slow to adopt it because they want to have that ownership of every data packet that goes to the network. They own it. They can see it. But I think in the future, people will start to realize that they can still get the data that they need to run this business. They can still have visibility and control over the network the way they need to run their business without controlling every single packet. When that happens, I think it's going to be a revolution for the internet as a whole but it's really going to revolutionize IoT and devices will get lower power. They'll get faster and they'll get more secure. CHARLES: When you say being able to get the data that they need, is it just being able to asynchronously spool off the data later? I guess I'm trying to understand how they get the data if it's never talking to some central servers? Or is it just you will get the data at the time you want it or there will be some delay? I assume you can also have your server being part of... I don't know. I'm just curious how you see that playing out. JOHN: I think, every developer is going to have to tackle that on a case-by-case scenario but take for example a big brand smart thermostat company. They have a device that's going to control your AC heating and air and the house and it also collects a lot of the data from when you're home and when you're using it to be smart and adjust the temperature at certain times of day even when you're not home. Again, I don't work for any company that does that and I don't know how they're doing their devices under the hood but traditionally, they tap to a centralized server and they send a lot of this information whenever it's happening, always to the server. Every time the user adjust the temperature, it sends an update to the server and says, "The user just updated." In a decentralized network, these devices can just talk to themselves and say maybe periodically or every day and it'll just send one update and say, "The user adjusted it." You can still talk to a central server but it doesn't have to rely on the central server. CHARLES: Right. It's just what we call, an out of band process. JOHN: Exactly, not mission critical. CHARLES: Okay, I got it. Talking about the decentralization and interacting with other devices, how do you manage the ecosystem right now with Knocki? It's a general purpose interface to rise. It serves really the role of a keyboard or a mouse or some way of controlling other devices and other systems. I assume that in order to do that, you have to understand the capabilities of those systems or maybe you don't. How do you integrate these two devices? Let's go with the thermostat and the Knocki or maybe one that you're more familiar with that you've done. Do you all have to write the integration? Can a third party write the integration? Or is there some way to automatically discover and map the existing inputs of the device. I feel like we've got all these new devices are coming out day to day then and now, there's more and more permutations in which to confine these devices into a coherent system and I'm just curious to hear about that integration story from your perspective. JOHN: Certainly. If we want to configure our Knocki to tap three times and turn on our Philips Hue light bulbs -- I keep using Philips Hue just because that's what I've been actively working on lately. We currently rewrite the integrations in our own backend so the user pulls up the mobile app and says, "Knocki on my desk, every time I tap three times, listen to this Philips Hue," and then we have an integration where in the mobile app, they can essentially set a lot of the parameters that a Philips Hue light would use based on API that Philips Hue would provide us. That's the way most integrations are going to happen with third party products. They expose an API and we can write a little module and the user can configure that API. CHARLES: I see and as far as making affordances for third party people, if they want to change the behavior or add like intelligence, obviously they can configure it from the app but if I want to say add behaviors or something like that. JOHN: When you say add behaviors, you mean add new -- CHARLES: I mean like, rather than turning the lights on and off, say I want to strobe the lights or flash the lights, maybe I'm someone who's running a theater or something and during intermission, I want to knock three times to flash the overhead lights. I don't know if that's something that your integration with Hue could do but if I want to be able to add that. JOHN: Okay, I see your question. We try to enable as much of the products functionality as possible through our own integration on our mobile app but say, you're a hacker and you've come up with your own smart light that turns on any sort of party mode and flashes different colors whenever you want and your Philips Hue or any other smart light just can't quite do what you wanted to do. In the future, our goal is to have an open API that people can access and they can hopefully control their own homemade IoT devices. CHARLES: Now, what about for existing ones. You can definitely flash the lights with the Philip Hue but you're going have to have some custom software to do it, right? Do you see what I mean? You have to send a series of messages to it in sequence. JOHN: In that scenario, we currently don't support that and don't have a plan to support that. In our research, that's a really small use case of people that would be interested in that. Also, it's difficult now if we wanted to do some sort repeated command, you knock three times and then every 30 seconds, it's going to send a command in your light bulbs. We have to be careful about having processes that run away and you have a bunch of CPU power forever in the cloud. We may include features like that in the future. I think the most likely path for that sort of stuff is we'll have an open API that people can direct Knocki's inputs to their own server and then their own server can flash their Philips Hue lights as much as they want. ELRICK: Is there any standardization between the communication and what these API supposed to look, like the communication between devices? anyone can have an API, expecting one thing and someone that's writing software to communicate with that, wouldn't have to go look it up. Do you know of any standardization? JOHN: Yes. I know there have been a couple of companies out there trying to put a standard on the market and I think a standard would be a great idea. ELRICK: Yeah, I think so too. JOHN: It would be wonderful if we could just write generic control structures or information flow structures and anybody can hook their stuff up to it. As far as I know, I haven't seen any that really fit the bill. CHARLES: It feel like there's something that programming systems like software developers have been chasing for a long time is to have some distributed set of peers that they can look each other up. You can discover the capabilities of a thing without ever having to even know about in the first place. But I haven't really known that worked really well and hit that sweet spot. I'm thinking of DCOM and Java. There's like Java distributed beans or something like that. You have this idea of these objects in the cloud, which seems kind of analogous to what we're talking about now, except we're talking about actual devices, rather the software devices but who knows. Maybe it'll pan out where we'll have some standard for discovery and integration. JOHN: It's interesting that there hasn't been one already. You look at IoT and it's really ripe for standardization because a lot of the communication between devices takes the same format. You're generally just passing a small message saying, on or off or, "I read this temperature at 75 degrees. Who knows, maybe someone will solve it. CHARLES: Yeah, maybe so. Maybe the folks at Kasita. They're active integrators. They were on the podcast two episodes ago and one of their challenges was getting all these 30 things to talk together well. Maybe we can follow up with them and if they could have a standard, what they would like it to look like? JOHN: If they get on that, I would love to hear what they were working on. CHARLES: I think, maybe they mostly, have a wish list. It is like, "I wish it did this. I wish it did this. I wish it did this." ELRICK: Maybe we need to have like a 10-way podcast. It's like IoT companies and we can hash it out like the TC39 of IoT on the Frontside Podcast. CHARLES: Right, and then everybody punches each other. All right, well thank you so much, John for coming and talking with us. It's always fascinating. You can find Knocki at @Knocki on Twitter and Knocki.com. It's a great product and like I said, always a fascinating conversation so thank you so much for coming on the show. JOHN: Yeah, thank you very much for having me. It was a great conversation. CHARLES: With that, we'll say goodbye. Thank you, Elrick. ELRICK: Thank you. CHARLES: We are, as always, the Frontside at @TheFrontside on Twitter, Frontside.io on the web or just drop us a line over email, Contact@Frontside.io. Thanks everybody.

Jason Jaynes: @jasoncjaynes Jeff Wilson: @ProfDumpster Show Notes: 00:53 - “Professor Dumpster” and Founding Kasita 05:33 - The Startup Industry 07:45 - Building the Kasita Team and Creating the Design 12:25 - Integrating Devices 16:33 - Challenges of Building These Ecosystems 24:36 - Controlling the Ecosystem: Will there be third-party developers and applications? 30:16 - Device Cohesion and User Experience 33:23 - Privacy Resources: Data for the People: How to Make Our Post-Privacy Economy Work for You by Andreas Weigend Kasita is hiring! Transcript: CHARLES: Hello, everybody and welcome to The Frontside Podcast, Episode 78. My name is Charles Lowell, a developer here at The Frontside and your podcast host-in-training. With me today are Jeff and Jason from Kasita. Now, Kasita is one of the most exciting products that I think we've gotten to work on here at Frontside in the last five years. We're going to be just talking about it because, I think it touches on a lot of the aspects of what makes software development and startups and just the emerging economy exciting. I'm really thankful that we get to have you all on the podcast. Welcome Jeff and welcome Jason. JEFF: Thanks for having us. JASON: Excited to be here. Thanks, Charles. CHARLES: Now Jeff, you are the founder of Kasita, the CEO and I believe your official title over there is 'Professor Dumpster.' Maybe you could actually unpack for us a little bit of what does that title mean? How did Kasita come about and what is it today? JEFF: A couple of years ago, I did a radical, social experiment around housing. I went and sold everything I own for a dollar an item out of a 3000-square foot house and moved into a 33-square foot used trash dumpster for a year. The idea of that project was to live in 1% the size of an average American home and try to use 1% the energy and water of the average American home. The project took a little bit of a twist, you might say and about part way through it when the dumpster started getting tricked out, I started thinking about the whole nature of housing and how we need to do something different and how that grand future probably would not be a gated community of dumpsters. CHARLES: Now, I assume you cleaned out the dumpster before you actually went to live in it. JEFF: Yeah, it was a fixer-upper. We give it a bit of a scrub and did some testing to make sure there wasn't anything nasty left in there. That went for about a year and a couple of months after that, I actually first set down with Jason because he was the only person that I knew in the entire startup scene, in the entire world. He said, "Wilson, you had some crazy ass ideas like this dumpster thing you told me about. This one might actually work, this Kasita thing." Here we are today, we're working together. CHARLES: Wow. This was something you just did on a lark. You didn't have the idea of starting this business but it was actually through the process of actually living in this dumpster for a year that the idea emerged or was there a master plan going in? JEFF: I don't know, Jason do you remember any kind of master plan when I first told you about the dumpster? JASON: No. When we first met to talk about the dumpster, it was an early morning, I believe in 2010 or 2011 and you're incubating the idea. At that point in time, there was nothing on your mind or you aren't looking towards the future of housing at all. You were just trying to figure out how you were going to move into a dumpster and people thought you would be crazy. Of course, I've validate it and I thought people would think you would be crazy. CHARLES: That is a pretty radical idea, the future of housing being 1% of what it is now. How do you see that playing out? How is that possible? How do you shift people's mindset away from that? JEFF: One of the bigger things we're trying to do with Kasita, there needs to be a massive shift in the wider way that we live in our homes. As everything else is moving towards on demand and as a service and as everything's being sort of productized, those are some of the core ideas behind Kasita. We think about Kasita a lot more like an iPhone or a Tesla than we would think about it as a single family home or an apartment block or even a micro-unit. That's why Jason and I are standing together here today is I represent a lot of ways, a kind of vision and origin story of Kasita but in a lot of ways, Jason represents the future of the software and integrated IoT that's going into these things. CHARLES: There is definitely a lot going into these things. I remember when Jason first started telling me about it because it is like an iPhone or a Tesla but, I think especially the Tesla is a great analogy because you have not just like a normal software or even really a hardware project, you've got architectural concerns. You've got manufacturing concerns. You've got, I assumed geopolitical concerns in terms of the politics around zoning and housing and real estate, all rolled up into a big startup. When I think startup, I think let's get a web application up and running and we're providing some service. This is cross-cutting at least five industries, it feels like if not more. I'm curious, what's been the experience in terms of wrangling that aspect because I think it is very unique in a startup today but it got me wondering is this going to be the normal in five years? JEFF: We've seen a movement recently in the venture community. Even a few years ago when we first started raising money was highly-regulated industries are hard, hardware is hard, "Thank you very much. We're going to go looking for our next two Stanford computer science dropouts to shove into a wee work and not have to deal with all of this kind of stuff." I think I've seen a shift to where people from the individual level up to the folks funding these things, see the massive opportunity in highly-regulated complex problems like housing and you're right. Jason and I are looking out over our shop floor here where we've got guys out there that are plumbers or traditional electricians all the way upstairs here to folks that have been mayor pro tem of large cities with PhDs. Bridging all of those individuals into a startup culture and then looking at the complexity of the landscape from a regulatory standpoint, autonomous cars are a breeze relative to the kind of complexity we're dealing with. CHARLES: Did you know this complexity walking in or was it a classic overoptimism? JEFF: No, it wasn't classic overoptimism. I'm always asked, "Are you a designer? Are you an architect? Are you a real estate developer? Are you a technology guy?" and I think if I would have been any of those besides a guy living in a dumpster, I wouldn't ever been crazy enough to try this. It's one of our core precepts as well. Jason had never worked with IoT stuff before. Our head of manufacturing used to build LEDs for Philips. Our quality guy inspected Cadillacs. Our manufacturing engineer built Boeing jets. The ideas that we're not pulling a lot of people from these traditional industries, we're pulling smart people that are passionate about our mission and to solve this, what is really a Rubik's Cube of a problem. JASON: Yeah, I think the other thing to add to that that Jeff is not getting himself enough credit is that from very early on, Jeff always looked at Kasita as a product that was going to incorporate multiple disciplines. He was very careful in how he orchestrate it and built the team to make sure that he was bringing the right expertise and the right areas together and then forcing those different disciplines to figure out how to meld and work together to build the Kasita. But the Kasita was from the beginning just about building a micro-urban home. It was about building a product of which part of that was a home, where people live obviously, but there's a whole lot more to it that we're working towards. I think even go back and Jeff, it might be relevant for you to talk a little bit about the approach that you took to just create an initial design for Kasita, which I think is revolutionary in itself. JEFF: A big part of our DNA was product from conception. When I was living in the dumpster, I recruited a couple of the top architects in the country really to help me turn that dumpster into a home. The way you're trained in architecture school, I think a lot of folks come in there with Buckminster Fuller kind of dreams and you're told pretty quick that you better bring things up to code and you better make things that sell or you're not going to eat when you get out of here. The idea was that we would start off with a product designer and not design a home. The kind of struggles in the dumpster taught me that we needed to go at a different approach so I went and recruited an industrial designer. One of the requirements for that person that he or she had never designed a home. This person had lived under a staircase and never designed a home so I said, "You're perfect." CHARLES: I like that and I'm curious, Jason from your perspective, what was it like to have gone through this? It sounds like what you're doing is asking people to bring their expertise but not their set of expectations like the industrial designer. What was it like for you coming primarily from the software development world to step into this pan-technological realm and what was that experience like and what were the things that stretched you and you found surprising? JASON: I think early on, I realized that it was going to be a bit more challenging maybe than I thought. Really, what it required was me to think outside of my discipline. Obviously, not only from the perspective of what we were doing on the IoT frontend, how we were melding software and hardware together but then going all the way over to the physical building structure and thinking about on a weekly, daily, hourly basis on how we are interacting with the other disciplines. An early example was, and this is one that I remember that's quite funny is one area that we wanted to make sure that we had covered in our research and understanding from IoT perspective was smart locks and how we were going to provide a smart locks for the data. We went out and did a lot of investigation, brought a number of leading smart lock solutions into the lab and tested them and narrow our list down. Then I recall vividly walking over to the architects to excitedly tell them we had selected our smart lock that we were going to use. They very quickly inform me that that lock wouldn't work because we needed a mortise lock and not a standard door lock. I realized that you can't work in a vacuum and just solve your problems. You have to be working together to make sure the solutions and the products you're selecting at work in accord with the overall design. That's continued to manifest itself. Every day, I'm down on the manufacturing floor, working directly with the electricians and others to make sure that our equipment is placed properly, where are we going to place our equipment, how are we routing around plumbing and pipes and other things that exist there and how are we locating things properly. It's an ongoing experience, which has definitely taken me out of my traditional software role but it's done so in a very exciting way and I've enjoyed it. It's just realizing that you have to actively be communicating across the organization with all groups and really, you can't take anything for granted. CHARLES: The number of different disciplines and technologies is really staggering, even if you limit it to just considering the set of devices that you're integrating. I was actually hoping we could talk a little bit about that. Now inside each Kasita, at least the ones that you're building right now, how many different devices do you have? How do you take all these different devices and turn them into a product or integrate them into something that itself is one product? JASON: If you were just to look at the technology bill of materials, what the products are that we're incorporating into our current Kasita design, there is around 50 different products and product parts that we're bringing together to build out the technology solution. If you narrow that down to what the end user is actually seeing and looking at, there are about seven noticeable products that the end user would see or they would recognize everything from a Sonos connecting amplifier to an Amazon Dot to a Nest Thermostat. Obviously, getting to that list of bill of materials and deciding on that 'subassembly of technology pieces,' took us quite some time in a number of iterations and a lot of outside engagement and talking to experts and trying to decide what were the best devices to bring in. But the other side of the equation was something that we kind of decided very early on in the process and kind of thinking the world of first principles was that, we wanted to make sure that Kasita was the primary interface to the user. We didn't want somebody else sitting between us and the end user. We wanted to be able to work with other products but we still felt at the end of the day that the end user, when they were living inside of a Kasita, when they were controlling the Kasita, when they were changing the state of the Kasita, they needed to go through our interface. With that as an initial first principle, you can begin to imagine that all the other parts of the system architecture and the way that we design things, the way that we select products and built things, it begin to derive themselves. Everything from that, immediately we needed an app and lo and behold. We were able, fortunately to work with you guys, the Frontside, to help us get our initial app concept up and going. It went from there and I can talk more about it. CHARLES: I think I really like that as a first principle. I really just want to inject a vigorous sense of agreement because I think it's so important, especially when this is the place where you're living. You want to imbue that inhabitant with a sense of ownership and control. I don't know if you would be able to do that if there were a bunch of different touch points and it didn't feel integrated under one product. In other words, this is my home, this is my Kasita. Is that the idea behind making sure that there was really only one interface? JEFF: We prefer to say 'Mi Kasita.' CHARLES: I love it. JASON: Absolutely, that's the idea. I think from a consumer perspective, if you've ever personally gone out and ventured through the halls of Home Depot or Best Buy and purchased some smart products off the shelf and brought them into your house and try to get them up and running, you very quickly learn that. It's not only challenging to get these devices connected in a way that you can control them but there's also this notion of there's an app for that. Every physical device you ended up putting in your how, has its own app for control and that becomes very overwhelming in a very short amount of time for the user. We did not want that to be the case with the Kasita. We wanted them to walk in the door from day one and immediately feel at home and feel like they have complete control of the Kasita, in much the same way when you go purchase an iPhone or you purchase a new Garmin watch or you purchase a new Android device, you're up and running with that ecosystem and you're interacting with that interface. We wanted people to be interacting with the Kasita interface to control their home because that's part of the product. CHARLES: I like that. It must present some unique challenges because I think you said it best. Every single device that you have comes with its own ecosystem and that ecosystem has its own APIs, its own web interfaces, its own applications and though there are walls around those ecosystems, what are some of the challenges you encounter in trying to punch holes through those walls so that you can hand information and control from one ecosystem to the other while providing a seamless experience to the user? JEFF: When you're talking about that, Jason one of the things that is often left out of this equation is at this specific point in space-time, it's very difficult to do that. But then to have any sort of semblance of planning for the future and future-proofing the system as developers usually call it, one of the reasons why you don't see a lot of Nest thermostats in multifamily development is because a developer knows that they're not going to ever have to replace a normal light switch. If it's a Lutron switch or if it is a Nest thermostat at some point, it's going to have to be replaced. Not only the physical replacement of the stuff but from a software side, making sure that we can continue to communicate with these devices in the future, I think is a big problem to solve. JASON: That's absolutely right. I think very early on, we recognize and realize that we were going to have to build software and a component that acted, if you will as a gateway for sitting between the end user and the end devices and facilitated the control of the end devices. Obviously, being able to accomplish that, one of the challenges is and I think, Charles you've seen this in your world because I know you've got experience with IoT is this whole proliferation of standards and protocols like if we're going to talk to the lightbulb or we're talking via Z-Wave or ZigBee, or do we have to go through a Philips Hue hub because that's the only way to actually communicate with it. Is there a separate way via Thread or Bluetooth you communicate with this device? In a very quick fashion, you get to this point where you can imagine that you've got a physical hardware controller that has four different radios in talking to four different device types. One for talking to Z-Wave, one for talking to ZigBee and it becomes overwhelming. We did a lot of research across the protocols that were available, mapping them across the devices. Early on, we were excited about the potential of Z-Wave but more recently, where we've shifted our attention quite honestly is looking for devices and device manufacturers who see the opportunity and Wi-Fi enabling their hardware devices and then providing either direct control of those devices in an IP-centric way over a local area network or even through the cloud. What that affords us back to Jeff's future-proofing concept is if you have Wi-Fi up and running and the device can get on the Wi-Fi network and there's a way to communicate with it, then it makes it a lot easier for us to sit between the user and that device and send commands and control that device. The other side of that, which I think continues to be a challenge and will be a challenged for the foreseeable future is a lot of the device manufacturers to the point that you brought up are still forcing you to go through the cloud to communicate with their devices. They don't allow for a local area network communication directly with the device and there's good reasons for doing that. But what that means is if you lose internet connectivity, you no longer have control of that device. CHARLES: Obviously, you've got probably pretty strict criteria about what it takes for a device to be integrated with Kasita. Is that a nonstarter right there? JASON: It's actually not. A nonstarter with be the device communicates via protocol that we can't interface with or the device works over a Wi-Fi network but has no API for controlling cloud or local. The third piece of that equation and fundamentally is the final nonstarter and really probably should be the first one and it's one that we take into consideration every time is that there should be a physical override for the user if internet connectivity is lost. What I mean by that is if we select a smart switch and the smart switch goes offline and there's no more connectivity, the user has still be able to walk to the wall and press the power button and the light should come on. There always has to be an ability for the user to fall back to the same old fashioned physical control in the absence of Internet connectivity or local area network connectivity. But the primary things are ability to fall back to physical control, ability to communicate over Wi-Fi or standard IP-based protocol, then the third one would be some form of API access, either remotely via the cloud or locally via the local area network. CHARLES: Wow, that's actually a great list. It's got me wondering, obviously you've encountered devices that have fallen on both sides of that divide. Do you feel like that's just a blip and we're going to be trending more towards devices that are happily and easily integrated or are we still seeing some moving and jostling as people maybe try and corner little parts of the market and make their device deliberately make it not easy so that you'll try and force people into that ecosystem? JASON: The latter, however we have two guerillas in the market right now that I think are helping drive the other direction in the way of Amazon and Google with Google Home and Amazon Echo. What they're doing is they're saying, "If we sit in the center and one of the interfaces for voice control for the user to control their home, then we're only going to work with devices that we can communicate with and that we can control through the cloud," and quite frankly, what that does is it puts the burden back on the device manufacturer. You could actually say three if you threw Apple in there. I don't want to leave Apple out with HomeKit. But my point is that the device manufacturer now has to find a way that the end device can either communicate via standard TCP/IP network-based connectivity that we all know and love from a developer community perspective or they have to insert a hub into the equation that can handle that form of communication and then communicate over its own proprietary wireless connection, which is in the case of Philips Hue, it's exactly what they do. JEFF: I would draw analogies here to some people get really tired of this, particularly the real estate people of me talking about the iPhone but that kind of leap into and integrated piece of hardware and software. There were certain things happening in 2007 that didn't make the iPhone or something like it, something that might happen but something that had to happen. This kind of cold death to the universe that we could see with all of these walled-off ecosystems, go in their directions and iterating into a space to a nobody owns anything and nothing talks, I think Kasita is a solution to that to where we're looking like combine all this stuff under one roof and build a single user experience, much like not having to pull your Palm Pilot out of one pocket, you're Rio MP3 player out of another and you're your Razor or whatever it was out of the other like integrating into a single experience, rather than a sort of convenience, which is what a lot of the IoT spaces right now in these walled-off ecosystems. CHARLES: That actually makes a lot of sense and clarifies it in my mind quite a bit. It clarifies one thing but then, immediately raises new questions. When the iPhone first came out, you had a set of basic integrations between your MP3 player and your web browsing and your calling and calendaring, so and so forth. Then, I don't know what was it like, a year and a half later, they actually came out with an SDK so that you could actually develop apps -- third-party developers could actually develop. Sell and distribute in apps -- to the iPhone. We're all really happy with the way that worked out. I guess my question is does this analogy carry forward then also for Kasita? Is there a future where you have third-party developers who are actually selling integrations or apps that would run on this integrated IoT product that is Kasita or am I stretching the analogy too far? JASON: I think the analogy is good with the exception that we're not looking to control the entire IoT ecosystem in a way that Apple maybe had look to control the mobile phone ecosystem with providing all of that in one box and the iPhone. We want to work with numerous hardware providers and even from that perspective, numerous folks that want to provide interfaces into our system. As we develop an architected Kasita technology system, we've taken an API-first approach and that's allowed us to build our user application layer right on top of that API but in the future, we see the opportunity to work with third-party developers to extend that, up on that and build their own interfaces to the end user. Then on the other side of the equation, if you think about what's actually controlling the devices, we're architecting that system in a way that a hardware manufacturer could take an SDK and add Kasita support for their product directly in and make it plug and play when it gets to the Kasita. We definitely see the opportunity, Charles to reach out and allow everybody to be part of this. We consider it quite frankly, a necessary thing. But we don't also want to pretend that we would look to control the whole ecosystem because we just don't have that level of scale, if you will. JEFF: And you know -- CHARLES: Not yet. JEFF: Yeah, and we try to keep our ego in the dumpster, so to speak as well. CHARLES: What would a third-party app even look like in the context of Kasita? Have you thought of like what are some things that you might be able to do? JEFF: If you don't want to call it directly an app, I think the first stage -- Jason and I haven't talked about this -- maybe more like an Alexa Skill to where you can have the Kasita do certain sets of tasks around a particular experience, which we're already building into the system the idea of moods but I don't know in terms of apps. JASON: Yeah, it's actually a really good idea. Even though we haven't talked about it, it always scares me a little bit when my boss is coming up with ideas on the fly that we have to implement but -- JEFF: But actually we will have our first -- we're going to call it a skill app, a Kasita skill app. We'll be releasing that say, October 1st. CHARLES: You heard it here first, folks. JASON: To take Jeff's idea a little further, I think that is an interesting concept when you think about the Kasita as being an end product and you provide interfaces whether it's the ability for people to write skills that tie into the Amazon Echo or an IFTTT-type capability. The Kasita, as a whole can be controlled -- all the lighting, the sound, all the different temperature, etcetera -- so now you're asking end users to write skills, to control the entire state of the building or of the home and not just doing it on a one-off basis writing skill to turn this light on and off or set the thermostat to this level. You basically box all of that together and make it much easier for people to get from Point A to Point B through our system. JEFF: Could you say that we're turning the entire Kasita into a board for people to play with, like treat the Kasita as your breadboard? JASON: I think there is some opportunity for that to the degree that will allow the user to have that much flexibility on the hardware side. I think it is still up for question but I think there's a lot of opportunity there, Charles and not only inside of the Kasita but then you can begin to see other applications as Kasita begin to multiply and people use them from many purposes. Let's take a sample of somebody owns 10 Kasitas and they use them as Airbnb properties and they allow users that live in Kasitas to come in for a short period of time into their Kasita and bring their Kasita profile with them. Immediately, they can make the Airbnb Kasita feel exactly like their Kasita feels when they're at home. Those are some interesting opportunities and ways that we see this technology potentially evolving. CHARLES: So it will have the same moods, the same behaviors. Any customizations or third-party extensions would also be in effect provided they were software-based? JASON: Yep. CHARLES: That would actually be quite amazing. I guess the other question I have in terms of hackability of Kasita is we're very interested in the IoT space and very interested in these products and we have some side projects here at Frontside also like I do a bunch of hobby stuff at home, where I try to integrate a bunch of these things. But one of the things that I really like about what you all are doing is that it's very much 'omakase' in the sense of there's an option of 10 smart locks, there's an option of this thermostat, there's an option of a million different devices but what we've done or what you've done is selected ones that we know are going to work well together. We've built the software, the control systems, both computer control systems and human control systems to get them to work together as a cohesive product. I would love to do is say, "I would just like to buy that product for my house," even though my lame tinkerings with smart switches, smart locks and audio controls and lighting, which are fun and gratifying the first few times but they don't really play nice together, give you that super sweet feeling. JEFF: This goes to the overall philosophy of Kasita. We want a turnkey, one-click housing solution. Not only for finding you a place to rent so that you're not fishing around on Craigslist for roommate or having to pay some outrageous fee in New York. You don't have to go mattress shopping. At some point, you should just have to show up with your iPhone and your toothbrush. When you start thinking about the technology inside, it's almost like folks don't really care what kind of Foxconn chip is in their iPhone or even if it was Foxconn that put it there, they just want it to work and they want it to be seamless and turnkey. It sets up a whole philosophy around, not only our smart kid in the Kasitas but it shouldn't even be a smart kid anymore. At some point, it should just be an experience so ultimately, what sort of UX inside of the Kasita are all of these things bringing you. I shouldn't have to really look at a blue glowing dot that lights up every time I walk by it to be at a comfortable temperature in my house. I shouldn't need a black tube over on my desktop that I yell commands at. I just talk or it should anticipate those actions. That's a future that I look forward to in Kasita to where we move away from having to tinker with devices and even knowing what those devices are to a true-like depth of experience. CHARLES: I like that a lot. Now, one thing that we haven't covered. We touched on it a little bit at the very beginning of the show when we talked about people feeling in control and feeling like they're truly the owner of the space is the issue of privacy. Obviously, there's a lot of a user's behavior that's going to be passing through software channels as their intentions move through the devices in the Kasita. Of course, all of these devices, they have their own ecosystems, their own vendors so how do you ensure that people's data is going to be protected, especially as it moves through potentially a bunch of different public clouds. JEFF: Yes, we gave a lot of thoughts to this. Actually, Jason put me on to this book called 'Data for the People' by Andreas Weigend. We took some inspiration on that, from that and set out on what we call it the four cornerstones of this future of the connected home. Those are agency, transparency, security and then the actual benefit that you get from this home. I gave a talk at South by called, 'The final frontier of AI is in your living room.' If that isn't black mirror, creepy enough to attract enough people, I don't know what is. In that talk, I won't take them out of order. First, we need to make sure that we're focused on transparency. Do people know what's actually being collected on them? I've been toting around my iPhone for 10 years. I'm pretty sure they know everywhere that I have been since then. I'm not really all that sure. Second, agency. Can I actually do something about it? Are we allowing people the ability to switch off, switch on, control where that data goes? Then third, security. Are we providing another level of security above what you would get out of the box? I'll let Jason talk about that in a minute. Then, the last is benefit. Am I getting ads? Am I getting a slightly better news feed focused on ads or am I getting my rent subsidized? Am I getting a better user experience, better sleep within the connected home? Those are the ways that we think about that in a bigger level. CHARLES: Is the idea that there's no benefit than it's exploitative? You want to make sure that there's benefit? JASON: Yeah, I think that onus is if you taking individual data and using it, then the onus is on you as a data collector to try to provide benefit back to the end user. If you can't do that, then I think the question should be why are you collecting the data in the first place? our goal is really looking at it from the perspective of if we know when users are turning lights on and off and what they're setting the temperature in their house to and when they're going to sleep at night, when they're waking up because we know when they turn everything off and turn it back on -- JEFF: Or where this things on the floor are from the vacuum robot. JASON: Yeah, exactly. If we have insight into that information, how are we taking that information and combining it in a valuable way that benefits the end user? I think that's the first question that we have to ask when we start looking at the data that we're collecting. But at the same time as Jeff said, that data collection really has to be based on this notion of agency, transparency and privacy or security. An agency is simply I have control over whether my data is collected. Transparency, from the perspective of I understand how my data is being used and where it's being sent and then of course, security, I know that my data is being securely transmitted and stored. When you think about security, we spend a lot of time thinking about not only the data at rest -- once it's been collected is it properly being stored and encrypted and protected -- but then how is that data being transmitted and are we putting the proper fail safes in place to make sure that somebody else can easily gain access and take control of my home and of the things that are important to me by finding back doors into the system and ways to breach them? Those are the cornerstones that we think about and we put first and foremost in our mind as we build out our architecture, build out our system and as we begin to take that data and to turn it back in useful and interesting ways for the end user. CHARLES: I think that's really important. I think it's a great comfort to hear that you all have a framework for thinking about this so that it's going to be integrated into every aspect of it. I think it's just so important, especially when it's something as critical as the space in which you're living. It's good to hear that it's not just an afterthought but that it's something that's been integrated from the start. Well, Jeff, Jason thank you guys so much for coming by and talking with us. I really think that Kasita is an exciting product and I think that it was an exciting project, certainly for us to get to work on, even though we were only seeing a very small sliver of it. We still got to perceive the whole enchilada that you guys were working on and see that just what a unique startup that really is, not just you're moving outside of software, integrating a bunch of different devices, integrating that with a unique home that's going to be designed, architected, manufactured and then thinking, then even rolling it up a degree further about how is this going to be integrated into the urban spaces in which we live. I hope that we see more startups that really engaged all those different disciplines. I think that with the technological changes that are happening, that's more and more a possibility. The price on software, the price on materials, the price on these smart devices is all coming down so it really enables people to take on scopes that might have been just completely impossible, even with someone who's overly optimistic. I hope that people look to it as an inspiration and it really was a great project for us to work on. I also understand that if someone does want to jump into this space and get involved, you all are hiring. JEFF: That's right. We are hiring for a broad range of positions. We're expecting to be doing a lot, more hiring soon. You can go to Kasita.com/Work and at the bottom of the page, you can also see that we have an open house here in Austin every Thursday morning from 9:30 to 11:30. The folks can come in and check out the crib. CHARLES: All right. Fantastic. I certainly really enjoyed getting the tour the space, what was that? Back in March? When you revealed the baby units? JEFF: Yeah, it was March at South by. CHARLES: Yeah, it's really something to see. If you are in Austin or you live here, take the time, go see it. It's really cool. With that, I guess we'll wrap it up. Thank you everybody for listening and as always, you can get in touch with us at @Frontside on Twitter or Frontside.io or send us an email at Contact@Frontside.io. Thank you all and see you next week.

Drew Covi: @drewcovi | about.me Show Notes: 01:04 - Honeywell User Experience (HUE) 05:00 - Deliverables 06:55 - Being a “Devsigner” 17:26 - Flash and Leading to Unique Skills 30:00 - Advice for People Straddling Roles 35:27 - Leveraging Design and Development Skills Together 39:41 - Embracing the Hardware Element 42:05 - Why the “Devsigner”? Resources: AOLpress CSS Beauty CSS Zen Garden Contribute Crave Transcript: CHARLES: Hello, everybody and welcome to The Frontside Podcast, Episode #76. My name is Charles Lowell. I'm a developer here at The Frontside and your podcast host-in-training. With me is Elrick Ryan, also a developer at The Frontside. Hello. ELRICK: Hey, what's going on? CHARLES: Not much. Are you excited about today's topic? ELRICK: Very excited. CHARLES: Yeah. You got a personal stake in it because today, we have in the room, not only you but also two developers who are also designers or designers who are also developers. Our guest today is actually the first person who fit this description that I ever worked with. It was a great experience, a great collaboration and his name is Drew Covi. Drew is a senior supervisor of product design at HUE Studios in Golden Valley, Minnesota. DREW: Howdy. How are you doing? CHARLES: Good. Thanks for joining us. Now, you're going to have to explain to us two things, one, what is a super senior product designer and let's start off talking about HUE first. What exactly is HUE because I think it's a cool organization? DREW: I'm working with four people and I'm working on all sorts of brand new ideas. I think the greatest opportunity that I've had in my career at this company, Honeywell is just working with physical product and the digital space. It's a unique opportunity. Not all companies focus on both so it's really been a learning experience for me and working with a great group of creative individuals is also been a real privilege. They say that at the end of the day, the most important thing is other people that you work with and really the entire team here has been fantastic in welcoming me and letting me explore and grow as a developer and as a designer. It's been great so far. CHARLES: Fantastic. Working with that group was absolutely wonderful. What does HUE stand for? DREW: HUE is Honeywell User Experience. Our previous CEO, Dave Cote often called it 'huey' but it's just HUE, without the Jersey accent. I'm going to probably misrepresent but we have over eight to 10 studios throughout the world. Each one focuses on different businesses for the most part. The one here in Golden Valley tends to focus on homes and buildings technologies. The studio out of Seattle, actually tends to focus on, again I'm going to get the acronym wrong here but it's essentially worker safety in industrial safety. CHARLES: What is it that you all do at HUE? DREW: What we do here at the studio here in Golden Valley is we support various businesses throughout the homes and buildings technology space. About fall of last year, Honeywell went through a bit of a shift in their business and they used to do all automation control solutions. Last fall essentially, we saw that one large business that was headquartered and based out of Golden Valley, break into two areas of more direct focus. Out of Seattle, we have folks working on, I think I mentioned before but Seattle works on sensing and productivity solutions. We focus on homes and building space so we're both providing upfront research to understand what the customer needs. We're actually creating everything from very rough user flows to final UIs and we're also working with industrial designers to create final products. Those industrial designers work very closely with engineering. Honeywell has a long reputation of very strong engineering when it comes to the hardware space. We've prided ourselves on excellent instruments and excellent performance. One thing that very few people understand is that we don't just do thermostats. We're in the business of turbos. We're creating the turbos for your car. We're creating all sorts of HVAC equipment. We're also handling various safety equipment. All of these items need designing, not just for end users and consumers but they also need designing for the workers in the field. If we make a product that is more efficient, easier to use and in some cases, more attractive, not only it does lead to more sales, it leads to more efficient work forces that can work quicker essentially. You could get up on a roof and get off in record time. We're not just designing consumer products. We're actually focused on a lot of other items as well, with oftentimes very large returns on investment. CHARLES: In the work that you do and HUE does in general, it sounds like there might be a large software component. Digital design is kind of we know in the web space but then also a lot of industrial design of just how does this thing going to look, how is it going to feel, how is it going to persist, how durable is it going to be, how is it going to withstand usage. Would you get involved in that process? DREW: Usually, the entire organization gets involved with the process very early on. One of the other shifts that happen in the fall as we get involved less in the production and more on the actual marketing side, like marketing deciding what's going to be built. We're actually really at the beginning and understanding what problems need to be solved at first. As far as my practice and my skill set, we do get involved with all that discovery phase work but when it comes to actual deliverables, we oftentimes see our deliverables around the actual creation of understanding user interactions. We will take research from our user research in OVOC, which is an acronym for Observational Voice of the Customer and we'll take those learnings and translate them into whatever solution we decide to build as a team. My output is going to look like a user flow, something you build in OmniGraffle or Visio and then it can start there, which is in the physical space and then we'll actually revolve those concepts into wireframes as well. Wireframes that will then be handed off to other team members who specialize and focus on visual design. Basically, it's kind of a very hands on process from the very beginning to the very end. It's essentially just understanding everything from the physical to the digital. CHARLES: When we were working together, at least in your case, it doesn't stop there. You're actually doing a significant amount of the implementation as well. Let's explore how did you actually end up getting to that position where you were working through interactions, wireframes and workflows and then also, getting to actually build the product in the form of a complex single-page application. DREW: Sure. Absolutely. One of the components that I kind of brought here to the team was a bit of a deeper understanding of frontend web development. I'm often pulled into conversations here and there. In the case of the project that we were working on specifically, it was essentially kind of early days on that project. We had a product that was pretty old and need a lot of work and it was basically, need to be rebuilt. We hadn't seen a lot of single-page applications at that time. In my case, I actually had worked on a couple small projects in my previous job and we can get into that in a little bit, where my career path took me. But essentially, it was me trying to kind of pave the way and eventually have that work scale. It was kind of proving that it could be done, showing how it could be done and then getting other developers on board. My role here has oftentimes involved, basically becoming a liaison between our design teams and our development teams. Ultimately in this case like you mentioned, it did wind up in turning into code that ultimately got factored into production code. It was definitely a time where we were experimenting with what role we would play. I will say in full disclosure that more or less which we're trying to move towards, basically making better informed decisions but not playing as much of a role in actual production code writing. It's something that we want to help scale. I think we'll talk about that kind of role and how well it scales hopefully in a little bit here but ultimately, it kind of changed a little bit. I don't do as much code as I used to. CHARLES: Right but nevertheless, the skill is there. Don't sell yourself short. You weren't slapping together a bunch of jQuery plugins. You were standing up, basically a full stack system with a StubDeck background, then Node.JS. This is back in early days where there was a custom-build tooling. You were using CoffeeScript. There was a lot of exploration and clearly, there is a fierce curiosity which you are actually exploring and actively kind of skinning and moving into the development space, which doesn't happen until people achieve a certain level of comfort. Whether or not you're exercising those skills, I think they have served you well in terms of the things that you've been able to build but also acting in that liaison and understanding what's possible and stuff like that. Obviously, once I met you, you were already there. I'm curious in exploring that journey of coming up the design ladder but also coming up the development ladder too. Maybe we can talk about each one separately and then see how they intertwine. Let's start with the design side. How did you get into that? DREW: I can take you way, way back. I love to talk more about this in a little bit but I think we, as a generation, are kind of very unique in that. We were raised in the birth of the internet. Some of us are old enough to remember the early dial up days and I certainly was one of those. I grew up basically obsessed with drawing and art and painting. I was a designer and artist raised by an engineer, essentially. My dad didn't really have a lot of opportunities to explore his creative side to basically make a living. I want to say that although graphic design existed to a certain extent, there wasn't really the same blend of engineering skills required so he decided to take the tack of I'm going to become an engineer so I was raised in a household where he was building everything but he was also a talented artist. As a kid, I basically did a lot of advanced art classes. I'm kind of a nerd, pretty much a huge nerd. I dropped my entire tenure as a high school student. It was also kind of dawn of video games as well so we had computers coming of age. We had video games coming of age so I was raised looking at digital art effectively, 8-bit, super accessible. It's kind of so early on that it was something that I could actually fathom getting into and creating on my own. I never got to creating any games but I will say that by my late high school years, I was using a tool called AOLpress. For anybody who has ever heard of that, congratulations. You're one of the few. CHARLES: I've never heard of that. AOLpress, we're going to have to link to that in the show notes. ELRICK: I've never heard of that either. DREW: It's awesome. It's got a Wikipedia page. It's got hieroglyphs and stuff. They really went all out on this product. It's basically the precursor to the Dreamweaver. It was a very, very WYSIWYG. I'm sure you've heard of Microsoft FrontPage, maybe. It was basically a precursor to FrontPage, I would say. Same thing, those are the days of framesets and all of that. I was a kid in scouting at the time and I wanted to build a web page for the troops so I built one and put it out there. I kind of remember that moment where I was like, "I'm going to write something and put it on the internet and anybody can see it." That whole experience was just super exciting. I know that if anybody's following Kickstarter, there's one that was started called 'What Comes Next Is the Future.' It was made by Matt Braun and Matt Griffin and it really explored the birth of the web. I would recommend it on your listeners to want to really dive deep if you didn't live through it, check it out. It's a great, great film. All the regulars are there as you'd expect. Zeldman on there, talking about it amongst others. But if it were for the web, I don't know that I would be who I am or where I am today, just because it's such a unique platform. It's so open. It's so readily available. There's no barriers. I would say that I was just an arts student in high school that picked up AOLpress and then got addicted to the web. From there, it was kind of off to the races. In fact, I didn't even know that I could make a living as a graphic designer until late high school. I decided that I wanted to go to school for graphic design, went a year at the University of Minnesota-Twin Cities and at that point in time, it was pretty much all print design and then Flash. Flash took over in my second year and at that point in time, it was Flash and framesets and tables. There was no CSS for layout. It's very early days. It sounds like you might know what I'm talking about. Have you been there? ELRICK: Yeah. You know, they say everyone in the world has like a twin and I'm like, "Drew is like my technology twin." DREW: Yeah. When we were raised in that time and we had to hack it with framesets and whatever tool -- FrontPage or AOLpress -- you basically, from very early days, realized that you had to force this stuff to happen. It was not easy. There was no documentation and where there was documentation, you were grateful to have it. I remember when I was, probably just about to graduate and if I look back at my portfolio piece, it was definitely still Flash. It was Timeline-based Flash. I also think that in many ways the way the web evolved was perfect. As a designer, I was very comfortable in the Timeline tool. Before ActionScript 3.0 and before they went on object-oriented on us, it was super accessible. You could add little bits of code here and there and create animations. It kind of got you hooked. Then suddenly, I found myself needing to create full screen Flash applications and needing to actually write code. I actually having to say, "If I want this Flash experience to scale, then I need to calculate where things go. I can't just X-Y coordinate and done," so that's where I jumped off and started getting into CSS. CSS was kind of early days as well. Again, this is before iPhone. This is like people were using CSS but people didn't really think it was that important. It was actually kind of discouraged because everybody in the world was using Internet Explorer and why would you need to know CSS. It was unreliable for different browsers and Internet Explorer was the worst. I remember sitting in a Dreamweaver conference, when it was Macromedia had a conference and they showed a webpage and then they hit the print button and they said, "Does anybody here know how this happened?" because the layout had changed, everything looked better and different. It was perfect for print. I remember my hand shot up because they was like, "Nobody was really familiar yet with that print style sheets?" Incidentally, I don't think that people still are familiar with print style sheets but it was a time when finally people were starting to understand that style sheets were more than just a layout tool. You could change them for all these different form, factors and all these different platforms. It was a fun time to be coming up in this age. CHARLES: It sounds like one, CSS and two, Flash were actually kind of gateway drugs into the development world? DREW: Absolutely. CHARLES: We still have CSS, clearly but do you feel like Flash, despite what some people might think about it, it was a full virtual machine that was running. You could code on it with ActionScript. It's kind of like the JVM but only for running inside the browser. Do you feel like designers might not have that gateway available to them anymore or maybe is the web just as big of a gateway to move into that? DREW: Yeah, for sure. I certainly think, beyond a doubt that had it not been for Flash, we would see a lot less creativity in the space. I say that only because at the time, if we had just gone from tables and tried to slowly evolve things, we'd have a much different feel, I believe. Certainly, it's a gateway drug. We'll be in a different web today without it. Is it still required? Are there any equivalents? I've seen a number of drag and drop web UI on the web tools out there and many of them claim to create production quality code. It's certainly possible to get there without Flash. I think, it's certainly its time has passed but we do see tools like Sketch for instance. These are all very much screen-based design tools that seem to leverage a lot of the same web styles and the web approaches. I think we definitely have the tools there to replace Flash. But I think from my perspective, it would be very interesting to go back and imagine, would we have immersive full screen web experiences without that Flash? CHARLES: Yeah. I remember it being very much a topic of conversation, certainly at the beginning of each project or when you were going to implement a feature is, "Are we going to do this using Flash? Are we trying to do this with native HTML? Are we going to use EGADS or Java applet?" ELRICK: Oh, man. Java applets. CHARLES: That was a conversation that was had before the web eventually went out but I think when it was, everything was very, very static. I do think that Flash definitely set the expectation higher and forced the web to evolve so that it could be the natural choice in those conversations. ELRICK: The time when Flash was around, I called it the 'golden age of user interface' because you can literally build any user experience, any user interface with Flash that you could dream up. There was no limitations creatively in the world of Flash. Nowadays, we're kind of limited without box model but it's getting better year-by-year. DREW: It's interesting to me because before Flash really died out, we had these... Let's put it this way. I feel as though, for a long time the web was a very much like a poster site kind of approach. You would have tools that were pretty rough on the eyes, pretty hard to use and then like for certain films, you have these very high budget, fully immersive Flash experiences. For a blip, that did actually translate at some point into Canvas-based and then Three.JS, like 3D WebGL-based experiences in native HTML but I don't see a whole lot of that anymore. It seems as though, it kind of settled down and in many ways, I would say killing Flash kind of evolved the web from more of a presentational platform to more of a usability first platform. It was a bit of a double-edged sword. You could build anything you want like you said but there wasn't a framework to it. It wasn't really responsive and then certainly, when Steve Jobs decided he wasn't going to Flash an iPhone, that was the end of it. Essentially now, we have -- ELRICK: Steve Job dropped the hammer. CHARLES: That was the memo that was heard around the world, right? DREW: Yeah. CHARLES: I just realized that was like 10 years ago. DREW: Yeah, they're celebrating the anniversary for the last couple of months here. It's been a huge deal. CHARLES: There's probably listeners that never heard that memo but it's definitely worth a read. The memo obviously, that you guys are referring to is when Steve Jobs basically said that Flash would not be on iPhone or iPad, not now, not ever. That was the end of it. DREW: People often forget too that when it was first launched, there was no app store. He basically said point blank, "Anything you need to do on this phone, you should be able to do using the web, using native web coding," and Safari at that point in time is really paving the way to bringing those native APIs into the web. You had geolocation through web. In many ways, that too is a huge gateway drug. Suddenly, you start looking at the web, not as just like, "I could use this as a poster site or as an informational site or a new site. I can actually use this to get things done." They're actually treating this platform as a first-class citizen. That to me was super exciting. I don't know if it gets as much attention anymore in the days of Swift and the App Store but I will say that if your listeners do get a chance to check out the show I mentioned earlier, 'What Comes Next Is the Future,' they even dive deep into just how limiting the app store experience can be. At least with the web, you can create whatever you want to create and people seemed to go that you URL and install on their home screen. This is a feature that nobody uses from what I've seen but if you bookmark a web app on your home screen, you can have an icon, you can have a loading screen, you can have all this stuff and nobody really uses it for whatever reason. CHARLES: I think it's the install, it's getting the knowledge about the fact that you can do that. It's not widely disseminated. ELRICK: Yeah, I think its capabilities starting to come up now with people making progressive web apps. They're starting to utilize that being able to put icons on people screens and loading screen and splash and etcetera. CHARLES: Flash really was kind of the gateway into the development world. I'm curious what opportunities do you feel opened up as you started taking on more web technologies, more JavaScript, more CSS and mixing that with the design that you were doing? What unique skills/superpowers do you think that gave you, that made you, that helped you at that stage in your career? DREW: Yeah, for better or worse, it really was the opportunity to get a job first of all. I know that the job market has been in all sorts of flux in the last couple of decades but I would say 12 years ago, in 2005 when I was entering the workforce, graphic design was not necessarily a hot field. I can say with relative certainty that the majority of the people I graduate with, didn't necessarily make their way into graphic design as a profession. I would say probably maybe 30% to 40% actually wound up following their degrees. For the obvious reason at that time, we were starting to see digital replace print. It meant that I was able to get a job for one. It wasn't a dream job necessarily but I was basically a one-stop-shop. I was designing and developing websites as working for a company but in many ways, shapes and forms, I was kind of freelancing as things were. I had a very direct relationship with the clients that I worked with. It was basically churning out websites. If I recall correctly at the time the company wanted to essentially create a Domino's Pizza of the web where we could use CSS to essentially build the actual HTML once and then restyle it. This is actually was a time when a site called CSS Beauty was just coming of age, I think the site still exists but back then, if you want the CSS Beauty, it's big thing was you have one website and people could upload their own CSS and completely change the layout, completely change the look. CHARLES: Are you talking about CSS Zen Garden? DREW: Maybe that was it. There's two of them. CHARLES: I remember that one. DREW: CSS Zen Garden was one of them and I think CSS beauty was a clone maybe of Zen Garden for sure. Maybe you're right, Zen Garden was the one where you actually had a website and Beauty was just showcasing certain CSS sites. I think you're right. Zen Garden was the one. When they saw that, they're like, "Wow, business opportunity. We can build a whole site." We were using something called 'Cold Fusion' and... Oh, it will escape me now. I think it was called 'Contribute.' There's a product called 'Contribute' that Macromedia come up with that worked on Cold Fusion. It was basically a WordPress. You basically set up editable regions, you basically code the site once in that regard in the backend coding and then just rework CSS to create multiple sites. Actually, the opportunity to open up for me, that job was very squarely-focused around the benefits of leveraging CSS. Eventually, that grew tiring. I kind of wanted to get into the actual marketing and advertising space. From there, I started to just jump to the next job. I worked for a very, very small marketing agency. It was called 'Vetta-Zelo' at that time and we focused on lots more Flash, a little bit of CSS websites but mostly Flash Experiences and they actually used Flash in a lot of kiosks and physical spaces. I started to jump into that, understanding PHP, understanding databases because we would do things like we would install Flash Experience on little portable tablets that would then sync up survey responses to a web URL that it would then dump it into a database. About that time, I was always trying to teach myself how to get really deep into the backend of the stack. CHARLES: That was just to make sure that these Flash sites that you're developing would be scalable and more robust? Was that the natural next layer to dig down? DREW: Absolutely. At the end of the day, we wanted to have immersive Flash experiences and we wanted to have the content easy to update. I would build these really crude backend with text areas and they would update a database and then the Flash Experience would pull that in as content. In that way, we didn't have to go in and re-publish the Flash every time, essentially. It was a much more streamlined process. I think we even gave some of our clients the keys, gave them a login and password and they could change certain things. There's an outfit around here called 'Crave.' They are a restaurant in town and we built the website for them -- one of the earlier websites. When you have to do things like update times and menus and things like that, it became pretty essential to having some sort of a CMS behind it. It was all based on necessity, in other words. What you said is absolutely true. We had to evolve what we learned and I had to push what I did to lever on different needs. Throughout my career, I've been the guy who does web and design. One of the things about that is it's kind of a lonely place to be and find yourself in creative agencies, where the majority of skill sets are not in development and trying to explain what's going on or make commitments on timelines and deliver on them. Whenever a bug shows up, it's never really fully understood. It's also a challenge to manage expectations, certainly as a young professional at that point. CHARLES: Yeah, I would say, what would be some advice you would give to somebody who is straddling these roles at that early career stage where they're maybe working for creative agency and fulfilling these two roles but most of their surroundings is towards the design end. DREW: Yeah, I would say for the most part, just be upfront. If there's anything that's unknown, be upfront about it and explain. If you are early in your development career as a designer, do your homework before you committing any commitment certainly. I think it's always better to be upfront about these things than to try to over-promise and then scramble at the end. I will say that a lot of my career has been marked with the term code 'code cowboy' as a designer and teaching myself to code. It was a disparaging term, I guess. I didn't really necessarily take it that way but I think other developers are trying to use it in that way. CHARLES: [Singing to the tune of Mammas Don't Let Your Babies Grow Up to Be Cowboys] Cowboys ain't easy to love and they're harder to hold... ELRICK: It's so true. DREW: You know, I'm not even embarrassed to say it because the truth of the matter is when you're a designer, you're used to just making a mess before you kind of landed on what you're done and what's right. The entire creative process is messy. I think it's inherent. If you're one of these designers turned devs and you basically just hack it until it comes together, that's kind of a natural flow from the creative process. Certainly, as you get more experienced, you want to reduce all that uncertainty and potential for error so you do learn to hone your craft, to use version control, to embrace a framework or embrace some model-view controller approach but none of that really existed in the early days of the web. I kind of came up in a time when you had to hack it. CHARLES: Well, there's a lot of learning that can happen when you're hacking and building things that are kind of ad hoc. As you go, you get to perceive firsthand the problems with them. Without perceiving those problems first, it's hard to really understand the solutions that the internet has come up with to deal with those complexities. DREW: I would say I was like a solo designer developer throughout the early years, because at 2010, I found my people in a local agency called 'Clockwork' and for the first time, I wasn't the only developer on staff. There was a whole team of developers. In fact, the shop was started as a development shop and they were making headway into the creative space and eventually, becoming full digital partners. But had it not been for my opportunities at Clockwork, I wouldn't have picked up my skill set as a backend coder. From the very beginning at Clockwork, they expected you to get your hands dirty and code and get your hands dirty in the terminal, honestly. Command line was required even in our design work. CHARLES: And this is all designers needed to be familiar with the terminal tools --? DREW: Correct. CHARLES: -- Basic coding? DREW: Yeah. Essentially, all of our work, whether it was creative or whether it was documents, were all managed in Subversion. As a part of onboarding, you basically learned how to use Subversion. There were some GUI tools for it but for the most part, it wasn't that steep of a learning curve. It was pretty easy to follow instructions and that was the second gateway drug, I would say. My first gateway drug, again was kind of coming up in the age of the web and getting into CSS and Flash. The second gateway drug was basically being required to learn command line and learning how to navigate a computer without a display. Had not been for that, I don't think my career would have taken the turns that it did. I basically got more into the IoT space. I had set up a home NAS server with Drobo FS, is what it was called at the time and it was just a really basic machine but by jumping into that, I could start to play around with UNIX and tools there. I started using home automation, playing with that and at some point in time, I made the jump from just web into the role that I play here at Honeywell, which is Internet of Things. We do a lot of Internet of Things. In fact, our latest tagline is 'the Power of Connected' so we've embraced it all the way down to our wood mark. It's becoming the new normal for most products so it's a good time to be at the center of all these different areas of expertise, to be in development, to be in IoT and to be in design. That's my path. That's my journey. I would kind of pick it up at a bunch of fortunate circumstances, honestly. ELRICK: Having these two skill sets: your design skills and your development skills, what do you believe that that gives you in terms of an advantage? Having these two skills set and being able to leverage these two? DREW: From my perspective, having both skill sets allows me to understand. I think the biggest challenge when working with large teams, particularly in this space or in any space is to really have a common level of understanding, stepping aside from a functional role and becoming more of a liaison between design development and to be honest with you, as we look beyond that, I took a three or four or five month course in business administration, actually. It was just a night class but I wanted to be able to speak to those needs as well. I think it really is becoming a translator. Serving as a translator between those items and then also being able to understand where the actual boundaries lie, there are a lot of very talented engineers and talented designers and sometimes opportunities are missed because, either timelines are pushing engineers to cut certain functionalities or certain features and there's a lot of pressure. Where we can lend a hand, where we can point to possible alternatives, I think that's where we really build cutting edge products. When we really know each domain, we can push those boundaries. That's where I'd enjoy bringing my skill set to the table. CHARLES: Yeah. I can second that. Having actually worked with you, I think one of the greatest things was the one just with the interactions that you were coming up with, were just really spot on. It wasn't ad hoc. It wasn't some -- ELRICK: Helter-skelter? CHARLES: Yeah, it wasn't helter-skelter. It wasn't some developer coming up with like, "Hey, this is what this looks like," Or, "This is some designer putting up pie in the sky stuff." It was, "I understand what's possible and I'm going to use that to design the best thing that can be possible." It made the designs very pleasant and some of them were just really fun, I think. Thinking especially like that, the hierarchical tree selector was one -- ELRICK: Yeah, that was fun. CHARLES: -- Which the implementation of that was just a joy. But then the second thing is being able to speak with you on the development challenges and really know that you understood that language. It really is being bilingual, I guess in the sense that I'm talking to you in French and you're talking to product owners in German or whatever. But because you're bilingual, the flow of information is as frictionless as possible. DREW: I will say that it was a real pleasure from our end working with your team as well because one of the trends in many businesses throughout the world today is embracing a lean and agile approach to product design development. One of the growth opportunities, I would say in any business is fully understanding how that process works, having the courage to be upfront about what can be accomplished in the time available. I think one of the other things is fully understanding those three pegs of the stool. There's always the budget, the time and then the features of any projects. I think that working with a team that understands that really changes the dynamic. I will say that it was equally a pleasure for us to work with your team because there was just a level of courage in being very forthright and very upfront about what do we need to get the job done? What has to happen? You made my job as a translator, essentially. CHARLES: We aim to please. ELRICK: Absolutely. DREW: Absolutely. The latest evolution of kind of where my career has taken us in the company is embracing the hardware element. We've talked a bit about the web and then how that evolved and then having to get comfortable of the command line and where that took place. I've always wanted to build. I've loved designing but I always want to build it and I want to put it out there. In the last six months actually, I finally decided that I would pull the Band-Aid off and jump into soldering hardware, writing what code I could and building actual physical hardware prototypes. I think the next step for anybody who likes to follow this maker trajectory, for a creative looking to become a maker or a developer looking to get into creative is just not stopping. There's always something there and we're also fortunate to live in a time when I can go on at Adafruit, pick up a kit of parts for under $100 and build something that's completely new. Then by the way, they have a full-on tutorial that takes you through every step of the process and gives you bits of code to get started so what's your excuse at that point? If you've got $100, then you can throw and toss into a hobby, pick up a soldering iron and go to town because there are videos, there's the documentation. Documentation is just everywhere now, where it was never there before. I think the next step for us is seeing how can we very early on show real physical world products to end users and get feedback. How we're taking design now is beyond the digital and into the physical. CHARLES: That's fascinating. I feel like there's this pendulum that swings through the tech industry of things moving from hardware to software and back again. We're in the middle of the swing towards the outside or towards the hardware again, like the distributed hardware versus the dumb terminals. It's distributed across a bunch of devices rather than concentrated on one super-powered desktop computer. The pendulum is going to swing in it but it's just always fascinated to see what the actual arc that it takes is going to be. This has been a fascinating conversation and the reason I wanted to have it and we were actually talking about this before the show started officially, why this topic of 'devsigner?' I think that it's a role that is emerging. I think it's still in the early days. I think that I went from three years ago having never really met this type of person to having met and worked with you. Now, I would say having met and worked with three people here at Frontside who fulfill that role and now knowing a couple professed devsigner or people who operate clearly in the design and the developer space on Twitter. I feel like it's this emerging career track that might not be fully understood or defined right now but clearly, there's something there so we wanted to explore that. I'm curious if we might be able to open up the discussion a little bit on what is the future of this role? What tasks will it be set to accomplish? When you're assembling your team, you say, "Get me one of those because we're going to need that." How is that going to be further refined and designed so that it scales as, perhaps an official career in one, two, five, 10 or 20 years? DREW: I can only speak to my experience in this area and I can say that for the most part, it is a very unique skill set and sometimes, it's hard to come but like you said, you're working now with three people. I think it's growing in prevalence. I believe that where coding was less common in the past, it's becoming so much more common now that it's almost like an expectation just like typing. It is an expectation now. People expect you know how to type. It's not a surprise that we're going to see more and more of these individuals. I would say that any design team out there could almost invariably benefit from having somebody with this skill set, somebody who can translate design concept into a working prototype. I've seen it manifest as a prototyping role, more or less just so that we can have a tangible deliverable for developers. I think it does depend on the team, certainly. If you have small teams with talented frontend developers, then certainly you can work in a lean and agile environment and make very quick iterative change. If you have very large design teams and very large development teams, I would say that having a frontend developer with the skill set in a creative team allows that communication to happen without routine phone calls and lots of meetings, essentially. It's a crystal clear example. I've see it manifest as a prototyping role because the expectation is this code will end up in production but some of the code may. The layout code may end up in production but the functional bits may not. That's not to say that the functionality isn't a part of the experience and that, designers don't care about how well an experience performs. But typically where many designers see the disconnect is in the presentation layer. Having somebody who can carry that over is usually something that is far smaller team can handle. Does that align with your experiences? CHARLES: Yeah, that makes a lot of sense and I would say that the compliment from having this person on your development team, if you're in mainline development mode or maybe you are a small team, even if it's a production system but you don't have full time design resources, this person can slice and dice the features and understand the hierarchy of interactions and being able to put together some wireframe, some very concrete goals and set those goals for the rest of the development team. But yet also understand what goals are achievable in the iteration. I think it works from the flipside as well. Maybe what we're seeing is the agile of the [inaudible] of everything. What we've seen over the past 15 years or 20 years, what has been the arc of my career is just seeing these feedback loops in every element of product development getting smaller and smaller and smaller. On the development side, we recognize this as being able to feedback loops and verification. Having your tests, you don't actually have to deploy your system to be able to get feedback about whether it works or have it be fully assembled to get feedback about whether it works. But then that manifests in terms of continuous integration and deployment. You're bringing down the feedback loop of getting this out in front of people versus these long deployment cycles that maybe you really have a release every year. It was hard to believe but that was the norm when I started. It was yearly, maybe even once every 18 months. It was not uncommon at all to have released cycles like that. Certainly, three months was very, very short but then those tight feedback loops can also manifest itself, internally in terms of team communication and I think having people who can make those feedback loops between the product and between the implementation, every time you shorten that feedback loop, you're unlocking an exponential amount of time. DREW: Yeah, I think you kind of hit the nail on the head when you talk about setting scope and understanding things as well. Strictly speaking from agile terminology, having a product or a role that can bridge those gaps is critical. I think that the best product owners that I've worked with have understood, have had an appreciation for design but also have had some degree of a development backend as well so they know how to make those critical decisions. In any sort of iterative or agile environment, you have to dice up these features and figure out which ones are going to ship when they're going to ship. I think, yeah you hit it right out of the park with that. Whether or not you can ever have a full-on team of just prototypers, I'm not as convinced that that's necessarily scalable. It seems like there's certainly a role for teams of developer that will break down features and then there's teams of creative as well. CHARLES: I think in terms of the person who would lead that team, this role definitely seems very well fit. DREW: Exactly. CHARLES: I think it's a great opportunity for someone who's looking for a leadership position in terms of developing and seeing products to market, which is kind of similar to what you're finding yourself in today or where you're headed towards, it sounds like. DREW: Yeah, for the most part. It seems like I do find myself in a number of calls in kind of bridging those gaps. It's certainly a different dynamic in the agile environment when work with hardware. That's something that I think we're still exploring and still understanding. Certainly, there are companies that do agile with hardware but there's a whole slew of different challenges. You're not just deploying anymore. You're actually building manufacturing understanding what needs to ship with what. I think the next evolution of our company's growth into this space is how do iteratively produce hardware. ELRICK: Interesting. CHARLES: You got to keep me posted. The next time we have you on the podcast, you're going to have it all figured out, you're going to be presenting your thesis, it's a conference talk upcoming, agile hardware. ELRICK: Yeah, that would be pretty interesting. DREW: Yeah, I'll let you know. CHARLES: In the first iteration, you just throw a bunch of boiling solder on the breadboard and see what works. "Okay, now, that didn't work." DREW: I'll be honest with you. The 3D printing is making lots of possibilities open up in that space but ultimately, you got to ship. We use 3D printing and now we are using these low-cost computers to really prototype real world experiences and near-to-final industrial design. We can do that. CHARLES: Drew, this sounds like you have the coolest job. ELRICK: I know, it sounds awesome. DREW: It become even more exciting than I had initially intended. It's fun times. I think, again we're living in a time when we can 3D print stuff and have it done within a couple of hours. What better time to embrace these technologies and this creative spirit. It's kind of all around us. Honestly, it's just being fortunate. CHARLES: Yeah. Fantastic. This has been a great conversation. Thank you so much, Drew for coming on. DREW: My pleasure. Thanks for having me, guys. CHARLES: It's an amazing place. It sounds like even more fun since we got to work with you. If anybody is out there and they're in the design space and they think that, "Oh, maybe I can't do development," or it's too hard. It's not. There's a lot of people out there who are doing it and experiencing lots of good benefits. I would say that the other thing is if you're a developer, you should think about looking into the design space, something that you might be interested in. I think it's probably less common that the vectors people move from development into design and not vice versa but there's nothing that says that it can't go that way. Mostly, it's because people just aren't doing and they think that that option is not available to them but clearly, it is and clearly, it's a valuable role. I think this role is going to only get more valuable in the future. DREW: I would second that thought and that notion. I give a quick shout out to Erin O'Neal. She's a former colleague of mine who's given a number of talks about that very topic -- backend developers caring about user experience, caring about the design. She's given some talks. You could probably find her on YouTube. Anybody who wants to talk about it, I'm all over the web as DrewCovi. I think I pretty much have that user name in every platform so if you Google me, you'll find me. CHARLES: We'll look for you. Obviously, you can find us at @TheFrontside on Twitter, TheFrontside on GitHub and feel free to drop us a line at Contact@Frontside.io. Thank you for listening everybody and we'll see you next week.

Robert Jackson: @rwjblue | rwjblue.com Show Notes: 01:00 - Build Tooling in JavaScript 02:19 - Ember and Babel 07:14 - Deciding on Features 11:46 - Class 13:29 - Workflow 14:39 - Payload Size 15:24 - Config Targets 17:18 - Source Maps 25:05 - Ember Decorators, Objects and ES6 Classes 36:07 - What's next and when can we get it?! Resources: Babel.js esperanto Ember CLI Targets

Anissa Willyard: @team_giveback | GiveBack2Schools Show Notes: 01:28 - The Mission of Mission Driven Businesses 05:05 - Defining Moments and Leaving a Legacy 11:30 - PPG (Plan, People, Go) 13:20 - Finding Your People 16:22 - Choosing a Mission 22:30 - Defining a Problem 34:19 - About GiveBack2Schools Resources: Boston: Peace of Mind Lyrics Transcript: CHARLES: Hello everybody and welcome to The Frontside Podcast, Episode 74. My name is Charles Lowell, a developer here at The Frontside and your podcast host-in-training. With me from The Frontside also is Ginger Whalen, our head of business development. Today, I'm actually excited about this episode, I'm excited about every episode but I'm excited about this one in particular because we're going to get to see something from the other side of the coin. Our listeners are on the implementation side and sometimes, when it comes to starting up a business or being a founder, the driver and the implementer are met in one person but more often, and I think in a more healthy way, they're split between multiple people serving multiple roles. Today, we're going to interview somebody who is a serial entrepreneur, who is currently a founder at a business that we came very, very close to working with. Please welcome, Anissa Willyard to the show. Hello, Anissa. ANISSA: Hello Charles. Thank you so much and thanks to The Frontside Team and Ginger. You guys are an incredible team and I really enjoy having the opportunity to be able to be here today so thanks so much. CHARLES: No problem. You're currently a founder at GiveBack2Schools, which is going to be launching soon. ANISSA: Yes, that is correct. Our goal is to launch in July 19th. We've kind of pushed it back a little just to make sure we get through the holidays and make sure that i's are dotted, t's are crossed and now, we are locked and loaded. The official date right now is the 19th and we're really excited to launch in Olathe, Kansas. CHARLES: Fantastic. We're going to actually talk a lot about that later on but first of all, there's obviously a large backstory for how you came to be doing what it is that you're doing today. I wanted to kind of delve into that a little bit. ANISSA: First of all, I think right now, we all live in such interesting times that I believe by having a business with a mission is actually is going to allow us as individuals to create such a positive change. The change that we create will actually feel us through as founders or as anybody who decides to move forward to start any kind of company. I do believe there was a quote that I live by. It's one by Zig Ziglar and it says, "It's not where you start but where you finish." To kind of take you through that journey, my hope today is that I give everybody hope, that if I can make a difference, that somebody else can too and that will give them and inspire one person to take that action to make that difference. I look back on everything in my life and kind of where I was in my journey, as far as the why. The why behind what I did. I sat down throughout my day and I broke it up into three different parts. In the first part is kind of called, 'the grownup phase.' That's where you go to school and you come through college and then your middle years where you're out, you're fighting for that career and you really are starting to climb that corporate ladder if that's the road that you decided to go down or raise families or whatever that is. Then the last part where it brings you to that moment, where you question yourself, "When this is all done, did I really make a difference? Am I leaving behind and making a better place here?" That all brings me through, like you said, "Where did it all come from and everything?" From my days with Mary Kay Cosmetics, back when I thought for sure, I was going to be an oncologist and then went out, that's with Mary Kay where I had the honor and the privilege to be trained by the woman herself back, growing up in the Pacific Northwest and having a great role model that just inspired me and be enabled at the right age of 22 years old to have earned a free car. My father thought that was just crazy, that I would sell lipstick so I might as well move on and pursue the corporate dream. Now today, to come full circle and think when it's all over, where am I heading? It kind of in a roundabout way who I was and from working in advertising, to marketing, to implementation, the jack-of-all trades and probably the master of none when it's all said and done. CHARLES: When you say you came to this moment where you had this time of reflection of saying, "Is this the legacy that I want to leave behind?" What is that when you start thinking about the things that you want to leave both to your family and to the world, maybe even long after you're gone? When did that happen and how long did that take and what was the end result of that? What path did that set you on? ANISSA: It's so interesting that you asked. The next part that probably I would share was hopefully I can make a future and take you down that path. I grew up in a very small town in the Pacific Northwest. The population was under 500. My mom actually passed away when I was in the fifth grade and my father was a logger, a timber faller, a real blue collar gentleman. My father knew how to work but he never really had an idea how to raise two small daughters from [inaudible]. He was by far no means a perfect man. He would leave us girls sometimes home alone and he only had a sixth grade education. Some of the defining moment since I look back and had time to reflect is my mom actually, when she passed away, my father was illiterate at that time. I was actually responsible for filling out the paperwork. The name Lyn spelled with a 'Y', not an 'I', my mom's, to this day, even on headstone, it's spelled wrong but it's a reminder that it's really not where you start, it's where you finish. But my mom was a great inspiration in my life. She told me every day, "Someday you're going to be somebody," and I didn't really know who that person was and she had battle with cancer. She was always encouraging and she always taught us girls to have no fear. Even I can share stories about that but then, if I looked back in my life, even though my mom wasn't around and my father was gone all the time, there was a long consistent thing I had in my life and that was school. It really allowed me to dream. School was something I looked forward to every day because they help me escape the reality that I still lived it. There was music and sports and outdoor school fun activities, the opportunity to learn and grow. My family life was so different. If I would have went to college they wouldn't have cared, nor if I would have been that home. It didn't really matter because there wasn't really a purpose. As I was decided to want to move out and went on to climb that corporate ladder, it always felt like there was something missing. I continue to rise to the top and of course, you work hard and you make your money but there was always that missing gap. I found myself in so many ways following in my father's footsteps: great worker, tons of awards, he traveled all the time and never around. For several years, I guess I question myself, "Is this really what it's all cracked up to be in." I don't know if anybody here that listens and have never heard of the music group, 'Boston.' While I enjoy classic rock and I don't know if you've ever heard the lyrics of Peace of Mind and I guess, I'll just open that up because that's not necessarily from my generation but a generation that's older than I am. But it just came on the radio one day and it made me take that pause. CHARLES: You know, I don't know the song out of hand but maybe I can look it up and sing it as the outro music for the podcast. [Laughter] CHARLES: I don't give any opportunities to sing on a podcast but when I do, I [inaudible] that. ANISSA: I would so love to hear it and sometimes, just a few lyrics have really resonated with me as I recall that moment because with my childhood, a lot of people may think, "Oh, my gosh. That poor girl..." I actually had a great childhood. I got to use my imagination and become creative and create something that most people never got to create in their mind. But the few of the lyrics -- just to share with you -- actually did define that moment. It was about five years ago and here is how just a few of them go. It says, "If you're feeling somewhat low about the dues you've been paying, you want to run but somehow, you're just not sure if you should run or stay. You just can't decide on which way to go. I understand about indecision and I don't care if I get left behind. People are always living in competition and all I want is peace of mind." I thought, "Oh, isn't that true? Well then, they kept going." They kept going to the next part and I thought, "Really, is this happening to me?" I'm driving down the road and I feel like this artist too was way before my time is talking to me and just going, "Seriously?" Then it said, "As you're climbing to the top of the company ladder, I hope it doesn't take too long because there'll come a day that it won't matter and that they you'll be gone." And I thought, "Oh, my gosh," and it just hit me like a ton of bricks and I thought, "You know what? Time to make a difference." I remember catching an airplane, flying home that evening and coming home to my husband and saying, "Guess what, baby? I'm going to do it," and that was the defining moment. GINGER: That is so great tip in such a powerful moment like that and those lyrics, isn't that the human condition? We all have encountered that indecision and questioning our purpose or our impact at one time or another and to be able to make a change and then develop this company, launch this company, hatch this idea and we'd love to hear how you then transition into this purpose-driven, mission-driven, cause-driven company. It's very powerful. ANISSA: Thank you. When you decide to do something, like I said that was five years ago, where I made that decision, and then you jump back and you go, "Well, how am I going to do this?" Okay, I have a great idea and I am going to make a difference but then you start to look at this whole picture and this is almost like an elephant sitting there in front of you, "How am I going to eat this?" Really, the next part I call and it's something that I coined to myself is called the PPG. I needed a plan, I needed people and I needed to go. That's really what I live by, was that PPG. The point on when I first started, I knew what the end game was. I thought I have a mission, I have a plan but where do you go? You start writing down all these ideas, I don't know if you guys are like me, I am a left brain, right brain person. One minute I'm analyzing something, next minute I'm trying to create it so I can go back and forth in the same sentence. I'm an oxymoron and I can really create a lot of chaos within my own world really fast where you start to doubt yourself when you think should I move forward. Even though you make that decision, it's like, "I'm going to cut off all options. I'm going to do it." When I first started, I didn't have all the answers to achieve the end game. You know, right now sitting where we are and everything that we do and all the different people we've spoken with, I don't know if you'll ever have all the answers. You just have to follow the heart. GINGER: If there's the cause that you find that mean something to you and mean something to your history. You said school was really important for you growing up as an escape in your safe place. How do you find people that will enlist or support or participate in that same cause? How did you find your people? ANISSA: The really interesting is that two of my partners and I have worked together in a different role in a different capacity for almost 15 years. What's really interesting about it is certain people gravitate together in so many ways. One of them was actually a teacher and a coach but there wasn't really any money to be made in that at that point in time so they ended up going into sales. The other one actually had the marketing background, single mom, raised three daughters, put them school as the pomp squad, head person and then my last partner is actually a former superintendent of schools. It's weird how we were all pointed in the same direction. To have that, I'm more of an outsider. I always say, they're more of a pedigree. I'm more of a mutt that kind of take care of me along the way. But they had similar backgrounds, similar heart and similar care. When she started to collaborate and share ideas, it just grew. I remember back in school where one of the founder says, "I was the child that maybe had to have that free lunch and sat alone," then you hear different ideas and then you put their friends in the room and you start talking with different coaches because they know a lot of coaches and activity director. It's just grows, Ginger like you wouldn't believe in. You get more excited and more ideas flow. I do think how we all came together and how we have all worked in different capacities made a big difference but that synergy of opening it up and being able to talk openly and freely about ideas, really is what grew this to where it is today. GINGER: The partnership is just so strong because you all have the same sense of purpose or you want to make an impact in the same area. Imagine it's really gratifying and you're really connected to your partner in ways that are unique. ANISSA: Right. It's funny because we've really have this saying, "No idea is a bad idea," and I am so glad they have those ideas for me because I'm the one who brings up the off the wall ideas and the thoughts and they encourage it every day. I think to have that synergy to pull that together has been priceless. GINGER: How would you recommend other small businesses if they're not built on a cause or built on a mission but they'd like to choose behind a certain mission. How do you say that you could go by choosing that, if they didn't all come from that background or come from that cause throughout their life? ANISSA: That's a great question. I do think that once you sit down and you get real still, which is not always easy. We have a 27-year old and we have an 11-year old daughter. To really sit still and to get close to yourself and go, "Where am I today?" If there is something that just really bugs me or something out there that is broken or if there's something that if I could fix it would make a difference, I know I'm speaking for myself. I can't solve all the world's problems but if I can make a difference in one or make a difference in one person's life, that they can leave this Earth just looking back, knowing it made a difference. What do you have to lose? A lot of times people think, "Oh, if I do a mission-driven business, I may not make money. I may not be for-profit," or, "What about this? What about that?" Put all of that aside for a minute and just start taking your old sticky notes and writing it down. This actually got even deeper with us. Our daughter play soccer at a real high level. She plays for a select team here in Austin, Texas. At one time, I even questioned myself, "Am I doing the right thing?" I thought, "You know what? I'm going to really get in the saddle to point myself right in those shoes of what our children go through and everything else again." They were doing that fundraising mission to give scholarships to children for balls and to be able to play soccer and I told our daughter and I said, "This isn't necessarily what our business is going to do but it has similar path and I asked that we go out and we try and make a difference and see what it takes and what it does." Once you make that decision, it will grow many legs no matter what it is. Just keep going with it and keep trying new things and keep experimenting. Eventually, your vision will be so clear that no matter what you woke up, this is it. GINGER: Yeah, I love it. It usually does start with one individual with the really strong passion. Also, I agree that it used to be saved for non-profit organization or some philanthropic organization or religious organization but now, corporation is for-profit, public-private, even governmental agencies and they now have a mission and they're seeing for something. I think employees these days demand it too. They really want to make a greater impact, even then just locally, not even one with global compact. They also want to stand for something collectively and going for some purpose. ANISSA: I so agree. People are changing the world by changing our ecosystem in business. It sounds so cliché. It really does. I remember the first time I heard someone actually state that, I laughed. I thought, "Ecosystem in business? What?" That has to be somebody is in a cool buzzword today. Then when you start to define how it all works together, the cliché actually means something and it starts to take a life of its own. Just like you are stating in business today, people demand more. They want to be part of something. People supporting what they helped create. It sounds like such a simple statement but in then encompasses some of the most difficult problem leaders today face. You have to always ask yourself, "Am I solving a problem?" But solving the problem will have an effect and it's just one of those, "What effects will I have?" I believe in my life, if we use such as fundraising, I always look back and think of how much cookie dough between the oldest selling and the youngest selling and all of that and I laugh and I'm thinking, "If all the cookie dough we buy, if I would have actually eaten it all, I would be a cookie today," so I am solving a problem, you know? I don't need no more cookie dough at my house. CHARLES: I think it's an important point though, any time there is a need and a great need, if you can make it exponentially easier for people to fulfill that need, then there's absolutely a business there and there's absolutely a mutually beneficial relationship that can be developed where it's okay to say, "You know what? I'm going to charge some money for this service," but the net effect is going to be everybody gets to participate and this thing is going to make things better. ANISSA: I so agree. You guys may find this funny but I believe podcast is something that make a difference in people's lives too. I really enjoy running. I run between five to seven miles a day and there's not a day that I don't turn on something that I listened to or I dive into. I do think the sharing out there today has made the difference. Just by you guys doing your podcast or simple things like that, it does bring that unity together in so many ways. GINGER: But we do hope to inspire and we are inspired by doing this, definitely. ANISSA: Oh, I can only imagine, definitely. But back to a little bit about people do help support what they help create. I think when you can define a problem, I know sometimes that does get deep and sometimes you may think it's shallow and sometimes when you start writing it down, you don't really know what the real deep problem is. You just start with a lot of different parts. You may go through days that you think, "Does this really matter at all?" Really does it? But deep down inside, that's your fuel. That's your passion that keeps you going. It also may take some time. When I first started out back years ago after my mom passed away with breast cancer, I thought, "I'm going to be an oncologist," so as I was going to college and everything, I realized that I really don't enjoy draw on your blood and dealing with all of that. I thought, I am more of that people person than I am sitting there. Even though they have a life and passion and I learned so much from that experience, my mom used to have a great, great saying and she used to always say, "Anissa, there's a lot of dead folks on this Earth. Don't be one. They're living but they're dead. Don't be one." That never came to me until many years later in life what she really meant. She meant live life with passion and go for it and never look back and I thought, "There is some to that saying, just don't walk around like a zombie. Be someone." That was when you bring it back, to define that problem like I started with, you may not even know what your problem is but you know that deep down inside. You're not excited to wake up. You're not excited to get busy at something because you're not being fulfilled and you know that somehow you did not make that difference. GINGER: Yeah, I think that's the bottom, like figure out what you want to stand for, figure out how to make an impact besides yourself and beside your community, more than global level and get behind it, get passionate about it. You keep doing that. You listen to your message. You listen to your mom. ANISSA: I did. I did. There are so many times I didn't listen to my mom and I don't know if it turned out good or bad because I've made a lot of mistakes. GINGER: We all have. ANISSA: Right. You know what, if you take that forward, you just challenge that problem from every angle. Who do I need to help solve this problem? Do you need your merchants, your consumers? Who is the [inaudible]? You may think there's a problem but they may not even know they have a problem. You know, sometimes people are in such denial. Others are not a problem and you're thinking, "Oh, you have a problem," but that's kind of where I come from and to do that. Then when you have a vision and you have people, there will be people who will not support you. They were actually say you are crazy. In that sometimes, people can take that internally and go, "Am I crazy?" Just know that that's a true compliment. If you're not always speak in everybody's language and people think you're in left-field sometimes, that's okay. Don't worry about that. The other part is when you start out on your own, you're going to hit roadblocks. Anybody who starts the business will tell you, you are always, always 100% never for sure and willing to change, no matter what as you continue towards your end game. But really, I live a lot of my life by a movie. The movie, my favorite movie of all times is The Wizard of Oz. I really do think it has several meanings to it. It does take a brain like the Scarecrow. If you're not the brains, get somebody who has those brains behind you. I always knew that when we were going, I needed somebody who has a lot smarter than I was in so many ways. It takes the heart. I've got a great heart and compassion and all of that but it does take that heart. Maybe you are more of the brains. Find somebody with that compassion at heart and it takes courage. I know that your team are there to build what you guys have and to do what you've done. It took a lot of courage. I think anybody today who is in business, I do think the mission sometimes is what get us through when we are in business because without that, there really is no end game. GINGER: Although all of us are going to start businesses based on a cause or a mission, there's a tons of ways you can get involved doing volunteering. Then you're involved with like-minded individuals chasing some cause, accomplishing something but those lessons can be an individual level, group level, company level. I completely agree. CHARLES: Yeah and I think it's important too. It's thematic on this podcast of don't go alone. I think we have some very toxic myths out there of the people, this person, kind of founder who locks himself in a room and then they sit there, eating nothing but pizza and Dr Pepper for 48 hours and they emerge with the solution to everything. That's just not how it works. Time and time again you'll see that what really makes a robust, resilient business is exactly like you said: having people who can truly fulfill each one of those roles, the brains, the heart, etcetera and really step into them completely. It's a great point that just can't be repeated enough. ANISSA: Oh, I agree. I don't know if my family would even be my family if I didn't surround myself with good people. If I lock myself behind closed doors, I may come out crazy, literally. They would not want to be around me. GINGER: Again, we can all say that at times. ANISSA: But Charles, to go back on your point, don't take this alone guys. There are people out there that have the different parts that maybe you're missing. I'm telling you, there is strength in numbers. I don't mean numbers like 10 or 20. That's great but when you're really collaborating and getting down in the nights that we worked and the early mornings and the times, we were told basically no. When you're out there fundraising and you're trying to get the money, to go where you need to go, to continue your own way, not everybody is excited about you or what you're doing. You'll have to have those partners to keep you going at every turn because they will make the difference. There are times where it gets really dark and that darkness can be just one more knock or one more call to make it happen. I encourage that spirit together. CHARLES: Yeah, I think there's also another point that I want to bring to the forth and while we're on the subject of toxic myths, I think we have also this myth and it is prevalent of the idea of startups and people embarking on businesses being very young, basically young men and I think that while young people have a lot of time on their hands, the people that you know and you can cultivate over a lifetime. They are the true asset. You have people with 10, 20, maybe even 30 years under their belts who have developed these networks which are just these -- we're from Austin. I love trees and I love the trees in Austin and one of the things that makes live oak trees, which is a very hardy native tree that survives around here, it goes through periods of sometimes it rains a lot but sometimes we go through five, six years where it doesn't rain a lot. What happens is these gigantic oak trees have these network of roots that intermesh and intermingle underneath, sometimes 10, 20, 30 feet under the ground. If you have that time in your life and you've spent that time being around extraordinary people, that will come to fruition in terms of you'll be able to grow new trees based on that shared interlocking network and the strength of that network is really a true asset. I think that another message here is that people who are advanced in their careers are almost the opposite of saying that a startup is not an option for you. It's almost like you're going to be better suited for it in many ways. ANISSA: Oh, I love that analogy. That is such a great analogy. Looking out in my backyard here in Austin, we have oak tree. I'm going to remember that every day. I agree so much. That was probably one of my biggest fears. You look at some of these great companies that are born today and come out and you're thinking, "Gosh, have I missed my prime?" Also, being a woman in business, sometimes I've been the lady in the boardroom. I've been the one that has earned that executive title. To climb that ladder isn't easy but to be a woman in business sometimes, also makes you stop and wonder, "Is anyone going to listen?" I think that the wisdom in business that we have, the so-called bloody knees from falling down and getting back up, the balance back ability, that in order to make those things happen, it was a [inaudible]. I just appreciate the fact that there are other people out there just like me who maybe want to make a difference and don't know where to go. Just realize that there is more than just me out there that has this feeling. You just need to take that chance and make it happen. GINGER: Or classic rock or classic movies. ANISSA: Right. No doubt. CHARLES: We talked about how to organize a business around a mission, I'd like to actually take a moment just a couple of minutes to talk about GiveBack straight on and let you explain because I think it's just a great concept. ANISSA: It's really funny. A little bit about GiveBack. GiveBack was actually born from an idea of being sick and tired of fundraising, if you can believe that. During the time when I was a child growing up, fundraising really hasn't changed for our educational system. Everything gets cut. I remember looking back and being able to sing songs from The Beatles, Little Submarine and all of those things. Nowadays, you don't have those things anymore. We always get these different, "Oh, got to sell cookie dough, got to sell this." In our time as a family, I was already at the end of its rope. I'm traveling. My husband's got a great career and our oldest son was actually raised by a full-time live-in nanny. Now, that's not something I'm excited to brag about but it happen. As we had our second child, we were heading down that same road. When it came to fundraising in schools and all of that, I thought how many kids out there don't even get to go to outdoor school. What we did, the concept and the idea was how can people just live and we give. That was really how it was born. From my marketing background, from working with businesses, people want to make a difference. But yet, there's really no trackability on their ROI. Yet, they also need the cost to get new clients in so there is that cost of acquisition, bringing new clients into their business. Schools need money. In order to keep going for these athletics activities and all of this and how can I tie this together that the consumer can dictate wherever they are in America that they can and maybe someday global, give back to their child soccer or maybe the chess team at the middle school or the high school. Maybe they have three kids so they're not giving up their precious time, their own resources, to go out to a school function night, when it's only Tuesday, service is poor. The restaurant wants to help but it's just not working into our time and who has the time? I remember the wagon full of cookie dough and taken it door to door and I was thinking, "If we don't get it out, it's going to spoil in our garage because our son had to hit the record and sell the most cookie dough. Yay! Great job. Now, Mommy's got to go deliver it with you." It's eight o'clock at night. We're driving around in the car, knocking on people's door to give them cookie dough. Then what percentage was really going back? That's kind of what we did. We connected the dots and we brought fundraising or funding of those activities at education and all of that into a way that it would just basically, as we always say, you get to live and let us do the giving. GINGER: That's great. Then people can really know what kind of impact they're making, how much they're giving back. I think that was the missing piece. People got frustrated in fundraising is it feels kind of good and giving to the cause, I support but what is it really doing and how much is really going back to them? Those were open questions back then. ANISSA: Right. Believe or not, they're still open today. Here we are going into the Year 2018 and they're still using envelops out there and who knows where that money's going or what percentage went back. You hear about that all the time. You see it firsthand. I agree with you and that was really something as that transparency to be able to show. You know, the school was gone, now what you did and how much went to this activity? Also, it was that full circle transparency and boy, doesn't that make a difference and to be able to let people start to dream again, if we have instruments maybe these kids could actually play them again, versus, "Oh we've got a company [inaudible]. Forget about outdoor school. Let's just go to more textbooks." It gives us a different way to bring back to our schools. GINGER: You want to talk about this going national at some point for people that are in the Austin area? ANISSA: To be honest right now, our end mission was to do what was right. As we soft launch and get out there, yes our goal is to go nationwide. We're looking more towards about the scalability but I would rather under-promise versus over-promise and not deliver. Our goal right now is just to watch for upcoming events. It was hard for me to keep it under wraps for so long. Now, we start to leak it out a little bit. I think probably the best way to put it is stay tuned and keep your eyes open. That would probably be the best verbiage to use at this point but I thank you guys for your hospitality and allowing me to do that. CHARLES: Oh, no. It was our pleasure. It was great meeting you. It was great getting to meet other members of your team. You come across folks in the line of work that we do and you all really stood out as folks who knew what they were about and we're on a mission to achieve it. I think it's important for people to hear that that you can do that, really at any point. Thank you so much. Thanks for GiveBack. ANISSA: Thank you. I just really want to say thank to all of you and I want to wish you a Happy Fourth and safe and good health out there.

Audrey Eschright: @ameschright | The Recompiler Show Notes: 00:50 - Background in Publishing and Open Source 06:53 - The Contributor Pool 12:37 - Open Source Bridge 15:29 - Mistakes Open Source Contributors Make 17:21 - Tools for Maintaining an Open Source Project 19:09 - Roles 23:33 - Open Source Bridge (Cont'd) 27:47 - Governance and Decision-Making 36:20 - Making Open Source Accessible, Safe, and Welcoming Resources: Free Geek Calagator PDX Activist Dreamwidth Safety First PDX Open Source Bridge: Enter the coupon code PODCAST to get $50 off a ticket! The conference will be held June 20-23, 2017 at The Eliot Center in downtown Portland, Oregon. Transcript: CHARLES: Hello, everybody and welcome to The Frontside Podcast, Episode #71. My name is Charles Lowell. I'm a developer here at The Frontside. With me also is Joe LaSala. JOE: Hello. CHARLES: Hey, Joe, another developer here at The Frontside. With us today is the publisher of The Recompiler Mag and a long-time open source contributor Audrey Eschright. Welcome Audrey. AUDREY: Hey! CHARLES: Thanks for being on the show. AUDREY: Oh, thank you. CHARLES: Today, we're going to be talking about open source and in particular, the labor that goes into open source and making that sustainable but before we get into that, I wanted to first talk about your background, both in terms of how you came to be publishing the magazine and also your background on open source, how we're arriving at the subject today. AUDREY: The magazine, in a lot of ways, I refer to it as a feminist hacker magazine. It holds together a lot of different things that I've worked on over the years so I'm going to jump all the way back to when I first encountered open source and then maybe that will fit together. When I was in high school, I first encountered the internet and the internet that was available to me at that time use things like Gopher. Gopher is a pretty web protocol and it was free software. I didn't really understand that it was free software at that point but I did understand that if I wanted to learn how to write code and the computer that I have access to were things like a bunch of really old PCs like 286's and an old Macintosh. Then there were commercial compilers for writing code and there were free compilers for writing code. There was a thing called GCC and I knew that it was on university computers and if I got access to those, then I could write code. Then I got to college and write about when open source really started to take off as this concept of how free software comes into business world. I've had that as a background of becoming a programmer and getting involved in things but after college I wasn't really sure that I want to work in technology so I took a break. When I came back, I needed a way to get myself up to date so I started volunteering with this local group called Free Geek that recycles computers. What they do is they take those computer parts and the ones that are usable, they build them into Linux boxes for people, like Linux desktop boxes. How I got back up and running was learning how to work and volunteering in an organization that was very open source based, like all of the tools that they used are just completely open source. CHARLES: Was that for budgetary reasons or they didn't want the people to burden the recipients of these computers with any licensing fees or obligations to third parties? AUDREY: It's budgetary but it's also ideological. The organization was started out of environmental interests. The original folks, they pointed to us this computer monitor that they [inaudible] as the reason that they do this, that the way computer waste is being handled was so unfriendly that you might as well just dump it in the river. They started from there but I think because those kinds of interests of creating something that was really accessible for people are really educational and accessible to lower income patrons has always been a really big part of it. I think that using Linux and using open source tools has been a big part of that. CHARLES: I think open source is so pervasive, a lot of people forget that in those days, there was a lot of radical thinking behind it, of radical accessibility like it's your basic right to be able to access every layer of your stack. It's a little bit unfortunate that you mentioned GCC that like the GNU, the Free Software Foundation isn't as much part of the conversation as they were back then. AUDREY: Yeah. I think that as more people come in to, we've shifted through these different generations basically in open source contribution and how it's formulated. The fact that I even default to open source is really interesting because a lot of the values that I referencing are those free software values. CHARLES: Fast forward to the present... AUDREY: Part of how I built my skills was by starting open source projects called Calagator. It's a community calendaring platform that makes it very easy to import things from other sources like Facebook. It's interesting, it wasn't our primary thing but it's so big now. We've been doing this for 10 years so a lot of recent change around us. We have a 10-year old [inaudible] app that is still up and running and is now in Rails engine. CHARLES: Wow. Is this an application that you can run yourself or when you say it's an engine, if I've got a Rails app, I can just drop it into any Rails app? AUDREY: Yeah, that was a direction we decided to go in a couple of years ago because my experience was that handing people in Rails app and saying, "Go fork it and then go sell it and use it in your community." That's a pretty big technical burden. At least, as an engine, it makes it a little bit more flexible for people to really come in and make some of those changes. We can bootstrap a little bit more for them. CHARLES: It's always funny to me know how some projects always run off the fork model, like there's a lot of HTML starters or editor starters where the thing is you fork it. I always hate that model because eventually, you ended up having to do this terrible dance with the upstream in order to jump around the changes that are coming through and stuff. AUDREY: Yeah and that was definitely one of the problems that we would run into. We would make changes to functionality and the frontend and the visual display of it. It was really difficult for people to pick and choose the parts that were useful for them. CHARLES: Yeah. Okay, so you've got a 10-year old Rails applications/engine, now you are actually running an instance of this engine yourself or just maintaining the open source? AUDREY: Yeah, there's actually two of them, that I'm in involved with right now. One of them is that Calagator.org. It's a Portland's techs events calendar. That was really our original site and the reason that we created this. The other one just as of a few months ago is PDXActivist.org and that is a way to get a lot of activism and political organizing off of Facebook, basically. That's really our primary target. It's just getting people an alternative to using Facebook for all of their events. CHARLES: I see. Now, having to maintain an open source project for 10 years, that's a really, really long time. AUDREY: Yeah. CHARLES: How big is the community now and how many different users have you seen as you developed this? AUDREY: Well, it's a little hard to tell. We deliberately don't do a lot of tracking, especially on PDX Activist side. I can tell you that there are a lot of events on both calendars. For the tech events, there are probably five things that you can do on any given day, maybe 10. During design week, they put all that on there too. This has been very consistent over the history of the project. I can also tell you that we've had dozens of contributors. CHARLES: Yeah, that's more what I meant when I said users. Not necessarily the consumers of the calendar but the consumers of the software that makes the calendar. AUDREY: It goes without saying that I think that those users are creating events, they are part of that because they help curate content. Like with the wiki, your user base isn't just the people who update MediaWiki. It's that people who really work on the content too. We've had dozens of people. There's a contributor's file that I didn't pull up but we can go and look at it. We made a point of crediting everybody who contributed at Code Sprint, whether or not they check in code. We have a really great documentation over the history of the project about how the different ways that people contributed and who they are. CHARLES: Yeah. I feel like that's something that often goes missing in projects, especially open source projects that you find on GitHub where there's so many people that are involved in creating software beyond just what you see in the commit history. It's kind of a poor showing of what it was all involved in the whole creative act. Sure, it's an accurate reflection if it's a one-person project who's hacking away on weekends but as your project scales, there's a lot of different stuff going on. AUDREY: Yeah, definitely. I think the other part that's really interesting for me about this is that I can point to that big contributor pool, people who have come to sprints so they've work on a project. They help define the shape of the project. Then I can tell you that we had a three-person core team for a very long time and then it was down to a two-person core team. Now, I'm not really sure which one of those is in charge. I don't look at GitHub often enough and a couple of the other computers. There isn't a lot of coaching happening anymore. We should have a wish list but there's nothing so urgent that we stop all other work and go back to making this our primary effort. CHARLES: Of the people on the core team, how many of them are developers? AUDREY: All of us. All three of us were. We come into with different cross skills. I've done a lot of documentation and mentorship. As of the others, I would say we have one person who were in design or one person who was more apps-oriented. We fill those different layers too. CHARLES: Of that group of the core contributors, outside that group of core contributors, you said you accumulate a list of all the people who contributed. What's the breakdown in the roles that those people are playing? AUDREY: You know, it has changed a lot over the course of the project. Early on, we had maybe half of the people were really doing development and the other half were helping. We took a very agile approach like index cards and users story. Maybe half of the people that show up at a given time, we just talk through the feature and do research. We were looking at a lot of integration so what needed to know what would be required to integrate it. We brainstorm a lot of things. We did in-person Code Sprints every two weeks from the year that we started, at late of January to the end of July. We had this whole set of in-person work that really shape in that. Also a lot of people who weren't necessarily contributing code that had disappeared. CHARLES: I see, so people who had a vested interest in a particular set of features could show up and voice that interest and be heard, as opposed to what you're having, it just be limited to the people who are writing the actual code. AUDREY: Yeah and we would ask people to spec it out. Just sit down with somebody and figure out how the feature could work and whether it fit with everything else to what we're doing. I do that research and investigation. Over the years, we've had this come and go in waves. Every so often, we need to go up a Rails version or make certain kinds of major updates so we get people together for that. We had some different pools of Codeschool students that have come in and really been interested in working on this to get a little bit more development experience, get some experience working with other people, have open source some resume to show off. I've been very enthusiastic about giving people that resume credit that if they need an open source of it so that they could say, "I know how to write with other people," then our projects is very happy to help them with that. CHARLES: What is the conference that you run? AUDREY: I am on the committee for Open Source Bridge. It's an annual conference for open source citizens, which is the same people who participate and benefit from open source. CHARLES: Which is pretty much the planet at this point. AUDREY: Yeah. It's funny because, I think it's just so interesting who does or doesn't identify themselves as part of that. Anybody using a computer these days is in some way benefiting from open source and could potentially contribute to it and be part of that. It's not just awareness, there are a lot of actual barriers so that, to everyone having a role in it. But the conference I co-founded it with Selena Deckelmann who's at Mozilla now. We do say over time to ask a lot of questions about how across technologies, open source comes together to build things? How projects work? What kinds of skills are involved? How we become better maintainers by being aware of our users, by communicating better, by being good moderators of online message boards and mailing lists and things like that? We've had a chance to really just look at broad swath of elements that come in. CHARLES: I think that literally every bullet point that you mentioned, I feel is something that we've come across and it has been a challenge for us, in our efforts to maintain our open source projects. Ours are mostly just libraries. There's very little by way of big, big frameworks or big, big applications. We've got it kind of easy, I would say and we still struggle with those things really understanding our users, understanding how your open source project should run and how it even fits into the bigger ecosystem. Is there a guide out there somewhere like how to how to open source? AUDREY: You know, I don't know that I've seen a single guide but there is really a lot of good writing and a lot of good conference talks on this topics. Like you said, it's just this broad set of skills and we focus so much on teaching people how to code and maybe teaching people how to code together, to be good contributors together but if you ever to maintain a project, there's leadership involved. There's communication involved. CHARLES: It seems to me that's the bulk of it, right? AUDREY: Yeah. I don't know, did you get training on that? [Laughter] AUDREY: I just decided to try things. I'm very lucky that I'm mostly made good guesses but there's some really bad ones too where later I look back at it and realized we could have done better. CHARLES: What are some of this mistakes that open source contributors often make, where they could save themselves a lot of trouble? AUDREY: I think a big one is thinking about it only in that technical framework. Even just by tools that we use, we tend to force people into contributing solely through GitHub, which means that you've got to understand somethings about the bug tracker and how tickets go and the workflow around that. CHARLES: Yeah. I've literally looking at a message in our Slack from yesterday where someone on our team who doesn't interact with GitHub said literally, "Someone is going to have to show me how because GitHub is the most confusing thing I have ever logged into." JOE: I thought about that message today too and yeah, I guess I'm wondering how do you attract those more non-technical skill sets to a project? AUDREY: It takes a lot of direct mentoring and coaching. You already has some people that are identifying themselves to you if you're having that conversation. I think I've really benefited from looking at who else is like them, who else do they know that might want to get involved and starting conversations that way. Because the biggest projects that I have worked on are these calendars, it does give us so many users that maybe are interested in having more technical involvement. If I can start looking at who's doing a lot of cleanup on there, who's paying a lot of attention to the content and the structure of the content and structuring information is also a technical skill. But people don't necessarily go from that to thinking, "I can write code," or, "I could submit a ticket and debug that thing and tell you what needs fixing now." But people can get there. We just have to be willing to talk to them about it and willing to look at it from their point of view. CHARLES: One thing that I dig out of there is that if you're running your open source project solely on GitHub, it's not going to be enough. You're going to be constrained in your growth just by the toolset and the implicit exclusivity of that toolset. What are some tools that you can bring in that are going to be more attractive? AUDREY: I think mailing list have turnout to be one of the most open-ended things that we've done. People who want to find out a little bit more, sometimes post there but also just having a good webpage, a good info pages or some sort, having your wiki actually talked about some of the less technical aspects of it. Even explaining what your project is for can be really good. You know, you start to make these assumptions like, "If they're going to go and install it, do they know?" Maybe not. I think just looking at it as a broader set of communications. CHARLES: Right. What seems self-evident to you and maybe someone who shares a lot of context to you is a mystery to someone else. It never hurts to state the obvious. It seems to me you have to be able to use tools that people are familiar with but also part of the leadership is giving people things to do, giving them a way to think about your project or giving them a way to act independently. How do you think about the different roles in an open source project so that you can then elucidate those roles so that someone coming, who is going to look at your website or who's going to be reading your e-mail list is going to be participating in your community in some way and particularly not in a code contribution way, how do you think about the different roles of your open source project so that you can kind of hand that to them? So that they can act independently like, "Here's this thing that you could do. Here's this thing that you could do. Here's this thing that you can do." What is that kind of core set of roles? CHARLES: We could think about it in terms of the actions that we take. If you go back to our lone weekend coder who put something on GitHub, you're already writing the code, making design decisions about the shape of the code, you are writing about it in some way, even if all you do is update the ReadMe to have two lines of something you're writing. You are managing any bug tickets that come in, any future request so you're doing some project management, some kind of general analysis of that. They don't necessarily have to be different roles. People implicitly take on the whole thought of that when they start a project. But they can also be split out. I hate to say like, "Give away your least favorite thing," because people sometimes do that, may dump it out there and it never gets handled well because they don't really understand what they're looking for. But it's okay to say, "I am really great at this one thing and I really struggle with this other thing." I bet there's somebody else who is just way better at organizing the stack communication and they can help me with that. If I can tell them what I need it for, maybe they can help with that. CHARLES: So you have to admit your weaknesses? AUDREY: Yeah. I think a lot of leadership is that kind of self-analysis: really seeing where you are helping the most, where you're strongest, what things absolutely have to be done with you. I don't know. I'd learned you to be really honest about that. Sometimes, the thing you enjoy doing is not the thing that you have to do because nobody else can. But often barred things that are really not fun for me, turned out to be the thing that nobody else can do. I just think that you have to spent some time thinking about that and thinking about what you can teach people too. You already have the knowledge of your project and what you're trying to do so I think what you can teach is what your mission is, what your goals are and maybe they can help you to communicate that too. CHARLES: Yeah, because it seems to me if you actually can very clearly communicate your target, then people can begin to walk towards it independently and that's almost more important than the actual taking the steps. Or the steps needed to be taken but that's something that you can provide. AUDREY: Yeah, you need that kind of definition regardless in order to make your decision and have your work actually function and the less conscious we are about, the more we tend to get a big pile of something and you go, "Now what? What do we do with that?" CHARLES: Right. I think it also flushes out if you have a clear target and you have a clear mission, by externalizing it, it makes you reflect on it more and hardens it, if that makes any sense. You have all these ideas bouncing around in your own head about the things that you might want to do or might like to do but once you actually try to express it to people and say, "You know what? We're going to do this." Then it takes on a reality of its own that is subject to more scrutiny but also subject to the constraints of the real world and that's a good thing. It means that whatever you're going to come up with is going to be more resilient. AUDREY: Yeah. I think we can be scared about putting that out there. They won't see what you see or they won't like it. Those who disagree with your goals there will go, "You really should have been building an eggplant slicer and not a tomato slicer." Yeah, I don't like tomatoes. But for more definition that we put out there, the clearer we are, the more that the people who want to [inaudible] they can find us. That's why it's so important to do it and not to dodge those kinds of questions. CHARLES: Yeah, absolutely. Now, I'm wondering so when is this conference that you're running? Is this the first one or is this the second, the third? AUDREY: Oh, no. We're on our ninth. CHARLES: You are on your ninth? Oh, my goodness. AUDREY: Yeah, it's actually just in a few weeks. It's in June, the week of the 20th, I want to say. Tickets are for sale. If you're in Portland, we had a great volunteer program where you put in eight hours over the course of the entire week. You can split out with everyone and you get a free ticket. CHARLES: Nice. This is the problem with the internet is I'm always finding out things that I wish I'd known 10 years ago. I wish I'd known about this before it actually tried to do any open source. This is the Open Source Bridge so what's a sample of what you guys are going to be talking about? AUDREY: The thing that we've added this year and it's really exciting is the activism track. We're having a lot more people to talk about what they do as code. In this other way, more of public facing way. We have Nicole Sanchez from GitHub. She's going to talk about diversity inclusion and some of the biggest [inaudible] there. We also had Emily Gorcenski doing another keynote and she talks a lot about data and ethics and has a lot of interesting things to say about how we collect and sort and process information and the impacts of that. We have a couple of workshops that are really great. One on technical interviewing and the personal skills that you need. There is a session on keyboard hacking. CHARLES: Keyboard hacking? This is in the activism track? AUDREY: No. This are across all the tracks. CHARLES: How many different tracks are there? AUDREY: There's five. CHARLES: This is a big conference. AUDREY: Yeah. It is such a great community for me to be a part of. Like I said, the different kinds of projects that people come from and bring into it and the different skills, we'll have people that are everywhere from kernel hackers to working in devops to people that kind of fit, I think what we think of it are more typical like web developer or mobile developer kind of skill set. People who run their projects, folks from Dreamwidth often come and participate and they have a lot of really great things to share because they have such an inclusive focus on how they do their project. CHARLES: Where was that? AUDREY: Dreamwidth. It's a LiveJournal spinoff. It's online community journaling website. It's in Perl, which is cool. There aren't as many outward facing things, hiring Perl programmer these days, I think. CHARLES: It's still a very active Perl project? AUDREY: Yeah. CHARLES: Wow. I did Perl a long, long time ago. AUDREY: I think it's really useful to remember that programming languages never actually die. There is always code. JOE: There's still plenty of COBOL positions out there. AUDREY: Yeah. Actually my uncle is a COBOL programmer. CHARLES: Yeah, I remember it was only some statistic where it was something like five years ago, Java, Eclipse, COBOL is the most popular programming language. The cycles are much larger than we tend to think. Surfing on the beach as we do, not realizing there's a whole ocean generating those waves. AUDREY: Yeah, I think if you're in a certain kind of technology startup plan, there's always this push to go for the nearest and shiniest on the number of JavaScript frameworks that we've gone through in the last five years. You kind of [inaudible] of all of these things that come before that are still in use. What I really loved about doing devops is that all of this pieces are still in play and there's something to learn from that. If they don't die, you don't get rid of them. You just try to build on them and keep them working usefully. CHARLES: Right. Man, that's exciting, so you have a very, very huge cross-section of the development community. It sounds like participating in here which is a quality in of itself. That must give you a pretty unique perspective being with that level of cross-discipline. Are there any insights that can only be gleaned by being able to perceive it from that high of a level? AUDREY: Well, a big one is that we all struggle with governance. We don't really talk outside of just a couple of forms for events that focus on open source maintainers. We don't talk about the governance of projects, like who was in charge and how decisions are made. But it turns out that that has just an enormous impact on what a project can actually do and how it survives. I think I might not have seen that as clearly without having people from so many different angles participating. CHARLES: I'm just trying to think of keeping it in the area of web frameworks because that's something that I'm familiar with. If we were to compare, say the governance model something like React, which is basically whatever Facebook wants, versus something in the middle like Angular, which is like an explicit governance model but also is heavily influenced by Google, versus something like... I don't know, well something like JavaScript itself, which has an open democratic model but heavily represented by major, major, major companies, versus something like Rust, which is I certainly get the feeling is a very explicit, very democratic model. All of those seem to have achieved a lot of success and this seemed like a very healthy projects but on the one hand of the spectrum like Rust, you have the super-transparent, super-democratic model and then on the React side, you've got this authoritarian model. That's opaque. How do you reconcile that those are both successful? AUDREY: I think a lot of what actually determines this stuff is who pays the developers. In both of those cases, meaning projects that present information and decision making differently but there are corporations that pay those developers and that's where the primary source of that code. Because of that, really who pays the developers determines what gets made, what code gets written. In a way, they're both doing some of the same things. They're just not giving you inside into that decision making, in some cases. CHARLES: The decision making apparatus is there, I guess the thing is this transparency to the user base matter. I would say that the user base of a thing like React dwarfs the actual corpus of decision makers. That doesn't seem to be that that decision making process is opaque. AUDREY: Well, I might be opening too much of a larger conversation by saying this but if you're familiar with the idea of algorithm transparency, decision making is encoded into things like algorithms and when we can't examine them, then we don't know how that decision was made so we don't know what biases are encoded into it. The same thing happens with code in general. You might say, "Let the outcome of this and this working really great," but there are still biases and preferences that are encoded into that that you don't have insight into. If they start to ship the project in a certain way, that include some users and excludes others. Even on just purely technical levels, you don't know what. You don't know how they got to that, you don't know if they're going to keep steering in that direction. If you're one of those people that is starting to be excluded, you don't know what you can do about it. I've seen these kinds of governance discussion even happen within Ruby in Rails. CHARLES: Yeah, it does seem like these political questions come up constantly. I remember an example that leaps to mind is a project that I was involved with was the Jenkins project, which originally was Hudson, which came out of Sun Microsystems. When Oracle bought Sun, they were basically trying to, I want to say there's always three sides to every story but from where I was sitting, they were essentially trying to subvert the project to their own needs and end up being in a fork of the project. Luckily, there was recourse there where because it was open source and because it was mostly maintained by the community and not by the company, they were able to fork it. They changed the name. They changed the logo and that was the end of the story. There was a question of which fork would survive but that was resolved within probably six months. But Jenkins lived and I think it's better off for it but I guess maybe then a question that you can one kind of stress test that you can put like, "Is it okay to put weight on this technology?" What would happen? Would my community be represented and would I be able to fork this, essentially? Maybe in that sense, React would pass that test. In the sense that it would be reasonable to fork it or something like that. I don't know. I'm just thinking of ways to try and validate if something safe to use. AUDREY: I think it's really interesting that you commented on the new change and the logo change because those kinds of trademarks are actually the most readily protected of all of the intellectual property in an open source projects. If things are going to go off and become a community project and it's being released under some open source model, often where the corporate control stays over those assets -- the name, the logo, the graphics -- maybe even some of the work [inaudible]. You have to ask if that code is still useful without that infrastructure that they provided. If you take the whole codebase and you walk off and you don't have the same developers and you don't have the same, even hosting resources or whatever, is that code still useful to you? What if you use a bug tracker? CHARLES: Right, now you own it. What's the cost now of maintaining? And are you going to get a return on that investment? AUDREY: Yeah. There's been some pretty big open source projects that have struggled with that, especially for end user facing software. Those turned out to be easy things for community to pick up. CHARLES: Can you provide any examples? AUDREY: I'm thinking of some of the stuff that happened with Open Office LibreOffice. CHARLES: Yeah, I remember that. AUDREY: There's still two different batches of people working on this and from what I understand, a whole lot of intellectual property complications. CHARLES: Yeah, it's funny how sometimes, it would be interesting to see a case study of all the major forks and the outcomes of what they were. Some I can think of, there was a fork of Ruby gems, for example I think back in 2009 that went off and was mainly, I think was a way of protest. I think some of those concerns were addressed in the main thing so that fork ended up dying, then you got the fork of io.js, which was ended up. There was a fork and then a rejoining with the Node community but I would say it was an effective tool so there was a fork but then it joined. It was a source code fork but it was a political fork. Then you have the Jenkins fork where the fork basically swallowed its ancestor and there's all these fascinating outcomes and then you've got this LibreOffice Open Office where the waters are very murky about what happened with that fork. AUDREY: I heard people say like, "If you don't like this decision, then just go fork the project." CHARLES: Because that's easy. AUDREY: And if one of your major developers does it, then maybe, like you said, they have some leverage and they can make the changes they want to see happen, [inaudible]. But in general, that's a really hard thing to pull off. You've got to be able to take your entire community with you. Part of this is have to be functional and I think people are very rarely actually make that happen. CHARLES: Right. I feel like that's a dishonest thing to say when people are like, "If you want to go fork it," because really forking the code is the easy part. It's forking the community. AUDREY: Well, if you do that, then you've got a lot of conflicts. You've got a lot of people's feelings to address. It's not a very simple thing to recover from. CHARLES: Yeah. Some people do it. We have some good examples of that happening but it doesn't always pan out for the best. How can we make open source more accessible and supportive of contributors? We've mentioned a lot of that stuff in terms of how you can support people who are contributing but there might be more to talk about that. AUDREY: Yeah, we haven't really talked about who gets to participate. We talked about what kinds of things you can do when you see that people are interested but we don't talk about how in order to be a week encoder, you've got to have those weekends free. Certainly, I am right now. CHARLES: Yeah, neither do I. AUDREY: You have to have access to a laptop if you want to go to Code Sprints or [inaudible]. Not everybody has that, even people who are programming or your own computer not owned by your employer. That can be really important. You have to have a knowledge of how open source works. I do see fairly often in conferences that focus on a lot in open source, there will be how to become an open source contributor kind of talk. That kind of cultural knowledge is really important because otherwise, you're going to GitHub and you look at it and you say, "What am I supposed to do here? What am I actually supposed to do with this?" It's just a wall of information. There's something about a project on GitHub that creates these entry points for somebody who doesn't know how open source projects work. CHARLES: Yeah and it's so hard to be able to perceive it from that person's perspective, especially if you're frog-boiled, so to speak in the community. You've been doing this for so long, these things seem self-evident that it takes a computer, it takes the time, it takes knowing where to establish a toehold. These are all non-problems for you but they're insurmountable for someone else. AUDREY: There's one other aspect of this that we haven't really talked about, which is the friendliness to the kinds of contributors that you have, the diversity of the project versus the homogeneity of the contributors, whether or not you have a code of conduct and you know how to do something with it so that people feel safe and welcome in your environment. There's a lot of people that stay away from open source projects because all they've ever seen is harassment and that behavior. You can have a counterexample but if you don't have some mechanism for showing that that won't happen in your projects, then there are folks that are never going to submit about. They're never going to make a commit. They're not going to put anything on the wiki. CHARLES: Why would voluntarily subject myself to, if the only thing on the other end of the phone is pain? AUDREY: There are plenty of people that decided just to opt out because of that. If open source projects want to see more contribution, you have to be very proactive in dealing with that. CHARLES: Yeah, I feel like it almost would be nice to have some sort of training. Even if you have a code of conduct on your open source project, I think as you grow it from something that's maybe just one or two people to where there's a larger community, the first time you have a bad actor who shows up and start slinging turds, it's shocking and you're taken aback. But just as the number of people grow in a community, that is going to happen. It's just an unfortunate fact of human nature so not having to react to it, but be prepared for it, I think is something that's extraordinarily valuable. I don't know if there's a guide for that on GitHub or guide for that on anywhere else but I think it would be very useful skill to have. AUDREY: It's just very funny that you say this because this is actually a training idea. CHARLES: Oh, really? I promise there was no payment under the table to ask that question. AUDREY: Yeah. There was some consulting around this and I started a program with a local non-profit called Safety First PDX and what we do is train user group leaders, conference organizers, open source project maintainers on exactly that: what to do with their code of conduct to enforce it and help people feel welcome in their community. I worked through a really specific examples with people about how you respond, how you have this conversations and what kinds of things you need to do to protect your contributors who are participants and be really firm about what is next in your space. CHARLES: Absolutely a critical skill for any open source project, for any open source community, for any large accumulation of people. AUDREY: And GitHub made it very easy to put a code of conduct on your project now but without these kinds of resources, I think what happens is that people get that first incident and they panic because it is scary to tell somebody that their behavior isn't okay. To tell them that they might have to step away from the project or stop doing that or even leave indefinitely, those are really hard things to get started doing. I really enjoy doing the training and getting to walkthrough that to people. CHARLES: Are you going to be offering that training anytime soon? AUDREY: We just had one here in Portland last week. We're doing it a quarterly thing but I'm also really open to bringing it elsewhere like a place to host and some sponsorship that they can throw at that and people that want to take this. CHARLES: That'll be awesome. Maybe we can have you in Austin. AUDREY: [inaudible]. CHARLES: Thank you, Joe. Thank you, Audrey for coming on the show. AUDREY: Thanks. CHARLES: It was really great to talk to you. It's great to talk about your history in open source and the things that you're doing in the community, especially the insights that you have around running sustainable open source projects. Also, thank you for talking to us about Open Source Bridge which is, I understand coming up right around the corner. If you want you can go to our podcast page and there will be a link to get $50 off if you enter in the discount code 'PODCAST.' That's $50 off of your open source bridge ticket. Be sure to go check it out. That's it for today, from The Frontside. If you're interested in hiring us, we do have availability starting in July so reach out to us. All right, everybody. Take care.

Toran Billups @toranb | GitHub | Blog Show Notes: 01:44 - New Developments in ember-redux 04:23 - New Developments in the Wider Redux Community 06:26 - Using Redux in Ember 09:40 - Omit 10:45 - Reducers 25:42 - Fulfilling the Role of Middleware in Ember 28:12 - Ember Data in Redux-land 31:24 - What does Toran do with this stuff?? Links: The Frontside Podcast Episode 55: Redux and Ember with Toran Billups The Frontside Podcast Episode 18: Back-End Devs and Bridging the Stack with Toran Billups redux-offline ember-redux-yelp create-react-app "Mega Simple redux” Twiddle ember-concurrency Thomas Chen: ember-redux The Frontside Podcast Episode 067: ember-concurrency with Alex Matchneer normalizr Rich Hickey: Simple Made Easy Other Noteable Resources: ember redux: The talk Toran prepared for EmberJS DC in April 2017 github.com/foxnewsnetwork/ember-with-redux Transcript CHARLES: Hello everybody and welcome to The Frontside Podcast, Episode 69. My name is Charles Lowell. I'm a developer here at The Frontside and your podcast host-in-training. With me Wil Wilsman, also a developer here at The Frontside. Hello, Wil. WIL: Hello. CHARLES: Today, we have a special guest, an actual elite member of the three-timers club, counting this appearance. We have with us Toran Billups. Thank you for coming on to the show today. TORAN: Absolutely. I'm not sure how the third time happened but I'll take it. CHARLES: Well, this is going to be the second one, we're going to be talking about Redux and then I believe you're on the podcast back in 2014 or 2015. TORAN: That's right. CHARLES: That's one of our first episodes. Make sure to get in touch with our producer afterwards to pick up your commemorative mug and sunglasses to celebrate your third time on the show. Awesome. I'm glad to have you. We actually tend to have people back who are good podcast guests. TORAN: Thank you. CHARLES: Yeah, I'm looking forward to this one. This is actually a continuation of a podcast that we did back in January that was actually one of our more popular episodes. There was a big demand to do a second part of it. That podcast we talked about the ember-redux library, which you're a maintainer and just kind of working with Redux in Ember in general. We're going to continue where we left off with that but obviously, that was what? Almost six months ago? I was wondering maybe you can start there and there been any kind of new developments, exciting things, what's kind of the state of the state or the state of the reducer or the state of the store in ember-redux. TORAN: For ember-redux, in particular, we're working on three initiatives right now. The first is making the store creation more customizable. A lot of people that come from the React background in particular are very used to hand crafting how the stores put the other with the right middleware and enhancers and reducers and that's been fine. I wanted to drop people into the pit of success and everybody's cool with that but now we're getting to a point where there are people wanted to do different things and it's great to open the door for those people if they can, while keeping it very simple so we're working on that. We have here that's just undergoing some discussion. We're also, just as the wider Ember community -- you guys maybe involved in this as well -- and trying to get the entire stack over to Babel 6, the ember-cli Babel 6.10 plus stack. There is a breaking change between Babel 5 and 6 so we're also having some discussions about the ember-redux 3.0 version bump at some point later this year, just because we really can't adopt this without introducing basically a breaking change for older ember-cli users. CHARLES: Just in general, this is a little bit off topic, what does it mean to go from Babel 5 to Babel 6, if I'm an add-on author. TORAN: I would probably ensure that need to speak more with Robert Jackson about this. We just kind went back and forth because I thought I had a Babel compile error. He's like, "No, you're missing this dependency which is the object spread." Unfortunately, the object spread is rampant in React projects and this is totally cool. I had to actually add that and that's just a breaking difference between these two. If we adapt the new version of this in the shims underneath of it as an Ember 2.43 user, if you're on node four which is still supported, you will break without this. I'm trying to get some discussion going about what we should do here and if we even should push ahead and just say only node six is supported. There's some discussion and then back to your original question, the third piece is we've introduced the ability to replace the reducer but we need to get some examples for hot reloading the reducer. That's a separate project but it needs to be enabled by ember-redux. Those are the three main initiatives. CHARLES: Being able to you hot load your reducers, just to make changes to your reducer and you just thunk them into the application without having to lose any of the application state and that one of the reasons that's possible then is because you're reducers have no state themselves. They're just pure functions, right? TORAN: Exactly. CHARLES: Okay. Awesome. That sounds like there's a lot of cool stuff going on. Beyond ember-redux, are there any developments to look for on the horizon in the wider Redux community that might be coming to Ember soon? TORAN: Actually one of them is actually fairly new and it's already in Ember and just because I have already got a shim up for it is redux-offline, which I remember you had Alex on two episodes ago about breaking your brain around Rx. I feel like that happened for me trying to build apps offline first. This is, of course just another library that can drop in that place nicely with Redux but I feel like the community, at least it's got me thinking now about what an absolute that would really disrupt someone who's a big player today. I feel like you've built a great offline experience with true and well done data syncing. You could really step in and wreck someone who's in the space today. CHARLES: Right, so this is a sim around... what was the name of the library again? TORAN: Redux-offline. CHARLES: Okay, so it's just tooling for taking your store and making sure that you can work with your store if you're not actually connected to the network and like persisting your store across sessions? TORAN: Yeah. It uses a library that called redux-persist that takes care of and kind of hydrating the store if you have no network connectivity. But it's also beginning to apply some conventional pattern around how to retry and how to roll back. It's just an interesting look at the offline problem through the lens of an action-based immutable data flow story. It's interesting. I don't have a ton of experience with that kind of tweet and rewrote my Yelp clone with it and that was tough. That's what I mean by this. It's like I thought this is very trivial but you have to do a lot of optimistic rendering and then sort of optimistically gen primary keys that gets swapped out later and it's tricky. If you've never done offline first, which I have not, I just think offline is pretty cool and along those lines, there's been a lot of discussion around convention. There's of course, Create React app which is like a little library or CLI tool to kick off your Redux in React projects. It's kind of ember-cli, very trimmed down version of that right now and that's just getting incrementally better. Of course, you guys are in the React space so you may touch upon that story if you haven't already. CHARLES: All right. We talked to the very high level, I think the last time we had you on the show but now that the idea is gaining traction, kind of delve into more specifics about how you use Redux in Ember. I asked at the end of the last podcast, let's step through a use case like what would deleting an item look like in ember-redux land. Maybe we could pick that up right now and just understand how it all connects together. TORAN: Yeah, absolutely. Without understanding how this entire flow or this event bubbling happens is hard to get your head around it. The process we're going to walk through is exactly that use case you laid out, Charles. We're going to have a button in our component and that button, on-click the idea is to remove an item in a list that we happen to be rendering, let's say. If this is a child component like the very primitive literally button that you have and you just have your on-click equal probably a closure action in Ember, the parent component or the outer context is going to be responsible for providing the implementation details for this closure action and what it does. This is kind of the meat of what you're getting at. The high level here is there is a single method on the Redux store that you have access to and it is called dispatch. The nice thing about Redux again, the API surface area is very small. It's just a very handful of methods you need to get your head around. This one, in particular takes one parameter, this dispatch method. That parameter is a JavaScript object. Now, if you're just playing around, you just want to see the event flow up, there's only one requirement asked of you and that is type property. This JavaScript objects have a type that is often a string so it's very human friendly, you just put in there and the string remove item, let's say. Now of course, if you want to remove a particular item in this remove example, you of course want to pass some information as well beyond the type. The type is mostly just a Redux thing to help us identify it but in this case, you'll definitely know the primary key or the ID value, let's say of the item you want to remove. In addition to type, this JavaScript object, let's just say has an ID property and that can come up from the closure action somehow if you want. Once you click this, what's going to happen is you're going to fire this closure action, the closure action is going to invoke dispatch with the JavaScript object and dispatch is going to run through the reducer which is the very next step and what we do is we take the existing state. Let's say we have a list of three items, that's going to be the first argument now in the reducer. The second argument is this action which is just, as I describe it a JavaScript object with two properties: type and ID. You can imagine an ‘if' statement, conditional switch statement that says, "Is this the remove item action? If it is, okay." We had the ID of the item that want to remove and since we have a dictionary where the primary key of the item is the ID, we can just use lodash-omit and we basically use omit to filter out the ID and then use object assigned to a transform or produce a new state and then a callback occurs after this that tells your list component that somewhere in your Ember tree to re-render, now only showing two items. CHARLES: One of the things I want to point out there, you just touched on it but I think it's an interesting in the subtle point is that the lodash method that you used was omit and that's how this is kind of tangential or I'll say parallel to programming in this way is that you don't actually use methods that mutate any state. You calculate new states based on the old states. I think that's a great example of that -- that omit function -- omission is the way that you delete from something in an immutable fashion. You're actually filtering or you're returning a new copy of that dictionary that just doesn't have that entry. You're just a omitting it. You're not destroying the old one. You're not deleting anything. You're not changing it. You're just kind of Xeroxing it but without that one particular entry, which is ironic because the effect on the UI is that you have done a delete but really what you're doing is omitting there. I just think that's cool. I think it's one of the ways that using the systems teaches you to think about identity differently. Then the question I want to ask you was, this all happens in the reducer, what does that mean? Was that word mean -- reducer? I kind of like danced around that idea and I've tried to understand where that term came from and how it helps give insight into what it's doing and I come up short a lot. Maybe we can try and if we can't explain what the name of reducers, maybe there's some alternate term that we can help come up with to aid people's understanding of what is the responsibility of this thing? TORAN: Yeah, I think we can just break down reduce first then we'll talk about how it ends up looking. But I think it's reduced almost like it's defined in the array context. If I have an array of one, two and three and I invoke the reduce method on that, we actually just produced a single value, sort of flatten it out and produce a single value as the result to that, so three plus two plus one, six is the end result. What we've glossed over this entire time, probably last episode is that this reduce word, I believe is used because in Redux, we don't really just have one massive monolithic reducer for the entire state tree. We instead have many small reducers that are truly combined to do all the work across the tree. In my mind is I think reducer fits well here because we're actually going to combine all these reducers and they're all going to work on some small part of the state tree. But at the end of the day, we still have just one global add-on and that's the output. We want one global add-on with a new transform state and that is the reduced state. CHARLES: You take some set of arguments. One of which is the prior state and you reduce that into a single object. I like that. The other place where I've seen this term applied in a similar context is in Erlang. They talk about reduction so that you have an Erlang server where the way that the servers modeled is as a recursive function call where you pass in the prior state of the server plus any arguments if you're handling a request or something like that, bundled in there is going to be arguments of the request and then what that function returns is a new state, which is then used to pass in to the next state. They call this process reduction. We've got two data points. Maybe there, we can go search for the mathematical foundation of that later -- TORAN: I like it. CHARLES: -- if you want to geek out. I think that helps a lot. Essentially, to sum up the responsibility of the action is you take a set of arguments, it's going to take the existing state of the store, run it through your reducers and then it's going to set the next state of the store or yield the next state of the store. Is that a fair summary of what you would say the responsibility action is? TORAN: Yeah. I think you're right. In fact, in preparation for this talk, I just threw together really small Ember Twiddle that will link in a show notes what I call the mega trimmed down version of ember-redux. It's basically a really naive look but for conceptualizing this flow, it's about 24 lines of code that show exactly what you're saying which is I have this reducer, it's passed into this create store method in the syntax, how do this actually look? It better illustrates how the reducer is used when dispatch is invoked so a dispatch, if I was actually to walk line by line through this, which will be probably pretty terrible. But the very first line of dispatch is to just call the reducer. From that new state transformation, we just push in so the store gets a new entry into it's array of here's the next state and because we had never tampered or side effect-ted the old state, we could easily go back in time just by flipping the pointer in the array or that indexing the array back point. CHARLES: I guess my next question is we've talked a little bit about immutability and we know this reducer that we call at the very first point of the dispatch is a pure function. You were dealing with pure functions, immutable data but at least in perception for our users, our system is going to have a side effect. There's going to be calls to the network. We are, in the least in theory deleting something from that list. How do you go about modeling those side effects inside Redux? TORAN: This is a great intro because in fact a friend of mine is actually a teacher at a boot camp and he was telling me the other day that he was asked to do a brief look at Redux and his very first feeling when he was watching some of the Egghead.io videos is like, "Oh, so the reducer is pure but I have to side effects something so where do I do this?" It's not very clear, I think for the very beginner which is why we left it out of that part one podcast. Today, we're kind of hit that head on but before we get into that list, we can talk about what having actions in their simplest form look like today in your Ember app. As I mentioned earlier in this remove example, you got the button, it just takes the closure action the on-click. No big deal. The bigger work was on the parent contexts or the parent component to provide this action, which sounded very simple but imagine instead of just dispatching synchronously, which is we talked through. Imagine we only wants to dispatch that officially to change it if we have gotten a 204 back and the fetch request is deleted on the server -- normal Ajax or fetch-type flow. In this case, you start to add a little more code and imagine for the moment this is all inline in your component JS file. The component now is started to take on an additional responsibility. In addition to just providing a simple dispatch, as I say, "Go and remove this Mr Reducer later," you're now starting to put some asynchronous logic and as you imagine a real application, you grows and try catch stuff, some error handling, some loading, some modals. This gets out of hand. One of the things that I want to touch on briefly is, at least in the ember-redux case we ship this Promise-based middleware and I want to stop right there for just a second because I use that word 'middleware' and immediately we've got at least highlight what this is doing. In that pipeline -- we talked about earlier -- in the component I dispatch and it just goes right to the reducer. Well, technically there's actually a step or an extension point right before the reducer is involved and that is where middleware comes in. Technically, you could dispatch from this action and then you could handle and do the network IO type request in middleware instead, then porting on another dispatch of a different type with the final arguments to be transformed. That's really the role. CHARLES: Can middleware actually dispatch its own actions? TORAN: Correct, yep. In fact, one of the first big differences between this example as I'm kind of hacking around the component and I've got access to dispatch but there's two things I'm really actually lacking if I don't leverage middleware full on. The first is I do have some state in the component but often, it's actually just a very small slice of state that this component renders. If there's actually a little bit more information I need or actually need to tap into the full store, I don't have it and that might be considered a good constraint for most people. But there are times you imagine more complex apps, you need the store. You might even see a little bit more state, middleware provides that is where you trapped with just that slice of state and dispatch the keyword. That's about it in the component. But the other side effects are the benefit, as you break this out is you get another seam in the application where the component now is not involved with error handling and Promises and async flows or generators. It just does the basic closure action set up and fires dispatch almost synchronously like you did in our very simple example, allowing the middleware to actually step in and play the role of, "Okay, I'm going to do a fetch request and I'm going to use a generator." It's almost like the buffer for IO or asynchronous work that it was missing in our original equation. Imagine, you want to debounce something or you want to log something or you want to cancel a Promise, which you can't do, any of that stuff that's going to happen in this middleware component. That's one of the things I like about middleware as I learn more about it and the moment you get to a very complex async task where you're actually doing the typical type ahead, where you literally want to cancel and not do the JavaScript work or you like to cancel the Promise as quickly as you can, you can very quickly dive into something kind of like what you and Alex talk about with ember-currency in the Redux-land. It's called redux-saga. It's just a generator-based async work. CHARLES: Is saga kind of emerged as async library that everybody uses? I know it's very popular. TORAN: Yeah and a good reason, I mean it solves a lot of the problems that if you were to try and do the cancelation token Promise stuff that came out a while back where we're trying to figure out how to cancel Promises. There's a lot more ceremony and just a lot more state tracking on your own that generators and even when I played with this last week, which is actually redux-observable which is an Rx-based middleware. It's built by, I think Ben Lesh and Jay Phelps from Netflix or... sorry, Jay is still at Netflix but anyway... You could use Rx, you could use generators. This really is just the escape valve for async and complex side effect programming that can't or shouldn't take place in the reducer because it's pure. It shouldn't take place necessarily in a component because one of the best pieces of advice I got when I was younger was, " Toran, make sure you do or delegate," and we're talking really about levels of depth in your methods at the simplest. But it applies here as well, which is I would love it if I had just a very declarative component and I didn't have to get into the weeds as I was looking at it about, is this a Promise? Is this Redux thunk, as they call it? Is this a generator or is it Rx? I don't even care in the component for the most part. I just need to know the name of the action and the arguments. If I'm having a problem with the Rx side of the generator, I'll go into the middleware and work from that particular abstraction but you can see the benefits of the seam there. CHARLES: Are the middlewares match on the action payload in the same way that the reducers do? Is that fair to say? TORAN: That is fair and I will warn. If that seems very strange, you're probably not alone. In fact, the first time I did this with redux-saga, I was dispatching, only to turn around then dispatch again. It feels very strange the first time but again, keeping in mind that you're really trying to have a separation from the work that is side effect and the work that is pure. The first action in that scenario, we'll call it remove-saga because it's actually going to fire something to a middleware. That work is all going to be network-heavy and it's not really as easily undo-redo because it's not pure. But the second event invoked from the actual middleware itself that says, "Remove item. Here's the ID. We're good." That work could be undone-redone all day because it hits the reducer, which is sure you can in and out. CHARLES: It sounds like basically the middleware is allowing you to have a branching flow structure because they all do involve more actions getting dispatched back to the store to record any bookkeeping that needs to happen as part of that. If you want to set some spinner state, that will be an action that gets dispatched. But in terms of sync, they allow you to set up sequences of actions or if you basically have one action that will actually get resolved as ten actions or something like that. If you think about an asynchronous process, you have the action that starts it but that might end up being composed of five different actions, right? Like I want to set the application into some state that knows that I've started my delete and that means I want to show like the spinner. Then at some later point, I might want to show progress like this deletes really taking a long time and I might want to dispatch five different actions indicating each one of those little bits of progress. Then finally, I might want to say it's done or it fails so really those got decomposed into 10 actions or five actions or whatever so the middleware is really where you do that, where you decompose high level actions into smaller actions? Or it's one of the places? Is that a correct understanding? TORAN: Yeah, I think if you're an old school developer for a minute, it will cater to the audience that maybe came from early 2000s backend dev. Now, they're still pretty current in web dev. I see it talked about as business logic. I feel like this is really the bulk of the complex work, especially if you're using Rx. You're actually creating these declarative pipelines for the events to flow through. My components are much thinner by comparison. They're truly just fire off this action with the information to kick the async pipeline but in the async piece of it, there's a lot more work happening and that's I think because there's a lot of complexity in async programming. CHARLES: Right, and it's almost like with the reducers then, there's not so much business logic because you're just resolving the implications of the new state. Is that fair to say? It's like now we've got this information, what does this imply directly? TORAN: Yeah, I think there is this old [inaudible] thing where they're talking about what's should be thick. You know, thick controllers or thick models, what should it be? Of course, we never want 'thick' anything, is the right answer, I think. But the apps on building today, I feel like if any was thick-er -- a measure of degree bigger in effort or work -- it is these middleware components right now. I think the nature of what you describe, which is the reducer is not supposed to be doing anything complex. It's literally taking a piece of data in, producing a new piece of data out. Logically thinking about that takes much less effort, I think than the human brain applies to async programming in JavaScript. CHARLES: Right. I think it makes sense and some of these things are just going to be necessarily gnarly and hairy because that's where the system is coupled. I can't say anything about whether the delete succeeded or failed until I've actually fired off the request. Those are implicitly sequenced. There has to be some glue or some code declaring that those things are sequenced. That has to be specified somewhere, whereas theoretically with your reducers, you could just run them all in parallel, even. If JavaScript supported multithreading, there's absolutely no dependency between those bits of code. TORAN: I think so, yeah. I think there are still some challenges because in the reducer sometimes. We can talk about this in a few minutes but you may actually be changing several top level pieces of the tree. If you're de-normalizing, which is what we probably should touch on next, there are some cases where you want to be a little careful but like you said, generically immutable programming enables multithreading. We're not touching the same piece of state at same time. CHARLES: Right as long as that piece of state that you're touching, like you need to resolve the leaf nodes of the tree first but at any siblings, I guess is what I'm saying on there, ought to be able to be resolved in parallel. It's more an exercise in theory or just a way of thinking about it because like why you're able to do those reducers as kind of these pure functions is because there's no dependency between them. I guess I'm just trying to point out that to wrap my head around, there are places where there are just clear sequences and dependencies and those are things that would be in the middleware. TORAN: Gotcha. I came a little scared of service worker. [Laughter] CHARLES: Actually a great point is what kind is the analogous -- if there is anything analogous -- in Ember today that's fulfilling the role of middleware. What's the migration path? What's the alternative, just trying to explore like where you might be able to use these techniques that we've been describing inside your app? TORAN: I think, at least my look at it has been a service injected into a service, which sounds completely bad or sounds broken the first time you see because you're like, "We're injecting a service into an existing service." I say that because, for me at least there is a top level service that owns the data and provides read-only attributes but there should be some other piece of code -- not the component -- that is doing this asynchronous complex processing, we just talk about as middleware, that is often a different service than the service that owns the state add-on. That's what I meant by service-injected. There's some Ember service whose job is to manage the complexity and probably ended up in middleware from the Redux perspective or ember-concurrency is literally solving that in my opinion. They do a lot for you: solving the async problem generally and I haven't dug into ember-concurrency enough to know. The pipeline stuff, I think which you guys talked about, which is an RFC, that may have eventually be what I consider the Rx or redux-saga of the middleware today. CHARLES: Right. I think ember-concurrency is just absolutely fantastic but it is a hairy problem but there's some overlap in terms of what it is solving. I think that is interesting. I guess a case where you would use middleware would be anywhere that you would use ember-concurrency. I think the interesting thing to compare in contrast there is one of maybe advantages or disadvantages -- let's just call it a tradeoff -- is that with ember-concurrency, you have this middleware that is associated with an object. It's associated usually with a component or a route. When things happen to that component, you're able to affect your ember-concurrency process but it does mean, these things are sprinkles throughout your application and the rules that are governing them can be really different, depending on which part you're operating in or just because sometimes you're using them on a route. Sometimes, you're doing it on a component. Sometimes, you're doing it on a service, whereas with the putting in the middleware, it sounds like they're going to all be in one particular place. All right. Let's move on from the simple to the more complex because that's where it's at. We've talked about modeling async processes, we've talked about handling state transitions and all that, nothing typifies that more profoundly in Ember community than Ember Data as just a foundation for state and syncing it over to the network. Love it or hate it, it's very popular. What are the things that you do in ember-redux land? One, is it fundamentally incompatible with Ember Data or is it just more easily served with other alternatives? How do you handle those foundational interactions, those fun foundational async loading network interactions with Ember Data, just using an ember-redux? TORAN: Yeah, for myself, I don't have an experience actually using the two together on purpose. There is a gentleman who did a talk sometime last year and I'll dig up the YouTube clip for you guys, where he talked about his approach where you would actually produce new states so it's still Redux friendly. Ember Data itself just be a new Ember Data model, every time you transformed it. But one of the tricky points is the philosophy of both so in Ember Data, you're just invoking set on everything and not just how it works. That's how the events bubble through the system as you re-render. You never actually create a new state of the system that's a copy, minus or plus, other attributes. You just always touching a single source of truth. I felt like that was always a sticking point. Anyway, Thomas who did this talk and I'll point you guys to it, did a great job of saying like, "Look, if you're stuck in this world with a lot of Ember Data, you're having some pain points with it and you want to try Redux to see if this alleviates those by not changing the state, here is a middle ground," which he did, I think a fabulous job driving it out. Although I must admit, there's got to be some challenges in there just because of the philosophical difference between the two. CHARLES: Yeah. It definitely sounds like there's some challenges but I'm actually pretty eager to go and watch that, to see what they came up with. If you're using these snapshot states of your Ember objects, would it be possible then to take all of your save, delete records, even query and have them inside of middlewares like have a redux-saga for every single operation you want to take on the Ember Data store. TORAN: The example he showed is basically the best of both worlds. You don't want to ember-mutate so he has a special bit of code to do that. But because the rest of it is vanilla Ember, you could drop in concurrency if you want to do the saga-type generator stuff. But you could also just make your changes as you would otherwise. You use the adapters, you fetch, you save, you delete, whatever you want to do for the most part. It saves a lot on the de-normalized side, which you would have to do manually. You don't write any Ajax code, which you have to do manually on the Redux side. I think there are benefits if someone could get it to work where you're just not changing the state of Ember Data, which may actually just be the future Ember Data at some point as well. CHARLES: It sounds like there is a pathway forward if that's the way that you want to go and the road that you want to walk so we'll look for that in the show notes. But my question then is, you're here on the podcast, what do you do? TORAN: I do want to have one disclaimer here, just so that I'm not a complete poser but I am. If you guys don't know this, I'm not trying to hide this from the community but I don't work on ember-redux at work. I don't have a side gate, making money with it. I don't use it ever. I literally just build examples to try stuff out. There's both a blessing and a curse of that. The curse is that you're asking, "Hey, Toran, you're the author. How does this work, man?" I can give you my run at it which I will but there is the other side which is very clear is that I have not built and shipped Facebook -- the current company I'm with -- with X million people hitting every month. We're not using it. This isn't exactly 'Toran-stamp of approval' here but I do mess around with -- this week in particular -- Rx which I like. I think Rx is just something that it changes the way you think about the way programming, especially async programming works. I definitely cannot comment much on Rx other than I like Alex's challenge to the community on your podcasting. Go use Rx, even if you use ember-concurrency or don't use ember-concurrency, how to break your brain. It will be for the better. Actually, Jay did a mini code review with me because my first pass at Rx was just using the fetch-promise because I was like, "I want Rx for side effect modeling but I wanted to still work with Ember acceptance testing," because I still feel like Ember is leading in that way, as you guys talk about in podcast recently. What was really cool is actually there's a shim that obviously Rx has its own little Ajax thing but it is not actually Promise-based. The advantage of this that Jay called out is in the Promise-based, where I'm using ember-fetch, let's say and I'm just wrapping it with Rx, those Promises are still not really cancel-able so what Jay was warning me about is like, "If you're going to use this quick and dirty, great. but in a real app, these will still queue up in Chrome or IE and block the amount of network requests you can actually make," so don't use Promises, even though they're very familiar. Use this operator, I think it is or a helper inside Rx which is the Ajax non-Promise-based operator. Long story short though, there's a good amount of work involved that grass is greener. In Ember Data, if you've ever used 'belongs-to' or 'has-many', you have done the most magical thing right there. In all the right ways, it is amazing because once you're in Redux and you're like, "I have this very nested object wrap," but Redux isn't meant to operate on this nested object wrap. It's meant to operate on this single tree structure, at these many top level entities as I can. As a project that's pretty popular in React called normalizer, you will likely use this project eventually. Maybe, not your first 'Hello, world', but you'll use this to actually break apart or truly de-normalized the structure. What that does a lot of times is if you have a blog with comments nested all inside of it from your JSON API call or your GraphQL call, that's fine coming from the network. But since you're going to have different components listening for just the comments, maybe or different components that just listen and re-render when the blog name changes and they don't care about the comments, you want to actually de-structure that so you have a separate blog top level item and you have a separate comments top level item. They're still related so the blog can get through its comments and vice versa as belongs-to and has-many works in Ember Data but you've got to do that work now. There is, of course magical projects like redux-orm that I just can't speak to how well they work or don't work but they try and solve the more Ember Data, look at the problem which has define this and there's the RM take care of the magic for you. I actually don't mind normalizer. It's just something people should be aware of because it's more work. You've got to break that apart yourself, just as much as writing your own network requests. You're, hopefully not going to duplicate Ajax all over the place but you will have to do the work that you otherwise do not do in Ember Data for sure. CHARLES: It's very interesting. If you look at the Ember Data internals, it sounds like the Ember Data store is actually structured very similarly to the way you had structure a Redux store using something like normalizer where you have these top level collections and then some mechanism to both declare and then compute the relationships between these collections of top level objects. But I want to go back to your other point too. I just wanted to say this. Toran, you know, don't sell yourself short because you give an incredible amount of time, an incredible amount of support to the community. You're very active in the Ember Redux channel. When problems come up, you think about them, you fix them. Even if you're not actually watering the trees, you're planting the seeds. I think that's actually great. I think that is a very valuable and necessary role in any community is to have people who are essentially the Johnny Appleseed of a particular technology. I think you go around and you throw these seeds around and see where they take root, even if you're not there. You're on to the next shady lane to plant seeds, rather than stay and enjoy the shade and the fruit of the apple trees. TORAN: Yeah. I appreciate your kind words there because a couple of years ago, I got into open source because it provides good personal branding. It's like, "This project, it's Toran's. We should hire Toran." It just makes you look more from that perspective. It also gives you almost a way to skip out on tough interviews at times if people are like, "This guy have a decent program. Let's take a look at his PR. He communicates with other humans online." It gets rid of some of that. But there is a dark side. We don't talk about it because there is an upside to it, especially for consulting but the dark side can be time commitment: how much bandwidth do you have outside of your family life, your hobby, if you have one and any other open source or work-related stuff that you already have to do. For me, this is really an exercise in thought leader-y stuff. I saw the benefits of this. It made my Ember better. Even if I wasn't using Redux, it just made the Ember code I wrote at work better. It inspired me to look at different things like ember-concurrency and Rx, things that are just way out of my comfort zone two years ago. I think those are all the benefits that come with it but the easiest part is got to be some value from it. The juice is it worth the squeeze. I think the community we've built and the people using it and the problems we're solving are all definitely worth the squeeze. CHARLES: It definitely is and you can tell from the vibrancy of that community that a lot of people are experiencing that value from it. To your point, I think something that is often lost on people is that you can actually use a project without actually using it. I think that there might be many people, for example in the Ember community that have never use React but are actually in fact using it because of the wonderful patterns that have come out and it's brought to the fore. I had thought about immutability, certainly on the server side but I had thought about it really deeply on the client side until a library like React came along and people started talking about it. I would say before we actually started using React the library, we were using it in thought. You touched on that when you were saying, "It's changed the way I think, it's changed the way I code." Even it's changed the way that I do things at work, in fact you're using it in spirit, if not the actual structure but I almost feel like that's more important. It's longer-lasting and has a greater impact on you, 10 years down the road or even five years down the road when neither of the technologies that we're actually talking about today are even going to be in wide use. TORAN: Yeah, that's true. In fact the one thing I would call out that people check out, I think Alex mentioned this or at least you guys have to talked about it in passing but definitely, sometime this weekend, watch the Simple Made Easy talk by Rich Hickey. It will definitely make you think differently, regardless of the simple side or the easy side that you follow on, projects of course make tradeoffs both sides of that but it is a great talk. Especially, if someone who's been programming six months or a year or two years, they're going to get huge benefit from it, just as much as someone older like myself who has got 10 plus years in the biz. CHARLES: Yeah, I know. That is a fantastic talk. We need to link to it at every single show. TORAN: Exactly. CHARLES: Well, I think that is a fantastic note to end on so we will wrap it up. That's it from Frontside for this week. We're going to have you back, obviously Toran there's so much that we could cover. Six months down the road, we'll do part three but for now, that's it. Thank you, Wil. Thank you, Toran for podcasting with us this morning. TORAN: Thanks for having me on guys. I really appreciate it. CHARLES: Then everybody else, take it easy and we'll see you all next week.

Michael Coté: @cote | cote.io | Pivotal | Software Defined Talk Show Notes: 00:54 - Pivotal 04:39 - Being a Professional Muller aka Analyst 11:08 - Iterative Development 32:54 - Getting a Job as a Professional Muller aka Analyst Resources: Pivotal Cloud Foundry GemFire Greenplum Pivotal Labs Wardley Maps Software Defined Talk Episode #79: From a vegan, clothing optional co-op to working with banks and oil companies - Coté's professional life, part 1 Software Defined Talk Episode #85: Being an analyst without being an asshole - Coté's professional life, part 2 RedMonk Transcript: CHARLES: Hello everybody and welcome to The Frontside Podcast, Episode #66. I am a developer, Charles Lowell at The Frontside and also host-in-training for 65 episodes. This is my 66th and I'm flying alone this week but we do have on the show with us a very special guest. Actually, the person who taught me how to podcast, I think it was about 10 years ago and he was like, "Charles, we should do this podcasting thing." I started my very first podcast with him and I still haven't figured it out. But his name is Michael Coté and he's a fantastic guy and welcome to the show, Coté. MICHAEL: Thanks for having me, Charles. It's great to be here. CHARLES: Now, what are you up to these days? You're over at Pivotal. MICHAEL: That's right. I work at Pivotal and probably people who are in the developing world know them for Spring. We have most of the Spring people. Then we also have this thing Pivotal Cloud Foundry. We're not supposed to call it a platform as a service but for matters of concision, it's a platform as a service that's the runtime that you run your stuff in. Then we also have a bunch of data products like GemFire and Greenplum and things like that. Then, 'openymously', if that's a word, we have Pivotal Labs. Now -- CHARLES: I think, it's eponymously. MICHAEL: Eponymously, yes. Now, you might remember Pivotal Labs as the people who use Chef Scripts to configure their desktops. Remember that? CHARLES: Yeah, I remember that. I was into that. MICHAEL: Yeah, in coincidental kind of way, the inspiration for the project Sputnik thing, which is coincidentally because now Dell Technologies owns Pivotal so all of that stuff has come for a full circle. I guess also since I'm intro-ing myself, I work on what we call the Advocate Team because we don't call them evangelists. No one likes to be called that I guess. I guess there's 12 of us now. We just hired this person, also in Austin actually McNorma who's big in the Go community and apparently can make images of gophers really well. I'm sure she does many other extraordinary things, not just the illustrator master. Everyone else basically like codes or uses the terminal but I do slides. CHARLES: Well, that's your weapon of choice, right? It's a more elegant weapon for civilized time or something like that. I'm going to look it up on Wikia. MICHAEL: Yeah, basically what we do on our team is we just talk about all the stuff Pivotal does and problems that we solve in the way people in an organizations like would think to care about our stuff. Most of what I do is I guess you call it the management consultant type of stuff. Since I have a background as an analyst and I used to work on corporate strategy and M&A at Dell so I have a vantage point in addition to having programmed a long time ago. If you're changing your organization over to be more agile or trying to devops, we would say cloud-native with a hyphen. How do you change your organization over what works and doesn't work? Most people in large organizations, they sort of pat you on your head. I'm sure you encounter this. That sounds really nice that we would be doing all of the good, correct ways of using computers but we're basically terrible and we could never make that happen here. Thanks for talking with us, we're going to go back and stew in our own juices of awfulness. You've got to pluck them out of that self-imposed cannibal pot there in the jungle and show them that they actually can improve and do things well. CHARLES: Would you say you feel like your job is being that person who shakes them away and can be like, "Good God! Get a grip on yourself!" MICHAEL: Sure. That's a very popular second or third slide in a presentation -- the FUD slide, the Fear of Uncertainty and Doubt slide where you're basically like, "Uber!" and then everyone just like soils their pants because they're afraid that are like Airbnb and Uber and [inaudible] and Google is going to come in and, as they say, disrupt their state industry. I try not to use the slides anymore because they're obnoxious. Also, most people in large organizations nowadays, they know all of that and they've already moved to putting on a new pair of pants stage of their strategizing. CHARLES: You've got the kind of the corporate wakeup call aspect of it but then it's also seems like a huge component of your job which is when you were at RedMonk, when you were at 451 and even to a lesser extent, it was Dell who was paid well to just kind of mull it over, like just kind of sit there and asynchronously process the tech industry, kind of like organizational yeast and let it ferment, kind of trying to see where the connections lie and then once you've made that presented, do you think that's fair? That's what sprung to mind when I heard you say like, "Yeah, we just kind of sit around and think about what is Pivotal and what does it do and what's it going," but like how do you get that job of like, "I'm just kind of a professional muller." MICHAEL: That's right. First of all, I think professional muller is accurate, as long as, I guess mulling is also for -- what's that thing you drink at Christmas that you put the little -- CHARLES: Mulled wine. Like low wine. MICHAEL: I can feel like that sometimes late at night. But having a job as an analyst, I was an industry analyst at two places for a total of about eight years or so. Then as you're saying doing strategy at a company, now what I do here, essentially a lot of what you do is very difficult. I know it sounds to people. You just read a lot of the Internet. You just consume a lot of the commentary and the ideas of things that are going out there and you try to understand it and then synthesize to use that cheesy word. Synthesize it into a new form that explains what it is and then finally, the consultant part comes in where you go and meet with people or you proactively think about what people might be asking and they say something like, "What does this mean for me? And how would I apply it to solve my problems?" I guess as an example of that -- I apologize for being a little commercial but these are just the ideas I have in my head -- Ford is a customer of ours and they also have invested in us which is kind of novel. We have GE and Ford invested in Pivotal and Microsoft and Dell Technologies as an interesting mix but anyways, they have this application called the Ford Pass Application. I drive a Ford Focus -- CHARLES: Like Subaru? But you do drive a Ford. MICHAEL: Yeah, because I don't care about cars. It's a bunch of nonsense. I see this app and basically the app, if you have a more advanced one, it might tell you your mileage and even like remotely start your car. But it doesn't really do that much. You have the app and it will tell you information about your car and where to park and it even has this thing where it links to another site to book a dealership thing, which is annoying. CHARLES: Why would you want to book a dealership? To buy another car? MICHAEL: Well because the Ford Focus I have is notorious for having transmission problems so you're like, "I got to go and take it into the dealer to get all this recall stuff taken care of," so wouldn't it be nice... I don't know if you've ever worked with a car dealer but it's not desirable. CHARLES: Yeah, it would be nice if they didn't charge $6000 for everything. MICHAEL: Right. It's a classic system of having a closed market, therefore that jacks up prices and lowers customer service usually. What's the fancy word if there is a negative correlation, if you were to chart it out? Like price is negatively correlated to your satisfaction with it. Kind of like the airline industry, not to bring up a contemporary topic. You pay a lot of money to fly and you're like, "This is one of the worst experiences I've had in my life," whereas you go to the dentist and get a root canal and you're like $20 co-pay. Loving it. [Laughter] MICHAEL: Anyhow, this Ford Pass application doesn't really do very much so what does that mean for what I was explaining. If you go look up and read about it, starting back in the late-90s, your extreme programming and then your Agile Software Development and your devops nowadays, one of the major principles is what you should do is ship often. Maybe you should even ship every week or every day. Don't worry about this gigantic stack of requirements that you have and whatever you should be shipping all the time and then we've trained ourselves to no longer say failing fast. That was a fun cheeky thing back in the late-2000s. CHARLES: Did we trained ourselves not to say that anymore? MICHAEL: I don't hear it very often. CHARLES: Man, I got to go scrub my brain. MICHAEL: Yeah, well this is why you consult with me every 10 years as I tell you the new things. CHARLES: Okay, here we go. We're going to have you on the podcast again. MICHAEL: That's right. You have this idea of like, "We should be releasing weekly," but then if you go to Ford, you're like, "What does that mean?" To shave the shaggy dog here, essentially the idea that they're shipping this mobile application that doesn't really do very much is an embodiment of the idea that they should be shipping more frequently. This may be a stupid example. It's not that it's not going to do very much like permanently but as I have witnessed, very frequently they add new features so Ford is in this cadence but there's this app that instead of working on an application for two years and having everything in it, they're actually releasing it on, I don't know if it's weekly but they're releasing it on a very frequent basis, which allows them to add features. What that gets you is all the advantages of a fast iteration cycle small batch thing where they can study this actually a good feature. They can do all your Lean Startup nonsense. That's a very like weird, perhaps example of how you explain to someone like a large car manufacturer like Ford, this is what devops means for you. Therefore, why you should spend a lot of money on Pivotal? Now that's the part that lets me pay my mortgage every month, the last bit there. CHARLES: Right so Pivotal builds apps. MICHAEL: Well, the Labs people build apps for you. CHARLES: I'm kidding Coté. MICHAEL: Yeah, they actually do. The Labs people are like a boutique of another boutique like ThoughtWorks is kind of a boutique but they're kind of a boutique-y version of ThoughtWorks. That probably is terrible as someone who markets for Pivotal to do that. Do you ever notice how political candidates never really name their opposition? Like you never really want to name your competition but anyways... CHARLES: Pivotal marketing are going to come crashing through your window. Everybody, if we hear them in the next five seconds -- well, I guess you can't call 911 because this is not live. MICHAEL: Yeah, that's true. The Labs people build stuff for you and then the part that I work, in the Pivotal Cloud Foundry people, they have the actual runtime environment, the cloud platform that you would run all that stuff. Plus all the Spring nonsense for your microservices and your Spring Boot. I understand people like that. CHARLES: So good for Ford, for actually being able to experience, either in the development and the joys and the benefits that come with it. But this is actually something that I actually want to talk about independently was as I kind of advance in my career, I find myself pushing back a little bit against that incredibly tight, iterative schedule. Shipping things is fantastic and it's great but I find so much of my job these days is just trying to think out and chart a course for where those iterations will carry you and there is a huge amount of upfront design and upfront thought that it is speculatory but it's very necessary. You need to speculate about what needs to happen. Then you kind of measure against what's actually happening but I feel that kind of upfront design, upfront thought, we had this moment we're like, "We don't need that anymore. Let's throw it all in the garbage." In favor of doing things in these incredibly tight loops and finding where's the clutch point, that kind of long range thinking and long range planning comes and meets with the iterative development. I have no idea. What's the best way for those to match up those long cycles and those short cycles? Where is the clutch play? MICHAEL: I'll give you two and a half, so to speak trains of thoughts on that. One of them is I think -- CHARLES: Two and half trains of thought, I like that. Can we get straight to the half train of thought? MICHAEL: Yeah, I'm going to start with the half, which is just taking all of your questions and putting periods at the end of them before I round up to answering the question. I think a lot of the lore and the learnings you get from the Agile world is basically from consultants and teams of consultants. Necessarily, they are not domain experts in what they're doing so their notion is that we're going to learn about what it is we're doing and we don't actually know we can't predict ahead of time because we're not domain experts so they almost have this attitude of like, "We'll just figure it out on the job." Let's say The Frontside gets hired to go work on a system that allows the Forest Service to figure out which trees to go chop down or not -- CHARLES: If you're the Forest Service, we are available to do that. MICHAEL: I'm guessing you don't have a lot of arborists who have 10 or 20 years of experience working there. CHARLES: No, we don't. MICHAEL: And so you have no idea about that domain so in doing an iterative thing, you won't be able to sit down and predict like everyone knows that when you send the lumberjacks out, they're going to need these five things so we're going to have to put that that feature on there. They need to be able to call in flapjacks when they run out. That's just what's going to happen so you don't know all of these things they need to do so you just can't sit down and cogitate about it ahead of time. Also this comes in from the Lean Startup where there's a small percentage of software that's actually done globally and the notion of a Lean Startup is that when you're doing a startup, you're never going to be determined what your exit is, how you cash out, whether that's building a successful long term company while you get sold to someone or whether you IPO, you're not going to able to predict what that business model is so you just need to start churning and not think a lot ahead of time. Now, the problem becomes, I think that if you are a domain expert, as you can do the inverse of all the jokes I was just making there, you actually can sit down and start to predict things. You're like, "We know we're going to need a flapjack service," so we can predict that out and start to design around that and you can do some upfront thinking. Now similarly, developers often overlook the huge amount of governance and planning that they do for their own tools, which I know you're more cognizant of being older or more experienced, as they like to say. But basically, there's a bunch of, as we used to call it when I did real work and develop stuff, iteration zero work like we're going to need to build a build system, we're going to need a version control. You actually do know all these things you're going to need so there are all the things you can plan out and that's analogous to whatever domain you're working in. Sometimes, at least for your toolchain, it is worth sitting down and planning out what you want. Now, to hold back the people who are going to crash in my window, one of the things you should consider is using Pivotal Cloud Foundry. That's probably something you should cogitate on ahead of time. CHARLES: I think they're going to crash through your window and give you a Martini, if the marketing ninjas are going to do that and if you mention them in a positive light. MICHAEL: You know, it's 10:52 Central but if we were in London, it would probably be an appropriate time so we'll just think about that. Now, on the other hand, you don't want to go too overboard on this pre-planning. I'll give you an example from a large health insurance company that I was talking with recently. They had this mobile app -- it's always a mobile app -- that had been languishing for 15 months and it really wasn't doing anything very interesting. It was just not working well and they could never release it. This is a classic example of like, "We took a long time to release a mobile app and then we never released it again and then it blows." It's not achieving all of the business goals that we wanted. Mostly, what a health insurance company -- I've talked with a lot of the health insurance companies -- want with their mobile app is at least two things and probably many more but these would be the top of the list. One, they want their customers, their users to look up what their health insurance is, figure out doctors they can go to, the basic functioning that you expect from your health insurance company. And two, they want to encourage their customers to do healthy behaviors because if you think about it as a health insurance company, health insurance in my mind is basically like this weird gamble of like, "I'm gambling on the fact that you are going to be healthy," because then I pay out less to you and you just give me money so the healthier that your users can be, the more profit you're going to make. That's why they're always trying to encourage you to be healthy and stuff like that. The mobile app was not achieving, at least these two, if not other business goals they have. They basically were rebooting the effort. The way they started off is they had -- I don't know how many inches thick it was -- a big, old stack of requirements and the first few iterations, the product team was working on it and talking with the business analyst about this and going over it and what they sort of, as we were calling Pivotal Labs the product owner but the person who runs the team, realize is like -- to cut a long story short -- "This is kind of a waste of time. We shouldn't just prioritize these 300 features and put them in some back road and execute on them because these are the same features that we based the more abundant application on, we should probably just start releasing up the application," kind of like the FordPass app. That said, they did have a bunch of domain experience so they had a notion of basically what this app was going to do and they could start planning it out but they figured out a good balance of not paying attention to, as Martin Fowler used to call it the almighty thud, of all the requirements. What they ended up doing is they basically -- CHARLES: What's the almighty thud? MICHAEL: You know, he's got some bleaky or whatever. It's basically like we started a project and I think it's from 2004 and someone FedExed me about 600 pages of an MRD or whatever and I put it down on my table and it made a loud noise so he calls that the 'almighty thud', when you get this gigantic upfront requirement thing. What happened in this health insurance thing is they stopped listening and talking with those people and they kind of like chaff them out, not like when your rub your legs together but they kind of distracted them to that fact but eventually, they just got them out of the cycle and they started working on the app. Then lo and behold, they shipped it and things are working out better now. CHARLES: Hearing what you're saying and kind of thinking it over, I think if you're going to have an almighty thud, what you really want is you want all that upfront research and all that upfront requirements gathering or whatever, not necessarily to take the form of a set of features or some backlog of 300 things that the app 'needs' to do or 'should' do but just a catalogue of the problems, like a roadmap of the problems. MICHAEL: Exactly. CHARLES: You know, that actually is very valuable. If it's like, "These are things that are true about our users and these are the obstacles that they face. If we do choose that we want to go from Point A to Point B, where we are at Point A, then we actually have a map of what are the things that are sitting in front of that and what are the risks involved." It's like if you got -- you played, you're from my generation, you play the Oregon Trail, right? MICHAEL: Yeah. "You have dysentery." CHARLES: Right. I don't know where I'm going with this analogy but my point is developing that app is like going from Kansas City to Portland. But the thing about software is you don't necessarily have your corn meal. You don't need to say like, "We're going to need six pounds of cornmeal and we're going to need these wagons and we're going to need these mules," because this is software and you can just code a mule if you need it. But you might not need a mule, if the rivers are not in flood... I don't know. Like I said, I don't know where I'm going with this analogy. But do you see what I'm saying? The point I'm trying to make is that having the map of the Rockies and where the passes are is going to help you. MICHAEL: Yeah, this is probably where I'm supposed to expertly rattle off what Wardley maps are and how they help, which is fine. I think that's a great tool. There's this guy Simon Wardley and he's actually a great contemporary philosophizer on IT-led strategy. I think he works for CSC who no longer owns mercenaries but they used to -- Computer Science Corporations. I think they own a little bit of HP Services Division but he works for some think tank associated with CSC and he has got a couple of OSCON talks on it, where it's called a Wardley map and it's a way that you start figuring out what you're saying, which is to say your company's strategy. Using your front metaphor of the era of tall hats, if you remember that other movie, if you're on the Oregon Trail, broadly your strategy is -- and people get all up in your face about the difference between a plan and a strategy and we'll just put mute on them and edit them out of the audio because they're very annoying -- CHARLES: We'll call it an approach. MICHAEL: That's right. Your plan or your strategy -- and pardon me if I use these phrase free and loosely and everything -- is you would like to get to Oregon and you would like to live there and maybe grow apples or start a mustache wax company or some donuts, whatever it is you do out there once you get to Oregon and their strategy is -- what are the assets that I have. I have a family, I have some money and I also know some people who are going there so I'm going to buy a stagecoach and a mule, then I'm going to kind of wangle it out and we're going to go over there. Also, part of our strategy is we're going to go through the northern pass because we're used to winter versus the southern pass, which isn't the Oregon Trail because reasons. Maybe Texas isn't part of The Union yet so I don't want to deal with the transition between whatever that weird Texas thing down there -- CHARLES: The desert, there's the southwest and the desert. MICHAEL: I don't have the capabilities to survive in a desert so I need to go to the north and hopefully I won't be like that movie and have a grizzly bear rip up my backside and everything. You sort of put together this plan. Now going back to what you would do in IT world is to your point, someone does need to define what we would call the business value or the strategy, like what you want to do. Looking at the Ford thing, what Ford wants to do is they do cogitating thing ahead of time and they're like, "We manufacture cars," and you've got electric cars and Uber. That's where the scarce light comes in. In the future, who knows that people will still buy cars? It might be like that I-Robot movie where all the cars are automated and you just go into one. As a company, whose responsibility is to be as immortal as possible, we need to start making plans about how we can survive if individuals no longer buy cars. Let's do that. This is a huge upfront notion that you would have and then that does trickle down into things like my Ford thing -- I'm kind of speaking on their behalf -- if we have a direct connection with people, maybe eventually we introduce an Uber-like service. You can just check-out a Ford car. Then maybe this and maybe that. It's the strategy of how do we set ourselves up to do that. Now, I think the Agile people, what they would go for is it's really good to have that upfront strategy and you'll notice that in a lot of lean manufacturing in Agile talk, no one ever talks about this stuff, much to my extreme annoyance. They don't ever talk about who defines the strategy and who defines that you're working on this project. That's sort of left as an exercise to the reader. The Agile people would say like, "The implementation details of that are best left to the development team in an Agile model." Just like the developers are always arrogantly are like, "Hey, product manager. How about you f-off about how I should implement this? I am the expert here and let me decide how I'm going to implement the feature that you want for me." It's kind of like that rushing dolling down of things. To the development team, you worked on some, what was it? Band frame wire thing, a long time ago? It was basically like, "We don't know it. Maybe this is not the case. Let's pretend like it was." We don't know exactly how you're going to implement this stuff but our goal is that there's bands and they need sides and ways of interacting with their users so let's just figure out what that looks like but they had that upfront idea of ways that they were doing things. CHARLES: Let's start walking. MICHAEL: To add on some more. There's another edge case that you're making me think of, which is a good way of thinking through almighty thuds versus how much planning you have and that's government work. Government work that's done by contractors and especially, military contracting work. What you notice in government work is they have, seemingly way too much paperwork and process. They literally will have project managers for project managers and the project managers have to update how the project is going and they reports. If they don't do the reports correctly, their contract is penalize and you might even get fired for doing it. If anyone stops and says while the software is working, they were like, "No, no, no. don't be naive. It doesn't matter if the software is working or not, if we don't fill up the project report, we're fired." Until someone like yourself or me, it's just like your head explodes and you're like, "But working software, not a concern." In that case, it actually is part of the feature set, part of the deliverable is this nauseating amount of project reporting and upfront requirements, which has this trickle-down effect of annoyance but that's what you're getting paid for so that's what you do and if you want to make yourself feel better about it. I don't know how it is in the rest of the world but in the US, basically we think the only person worse than maybe, Lucifer is the government. I don't know why this comes about. We enjoy the fruits of the government all the time but for some reason, we just think they're awful. Whenever we give money over the government, we want to make sure that they're spending it well and if they're not corrupt and they don't hire their entire family to help them run the government and make sure that they're making extra money globally in their businesses, I wouldn't know anything about that. But essentially, you want to make sure there's no corruption so transparency is almost more important than working software. The way you achieve that transparency is with all this crazy documentation. CHARLES: Here's the thing. I agree the transparency is fantastic but nothing is more transparent than working software. Nothing is more transparent than monitored software. Nothing is more transparent than software whose, by its very nature is radiating information about itself. You can fudge a report but you can't fudge a million happy users. MICHAEL: Don't get me wrong. I'm not saying that the way that things currently operate is the ideal state. I'm saying that that desire for transparency has to be addressed and for example, using your example, let's say you were delivering working software but you were also skimming 20% off the top into some Swiss bank account -- you're basically embezzling -- and then it turns out that you need 500 developers but you only actually had 30 developers. There was corruption. The means even though the ends, even though the outcome was awesome, the means was corrupt so that's the thing in a lot of government work that you want to protect against. I just bring that up as an edge case so a principle to draw from that, when it comes to almighty thudding is like sometimes, that is part of the deliverable. We would aspire in our fail, fast, Agile world to not have a bunch of gratuitous documentation as part of the deliverable because it seems like a waste. It would be like every morning when you battle with your kids to get their shoes on, you had to write a two-page report about how you're getting ready to go to school stuff with your kids was going. As a parent you would be like, "I don't need that." However, maybe if you were like an abusive parent and it was required for you to fill out a daily status report for you to retain the parentship of your kids, maybe it would be worth of your time to fill out your daily status report. That was an awfully depressing example there. CHARLES: Let's go back to the Oregon Trail. What I'm hearing is that -- and we will take it back to the Oregon Trail -- you also need to consider, as were saying, you have some sort of strategy which is we want to go sell apples and moustache wax. But what we're going to do is we're just going to start walking, even though we don't have a map. But obviously, if you send out scouting missions, like you know where you're going, you know the West Coast is out there somewhere, you start walking but the stakes determine how much of your resources you spend on scouting and map drawing -- MICHAEL: Yeah. My way of thinking about strategy and again, people strategy is this overloaded word. But my way of thinking about strategy is you establish a goal: I would like to go to the West Coast. Now, how you figure that out could be a strategy on its own, like how did you figure out you want to go to the West Coast. But somehow, you've got to get to a prime mover. Maybe those tall hat people keep beating me up so I want to go to the West Coast. I want to go the West Coast is the prime mover. There's nothing before that. Then you've got to deal in a series of constraints. What capabilities do I have, which is another way of saying, what do I not have? And what's my current situation and context? On the Oregon Trail thing, you might be like, "I have a family of seven. I can't just get a horse and go buy a pack of cigarettes and never show up again." I guess I could do that. That's probably popular but I, as an individual have to take this family of six other people. Do I have the capabilities to do that? How could I get the cash for it? Because I need to defend against all the madness out there, I'm going to need to find some people to meet with. You're thinking and scenario planning out all of this stuff and this gets to your point of like, "If you're going to Oregon, it probably is a good idea to plan things out." You don't want to just like the next day, just figure it out. [inaudible] tell a joke. It's like, "Why do they sell luggage at the airport? Is anyone is just like, 'Screw it. Pack a clothes and we'll sort it out at the airport.'" It's an odd thing to sell at the airport. But you do some planning and you figure out ahead of time. Now, to continue the sort of pedantry of this metaphor, the other characteristic of going to the Oregon Trail, unless you're the first 10 people to do it is hundreds, if not thousands of people have done it already so you kind of know what it's going to be like. It's the equivalent, in a piece of software, if they were like, "This application is written in COBOL. I want you to now write it in --" I don't know, what are the kids do nowadays? Something.io? I-want-you-to-write-this-in-a-hot-new-language.io and basically just duplicate it. You're going to still have to discover how to do things and solve problems but if the job is just one-to-one duplicate something, then you can do a lot more upfront planning for it. CHARLES: While you're doing it, making the Uber and Airbnb. MICHAEL: Yes. CHARLES: Then you're done. MICHAEL: I think that's the truth and I want to put it another way. We used to be down here in Texas, the way we run government here is just lovely but we used to have this notion of a zero-budget, which is basically like, "Assume I'm going to give you nothing and justify every penny that I'm going to give you." I think that's a good way to think about defaults. I mean, about requirements is default is you don't need any and only get as many requirements as you need. If you're building tanks or going to the Oregon Trail, you might need a lot of requirements upfront that are actually helpful. CHARLES: But like a suit, you're just going to just strike out naked walking with. MICHAEL: That's probably a bad idea unless you -- CHARLES: Yeah, that is a bad idea but that's the bar but what happened if I were to do that? I might make it for 20 miles. MICHAEL: And build up from there and then have all the requirements that you need. I'm sure when Lewis and Clark went they were like, "We're going to need a quill and some paper and maybe a canoe and probably some guns and then let's see what happens." But that was a whole different situation than going to establish Portland. CHARLES: That was an ultimate Agile move. That was a pretty Agile project. They needed boats, they built them but they didn't leave St Louis carrying boats. MICHAEL: Right and they also didn't have a family of six that they needed to support and all this kind of stuff, right? CHARLES: Uhm-mm. MICHAEL: There was a question you asked a long time ago, not to steal the emceeing for you -- CHARLES: I would say, we need to get onto our topic -- MICHAEL: Oh, yeah. Well, maybe this is a good saying, what you're asking is, "How do you get this job?" and I don't think we ever addressed that. CHARLES: Yeah, that's a great question. You said you had to consume a lot of stuff on the internet. MICHAEL: Right. That's definitely how I do the job but I think how I get the job, there's an extended two-part interview with me on my Software Defined Talk Podcast Episode, available at SoftwareDefinedTalk.com, where I talk about my history of becoming an analyst and things like that but the way it happened is I don't have any visible hobbies, as you know Charles except reading the stuff in the Techworld. I would read about what's happening in the Techworld and would blog about it back in 2004, 2005 and I was discovered as it were by the people at RedMonk. I remember for some reason, I wrote some lengthy opinion piece about a release of Lotus Notes. I don't know why but that was a good example. This is back when all of the programming job were going to be off shored and I thought it was imminent that I was going to lose my job. I was looking for a job and I shifted over to being an analyst. That like the way that you get into this kind of business is you establish, there's two ways -- CHARLES: You established expertise, right? MICHAEL: Yeah, which is like always an unhelpful answer because it's sort of like, I was joking about this in another podcast, it's like Seth Godin's advice about doing good marketing, which is the way you do good marketing is you have an excellent product. If you have an excellent product that everyone wants to buy, then your marketing will take care of itself. I think if I'm asking how to market, I'm trying to figure out how to market a bad product. That's really what people want. CHARLES: That's also just not true. That's just like flat ass not true. That's a lie. MICHAEL: I mean, people who want to know how to diet better are not already healthy and dieting successful. You can't start with the base assumption of things are going well. CHARLES: Well, it is true. I like to think that we have an excellent product. We sell an excellent product but the thing is you can just sit on your excellent product all day and you have to tell people about it. If you want them to come sample it and try, maybe eventually buy it like the advice that you just need an excellent product. I'm amazed at anyone who can actually can say that with a straight face. MICHAEL: Well, he only writes like 150-word blogpost. I think his point is that you should aspire to have a unique situation and then marketing is easier. Similar with everyone's favorite example like an Apple or like a Pivotal or a ThoughtWorks. We eat all three of us and yourself as well, once someone gives you the benefit of the doubt of listening, you can explain why what you have is not available anywhere else. CHARLES: What it boils down to is if you want to easily differentiate, allow people to differentiate your products from others, then be different. That's fair. I'll give -- MICHAEL: To summarize it, it begets more of the tactics of how one gets a job like I do. What's the name of the short guy in Game of Thrones? 'Tyrian'? 'Tyran'? 'Tyron'? CHARLES: Tyrion. MICHAEL: At one point, Tyrion is like, "I do two things. I know things and I drink," so that's how you get into this type of business as you establish yourself as an expert and you know things. Now, the third thing which I guess Tyrion was not always required to do is you have to be able to communicate in pretty much all forms. You need to be good at written communication, at verbal communication, at PowerPoint communication, whatever all the mediums are. Just knowing something is not very useful. You also have to tell people these things. CHARLES: I think Tyrion is pretty good at that. MICHAEL: Yeah, that's true but he doesn't ever write anything. There is no Twitter or things like that. CHARLES: I feel like [inaudible] been a pretty big deal in the blogosphere. MICHAEL: Sure, no doubt. The metaphor kind of breaks down because the lattice for the continuing counterarguments do not exist in the Game of Thrones universe but whatever. CHARLES: They've got the ravens. That's like Twitter and it's bird. MICHAEL: That is true. Knowing how to deploy a raven at the right time, with the right message is valuable. CHARLES: We buffer up our ravens so that they fly right at eleven o'clock. MICHAEL: That's true. I could be convinced otherwise. CHARLES: That's why they arrived both at 6PM in the Westeros -- MICHAEL: I guess true to the metaphor of a tweet, most of the communications in Game of Thrones is either, what are they called? Little Birds? That the [inaudible] always has and then the Big Birds. You've got to tweets and the blogs. CHARLES: This is like it's nothing but Twitter. MICHAEL: Exactly. You got to really communicate across mediums. Now that the other thing that's helpful and you don't necessarily have to do this but this is what I think gets you into the larger margin. The more profitable parts of the work that I do is you have to be able to consult with people and give them advice and consulting is largely about, first figuring out the right opportunity to tell them how they can improve, which usually is it's good if they ask you first. I don't know about you but I've found that if you just pro-offer advice, especially with your spouse, you're basically told that you're a jerk. CHARLES: Well, it'd be like a personal trainer and walking around me like, "Hey man. Your muscle tone is kind of flabby. You got to really work on that." MICHAEL: The line between a good consultant and being overly-explain-y is difficult to discern but it's something that you have to master. Now, the other way you consult with people is you study them and understand what their problems are and you're sympathetic to them and I guess you can be like a British nanny and just scold them. That's a certain subset of consulting. CHARLES: Don Rickles of consulting? MICHAEL: That's right. You just help them understand how all of this knowledge that you have applies to them and hope solve their problems like the FordPass thing. When I went from being a developer to an analyst, it was a big risk to take on. I think I probably took like a $30,000 pay cut and I went from a big company health insurance to being on a $10.99 and buying your own health insurance which a whole other conversation. We talked about that every now and then but like it's a risky affair. It's not a promotion or even a lateral move. It's just an entirely different career that you go into. Then you talk with people a lot. As an analyst, you're constantly having to sort out the biases that you have with vendors who want to pay you to save things versus end-users who want to hear the truth. You can't really see a lot of Gartner and Forrester work but the work that you can see publicly from people like RedMonk, it's pretty straightforward. CHARLES: Yeah it is and whatever they did, a piece that was for one of their clients, there was always a big fat disclaimer. MICHAEL: Now, the other thing I would say is what I've noticed -- not to be all navel-gazing -- about myself and other people who are successful at whatever it is I do is there's two things. One, they constantly are putting themselves out there. I remember and this is probably still the case. This is probably all in Medium. There's probably a Medium post every quarter that's like, "If you're a developer, how do you give more talks. What your first conference talk?" Basically, the chief advice in there, other than bring business cards and rehearse is essentially like you just got to get over that idea of self-promotion. You basically have to self-promote yourself incessantly and do all those things that you find nauseous and be like, "Me, me, me," which is true. You've got to get over that thing. If you're like me and you're an introvert who actually doesn't really like that many people, except a handful of people like yourself that I'm friends or family with, you have to put on the mask of an extrovert and go out there and do all this extrovert stuff or you'll fail. I shouldn't say you'll fail, you won't increase your overall comp and margin and everything. You'll basically bottom out at about $120,000 a year or so because that's about as much as anyone will pay for someone who just write stuff but doesn't actually engage in the world and consult. You've got to do that. Then the other consequence of that is you always have to be trying out new types of content and mediums like here we are in a podcast. Long ago, you and I, in 2005 or 2004 -- CHARLES: You got me to sign up for Twitter. MICHAEL: Yeah, like we started off a podcast because I remember hearing the IT conversation stuff and John [inaudible], who is a big inspiration for me, a role model, I remember he was just trying out podcast and I was like, "All right. I'll try that out. That looks like fun," and then here we are. CHARLES: I remember you tried out the podcast and you're like, "Let's go into your backyard or my backyard. Let's talk about software for 15 minutes." I remember that very clearly and that was 12 years ago. Then I remember also like with Twitter, you're like, "Now, you should sign up for this Twitter thing," and I remember I did and that's when it was still coming through SMS on your phone and like "I'm walking around Teatown Lake. I'm going to get tea." And I was like, "Oh, my God. This is so fucking stupid." But little did I know, you were actually signed me up to a service that changed my life. MICHAEL: Yeah, it was the stage direction era of Web 2.0 where you're just supposed to give people your status updates, instead of your searing insights. But yeah, you've tried it all these different mediums because again it goes back to your job is to communicate. You need to tell people things that you know. CHARLES: Coté, what is your strategy on virtual reality? MICHAEL: My strategy in virtual reality. Well, you've caught me, Charles because I'm not into that. You remember when Time Magazine had that Chinese lady who was like a... Not Frontside. What was the name of the big virtual reality thing that was big...? CHARLES: Second Life. MICHAEL: Second Life, who is a Second Life millionaire. CHARLES: Yeah, she had armies of people. She was mining some resource in Second Life and then reselling it and she made a lot of money. MICHAEL: I don't really like visual mediums so as Marshall McLuhan would say 'hot mediums'. I guess I like the cool mediums. That's not my thing. That's where my principle fails. Maybe I'll do that one day. CHARLES: This is pretty hot. This medium is pretty like -- MICHAEL: I think maybe audio broadcast is hot. I'm just pretending like I know. This is another trick that you can deploy that my wife has picked on is most of the time, 78% of the time, I actually have no idea what I'm talking about. I just know words. I don't actually know Marshall McLuhan theory. I read that one book a long time ago and I remember that scene in Annie Hall where he gives a little diatribe to whatever the Woody Allen character is. That's the extent of my Marshall McLuhan knowledge. CHARLES: Was Marshall McLuhan actually in Annie Hall? MICHAEL: He was. CHARLES: Don't sell yourself short, Coté. MICHAEL: Sure. CHARLES: You know things and you drink so let's talk about that second aspect because I know that you like me whole tearing up as a role model. MICHAEL: I should say since we're both happily married, except for the third thing that he does which he -- CHARLES: Oh, right. MICHAEL: Another unmentionable word. He too freely hangs out with the ladies. CHARLES: Right, anyway aside from that, throughout doing all this stuff, you keep a very, very chill perspective on things. I feel like the tech world gets so wound up around itself and it gets so tight and so stressed about its own problems. There's constantly wars in JavaScript and before we were in the JavaScript world, we were warring in Ruby. I remember when Twitter went over to using Scala instead of Ruby. Oh, my goodness, it was terrible times. I feel like there's a lot of stress and yes, you want to take it seriously but I feel like you've always been able to maintain an even-keeled perspective about technology which actually allows you to commentate on it effectively and intelligently because you're able to unwind yourself from the squabbles of the day and see maybe a bigger picture or something like that. MICHAEL: That's nice of you to characterize me to use a -- is that a hanging, dangling participle there, when you're in [inaudible]? CHARLES: Yeah, I don't know. MICHAEL: I think that's also just a function of being old. CHARLES: So are you actually not stressed or is it just part of your persona of being an extrovert or something like that? MICHAEL: About the tech world? No, I'm not stressed about that. As you kind of outlined, especially I was not sent the demographics for the show, which is fine. I'll overlook that but I'm guessing that that was a joke. CHARLES: Who got some designers, developers -- MICHAEL: I'm guessing there's a lot of people who actually are on the frontlines of working on software. I think this happens also in the white collar set. But essentially, it's really easy to slip into over allegiance to something and I don't know what rhetorical fallacy this is but it's the bias of over allegiance to something, you get all wrapped up in defending a tool over something and the virtue of it, whether it's Emacs and vi. I'm sure reactive people, whatever that is, have all sorts of debates. The thing is when you're heads down on this stuff, you don't realize how petty all those discussions are. It's not so much that it's a waste of your time but it's just one battle in an overall war that you have. It's good to have opinions and figure things out but you should just relax about it because the more angry and emotional you get, you're going to make a lot of mistakes and decision and problems. I wish I had an example of this but this is one of those things that intuitively as you ages as developer, it's not like your literal age. It's just the amount of time you've been developing software. You could be a 25-year old who's been developing software for 10 years and you would probably get this notion but you just realize that stuff changes and you just learn the new things. It's kind of not a big deal like one day, you're going on and on about how vi is great and the next day you're using that Atom editor and then whatever and you just use the tool that's appropriate and it's annoying when you're younger and people are applying Hacker News with like, "You should use the tool that is appropriate," which is a stupid reply. That's just kind of how it is. Also the other thing, in the more white collar world, as an analyst, especially doing strategy for a company, you can't be biased by things because then you'll make poor decisions as an analyst. Also when you're doing strategy in M&A that result in bad business outcomes so you actually be very unbiased about things. CHARLES: I think it applies in everything. If you get too emotionally invested in one particular approach in software, literally in anything you do, it does result in bad outcomes. The problem is you may not actually realize the consequences of those bad outcomes far down the road from the poor decision that you made that caused you that outcome so you might not necessarily connect it back. MICHAEL: Yeah, and I keep bringing this up but I think another effect of being calmer in your nerd life is having something that you do outside of your programming life, which is either having a family or having hobbies or something like that but you know -- CHARLES: Or having a wild turkey. MICHAEL: Yeah but you've got to have something, a reason to stop thinking about your tech stuff or it'll consume you. I suspect when you see the older graybeards who go on and on about open source and they're very like... I don't know. What's the word? They're very over the top and fervent about tech stuff. It's probably because like me, that's their only hobby and they haven't figured out how to how to control it. It becomes part of their identity and it defines them and then they're down this twisty, turny path of annoyance to the rest of us. CHARLES: Again, don't sell yourself short, Coté. You've got plenty: you love the cooking and eating and the drinking so close this. Do you have a favorite drink that you've been mixing lately? MICHAEL: No. CHARLES: Or any kind of favorite food because every time I go over to your house, even if we're having pizza, there's always a nice hors d'oeuvre or something to drink, something to tweak that appetite for something special. I kind of wondering if there's anything that you're into. MICHAEL: I have some very basics. One, I don't know if I drink a lot or drink a little. I think the science on this is very confusing, kind of like drinking coffee. I try to drink less. I basically go back to the basics of I want cheap wine that's not terrible. That's what I'm always trying to discover. I think I've also started to rediscover just straight vodka. That's pretty good. I think that fits into the grand scheme. CHARLES: I just can't do it. I can't follow you there. I need some, what do they call them? Gin florals? I can drink gin -- MICHAEL: Oh yeah, that's good too. CHARLES: That's about as close as I can get to straight vodka. MICHAEL: And then food-wise, I just wrapped up finally figuring out how to cook fish and chicken without it tasting terrible. CHARLES: Oh! What's the secret? MICHAEL: No, I want to put a disclaimer out. There's a EULA on this. I'm not responsible for anything bad that happens but what you want to do is cook at about 10 degrees less than you're supposed to. A chicken is supposed to be 165 degrees but you take it out of the pot when it's like 150 or 155 on another part of the pan. Fish is supposed to be 145 degrees but you take it off when it's about 130 or 135. It cooks a little bit more but these guidelines to cook your meat to that thing, it ruins it. Also you can brine a chicken and things like that. Also, what you want to get is an instant meat thermometer. One of those that you can just poke in your meat so you're always checking the temperature. That's what I've been working on. CHARLES: I have a theory about that. I will laid out really quickly, maybe it's just because the juices. It's the juice that so yummy there so you want those to be locked in and boiling but not boiled away. I'm going to give that a try on my -- MICHAEL: And fish is particularly tricky. CHARLES: Because all it takes is five minutes. Sometimes, it's two minutes and 30 seconds too long and you ruin the fish. MICHAEL: Then the next theory I want to try out is that you can actually fry fish in pure butter but you've got to paper towel it off afterwards because too much butter ruins it. But I think if your paper tower it off like you do grease off of bacon, then I think that's how you achieve -- not as good as a restaurant because in a restaurant, they have those butane torches and the crisp it up on the outside or reverse sear or whatever -- CHARLES: Is that what they do? Do they just run their torch right over the fish? MICHAEL: That's all I can figure. They might also be professional cooks who know how to cook things. CHARLES: They might have done it a lot of times. They might have had someone like Gordon Ramsay yelling at them constantly. "I can't believe this fish is so terrible. Waah!" All right. I'm going to give the fish a try. I'm going to give the chicken a try and I'm going to give everything that you just spent the last hour talking about, also a try. MICHAEL: Well, thanks for having me on. It's always fun to have a show with you. I just posted yesterday our second revival of the Drunken Retired Podcast, which is over at Cote.show. It's just '.show'. URLs are crazy nowadays. I guess the only self-promotional thing I have is I'm over in Twitter @Cote. It'd be nice if everyone should just go follow me there because I'm always very sad that I don't have enough followers and they'll never verify me. I don't understand what the problem is. I'm clearly me. Then I mentioned earlier, the main podcast that I do is Software Defined Talk, which is at SoftwareDefinedTalk.com and you should come spend a lot of money on Pivotal stuff. I'm happy to tell you all about that. Just go check out Pivotal at Pivotal.io CHARLES: I guess that is about it. We will talk to everybody later. Thank you for staying tuned and listening to this supersized episode. Come check us out sometime!

Balint Erdi: @baaz | balinterdi.com | Rock and Roll with Ember.js Show Notes: 01:58 - What is JSON API? Advantages 03:22 - Tooling and Libraries 05:49 - Relationship Loading 07:51 - Designing a Data Loading Strategy 11:23 - Pitfalls of Not Designing a Data Loading Strategy 13:53 - Ember Data 16:37 - Pagination & Sorting 23:06 - Writing a Book 25:48 - Implementing Searches with Filters 31:08 - What's next for Balint? Resources: Balint Erdi: Data Loading Patterns with JSON API @ EmberConf 2017 (Talk) Balint Erdi: Data Loading Patterns @ EmberConf 2017 (Slides) jsonapi-resources GraphQL JSON API By Example by Adolfo Builes ember-cli 101 by Adolfo Builes 33 Page Minibook + Coupon Code! Transcript: CHARLES: Hello, everybody and welcome to The Frontside Podcast, Episode 65. My name is Charles Lowell. I'm a developer here at The Frontside. With me, also from The Frontside is Elrick Ryan. Thank you for being with us, Elrick. I know this is your first podcast. ELRICK: This is my first podcast. It's great to be here. CHARLES: All right. Fantastic. Yes, we hired Elrick a little bit ago and it's been fantastic. I'm glad to get you on. With us today is a really awesome guest. His name is Balint Erdi. I actually like to tell a little bit of a story when I have an anecdote and I do have one about you that I think you might like, although you might not even remember it. But it was shortly after EmberConf. Last year you and I got on a pairing session remotely and I don't even remember what we were working on but I was struggling with this way to decorate objects without changing them, without touching them or mutating them in any way and you showed me this technique of actually decorating it by creating a new object with the old object as the prototype. Do you remember that? BALINT: Yes. I totally knew. How could I forget? CHARLES: Yeah. That one hot tip changed my life. It is one of the best techniques that I have discovered in the last five years of working with JavaScript. It really was great and I use it all the time. BALINT: Wow. Amazing. CHARLES: Thank you. I don't know if I ever said, "Thank you," but thank you, thank you, thank you. BALINT: Yeah, no problem. I also learned a lot from this pairing session actually. I didn't know that my small contribution made such an impact. I'm glad to hear that. CHARLES: Yeah, that was fantastic. We need to actually make that happen again. I don't know why we only did that once. BALINT: Yeah, we should. CHARLES: Anyway, we're here to talk about data loading, it's something that is absolutely critical to building good frontend and building UI and yet, it's something that the users never really see. Sometimes, it feels like it's 90% of the problem. BALINT: Exactly, yeah. ELRICK: Yeah, that's so true. CHARLES: We're going to talk about techniques that we use and you use and, in particular JSON API, what it is and what's so great about it. So, what is JSON API for folks who've never heard of it? BALINT: JSON API is a standard way to build APIs. I think the specification has reached 1.0, I would say two years ago or three years ago. I remember it was in June, I'm not sure which year. It basically lays down everything that's usually consider when you build an API: how do fetch relationships, how to paginate data, how to sort, all of these things that I think developers tend to invent again and again. I think probably the biggest advantage of JSON API is that it just declare a standard way to do that. It basically reduces the byte sharing going on at the start of the project. Well, not just at the start, later on too. In my talk at EmberConf, I coined a JSON API the conventional over configuration for APIs. CHARLES: I see. Pagination is something that everybody does. Why need to byte share over the syntax, like the actual data from it? BALINT: All of these things are things that everybody does. It's just that everybody does it differently. There's a lot of discussion going on which the best ways, for example when there's a team, at least every team that I was involved with had several discussions going on about what data formats to send data in and how to paginate and all these, I think details where it's more important to get to an agreement just to agree on something and move on than to get it perfectly, if at all there is a perfect way to do it. CHARLES: I guess my question is if you have the standard way of doing of everything, what kind of tooling can you build, that can you kind of inherit for free? At what level, both from the low level and then up to the top level? When I say top level, I mean what the user is seeing. BALINT: By low level, I guess you mean the actual libraries that implement JSON API in different frameworks, right? CHARLES: Exactly. Are there now a lot of libraries out there so whatever I'm using, if I'm using JSON API, is it available in a lot of different ecosystems now? BALINT: Yes. It definitely is. There is a full page on the JSONAPI.org, on the official JSON API page that just list all of the different libraries that are now implementing all these languages. I have experience with Rails, probably at Ruby too and there are three libraries and I think all of them are pretty good just for implementing JSON API. The one I use is called JSON API resources and it's very telling. Well, it's a rather simple application but I basically didn't have to write a single line of code. I've only had to write very little code in the server to implement a JSON API specific feature. Most of the relationships could be implemented with just declaring JSON API resources and then the name of the resource in the Rails application. For all the other things, I really didn't have to do that much so in every time, it was just adding one or two lines or changing a configuration value, then it was just there. CHARLES: Now, how do you choose then what relationship you want to load and which order? Is that controlled by JSON API? BALINT: It's controlled by the frontend. It is a frontend application that's going to send these requests to the backend so that's where you should consciously think about what relationships you are fetching and how. CHARLES: Right so part of the specification is a way of specifying which relationships you want to load. In my understanding, part of JSON API is an interface along with say, the user, "I want to load all of the posts that this user has made." BALINT: Yes. JSON API indeed has a keyword called 'included', which you can implement on your backend which does this. If you specify 'include' and then the name of the relationship or relationships for several ones, the backend must comply with that request and also send back those related resources. That is called compound document in JSON API parlance. ELRICK: Is that the reverse of what they doing in GraphQL because at GraphQL, I think you have to request the relationships you want on the frontend and then it kicks it off to the backend and it gives you the information back. Is it like JSON API that includes that same thing but in reverse? BALINT: I'm not sure because I'm very familiar with GraphQL but all the things it does is that you are normally fetching a resource and then if you specify 'include' then you are telling the backend, "Please also include these related resources with a primary source." CHARLES: I think it occupies a very similar concept that you want to have control on the frontend about which resources or what data gets fetched in addition. ELRICK: Yeah, it sounds very similar. CHARLES: Well, I'd love actually to do a comparison because I know there's a lot of overlap between GraphQL and this. Maybe we can get into that a little bit later. One of the things that you said back there is having this, gives you kind of fine-grain control over when and how you load your data because that always seems a pretty difficult problem to attack on your frontend because as you're rendering your application, you have to incrementally fetch little pieces of data here and there and make sure that it's already, at the time that you actually need to render something like a component and then it's got the right data at the right time. It seems like it's this constant dance of whack-a-mole and like, "I'm loading too much here. It's taking too long," versus, "I've got too many loads happening. I've got 20 requests to run the single page." How can I back those up into a single thing? How do you go about thinking and designing that data loading strategy, as you're ready to render pages? BALINT: That's a really good question. I think the short answer is how you load data has to be part of the design process. You really have to spend time thinking about how you're doing that based on the needs of the UI. I think the way you need to render the UI will suggest the way to do data loading. Especially in Ember, I think do I really need ways to, for example block the page from rendering too early so anything that you want to render first, you can fetch in this non-blocking way into model hook of Ember. Then anything else, if you're okay with rendering later, you can fetch in other ways like to set up controller hook or from the templates or from controllers or whatever way but not in the model hook. CHARLES: But if you are doing something outside of the model hook -- because this is something I feel like a pattern that comes up a lot -- and regardless of where you're operating, if you're using Ember, if you're using another framework, you have kind of this top level data loading. But then you have your nested components might need more data, how do you go about loading that data. I guess you have to think about that also. Upfront it's like, "I've got this component that might want to request more data," how do I actually design that and think about that of I've got this data that's going to come, who knows, maybe minutes after the initial route is rendered? BALINT: That would be different but I think that you can apply the same principle. You can fetch some data in the model hook in this blocking way and pass it all it down to your component if you don't want to render the component before the page renderers or you can just even fetch it from the component while the component is rendered. CHARLES: You're thinking maybe, in terms of streamlining, the rendering process so that you can begin rendering while your data is loading. Is that the use case? BALINT: Yes. If you, for example fetch the data from your component, then it's just going to fetch the data as needed. You have the whole page load as render and then when the component is render and it fetches the data, then you're going to see other requests go out to the backend. The data comes back and then the component is going to be rerendered with the new data. I think in most cases, that's totally fine but you might have a use case when you don't want to render the page before the component is all over the data that it needs. In that case, what you can do is once again, if the fetched data is needed in the model hook, it will just pass it all down to the component. ELRICK: What are some of the pitfalls that you would run into by not thinking about your data loading strategy beforehand that you can pull out and explain? BALINT: I think the classic one is what was known name in Rails and other framework is 'N + 1 problem' and it's when you fetch many related resources, then you might end up with doing end requests for a number of resources. In the case of Ember, you have for example a blog post as your model and then in your templates you write model.comments, then what's going to happen -- depending on actual library that you use on the backend but I ran into this myself -- is by default, if you are going to make those end requests. Ember solution is really, they just works. I mean, the default solution just works and you might not even run into this because you might not have the scenario. You might just have a few records. But if you have a great number of them, then it's going to be a bad experience. ELRICK: So someone that just dealing with Ember and then they go and make a request and then see all these requests come back, that would be something that they would then have to turn around and asked or fix within the backend to say like, "Only give me a certain set of this." BALINT: Yes, exactly. ELRICK: Or is there something on the Ember side with Ember data that you can say, "Only fetch X amount." BALINT: Right. I think both. I can speak about using Ember data and JSON API resources so what you can do to mitigate this case is to use links or cause relationship links instead of fetching the comments one by one, the backend can actually drive Ember data to fetch all the comments in one request from the link that it sends the frontend. You first ask for the blog post itself and then a JSON API compliant backend will send back the blog post resource but it's going to have a relationship link inside. An Ember data automatically records this so when it needs the comments for the blog post then it's going to fetch down from that provided URL. Actually, I haven't really talk about it in a lot of detail at EmberConf. CHARLES: Yeah, I see. I'm going to go off on a little bit of a tangent because I feel like this is a pattern that's coming up more and more. To give a little bit of context, I feel like the way that our data loading strategies have evolved is we're used to the page loads, then we kind of analyze the URL, we decide what data needs to be loaded to render our components and then we pull that data from the server based on a decision and then we do our render. But it seems increasingly more prevalent than we are having a combination of both pulling the data from the server and having the server push data onto us. One is what are the strategies for dealing with that? Given kind of certainly, at least in the Ember world, routes aren't reactive. Then does JSON API actually help with that at all? BALINT: Yeah, a good question. I don't think that JSON API specifies how [inaudible]. You are thinking about something like web sockets like pushing data from the server sockets. I don't think JSON API covers this summary. CHARLES: Yeah. It definitely seems like beyond the scope but I'm wondering if there are any thoughts about general strategies, about how to handle this model state that sitting at the top of your render tree. In Ember for example, in your route and how do you handle the fact that the route requests data and how do you handle data coming in after the initial render? BALINT: If you use Ember data, then you can push data coming in from a web socket, for example to the store. You probably do some massaging on the data that comes in and then you push it to the store and then depending on how you fetch the data, you might have a live collection. For example, if you do a store, find all notifications and then notification is coming in that is going to get displayed right away on your page because then your template is bound to a live collection. But not all of things in Ember data are live collections. CHARLES: Okay, it's mostly library dependent but if you're using Ember data, then you just push those directly into the store, those live collections. It kind of like a real time query. BALINT: Yeah, exactly. But what I'm saying is that not all Ember data query that you do are live collections. For example, relationships are probably not, depending on what method you use to fetch that data. You might have to do some additional footwork. CHARLES: Okay. Now, getting back into the area in which JSON API really does shine at things that can really shave off a lot of time and consequently money from your work, let's talk about those a little bit. For example, you mentioned pagination. How is JSON API going to help me if I want to have paginated data? What are the scenarios on the client where I would need paginated data? Then maybe we can walk back from here's this user interaction that I need paginated data for and how is JSON API going to help me with that? BALINT: Sure. I guess the typical scenario on the frontend is there is a long list of items and you don't want to overwhelm the user by showing them what it wants. You need to just show them page by page so JSON API recommends to use so it's agnostic about the exact paging strategy that you use. You can use the classic page-based approach or cursor-based one or start of pagination technique. It doesn't really force you to choose the way you want to do that. It also mentions that, I think the way it frames it, you might want to use the page for your parameter. I think that the libraries, at least at JSON API resources for sure, you use that page parameter to send back paginated data. You have a page and a square brackets number and then page size. The request variables are page number and page size. Then the server knows that I just have to send back the second page if the page size is 25 and they just set that [inaudible]. CHARLES: Then if you're using a library on the server side, then you don't have to do any extra work. BALINT: Yeah, exactly, depending on the library I guess. JSON API resources made this one really simple. CHARLES: I see. Then in terms of library support on the client, I assume that there are libraries, like Ember data that automatically will support this so when you're creating these live queries, you can include information about the page. BALINT: Yes. I'm very in to Ember so I'm not sure about the frameworks but I suppose that there are some ways that make this very easy for the developer in many of these frameworks like Angular and React. CHARLES: Right. Something that has just bit me on the butt so many times is when I have paginated sorted data. Imagine you've got some infinite scrolling table or not infinite scrolling but you've got some a bunch of table rows that are maybe 300 of them or 3000 of them so you don't want to load them all at the same time. But at the same time, you've got complex sorting that's happening on each column. You might have seven layers of sort. You want to sort by name, followed by ID, followed by date. One of the biggest problems I've encountered is trying to reconcile and sort on the client versus trying to sort on the server. Are there any facilities to help you deal with that? BALINT: Yes. I think the approach I usually take is if you do it on the server, then do it on the server. Do not mix the two things. In this case for example, if you are sorting and then you change just sort field, I would just send a request to the server to have the items returned according to the new sort criteria. I think that's the simplest approach because as you said, I've probably experienced things can get really messy, if you want to do that on the client. CHARLES: Then you've got the third page but when you do the sort, the contents of the third page could be anything. BALINT: Yeah, that's the other thing. I think if you change the sorting criteria, you'd probably want to go to the first page. I haven't thought through all of the scenarios but I think it's really rare that you want to stay on the fourth page while you change the sorting criteria to be created at descending. You probably want to see the first item the way it was created. CHARLES: Someone should seriously write a book about sorting and pagination and loading these data sets because seriously, I feel there's this tribal knowledge of things that people have learned from screwing it up. There's not a written down way of this is how you build the data loading layer for an infinite dataset so that you can sort, you can paginate and here are the problems that you'll encounter. BALINT: There's actually a book written by Adolfo Builes. CHARLES: He wrote a book on Ember CLI, too, right? BALINT: Exactly, Ember CLI 101. He's the same guy and he wrote JSON API By Example. I have it on my mental to-do-list to buy that book. I'm not sure if it covers these exact scenarios but he must cover several in that book. CHARLES: Well, I'll be sure to reach out from because certainly there are a couple of scenarios that have bit me too. The other one is where you have some collection that's paginated and sorted, then someone adds on the client side a thing to that collection. Say, you want to create a new row in that table, well then what do you do with that new row? chances are it's not going to be anywhere on the screen because who knows what the sort order is and the terms of the total sort, which is only the server knows and who knows what page it's going to be on? You get all these problems that compound and it would be great to have one place where people could reference them or have a little cookbook. BALINT: Sure, absolutely. I think in that scenario, the simplest thing, which I think probably works best like 99% of the cases is again, just to reset the sorting and the pagination. If I create a new record, I really want to see that new record. CHARLES: Yeah, maybe you put the new record up in a box up, at the top in a special new record place. BALINT: Sure you can go fancy and do that. That would be a good solution too. But probably you can just reset the page number and show the first page with the new record. CHARLES: Ah, yeah. I see. BALINT: It's tricky. It's going to get more complicated than this. CHARLES: You might have a problem where you don't want to lose that context to the records that you were looking at. BALINT: That's right. Somebody should write the book in this. I know somebody who wrote a book [inaudible]. CHARLES: It could be you. You haven't written a book in over a year, right? [Laughter] BALINT: It's been two years already. CHARLES: It's been two years. BALINT: Exactly. CHARLES: I'm never going to write a book again. I don't know. Do you think you might? BALINT: I think I might. I actually have this urge. CHARLES: If I recall correctly, you're one of the few people I've talked to who was like, "Yeah, you know, writing a book, it wasn't that bad. It was kind of an amazing experience." And literally, everyone else I talked to have been like, "Urgh!" ELRICK: That's really interesting too because his book keeps up with the releases of Ember so that makes it even harder. That's surprising to hear that like, "Oh, it wasn't that bad." BALINT: Exactly. Well, I kind of put off writing the second book for a while because if I just wrote a book, then I could just be done with it, I would be happy to start writing the second book. But if I have to maintain it for years, I don't have to do that but it's just so much extra work that I'd have to take this into consideration. CHARLES: Yeah. Essentially, what you've done is you've rewritten the book. In terms of absolute content, given how much everything has changed, you probably flipped over the content of the book in the same way that people flip over the atoms in their body. I think there was something like you don't have a single atom in your body three years from now that you have today. It takes about three years and you have all the matters completely and totally exchanged. It's kind of like that. BALINT: Yes, it's kind of like that. Well, I think maybe half of the book is still relatively as it was when I released it because since Ember 2 came out, Ember didn't change that much so in the 1.X series it did change a lot and I think my book originally came out when Ember was 1.10, I would say. There's a lot less work that require now to maintain that it was back then. But it had changed a lot for sure. ELRICK: Yeah, I think I bought the book on its first release. It was 1.10. I guess you automatically get assigned to the GitHub repo so you just see a constant barrage of updates, updates, updates and I'm like, "Wow, Balint is really killing it in updating this book." BALINT: Yeah. CHARLES: It is a good book and everybody should go buy it. The other thing that I want to cover to, as long as we're talking about scenarios that come up again and again, we talked about pagination, we talked about sorting, what about things like search? Is there a uniform mechanism to help you out there? BALINT: You can implement searches by using filters. It's a JSON API concept of using filters. You can pass a parameter called filter to your query and then a square bracket. You have the name of the field that you want to search and then just pass the value, the search term basically and then backend should return the items that matched according to some criteria. That's the simple case. CHARLES: It's a simple case but clearly, it's up to the backend to implement that API. BALINT: Yes. CHARLES: So I'm wondering what libraries are available if I'm doing something in Elixir or I'm doing something in Sinatra or I'm doing something using Express, how seamless is it because I feel like a lot of times you can run into problems where these leaky abstractions about the fact that one thing is a Mongo backend, then one is based on Postgres. Maybe that's a better example than a different server technology but more of sticking with a single server technology -- let's use Ruby -- but one is I'm using a Postgres backend and when I'm using, say MongoDB or some other key value store. In your experience, if you've seen this, how much does the backing store leak into the frontend? Is JSON API a good protection and the ecosystem around it from those leaks? BALINT: Yeah, that's a good question. I was about the say that the backend needs to be the abstraction that's use you from having to know what kind of persistence layer you use. The frontend shouldn't care about -- or any client of that backend -- whether you use MongoDB or Postgres. That's a responsibility of the API. You can still send, in this case for example, a filter query. However, the backend translates this to database queries. That's his job. I think the answer to your question is that JSON API does protect you from having to know the intricate details of the database. CHARLES: You might have some work to do but it's possible. BALINT: Sure. You might have a lot of work to do on the backend but it's possible. But it's not just JSON API that protects you. Any kind of API should protect you from this kind of knowledge. CHARLES: Right, unless you're using GraphQL. BALINT: Could be. I don't know exactly how GraphQL works. CHARLES: Yeah. I think there would be actually nice to read something from somebody who's got a lot of good experience with all of these, like different technologies to make a comparison. I feel kind of in the dark. Unfortunately, the problem with any technology, I feel like most of the comparisons out there, if you're going to compare, most people have a huge implicit bias for one tool and a little bit of experience with the other, maybe dangerous of like, "I don't definitely want to render opinions on my GraphQL and certainly not versus the API that I'm used to because I feel like I can't make a good comparison." BALINT: Yeah, absolutely. That's a good one. CHARLES: But it is something that I'm so intrigued because I feel like there's a lot of overlap there but who knows. BALINT: Yeah. ELRICK: They need that to do API like how they had to do MVC, they need to do API comparison. CHARLES: Yeah. One thing we could do, we could implement JSON API over GraphQL and kind of just move your backend on to the frontend. BALINT: I think you could but you just said that with GraphQL, you have to know the client on the frontend, like feels the database. CHARLES: Right. You could technically have a JSON API on top of your GraphQL backend because I think the thing that kind of freaks me out -- this is a crazy idea that no one should ever do it but I hope someone does -- about GraphQL is being as old as I am, I saw so many projects ruined by the visual basic kind of mantra of just like, "Just query your database right inside your components," and that was literally the rope that hung ten thousand projects and just made people despise visual basic development because there was no shield and then literally every button was coupled to your database. But you could theoretically have the best of both worlds where you're sitting on an abstraction that's also on your client but just moving your query language from your server over to your client or something like that. BALINT: Yes something like that could work, in theory. CHARLES: In theory. Like I said, no one should ever do that unless they really want to. BALINT: Yes. CHARLES: Fantastic. The other thing I want to ask you is kind of what do you have cookin'. You got your book, you wrote but you keep updated, you recently have been evangelizing data loading patterns most recently at Ember Conf. What's next? What's now? Or you're just kind of taking a break? BALINT: I already have published book a mini-book about these data loading scenarios. Actually, I just cover the things I told them out at EmberConf then add some more but I might make this into a full-fledged book, providing I don't have to update it. [Laughter] CHARLES: I'm going to write this book -- BALINT: But just once. CHARLES: -- But just once. BALINT: Yeah, I still have to find a way of doing that. CHARLES: All right. We will look for all of those things. One thing that just occurred to me is just how much of actually building UI and building frontend really is about thinking about the structure and flow of how you load your data and how much the user doesn't see that but how important that is to provide a good experience to that user. That's one of the things that sometimes we, as UI engineers don't like to think about but I think it is absolutely true and crucial and foundational. Thank you so much, Balint for coming by and talking with us about these important topics. We will see everybody next week with that. I will bid everybody to do. Good bye, Elrick. Good bye, Balint. BALINT: Yeah, thank you very much for having me. Goodbye.

Ginger Whalen: @gingerwhalen Show Notes: 01:28 - Everyone Does Sales 02:11 - Sales is an Exchange: Understanding and Solving Others' Problems 05:58 - “Personas”; Empathy vs Sympathy 11:47 - Empathy Requires Introspection 15:12 - Persona Example 20:33 - Making Incremental Improvements 23:15 - Challenges in the UI Business Space 29:25 - TL;DR on the Business Development Process Resources: Thinking, Fast and Slow by Daniel Kahneman Transcript: CHARLES: Hello everybody and welcome to The Frontside Podcast, Episode 64. My name is Charles Lowell. I'm a developer here at The Frontside. I'm here with Jeffrey Cherewaty. Hello, Jeffery. JEFFREY: Hey, there. CHARLES: With us today is someone also at The Frontside but not one of our developers. Her name is Ginger Whalen. Just to give you a little bit of backstory on her and why I'm so excited to have her on the podcast is we, about the middle of last year, began a search to bring someone on to help manage and grow and just have their eye on our business pipeline because that's something that's really, really critical, it turns out, to a software agency is making sure that we have clients. We put together a pretty extensive search plan and then we executed it and it was, I think about six months but in the end, we ended up hiring her. The reason we did was because she had a very unique take on what we would typically deemed the sales process so we learned a lot about it and I wanted to have her on the show so that we could just kind of share with our, I would say, mostly skewed on the technical side audience about what a healthy sales process might look like. Welcome, Ginger. Thank you for coming. GINGER: Thank you. My pleasure. It's so funny when people talk about sales and they say to me, "Oh, you're a sales person," I think that's so funny because I've never viewed people to be in sales and people not to be in sales. To me, everybody is in sales because all sales is to me is somebody just exploring somebody's problems with them and then when the time's right, you educate them and then you help them see the value of acting on some solution. Who doesn't do that? We all do that. CHARLES: We all do that. We do it in our day-to-day when we're proposing and discussing and talking about technical solutions, the same principle may be applied in a different level. GINGER: Yes, some people are just on the hook for it, that their professional goals are tied to that end game. CHARLES: Yeah. One of the things that struck me as kind of setting you apart when we were doing the interview process is the way in which, I want to say the focus of building those relationships and kind of the focus on understanding and understanding who exactly potential customers are and trying to categorize them and tailor our message so that we can actually maybe help them. Maybe you could explain a little bit about that. What's the front half of that process looks like? GINGER: That's kind of the bigger subject. I guess sales is the exchange, the educating and helping somebody solve a problem, get what they want to solve that problem. But if you're not speaking in their language and sometimes you hear the term 'knocked around personas' -- the persona you're talking to or the person you're talking to -- if you're not speaking in their language or addressing their pain, talking about their problems, it just so boring to them. It doesn't really seemed like it's going to solve their problem and you just don't hit the target right. You're probably not going make a sale in the end. That's what we talked about when we first start talking with Charles and The Frontside team. We talked about personas, who are you approaching, what are their pains -- their business pains, their pains with their engineering team? What do they come to the table with and what would they like you to help them solve? The subject today, we're going to get into this a little bit later about empathy and then is really what it's all about is that gift of really understanding somebody else's experiences, somebody else's problems, their emotions and how we use that in sales as we're going to talk about a little bit more today. CHARLES: Yeah, I really like that because the drive there is to maybe not always shoot for a sale, to really try and understand like you said, someone's problems and just go out there and talk with interview, listen to a lot of different people and realized you're not going to be a good match for most of those people out there. If that's the case, not trying to view that as a potential opportunity that you want to just kind of force through but rather say, "How can I help this person even when it's not through my services and developing that and having this sales process?" because I think we tend to have a stereotype of what it is as being like I have a goal. When I want to interact with this person, what my agenda is actually to make a sale to them. It's clearly not like that. I think it varies from business to business but that's kind of the stereotype that I had in my head before this process began. JEFFREY: It feels easier as an engineer to be able to say what I'm trying to do is solve your problem versus what I'm trying to do is a self-centered like, "I want to have your business." That's very different approach to the problem. GINGER: That's so funny because I view it as exactly the same as what you guys do. You're solving problems, you know the end game is you're going to write this code to figure this out to do this thing and this thing is your end goal. That's the same thing I'm doing as I'm trying to fit my solution with this person but the means to get there, I just have to let it unfold. Sometimes, it's really hard and sometimes it's really complicated and takes a lot of code and lot of trying and then trying something different, then that broke and then trying something different again. In sales, it's the same thing. Especially at this level, when you're with a consultancy, you have multiple buyers or multiple personas out there. You're selling up into an organization that can get pretty complex and you can have some fits and starts, just like you do when you guys are coding and trying to solve a problem. CHARLES: Now these personas, this was actually something that was really interesting to me, the kind of introduction is this as you understand the problems and pain points, you're trying to collect them into related that this person has these general challenges and problems that need to be solved. How do you go about developing those personas? What is a persona and how do you go about developing them? GINGER: First, you have to know who your decision maker is in an organization. From that, you develop your different personas. Now, some of these personas are the absolute decision maker at the end so the guys can assign. Then some of those personas are influencers throughout an organization. For example, in this field, we talk often to a lead developer. We talk to VP Engineering. We talk to CTOs. We talk to presidents and CEOs of organizations. They probably have different organizational goals, different things they're on the hook for. They have a common vision, absolutely that they're all working toward so maybe a business growth goal that they're working towards. But the VP Engineering might care more about the cohesiveness and the development of his team. He's going to be really looking for an engineering team that fits well with his team. He wants to make sure that their productivity doesn't go down. He wants to make sure that this team merges well with his so he's thinking about lowering his risk through this experience, keeping everything smooth. Where somebody at a different level, a CEO is maybe looking at something a little bit different. Are we going to hit this deadline so we can get this product out, so we can get our first customer, so we can get some revenue in? Maybe he's just looking at bottom line numbers. If they come to the table and they have all these different problems or different pains, you really have to speak to each one of them about their pain and make sure you're addressing that. That takes some empathy. Not sympathy, not saying, "Oh, my gosh. I feel so sorry for you having all these problems." That's sympathy. That's pity. But empathy is not about telling your story, "Oh, yeah. That happened to me." No. It's about understanding their story. It really does take some listening in the beginning to develop your personas and to really get, not coming to the table and saying, "I think this is what his problem should be because he's this guy, this level, in this organization," but really listening to them and letting them tell you what their concerns might be. Don't be afraid to ask for those because that is how you develop those personas. CHARLES: Right. You really just have to walk in a lot of different shoes and then write down those stories. That's what a persona is, it's writing down the narrative based on trying to discover people's actual experiences. GINGER: In the old days, they say personas are about demographics like this guy is about this age, eats this for breakfast and lives here and has this sort of family life. It's a lot more developed than that now and it really again focuses on their pain because you're the consultant in there. That's what they want. They want somebody that's going to be able to educate them on their services or something in the industry, teach them something about their business would be great. But what they're really looking for is someone to help them solve these problems, "Help me solve these problems. Can you do that?" and I would choose that person to work with if they can help me solve my problems. JEFFREY: So much of this process feels like it's a lot of listening. It's a lot of deducing what the problems are based on what you're being told. At what point in the process do we present a solution? GINGER: Yeah, a good question. Just like the timing of a joke. That's really important: the pauses and when to do what in the joke or just the flat. During the process, when you're gathering information, you're also qualifying them. As you're asking these questions and getting them to reveal their problems and pains, you have some questions. You really need to get answer to. This isn't just, "Let's go have coffee together and talk about your pains." It's really you're qualifying them. You have some strict qualifying questions that you know about an ideal client for your company. Some of those qualifiers might be how many employees do they have, what size company are they, what stage of growth are they in? Are they hypergrowth? Are they scaling? Are they a startup? Are they in a certain silo of business? Do you do especially well in education or energy? Then triggers: what's going on with their business? Did they have a lot of recent personnel changes? Is that a trigger that usually helps you segment your ideal client profile? They just have an acquisition. Is that a good trigger or signal that they're qualified prospects for you? While you're asking these questions and again, identifying your personas going through their pains, you're also qualifying them. I'd say when do you go to the part where you're selling -- Charles has been on sales calls with me and he's probably already noticed -- that you're always selling the value of your organization. You're always selling but as far as giving them a solution to their problem, you wait for that. You don't give that away, right away and you can miss your chance or miss the punchline if you give that away too soon. I'd say, once you truly understand their problem and then you get agreement from them that this is a problem, this is really important to solve and there's some urgency tied to that, when you have those things, then you can begin your presentation and show them the solution to that problem. But there's two parts: one is identifying the problem, two really find out that they agree. Play it back to them that this is a problem, how does it affect them and if there's the urgency tied to it. That's what you need to know before you propose a solution, I'd say. JEFFREY: It's interesting that you mentioned how much empathetic sales process also requires introspection and besides understanding the customer's pain points and what problems they need to solve, you also have to recognize what problems we are able to solve and what problems we enjoy solving and that I think is counterintuitive that empathy requires introspection. GINGER: I agree. You have to intellectually and emotionally identify with somebody and to the point where you can really experience what their attitude or their emotion or their feelings is. That's definitely an introspective thing. Yeah, you're right. Not the typical old school salesperson that just blabbering all over the place, being an extrovert telling their story. It really is more of an introspective. I'll even say introvert way of selling and that can be very powerful. CHARLES: As you do that, listening and do that introspection and then develop this understanding of your clientele via these personas, once these personas are developed, how do you integrate them back into the business development process? GINGER: That's a good test to say, "Can I really put myself in this person's shoes? Do I understand them? Do I have a better bond with them now because there's a mutual respect because we understand each other's experiences and thoughts and attitudes, pains and problems about something? Can I put myself in this person's shoes?" CHARLES: You spend time listening and talking and introspecting and developing these personas of your clientele so then how do you actually, with that information in hand, use that to build your business development process? Now that you have this information, how do you actually leverage it on a macroscopic scale? Like I'm kind of out there in the world now, I've got these personas in my tool box, how do I go out there and interact with people based on that? GINGER: I view that as an overlay over a sales process. That's the umbrella over the whole sales process. In a sales process, we do our first calls or connect calls or exploratory calls, again where we're learning about the company and their products, getting to know our personas. Then as you understand that persona, when you get into their goals and their plans and their challenges, when you're in the discovery part of your sales process, that's the part where you can transfer those points of pain and their challenges into your solutions. Again, that's just a part of the conversation where you're speaking about value of the company, you're letting them know you understand their pains. When you get to the presentation, this isn't really explicit. It's not explicit in saying, "I know this is a pain. This is a problem you have," you turn it on its head and you take those challenges that they've told you about and weave those into your solutions. It's kind of where the rubber hits the road there because if your solutions don't solve their problem, it's absolutely where you're going to figure that out when you're presenting or when you get ready for that presentation. Would it help if we talked about, maybe one of those personas and wove that through the sales process? CHARLES: Yeah, I think that's a good idea. GINGER: Okay, a popular one with our type of business -- the VP Engineering. The VP Engineering, when we're talking to him, we're really trying to understand what he values, what's important to him in his work, what problems need to get solved. This person is really caring about quality, the quality of the work and the value that they're getting for their money. We talked about their team developing, learning as they work with us. If that's important, we're probably going to really stress the quality of our service. How do we do that? We're going to talk about the way we code, the way we test, the way build software. We're going to give them supporting evidence and case studies of how this is played out. We're going to talk about the value in this. We're going to talk about, maybe how we've reduce some time, for example [inaudible]. We've done that recently with a client where he's getting better value for his money because we're talking about his [inaudible] in reducing some of the time to build. If we're talking about the development of his team, that something he's really concerned about. Is my team going to continue learning? Maybe we can present a story or a case study about a team that when we arrived, it looked this way and when we left, it looked this way. They had these additional skills... I don't know... Maybe you can jump in here but there's a client we worked in where their skills were totally transformed with us pair programming or working alongside them. Something like that are really important to VP Engineering. CHARLES: Yeah. One of the things that this process has taught me and made me see and try to cast for wherever I can find it is actually trying to really quantify and measure that value. I know that's really hard because what we do is development. That's a really long process. If the product you're developing is successful, that thing that you put in there is worth a lot of money. If, 'thanks,' then it's worth nothing at all and the thing that you did was negative value because it was just sunk cost. Those are two huge extremes and it feels like how do you get a little bit more fine grained in that and say, "Here's this activity that we're proposing and it's going to save this much money." You hear a lot of this people saying that you should try and do this this but how do you actually go about it? One of the things that kind of working in this way has taught me is really to try and wherever possible, it's not always possible but attach monetary value to the tradeoffs that you're proposing and the solutions like saying, "What is the impact of the work that we're going to be proposing?" "We think it will save you $30,000." When someone actually takes a look at that and says, "That is a lot of money." That is, "We think that this could save you 50 developer hours per month." They say, "That is a lot of time," which again equates into a lot of money. GINGER: That would be huge. I'm sure that would make him smile. CHARLES: Right, exactly. You always hear that but it's trying to be more deliberate about it and really you have to think a lot about it to try and get to that level. GINGER: Yeah and you just help me thinking about it all the way to the bottom line like that is terrific because in this situation, a VP Engineering, that person might have to sell it up the organization and maybe he has to sell it to the CEO and maybe the CEO is not going to spend as much time with us, may not even meet us directly. Those bottom line numbers might be really important for this persona to internally sell to the person that's actually going to give the nod and about their concern about we're going to get it right the first time, how do we help them get comfortable with that? That's one of those risk issues that a VP Engineering might have. Knowing these thing about your persona, knowing the pain that your persona has, all these things can be woven into your presentation. This is where, again the empathy comes in because you are speaking to their pains, their problems. This is an emotional sale. I don't care how technical something is, it is always in the end is going to be an emotional decision. It really is. Who's that author we're talking about that? Daniel Kahneman? His book out there: Thinking, Fast and Slow. He did this study where he analyzed super, super wicked, smart people and then the rest of us, smart people but normal people. He gave them different decision making challenges and they watched the brain patterns. In the end, the part of the brain that lit up was the emotional part lit up and not the rational side. Even though they may have weighed all the evidence and maybe had a much more complex rational decision making process with their data, in the end, the part of the brain that lit up was emotional side. No matter who it was. CHARLES: Yeah. GINGER: Good to know, right? CHARLES: It's an adaptation that our brains work so that we actually think that we're acting rationally so we fool ourselves into that. But most of the time, people are making decision with their guts. GINGER: That's how smart we are. We fool ourselves that way. CHARLES: It's the ultimate trick. I'm curious. When you have this process in place, you're developing, leaning on these personas, weaving them in to your solutions and the story that you going to be presenting to your clients, how do you iterate on that? Just like we iterate on software, how do you make those incremental improvements and take what you're learning and then feed that back into the rest of the process? GINGER: First, I'd like to answer that on a human level. I think the way we all learn this is we just practice it. As we put ourselves in another person's shoes, here's the part about it is when we're really dialed into empathy, we're dialed into that feeling of that emotion that somebody has, we're sharing in that feeling. To do that effectively and even when you're presenting a solution to your persona sale situation, talking to your husband or wife, talking to your daughter, words matter. You're really do have to choose wisely and your tone of voice matters. If you're in person with them, your body language and the volume you're using. Those are some of the things that, I think will help when you're presenting or really in any situation to express that you're with them and you're with among the emotion. Let's say that it's something you have an experienced, this VP Engineering, maybe he just had a huge failure. They did not get it right the first time. Their software broke. The money once spend on it proving that it was great, he had to spend on fixing it and starting over. Maybe that never happened to you, hopefully it hasn't but you feel the heaviness of it and you know sadness. He's sad, he's desperate, he's scared, he's going to lose his job, you've had those emotions. That's what you can join him in that emotion. Not pity, you poor thing. More like, "Oh, that sounds really tough. Wow, that doesn't sound like fun." That's empathy so you're there with them and weaving that into your presentation, again not just blowing it off, "Let's not have that happen again because it won't happen that way with us. Here's our solution." That is blowing your chance to have some empathy and sit with them in that emotion for a bit before you get onto your solution. That would be one way to weave it into the presentation. But that takes something for him to share something like that with you too because he knows you're going to counter it and say, "That's not us," but does he know you're going to share in that emotion? Like, "Hey, that didn't sound like fun. Wow. Heavy." CHARLES: Yeah. I'm curious and this is maybe a little bit of a curve ball for you but as you come into, every business is unique. I think doing UI consulting is one unique business out of many. What has been the biggest challenge of coming into this space and trying to develop a clientele inside of it? GINGER: The trick for me since I don't have a heavy technical background is to get away from the technical conversations. Since that's not my strength, I can bring in you guys, bring in a team when it's right, when that person is qualified. But the challenge with me is to find the business people that really want to talk about, "We had this great idea but we don't have enough of bandwidth to execute on it." I can get the idea sold internally and it was just too complicated for us to achieve. I can get those conversations going where really, we're talking about pains, we're talking about business problems that you can approach in any business you go into. It doesn't matter what the end game is and what we're selling. It really doesn't. Everybody has these business problems. To get people to talk about these business problems is the key to make those context to get them to open up about those business problems, business pains then you can qualify and see if it's a fit for your solution. The challenge is to initiate those conversations with people because people do want to get technical quickly and to back them up and to talk about the business first because in the end, it probably does have to get approved by somebody that really is thinking about the business pains and business problems, maybe differently than this person's thinking about it so you want to do the work in the beginning to let all the surface before getting into the technical stuff. That's been always a challenge, whether it's web design development, marketing technology or an engineering UI company like us. CHARLES: Yeah, it's always a challenge. I think you said this at the very beginning of the podcast is you need to be speaking the language of the person that you're speaking with. It seemed so obvious but the reality is though, everybody might be speaking English or one of several human languages within each one of those human languages is literally a million different actual languages, which is carrying on the context of what those people experience in their daily lives. Sounds like what I'm hearing is that there's a specific language within this business and part of it is learning to speak parts of that language, then also be like, "I need an interpreter. I need to bring in someone to speak to this person in their language." On the flipside, I think it's also great too. I'm not a native speaker of some of these higher level concepts and I think that our strength here is in speaking that strong technical language that other technical people can bond with immediately but when you start backing up and just talking about those higher level business concerns, it's really great also to have someone who speaks that natively. Also, that language of even higher than that of, "Let's understand each other's problems here," which is something that you're very fluent in. It is interesting to see that there really is many, many different languages within one language. JEFFREY: I think even within the technical conversations, there are multiple languages. There are different levels of, I don't want to say understanding but different levels of comfort and familiarity with different technologies, with different stacks. A lot of the conversations we have with companies that ours is comfortable in the stack that we're comfortable in and we're presenting that to them and that's where they're coming to us in the first place is because we have that expertise and we have to translate it into, I think that they understand and that fits their needs. GINGER: And that's something, I think you guys are really good at. I don't know if you're giving yourself enough credit there but I think you do ask really good questions about, "Tell me about the experience you want to build? What type of goals are attached to this project?" You guys asked about timeline. That's a goal when you get launched by then. You guys asked what would you say the strengths to your team are. I've heard these questions when you're talking about requirements so you can start to assess, how complex, how much we're connected, pair program with them, help them. I think you guys do ask about goals and plans and challenges. It's just kind of packaged up a different way. CHARLES: Yeah, part of that is just having borne the battle scars of not asking those questions upfront, rather than gravitating through intuition. GINGER: I think we all unfortunately or fortunately learned that way, right? "Don't step in the puddle. Don't step in the puddle. Oh! Did it again." [Laughter] CHARLES: Is there any way to learn that doesn't involve stepping in the puddle or stepping on the tack or falling out of your chair? GINGER: I guess, if you're a baby and you don't know anything yet, you're maybe 50/50. You'll get some of them right. CHARLES: Yeah, it is funny. But just a brief diversion, I just remember there was a period in her life where my daughter would just randomly fall out of chair. It was out of nowhere. It wasn't like she was sitting precariously or whatever. She would just fall out. It happened a lot and I was like, "Wow, sitting in a chair is a learned skill." Anyway, it just made me think about that is that there was actually this pain-based learning process there. GINGER: I wonder what the response was. Did the room laugh? Did the room say, "Oh, let me help you?" Maybe she was ahead of the game. [Laughter] CHARLES: A lot of times it involved tears, which made the laughing worse. I mean, you can't help but laugh. You can also hide it. GINGER: Yeah. Maybe we do need to fall a few times to learn stuff. CHARLES: Ginger, if you could summarize what your business development process looks like in a few sentences before we head out. GINGER: I would say, take your time when you're talking to new people. Get to know them. Get to know who the person is. Get to know what their pains are before you tell them how you can help them. On the empathy side of it, I think the key to empathy is really to remember that it's really not about telling your story: matching them at every step, it's really about understanding their story. CHARLES: All right. I really like that and I really like the process that you're building here. There you have it folks. With that, I'm Charles Lowell from The Frontside. I hope you enjoyed this episode. Thank you, Jeffrey. Thank you, Ginger and we will see you all next week.

Marcy Sutton: @marcysutton | marcysutton.com | Deque Systems Show Notes: 01:07 - Deque Systems 01:54 - Accessibility Tool Integration and Testing 05:26 - Configuration and Success Criteria 07:04 - What is accessibility? WCAG 09:22 - Spurring Adoption of Accessibility 12:09 - The Accessibility Matrix 16:56 - Accessibility-First Development 18:12 - WCAG and ARIA Roles 24:57 - Test Automation vs Human Interaction 28:56 - Empathy Building 30:45 - Porting to the Web 35:57 - Accessibility in Single-page Apps and Focus Management Resources: axe-core aXe aXe Developer Tools WCAG (Web Content Accessibility Guidelines) Web Accessibility for Designers WAI-ARIA Authoring Practices 1.1 First rule of ARIA use Access Works: Usability and Accessibility Training Marcy Sutton: Notes On Client-Rendered Accessibility a11y on Slack Transcript: CHARLES: Hello everybody. Welcome to The Frontside Podcast Episode 61. My name is Charles Lowell. I'm a developer here at The Frontside. With me also is Mr Robert De Luca, a developer at The Frontside and today we have with us, Marcy Sutton who is going to be talking with us a little bit about accessibility, both in the large and the small. Welcome, Marcy. MARCY: Good morning, everyone. Happy to be here. CHARLES: I know, I understand you're actually calling us from the parking lot of a ski area. MARCY: I am at the legendary Mount Baker ski area outside of Bellingham, Washington where we have the winter that is just going on and on and on and getting after it on the last few days of my birthday vacation. ROBERT: Oh, wait. Happy birthday. CHARLES: Yeah, happy birthday. ROBERT: Happy belated or happy birthday. MARCY: Yeah, it was Sunday so still on that shiny birthday week. CHARLES: Well, thank you for getting with us on your vacation and on your birthday but doing a little bit of work, you actually work at Deque Labs. What is it that you guys do over there and what's your particular area of interest and work there? MARCY: Deque is an accessibility company. We have people who work on products and services for accessibility. We help people avoid lawsuits and make their websites and mobile apps more accessible to people with disabilities. My slice of that work is on the product team, where I work on browser extensions, APIs for developers. Basically to make it so you don't have to write every single accessibility tool or test yourself. You can pull in these APIs and get some of that experience that Deque has built up for years and years and years, which was part of the reason I went to work there was to learn from them. We make tools that make it easy for you to make use of that knowledge in your applications. ROBERT: That's awesome. It's like a base JavaScript library that can be ported anywhere, like to browser extensions. I know we use it in Ember accessibility testing. That's really cool. That's where I've gone for the way I write JavaScript. It's in a base library so everybody can use it and it's even more awesome that it's testing and like wrapping tooling around accessibility because I know a lot of developer-minded people want to see like a failed built. CHARLES: Yes, what does that experience look like? I mean, coming from someone who's never even heard of these tools, how would I integrate them into my project and what would change about my workflow? What information would it surface? MARCY: The best place and the reason I work on these products is that I saw projects go out the door broken a lot of times, when working in agencies or maybe testing isn't part of your methodology. Personally in my career, I just knew there had to be a better way. I got into software testing and the more I learned about it, the more I thought that it was sustainable, you could pull in other APIs to help you write better tests. I went to work on axe-core, which is the JavaScript library that we've talked about a second ago. That really is bottling up all of these accessibility tests that you can automate some of the accessibility checks for things like if your HTML markup is in a good state and you're using attributes correctly. Basically, saving you from having to write all of those little microtests, some of which can be sort of complicated. It's all about getting test coverage for the automated things that we can actually test for. CHARLES: You described a pretty wide-ranging coverage. How do you go about actually implementing that into your CI process? Do you just install the axe-core? Do you have to load up your browser and then pointed it out? What does that look like? MARCY: Ideally, you would already have a test suite and you could just pull in the test harness. There's all different versions of aXe. There's versions in JavaScript and in Node. The core thing that you need to test is get your app running in a browser, whether it's a headless browser or could be a mounted browser but we need those actual DOM browser APIs to check things like color contrast. We need to be sort of coupled to the DOMs so that we can run our full set of tests, which is a distinction from, say some shallow rendering that you might be doing in React testing or something like that. For accessibility tests, we need an actual DOM so you could get axe-core on npm and then pull it into your project and then you basically either require or import it, depending on what your stack looks like in JavaScript. Then you have access to all of these tests. It's pretty useful since our ecosystem has evolved to cover things like npm. I've found that it works pretty well. ROBERT: That is pretty neat. You require it into your test and then you visit a page that's fully rendered and then you do aXe check, like you call a method that runs all these checks? MARCY: Exactly. You would call axe.run and then you configure it to run, either specific tests or just one test. One of the tricks that has been helpful to know is that if you disable the color contrast rule, you don't need quite as many of the DOM APIs so it will run faster in things like JSDOM, which doesn't implement the entire browser APIs. But you could call axe.run, either in your unit tests or more likely it would be in your integration tests because you'd already have a browser instance, either through Selenium-WebDriver or karma-chrome-launcher or something like that. Then you basically call axe.run, passing a callback function and then it will return to you at set of JSON results and then you can do things with those. ROBERT: When you call run, can you pass options of what you want to check? Can you filter out things that you know might -- because I imagine like if you put this into an existing app that's been going for a while, I imagine you're going to get a bunch of fails and it might be overwhelming. Is there a way to peel a back like an onion and start working at it that way? MARCY: Yes. You can get pretty specific with our API. The GitHub for axe-core has our entire API configuration. You can get pretty specific. You can filter by tags. I imagined we're going to talk a little bit more about what WCAG is but there's a set of standards that you can break accessibility down into things that you can actually assert that they are either accessible or not. There's all different kinds of what we call success criteria. All of our rules are mapped to these actual guidelines and standards because that means that our tests are helping you solve things that are actually helpful so you could filter by the different levels. Maybe you want to configure it with custom rules. We have some additional products for that. You can get pretty specific with what you want it to run. ROBERT: It's extensible too so you can add your own stuff. MARCY: You can and we do a lot of work with some of our clients to actually help them write custom roles so that's a service that we offer. But the API is pretty configurable on the JavaScript side so you can do quite a bit of configuring on your own as well, which is cool. ROBERT: That is pretty awesome. You alluded to WCAG, I guess now we know how you can integrate a testing library into your JavaScript apps, let's take a step back a little bit and what exactly is accessibility and then you can start explaining WCAG because WCAG is a very big document that tells you how to go and be accessible. CHARLES: I assume WCAG is some acronym? MARCY: It is. Peeling that back a little bit to what is accessibility. I'm more on the digital side. There is physical accessibility as well for spaces. But when we're talking about digital accessibility, we're talking about making apps and websites that work for people with a broad range of abilities. Say, you had color blindness or a low vision or you're fully blind, you would need to be able to zoom in, you need high-contrast colors, you might use a screen reader if you're blind. But then there's other categories. People might actually fall into more than one category including motor disabilities, where maybe you can't use a mouse and you have to use a keyboard only or a keyboard with one button, which is how we think about a switch control --that's another device. You might be deaf or hard of hearing and need transcripts or close captions so any audio or video content needs an alternative of some kind. Then there's cognitive disabilities where people have learning disabilities. Maybe the language used on a website is too vague or too marketing copy speak and we need to simplify, people with traumatic brain injury like Stephen Hawking has ALS. I discovered at some point in my career that I could actually make the web a better place by supporting all different kinds of people. That's really what it's about for me is doing good craftsmanship and making sure that you're actually making things as accessible as you can. The WCAG thing that we mentioned, it stands for Web Content Accessibility Guidelines. It's just that. It's a set of guidelines, sort of a map to help you get there. You have to actually interpret those guidelines and put in the work to do it. The guideline is just a guideline but it gives us a really good roadmap of how to implement all of these different areas of accessibility. CHARLES: I actually had a question and this is a little bit harkening back to the discussion about the axe-core but also kind of straddling. How do you spur adoption, both the technology and the value inside of your development team? You know, we definitely make our web apps as accessible as we can because we have Rob on the team. But for teams that don't have Rob, how do you spurred option? How do you pitch it to your team and to your management structure? Like testing. Testing used to be controversial. I think in some pockets, it still is but it was something that you had to pitch or agile methodologies was something that you had to pitch. Now it's kind of accepted. It's a core-value of development, I think. I hope. MARCY: Definitely more so. I agree. CHARLES: Do you see a future where making applications accessible is just a tenet of development in the modern era and how do we get to that point? How do we pitch our teams to adopt that value? MARCY: Part of what I'm trying to do is meet developers where they're at and make tools that make it really easy and free to integrate things so it doesn't cost you anything to npm install a library and pull it in your project or to use a free browser extension. What we're trying to do is really help developers get there by lowering the barriers, just kind of a funny way to put it because that's what we're doing with accessibility is removing barriers for people that get access to things. I'm pretty optimistic about it. We talked a lot in the accessibility world about education is really needed because often, it's just that people don't know about it. I've made it my mission to spread the word as much as possible by doing talks and blog posts and just trying to get as many people on board as possible, instead of making them feel bad about it like, "Oh, you don't know about this? You're terrible." ROBERT: Oh man, you're speaking to me. MARCY: "-- You can do this." I try to bring people along and make them feel welcome because it's not really a fun experience to be like, "Oh you're bad because you didn't do this. You don't think about this thing." That's what I try to do. ROBERT: One of my first experiences in accessibility was like somebody giving me that moral argument like, "You're ruining people's lives. They can't do things on their computer." I just remember the response I had and it wasn't that, "Oh, you're right. I should go make this accessible. It was more of like I had a flight or fight response. I start to justify the reasons I didn't do and that wasn't a good experience so the way you put it, like meet the developers where they're at, I love that because that's how I've been operating too. I think accessibility is just another engineering problem and it can be an engineering problem that would be fun to solve. The accessibility matrix gets really hard and hairy as you get into it like -- CHARLES: Oops! Jargon alert! What is the accessibility matrix? Does the accessibility matrix has Neo? ROBERT: The different AT combos and since my experience stems from screen readers -- MARCY: Assistive technologies? ROBERT: Yeah, assistive technologies -- I'm doing a poor job here -- Basically, you have three levels that you work with here. It's the operating system, the type of assistive technology and if we're talking about the web, it's the browser. You could have like the matrix, the beaten path is MacOS, VoiceOver and Safari. That's going to be your matrix. Then on Windows, it could be Windows JAWS and Internet Explorer or Windows NVDA, which is another screen reader on Windows. JAWS is also a screen reader. The browser for NVDA would be Firefox. Then it can just fork in any of those different combinations that you could possibly imagine that makes it hard to debug for. But that's why I think this is a cool programming problem is because we can build awesome tools to help us do this and test for it like aXe. MARCY: Yeah. I would also argue that it's almost even more of a design problem. It's part of the additional challenges that we have to get our design friends and colleagues on board as well because the more that they are thinking about it before they handed off to us, the less we're going to be caught in these situations where we have to make it work in one browser and assistive technology but then it's broken somewhere else because we're trying to use really experimental APIs or we're just trying to do things differently for the mouse versus the keyboard. I can tell you that could be really difficult. The more we're thinking about making things straightforward and intuitive from the design side, not to say the easier job is going to be but the more successful, I think we can be as a team because it's more than just developments responsibility. There's good resources for designers as well, like a web accessibility for designers. If you just Google that, there's a great checklist from WebAIM. I think it's helpful to make it inclusive to people that we work with, not just in the development side because we really want them to set us up for success or else were really just fixing problems that not at their core. You know what I mean? ROBERT: Yeah, as they come down the pipe, we're kind of dealing with them instead of getting ahead of it. CHARLES: That reminds me actually of an experience that I had, a pair programming with Rob, probably about a year ago as we were making an interaction model for a select box. This was for a custom client. We actually stripped it away and we're like, "Let's just focus on what is the state machine behind this thing," so we drew it out on the board and it turned out that we were really just capturing the interaction apart from any rendering so we had a very strong model. With each state's transition, we were able to basically radiate that information with a screen reader in this case. But it was actually very trivial to do because we've actually forgotten about the DOM, forgotten about the fact that we were actually chasing a visual interaction and like I said, what is the actual user interaction? What is the information coming in and coming out? It turned out once we kind of flush that out and have developed that, hanging the interface on that skeleton was very easy and we could do it in multiple media. It feels like a similar concept where if your designers are very upfront about really exploring the information architecture of an application then being able to represent that information architecture in multiple forms becomes much easier because the joints and beams are very, very clear and they aren't bound to a particular form of representation. MARCY: Yeah, I think it a way that's definitely true. One challenge I would issue for this part of prototyping would be to consider all of the user inputs. Make sure that you're considering a keyboard user hitting an escape key to close that select or maybe they're using a screen reader on a touch device and like the single finger swipe, it's already allocated when that screen reader is running so if you have an interface that was only swipe left or right and there were no other affordances like buttons that you could actually activate, that would be an unusable interface to a mobile screen reader user. What really helps to make that information architecture stand up or hold out when you're developing it, like stay true to your vision through the process is making sure that you're considering all of those user inputs. Sometimes, developers aren't thinking about keyboard user so they're not thinking about focus styles, really trying to activate it another way. I do think that's a helpful exercise. ROBERT: Yeah, and to be fair at Frontend developers, we already have a lot to think about. It's just a lot to juggle so I can understand that's why we have tools like aXe. But what Charles is talking about, I think is actually kind of neat is we were experimenting with accessibility-first development so the people do TDD -- test driven development -- and I was trying to see if we could build something. I wanted to see if what we're writing would yield better software if we did it with an accessibility in mind from the outset. I think that's true. It was a more accessible typeahead. It was better, more well-defined user experience around the typeahead and it was because we thought about accessibility and all of the different edge cases. We really boil it down to the core problem. CHARLES: Right. We were driving it first with keys and nonstandard interaction methods. It meant that we actually got more clear interaction model lying underneath. It was decoupled from the actions that drove it completely because we had to support too from the get go. ROBERT: I thought that was neat. CHARLES: Yeah that was a fun exercise. You know, we should have blog about that because I think that actually results in better software. ROBERT: Yeah, I had a conference talk brewing in there somewhere. Just never got around to it. Talking about the web accessibility guidelines. There's different levels to it. Now, you have an A, AA, and AAA. What do those mean and where does that play into ARIA roles and stuff? MARCY: There's WCAG 2.0 and actually 2.1 is an update that they're working on right now but WCAG 2.0 is -- ROBERT: Oh, yeah. I saw that. MARCY: Yeah, there's some new stuff coming out. It's mainly for low-vision users and mobile touch things. But the WCAG 2.0 is the blessed standard that we're working with right now and the levels are different conformance levels. There's different things that you can achieve with A, AA, or AAA. Most people go for AA. AAA is pretty restrictive in what you can do and if you make it support WCAG 2.0 AA, it doesn't necessarily mean it's going to be intuitive to use. You could make it technically conformant but it won't necessarily be that beautiful or accessible. There's a bit of a dance that we have to do around that to meet these guidelines but do them in an intentional way so that we're actually making something usable. I think that goes back to that idea of craftsmanship and caring about your user to know if this actually going to work for them. There's a number of success criteria in WCAG that are broken up into different categories. There's perceivable, operable, understandable and robust. Within each of those, there's all kinds of different checkpoints that you can look at to inform how do I make this keyboard accessible. There's all kinds of really helpful documentation. That's the WCAG guidelines and within each of those, there are a number of different ways that you can code something. As I'm sure you know, there are infinite ways to code the same thing, pretty much and part of what that cover is techniques for making things accessible. They'll tell you all about Native HTML and what tools you can use within that standard. Then there's this other standard called WAI-ARIA and that's the Web Accessibility Initiative – Accessible Rich Internet Applications. That was originally created back in the day when we didn't have as many browser APIs and we didn't have great ways to expose accessibility information to screen readers. They made this API in browsers that implemented that you can actually bolt on some of the same information that you get from HTML. It's helpful if you're writing as VG or XML, where you just don't have those built in semantics so we have things ARIA role states and properties. You may have seen things like 'role="button"' or 'role="main"' or 'role="search"'. You might see that somewhere and that is just exposing programmatically bolting on a role to any element. You could put on 'div role="button"' and there's a little more that goes into that to make it an accessible button. Anytime we mentioned -- ROBERT: The tab index. MARCY: Yeah, the tab index. You have to make sure you have a keyboard event but that would be a programmatic way to create a button element. You should always start with the native button element because you get all that stuff for free but ARIA gives us an API to actually implement accessibility information. You'll see those techniques come up a lot in WCAG of how you can accomplish the same thing multiple ways. Those are some of the things that we test for in our animated tests in aXe. We check to make sure that you've only use roles that are actual roles because there is a set standard of them. We check to make sure that all of the ARIA values that you might use are actually allowed for that. Sometimes, if you're using 'role="list"' for whatever reason, you can't use a real list. It is possible to create a list with ARIA but if you had the wrong child role or something, that's a pretty easy thing that we can flag with aXe so we're sort of saving you from yourself. It helps me sometimes when I get a role wrong because we're human and we do make mistakes. There's a lot of things to remember so that's pretty key technique that aXe will help you with. That's making sure that your ARIAs used correctly because it is pretty easy -- ROBERT: That's really nice. MARCY: -- to get it wrong, to be honest. ROBERT: Yes. I've definitely done that. Being through the spec document is not the most fun. Trying to read the standards language is a little bit complicated so having a tool like aXe is really helpful for me to pick my way through it like, "aXe will tell me that this is wrong," so it narrows the problem set down for me where I can go and look at the standard and kind of tunnel vision in on that one, rather than get overwhelmed looking at that whole standard documents like there's so much here. MARCY: Yes, there is. One thing that might help with the is the initiative that people are working on called the ARIA Practices Guide, the ARIA Authoring Practices and it sort of breaks down these techniques into what is the keyboard navigation model for that component or it will break it into known patterns. This is really helpful also for designers to know what are some known patterns and how can I implement accessibly. They can really help you jumpstart to using those patterns with this more easily digestible information to tell you how to do it correctly. That has come up in the last few years that I found really useful. ROBERT: That's awesome. I think I've seen this. Is it where they tell you like, "If you're going to reimplement a checkbox, here's how you would do with ARIA?" MARCY: Exactly. I've dropped a link in the chat so we'll expose that in the show notes, I'm sure. There's more resources out there now that are really helpful. There's another one called ARIA in HTML and that one is also from the W3C and it's a note on using ARIA and HTML. That one I found to be very useful as well because they tell you this first, second, third, fourth, and fifth rules of ARIA use. The first rule of ARIA use is if you can use a Native HTML element or attribute, you should absolutely use the built-in one first. That's a big -- ROBERT: Yeah, let's stop reinventing. MARCY: Yeah, you know it's tempting because you can create these custom elements and try to bolt on ARIA but the reality is that if you're trying to make it really backwards compatible, it's just so much easier to support the native things. There is an assisted technology called Dragon NaturallySpeaking, that's a dictation method and they didn't support ARIA until 2014 so you can easily imagine some of your user base with an older assistive technology. That might be completely broken for them so that's why we really push using the native things first just because of the better support on every platform. CHARLES: I have a question about the test automation. We've been talking a lot about aXe in the way that you can do this. Did I get it right? Are my roles correct? And all these things. What's an example of something that you just can't test for in an automated fashion? It just requires human interaction just to perceive it. I mean, this would be right now, kind of in the visual sphere, the state of automation for testing like did I break the layout still requires a human. What are examples of that in terms of accessible interface where you just do the things that you have to be on the lookout for that you can't cover with automation right now? MARCY: I think context and content are some of the most difficult like writing good all text. That can be really challenging just because what makes a good alt for an image and that supposed to be a text alternative to say, "This is something useful," and Facebook has solved that by using artificial intelligence to dynamically guess what's in an image. A blind colleague of mine that works there has written about and he said he always felt left out when he would read his news feed and someone would be talking about their first love or some kind of vague status update. With this new feature, it could say, "Oh, this image that they're talking about their love is a pepperoni pizza," or something where -- [Laughter] MARCY: It's really missing the context so they've started to do automatic all text. For us doing accessibility checks, we try to keep our solution as light weight as possible and without false positives. We can check whether you have an all attribute missing like you don't even have the alt attribute at all which means that the file name would be read in the screen reader which is often terrible, depending on what your filenames are so we can check if that's missing but we can't really tell you what would make a better alt attribute, if you already have one. That's one is a bit difficult. There's another one that we're working on right now with color contrast where we can't really tell if you have a background image that's behind some text. If it has multiple pixel color values in it, even if we could read those colors, it gets really hard for us to say whether text meets color contrast when it's over an image for multiple reasons. That one's a bit tricky. I think there are some other examples throughout WCAG that we can only automate. Depending on which rule set you're using, we estimate between 30% to 40% of issues, we can actually catch with automated tests so there is quite a bit that we still need humans for. But however, I think some of these really basic ones that we can check to help you do those easy wins so that you're not getting messed up by using the attribute Aria-role when it's just role. Those kind of things. It's like we're helping you so you can save that time for those more complex task that might require a human. There's definitely no substitute for trying to use the keyboard to make sure that your app is usable from the keyboard. Test it with a screen reader, you can find people in the web accessibility Slack that might be willing to help you test it, if you're extra nice or maybe you can give them a gift card or something. There is an organization called Knowbility and they have this thing called Access Works where if you need to find a user with a disability to deduce a user testing for you because that's a great thing to do. It's very important. They can help you, as a business think up with someone who can test your app. I would definitely check out Access Works. That's really what's the missing piece. As a developer, I'm okay using a screen reader after doing accessibility for a few years but it's not my primary way of navigating so it's really helpful to have real users to test your app and that's a good way to find someone to actually test it. That sort of makes up the rest so you can get that really valuable feedback. ROBERT: I'm a firm believer in testing but also, I really do think a lot of accessibility work is just kind of empathy building and the way you do that is just sit down and actually use this assistive tech that these people will be using. In that way, you can understand as you're building it, how somebody might move their screen or cursor over the top of this and you can start to think about what the screen will read off and stuff like that. I think using a screen reader as a developer is powerful. But I agree, it will never reach the level like my mom that has been using a screen reader for seven years now. I'll never be able to use it as well as she does. It actually putting in the hands of people that do this day-to-day and live this. A far better idea and that goes beyond accessibility too. You want to user test all your apps anyway. MARCY: Yeah, exactly. I think that should be a big thing that we demand just from our organizations like how you were saying it was kind of controversial. I feel like user testing is another flavor of that where we have a bit of emotional tide of these things that we create and we want them to be perfect in the way that we have envisioned but not everyone interacts with things the same and it's really humbling to watch someone use something that you made and have it completely not get it at all. I think that's a really valuable experience. I've watched my mom or my dad or people try to use something that we assume is really intuitive and it's just not. We look at the web all day -- day-in and day-out being professionals and it's really helpful to show it to people who maybe aren't as fluent, aren't digital natives like that. CHARLES: We talked about actual user testing. We talked about the checking where you render your application and you run a set of checks. Do you have any experience with actually -- this is kind of an idea that just occurred to me, although we did a little bit of it when we were doing native applications -- using the accessible interfaces to actually drive your acceptance tests? Is that anything that you have experience with? Because it seems like on the face of it, you've got this assistive technology that surfaces the key levers of your application so is it a good idea to grab those levers from within your test case? Within your acceptance test to manipulate your application and thereby kind of front load your accessibility because in order to verify it, you must have those levers in place. MARCY: Yeah, from understanding your question correctly, you're wanting to just run your tests using accessibility features? CHARLES: Yes. For example, when we write our acceptance tests in our application, what we do is as part of setting them up, say we want to click here and I want to enter this text into this text box and I want to move this over here and that implies actually dispatching mouse events, keyboard events and then also being able to find the elements in the DOM that I want to dispatch those events on so we're kind of doing it in, I think we use CSS selectors to find them and then we use the jQuery event interface to actually create the events and send them to those elements. But it seems that part of ARIA roles or something else is like identifying the role that this element has in your application and basically saying, "For my test cases, I'm going to use these roles and I'm going to use these things and I'm going to use different access methods, keyboard mouse or whatever to manipulate my interface." Does that makes sense? MARCY: Yeah. ROBERT: I think this makes sense in the native world where in order to get the label, I think you have to use the accessibility label. CHARLES: They do that when you're functionally testing iOS apps so why not -- ROBERT: Does it port to the web, basically. CHARLES: Yeah, does that port to the web? MARCY: It does -- CHARLES: It's really long, way of saying that, I guess. Sorry you all. [Laughter] MARCY: No, and I wanted to clarify because I was wondering if you're talking about driving it with actual assistive technology, which we can't quite yet. We don't have any tools for that. But yes, you should -- ROBERT: We should explore that in Ember. MARCY: Yeah, we just don't have the hooks for that. Maybe Python and NVDAs, since it's open source, maybe AppleScripts. CHARLES: What would that look like to drive it with assistive technologies? ROBERT: We talked to some people at Apple with Ember accessibility team and if I remember correctly, we could only drive VoiceOver on MacOS with AppleScripts but there was no way to do it in any other way so you only could do it with VoiceOver on MacOS and that was still kind of murky. MARCY: Yeah, exactly. The idea would be, rather than just testing the browser, we would actually be able to run a simulator programmatically, to know is the screen reader actually exposing this information. Because a lot of it is there are things that get lost in translation, sometimes where we're following best practices and standards because we have this agreement that people who implement browsers and screen readers are going to follow those standards. It's definitely is not always smooth sailing with that. But there's sort of this disconnect between the browser testing and then actually firing it up in the screen reader and make sure it worked. We take that on faith a lot of time, which is getting back to your original question, why it's so valuable to have tests that use these interaction methods. Absolutely, either in your unit tests or even in your integration test, they can live in either place to have tests that assert and closes with the escape key or it operates with the enter key or whatever the user interaction should be, that we have tests that assert that because that way, if you leave your team or heaven forbid, you got hit by a bus or something, you have a test coverage that makes a contract of how this component should work and you have accessibility support, actually built into your test infrastructure. That is super valuable. At least we know that that part of it is there. We know we can drive it from the keyboard, which is how a lot of screen readers work. They operate on top of the keyboard so we can get really far just by having basic keyboard support. Then, if you pull in an API like axe-core, you can have it tell you if you were using ARIA wrong or something. It's sort of a combination of both where those feature tests in your actual project where you're writing something that it works with the escape key, those are custom tests for your application. I find that they're really valuable just to have in there, especially if you work on a component library or something reusable so that everybody who is contributing knows how this thing is supposed to work. I think that is really valuable. ROBERT: Absolutely. I want to talk about accessibility in single-page apps. The problem with accessibility in single-page apps is while using a screen reader, you click a link and to the screen reader user, all it says is the link was pressed. They don't actually know that the content has changed. But in Ember, we kind of solve this by focusing the outlet that has changed but in other frameworks, in your experience everywhere else, how do you combat this? What are the best ways of attacking this? CHARLES: Yeah, what are the problems that you encounter in single-page applications? MARCY: I've done quite a bit of research and blogging and conference talks on this. I'm working on the Angular team for a while. The issue with the single-page app is the page isn't being refreshed when you make a raving change or something happens dynamically. The user's focus is never refresh to the top of the page so they don't hear a title change or things like that. There's different techniques that you can employ to make that experience more accessible. The first and foremost tool to have in your toolbox is focus management so that you're programmatically sending the user's focus to this new content. Say, I have a sidebar with links in it and I click one of them, I can send focus to content wherever it loaded on the page. That way, they are both alerted to the new content because depending on where you send it. There's different techniques for this but often, we will send focus to the wrapping element so that everything will be read aloud and you can accomplish that by using tab index of -1 in your HTML. That will make this wrapper catch the focus, essentially but it won't add it to the tab order of the entire page. That's a technique that we used to shuffle focus around. I've also seen people use what's called an ARIA Live Region where you have this element somewhere on your page that's not visible. It has to be rendered so you can't use 'display: none' but you can basically pipe messages to these live regions to announce what's happening on the screen. I've just saw a React example where they put an ARIA Live attribute just on that wrapping element, instead of the focus management so anytime new content went into that element, it would just be announced. The challenge with that is that you can't always control everything on the page. That works if you control everything and you know that only this one element is getting updated at the time. But often, we work in this big ecosystem where there's a bunch of things happening. Depending on how complex your app is, you might need some sort of a focus manager, some sort of a utility that will keep track of what's focused and routed around at a correct place. That's the biggest tool for creating accessible single-page apps, that's focused management. I mean, not only for the reading content purpose but also to have their focus in the more accurate place so if they hit tab or they try to start interacting with something that they're in the right part of the page. A good example, if you think about like a modal window -- a modal window may open as a new layer over something -- that requires focus management on open so that your focus is sent into it, either to the first focusable element or to the wrapper. Then when you hit escape or close the modal, it just send your focus back. ROBERT: To the previously focused element, right? MARCY: Exactly, so that if you are using a keyboard and you can't actually use a trackpad or a mouse to get back then you're in the right place or if you're screen reader user and you can't even see the screen, then you're always in the right spot. That's actually, I think really cool. Something that's become more common place with dynamic JavaScript apps is that we can do these really cool focus management techniques. I think they're really cool, they can be challenging but that is something that we definitely need to think about as developers of single-page apps. ROBERT: Absolutely, especially since none of the single-page app frameworks out there were libraries. Actually maybe with the exception of your work on Angular, they don't come with a router focused-library built in so this is something that you have to actually think about and then pull in and do yourself. Does Angular have it, by default? MARCY: No, we never added a focus manager utility. There were some things to try and clean up that HTML, which ended up being, honestly worse than the original problem. But I've written a blog post about focus management techniques. I just dropped that in the chat. There's a smashing magazine article I wrote and it really is framework-agnostic so it sort of covers all of the things that you need to think about if you're writing a client-rendered application using Ember, React or Angular. It is something that we have to think about as developers because from the framework level, it's impossible to know what the right situation would be in your app in a given moment so we can only get so far with magic at the framework level. It's something I would like to see more of. Maybe if there is some sort of a layer manager, I think that is a tool that someone could write that would be super useful -- to make sort of an intelligent layer managing system for focus management. I've heard the Facebook team talked about how they do it internally but it's not open source so I have yet to see an open source solution for this. We have to tackle it in our own apps but once you know that that's the thing, you can really make sure that you're covering it. If you have someone with a visual disability or impairment that try and use your app, they'll probably uncover that problem pretty quickly. That's the value of user testing in case you forget. Maybe there's a few views -- ROBERT: Need to sell it. MARCY: Yeah, or maybe with your application, if you don't have visible focus styles turned on, you might not see that the focus isn't being sent. That is one trick, I will tell you in development. If you're working with focus management, turn the focus outlines on and then if you were trying to send focus before it got fully rendered or something because it has to actually be rendered to catch the focus. That is good debug flag, if you can all agree on the focus styles, for all users. I found that to be really useful in our app. You just to have those turned on so you can debug it. ROBERT: And make it really loud like this is a giant red outline. MARCY: Yeah, then you'll know, if you forgot to add tab index of -1, to make it catch the focus or like I said, maybe there's a rendering thing where you need to wait a tick by using a set time out or something. That is a good technique that I've used recently. ROBERT: Awesome. Basically, what it boils down to in single-page apps is manage your focus and enhance your focus, some might say. MARCY: Yeah, let's think about keyboard ergonomics, like if you are doing things dynamically on the screen and then you want to start typing, I think the most common example I see is autofocus. The developers, even if they aren't thinking about accessibility, they'll ask for autofocus. That in a way is focus management. The difference with autofocus is that you can only use it once and it will send your focus there automatically. But in a similar way, that's the idea of what we want is to get the user's focus point into the right spot so that they can do the right activity on the screen and they know what content they're looking at. ROBERT: Right. Sometimes, it's like navigating around a website with your keyboard, that's like power users who have Vim or Emacs or anybody that's a power user of computer that doesn't like to leave the home row, you can make your application awesome for you to use and also lay the groundwork for accessibility, if you can navigate your website with just a keyboard. MARCY: Exactly. ROBERT: Let's try to pitch it to people in that way. It's still a developer problem. CHARLES: I like that because it really highlights the fact that there is this kind of deep interaction model. The user actually is focused on one thing at a time in the application and if you track that, then it's going to be a benefit for all of your users. If you are deliberate about thinking like this is the subject of interest at this moment. You're just going to reap a lot of benefit for everybody. ROBERT: Keep coming back to it, building accessible applications yields a better application for everybody. MARCY: Absolutely. It might enable you to support some futuristic device that you haven't even thought of yet. If you have your actions decoupled from the actual input and you can do everything declaratively, that really makes it easier to try and support of use cases you haven't thought of like we need to borrow up that other keyboard combination or some touch device. It just really helps to not have everything buried in a jQuery event. ROBERT: Yes. [Laughter] MARCY: Like, "Oh, man I need to call that same functionality for multiple events. Crap." You need to decouple that real quick. ROBERT: "Let's obstruct this." CHARLES: Right. I think we're about the time. I know you've got a hard stop. You got some skiing to do. MARCY: I do. CHARLES: So we will let you get up on the mountain but thank you so much for coming by. This is been a great conversation. ROBERT: Yes, thank you for dropping all the knowledge. CHARLES: Yeah, I'm feeling lots of knowledge right on top of my head -- MARCY: Awesome. CHARLES: -- That I got to go and process. But for everybody else out there, I would say go experiment with aXe. The idea is going to be easy for developers. I know I'm going to experiment with it and then you said, there was a browser extension as well to help you out and probably call out every website that you ever use, right? MARCY: I'm dropping some links for you, just now. CHARLES: There's some links to go along with the knowledge so go check them out and you are @MarcySutton on Twitter? MARCY: That is correct. CHARLES: All right. Fantastic. Thank you so much for coming by. MARCY: Yeah, no problem. Thanks so much for having me.