Podcast appearances and mentions of mike perham

Mike Perham is the creator of Sidekiq, a background job processor for Ruby. He's also the creator of Faktory a similar product for multiple language environments. We talk about the RubyConf keynote and Ruby's limitations, supporting products as a solo developer, and some ideas for funding open source like a public utility. Recorded at RubyConf 2023 in San Diego. -- A few topics covered: Sidekiq (Ruby) vs Faktory (Polyglot) Why background job solutions are so common in Ruby Global Interpreter Lock (GIL) Ractors (Actor concurrency) Downsides of Multiprocess applications When to use other languages Getting people to pay for Sidekiq Keeping a solo business Being selective about customers Ways to keep support needs low Open source as a public utility Mike Mike's blog mastodon Sidekiq faktory From Employment to Independence Ruby Ractor The Practical Effects of the GVL on Scaling in Ruby Transcript You can help correct transcripts on GitHub. Introduction [00:00:00] Jeremy: I'm here at RubyConf San Diego with Mike Perham. He's the creator of Sidekiq and Faktory. [00:00:07] Mike: Thank you, Jeremy, for having me here. It's a pleasure. Sidekiq [00:00:11] Jeremy: So for people who aren't familiar with, I guess we'll start with Sidekiq because I think that's what you're most known for. If people don't know what it is, maybe you can give like a small little explanation. [00:00:22] Mike: Ruby apps generally have two major pieces of infrastructure powering them. You've got your app server, which serves your webpages and the browser. And then you generally have something off on the side that... It processes, you know, data for a million different reasons, and that's generally called a background job framework, and that's what Sidekiq is. [00:00:41] It, Rails is usually the thing that, that handles your web stuff, and then Sidekiq is the Sidekiq to Rails, so to speak. [00:00:50] Jeremy: And so this would fit the same role as, I think in Python, there's celery. and then in the Ruby world, I guess there is, uh, Resque is another kind of job. [00:01:02] Mike: Yeah, background job frameworks are quite prolific in Ruby. the Ruby community's kind of settled on that as the, the standard pattern for application development. So yeah, we've got, a half a dozen to a dozen different, different examples throughout history, but the major ones today are, Sidekiq, Resque, DelayedJob, GoodJob, and, and, and others down the line, yeah. Why background jobs are so common in Ruby [00:01:25] Jeremy: I think working in other languages, you mentioned how in Ruby, there's this very clear, preference to use these job scheduling systems, these job queuing systems, and I'm not. I'm not sure if that's as true in, say, if somebody's working in Java, or C sharp, or whatnot. And I wonder if there's something specific about Ruby that makes people kind of gravitate towards this as the default thing they would use. [00:01:52] Mike: That's a good question. What makes Ruby... The one that so needs a background job system. I think Ruby, has historically been very single threaded. And so, every Ruby process can only do so much work. And so Ruby oftentimes does, uh, spin up a lot of different processes, and so having processes that are more focused on one thing is, is, is more standard. [00:02:24] So you'll have your application server processes, which focus on just serving HTTP responses. And then you have some other sort of focused process and that just became background job processes. but yeah, I haven't really thought of it all that much. But, uh, you know, something like Java, for instance, heavily multi threaded. [00:02:45] And so, and extremely heavyweight in terms of memory and startup time. So it's much more frequent in Java that you just start up one process and that's it. Right, you just do everything in that one process. And so you may have dozens and dozens of threads, both serving HTTP and doing work on the side too. Um, whereas in Ruby that just kind of naturally, there was a natural split there. Global Interpreter Lock [00:03:10] Jeremy: So that's actually a really good insight, because... in the keynote at RubyConf, Mats, the creator of Ruby, you know, he mentioned the, how the fact that there is this global, interpreter lock, [00:03:23] or, or global VM lock in Ruby, and so you can't, really do multiple things in parallel and make use of all the different cores. And so it makes a lot of sense why you would say like, okay, I need to spin up separate processes so that I can actually take advantage of, of my, system. [00:03:43] Mike: Right. Yeah. And the, um, the GVL. is the acronym we use in the Ruby community, or GIL. Uh, that global lock really kind of is a forcing function for much of the application architecture in Ruby. Ruby, uh, applications because it does limit how much processing a single Ruby process can do. So, uh, even though Sidekiq is heavily multi threaded, you can only have so many threads executing. [00:04:14] Because they all have to share one core because of that global lock. So unfortunately, that's, that's been, um, one of the limiter, limiting factors to Sidekiq scalability is that, that lock and boy, I would pay a lot of money to just have that lock go away, but. You know, Python is going through a very long term experiment about trying to remove that lock and I'm very curious to see how well that goes because I would love to see Ruby do the same and we'll see what happens in the future, but, it's always frustrating when I come to another RubyConf and I hear another Matt's keynote where he's asked about the GIL and he continues to say, well, the GIL is going to be around, as long as I can tell. [00:04:57] so it's a little bit frustrating, but. It's, it's just what you have to deal with. Ractors [00:05:02] Jeremy: I'm not too familiar with them, but they, they did mention during the keynote I think there Ractors or something like that. There, there, there's some way of being able to get around the GIL but there are these constraints on them. And in the context of Sidekiq and, and maybe Ruby in general, how do you feel about those options or those solutions? [00:05:22] Mike: Yeah, so, I think it was Ruby 3. 2 that introduced this concept of what they call a Ractor, which is like a thread, except it does not have the global lock. It can run independent to the global lock. The problem is, is because it doesn't use the global lock, it has pretty severe constraints on what it can do. [00:05:47] And the, and more specifically, the data it can access. So, Ruby apps and Rails apps throughout history have traditionally accessed a lot of global data, a lot of class level data, and accessed all this data in a, in a read only fashion. so there's no race conditions because no one's changing any of it, but it's still, lots of threads all accessing the same variables. [00:06:19] Well, Ractors can't do that at all. The only data Ractors can access is data that they own. And so that is completely foreign to Ruby application, traditional Ruby applications. So essentially, Ractors aren't compatible with the vast majority of existing Ruby code. So I, I, I toyed with the idea of prototyping Sidekiq and Ractors, and within about a minute or two, I just ran into these, these, uh... [00:06:51] These very severe constraints, and so that's why you don't see a lot of people using Ractors, even still, even though they've been out for a year or two now, you just don't see a lot of people using them, because they're, they're really limited, limited in what they can do. But, on the other hand, they're unlimited in how well they can scale. [00:07:12] So, we'll see, we'll see. Hopefully in the future, they'll make a lot of improvements and, uh, maybe they'll become more usable over time. Downsides of multiprocess (Memory usage) [00:07:19] Jeremy: And with the existence of a job queue or job scheduler like Sidekiq, you're able to create additional processes to get around that global lock, I suppose. What are the... downsides of doing so versus another language like we mentioned Java earlier, which is capable of having true parallelism in the same process. [00:07:47] Mike: Yeah, so you can start up multiple Ruby processes to process things truly in parallel. The issue is that you do get some duplication in terms of memory. So your Ruby app maybe take a gigabyte per process. And, you can do copy on write forking. You can fork and get some memory sharing with copy on write semantics on Unix operating systems. [00:08:21] But you may only get, let's say, 30 percent memory savings. So, there's still a significant memory overhead to forking, you know, let's say, eight processes versus having eight threads. You know, you, you, you may have, uh, eight threads can operate in a gigabyte process, but if you want to have eight processes, that may take, let's say, four gigabytes of RAM. [00:08:48] So you, you still, it's not going to cost you eight gigabytes of RAM, you know, it's not like just one times eight, but, there's still a overhead of having those separate processes. [00:08:58] Jeremy: would you say it's more of a cost restriction, like it costs you more to run these applications, or are there actual problems that you can't solve because of this restriction. [00:09:13] Mike: Help me understand, what do you mean by restriction? Do you mean just the GVL in general, or the fact that forking processes still costs memory? [00:09:22] Jeremy: I think, well, it would be both, right? So you're, you have two restrictions right now. You have the, the GVL, which means you can't have parallelism within the same process. And then your other option is to spin up a bunch of processes, which you have said is the downside there is that you're using a lot more RAM. [00:09:43] I suppose my question is that Does that actually stop you from doing anything? Like, if you throw more money at the problem, you go like, we're going to have more instances, I'll pay for the RAM, it's fine, can that basically get you out of these situations or are these limitations actually stopping you from, from doing things you could do in other languages? [00:10:04] Mike: Well, you certainly have to manage the multiple processes, right? So you've gotta, you know, if one child process crashes, you've gotta have a parent or supervisor process watching all that and monitoring and restarting the process. I don't think it restricts you. Necessarily, it just, it adds complexity to your deployment. [00:10:24] and, and it's just a question of efficiency, right? Instead of being able to deploy on a, on a one gigabyte droplet, I've got to deploy to a four gigabyte droplet, right? Because I just, I need the RAM to run the eight processes. So it, it, it's more of just a purely a function of how much money am I going to have to throw at this problem. [00:10:45] And what's it going to cost me in operational costs to operate this application in production? When to use other languages? [00:10:53] Jeremy: So during the. Keynote, uh, Matz had mentioned that Rails, is really suitable as this one person framework, like you can have a very small team or maybe even yourself and, and build this product. And so I guess from... Your perspective, once you cross a certain threshold, is like, what Ruby and what Sidekiq provides not enough, and that's why you need to start looking into other languages? [00:11:24] Or like, where's the, turning point, or the, if you [00:11:29] Mike: Right, right. The, it's all about the problem you're trying to solve, right? At the end of the day, uh, the, the question is just what are we trying to solve and how are we trying to solve it? So at a higher level, you got to think about the architecture. if the problem you're trying to solve, if the service you're trying to build, if the app you're trying to operate. [00:11:51] If that doesn't really fall into the traditional Ruby application architecture, then you, you might look at it in another language or another ecosystem. something like Go, for instance, can compile down to a single binary, which makes deployment really easy. It makes shipping up a product. on to a user's machine, much simpler than deploying a Ruby application onto a user's desktop machine, for instance, right? [00:12:22] Um, Ruby does have this, this problem of how do you package everything together and deploy it somewhere? Whereas Go, when you can just compile to a single binary, now you've just got a single thing. And it's just... Drop it on the file system and execute it. It's easy. So, um, different, different ecosystems have different application architectures, which empower different ways of solving the same problems. [00:12:48] But, you know, Rails as a, as a one man framework, or sorry, one person framework, It, it, I don't, I don't necessarily, that's a, that's sort of a catchy marketing slogan, but I just think of Rails as the most productive framework you can use. So you, as a single person, you can maximize what you ship and the, the, the value that you can create because Rails is so productive. [00:13:13] Jeremy: So it, seems like it's maybe the, the domain or the type of application you're making. Like you mentioned the command line application, because you want to be able to deliver it to your user easily. Just give them a binary, something like Go or perhaps Rust makes a lot more sense. and then I could see people saying that if you're doing something with machine learning, like the community behind Python, it's, they're just, they're all there. [00:13:41] So Room for more domains in Ruby [00:13:41] Mike: That was exactly the example I was going to use also. Yeah, if you're doing something with data or AI, Python is going to be a more, a more traditional, natural choice. that doesn't mean Ruby can't do it. That doesn't mean, you wouldn't be able to solve the problem with Ruby. And, and there's, that just also means that there's more space for someone who wants to come in and make an impact in the Ruby community. [00:14:03] Find a problem that Ruby's not really well suited to solving right now and build the tooling out there to, to try and solve it. You know, I, I saw a talk, from the fellow who makes the Glimmer gem, which is a native UI toolkit. Uh, a gem for building native UIs in Ruby, which Ruby traditionally can't do, but he's, he's done an amazing job at sort of surfacing APIs to build these, um, these native, uh, native applications, which I think is great. [00:14:32] It's awesome. It's, it's so invigorating to see Ruby in a new space like that. Um, I talked to someone else who's doing the Polars gem, which is focused on data processing. So it kind of takes, um, Python and Pandas and brings that to Ruby, which is, is awesome because if you're a Ruby developer, now you've got all these additional tools which can allow you to solve new sets of problems out there. [00:14:57] So that's, that's kind of what's exciting in the Ruby community right now is just bring it into new spaces. Faktory [00:15:03] Jeremy: In addition to Sidekiq, you have, uh, another product called Faktory, I believe. And so does that serve a, a similar purpose? Is that another job scheduling, job queueing system? [00:15:16] Mike: It is, yes. And it's, it's, it's similar in a way to Sidekiq. It looks similar. It's got similar concepts at the core of it. At the end of the day, Sidekiq is limited to Ruby. Because Sidekiq executes in a Ruby VM, it executes the jobs, and the jobs are, have to be written in Ruby because you're running in the Ruby VM. [00:15:38] Faktory was my attempt to bring, Sidekiq functionality to every other language. I wanted, I wanted Sidekiq for JavaScript. I wanted Sidekiq for Go. I wanted Sidekiq for Python because A, a lot of these other languages also could use a system, a background job system. And the problem though is that. [00:16:04] As a single man, I can't port Sidekiq to every other language. I don't know all the languages, right? So, Faktory kind of changes the architecture and, um, allows you to execute jobs in any language. it, it replaces Redis and provides a server where you just fetch jobs, and you can use it from it. [00:16:26] You can use that protocol from any language to, to build your own worker processes that execute jobs in whatever language you want. [00:16:35] Jeremy: When you say it replaces Redis, so it doesn't use Redis, um, internally, it has its own. [00:16:41] Mike: It does use Redis under the covers. Yeah, it starts Redis as a child process and, connects to it over a Unix socket. And so it's really stable. It's really fast. from the outside, the, the worker processes, they just talk to Faktory. They don't know anything about Redis at all. [00:16:59] Jeremy: I see. And for someone who, like we mentioned earlier in the Python community, for example, there is, um, Celery. For someone who is using a task scheduler like that, what's the incentive to switch or use something different? [00:17:17] Mike: Well, I, I always say if you're using something right now, I'm not going to try and convince you to switch necessarily. It's when you have pain that you want to switch and move away. Maybe you have Maybe there's capabilities in the newer system that you really need that the old system doesn't provide, but Celery is such a widely known system that I'm not necessarily going to try and convince people to move away from it, but if people are looking for a new system, one of the things that Celery does that Faktory does not do is Celery provides like data adapters for using store, lots of different storage systems, right? [00:17:55] Faktory doesn't do that. Faktory is more, has more of the Rails mantra of, you know, Omakase where we choose, I choose to use Redis and that's it. You don't, you don't have a choice for what to use because who cares, you know, at the end of the day, let Faktory deal with it. it's, it's not something that, You should even necessarily be concerned about it. [00:18:17] Just, just try Faktory out and see how it works for you. Um, so I, I try to take those operational concerns off the table and just have the user focus on, you know, usability, performance, and that sort of thing. but it is, it's, it's another background job system out there for people to try out and see if they like that. [00:18:36] And, and if they want to, um, if they know Celery and they want to use Celery, more power to Faktory them. Sidekiq (Ruby) or Faktory (Polyglot) [00:18:43] Jeremy: And Sidekiq and Faktory, they serve a very similar purpose. For someone who they have a new project, they haven't chosen a job. scheduling system, if they were using Ruby, would it ever make sense for them to use Faktory versus use Sidekiq? [00:19:05] Mike: Uh Faktory is excellent in a polyglot situation. So if you're using multiple languages, if you're creating jobs in Ruby, but you're executing them in Python, for instance, um, you know, if you've, I have people who are, Creating jobs in PHP and executing them in Python, for instance. That kind of polyglot scenario, Sidekiq can't do that at all. [00:19:31] So, Faktory is useful there. In terms of Ruby, Ruby is just another language to Faktory. So, there is a Ruby API for using Faktory, and you can create and execute Ruby jobs with Faktory. But, you'll find that in the Ruby community, Sidekiq is much widely... much more widely used and understood and known. So if you're just using Ruby, I think, I think Sidekiq is the right choice. [00:19:59] I wouldn't look at Faktory. But if you do need, find yourself needing that polyglot tool, then Faktory is there. Temporal [00:20:07] Jeremy: And this is maybe one, maybe one layer of abstraction higher, but there's a product called Temporal that has some of this job scheduling, but also this workflow component. I wonder if you've tried that out and how you think about that product? [00:20:25] Mike: I've heard of them. I don't know a lot about the product. I do have a workflow API, the Sidekiq batches, which allow you to fan out jobs and then, and then execute callbacks when all the jobs in that, in that batch are done. But I don't, provide sort of a, a high level. Graphical Workflow Editor or anything like that. [00:20:50] Those to me are more marketing tools that you use to sell the tool for six figures. And I don't think they're usable. And I don't think they're actually used day to day. I provide an API for developers to use. And developers don't like moving blocks of code around in a GUI. They want to write code. And, um, so yeah, temporal, I, like I said, I don't know much about them. [00:21:19] I also, are they a venture capital backed startup? [00:21:22] Jeremy: They are, is my understanding, [00:21:24] Mike: Yeah, that, uh, any, any sort of venture capital backed startup, um, who's building technical infrastructure. I, I would look long and hard at, I'm, I think open source is the right core to build on. Of course I sell commercial software, but. I'm bootstrapped. I'm profitable. [00:21:46] I'm going to be around forever. A VC backed startup, they tend to go bankrupt, because they either get big or they go out of business. So that would be my only comment is, is, be a little bit leery about relying on commercial venture capital based infrastructure for, for companies, uh, long term. Getting people to pay for Sidekiq [00:22:05] Jeremy: So I think that's a really interesting part about your business is that I think a lot of open source maintainers have a really big challenge figuring out how to make it as a living. The, there are so many projects that they all have a very permissive license and you can use them freely one example I can think of is, I, I talked with, uh, David Kramer, who's the CTO at Sentry, and he, I don't think they use it anymore, but they, they were using Nginx, right? [00:22:39] And he's like, well, Nginx, they have a paid product, like Nginx. Plus that or something. I don't know what the name is, but he was like, but I'm not going to pay for it. Right. I'm just going to use the free one. Why would I, you know, pay for the, um, the paid thing? So I, I, I'm kind of curious from your perspective when you were coming up with Sidekiq both as an open source product, but also as a commercial one, how did you make that determination of like to make a product where it's going to be useful in its open source form? [00:23:15] I can still convince people to pay money for it. [00:23:19] Mike: Yeah, the, I was terrified, to be blunt, when I first started out. when I started the Sidekiq project, I knew it was going to take a lot of time. I knew if it was successful, I was going to be doing it for the next decade. Right? So I started in 2012, and here I am in 2023, over a decade, and I'm still doing it. [00:23:38] So my expectation was met in that regard. And I knew I was not going to be able to last that long. If I was making zero dollars, right? You just, you burn out. Nobody can last that long. Well, I guess there are a few exceptions to that rule, but yeah, money, I tend to think makes things a little more sustainable for sure. [00:23:58] Especially if you can turn it into a full time job solving and supporting a project that you, you love and, and is, is, you know, your, your, your baby, your child, so to speak, your software, uh, uh, creation that you've given to the world. but I was terrified. but one thing I did was at the time I was blogging a lot. [00:24:22] And so I was telling people about Sidekiq. I was telling people what was to come. I was talking about ideas and. The one thing that I blogged about was financial experiments. I said bluntly to the, to, to the Ruby community, I'm going to be experimenting with financial stability and sustainability with this project. [00:24:42] So not only did I create this open source project, but I was also publicly saying I I need to figure out how to make this work for the next decade. And so eventually that led to Sidekiq Pro. And I had to figure out how to build a closed source Ruby gem, which, uh, There's not a lot of, so I was kind of in the wild there. [00:25:11] But, you know, thankfully all the pieces came together and it was actually possible. I couldn't have done it if it wasn't possible. Like, we would not be talking if I couldn't make a private gem. So, um, but it happened to work out. Uh, and it allowed me to, to gate features behind a paywall effectively. And, and yeah, you're right. [00:25:33] It can be tough to make people pay for software. but I'm a developer who's selling to other developers, not, not just developers, open source developers, and they know that they have this financial problem, right? They know that there's this sustainability problem. And I was blunt in saying, this is my solution to my sustainability. [00:25:56] So, I charge what I think is a very fair price. It's only a thousand dollars a year to a hobbyist. That may seem like a lot of money to a business. It's a drop in the bucket. So it was easy for developers to say, Hey, listen, we want to buy this tool for a thousand bucks. It'll ensure our infrastructure is maintained for the next decade. [00:26:18] And it's, and it's. And it's relatively cheap. It's way less than, uh, you know, a salary or even a laptop. So, so that's, that's what I did. And, um, it's, it worked out great. People, people really understood. Even today, I talk to people and they say, we, we signed up for Sidekiq Pro to support you. So it's, it's, it's really, um, invigorating to hear people, uh, thank me and, and they're, they're actively happy that they're paying me and our customers. [00:26:49] Jeremy: it's sort of, uh, maybe a not super common story, right, in terms of what you went through. Because when I think of open core businesses, I think of companies like, uh, GitLab, which are venture funded, uh, very different scenario there. I wonder, like, in your case, so you started in 2012, and there were probably no venture backed competitors, right? [00:27:19] People saying that we're going to make this job scheduling system and some VC is going to give me five million dollars and build a team to work on this. It was probably at the time, maybe it was Rescue, which was... [00:27:35] Mike: There was a venture backed system called IronMQ, [00:27:40] Jeremy: Hmm. [00:27:41] Mike: And I'm not sure if they're still around or not, but they... They took, uh, one or more funding rounds. I'm not sure exactly, but they were VC backed. They were doing, background jobs, scheduled jobs, uh, you know, running container, running container jobs. They, they eventually, I think, wound up sort of settling on Docker containers. [00:28:06] They'll basically spin up a Docker container. And that container can do whatever it wants. It can execute for a second and then shut down, or it can run for, for however long, but they would, um, yeah, I, yeah, I'll, I'll stop there because I don't know the actual details of exactly their system, but I'm not sure if they're still around, but that's the only one that I remember offhand that was around, you know, years ago. [00:28:32] Yeah, it's, it's mostly, you know, low level open source infrastructure. And so, anytime you have funded startups, they're generally using that open source infrastructure to build their own SaaS. And so SaaS's are the vast majority of where you see sort of, uh, commercial software. [00:28:51] Jeremy: so I guess in that way it, it, it gave you this, this window or this area where you could come in and there wasn't, other than that iron, product, there wasn't this big money that you were fighting against. It was sort of, it was you telling people openly, I'm, I'm working on this thing. [00:29:11] I need to make money so that I can sustain it. And, if you, yeah. like the work I do, then, you know, basically support me. Right. And, and so I think that, I'm wondering how we can reproduce that more often because when you see new products, a lot of times it is VC backed, right? [00:29:35] Because people say, I need to work on this. I need to be paid. and I can't ask a team to do this. For nothing, right? So [00:29:44] Mike: Yeah. It's. It's a wicked problem. Uh, it's a really, really hard problem to solve if you take vc you there, that that really kind of means that you need to be making tens if not hundreds of millions of dollars in sales. If you are building a small or relatively small. You know, put small in quotes there because I don't really know what that means, but if you have a small open source project, you can't charge huge amounts for it, right? [00:30:18] I mean, Sidekiq is a, I would call a medium sized open source project, and I'm charging a thousand bucks for it. So if you're building, you know, I don't know, I don't even want to necessarily give example, but if you're building some open source project, and It's one of 300 libraries that people's applications will depend on. [00:30:40] You can't necessarily charge a thousand dollars for that library. depending on the size and the capabilities, maybe you can, maybe you can't. But there's going to be a long tail of open source projects that just, they can't, they can't charge much, if anything, for them. So, unfortunately, we have, you know, these You kind of have two pathways. [00:31:07] Venture capital, where you've got to sell a ton, or free. And I've kind of walked that fine line where I'm a small business, I can charge a small amount because I'm bootstrapped. And, and I don't need huge amounts of money, and I, and I have a project that is of the right size to where I can charge a decent amount of money. [00:31:32] That means that I can survive with 500 or a thousand customers. I don't need to have a hundred million dollars worth of customers. Because I, you know, when I started the business, one of the constraints I said is I don't want to hire anybody. I'm just going to be solo. And part of the, part of my ability to keep a low price and, and keep running sustainably, even with just You know, only a few hundred customers is because I'm solo. [00:32:03] I don't have the overhead of investors. I don't have the overhead of other employees. I don't have an office space. You know, my overhead is very small. So that is, um, you know, I just kind of have a unique business in that way, I guess you might say. Keeping the business solo [00:32:21] Jeremy: I think that's that's interesting about your business as well But the fact that you've kept it you've kept it solo which I would imagine in most businesses, they need support people. they need, developers outside of maybe just one. Um, there's all sorts of other, I don't think overhead is the right word, but you just need more people, right? [00:32:45] And, and what do you think it is about Sidekiq that's made it possible for it to just be a one person operation? [00:32:52] Mike: There's so much administrative overhead in a business. I explicitly create business policies so that I can run solo. you know, my support policy is officially you get one email ticket or issue per quarter. And, and anything more than that, I can bounce back and say, well, you're, you're requiring too much support. [00:33:23] In reality, I don't enforce that at all. And people email me all the time, but, but things like. Things like dealing with accounting and bookkeeping and taxes and legal stuff, licensing, all that is, yeah, a little bit of overhead, but I've kept it as minimal as I can. And part of that is I don't want to hire another employee because then that increases the administrative overhead that I have. [00:33:53] And Sidekiq is so tied to me and my knowledge that if I hire somebody, they're probably not going to know Ruby and threading and all the intricate technical detail necessary to build and maintain and support the system. And so really you'll kind of regress a little bit. We won't be able to give as good support because I'm busy helping that other employee. Being selective about customers [00:34:23] Mike: So, yeah, it's, it's a tightrope act where you've got to really figure out how can I scale myself as far as possible without overwhelming myself. The, the overwhelming thing that I have that I've never been able to solve. It's just dealing with billing inquiries, customers, companies, emailing me saying, how do we buy this thing? [00:34:46] Can I get an invoice? Every company out there, it seems wants an invoice. And the problem with invoicing is it takes a lot more. manual labor and administrative overhead to issue that invoice to collect payment on the invoice. So that's one of the reasons why I have a very strict policy about credit card only for, for the vast majority of my customers. [00:35:11] And I demand that companies pay a lot more. You have to have a pretty big enterprise license if you want an invoice. And if the company, if the company comes back and complains and says, well, you know, that's ridiculous. We don't, we don't want to pay that much. We don't need it that much. Uh, you know, I, I say, okay, well then you have two, two things, two, uh, two things. [00:35:36] You can either pay with a credit card or you can not use Sidekiq. Like, that's, that's it. I'm, I don't need your money. I don't want the administrative overhead of dealing with your accounting department. I just want to support my, my customers and build my software. And, and so, yeah, I don't want to turn into a billing clerk. [00:35:55] So sometimes, sometimes the, the, the best thing in business that you can do is just say no. [00:36:01] Jeremy: That's very interesting because I think being a solo... Person is what probably makes that possible, right? Because if you had the additional staff, then you might say like, Well, I need to pay my staff, so we should be getting, you know, as much business as [00:36:19] Mike: Yeah. Chasing every customer you can, right. But yeah. [00:36:22] Every customer is different. I mean, I have some customers that just, they never contact me. They pay their bill really fast or right on time. And they're paying me, you know, five figures, 20, a year. And they just, it's a, God bless them because those are, are the. [00:36:40] Best customers to have and the worst customers are the ones who are paying 99 bucks a month and everything that they don't understand or whatever is a complaint. So sometimes, sometimes you, you want to, vet your customers from that perspective and say, which one of these customers are going to be good? [00:36:58] Which ones are going to be problematic? [00:37:01] Jeremy: And you're only only person... And I'm not sure how many customers you have, but [00:37:08] Mike: I have 2000 [00:37:09] Jeremy: 2000 customers. [00:37:10] Okay. [00:37:11] Mike: Yeah. [00:37:11] Jeremy: And has that been relatively stable or has there been growth [00:37:16] Mike: It's been relatively stable the last couple of years. Ruby has, has sort of plateaued. Um, it's, you don't see a lot of growth. I'm getting probably, um, 15, 20 percent growth maybe. Uh, so I'm not growing like a weed, like, you know, venture capital would want to see, but steady incremental growth is, is, uh, wonderful, especially since I do very little. [00:37:42] Sales and marketing. you know, I come to RubyConf I, I I tweet out, you know, or I, I toot out funny Mastodon Toots occasionally and, and, um, and, and put out new releases of the software. And, and that's, that's essentially my, my marketing. My marketing is just staying in front of developers and, and, and being a presence in the Ruby community. [00:38:06] But yeah, it, it's, uh. I, I, I see not a, not a huge amount of churn, but I see enough sales to, to, to stay up and keep my head above water and to keep growing, um, slowly but surely. Support needs haven't grown [00:38:20] Jeremy: And as you've had that steady growth, has the support burden not grown with it? [00:38:27] Mike: Not as much because once customers are on Sidekiq and they've got it working, then by and large, you don't hear from them all that much. There's always GitHub issues, you know, customers open GitHub issues. I love that. but yeah, by and large, the community finds bugs. and opens up issues. And so things remain relatively stable. [00:38:51] I don't get a lot of the complete newbie who has no idea what they're doing and wants me to, to tell them how to use Sidekiq that I just don't see much of that at all. Um, I have seen it before, but in that case, generally, I, I, I politely tell that person that, listen, I'm not here to educate you on the product. [00:39:14] It's there's documentation in the wiki. Uh, and there's tons of, of more Ruby, generic Ruby, uh, educational material out there. That's just not, not what I do. So, so yeah, by and large, the support burden is, is not too bad because once people are, are up and running, it's stable and, and they don't, they don't need to contact me. [00:39:36] Jeremy: I wonder too, if that's perhaps a function of the price, because if you're a. new developer or someone who's not too familiar with how to do job processing or what they want to do when you, there is the open source product, of course. but then the next step up, I believe is about a hundred dollars a month. [00:39:58] And if you're somebody who is kind of just getting started and learning how things work, you're probably not going to pay that, is my guess. And so you'll never hear from them. [00:40:11] Mike: Right, yeah, that's a good point too, is the open source version, which is what people inevitably are going to use and integrate into their app at first. Because it's open source, you're not going to email me directly, um, and when people do email me directly, Sidekiq support questions, I do, I reply literally, I'm sorry I don't respond to private email, unless you're a customer. [00:40:35] Please open a GitHub issue and, um, that I try to educate both my open source users and my commercial customers to try and stay in GitHub issues because private email is a silo, right? Private email doesn't help anybody else but them. If I can get people to go into GitHub issues, then that's a public record. [00:40:58] that people can search. Because if one person has that problem, there's probably a dozen other people that have that same problem. And then that other, those other 11 people can search and find the solution to their problem at four in the morning when I'm asleep. Right? So that's, that's what I'm trying to do is, is keep, uh, keep everything out in the open so that people can self service as much as possible. Sidekiq open source [00:41:24] Jeremy: And on the open source side, are you still primarily the main contributor? Or do you have other people that are [00:41:35] Mike: I mean, I'd say I do 90 percent of the work, which is why I don't feel guilty about keeping 100 percent of the money. A lot of open source projects, when they look for financial sustainability, they also look for how can we split this money amongst the team. And that's, that's a completely different topic that I've. [00:41:55] is another reason why I've stayed solo is if I hire an employee and I pay them 200, 000 a year as a developer, I'm meanwhile keeping all the rest of the profits of the company. And so that almost seems a little bit unfair. because we're both still working 40 hours a week, right? Why am I the one making the vast majority of the, of the profit and the money? [00:42:19] Um, so, uh, I've always, uh, that's another reason why I've stayed solo, but, but yeah, having a team of people working on something, I do get, regular commits, regular pull requests from people, fixing a bug that they found or just making a tweak that. that they saw, that they thought they could improve. [00:42:42] A little more rarely I get a significant improvement or feature, as a pull request. but Sidekiq is so stable these days that it really doesn't need a team of people maintaining it. The volume of changes necessary, I can easily keep up with that. So, I'm still doing 90 95 percent of the work. Are there other Sidekiq-like opportunities out there? [00:43:07] Jeremy: Yeah, so I think Sidekiq has sort of a unique positioning where it's the code base itself is small enough where you can maintain it yourself and you have some help, but primarily you're the main maintainer. And then you have enough customers who are willing to, to pay for the benefit it gives them on top of what the open source product provides. [00:43:36] cause it's, it's, you were talking about how. Every project people work on, they have, they could have hundreds of dependencies, right? And to ask somebody to, to pay for each of them is, is probably not ever going to happen. And so it's interesting to think about how you have things like, say, you know, OpenSSL, you know, it's a library that a whole bunch of people rely on, but nobody is going to pay a monthly fee to use it. [00:44:06] You have things like, uh, recently there was HashiCorp with Terraform, right? They, they decided to change their license because they, they wanted to get, you know, some of that value back, some of the money back, and the community basically revolted. Right? And did a fork. And so I'm kind of curious, like, yeah, where people can find these sweet spots like, like Sidekiq, where they can find this space where it's just small enough where you can work on it on your own and still get people to pay for it. [00:44:43] It's, I'm trying to picture, like, where are the spaces? Open source as a public utility [00:44:48] Mike: We need to look at other forms of financing beyond pure capitalism. If this is truly public infrastructure that needs to be maintained for the long term, then why are we, why is it that we depend on capitalism to do that? Our roads, our water, our sewer, those are not Capitalist, right? Those are utilities, that's public infrastructure that we maintain, that the government helps us maintain. [00:45:27] And in a sense, tech infrastructure is similar or could be thought of in a similar fashion. So things like Open Collective, things like, uh, there's a, there's a organization in Europe called NLNet, I think, out of the Netherlands. And they do a lot of grants to various open source projects to help them improve the state of digital infrastructure. [00:45:57] They support, for instance, Mastodon as a open source project that doesn't have any sort of corporate backing. They see that as necessary social media infrastructure, uh, for the long term. And, and I, and I think that's wonderful. I like to see those new directions being explored where you don't have to turn everything into a product, right? [00:46:27] And, and try and market and sale, um, and, and run ads and, and do all this stuff. If you can just make the case that, hey, this is, this is useful public infrastructure that so many different, um, Technical, uh, you know, applications and businesses could rely on, much like FedEx and DHL use our roads to the benefit of their own, their own corporate profits. [00:46:53] Um, why, why, why shouldn't we think of tech infrastructure sort of in a similar way? So, yeah, I would like to see us explore more. in that direction. I understand that in America that may not happen for quite a while because we are very, capitalist focused, but it's encouraging to see, um, places like Europe, uh, a little more open to, to trialing things like, cooperatives and, and grants and large long term grants to, to projects to see if they can, uh, provide sustainability in, in, you know, in a new way. [00:47:29] Jeremy: Yeah, that's a good point because I think right now, a lot of the open source infrastructure that we all rely on, either it's being paid for by large companies and at the whim of those large companies, if Google decides we don't want to pay for you to work on this project anymore, where does the money come from? [00:47:53] Right? And on the other hand, there's the thousands, tens of thousands of people who are doing it. just for free out of the, you know, the goodness of their, their heart. And that's where a lot of the burnout comes from. Right. So I think what you're saying is that perhaps a lot of these pieces that we all rely on, that our, our governments, you know, here in the United States, but also around the world should perhaps recognize as this is, like you said, this is infrastructure, and we should be. [00:48:29] Paying these people to keep the equivalent of the roads and, and, uh, all that working. [00:48:37] Mike: Yeah, I mean, I'm not, I'm not claiming that it's a perfect analogy. There's, there's, there's lots of questions that are unanswered in that, right? How do you, how do you ensure that a project is well maintained? What does that even look like? What does that mean? you know, you can look at a road and say, is it full of potholes or is it smooth as glass, right? [00:48:59] It's just perfectly obvious, but to a, to a digital project, it's, it's not as clear. So, yeah, but, but, but exploring those new ways because turning everybody into a businessman so that they can, they can keep their project going, it, it, it itself is not sustainable, right? so yeah, and that's why everything turns into a SaaS because a SaaS is easy to control. [00:49:24] It's easy to gatekeep behind a paywall and it's easy to charge for, whereas a library on GitHub. Yeah. You know, what do you do there? You know, obviously GitHub has sponsors, the sponsors feature. You've got Patreon, you've got Open Collective, you've got Tidelift. There's, there's other, you know, experiments that have been run, but nothing has risen to the top yet. [00:49:47] and it's still, it's still a bit of a grind. but yeah, we'll see, we'll see what happens, but hopefully people will keep experimenting and, and maybe, maybe governments will start. Thinking in the direction of, you know, what does it mean to have a budget for digital infrastructure maintenance? [00:50:04] Jeremy: Yeah, it's interesting because we, we started thinking about like, okay, where can we find spaces for other Sidekiqs? But it sounds like maybe, maybe that's just not realistic, right? Like maybe we need more of a... Yeah, a rethinking of, I guess the, the structure of how people get funded. Yeah. [00:50:23] Mike: Yeah, sometimes the best way to solve a problem is to think at a higher level. You know, we, the, the sustainability problem in American Silicon Valley based open source developers is naturally going to tend toward venture capital and, and capitalism. And I, you know, I think, I think that's, uh, extremely problematic on a, on a lot of different, in a lot of different ways. [00:50:47] And, and so sometimes you need to step back and say, well, maybe we're, maybe we just don't have the right tool set to solve this problem. But, you know, I, I. More than that, I'm not going to speculate on because it is a wicked problem to solve. [00:51:04] Jeremy: Is there anything else you wanted to, to mention or thought we should have talked about? [00:51:08] Mike: No, I, I, I loved the talk, of sustainability and, and open source. And I, it's, it's a, it's a topic really dear to my heart, obviously. So I, I am happy to talk about it at length with anybody, anytime. So thank you for having me. [00:51:25] Jeremy: All right. Thank you very much, Mike.

saas.unbound is a podcast for and about founders who are working on scaling inspiring products that people love brought to you by https://saas.group/ . I'm your host Anna Nadeina, Head of Growth for saas.group. In this episode #2a we talk with Mike Perham, founder and CEO at Sidekiq (https://sidekiq.org/), a full-featured background processing framework for Ruby. Mike unveils the concept behind Sidekiq explaining how this tool allows developers to build and scale Ruby applications. He explains how Sidekiq works by creating jobs to process data and distribute them across many machines. Listen as Mike discusses the synergy between the open-source project and the commercial aspect of Sidekiq, contributing to its ongoing development and success. Get a look into the process of building dependable software and how to monetize an open-source project as Mike shares his journey. Discover the benefits of an open-core model and learn about Mike's strategies for building the project, finding contributors, and pricing his product.Subscribe to our channel to be the first to see the interviews that we publish twice a week - https://www.youtube.com/@saas-group?sub_confirmation=1 Stay up to date: LinkedIn - https://www.linkedin.com/company/14790796 Twitter - https://twitter.com/SaaS_group Website - https://saas.group/

In episode 661, Rob Walling chats with Mike Perham, the founder of Sidekiq, who is a solo founder doing millions in revenue as a one-person business.If this isn't unique enough, Sidekiq originally started as an open-source project before he later monetized it by selling features that aren't available in the core product. You'll also hear how it took him ten years to become "an overnight success" because of all the things Mike tried before launching Sidekiq. Episode Sponsor: Find your perfect developer or a team at Lemon.io/startupsThe competition for incredible engineers and developers has never been more fierce. Lemon.io helps you cut through the noise and find great talent through its network of engineers in Europe and Latin America.They take care of the vetting, interviewing, and testing of candidates to make sure that you are working with someone who can hit the ground running.When it comes to hiring, the time it takes to write your job description, list...Read more... »Click the icon below to listen.  

Guest Naytri Sramek Panelists Richard Littauer | Justin Dorfman Show Notes Hello and welcome to Sustain! The podcast where we talk about sustaining open source for the long haul. Today, we're super excited to have joining us as our guest, Naytri Sramek, who's the Senior Director of Strategy at GitHub. Have you heard of the GitHub Accelerator and M12 GitHub Fund? Well, this is a great day to be joining us because Naytri is here to talk about these programs that they've been launching to help support and sustain OSS over the long haul. Naytri shares GitHub's journey which began with the GitHub Sponsors launch in 2019, bringing on enterprise sponsors, and how it led into launching the GitHub Accelerator program and the M12 GitHub Fund. Go ahead and download this episode now to learn more. [00:01:23] Naytri reveals the two things they've been launching which are the GitHub Accelerator and the M12 GitHub Fund. She also tells us about bringing on enterprise sponsors since they've benefited from open source. [00:06:25] Peter Thomas, who worked at Intuit and is creator of Karate Labs, is brought up and Justin wonders if he's involved in this venture or if there are others. [00:09:37] A question comes up regarding if the growth of the projects has been tracked with the money that GitHub has given to developers, if they've been able to quit their jobs since the money was given to them, and if those projects have improved. [00:15:35] We hear the focus of the GitHub sponsors, the Accelerator, and the M12 Fund. [00:19:57] Justin brings up the difficult issue of how to deal with developers that build these critical pieces of software, but they don't want to deal with the responsibility and wonders how Naytri and her team deal with this issue. [00:23:18] There's a 10-week course for the accelerator program and we hear how it works, and if it will be available to everyone in the future on GitHub. [00:29:28] Naytri explains how the communities are being funded. [00:32:47] A point is brought up about how long can these strategies and programs live on so maintainers and open source developers can make a good living, and Naytri goes in depth about the need for more sources of funding and funding models. [00:36:34] Find out where you can learn more about the GitHub Accelerator and the M12 Fund. Quotes [00:17:40] “The M12 GitHub Fund is all about how we do invest in the tools that are built on GitHub's platform.” [00:24:33] “I want 20 people making $200,000 a year.” [00:24:58] “The GitHub Accelerator course itself will be open source.” [00:28:08] “As we've expanded the program into more countries, we've doubled the number of countries that sponsors cover right now.” [00:30:10] “Commits aren't universal. You shouldn't just be rewarded for the code.” [00:33:07] “The way we're thinking about the accelerator and the fund is we need so many more sources of funding and funding models to be able to support open source creators as well as communities.” Spotlight [00:37:44] Justin's spotlight is Jessica Lord, who's the GitHub Sponsors Product Lead. [00:38:14] Richard's spotlight is Bill Watterson, author of Calvin and Hobbes. [00:38:23] Naytri's spotlight is Mike Perham and a 10 year anniversary post he wrote to Sidekiq. Links SustainOSS (https://sustainoss.org/) SustainOSS Twitter (https://twitter.com/SustainOSS?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Eauthor) SustainOSS Discourse (https://discourse.sustainoss.org/) podcast@sustainoss.org (mailto:podcast@sustainoss.org) Richard Littauer Twitter (https://twitter.com/richlitt?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Eauthor) Justin Dorfman Twitter (https://twitter.com/jdorfman?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Eauthor) Naytri Sramek LinkedIn (https://www.linkedin.com/in/naytri-sramek-a6872516?trk=people-guest_people_search-card) naytri@github.com (mailto:naytri@github.com) fund@github.com (mailto:fund@github.com) GitHub Accelerator (https://accelerator.github.com/) GitHub Blog- An open source economy-built by developers, for developers by Naytri Sramek (https://github.blog/2022-11-09-an-open-source-economy-built-by-developers-for-developers/) Sustain Podcast-Episode 56: Dominic Tarr on Coding What You Want, Living on A Boat, and the Early Days of Node.js (https://podcast.sustainoss.org/56) Karate Labs (https://www.karatelabs.io/) Hopscotch (https://www.gethopscotch.com/) Justin Dorfman Tweet: The hard decisions popular open source project maintainers need to make…daily. (https://twitter.com/jdorfman/status/1597991465673596935) Jessica Lord-GitHub (https://github.com/jlord) Bill Watterson-Wikipedia (https://en.wikipedia.org/wiki/Bill_Watterson) Happy 10th Birthday, Sidekiq! -by Mike Perham (https://www.mikeperham.com/2022/01/17/happy-10th-birthday-sidekiq/) Credits Produced by Richard Littauer (https://www.burntfen.com/) Edited by Paul M. Bahr at Peachtree Sound (https://www.peachtreesound.com/) Show notes by DeAnn Bahr Peachtree Sound (https://www.peachtreesound.com/) Special Guest: Naytri Sramek.

[00:01:22] Andrew tells us about a blog post he put out about his Twitter profile image and some stuff he did with his GitHub actions repo.[00:05:29] The Tweet sharing begins. Tweet #1 topic is someone who's learning Rails, has no idea how to read the documentation or where to go, and getting lost easily. [00:13:55] Tweet #2 topic is about Matestack, and Jason brings up a previous episode they did where they talked about it with Jonas Jabari.[00:14:47] Tweet #3 topic is about what first timers at RailsConf should know or do by the time this episode airs.  [00:19:02] Andrew tells us about people giving massages in the exhibition hall and he's going to be devasted if they don't do them this year. ☺[00:19:36] The guys share some conference advice for first timers going to RailsConf, such as meeting new friends, taking notes, and talking to people who are speakers. [00:24:54] Andrew brings up how to choose between talks and workshops, and Jason encourages everyone to go to the podcast panel recording since they will all be there.[00:26:44] Jason brings back the meeting people topic and mentions some dinner ideas, and Chris mentions meeting people at the evening events they have.[00:28:31] Go to Mike Perham's website where he has all the events listed going on during the week. [00:29:48] Andrew talks about seeing your Ruby Heroes at RailsConf and not to be intimidated.[00:35:34] Tweet #4 is “Bet!”Panelists:Jason CharnesChris OliverAndrew MasonSponsor:HoneybadgerLinks:Ruby Radar NewsletterRuby Radar TwitterGitHub actions Tweet #1Remote Ruby Podcast-Episode 103: Reimagined Rails views using Matestack with Jonas JabariTweet #2Tweet #3RailsConf 2022 Events

Mike Perham, the creator of Sidekiq

Chris is making hiring progress and loves asdf and M1 laptops. Steph is anticipating the arrival of one dongle to rule them all and talks about moving away from having a lot of Bluetooth connections. Two other big things on Steph's mind are education around factories because they're v important and shared examples and how they can be overused. She and Chris agree that it is better to tell stories in tests instead. This episode is brought to you by ScoutAPM (https://scoutapm.com/bikeshed). Give Scout a try for free today and Scout will donate $5 to the open source project of your choice when you deploy. Services down? New Relic (https://newrelic.com/bikeshed) offers full stack visibility with 16 different monitoring products in a single platform. GitHub - asdf-vm/asdf: Extendable version manager with support for Ruby, Node.js, Elixir, Erlang & more (https://github.com/asdf-vm/asdf) Factories Should be the Bare Minimum (https://thoughtbot.com/blog/factories-should-be-the-bare-minimum) Mystery Guest (https://thoughtbot.com/blog/mystery-guest) GitHub - varvet/pundit: Minimal authorization through OO design and pure Ruby classes (https://github.com/varvet/pundit) Become a Sponsor (https://thoughtbot.com/sponsorship) of The Bike Shed! Transcript: STEPH: Hello and welcome to another episode of The Bike Shed. [laughs] CHRIS: Hello, and I'm singing, and I love singing. STEPH: It's Buddy the Elf; what's your favorite color? [laughter] For reals, here we go. Hello and welcome to another episode of The Bike Shed, a weekly podcast from your friends at thoughtbot about developing great software. I'm Steph Viccari. CHRIS: And I'm Chris Toomey. STEPH: And together, we're here to share a bit of what we've learned along the way. So hey, Chris. What's new in your world? CHRIS: My world continues to be focused on hiring as a pretty core aspect of things. We have happily had one offer extended and accepted, so that's great. We've got a person who will be joining the team in a couple of weeks. That's very exciting. And we're continuing in conversations with some other folks. So I look forward to the place where I can be on the other side of this and have that team and be growing the team and not having to focus because hiring takes a lot of effort. It is something that I believe should be done as well as possible and intentionally as possible and then just outreach and all that. So yeah, I'll be fine with being on the other side of that. But it's going well, so that is nice. STEPH: That's awesome that you're making progress. Once you have hired your team, will you then add to the agenda to hire someone to help with hiring? CHRIS: I don't actually know if the organization, if the whole company has someone who's focused on hiring. I think that can make sense. Working through recruiters and things like that is something that I've seen in the past. I've seen it work for certain organizations. I've also been on the receiving end of plenty of obviously copy and pasted very generic "Hey, person, I saw that you do lots of Java and other enterprise code software. Would you like to come work with us?" I'm like, none of those are true, and I do not want to go work with you. But thanks, I still appreciate the outreach. [laughs] So I am intrigued to see how we think about it. More generally, this is something that you and I have talked about offline but the idea that you kind of always want to be hiring. We do have specific roles that we've identified that the budget has space for. But more generally, ideally, we're going to need to hire more people down the road, and that will happen at a particular point. But having those conversations, starting to talk to people, now planting the idea of like, hey, you're great, and I would love to work with you someday and just keeping those lines of communication open. Networking is perhaps what the people call it. I don't know; I've never felt super comfortable with that word, but I think it's that and being friendly and staying connected with people whose work I respect and would love to work with more. So that's part of what I will come out of this with is yeah, let's always be hiring in a certain sense. STEPH: I'm glad you expanded on it because I was just thinking I have specific ideas as to what always be hiring means to me and what those activities would include. So I was curious what it means to you. And I agree, I think it's a lot of networking. It's a lot of taking chats and social chats with folks and just talking about the company and finding out where they're at. And then one day, if it works out that then they want to make a shift, then you've already got that relationship that started, and they're already potentially interested in your team. I guess some of the other big stuff that comes to mind, too, is like thoughtbot we have the blog. I feel like that's always really helpful too. Like when you help somebody, when you publish information that then helps them in their career, I feel like that will then draw people towards you as well. CHRIS: Yeah, the thoughtbot blog and basically everything that thoughtbot does, the podcast here, or Upcase, or all those things were so incredibly helpful in the hiring. But I also know they're hard to spin up, is what I would say. The thoughtbot blog has I don't even know how many hundreds of thousands of hours maybe. It's weird to try and put a number to it. But I've written a handful of posts for it, and I'm not great at writing them. They take me way longer than they should, but they took many hours. And then I had wonderful peer review by other developers at thoughtbot. And so, the amount of effort that goes into the thoughtbot blog absolutely produces wonderful benefits. But it's not free by any means, and similarly, the podcasts or Upcase or any of those sort of things. Similarly, the one that's actually most interesting that I see a lot of organizations go for initially and then often walk back is open source. Like, oh, we have this internal library that we built to do something. What we'll do is we'll just package it up and share it with the world, and then it'll be great. And the maintenance burden and support necessity of an open-source project is so high. I've actually historically gotten into the mode of suggesting...when I was working with clients, they would start to mention this and be like, "Oh yeah, we think we'll open source this thing, and it'll be great." I'm like, "Are you sure, though? Do you definitely want to?" There's definitely a difference between open sourcing and just putting an idea out there is one thing that I would say. Can you just write a blog post that has code snippets but not reusable code that you have to maintain that people, unfortunately, I think unfairly expect responsiveness and maintenance over time? And what if you stopped using that technology? What if you stop using this thing, but your name is still attached to it? And people have expectations of what that looks like. Or people come in and say, "Hey, this is great, but I want to change it in this way." And you're like, "Yeah, but that actually doesn't work for us. That's not how we use it. But we would be on the hook to maintain that code if we accept your pull request." And so, as wonderful as open source is, I tend to be on the more conservative end of the spectrum of like, are you definitely sure you want to open source this? Is there another way that you can share this with the world? Can it be a conference talk, or a blog post, or something like that? But it is an interesting one. STEPH: Yeah, I've been a part of several teams that have started with that; let's start an engineering blog. And their hearts are totally in the right place, and I understand why they want to do it. But like you just said, there's a cost to that. And if you don't have something like thoughtbot has like an investment day or a time for engineers to then be able to contribute to that blog, then either they're just not because they're not going to have their downtime to be able to do that. And it is hard to write and publish and be happy with what you're going to publish with the world. I really like what you're talking about in terms of the maintenance burden because I can't remember if it was an Upcase conversation or if there was something...but I was early on at thoughtbot and had a similar thought of why can't we just open source it? Why can't we make it public? And there was a very big thoughtful discussion around well, we have to have all these considerations in place. Who's going to maintain it? Just like FactoryBot is a really big internal project at thoughtbot. And there's typically a rotation of folks who will then take ownership and then onboard other people who are interested in it and curate the issues. And it's very important work, but you have to allocate time for it. All of that to say, I totally agree. There's a big burden that goes with it. CHRIS: Yeah, it's interesting that this has been an evolving thought in my head, and it makes me sad is another thing I'll say about it. I wish it were easier to just put code out there in the world and have the expectations properly calibrated for like, hey, I did this thing. Here's a code sample. It worked for me. Actually, I found dropping something in a Gist...a Gist just has a point-in-time connotation to it that I like. Like, if I see a code sample in a Gist, I'm like, I have no expectation that that person is going to do anything or respond to anything I have to say. But this is great because I now have this sample code that helps me get a little bit further. And I may have to vendor that code or take it on myself, and I now own it. It's not this person's responsibility. But the minute you have a repo with a README that says stuff and like, here are the installation instructions, the expectations just flip in a way that I don't think is...at least I become cautious around. And that does make me sad, though. STEPH: Yeah, it feels like you went from offering an example to I'm offering a product. And so then as soon as people feel like, oh, you're giving me something as a product that you maintain, then I'm going to have higher expectations of it should work how I expected it to work. I'm going to ask questions. And yeah, you make a lot of good points. CHRIS: Would you like to pay me $0 for me to build software for you? That sounds fun. STEPH: [laughs] CHRIS: And open source is such a wonderful thing. And so I'm interested in...like, I follow a lot of folks who are in the open-source world and have found ways to make it make sense financially or otherwise or organizationally. Open Collective and things like that is one option or OpenCore and then paid pro models and things like that like Sidekiq as an example. Sidekiq just celebrated ten years with some wild numbers in terms of the revenue, and it's like, yeah, that's fantastic. This is a cornerstone piece of software in the Ruby and Rails community. And also, Mike Perham had a great outcome from it. I think that's a win. So maybe blogging, maybe, but not sure. Probably not open source is my suggestion, at least for me. But one thing that I am interested in that hasn't been an option in my mind for a long time, but I'd love to get back to is conferences and going there, especially with a small team from an organization. The three developers we go, and we hang out at a conference and the company has a space there. And there's room to have conversations and meet people. That is one that I would love to continue in a way of making sure that our name is in people's minds as a place that they could work in this world. It is interesting, though, that it gets scoped a little bit like we are definitely a Rails shop. But that's not all that we are, or that's not the complete totality of our technical identity, so it becomes interesting. But I think it's probably the most representative. And I definitely see the Ruby and Rails community is having a good product-centric mindset that is definitely the sort of thing that I want in the teams that I'm building. STEPH: Yeah, I think that's an awesome idea because it's a way that you could focus on creating content. It'll likely have a big impact. But then you can also replay that content, but it's not the commitment of a blog or a commitment of open source. CHRIS: But yeah, so hiring has been, I would say, most of what I've been doing. One other thing that was fun this week, so I have my new laptop that I've had now for a couple of months, I'd say. And just this week, we had a very frustrating issue where Heroku stopped deploying our application. Just one day, it was like, nah, it doesn't work anymore. And I was like, well, that's less cool than I want it to be. And so one of the developers on the team dug into it, and it turned out Node-sass was the answer, which we're not even using is extra unpleasant. It's just part of Sprockets and Webpack or something like that. There's some downstream dependency sequence. We're using Tailwind and PostCSS. So we don't even need Node-sass. I think maybe PostCSS does. But anyway, turns out Heroku had switched to using version 16 of Node just without telling us. We were previously on 14, and then Node-sass didn't build on that. There was just this weird dependency chain that stopped working one day. And we weren't pinning the Node version within our application. So one of the developers figured this out, pinned us back to version 14 something of Node, and that was fine. But then my computer got confused because the versions were out of sync. Anyway, asdf is great. That's the first thing I'm going to say. So I use asdf to manage the versions of Ruby, and Node, and Yarn, and Elm, and basically everything else that I use. And I love that it's all under one hood, so asdf, wonderful. Also, my laptop, wonderful. I really love the M1 fancy laptop. But what was fun was I had to install the new Node version. And this was the first time in the three months I've had this computer that I've heard the fans come on. Finally, I asked it to do something hard enough that it was like, whoa, whoa, whoa, I'm going to need some backup here. And so the fans finally kicked in. So I don't know what's going on installing Node, but good for everyone involved, [laughter] impressive to make such aggressive use of all of the hardware in my computer. STEPH: Yeah, I love asdf. I miss it right now because I'm on my client machine, and we're not using asdf. Instead, we are using Chruby, C-H Ruby to manage Ruby versions. asdf is awesome. That's fun. It's the first time that the fans kicked on. I'm intrigued with my machine. I haven't really paid attention to it when the fans kick on except the one time where I had like a Ruby process that was running away, and I had to figure out what was going on there. Because then all the CPU was just being dedicated to Ruby even when I wasn't using Ruby. But since then, I haven't heard the fans. It's been very, very quiet. It's lovely. I like when it's quiet. CHRIS: Oh, it's been great. It was interesting because it was this weird noise that I'd forgotten about. STEPH: [laughs] CHRIS: My previous computer was so old that this was happening regularly whenever my backup process would run. Apparently, that is a very computationally intensive activity. So I would hear the fans kick in, immediately go find the backup process and say pause for 60 minutes or whatever it was. Just like, leave me alone. Stop it. The computer is getting too hot. You need to calm down. But now, with the new computer, there was nothing I could do to make it happen. And then finally it happened, and I was like, oh yeah, I guess this computer has fans. That's neat. But yeah, so things that are great, asdf and the M1 laptops. STEPH: Nice. Yeah, you're one of the few individuals I know that's using one of the M1 chip. So it's been reassuring to hear how well it's going because I did not opt in to that new-new. I opted in to the give me something stable and steady that I know so that way I don't have to fuss with it because I can then fuss with all the other things that I need to fuss about. CHRIS: So much fussing to do. STEPH: Lots of fussing. Fussing and cussing is what I do over here. CHRIS: [laughs] Mid-roll ad Hey, friends, let's take a quick break to hear from today's sponsor, New Relic. All right, so you've probably experienced this before where you're just starting to fall asleep, and it's a calm, code-free peaceful sleep, and then you're jolted awake by an emergency page. It's your night on call, and something is wrong. But I have some good news because you have New Relic, which means you can quickly run down the incident checklist and find that problem. So let's see, our real user monitoring metrics look good. And that's where New Relic measures the speed and performance of your end-users as they navigate the site. But it looks like there's an error in application performance monitoring. If we click on the error, we can find the deployment marker where it all began, roll back the change, and, ooh, problem is solved. We can go back to bed, back to sleep, and back to happy. That's the power of combining 16 different monitoring products into one platform. You can pinpoint issues down to the line of code so you know exactly why the problem happened and can resolve it quickly. That's why more than 14,000 other companies, including GitHub and Epic Games, use New Relic to improve their software. So you know that next late-night call is just waiting to happen, so get New Relic before it does. And you can get access to the whole New Relic platform and 100 gigabytes of data free forever. No credit card required. Sign up at newrelic.com/bikeshed. That's New Relic N-E-W-R-E-L-I-C .com/bikeshed, newrelic.com/bikeshed. CHRIS: Well, speaking of, what have you been fussing and cussing about this week, Steph? STEPH: So this is more in the pranting area, which is our portmanteau for praise and rant, where I'm super excited. I have a delivery coming from Amazon today. So I'm that person that keeps checking and waiting for it to show up. But I'm finally going to have one dongle to rule them all. I have a very messy approach right now [laughs] where I have all the dongles and have to plug everything in. And you know what? Normally it's fine. It's fine because I do it once, and I don't have to mess with it that much. But because I now have my thoughtbot laptop and I have a client laptop, and I needed to be able to switch back and forth, it is just too much. And I was realizing how many dongles I'm having to use. So I have one dongle to rule them all. It's showing up today. It's a very exciting day. CHRIS: I'm very excited for you. I recently made a similar switch when I got this new laptop. I was like, you know what? I'm going to look into it because power can come over USB-C and whatnot. And I was like, all right, it's finally time. I want to be able to just click in. And it's one of those things that feels trivial, or at least in my mind, I'm like, this doesn't feel like it'll make that big of a difference. But it makes it so much easier to disconnect my laptop, go somewhere else, and then come back. And I noticed myself doing that more, which I think is a positive thing. Otherwise, I'm just anchored to my desk. I'm like, I don't want to unplug everything and then have to replug it. That's like a whole thing. But now that it's not, I am more mobile, more flexible in where I'm working from, and I found benefits from that. So I'm a fan. I'm very happy that this is going to show up for you [laughs] and really change the way you're working. STEPH: Well, I've started moving away from a lot of Bluetooth connections as well because my keyboard will support Bluetooth, my headphones support Bluetooth. And I liked the idea of being wireless. But then, especially from switching laptops back and forth and then having to reconnect and all of it, it was just too tedious to go back and forth. And yeah, I'm with you where I didn't want to have to leave my desk and unplug everything and then bring it back where I'm playing, you know, like the game Operation where you had to reach in and then you had to grab different little bones? If you don't know the game Operation, that sounds really weird. But it felt like a game of Operation where then I was having to find all the dongles and connect them and plug them all in. And yeah, so it's going to be wonderful. CHRIS: Even knowing the game Operation, that still sounds kind of weird. STEPH: [laughs] CHRIS: But I really love that there are people out there listening that are like, what are they talking about? STEPH: What weird childhood did you have? CHRIS: Yeah, I'm definitely Team Wired-Almost-Everything. The only thing that I have that's wireless is my headphones. And it only works kind of, and I have to trick them sometimes. And the worst thing is occasionally my computer will have control, whatever, they're connected. So I'm listening to music on my computer and then suddenly, my phone will just steal it. It's like, what are you doing? No. Or, randomly, my headphones will be sitting away from me, and they'll just connect. And I'll be in the middle of a call on something else. Like, I'm here talking to you, and suddenly my headphones are like, hey, we wanted to join the party. It's like no, absolutely not, [laughs] not at this moment, under no circumstances. So I don't really believe in Bluetooth as a technology. I'm very much a fan, particularly with things like keyboards and whatnot. Bluetooth I've yet to be convinced that it is a sound technology. STEPH: I have the headphones where they try to be very smart, and they are pretty smart where they will block out sound. But then, if I am talking, then it will put me in more of an auditory space where then I can more easily hear, and it won't filter out sound as aggressively. But I've noticed a problem. And it's when I'm watching anything that's funny that then I'm laughing. So every time I laugh, my headphones think I'm talking to someone, and then it will switch over to where it's trying to let me hear more sounds out in the universe. And then it kicks back on because it's like, okay, she's done talking. It's a very jarring experience. [laughs] And I haven't figured out how to turn that setting off. It's like, oh, I just can't watch funny stuff with my headphones right now, which is also problematic with pairing because I tend to laugh a lot with pairing. It's a thing. I'm working on it. The struggles of Shteph. CHRIS: Well, at a minimum, it sounds like your dongle life is going to be improving very soon, and that's exciting. STEPH: Dongle life, it'll be single dongle life. That's it. [singing] All the single dongles, all the single dongles. Put your adapters up. [laughter] On a different note, talking about some of the work that I've been doing this past week with Joël Quenneville on our client work, is that we have been looking for ways that we still want to build up CI time. We've talked about the fact that we're working on some of that horizontal scaling. And I don't have an update there. But the other update I have is where we want to be very strategic about where we invest our time because improving the test is not trivial work. A lot of the low-hanging fruit has already been done, so triaging a flaky test can be very difficult, and it can take us a while. So we just want to make sure and verify that before we invest a lot of time into a portion of the test that then we know what the outcome is going to be. Are we improving developers' lives by this much? And how do we measure that? Are we reducing the CI build time, and how do we know that? And one of the areas that I really wanted to focus on is FactoryBot because there are a lot of factories. The factories tend to do a ton. So they are calling out to the database and building a lot of associations. And that's something that the team knows about as well is that there are just so many SQL queries that get executed in tests. And it would be great if we could reduce the number of SQL queries that are going out. And FactoryBot includes some ActiveSupport notifications, which means you can subscribe to factories being run which then gives you access to details like which factories are being used? What build strategy is used? Are you calling build build_stubbed or create? And the factory's execution time. So then the idea of this is that if we can harness a lot of the data that we can collect from FactoryBot, then can we ask questions around what's our slowest factory? How long does it take, and how many places is it being called? Because then ideally, we can calculate to say, okay, if this factory takes this long and it's used in this number of places, then we can have a formula to figure out how many minutes of our test suite is spent just on executing that factory. And then if we can reduce the time of that factory, let's say by half, then we know how much time we're shaving off of our CI build. And then we have this more concrete verified okay; this is worth our investment. We want to pursue this, even if the factory may take us a full day to improve because it does so much. And it's just gnarly. So it's going to take some time to really refactor it into a more simplified state. So, in theory, this sounds really, really great, and it was a lot of thanks to Josh Clayton, who was the one that advocated saying that we could use the ActiveSupport notifications to find a lot of this data. And so Josh and I paired on this for a bit to look into can we answer some of those other questions as well? And we were testing it on a small side project that he had, which was great because the other codebase is very big, and feedback is just a lot slower. So we wanted to first prototype it and have a proof of concept in a very quick space and just to be able to look through the data and make sure the assumptions that we had and the value would be there. So we applied that first, and that was going really well. And then Joël Quenneville took that strategy and then applied it to all the specs in the spec models directory and ran it for the much larger client codebase and got some really great results. And we also used a low fidelity approach where we wanted to be able to see which factories were the most popular. So how often are they getting called? And the average execution time. So that way, we could then quickly look at this scatter plot, and then we could see, okay, who's in the far upper right quadrant? Because those are the factories that are causing the most pain. But we started looking into a graphing library and what are we going to pull in. And Josh had the great idea. He's like, "I wonder if Google Sheets has a scatter plot. Can we export this to CSV data and then copy it from the terminal and import it into Google Sheets?" And it turns out that you can. So then we grabbed it and put it in Google Sheets and then just converted it into a scatter plot, which was really nice because then we didn't have to incorporate any chart library or any graphics or anything. We could just plop it into Google Sheets and then easily share it. So we now have this list thanks to Joël because he ran it through the spec models directory of all the factories that are getting called. And it's really interesting. And there's one, in particular, that is high on the list. And it was actually one of the first ones that we worked with when we were troubleshooting a test that took us a while when we first joined the project. And the average time for this factory is four seconds, and it gets called over 500 times. It's like 527 times. So then if we multiply that, so if we say, all right, it takes about 4 seconds times 527 and then divide it for 60 for minutes, that's 35 minutes, 35 minutes for that factory. Now, granted, these are getting parallelized across different processes. But still, if you divide that up across four processes, that's a non-trivial amount of time. So I think this is going to be really helpful and really interesting data that we can then use to drive our decisions to say, okay, we want to take this factory and let's say even if we can cut it into half, let's say if we bring it from 4 seconds to 2 seconds, it'll go from 35 minutes to 17 and a half. CHRIS: Oh wow, I love the methodical approach. I love that actually having a number you're like; this is how much pain or the cost of this right now. And so we've identified that this is this high-level thing. I love the intentional starting with, like, let's measure it. Let's understand where the most bang for the buck is. In particular, the graph that you're describing reminds me of one I haven't actually worked with it much. But Code Climate has a graph that they use, which is it's churn versus complexity. So it's like, you may have a very complex piece of your code, but someone wrote it once, and it just sits in the corner. And you know what? It quietly does its job. And yes, it's very complex, but nobody needs to touch it. So it's not a big deal. And then you have stuff that changes constantly, but it's super simple, so that's fine too. Your UsersController is probably going to change a bunch; that's fine. But the stuff that is constantly changing and very complex that's the magic quadrant that, like, pay a lot of attention to that. And similarly, which are the ones that are being used a lot and take a while? That's the magic quadrant. I'm intrigued now. I want to search for more magic quadrants that deserve attention. But for now, that sounds like a lot of fun. So now, what's the approach that you're going to take? I imagine you need to alias that factory and have it exist because some tests will rely on certain details of it. This is my guess. So let me see if this is the way that you're thinking about it, alias the factory, so you have a representation that does all the stuff that the current one does. But then you have a new one that is much more pared down. And then, on a test by test basis, you start switching it over and trying to move things to the lower weight, the slimmer version of the factory. But I would think you would want to do a gradual process if there are 520-ish usages. Because otherwise, just changing that factory out from under all the tests, I imagine you'd break some tests if you just were like, what if it did less? STEPH: Yeah, I like that idea of the incremental approach. And that all sounds great, especially the alias because you're right; we want to change it incrementally and not all of them at once. But then essentially implement, one, because I want to see what does the pared-down factory look like? What is the basic factory that we can get away with? And then how long does it take for that factory to execute? Because then that will help confirm, can we really get it down to two seconds? Or is this just a factory that's always going to take three and a half seconds, and then it's not really that much of a payoff? Maybe we should look for a different factory to investigate. And then also understanding from the test are people reaching for this factory all the time because it builds up the world and all of these tests need the full world? Or are people just reaching for this because it does the one or two things that they need, and we can get away with a much slimmer factory? So right now, it's in the space of understanding why are people reaching for this? What are the tests they actually need? And yeah, how can we do it incrementally? At one point, we may even be able to try to programmatically switch it out. Maybe we just find 50 tests that are using this once we have the slimmed-down version and we replace...50 is probably too big. But if we replace X number of tests with this factory, how many of them fail? Maybe 10% of them passed. Cool, let's just take those 10% as a win and issue those as a PR. So that could be a strategy as well just to find if there's any that are super easy to change. All we had to do is literally change the name of the factory. The other big part that's on my mind is education around factories. I think a lot of people on the team understand that factories are very important. They can be very helpful. They can also be very cumbersome. But it feels like a good opportunity to say, "Hey, we are specifically working on these factories. Here's the reasoning that led us to work on these factories. When you're in the space of factories, please be mindful about what are you reaching for? Is there a slimmed-down factory that you can reach for? Maybe you can implement your own slimmed-down factory if one doesn't already exist." So I like the idea of coupling it with also just broader awareness because we are but two people. So I would love for more people to be part of the changes. CHRIS: Unsurprisingly, there are some wonderful blog posts on the thoughtbot blog that speak to this topic. One that I'm a fan of is Factories Should be the Bare Minimum. This was written by Matt Sumner. And it describes basically that idea of factories shouldn't build the worlds. They should give you the pieces that you can use to build the world but not build the world entirely. And so I'm a big believer in that, having your factories be as minimal as possible. They should be valid, but that's about it. And then I will often reach for extracted helper methods and keeping those as locally scoped as possible often in the spec file, or if not, maybe they're sharing spec support. But being intentional with where we reach for them and not having everyone use the same thing that just slowly gets added to. And it's like, do I actually need everything that's in there? The other thing that's interesting is the idea of having a factory that does a ton is, in my mind, sort of in direct contradiction to what I believe factories exist for which is when I think of factories, they're useful to fill in the rest of the details such that you don't have mystery guests in your test. But you can explicitly say build me a user who has an email that looks like this because, in this test, I care about the email, but I don't care about the rest of the details. I don't care about their name. I don't care about their password, or the roles, or any of the other details. Just let the factory deal with that because it's not important to the test. But I want to make sure that the relevant detail is present and specified within the spec. If you have a factory that builds everything in the world, that's like build a user and then grabs the first action from the project that that user has, because I know that they do because they use the big factory, that is just in direct contradiction to what we want factories to do. We want tests to tell their story. We want to avoid mystery guests. Factories are a great way to do that while still remaining concise. But if your factories just build the world, then there are some mystery guests in the world, I can assure you. STEPH: Yeah, I agree where factories have served as an abstraction for what I think is important to the test. But then there becomes this moment of where someone thinks, well, I need to build up these records, but I don't really need to reference them directly. I just have some coupled code that's going to rely on these. And so I don't explicitly need them, but they need to be there. So I'm going to extract it away, and a factory feels like a good place for me to extract that too. And I would take the very hard opposite approach where if you have coupled code and you have these dependencies that aren't necessarily explicitly used in the test, but they are required for the test, I'd rather see a painful test setup than have all of that extracted away from me. Because then if I do need to triage or troubleshoot that test, it's going to take a lot of just mental overhead to work through what do I actually need here and why? So I'd rather see that painful test set up then have it moved somewhere else. But I think a lot of people take the opposite approach of where if I abstracted away, my test looks prettier. And I'm like, yeah, but maybe to you in the moment, but it's going to cause me a lot of pain further down the road when I have to work with this. So show me all the crap that you had to do upfront. Just let me know. [laughs] I'd rather the test be honest with me. And then it's a really nice jumping-off point because you can see a test that does all of this. And instead of blaming the test and thinking it's the test's fault, you recognize this test has a lot of complicated setup, and it's probably because of the code and how the code was written. And we should look at refactoring the code, not at how can we make our tests look prettier? CHRIS: Unsurprisingly, I agree with 100% of that. Someday we'll find things other than Pop-Tarts and IPAs that we disagree on. But today is not that day. [laughs] Once again, today is not that day. Mid-roll Ad Hi, friends. And now a quick break to hear from today's sponsor, Scout APM. Scout APM is an application performance monitoring tool that's designed to help developers find and fix performance issues quickly. With an intuitive interface, Scout will tie bottlenecks to source code, so you can quickly pinpoint and resolve those performance abnormalities like N+1 queries, slow database queries, and memory bloat. Scout also recently implemented external service monitoring, adding even more granularity when it comes to HTTP requests and API calls. So give Scout a try today with a free 14-day trial and experience first-hand why developers worldwide call Scout their best friend. And as an added bonus for Bike Shed listeners, Scout will donate $5 to the open-source project of your choice when you deploy. To learn more, visit scoutapm.com/bikeshed. That's scoutapm.com/bikeshed. STEPH: Well, here's one more that maybe you'll agree with, maybe you won't. We'll see. I'll try not to lead you in either direction, but shared examples. If I'm going to rant for a little bit, shared examples are in that space of where they just get used so heavily, and they abstract away important information about the test. And it makes the test so succinct that I don't actually know what the test is doing. And I've seen a number of places where a shared example has been extracted, and it is only used within that test file once, maybe twice. And I'm just like, friends, too much abstraction. Please keep it close. [laughs] We don't need to move it away. We want our test to be friendly and just full of context, which is what I mean when I say friendly. I want full of context is what I'm looking for, well-named variables. And I won't be able to read the test and see what's happening. So my little complaint for today would also be about shared examples and how they can be overused. And they do have a really neat purpose. They can be helpful for if you're testing maybe a controller action and you want to say you're extracting that authentication, making sure that a controller always has authentication and then that is getting included. Sure, that feels very helpful. But that's really one of the few cases I can think of where a shared example comes into play. And if you are testing code over and over throughout different parts of your codebase, there's probably a part of your codebase there that needs to then be pulled out into a class and test that class in isolation. And then you don't need to retest it throughout all of your other classes. Have I already ranted about shared examples? I can't recall at this point if I have or not. [laughs] CHRIS: I don't think we have talked about shared examples before. And I appreciate you not leading the witness here. But I think I'm in agreement with you, particularly the way you refined it there at the end because that controller example is the one rare case where I might reach for it. But in general, I think this is one of those things that I saw early on in my career. I was like, oh, cool; this is a way to clean stuff up and DRY and all those wonderful things. And then I've definitely felt the pain of just overuse of shared examples and ways to pull details out of tests. But it's like, I want to see the details. And I think broadly, that's the theme that you and I are very aligned on is like, no, no, no, tell me the story in tests. I am much less interested in having these concise tests that have a single line, and it's like, expect foo to have bar. And it's like, why? Because...oh, there's a let and then a subject, and it's a shared...oh okay. Now that I can put it together, I can tell the story, but I cannot look at this test and see a story. I want to see a story, friends. So yes, I'm totally in alignment, especially with the slight caveat at the end of like, there are cases where it's useful. Similarly, I've used let. I definitely have not even that long ago. And I stand by the usage, but it was very rare. It's very rare, and it is something that I'll look at and be like, am I sure? Definitely, is this the right thing, or did I do something wrong? Because if I find myself leaning towards let, it's like there's something that I don't think is important to the story of this test that still needs to happen. Why is that? What's going on here? Something feels off about that. And similarly, with the shared examples, it's like, is there not a different way to extract this such that I can test it in a way that I have confidence in, and then we're good? I occasionally will talk myself into using shared examples or something like it where I'm like, oh, but it's really important that everything in the app has that authentication layer put in. And so, I should definitely have this very easily reusable test that can ensure that I have it. But there's a tautology there of well, if I write the test, then I'm definitely thinking about the implementation. But if I forget the implementation, I might also forget the test. And so, it actually doesn't provide any real safety. And in those cases, that's a rare case where I would reach for some weird metaprogramming thing that's like, controllers must do the thing. And we say that in application control and then everything inherited from that will raise if it doesn't implement the authentication layer. Something like weird code that says, "You shall not pass. You must, in fact, implement the authentication layer." Rather than saying, "Oh, we'll just make it really easy to test it so that we always test and, therefore, always use the necessary authentication layer." But yeah, that's a hard one to describe in the radio. So I don't know if that came through clearly. But that's sort of my headspace on this. STEPH: Yeah, and all of that makes sense. I'm trying to think of a good example. And it's been a while since I've used Pundit, but I feel like Pundit may have a really good example of this where it's very easy to document to say, hey, all of these controllers need to make sure that they call out to this class or that there's authentication. I can't remember the exact code and how that works. But I feel like Pundit has a really good example of that behavior. CHRIS: I think they do. It's something where I think it's a configuration level thing, but you say, "Hey, Pundit, we should definitely authorize any access to models." And so Pundit then has a before action, or it's an around filter one of those. But it will raise an unauthorized error, I want to say. Like, you did not do the authorization dance in this. And that's a great example of like, I like that it is loud and annoying and in your face. And it is not possible for me to forget it because we configured it throughout all controllers. And so it's that sort of thing that I would probably reach for even though that code gets complicated and messy, and actions at a distance. But it's worth that trade-off in my mind to have, like, I don't want to forget to do the authorization stuff. Permissions matter. STEPH: That was a really nice pre-emptive approach as well. Because in most cases that we're describing, it's the I'm going to write a controller, and then I need to add this test to verify and prove that yes, I didn't forget the authentication stuff versus upfront, you're setting in a configuration to say, "Hey, please remind me to do the configuration or the authentication step that I don't miss this." So that's also a really, really nice approach. CHRIS: Yeah, the same version of me that's going to forget to write the test is going to forget to write the implementation. So I don't want to trust that version of me to save that version of me. I'm equally untrustworthy in those situations. STEPH: You want to trust the version of you that's going to get yelled at by the code if you don't do it. CHRIS: Yep, I'm going to trust the version of me that was like, I don't trust any future version of me. I will yell at myself if I have not done the necessary things. STEPH: [laughs] CHRIS: To be clear, this is like a life philosophy of mine. I don't try to remember things because I forget stuff a lot. It just happens. And so if I need to take something out the door with me, it goes in front of the door but extra critically, and this is the subtle line. Because plenty of people do that trick where you put a thing in front of the door because then you can't leave without it. There's no way to forget it. But by virtue of that, you cannot put something in front of the door until it is time to use it. Like, if ever you have to go and be like, oh, I don't need it now, though, so I'm going to move it out of the way, open the door, and then leave. No, no, no, because then you've broken the magic of the thing in front of the door must leave with you. So it's a very subtle line. I will play games with myself. I'll be like, I am forgetful. I will not remember this. I do not trust future me, so I'm going to play a trick on them. But you got to calibrate it just right. STEPH: That's really funny because I totally [laughs] didn't think about it until now how you described it. But I have definitely done that where I set a rule for myself, but then I'll break it. And then, of course, everything all of it collapses. There is a time when Tim, my husband, was going through a developer bootcamp. And as he was learning the whole world and everything that's out there, he would ask me all these questions. And he's like, "Do you know this?" And I'd be like, "No." He's like, "Do you know that?" I was like, "No." He was like, "I thought you knew this stuff." He's like, "I thought this was your job." And I was like, "Yeah, I'm really good at finding it and Googling it. But I work really hard to not store this in short-term memory because I'm filling it up with other stuff. So I work really hard to be able to find this stuff and track it and Google it." But now, there's a lot of stuff that I try very hard to not hold on to until I need it. But that was a funny moment where he seemed very upset that I didn't know stuff. And I was like, "Well, welcome to web development. There is too much to know. You're going to have to have a really good catalog system." CHRIS: Also, just so we're clear, it's going to change by next Thursday, so don't hang on to anything like it's just true forever. STEPH: [laughs] CHRIS: SQL will probably be around. That's about it. That's the one thing that I'm really confident in. STEPH: Yeah, that feels fair. Get really good at understanding HTTP forms, SQL, all that feels like some really good groundwork. CHRIS: There are some foundations. We should have a foundations episode where we talk about what we think the foundations are, the stuff that we bet won't be different in 10 years. But everything else is going to change by next Thursday, specifically. STEPH: Yeah, I like the idea of foundations. I'd be intrigued to see what we talk about and what happens there because I feel like that's going to be very representative of already what we talk about. We often will sprinkle in some new-new, especially thanks to a lot of the adventures that you go on. But I feel like a lot of the stuff that we talk about we always bring it back to the foundation because we do want the experiences that we're having to be applicable to everyone else as well. So yeah, that would be interesting to see what comes out of that. On that note, shall we wrap up? CHRIS: Let's wrap up. The show notes for this episode can be found at bikeshed.fm. STEPH: This show is produced and edited by Mandy Moore. CHRIS: If you enjoyed listening, one really easy way to support the show is to leave us a quick rating or even a review on iTunes, as it really helps other folks find the show. STEPH: If you have any feedback for this or any of our other episodes, you can reach us at @_bikeshed or reach me on Twitter @SViccari. CHRIS: And I'm @christoomey. STEPH: Or you can reach us at hosts@bikeshed.fm via email. CHRIS: Thanks so much for listening to The Bike Shed, and we'll see you next week. ALL: Byeeeeeeeee!!! ANNOUNCER: This podcast was brought to you by thoughtbot. thoughtbot is your expert design and development partner. Let's make your product and team a success.

Jonan Scheffler interviews Mike Perham of Contributed Systems about his work on Sidekiq: a framework for building and executing background jobs.Should you find a burning need to share your thoughts or rants about the show, please spray them at devrel@newrelic.com. While you're going to all the trouble of shipping us some bytes, please consider taking a moment to let us know what you'd like to hear on the show in the future. Despite the all-caps flaming you will receive in response, please know that we are sincerely interested in your feedback; we aim to appease. Follow us on the Twitters:@PolyglotShow.

It’s been a crazy week of weather in Jason, Chris, and Andrew’s hometowns, but thankfully we have a great guest to take away the winter blues. Today, we have Mike Perham from Sidekiq and Faktory fame. We find out what Sidekiq is and a great story about what led Mike into starting it, as well as Faktory. Other topics in the discussion are how Active Jobs plays into Sidekiq, an experiment Mike did with Sidekiq and Crystal, and Mike giving some inspiring advice on how teaching and building trust work hand in hand.

This conversation covers: Mirage's role as an API mocking library, the value that it offers for developers, and who can benefit from using it. How Mirage empowers front end developers to create production-ready UIs as quickly as possible. How Mirage evolved into an API mocking library  How Mirage differs from JSON Server  Sam's relationship to Mirage, and how it fits in with his business. Sam also talks about open source business models, and whether Mirage could work as a SaaS offering. One interesting use case for Mirage, which involves demoing software and driving sales. Links Mirage Sam's teaching site Follow Sam on Twitter Subscribe to Sam's YouTube Channel TranscriptEmily: Hi everyone. I'm Emily Omier, your host, and my day job is helping companies position themselves in the cloud-native ecosystem so that their product's value is obvious to end-users. I started this podcast because organizations embark on the cloud naive journey for business reasons, but in general, the industry doesn't talk about them. Instead, we talk a lot about technical reasons. I'm hoping that with this podcast, we focus more on the business goals and business motivations that lead organizations to adopt cloud-native and Kubernetes. I hope you'll join me.Emily: Welcome to the Business of Cloud Native. My name is Emily, I'm your host, and today I'm chatting with Sam Selikoff. Thank you so much for joining us, Sam.Sam: Thanks for having me.Emily: Yeah. So, today, we're going to do something a little bit different, and we're going to talk about positioning for open source projects. A lot of people talk about positioning for companies, which is also really important. And they don't always think about how positioning is important for open source. Open source maintainers often don't like to talk about marketing because you're not selling anything. But you are asking people to give you their time which, at least for some people, is actually more valuable than their money. And that means you have to make a compelling case for why it's worth it to contribute to your project, and also why they should use it, why they should care about it? So, anyway, we're going to talk with Sam, about Mirage. But first, I should let you introduce yourself. Sam, thank you so much for joining me, and can you introduce yourself a little bit?Sam: Sure. My name is Sam Selikoff. These days, I spend most of my time teaching people how to code in the form of videos on my YouTube channel, and my website, embermap.com. Most of it is front end web development focused. So, we focus on JavaScript. I have a business partner who also works with me. And then we also do custom app development, you know, some consulting throughout the year.Emily: Cool. And then tell me a little bit about Mirage.Sam: Yeah, so Mirage is the biggest open source project I've been a part of since falling into web development, I'd say about eight years ago, I got into open source pretty early on in programming, kind of what made me fall in love with web development and JavaScript. So, I was starting to help out and just get involved with existing projects and things that I was using. Eventually, I made my way to TED Talks, the conference company where I was a front end developer, and that's actually where I met my business partner, Ryan. And we were using Ember.js, which is a JavaScript framework, and we had lots of different apps at TED that were helping with various parts of publishing talks, and running conferences, and all that stuff. And we were seeing some common setup code that we were using across all these apps to help us test them, and that's where Mirage came from. There was another project called Pretender, which helped you mock out servers so that you could test your front end against different server states. And we first wrapped that with something called Pretenderify, and then it grew in complexity. So, I was working on it on my learning Wednesdays, renamed it to Mirage, and then I've been working on it basically ever since. And then, the other big step, I guess, in the history is that originally was an Ember only project, and then last year, we worked on generalizing it so that it can be used by React developers, React Native developers, Vue developers, so now it's just a general-purpose JavaScript API mocking library.Emily: So, we would say that the position is an API mocking library. And—does that sound right?Sam: Yeah. If I had to say what it is, I would say it's a mocking library that helps front end developers mock out backend API's so that they can develop and test the user interfaces without having to rely on back end services.Emily: Why does that matter?Sam: It matters because back end services can be very complicated, there can be multiple back end services that need to run in order to support a UI, and if you're a front end developer, and you just want to make a change and see what the shopping cart looks like when it's empty. What does the shopping cart look like when there's one item? What does it look like when there's 100 items, and we have to have multiple pages? All three of those states correspond to different data in some back end service, usually in a database. And so, for a front end developer, or anyone working on the user interface, really, it can be time-consuming and complex to put that actual server in that state that they need to help them develop the UI. That can involve anything from running, like, a Rails server on their computer to getting other API's that other teams manage into the state they need to develop the UI. So, Mirage lets them mock that out and basically have a fake server that they control and they can put into any state they need. So, it's like a simplified version of back end services that the front end developer can control to help them develop and test the UI.Emily: And when you first started Mirage, did you think of it as an API mocking library?Sam: Not exactly. We used it mostly because of testing. So, in a test, it's usually a best practice to not have your test rely on an actual network. You want to be able to run your test suite of your user interface anywhere, let's say on an airplane or something like that. So, if your user interface relies on live back end services, that's usually where you would bring in a mocking library. And then you would say, okay, when the user visits amazon.com/cart, normally, it would go try to fetch the items in your cart from a real server, but in the test, we're going to say, “Oh, when my app does that, let's just respond with zero items. And then in this next test, when my app does that, let's respond with three items.” So, that's the motivation originally, is in a testing environment, giving the UI developer control over that. And then what happened was that it was so useful, we started using it in development as well, just to help during normal times, just because it was faster than working with the real back end services.Emily: Do you think there are any other projects that do something similar?Sam: Yeah, for sure. I think the most popular one is called JSON Server, which is a popular open source library that lets a front end developer put some data in a file, and then you point JSON Server to it, and then it just gives you an instant mock server you can use to help develop your app.Emily: So, what's the difference between Mirage and JSON Server?Sam: The difference is that JSON Server is made—it's really optimized for giving you a development—kind of a fake server as fast as possible, but it comes with a certain format that it gives you the data in. So, what ends up happening is that it can help you get feedback and build your UI faster, but eventually, you're going to need to point your app at a real API server, whatever you planning on using in production. And so the way JSON Server works might not correspond—in fact, often doesn't correspond with your actual API. So, Mirage fills that gap because Mirage is designed to be able to faithfully reproduce any production API; there's ways to customize how the data comes back so that it matches so that as you're developing your actual user interface against Mirage, you can have confidence that it'll work once you switch over to production.Emily: Is Mirage slower than other options?Sam: Not performance-wise because they're all JavaScript code that runs in the browser, but JSON Server is really optimized for just getting started as fast as possible because it comes with all of those pre-baked conventions about how the data is going to be moving back and forth. So, with Mirage—it can be faster, it depends—but with Mirage, you need to learn a little bit more in order to understand how to faithfully reproduce your production API. But I think it's faster because in the long run, if you're writing code against a mock server that doesn't match the interface of your production API, then you're just going to be having to change that application code that you're writing.Emily: How much do you talk to other people in the Mirage community, and talk about how they're actually using it?Sam: I felt more in touch with the users when it was an Ember project only because Ember is a more niche-type community. Whereas now, there's folks using it in React and Vue, like I was saying, and Angular. And so, we get issues almost every day on the project. It's not like a mega-popular project, but it does have enough people using it that people will ask questions, or open an issue almost every single day. And so I try to stay in touch with the users through that, basically. And then when I went to conferences—you know, before 2020—I would love to talk with people about it, or people would just bring it up. That's kind of how a lot of people know me on the internet. So, I would say that I do it in kind of a passive way. I haven't actively gone out to talk to them, partly because it's an open source project so it doesn't contribute to our revenue. We have some ideas for how that might happen one day, but as of right now, we can't justify doing proper product development on it in the sense of spending time doing customer interviews and stuff because it's a free project right now.Emily: That is my next question, which is, how does it fit in with your business in the larger sense? You know, how you put food on the table? And what are your goals for the project?Sam: A good question. I mean, it's really aligned with our overall mission of, of everything that we do because me and my business partner, Ryan, we really want to just help people get better at UI development, we want it to be easier, we want to help empower more people to do it because we think it's powerful tool in the world, and it's just too hard right now; there's so many things that are hard about it. So, that goes back to our consulting and our teaching; mainly our teaching. That's our main mission. And then Mirage is really—the purpose of Mirage is to enable front end developers to do more kind of with less, so they don't have to run a Docker container or get an SQL database up just to change some CSS for a given server state. So, it fits into that mission. I've been doing open source long enough to know the pattern, and it happens over and over again, where people work on something, it gets popular enough that they start opening issues, and it becomes a maintenance burden for the maintainers, and then they try to stay up late closing issues, they get burned out, and then the project kind of rots. So, that's something that happens a lot in open source. And so, over the last five years or so of working on Mirage, I've been more involved, or sometimes step back if I need to spend more time on other things, but I'm really interested in making it sustainable, and I know some people in the open source community who have made their projects sustainable financially, other with a pro plan, or support plan, or different related services. So, if we could snap our fingers, that would be what would happen, and that's what we're working on now.Emily: Which one of those open source business models do you think is most appropriate?Sam: At this point, we basically are trying to not guess that answer because we think the way to find out which one it is, is to get a critical mass of users and listen to what they're saying. So, on our podcast, we interviewed Mike Perham from Sidekick, who runs Sidekiq and Sidekiq Pro in the Rails community for about 10 years, and he makes really good money. Sidekiq Pro came about because the enterprise customers of Sidekiq were asking for more robust job servers and all this kind of stuff, so there was a natural path for him to making a pro version that he could sell to enterprise clients. And so that's worked out really well for him, and the rest of the community gets to use the base version for free. And I love that because I do love this zero-cost to entry to open source. And then my other friend, Adam Wathan who works on Tailwind, he's made money to help Tailwind be sustainable through education and courses, and then more recently, a project called Tailwind UI, which is pre-built UI components with Tailwind. And that, again, came about from people asking for that after he was working on Tailwind. So, I think the best way to do it is to get that critical mass of users to the point where you know what they're asking for, you hear over and over again, and then it makes sense to go forward with that.Emily: There's also a third option, which is a cloud service, and I'm just curious—like a complete SaaS offering—have you ever considered that?Sam: Absolutely. There's a really cool opportunity there for Mirage because your Mirage server usually lives alongside your front end code so that when every individual front end developer pulls the project down, starts working on it locally, they're running their own Mirage server. But some of the things that people have done organically with Mirage is, create a certain configuration of a server state—let's say for a demo—so let's say they create a shopping cart, or let's say they're working on a financial piece of software, and they need to show what it looks like with three clients and four contracts, and here are the products that we sold and how much money they are. People will make up a Mirage scenario with that specific set of data that their salespeople take and use on sales calls. And that way, again, the user interface looks fully realistic, it's a fully working UI that is talking to Mirage, but now you don't have to worry about the actual back end servers going down or anything like that. And so, we've had this thought of a hosted Mirage, basically like a Mirage cloud, where even non-technical people could tweak the data there, and then again, they don't have to get involved with ops people or anything like that. And that could be really powerful. So, there's a ton of ideas there. I think our hesitation with our company, Embermap, it's worked to some extent, but it's not grown to the point where it can sustain us, and so that's partly because of the market issue, Ember being a little smaller. So, we're nervous to jump into any particular solution before we feel there's a proven market need for it. And so that's why, even though we have a lot of these fun ideas that I think could really work out, we first want to wait until we get that critical mass, the audience size that we'd feel comfortable could sustain a business.Emily: How many people is that? What is the audience size?Sam: That's a tough question. Let's say Mirage cloud was the goal. I think instead of waiting to a certain point, and then trying to build Mirage cloud, we would do a few things in the interim that would be little experiments and lower risk. So, I think the first thing would be, let's say, a Mirage course. And if we can sell a course on Mirage—or even we make a free Mirage course—that gets enough attention, that would tell us that there is enough of a need there, that the positioning resonates, that people are seeing it's a valuable thing, and that would tell us, okay, let's try the next thing. So, we've been trying to do that. I make some YouTube videos over this last year, and I've been tweeting more about it and the work we've been doing, and so all of those are little experiments that I'm trying to pay attention to what resonates. Again, we have users who really love Mirage, and it's changed their workflow, but it's not at the point where I feel like it's a slam dunk and it makes sense to go on the next phase. So, I think there's been some frictions with Mirage, as we've brought it out to the wider JavaScript community. So, we have some things we want to tweak, and then ship a 1.0, and then I think maybe a 10 video course, would be a good next step, and just see the response to that, and basically take it from there.Emily: Yeah. It's always interesting to think like, how will that you have reached the critical mass?Sam: Yeah, I mean—Emily: Hard. Sam: It's really hard. And there's other strategies. I mean, a lot of businesses just have an idea and go try it out. There's this book I read, called Nail It and Scale It, and they talked about finding a problem. And we've have had some users of Mirage who use it at big, bigger companies, and it's really a big part of their workflow. And we've thought about, “Hey, what if we were to build something for them that we could generalize?” They actually have a need for something like a Mirage cloud because every time they develop a feature, they have to sit down with their product people, and their front end developer will basically run through all these different Mirage scenarios to show them how the feature works in every case, and they would love to be able to just send a link to a hosted version where they could do that. So, we've thought about building that kind of thing. But again, it's just a big risk. And then the market risk is there. So, what I've seen, I feel like, working in the past few years with a small circle of small business people and open source that I hang out with is this audience-first approach. So, it's like, if you're delivering a lot of value in the form of open source, and education, and talks, and you build an audience that believes what you have to say, and likes your opinion on things, likes your point of view, and again, resonates with the value of your work, then it becomes more and more obvious. You have a lot of people giving you feedback, you start seeing the same thing over and over. And that's just what we do with developing Mirage itself. We know a lot of what we work on next is driven by the same issues that come up, the same problems that come up in the issues, over and over again. So, I think it is hard to know, but it's one of those lukewarm things. And basically, right now, it feels too early. You know?Emily: And do you feel like, when you say to somebody, let's say, somebody who isn't involved with Mirage at the moment, maybe you're at a developer meetup or something, and you say, “I created Mirage. It's an API mocking library.” Do you have an aha moment? Are they like, “Oh, yeah. I know what that is. I know why you would use that.”Sam: Mm-hm. Sometimes yes, and sometimes no. There is a form of position I feel like, sometimes, really resonates well with people, which is like, “Don't get blocked by your back end developers.” So, if you're a front end developer, and the API is not ready, you can still build your app, including all the dynamic parts of it which would normally require a real server to be running. So, you're the front end developer, and you're trying to wire up the interface, and what happens when you click save on a cart, and it saves it in the back end, and then you show a success message. But your API is not ready, and you're working with another team that's working on the API, so you're kind of frustrated because you're stuck and you feel like you can't do anything, well, that's where Mirage can come in because you don't have to wait on them at all. You can just build it out yourself with Mirage. It's much simpler because it abstracts away all the complexity of a real server enough that you can actually build your UI against this kind of faithful reproduction of the back end. So, people really like that. And then people really like the testing use cases as well. They get confused about the best way to test user interfaces in this new distributed world we're living in where you're maybe working on a React app, and the back end is a separate API that's also serving up iPhone clients and things like that. So, there's really multiple apps involved with running a website like amazon.com. So, the question is, how do you test the front end? And Mirage is a great answer for that as well. And that's where it originally came from, so.Emily: Yeah. I can definitely see the positioning is sort of like a way for front end developers to decrease their dependence on their back end colleagues.Sam: Yeah, exactly. It's hard because there's a lot that it helps you with. And the people who use Mirage the most and have used it the most, it's really transformed their workflow because it lets the front end developer just move so much faster. So, it's really just, like, the fastest way to build a front end. That's really what the whole point of—every change we make to the library, every decision that went into it, it's how to empower a front end developer to build a production-ready user interface as fast as possible. And that includes accounting for all these different server states that are usually a pain in the butt to get.Emily: I'm just taking notes. I mean, that that actually sounded really powerful. Like, “Mirage lets front end developers work as fast as possible,” basically. It's fairly high level, but ultimately, that is a pretty compelling value statement.Sam: Yeah. And I believe it to be true, too. And I mean, that's how—I mean, I've been working on apps recently, and I still use Mirage because it's just faster than using a real server because it's right there in your code. It's right alongside your front end code, so you don't have to switch over to another process running, or open up a browser and see it; it's all right there. If you want to switch from being an admin to being a user, it's like you just uncomment a line alongside the code that you're already writing. So, I truly believe it is the best way to build a user interface. I think the positioning question is interesting because that is high-level. Sometimes developers in open source, they just want to know what it is, so there are some people who would see, “Just tell me what it is.” “It's an API mocking library.” “Okay, got it.” And that's what they want to know. “And what makes it different from other API mocking libraries?” Okay, we can talk about that. But then there's other people I think, who could stand to benefit from it, and if they saw, “Mirage is an API mocking library,” they're going to be like, “I don't really need that.” But then they go back to their job, and they have to work on their app, and they find themselves spinning up a Docker container, going to auth0 to sign in just so they can run their app in an authenticated state, and it's like, Mirage would actually be perfect for this. You could just mock all that stuff out in Mirage once, all your front end developers could use it. And now anytime you want to just work on your app in an auth state, it's just right there in Mirage, you don't have to deal with any of the complexities of the production services.Emily: Yeah. I mean, I also like the idea of sort of the fastest way to build a front end app.Sam: Yeah.Emily: Or to build a UI. The fastest way to build a UI.Sam: Yeah. That is, that's nice. I mean, it's interesting. It's a pretty interesting thing to say, and it's pretty compelling, and it's like, if you can back it up, that's a pretty strong claim.Emily: Yeah. And I mean, one of the things that I also talk to clients about—so I work with all technical founders, usually, and we're often really focused on features, all the cool features, but—you don't have to ignore the features, but you sort of use them to prove that the thing that you're talking about, that the value you say you provide is not BS. But you still want to connect the dots. Like you were saying, sometimes even the most technical person is like, “Oh, a mocking library. But I don't need that.” They're not going to necessarily connect the dots that, like, “Oh, that's going to make my workflow way simpler.” Sam: Right.Emily: Does everybody know what a mocking library is? Like, everybody who needs one?Sam: Mm, depends. If you're a front end developer. I mean, these days, people are becoming more and more specialized, so there's some front end developers who don't even deal with the data fetching side of an app at all. They're working on a part of the app that already has the data, and they're just working on maybe styling, layout, things like that, but they're never writing code or refactoring code that actually interacts with the server. And then even if you are doing that, you might not be writing tests. So, a lot of people just do the lowest friction thing, which is, just point their local UI at whatever server they can. Maybe their company has a staging server or maybe they run one locally in development and they just build it like that. But again, that's the lowest friction, but it's slow because now you're running a Rails server. If you want to change the data, you have to know how to do it in Rails, and you have to run different commands. And then it comes to testing, it's really hard, too. So, I think most people would know, if you were to say, “Yeah, it mocks out the API.” Most people would know that. But people might have different conceptions of what that means. So, there are some approaches to mocking—or stubbing, or faking, there's some technical difference between those terms, but for the purpose of this conversation, it's just faking out that functionality—there's some people who would hear that and say, “Oh, okay, I get it. A function that my code calls when it normally goes to the server, I'm going to replace it with a different function that returns this fake data.” But Mirage works differently because it operates at the boundary of the app. So, when you mock your API with Mirage, you don't have to change anything about your application because it intercepts the network request before it goes to the server. So, that's another big benefit of Mirage, that Mirage has over other solutions for mocking out network functionality is that your application code stays exactly the same. And that's an important point because the whole goal with Mirage is to be the fastest way to build a production-ready user interface. And by production-ready, I mean an interface that can be plugged into your production API and you'll have confidence that it works. So, that boundary thing is another important point of Mirage. So, that would maybe be the one thing that people could mean different things when they say ‘mocking.' But by and large, I would say most front end developers would understand if you said ‘API mocking,' what they mean.Emily: And do you think they generally are also able to figure out what that's used for, why they should care?Sam: Yeah, that's the thing is that that's where I think the real opportunity is because I think, if you were to say ‘API mocking,' people are going to immediately go to testing because that's the kind of environment where you would want to mock the API so you have control over it. But people who haven't used Mirage or something like Mirage, don't realize how powerful it can be, to mock it just during your normal development flow. So, again, once people use it and get it, they use Mirage for everything; it's just part of their workflow. I just start up my UI, I'm running Mirage in a specific scenario, and if I need to see the UI in a different state, I just changed my Mirage scenario, or put some new data in there, or empty out the database there. And so it totally affects your whole workflow because now you're just disconnected from the back end services. So, I think a lot of people aren't doing that. They are just stuck—you know, they're just used to the hassle of getting these three services up and running just so that they can run their UI, and it's a pain in the butt. I mean, I was just talking to someone on another podcast, and he was saying it's the same thing. And it's really hard when they onboard new people because they have to get them set up with all these auth keys just so they can run the front end, even though they don't really care about the auth setup, they just want to start working on this page. So, again, the companies that have used Mirage and adopted it don't have any of those problems because the Mirage server is right there with the user interface, and they don't have to worry about it. So, I think that is a big gap that we could probably do a lot better job of closing with information on the homepage, or examples, or something like that.Emily: So, Mirage also helps new hires get up to speed a lot faster, or get productive, I should say?Sam: So, yeah. If you were hired by Facebook, and you're a front end developer, and your first task is to update the way the colors look on the home feed, to get that running on your computer so you can make changes and see how it looks in the code, at many companies—probably most companies—it's going to involve a lot of moving pieces because you have this React app on the front end, but it has to fetch data from somewhere so you can see how it works. Maybe they have a server that is just used for development that the person can point to. But again, if you have this shared hosted server, you only get what's on there; maybe you don't have an easy way to change what data it gives you. So, usually, front end developers need to see a dynamic user interface in all these different states. But if you're using a shared server, you don't get all those different states. But it's easier to set up because it's already set up. So, the alternative is to say, “All right, Sam, you're new here. Let's get you set up so that you can run a copy of Facebook locally.” So, that usually involves a ton of steps. I mean, I've seen that take, like, a week just to get all the pieces that are required to run the back end up so that you can actually power your front end. So, yeah, companies definitely, definitely have a problem with this. They have a problem with shared staging servers, I've talked to tons of people over the years who hate their staging servers, the back end teams, the ops teams hate them because everyone's always trying to change it for the salespeople to go take demo calls or for people to test. And so basically a shared server like that is just a nightmare because it's basically another production server to maintain that your internal team is trying to use and people want it to do different things. So, that's why Mirage is nice because it's just local to the code, and every front end developer can just tweak it in exactly the way they need for what they're doing at that time. So, I think that's a big opportunity as well.Emily: One of the most interesting things, I think, about this conversation is that you've touched on this idea of salespeople doing demos, and I really like that because it's so different. It's such a different use case.Sam: Yeah. We had a thought about that, actually because we were talking to a handful of companies—did interviews with them, actually, a couple years ago—and we're like, this could be a cool product. And it's like, just the easiest way to show off your software. And again, we've even done consulting for companies that have had basically another server, just for the purposes of a demo. And again, it's just a pain in the butt because it's another production server to manage, you have to reset the database after each call you do, and with Mirage is not like that; it just restarts with the app. It's just heavier, it's a real server to manage, where again if you just need to show off the UI, you can do it with Mirage. So, we thought about doing that. It's an interesting use case, for sure.Emily: I think it's a really good illustration of how you can take the same technology, and reposition it, basically, to do something totally different, have a totally different set of value proposition. The people who would be actually benefiting from its use would be totally different. I mean, you have salespeople versus front end developers.Sam: Yep. We've thought about that. What would that look like to market to those people? And maybe one way you could do it would be, you know, the salespeople get frustrated because they want to just change the numbers in this part of the app on the spreadsheet summary because they know it's going to help them communicate the value to the people that they're talking to, but now they have to go and ask some back end developer or someone on the ops team, “Hey, can you change this database column so that my demo works better?” What if they had a Mirage-powered demo and they could tweak the Mirage data, maybe in some interface themselves, without having to involve anybody. And that's pretty compelling because Mirage, again, is designed to work with any API. So, no matter what your tech stack is on the back end, Mirage works for the front end team. And so now the back end team can do all their stuff, they can be switching from Rails to Elixir, they can be switching to Go or microservice architecture, and they have all this stuff going on, so they're the only ones who really know how to actually get this number from 100 to 200 in the system. Like, it's complicated. And so that's, again, a benefit of just having Mirage because, on the sales calls, the people are just wanting to show what it looks like when the data is a certain way. So, yeah, that was definitely a use case that we didn't anticipate at all.Emily: Yeah. And even you could have five salespeople trying to do calls at the same time and—Sam: Exactly.Emily: Wanting to have different data reflected, and I'm sure that's just like—Sam: A nightmare. I mean, people have told us just, it's the worst. Basically, the shared staging server is a—yeah, it's a huge problem for a lot of teams.Emily: It's so interesting. Well, anyway, taking us back from the rabbit hole, although it—I mean, it really is interesting, and just fascinating how you could change this to be something that's targeted at that totally different market.Sam: Right.Emily: To wrap up. I mean, I was thinking about how you're saying that the fastest way to build a production-ready UI, that possibly is the most powerful thing I think you've said.Sam: Yeah. We actually—I think that's how we had it when we first did the redesign of the site. And it was like, I don't know if that stuck in the sense of, what does production-ready mean? “Oh, I'm already writing production-ready code.” But then it's like, you have to—like you said, connect the dots and explain how you're not writing production-ready code unless the way you're mocking your API is matching your production API, so we kind of switched it to what it says now, which is, “Build complete front end features, even if your API doesn't exist.” Which basically came from the mouth of someone who was using it, and when they had their aha moment that was what they were saying. I've been thinking if we get to the 1.0 launch, I think I would want to go back to something like that because I do think it's compelling. So, “It's the fastest way to develop a UI, test it, and then share a working demo of it,” because that's is really the motivation for the library. So, yeah, it might be interesting to think about doubling down on that as the unique value proposition.Emily: Yeah. I'm curious what your immediate plans are.Sam: So, we've been working on this REPL playground area of the site where people can learn Mirage and see examples, and we can share them and stuff. So, that will hopefully—you'll see a lot of that kind of thing in the JavaScript community. If you go to Svelte, which is another front end framework, you can create little sandboxes and play around with Svelte and learn it. So, it's a really good way to learn things and just see how it works quickly right in the browser without having to install anything. So, we're about wrapping that up. And then I think it'll just be a matter of carving out the time to go through some of the issues and bugs that people have found, and getting it to a point where we feel good about slapping a 1.0 on it. At which point, we can take a look at all the feature requests and all the issues that people have run into and figure out okay, where do we want to focus our time? Again, considering, like, is there a story here where we can make it sustainable, and hopefully, dedicate more of our time to it? Because that's really, again, I think it's the biggest impact work I've done in my career so far, but it's just, sustainability in open source is tough. So, it's a matter of not jumping too early on any particular idea for how to sustain it and monetize it: if it's a pro version, if it's a support plan, people have asked for all these things, but not maybe in the strongest numbers that would make me feel comfortable diving in on that. But I do believe that it can work because I believe it's a really good idea and I know a lot of companies have gotten a lot of value from it. So, that's what our short term plan is.Emily: Well, fabulous. Thanks so much for talking about Mirage. This has been really interesting.Sam: Yeah, thanks a lot for having me, and I appreciate your input. It was fun to revisit the positioning stuff. It's been a while, so it's always good to be thinking about that.Emily: All right. I have one last question for you, which is what is an engineering tool you can't live without?Sam: These days, it's got to be Tailwind. It's a styling framework. And it's just really excellent. So, I would not want to work on a site without it because it would just be painful. So, [laugh] I love Tailwind. Yeah, it's a really good library.Emily: All right, cool. Well, thank you so much. And—oh, what—the very last thing it should be, how can listeners find you, follow you, keep up with you?Sam: Yeah, best place is Twitter, at @samselikoff. And I'm also making YouTube videos more and more these days: youtube.com/samselikoff. So, those would be the places to go.Emily: Right. Excellent.Emily: Thanks for listening. I hope you've learned just a little bit more about The Business of Cloud Native. If you'd like to connect with me or learn more about my positioning services, look me up on LinkedIn: I'm Emily Omier—that's O-M-I-E-R—or visit my website which is emilyomier.com. Thank you, and until next time.Announcer: This has been a HumblePod production. Stay humble.

Mike, of course, is most famous in the Ruby community for Sidekiq. Outside Ruby, you're more likely to know him for Faktory. Mike and I talk a lot about his education in and out of university and how it's served him. We also talk about how that's changed over the years as his career has continued. For show notes, links, comments and transcripts: http://justtheusefulbits.com/jtub/mike-perham-sidekiq-and-whitepapers-and-what-success-is-for/

Mike Perham of Sidekiq and Faktory fame joins us on FounderQuest this week. We talk about the origins of Sidekiq, COVID impacts, and having a plan for your business if you, the sole proprietor, kicks the bucket. Can you kick it? Yes you can!

Me and Mike start with a detailed discussion of how systemd and systemctl work in Linux, then transition into server infrastructure in general, then finally we talk about the business side of Sidekiq.

Topics include:- 0:00 - What is Faktory?- 2:28 – Why might I need a background job?- 13:26 – Why did you make Sidekiq?- 16:15 – What lead to Faktory?- 24:02 – Why'd you use Go to implement Faktory's server?- 25:36 – Who is Faktory for?- 31:58 – What's the most interesting thing you've learned about architecting background job systems?- 36:24 – How do you see job queuing work in a serverless world?- 41:23 – What are some of your thoughts on open source sustainability?- 46:48 – What makes a library productizable?- 48:30 – Were you thinking entrepreneurially when starting Sidekiq?- 53:30 – Could open source sustainability be solved by a marketplace or middleman?- 55:14 – How has your business model and financial incentives affected the development of your open source libraries?- 1:00:30 – How do you think about API additions and feature requests to Sidekiq? Links:- [Faktory](https://github.com/contribsys/faktory)- [Sidekiq](https://sidekiq.org)- [Mike on Twitter](https://twitter.com/getajobmike)- [Mike's blog](https://www.mikeperham.com)- [Building a $1 Million Business Solo with Mike Perham of Sidekiq](https://www.indiehackers.com/podcast/016-mike-perham-of-sidekiq)

Robby speaks with Mike Perham, Founder and CEO at Contribsys and author of Sidekiq and Faktory. They discuss the pros and cons of using external dependencies, how Mike built a business off of his open source project, Sidekiq, and the dIfference in maintaining Ruby vs Go software projects.Helpful LinksMike on GithubMike on Twittermikeperham.comContribsysSidekiq[Book] Rising by Elizabeth Rush[Book] The Watch, Thoroughly Revised by Gene Stone and Stephen PulvirentSubscribe to Maintainable on:Apple PodcastsOvercastSpotifyOr search "Maintainable" wherever you stream your podcasts.

Me and Mike discuss, among other things, good use cases for Sidekiq, deploying Sidekiq to production, and side topics like what the JVM is and what threads are.

Mike Perham is back for his 4th appearance to talk about his new project Faktory, a new background job system that’s aiming to bring the best practices developed over the last five years in Sidekiq to every programming language. We catch up with Mike on the continued success and model of Sidekiq, the future of background jobs, his thoughts on RocksDB in Faktory vs BoltDB, Redis, or SQLite, how he plans to support Sidekiq for the next 10 years, and his thoughts on Faktory being a SaaS option in the future.

Mike Perham is back for his 4th appearance to talk about his new project Faktory, a new background job system that’s aiming to bring the best practices developed over the last five years in Sidekiq to every programming language. We catch up with Mike on the continued success and model of Sidekiq, the future of background jobs, his thoughts on RocksDB in Faktory vs BoltDB, Redis, or SQLite, how he plans to support Sidekiq for the next 10 years, and his thoughts on Faktory being a SaaS option in the future.

Sidekiq creator Mike Perham talks about how he build a sustainble business model on top of an open source project.

Learn how Mike Perham turned his open-source side project into a business, quit his job, and grew revenue to $80,000/mo without making a single hire.

Mike Perham has successfully built a wildly profitable one-man business built on his open source efforts. We discuss what it's like balancing the seemingly opposing forces of open source and a for-profit business, managing support for the open source version as well as paying customers, and how businesses are all-too-willing to pay for things that provide value or help them save time. Special Guest: Mike Perham.

Ben is joined by Mike Perham, author of Sidekiq, to discuss running a business solo, taking a side project from hobby to full time job, and scaling support as an app grows. Upcase FormKeep Indie Hackers- Sidekiq Sidekiq Open Core Software Business Model MikePerham.com Kill Your Dependencies

This week I chat with Mike Perham who is the creator, owner, and operator of Sidekiq. We talk about running a successful open source project, how to make some money and keep your sanity, and a little bit about the future with Crystal. Also, have you heard about Pokemon Go? We'll catch up on GraphQL Summit and talk about being a solo co-host.

This week I chat with Mike Perham who is the creator, owner, and operator of Sidekiq. We talk about running a successful open source project, how to make some money and keep your sanity, and a little bit about the future with Crystal. Also, have you heard about Pokemon Go? We'll catch up on GraphQL Summit and talk about being a solo co-host.

Derek and Sean discuss the left-pad saga, how other programming communities are reacting to it, and what you should learn from it as a library or application author. Bash on Ubuntu on Windows I've Just Liberated My Modules by Azer Koçulu A discussion about the breaking of the internet (Kik's side of the story) by Mike Roberts Kik, left-pad, and npm by Isaac Z. Schlueter from npm npm Package Hijacking: From the Hijackers Perspective by Nathan Johnson Is gem yank a security concern? Kill Your Dependencies by Mike Perham To gem, or not to gem by Elle Meredith changes to npm's unpublish policy by Ashley Williams from npm ApplicationRecord in Rails 5 Thank you to Hired for sponsoring this episode!

Check out Angular Remote Conf and RailsClips!   03:15 - Mike Perham Introduction Twitter GitHub Blog Contributed Systems sidekiq dalli 03:43 - Sidekiq Overview resque JRuby 05:18 - Job Runners vs Queuing Systems, Background Jobs RabbitMQ sneakers 08:47 - Performance celluloid 09:49 - celluloid vs Ruby Threads 11:47 - The GIL (Global Interpreter Lock) 12:49 - Passing Data 14:01 - Performance Boost From Using JRuby? 15:48 - The Actor Model revactor Rubinius girl_friday 20:39 - Sidekiq Roadmap Statistics & History 21:44 - Sidekiq Enterprise 27:58 - Sidekiq vs Resque Scheduled-Jobs 29:50 - Adding Features to Sidekiq 30:28 - “Unique Job” 31:17 - Idempotency Sidekiq Best Practices Page 33:12 - Mixing In Other Data Stores Redis Kafka Apollo 38:42 - Encoding 40:04 - Format 40:36 - The Active Job Adapter 41:23 - Making Open Source Viable and Sustainable 44:04 - Launching An Open Source Project Kickstarter BSD & LGPL Licences Picks Mike Hoye: Citation Needed (David) Code Master (Coraline) Robot Turtles (Coraline) Zalando STUPS (Jessica) Elevator Saga (Chuck) Developer On Fire: Episode 017 - Charles Max Wood - Get Involved and Try New Things (Chuck) Model View Culture (Mike) Plasso (Mike) James Mickens: Not Even Close: The State of Computer Security (with slides) from NDC Conferences (Mike)

Check out Angular Remote Conf and RailsClips!   03:15 - Mike Perham Introduction Twitter GitHub Blog Contributed Systems sidekiq dalli 03:43 - Sidekiq Overview resque JRuby 05:18 - Job Runners vs Queuing Systems, Background Jobs RabbitMQ sneakers 08:47 - Performance celluloid 09:49 - celluloid vs Ruby Threads 11:47 - The GIL (Global Interpreter Lock) 12:49 - Passing Data 14:01 - Performance Boost From Using JRuby? 15:48 - The Actor Model revactor Rubinius girl_friday 20:39 - Sidekiq Roadmap Statistics & History 21:44 - Sidekiq Enterprise 27:58 - Sidekiq vs Resque Scheduled-Jobs 29:50 - Adding Features to Sidekiq 30:28 - “Unique Job” 31:17 - Idempotency Sidekiq Best Practices Page 33:12 - Mixing In Other Data Stores Redis Kafka Apollo 38:42 - Encoding 40:04 - Format 40:36 - The Active Job Adapter 41:23 - Making Open Source Viable and Sustainable 44:04 - Launching An Open Source Project Kickstarter BSD & LGPL Licences Picks Mike Hoye: Citation Needed (David) Code Master (Coraline) Robot Turtles (Coraline) Zalando STUPS (Jessica) Elevator Saga (Chuck) Developer On Fire: Episode 017 - Charles Max Wood - Get Involved and Try New Things (Chuck) Model View Culture (Mike) Plasso (Mike) James Mickens: Not Even Close: The State of Computer Security (with slides) from NDC Conferences (Mike)

Check out Angular Remote Conf and RailsClips!   03:15 - Mike Perham Introduction Twitter GitHub Blog Contributed Systems sidekiq dalli 03:43 - Sidekiq Overview resque JRuby 05:18 - Job Runners vs Queuing Systems, Background Jobs RabbitMQ sneakers 08:47 - Performance celluloid 09:49 - celluloid vs Ruby Threads 11:47 - The GIL (Global Interpreter Lock) 12:49 - Passing Data 14:01 - Performance Boost From Using JRuby? 15:48 - The Actor Model revactor Rubinius girl_friday 20:39 - Sidekiq Roadmap Statistics & History 21:44 - Sidekiq Enterprise 27:58 - Sidekiq vs Resque Scheduled-Jobs 29:50 - Adding Features to Sidekiq 30:28 - “Unique Job” 31:17 - Idempotency Sidekiq Best Practices Page 33:12 - Mixing In Other Data Stores Redis Kafka Apollo 38:42 - Encoding 40:04 - Format 40:36 - The Active Job Adapter 41:23 - Making Open Source Viable and Sustainable 44:04 - Launching An Open Source Project Kickstarter BSD & LGPL Licences Picks Mike Hoye: Citation Needed (David) Code Master (Coraline) Robot Turtles (Coraline) Zalando STUPS (Jessica) Elevator Saga (Chuck) Developer On Fire: Episode 017 - Charles Max Wood - Get Involved and Try New Things (Chuck) Model View Culture (Mike) Plasso (Mike) James Mickens: Not Even Close: The State of Computer Security (with slides) from NDC Conferences (Mike)

Josh Owens and Ben Strahan speak with Mike Perham and Mark Shropshire.

Mike Perham joined the show to talk about sustaining open source software, living a healthy life, how to treat one another, and more.

Mike Perham joined the show to talk about sustaining open source software, living a healthy life, how to treat one another, and more.

Adam and Jerod talk with Mike Perham about his new project Inspeqtor and his approach to better application infrastructure monitoring.

Adam and Jerod talk with Mike Perham about his new project Inspeqtor and his approach to better application infrastructure monitoring.

Mike Perham - creator of Sidekiq and Sidekiq Pro - talks about the history of Sidekiq, the brand new 3.0 release, the Sidekiq Pro commercial add on, money in open source, and related topics.

Mike Perham - creator of Sidekiq and Sidekiq Pro - talks about the history of Sidekiq, the brand new 3.0 release, the Sidekiq Pro commercial add on, money in open source, and related topics.

Adam Stacoviak and Andrew Thorp wrap-up with Mike Perham on The Changelog #92.

Adam Stacoviak and Andrew Thorp talk with Mike Perham about sustaining open source, sidekiq, message processing with Ruby, and more.

Adam Stacoviak and Andrew Thorp talk with Mike Perham about sustaining open source, sidekiq, message processing with Ruby, and more.