Podcasts about resque

Pour en savoir plus sur Hellodarwin : https://go.hellodarwin.com/hypercroissance?utm_source=helloDarwin&utm_medium=podcast&utm_campaign=grants-hypercroissance2800 subventions. 6 millions obtenus !Dans cet épisode d'Hypercroissance, Francis Desrochers, président de helloDarwin, et Antoine Gagné discutent des meilleures stratégies pour préparer son entreprise à l'exportation face à la crise des tarifs douaniers.

In this interview with Rosa Gutierrez of 37signals, Rosa elaborates on her work, particularly her contributions to Solid Queue and its implementation alongside Active Job. She describes the motivations behind developing Solid Queue, addressing performance issues with existing systems like Resque, and transitioning some workloads to database-backed queues. Rosa also explains the technical details of Solid Queue's architecture, including supervisor and dispatcher roles, how it handles job prioritization, concurrency, and the challenges of ensuring performance in high-load environments. Essential advice is provided on when to use or avoid Solid Queue, depending on specific needs and existing setups.Solid Queue37signals

Producer Sam goes around the world to bring you all the important stories not making the mainstream headlines. Learn more about your ad choices. Visit megaphone.fm/adchoices

Today on Elixir Wizards Office Hours, SmartLogic Engineer Joel Meador joins Dan Ivovich to discuss all things background jobs. The behind-the-scenes heroes of app performance and scalability, background jobs take center stage as we dissect their role in optimizing user experience and managing heavy-lifting tasks away from the main application flow. From syncing with external systems to processing large datasets, background jobs are pivotal to successful application management. Dan and Joel share their perspectives on monitoring, debugging, and securing background jobs, emphasizing the need for a strategic approach to these hidden workflows. Key topics discussed in this episode: The vital role of background jobs in app performance Optimizing user experience through background processing Common pitfalls: resource starvation and latency issues Strategies for effective monitoring and debugging of task runners and job schedulers Data integrity and system security in open source software Background job tools like Oban, Sidekiq, Resque, Cron jobs, Redis pub sub CPU utilization and processing speed Best practices for implementing background jobs Keeping jobs small, focused, and well-monitored Navigating job uniqueness, locking, and deployment orchestration Leveraging asynctask for asynchronous operations The art of continuous improvement in background job management Links mentioned in this episode: https://redis.io/ Oban job processing library https://hexdocs.pm/oban/Oban.html Resque Ruby library for background jobs https://github.com/resque Sidekiq background processing for Ruby https://github.com/sidekiq Delayed Job priority queue system https://github.com/collectiveidea/delayed_job RabbitMQ messaging and streaming broker https://www.rabbitmq.com/ Mnesia distributed telecommunications DBMS https://www.erlang.org/doc/man/mnesia.html Task for Elixir https://hexdocs.pm/elixir/1.12/Task.html ETS in-memory store for Elixir and Erlang objects https://hexdocs.pm/ets/ETS.html Cron - https://en.wikipedia.org/wiki/Cron Donate to Miami Indians of Indiana https://www.miamiindians.org/take-action Joel Meador on Tumblr https://joelmeador.tumblr.com/ Special Guest: Joel Meador.

A nova edição do Entender Direito tem como foco as dúvidas jurídicas recorrentes sobre condomínio edilício, que resulta do desmembramento de espaços compartilhados em partes de propriedade exclusiva e de propriedade comum dos condôminos. Participaram da entrevista os convidados Andréa Resque e Francisco Egito, ambos professores e advogados especializados em direito condominial e imobiliário. Na conversa conduzida pela jornalista Fátima Uchôa, foram discutidas as responsabilidades do síndico, as controvérsias em torno dos aluguéis de curta temporada via plataformas digitais e as restrições a condôminos inadimplentes, entre outros assuntos. Grandes temas jurídicos e as teses do STJ Entender Direito é um programa quinzenal que aborda discussões relevantes no meio jurídico, com a participação de juristas e operadores do direito debatendo cada tema à luz da legislação e da jurisprudência do STJ.   Confira a entrevista na TV Justiça, às quartas-feiras, às 10h, com reprises aos sábados, às 14h, e às terças, às 22h. Na Rádio Justiça (104,7 FM – Brasília), o programa é apresentado de forma inédita aos sábados, às 7h, com reprise aos domingos no mesmo horário.     Também está disponível no canal do STJ no YouTube e nas principais plataformas de podcast, como Spotify e SoundCloud.    

On this episode of The Barbeque Life Kurt talks with Josh Hall out of Cumming, GA and is the Co-Owner of ResQue Rubs, a BBQ rub business that donates 10% of their proceeds to Fur Kids, a no kill shelter for cats and dogs about how COVID gave them the idea to start the ResQue Rubs, how they have built the business, gotten customers, and spread the word about their company, how they came up with the names of products, get feedback, deal with adversity, and so much more. “[Social media] is a necessary evil. It's free advertising. You gotta do it.” Watch on YouTube here: https://youtu.be/sXjWuahEMV8 Follow and connect with ResQue Rubs at the links below! Website: https://www.resquerubs.com/ Instagram: https://www.instagram.com/resquerubs/ Facebook: https://www.facebook.com/resquerubs TikTok: https://www.tiktok.com/@resquerubs_1117 Fur Kids: https://www.instagram.com/furkidsatlanta/ Because Coffee: https://because.coffee/ Leave us a review on Apple Podcasts, Spotify, or Facebook! Apple Podcasts: https://apple.co/3YkKv7h Spotify: http://bit.ly/3RvpAwc Facebook: https://www.facebook.com/thebarbequelife/reviews Follow us on Facebook, Instagram, TikTok, and Twitter: https://linktr.ee/thebarbequelife Want to reach us? Email us at kurt@thebarbequelife.com KEEP CALM AND SMOKE ON!  --- Send in a voice message: https://podcasters.spotify.com/pod/show/thebarbequelife/message

Mike Perham is the creator of Sidekiq, a background job processor for Ruby. He's also the creator of Faktory a similar product for multiple language environments. We talk about the RubyConf keynote and Ruby's limitations, supporting products as a solo developer, and some ideas for funding open source like a public utility. Recorded at RubyConf 2023 in San Diego. -- A few topics covered: Sidekiq (Ruby) vs Faktory (Polyglot) Why background job solutions are so common in Ruby Global Interpreter Lock (GIL) Ractors (Actor concurrency) Downsides of Multiprocess applications When to use other languages Getting people to pay for Sidekiq Keeping a solo business Being selective about customers Ways to keep support needs low Open source as a public utility Mike Mike's blog mastodon Sidekiq faktory From Employment to Independence Ruby Ractor The Practical Effects of the GVL on Scaling in Ruby Transcript You can help correct transcripts on GitHub. Introduction [00:00:00] Jeremy: I'm here at RubyConf San Diego with Mike Perham. He's the creator of Sidekiq and Faktory. [00:00:07] Mike: Thank you, Jeremy, for having me here. It's a pleasure. Sidekiq [00:00:11] Jeremy: So for people who aren't familiar with, I guess we'll start with Sidekiq because I think that's what you're most known for. If people don't know what it is, maybe you can give like a small little explanation. [00:00:22] Mike: Ruby apps generally have two major pieces of infrastructure powering them. You've got your app server, which serves your webpages and the browser. And then you generally have something off on the side that... It processes, you know, data for a million different reasons, and that's generally called a background job framework, and that's what Sidekiq is. [00:00:41] It, Rails is usually the thing that, that handles your web stuff, and then Sidekiq is the Sidekiq to Rails, so to speak. [00:00:50] Jeremy: And so this would fit the same role as, I think in Python, there's celery. and then in the Ruby world, I guess there is, uh, Resque is another kind of job. [00:01:02] Mike: Yeah, background job frameworks are quite prolific in Ruby. the Ruby community's kind of settled on that as the, the standard pattern for application development. So yeah, we've got, a half a dozen to a dozen different, different examples throughout history, but the major ones today are, Sidekiq, Resque, DelayedJob, GoodJob, and, and, and others down the line, yeah. Why background jobs are so common in Ruby [00:01:25] Jeremy: I think working in other languages, you mentioned how in Ruby, there's this very clear, preference to use these job scheduling systems, these job queuing systems, and I'm not. I'm not sure if that's as true in, say, if somebody's working in Java, or C sharp, or whatnot. And I wonder if there's something specific about Ruby that makes people kind of gravitate towards this as the default thing they would use. [00:01:52] Mike: That's a good question. What makes Ruby... The one that so needs a background job system. I think Ruby, has historically been very single threaded. And so, every Ruby process can only do so much work. And so Ruby oftentimes does, uh, spin up a lot of different processes, and so having processes that are more focused on one thing is, is, is more standard. [00:02:24] So you'll have your application server processes, which focus on just serving HTTP responses. And then you have some other sort of focused process and that just became background job processes. but yeah, I haven't really thought of it all that much. But, uh, you know, something like Java, for instance, heavily multi threaded. [00:02:45] And so, and extremely heavyweight in terms of memory and startup time. So it's much more frequent in Java that you just start up one process and that's it. Right, you just do everything in that one process. And so you may have dozens and dozens of threads, both serving HTTP and doing work on the side too. Um, whereas in Ruby that just kind of naturally, there was a natural split there. Global Interpreter Lock [00:03:10] Jeremy: So that's actually a really good insight, because... in the keynote at RubyConf, Mats, the creator of Ruby, you know, he mentioned the, how the fact that there is this global, interpreter lock, [00:03:23] or, or global VM lock in Ruby, and so you can't, really do multiple things in parallel and make use of all the different cores. And so it makes a lot of sense why you would say like, okay, I need to spin up separate processes so that I can actually take advantage of, of my, system. [00:03:43] Mike: Right. Yeah. And the, um, the GVL. is the acronym we use in the Ruby community, or GIL. Uh, that global lock really kind of is a forcing function for much of the application architecture in Ruby. Ruby, uh, applications because it does limit how much processing a single Ruby process can do. So, uh, even though Sidekiq is heavily multi threaded, you can only have so many threads executing. [00:04:14] Because they all have to share one core because of that global lock. So unfortunately, that's, that's been, um, one of the limiter, limiting factors to Sidekiq scalability is that, that lock and boy, I would pay a lot of money to just have that lock go away, but. You know, Python is going through a very long term experiment about trying to remove that lock and I'm very curious to see how well that goes because I would love to see Ruby do the same and we'll see what happens in the future, but, it's always frustrating when I come to another RubyConf and I hear another Matt's keynote where he's asked about the GIL and he continues to say, well, the GIL is going to be around, as long as I can tell. [00:04:57] so it's a little bit frustrating, but. It's, it's just what you have to deal with. Ractors [00:05:02] Jeremy: I'm not too familiar with them, but they, they did mention during the keynote I think there Ractors or something like that. There, there, there's some way of being able to get around the GIL but there are these constraints on them. And in the context of Sidekiq and, and maybe Ruby in general, how do you feel about those options or those solutions? [00:05:22] Mike: Yeah, so, I think it was Ruby 3. 2 that introduced this concept of what they call a Ractor, which is like a thread, except it does not have the global lock. It can run independent to the global lock. The problem is, is because it doesn't use the global lock, it has pretty severe constraints on what it can do. [00:05:47] And the, and more specifically, the data it can access. So, Ruby apps and Rails apps throughout history have traditionally accessed a lot of global data, a lot of class level data, and accessed all this data in a, in a read only fashion. so there's no race conditions because no one's changing any of it, but it's still, lots of threads all accessing the same variables. [00:06:19] Well, Ractors can't do that at all. The only data Ractors can access is data that they own. And so that is completely foreign to Ruby application, traditional Ruby applications. So essentially, Ractors aren't compatible with the vast majority of existing Ruby code. So I, I, I toyed with the idea of prototyping Sidekiq and Ractors, and within about a minute or two, I just ran into these, these, uh... [00:06:51] These very severe constraints, and so that's why you don't see a lot of people using Ractors, even still, even though they've been out for a year or two now, you just don't see a lot of people using them, because they're, they're really limited, limited in what they can do. But, on the other hand, they're unlimited in how well they can scale. [00:07:12] So, we'll see, we'll see. Hopefully in the future, they'll make a lot of improvements and, uh, maybe they'll become more usable over time. Downsides of multiprocess (Memory usage) [00:07:19] Jeremy: And with the existence of a job queue or job scheduler like Sidekiq, you're able to create additional processes to get around that global lock, I suppose. What are the... downsides of doing so versus another language like we mentioned Java earlier, which is capable of having true parallelism in the same process. [00:07:47] Mike: Yeah, so you can start up multiple Ruby processes to process things truly in parallel. The issue is that you do get some duplication in terms of memory. So your Ruby app maybe take a gigabyte per process. And, you can do copy on write forking. You can fork and get some memory sharing with copy on write semantics on Unix operating systems. [00:08:21] But you may only get, let's say, 30 percent memory savings. So, there's still a significant memory overhead to forking, you know, let's say, eight processes versus having eight threads. You know, you, you, you may have, uh, eight threads can operate in a gigabyte process, but if you want to have eight processes, that may take, let's say, four gigabytes of RAM. [00:08:48] So you, you still, it's not going to cost you eight gigabytes of RAM, you know, it's not like just one times eight, but, there's still a overhead of having those separate processes. [00:08:58] Jeremy: would you say it's more of a cost restriction, like it costs you more to run these applications, or are there actual problems that you can't solve because of this restriction. [00:09:13] Mike: Help me understand, what do you mean by restriction? Do you mean just the GVL in general, or the fact that forking processes still costs memory? [00:09:22] Jeremy: I think, well, it would be both, right? So you're, you have two restrictions right now. You have the, the GVL, which means you can't have parallelism within the same process. And then your other option is to spin up a bunch of processes, which you have said is the downside there is that you're using a lot more RAM. [00:09:43] I suppose my question is that Does that actually stop you from doing anything? Like, if you throw more money at the problem, you go like, we're going to have more instances, I'll pay for the RAM, it's fine, can that basically get you out of these situations or are these limitations actually stopping you from, from doing things you could do in other languages? [00:10:04] Mike: Well, you certainly have to manage the multiple processes, right? So you've gotta, you know, if one child process crashes, you've gotta have a parent or supervisor process watching all that and monitoring and restarting the process. I don't think it restricts you. Necessarily, it just, it adds complexity to your deployment. [00:10:24] and, and it's just a question of efficiency, right? Instead of being able to deploy on a, on a one gigabyte droplet, I've got to deploy to a four gigabyte droplet, right? Because I just, I need the RAM to run the eight processes. So it, it, it's more of just a purely a function of how much money am I going to have to throw at this problem. [00:10:45] And what's it going to cost me in operational costs to operate this application in production? When to use other languages? [00:10:53] Jeremy: So during the. Keynote, uh, Matz had mentioned that Rails, is really suitable as this one person framework, like you can have a very small team or maybe even yourself and, and build this product. And so I guess from... Your perspective, once you cross a certain threshold, is like, what Ruby and what Sidekiq provides not enough, and that's why you need to start looking into other languages? [00:11:24] Or like, where's the, turning point, or the, if you [00:11:29] Mike: Right, right. The, it's all about the problem you're trying to solve, right? At the end of the day, uh, the, the question is just what are we trying to solve and how are we trying to solve it? So at a higher level, you got to think about the architecture. if the problem you're trying to solve, if the service you're trying to build, if the app you're trying to operate. [00:11:51] If that doesn't really fall into the traditional Ruby application architecture, then you, you might look at it in another language or another ecosystem. something like Go, for instance, can compile down to a single binary, which makes deployment really easy. It makes shipping up a product. on to a user's machine, much simpler than deploying a Ruby application onto a user's desktop machine, for instance, right? [00:12:22] Um, Ruby does have this, this problem of how do you package everything together and deploy it somewhere? Whereas Go, when you can just compile to a single binary, now you've just got a single thing. And it's just... Drop it on the file system and execute it. It's easy. So, um, different, different ecosystems have different application architectures, which empower different ways of solving the same problems. [00:12:48] But, you know, Rails as a, as a one man framework, or sorry, one person framework, It, it, I don't, I don't necessarily, that's a, that's sort of a catchy marketing slogan, but I just think of Rails as the most productive framework you can use. So you, as a single person, you can maximize what you ship and the, the, the value that you can create because Rails is so productive. [00:13:13] Jeremy: So it, seems like it's maybe the, the domain or the type of application you're making. Like you mentioned the command line application, because you want to be able to deliver it to your user easily. Just give them a binary, something like Go or perhaps Rust makes a lot more sense. and then I could see people saying that if you're doing something with machine learning, like the community behind Python, it's, they're just, they're all there. [00:13:41] So Room for more domains in Ruby [00:13:41] Mike: That was exactly the example I was going to use also. Yeah, if you're doing something with data or AI, Python is going to be a more, a more traditional, natural choice. that doesn't mean Ruby can't do it. That doesn't mean, you wouldn't be able to solve the problem with Ruby. And, and there's, that just also means that there's more space for someone who wants to come in and make an impact in the Ruby community. [00:14:03] Find a problem that Ruby's not really well suited to solving right now and build the tooling out there to, to try and solve it. You know, I, I saw a talk, from the fellow who makes the Glimmer gem, which is a native UI toolkit. Uh, a gem for building native UIs in Ruby, which Ruby traditionally can't do, but he's, he's done an amazing job at sort of surfacing APIs to build these, um, these native, uh, native applications, which I think is great. [00:14:32] It's awesome. It's, it's so invigorating to see Ruby in a new space like that. Um, I talked to someone else who's doing the Polars gem, which is focused on data processing. So it kind of takes, um, Python and Pandas and brings that to Ruby, which is, is awesome because if you're a Ruby developer, now you've got all these additional tools which can allow you to solve new sets of problems out there. [00:14:57] So that's, that's kind of what's exciting in the Ruby community right now is just bring it into new spaces. Faktory [00:15:03] Jeremy: In addition to Sidekiq, you have, uh, another product called Faktory, I believe. And so does that serve a, a similar purpose? Is that another job scheduling, job queueing system? [00:15:16] Mike: It is, yes. And it's, it's, it's similar in a way to Sidekiq. It looks similar. It's got similar concepts at the core of it. At the end of the day, Sidekiq is limited to Ruby. Because Sidekiq executes in a Ruby VM, it executes the jobs, and the jobs are, have to be written in Ruby because you're running in the Ruby VM. [00:15:38] Faktory was my attempt to bring, Sidekiq functionality to every other language. I wanted, I wanted Sidekiq for JavaScript. I wanted Sidekiq for Go. I wanted Sidekiq for Python because A, a lot of these other languages also could use a system, a background job system. And the problem though is that. [00:16:04] As a single man, I can't port Sidekiq to every other language. I don't know all the languages, right? So, Faktory kind of changes the architecture and, um, allows you to execute jobs in any language. it, it replaces Redis and provides a server where you just fetch jobs, and you can use it from it. [00:16:26] You can use that protocol from any language to, to build your own worker processes that execute jobs in whatever language you want. [00:16:35] Jeremy: When you say it replaces Redis, so it doesn't use Redis, um, internally, it has its own. [00:16:41] Mike: It does use Redis under the covers. Yeah, it starts Redis as a child process and, connects to it over a Unix socket. And so it's really stable. It's really fast. from the outside, the, the worker processes, they just talk to Faktory. They don't know anything about Redis at all. [00:16:59] Jeremy: I see. And for someone who, like we mentioned earlier in the Python community, for example, there is, um, Celery. For someone who is using a task scheduler like that, what's the incentive to switch or use something different? [00:17:17] Mike: Well, I, I always say if you're using something right now, I'm not going to try and convince you to switch necessarily. It's when you have pain that you want to switch and move away. Maybe you have Maybe there's capabilities in the newer system that you really need that the old system doesn't provide, but Celery is such a widely known system that I'm not necessarily going to try and convince people to move away from it, but if people are looking for a new system, one of the things that Celery does that Faktory does not do is Celery provides like data adapters for using store, lots of different storage systems, right? [00:17:55] Faktory doesn't do that. Faktory is more, has more of the Rails mantra of, you know, Omakase where we choose, I choose to use Redis and that's it. You don't, you don't have a choice for what to use because who cares, you know, at the end of the day, let Faktory deal with it. it's, it's not something that, You should even necessarily be concerned about it. [00:18:17] Just, just try Faktory out and see how it works for you. Um, so I, I try to take those operational concerns off the table and just have the user focus on, you know, usability, performance, and that sort of thing. but it is, it's, it's another background job system out there for people to try out and see if they like that. [00:18:36] And, and if they want to, um, if they know Celery and they want to use Celery, more power to Faktory them. Sidekiq (Ruby) or Faktory (Polyglot) [00:18:43] Jeremy: And Sidekiq and Faktory, they serve a very similar purpose. For someone who they have a new project, they haven't chosen a job. scheduling system, if they were using Ruby, would it ever make sense for them to use Faktory versus use Sidekiq? [00:19:05] Mike: Uh Faktory is excellent in a polyglot situation. So if you're using multiple languages, if you're creating jobs in Ruby, but you're executing them in Python, for instance, um, you know, if you've, I have people who are, Creating jobs in PHP and executing them in Python, for instance. That kind of polyglot scenario, Sidekiq can't do that at all. [00:19:31] So, Faktory is useful there. In terms of Ruby, Ruby is just another language to Faktory. So, there is a Ruby API for using Faktory, and you can create and execute Ruby jobs with Faktory. But, you'll find that in the Ruby community, Sidekiq is much widely... much more widely used and understood and known. So if you're just using Ruby, I think, I think Sidekiq is the right choice. [00:19:59] I wouldn't look at Faktory. But if you do need, find yourself needing that polyglot tool, then Faktory is there. Temporal [00:20:07] Jeremy: And this is maybe one, maybe one layer of abstraction higher, but there's a product called Temporal that has some of this job scheduling, but also this workflow component. I wonder if you've tried that out and how you think about that product? [00:20:25] Mike: I've heard of them. I don't know a lot about the product. I do have a workflow API, the Sidekiq batches, which allow you to fan out jobs and then, and then execute callbacks when all the jobs in that, in that batch are done. But I don't, provide sort of a, a high level. Graphical Workflow Editor or anything like that. [00:20:50] Those to me are more marketing tools that you use to sell the tool for six figures. And I don't think they're usable. And I don't think they're actually used day to day. I provide an API for developers to use. And developers don't like moving blocks of code around in a GUI. They want to write code. And, um, so yeah, temporal, I, like I said, I don't know much about them. [00:21:19] I also, are they a venture capital backed startup? [00:21:22] Jeremy: They are, is my understanding, [00:21:24] Mike: Yeah, that, uh, any, any sort of venture capital backed startup, um, who's building technical infrastructure. I, I would look long and hard at, I'm, I think open source is the right core to build on. Of course I sell commercial software, but. I'm bootstrapped. I'm profitable. [00:21:46] I'm going to be around forever. A VC backed startup, they tend to go bankrupt, because they either get big or they go out of business. So that would be my only comment is, is, be a little bit leery about relying on commercial venture capital based infrastructure for, for companies, uh, long term. Getting people to pay for Sidekiq [00:22:05] Jeremy: So I think that's a really interesting part about your business is that I think a lot of open source maintainers have a really big challenge figuring out how to make it as a living. The, there are so many projects that they all have a very permissive license and you can use them freely one example I can think of is, I, I talked with, uh, David Kramer, who's the CTO at Sentry, and he, I don't think they use it anymore, but they, they were using Nginx, right? [00:22:39] And he's like, well, Nginx, they have a paid product, like Nginx. Plus that or something. I don't know what the name is, but he was like, but I'm not going to pay for it. Right. I'm just going to use the free one. Why would I, you know, pay for the, um, the paid thing? So I, I, I'm kind of curious from your perspective when you were coming up with Sidekiq both as an open source product, but also as a commercial one, how did you make that determination of like to make a product where it's going to be useful in its open source form? [00:23:15] I can still convince people to pay money for it. [00:23:19] Mike: Yeah, the, I was terrified, to be blunt, when I first started out. when I started the Sidekiq project, I knew it was going to take a lot of time. I knew if it was successful, I was going to be doing it for the next decade. Right? So I started in 2012, and here I am in 2023, over a decade, and I'm still doing it. [00:23:38] So my expectation was met in that regard. And I knew I was not going to be able to last that long. If I was making zero dollars, right? You just, you burn out. Nobody can last that long. Well, I guess there are a few exceptions to that rule, but yeah, money, I tend to think makes things a little more sustainable for sure. [00:23:58] Especially if you can turn it into a full time job solving and supporting a project that you, you love and, and is, is, you know, your, your, your baby, your child, so to speak, your software, uh, uh, creation that you've given to the world. but I was terrified. but one thing I did was at the time I was blogging a lot. [00:24:22] And so I was telling people about Sidekiq. I was telling people what was to come. I was talking about ideas and. The one thing that I blogged about was financial experiments. I said bluntly to the, to, to the Ruby community, I'm going to be experimenting with financial stability and sustainability with this project. [00:24:42] So not only did I create this open source project, but I was also publicly saying I I need to figure out how to make this work for the next decade. And so eventually that led to Sidekiq Pro. And I had to figure out how to build a closed source Ruby gem, which, uh, There's not a lot of, so I was kind of in the wild there. [00:25:11] But, you know, thankfully all the pieces came together and it was actually possible. I couldn't have done it if it wasn't possible. Like, we would not be talking if I couldn't make a private gem. So, um, but it happened to work out. Uh, and it allowed me to, to gate features behind a paywall effectively. And, and yeah, you're right. [00:25:33] It can be tough to make people pay for software. but I'm a developer who's selling to other developers, not, not just developers, open source developers, and they know that they have this financial problem, right? They know that there's this sustainability problem. And I was blunt in saying, this is my solution to my sustainability. [00:25:56] So, I charge what I think is a very fair price. It's only a thousand dollars a year to a hobbyist. That may seem like a lot of money to a business. It's a drop in the bucket. So it was easy for developers to say, Hey, listen, we want to buy this tool for a thousand bucks. It'll ensure our infrastructure is maintained for the next decade. [00:26:18] And it's, and it's. And it's relatively cheap. It's way less than, uh, you know, a salary or even a laptop. So, so that's, that's what I did. And, um, it's, it worked out great. People, people really understood. Even today, I talk to people and they say, we, we signed up for Sidekiq Pro to support you. So it's, it's, it's really, um, invigorating to hear people, uh, thank me and, and they're, they're actively happy that they're paying me and our customers. [00:26:49] Jeremy: it's sort of, uh, maybe a not super common story, right, in terms of what you went through. Because when I think of open core businesses, I think of companies like, uh, GitLab, which are venture funded, uh, very different scenario there. I wonder, like, in your case, so you started in 2012, and there were probably no venture backed competitors, right? [00:27:19] People saying that we're going to make this job scheduling system and some VC is going to give me five million dollars and build a team to work on this. It was probably at the time, maybe it was Rescue, which was... [00:27:35] Mike: There was a venture backed system called IronMQ, [00:27:40] Jeremy: Hmm. [00:27:41] Mike: And I'm not sure if they're still around or not, but they... They took, uh, one or more funding rounds. I'm not sure exactly, but they were VC backed. They were doing, background jobs, scheduled jobs, uh, you know, running container, running container jobs. They, they eventually, I think, wound up sort of settling on Docker containers. [00:28:06] They'll basically spin up a Docker container. And that container can do whatever it wants. It can execute for a second and then shut down, or it can run for, for however long, but they would, um, yeah, I, yeah, I'll, I'll stop there because I don't know the actual details of exactly their system, but I'm not sure if they're still around, but that's the only one that I remember offhand that was around, you know, years ago. [00:28:32] Yeah, it's, it's mostly, you know, low level open source infrastructure. And so, anytime you have funded startups, they're generally using that open source infrastructure to build their own SaaS. And so SaaS's are the vast majority of where you see sort of, uh, commercial software. [00:28:51] Jeremy: so I guess in that way it, it, it gave you this, this window or this area where you could come in and there wasn't, other than that iron, product, there wasn't this big money that you were fighting against. It was sort of, it was you telling people openly, I'm, I'm working on this thing. [00:29:11] I need to make money so that I can sustain it. And, if you, yeah. like the work I do, then, you know, basically support me. Right. And, and so I think that, I'm wondering how we can reproduce that more often because when you see new products, a lot of times it is VC backed, right? [00:29:35] Because people say, I need to work on this. I need to be paid. and I can't ask a team to do this. For nothing, right? So [00:29:44] Mike: Yeah. It's. It's a wicked problem. Uh, it's a really, really hard problem to solve if you take vc you there, that that really kind of means that you need to be making tens if not hundreds of millions of dollars in sales. If you are building a small or relatively small. You know, put small in quotes there because I don't really know what that means, but if you have a small open source project, you can't charge huge amounts for it, right? [00:30:18] I mean, Sidekiq is a, I would call a medium sized open source project, and I'm charging a thousand bucks for it. So if you're building, you know, I don't know, I don't even want to necessarily give example, but if you're building some open source project, and It's one of 300 libraries that people's applications will depend on. [00:30:40] You can't necessarily charge a thousand dollars for that library. depending on the size and the capabilities, maybe you can, maybe you can't. But there's going to be a long tail of open source projects that just, they can't, they can't charge much, if anything, for them. So, unfortunately, we have, you know, these You kind of have two pathways. [00:31:07] Venture capital, where you've got to sell a ton, or free. And I've kind of walked that fine line where I'm a small business, I can charge a small amount because I'm bootstrapped. And, and I don't need huge amounts of money, and I, and I have a project that is of the right size to where I can charge a decent amount of money. [00:31:32] That means that I can survive with 500 or a thousand customers. I don't need to have a hundred million dollars worth of customers. Because I, you know, when I started the business, one of the constraints I said is I don't want to hire anybody. I'm just going to be solo. And part of the, part of my ability to keep a low price and, and keep running sustainably, even with just You know, only a few hundred customers is because I'm solo. [00:32:03] I don't have the overhead of investors. I don't have the overhead of other employees. I don't have an office space. You know, my overhead is very small. So that is, um, you know, I just kind of have a unique business in that way, I guess you might say. Keeping the business solo [00:32:21] Jeremy: I think that's that's interesting about your business as well But the fact that you've kept it you've kept it solo which I would imagine in most businesses, they need support people. they need, developers outside of maybe just one. Um, there's all sorts of other, I don't think overhead is the right word, but you just need more people, right? [00:32:45] And, and what do you think it is about Sidekiq that's made it possible for it to just be a one person operation? [00:32:52] Mike: There's so much administrative overhead in a business. I explicitly create business policies so that I can run solo. you know, my support policy is officially you get one email ticket or issue per quarter. And, and anything more than that, I can bounce back and say, well, you're, you're requiring too much support. [00:33:23] In reality, I don't enforce that at all. And people email me all the time, but, but things like. Things like dealing with accounting and bookkeeping and taxes and legal stuff, licensing, all that is, yeah, a little bit of overhead, but I've kept it as minimal as I can. And part of that is I don't want to hire another employee because then that increases the administrative overhead that I have. [00:33:53] And Sidekiq is so tied to me and my knowledge that if I hire somebody, they're probably not going to know Ruby and threading and all the intricate technical detail necessary to build and maintain and support the system. And so really you'll kind of regress a little bit. We won't be able to give as good support because I'm busy helping that other employee. Being selective about customers [00:34:23] Mike: So, yeah, it's, it's a tightrope act where you've got to really figure out how can I scale myself as far as possible without overwhelming myself. The, the overwhelming thing that I have that I've never been able to solve. It's just dealing with billing inquiries, customers, companies, emailing me saying, how do we buy this thing? [00:34:46] Can I get an invoice? Every company out there, it seems wants an invoice. And the problem with invoicing is it takes a lot more. manual labor and administrative overhead to issue that invoice to collect payment on the invoice. So that's one of the reasons why I have a very strict policy about credit card only for, for the vast majority of my customers. [00:35:11] And I demand that companies pay a lot more. You have to have a pretty big enterprise license if you want an invoice. And if the company, if the company comes back and complains and says, well, you know, that's ridiculous. We don't, we don't want to pay that much. We don't need it that much. Uh, you know, I, I say, okay, well then you have two, two things, two, uh, two things. [00:35:36] You can either pay with a credit card or you can not use Sidekiq. Like, that's, that's it. I'm, I don't need your money. I don't want the administrative overhead of dealing with your accounting department. I just want to support my, my customers and build my software. And, and so, yeah, I don't want to turn into a billing clerk. [00:35:55] So sometimes, sometimes the, the, the best thing in business that you can do is just say no. [00:36:01] Jeremy: That's very interesting because I think being a solo... Person is what probably makes that possible, right? Because if you had the additional staff, then you might say like, Well, I need to pay my staff, so we should be getting, you know, as much business as [00:36:19] Mike: Yeah. Chasing every customer you can, right. But yeah. [00:36:22] Every customer is different. I mean, I have some customers that just, they never contact me. They pay their bill really fast or right on time. And they're paying me, you know, five figures, 20, a year. And they just, it's a, God bless them because those are, are the. [00:36:40] Best customers to have and the worst customers are the ones who are paying 99 bucks a month and everything that they don't understand or whatever is a complaint. So sometimes, sometimes you, you want to, vet your customers from that perspective and say, which one of these customers are going to be good? [00:36:58] Which ones are going to be problematic? [00:37:01] Jeremy: And you're only only person... And I'm not sure how many customers you have, but [00:37:08] Mike: I have 2000 [00:37:09] Jeremy: 2000 customers. [00:37:10] Okay. [00:37:11] Mike: Yeah. [00:37:11] Jeremy: And has that been relatively stable or has there been growth [00:37:16] Mike: It's been relatively stable the last couple of years. Ruby has, has sort of plateaued. Um, it's, you don't see a lot of growth. I'm getting probably, um, 15, 20 percent growth maybe. Uh, so I'm not growing like a weed, like, you know, venture capital would want to see, but steady incremental growth is, is, uh, wonderful, especially since I do very little. [00:37:42] Sales and marketing. you know, I come to RubyConf I, I I tweet out, you know, or I, I toot out funny Mastodon Toots occasionally and, and, um, and, and put out new releases of the software. And, and that's, that's essentially my, my marketing. My marketing is just staying in front of developers and, and, and being a presence in the Ruby community. [00:38:06] But yeah, it, it's, uh. I, I, I see not a, not a huge amount of churn, but I see enough sales to, to, to stay up and keep my head above water and to keep growing, um, slowly but surely. Support needs haven't grown [00:38:20] Jeremy: And as you've had that steady growth, has the support burden not grown with it? [00:38:27] Mike: Not as much because once customers are on Sidekiq and they've got it working, then by and large, you don't hear from them all that much. There's always GitHub issues, you know, customers open GitHub issues. I love that. but yeah, by and large, the community finds bugs. and opens up issues. And so things remain relatively stable. [00:38:51] I don't get a lot of the complete newbie who has no idea what they're doing and wants me to, to tell them how to use Sidekiq that I just don't see much of that at all. Um, I have seen it before, but in that case, generally, I, I, I politely tell that person that, listen, I'm not here to educate you on the product. [00:39:14] It's there's documentation in the wiki. Uh, and there's tons of, of more Ruby, generic Ruby, uh, educational material out there. That's just not, not what I do. So, so yeah, by and large, the support burden is, is not too bad because once people are, are up and running, it's stable and, and they don't, they don't need to contact me. [00:39:36] Jeremy: I wonder too, if that's perhaps a function of the price, because if you're a. new developer or someone who's not too familiar with how to do job processing or what they want to do when you, there is the open source product, of course. but then the next step up, I believe is about a hundred dollars a month. [00:39:58] And if you're somebody who is kind of just getting started and learning how things work, you're probably not going to pay that, is my guess. And so you'll never hear from them. [00:40:11] Mike: Right, yeah, that's a good point too, is the open source version, which is what people inevitably are going to use and integrate into their app at first. Because it's open source, you're not going to email me directly, um, and when people do email me directly, Sidekiq support questions, I do, I reply literally, I'm sorry I don't respond to private email, unless you're a customer. [00:40:35] Please open a GitHub issue and, um, that I try to educate both my open source users and my commercial customers to try and stay in GitHub issues because private email is a silo, right? Private email doesn't help anybody else but them. If I can get people to go into GitHub issues, then that's a public record. [00:40:58] that people can search. Because if one person has that problem, there's probably a dozen other people that have that same problem. And then that other, those other 11 people can search and find the solution to their problem at four in the morning when I'm asleep. Right? So that's, that's what I'm trying to do is, is keep, uh, keep everything out in the open so that people can self service as much as possible. Sidekiq open source [00:41:24] Jeremy: And on the open source side, are you still primarily the main contributor? Or do you have other people that are [00:41:35] Mike: I mean, I'd say I do 90 percent of the work, which is why I don't feel guilty about keeping 100 percent of the money. A lot of open source projects, when they look for financial sustainability, they also look for how can we split this money amongst the team. And that's, that's a completely different topic that I've. [00:41:55] is another reason why I've stayed solo is if I hire an employee and I pay them 200, 000 a year as a developer, I'm meanwhile keeping all the rest of the profits of the company. And so that almost seems a little bit unfair. because we're both still working 40 hours a week, right? Why am I the one making the vast majority of the, of the profit and the money? [00:42:19] Um, so, uh, I've always, uh, that's another reason why I've stayed solo, but, but yeah, having a team of people working on something, I do get, regular commits, regular pull requests from people, fixing a bug that they found or just making a tweak that. that they saw, that they thought they could improve. [00:42:42] A little more rarely I get a significant improvement or feature, as a pull request. but Sidekiq is so stable these days that it really doesn't need a team of people maintaining it. The volume of changes necessary, I can easily keep up with that. So, I'm still doing 90 95 percent of the work. Are there other Sidekiq-like opportunities out there? [00:43:07] Jeremy: Yeah, so I think Sidekiq has sort of a unique positioning where it's the code base itself is small enough where you can maintain it yourself and you have some help, but primarily you're the main maintainer. And then you have enough customers who are willing to, to pay for the benefit it gives them on top of what the open source product provides. [00:43:36] cause it's, it's, you were talking about how. Every project people work on, they have, they could have hundreds of dependencies, right? And to ask somebody to, to pay for each of them is, is probably not ever going to happen. And so it's interesting to think about how you have things like, say, you know, OpenSSL, you know, it's a library that a whole bunch of people rely on, but nobody is going to pay a monthly fee to use it. [00:44:06] You have things like, uh, recently there was HashiCorp with Terraform, right? They, they decided to change their license because they, they wanted to get, you know, some of that value back, some of the money back, and the community basically revolted. Right? And did a fork. And so I'm kind of curious, like, yeah, where people can find these sweet spots like, like Sidekiq, where they can find this space where it's just small enough where you can work on it on your own and still get people to pay for it. [00:44:43] It's, I'm trying to picture, like, where are the spaces? Open source as a public utility [00:44:48] Mike: We need to look at other forms of financing beyond pure capitalism. If this is truly public infrastructure that needs to be maintained for the long term, then why are we, why is it that we depend on capitalism to do that? Our roads, our water, our sewer, those are not Capitalist, right? Those are utilities, that's public infrastructure that we maintain, that the government helps us maintain. [00:45:27] And in a sense, tech infrastructure is similar or could be thought of in a similar fashion. So things like Open Collective, things like, uh, there's a, there's a organization in Europe called NLNet, I think, out of the Netherlands. And they do a lot of grants to various open source projects to help them improve the state of digital infrastructure. [00:45:57] They support, for instance, Mastodon as a open source project that doesn't have any sort of corporate backing. They see that as necessary social media infrastructure, uh, for the long term. And, and I, and I think that's wonderful. I like to see those new directions being explored where you don't have to turn everything into a product, right? [00:46:27] And, and try and market and sale, um, and, and run ads and, and do all this stuff. If you can just make the case that, hey, this is, this is useful public infrastructure that so many different, um, Technical, uh, you know, applications and businesses could rely on, much like FedEx and DHL use our roads to the benefit of their own, their own corporate profits. [00:46:53] Um, why, why, why shouldn't we think of tech infrastructure sort of in a similar way? So, yeah, I would like to see us explore more. in that direction. I understand that in America that may not happen for quite a while because we are very, capitalist focused, but it's encouraging to see, um, places like Europe, uh, a little more open to, to trialing things like, cooperatives and, and grants and large long term grants to, to projects to see if they can, uh, provide sustainability in, in, you know, in a new way. [00:47:29] Jeremy: Yeah, that's a good point because I think right now, a lot of the open source infrastructure that we all rely on, either it's being paid for by large companies and at the whim of those large companies, if Google decides we don't want to pay for you to work on this project anymore, where does the money come from? [00:47:53] Right? And on the other hand, there's the thousands, tens of thousands of people who are doing it. just for free out of the, you know, the goodness of their, their heart. And that's where a lot of the burnout comes from. Right. So I think what you're saying is that perhaps a lot of these pieces that we all rely on, that our, our governments, you know, here in the United States, but also around the world should perhaps recognize as this is, like you said, this is infrastructure, and we should be. [00:48:29] Paying these people to keep the equivalent of the roads and, and, uh, all that working. [00:48:37] Mike: Yeah, I mean, I'm not, I'm not claiming that it's a perfect analogy. There's, there's, there's lots of questions that are unanswered in that, right? How do you, how do you ensure that a project is well maintained? What does that even look like? What does that mean? you know, you can look at a road and say, is it full of potholes or is it smooth as glass, right? [00:48:59] It's just perfectly obvious, but to a, to a digital project, it's, it's not as clear. So, yeah, but, but, but exploring those new ways because turning everybody into a businessman so that they can, they can keep their project going, it, it, it itself is not sustainable, right? so yeah, and that's why everything turns into a SaaS because a SaaS is easy to control. [00:49:24] It's easy to gatekeep behind a paywall and it's easy to charge for, whereas a library on GitHub. Yeah. You know, what do you do there? You know, obviously GitHub has sponsors, the sponsors feature. You've got Patreon, you've got Open Collective, you've got Tidelift. There's, there's other, you know, experiments that have been run, but nothing has risen to the top yet. [00:49:47] and it's still, it's still a bit of a grind. but yeah, we'll see, we'll see what happens, but hopefully people will keep experimenting and, and maybe, maybe governments will start. Thinking in the direction of, you know, what does it mean to have a budget for digital infrastructure maintenance? [00:50:04] Jeremy: Yeah, it's interesting because we, we started thinking about like, okay, where can we find spaces for other Sidekiqs? But it sounds like maybe, maybe that's just not realistic, right? Like maybe we need more of a... Yeah, a rethinking of, I guess the, the structure of how people get funded. Yeah. [00:50:23] Mike: Yeah, sometimes the best way to solve a problem is to think at a higher level. You know, we, the, the sustainability problem in American Silicon Valley based open source developers is naturally going to tend toward venture capital and, and capitalism. And I, you know, I think, I think that's, uh, extremely problematic on a, on a lot of different, in a lot of different ways. [00:50:47] And, and so sometimes you need to step back and say, well, maybe we're, maybe we just don't have the right tool set to solve this problem. But, you know, I, I. More than that, I'm not going to speculate on because it is a wicked problem to solve. [00:51:04] Jeremy: Is there anything else you wanted to, to mention or thought we should have talked about? [00:51:08] Mike: No, I, I, I loved the talk, of sustainability and, and open source. And I, it's, it's a, it's a topic really dear to my heart, obviously. So I, I am happy to talk about it at length with anybody, anytime. So thank you for having me. [00:51:25] Jeremy: All right. Thank you very much, Mike.

In this episode, Jason and Chris welcome DHH, who joins them after the recent Rails World event. They cover a wide range of topics from the Rails Foundation's mission to attract new talent to open source misconceptions, the value of open source contributions as gifts, and the importance of boundaries between contributions and vendor relationships.  DHH shares insights into his current projects, including “Prop Shaft” and “Skiff,” addressing deployment challenges and building static sites. [00:00:29] DHH describes the incredible energy and positive atmosphere at Rails World, emphasizing the importance of in-person gatherings. [00:05:02] A discussion comes up about the foundation's role in supporting open source and attracting sponsors like Shopify for the benefit of both the community and businesses.[00:11:54] DHH talks about the misconception that open source is primarily about unpaid labor and how it's important to avoid becoming an unpaid employee.[00:15:47] DHH announced in his keynote at Rails World seven new things coming out and he tells us some he most excited about.[00:20:00] DHH describes the development journey from initial concept to validating in production applications, extracting into a library or framework, and ultimately making it the default for broad use. [00:22:12] Jason asks about the static site work that DHH is thinking about, and he introduces a project he's working on called “Skiff,” built on top of Kamal for deploying static sites.[00:26:28] Chris brings up a question about when to build your own solutions or use existing ones, and DHH highlights that it depends on the domain and the impact it has on daily work. [00:29:30] DHH talks about the problems with the existing job running solution, Resque, and the need to maintain multiple gems to patch it. [00:34:46] Jason brings up Webpacker and DHH discusses his frustration with complex bundling systems like Webpacker and his eagerness to simplify them. [00:36:02] Chris talks about the concept of finding the right abstraction layer where there's a balance between providing a simple interface and allowing users to dig deep into specific features when necessary.[00:38:32] The importance of recognizing fundamental improvements like esbuild and adopting them is highlighted.[00:40:59] The conversation shifts to the maintenance of separate frameworks like Hotwire and Kamal, and the question of separate maintainer teams and regular Rails releases is brought up.[00:43:55] DHH describes Hotwire as a “two and a half party” with substantial development happening with Basecamp but contributions from a considerable external community.[00:45:14] DHH talks about the evolving nature of projects like Turbo and the need for experimentation to address real-world issues.[00:50:37] We end with DHH highlighting the inherent tension between project creators and users and clarifies that not all open source projects operate as democracies. Links:Jason Charnes TwitterChris Oliver TwitterAndrew Mason TwitterDHH TwitterRails World 2023 Opening Keynote-David Heinemeier Hansson (YouTube)The Rails FoundationHoneybadger Honeybadger is an application health monitoring tool built by developers for developers.Disclaimer: This post contains affiliate links. If you make a purchase, I may receive a commission at no extra cost to you.

Mental models are metaphors that help us understand complex problems we work on. They can be a simplified roadmap over an infinite area of complexity. How does one come up with mental models? How are they useful? Are they primarily a solo thing, or can they be used to communicate with the team? What happens when your model is inaccurate? Today, Joël is joined by Eebs Kobeissi, a Developer and Dev Manager at You Need a Budget, to discuss. This episode is brought to you by Airbrake (https://airbrake.io/?utm_campaign=Q3_2022%3A%20Bike%20Shed%20Podcast%20Ad&utm_source=Bike%20Shed&utm_medium=website). Visit Frictionless error monitoring and performance insight for your app stack. Eebs on Twitter (https://twitter.com/EebsKobeissi) You Need a Budget (https://www.youneedabudget.com/) Skill floors and skill ceilings (https://thoughtbot.com/blog/who-is-empowered-by-your-design) Transcript: JOËL: Hello and welcome to another episode of The Bike Shed, a weekly podcast from your friends at thoughtbot about developing great software. I'm Joël Quenneville. And today, I'm joined by Eebs Kobeissi, a Developer and Dev Manager at You Need a Budget. EEBS: Hi, Joël. It's really good to be here. JOËL: And together, we're here to share a little bit about what we've learned along the way. So, Eebs, what's new in your world? EEBS: Oh, a whole lot. I'm a new dad, so I'm getting to experience all those things. But in the developer world, I've recently picked up programming on an ESP32, which controls LED lights. And so I'm having fun lighting up my office. JOËL: Is that like one of those little microboards, kind of like a Raspberry Pi? EEBS: Yeah, exactly. It's a little board that's compatible with the Arduino IDE. And I literally only played with it last weekend, so it's still very new to me. JOËL: Nice. Have you done any Arduino development or Raspberry Pi or anything like that before? EEBS: No, I have a Raspberry Pi that I run like a DNS server on, but I haven't done any actual programming. I did make an LED blink, which is pretty cool. JOËL: What kind of programming is required for a board like that? EEBS: From my understanding, it's either in Python or C. Those are, I think, the two languages that you can program on it. I definitely do not know C. And so I'm just going through a bunch of tutorials and reading some sample code. But I think if I ever end up trying to implement something more complex, I'll probably switch over to Python because that's a little more familiar. JOËL: So the coding feels fairly high level even though you're writing controller code for LEDs. EEBS: I hope so. I'd love to be able to take advantage of whatever abstractions I can. JOËL: Do you have any fun goals you're trying to do with this? Or is this just for the fun of trying a completely different environment than web development? EEBS: No, it's actually rooted in something visual. So I have these shelves behind me that are in my webcam when I'm in meetings or whatever. And so I want to be able to put a light strip across these shelves and have some sort of visual thing in the background. JOËL: Like LED mood ring? EEBS: Yeah, kind of. My eventual goal would be that as I'm talking, a little equalizer display pops up behind me. I thought that would be pretty neat. JOËL: That is amazing. That will give you all of the cred in the meetings. EEBS: Right? I thought that'd be pretty cool. What have you been thinking about recently, Joël? JOËL: I've been submitting to the RubyConf call for proposals which, as of the recording of this episode, has just closed this week. And like many people, I submitted on the last day. EEBS: [laughs] JOËL: And it was really fun trying to take some ideas that I'm excited about and then turn them into a proposal that is accessible to other people. EEBS: Nice. Do you want to share a little bit about what the talk is, or is it under wraps for now? JOËL: I don't know if anyone on the committee will listen to this before the review goes out. This might break the anonymity of the proposal. EEBS: Oh, right, right. JOËL: One thing I will share that's interesting is that there are topics that I'm excited about. It's like, oh, here are a bunch of cool things about something, some technical topic. But talks that are just ten cool things about X are not that great. And so I needed to find some sort of unifying idea that I could use to share that. And that generally is in the form of trying to find a story that I can tell. What unifies all of these things together? What tells a compelling story? Is there some metaphor I can lean into? EEBS: Nice. I think that's a really powerful way of communicating something deeper is through telling something that people can relate to. JOËL: One way that thinking about metaphors has been really impactful for me recently is the idea of mental models and how those can help us in development. I'm curious; we've thrown around the phrase a little bit you and I in past conversations; what does a mental model mean to you? EEBS: I tend to be a visual thinker. And from talking to others, I've heard similar statements. So for me, a mental model is how I think about a particular domain or how I think about code flow or structure. And for me, it's usually either as two-dimensional objects or occasionally three-dimensional objects that I have floating in my visual space. So, for example, if we had two classes that are collaborators in some way, I often think of them maybe as two rectangles that are side by side. And when they interact, there's some little amorphous blob from one of those rectangles that reaches out into the other one or passes a message from one to the other. And I sort of have this idea of how many connections are there between these two physical things. Or, if I'm thinking about code flow and the path of execution that code might take, sometimes I visualize it as maybe a tree or potentially loops if there are such cases. JOËL: So when you think of these concepts, just in general, you're seeing in your mind's eye squares and rectangles floating in the air. EEBS: Yeah, pretty often. Sometimes it takes those shapes, and as I build up a mental model of some code, I'm usually adding new shapes into that picture I have in my mind. I tend to view things sort of top-down. So like, the start of code or the start of execution is usually at the top or maybe the far left or far right. And as execution happens, I usually view that as moving in towards the middle and potentially going back out when a response is returned. If it's a web request, something like that, I view it as this sort of outside in. And there's a bunch of pieces in there that are all talking to each other. JOËL: That's really cool. So not only is there a geometric aspect to it, but there's a spatial aspect to it as well. EEBS: Yeah. And it's interesting, like, I haven't actually thought about it [laughs] in this level of detail before. But yeah, there certainly is a spatial aspect to it. And I have this idea in my mind of like things and domain objects kind of belong at the bottom, and they should have well-defined boundaries. But the pieces that are a little bit towards the outer edges may be a little more fuzzy and may have less definition around them. JOËL: That's really interesting because I also have this sort of in my mind's eye see these things when I'm thinking about concepts like that. But I've talked to other people, and some people don't even have much of a mind's eye at all. They don't tend to visualize things in their mind in that way. EEBS: Yeah, it's really interesting how different people approach this thinking about code. A lot of people write things down. And I write things down, too, and draw little arrows that don't really make any sense. But it helps me do something physically sometimes as well as just thinking about it. JOËL: Have you ever tried to convert these pictures you see in your mind and actually draw them on paper? EEBS: Occasionally. And for the most part, that usually takes the form of some kind of domain modeling, whether it's based on database tables or just domain objects. And sometimes I will try and draw them out and then specify the relationships between them like, oh, you know, this one model talks to this other model in this particular way. And I'll define a relationship between them, which helps me think about them and how they interact. JOËL: I've found that even though for some things I can see it very vividly in my mind's eye, I struggle to then concretely translate that onto paper or digital paper if you will. It's almost like trying to, say, translate an emotion into words and that even though I feel like I see a visual picture, I can't reproduce it by drawing necessarily. EEBS: It's interesting you brought up feeling because a lot of the times, I have this gut feeling about a mental model, like whether I think it is correct or not. And sometimes I have this uneasy feeling of, like, that doesn't feel right to me, but it's hard to articulate why. And I think sometimes that's when I have to pull out something physical, start making those relations, start connecting things. And that's when I might uncover, like, oh, this feels odd because I have a circle here or a cycle or something. Or I've sort of represented the truth of something in two different places. Do you have any techniques for getting it out of your head and into something physical that you could share with someone else, maybe it's text or a picture? JOËL: I think I do struggle with that conversion sometimes. Practice definitely helps. I think maybe there is a metaphor here between converting these, let's call them, pictures that I see in my mind's eye and then drawing diagrams with trying to take feelings and expressing them in words. In the same way that, maybe I might have some feelings, and then I want to journal how I feel, and I struggle to express that. But finding a way to express that gives me a certain amount of precision and a more concrete thing. In the same way, these things that flash in front of my mind's eye, if I can take the time to put them on paper, they're now more real. They're more concrete. I think you can probe the edges, the ways that it kind of falls apart more easily. EEBS: Yeah, that makes a lot of sense. There's a lot of value in writing that down and going through those details in a methodical way because oftentimes, you'll catch inconsistencies, or you'll find better ways to describe it. And being able to share your mental model with someone else is often...well, it can be really tricky. And I think that's why it's important to go through and maybe find a common medium that you can share because I can't see into your brain. You can't see into mine. But if we can share our mental models, then hopefully, we have a better chance of agreeing on the solution or finding inconsistencies. JOËL: Exactly. I think, in many ways, there are almost multiple layers of mental models and that you might have an abstraction or a metaphor for a concept that you're working with separate from the diagram. And then the diagram is yet another metaphor, but now we're going geometric to represent a broader idea. EEBS: Yeah. Are there any other ways that you take that picture from your mind's eye besides written documents or conversations? Do you use any diagramming tools that specifically help with that? Or is it just kind of free-form? JOËL: I do a mix. I am a big fan of draw.io, which allows you to just free-hand or pull shapes together, things like that. There are some more structured tools that I will use. I'll use Mermaid.js. EEBS: Yeah, I've been using that a lot too. JOËL: Yeah, that's great. I've been digging into more structured diagrams recently, particularly the idea of graphs, directed graphs. And those have interesting properties. EEBS: Can you share a little more detail about what you mean? JOËL: So a graph in the computer science sense is a bunch of nodes. They are typically represented as circles and then edges which are the connections between them. A directed graph is now there's an arrow pointing in a particular direction. A really interesting property that you can have with directed graphs is whether or not they include cycles. So can you only by following the arrows effectively create a loop? Or will the arrows always lead you to some kind of terminal node? EEBS: Gotcha. Is that a directed acyclic graph? JOËL: If there are no cycles, yes, it is a directed acyclic graph or DAG, as you'll often see it abbreviated. EEBS: [laughs] How do you relate that graph to code? And what benefits do you get from expressing it that way? JOËL: So this shows up in a lot of places. And I'd even say that thinking of certain aspects of my code as a graph and a potentially directed acyclic graph is itself a mental model or a metaphor that helps bring clarity to the way I think about things. So, for example, code, you know, you invoke some main function at some point to call the code, and then that's going to call out some other functions, which call out some other functions, and so on. You may have heard that referred to as a call graph. But that is a graph of calls. There might be cycles in there for co-recursive functions and things like that. But that is one way you can then sift through and analyze how control flow or how logic flows through your application is through a function graph. You mentioned earlier the idea of objects and how they're connected to each other. That's an object graph. EEBS: Right. Recently, I had to work through a state transition problem where a customer has some billing, and they can go through many different states, whether it's active, or canceled, or past due, those sorts of things. And so actually, I reached for Mermaid.js and built a graph of, okay, they start here in this empty state. And then they subscribe, which then they become active. They might cancel their subscription, which moves them to a different state. And by listing out all the states and the transitions between them, it helped me to understand what methods I might need to define on which objects in order to allow those transitions to happen and what checks I might need to make before allowing those transitions depending on the state of the system. JOËL: I'm hearing the keywords states and transitions. And that's making me think of finite-state machines. Are you drawing a finite-state machine graph or something a bit more free-handed? EEBS: It's a bit more of free-handed. I don't think I've actually drawn out a state machine since college but just representing the different states as different boxes and the transitions that are possible from those states. I mean, I guess that kind of is a state machine in some way. So graphs are great visual approaches. Are there any non-visual approaches that you take? JOËL: That's a great question because not all mental models have to be visual. I think the power of a mental model exists in a metaphor. And one that's kind of broad but that I've applied to a lot of different areas is the general idea of something being parallel or in series. I think I first came across this concept talking about electric circuits. And are we talking about two little light bulbs that are in parallel, and if the electricity to one is cut, the other one still lights up? Or are they chained together in series? EEBS: Yeah, like my LEDs. JOËL: Exactly, going back to Arduino, but it can also be applied to a bunch of other things. We can talk about code being in parallel or in series. We can talk about work being in parallel or in series. Interestingly, I took that mental model as a sort of quick shortcut when I was digging into some functional programming ideas. Monads and applicatives are the fancy terms here. EEBS: Oh boy, I'm ready. JOËL: In general, and there's a hefty asterisk here, I think of monads as being serial, so you're chaining something; one thing happens, then another. So you can think of, for example, chaining promises in JavaScript, promise one, then promise two, as opposed to applicatives which are parallel. So you might think of maybe zipping two lists or two arrays in Ruby. The two arrays, there are no dependencies between the two of them. They get processed side by side as you're traversing both of them together. EEBS: Interesting. I've heard the term monad a lot, but I haven't heard the term applicative. Are there any other details you can share about them and what makes them different or how they might be seen in our code? JOËL: I think that the key difference is that distinction in how they're processed. Applicatives are a way of combining two independent, let's call them data sources, and then you find a way to combine them together. So it could be two independent arrays, and you're zipping through them. It might be two independent HTTP requests, and they can both fire in parallel. But then you want to combine their outputs. So you say wait until both are successful and then combine their output. EEBS: Oh, okay, gotcha. JOËL: It could even be nullable values. So you say do this thing if both values are present. But you're not...the value of one or the fact that one is null or not is not dependent on whether the other one is null or not. They're independently null or not as opposed to something...Monads are, again, a different way of combining. You might call them data sources or operations. But in this case, there is a clear dependency one, and then its output influences the next one. You might say check the value is null or present or not. And then, if it is present, take that value and then put it as the input of my next operation. And then, if it is null or not, do another thing. See, now you have a sort of chain. EEBS: Where do you see these chains happening in code? Or is it everywhere? JOËL: Once you know that pattern which, again, could be thought of as another mental model, you start seeing it everywhere. So promises in JavaScript chaining together that's effectively monads. Don't @ me, all the functional programming people. EEBS: [laughs] JOËL: I know that's not quite true. Anything dealing with multiple operations that could succeed or fail depending on, again, whether you're treating them as dependent or independent, that's probably going to look very similar to either monad or applicative. EEBS: So the first thing that actually comes to mind here is things like background jobs. Using Sidekiq or Resque or other job processors, you can have a queue of jobs that need to be executed, and they might need to run in serial, or potentially you have multiple workers pulling from a single queue, and thus the work is happening in parallel. Is that a reasonable analogy? JOËL: I think it's good for the serial versus parallel, but it's not necessarily a good analogy for understanding monads and applicatives. EEBS: Gotcha. JOËL: So with two workers, you can process a queue in parallel, and a bunch of things happen. EEBS: But there's not necessarily anything that is bringing those two workers together to produce a single output. JOËL: Yes. And there's no dependency between the tasks in the queue. EEBS: Right, right, gotcha. JOËL: So if you have a task that says execute this task and then only if this task succeeds, then do the second task, now you've created a dependency. And you couldn't process that in parallel because if task one, which has to be executed first, is executed by worker one, task two should not get processed unless task one is successful. You can't just say, oh, I've got another worker free. I haven't processed task two because it's waiting to know does task one succeed. EEBS: Right. So an example in code would be a user creates a new order. And when they create a new order, we send them a confirmation email. That would be an example of that happening in serial or a monad-like thing. [chuckle] JOËL: Yes, I found that thinking of things as serial or parallel is a good shortcut for thinking about monads and applicatives. I don't know that the reverse is necessarily true. They don't necessarily transfer one-to-one with each other. And maybe that's a danger of mental models, right? You find a mental model that describes a situation, and then you try to reverse it, and then you make false assumptions about the world. MID-ROLL AD: Debugging errors can be a developer's worst nightmare...but it doesn't have to be. Airbrake is an award-winning error monitoring, performance, and deployment tracking tool created by developers for developers that can actually help cut your debugging time in half. So why do developers love Airbrake? It has all of the information that web developers need to monitor their application - including error management, performance insights, and deploy tracking! Airbrake's debugging tool catches all of your project errors, intelligently groups them, and points you to the issue in the code so you can quickly fix the bug before customers are impacted. In addition to stellar error monitoring, Airbrake's lightweight APM helps developers to track the performance and availability of their application through metrics like HTTP requests, response times, error occurrences, and user satisfaction. Finally, Airbrake Deploy Tracking helps developers track trends, fix bad deploys, and improve code quality. Since 2008, Airbrake has been a staple in the Ruby community and has grown to cover all major programming languages. Airbrake seamlessly integrates with your favorite apps to include modern features like single sign-on and SDK-based installation. From testing to production, Airbrake notifiers have your back. Your time is valuable, so why waste it combing through logs, waiting for user reports, or retrofitting other tools to monitor your application? You literally have nothing to lose. Head on over to airbrake.io/try/bikeshed to create your FREE developer account today! JOËL: Another mental model that is not necessarily visual that I like actually comes from the video game community, and that's thinking of skill ceilings and skill floors. So in, I think, particularly the MOBA Community, that's a Multiplayer Online Battle Arena, they'll talk about characters as having a high skill floor or a low skill ceiling. And generally, what that means, and again, the meaning varies a little bit by community, is that a character with a low-skill floor is an easily accessible character. They might not have a lot of skill shots, like, you press a button, and things happen around your character. You don't need to aim, things like that. A high skill ceiling means that there's a lot of room for you to grow, and as you get more skilled, you can get significantly better with that character. EEBS: Gotcha. So the opportunity is greater with a higher skill ceiling. JOËL: Correct. And depending on how the character is set up, you might have a very narrow range that could be in the low range where it has a low skill floor and a low skill ceiling, which means that the character is easy to learn. But once you've learned it, there's not really a lot you can do with it. It's a fairly basic character. So getting better at the game is not necessarily going to make you that much more impactful. And then you could have one that's the opposite that is high both skill floor and skill ceiling where a character is very hard to learn. But once you learn it, that's kind of all there is to it. And then you might have one that has a large range somewhere; maybe it's easy to learn, but it's hard to master, or there's a lot of room for growth. And so, taking this framework for analyzing characters and video games, I think we can apply that to technology in general. This could be language design. This could be just API design. And you might say, well, I want this to be very accessible. People can jump in very easily. You might say I want this to be very powerful and have a lot of high-end features that make your power users very happy and very productive. EEBS: That's interesting. When you first were talking about it, I was actually less thinking about it from a user's perspective of what maybe they could do in the application but potentially from the standpoint of a developer writing the system itself. One of the pieces I always come back to in software development is that change is inevitable. And so, making something easy to change often pays great benefits down the road. And so I wonder how that fits into this idea of a low skill ceiling or a high skill ceiling in terms of perhaps flexibility or being decoupled such that you can take one idea and easily extend it or easily get more from it than you originally set out to build. JOËL: There's often a trade-off. So you make something easy to change. It's highly decoupled. But you maybe introduce more indirection to the system. So while it's easier to change one single piece, it's harder to understand the system as a whole. EEBS: Yeah, that's true. And sometimes, you bake in assumptions that you make about the future, which turn out not to be true. JOËL: [laughs] Yes, that is definitely something I'm guilty of. EEBS: I think we all are. JOËL: One thing that I find interesting is as you evolve the design of an architecture pattern, a system, a whole language, you might want to move one of those if I think of them like two independent sliders on a one-dimensional scale. So maybe you want to move the upper boundary a little bit and say I want a higher skill ceiling for this, but they don't actually move completely independently. So introducing some advanced features might inadvertently also raise the skill floor. And conversely, making the language super accessible so that it has a low-skill floor, you might have to decide I will not introduce certain features. EEBS: One thing I wanted to ask you about is, do you view different languages as having different skill floors and ceilings? And, you know, I love Ruby. I know you love Element. I've played with Element. It's been a great learning tool for me. How do you view those two languages in terms of skill ceilings and skill floors in terms of, I guess, what you can do with them? JOËL: That's a great question. And I think you can definitely apply that to languages. Admittedly, I think you could probably start a lot of flame wars with that. EEBS: [chuckles] Let's not do that. JOËL: I wrote an article a while back where I applied that mental model to look at the F# programming language. And there was a debate in that community about certain features to add and whether they would allow advanced programming but potentially at the cost of accessibility to newer members of the community and how to balance those. And so I thought, hey, let's throw this video game metaphor at the problem and talk about it through that lens. EEBS: That's really cool. Did you draw any conclusions, or was it as a way to start a conversation? JOËL: It is a way to start a conversation. I don't think there is a single correct or best distribution of your skill, ceiling, and floor. It has to match the goals you set out for your project. Just like in games, people love to rank which characters are best and not. And sometimes you can show that, in general, this character is better. But oftentimes, in a balanced game, you can talk about this character being easier to get started with or this character working very well if you're a pro. But the fact that you have a higher or lower skill ceiling or floor doesn't necessarily make the character better or worse. EEBS: So, this conversation about differing mental models, I think I hadn't realized that there can be so many different types of mental models. And some things that I do in my thinking I haven't classified as a mental model. But now that you bring it up, I think one that I think about fairly often is this idea of two objects that are collaborators and reaching into the internals of one of those objects from the other object. So A and B are two separate things. And if A reaches into B's bucket and messes with the state of B, I view that as sort of a bad practice. You're not really adhering to maybe the public API that that object is exposing. You're kind of reaching in and going around behind its back and changing some stuff that it may not expect. JOËL: Would you refer to that maybe as tight coupling? EEBS: Yeah, it's definitely tight coupling. It's not just tightly coupled; it's almost worse than that. It's almost like going behind somebody's back and making a change without them knowing. And so when I see that in code or when I write code that does that, I have this really intense desire to separate that and to say, no, no, you can't go in and update this record directly in the database. You have to send it a message and say, "Hey, I would like you to be aware of something," and then it goes and changes its own internal state as a response to that. And so I have this very vivid sort of mental feeling of it being wrong, of it being like, I'm being sneaky, or I'm not being gracious to the person I'm interacting with as though I were one of these objects. JOËL: That's fascinating. You've practically anthropomorphized these objects. EEBS: I do. I view them as little people. JOËL: You describe this interaction as going behind someone's back. That is the thing that I, a person, do to someone else. It's not a function making a direct call. And yet, it's such a strong...we use a social mental model to talk about objects and interactions. EEBS: Yeah, I almost want them to be friends. And I think that applies to real-life relationships, right? If you have a nice dialogue back and forth, there's an understanding. There's commonality that you can find. But if I were to go do something behind your back without chatting with you about it first, you might not be so happy with me. JOËL: I'd feel betrayed. EEBS: Right. JOËL: I feel like there's probably a really fun conference talk to be done about that. We often use that metaphor; I think when talking about objects sort of subconsciously but making it explicit and just being, hey, let's talk about these objects as if they were people. Why don't we want to do this? Because this one here is betraying the other object there. This one here is being impolite. EEBS: We could have two people get on stage and talk to each other. And I might then go and reach in your pocket and pull out some change without you knowing, and you might be upset with me. JOËL: That would be great. Get a little skit going up on stage. Or even if you're artistically inclined, you could probably draw some really fun little characters to illustrate this. EEBS: That would be really cool. I, unfortunately, don't have the artistic talent to do that. JOËL: Well, free conference talk idea to all listeners of the podcast. I expect to see this for RailsConf 2023, maybe. EEBS: I'll be looking for it. So I've shared a mental model that I didn't really know was a mental model. Are there other mental models that you want to share that I may not be thinking of? JOËL: Here's one that I've just come to realize recently that I'm actually quite excited about: when you think about the word refactoring, how would you describe that idea? EEBS: Well, refactoring to me is changing the implementation without changing the behavior. JOËL: Yes, I think that is the classical definition. You should be able to change the implementation of a method, and the tests without changing are still green after you've done that. EEBS: I guess, mentally, I think about that as perhaps drawing a box around some of the objects that are floating in my mind's eye, rearranging how they exist within that box, and then the box dissolves. And the tests still pass, but the structure of the objects or the code has changed in my mind. JOËL: I love that you immediately went to a visual approach there. And I think I have something similar, but I'm coming at it from a slightly more domain modeling perspective. So thinking maybe less from an individual method approach but looking at maybe a larger system, what you're trying to do is use code to describe some version of reality. So it might be a business process that you have. It might be trying to describe some aspect of your customer's life that you're trying to automate for them. Oftentimes, this thing you're trying to describe in code terms is going to be a simplification because life has a ton of edge cases, and many of them we don't care about. So if we go with a visual metaphor here, you're trying to draw some kind of shape using only straight lines to approximate some weird curve. And so, let's say you draw something with only four lines. It's really simple, how you have a diamond. That's the shape you're trying to create. And then you're going to fill it in with little other shapes that approximate a diamond. And those are your different models and functions and all the other components that we use to build software. At some point, your understanding of the underlying reality might change. Maybe you need more precision, or maybe the actual feature requirements have changed. The thing you're trying to approximate with your code is not a diamond. Maybe you've added a few more sides to it. It's a pentagon. So we've gone from four sides to five. And the little components, and modules, and things that you have there approximate that diamond work. They still mostly approximate your pentagon, but it's really clunky because the initial design was to approximate something else. They were really good for fitting in really tightly and being very loosely coupled to each other when we were trying to do a diamond, but then they don't work as well in the pentagon. EEBS: So maybe some of the internal shapes need to change or adjust to fill the space that the pentagon has now created. JOËL: To fill the space or maybe even just to fill it in a way that's less clunky. And so the idea here in this metaphor is that the reality we're targeting in software is always changing. And so the underlying reality changes, and so we're changing that shape that we're creating all the time. But also, we're getting more precision as we decide; oh, we care about this edge case now. We didn't in version one, and so as part of that, we're constantly having to take the modules that maybe were very well designed initially but then restructure them to fit the new requirements because now there's a fourth object coming in, and it's kind of clunky with our current configuration. EEBS: That's interesting. One of the first things that jumps to mind is that maybe there are better ways or worse ways to do that refactoring to fit that new shape. Do you think there's any truth to that in the sense that you might initially design a system that perfectly fits that diamond or very closely fits that diamond but then as it changes to a pentagon, do you need to simply add a new piece to fill in that empty space? Or do you need to restructure everything within the diamond now to fit the shape of the pentagon? JOËL: Oftentimes, you do need to restructure. And I think there's this wonderful little phrase from; I believe it's Kent Beck that says, "Make the change easy, and then make the easy change." EEBS: Yep. JOËL: And so, to me, that makes the change easy is that initial restructuring that you need to do of those first shapes so that you can finally bring in the new one. EEBS: Oh, that's a cool visual. I immediately can imagine the pieces in the pentagon moving around to make space for a new piece that you need to now bring in. And that movement of all those pieces can be really difficult. Have you ever played that game where it's a square, and you're trying to get a ship out of a port, but there's a whole bunch of other ships, and you can only move them left and right and up and down? And you can do that. And that's what I'm picturing right now is moving shapes within that pentagon to then make space for either a new shape or to allow a shape to escape that is no longer relevant. JOËL: I played a version of that that had cars, cars, and trucks. EEBS: Gotcha. Yeah, I think I played that too. JOËL: That would also be a fun conference talk, right? Like, start with that game as your initial metaphor. And then you use that as a way to talk about refactoring. EEBS: That would be really cool. JOËL: I would watch that talk. To anybody listening who wants to give that talk, I want to see you at RailsConf 2023. EEBS: [laughs] Are we just a talk factory now? [laughter] JOËL: I love talk ideas. Maybe this should become a segment. Just have Eebs come in for five minutes once a month and give us a talk idea. It could even be fun to see a talk idea that multiple people implemented differently. EEBS: That would be really cool, actually. I always get nervous about giving talks or being on podcasts like this one. I would love to be the person that gets to sit there and throw out random ideas and have other people fulfill my dreams. JOËL: Well, thank you so much, Eebs, for joining us to talk about mental models. And to all of our listeners, I'd love to hear about what mental models you find are helpful, and so please share them with us. On Twitter, you can reach us at @_bikeshed. EEBS: Thanks for having me, JOËL. This has been super fun. JOËL: And on that note, let's wrap up. The show notes for this episode can be found at bikeshed.fm. This show is produced and edited by Mandy Moore. If you enjoyed listening, one really easy way to support the show is to leave us a quick rating or even a review in iTunes. It really helps other folks find the show. If you have any feedback, you can reach us at @_bikeshed or reach me at @joelquen on Twitter or at hosts@bikeshed.fm via email. Thank you so much for listening to The Bike Shed, and we'll see you next week. Byeeeeeee!!!!!! ANNOUNCER: This podcast was brought to you by thoughtbot. thoughtbot is your expert design and development partner. Let's make your product and team a success.

No episódio #56, os podcasters se reúnem com uma convidada para comentar uma lista de figuras reais e fictícias que foram referências importantes no processo de descoberta de pessoas LGBTQIA+. O episódio é apresentado por Raphael Mendes e comentado pelos podcasters Gabriel Bandeira e Ana Brasileiro e pela convidada Laís Resque. Curtam o nosso episódio em homenagem ao mês do orgulho LGBTQIA+, e sigam o @maladourada nas redes sociais! Vinheta por: Uriel Bestene. Roteiro: Raphael Mendes, Gabriel Bandeira, Ana Brasileiro e Laís Resque. Edição: Raphael Mendes.

Vaya tela cómo pasa el tiempo, ¿no? Hace nada estábamos comenzando con los propósitos de 2021, deseando que por favor no fuera como 2020... Y ahora ya estamos casi con las uvas en la boca. Bueno. Lo más importante que ha tenido este año ha sido, sin duda, crear Audiencias Marcianas y toda la gente que rodea a la empresa. Desde el equipo, que es fantástico, hasta cada uno de nuestros suscriptores y clientes. Ha sido un comienzo de viaje emocionante, pero... Calma. Lo mejor está aún por llegar. Para el último episodio del podcast de 2021 queríamos hacer algo especial. Y dado que llevábamos mucho tiempo diciendo "queremos traer gente" pero nunca lo hacíamos, esta vez sí lo hemos cumplido. Así que contamos con dos invitados de lujo (no te voy a desvelar quiénes son, para eso tendrás que ir al podcast) con los que tendremos una conversación muy entretenida, mientras Resque se infla a chucherías y pipas. Y hablan principalmente de: -Qué cambios más fuertes han visto dentro de las fuentes de tráfico -Oportunidades y sustos con el tráfico en 2022 -Qué oportunidades pueden surgir en cuanto a publicidad con el famoso metaverso de Facebook -Qué recomiendan a los emprendedores que quieren comenzar a invertir en publi Y más cosas. Espero que lo disfrutes tanto como nosotros, y sobre todo, que te sea muy útil y te aporte unas cuantas ideas de lo que está por venir este próximo año. Para escuchar el podcast ¡dale al play! Para más información puedes consultar estos enlaces: ➡️ Canal de Telegram ➡️ Web Y para aprender más consejos marcianos sobre marketing, no olvides seguirnos. ➡️ Instagram: Audiencias Marcianas + Javier Santos + Diego Fernández ➡️ Youtube: Audiencias Marcianas ➡️ Facebook: Audiencias Marcianas ➡️ Tiktok: Audiencias Marcianas

Una cuestión suculenta, la verdad. Cómo os gusta el morbo a los terrícolas, vaya cosa. De hecho, creo que este tema se ha convertido (si no lo es ya) en la "carrera espacial" del marketing. A ver... La respuesta rápida es no. No podemos saber cómo funcionan los algoritmos porque son más secretos que la colección de vestidos de Putin. Ahh.. que... ¿no tiene una colección de vestidos? Bueno, eso no se sabe. Aunque si fuese cierto, ¿no crees que él lo mantendría en secreto? Pues con el algoritmo, igual. Y de todas formas, la verdad es que nadie sabe a día de hoy cómo funciona el algoritmo de Instagram, solo son interpretaciones de lo que pensamos que puede ser. Por no hablar de que todos los días pueden cambiar cosas en los algoritmos. No obstante... Sí que hay ciertos factores y elementos que podemos analizar y utilizar a nuestro favor para que el algoritmo nos preste más atención, y tener la posibilidad de que nuestros contenidos lleguen a más gente. Y en el episodio del podcast de esta semana hay bastantes pepitas de oro sobre esto. Explicamos, entre otras cosas: -A qué porcentaje de tu público muestra Instagram tus contenidos -Qué criterios sigue el algoritmo para dar más alcance a las publicaciones -Un truco mega simple de menos de 5 segundos, que nos da una pista de qué clase de contenidos prefiere Instagram -Cuántas veces al día hay que publicar dependiendo del tipo de cuenta que tengas Y además, en nuestros peculiares momentos marcianos, tenemos... *Consejos de seducción de Ñesque para camelar al algoritmo y que le dé más vidilla a tus contenidos *La primera vez que verás a Resque descojonarse de risa (literal) hasta el punto de sacar de quicio a Ñesque Si quieres estar lo más cerca posible de descifrar el funcionamiento de Instagram ¡dale al play! Para más información puedes consultar estos enlaces: ➡️ Canal de Telegram ➡️ Web Y para aprender más consejos marcianos sobre marketing, no olvides seguirnos. ➡️ Instagram: Audiencias Marcianas + Javier Santos + Diego Fernández ➡️ Youtube: Audiencias Marcianas ➡️ Facebook: Audiencias Marcianas ➡️ Tiktok: Audiencias Marcianas

Aquí una lección/reflexión algo atípica. Si algo nos ha enseñado nuestra evolución cultural y como sociedad, es que todo tiende a dirigirse hacia la inclusión. Es decir, algo se va expandiendo hasta que se mezcla con otros elementos, con los que acaba formando algo totalmente nuevo. Por ejemplo, la Ciencia avanza gracias a plantear nuevas hipótesis dentro de las teorías que ya existen, bien para afianzarlas o para buscar la forma de romperlas y encontrar otras nuevas y mejores. Y todavía están tratando de encontrar LA teoría que una la Física de la Relatividad con la Física moderna (cuántica). Otro ejemplo es la gastronomía. Actualmente cada vez están más de moda los restaurantes pijos donde los platos tienen una cantidad inhumana de mezclas de ingredientes y condimentos de todas las partes del planeta. Y otro ejemplo podrían ser las preferencias sesssuales. Primero se pensaba en modo "hetero/homo" pero ahora lo que se lleva es ser bi. ¿Para qué excluir uno de los dos géneros cuando puedes disfrutar de los dos? Pues ¡claro que sí, guapi! Incluir, expandirse y combinar es lo que se lleva. Y nosotros estamos aplicando esto a las estrategias de marketing, donde todavía hay mucho terreno virgen. Por ejemplo, ¿qué es mejor a día de hoy, hacer lanzamientos u optar por estrategias evergreen? ¿Cuál es más rentable? ¿Cuál da mejores resultados? Y lo que es más intrigante... ¿Podrían funcionar las dos juntas para poder así abarcar una mayor parte del pastel? Pues a todas esas cuestiones damos respuesta en el episodio del podcast de hoy. Bueno, a esas cuestiones y otras cosas, como: -Por qué a Ñesque le dieron una paliza en sus glúteos... que le ha hecho tener que grabar de pie -Las oscuras intenciones de Resque que han ofendido (y mucho) a Ñesque -Un nuevo elemento hasta ahora inexplorado en los embudos de marketing: el TOFU Si quieres aprender algo que te será muy útil en los meses y años que están por venir, dale al play y escucha ahora el episodio. Para más información puedes consultar estos enlaces: ➡️ Canal de Telegram ➡️ Web ¡Síguenos! ➡️ Instagram: Audiencias Marcianas + Javier Santos + Diego Fernández ➡️ Youtube: Audiencias Marcianas ➡️ Facebook: Audiencias Marcianas ➡️ Tiktok: Audiencias Marcianas Y para aprender más consejos marcianos sobre marketing, no olvides seguirnos.

Chris evaluates the pros and cons between using Sidekiq or Active Job with Sidekiq. He sees exceptions everywhere. Steph talks about an SSL error that she encountered recently. It's officially spooky season, y'all! sidekiq-symbols (https://github.com/aprescott/sidekiq-symbols) Transcript: CHRIS: Additional radiation just makes Spider-Man more powerful. STEPH: [laughs] Hello and welcome to another episode of The Bike Shed, a weekly podcast from your friends at thoughtbot about developing great software. I'm Steph Viccari. CHRIS: And I'm Chris Toomey. STEPH: And together, we're here to share a bit of what we've learned along the way. Hey, Chris, what's new in your world? CHRIS: Fall is in the air. It's one of those, like, came out of nowhere. I knew it was coming. I knew it was going to happen. But now it's time for pumpkin beer and pumpkin spice lattes, and exclusively watching the movie Hocus Pocus for the next month or so or some variation of those themes. But unrelated to that, I did a thing that I do once, let's call it every year or so, where I had to make the evaluation between Sidekiq or Active Job with Sidekiq, as the actual implementation as the background job engine that is running. And I just keep running through this same cycle. To highlight it, Active Job is the background job system within Rails. It is a nice abstraction that allows you to connect to any of a number of them, so I think Delayed Job is one. Sidekiq is one. Resque is probably another. I'm sure there's a bunch of others. But historically, I've almost always used Sidekiq. Every project I've worked on has used Sidekiq. But the question is do you use Active Job with the adapter set to Sidekiq and then you're sort of living in both worlds, or do you lean in entirely and you use Sidekiq? And so that would mean that your jobs are defined to include Sidekiq::Worker because that's the actual thing that provides the magic as opposed to inheriting from Application Job. And then do you accept all of the trade-offs therein? And every time I go back and forth. And I'm like, well, but I want this feature, but I don't want that feature. But I want these things. So I've made a decision, but I want to talk ever so briefly through the decision points that were part of it. Have you done this back and forth? Are you familiar with the annoying choice that exists here? STEPH: It's been a while since I've had the opportunity to make that choice. I'm usually joining projects where that decision has already been made. So I can't think of a recent time that I've thought through it. And my current project is using that combination of where we are using Active Job and Sidekiq. CHRIS: So I think there's even a middle ground there where that was the configuration that I'd set up on the project that I'm working on. But you can exist in both worlds. And you can selectively opt for certain background jobs to be fully Sidekiq. And if you do that, then instead of saying, "Performlater," You say, "Performasync." And there are a couple of other configurations. It gives you access to the full Sidekiq API. And you can do things like hey, Sidekiq, here's the maximum number of retries or a handful of other things. But then you have to trade away a bunch of the niceties that Active Job gives. So as an example, one thing that Active Job provides that's really nice is the use of GlobalID. So GlobalID is a feature that they added to Rails a while back. And it's a way to uniquely identify a given record within your system such that when you say performlater, you can say, InvitationMailer.performlater and then pass it a user record so like an instance of a user model. And what will happen in the background is that gets serialized, but instead of serializing the whole user object because we don't actually want that, it will do the GlobalID magic. And so it'll turn into, I think it's GID:// so almost like a URL. But then it'll be, I think, your application name/model name down the road. And the Perform method actually gets invoked via the background system. Then you will just get handed that user record back, but it's not the same instance of the user record. It sort of freezes and thaws it. It's really nice. It's a wonderful little feature. Sidekiq wants nothing to do with that. STEPH: I'm so glad that you highlighted that feature because that was on my mind; I think this week where I was reviewing...somebody had made the comment where they were concerned about passing a record to a job and saying how that wouldn't play nicely with Sidekiq. And in the back of my mind, I'm like, yeah, that's right. But then I was also I'm pretty sure this got addressed, though. And I couldn't recall specifically if it was a Sidekiq enhancement or if it was a Rails enhancement. So you just cleared something up for me that I had not had time to confirm myself. So thanks. CHRIS: Well, to be clear, this works if you are using Active Job with Sidekiq as the adapter, but not if you are using a true Sidekiq worker. So if you opt-out of the Active Job flow, then you have to say, "Perform_async," and if you pass it a record, that's not going to work out particularly nicely. The other similar thing is that Sidekiq does not allow the use of keyword args, which, I'm going, to be honest, I really like keyword arguments, especially for background jobs or shuttling data through your system. And there's almost a lazy evaluation. I want some nicety to make sure that when I am putting something into a background job that I'm actually using the correct call signature, essentially passing the correct data in the correct shape. Am I passing a record, or am I passing the ID? Am I passing a list of options or a single option? Those sort of trade-offs that are really easy to subtly get wrong. I came around on this one because I realized although Active Job does support keyword arguments, the way it does that is it just has a JSON serialization format for them. So a keyword argument turns into a positional array with an associated hash that allows for the lookup or whatever. Basically, again, they handle the details. You get to use keyword args, which is great, with the exception that when you're actually calling performlater, that method performlater is a method missing type magic method. So it does not actually check the keyword arguments at that point. You're basically just passing an options hash as opposed to true keyword arguments that would error because they don't match up. And so when I figured that out, I was like, oh, never mind. This doesn't actually do the thing that I care about. It's a little bit nicer in terms of the signature of the method when you're defining your background job itself, but it doesn't actually do any logical checking. It doesn't give me any safety or robustness within my system. So I don't care about that. I did find a project called sidekiq-symbols, which does some things under the hood to how Sidekiq serializes and deserializes jobs, which I think gives largely the same behavior as Active Job. So I can now define my Sidekiq jobs with keyword arguments. Things will work. I can't use GlobalID. That's still out. But that's fine. I can do a little helper method that basically does the same thing as GlobalID or at least close approximation. But sidekiq-symbols lets me have keyword arg-like signatures in my methods; basically, it is. But again, it doesn't actually do any check-in when I'm enqueueing a job, and I am sad about that. STEPH: Yeah, that's another interesting distinction. And I'm unsurprisingly with you that I would favor having keyword args and having that additional safety in place. Okay, so I've been keeping track. And so far, it sounds like we have two points because I'm doing a little scorecard here between Active Job and Sidekiq. And we have two points in favor of Active Job because they offer a GlobalID, which then allows us to pass in a record, and then it takes care of the serialization for us. And then also, keyword args, which I agree with you that's a really nice feature to have in place as well. So I'm curious, so it sounded like you're leaning towards Active Job, but I don't want to spoil the ending. CHRIS: Yes, I could see why that's what you would be taking away from the conversation thus far. So again, just to reiterate, Active Job and Sidekiq with this sidekiq-symbols extension they both support keyword args, kind of. They support defining your job with keyword args and then enqueueing a job passing something that looks like keyword args. But it ends up...nobody's actually checking anything, so it's mostly like a syntactic nicety as opposed to any sort of correctness, which is still nicer, but it's not the thing that I actually want. Either way, nobody supports it, so it is not available to me. Therefore, it is not a consideration point. The GlobalID thing is nice, but it is really, again, it's a nicety more than anything. I have gone, and I'm leaning in the direction of full Sidekiq and Sidekiq everywhere as opposed to Active Job in most cases, but then Sidekiq when we need it. And that's because Sidekiq just has a lot more power and a lot more functionality. So, in particular, Sidekiq has a feature which allows you to say...it's a block that you put at the top of your Sidekiq job that says retries exhausted or something. I think Sidekiq retries exhausted is the actual full name of that at that point, which is really unfortunate in my mind, but anyway, I'll deal. At that point, you know that Sidekiq has exhausted all of the retries, and you can treat it as failed. I'm going, to be honest, I went on a quest to find a way to say, hey, I'm going to put some work into the background. It's really important for me to know if this work succeeds or if it fails. It's very easy to know if it succeeds because that just happens in-line in the method. But we can have an exception raised at basically any point; Sidekiq does a great job of catching those, of retrying, of having fundamental mechanisms there. But this is the best that I can get for this job failed. And so Active Job, as far as I can tell, does not have anything for this in order to say, yep, we are done. We are not going to keep working on this. This work has failed. It is dead. Dead is; actually, I think the more correct term for where we're at because failed is a temporary state, and then you retry after a failure. Whereas dead is, this has gone through all of its retries, and it will never be run again. Therefore, we should treat this as not having run. And in my case, the thing that I want to do is inform the user that this operation that we were trying to do on their behalf has not succeeded, will not succeed. And please reach out or otherwise deal with the fact that we were unable to do the thing that they asked us to do. That feels like a really important thing for me to be able to do, to be able to communicate back to my users. This is one of those situations where I'm looking at the available options, and I'm like, I feel like I can't be the only one who wants to know when something goes wrong. This feels like a thing that's important. But this is the best example that I've found, the Sidekiq retries exhausted block. And unfortunately, when I'm using it, it gets yielded the Sidekiq JSON blob deserialized, so it's like Ruby hash. But it's still like this blob of data. It's not the same data that gets passed into perform. And so, as a result, when I want to look up the record that was associated with it, I have to do this nested dig into the available hash of data. And it just feels like this is not a well-paved path. This is not something that is a deeply thought about or recommended use case. But again, I don't feel like I'm doing something weird here. Am I doing something weird, Steph, wanting to tell my users when I was unable to do the thing they asked me to do? [chuckles] STEPH: That feels like a very rhetorical question. [laughs] CHRIS: It does. I apologize. I'm leading the witness. But in your sincere heart of hearts, what do you think? STEPH: No, that certainly doesn't sound weird. I'm actually thinking back to some of the jobs that cause me stress in regards to knowing when they failed and then having that communication of knowing that we've exhausted all the retries. And, of course, knowing when those retries are exhausted is incredibly helpful. I am intrigued, though,, because you're highlighting that Active Job doesn't have the same option around setting the retry. And I'm trying to recall exactly how it's set. But I feel like I have set the retry count for Active Job. And maybe, as you mentioned before, that's because it's an abstraction, or I'm not sure if Active Job actually has that native support. So I feel a little confused there where I think my default instinct would have been Active Job does have that retry capability. But it sounds like you've discovered otherwise. CHRIS: I'm not actually sure what Active Jobs core retry logic or option looks like. So fundamentally, as far as I understand it, Active Job is an abstraction. And under the hood, you're always connecting an adapter. So it's either going to be Sidekiq, or Resque, or Delayed Job, or other. And each of those systems, whichever system you have as the adapter, is the one that's actually going to be managing retries. And so I know Sidekiq happens to have as a default 25 retries. And that spans, I think it's a two-week exponential back off. And Sidekiq has some very robust logic that they have implemented as the way retries exist within Sidekiq. I'm not sure what that would look like if you're trying to express it abstractly because it is slightly different. I know there was some good work that was done on Sidekiq to allow the Sidekiq options that's a method at the top level of the job, even if it's an Active Job job to express the retries. So that may be what you've seen, or there may be truly an abstraction that exists within Active Job, and then each adapter needs to know how to handle retries. But frankly, the what can Sidekiq do that Active Job can't? There's a whole bunch of stuff around limiting when you would retry limiting, enqueuing a job if there already exists one, when and how do those records get locked. There's a whole bunch of stuff. Sidekiq has a lot of power under the hood. And so if we want to be leaning into that, that's why I'm leaning towards let's just be Sidekiq all the time. Let's become Sidekiq experts. Let's accept that as a deep architectural decision within the app as opposed to just relying on the abstraction. Because fundamentally, if we're just using Active Job, we're not going to have access to the full power of Sidekiq or whatever the underlying system is, so sort of that decision that I'm making, but I don't know specifically around the retries. STEPH: Okay, thanks. That's really helpful. It's been a while since I've had to make this decision. I'm really enjoying you sharing your adventure because I'm trying to think what's the risk? If you don't use Active Job, what are the trade-offs? And you'd mentioned some of them around the GlobalID and keyword args, which are some niceties. But overall, if you don't go with the abstraction, if you lean into Sidekiq, the risk is then you want to migrate to a different enqueuing service. And something that we talk about is mitigating that risk, so then you can swap it out. That's also something I have never done or encountered where we've had to make that change. And it feels like a very low risk in my mind. CHRIS: Sidekiq feels like the thing you would migrate to, not a thing you would migrate from. It feels like it is the most powerful. And if anything, I expect at some point we'll be upgrading to Sidekiq pro or enterprise or whatever the higher versions that you pay for, but you get more features there. So in that sense, that is the calculation. That's the risk trade-off in my mind is that we're leaning into this technology and coupling ourselves more closely to it. But I don't see that as one that will reassess in the same way that people talk about Active Record and it being an ORM. And it's like, oh, we're abstracting the database underneath, and I'm like, no, I'm not. I'm always using Postgres. Please do not take Postgres. I'm not going to switch over to MySQL next week. That's totally fine if you start on MySQL. It's unlikely you're going to port over to Postgres. We may port to an entirely…like it's a Cassandra column store with a Kafka queue, I don't know, something weird down the road. But it's not going to be swapping out Postgres for MySQL or vice versa. Like you said, that's probably not a change that's going to happen. But that I think is the consideration. The other consideration I have in my mind is Active Job is the abstraction that exists within Rails. And so I can treat it as the lowest common denominator, and folks joining the project, it's nice to have that familiarity. So perform_later is the method on the Active Job jobs, and it has a certain shape to it. People may be familiar with that. Mailers will automatically use Active Job just implicitly under the hood. And so there's a familiarity, a discoverability. It's just kind of up the middle choice. And so if I can stick with that, I think there's a nicety there. But in this case, I think I'm choosing I would like the power and consistency on the Sidekiq side, and so I'm leaning into that. STEPH: Yeah, that makes a lot of sense to me. And I liked the other example you provided around things that were not likely to swap out and Postgres, MySQL, your database being one of them. And in favor of an example that I do have for something that...I do enjoy wrapping. It's not something that I adhere to strictly, but I do enjoy it when I have the space to make this choice. So I do enjoy wrapping HTTPClients, not just because then I can swap it out for a different HTTPClient, which frankly, that's also rare that I do that. Once I choose an HTTPClient, I'm probably pretty happy, and I don't need to swap it out. But I really like being able to extend to the API specifically if they don't handle error responses in a way that I would like to or if they raise, and then I want to change the API to have a more thoughtful interface and where I don't have to rescue those errors. But instead, I can interact with this object that then represents an error state. So that was just one example that came to mind for things that I do enjoy having an abstraction around and not just so I can swap it out because that feels like a very low risk, but more frankly, so I can extend the API. CHRIS: I definitely share the I almost always wrap APIs, or I try and hide whatever the implementation detail whether it be HTTPParty, or Faraday or whatever it is that I'm using and trying to hide that deeply within the system. And then I have whatever API client that we define. And that's what we're interacting with. It's interesting that you bring up errors and exceptions there because that's the one other thing that has caused me this...what I'm describing now seems perhaps like, oh, here's just a list of pros and cons, a simple decision was made, and there we are. This represents some real soul searching on my part, if we will. And one of the last things that I ran into that was just so frustrating is that Sidekiq is explicitly built around the idea of exceptions; Sidekiq retries if there is an exception raised in the job, otherwise, it treats it as success, and that's it. That is the entirety of it. That is the story. But if you raise an exception in a job, then you can't test that job because now it's raising an exception. You can't test retries or this retry exhausted block that I'm trying to lean into. I'm like, I want to put that in a feature spec and say, oh, this job goes in the background, but it's in a failure state, and therefore, the user sees the failure message. Sorry, I can't do that because the only way to actually fail a job is via an exception. And I've actually gone to some links in this application to try to introduce more structured data flow. I've talked a bunch about the command objects and the dry-monads and all those things. And I've really loved them where I've gotten to use them. But then I run into one of these edge cases where Sidekiq is like, no, no, no, you can't do that. And so now I have parts of my system that very purposefully return data as opposed to raising an exception. And I just have to turn around and directly raise that failure as an exception, and it just feels less expressive. I actually just ran into the identical thing with Pundit. They have a little bit better control over it; I can choose whether or not I want the raising version or not. But I see exceptions everywhere, and I want a little more discrete data flow. [chuckles] That is my dream. So anyway, I chose Sidekiq is the summary here. And slowly, we're going to migrate entirely to Sidekiq. And I'm going to be totally fine with it. And I'm done griping now. STEPH: This is your own little October Halloween movie, that I see exceptions everywhere. CHRIS: They're so spooky. STEPH: [laughs] That's cool about Pundit. I'm not sure I knew that, that you get to essentially turn on or off that exception flow behavior. On one hand, I'm like, that's nice. You get the option. On the other hand, I'm like, well, let's just not do it. Let's just never raise on people. But at least they give people options; that seems really cool. CHRIS: They do give the option. I think you can choose different strategies there. And also, if we're being honest, I'm newer to Pundit. And I used a different thing, which was to get the Policy Object and ask it a question. I wanted to ask, is this enabled or not? Can a user do this or not? That should not raise an exception. I'm just asking a question. We're just being real chill about this. I just want to know some information. Let's flow some data through our system. We don't need exceptions for that. STEPH: Why are you yelling at me? I just have a question. [laughs] CHRIS: Yeah. I figured out how to be easy on that front. Sidekiq apparently has no be easy mode, but that's fine. You know what? We're going to make it work, and it's going to be fine. But it is interesting deciding which of these facets of the system that I'm building do I really care about? Which are the ones where I'm like, whatever, just pick something, and we'll move forward, it's not a big deal? Versus, we're actually going to be doing a lot of work in the background. This is the thing that I care about deeply. I want to know about failure and success. I want to really understand that and have a robust answer to what our architecture looks like there. Similarly, Pundit for authorization. I believe that authorization will be a critical aspect of our system. It's typically a pretty important thing. But for us, I think we're going to have different types of users who can log in and see different subsets of data and having a consistent and concrete way that we have chosen to implement that we are able to test, that we're able to verify. I think that's another core competency within the app. But you only get to have so many of those. You can only be really good at a couple of things. And so I'm in that place where I'm like, which are our top five when I say are the things that I care a lot about? And then which are the things where I'm like, I don't know, whatever, just run with it? STEPH: Just a little bit ago, I came so close to singing because you said the I want to know phrase again. And that, I'm realizing, [laughs] is a trigger for me and a song where I want to sing. I held it back this time. CHRIS: It's smart. You got to learn anytime you sing on mic that is part of the permanent record. STEPH: Edward Loveall at thoughtbot, since I sang in a recent episode, did the delightful thing where then he grabbed that clip of where you talk a little bit, and then I sing and then encouraged everyone to go listen to it. And in which I responded, like, I would highly recommend that you save your ears and don't listen to it. But yes, singing on the mic is a thing. I do it from time to time. I can't hold it back. CHRIS: We all do. But since it doesn't seem that you're going to sing in this moment, I think I can probably wrap up my Odyssey of choosing between Sidekiq and Active Job. I hope those details were useful to anyone other than me. It was an adventure, so I figured I'd share it. But yeah, that about wraps it up on my side. Mid-roll Ad And now a quick break to hear from today's sponsor, Scout APM. Scout APM is leading-edge application performance monitoring that's designed to help Rails developers quickly find and fix performance issues without having to deal with the headache or overhead of enterprise platform feature bloat. With a developer-centric UI and tracing logic that ties bottlenecks to source code, you can quickly pinpoint and resolve those performance abnormalities like N+1 queries, slow database queries, memory bloat, and much more. Scout's real-time alerting and weekly digest emails let you rest easy knowing Scout's on watch and resolving performance issues before your customers ever see them. Scout has also launched its new error monitoring feature add-on for Python applications. Now you can connect your error reporting and application monitoring data on one platform. See for yourself why developers call Scout their best friend and try our error monitoring and APM free for 14 days; no credit card needed. And as an added-on bonus for Bike Shed listeners, Scout will donate $5 to the open-source project of your choice when you deploy. Learn more at scoutapm.com/bikeshed. That's scoutapm.com/bikeshed. STEPH: So, I would love to talk about an SSL error that I encountered recently. So one of the important processes in our application is sending data to another system. And while sending data to that other system, we started seeing the following error that the read "Certificate verify failed." And then in parens, it states, "Unable to get local issuer certificate." So upon seeing that error, I initially thought, okay, something is wrong with their SSL certificate or their SSL configuration. And that's not something that I have control over and can fix. So we should reach out and let them know to take a look at their SSL config. But it turns out that their team already knew about the issue. They had recently updated or renewed their SSL cert, and they saw our messages were no longer being processed, and they were reaching out to us for help. So at that point, I'm still pretty sure that it's related to something on their end, and it's not something that I can really fix on our end. But we can help them troubleshoot. Maybe there's a workaround that we can add to still get messages processing while they're looking into their SSL config. It seemed like they still just needed help. So it was something that was still worth diving into. So going back to the first error, I want to talk a little bit about it because I realized that I understand SSL just enough, just the surface to get by as a developer. But then, every time that I run into a specific error with it, then I really have to refresh my understanding as to what could be wrong, so then I can troubleshoot more effectively. So for anyone that could use a refresher on that certificate verification process, when your browser or your server is connecting to a site that uses SSL, then your browser server, whichever one you're using, is going to download that site certificate and verify a couple of things. So it's going to check does the certificate contain the domain name of the website? So essentially, you gave us a certificate. Is this your certificate? Does it match the site that we're connecting to? Is this cert issued by a trusted certificate authority? So did someone that we trust give you this certificate? And is the cert still valid, or has it expired? So that part is pretty straightforward. The second part, "Unable to get local issuer certificate," so that's the part I was less certain about. And I took this to mean that they had passed two of those three checks that their cert included the site's name, and it had not expired. But for some reason, we aren't able to determine if their cert was issued by someone that we should trust. So following that journey, my next question was, so what are they giving us? So this is a tool that I don't get to use very often, but I reached for OpenSSL and, specifically, the s_client command, which connects to a specified domain and prints all certificates in the certificate chain. You may already know this, but the certificate chain is basically a fancy way of saying, show me all the certificates necessary to prove your site certificate was authorized by a trusted certificate authority. CHRIS: I did not know that. STEPH: Okay, I honestly didn't either. [laughs] CHRIS: I liked that you thought I would, though. So thank you, but no. [chuckles] STEPH: Yeah, it's one of those areas of SSL where I know just enough. But that was something that was new to me. I thought there was a site certificate, and I didn't realize that there is this chain of certificates that has to be honored. So going back and looking through that output of the certificate chain, that's what highlighted to me that their server was giving us their certificate and saying, hey, you should trust our site certificate. It's legit because it was authorized by, let's say, XYZ certificate. And so if it were a proper certificate chain, then they would give us that XYZ cert. And essentially, we can use this chain of certificates to get back to a trusted authority that then everybody knows that we can trust. However, they weren't actually giving us a reference certificate; they were giving us something else. So essentially, they were saying, "Hey, look at our certificate and look at this very trustworthy reference that we have." But they're actually failing to give us that reference. So to bring it all home, we can download that intermediate certificate that they reference; that is something that is publicly accessible. That's why we're able to then verify each certificate that's provided in that chain. We could go and download that intermediate certificate from that certificate authority. We could combine that with their site-specific certificate, include that in our request to their system, and then complete the certificate chain. And boom, we're back in business. But it was quite a journey. CHRIS: That is quite the journey. And yeah, I definitely knew very little of that, although everything you're saying makes sense. And I have a bunch of cubbyholes in my brain for SSL knowledge. And the words you said all fit into the spaces that I have in my brain, but I didn't know a bunch of those pieces. So thank you for sharing that. SSL and cryptography, more generally or password hashing or things like that, occupy this special place in my brain where I'm both really interested in them. And I will occasionally research them. If I see a blog article, I'll be like, oh yeah, I want to read more about this password hashing. And what's a Salt? And what's a Pepper? And what are we doing there? And what is BCrypt versus SCrypt? What are all these things? This is cool. And almost the arms race on the two sides of how do we demonstrate trust in a secure manner on the internet? But at the same time, I am not allowed to do anything with this information. I outsource this as much as humanly possible because it's one of those things that you just should not do yourself and SSL perhaps even more so. So I have configured aspects of my password hashing. But I 100% just lean on the fact that Let's Encrypt exists in the world. And prior to that, it was a little more work. But frankly, earlier on in my career, I wasn't dealing with the SSL parts of things. But I'm so grateful to Let's Encrypt as a project that exists. And now, on almost every platform that I work with, there's just a checkbox for please do the SSL work for me, make it good, make it work, and then I will be happy. And I'm so glad that that organization exists and really pushed the envelope also. I forget what it was, but it was only like three years ago where SSL was not actually nearly as common as it is now. And now it is pervasive and everywhere. And all of the sites have it, and so that is a wonderful thing. But I don't actually know much. I know that I should have it. I must have it. I should force it. That's true. So I push that out… STEPH: Hello. CHRIS: Are you trying to get me to sing? [chuckles] STEPH: [laughs] No, but I did want to know if you get the reference, the Salt-N-Pepa. CHRIS: Push It Real Good the song? Yeah, okay. STEPH: Yeah, you got it. [chuckles] CHRIS: I will just say the lyrics. I shall not sing the lyrics. I would say that, though, that yes, yes, they do that. STEPH: Thank you for acknowledging my very terrible reference. Circling back just a little bit too in regards to...I'm with you; this is a world that is not one that I am very deeply technical in and something that I learned a fair amount while troubleshooting this particular SSL error. And it was very interesting. But there's also that concern where it's like, that was interesting. And we worked around the issue, but this also feels very fragile. So we still haven't fixed it on their end where they are sending the wrong certificate. So then that's why we had to do more investigative work, and then download the certificate that they meant to send us, and then send back a complete certificate chain so that we don't have this error anymore. But should they change anything about their certificate, should they renew anything like that, then suddenly, we're going to break again. And then, the next developer is going to have to go through the same journey. And this wasn't a light journey. This was a good half-day journey to figure out what was going on and to spend the time, and then to also get that fix out to production. So it's a meaningful task that I don't want anyone else to have to go through. But we are relying on someone else updating their configuration. So, on one hand, we're in a good spot until they are able to update. But on the other hand, I wrote a heck of a commit message for the next person just describing like, friend, just grab some coffee if we're going to chat. It's a very small code change, but you need to know the scoop. So should you need to replicate this because they've changed something, or if this happens…because we work with a number of systems that we send data to. So if someone else should run into a similar issue, they will understand some of the troubleshooting techniques that I used and be able to look up that chain and find out if there's a missing cert or something else they need to provide. So it feels like a win, but I'm also nervous for future selves, future developers. So there's another approach that I haven't mentioned yet, but it was often a top recommendation for when dealing with SSL errors. And specifically, it was turning off SSL verification. And I saw that, and I was like, well, that won't work. I'm definitely sending sensitive, important data. And I need to verify that who I'm sending this to is really the person that I want to send this data to. So that was not an option for me. But it made me very nervous how often that was an approach that people would recommend and be like, oh, it's okay, just turn off SSL. You'll be fine. Like, don't worry about it. CHRIS: I feel like this so perfectly fits into the...some of our work is finding the information and connecting the pieces together and making it work. But some of it is that heuristic sense, that voice in the back of your head that is like, wait, I'm sorry, what? You want me to just turn off the security perimeter and hope that the velociraptors won't come in? That doesn't seem like it's going to end well. I get that that's an easy option that we have available to us right now and will solve the immediate problem but then let's play this out. There are four or five Jurassic Park movies now that tell the story of that. So let's be careful. STEPH: It always ends super well, though, right? Like, it's totally fine. [laughs] CHRIS: [laughs] Exclusively. Although it's funny that you mentioned OpenSSL no verify because just this past week, I used that very same configuration. I think it was okay in my case; I'm pretty sure. But it is interesting because when I saw it, I was like, oh no, can't do that. Certainly not that. Don't turn off the security feature. That's the wrong way to deal with the issue. But in the particular case that I'm working with, I'm using Redis, Heroku Redis, in particular, in a Heroku configuration. And the nature of how Heroku configures the Redis instances and the connectivity to our app into our dyno...I forget why. I read an article. They wrote it; Heroku wrote it. I trust them; they're good. I've outsourced my trust to people that I do trust. The trust chain actually maps really well to the certificate trust chain. I trust that Heroku has taken security deeply seriously. And for some reason, their configuration of Redis requires that I turn on OpenSSL no verify mode. So I'm using this now both in Sidekiq, and then we're using our Redis instance for our Rails cache as well. So in both cases, I said, "It's fine. Don't worry about it." I used the Don't worry about it configuration. And I didn't love it but I think it's okay. And partly, I'm trying to say this into the internet radio right now just in case anyone's listening who's like, no, no, no, you can't do that. That's bad. So I'm willing to be deeply wrong on the internet in favor of someone telling me and then I get to get out in front of it. But I think it's fine. Pretty sure it's fine. It should be fine. STEPH: I love love love that you gave a very visual example of velociraptors, and then you're like, oh, but I turned it off. [laughs] So I'm going to start sending you a velociraptor gif each day. CHRIS: I hope you do. I hope the internet holds you accountable to that. STEPH: [laughs] CHRIS: And I really look forward to [laughs] moving forward because that's a great way to start the day. Well, it doesn't need to start the day, but I look forward to them. STEPH: [laughs] I am really intrigued because I'm with you. Like you said, there are certain entities that are in our trust chain where it's like, hey, you are running this for us, and so I do have faith and trust in you that you wouldn't steer me wrong and provide a bad recommendation. Someone on Stack Overflow telling me to turn off SSL verify uh; that's not my trust chain. Heroku or someone else telling me I'm going to take it a little more seriously. And so I'm also interested in hearing from...what'd you say? You're speaking into the internet phone. [laughs] What'd you say? CHRIS: I think I said internet radio. But yeah, in a way. I mean, we're recording over Skype right now. So in a manner of speaking, we're on the internet phone to make our internet radio show. STEPH: [laughs] Oh goodness, the internet radio. I'm also intrigued to hear if other people are like, oh, no, no, no. Yeah, that sounds like an interesting scenario. Because I would think you'd still want your connection to...you said it's for Redis. So you still want that connection to be verified. But then if Redis itself can't have a specific...yeah, we're testing the boundaries of my SSL knowledge here as to how the heck you would even establish that SSL connection or the verification process. CHRIS: Me too. And it also exists in an interesting space where Heroku is rather clear in their documentation about this. And it was a surprising claim when I saw it. And so, I don't expect them to be flippant about a thing that is important. Like, if they're like, "No, no, no, it is okay. You can turn off the security thing, don't worry." I trust that they're not just like, oh, we didn't think about it too much. But we figured why not? It's not a big deal. I'm sure that they have thought about it deeply because it is an important thing. And so in a weird way, my trust of them and the severity of what this thing represents, I'm like, oh yeah, I super trust that because you're not going to get a major thing wrong. You might get a minor, small, subtle thing wrong. But this is a pretty major configuration change. As I say it, I'm now getting more worried. I'm now like, I feel fine about this. This doesn't seem like a problem at all. But then I keep saying stuff, and I'm like, oh no. That's why I love having a podcast; I find out things about myself as I talk into a microphone to you. STEPH: We come here to share our deep, dark developer secrets. Chris: Spooky developer therapy. STEPH: But just to clarify, even though you've turned off the SSL verify, you're still connecting over SSL. CHRIS: Yes, I believe that's the case. And if I'm remembering, I think the nature of how this works is they're using a self-signed certificate because of shared infrastructure or something, something that made sense when I read it. But it was the idea that they are doing a self-signed certificate. Therefore, to what you were talking about earlier, there isn't the certificate authority in the chain of those because it's self-signed. And so, they are not a trusted certificate authority. Therefore, that certificate that they have generated would not be trusted. But it does still allow for the SSL handshake and then communication to happen over SSL. It's just that fundamental question of trust. I'm saying, in this case, for reasons, it's okay. Trust me that I trust them. We're good. Which, again, I don't feel great about, but I think yes, it is still SSL, but it is a self-signed certificate. So we have to make this configuration change. STEPH: Yeah, all of that makes sense. And it certainly sounds like you have been very thoughtful about that change and put in some investigative work. So on that note, I have a very unrelated bad joke for you. CHRIS: I'm very excited. STEPH: All right, here we go. All right, so what do you call an alligator wearing a vest? CHRIS: I don't know. What do you call an alligator wearing a vest? STEPH: An investigator. [laughter] On that note, shall we wrap up? CHRIS: Oh, let's wrap up. We should also include a link in the show notes to the episode where you told the joke about the elephant hiding in the trees because that's one of my favorite jokes. You slayed me with that one. [laughs] But on that note, yes, let us wrap up. The show notes for this episode can be found at bikeshed.fm. STEPH: This show is produced and edited by Mandy Moore. CHRIS: If you enjoyed listening, one really easy way to support the show is to leave us a quick rating or even a review in iTunes,,as it really helps other folks find the show. STEPH: If you have any feedback for this or any of our other episodes, you can reach us at @_bikeshed or reach me on Twitter @SViccari. CHRIS: And I'm @christoomey STEPH: Or you can reach us at hosts@bikeshed.fm via email. CHRIS: Thanks so much for listening to The Bike Shed, and we'll see you next week. All: Byeeeeeeeeee!!! Announcer: This podcast was brought to you by thoughtbot. thoughtbot is your expert design and development partner. Let's make your product and team a success.

Bob Quillin and Karthik Gaekwad are on the Oracle developer relations team. Karthik has been on Ruby Rogues previously, and he explains how he went from the Kubernetes team to developer relations. They begin the show by explaining what Kafka is, the leading open-source event streaming platform that Oracle is compatible with. It allows cloud developers to build, publish, and subscribe models for streams of records in addition to many other functions. Systems that used to take a long time to make have become very small and simple with Kafka. Kafka stands out from other message queueing systems because of its robust nature and scalability.  Bob goes into more depth about the evolution of Kafka and the panel discusses some different use cases, concluding that Kafka works best for projects with a large amount of data coming in and for making real-time decisions. Bob and Karthik talk about other things Kafka can do beyond the message queue, such as building streams from specific patterns. They talk about when you should consider moving over to Kafka. Karthik talks about how to get started with Kafka. One of the best ways to do this is to set up a service with Oracle and to just play around with it, which won’t cost you much if you aren’t pushing a lot of data through it. Bob and Karthik talk about some of the features offered by Oracle and Kafka. While the offerings are somewhat vanilla, you get the advantage of it being an open-source driven service on top of a cloud that’s highly secure, available, and built to last. The panel discusses security within Kafka. They talk briefly about the framework Karafka and tools and resources available through Oracle for Kafka. The show concludes with the panel talking about compatibility between Kafka and Docker. Panelists John Epperson Charles Max Wood Guests Bob Quillin Karthik Gaekwad Sponsors Cloud 66 - Pain-Free Rails Deployments Try Cloud 66 Rails for FREE & get $100 of free credits with promo code RubyRogues-19 RedisGreen ____________________________ > "The MaxCoders Guide to Finding Your Dream Developer Job" by Charles Max Wood is now available on Amazon. Get Your Copy Today! ____________________________________________________________ Links RR 397: Oracle with Karthik Gaekwad RR 402: Kafka and Karafka with Maciej Menfeld Kafka Redis Amazon SQS Sidekick Resque RabbitMQ Zookeeper Karafka Picks Charles Max Wood: Discord HubSpot John Epperson: Aberlour A'Bunadh The Witcher Bob Quillin: Balcones Distillery Milam and Greene Whiskeys The Overstory Karthik Gaekwad: Follow Karthik on LinkedIn, Github, and Twitter

Bob Quillin and Karthik Gaekwad are on the Oracle developer relations team. Karthik has been on Ruby Rogues previously, and he explains how he went from the Kubernetes team to developer relations. They begin the show by explaining what Kafka is, the leading open-source event streaming platform that Oracle is compatible with. It allows cloud developers to build, publish, and subscribe models for streams of records in addition to many other functions. Systems that used to take a long time to make have become very small and simple with Kafka. Kafka stands out from other message queueing systems because of its robust nature and scalability.  Bob goes into more depth about the evolution of Kafka and the panel discusses some different use cases, concluding that Kafka works best for projects with a large amount of data coming in and for making real-time decisions. Bob and Karthik talk about other things Kafka can do beyond the message queue, such as building streams from specific patterns. They talk about when you should consider moving over to Kafka. Karthik talks about how to get started with Kafka. One of the best ways to do this is to set up a service with Oracle and to just play around with it, which won’t cost you much if you aren’t pushing a lot of data through it. Bob and Karthik talk about some of the features offered by Oracle and Kafka. While the offerings are somewhat vanilla, you get the advantage of it being an open-source driven service on top of a cloud that’s highly secure, available, and built to last. The panel discusses security within Kafka. They talk briefly about the framework Karafka and tools and resources available through Oracle for Kafka. The show concludes with the panel talking about compatibility between Kafka and Docker. Panelists John Epperson Charles Max Wood Guests Bob Quillin Karthik Gaekwad Sponsors Cloud 66 - Pain-Free Rails Deployments Try Cloud 66 Rails for FREE & get $100 of free credits with promo code RubyRogues-19 RedisGreen ____________________________ > "The MaxCoders Guide to Finding Your Dream Developer Job" by Charles Max Wood is now available on Amazon. Get Your Copy Today! ____________________________________________________________ Links RR 397: Oracle with Karthik Gaekwad RR 402: Kafka and Karafka with Maciej Menfeld Kafka Redis Amazon SQS Sidekick Resque RabbitMQ Zookeeper Karafka Picks Charles Max Wood: Discord HubSpot John Epperson: Aberlour A'Bunadh The Witcher Bob Quillin: Balcones Distillery Milam and Greene Whiskeys The Overstory Karthik Gaekwad: Follow Karthik on LinkedIn, Github, and Twitter

Bob Quillin and Karthik Gaekwad are on the Oracle developer relations team. Karthik has been on Ruby Rogues previously, and he explains how he went from the Kubernetes team to developer relations. They begin the show by explaining what Kafka is, the leading open-source event streaming platform that Oracle is compatible with. It allows cloud developers to build, publish, and subscribe models for streams of records in addition to many other functions. Systems that used to take a long time to make have become very small and simple with Kafka. Kafka stands out from other message queueing systems because of its robust nature and scalability.  Bob goes into more depth about the evolution of Kafka and the panel discusses some different use cases, concluding that Kafka works best for projects with a large amount of data coming in and for making real-time decisions. Bob and Karthik talk about other things Kafka can do beyond the message queue, such as building streams from specific patterns. They talk about when you should consider moving over to Kafka. Karthik talks about how to get started with Kafka. One of the best ways to do this is to set up a service with Oracle and to just play around with it, which won’t cost you much if you aren’t pushing a lot of data through it. Bob and Karthik talk about some of the features offered by Oracle and Kafka. While the offerings are somewhat vanilla, you get the advantage of it being an open-source driven service on top of a cloud that’s highly secure, available, and built to last. The panel discusses security within Kafka. They talk briefly about the framework Karafka and tools and resources available through Oracle for Kafka. The show concludes with the panel talking about compatibility between Kafka and Docker. Panelists John Epperson Charles Max Wood Guests Bob Quillin Karthik Gaekwad Sponsors Cloud 66 - Pain-Free Rails Deployments Try Cloud 66 Rails for FREE & get $100 of free credits with promo code RubyRogues-19 RedisGreen ____________________________ > "The MaxCoders Guide to Finding Your Dream Developer Job" by Charles Max Wood is now available on Amazon. Get Your Copy Today! ____________________________________________________________ Links RR 397: Oracle with Karthik Gaekwad RR 402: Kafka and Karafka with Maciej Menfeld Kafka Redis Amazon SQS Sidekick Resque RabbitMQ Zookeeper Karafka Picks Charles Max Wood: Discord HubSpot John Epperson: Aberlour A'Bunadh The Witcher Bob Quillin: Balcones Distillery Milam and Greene Whiskeys The Overstory Karthik Gaekwad: Follow Karthik on LinkedIn, Github, and Twitter

Sponsors Sentry use code “devchat” for $100 credit Cloud 66 - Pain Free Rails Deployments: Try Cloud 66 Rails for FREE & get $66 free credits with promo code RubyRogues RedisGreen Panel Charles Max Wood Nate Hopkins Andrew Mason With Special Guest: Kir Shatrov Episode Summary Today’s guest Kir Shatrov is a production engineer on Shopify based in London, UK. Today, he and the panel are discussing capacity planning. Kir believes that capacity planning becomes a priority when your company starts losing money and your customers are suffering. When someone does get to the point of scaling their app, it’s important to look at the limitations of the hosting service. It is also important to remember that scaling is not a job that ever completes. Kir talks about his experience and time with Shopify and what types of changes have happened in the four years he’s been with the company. Kir explains that when Shopify was founded about 12 years ago, they were some of the first contributors to Rails, and Rails was just a zip file they shared over an email. This is important to know because the monolith code for Shopify has never been rewritten, so they put a lot of care into keeping it working. He talks about some of the techniques Shopify uses to avoid splitting into microservices when scaling their organization and how the multiple instances of the database is structured and managed from an opps point of view. He talks about what aspects of Shopify are open source and the approach to the architecture of the background jobs system. The panel discusses what should be done if you want to scale your project and move away from background jobs. Kir talks about what criteria his company uses to determine what moves to a background job and when it is too much to background something. The show finishes with Kir sharing some of his favorite tips, tricks, and approaches he’s used at Shopify.   Links Heroku Digital Ocean Sidekiq Shopify job iteration Redis PostgreSQL Resque Strace  RB Spy RB Trace New Relic Data Dog Fat Free CRM Follow DevChat on Facebook and Twitter Picks Nate Hopkins: Open source Cats (Maine Coon and Russian Blue) Charles Max Wood: 75 Hard challenge Kirill Shatrov:  Ruby Hack Challenge Follow Kirill Shatrov @kirshatrov on Twitter and @kirs on Github

Sponsors Sentry use code “devchat” for $100 credit Cloud 66 - Pain Free Rails Deployments: Try Cloud 66 Rails for FREE & get $66 free credits with promo code RubyRogues RedisGreen Panel Charles Max Wood Nate Hopkins Andrew Mason With Special Guest: Kir Shatrov Episode Summary Today’s guest Kir Shatrov is a production engineer on Shopify based in London, UK. Today, he and the panel are discussing capacity planning. Kir believes that capacity planning becomes a priority when your company starts losing money and your customers are suffering. When someone does get to the point of scaling their app, it’s important to look at the limitations of the hosting service. It is also important to remember that scaling is not a job that ever completes. Kir talks about his experience and time with Shopify and what types of changes have happened in the four years he’s been with the company. Kir explains that when Shopify was founded about 12 years ago, they were some of the first contributors to Rails, and Rails was just a zip file they shared over an email. This is important to know because the monolith code for Shopify has never been rewritten, so they put a lot of care into keeping it working. He talks about some of the techniques Shopify uses to avoid splitting into microservices when scaling their organization and how the multiple instances of the database is structured and managed from an opps point of view. He talks about what aspects of Shopify are open source and the approach to the architecture of the background jobs system. The panel discusses what should be done if you want to scale your project and move away from background jobs. Kir talks about what criteria his company uses to determine what moves to a background job and when it is too much to background something. The show finishes with Kir sharing some of his favorite tips, tricks, and approaches he’s used at Shopify.   Links Heroku Digital Ocean Sidekiq Shopify job iteration Redis PostgreSQL Resque Strace  RB Spy RB Trace New Relic Data Dog Fat Free CRM Follow DevChat on Facebook and Twitter Picks Nate Hopkins: Open source Cats (Maine Coon and Russian Blue) Charles Max Wood: 75 Hard challenge Kirill Shatrov:  Ruby Hack Challenge Follow Kirill Shatrov @kirshatrov on Twitter and @kirs on Github

Sponsors Sentry use code “devchat” for $100 credit Cloud 66 - Pain Free Rails Deployments: Try Cloud 66 Rails for FREE & get $66 free credits with promo code RubyRogues RedisGreen Panel Charles Max Wood Nate Hopkins Andrew Mason With Special Guest: Kir Shatrov Episode Summary Today’s guest Kir Shatrov is a production engineer on Shopify based in London, UK. Today, he and the panel are discussing capacity planning. Kir believes that capacity planning becomes a priority when your company starts losing money and your customers are suffering. When someone does get to the point of scaling their app, it’s important to look at the limitations of the hosting service. It is also important to remember that scaling is not a job that ever completes. Kir talks about his experience and time with Shopify and what types of changes have happened in the four years he’s been with the company. Kir explains that when Shopify was founded about 12 years ago, they were some of the first contributors to Rails, and Rails was just a zip file they shared over an email. This is important to know because the monolith code for Shopify has never been rewritten, so they put a lot of care into keeping it working. He talks about some of the techniques Shopify uses to avoid splitting into microservices when scaling their organization and how the multiple instances of the database is structured and managed from an opps point of view. He talks about what aspects of Shopify are open source and the approach to the architecture of the background jobs system. The panel discusses what should be done if you want to scale your project and move away from background jobs. Kir talks about what criteria his company uses to determine what moves to a background job and when it is too much to background something. The show finishes with Kir sharing some of his favorite tips, tricks, and approaches he’s used at Shopify.   Links Heroku Digital Ocean Sidekiq Shopify job iteration Redis PostgreSQL Resque Strace  RB Spy RB Trace New Relic Data Dog Fat Free CRM Follow DevChat on Facebook and Twitter Picks Nate Hopkins: Open source Cats (Maine Coon and Russian Blue) Charles Max Wood: 75 Hard challenge Kirill Shatrov:  Ruby Hack Challenge Follow Kirill Shatrov @kirshatrov on Twitter and @kirs on Github

On this week's episode, we revisit RailsConf 2019 for another live recording, this time with Eileen M. Uchitelle, GitHubber and rails core team member. Eileen joins Chris to discuss her RailsConf talk on how GitHub maintained a custom fork of Rails for years, how they finally moved off it, and what lessons we can take away from their experience. They also discussed Eileen's recent work on automatic database switching coming in Rails 6, microservices and monoliths, and getting into working on Rails. This episode of The Bike Shed is sponsored by Indeed Prime Eileen M. Uchitelle - eileencodes Eileen's talk - The Past, Present, and Future of Rails at GitHub Rails 6 connection switching for databases Circuit break pattern ActiveJob Resque The Success of Open Source ActiveRecord Enums ActionCable S3 Service Disruption Indident IOT DDOS on DNS Aaron Patterson

Volunteering and being involved in the community is important for Jeff, Dennis, and Peggy.  In fact so important that they are dedicating a whole episode to talk about local non-profits that make an impact in their community.  There’s a wide diversity of organizations and the ones mentioned in this episode are only the tip of the iceberg. Check out our show notes for a list of more local and nationwide nonprofits!

Panel: Charles Max Wood Mark Erikson Josh Adams Eric Berry Special Guests: Sam Davies In this episode of Elixir Mix, the panel talks to Sam Davies about asynchronicity in Elixir. Sam has been programming for about six years and got into Elixir about a year ago. Before working with Elixir, he was a Ruby programmer and he currently works for a company called Nested and introduced them to Elixir there. They talk about asynchronous programming, different Elixir libraries, and his creation Rihanna. In particular, we dive pretty deep on: Sam intro ProGolfMe Startup Contributor to Rails and Elixir core Worked in Ruby before Elixir Loved the Ruby community Why Elixir? Asynchronous programming Understanding the BEAM Erlang Idea of supervisors What you do when a job fails Is the company going to lose money if we implement this? Exq library Resque and Sidekiq Issues with Exq Codefund Rihanna Rihanna inspired by Que Delayed Job And much, much more! Links: Elixir Nested ProGolfMe Rails Ruby Why Elixir? Erlang Exq Resque Sidekiq Codefund Rihanna Que Delayed Job SamuelDavies.net Sam’s GitHub @samphilipd Picks: Charles Mattermost Documenting processes for the podcasts Spend time with the people you care about JavaScript Jabber, Adventures in Angular, React Round Up, and Views on Vue Mark gproc Eric CodePilot Working from home Josh Talk: Efficient data loading in Elixir using the deferrable pattern Event Sourcing made Simple by Philippe Creux Sam Node.js and Elixir presentation Talk: Elixir Umbrella

Panel: Charles Max Wood Mark Erikson Josh Adams Eric Berry Special Guests: Sam Davies In this episode of Elixir Mix, the panel talks to Sam Davies about asynchronicity in Elixir. Sam has been programming for about six years and got into Elixir about a year ago. Before working with Elixir, he was a Ruby programmer and he currently works for a company called Nested and introduced them to Elixir there. They talk about asynchronous programming, different Elixir libraries, and his creation Rihanna. In particular, we dive pretty deep on: Sam intro ProGolfMe Startup Contributor to Rails and Elixir core Worked in Ruby before Elixir Loved the Ruby community Why Elixir? Asynchronous programming Understanding the BEAM Erlang Idea of supervisors What you do when a job fails Is the company going to lose money if we implement this? Exq library Resque and Sidekiq Issues with Exq Codefund Rihanna Rihanna inspired by Que Delayed Job And much, much more! Links: Elixir Nested ProGolfMe Rails Ruby Why Elixir? Erlang Exq Resque Sidekiq Codefund Rihanna Que Delayed Job SamuelDavies.net Sam’s GitHub @samphilipd Picks: Charles Mattermost Documenting processes for the podcasts Spend time with the people you care about JavaScript Jabber, Adventures in Angular, React Round Up, and Views on Vue Mark gproc Eric CodePilot Working from home Josh Talk: Efficient data loading in Elixir using the deferrable pattern Event Sourcing made Simple by Philippe Creux Sam Node.js and Elixir presentation Talk: Elixir Umbrella

今回はこんな会議室での収録でした。 対話式UIの「ペライチ」プロトタイプ iTunesで購読する 株式会社ホットスタートアップ代表取締役の橋田さんとこれまでのキャリア、利用しているテクノロジースタックについて話しました。 橋田一秀（@hassy0607） 山本大策（@daisaku） 「今回のキーワード」MashupAwardsハッカソンで出会う/ペライチ/Webサービスカタログ/Product Hunt/SIerに就職/みずほ銀行次期システム/ペライチを支える技術/インスタントチームで利用したサービス/チャットボット/対話式UI/Pokemon Go/ライブ配信はじめました ペライチ Product Hunt みずほ銀行次期システム関連のまとめ Webサービス「ペライチ」を支える技術とチームとその変遷 「ペライチ」のスタックフロント： BackboneJSサーバー： CakePHPインフラ： AWSデプロイ： Capistrano開発環境： Vagrant, flyJSユーザー行動解析： Mixpanelソースコード管理： GitHubその他開発ツール： SideCI, CircleCI, Ansibleコミュニケーション： Slackプロジェクト管理： Trelloドキュメント共有： Qiita::Team, esa.io 「インスタントチーム」のスタックフレームワーク: Ruby On Railsプラットフォーム: Herokuデータベース: PostgreSQL、Redisメール配信: SendGrid画像サーバ: Cloudinary検索: Algoliaリアルタイム: PusherWebプッシュ通知: Roost決済: WebPay、Stripeキューイング: Resqueログ監視: Logentriesパフォーマンス監視: NewRelicバージョン管理: GitHub 断言しよう、チャットボットブームは去るし関連ビジネスも失敗するよ Pokemon Go このポッドキャストへのお問い合わせ、ご感想、ご質問、ご要望は右下のお問い合わせからメッセージしてください。Twitterの場合は #prototypefm をつけてツイートしてください。 Special Guest: 橋田 一秀.

Check out Angular Remote Conf and RailsClips!   03:15 - Mike Perham Introduction Twitter GitHub Blog Contributed Systems sidekiq dalli 03:43 - Sidekiq Overview resque JRuby 05:18 - Job Runners vs Queuing Systems, Background Jobs RabbitMQ sneakers 08:47 - Performance celluloid 09:49 - celluloid vs Ruby Threads 11:47 - The GIL (Global Interpreter Lock) 12:49 - Passing Data 14:01 - Performance Boost From Using JRuby? 15:48 - The Actor Model revactor Rubinius girl_friday 20:39 - Sidekiq Roadmap Statistics & History 21:44 - Sidekiq Enterprise 27:58 - Sidekiq vs Resque Scheduled-Jobs 29:50 - Adding Features to Sidekiq 30:28 - “Unique Job” 31:17 - Idempotency Sidekiq Best Practices Page 33:12 - Mixing In Other Data Stores Redis Kafka Apollo 38:42 - Encoding 40:04 - Format 40:36 - The Active Job Adapter 41:23 - Making Open Source Viable and Sustainable 44:04 - Launching An Open Source Project Kickstarter BSD & LGPL Licences Picks Mike Hoye: Citation Needed (David) Code Master (Coraline) Robot Turtles (Coraline) Zalando STUPS (Jessica) Elevator Saga (Chuck) Developer On Fire: Episode 017 - Charles Max Wood - Get Involved and Try New Things (Chuck) Model View Culture (Mike) Plasso (Mike) James Mickens: Not Even Close: The State of Computer Security (with slides) from NDC Conferences (Mike)

Check out Angular Remote Conf and RailsClips!   03:15 - Mike Perham Introduction Twitter GitHub Blog Contributed Systems sidekiq dalli 03:43 - Sidekiq Overview resque JRuby 05:18 - Job Runners vs Queuing Systems, Background Jobs RabbitMQ sneakers 08:47 - Performance celluloid 09:49 - celluloid vs Ruby Threads 11:47 - The GIL (Global Interpreter Lock) 12:49 - Passing Data 14:01 - Performance Boost From Using JRuby? 15:48 - The Actor Model revactor Rubinius girl_friday 20:39 - Sidekiq Roadmap Statistics & History 21:44 - Sidekiq Enterprise 27:58 - Sidekiq vs Resque Scheduled-Jobs 29:50 - Adding Features to Sidekiq 30:28 - “Unique Job” 31:17 - Idempotency Sidekiq Best Practices Page 33:12 - Mixing In Other Data Stores Redis Kafka Apollo 38:42 - Encoding 40:04 - Format 40:36 - The Active Job Adapter 41:23 - Making Open Source Viable and Sustainable 44:04 - Launching An Open Source Project Kickstarter BSD & LGPL Licences Picks Mike Hoye: Citation Needed (David) Code Master (Coraline) Robot Turtles (Coraline) Zalando STUPS (Jessica) Elevator Saga (Chuck) Developer On Fire: Episode 017 - Charles Max Wood - Get Involved and Try New Things (Chuck) Model View Culture (Mike) Plasso (Mike) James Mickens: Not Even Close: The State of Computer Security (with slides) from NDC Conferences (Mike)

Check out Angular Remote Conf and RailsClips!   03:15 - Mike Perham Introduction Twitter GitHub Blog Contributed Systems sidekiq dalli 03:43 - Sidekiq Overview resque JRuby 05:18 - Job Runners vs Queuing Systems, Background Jobs RabbitMQ sneakers 08:47 - Performance celluloid 09:49 - celluloid vs Ruby Threads 11:47 - The GIL (Global Interpreter Lock) 12:49 - Passing Data 14:01 - Performance Boost From Using JRuby? 15:48 - The Actor Model revactor Rubinius girl_friday 20:39 - Sidekiq Roadmap Statistics & History 21:44 - Sidekiq Enterprise 27:58 - Sidekiq vs Resque Scheduled-Jobs 29:50 - Adding Features to Sidekiq 30:28 - “Unique Job” 31:17 - Idempotency Sidekiq Best Practices Page 33:12 - Mixing In Other Data Stores Redis Kafka Apollo 38:42 - Encoding 40:04 - Format 40:36 - The Active Job Adapter 41:23 - Making Open Source Viable and Sustainable 44:04 - Launching An Open Source Project Kickstarter BSD & LGPL Licences Picks Mike Hoye: Citation Needed (David) Code Master (Coraline) Robot Turtles (Coraline) Zalando STUPS (Jessica) Elevator Saga (Chuck) Developer On Fire: Episode 017 - Charles Max Wood - Get Involved and Try New Things (Chuck) Model View Culture (Mike) Plasso (Mike) James Mickens: Not Even Close: The State of Computer Security (with slides) from NDC Conferences (Mike)

Sean has a conversation with Terence Lee. Terence is known for his work at Heroku and his open source contributions to Ruby Core, Bundler and Resque.

Sean has a conversation with Terence Lee. Terence is known for his work at Heroku and his open source contributions to Ruby Core, Bundler and Resque.

Новости Костя про Ruby 2.1 и русский перевод Как Групон перешел на Node.js Что модно в Rails Rubinius 2.0 и 2.1 зарелизился Resque для Go Resque 1.25.0 зарелизился Изучение Ruby через айфон Ruby 2.1 preview1 Шедулер для Sidekiq Capistrano 3 http gem Свежий релиз Rails 4.0.1 Ruby Hacking Guide на английском Самовыпиливание wmeissner Обсуждение Подкаст rwpod Подкаст ДевОпс Дефлопе Рубиниус Икс Руби упал в индексе TIOBE Вагрант Пакер Виви Серф К.Г. Юнг “Воспоминания, сновидения, размышления” Фильм Ивана Вырыпаева “Танец Дели”

本期由 Daniel 主持，参与嘉宾有 SaitoWu, Dingding Ye。武鑫(Saito) 是著名自托管 Git 项目仓库开源项目 GitLab 的核心开发者之一，也是 RubyConfChina 2012 的讲师，给大家做了期很精彩的 GitLab 实现介绍。本期我们很荣幸能请到 SaitoWu 同学来跟大家聊聊他的从业经历，他所感兴趣的话题，包括 Git，GitHub，以及 GitLab。 Why Git is better than X Git 为什么这么好? Unlocking the Secrets of Git Git scaling at GitHub Chatops at Github Sinatra How gitlab works SVN VPN git-svn 蓝光党 Ruby Tuesday http://clojure.org/ Haskell Rich Hickey 七周七语言 JDK8 Github Enterprise authorized_keys Gitosis Gitolite Twisted GitHub is Moving to Rackspace! Engine Yard And GitHub Transition Zach Holman Github Boxen libgit2 Hubot Play - Company's DJ Redcarpet Unicorn html-pipeline Resque Gitcafe Redmine Use pull request Scrum要素 Component Hexo Special Guest: saitowu.

Sidekiq allows you to move jobs into the background for asynchronous processing. It uses threads instead of forks so it is much more efficient with memory compared to Resque.

Sidekiq allows you to move jobs into the background for asynchronous processing. It uses threads instead of forks so it is much more efficient with memory compared to Resque.

Etienne de Bruin demonstrates how to visualize Twitter retweets in real time using the tweetstream gem, arbor.js, Redis, and Resque. He also shares his personal journey as a developer and how he came to love Ruby.

Resque creates background jobs using Redis. It supports multiple queue and comes with an administration interface for monitoring and managing the queues.

Resque creates background jobs using Redis. It supports multiple queue and comes with an administration interface for monitoring and managing the queues.

Emanuele Tozzato gives a brief introduction to Redis, then demonstrates real-life database performance optimization with redis-object and how to manage background jobs using Resque.