Podcasts about MySQL

Share on
Share on Facebook
Share on Twitter
Share on Reddit
Copy link to clipboard

SQL database engine software

  • 409PODCASTS
  • 891EPISODES
  • 43mAVG DURATION
  • 5WEEKLY NEW EPISODES
  • Jan 19, 2022LATEST
MySQL

POPULARITY

20122013201420152016201720182019202020212022


Best podcasts about MySQL

Show all podcasts related to mysql

Latest podcast episodes about MySQL

Screaming in the Cloud
Find, Fix and Eliminate Cloud Vulnerabilities with Shir Tamari and Company

Screaming in the Cloud

Play Episode Listen Later Jan 19, 2022 33:53


About ShirShir Tamari is the Head of Research of Wiz, the cloud security company. He is an experienced security and technology researcher specializing in vulnerability research and practical hacking. In the past, he served as a consultant to a variety of security companies in the fields of research, development and product.About SagiSagi Tzadik is a security researcher in the Wiz Research Team. Sagi specializes in research and exploitation of web applications vulnerabilities, as well as network security and protocols. He is also a Game-Hacking and Reverse-Engineering enthusiast.About NirNir Ohfeld is a security researcher from Israel. Nir currently does cloud-related security research at Wiz. Nir specializes in the exploitation of web applications, application security and in finding vulnerabilities in complex high-level systems.Links: Wiz: https://www.wiz.io Cloud CVE Slack channel: https://cloud-cve-db.slack.com/join/shared_invite/zt-y38smqmo-V~d4hEr_stQErVCNx1OkMA Wiz Blog: https://wiz.io/blog Twitter: https://twitter.com/wiz_io TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: This episode is sponsored in part by our friends at Redis, the company behind the incredibly popular open source database that is not the bind DNS server. If you're tired of managing open source Redis on your own, or you're using one of the vanilla cloud caching services, these folks have you covered with the go to manage Redis service for global caching and primary database capabilities; Redis Enterprise. To learn more and deploy not only a cache but a single operational data platform for one Redis experience, visit redis.com/hero. Thats r-e-d-i-s.com/hero. And my thanks to my friends at Redis for sponsoring my ridiculous non-sense.  Corey: This episode is sponsored in part by our friends at Rising Cloud, which I hadn't heard of before, but they're doing something vaguely interesting here. They are using AI, which is usually where my eyes glaze over and I lose attention, but they're using it to help developers be more efficient by reducing repetitive tasks. So, the idea being that you can run stateless things without having to worry about scaling, placement, et cetera, and the rest. They claim significant cost savings, and they're able to wind up taking what you're running as it is in AWS with no changes, and run it inside of their data centers that span multiple regions. I'm somewhat skeptical, but their customers seem to really like them, so that's one of those areas where I really have a hard time being too snarky about it because when you solve a customer's problem and they get out there in public and say, “We're solving a problem,” it's very hard to snark about that. Multus Medical, Construx.ai and Stax have seen significant results by using them. And it's worth exploring. So, if you're looking for a smarter, faster, cheaper alternative to EC2, Lambda, or batch, consider checking them out. Visit risingcloud.com/benefits. That's risingcloud.com/benefits, and be sure to tell them that I said you because watching people wince when you mention my name is one of the guilty pleasures of listening to this podcast.Corey: Welcome to Screaming in the Cloud, I'm Corey Quinn. One of the joyful parts of working with cloud computing is that you get to put a whole lot of things you don't want to deal with onto the shoulders of the cloud provider you're doing business with—or cloud providers as the case may be, if you fallen down the multi-cloud well. One of those things is often significant aspects of security. And that's great, right, until it isn't. Today, I'm joined by not one guest, but rather three coming to us from Wiz, which I originally started off believing was, oh, it's a small cybersecurity research group. But they're far more than that. Thank you for joining me, and could you please introduce yourself?Shir: Yes, thank you, Corey. My name is Shir, Shir Tamari. I lead the security research team at Wiz. I working in the company for the past year. I'm working with these two nice teammates.Nir: Hi, my name is Nir Ohfield,. I'm a security researcher at the Wiz research team. I've also been working for the Wiz research team for the last year. And yeah.Sagi: I'm Sagi, Sagi Tzadik. I also work for the Wiz research team for the last six months.Corey: I want to thank you for joining me. You folks really burst onto the scene earlier this year, when I suddenly started seeing your name come up an awful lot. And it brought me back to my childhood where there was an electronics store called Nobody Beats the Wiz. It was more or less a version of Fry's on a different coast, and they went out of business and oh, good. We're going back in time. And suddenly it felt like I was going back in time in a different light because you had a number of high profile vulnerabilities that you had discovered, specifically in the realm of Microsoft Azure. The two that leap to mind the most readily for me are ChaosDB and the OMIGOD exploits. There was a third as well, but why don't you tell me, in your own words, what it is that you discovered and how that played out?Shir: We, sort of, found the vulnerabilities in Microsoft Azure. We did report multiple vulnerabilities also in GCP, and AWS. We had multiple vulnerabilities in AWS [unintelligible 00:02:42] cross-account. It was a cross-account access to other tenants; it just was much less severe than the ChaosDB vulnerability that we will speak on more later. And a both we've present in Blackhat in Vegas in [unintelligible 00:02:56]. So, we do a lot of research. You mentioned that we have a third one. Which one did you refer to?Corey: That's a good question because you had the I want to say it was called as Azurescape, and you're doing a fantastic job with branding a number of your different vulnerabilities, but there's also, once you started reporting this, a lot of other research started coming out as well from other folks. And I confess, a lot of it sort of flowed together and been very hard to disambiguate, is this a systemic problem; is this, effectively, a whole bunch of people piling on now that their attention is being drawn somewhere; or something else? Because you've come out with an awful lot of research in a short period of time.Shir: Yeah, we had a lot of good research in the past year. It's a [unintelligible 00:03:36] mention Azurecape was actually found by a very good researcher in Palo Also. And… do you remember his name?Sagi: No, I can't recall his name is.Corey: Yeah, they came out of unit 42 as I recall, their cybersecurity division. Every tech company out there seems to have some sort of security research division these days. What I think is, sort of, interesting is that to my understanding, you were founded, first and foremost, as a security company. You're not doing this as an ancillary to selling something else like a firewall, or, effectively, you're an ad comp—an ad tech company like Google, we you're launching Project Zero. You are first and foremost aimed at this type of problem.Shir: Yes. Wiz is not just a small research company. It's actually pretty big company with over 200 employees. And the purpose of this product is a cloud security suite that provides [unintelligible 00:04:26] scanning capabilities in order to find risks in cloud environments. And the research team is a very small group. We are [unintelligible 00:04:35] researchers.We have multiple responsibilities. Our first responsibility is to find risks in cloud environments: It could be misconfigurations, it could be vulnerabilities in libraries, in software, and we add those findings and the patterns we discover to the product in order to protect our customers, and to allow them for new risks. Our second responsibility is also to do a community research where we research everyone vulnerabilities in public products and cloud providers, and we share our findings with the cloud providers, then also with the community to make the cloud more secure.Corey: I can't shake the feeling that if there weren't folks doing this sort of research and shining a light on what it is that the cloud providers are doing, if they were to discover these things at all, they would very quietly, effectively, fix it in the background and never breathe a word of it in public. I like the approach that you're taking as far as dragging it, kicking and screaming, into the daylight, but I also have to imagine that probably doesn't win you a whole lot of friends at the company that you're focusing on at any given point in time. Because whenever you talk to a company about a security issue, it seems like the first thing they're concerned about is, “Okay, how do we wind up spinning this or making sure that we minimize the reputational damage?” And then there's a secondary reaction of, “Oh, and how do we protect our customers? But mostly, how do we avoid looking bad as a result?” And I feel like that's an artifact of corporate culture these days. But it feels like the relationship has got to be somewhat interesting to navigate from your perspective.Shir: So, once we found a vulnerability and we discuss it with the vendor, okay, first, I will mention that most cloud providers have a bug bounty program where they encourage researchers to find vulnerabilities and to discover new security threats. And all of them, as a public disclosure, [unintelligible 00:06:29] program will researchers are welcome and get safe harbor, you know, where the disclosure vulnerabilities. And I think it's, like, common interest, both for customers, but for researchers, and the cloud providers to know about those vulnerabilities, to mitigate it down. And we do believe that sometimes cloud providors does resolve and mitigate vulnerabilities behind the scenes, and we know—we don't know for sure, but—I don't know about everything, but just by the vulnerabilities that we find, we assume that there is much more of them that we never heard about. And this is something that we believe needs to be changed in the industry.Cloud providers should be more transparent, they should show more information about the result vulnerabilities. Definitely when a customer data was accessible, or where it was at risk, or at possible risk. And this is actually—it's something that we actually trying to change in the industry. We have a community and, like, innovative community. It's like an initiative that we try to collect, we opened a Slack channel called the Cloud CVE, and we try to invite as much people as we can that concern about cloud's vulnerabilities, in order to make a change in the industry, and to assist cloud providers, or to convince cloud providers to be more transparent, to enumerate cloud vulnerabilities so they have an identifier just, like cloud CVE, like a CVE, and to make the cloud more protected and more transparent customers.Corey: The thing that really took me aback by so much of what you found is that we've become relatively accustomed to a few patterns over the past 15 to 20 years. For example, we're used to, “Oh, this piece of software you run on your desktop has a horrible flaw. Great.” Or this thing you run in your data center, same story; patch, patch, patch, patch patch. That's great.But there was always the sense that these were the sorts of things that were sort of normal, but the cloud providers were on top of things, where they were effectively living up to their side of the shared responsibility bargain. And that whenever you wound up getting breached, for whatever reason—like in the AWS world, where oh, you wound up losing a bunch of customer data because you had an open S3 bucket? Well, yeah, that's not really something you can hang super effectively around the neck of the cloud provider, given that you're the one that misconfigured that. But what was so striking about what you found with both of the vulnerabilities that we're talking about today, the customer could have done everything absolutely correctly from the beginning and still had their data exposed. And that feels like it's something relatively new in the world of cloud service providers.Is this something that's been going on for a while and we're just now shining a light on it? Have I just missed a bunch of interesting news stories where the clouds have—“Oh, yeah, by the way, people, we periodically have to go in and drag people out of our cloud control plane because oops-a-doozy, someone got in there again with the squirrels,” or is this something that is new?Shir: So, we do see an history other cases where probability [unintelligible 00:09:31] has disclosed vulnerabilities in the cloud infrastructure itself. There was only few, and usually, it was—the research was conducted by independent researchers. And I don't think it had such an impact, like ChaosDB, which allowed [cross-system 00:09:51] access to databases of other customers, which was a huge case. And so if it wasn't a big story, so most people will not hear about it. And also, independent researchers usually don't have the back that we have here in Wiz.We have a funding, we have the marketing division that help us to get coverage with reporters, who make sure to make—if it's a big story, we make sure that other people will hear about it. And I believe that in most bug bounty programs where independent researchers find vulnerabilities, usually they more care about the bounty than the aftereffect of stopping the vulnerability, sharing it with the community. Usually also, independent [unintelligible 00:10:32] usually share the findings with the research community. And the research community is relatively small to the IT community. So, it is new, but it's not that new.There was some events back in history, [unintelligible 00:10:46] similar vulnerabilities. So, I think that one of the points here is that everyone makes a mistake. You can find bugs which affected mostly, as you mentioned previously, this software that you installed on your desktop has bugs and you need to patch it, but in the case of cloud providers, when they make mistakes, when they introduce bugs to the service, it affects all of their customers. And this is something that we should think about. So, mistakes that are being made by cloud providers have a lot of impact regarding their customers.Corey: Yeah. It's not a story of you misconfigured, your company's SAN, so you're the one that was responsible for a data breach. It's suddenly, you're misconfiguring everyone's SAN simultaneously. It's the sheer scale and scope of what it is that they've done. And—Shir: Yeah, exactly.Corey: —I'm definitely on board with that. But the stuff I've seen in the past, from cloud providers—AWS, primarily, since that is admittedly where I tend to focus most of my time and energy—has been privilege escalation style stuff, where, okay, if you assign some users at your company—or wherever—access to this managed IAM policy, well, they'll have suddenly have access to things that go beyond the scope of that. And that's not good, let's be very clear on that, but it is a bit different between that and oh, by the way, suddenly, someone in another company that has no relationship established with you at all can suddenly rummage through your data that you're storing in Cosmos DB, their managed database offering. That's the thing to me that I think was the big head-turning aspect of this, not just for me, but for a number of folks I've spoken to, in financial services, in government, in a bunch of environments where data privacy is not optional in the same way that it is when, you know, you're running a social media for pets app.Nir: [laugh]. Yeah, but the thing is, that until the publication of ChaosDB, no one ever heard about the [unintelligible 00:12:40] data tampering in any cloud providers. Meaning maybe in six months, you can see a similar vulnerabilities in other cloud providers that maybe other security research groups find. So yeah, so Azure was maybe the first, but we don't think they will be the last.Shir: Yes. And also, when we do the community research, it is very important to us to take big targets. We enjoy the research. One day, the research will be challenging and we want to do something that it was new and great, so we always put a very big targets. To actually find vulnerability in the infrastructure of the cloud provider, it was very challenging for us.When didn't came ChaosDB by that; we actually found it by mistake. But now we think actively that this is our next goal is to find vulnerabilities in the infrastructure and not just vulnerabilities that affect only the—vulnerabilities within the account itself, like [unintelligible 00:13:32] or bad scoped policies that affects only one account.Corey: That seems to be the transformative angle that you don't see nearly as much in existing studies around vulnerabilities in this space. It's always the, “Oh, no. We could have gotten breached by those people across the hallway from us in our company,” as opposed to folks on the other side of the planet. And that is, I guess, sort of the scary thing. What has also been interesting to me, and you obviously have more experience with this than I do, but I have a hard time envisioning that, for example, AWS, having a vulnerability like this and not immediately swinging into disaster firefighting mode, sending their security execs on a six month speaking tour to explain what happened, how it got there, all of the steps that they're taking to remediate this, but Azure published a blog post explaining this in relatively minor detail: Here are the mitigations you need to take, and as far as I can tell, then they sort of washed their hands of the whole thing and have enthusiastically begun saying absolutely nothing since.And that I have learned is sort of fairly typical for Microsoft, and has been for a while, where they just don't talk about these things when it arises. Does that match your experience? Is this something that you find that is common when a large company winds up being, effectively, embarrassed about their security architecture, or is this something that is unique to Microsoft tends to approach these things?Shir: I would say in general, we really like the Microsoft MSRC team. The group in Microsoft that's responsible for handling vulnerabilities, and I think it's like the security division inside Microsoft, MSRC. So, we have a really good relationship and we had really good time working with them. They're real professionals, they take our findings very seriously. I can tell that in the ChaosDB incident, they didn't plan to publish a blog post, and they did that after the story got a lot of attention.So, I'm looking at a PR team, and I have no idea out there decide stuff and what is their strategy, but as I mentioned earlier, we believe that there is much more cloud vulnerabilities that we never heard of, and it should change; they should publish more.Nir: It's also worth mentioning that Microsoft acted really quick on this vulnerability and took it very seriously. They issued the fix in less than 48 hours. They were very transparent in the entire procedure, and we had multiple teams meeting with them. The entire experience was pretty positive with each of the vulnerability we've ever reported to Microsoft.Sagi: So, it's really nice working with the guys that are responsible for security, but regarding PR, I agree that they should have posted more information regarding this incident.Corey: The thing that I found interesting about this, and I've seen aspects of it before, but never this strongly is, I was watching for, I guess, what I would call just general shittiness, for lack of a better term, from the other providers doing a happy dance of, “Aha, we're better than you are,” and I saw none of that. Because when I started talking to people in some depth at this at other companies, the immediate response—not just AWS, to be clear—has been no, no, you have to understand, this is not good for anyone because this effectively winds up giving fuel to the slow-burning fire of folks who are pulling the, “See, I told you the cloud wasn't secure.” And now the enterprise groundhog sees that shadow and we get six more years of building data centers instead of going to the cloud. So, there's no one in the cloud space who's happy with this kind of revelation and this type of vulnerability. My question for you is given that you are security researchers, which means you are generally cynical and pessimistic about almost everything technological, if you're like most of the folks in that space that I've spent time with, is going with cloud the wrong answer? Should people be building their own data centers out? Should they continue to be going on this full cloud direction? I mean, what can they do if everything's on fire and terrible all the time?Shir: So, I think that there is a trade-off when you embrace the cloud. On one hand, you get the fastest deployment times, and a good scalability regarding your infrastructure, but on the other end, when there is a security vulnerability in the cloud provider, you are immediately affected. But it is worth mentioning that the security teams or the cloud providers are doing extremely good job. Most likely, they are going to patch the vulnerability faster than it would have been patched in on-premise environment. And it's good that you have them working for you.And once the vulnerability is mitigated—depends on the vulnerability but in the case of ChaosDB—when the vulnerability was mitigated on Microsoft's end, and it was mitigated completely. No one else could have exploited after the mitigated it once. Yes, it's also good to mention that the cloud provides organization and companies a lot of security features, [unintelligible 00:18:34] I want to say security features, I would say, it provides a lot of tooling that helps security. The option to have one interface, like one API to control all of my devices, to get visibility to all of my servers, to enforce policies very easily, it's much more secure than on-premise environments, where there is usually a big mess, a lot of vendors.Because the power was in the on-prem, the power was on the user, so the user had a lot of options. Usually used many types of software, many types of hardware, it's really hard to mitigate the software vulnerability in on-prem environments. It's really helped to get the visibility. And the cloud provides a lot of security, like, a good aspects, and in my opinion, moving to the cloud for most organization would be a more secure choice than remain on-premise, unless you have a very, very small on-prem environment.Corey: This episode is sponsored by our friends at Oracle HeatWave is a new high-performance accelerator for the Oracle MySQL Database Service. Although I insist on calling it “my squirrel.” While MySQL has long been the worlds most popular open source database, shifting from transacting to analytics required way too much overhead and, ya know, work. With HeatWave you can run your OLTP and OLAP, don't ask me to ever say those acronyms again, workloads directly from your MySQL database and eliminate the time consuming data movement and integration work, while also performing 1100X faster than Amazon Aurora, and 2.5X faster than Amazon Redshift, at a third of the cost. My thanks again to Oracle Cloud for sponsoring this ridiculous nonsense.Corey: The challenge I keep running into is that—and this is sort of probably the worst of all possible reasons to go with cloud, but let's face it, when us-east-1 recently took an outage and basically broke a decent swath of the internet, a lot of companies were impacted, but they didn't see their names in the headlines; it was all about Amazon's outage. There's a certain value when a cloud provider takes an outage or a security breach, that the headlines screaming about it are about the provider, not about you and your company as a customer of that provider. Is that something that you're seeing manifest across the industry? Is that an unhealthy way to think about it? Because it feels almost like it's cheating in a way. It's, “Yeah, we had a security problem, but so did the entire internet, so it's okay.”Nir: So, I think that if there would be evidence that these kind of vulnerabilities were exploited while disclosure, then you wouldn't see headlines of companies, shouting in the headlines. But in the case of the us reporting the vulnerabilities prior to anyone exploiting them, results in nowhere a company showing up in the headlines. I think it's a slightly different situation than an outage.Shir: Yeah, but also, when one big provider have an outage or a breach, so usually, the customers will think it's out of my responsibility. I mean, it's bad; my data has been leaked, but what can I do? I think it's very easy for most people to forgive companies [unintelligible 00:21:11]. I mean, you know what, it's just not my area. So, maybe I'm not answer that into that. [laugh].Corey: No, no, it's very fair. The challenge I have, as a customer of all of these providers, to be honest, is that a lot of the ways that the breach investigations are worded of, “We have seen no evidence that this has been exploited.” Okay, that simultaneously covers the two very different use cases of, “We have pored through our exhaustive audit logs and validated that no one has done this particular thing in this particular way,” but it also covers the use case, “Of, hey, we learned we should probably be logging things, but we have no evidence that anything was exploited.” Having worked with these providers at scale, my gut impression is that they do in fact, have fairly detailed logs of who's doing what and where. Would you agree with that assessment, or do you find that you tend to encounter logging and analysis gaps as you find these exploits?Shir: We don't really know. Usually when—I mean, ChaosDB scenario, we got access to a Jupyter Notebook. And from the Jupyter Notebook, we continued to another internal services. And we—nobody stopped us. Nobody—we expected an email, like—Corey: “Whatcha doing over there, buddy?”Shir: Yeah. “Please stop doing that, and we're investigating you.” And we didn't get any. And also, we don't really know if they monitor it or not. I can tell from my technical background that logging so many environments, it's hard.And when you do decide to log all these events, you need to decide what to log. For example, if I have a database, a managed database, do I log all the queries that customers run? It's too much. If I have an HTTP application—a managed HTTP application—do I save all the access logs, like all the requests? And if so, what will be the retention time? For how long?We believe that it's very challenging on the cloud provider side, but it just an assumption. And doing the discussion with Microsoft, the didn't disclose any, like, scenarios they had with logging. They do mention that they're [unintelligible 00:23:26] viewing the logs and searching to see if someone exploited this vulnerability before we disclosed it. Maybe someone discovered before we did. But they told us they didn't find anything.Corey: One last area I'd love to discuss with you before we call it an episode is that it's easy to view Wiz through the lens of, “Oh, we just go out and find vulnerabilities here and there, and we make companies feel embarrassed—rightfully so—for the things that they do.” But a little digging shows that you've been around for a little over a year as a publicly known entity, and during that time, you've raised $600 million in funding, which is basically like what in the world is your pitch deck where you show up to investors and your slides are just, like, copies of their emails, and you read them to them?[laugh]I mean, on some level, it seems like that is a… as-, astounding amount of money to raise in a short period of time. But I've also done a little bit of digging, and to be clear, I do not believe that you have an extortion-based business model, which is a good thing. You're building something very interesting that does in-depth analysis of cloud workloads, and I think it's got an awful lot of promise. How does the vulnerability research that you do tie into that larger platform, other than, let's be honest, some spectacularly effective marketing.Sagi: Specifically in the ChaosDB vulnerability, we were actually not looking for a vulnerability in the cloud service providers. We were originally looking for common misconfigurations that our customers can make when they set up their Cosmos DB accounts, so that our product will be able to alert our customers regarding such misconfigurations. And then we went to the Azure portal and started to enable all of the features that Cosmos DB has to offer, and when we enabled enough features, we noticed some feature that could be vulnerable, and we started digging into it. And we ended up finding ChaosDB.But our original work was to try and find misconfigurations that our customers can make in order to protect them and not to find a vulnerability in the [CSP 00:25:31]. This was just, like, a byproduct of this research.Shir: Yes. There is, as I mentioned earlier, our main responsibility is to add a little security rist content to the product, to help customers to find new security risks in their environment. As you mentioned, like, the escalation possibilities within cloud accounts, and bad scoped policies, and many other security risks that are in the cloud area. And also, we are a very small team inside a big company, so most of the company, they are doing heavy [unintelligible 00:26:06] and talk with customers, they understand the risks, they understand the market, what the needs for tomorrow, and maybe we are well known for our vulnerabilities, but it just a very small part of the company.Corey: On some level, it says wonderful things about your product, and also terrifying things from different perspectives of, “Oh, yeah, we found one of the worst cloud breaches in years by accident,” as opposed to actively going in trying to find the thing that has basically put you on the global map of awareness around these things. Because there a lot of security companies out there doing different things. In fact, go to RSA, and you'll see basically 12 companies that just repeated over and over and over with different names and different brandings, and they're all selling some kind of firewall. This is something actively different because everyone can tell beautiful pictures with slides and whatnot, and the corporate buzzwords. You're one of those companies that actually did something meaningful, and it felt almost like a proof of concept. On some level, the fact that you weren't actively looking for it is kind of an amazing testament for the product itself.Shir: Yeah. We actually used the product in the beginning, in order to overview our own environment, and what is the most common services we use. In order—and we usually we mix this information with our product managers, know to understand what customers use and what products and services we need to research in order to bring value to the product.Sagi: Yeah, so the reason we chose to research Cosmos DB was that, we found that a lot of our Azure customers are using Cosmos DB on their production environments, and we wanted to add mitigations for common misconfigurations to our product in order to protect our customers.Nir: Yeah, the same goes with our other research, like OMIGOD, where we've seen that there is a excessive amount of [unintelligible 00:27:56] installations in an Azure environment, and it raised our [laugh] it raised our attention, and then found this vulnerability. It's mostly, like, popularity-guided research. [laugh].Shir: Yeah. And also [unintelligible 00:28:11] mention that maybe we find vulnerabilities by accident, but the service, we are doing vulnerability itself for the past ten years, and even more. So, we are very professional and this is what we do, and this is what we like to do. And we came skilled to the [crosstalk 00:28:25].Corey: It really is neat to see, just because every other security tool that I've looked at in recent memory tells you the same stuff. It's the same problem you see in the AWS billing space that I live in. Everyone says, “Oh, we can find these inactive instances that could be right-sized.” Great, because everyone's dealing with the same data. It's the security stuff is no different. “Hey, this S3 bucket is open.” Yes, it's a public web server. Please stop waking me up at two in the morning about it. It's there by design.But it goes back and forth with the same stuff just presented differently. This is one of the first truly novel things I've seen in ages. If nothing else, you convince me to kick the tires on it, and see what kind of horrifying things I can learn about my own environments with it.Shir: Yeah, you should. [laugh]. Let's poke [unintelligible 00:29:13].[laugh].Corey: I want to thank you so much for taking the time to speak with me today. If people want to learn more about the research you're up to and the things that you find interesting, where can they find you all?Shir: Most of our publication—I mean, all of our publications are under the Wiz, which is wiz.io/blog, and people can read all of our research. Just today we are announcing a new one, so feel free to go and read there. And they also feel free to approach us on Twitter, the service, we have a Twitter account. We are open for, like, messages. Just send us a message.Corey: And we will certainly put links to all of that in the [show notes 00:29:49]. Shir, Sagi, Nir, thank you so much for joining me today. I really appreciate your time.Shir: Thank you.Sagi: Thank you.Nir: Thank you much.Shir: It was very fun. Yeah.Corey: This has been Screaming in the Cloud. I'm Cloud Economist Corey Quinn and thank you for listening. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice along with an angry insulting comment from someone else's account.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.

Screaming in the Cloud
The re:Invent Wheel in the Sky Keeps on Turning with Pete Cheslock

Screaming in the Cloud

Play Episode Listen Later Jan 18, 2022 54:52


About PetePete does many startup things at Allma. Links: Last Tweet in AWS: https://lasttweetinaws.com Twitter: https://twitter.com/petecheslock LinkedIn: https://www.linkedin.com/in/petecheslock/ TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: This episode is sponsored in part byLaunchDarkly. Take a look at what it takes to get your code into production. I'm going to just guess that it's awful because it's always awful. No one loves their deployment process. What if launching new features didn't require you to do a full-on code and possibly infrastructure deploy? What if you could test on a small subset of users and then roll it back immediately if results aren't what you expect? LaunchDarkly does exactly this. To learn more, visitlaunchdarkly.com and tell them Corey sent you, and watch for the wince.Corey: This episode is sponsored in part by our friends at Redis, the company behind the incredibly popular open source database that is not the bind DNS server. If you're tired of managing open source Redis on your own, or you're using one of the vanilla cloud caching services, these folks have you covered with the go to manage Redis service for global caching and primary database capabilities; Redis Enterprise. To learn more and deploy not only a cache but a single operational data platform for one Redis experience, visit redis.com/hero. Thats r-e-d-i-s.com/hero. And my thanks to my friends at Redis for sponsoring my ridiculous non-sense.  Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. I am joined—as is tradition, for a post re:Invent wrap up, a month or so later, once everything is time to settle—by my friend and yours, Pete Cheslock. Pete, how are you?Pete: Hi, I'm doing fantastic. New year; new me. That's what I'm going with.Corey: That's the problem. I keep hoping for that, but every time I turn around, it's still me. And you know, honestly, I wouldn't wish that on anyone.Pete: Exactly. [laugh]. I wouldn't wish you on me either. But somehow I keep coming back for this.Corey: So, in two-thousand twenty—or twenty-twenty, as the children say—re:Invent was fully virtual. And that felt weird. Then re:Invent 2021 was a hybrid event which, let's be serious here, is not really those things. They had a crappy online thing and then a differently crappy thing in person. But it didn't feel real to me because you weren't there.That is part of the re:Invent tradition. There's a midnight madness thing, there's a keynote where they announce a bunch of nonsense, and then Pete and I go and have brunch on the last day of re:Invent and decompress, and more or less talk smack about everything that crosses our minds. And you weren't there this year. I had to backfill you with Tim Banks. You know, the person that I backfield you with here at The Duckbill Group as a principal cloud economist.Pete: You know, you got a great upgrade in hot takes, I feel like, with Tim.Corey: And other ways, too, but it's rude of me to say that to you directly. So yeah, his hot takes are spectacular. He was going to be doing this with me, except you cannot mess with tradition. You really can't.Pete: Yeah. I'm trying to think how many—is this third year? It's at least three.Corey: Third or fourth.Pete: Yeah, it's at least three. Yeah, it was, I don't want to say I was sad to not be there because, with everything going on, it's still weird out there. But I am always—I'm just that weird person who actually likes re:Invent, but not for I feel like the reasons people think. Again, I'm such an extroverted-type person, that it's so great to have this, like, serendipity to re:Invent. The people that you run into and the conversations that you have, and prior—like in 2019, I think was a great example because that was the last one I had gone to—you know, having so many conversations so quickly because everyone is there, right? It's like this magnet that attracts technologists, and venture capital, and product builders, and all this other stuff. And it's all compressed into, like, you know, that five-day span, I think is the biggest part that makes so great.Corey: The fear in people's eyes when they see me. And it was fun; I had a pair of masks with me. One of them was a standard mask, and no one recognizes anyone because, masks, and the other was a printout of my ridiculous face, which was horrifyingly uncanny, but also made it very easy for people to identify me. And depending upon how social I was feeling, I would wear one or the other, and it worked flawlessly. That was worth doing. They really managed to thread the needle, as well, before Omicron hit, but after the horrors of last year. So, [unintelligible 00:03:00]—Pete: It really—Corey: —if it were going on right now, it would not be going on right now.Pete: Yeah. I talk about really—yeah—really just hitting it timing-wise. Like, not that they could have planned for any of this, but like, as things were kind of not too crazy and before they got all crazy again, it feels like wow, like, you know, they really couldn't have done the event at any other time. And it's like, purely due to luck. I mean, absolute one hundred percent.Corey: That's the amazing power of frugality. Because the reason is then is it's the week after Thanksgiving every year when everything is dirt cheap. And, you know, if there's one thing that I one-point-seve—sorry, their stock's in the toilet—a $1.6 trillion company is very concerned about, it is saving money at every opportunity.Pete: Well, the one thing that was most curious about—so I was at the first re:Invent in-what—2012 I think it was, and there was—it was quaint, right?—there was 4000 people there, I want to say. It was in the thousands of people. Now granted, still a big conference, but it was in the Sands Convention Center. It was in that giant room, the same number of people, were you know, people's booths were like tables, like, eight-by-ten tables, right? [laugh].It had almost a DevOpsDays feel to it. And I was kind of curious if this one had any of those feelings. Like, did it evoke it being more quaint and personable, or was it just as soulless as it probably has been in recent years?Corey: This was fairly soulless because they reduced the footprint of the event. They dropped from two expo halls down to one, they cut the number of venues, but they still had what felt like 20,000 people or something there. It was still crowded, it was still packed. And I've done some diligent follow-ups afterwards, and there have been very few cases of Covid that came out of it. I quarantined for a week in a hotel, so I don't come back and kill my young kids for the wrong reasons.And that went—that was sort of like the worst part of it on some level, where it's like great. Now I could sit alone at a hotel and do some catch-up and all the rest, but all right I'd kind of like to go home. I'm not used to being on the road that much.Pete: Yeah, I think we're all a little bit out of practice. You know, I haven't been on a plane in years. I mean, the travel I've done more recently has been in my car from point A to point B. Like, direct, you know, thing. Actually, a good friend of mine who's not in technology at all had to travel for business, and, you know, he also has young kids who are under five, so he when he got back, he actually hid in a room in their house and quarantine himself in the room. But they—I thought, this is kind of funny—they never told the kids he was home. Because they knew that like—Corey: So, they just thought the house was haunted?Pete: [laugh].Corey: Like, “Don't go in the west wing,” sort of level of nonsense. That is kind of amazing.Pete: Honestly, like, we were hanging out with the family because they're our neighbors. And it was like, “Oh, yeah, like, he's in the guest room right now.” Kids have no idea. [laugh]. I'm like, “Oh, my God.” I'm like, I can't even imagine. Yeah.Corey: So, let's talk a little bit about the releases of re:Invent. And I'm going to lead up with something that may seem uncharitable, but I don't think it necessarily is. There weren't the usual torrent of new releases for ridiculous nonsense in the same way that there have been previously. There was no, this service talks to satellites in space. I mean, sure, there was some IoT stuff to manage fleets of cars, and giant piles of robots, and cool, I don't have those particular problems; I'm trying to run a website over here.So okay, great. There were enhancements to a number of different services that were in many cases appreciated, in other cases, irrelevant. Werner said in his keynote, that it was about focusing on primitives this year. And, “Why do we have so many services? It's because you asked for it… as customers.”Pete: [laugh]. Yeah, you asked for it.Corey: What have you been asking for, Pete? Because I know what I've been asking for and it wasn't that. [laugh].Pete: It's amazing to see a company continually say yes to everything, and somehow, despite their best efforts, be successful at doing it. No other company could do that. Imagine any other software technology business out there that just builds everything the customers ask for. Like from a product management business standpoint, that is, like, rule 101 is, “Listen to your customers, but don't say yes to everything.” Like, you can't do everything.Corey: Most companies can't navigate the transition between offering the same software in the Cloud and on a customer facility. So, it's like, “Ooh, an on-prem version, I don't know, that almost broke the company the last time we tried it.” Whereas you have Amazon whose product strategy is, “Yes,” being able to put together a whole bunch of things. I also will challenge the assertion that it's the primitives that customers want. They don't want to build a data center out of popsicle sticks themselves. They want to get something that solves a problem.And this has been a long-term realization for me. I used to work at Media Temple as a senior systems engineer running WordPress at extremely large scale. My websites now run on WordPress, and I have the good sense to pay WP Engine to handle it for me, instead of doing it myself because it's not the most productive use of my time. I want things higher up the stack. I assure you I pay more to WP Engine than it would cost me to run these things myself from an infrastructure point of view, but not in terms of my time.What I see sometimes as the worst of all worlds is that AWS is trying to charge for that value-added pricing without adding the value that goes along with it because you still got to build a lot of this stuff yourself. It's still a very janky experience, you're reduced to googling random blog posts to figure out how this thing is supposed to work, and the best documentation comes from externally. Whereas with a company that's built around offering solutions like this, great. In the fullness of time, I really suspect that if this doesn't change, their customers are going to just be those people who build solutions out of these things. And let those companies capture the up-the-stack margin. Which I have no problem with. But they do because Amazon is a company that lies awake at night actively worrying that someone, somewhere, who isn't them might possibly be making money somehow.Pete: I think MongoDB is a perfect example of—like, look at their stock price over the last whatever, years. Like, they, I feel like everyone called for the death of MongoDB every time Amazon came out with their new things, yet, they're still a multi-billion dollar company because I can just—give me an API endpoint and you scale the database. There's is—Corey: Look at all the high-profile hires that Mongo was making out of AWS, and I can't shake the feeling they're sitting there going, “Yeah, who's losing important things out of production now?” It's, everyone is exodus-ing there. I did one of those ridiculous graphics of the naming all the people that went over there, and in—with the hurricane evacuation traffic picture, and there's one car going the other way that I just labeled with, “Re:Invent sponsorship check,” because yeah, they have a top tier sponsorship and it was great. I've got to say I've been pretty down on MongoDB for a while, for a variety of excellent reasons based upon, more or less, how they treated customers who were in pain. And I'd mostly written it off.I don't do that anymore. Not because I inherently believe the technology has changed, though I'm told it has, but by the number of people who I deeply respect who are going over there and telling me, no, no, this is good. Congratulations. I have often said you cannot buy authenticity, and I don't think that they are, but the people who are working there, I do not believe that these people are, “Yeah, well, you bought my opinion. You can buy their attention, not their opinion.” If someone changes their opinion, based upon where they work, I kind of question everything they're telling me is, like, “Oh, you're just here to sell something you don't believe in? Welcome aboard.”Pete: Right. Yeah, there's an interview question I like to ask, which is, “What's something that you used to believe in very strongly that you've more recently changed your mind on?” And out of politeness because usually throws people back a little bit, and they're like, “Oh, wow. Like, let me think about that.” And I'm like, “Okay, while you think about that I want to give you mine.”Which is in the past, my strongly held belief was we had to run everything ourselves. “You own your availability,” was the line. “No, I'm not buying Datadog. I can build my own metric stack just fine, thank you very much.” Like, “No, I'm not going to use these outsourced load balancers or databases because I need to own my availability.”And what I realized is that all of those decisions lead to actually delivering and focusing on things that were not the core product. And so now, like, I've really flipped 180, that, if any—anything that you're building that does not directly relate to the core product, i.e. How your business makes money, should one hundred percent be outsourced to an expert that is better than you. Mongo knows how to run Mongo better than you.Corey: “What does your company do?” “Oh, we handle expense reports.” “Oh, what are you working on this month?” “I'm building a load balancer.” It's like that doesn't add the value. Don't do that.Pete: Right. Exactly. And so it's so interesting, I think, to hear Werner say that, you know, we're just building primitives, and you asked for this. And I think that concept maybe would work years ago, when you had a lot of builders who needed tools, but I don't think we have any, like, we don't have as many builders as before. Like, I think we have people who need more complete solutions. And that's probably why all these businesses are being super successful against Amazon.Corey: I'm wondering if it comes down to a cloud economic story, specifically that my cloud bill is always going to be variable and it's difficult to predict, whereas if I just use EC2 instances, and I build load balancers or whatnot, myself, well, yeah, it's a lot more work, but I can predict accurately what my staff compensation costs are more effectively, that I can predict what a CapEx charge would be or what the AWS bill is going to be. I'm wondering if that might in some way shape it?Pete: Well, I feel like the how people get better in managing their costs, right, you'll eventually move to a world where, like, “Yep, okay, first, we turned off waste,” right? Like, step one is waste. Step two is, like, understanding your spend better to optimize but, like, step three, like, the galaxy brain meme of Amazon cost stuff is all, like, unit economics stuff, where trying to better understand the actual cost deliver an actual feature. And yeah, I think that actually gets really hard when you give—kind of spread your product across, like, a slew of services that have varying levels of costs, varying levels of tagging, so you can attribute it. Like, it's really hard. Honestly, it's pretty easy if I have 1000 EC2 servers with very specific tags, I can very easily figure out what it costs to deliver product. But if I have—Corey: Yeah, if I have Corey build it, I know what Corey is going to cost, and I know how many servers he's going to use. Great, if I have Pete it, Pete's good at things, it'll cut that server bill in half because he actually knows how to wind up being efficient with things. Okay, great. You can start calculating things out that way. I don't think that's an intentional choice that companies are making, but I feel like that might be a natural outgrowth of it.Pete: Yeah. And there's still I think a lot of the, like, old school mentality of, like, the, “Not invented here,” the, “We have to own our availability.” You can still own your availability by using these other vendors. And honestly, it's really heartening to see so many companies realize that and realize that I don't need to get everything from Amazon. And honestly, like, in some things, like I look at a cloud Amazon bill, and I think to myself, it would be easier if you just did everything from Amazon versus having these ten other vendors, but those ten other vendors are going to be a lot better at running the product that they build, right, that as a service, then you probably will be running it yourself. Or even Amazon's, like, you know, interpretation of that product.Corey: A few other things that came out that I thought were interesting, at least the direction they're going in. The changes to S3 intelligent tiering are great, with instant retrieval on Glacier. I feel like that honestly was—they talk a good story, but I feel like that was competitive response to Google offering the same thing. That smacks of a large company with its use case saying, “You got two choices here.” And they're like, “Well, okay. Crap. We're going to build it then.”Or alternately, they're looking at the changes that they're making to intelligent tiering, they're now shifting that to being the default that as far as recommendations go. There are a couple of drawbacks to it, but not many, and it's getting easier now to not have the mental overhead of trying to figure out exactly what your lifecycle policies are. Yeah, there are some corner cases where, okay, if I adjust this just so, then I could save 10% on that monitoring fee or whatnot. Yeah, but look how much work that's going to take you to curate and make sure that you're not doing something silly. That feels like it is such an in the margins issue. It's like, “How much data you're storing?” “Four exabytes.” Okay, yeah. You probably want some people doing exactly that, but that's not most of us.Pete: Right. Well, there's absolutely savings to be had. Like, if I had an exabyte of data on S3—which there are a lot of people who have that level of data—then it would make sense for me to have an engineering team whose sole purpose is purely an optimizing our data lifecycle for that data. Until a point, right? Until you've optimized the 80%, basically. You optimize the first 80, that's probably, air-quote, “Easy.” The last 20 is going to be incredibly hard, maybe you never even do that.But at lower levels of scale, I don't think the economics actually work out to have a team managing your data lifecycle of S3. But the fact that now AWS can largely do it for you in the background—now, there's so many things you have to think about and, like, you know, understand even what your data is there because, like, not all data is the same. And since S3 is basically like a big giant database you can query, you got to really think about some of that stuff. But honestly, what I—I don't know if—I have no idea if this is even be worked on, but what I would love to see—you know, hashtag #AWSwishlist—is, now we have countless tiers of EBS volumes, EBS volumes that can be dynamically modified without touching, you know, the physical host. Meaning with an API call, you can change from the gp2 to gp3, or io whatever, right?Corey: Or back again if it doesn't pan out.Pete: Or back again, right? And so for companies with large amounts of spend, you know, economics makes sense that you should have a team that is analyzing your volumes usage and modifying that daily, right? Like, you could modify that daily, and I don't know if there's anyone out there that's actually doing it at that level. And they probably should. Like, if you got millions of dollars in EBS, like, there's legit savings that you're probably leaving on the table without doing that. But that's what I'm waiting for Amazon to do for me, right? I want intelligent tiering for EBS because if you're telling me I can API call and you'll move my data and make that better, make that [crosstalk 00:17:46] better [crosstalk 00:17:47]—Corey: Yeah it could be like their auto-scaling for DynamoDB, for example. Gives you the capacity you need 20 minutes after you needed it. But fine, whatever because if I can schedule stuff like that, great, I know what time of day, the runs are going to kick off that beat up the disks. I know when end-of-month reporting fires off. I know what my usage pattern is going to be, by and large.Yeah, part of the problem too, is that I look at this stuff, and I get excited about it with the intelligent tiering… at The Duckbill Group we've got a few hundred S3 buckets lurking around. I'm thinking, “All right, I've got to go through and do some changes on this and implement all of that.” Our S3 bill's something like 50 bucks a month or something ridiculous like that. It's a no, that really isn't a thing. Like, I have a screenshot bucket that I have an app installed—I think called Dropshare—that hooks up to anytime I drag—I hit a shortcut, I drag with the mouse to select whatever I want and boom, it's up there and the URL is not copied to my clipboard, I can paste that wherever I want.And I'm thinking like, yeah, there's no cleanup on that. There's no lifecycle policy that's turning into anything. I should really go back and age some of it out and do the rest and start doing some lifecycle management. It—I've been using this thing for years and I think it's now a whopping, what, 20 cents a month for that bucket. It's—I just don't—Pete: [laugh].Corey: —I just don't care, other than voice in the back of my mind, “That's an unbounded growth problem.” Cool. When it hits 20 bucks a month, then I'll consider it. But until then I just don't. It does not matter.Pete: Yeah, I think yeah, scale changes everything. Start adding some zeros and percentages turned into meaningful numbers. And honestly, back on the EBS thing, the one thing that really changed my perspective of EBS, in general, is—especially coming from the early days, right? One terabyte volume, it was a hard drive in a thing. It was a virtual LUN on a SAN somewhere, probably.Nowadays, and even, like, many years after those original EBS volumes, like all the limits you get in EBS, those are actually artificial limits, right? If you're like, “My EBS volume is too slow,” it's not because, like, the hard drive it's on is too slow. That's an artificial limit that is likely put in place due to your volume choice. And so, like, once you realize that in your head, then your concept of how you store data on EBS should change dramatically.Corey: Oh, AWS had a blog post recently talking about, like, with io2 and the limits and everything, and there was architecture thinking, okay. “So, let's say this is insufficient and the quarter-million IOPS a second that you're able to get is not there.” And I'm sitting there thinking, “That is just ludicrous data volume and data interactivity model.” And it's one of those, like, I'm sitting here trying to think about, like, I haven't had to deal with a problem like that decade, just because it's, “Huh. Turns out getting these one thing that's super fast is kind of expensive.” If you paralyze it out, that's usually the right answer, and that's how the internet is mostly evolved. But there are use cases for which that doesn't work, and I'm excited to see it. I don't want to pay for it in my view, but it's nice to see it.Pete: Yeah, it's kind of fun to go into the Amazon calculator and price out one of the, like, io2 volumes and, like, maxed out. It's like, I don't know, like $50,000 a month or a hun—like, it's some just absolutely absurd number. But the beauty of it is that if you needed that value for an hour to run some intensive data processing task, you can have it for an hour and then just kill it when you're done, right? Like, that is what is most impressive.Corey: I copied 130 gigs of data to an EFS volume, which was—[unintelligible 00:21:05] EFS has gone from “This is a piece of junk,” to one of my favorite services. It really is, just because of its utility and different ways of doing things. I didn't have the foresight, just use a second EFS volume for this. So, I was unzipping a whole bunch of small files onto it. Great.It took a long time for me to go through it. All right, now that I'm done with that I want to clean all this up. My answer was to ultimately spin up a compute node and wind up running a whole bunch of—like, 400, simultaneous rm-rf on that long thing. And it was just, like, this feels foolish and dumb, but here we are. And I'm looking at the stats on it because the instance was—all right, at that point, the load average [on the instance 00:21:41] was like 200, or something like that, and the EFS volume was like, “Ohh, wow, you're really churning on this. I'm now at, like, 5% of the limit.” Like, okay, great. It turns out I'm really bad at computers.Pete: Yeah, well, that's really the trick is, like, yeah, sure, you can have a quarter-million IOPS per second, but, like, what's going to break before you even hit that limit? Probably many other things.Corey: Oh, yeah. Like, feels like on some level if something gets to that point, it a misconfiguration somewhere. But honestly, that's the thing I find weirdest about the world in which we live is that at a small-scale—if I have a bill in my $5 a month shitposting account, great. If I screw something up and cost myself a couple hundred bucks in misconfiguration it's going to stand out. At large scale, it doesn't matter if—you're spending $50 million a year or $500 million a year on AWS and someone leaks your creds, and someone spins up a whole bunch of Bitcoin miners somewhere else, you're going to see that on your bill until they're mining basically all the Bitcoin. It just gets lost in the background.Pete: I'm waiting for those—I'm actually waiting for the next level of them to get smarter because maybe you have, like, an aggressive tagging system and you're monitoring for untagged instances, but the move here would be, first get the creds and query for, like, the most used tags and start applying those tags to your Bitcoin mining instances. My God, it'll take—Corey: Just clone a bunch of tags. Congratulations, you now have a second BI Elasticsearch cluster that you're running yourself. Good work.Pete: Yeah. Yeah, that people won't find that until someone comes along after the fact that. Like, “Why do we have two have these things?” And you're like—[laugh].Corey: “Must be a DR thing.”Pete: It's maxed-out CPU. Yeah, exactly.Corey: [laugh].Pete: Oh, the terrible ideas—please, please, hackers don't take are terrible ideas.Corey: I had a, kind of, whole thing I did on Twitter years ago, talking about how I would wind up using the AWS Marketplace for an embezzlement scheme. Namely, I would just wind up spinning up something that had, like, a five-cent an hour charge or whatnot on just, like, basically rebadge the CentOS Community AMI or whatnot. Great. And then write a blog post, not attached to me, that explains how to do a thing that I'm going to be doing in production in a week or two anyway. Like, “How to build an auto-scaling group,” and reference that AMI.Then if it ever comes out, like, “Wow, why are we having all these marketplace charges on this?” “I just followed the blog post like it said here.” And it's like, “Oh, okay. You're a dumbass. The end.”That's the way to do it. A month goes by and suddenly it came out that someone had done something similarly. They wound up rebadging these community things on the marketplace and charging big money for it, and I'm sitting there going like that was a joke. It wasn't a how-to. But yeah, every time I make these jokes, I worry someone's going to do it.Pete: “Welcome to large-scale fraud with Corey Quinn.”Corey: Oh, yeah, it's fraud at scale is really the important thing here.Corey: This episode is sponsored by our friends at Oracle HeatWave is a new high-performance accelerator for the Oracle MySQL Database Service. Although I insist on calling it “my squirrel.” While MySQL has long been the worlds most popular open source database, shifting from transacting to analytics required way too much overhead and, ya know, work. With HeatWave you can run your OLTP and OLAP, don't ask me to ever say those acronyms again, workloads directly from your MySQL database and eliminate the time consuming data movement and integration work, while also performing 1100X faster than Amazon Aurora, and 2.5X faster than Amazon Redshift, at a third of the cost. My thanks again to Oracle Cloud for sponsoring this ridiculous nonsense.Corey: I still remember a year ago now at re:Invent 2021 was it, or was it 2020? Whatever they came out with, I want to say it wasn't gp3, or maybe it was, regardless, there was a new EBS volume type that came out that you were playing with to see how it worked and you experimented with it—Pete: Oh, yes.Corey: —and the next morning, you looked at the—I checked Slack and you're like well, my experiments yesterday cost us $5,000. And at first, like, the—my response is instructive on this because, first, it was, “Oh, my God. What's going to happen now?” And it's like, first, hang on a second.First off, that seems suspect but assume it's real. I assumed it was real at the outset. It's “Oh, right. This is not my personal $5-a-month toybox account. We are a company; we can absolutely pay that.” Because it's like, I could absolutely reach out, call it a favor. “I made a mistake, and I need a favor on the bill, please,” to AWS.And I would never live it down, let's be clear. For a $7,000 mistake, I would almost certainly eat it. As opposed to having to prostrate myself like that in front of Amazon. I'm like, no, no, no. I want one of those like—if it's like, “Okay, you're going to, like, set back the company roadmap by six months if you have to pay this. Do you want to do it?” Like, [groans] “Fine, I'll eat some crow.”But okay. And then followed immediately by, wow, if Pete of all people can mess this up, customers are going to be doomed here. We should figure out what happened. And I'm doing the math. Like, Pete, “What did you actually do?” And you're sitting there and you're saying, “Well, I had like a 20 gig volume that I did this.” And I'm doing the numbers, and it's like—Pete: Something's wrong.Corey: “How sure are you when you say ‘gigabyte,' that you were—that actually means what you think it did? Like, were you off by a lot? Like, did you mean exabytes?” Like, what's the deal here?Pete: Like, multiple factors.Corey: Yeah. How much—“How many IOPS did you give that thing, buddy?” And it turned out what happened was that when they launched this, they had mispriced it in the system by a factor of a million. So, it was fun. I think by the end of it, all of your experimentation was somewhere between five to seven cents. Which—Pete: Yeah. It was a—Corey: Which is why you don't work here anymore because no one cost me seven cents of money to give to Amazon—Pete: How dare you?Corey: —on my watch. Get out.Pete: How dare you, sir?Corey: Exactly.Pete: Yeah, that [laugh] was amazing to see, as someone who has done—definitely maid screw-ups that have cost real money—you know, S3 list requests are always a fun one at scale—but that one was supremely fun to see the—Corey: That was a scary one because another one they'd done previously was they had messed up Lightsail pricing, where people would log in, and, like, “Okay, so what is my Lightsail instance going to cost?” And I swear to you, this is true, it was saying—this was back in 2017 or so—the answer was, like, “$4.3 billion.” Because when you see that you just start laughing because you know it's a mistake. You know, that they're not going to actually demand that you spend $4.3 billion for a single instance—unless it's running SAP—and great.It's just, it's a laugh. It's clearly a mispriced, and it's clearly a bug that's going to get—it's going to get fixed. I just spun up this new EBS volume that no one fully understands yet and it cost me thousands of dollars. That's the sort of thing that no, no, I could actually see that happening. There are instances now that cost something like 100 bucks an hour or whatnot to run. I can see spinning up the wrong thing by mistake and getting bitten by it. There's a bunch of fun configuration mistakes you can make that will, “Hee, hee, hee. Why can I see that bill spike from orbit?” And that's the scary thing.Pete: Well, it's the original CI and CD problem of the per-hour billing, right? That was super common of, like, yeah, like, an i3, you know, 16XL server is pretty cheap per hour, but if you're charged per hour and you spin up a bunch for five minutes. Like, it—you will be shocked [laugh] by what you see there. So—Corey: Yeah. Mistakes will show. And I get it. It's also people as individuals are very different psychologically than companies are. With companies it's one of those, “Great we're optimizing to bring in more revenue and we don't really care about saving money at all costs.”Whereas people generally have something that looks a lot like a fixed income in the form of a salary or whatnot, so it's it is easier for us to cut spend than it is for us to go out and make more money. Like, I don't want to get a second job, or pitch my boss on stuff, and yeah. So, all and all, routing out the rest of what happened at re:Invent, they—this is the problem is that they have a bunch of minor things like SageMaker Inference Recommender. Yeah, I don't care. Anything—Pete: [laugh].Corey: —[crosstalk 00:28:47] SageMaker I mostly tend to ignore, for safety. I did like the way they described Amplify Studio because they made it sound like a WYSIWYG drag and drop, build a React app. It's not it. It basically—you can do that in Figma and then it can hook it up to some things in some cases. It's not what I want it to be, which is Honeycode, except good. But we'll get there some year. Maybe.Pete: There's a lot of stuff that was—you know, it's the classic, like, preview, which sure, like, from a product standpoint, it's great. You know, they have a level of scale where they can say, “Here's this thing we're building,” which could be just a twinkle in a product managers, call it preview, and get thousands of people who would be happy to test it out and give you feedback, and it's a, it's great that you have that capability. But I often look at so much stuff and, like, that's really cool, but, like, can I, can I have it now? Right? Like—or you can't even get into the preview plan, even though, like, you have that specific problem. And it's largely just because either, like, your scale isn't big enough, or you don't have a good enough relationship with your account manager, or I don't know, countless other reasons.Corey: The thing that really throws me, too, is the pre-announcements that come a year or so in advance, like, the Outpost smaller ones are finally available, but it feels like when they do too many pre-announcements or no big marquee service announcements, as much as they talk about, “We're getting back to fundamentals,” no, you have a bunch of teams that blew the deadline. That's really what it is; let's not call it anything else. Another one that I think is causing trouble for folks—I'm fortunate in that I don't do much work with Oracle databases, or Microsoft SQL databases—but they extended RDS Custom to Microsoft SQL at the [unintelligible 00:30:27] SQL server at re:Invent this year, which means this comes down to things I actually use, we're going to have a problem because historically, the lesson has always been if I want to run my own databases and tweak everything, I do it on top of an EC2 instance. If I want to managed database, relational database service, great, I use RDS. RDS Custom basically gives you root into the RDS instance. Which means among other things, yes, you can now use RDS to run containers.But it lets you do a lot of things that are right in between. So, how do you position this? When should I use RDS Custom? Can you give me an easy answer to that question? And they used a lot of words to say, no, they cannot. It's basically completely blowing apart the messaging and positioning of both of those services in some unfortunate ways. We'll learn as we go.Pete: Yeah. Honestly, it's like why, like, why would I use this? Or how would I use this? And this is I think, fundamentally, what's hard when you just say yes to everything. It's like, they in many cases, I don't think, like, I don't want to say they don't understand why they're doing this, but if it's not like there's a visionary who's like, this fits into this multi-year roadmap.That roadmap is largely—if that roadmap is largely generated by the customers asking for it, then it's not like, oh, we're building towards this Northstar of RDS being whatever. You might say that, but your roadmap's probably getting moved all over the place because, you know, this company that pays you a billion dollars a year is saying, “I would give you $2 billion a year for all of my Oracle databases, but I need this specific thing.” I can't imagine a scenario that they would say, “Oh, well, we're building towards this Northstar, and that's not on the way there.” Right? They'd be like, “New Northstar. Another billion dollars, please.”Corey: Yep. Probably the worst release of re:Invent, from my perspective, is RUM, Real User Monitoring, for CloudWatch. And I, to be clear, I wrote a shitposting Twitter threading client called Last Tweet in AWS. Go to lasttweetinaws.com. You can all use it. It's free; I just built this for my own purposes. And I've instrumented it with RUM. Now, Real User Monitoring is something that a lot of monitoring vendors use, and also CloudWatch now. And what that is, is it embeds a listener into the JavaScript that runs on client load, and it winds up looking at what's going on loading times, et cetera, so you can see when users are unhappy. I have no problem with this. Other than that, you know, liking users? What's up with that?Pete: Crazy.Corey: But then, okay, now, what this does is unlike every other RUM tool out there, which charges per session, meaning I am going to be… doing a web page load, it charges per data item, which includes HTTP errors, or JavaScript errors, et cetera. Which means that if you have a high transaction volume site and suddenly your CDN takes a nap like Fastly did for an hour last year, suddenly your bill is stratospheric for this because errors abound and cascade, and you can have thousands of errors on a single page load for these things, and it is going to be visible from orbit, at least with a per session basis thing, when you start to go viral, you understand that, “Okay, this is probably going to cost me some more on these things, and oops, I guess I should write less compelling content.” Fine. This is one of those one misconfiguration away and you are wailing and gnashing teeth. Now, this is a new service. I believe that they will waive these surprise bills in the event that things like that happen. But it's going to take a while and you're going to be worrying the whole time if you've rolled this out naively. So it's—Pete: Well and—Corey: —I just don't like the pricing.Pete: —how many people will actively avoid that service, right? And honestly, choose a competitor because the competitor could be—the competitor could be five times more expensive, right, on face value, but it's the certainty of it. It's the uncertainty of what Amazon will charge you. Like, no one wants a surprise bill. “Well, a vendor is saying that they'll give us this contract for $10,000. I'm going to pay $10,000, even though RUM might be a fraction of that price.”It's honestly, a lot of these, like, product analytics tools and monitoring tools, you'll often see they price be a, like, you know, MAU, Monthly Active User, you know, or some sort of user-based pricing, like, the number of people coming to your site. You know, and I feel like at least then, if you are trying to optimize for lots of users on your site, and more users means more revenue, then you know, if your spend is going up, but your revenue is also going up, that's a win-win. But if it's like someone—you know, your third-party vendor dies and you're spewing out errors, or someone, you know, upgraded something and it spews out errors. That no one would normally see; that's the thing. Like, unless you're popping open that JavaScript console, you're not seeing any of those errors, yet somehow it's like directly impacting your bottom line? Like that doesn't feel [crosstalk 00:35:06].Corey: Well, there is something vaguely Machiavellian about that. Like, “How do I get my developers to care about errors on consoles?” Like, how about we make it extortionately expensive for them not to. It's, “Oh, all right, then. Here we go.”Pete: And then talk about now you're in a scenario where you're working on things that don't directly impact the product. You're basically just sweeping up the floor and then trying to remove errors that maybe don't actually affect it and they're not actually an error.Corey: Yeah. I really do wonder what the right answer is going to be. We'll find out. Again, we live, we learn. But it's also, how long does it take a service that has bad pricing at launch, or an unfortunate story around it to outrun that reputation?People are still scared of Glacier because of its original restore pricing, which was non-deterministic for any sensible human being, and in some cases lead to I'm used to spending 20 to 30 bucks a month on this. Why was I just charged two grand?Pete: Right.Corey: Scare people like that, they don't come back.Pete: I'm trying to actually remember which service it is that basically gave you an estimate, right? Like, turn it on for a month, and it would give you an estimate of how much this was going to cost you when billing started.Corey: It was either Detective or GuardDuty.Pete: Yeah, it was—yeah, that's exactly right. It was one of those two. And honestly, that was unbelievably refreshing to see. You know, like, listen, you have the data, Amazon. You know what this is going to cost me, so when I, like, don't make me spend all this time to go and figure out the cost. If you have all this data already, just tell me, right?And if I look at it and go, “Yeah, wow. Like, turning this on in my environment is going to cost me X dollars. Like, yeah, that's a trade-off I want to make, I'll spend that.” But you know, with some of the—and that—a little bit of a worry on some of the intelligent tiering on S3 is that the recommendation is likely going to be everything goes to intelligent tiering first, right? It's the gp3 story. Put everything on gp3, then move it to the proper volume, move it to an sc or an st or an io. Like, gp3 is where you start. And I wonder if that's going to be [crosstalk 00:37:08].Corey: Except I went through a wizard yesterday to launch an EC2 instance and its default on the free tier gp2.Pete: Yeah. Interesting.Corey: Which does not thrill me. I also still don't understand for the life of me why in some regions, the free tier is a t2 instance, when t3 is available.Pete: They're uh… my guess is that they've got some free t—they got a bunch of t2s lying around. [laugh].Corey: Well, one of the most notable announcements at re:Invent that most people didn't pay attention to is their ability now to run legacy instance types on top of Nitro, which really speaks to what's going on behind the scenes of we can get rid of all that old hardware and emulate the old m1 on modern equipment. So, because—you can still have that legacy, ancient instance, but now you're going—now we're able to wind up greening our data centers, which is part of their big sustainability push, with their ‘Sustainability Pillar' for the well-architected framework. They're talking more about what the green choices in cloud are. Which is super handy, not just because of the economic impact because we could use this pretty directly to reverse engineer their various margins on a per-service or per-offering basis. Which I'm not sure they're aware of yet, but oh, they're going to be.And that really winds up being a win for the planet, obviously, but also something that is—that I guess puts a little bit of choice on customers. The challenge I've got is, with my serverless stuff that I build out, if I spend—the Google search I make to figure out what the most economic, most sustainable way to do that is, is going to have a bigger carbon impact on the app itself. That seems to be something that is important at scale, but if you're not at scale, it's one of those, don't worry about it. Because let's face it, the cloud providers—all of them—are going to have a better sustainability story than you are running this in your own data centers, or on a Raspberry Pi that's always plugged into the wall.Pete: Yeah, I mean, you got to remember, Amazon builds their own power plants to power their data centers. Like, that's the level they play, right? There, their economies of scale are so entirely—they're so entirely different than anything that you could possibly even imagine. So, it's something that, like, I'm sure people will want to choose for. But, you know, if I would honestly say, like, if we really cared about our computing costs and the carbon footprint of it, I would love to actually know the carbon footprint of all of the JavaScript trackers that when I go to various news sites, and it loads, you know, the whatever thousands of trackers and tracking the all over, like, what is the carbon impact of some of those choices that I actually could control, like, as a either a consumer or business person?Corey: I really hope that it turns into something that makes a meaningful difference, and it's not just greenwashing. But we'll see. In the fullness of time, we're going to figure that out. Oh, they're also launching some mainframe stuff. They—like that's great.Pete: Yeah, those are still a thing.Corey: I don't deal with a lot of customers that are doing things with that in any meaningful sense. There is no AWS/400, so all right.Pete: [laugh]. Yeah, I think honestly, like, I did talk to a friend of mine who's in a big old enterprise and has a mainframe, and they're actually replacing their mainframe with Lambda. Like they're peeling off—which is, like, a great move—taking the monolith, right, and peeling off the individual components of what it can do into these discrete Lambda functions. Which I thought was really fascinating. Again, it's a five-year-long journey to do something like that. And not everyone wants to wait five years, especially if their support's about to run out for that giant box in the, you know, giant warehouse.Corey: The thing that I also noticed—and this is probably the—I guess, one of the—talk about swing and a miss on pricing—they have a—what is it?—there's a VPC IP Address Manager, which tracks the the IP addresses assigned to your VPCs that are allocated versus not, and it's 20 cents a month per IP address. It's like, “Okay. So, you're competing against a Google Sheet or an Excel spreadsheet”—which is what people are using for these things now—“Only you're making it extortionately expensive?”Pete: What kind of value does that provide for 20—I mean, like, again—Corey: I think Infoblox or someone like that offers it where they become more cost-effective as soon as you hit 500 IP addresses. And it's just—like, this is what I'm talking about. I know it does not cost AWS that kind of money to store an IP address. You can store that in a Route 53 TXT record for less money, for God's sake. And that's one of those, like, “Ah, we could extract some value pricing here.”Like, I don't know if it's a good product or not. Given its pricing, I don't give a shit because it's going to be too expensive for anything beyond trivial usage. So, it's a swing and a miss from that perspective. It's just, looking at that, I laugh, and I don't look at it again.Pete: See I feel—Corey: I'm not usually price sensitive. I want to be clear on that. It's just, that is just Looney Tunes, clown shoes pricing.Pete: Yeah. It's honestly, like, in many cases, I think the thing that I have seen, you know, in the past few years is, in many cases, it can honestly feel like Amazon is nickel-and-diming their customers in so many ways. You know, the explosion of making it easy to create multiple Amazon accounts has a direct impact to waste in the cloud because there's a lot of stuff you have to have her account. And the more accounts you have, those costs grow exponentially as you have these different places. Like, you kind of lose out on the economies of scale when you have a smaller number of accounts.And yeah, it's hard to optimize for that. Like, if you're trying to reduce your spend, it's challenging to say, “Well, by making a change here, we'll save, you know, $10,000 in this account.” “That doesn't seem like a lot when we're spending millions.” “Well, hold on a second. You'll save $10,000 per account, and you have 500 accounts,” or, “You have 1000 accounts,” or something like that.Or almost cost avoidance of this cost is growing unbounded in all of your accounts. It's tiny right now. So, like, now would be the time you want to do something with it. But like, again, for a lot of companies that have adopted the practice of endless Amazon accounts, they've almost gone, like, it's the classic, like, you know, I've got 8000 GitHub repositories for my source code. Like, that feels just as bad as having one GitHub repository for your repo. I don't know what the balance is there, but anytime these different types of services come out, it feels like, “Oh, wow. Like, I'm going to get nickeled and dimed for it.”Corey: This ties into the re:Post launch, which is a rebranding of their forums, where, okay, great, it was a little crufty and it need modernize, but it still ties your identity to an IAM account, or the root email address for an Amazon account, which is great. This is completely worthless because as soon as I change jobs, I lose my identity, my history, the rest, on this forum. I'm not using it. It shows that there's a lack of awareness that everyone is going to have multiple accounts with which they interact, and that people are going to deal with the platform longer than any individual account will. It's just a continual swing and a miss on things like that.And it gets back to the billing question of, “Okay. When I spin up an account, do I want them to just continue billing me—because don't turn this off; this is important—or do I want there to be a hard boundary where if you're about to charge me, turn it off. Turn off the thing that's about to cost me money.” And people hem and haw like this is an insurmountable problem, but I think the way to solve it is, let me specify that intent when I provision the account. Where it's, “This is a production account for a bank. I really don't want you turning it off.” Versus, “I'm a student learner who thinks that a Managed NAT Gateway might be a good thing. Yeah, I want you to turn off my demo Hello World app that will teach me what's going on, rather than surprising me with a five-figure bill at the end of the month.”Pete: Yeah. It shouldn't be that hard. I mean, but again, I guess everything's hard at scale.Corey: Oh, yeah. Oh yeah.Pete: But still, I feel like every time I log into Cost Explorer and I look at—and this is years it's still not fixed. Not that it's even possible to fix—but on the first day of the month, you look at Cost Explorer, and look at what Amazon is estimating your monthly bill is going to be. It's like because of your, you know—Corey: Your support fees, and your RI purchases, and savings plans purchases.Pete: [laugh]. All those things happened, right? First of the month, and it's like, yeah, “Your bill's going to be $800,000 this year.” And it's like, “Shouldn't be, like, $1,000?” Like, you know, it's the little things like that, that always—Corey: The one-off charges, like, “Oh, your Route 53 zone,” and all the stuff that gets charged on a monthly cadence, which fine, whatever. I mean, I'm okay with it, but it's also the, like, be careful when that happen—I feel like there's a way to make that user experience less jarring.Pete: Yeah because that problem—I mean, in my scenario, companies that I've worked at, there's been multiple times that a non-technical person will look at that data and go into immediate freakout mode, right? And that's never something that you want to have happen because now that's just adding a lot of stress and anxiety into a company that is—with inaccurate data. Like, the data—like, the answer you're giving someone is just wrong. Perhaps you shouldn't even give it to them if it's that wrong. [laugh].Corey: Yeah, I'm looking forward to seeing what happens this coming year. We're already seeing promising stuff. They—give people a timeline on how long in advance these things record—late last night, AWS released a new console experience. When you log into the AWS console now, there's a new beta thing. And I gave it some grief on Twitter because I'm still me, but like the direction it's going. It lets you customize your view with widgets and whatnot.And until they start selling widgets on marketplace or having sponsored widgets, you can't remove I like it, which is no guarantee at some point. But it shows things like, I can move the cost stuff, I can move the outage stuff up around, I can have the things that are going on in my account—but who I am means I can shift this around. If I'm a finance manager, cool. I can remove all the stuff that's like, “Hey, you want to get started spinning up an EC2 instance?” “Absolutely not. Do I want to get told, like, how to get certified? Probably not. Do I want to know what the current bill is and whether—and my list of favorites that I've pinned, whatever services there? Yeah, absolutely do.” This is starting to get there.Pete: Yeah, I wonder if it really is a way to start almost hedging on organizations having a wider group of people accessing AWS. I mean, in previous companies, I absolutely gave access to the console for tools like QuickSight, for tools like Athena, for the DataBrew stuff, the Glue DataBrew. Giving, you know, non-technical people access to be able to do these, like, you know, UI ETL tasks, you know, a wider group of a company is getting access into Amazon. So, I think anything that Amazon does to improve that experience for, you know, the non-SREs, like the people who would traditionally log in, like, that is an investment definitely worth making.Corey: “Well, what could non-engineering types possibly be doing in the AWS console?” “I don't know, jackhole, maybe paying the bill? Just a thought here.” It's the, there are people who look at these things from a variety of different places, and you have such sprawl in the AWS world that there are different personas by a landslide. If I'm building Twitter for Pets, you probably don't want to be pitching your mainframe migration services to me the same way that you would if I were a 200-year-old insurance company.Pete: Yeah, exactly. And the number of those products are going to grow, the number of personas are going to grow, and, yeah, they'll have to do something that they want to actually, you know, maintain that experience so that every person can have, kind of, the experience that they want, and not be distracted, you know? “Oh, what's this? Let me go test this out.” And it's like, you know, one-time charge for $10,000 because, like, that's how it's charged. You know, that's not an experience that people like.Corey: No. They really don't. Pete, I want to thank you for spending the time to chat with me again, as is our tradition. I'm hoping we can do it in person this year, when we go at the end of 2022, to re:Invent again. Or that no one goes in person. But this hybrid nonsense is for the birds.Pete: Yeah. I very much would love to get back to another one, and yeah, like, I think there could be an interesting kind of merging here of our annual re:Invent recap slash live brunch, you know, stream you know, hot takes after a long week. [laugh].Corey: Oh, yeah. The real way that you know that it's a good joke is when one of us says something, the other one sprays scrambled eggs out of their nose. Yeah, that's the way to do it.Pete: Exactly. Exactly.Corey: Pete, thank you so much. If people want to learn more about what you're up to—hopefully, you know, come back. We miss you, but you're unaffiliated, you're a startup advisor. Where can people find you to learn more, if they for some unforgivable reason don't know who or what a Pete Cheslock is?Pete: Yeah. I think the easiest place to find me is always on Twitter. I'm just at @petecheslock. My DMs are always open and I'm always down to expand my network and chat with folks.And yeah, right, now, I'm just, as I jokingly say, professionally unaffiliated. I do some startup advisory work and have been largely just kind of—honestly checking out the state of the economy. Like, there's a lot of really interesting companies out there, and some interesting problems to solve. And, you know, trying to spend some of my time learning more about what companies are up to nowadays. So yeah, if you got some interesting problems, you know, you can follow my Twitter or go to LinkedIn if you want some great, you know, business hot takes about, you know, shitposting basically.Corey: Same thing. Pete, thanks so much for joining me, I appreciate it.Pete: Thanks for having me.Corey: Pete Cheslock, startup advisor, professionally unaffiliated, and recurring re:Invent analyst pal of mine. I'm Cloud Economist Corey Quinn and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice along with an angry comment calling me a jackass because do I know how long it took you personally to price CloudWatch RUM?Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.

North Meets South Web Podcast
New year, new job, new start

North Meets South Web Podcast

Play Episode Listen Later Jan 17, 2022 63:54


In this episode, Jake and Michael discuss Michael's new job, YAGNI, and approaches to working your way into a new codebase and a new industry.This episode is sponsored by Workvivo and Makeable.dk and was streamed live.Show links Laravel Transporter Saloon YAGNI ClickUp Allen Holub on building software better and building better software 

Screaming in the Cloud
Slinging CDK Knowledge with Matt Coulter

Screaming in the Cloud

Play Episode Listen Later Jan 12, 2022 37:37


About MattMatt is an AWS DevTools Hero, Serverless Architect, Author and conference speaker. He is focused on creating the right environment for empowered teams to rapidly deliver business value in a well-architected, sustainable and serverless-first way.You can usually find him sharing reusable, well architected, serverless patterns over at cdkpatterns.com or behind the scenes bringing CDK Day to life.Links: AWS CDK Patterns: https://cdkpatterns.com The CDK Book: https://thecdkbook.com CDK Day: https://www.cdkday.com TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: It seems like there is a new security breach every day. Are you confident that an old SSH key, or a shared admin account, isn't going to come back and bite you? If not, check out Teleport. Teleport is the easiest, most secure way to access all of your infrastructure. The open source Teleport Access Plane consolidates everything you need for secure access to your Linux and Windows servers—and I assure you there is no third option there. Kubernetes clusters, databases, and internal applications like AWS Management Console, Yankins, GitLab, Grafana, Jupyter Notebooks, and more. Teleport's unique approach is not only more secure, it also improves developer productivity. To learn more visit: goteleport.com. And not, that is not me telling you to go away, it is: goteleport.com.Corey: This episode is sponsored in part by our friends at Rising Cloud, which I hadn't heard of before, but they're doing something vaguely interesting here. They are using AI, which is usually where my eyes glaze over and I lose attention, but they're using it to help developers be more efficient by reducing repetitive tasks. So, the idea being that you can run stateless things without having to worry about scaling, placement, et cetera, and the rest. They claim significant cost savings, and they're able to wind up taking what you're running as it is in AWS with no changes, and run it inside of their data centers that span multiple regions. I'm somewhat skeptical, but their customers seem to really like them, so that's one of those areas where I really have a hard time being too snarky about it because when you solve a customer's problem and they get out there in public and say, “We're solving a problem,” it's very hard to snark about that. Multus Medical, Construx.ai and Stax have seen significant results by using them. And it's worth exploring. So, if you're looking for a smarter, faster, cheaper alternative to EC2, Lambda, or batch, consider checking them out. Visit risingcloud.com/benefits. That's risingcloud.com/benefits, and be sure to tell them that I said you because watching people wince when you mention my name is one of the guilty pleasures of listening to this podcast.Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. I'm joined today by Matt Coulter, who is a Technical Architect at Liberty Mutual. You may have had the privilege of seeing him on the keynote stage at re:Invent last year—in Las Vegas or remotely—that last year of course being 2021. But if you make better choices than the two of us did, and found yourself not there, take the chance to go and watch that keynote. It's really worth seeing.Matt, first, thank you for joining me. I'm sorry, I don't have 20,000 people here in the audience to clap this time. They're here, but they're all remote as opposed to sitting in the room behind me because you know, social distancing.Matt: And this left earphone, I just have some applause going, just permanently, just to keep me going. [laugh].Corey: That's sort of my own internal laugh track going on. It's basically whatever I say is hilarious, to that. So yeah, doesn't really matter what I say, how I say it, my jokes are all for me. It's fine. So, what was it like being on stage in front of that many people? It's always been a wild experience to watch and for folks who haven't spent time on the speaking circuit, I don't think that there's any real conception of what that's like. Is this like giving a talk at work, where I just walk on stage randomly, whatever I happened to be wearing? And, oh, here's a microphone, I'm going to say words. What is the process there?Matt: It's completely different. For context for everyone, before the pandemic, I would have pretty regularly talked in front of, I don't know, maybe one, two hundred people in Liberty, in Belfast. So, I used to be able to just, sort of, walk in front of them, and lean against the pillar, and use my clicker, and click through, but the process for actually presenting something as big as a keynote and re:Invent is so different. For starters, you think that when you walk onto the stage, you'll actually be able to see the audience, but the way the lights are set up, you can pretty much see about one row of people, and they're not the front row, so anybody I knew, I couldn't actually see.And yeah, you can only see, sort of like, the from the void, and then you have your screens, so you've six sets of screens that tell you your notes as well as what slides you're on, you know, so you can pivot. But other than that, I mean, it feels like you're just talking to yourself outside of whenever people, thankfully, applause. It's such a long process to get there.Corey: I've always said that there are a few different transition stages as the audience size increases, but for me, the final stage is more or less anything above 750 people. Because as you say, you aren't able to see that many beyond that point, and it doesn't really change anything meaningfully. The most common example that you see in the wild is jokes that work super well with a small group of people fall completely flat to large audiences. It's why so much corporate numerous cheesy because yeah, everyone in the rehearsals is sitting there laughing and the joke kills, but now you've got 5000 people sitting in a room and that joke just sounds strained and forced because there's no longer a conversation, and no one has the shared context that—the humor has to change. So, in some cases when you're telling a story about what you're going to say on stage, during a rehearsal, they're going to say, “Well, that joke sounds really corny and lame.” It's, “Yeah, wait until you see it in front of an audience. It will land very differently.” And I'm usually right on that.I would also advise, you know, doing what you do and having something important and useful to say, as opposed to just going up there to tell jokes the whole time. I wanted to talk about that because you talked about how you're using various CDK and other serverless style patterns in your work at Liberty Mutual.Matt: Yeah. So, we've been using CDK pretty extensively since it was, sort of, Q3 2019. At that point, it was new. Like, it had just gone GA at the time, just came out of dev preview. And we've been using CDK from the perspective of we want to be building serverless-first, well-architected apps, and ideally we want to be building them on AWS.Now, the thing is, we have 5000 people in our IT organization, so there's sort of a couple of ways you can take to try and get those people onto the cloud: You can either go the route of being, like, there is one true path to architecture, this is our architecture and everything you want to build can fit into that square box; or you can go the other approach and try and have the golden path where you say this is the paved road that is really easy to do, but if you want to differentiate from that route, that's okay. But what you need to do is feed back into the golden path if that works. Then everybody can improve. And that's where we've started been using CDK. So, what you heard me talk about was the software accelerator, and it's sort of a different approach.It's where anybody can build a pattern and then share it so that everybody else can rapidly, you know, just reuse it. And what that means is effectively you can, instead of having to have hundreds of people on a central team, you can actually just crowdsource, and sort of decentralize the function. And if things are good, then a small team can actually come in and audit them, so to speak, and check that it's well-architected, and doesn't have flaws, and drive things that way.Corey: I have to confess that I view the CDK as sort of a third stage automation approach, and it's one that I haven't done much work with myself. The first stage is clicking around in the console; the second is using CloudFormation or Terraform; the third stage is what we're talking about here is CDK or Pulumi, or something like that. And then you ascend to the final fourth stage, which is what I use, which is clicking around in the AWS console, but then you lie to people about it. ClickOps is poised to take over the world. But that's okay. You haven't gotten that far yet. Instead, you're on the CDK side. What advantages does CDK offer that effectively CloudFormation or something like it doesn't?Matt: So, first off, for ClickOps in Liberty, we actually have the AWS console as read-only in all of our accounts, except for sandbox. So, you can ClickOps in sandbox to learn, but if you want to do something real, unfortunately, it's going to fail you. So.—Corey: I love that pattern. I think I might steal that.Matt: [laugh]. So, originally, we went heavy on CloudFormation, which is why CDK worked well for us. And because we've actually—it's been a long journey. I mean, we've been deploying—2014, I think it was, we first started deploying to AWS, and we've used everything from Terraform, to you name it. We've built our own tools, believe it or not, that are basically CDK.And the thing about CloudFormation is, it's brilliant, but it's also incredibly verbose and long because you need to specify absolutely everything that you want to deploy, and every piece of configuration. And that's fine if you're just deploying a side project, but if you're in an enterprise that has responsibilities to protect user data, and you can't just deploy anything, they end up thousands and thousands and thousands of lines long. And then we have amazing guardrails, so if you tried to deploy a CloudFormation template with a flaw in it, we can either just fix it, or reject the deploy. But CloudFormation is not known to be the fastest to deploy, so you end up in this developer cycle, where you build this template by hand, and then it goes through that CloudFormation deploy, and then you get the failure message that it didn't deploy because of some compliance thing, and developers just got frustrated, and were like, sod this. [laugh].I'm not deploying to AWS. Back the on-prem. And that's where CDK was a bit different because it allowed us to actually build abstractions with all of our guardrails baked in, so that it just looked like a standard class, for developers, like, developers already know Java, Python, TypeScript, the languages off CDK, and so we were able to just make it easy by saying, “You want API Gateway? There's an API Gateway class. You want, I don't know, an EC2 instance? There you go.” And that way, developers could focus on the thing they wanted, instead of all of the compliance stuff that they needed to care about every time they wanted to deploy.Corey: Personally, I keep lobbying AWS to add my preferred language, which is crappy shell scripting, but for some reason they haven't really been quick to add that one in. The thing that I think surprises me, on some level—though, perhaps it shouldn't—is not just the adoption of serverless that you're driving at Liberty Mutual, but the way that you're interacting with that feels very futuristic, for lack of a better term. And please don't think that I'm in any way describing this in a way that's designed to be insulting, but I do a bunch of serverless nonsense on Twitter for Pets. That's not an exaggeration. twitterforpets.com has a bunch of serverless stuff behind it because you know, I have personality defects.But no one cares about that static site that's been a slide dump a couple of times for me, and a running joke. You're at Liberty Mutual; you're an insurance company. When people wind up talking about big enterprise institutions, you're sort of a shorthand example of exactly what they're talking about. It's easy to contextualize or think of that as being very risk averse—for obvious reasons; you are an insurance company—as well as wanting to move relatively slowly with respect to technological advancement because mistakes are going to have drastic consequences to all of your customers, people's lives, et cetera, as opposed to tweets or—barks—not showing up appropriately at the right time. How did you get to the, I guess, advanced architectural philosophy that you clearly have been embracing as a company, while having to be respectful of the risk inherent that comes with change, especially in large, complex environments?Matt: Yeah, it's funny because so for everyone, we were talking before this recording started about, I've been with Liberty since 2011. So, I've seen a lot of change in the length of time I've been here. And I've built everything from IBM applications right the way through to the modern serverless apps. But the interesting thing is, the journey to where we are today definitely started eight or nine years ago, at a minimum because there was something identified in the leadership that they said, “Listen, we're all about our customers. And that means we don't want to be wasting millions of dollars, and thousands of hours, and big trains of people to build software that does stuff. We want to focus on why are we building a piece of software, and how quickly can we get there? If you focus on those two things you're doing all right.”And that's why starting from the early days, we focused on things like, okay, everything needs to go through CI/CD pipelines. You need to have your infrastructure as code. And even if you're deploying on-prem, you're still going to be using the same standards that we use to deploy to AWS today. So, we had years and years and years of just baking good development practices into the company. And then whenever we started to move to AWS, the question became, do we want to just deploy the same thing or do we want to take full advantage of what the cloud has to offer? And I think because we were primed and because the leadership had the right direction, you know, we were just sitting there ready to say, “Okay, serverless seems like a way we can rapidly help our customers.” And that's what we've done.Corey: A lot of the arguments against serverless—and let's be clear, they rhyme with the previous arguments against cloud that lots of people used to make; including me, let's be clear here. I'm usually wrong when I try to predict the future. “Well, you're putting your availability in someone else's hands,” was the argument about cloud. Yeah, it turns out the clouds are better at keeping things up than we are as individual companies.Then with serverless, it's the, “Well, if they're handling all that stuff for you on their side, when they're down, you're down. That's an unacceptable business risk, so we're going to be cloud-agnostic and multi-cloud, and that means everything we build serverlessly needs to work in multiple environments, including in our on-prem environment.” And from the way that we're talking about servers and things that you're building, I don't believe that is technically possible, unless some of the stuff you're building is ridiculous. How did you come to accept that risk organizationally?Matt: These are the conversations that we're all having. Sort of, I'd say once a week, we all have a multi-cloud discussion—and I really liked the article you wrote, it was maybe last year, maybe the year before—but multi-cloud to me is about taking the best capabilities that are out there and bringing them together. So, you know, like, Azure [ID 00:12:47] or whatever, things from the other clouds that they're good at, and using those rather than thinking, “Can I build a workload that I can simultaneously pay all of the price to run across all of the clouds, all of the time, so that if one's down, theoretically, I might have an outage?” So, the way we've looked at it is we embraced really early the well-architected framework from AWS. And it talks about things like you need to have multi-region availability, you need to have your backups in place, you need to have things like circuit breakers in place for if third-party goes down, and we've just tried to build really resilient architectures as best as we can on AWS. And do you know what I think, if [laugh] it AWS is not—I know at re:Invent, there it went down extraordinarily often compared to normal, but in general—Corey: We were all tired of re:Invent; their us-east-1 was feeling the exact same way.Matt: Yeah, so that's—it deserved a break. But, like, if somebody can't buy insurance for an hour, once a year, [laugh] I think we're okay with it versus spending millions to protect that one hour.Corey: And people make assumptions based on this where, okay, we had this problem with us-east-1 that froze things like the global Route 53 control planes; you couldn't change DNS for seven hours. And I highlighted that as, yeah, this is a problem, and it's something to severely consider, but I will bet you anything you'd care to name that there is an incredibly motivated team at AWS, actively fixing that as we speak. And by—I don't know how long it takes to untangle all of those dependencies, but I promise they're going to be untangled in relatively short order versus running data centers myself, when I discover a key underlying dependency I didn't realize was there, well, we need to break that. That's never going to happen because we're trying to do things as a company, and it's just not the most important thing for us as a going concern. With AWS, their durability and reliability is the most important thing, arguably compared to security.Would you rather be down or insecure? I feel like they pick down—I would hope in most cases they would pick down—but they don't want to do either one. That is something they are drastically incentivized to fix. And I'm never going to be able to fix things like that and I don't imagine that you folks would be able to either.Matt: Yeah, so, two things. The first thing is the important stuff, like, for us, that's claims. We want to make sure at any point in time, if you need to make a claim you can because that is why we're here. And we can do that with people whether or not the machines are up or down. So, that's why, like, you always have a process—a manual process—that the business can operate, irrespective of whether the cloud is still working.And that's why we're able to say if you can't buy insurance in that hour, it's okay. But the other thing is, we did used to have a lot of data centers, and I have to say, the people who ran those were amazing—I think half the staff now work for AWS—but there was this story that I heard where there was an app that used to go down at the same time every day, and nobody could work out why. And it was because someone was coming in to clean the room at that time, and they unplugged the server to plug in a vacuum, and then we're cleaning the room, and then plugging it back in again. And that's the kind of thing that just happens when you manage people, and you manage a building, and manage a premises. Whereas if you've heard that happened that AWS, I mean, that would be front page news.Corey: Oh, it absolutely would. There's also—as you say, if it's the sales function, if people aren't able to buy insurance for an hour, when us-east-1 went down, the headlines were all screaming about AWS taking an outage, and some of the more notable customers were listed as examples of this, but the story was that, “AWS has massive outage,” not, “Your particular company is bad at technology.” There's sort of a reputational risk mitigation by going with one of these centralized things. And again, as you're alluding to, what you're doing is not life-critical as far as the sales process and getting people to sign up. If an outage meant that suddenly a bunch of customers were no longer insured, that's a very different problem. But that's not your failure mode.Matt: Exactly. And that's where, like, you got to look at what your business is, and what you're specifically doing, but for 99.99999% of businesses out there, I'm pretty sure you can be down for the tiny window that AWS is down per year, and it will be okay, as long as you plan for it.Corey: So, one thing that really surprised me about the entirety of what you've done at Liberty Mutual is that you're a big enterprise company, and you can take a look at any enterprise company, and say that they have dueling mottos, which is, “I am not going to comment on that,” or, “That's not funny.” Like, the safe mode for any large concern is to say nothing at all. But a lot of folks—not just you—at Liberty have been extremely vocal about the work that you're doing, how you view these things, and I almost want to call it advocacy or evangelism for the CDK. I'm slightly embarrassed to admit that for a little while there, I thought you were an AWS employee in their DevRel program because you were such an advocate in such strong ways for the CDK itself.And that is not something I expected. Usually you see the most vocal folks working in environments that, let's be honest, tend to play a little bit fast and loose with things like formal corporate communications. Liberty doesn't and yet, there you folks are telling these great stories. Was that hard to win over as a culture, or am I just misunderstanding how corporate life is these days?Matt: No, I mean, so it was different, right? There was a point in time where, I think, we all just sort of decided that—I mean, we're really good at what we do from an engineering perspective, and we wanted to make sure that, given the messaging we were given, those 5000 teck employees in Liberty Mutual, if you consider the difference in broadcasting to 5000 versus going external, it may sound like there's millions, billions of people in the world, but in reality, the difference in messaging is not that much. So, to me what I thought, like, whenever I started anyway—it's not, like, we had a meeting and all decided at the same time—but whenever I started, it was a case of, instead of me just posting on all the internal channels—because I've been doing this for years—it's just at that moment, I thought, I could just start saying these things externally and still bring them internally because all you've done is widened the audience; you haven't actually made it shallower. And that meant that whenever I was having the internal conversations, nothing actually changed except for it meant external people, like all their Heroes—like Jeremy Daly—could comment on these things, and then I could bring that in internally. So, it almost helped the reverse takeover of the enterprise to change the culture because I didn't change that much except for change the audience of who I was talking to.Corey: This episode is sponsored by our friends at Oracle HeatWave is a new high-performance accelerator for the Oracle MySQL Database Service. Although I insist on calling it “my squirrel.” While MySQL has long been the worlds most popular open source database, shifting from transacting to analytics required way too much overhead and, ya know, work. With HeatWave you can run your OLTP and OLAP, don't ask me to ever say those acronyms again, workloads directly from your MySQL database and eliminate the time consuming data movement and integration work, while also performing 1100X faster than Amazon Aurora, and 2.5X faster than Amazon Redshift, at a third of the cost. My thanks again to Oracle Cloud for sponsoring this ridiculous nonsense.Corey: One thing that you've done that I want to say is admirable, and I stumbled across it when I was doing some work myself over the break, and only right before this recording did I discover that it was you is the cdkpatterns.com website. Specifically what I love about it is that it publishes a bunch of different patterns of ways to do things. This deviates from a lot of tutorials on, “Here's how to build this one very specific thing,” and instead talks about, “Here's the architecture design; here's what the baseline pattern for that looks like.” It's more than a template, but less than a, “Oh, this is a messaging app for dogs and I'm trying to build a messaging app for cats.” It's very generalized, but very direct, and I really, really like that model of demo.Matt: Thank you. So, watching some of your Twitter threads where you experiment with new—Corey: Uh oh. People read those. That's a problem.Matt: I know. So, whatever you experiment with a new piece of AWS to you, I've always wondered what it would be like to be your enabling architect. Because technically, my job in Liberty is, I meant to try and stay ahead of everybody and try and ease the on-ramp to these things. So, if I was your enabling architect, I would be looking at it going, “I should really have a pattern for this.” So that whenever you want to pick up that new service the patterns in cdkpatterns.com, there's 24, 25 of them right there, but internally, there's way more than dozens now.The goal is, the pattern is the least amount to code for you to learn a concept. And then that way, you can not only see how something works, but you can maybe pick up one of the pieces of the well-architected framework while you're there: All of it's unit tested, all of it is proper, you know, like, commented code. The idea is to not be crap, but not be gold-plated either. I'm currently in the process of upgrading that all to V2 as well. So, that [unintelligible 00:21:32].Corey: You mentioned a phrase just now: “Enabling architect.” I have to say this one that has not crossed my desk before. Is that an internal term you use? Is that an enterprise concept I've somehow managed to avoid? Is that an AWS job role? What is that?Matt: I've just started saying [laugh] it's my job over the past couple of years. That—I don't know, patent pending? But the idea to me is—Corey: No, it's evocative. I love the term, I'd love to learn more.Matt: Yeah, because you can sort of take two approaches to your architecture: You can take the traditional approach, which is the ‘house of no' almost, where it's like, “This is the architecture. How dare you want to deviate. This is what we have decided. If you want to change it, here's the Architecture Council and go through enterprise architecture as people imagine it.” But as people might work out quite quickly, whenever they meet me, the whole, like, long conversational meetings are not for me. What I want to do is teach engineers how to help themselves, so that's why I see myself as enabling.And what I've been doing is using techniques like Wardley Mapping, which is where you can go out and you can actually take all the components of people's architecture and you can draw them on a map for—it's a map of how close they are to the customer, as well as how cutting edge the tech is, or how aligned to our strategic direction it is. So, you can actually map out all of the teams, and—there's 160, 170 engineers in Belfast and Dublin, and I can actually go in and say, “Oh, that piece of your architecture would be better if it was evolved to this. Well, I have a pattern for that,” or, “I don't have a pattern for that, but you know what? I'll build one and let's talk about it next week.” And that's always trying to be ahead, instead of people coming to me and I have to say no.Corey: AWS Proton was designed to do something vaguely similar, where you could set out architectural patterns of—like, the two examples that they gave—I don't know if it's in general availability yet or still in public preview, but the ones that they gave were to build a REST API with Lambda, and building something-or-other with Fargate. And the idea was that you could basically fork those, or publish them inside of your own environment of, “Oh, you want a REST API; go ahead and do this.” It feels like their vision is a lot more prescriptive than what yours is.Matt: Yeah. I talked to them quite a lot about Proton, actually because, as always, there's different methodologies and different ways of doing things. And as I showed externally, we have our software accelerator, which is kind of our take on Proton, and it's very open. Anybody can contribute; anybody can consume. And then that way, it means that you don't necessarily have one central team, you can have—think of it more like an SRE function for all of the patterns, rather than… the Proton way is you've separate teams that are your DevOps teams that set up your patterns and then separate team that's consumer, and they have different permissions, different rights to do different things. If you use a Proton pattern, anytime an update is made to that pattern, it auto-deploys your infrastructure.Corey: I can see that breaking an awful lot.Matt: [laugh]. Yeah. So, the idea is sort of if you're a consumer, I assume you [unintelligible 00:24:35] be going to change that infrastructure. You can, they've built in an escape hatch, but the whole concept of it is there's a central team that looks to what the best configuration for that is. So, I think Proton has so much potential, I just think they need to loosen some of the boundaries for it to work for us, and that's the feedback I've given them directly as well.Corey: One thing that I want to take a step beyond this is, you care about this? More than most do. I mean, people will work with computers, yes. We get paid for that. Then they'll go and give talks about things. You're doing that as well. They'll launch a website occasionally, like, cdkpatterns.com, which you have. And then you just sort of decide to go for the absolute hardest thing in the world, and you're one of four authors of a book on this. Tell me more.Matt: Yeah. So, this is something that there's a few of us have been talking since one of the first CDK Days, where we're friends, so there's AWS Heroes. There's Thorsten Höger, Matt Bonig, Sathyajith Bhat, and myself, came together—it was sometime in the summer last year—and said, “Okay. We want to write a book, but how do we do this?” Because, you know, we weren't authors before this point; we'd never done it before. We weren't even sure if we should go to a publisher, or if we should self-publish.Corey: I argue that no one wants to write a book. They want to have written a book, and every first-time author I've ever spoken to at the end has said, “Why on earth would anyone want to do this a second time?” But people do it.Matt: Yeah. And that's we talked to Alex DeBrie, actually, about his book, the amazing Dynamodb Book. And it was his advice, told us to self-publish. And he gave us his starter template that he used for his book, which took so much of the pain out because all we had to do was then work out how we were going to work together. And I will say, I write quite a lot of stuff in general for people, but writing a book is completely different because once it's out there, it's out there. And if it's wrong, it's wrong. You got to release a new version and be like, “Listen, I got that wrong.” So, it did take quite a lot of effort from the group to pull it together. But now that we have it, I want to—I don't have a printed copy because it's only PDF at the minute, but I want a copy just put here [laugh] in, like, the frame. Because it's… it's what we all want.Corey: Yeah, I want you to do that through almost a traditional publisher, selfishly, because O'Reilly just released the AWS Cookbook, and I had a great review quote on the back talking about the value added. I would love to argue that they use one of mine for The CDK Book—and then of course they would reject it immediately—of, “I don't know why you do all this. Using the console and lying about it is way easier.” But yeah, obviously not the direction you're trying to take the book in. But again, the industry is not quite ready for the lying version of ClickOps.It's really neat to just see how willing you are to—how to frame this?—to give of yourself and your time and what you've done so freely. I sometimes make a joke—that arguably isn't that funny—that, “Oh, AWS Hero. That means that you basically volunteer for a $1.6 trillion company.”But that's not actually what you're doing. What you're doing is having figured out all the sharp edges and hacked your way through the jungle to get to something that is functional, you're a trailblazer. You're trying to save other people who are working with that same thing from difficult experiences on their own, having to all thrash and find our own way. And not everyone is diligent and as willing to continue to persist on these things. Is that a somewhat fair assessment how you see the Hero role?Matt: Yeah. I mean, no two Heroes are the same, from what I've judged, I haven't met every Hero yet because pandemic, so Vegas was the first time [I met most 00:28:12], but from my perspective, I mean, in the past, whatever number of years I've been coding, I've always been doing the same thing. Somebody always has to go out and be the first person to try the thing and work out what the value is, and where it'll work for us more work for us. The only difference with the external and public piece is that last 5%, which it's a very different thing to do, but I personally, I like even having conversations like this where I get to meet people that I've never met before.Corey: You sort of discovered the entire secret of why I have an interview podcast.Matt: [laugh]. Yeah because this is what I get out of it, just getting to meet other people and have new experiences. But I will say there's Heroes out there doing very different things. You've got, like, Hiro—as in Hiro, H-I-R-O—actually started AWS Newbies and she's taught—ah, it's hundreds of thousands of people how to actually just start with AWS, through a course designed for people who weren't coders before. That kind of thing is next-level compared to anything I've ever done because you know, they have actually built a product and just given it away. I think that's amazing.Corey: At some level, building a product and giving it away sounds like, “You know, I want to never be lonely again.” Well, that'll work because you're always going to get support tickets. There's an interesting narrative around how to wind up effectively managing the community, and users, and demands, based on open-source maintainers, that we're all wrestling with as an industry, particularly in the wake of that whole log4j nonsense that we've been tilting at that windmill, and that's going to be with us for a while. One last thing I want to talk about before we wind up calling this an episode is, you are one of the organizers of CDK Day. What is that?Matt: Yeah, so CDK Day, it's a complete community-organized conference. The past two have been worldwide, fully virtual just because of the situation we're in. And I mean, they've been pretty popular. I think we had about 5000 people attended the last one, and the idea is, it's a full day of the community just telling their stories of how they liked or disliked using the CDK. So, it's not a marketing event; it's not a sales event; we actually run the whole event on a budget of exactly $0. But yeah, it's just a day of fun to bring the community together and learn a few things. And, you know, if you leave it thinking CDK is not for you, I'm okay with that as much as if you just make a few friends while you're there.Corey: This is the first time I'd realized that it wasn't a formal AWS event. I almost feel like that's the tagline that you should have under it. It's—because it sounds like the CDK Day, again, like, it's this evangelism pure, “This is why it's great and why you should use it.” But I love conferences that embrace critical views. I built one of the first talks I ever built out that did anything beyond small user groups was “Heresy in the Church of Docker.”Then they asked me to give that at ContainerCon, which was incredibly flattering. And I don't think they made that mistake a second time, but it was great to just be willing to see some group of folks that are deeply invested in the technology, but also very open to hearing criticism. I think that's the difference between someone who is writing a nuanced critique versus someone who's just [pure-on 00:31:18] zealotry. “But the CDK is the answer to every technical problem you've got.” Well, I start to question the wisdom of how applicable it really is, and how objective you are. I've never gotten that vibe from you.Matt: No, and that's the thing. So, I mean, as we've worked out in this conversation, I don't work for AWS, so it's not my product. I mean, if it succeeds or if it fails, it doesn't impact my livelihood. I mean, there are people on the team who would be sad for, but the point is, my end goal is always the same. I want people to be enabled to rapidly deliver their software to help their customers.If that's CDK, perfect, but CDK is not for everyone. I mean, there are other options available in the market. And if, even, ClickOps is the way to go for you, I am happy for you. But if it's a case of we can have a conversation, and I can help you get closer to where you need to be with some other tool, that's where I want to be. I just want to help people.Corey: And if I can do anything to help along that axis, please don't hesitate to let me know. I really want to thank you for taking the time to speak with me and being so generous, not just with your time for this podcast, but all the time you spend helping the rest of us figure out which end is up, as we continue to find that the way we manage environments evolves.Matt: Yeah. And, listen, just thank you for having me on today because I've been reading your tweets for two years, so I'm just starstruck at this moment to even be talking to you. So, thank you.Corey: No, no. I understand that, but don't worry, I put my pants on two legs at a time, just like everyone else. That's right, the thought leader on Twitter, you have to jump into your pants. That's the rule. Thanks again so much. I look forward to having a further conversation with you about this stuff as I continue to explore, well honestly, what feels like a brand new paradigm for how we manage code.Matt: Yeah. Reach out if you need any help.Corey: I certainly will. You'll regret asking. Matt [Coulter 00:33:06], Technical Architect at Liberty Mutual. I'm Cloud Economist Corey Quinn and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice, write an angry comment, then click the submit button, but lie and say you hit the submit button via an API call.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.

Screaming in the Cloud
GCP's Many Profundities with Miles Ward

Screaming in the Cloud

Play Episode Listen Later Jan 11, 2022 42:06


About MilesAs Chief Technology Officer at SADA, Miles Ward leads SADA's cloud strategy and solutions capabilities. His remit includes delivering next-generation solutions to challenges in big data and analytics, application migration, infrastructure automation, and cost optimization; reinforcing our engineering culture; and engaging with customers on their most complex and ambitious plans around Google Cloud.Previously, Miles served as Director and Global Lead for Solutions at Google Cloud. He founded the Google Cloud's Solutions Architecture practice, launched hundreds of solutions, built Style-Detection and Hummus AI APIs, built CloudHero, designed the pricing and TCO calculators, and helped thousands of customers like Twitter who migrated the world's largest Hadoop cluster to public cloud and Audi USA who re-platformed to k8s before it was out of alpha, and helped Banco Itau design the intercloud architecture for the bank of the future.Before Google, Miles helped build the AWS Solutions Architecture team. He wrote the first AWS Well-Architected framework, proposed Trusted Advisor and the Snowmobile, invented GameDay, worked as a core part of the Obama for America 2012 “tech” team, helped NASA stream the Curiosity Mars Rover landing, and rebooted Skype in a pinch.Earning his Bachelor of Science in Rhetoric and Media Studies from Willamette University, Miles is a three-time technology startup entrepreneur who also plays a mean electric sousaphone.Links: SADA.com: https://sada.com Twitter: https://twitter.com/milesward Email: miles@sada.com TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: It seems like there is a new security breach every day. Are you confident that an old SSH key, or a shared admin account, isn't going to come back and bite you? If not, check out Teleport. Teleport is the easiest, most secure way to access all of your infrastructure. The open source Teleport Access Plane consolidates everything you need for secure access to your Linux and Windows servers—and I assure you there is no third option there. Kubernetes clusters, databases, and internal applications like AWS Management Console, Yankins, GitLab, Grafana, Jupyter Notebooks, and more. Teleport's unique approach is not only more secure, it also improves developer productivity. To learn more visit: goteleport.com. And not, that is not me telling you to go away, it is: goteleport.com.Corey: This episode is sponsored in part by our friends at Redis, the company behind the incredibly popular open source database that is not the bind DNS server. If you're tired of managing open source Redis on your own, or you're using one of the vanilla cloud caching services, these folks have you covered with the go to manage Redis service for global caching and primary database capabilities; Redis Enterprise. To learn more and deploy not only a cache but a single operational data platform for one Redis experience, visit redis.com/hero. Thats r-e-d-i-s.com/hero. And my thanks to my friends at Redis for sponsoring my ridiculous non-sense.  Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. I am joined today, once again by my friend and yours, Miles Ward, who's the CTO at SADA. However, he is, as I think of him, the closest thing the Google Cloud world has to Corey Quinn. Now, let's be clear, not the music and dancing part that is Forrest Brazeal, but Forrest works at Google Cloud, whereas Miles is a reasonably salty third-party. Miles, thank you for coming back and letting me subject you to that introduction.Miles: Corey, I appreciate that introduction. I am happy to provide substantial salt. It is easy, as I play brass instruments that produce my spit in high volumes. It's the most disgusting part of any possible introduction. For the folks in the audience, I am surrounded by a collection of giant sousaphones, tubas, trombones, baritones, marching baritones, trumpets, and pocket trumpets.So, Forrest threw down the gauntlet and was like, I can play a keyboard, and sing, and look cute at the same time. And so I decided to fail at all three. We put out a new song just a bit ago that's, like, us thanking all of our customers and partners, covering Kool & the Gang “Celebration,” and I neither look good, [laugh] play piano, or smiling, or [capturing 00:01:46] any of the notes; I just play the bass part, it's all I got to do.Corey: So, one thing that I didn't get to talk a lot about because it's not quite in my universe, for one, and for another, it is during the pre re:Invent—pre:Invent, my nonsense thing—run up, which is Google Cloud Next.Miles: Yes.Corey: And my gag a few years ago is that I'm not saying that Google is more interested in what they're building and what they're shipping, but even their conference is called Next. Buh dum, hiss.Miles: [laugh].Corey: So, I didn't really get to spend a lot of attention on the Google Cloud releases that came out this year, but given that SADA is in fact the, I believe, largest Google Cloud partner on the internet, and thus the world—Miles: [unintelligible 00:02:27] new year, three years in a row back, baby.Corey: Fantastic. I assume someone's watch got stuck or something. But good work. So, you have that bias in the way that I have a bias, which is your business is focused around Google Cloud the way that mine is focused on AWS, but neither of us is particularly beholden to that given company. I mean, you do have the not getting fired as partner, but that's a bit of a heavy lift; I don't think I can mouth off well enough to get you there.So, we have a position of relative independence. So, you were tracking Google Next, the same way that I track re:Invent. Well, not quite the same way I track re:Invent; there are some significant differences. What happened at Cloud Next 2021, that the worst of us should be paying attention to?Miles: Sure. I presented 10% of the material at the first re:Invent. There are 55 sessions; I did six. And so I have been at Cloud events for a really long time and really excited about Google's willingness to dive into demos in a way that I think they have been a little shy about. Kelsey Hightower is the kind of notable deep exception to that. Historically, he's been ready to dive into the, kind of, heavy hands-on piece but—Corey: Wait, those were demos? [Thought 00:03:39] was just playing Tetris on stage for the love of it.Miles: [laugh]. No. And he really codes all that stuff up, him and the whole team.Corey: Oh, absol—I'm sorry. If I ever grow up, I wish to be Kelsey Hightower.Miles: [laugh]. You and me both. So, he had kind of led the charge. We did a couple of fun little demos while I was there, but they've really gotten a lot further into that, and I think are doing a better job of packaging the benefits to not just developers, but also operators and data scientists and the broader roles in the cloud ecosystem from the new features that are being launched. And I think, different than the in-person events where there's 10, 20,000, 40,000 people in the audience paying attention, I think they have to work double-hard to capture attention and get engineers to tune in to what's being launched.But if you squint and look close, there are some, I think, very interesting trends that sit in the back of some of the very first launches in what I think are going to be whole veins of launches from Google over the course of the next several years that we are working really hard to track along with and make sure we're extracting maximum value from for our customers.Corey: So, what was it that they announced that is worth paying attention to? Now, through the cacophony of noise, one announcement that [I want to note 00:04:49] was tied to Next was the announcement that GME group, I believe, is going to be putting their futures exchange core trading systems on Google Cloud. At which point that to me—and I know people are going to yell at me, and I don't even slightly care—that is the last nail in the coffin of the idea that well, Google is going to turn this off in a couple years. Sorry, no. That is not a thing that's going to happen. Worst case, they might just stop investing it as aggressively as they are now, but even that would be just a clown-shoes move that I have a hard time envisioning.Miles: Yeah, you're talking now over a dozen, over ten year, over a billion-dollar commitments. So, you've got to just really, really hate your stock price if you're going to decide to vaporize that much shareholder value, right? I mean, we think that, in Google, stock price is a material fraction of the recognition of the growth trajectory for cloud, which is now basically just third place behind YouTube. And I think you can do the curve math, it's not like it's going to take long.Corey: Right. That requires effectively ejecting Thomas Kurian as the head of Google Cloud and replacing him with the former SVP of Bad Decisions at Yahoo.Miles: [laugh]. Sure. Google has no shyness about continuing to rotate leadership. I was there through three heads of Google Cloud, so I don't expect that Thomas will be the last although I think he may well go down in history as having been the best. The level of rotation to the focuses that I think are most critical, getting enterprise customers happy, successful, committed, building macroscale systems, in systems that are critical to the core of the business on GCP has grown at an incredible rate under his stewardship. So, I think he's doing a great job.Corey: He gets a lot of criticism—often from Googlers—when I wind up getting the real talk from them, which is, “Can you tell me what you really think?” Their answer is, “No,” I'm like, “Okay, next question. Can I go out and buy you eight beers and then”— and it's like, “Yeah.” And the answer that I get pretty commonly is that he's brought too much Oracle into Google. And okay, that sounds like a bad thing because, you know, Oracle, but let's be clear here, but what are you talking about specifically? And what they say distills down to engineers are no longer the end-all be-all of everything that Google Cloud. Engineers don't get to make sales decisions, or marketing decisions, or in some cases, product decisions. And that is not how Google has historically been run, and they don't like the change. I get it, but engineering is not the only hard thing in the world and it's not the only business area that builds value, let's be clear on this. So, I think that the things that they don't like are in fact, what Google absolutely needs.Miles: I think, one, the man is exceptionally intimidating and intentionally just hyper, hyper attentive to his business. So, one of my best employees, Brad [Svee 00:07:44], he worked together with me to lay out what was the book of our whole department, my team of 86 people there. What are we about? What do we do? And like I wanted this as like a memoriam to teach new hires as got brought in. So, this is, like, 38 pages of detail about our process, our hiring method, our promotional approach, all of it. I showed that to my new boss who had come in at the time, and he thought some of the pictures looked good. When we showed it to TK, he read every paragraph. I watched him highlight the paragraphs as he went through, and he read it twice as fast as I can read the thing. I think he does that to everybody's documents, everywhere. So, there's a level of just manual rigor that he's brought to the practice that was certainly not there before that. So, that alone, it can be intimidating for folks, but I think people that are high performance find that very attractive.Corey: Well, from my perspective, he is clearly head and shoulders above Adam Selipsky, and Scott Guthrie—the respective heads of AWS and Azure—for one key reason: He is the only one of those three people who follows me on Twitter. And—Miles: [laugh].Corey: —honestly, that is how I evaluate vendors.Miles: That's the thing. That's the only measure, yep. I've worked on for a long time with Selipsky, and I think that it will be interesting to see whether Adam's approach to capital allocation—where he really, I think, thinks of himself as the manager of thousands of startups, as opposed to a manager of a global business—whether that's a more efficient process for creating value for customers, then, where I think TK is absolutely trying to build a much more unified, much more singular platform. And a bunch of the launches really speak to that, right? So, one of the product announcements that I think is critical is this idea of the global distributed cloud, Google Distributed Cloud.We started with Kubernetes. And then you layer on to that, okay, we'll take care of Kubernetes for you; we call that Anthos. We'll build a bunch of structural controls and features into Anthos to make it so that you can really deal with stuff in a global way. Okay, what does that look like further? How do we get out into edge environments? Out into diverse hardware? How do we partner up with everybody to make sure that, kind of like comparing Apple's approach to Google's approach, you have an Android ecosystem of Kubernetes providers instead of just one place you can buy an outpost. That's generally the idea of GDC. I think that's a spot where you're going to watch Google actually leverage the muscle that it already built in understanding open-source dynamics and understanding collaboration between companies as opposed to feeling like it's got to be built here. We've got to sell it here. It's got to have our brand on it.Corey: I think that there's a stupendous and extreme story that is still unfolding over at Google Cloud. Now, re:Invent this year, they wound up talking all about how what they were rolling out was a focus on improving primitives. And they're right. I love their managed database service that they launched because it didn't exist.Miles: Yeah Werner's slide, “It's primitives, not frameworks.” I was like, I think customers want solutions, not frameworks or primitives. [laugh]. What's your plan?Corey: Yeah. However, I take a different perspective on all of this, which is that is a terrific spin on the big headline launches all missed the re:Invent timeline, and… oops, so now we're just going to talk about these other things instead. And that's great, but then they start talking about industrial IOT, and mainframe migrations, and the idea of private 5G, and running fleets of robots. And it's—Miles: Yeah, that's a cool product.Corey: Which one? I'm sorry, they're all very different things.Miles: Private 5G.Corey: Yeah, if someone someday will explain to me how it differs from Wavelength, but that's neither here nor there. You're right, they're all interesting, but none of them are actually doing the thing that I do, which is build websites, [unintelligible 00:11:31] looking for web services, it kind of says it in the name. And it feels like it's very much broadening into everything, and it's very difficult for me to identify—and if I have trouble that I guarantee you customers do—of, which services are for me and which are very much not? In some cases, the only answer to that is to check the pricing. I thought Kendra, their corporate information search thing was for me, then it's 7500 bucks a month to get started with that thing, and that is, “I can hire an internal corporate librarian to just go and hunt through our Google Drive.” Great.Miles: Yeah.Corey: So, there are—or our Dropbox, or our Slack. We have, like, five different information repositories, and this is how corporate nonsense starts, let me assure you.Miles: Yes. We call that luxury SaaS, you must enjoy your dozens of overlapping bills for, you know, what Workspace gives you as a single flat rate.Corey: Well, we have [unintelligible 00:12:22] a lot of this stuff, too. Google Drive is great, but we use Dropbox for holding anything that touches our customer's billing information, just because I—to be clear, I do not distrust Google, but it also seems a little weird to put the confidential billing information for one of their competitors on there to thing if a customer were to ask about it. So, it's the, like, I don't believe anyone's doing anything nefarious, but let's go ahead and just make sure, in this case.Miles: Go further man. Vimeo runs on GCP. You think YouTube doesn't want to look at Vimeo stats? Like they run everything on GCP, so they have to have arrived at a position of trust somehow. Oh, I know how it's called encryption. You've heard of encryption before? It's the best.Corey: Oh, yes. I love these rumors that crop up every now and again that Amazon is going to start scanning all of its customer content, somehow. It's first, do you have any idea how many compute resources that would take and to if they can actually do that and access something you're storing in there, against their attestations to the contrary, then that's your story because one of them just makes them look bad, the other one utterly destroys their entire business.Miles: Yeah.Corey: I think that that's the one that gets the better clicks. So no, they're not doing that.Miles: No, they're not doing that. Another product launch that I thought was super interesting that describes, let's call it second place—the third place will be the one where we get off into the technical deep end—but there's a whole set of coordinated work they're calling Cortex. So, let's imagine you go to a customer, they say, “I want to understand what's happening with my business.” You go, “Great.” So, you use SAP, right? So, you're a big corporate shop, and that's your infrastructure of choice. There are a bunch of different options at that layer.When you set up SAP, one of the advantages that something like that has is they have, kind of, pre-built configurations for roughly your business, but whatever behaviors SAP doesn't do, right, say, data warehousing, advanced analytics, regression and projection and stuff like that, maybe that's somewhat outside of the core wheelhouse for SAP, you would expect like, oh okay, I'll bolt on BigQuery. I'll build that stuff over there. We'll stream the data between the two. Yeah, I'm off to the races, but the BigQuery side of the house doesn't have this like bitching menu that says, “You're a retailer, and so you probably want to see these 75 KPIs, and you probably want to chew up your SKUs in exactly this way. And here's some presets that make it so that this is operable out of the box.”So, they are doing the three way combination: Consultancies plus ISVs plus Google products, and doing all the pre-work configuration to go out to a customer and go I know what you probably just want. Why don't I just give you the whole thing so that it does the stuff that you want? That I think—if that's the very first one, this little triangle between SAP, and Big Query, and a bunch of consultancies like mine, you have to imagine they go a lot further with that a lot faster, right? I mean, what does that look like when they do it with Epic, when they go do it with Go just generally, when they go do it with Apache? I've heard of that software, right? Like, there's no reason not to bundle up what the obvious choices are for a bunch of these combinations.Corey: The idea of moving up the stack and offering full on solutions, that's what customers actually want. “Well, here's a bunch of things you can do to wind up wiring together to build a solution,” is, “Cool. Then I'm going to go hire a company who's already done that is going to sell it to me at a significant markup because I just don't care.” I pay way more to WP Engine than I would to just run WordPress myself on top of AWS or Google Cloud. In fact, it is on Google Cloud, but okay.Miles: You and me both, man. WP Engine is the best. I—Corey: It's great because—Miles: You're welcome. I designed a bunch of the hosting on the back of that.Corey: Oh, yeah. But it's also the—I—well, it costs a little bit more that way. Yeah, but guess what's not—guess what's more expensive than that bill, is my time spent doing the care and feeding of this stuff. I like giving money to experts and making it their problem.Miles: Yeah. I heard it said best, Lego is an incredible business. I love their product, and you can build almost any toy with it. And they have not displaced all other plastic toy makers.Corey: Right.Miles: Some kids just want to buy a little car. [laugh].Corey: Oh, yeah, you can build anything you want out of Lego bricks, which are great, which absolutely explains why they are a reference AWS customer.Miles: Yeah, they're great. But they didn't beat all other toy companies worldwide, and eliminate the rest of that market because they had the better primitive, right? These other solutions are just as valuable, just as interesting, tend to have much bigger markets. Lego is not the largest toy manufacturer in the world. They are not in the top five of toy manufacturers in the world, right?Like, so chasing that thread, and getting all the way down into the spots where I think many of the cloud providers on their own, internally, had been very uncomfortable. Like, you got to go all the way to building this stuff that they need for that division, inside of that company, in that geo, in that industry? That's maybe, like, a little too far afield. I think Google has a natural advantage in its more partner-oriented approach to create these combinations that lower the cost to them and to customers to getting out of that solution quick.Corey: So, getting into the weeds of Google Next, I suppose, rather than a whole bunch of things that don't seem to apply to anyone except the four or five companies that really could use it, what things did Google release that make the lives of people building, you know, web apps better?Miles: This is the one. So, I'm at Amazon, hanging out as a part of the team that built up the infrastructure for the Obama campaign in 2012, and there are a bunch of Googlers there, and we are fighting with databases. We are fighting so hard, in fact, with RDS that I think we are the only ones that [Raju 00:17:51] has ever allowed to SSH into our RDS instances to screw with them.Corey: Until now, with the advent of RDS Custom, meaning that you can actually get in as root; where that hell that lands between RDS and EC2 is ridiculous. I just know that RDS can now run containers.Miles: Yeah. I know how many things we did in there that were good for us, and how many things we did in there that were bad for us. And I have to imagine, this is not a feature that they really ought to let everybody have, myself included. But I will say that what all of the Googlers that I talk to, you know, at the first blush, were I'm the evil Amazon guy in to, sort of, distract them and make them build a system that, you know, was very reliable and ended up winning an election was that they had a better database, and they had Spanner, and they didn't understand why this whole thing wasn't sitting on Spanner. So, we looked, and I read the white paper, and then I got all drooly, and I was like, yes, that is a much better database than everybody else's database, and I don't understand why everybody else isn't on it. Oh, there's that one reason, but you've heard of it: No other software works with it, anywhere in the world, right? It's utterly proprietary to Google. Yes, they were kind—Corey: Oh, you want to migrate it off somewhere else, or a fraction of it? Great. Step one, redo your data architecture.Miles: Yeah, take all of my software everywhere, rewrite every bit of it. And, oh all those commercial applications? Yeah, forget all those, you got, too. Right? It was very much where Google was eight years ago. So, for me, it was immensely meaningful to see the launch at Next where they described what they are building—and have now built; we have alpha access to it—a Postgres layer for Spanner.Corey: Is that effectively you have to treat it as Postgres at all times, or is it multimodal access?Miles: You can get in and tickle it like Spanner, if you want to tickle it like Spanner. And in reality, Spanner is ANSI SQL compliant; you're still writing SQL, you just don't have to talk to it like a REST endpoint, or a GRPC endpoint, or something; you can, you know, have like a—Corey: So, similar to Azure's Cosmos DB, on some level, except for the part where you can apparently look at other customers' data in that thing?Miles: [laugh]. Exactly. Yeah, you will not have a sweeping discovery of incredible security violations in the structure Spanner, in that it is the control system that Google uses to place every ad, and so it does not suck. You can't put a trillion-dollar business on top of a database and not have it be safe. That's kind of a thing.Corey: The thing that I find is the most interesting area of tech right now is there's been this rise of distributed databases. Yugabyte—or You-ji-byte—Pla-netScale—or PlanetScale, depending on how you pronounce these things.Miles: [laugh]. Yeah, why, why is G such an adversarial consonant? I don't understand why we've all gotten to this place.Corey: Oh, yeah. But at the same time, it's—so you take a look at all these—and they all are speaking Postgres; it is pretty clear that ‘Postgres-squeal' is the thing that is taking over the world as far as databases go. If I were building something from scratch that used—Miles: For folks in the back, that's PostgreSQL, for the rest of us, it's okay, it's going to be, all right.Corey: Same difference. But yeah, it's the thing that is eating the world. Although recently, I've got to say, MongoDB is absolutely stepping up in a bunch of really interesting ways.Miles: I mean, I think the 4.0 release, I'm the guy who wrote the MongoDB on AWS Best Practices white paper, and I would grab a lot of customer's and—Corey: They have to change it since then of, step one: Do not use DocumentDB; if you want to use Mongo, use Mongo.Miles: Yeah, that's right. No, there were a lot of customers I was on the phone with where Mongo had summarily vaporized their data, and I think they have made huge strides in structural reliability over the course of—you know, especially this 4.0 launch, but the last couple of years, for sure.Corey: And with all the people they've been hiring from AWS, it's one of those, “Well, we'll look at this now who's losing important things from production?”Miles: [laugh]. Right? So, maybe there's only actually five humans who know how to do operations, and we just sort of keep moving around these different companies.Corey: That's sort of my assumption on these things. But Postgres, for those who are not looking to depart from the relational model, is eating the world. And—Miles: There's this, like, basic emotional thing. My buddy Martin, who set up MySQL, and took it public, and then promptly got it gobbled up by the Oracle people, like, there was a bet there that said, hey, there's going to be a real open database, and then squish, like, the man came and got it. And so like, if you're going to be an independent, open-source software developer, I think you're probably not pushing your pull requests to our friends at Oracle, that seems weird. So instead, I think Postgres has gobbled up the best minds on that stuff.And it works. It's reliable, it's consistent, and it's functional in all these different, sort of, reapplications and subdivisions, right? I mean, you have to sort of squint real hard, but down there in the guts of Redshift, that's Postgres, right? Like, there's Postgres behind all sorts of stuff. So, as an interface layer, I'm not as interested about how it manages to be successful at bossing around hardware and getting people the zeros and ones that they ask for back in a timely manner.I'm interested in it as a compatibility standard, right? If I have software that says, “I need to have Postgres under here and then it all will work,” that creates this layer of interop that a bunch of other products can use. So, folks like PlanetScale, and Yugabyte can say, “No, no, no, it's cool. We talk Postgres; that'll make it so your application works right. You can bring a SQL alchemy and plug it into this, or whatever your interface layer looks like.”That's the spot where, if I can trade what is a fairly limited global distribution, global transactional management on literally ridiculously unlimited scalability and zero operations, I can handle the hard parts of running a database over to somebody else, but I get my layer, and my software talks to it, I think that's a huge step.Corey: This episode is sponsored in part by my friends at Cloud Academy. Something special just for you folks. If you missed their offer on Black Friday or Cyber Monday or whatever day of the week doing sales it is—good news! They've opened up their Black Friday promotion for a very limited time. Same deal, $100 off a yearly plan, $249 a year for the highest quality cloud and tech skills content. Nobody else can get this because they have a assured me this not going to last for much longer. Go to CloudAcademy.com, hit the "start free trial" button on the homepage, and use the Promo code cloud at checkout. That's c-l-o-u-d, like loud, what I am, with a “C” in front of it. It's a free trial, so you'll get 7 days to try it out to make sure it's really a good fit for you, nothing to lose except your ignorance about cloud. My thanks again for sponsoring my ridiculous nonsense.Corey: I think that there's a strong movement toward building out on something like this. If it works, just because—well, I'm not multiregion today, but I can easily see a world in which I'd want to be. So, great. How do you approach the decision between—once this comes out of alpha; let's be clear. Let's turn this into something that actually ships, and no, Google that does not mean slapping a beta label on it for five years is the answer here; you actually have to stand behind this thing—but once it goes GA—Miles: GA is a good thing.Corey: Yeah. How do you decide between using that, or PlanetScale? Or Yugabyte?Miles: Or Cockroach or or SingleStore, right? I mean, there's a zillion of them that sit in this market. I think the core of the decision making for me is in every team you're looking at what skills do you bring to bear and what problem that you're off to go solve for customers? Do the nuances of these products make it easier to solve? So, I think there are some products that the nature of what you're building isn't all that dependent on one part of the application talking to another one, or an event happening someplace else mattering to an event over here. But some applications, that's, like, utterly critical, like, totally, totally necessary.So, we worked with a bunch of like Forex exchange trading desks that literally turn off 12 hours out of the day because they can only keep it consistent in one geographical location right near the main exchanges in New York. So, that's a place where I go, “Would you like to trade all day?” And they go, “Yes, but I can't because databases.” So, “Awesome. Let's call the folks on the Spanner side. They can solve that problem.”I go, “Would you like to trade all day and rewrite all your software?” And they go, “No.” And I go, “Oh, okay. What about trade all day, but not rewrite all your software?” There we go. Now, we've got a solution to that kind of problem.So like, we built this crazy game, like, totally other end of the ecosystem with the Dragon Ball Z people, hysterical; your like—you literally play like Rock, Paper, Scissors with your phone, and if you get a rock, I throw a fireball, and you get a paper, then I throw a punch, and we figure out who wins. But they can play these games like Europe versus Japan, thousands of people on each side, real-time, and it works.Corey: So, let's be clear, I have lobbied a consistent criticism at Google for a while now, which is the Google Cloud global control plane. So, you wind up with things like global service outages from time to time, you wind up with this thing is now broken for everyone everywhere. And that, for a lot of these use cases, is a problem. And I said that AWS's approach to regional isolation is the right way to do it. And I do stand by that assessment, except for the part where it turns out there's a lot of control plane stuff that winds up single tracking through us-east-1, as we learned in the great us-east-1 outage of 2021.Miles: Yeah, when I see customers move from data center to AWS, what they expect is a higher count of outages that lasts less time. That's the trade off, right? There's going to be more weird spurious stuff, and maybe—maybe—if they're lucky, that outage will be over there at some other region they're not using. I see almost exactly the same promise happening to folks that come from AWS—and in particular from Azure—over onto GCP, which is, there will be probably a higher frequency of outages at a per product level, right? So, like sometimes, like, some weird product takes a screw sideways, where there is structural interdependence between quite a few products—we actually published a whole internal structural map of like, you know, it turns out that Cloud SQL runs on top of GCE not on GKE, so you can expect if GKE goes sideways, Cloud SQL is probably not going to go sideways; the two aren't dependent on each other.Corey: You take the status page and Amazon FreeRTOS in a region is having an outage today or something like that. You're like, “Oh, no. That's terrible. First, let me go look up what the hell that is.” And I'm not using it? Absolutely not. Great. As hyperscalers, well, hyperscale, they're always things that are broken in different ways, in different locations, and if you had a truly accurate status page, it would all be red all the time, or varying shades of red, which is not helpful. So, I understand the challenge there, but very often, it's a partition that is you are not exposed to, or the way that you've architected things, ideally, means it doesn't really matter. And that is a good thing. So, raw outage counts don't solve that. I also maintain that if I were to run in a single region of AWS or even a single AZ, in all likelihood, I will have a significantly better uptime across the board than I would if I ran it myself. Because—Miles: Oh, for sure.Corey: —it is—Miles: For sure they're way better at ops than you are. Me, right?Corey: Of course.Miles: Right? Like, ridiculous.Corey: And they got that way, by learning. Like, I think in 2022, it is unlikely that there's going to be an outage in an AWS availability zone by someone tripping over a power cable, whereas I have actually done that. So, there's a—to be clear in a data center, not an AWS facility; that would not have flown. So, there is the better idea of of going in that direction. But the things like Route 53 is control plane single-tracking through the us-east-1, if you can't make DNS changes in an outage scenario, you may as well not have a DR plan, for most use cases.Miles: To be really clear, it was a part of the internal documentation on the AWS side that we would share with customers to be absolutely explicit with them. It's not just that there are mistakes and accidents which we try to limit to AZs, but no, go further, that we may intentionally cause outages to AZs if that's what allows us to keep broader service health higher, right? They are not just a blast radius because you, oops, pulled the pin on the grenade; they can actually intentionally step on the off button. And that's different than the way Google operates. They think of each of the AZs, and each of the regions, and the global system as an always-on, all the time environment, and they do not have systems where one gets, sort of, sacrificed for the benefit of the rest, right, or they will intentionally plan to take a system offline.There is no planned downtime in the SLA, where the SLAs from my friends at Amazon and Azure are explicit to, if they choose to, they decide to take it offline, they can. Now, that's—I don't know, I kind of want the contract that has the other thing where you don't get that.Corey: I don't know what the right answer is for a lot of these things. I think multi-cloud is dumb. I think that the idea of having this workload that you're going to seamlessly deploy to two providers in case of an outage, well guess what? The orchestration between those two providers is going to cause you more outages than you would take just sticking on one. And in most cases, unless you are able to have complete duplication of not just functionality but capacity between those two, congratulations, you've now just doubled your number of single points of failure, you made the problem actively worse and more expensive. Good job.Miles: I wrote an article about this, and I think it's important to differentiate between dumb and terrifyingly shockingly expensive, right? So, I have a bunch of customers who I would characterize as rich, as like, shockingly rich, as producing businesses that have 80-plus percent gross margins. And for them, the costs associated with this stuff are utterly rational, and they take on that work, and they are seeing benefits, or they wouldn't be doing it.Corey: Of course.Miles: So, I think their trajectory in technology—you know, this is a quote from a Google engineer—it's just like, “Oh, you want to see what the future looks like? Hang out with rich people.” I went into houses when I was a little kid that had whole-home automation. I couldn't afford them; my mom was cleaning house there, but now my house, I can use my phone to turn on the lights. Like—Corey: You know, unless us-east-1 is having a problem.Miles: Hey, and then no Roomba for you, right? Like utterly offline. So—Corey: Roomba has now failed to room.Miles: Conveniently, my lights are Philips Hue, and that's on Google, so that baby works. But it is definitely a spot where the barrier of entry and the level of complexity required is going down over time. And it is definitely a horrible choice for 99% of the companies that are out there right now. But next year, it'll be 98. And the year after that, it'll probably be 97. [laugh].And if I go inside of Amazon's data centers, there's not one manufacturer of hard drives, there's a bunch. So, that got so easy that now, of course you use more than one; you got to do—that's just like, sort of, a natural thing, right? These technologies, it'll move over time. We just aren't there yet for the vast, vast majority of workloads.Corey: I hope that in the future, this stuff becomes easier, but data transfer fees are going to continue to be a concern—Miles: Just—[makes explosion noise]—Corey: Oh, man—Miles: —like, right in the face.Corey: —especially with the Cambrian explosion of data because the data science folks have successfully convinced the entire industry that there's value in those mode balancer logs in 2012. Okay, great. We're never deleting anything again, but now you've got to replicate all of that stuff because no one has a decent handle on lifecycle management and won't for the foreseeable future. Great, to multiple providers so that you can work on these things? Like, that is incredibly expensive.Miles: Yeah. Cool tech, from this announcement at Next that I think is very applicable, and recognized the level of like, utter technical mastery—and security mastery to our earlier conversation—that something like this requires, the product is called BigQuery Omni, what Omni allows you to do is go into the Google Cloud Console, go to BigQuery, say I want to do analysis on this data that's in S3, or in Azure Blob Storage, Google will spin up an account on your behalf on Amazon and Azure, and run the compute there for you, bring the result back. So, just transfer the answers, not the raw data that you just scanned, and no work on your part, no management, no crapola. So, there's like—that's multi-cloud. If I've got—I can do a join between a bunch of rows that are in real BigQuery over on GCP side and rows that are over there in S3. The cross-eyedness of getting something like that to work is mind blowing.Corey: To give this a little more context, just because it gets difficult to reason about these things, I can either have data that is in a private subnet in AWS that traverses their horribly priced Managed NAT Gateways, and then goes out to the internet and sent there once, for the same cost as I could take that same data and store it in S3 in their standard tier for just shy of six full months. That's a little imbalanced, if we're being direct here. And then when you add in things like intelligent tiering and archive access classes, that becomes something that… there's no contest there. It's, if we're talking about things that are now approaching exabyte scale, that's one of those, “Yeah, do you want us to pay by a credit card?”—get serious. You can't at that scale anyway—“Invoice billing, or do we just, like, drive a dump truck full of gold bricks and drop them off in Seattle?”Miles: Sure. Same trajectory, on the multi-cloud thing. So, like a partner of ours, PacketFabric, you know, if you're a big, big company, you go out and you call Amazon and you buy 100 gigabit interconnect on—I think they call theirs Direct Connect, and then you hook that up to the Google one that's called Dedicated Interconnect. And voila, the price goes from twelve cents a gig down to two cents a gig; everybody's much happier. But Jesus, you pay the upfront for that, you got to set the thing up, it takes days to get deployed, and now you're culpable for the whole pipe if you don't use it up. Like, there are charges that are static over the course of the month.So, PacketFabric just buys one of those and lets you rent a slice of it you need. And I think they've got an incredible product. We're working with them on a whole bunch of different projects. But I also expect—like, there's no reason the cloud providers shouldn't be working hard to vend that kind of solution over time. If a hundred gigabit is where it is now, what does it look like when I get to ten gigabit? When I get to one gigabit? When I get to half gigabit? You know, utility price that for us so that we get to rational pricing.I think there's a bunch of baked-in business and cost logic that is a part of the pricing system, where egress is the source of all of the funding at Amazon for internal networking, right? I don't pay anything for the switches that connect to this machine to that machine, in region. It's not like those things are cheap or free; they have to be there. But the funding for that comes from egress. So, I think you're going to end up seeing a different model where you'll maybe have different approaches to egress pricing, but you'll be paying like an in-system networking fee.And I think folks will be surprised at how big that fee likely is because of the cost of the level of networking infrastructure that the providers deploy, right? I mean, like, I don't know, if you've gone and tried to buy a 40 port, 40 gig switch anytime recently. It's not like they're those little, you know, blue Netgear ones for 90 bucks.Corey: Exactly. It becomes this, [sigh] I don't know, I keep thinking that's not the right answer, but part of it also is like, well, you know, for things that I really need local and don't want to worry about if the internet's melting today, I kind of just want to get, like, some kind of Raspberry Pi shoved under my desk for some reason.Miles: Yeah. I think there is a lot where as more and more businesses bet bigger and bigger slices of the farm on this kind of thing, I think it's Jassy's line that you're, you know, the fat in the margin in your business is my opportunity. Like, there's a whole ecosystem of partners and competitors that are hunting all of those opportunities. I think that pressure can only be good for customers.Corey: Miles, thank you for taking the time to speak with me. If people want to learn more about you, what you're up to, your bad opinions, your ridiculous company, et cetera—Miles: [laugh].Corey: —where can they find you?Miles: Well, it's really easy to spell: SADA.com, S-A-D-A dot com. I'm Miles Ward, it's @milesward on Twitter; you don't have to do too hard of a math. It's miles@sada.com, if you want to send me an email. It's real straightforward. So, eager to reach out, happy to help. We've got a bunch of engineers that like helping people move from Amazon to GCP. So, let us know.Corey: Excellent. And we will, of course, put links to this in the [show notes 00:37:17] because that's how we roll.Miles: Yay.Corey: Thanks so much for being so generous with your time, and I look forward to seeing what comes out next year from these various cloud companies.Miles: Oh, I know some of them already, and they're good. Oh, they're super good.Corey: This is why I don't do predictions because like, the stuff that I know about, like, for example, I was I was aware of the Graviton 3 was coming—Miles: Sure.Corey: —and it turns out that if your—guess what's going to come up and you don't name Graviton 3, it's like, “Are you simple? Did you not see that one coming?” It's like—or if I don't know it's coming and I make that guess—which is not the hardest thing in the world—someone would think I knew and leaked. There's no benefit to doing predictions.Miles: No. It's very tough, very happy to do predictions in private, for customers. [laugh].Corey: Absolutely. Thanks again for your time. I appreciate it.Miles: Cheers.Corey: Myles Ward, CTO at SADA. I'm Cloud Economist Corey Quinn, and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice and be very angry in your opinion when you write that obnoxious comment, but then it's going to get lost because it's using MySQL instead of Postgres.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.

Screaming in the Cloud
An Enterprise Level View of Cloud Architecture with Levi McCormick

Screaming in the Cloud

Play Episode Listen Later Jan 6, 2022 33:52


About LeviLevi's passion lies in helping others learn to cloud better.Links: Jamf: https://www.jamf.com Twitter: https://twitter.com/levi_mccormick TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: It seems like there is a new security breach every day. Are you confident that an old SSH key, or a shared admin account, isn't going to come back and bite you? If not, check out Teleport. Teleport is the easiest, most secure way to access all of your infrastructure. The open-source Teleport Access Plane consolidates everything you need for secure access to your Linux and Windows servers, and I assure you there is no third option there. Kubernetes clusters, databases, and internal applications like AWS Management Console, Yankins, GitLab, Grafana, Jupyter Notebooks, and more. Teleport's unique approach is not only more secure, it also improves developer productivity. To learn more visit: goteleport.com. And not, that is not me telling you to go away, it is: goteleport.com.Corey: This episode is sponsored in part by our friends at Rising Cloud, which I hadn't heard of before, but they're doing something vaguely interesting here. They are using AI, which is usually where my eyes glaze over and I lose attention, but they're using it to help developers be more efficient by reducing repetitive tasks. So, the idea being that you can run stateless things without having to worry about scaling, placement, et cetera, and the rest. They claim significant cost savings, and they're able to wind up taking what you're running as it is in AWS with no changes, and run it inside of their data centers that span multiple regions. I'm somewhat skeptical, but their customers seem to really like them, so that's one of those areas where I really have a hard time being too snarky about it because when you solve a customer's problem and they get out there in public and say, “We're solving a problem,” it's very hard to snark about that. Multus Medical, Construx.ai and Stax have seen significant results by using them. And it's worth exploring. So, if you're looking for a smarter, faster, cheaper alternative to EC2, Lambda, or batch, consider checking them out. Visit risingcloud.com/benefits. That's risingcloud.com/benefits, and be sure to tell them that I said you because watching people wince when you mention my name is one of the guilty pleasures of listening to this podcast.Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. I am known-slash-renowned-slash-reviled for my creative pronunciations of various technologies, company names, et cetera. Kubernetes, for example, and other things that get people angry on the internet. The nice thing about today's guest is that he works at a company where there is no possible way for me to make it more ridiculous than it sounds because Levi McCormick is a cloud architect at Jamf. I know Jamf sounds like I'm trying to pronounce letters that are designed to be silent, but no, no, it's four letters: J-A-M-F. Jamf. Levi, thanks for joining me.Levi: Thanks for having me. I'm super excited.Corey: Exactly. Also professional advice for anyone listening: Making fun of company names is hilarious; making fun of people's names makes you a jerk. Try and remember that. People sometimes blur that distinction.So, very high level, you're a cloud architect. Now, I remember the days of enterprise architects where their IDEs were basically whiteboards, and it was a whole bunch of people sitting in a room. They call it an ivory tower, but I've been in those rooms; I assure you there is nothing elevated about this. It's usually a dank sub-basement somewhere. What do you do, exactly?Levi: Well, I am part of the enterprise architecture team at Jamf. My roles include looking at our use of cloud; making sure that we're using our resources to the greatest efficacy possible; coordinating between many teams, many products, many architectures; trying to make sure that we're using best practices; bringing them from the teams that develop them and learn them, socializing them to other teams; and just trying to keep a handle on this wild ride that we're on.Corey: So, what I find fun is that Jamf has been around for a long time. I believe it is not your first name. I want to say Casper was originally?Levi: I believe so, yeah.Corey: We're Jamf customers. You're not sponsoring this episode or anything, to the best of my knowledge. So, this is not something I'm trying to shill the company, but we're a customer; we use you to basically ensure that all of our company MacBooks, and laptops, et cetera, et cetera, are basically ensured that there's disk encryption turned on, that people have a password, and that screensaver is turned on, basically to mean that if someone gets their laptop stolen, it's a, “Oh, I have to spend more money with Apple,” and not, “Time to sound the data breach alarm,” for reasons that should be blindingly obvious. And it's great not just at the box check, but also fixing the real problem of I [laugh] don't want to lose data that is sensitive for obvious reasons. I always thought of this is sort of a thing that worked on the laptops. Why do you have a cloud team?Levi: Many reasons. First of all, we started in the business of providing the software that customers would run in their own data centers, in their own locations. Sometime in about 2015, we decided that we are properly equipped to run this better than other people, and we started to provide that as a service. People would move in, migrate their services into the cloud, or we would bring people into the cloud to start with.Device management isn't the only thing that we do. We provide some SSO-type services, we recently acquired a company called Wandera, which does endpoint security and a VPN-like experience for traffic. So, there's a lot of cloud powering all of those things.Corey: Are you able to disclose whether you're focusing mostly on AWS, on Azure, on Google Cloud, or are you pretending a cloud with something like IBM?Levi: All of the above, I believe.Corey: Excellent. That tells you it's a real enterprise, in seriousness. It's the—we talk about the idea of going all in on one providers being a general best practice of good place to start. I believe that. And then there are exceptions, and as companies grow and accumulate technical debt, that also is load-bearing and generates money, you wind up with this weird architectural series of anti-patterns, and when you draw it on a whiteboard of, “Here's our architecture,” the junior consultant comes in and says, “What moron built this?” Usually two said quote-unquote, “Moron,” and then they've just pooched the entire engagement.Yeah, most people don't show up in the morning hoping to do a terrible job today, unless they work at Facebook. So, there are reasons things are the way they are; they're constraints that shape these things. Yeah, if people were going to be able to shut down the company for two years and rebuild everything from scratch from the ground up, it would look wildly different. But you can't do that most of the time.Levi: Yeah. Those things are load bearing, right? You can't just stop traffic one day, and re-architect it with the golden image of what it should have been. We've gone through a series of acquisitions, and those architectures are disparate across the different acquired products. So, you have to be able to leverage lessons from all of them, bring them together and try and just slowly, incrementally march towards a better future state.Corey: As we take a look at the challenges we see The Duckbill Group over on my side of the world, where we talk to customers, it's I think it is surprising to folks to learn that cloud economics as I see it is—well, first, cost and architecture the same thing, which inherently makes sense, but there's a lot more psychology that goes into it than math. People often assume I spend most of my time staring into spreadsheets. I assure you that would not go super well. But it has to do with the psychological elements of what it is that people are wrestling with, of their understanding of the environment has not kept pace with reality, and APIs tend to, you know, tell truths.It's always interesting to me to see the lies that customers tell, not intentionally, but the reality of it of, “Okay, what about those big instances you're running in Australia?” “Oh, we don't have any instances in Australia.” “Look, I understand that you are saying that in good faith, however…” and now we're in a security incident mode and it becomes a whole different story. People's understanding always trails. What do you spend the bulk of your time doing? Is it building things? Is it talking to people? Is it trying to more or less herd cats in certain directions? What's the day-to-day?Levi: I would say it varies week-to-week. Depends on if we have a new product rolling out. I spend a lot of my time looking at architectural diagrams, reference architectures from AWS. The majority of the work I do is in AWS and that's where my expertise lies. I haven't found it financially incentivized to really branch out into any of the other clouds in terms of expertise, but I spend a lot of my time developing solutions, socializing them, getting them in front of teams, and then educating.We have a wide range of skills internally in terms of what people know or what they've been exposed to. I'd say a lot of engineers want to learn the cloud and they want to get opportunities to work on it, and their day-to-day work may not bring them those opportunities as often as they'd like. So, a good portion of my time is spent educating, guiding, joining people's sprints, joining in their stand-ups, and just kind of talking through, like, how they should approach a problem.Corey: Whenever you work at a big company, you invariably wind up with—well, microservices becomes the right answer, not because of the technical reasons; because of the people reason, the way that you get a whole bunch of people moving in roughly the same direction. You are a large scale company; who owns services in your idealized view of the world? Is it, “Well, I wrote something and it's five o'clock. Off to production with it. Talk to you in two days, if everything—if we still have a company left because I didn't double-check what I just wrote.”Do you think that the people who are building services necessarily should be the ones supporting it? Like, in other words, Amazon's approach of having the software engineers being responsible for the ones running it in production from an ops perspective. Is that the direction you trend towards, or do you tend to be from my side of the world—which is grumpy sysadmin—where people—developers hurl applications into your yard for you to worry about?Levi: I would say, I'm an extremist in the view of supporting the Amazon perspective. I really like you build it, you run it, you own it, you architect it, all of it. I think the other teams in the organization should exist to support and enable those paths. So, if you have platform teams are a really common thing you see hired right now, I think those platforms should be built to enable the company's perspective on operating infrastructure or services, and then those service teams on top of that should be enabled to—and empowered to make the decisions on how they want to build a service, how they want to provide it. Ultimately, the buck should stop with them.You can get into other operational teams, you could have a systems operation team, but I think there should be an explicit contract between a service team, what they build, and what they hand off, you know, you could hand off, like, a tier one level response, you know, you can do playbooks, you could do, you know, minimal alert, response, routing, that kind of stuff with a team, but I think that even that team should have a really strong contract with, like, here's what our team provides, here's how you engage with our team, here's how you will transition services to our team.Corey: The challenge with doing that, in some shops, has been that if you decide to roll out a, you build it, you own it, approach that has not been there since the beginning, you wind up with a lot of pushback from engineers who until now really enjoyed their 5:30 p.m. quitting time, or whenever it was they wound up knocking off work. And they started pushing back, like, “Working out of hours? That's inhumane.” And the DevOps team would be sitting there going, “We're right here. How dare you? Like, what do you think our job is?” And it's a, “Yes, but you're not people.” And then it leads to this whole back and forth acrimonious—we'll charitably call it a debate. How do you drive that philosophy?Levi: It's a challenge. I've seen many teams fracture, fall apart, disperse, if you will, under the transition of going through, like, an extreme service ownership. I think you balance it out with the carrot of you also get to determine your own future, right? You get to determine the programming language you use, you get to determine the underlying technologies that you use. Again, there's a contract: You have to meet this list of security concerns, you need to meet these operational concerns, and how you do that is up to you.Corey: When you take a look across various teams—let's bound this to the industry because I don't necessarily want you to wind up answering tough questions at work the day this episode airs—what do you see the biggest blockers to achieving, I guess, a functional cultural service ownership?Levi: It comes down to people's identity. They've established their own identity, “As I am X,” right? I'm a operations engineer. I'm a developer, I'm an engineer. And getting people to kind of branch out of that really fixed mindset is hard, and that, to me, is the major blocker to people assuming ownership.I've seen people make the transition from, “I'm just an engineer. I just want to write code.” I hate those lines. That frustrates me so much: “I just want to write code.” Transitioning into that, like, ownership of, “I had an idea. I built the platform or the service. It's a huge hit.” Or you know, “Lots of people are using it.” Like, seeing people go through that transformation become empowered, become fulfilled, I think is great.Corey: I didn't really expect to get called out quite like this, but you're absolutely right. I was against the idea, back when I was a sysadmin type because I didn't know how to code. And if you have developers supporting all of the stuff that they've built, then what does that mean for me? It feels like my job is evaporating. I don't know how to write code.Well, then I started learning how to write code incredibly badly. And then wow, it turns out, everyone does this. And here we are. But it's—I don't build applications, for obvious reasons. I'm bad at it, but I found another way to proceed in the wide world that we live in of high technology.But yeah, it was hard because this idea of my sense of identity being tied to the thing that I did, it really was an evolve-or-die dinosaur kind of moment because I started seeing this philosophy across the board. You take a look, even now at modern SRE is, or modern DevOps folks, or modern sysadmins, what they're doing looks a lot less like logging into Linux systems and tinkering on the command line a lot more like running and building distributed applications. Sure, this application that you're rolling out is the one that orchestrates everything there, but you're still running this in the same way the software engineers do, which is, interestingly.Levi: And that doesn't mean a team has to be only software engineers. Your service team can be multiple disciplines. It should be multiple disciplines. I've seen a traditional ops team broken apart, and those individuals distributed into the services that they were chiefly skilled in supporting in the past, as the ops team, as we transitioned those roles from one of the worst on-call rotations I've ever seen—you know, 13 to 14 alerts a night—transitioning those out to those service teams, training them up on the operations, building the playbooks. That was their role. Their role wasn't necessarily to write software, day one.Corey: I quit a job after six weeks because of that style of, I guess, mismanagement. Their approach was that, oh, we're going to have our monitoring system live in AWS because one of our VPs really likes AWS—let's be clear, this was 2008, 2009 era—latency was a little challenging there. And [unintelligible 00:17:04] he really liked Big Brother, which was—not to—now before that became a TV show and at rest, it was a monitoring system—but network latency was always a weird thing in AWS in those days, so instead, he insisted we set up three of them. And whenever—if we just got one page, it was fine. But if we got three, then we had to jump in. And two was always undefined.And they turned this off from I think, 10 p.m. to 6 a.m. every night, just so the person I call could sleep. And I'm looking at this, like, this might be the worst thing I've ever seen in my life. This was before they released the Managed NAT Gateway, so possibly it was.Levi: And then the flood, right, when you would get—Corey: Oh, God this was the days, too—Levi: Yeah.Corey: —when you were—if you weren't careful, you'd set this up to page you on the phone with a text message and great, now it takes time for my cell provider to wind up funneling out the sudden onslaught of 4000 text messages. No thanks.Levi: If your monitoring system doesn't have the ability to say, you know, the alert flood, funnel them into one alert, or just pause all alerts, while—because we know there's an incident; you know, us-east-1 is down, right? We know this; we don't need to get 500 text messages to each engineer that's on call.Corey: Well, my philosophy at that point was no, I'm going to instead take a step beyond. If I'm not empowered to fix this thing that is waking me up—and sometimes that's the monitoring system, and sometimes it's the underlying application—I'm not on call.Levi: Yes, exactly. And that's why I like the model of extre—you know, the service ownership: Because those alerts should go to the people—the pain should be felt by the people who are empowered to fix it. It should not land anywhere else. Otherwise, that creates misaligned incentives and nothing gets better.Corey: Yeah. But in large distributed systems, very often the person is on call more or less turns into a traffic router.Levi: Right. That's unfair to them.Corey: That's never fun—yeah, that's unfair, and it's not fun, either, and there's no great answer when you've all these different contributory factors.Levi: And how hard is it to keep the team staffed up?Corey: Oh, yeah. It's a, “Hey, you want a really miserable job one week out of every however many there are in the cycle?” Eh, people don't like that.Levi: Exactly.Corey: This episode is sponsored by our friends at Oracle HeatWave, a new high-performance accelerator for the Oracle MySQL Database Service, although I insist on calling it, “My squirrel.” While MySQL has long been the world's most popular open source database, shifting from transacting to analytics required way too much overhead and, you know, work. With HeatWave you can run your OLAP and OLTP—don't ask me to ever say those acronyms again—workloads directly from your MySQL database and eliminate the time consuming data movement and integration work, while also performing 1100X faster than Amazon Aurora, and 2.5X faster than Amazon Redshift, at a third of the cost. My thanks again to Oracle Cloud for sponsoring this ridiculous nonsense.Corey: So, I've been tracking what you're up to for little while now—you're always a blast to talk with—what is this whole Cloud Builder thing that you were talking about for a bit, and then I haven't seen much about it.Levi: Ah, so at the beginning of the pandemic, our mutual friend, Forrest Brazeal, released the Cloud Resume Challenge. I looked at that, and I thought, this is a fantastic idea. I've seen lots of people going through it. I recommend the people I mentor go through it. Great way to pick up a couple cloud skills here and there, tell an interesting story in an interview, right? It's a great prep.I intended the Cloud Builder Challenge to be a natural kind of progression from that Resume Challenge to the Builder Challenge where you get operational experience. Again, back to that, kind of, extreme service ownership mentality, here's a project where you can build, really modeled on the Amazon GameDays from re:Invent, you build a service, we'll send you traffic, you process those payloads, do some matching, some sorting, some really light processing on these payloads, and then send it back to us, score some points, we'll build a public dashboard, people can high five each other, they can razz each other, kind of competition they want to do. Really low, low pressure, but just a fun way to get more operational experience in an area where there is really no downside. You know, playing like that at work, bad idea, right?Corey: Generally, yes. [crosstalk 00:21:28] production, we used to have one of those environments; oops-a-doozy.Levi: Yeah. I don't see enough opportunities for people to gain that experience in a way that reflects a real workload. You can go out and you can find all kinds of Hello Worlds, you can find all kinds of—like, for front end development, there are tons of activity activities and things you can do to learn the skills, but for the middleware, the back end engineers, there's just not enough playgrounds out there. Now, standing up a Hello World app, you know, you've got your infrastructures code template, you've got your pre-written code, you deploy it, congratulations. But now what, right?And I intended this challenge to be kind of a series of increasingly more difficult waves, if you will, or levels. I really had a whole gamification aspect to it. So, it would get harder, it would get bigger, more traffic, you know, all of those things, to really put people through what it would be like to receive your, “Post got slash-dotted today,” or those kinds of things where people don't get an opportunity to deal with large amounts of traffic, or variable payloads, that kind of stuff.Corey: I love the idea. Where is it?Levi: It is sitting in a bunch of repos, and I am afraid to deploy it. [laugh].Corey: What is it that scares you about it specifically?Levi: The thing that specifically scares me is encouraging early career developers to go out there, deploy this thing, start playing with it, and then incur a huge cloud bill.Corey: Because they failed to secure something or other reasons behind that?Levi: There are many ways that this could happen, yeah. You could accidentally push your access key, secret key up into a public repo. Now, you've got, you know, Bitcoin miners or Monero miners running in your environment. You forget to shut things off, right? That's a really common thing.I went through a SageMaker demo from AWS a couple years ago. Half the room of intelligent, skilled engineers forgot to shut off the SageMaker instances. And everybody ran out of the $25 of credit they had from the demo—Corey: In about ten minutes. Yeah.Levi: In about ten minutes, yeah. And we had to issue all kinds of requests for credits and back and forth. But granted, AWS was accommodating to all of those people, but it was still a lot of stress.Corey: But it was also slow. They're very slow on that, which is fair. Like, if someone's production environment is down, I can see why you care more about that than you do about someone with, “Ah, I did something wrong and lost money.” The counterpoint to that is that for early career folks, that money is everything. We remember earlier this year, that tragic story from the Robinhood customer who committed suicide after getting a notification that he was $730,000 in debt. Turns out it wasn't even accurate; he didn't owe anything when all was said and done.I can see a scenario in which that happens in the AWS world because of their lack of firm price controls on a free tier account. I don't know what the answer on this is. I'm even okay with a, “Cool you will—this is a special kind of account that we will turn you off at above certain levels.” Fine. Even if you hard cap at the 20 or 50 bucks, yeah, it's going to annoy some people, but no one is going to do something truly tragic over that. And I can't believe that Oracle Cloud of all companies is the best shining example of this because you have to affirmatively upgrade your account before they'll charge you a dime. It's the right answer.Levi: It is. And I don't know if you've ever looked at—well, I'm sure you'd have. You've probably looked at the solutions provided by AWS for monitoring costs in your accounts, preventing additional spend. Like, the automation to shut things down, right, it's oftentimes more engineering work to make it so that your systems will shut down automatically when you reach a certain billing threshold than the actual applications that are in place there.Corey: And I don't for the life of me understand why things are the way that they are. But here we go. It's a—[sigh] it just becomes this perpetual strange world. I wish things were better than they are, but they're not.Levi: It makes me terribly sad. I mean, I think AWS is an incredible product, I think the ecosystem is great, and the community is phenomenal; everyone is super supportive, and it makes me really sad to be hesitant to recommend people dive into it on their own dime.Corey: Yeah. And that is a—[sigh] I don't know how you fix that or square that circle. Because I don't want to wind up, I really do not want to wind up, I guess, having to give people all these caveats, and then someone posts about a big bill problem on the internet, and all the comments are, “Oh, you should have set up budgets on that.” Yeah, that's thing still a day behind. So okay, great, instead of having an enormous bill at the end of the month, you just have a really big one two days later.I don't think that's the right answer. I really don't. And I don't know how to fix this, but, you know, I'm not the one here who's a $1.7 trillion company, either, that can probably find a way to fix this. I assure you, the bulk of that money is not coming from a bunch of small accounts that forgot to turn something off or got exploited.Levi: I haven't done my 2021 taxes yet, but I'm pretty sure I'm not there either.Corey: The world in which we live.Levi: [laugh]. I would love this challenge. I would love to put it out there. If I could, on behalf of, you know, early career people who want to learn—if I could issue credits, if I could spin up sandboxes and say, like, “Here's an account, I know you're going to be safe. I have put in a $50 limit.” Right?Corey: Yeah.Levi: “You can't spend more than $50,” like, if I had that control or that power, I would do this in a heartbeat. I'm passionate about getting people these opportunities to play, you know, especially if it's fun, right? If we can make this thing enjoyable, if we can gamify it, we can play around, I think that'd be great. The experience, though, would be a significant amount of engineering on my side, and then a huge amount of outreach, and that to me makes me really sad.Corey: I would love to be able to do something like that myself with a, “Look, if you get a bill, they will waive it, or I will cover it.” But then you wind up with the whole problem of people not operating in good faith as well. Like, “All right, I'm going to mine a bunch of Bitcoin and claim someone else did it.” Or whatnot. And it's just… like, there are problems with doing this, and the whole structure doesn't lend itself to that working super well.Levi: Exactly. I often say, you know, I face a lot of people who want to talk about mining cryptocurrency in the cloud because I'm a cloud architect, right? That's a really common conversation I have with people. And I remind them, like, it's not economical unless you're not paying for it.Corey: Yeah, it's perfectly economical on someone else's account.Levi: Exactly.Corey: I don't know why people do things the way that they do, but here we are. So, re:Invent. What did you find that was interesting, promising there, promising but not there yet, et cetera? What was your takeaway from it? Since you had the good sense not to be there in person?Levi: [laugh]. To me, the biggest letdown was Amplify Studio.Corey: I thought it was just me. Thank you. I just assumed it was something I wasn't getting from the explanation that they gave. Because what I heard was, “You can drag and drop, basically, a front end web app together and then tie it together with APIs on the back end.” Which is exactly what I want, like Retool does; that's what I want only I want it to be native. I don't think it's that.Levi: Right. I want the experience I already have of operating the cloud, knowing the security posture, knowing the way that my users access it, knowing that it's backed by Amazon, and all of their progressively improving services, right? You say it all the time. Your service running on Amazon is better today than it was two years ago. It was better than it was five years ago. I want that experience. But I don't think Amplify Studio delivered.Corey: I wish it had. And maybe it will, in the fullness of time. Again, AWS services do not get worse as they age they get better.Levi: Some gets stale, though.Corey: Yeah. The worst case scenario is they sit there and don't ever improve.Levi: Right. I thought the releases from S3 in terms of, like, the intelligent tiering, were phenomenal. I would love to see everybody turn on intelligent tiering with instant access. Those things to me were showing me that they're thinking about the problem the right way. I think we're missing a story of, like, how do we go from where we're at today—you know, if I've got trillions of objects in storage, how do I transition into that new world where I get the tiering automatically? I'm sure we'll see blog posts about people telling us; that's what the community is great for.Corey: Yeah, they explain these things in a way that the official docs for some reason fail to.Levi: Right. And why don't—Corey: Then again, it's also—I think—I think it's because the people that are building these things are too close to the thing themselves. They don't know what it's like to look at it through fresh eyes.Levi: Exactly. They're often starting from a blank slate, or from a greenfield perspective. There's not enough thought—or maybe there's a lot of thought to it, but there's not enough communication coming out of Amazon, like, here's how you transition. We saw that with Control Tower, we saw that with some of the releases around API Gateway. There's no story for transitioning from existing services to these new offerings. And I would love to see—and maybe Amazon needs a re:Invent Echo, where it's like, okay, here's all the new releases from re:Invent and here's how you apply them to existing infrastructure, existing environments.Corey: So, what's next for you? What are you looking at that's exciting and fun, and something that you want to spend your time chasing?Levi: I spend a lot of my time following AWS releases, looking at the new things coming out. I spend a lot of energy thinking about how do we bring new engineers into the space. I've worked with a lot of operations teams—those people who run playbooks, they hop on machines, they do the old sysadmin work, right—I want to bring those people into the modern world of cloud. I want them to have the skills, the empowerment to know what's available in terms of services and in terms of capabilities, and then start to ask, “Why are we not doing it that way?” Or start looking at making plans for how do we get there.Corey: Levi, I really want to thank you for taking the time to speak with me. If people want to learn more. Where can they find you?Levi: I'm on Twitter. My Twitter handle is @levi_mccormick. Reach out, I'm always willing to help people. I mentor people, I guide people, so if you reach out, I will respond. That's a passion of mine, and I truly love it.Corey: And we'll of course, include a link to that in the [show notes 00:32:28]. Thank you so much for being so generous with your time. I appreciate it.Levi: Thanks, Corey. It's been awesome.Corey: Levi McCormick, cloud architect at Jamf. I'm Cloud Economist Corey Quinn, and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice, along with a comment telling me that service ownership is overrated because you are the storage person, and by God, you will die as that storage person, potentially in poverty.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.

SCRIPTease
046 | Pipedrive – Jakub Kadlubiec, Head of Engineering & Georgy Marchuk, Senior Developer

SCRIPTease

Play Episode Listen Later Jan 4, 2022 59:04


Pipedrive je inovativní sales & CRM platforma, která původně vznikla v Estonsku, ale od roku 2018 má výraznou českou stopu. Obsluhovat 100 000 zákazníků po celém světě, a to včetně Amazonu nebo SpaceX, totiž není jen tak. Mezi 300 developery proto najdeme početný tým, který „jede bomby“ v pražském Karlíně.Unikátní feature Pipedrive spočívá v přesunu celého sales procesu do Kanban Boardu. Kluci a holky z Prahy na to navázali vývojem v oblasti Lead Generation, který pomohl firmu katapultovat do pozice dolarového jednorožce. Pozvání na tenhle SCRIPTease přijali Jakub Kadlubiec, ostřílený matador v programování, který šéfuje vývojářům v Praze a do Pipedrive přelétl ze skotského Skyscanneru, a jeho nepostradatelná pravá ruka Georgy Marchuk, Senior Developer nabytý zkušenostmi po spolupráci s velkými značkami jako jsou Adidas, Avast, Decathlon nebo český FinTech startup Twisto. Hot Tech Stack: Node.js, TypeScript, Go, PHP, React, GraphQL, Relay, Kafka, RabbitMQ, MySQL, KubernetesChcete se dozvědět, jak spravovat 150 integrací, 500 mikroslužeb a nezabít u toho ani jednoho člena týmu, který v Praze pro Pipedrive tvrdě maká?

TSR - The Server Room
Episode 101 - Devart DbForge Studio

TSR - The Server Room

Play Episode Listen Later Jan 1, 2022 32:24


Blog article: https://blog.tsr-podcast.com Summary: dbForge Studio for MySQL is a powerful GUI tool for MySQL and MariaDB for database administration, management, and development. It is a full-fledged IDE that helps create and execute queries, develop and debug stored routines, automate database object management, compare and synchronize databases, analyze table data, and much more. Its rich functionality is delivered under an intuitive interface. It is the perfect tool to transition to from MySQL Workbench for those who are looking for a broader set of features and increased performance. It contains everything you need to work with MySQL databases more effectively. Let me talk about some of its features. For a complete and exhaustive list of features and information please visit: https://www.devart.com/dbforge/mysql/studio/features.html Compare the different Editions: https://www.devart.com/dbforge/mysql/studio/editions.html

Screaming in the Cloud
Spreading the Networking Vibes with Serena (@shenetworks)

Screaming in the Cloud

Play Episode Listen Later Dec 30, 2021 38:43


About Serena Serena is a Network Engineer who specializes in Data Center Compute and Virtualization. She has degrees in Computer Information Systems with a concentration on networking and information security and is currently pursuing a master's in Data Center Systems Engineering. She is most known for her content on TikTok and Twitter as Shenetworks. Serena's content focuses on networking and security for beginners which has included popular videos on bug bounties, switch spoofing, VLAN hoping, and passing the Security+ certification in 24 hours.Links: TikTok: https://www.tiktok.com/@shenetworks Twitter: https://twitter.com/notshenetworks?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Eauthor TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: It seems like there is a new security breach every day. Are you confident that an old SSH key, or a shared admin account, isn't going to come back and bite you? If not, check out Teleport. Teleport is the easiest, most secure way to access all of your infrastructure. The open source Teleport Access Plane consolidates everything you need for secure access to your Linux and Windows servers—and I assure you there is no third option there. Kubernetes clusters, databases, and internal applications like AWS Management Console, Yankins, GitLab, Grafana, Jupyter Notebooks, and more. Teleport's unique approach is not only more secure, it also improves developer productivity. To learn more visit: goteleport.com. And not, that is not me telling you to go away, it is: goteleport.com.Corey: This episode is sponsored in part by our friends at Redis, the company behind the incredibly popular open source database that is not the bind DNS server. If you're tired of managing open source Redis on your own, or you're using one of the vanilla cloud caching services, these folks have you covered with the go to manage Redis service for global caching and primary database capabilities; Redis Enterprise. To learn more and deploy not only a cache but a single operational data platform for one Redis experience, visit redis.com/hero. Thats r-e-d-i-s.com/hero. And my thanks to my friends at Redis for sponsoring my ridiculous non-sense.  Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. Once upon a time, I was a grumpy Unix systems administrator—because it's not like there's a second kind of Unix systems administrator—then I decided it was time to get better at the networking piece, so I got a CCNA one year. Did this make me a competent network engineer? Absolutely not. But it made me a slightly better systems person.My guest today is coming from the other side of the world, specifically someone who is, in fact, good at the networking things. Serena—or @SheNetworks as you might know her from TikTok or @notshenetworks from the Twitters—thank you for joining me, I appreciate your time.Serena: Yeah, thanks for inviting me on.Corey: So, at a very high level, you are a network engineer, and you specialize in data center compute and virtualization, which is fun because I remember doing a lot of that once upon a time before I went basically all in on Cloud consulting, and then sort of forgot that data centers existed. That's still a thing that's still going well, and there are computers out there that don't belong to what are the three biggest tech companies in the world?Serena: Yeah. Shockingly, there's still a ton of data centers out there, still a lot of private hosting, and a lot of the environments that we see are mixed environment; they will have some cloud, some on-prem. But yes, data centers are still relevant. [laugh].Corey: On some level, it feels like once you get into the world of cloud, you don't have to really think about networking anymore. You know, until there's a big outage, and suddenly everyone had think about the networks. But it also feels like it is abstractions piled upon abstractions in the cloud infrastructure space. How much of what happens in data centers these days maps to what happens in these hyperscaler provider environments?Serena: That's a good question. I think—so I have two CCNAs; I'm very familiar with networking, I'm very familiar with virtualization, and I went and got my AWS certification because as we're talking about a lot of cloud things happening now, it's big, it's good to know about it. And underlying infrastructure under the cloud is all the data centers that I work with, all the networking things that I work with. So, it maps very well to me. I thought I had, like, a really easy time studying for my AWS certification because a lot of the concepts just had, like, a different fancy name for AWS versus just what you know, as, like, NAT, or, you know, DNS, different things like that.Corey: Of course, NAT used to be a thing that was—everyone would yell at you, “It's not security,” even though there are—I would argue there are security elements tied into it. But honestly, that feels like one of the best ways to pick fights with people who are way better at this than I am. Nowadays, of course, I just view NAT through a lens of, “Yeah, I totally want to pay an extra four-and-a-half cents per gigabyte passing through a managed NAT gateway,” which remains, of course, my nemesis. The intersection of security, networking, and billing leads to basically just being very angry all the time.Serena: Yeah. You come into the field, like, so ready to go, and then sometimes you do get beat down. But it's worth it, I think. I really like what I do.Corey: And what you do is something of an anomaly because most people who focus on this world of data center networking and the security aspects thereof, and the virtualization stuff, are all—how do I put it politely?—old, grumpy and unpleasant. I mean, I guess I'm not going to put it politely because I'm just going to be honest with it. Because I'm one of those people, let's be clear here. Instead, you are creating a whole bunch of content on Twitter and on TikTok, where I've got to say that the union set in the Venn diagram between TikTok and deep-dive networking and cybersecurity is basically you. How did you get there?Serena: That's a really good question. To your first point, the, you know, old grumpy, kind of, stereotype, those are honestly some of my favorite people, truly, because I don't know what it is, but I just vibe with them in a work environment so well. And it's funny, you know, when I got my first job out of college, I was definitely the youngest person on my team by far. And we would all go out to lunch, I would mess with all of them, we'd all play pranks on each other. Just integrating into the teams was always super easy for me, which I'm really lucky that—not everybody has that experience, especially in their first job; things are a little rough.But it's always great. Like, I love the diversity in tech. And to your second point, how did I end up here, right, with this kind of intersection from this networking world to TikTok? People are always confused. Like, how did that happen? How are you finding followers on TikTok that are interested in networking?And I'm just as shocked honestly. [laugh]. I started making this content this time last year, and… you know, at first I was like, nobody wants to learn about DNS on TikTok. This is where people dance and play pranks and all this stuff.Corey: And if there's dancing when it comes to DNS, at some point, something has gone other hilarious or terrifyingly. That again, I use it as a database, so who am I to talk?Serena: [laugh]. Yeah, but it's been fun. I am shocked. But there's such a wide variety of people now using TikTok and it's growing so quickly. Early on in my TikTok career, I had messages and emails from people who are vice presidents at major Fortune 100 companies asking me, you know, if I'd be interested in working there or, you know, something like that, and I was just—I was so shocked because there was a company that was a Fortune 100, and one of their VPs joined one of my Lives, and was asking me questions, just about, like, my background career, and then they sent me a follow up email [laugh] to be like, “Hey.”So, I was like, “Did I just get interviewed on my Live on TikTok?” And that they always, like, cracked me up. And at that point, I knew I was like, okay, this is something different; like, this is interesting. Because, you know, at the end of the day, you see the views and the numbers and the followers, but you don't have, really, faces to put to them or names, and you don't really know where a lot of these people are from, so you don't know who's seeing it. And a lot of times, I think I made the assumption that they are younger kids. Which is true, but there are also a lot of very seasoned professionals that have been in this field for a very long time that also follow me, and comment on my videos, and add great input and things like that.Corey: There's a giant misunderstanding, I think across the industry, that the executives at the big serious companies, you know, the ones whose mottos may as well be, “That's not funny,” have no personality themselves as people and that they live their entire lives in this corporate bubble where they talk to their kids primarily via I don't know, Microsoft Teams, or WebEx, or something else equally sad. And in practice, that just doesn't work that way. They're human beings, too. And granted, you have to present in certain ways in certain rooms, but the idea that, oh, you're only going to reach developers with attitude problems by having a personality of being on modern platforms. I mean, it's an easy mistake to make.I know this because I spent years making it myself with the nonsense that I do until suddenly people are reaching out and it's, “Huh. You sure did use a lot of high-level strategic terms for a developer.” And you start digging into it, and it's like, “Oh, you're your chief operating officer to giant company. I bet your code is terrible.” Is it? It's like, “Yeah. Turns out, maybe I'm not looking at that through the right lens.” Meeting people where they are with engaging content is important, and I think that a lot of folks completely miss that bus.Serena: Yeah, I agree. And this is a small field, right, so it gets kind of nerve wracking sometimes because sometimes you say things and it's so easy to be like, this is how I joke with my friends. But I'm still somewhat in a professional capacity because of me associating with my career, right? And then when my videos reach a million, half-a-million views, when we think about how many people are actually in this field that would be interested in viewing that content, you realize, oh, wow. Like, this is a huge mixed bag of people, which does include very high level executives, all the way to people that are in high school that are just interested in learning more. So, it's definitely been interesting to figure that out along the way. [laugh]. But yeah, they will have regular personalities. They all like TikTok too. If they don't, they're lying. [laugh].Corey: I used to be very down on the whole TikTok thing, but I started experimenting with it. And yeah, it turns out I have a face for radio and, you know, the social graces for Twitter. So, it's not really my cup of tea, but I enjoy watching it. I found that I'm not really a video person, but something about the TikTok format means I'm just going to start scrolling. And oh, dear, it's been six hours and my phone battery died. Thank God, or I'd still be there. There's something very captivating about it and I really like the format.The problem I always had with looking at a lot of the deeply technical content out there is so many companies are out there producing this and selling this. And that's fine. Like, money is not the end all, be all [of this 00:09:40]. I'm about to spend weeks of my life on something, the fact that it cost me 30 or 50 bucks or whatnot is really not economic thing I should be concerning myself with. But it all feels like it's classroom stuff. It's if you give people an option, are you going to go to a college lecture or are you going to go to a comedy show? Does the idea of, I want to be entertained. If you can teach me something while entertaining me, that feels like the winning combination, and you've absolutely nailed that.Serena: I think a lot of these companies that are producing content, hold themselves back a lot. And that is why they're not successful, right? Because there's so many stipulations, and there's teams of people, and boardrooms of approvals, and all these things, and me, all I'm doing—I record all my TikToks on my iPhone, and I just use in-app editing. I spend a lot of time kind of researching, right, maybe I will experiment with different formats, but the best format that's worked for me is just being authentic, kind of, not having that corporate vibe, right? And also not really expecting anything in return.So, a lot of times, corporations are putting out content because they obviously want to drive traffic to their websites, and different things like that, but the companies that do the best are the ones that are just putting out content for free, and really not necessarily expecting anything in return. And they also give themselves so much more leeway into the type of content that they create because they're not thinking about the numbers at the end of it, right? You just got to put stuff out there and people will see it. For me, I just put stuff out there, I don't need to wait for someone to approve my TikTok for me to push it out and have this content there. So, that is a big difference.And I've learned that through working with sponsors where they'll send you a giant list of talking points they want you to say and I'm like, “You guys know this is a 60-second video, right?” It needs to be really small. You need to, like, really learn how to get the really important stuff out there because the rest of the smaller stuff doesn't matter as much. Like, sell them on one big thing, and that really makes a difference.Corey: Oh, very much so. I see that sometimes with this show where people will reach out and ask about sponsoring, and they'll want to have a URL that I read into the microphone, and it's with UTM tracking parameters and the rest. And it's, like, “I appreciate where you're coming from and your intention here, however, that is not generally how this format works, so let's talk about this and the outcome.” And again, it's a brave new world out there. Yeah, if you're used to buying display ads in various places, that is exactly what you do.For some reason, there's this corporate mentality toward we're going to spend $25 million on a billboard saturation campaign, and not really give any thought about what we're actually going to say now that we have all of that visual real estate to get people's attention with. It's, there's not enough focus on the message itself, and I think that is a giant lost opportunity. Enterprise marketing doesn't have to be boring, it can be a lot of fun.Serena: I agree. And I think podcasting was the last, probably, big area that people budgeted for marketing, right? So, you have your traditional TV commercials and there was YouTube, and—you know, TV commercials, billboards, newspapers, then there's YouTube, and then podcasts, I would say, probably came a little bit later, as far as these companies look at for marketing potential. And now TikTok is so new and a lot of these marketing companies have no idea how to be successful on it because it's just so different. It's Gen Z, the humor is different.It's kind of like [laugh] the wild west on social media where things are just, like, crazy, and you have to fight the algorithm because on TikTok it's, if you don't like it, you just scroll within three seconds. The attention span is so short. So, you really have to capture people's attention within those first three seconds. Versus a podcast, you have the whole, let's say, first 20 minutes to get people, kind of, interested before you can be like, oh, hey, and here's my sponsor. So, it's very different versus TikTok, they'll just, like, oh, scroll. So, [laugh] you have to get creative and think differently.Corey: Many moons ago, when I was getting my CCNA, I worked at a company where we wound up getting a core switches for the data center, which was at the time, something like 65 grand. Great. And then we rented—because we had configured it in our office—and then a couple of us had to rent a commercial van, which I think ran something like $30,000 itself to transport this thing 20 miles to the data center, and I'm sitting there going, like, “Wow, the switch is worth way more than the van that's sitting within. Also were really shitty movers and that doesn't seem like the best idea for anything.” But I just think they remember that, and it left an impression on me.What I like about cloud with what I do is I can take a credit card and then spend less than $10 on AWS—or theoretically, Azure, or Google Cloud or, you know, $2 million on IBM because oops-a-doozy, but fine—and I wind up coming out the other side of that with having done some interesting disaster stuff. You are teaching people about how this stuff works, but in a data center world, it seems to me that the startup costs of, “Oh, I'm going to buy this random router or switch to wind up doing some demonstration stuff for,” it feels like the startup costs of getting hands on that equipment would be out of reach for an awful lot of people. Am I just completely out of touch with how that world works?Serena: No, you're right, you're one hundred percent, right. It is difficult. So, in college, my undergraduate degree is computer information systems, and they had a Cisco Networking Academy. And so we had old switches, old layer 3 switches, and then we had some routers, and this is all stuff that was EOL, donated equipment, right? And this is going to—Corey: It breaks down you're bidding against very faraway places with no budget on eBay for replacements. Oh, yes.Serena: Yeah, exactly. And it was a lot of IOS stuff, right? And so when I was in college, I had no idea that NX-OS existed, which is the data center Nexus version operating system for their switches and things. And so when I got to my first job and saw NX-OS, I was like, “Oh, crap, [laugh] like, what is this?” Right?Because I honestly didn't even know. I graduated and did not know that existed. And I didn't know a lot of the stuff that I was working on at my first shop existed. And I really had to rely on, kind of, the fundamentals. And they are transferable, right? That's why it's good to kind of get into—like, I know what these routing protocols are. I know, layer 2, I know this cabling, so let me just learn these command differences and things like that.And once you get into a production environment in general, out of a lab, it hits the fan. Like, everything you feel like you've learned is gone almost because there's so many layers and now all of a sudden, you have these firewalls, when before you were just trying to get, like, your routing neighborships to establish [laugh] and you weren't worried about rules on a firewall somewhere. And [crosstalk 00:16:39]—Corey: “Oh, and by the way, in this environment, that link that you're working on goes down, every minute it's down, here is the number of commas in the amount of money that we're losing, and yes, that's a plural.” It's, “Okay, so I guess I'm going to double-check everything I run first.” Yeah, it's that caution that gives people a bit of credence there. [unintelligible 00:16:58] do these things in a, more or less, cowboy style in these environments, at least not for very long. Because you can break individual servers; that's fine, but if you break the network suddenly, you may as well not have the computers.Serena: Yeah. It can be paralyzing, truly. It can be very overwhelming your first networking job. Especially for me, I was just dealing with outages constantly because I worked for a vendor, and I was [laugh] like, I was just scared, you know? Because I would get these cases and it would be a hospital outage.And I'm like, “I just graduated college. Like, what do you want from me?” You know, and back to your original point, it is difficult in a data center space because the equipment's so expensive. So, a lot of people ask, “Do you have a home lab?” And one—there's a couple of reasons I don't really have a significant home lab. One, I move so much.Corey: Oh, and in the spare room basically is always 90 degrees and sounds like a jet engine taking off.Serena: Yeah.Corey: Yeah, it's one of those, I should probably find a different place where I don't live, to have that equipment. Yeah.Serena: Yeah. And I have access, like, remotely to all the lab equipment that I really need. So, I don't personally have one, but a lot of things that I do work with are so expensive, that I'm like, I can't afford to put this data center equipment in my house. That doesn't make any sense.And there is luckily now a lot of virtual labs that you can do. There's some sandboxes by Cisco and other vendors, where you can kind of get a little bit of hands-on experience. A lot of it relates to their certifications. You can rent racks, but that gets pretty pricey, too. So, it is difficult, and sometimes that's why a lot of these jobs, I think I have a lot of people who are looking for entry-level work, and it's hard to get into a specifically a data center space.And aside from racking, stacking, working in a data center—maybe a NOC—if you want to get into the actual,s I'm configuring Nexus switches, I'm configuring, you know, Palo Alto firewalls, it can be difficult because it's hard to get to that point, there's not a clear path.Corey: What is the entry path these days? I entered tech by working on a help desk, and those aren't really the jobs that they once were, in a lot of different ways. So, I've stopped talking to entry-level folks with the position of, “Oh, yeah, this is what you should do because that's what I did.” It turns into, like, “Okay, Boomer. Great job. Tell me a little bit more, though, about what the Great War was like, first.” No, we aren't going to go down that path. It's just I don't know what the entry-level point is for someone who's legitimately interested in these things these days.Serena: Nobody does. It's crazy. And you're right at the, “Okay, Boomer,” thing. See, networking was one of those… things that just got pushed onto people in, just, a general IT department, right? So, that's when everything was like, “Okay, we need to get on the internet, so, you know, hey, you handle some of the computer stuff. It's your job now. Good luck. Figure it out.”And so, people started doing that and they kind of just got pushed into it, and then as the internet grew, as our capabilities grew, then the job became, like, a little bit more specialized. And now we have, you know, dedicated network engineers, we have people running data centers. But that's not necessarily a viable path now for people just because there's so much to it now. There's cloud, there's security risks, there's data center, wireless, pho—I mean, you can be an engineer just for phones, right? So, it's a little bit difficult for, especially, the younger people coming in, and the people that I talk to, and figuring out, well, how do I get to what you're doing?And the way that I did is I went and got a four-year degree and then joined a new college graduate program at a Fortune 100 company. Which is a great path, I highly recommend it to anybody that can do it, but it's also not available for everybody, right, because not everybody has the means to get a four-year education, nor do you necessarily need one to do what I do. So, everybody's kind of has this different path, and it's very confusing for people who are aspiring network engineers, or aspiring cloud engineers, even.Corey: This episode is sponsored by our friends at Oracle HeatWave is a new high-performance accelerator for the Oracle MySQL Database Service. Although I insist on calling it “my squirrel.” While MySQL has long been the worlds most popular open source database, shifting from transacting to analytics required way too much overhead and, ya know, work. With HeatWave you can run your OLTP and OLAP, don't ask me to ever say those acronyms again, workloads directly from your MySQL database and eliminate the time consuming data movement and integration work, while also performing 1100X faster than Amazon Aurora, and 2.5X faster than Amazon Redshift, at a third of the cost. My thanks again to Oracle Cloud for sponsoring this ridiculous nonsense.Corey: The narrative the cloud companies have been pushing for a while—like, and I'm in that space deeply enough that I haven't really thought to go super deep into questioning this—is that well, the future is all cloud, the data center is basically this legacy thing that the tide is slowly eroding, in the fullness of time, because everything will one day be cloud. Do you think that's accurate?Serena: I don't. I really don't think that's accurate. Don't get me wrong, I think that the cloud is here to stay, and a lot of people are going to be using it. And it's going to be—and it currently is a huge part of our lives. Like, as we've seen recently with a few of the AWS outages, when it goes down and goes down hard because everything's so centralized.And people like to think, like, oh, you know, we have all this redundancy, yadda, yadda. That has not protected us so far, [laugh] like, from these major outages, right? And a lot of places that I see—especially when you're looking at public sector—is a hybrid, where you do have data center on-prem and you have cloud. And I think that, personally, is the best way to go. Unless, you know, maybe you're a fast growing startup and AWS or Azure makes a lot of sense to you.And it does. There's great use cases for that, right? But they're—not only aside from the whole cloud shift, there's another shift of, you know, making our data centers eco-friendly, too, and workload optimization. So, maybe the price point that you're looking for, what's going to save your business the most money, is doing that hybrid. So, I'm going to store a lot of my private documents on site, I'm going to have this as a backup disaster recovery, but we're also going to operate in the cloud. I don't think that the data centers as we know them are going to go extinct. [laugh]. I think they will be around.Corey: Well, AWS finally made their Outpost—the smaller ones; read as servers that run AWS services on in your facility—available a year after announcing them. And I looked at it like, oh, wow, these things are 600 bucks a month. Which is not nothing, but certainly something I could afford to wind up exploring and doing some content. But okay, first, it's a three-year commitment. So, that's 20 grand or so. Okay, not ideal, but fine.That would effectively almost double my AWS bill, but that's not the hardest part because, oh, and to get one of these, you have to have enterprise support. And when I pointed this out to some Amazonian friends, their response was, “Well, what's the problem on this?” Yeah, enterprise support starts at $15,000 a month minimum, and that means that people aren't going to pick these up to do proof of concept work. They're going to do it when they already have a significant infrastructure out there, and I think that's leaving an awful lot of money on the table by making people jump through sales hoops, and getting proof of concept credits, and doing all the other stuff for this. It's just ship me a box for a few weeks and let me kick the tires on in my environment and see if it works or doesn't work.Worst case, I'll ship it back to you. Worst, worst case, I lose the thing, and then you charge me whatever it costs to replace this. But it still feels like they are really doing the whole, “Oh, it's only big legacy companies that have on-premises stuff.” I don't like that narrative.Serena: I don't either. And I honestly think it's a bad idea, right, because if you do put all of your eggs in the AWS basket and they have all the power, that's not going to give us a lot of bargaining, right? That's not going to give people a lot of—because they'll know. They know how hard it is to get off of AWS at that point: They know it's costly, it takes manpower, it takes knowledge, right? And I think that it is in people's best interest to kind of have that mixed environment. Just for long-term, I'm just very wary of centralizing everything in one area. I think it's a bad idea. [laugh]. I think that we need to be prepared for ourselves, and that means also relying a little bit on ourselves. We can't just, in my opinion, put everything in the AWS basket. [laugh].Corey: Not very long anyway. It just doesn't seem to work.Serena: Right. And it's a great product.Corey: Oh, it absolutely is, but—Serena: There's so many positive things about using cloud. Because I'm not the type of person that likes to, kind of, talk crap about any vendor. I think everybody has their pros, cons, flaws, whatever. It's really about what works best for your environment, and that's part of being a network engineer or an architect is evaluating your environment and figuring out what is going to be the best for you, right? There's no one size fits all, unfortunately.Corey: Yeah. And AWS is uniformly excellent, let's be very clear. Okay, not—maybe not uniformly. Some services are significantly better than others, but I have an opinion piece in the information—paywalled, unfortunately, but I'm working on i—the general thesis that AWS has gotten too big to fail, in that when it's not—like, first, they are going to have better uptime than you or I will running our own data centers, across the board.They are very good at keeping things up, but when they do go down, it's not just your company or my company anymore having an outage, it is a significant portion of, you know, the global economy, and that is an awful lot of systemic concentrated risk. I'm not suggesting they did anything wrong, as far as how they sold these things—though, some people will want to argue with that—but it's the, “What does this mean?” Are we ready to reckon with that as a society that whenever us-east-1 has a bad day, so does the stock market? Is that something we're really prepared to accept or wrangle with? Or worse than that, there are life-critical services now. Does that mean that we're going to accept there is some number of people who will die when there's an outage of a data center? And that's new territory for me. I have not worked in environments where it was life or death consequential. At least not directly.Serena: Yeah, I have. So, I have definitely worked in those environments, right, and it's very scary, and especially when it's outside of your control. So, if you are relying, or just waiting on AWS to get back up, you don't have the control to get in there and start fixing things yourself, which is my instinct, right? Like, I immediately want to get hands-on. I put my troubleshooting hat on, like, let's figure this out, let me look through logs, let me do this.And you don't have that option with AWS when it's a significant outage that's impacting multiple people, it's not some configuration internally to you, right?And that's scary. It's a scary place to be. And I think that we need to really consider the cascading effects that will happen, which a lot of these outages that are kind of starting to show us, right? And luckily, there hasn't been anything major catastrophic, but we do need to really consider life when we're talking about, you know, hospitals, 911 systems, all of these critical infrastructures that are going to be cloud managed, and out of our control, and centralized.So, you know, you lose one 911 system, okay, well, you can do a backup, right? You may be able to route all your calls to the city over because their 911 systems are up and running. Well, what if there's are out now, too, because you're both hosted on AWS?Corey: Or you're, “Ah, we're going to diversify and we're going to have this other one on a different cloud provider.” That's great, but there's a critical third-party dependency that's right back to the thing you're trying to avoid. And there you go again.Serena: Yep. And that's dependency hell, right? [laugh].Corey: Oh, yeah. And I don't know how we get away from that.Serena: Yeah.Corey: Like, we don't want everyone writing all their own stuff from scratch, like starting with assembly, move up the stack. But here we are.Serena: Right. And it's funny because these AWS outages specifically effects—or cloud outages, right? I feel like I'm picking on them. I'm not trying to—sorry, AWS, but [laugh] don't come for me.But you know, explaining to my mom, why her Ring doorbell is not working and her Roomba stopped working when that outage happened, right, she's like, “Why is this not—it won't connect.” Like, “I don't understand.” She's like, “What's AWS?” And then to tell my mom that the company that she buys her socks from, like, that she goes online and, like, buys on Amazon is the company that also is hosting her Roomba, you know, services, her Ring services, it's so interesting to have those conversations. And a lot of people who aren't in our field don't understand that. They don't understand cloud, they don't understand on-prem versus, you know, hosted by a third-party. So, it's interesting to watch that kind of unfold now because it's very new. It's very new territory.Corey: And one last question before we wind up calling it an episode. It is remarkably clear in talking to you that you are in no way, shape, or form, junior. You are not a beginner. You know exactly how this stuff works in significant depth. Your content that you put out is aimed at beginners. I do something very similar. So, to be very clear, this is not a criticism in the slightest, but I am curious as to why that's the direction you went in.Serena: I think there's a few reasons. Well, I might have this knowledge, right? I still consider myself very junior in my career, very early in my career. There's so many things that I don't know and I recognize that. When you're first starting out, you might have this kind of inflated sense of knowledge where you're like—like, me, I was like, “Oh, yeah. I know all about OSPF and running on IOS and the command line,” until I figured out there was an NX-OS and I'm like, “Oh crap, what else do I not know about?” Right? [laugh].Corey: Oh, by the way, that never goes away. I feel exactly the same way 20 years into my career, now. I still have absolutely no idea what I'm doing. So smile, nod, and get used to it is the only insight I've got there. But please, go on.Serena: And even on Twitter sometimes, I'm reading people's stuff, and I'm like, “How did you get into these obscure protocols and all these things?” And, you know, I just kind of dive deeper into there. But I think the big reason that I create a lot of my content for beginners is because I remember so well how it was at the beginning, learning about subnetting, and that IOS—[laugh]—[unintelligible 00:30:52] learning about subnetting, and all of the different models that we have, right? And I was overwhelmed, and I was stressed out, and it just seems so… just, like, a giant mountain to climb. It seems so daunting in the beginning, for me it did because there's so much, right?And it felt like everybody was so far ahead of me. And I don't want other people to really feel like that. Like, I don't want people to be turned off from networking because they feel like the bar is too high, that we're not letting enough new people enter because we're discouraging them from the beginning by saying, “Oh, well, you're going to have to know all this. And let me throw this certification book at you.” And they're big. Like, my certification books—and these are massive. And this is for one half of the CCNA.Corey: For those who aren't, like, on the video call—it's not being recorded video-wise—she's holding a book that you could use to kill a mid-sized dog by accident if it falls off a table. It looks like a phonebook with a hardcover on it.Serena: Yeah. [laugh]. It's huge, right? And there are thousands of pages, and we just give this to somebody and say, like, “Here you go. Make sure you remember all this.” And this is all new information.Corey: And does it still cover things like EIGRP? Like Cisco's proprietary routing protocols that I've never once seen in the wild?Serena: Yeah. So, sometimes you will have to learn that, and they've changed it recently, too. They update their certification exam. So, you will learn about some legacy protocols because sometimes you do run into them.Corey: Oh, yes. That's when I have the good sense to pay professionals who know what they're doing.Serena: [laugh]. Yeah. Exactly. So yeah, you do run into those sometimes. But it feels so daunting for new people, and I totally recognize that. And by nature of TikTok I, especially when I first start making content, I assume that most of the people on there are going to be people who are younger, who are interested in this career.And as you know, in tech in general, especially networking, security, cloud, there's a massive shortage of people, and how are we solving that, right? And my contribution to helping solve that is by getting people interested. And now I have people that DM me and say, “I passed my [Network+ 00:33:01],” or, “I just took the CCNA,” or, “This has been helping me with my class so much.” And that is like, okay, this is great.Like, that's exactly what I want. I want to help the pipeline, I want to get more people interested and help a diverse group of people get interested in tech and say, “Hey, like, this is, you know, where I came from. And I did it; you can do it; let's do it together,” type situation.Corey: I really want to thank you for being so generous with your time. If people want to learn more, as they absolutely should, where can they find you?Serena: I am on TikTok as @SheNetworks. I am on Twitter as @notshenetworks because somebody else—Corey: That is very confusing.Serena: [laugh]. I know. Well, my initial thing was like, I didn't really use Twitter that much, and I would just like—I kind of used it as, like, a backchannel to my TikTok, right, where I would just, like, “Hey, I'm going to go live,” or do this. And then my Twitter, kind of, got a little out of control [laugh] and out of my hands. And so—Corey: It does that sometimes.Serena: Yeah. I had no idea there would be so much interest. And it surprises me every day. So, it's exciting though. I really love all the people that I've met, and I feel like I fit in, and I've met so many good friends that it's been great. But yeah, so @notshenetworks on Twitter because somebody had shenetworks and it was a joke. And [laugh] so if you want to find me there, you could also find me there.Corey: And we will, of course, put links to that in the [show notes 00:34:20]. Thank you so much for taking the time to speak with me today. I really do appreciate it.Serena: Thank you for having me. This has been great. [laugh].Corey: Serena, also known as @SheNetworks, networking content creator to the stars. I'm cloud economist, Corey Quinn, and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice and then a long, angry, rambling comment about how the network isn't that important that you're then not going to be able to submit because the network isn't working.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.

Percona's HOSS Talks FOSS:  The Open Source Database Podcast
Highlight of Talks 2021 - Open source database, MySQL, Postgres, MongoDB, MariaDB - Part 01 - Special Edition 49

Percona's HOSS Talks FOSS: The Open Source Database Podcast

Play Episode Listen Later Dec 29, 2021 43:08


Special edition! All along 2021 Matt Yonkovit, The Head of Open Source Strategy, sat down with special guests to talk about the open-source community, databases features, trends, and more. In this episode, we have selected some short talks from previous podcasts. More talks are coming for 2022. Thank you for following us!

Screaming in the Cloud
Breaching the Coding Gates with Anil Dash

Screaming in the Cloud

Play Episode Listen Later Dec 29, 2021 39:03


About AnilAnil Dash is the CEO of Glitch, the friendly developer community where coders collaborate to create and share millions of web apps. He is a recognized advocate for more ethical tech through his work as an entrepreneur and writer. He serves as a board member for organizations like the Electronic Frontier Foundation, the leading nonprofit defending digital privacy and expression, Data & Society Research Institute, which researches the cutting edge of tech's impact on society, and The Markup, the nonprofit investigative newsroom that pushes for tech accountability. Dash was an advisor to the Obama White House's Office of Digital Strategy, served for a decade on the board of Stack Overflow, the world's largest community for coders, and today advises key startups and non-profits including the Lower East Side Girls Club, Medium, The Human Utility, DonorsChoose and Project Include.As a writer and artist, Dash has been a contributing editor and monthly columnist for Wired, written for publications like The Atlantic and Businessweek, co-created one of the first implementations of the blockchain technology now known as NFTs, had his works exhibited in the New Museum of Contemporary Art, and collaborated with Hamilton creator Lin-Manuel Miranda on one of the most popular Spotify playlists of 2018. Dash has also been a keynote speaker and guest in a broad range of media ranging from the Obama Foundation Summit to SXSW to Desus and Mero's late-night show.Links: Glitch: https://glitch.com Web.dev: https://web.dev Glitch Twitter: https://twitter.com/glitch Anil Dash Twitter: https://twitter.com/anildash TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: It seems like there is a new security breach every day. Are you confident that an old SSH key, or a shared admin account, isn't going to come back and bite you? If not, check out Teleport. Teleport is the easiest, most secure way to access all of your infrastructure. The open source Teleport Access Plane consolidates everything you need for secure access to your Linux and Windows servers—and I assure you there is no third option there. Kubernetes clusters, databases, and internal applications like AWS Management Console, Yankins, GitLab, Grafana, Jupyter Notebooks, and more. Teleport's unique approach is not only more secure, it also improves developer productivity. To learn more visit: goteleport.com. And not, that is not me telling you to go away, it is: goteleport.com.Corey: It seems like there is a new security breach every day. Are you confident that an old SSH key, or a shared admin account, isn't going to come back and bite you? If not, check out Teleport. Teleport is the easiest, most secure way to access all of your infrastructure. The open source Teleport Access Plane consolidates everything you need for secure access to your Linux and Windows servers—and I assure you there is no third option there. Kubernetes clusters, databases, and internal applications like AWS Management Console, Yankins, GitLab, Grafana, Jupyter Notebooks, and more. Teleport's unique approach is not only more secure, it also improves developer productivity. To learn more visit: goteleport.com. And not, that is not me telling you to go away, it is: goteleport.com.Corey: This episode is sponsored in part by our friends at Redis, the company behind the incredibly popular open source database that is not the bind DNS server. If you're tired of managing open source Redis on your own, or you're using one of the vanilla cloud caching services, these folks have you covered with the go to manage Redis service for global caching and primary database capabilities; Redis Enterprise. To learn more and deploy not only a cache but a single operational data platform for one Redis experience, visit redis.com/hero. Thats r-e-d-i-s.com/hero. And my thanks to my friends at Redis for sponsoring my ridiculous non-sense.  Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. Today's guest is a little bit off the beaten path from the cloud infrastructure types I generally drag, kicking and screaming, onto the show. If we take a look at the ecosystem and where it's going, it's clear that in the future, not everyone who wants to build a business, or a tool, or even an application is going to necessarily spring fully-formed into the world from the forehead of some God, knowing how to code. And oh, “I'm going to go to a boot camp for four months to learn how to do it first,” is increasingly untenable. I don't know if you would call it low-code or not. But that's how it feels. My guest today is Anil Dash, CEO of Glitch. Anil, thank you for joining me.Anil: Thanks so much for having me.Corey: So, let's get the important stuff out of the way first, since I have a long-standing history of mispronouncing the company Twitch as ‘Twetch,' I should probably do the same thing here. So, what is Gletch? And what does it do?Anil: Glitch is, at its simplest, a tool that lets you build a full-stack app in your web browser in about 30 seconds. And, you know, for your community, your audience, it's also this ability to create and deploy code instantly on a full-stack server with no concern for deploy, or DevOps, or provisioning a container, or any of those sort of concerns. And what it is for the users is, honestly, a community. They're like, “I looked at this app that was on Glitch; I thought it was cool; I could do what we call [remixing 00:02:03].” Which is to kind of fork that app, a running app, make a couple edits, and all of a sudden live at a real URL on the web, my app is running with exactly what I built. And that's something that has been—I think, just captured a lot of people's imagination to now where they've built over 12 or 15 million apps on the platform.Corey: You describe it somewhat differently than I would, and given that I tend to assume that people who create and run successful businesses don't generally tend to do it without thought, I'm not quite, I guess, insufferable enough to figure out, “Oh, well, I thought about this for ten seconds, therefore I've solved a business problem that you have been needling at for years.” But when I look at Glitch, I would describe it as something different than the way that you describe it. I would call it a web-based IDE for low-code applications and whatnot, and you never talk about it that way. Everything I can see there describes it talks about friendly creators, and community tied to it. Why is that?Anil: You're not wrong from the conventional technologist's point of view. I—sufficient vintage; I was coding in Visual Basic back in the '90s and if you squint, you can see that influence on Glitch today. And so I don't reject that description, but part of it is about the audience we're speaking to, which is sort of a next generation of creators. And I think importantly, that's not just age, right, but that could be demographic, that can be just sort of culturally, wherever you're at. And what we look at is who's making the most interesting stuff on the internet and in the industry, and they tend to be grounded in broader culture, whether they're on, you know, Instagram, or TikTok, or, you know, whatever kind of influencer, you want to point at—YouTube.And those folks, they think of themselves as creators first and they think of themselves as participating in the community first and then the tool sort of follow. And I think one of the things that's really striking is, if you look at—we'll take YouTube as an example because everyone's pretty familiar with it—they have a YouTube Creator Studio. And it is a very rich and deep tool. It does more than, you know, you would have had iMovie, or Final Cut Pro doing, you know, 10 or 15 years ago, incredibly advanced stuff. And those [unintelligible 00:04:07] use it every day, but nobody goes to YouTube and says, “This is a cloud-based nonlinear editor for video production, and we target cinematographers.” And if they did, they would actually narrow their audience and they would limit what their impact is on the world.And so similarly, I think we look at that for Glitch where the social object, the central thing that people organize around a Glitch is an app, not code. And that's this really kind of deep and profound idea, which is that everybody can understand an app. Everybody has an idea for an app. You know, even the person who's, “Ah, I'm not technical,” or, “I'm not really into technology,” they're like, “But you know what? If I could make an app, I would make this.”And so we think a lot about that creative impulse. And the funny thing is, that is a common thread between somebody that literally just got on the internet for the first time and somebody who has been doing cloud deploys for as long as there's been a cloud to deploy to, or somebody has been coding for decades. No matter who you are, you have that place that is starting from what's the experience I want to build, the app I want to build? And so I think that's where there's that framing. But it's also been really useful, in that if you're trying to make a better IDE in the cloud and a better text editor, and there are multiple trillion-dollar companies that [laugh] are creating products in that category, I don't think you're going to win. On the other hand, if you say, “This is more fun, and cooler, and has a better design, and feels better,” I think we could absolutely win in a walk away compared to trillion-dollar companies trying to be cool.Corey: I think that this is an area that has a few players in it could definitely stand to benefit by having more there. My big fear is not that AWS is going to launch stuff in your space and drive you out of business; I think that is a somewhat naive approach. I'm more concerned that they're going to try to launch something in your space, give it a dumb name, fail that market and appropriately, not understand who it's for and set the entire idea back five years. That is, in some cases, it seems like their modus operandi for an awful lot of new markets.Anil: Yeah, I mean, that's not an uncommon problem in any category that's sort of community driven. So, you know, back in the day, I worked on building blogging tools at the beginning of this, sort of, social media era, and we worried about that a lot. We had built some of the first early tools, Movable Type, and TypePad, and these were what were used to launch, like, Gawker and Huffington Post and all the, sort of, big early sites. And we had been doing it a couple years—and then at that time, major player—AOL came in, and they launched their own AOL blog service, and we were, you know, quaking in our boots. I remember just being kind of like, pit in your stomach, “Oh, my gosh. This is going to devastate the category.”And as it turns out, people were smart, and they have taste, and they can tell. And the domain that we're in is not one that is about raw computing power or raw resources that you can bring to bear so much as it is about can you get people to connect together, collaborate together, and feel like they're in a place where they want to make something and they want to share it with other people? And I mean, we've never done a single bit of advertising for Glitch. There's never been any paid acquisition. There's never done any of those things. And we go up against, broadly in the space, people that have billboards and they buy out all the ads of the airport and, you know, all the other kind of things we see—Corey: And they do the typical enterprise thing where they spend untold millions in acquiring the real estate to advertise on, and then about 50 cents on the message, from the looks of it. It's, wow, you go to all this trouble and expense to get something in front of me, and after all of that to get my attention, you don't have anything interesting to say?Anil: Right.Corey: [crosstalk 00:07:40] inverse of that.Anil: [crosstalk 00:07:41] it doesn't work.Corey: Yeah. Oh, yeah. It's brand awareness. I love that game. Ugh.Anil: I was a CIO, and not once in my life did I ever make a purchasing decision based on who was sponsoring a golf tournament. It never happened, right? Like, I never made a call on a database platform because of a poster that was up at, you know, San Jose Airport. And so I think that's this thing that developers in particular, have really good BS filters, and you can sort of see through.Corey: What I have heard about the airport advertising space—and I but a humble cloud economist; I don't know if this is necessarily accurate or not—but if you have a company like Accenture, for example, that advertises on airport billboards, they don't even bother to list their website. If you go to their website, it turns out that there's no shopping cart function. I cannot add ‘one consulting' to my cart and make a purchase.Anil: “Ten pounds of consult, please.”Corey: Right? I feel like the primary purpose there might very well be that when someone presents to your board and says, “All right, we've had this conversation with Accenture.” The response is not, “Who?” It's a brand awareness play, on some level. That said, you say you don't do a bunch traditional advertising, but honestly, I feel like you advertise—more successfully—than I do at The Duckbill Group, just by virtue of having a personality running the company, in your case.Now, your platform is for the moment, slightly larger than mine, but that's okay,k I have ambition and a tenuous grasp of reality and I'm absolutely going to get there one of these days. But there is something to be said for someone who has a track record of doing interesting things and saying interesting things, pulling a, “This is what I do and this is how I do it.” It almost becomes a personality-led marketing effort to some degree, doesn't it?Anil: I'm a little mindful of that, right, where I think—so a little bit of context and history: Glitch as a company is actually 20 years old. The product is only a few years old, but we were formerly called Fog Creek Software, co-founded by Joel Spolsky who a lot of folks will know from back in the day as Joel on Software blog, was extremely influential. And that company, under leadership of Joel and his co-founder Michael Pryor spun out Stack Overflow, they spun out Trello. He had created, you know, countless products over the years so, like, their technical and business acumen is off the charts.And you know, I was on the board of Stack Overflow from, really, those first days and until just recently when they sold, and you know, you get this insight into not just how do you build a developer community that is incredibly valuable, but also has a place in the ecosystem that is unique and persists over time. And I think that's something that was very, very instructive. And so when it came in to lead Glitch I, we had already been a company with a, sort of, visible founder. Joel was as well known as a programmer as it got in the world?Corey: Oh, yes.Anil: And my public visibility is different, right? I, you know, I was a working coder for many years, but I don't think that's what people see me on social media has. And so I think, I've been very mindful where, like, I'm thrilled to use the platform I have to amplify what was created on a Glitch. But what I note is it's always, “This person made this thing. This person made this app and it had this impact, and it got these results, or made this difference for them.”And that's such a different thing than—I don't ever talk about, “We added syntax highlighting in the IDE and the editor in the browser.” It's just never it right. And I think there are people that—I love that work. I mean, I love having that conversation with our team, but I think that's sort of the difference is my enthusiasm is, like, people are making stuff and it's cool. And that sort of is my lens on the whole world.You know, somebody makes whatever a great song, a great film, like, these are all things that are exciting. And the Glitch community's creations sort of feel that way. And also, we have other visible people on the team. I think of our sort of Head of Community, Jenn Schiffer, who's a very well known developer and her right. And you know, tons of people have read her writing and seen her talks over the years.And she and I talk about this stuff; I think she sort of feels the same way, which is, she's like, “If I were, you know, being hired by some cloud platform to show the latest primitives that they've deployed behind an API,” she's like, “I'd be miserable. Like, I don't want to do that in the world.” And I sort of feel the same way. But if you say, “This person who never imagined they would make an app that would have this kind of impact.” And they're going to, I think of just, like, the last couple of weeks, some of the apps we've seen where people are—it could be [unintelligible 00:11:53]. It could be like, “We made a Slack bot that finally gets this reporting into the right channel [laugh] inside our company, but it was easy enough that I could do it myself without asking somebody to create it even though I'm not technically an engineer.” Like, that's incredible.The other extreme, we have people that are PhDs working on machine learning that are like, “At the end of the day, I don't want to be responsible for managing and deploying. [laugh]. I go home, and so the fact that I can do this in create is really great.” I think that energy, I mean, I feel the same way. I still build stuff all the time, and I think that's something where, like, you can't fake that and also, it's bigger than any one person or one public persona or social media profile, or whatever. I think there's this bigger idea. And I mean, to that point, there are millions of developers on Glitch and they've created well over ten million apps. I am not a humble person, but very clearly, that's not me, you know? [laugh].Corey: I have the same challenge to it's, effectively, I have now a 12 employee company and about that again contractors for various specialized functions, and the common perception, I think, is that mostly I do all the stuff that we talk about in public, and the other 11 folks sort of sit around and clap as I do it. Yeah, that is only four of those people's jobs as it turns out. There are more people doing work here. It's challenging, on some level, to get away from the myth of the founder who is the person who has the grand vision and does all the work and sees all these things.Anil: This industry loves the myth of the great man, or the solo legend, or the person in their bedroom is a genius, the lone genius, and it's a lie. It's a lie every time. And I think one of the things that we can do, especially in the work at Glitch, but I think just in my work overall with my whole career is to dismantle that myth. I think that would be incredibly valuable. It just would do a service for everybody.But I mean, that's why Glitch is the way it is. It's a collaboration platform. Our reference points are, you know, we look at Visual Studio and what have you, but we also look at Google Docs. Why is it that people love to just send a link to somebody and say, “Let's edit this thing together and knock out a, you know, a memo together or whatever.” I think that idea we're going to collaborate together, you know, we saw that—like, I think of Figma, which is a tool that I love. You know, I knew Dylan when he was a teenager and watching him build that company has been so inspiring, not least because design was always supposed to be collaborative.And then you think about we're all collaborating together in design every day. We're all collaborating together and writing in Google Docs—or whatever we use—every day. And then coding is still this kind of single-player game. Maybe at best, you throw something over the wall with a pull request, but for the most part, it doesn't feel like you're in there with somebody. Certainly doesn't feel like you're creating together in the same way that when you're jamming on these other creative tools does. And so I think that's what's been liberating for a lot of people is to feel like it's nice to have company when you're making something.Corey: Periodically, I'll talk to people in the AWS ecosystem who for some reason appear to believe that Jeff Barr builds a lot of these services himself then writes blog posts about them. And it's, Amazon does not break out how many of its 1.2 million or so employees work at AWS, but I'm guessing it's more than five people. So yeah, Jeff probably only wrote a dozen of those services himself; the rest are—Anil: That's right. Yeah.Corey: —done by service teams and the rest. It's easy to condense this stuff and I'm as guilty of it as anyone. To my mind, a big company is one that has 200 people in it. That is not apparently something the world agrees with.Anil: Yeah, it's impossible to fathom an organization of hundreds of thousands or a million-plus people, right? Like, our brains just aren't wired to do it. And I think so we reduce things to any given Jeff, whether that's Barr or Bezos, whoever you want to point to.Corey: At one point, I think they had something like more men named Jeff on their board than they did women, which—Anil: Yeah. Mm-hm.Corey: —all right, cool. They've fixed that and now they have a Dave problem.Anil: Yeah [unintelligible 00:15:37] say that my entire career has been trying to weave out of that dynamic, whether it was a Dave, a Mike, or a Jeff. But I think that broader sort of challenge is this—that is related to the idea of there being this lone genius. And I think if we can sort of say, well, creation always happens in community. It always happens influenced by other things. It is always—I mean, this is why we talk about it in Glitch.When you make an app, you don't start from a blank slate, you start from a working app that's already on the platform and you're remix it. And there was a little bit of a ego resistance by some devs years ago when they first encountered that because [unintelligible 00:16:14] like, “No, no, no, I need a blank page, you know, because I have this brilliant idea that nobody's ever thought of before.” And I'm like, “You know, the odds are you'll probably start from something pretty close to something that's built before.” And that enabler of, “There's nothing new under the sun, and you're probably remixing somebody else's thoughts,” I think that sort of changed the tenor of the community. And I think that's something where like, I just see that across the industry.When people are open, collaborative, like even today, a great example is web browsers. The folks making web browsers at Google, Apple, Mozilla are pretty collaborative. They actually do share ideas together. I mean, I get a window into that because they actually all use Glitch to do test cases on different bugs and stuff for them, but you see, one Glitch project will add in folks from Mozilla and folks from Apple and folks from the Chrome team and Google, and they're like working together and you're, like—you kind of let down the pretense of there being this secret genius that's only in this one organization, this one group of people, and you're able to make something great, and the web is greater than all of them. And the proof, you know, for us is that Glitch is not a new idea. Heroku wanted to do what we're doing, you know, a dozen years ago.Corey: Yeah, everyone wants to build Heroku except the company that acquired Heroku, and here we are. And now it's—I was waiting for the next step and it just seemed like it never happened.Anil: But you know when I talked to those folks, they were like, “Well, we didn't have Docker, and we didn't have containerization, and on the client side, we didn't have modern browsers that could do this kind of editing experience, all this kind of thing.” So, they let their editor go by the wayside and became mostly deploy platform. And—but people forget, for the first year or two Heroku had an in-browser editor, and an IDE and, you know, was constrained by the tech at the time. And I think that's something where I'm like, we look at that history, we look at, also, like I said, these browser manufacturers working together were able to get us to a point where we can make something better.Corey: This episode is sponsored by our friends at Oracle HeatWave is a new high-performance accelerator for the Oracle MySQL Database Service. Although I insist on calling it “my squirrel.” While MySQL has long been the worlds most popular open source database, shifting from transacting to analytics required way too much overhead and, ya know, work. With HeatWave you can run your OLTP and OLAP, don't ask me to ever say those acronyms again, workloads directly from your MySQL database and eliminate the time consuming data movement and integration work, while also performing 1100X faster than Amazon Aurora, and 2.5X faster than Amazon Redshift, at a third of the cost. My thanks again to Oracle Cloud for sponsoring this ridiculous nonsense.Corey: I do have a question for you about the nuts and bolts behind the scenes of Glitch and how it works. If I want to remix something on Glitch, I click the button, a couple seconds later it's there and ready for me to start kicking the tires on, which tells me a few things. One, it is certainly not using CloudFormation to provision it because I didn't have time to go and grab a quick snack and take a six hour nap. So, it apparently is running on computers somewhere. I have it on good authority that this is not just run by people who are very fast at assembling packets by hand. What does the infrastructure look like?Anil: It's on AWS. Our first year-plus of prototyping while we were sort of in beta and early stages of Glitch was getting that time to remix to be acceptable. We still wish it were faster; I mean, that's always the way but, you know, when we started, it was like, yeah, you did sit there for a minute and watch your cursor spin. I mean, what's happening behind the scenes, we're provisioning a new container, standing up a full stack, bringing over the code from the Git repo on the previous project, like, we're doing a lot of work, lift behind the scenes, and we went through every possible permutation of what could make that experience be good enough. So, when we start talking about prototyping, we're at five-plus, almost six years ago when we started building the early versions of what became Glitch, and at that time, we were fairly far along in maturity with Docker, but there was not a clear answer about the use case that we're building for.So, we experimented with Docker Swarm. We went pretty far down that road; we spent a good bit of time there, it failed in ways that were both painful and slow to fix. So, that was great. I don't recommend that. In fairness, we have a very unusual use case, right? So, Glitch now, if you talk about ten million containers on Glitch, no two of those apps are the same and nobody builds an orchestration infrastructure assuming that every single machine is a unique snowflake.Corey: Yeah, massively multi-tenant is not really a thing that people know.Anil: No. And also from a security posture Glitch—if you look at it as a security expert—it is a platform allowing anonymous users to execute arbitrary code at scale. That's what we do. That's our job. And so [laugh], you know, so your threat model is very different. It's very different.I mean, literally, like, you can go to Glitch and build an app, running a full-stack app, without even logging in. And the reason we enable that is because we see kids in classrooms, they're learning to code for the first time, they want to be able to remix a project and they don't even have an email address. And so that was about enabling something different, right? And then, similarly, you know, we explored Kubernetes—because of course you do; it's the default choice here—and some of the optimizations, again, if you go back several years ago, being able to suspend a project and then quickly sort of rehydrate it off disk into a running app was not a common use case, and so it was not optimized. And so we couldn't offer that experience because what we do with Glitch is, if you haven't used an app in five minutes, and you're not a paid member, who put that app to sleep. And that's just a reasonable—Corey: Uh, “Put the app to sleep,” as in toddler, or, “Put the app to sleep,” as an ill puppy.Anil: [laugh]. Hopefully, the former, but when we were at our worst and scaling the ladder. But that is that thing; it's like we had that moment that everybody does, which is that, “Oh, no. This worked.” That was a really scary moment where we started seeing app creation ramping up, and number of edits that people were making in those apps, you know, ramping up, which meant deploys for us ramping up because we automatically deploy as you edit on Glitch. And so, you know, we had that moment where just—well, as a startup, you always hope things go up into the right, and then they do and then you're not sleeping for a long time. And we've been able to get it back under control.Corey: Like, “Oh, no, I'm not succeeding.” Followed immediately by, “Oh, no, I'm succeeding.” And it's a good problem to have.Anil: Exactly. Right, right, right. The only thing worse than failing is succeeding sometimes, in terms of stress levels. And organizationally, you go through so much; technically, you go through so much. You know, we were very fortunate to have such thoughtful technical staff to navigate these things.But it was not obvious, and it was not a sort of this is what you do off the shelf. And our architecture was very different because people had looked at—like, I look at one of our inspirations was CodePen, which is a great platform and the community love them. And their front end developers are, you know, always showing off, “Here's this cool CSS thing I figured out, and it's there.” But for the most part, they're publishing static content, so architecturally, they look almost more like a content management system than an app-running platform. And so we couldn't learn anything from them about our scaling our architecture.We could learn from them on community, and they've been an inspiration there, but I think that's been very, very different. And then, conversely, if we looked at the Herokus of the world, or all those sort of easy deploy, I think Amazon has half a dozen different, like, “This will be easier,” kind of deploy tools. And we looked at those, and they were code-centric not app-centric. And that led to fundamentally different assumptions in user experience and optimization.And so, you know, we had to chart our own path and I think it was really only the last year or so that we were able to sort of turn the corner and have high degree of confidence about, we know what people build on Glitch and we know how to support and scale it. And that unlocked this, sort of, wave of creativity where there are things that people want to create on the internet but it had become too hard to do so. And the canonical example I think I was—those of us are old enough to remember FTPing up a website—Corey: Oh, yes.Anil: —right—to Geocities, or whatever your shared web host was, we remember how easy that was and how much creativity was enabled by that.Corey: Yes, “How easy it was,” quote-unquote, for those of us who spent years trying to figure out passive versus active versus ‘what is going on?' As far as FTP transfers. And it turns out that we found ways to solve for that, mostly, but it became something a bit different and a bit weird. But here we are.Anil: Yeah, there was definitely an adjustment period, but at some point, if you'd made an HTML page in notepad on your computer, and you could, you know, hurl it at a server somewhere, it would kind of run. And when you realize, you look at the coding boot camps, or even just to, like, teach kids to code efforts, and they're like, “Day three. Now, you've gotten VS Code and GitHub configured. We can start to make something.” And you're like, “The whole magic of this thing getting it to light up. You put it in your web browser, you're like, ‘That's me. I made this.'” you know, north star for us was almost, like, you go from zero to hello world in a minute. That's huge.Corey: I started participating one of those boot camps a while back to help. Like, the first thing I changed about the curriculum was, “Yeah, we're not spending time teaching people how to use VI in, at that point, the 2010s.” It was, that was a fun bit of hazing for those of us who were becoming Unix admins and knew that wherever we'd go, we'd find VI on a server, but here in the real world, there are better options for that.Anil: This is rank cruelty.Corey: Yeah, I mean, I still use it because 20 years of muscle memory doesn't go away overnight, but I don't inflict that on others.Anil: Yeah. Well, we saw the contrast. Like, we worked with, there's a group called Mouse here in New York City that creates the computer science curriculum for the public schools in the City of New York. And there's a million kids in public school in New York City, right, and they all go through at least some of this CS education. [unintelligible 00:24:49] saw a lot of work, a lot of folks in the tech community here did. It was fantastic.And yet they were still doing this sort of very conceptual, theoretical. Here's how a professional developer would set up their environment. Quote-unquote, “Professional.” And I'm like, you know what really sparks kids' interests? If you tell them, “You can make a page and it'll be live and you can send it to your friend. And you can do it right now.”And once you've sparked that creative impulse, you can't stop them from doing the rest. And I think what was wild was kids followed down that path. Some of the more advanced kids got to high school and realized they want to experiment with, like, AI and ML, right? And they started playing with TensorFlow. And, you know, there's collaboration features in Glitch where you can do real-time editing and a code with this. And they went in the forum and they were asking questions, that kind of stuff. And the people answering their questions were the TensorFlow team at Google. [laugh]. Right?Corey: I remember those days back when everything seemed smaller and more compact, [unintelligible 00:25:42] but almost felt like a balkanization of community—Anil: Yeah.Corey: —where now it's oh, have you joined that Slack team, and I'm looking at this and my machine is screaming for more RAM. It's, like, well, it has 128 gigs in it. Shouldn't that be enough? Not for Slack.Anil: Not for chat. No, no, no. Chat is demanding.Corey: Oh, yeah, that and Chrome are basically trying to out-ram each other. But if you remember the days of volunteering as network staff on Freenode when you could basically gather everyone for a given project in the entire stack on the same IRC network. And that doesn't happen anymore.Anil: And there's something magic about that, right? It's like now the conversations are closed off in a Slack or Discord or what have you, but to have a sort of open forum where people can talk about this stuff, what's wild about that is, for a beginner, a teenage creator who's learning this stuff, the idea that the people who made the AI, I can talk to, they're alive still, you know what I mean? Like, yeah, they're not even that old. But [laugh]. They think of this is something that's been carved in stone for 100 years.And so it's so inspiring to them. And then conversely, talking to the TensorFlow team, they made these JavaScript examples, like, tensorflow.js was so accessible, you know? And they're like, “This is the most heartwarming thing. Like, we think about all these enterprise use cases or whatever. But like, kids wanting to make stuff, like recognize their friends' photo, and all the vision stuff they're doing around [unintelligible 00:26:54] out there,” like, “We didn't know this is why we do it until we saw this is why we do it.”And that part about connecting the creative impulse from both, like, the most experienced, advanced coders at the most august tech companies that exist, as well as the most rank beginners in public schools, who might not even have a computer at home, saying that's there—if you put those two things together, and both of those are saying, “I'm a coder; I'm able to create; I can make something on the internet, and I can share it with somebody and be inspired by it,” like, that is… that's as good as it gets.Corey: There's something magic in being able to reach out to people who built this stuff. And honestly—you shouldn't feel this way, but you do—when I was talking to the folks who wrote the things I was working on, it really inspires you to ask better questions. Like when I'm talking to Dr. Venema, the author of Postfix and I'm trying to figure out how this thing works, well, I know for a fact that I will not be smarter than he is at basically anything in that entire universe, and maybe most beyond that, as well, however, I still want to ask a question in such a way that doesn't make me sound like a colossal dumbass. So, it really inspires you—Anil: It motivates you.Corey: Oh, yeah. It inspires you to raise your question bar up a bit, of, “I am trying to do x. I expect y to happen. Instead, z is happening as opposed to what I find the documentation that”—oh, as I read the documentation, discover exactly what I messed up, and then I delete the whole email. It's amazing how many of those things you never send because when constructing a question the right way, you can help yourself.Anil: Rubber ducking against your heroes.Corey: Exactly.Anil: I mean, early in my career, I'd gone through sort of licensing mishap on a project that later became open-source, and sort of stepped it in and as you do, and unprompted, I got an advice email from Dan Bricklin, who invented the spreadsheet, he invented VisiCalc, and he had advice and he was right. And it was… it was unreal. I was like, this guy's one of my heroes. I grew up reading about his work, and not only is he, like, a living, breathing person, he's somebody that can have the kindness to reach out and say, “Yeah, you know, have you tried this? This might work.”And it's, this isn't, like, a guy who made an app. This is the guy who made the app for which the phrase killer app was invented, right? And, you know, we've since become friends and I think a lot of his inspiration and his work. And I think it's one of the things it's like, again, if you tell somebody starting out, the people who invented the fundamental tools of the digital era, are still active, still building stuff, still have advice to share, and you can connect with them, it feels like a cheat code. It feels like a superpower, right? It feels like this impossible thing.And I think about like, even for me, the early days of the web, view source, which is still buried in our browser somewhere. And you can see the code that makes the page, it felt like getting away with something. “You mean, I can just look under the hood and see how they made this page and then I can do it too?” I think we forget how radical that is—[unintelligible 00:29:48] radical open-source in general is—and you see it when, like, you talk to young creators. I think—you know, I mean, Glitch obviously is used every day by, like, people at Microsoft and Google and the New York Timesor whatever, like, you know, the most down-the-road, enterprise developers, but I think a lot about the new creators and the people who are learning, and what they tell me a lot is the, like, “Oh, so I made this app, but what do I have to do to put it on the internet?”I'm like, “It already is.” Like, as soon as you create it, that URL was live, it all works. And their, like, “But isn't there, like, an app store I have to ask? Isn't there somebody I have to get permission to publish this from? Doesn't somebody have to approve it?”And you realize they've grown up with whether it was the app stores on their phones, or the cartridges in their Nintendo or, you know, whatever it was, they had always had this constraint on technology. It wasn't something you make; it's something that is given to you, you know, handed down from on high. And I think that's the part that animates me and the whole team, the community, is this idea of, like, I geek out about our infrastructure. I love that we're doing deploys constantly, so fast, all the time, and I love that we've taken the complexity away, but the end of the day, the reason why we do it, is you can have somebody just sort of saying, I didn't realize there was a place I could just make something put it in front of, maybe, millions of people all over the world and I don't have to ask anybody permission and my idea can matter as much as the thing that's made by the trillion-dollar company.Corey: It's really neat to see, I guess, the sense of spirit and soul that arises from a smaller, more, shall we say, soulful company. No disparagement meant toward my friends at AWS and other places. It's just, there's something that you lose when you get to a certain point of scale. Like, I don't ever have to have a meeting internally and discuss things, like, “Well, does this thing that we're toying with doing violate antitrust law?” That is never been on my roadmap of things I have to even give the slightest crap about.Anil: Right, right? You know, “What does the investor relations person at a retirement fund think about the feature that we shipped?” Is not a question that we have to answer. There's this joy in also having community that sort of has come along with us, right? So, we talk a lot internally about, like, how do we make sure Glitch stays weird? And, you know, the community sort of supports that.Like, there's no reason logically that our logo should be the emoji of two fish. But that kind of stuff of just, like, it just is. We don't question it anymore. I think that we're very lucky. But also that we are part of an ecosystem. I also am very grateful where, like… yeah, that folks at Google use Glitch as part of their daily work when they're explaining a new feature in Chrome.Like, if you go to web.dev and their dev portal teaches devs how to code, all the embedded examples go to these Glitch apps that are running, showing running code is incredible. When we see the Stripe team building examples of, like, “Do you want to use this new payment API that we made? Well, we have a Glitch for you.” And literally every day, they ship one that sort of goes and says, “Well, if you just want to use this new Stripe feature, you just remix this thing and it's instantly running on Glitch.”I mean, those things are incredible. So like, I'm very grateful that the biggest companies and most influential companies in the industry have embraced it. So, I don't—yeah, I don't disparage them at all, but I think that ability to connect to the person who'd be like, “I just want to do payments. I've never heard of Stripe.”Corey: Oh yeah.Anil: And we have this every day. They come into Glitch, and they're just like, I just wanted to take credit cards. I didn't know there's a tool to do that.Corey: “I was going to build it myself,” and everyone shrieks, “No, no. Don't do that. My God.” Yeah. Use one of their competitors, fine,k but building it yourself is something a lunatic would do.Anil: Exactly. Right, right. And I think we forget that there's only so much attention people can pay, there's only so much knowledge they have.Corey: Everything we say is new to someone. That's why I always go back to assuming no one's ever heard of me, and explain the basics of what I do and how I do it, periodically. It's, no one has done all the mandatory reading. Who knew?Anil: And it's such a healthy exercise to, right, because I think we always have that kind of beginner's mindset about what Glitch is. And in fairness, I understand why. Like, there have been very experienced developers that have said, “Well, Glitch looks too colorful. It looks like a toy.” And that we made a very intentional choice at masking—like, we're doing the work under the hood.And you can drop down into a terminal and you can do—you can run whatever build script you want. You can do all that stuff on Glitch, but that's not what we put up front and I think that's this philosophy about the role of the technology versus the people in the ecosystem.Corey: I want to thank you for taking so much time out of your day to, I guess, explain what Glitch is and how you view it. If people want to learn more about it, about your opinions, et cetera. Where can they find you?Anil: Sure. glitch.com is easiest place, and hopefully that's a something you can go and a minute later, you'll have a new app that you built that you want to share. And, you know, we're pretty active on all social media, you know, Twitter especially with Glitch: @glitch. I'm on as @anildash.And one of the things I love is I get to talk to folks like you and learn from the community, and as often as not, that's where most of the inspiration comes from is just sort of being out in all the various channels, talking to people. It's wild to be 20-plus years into this and still never get tired of that.Corey: It's why I love this podcast. Every time I talk to someone, I learn something new. It's hard to remain too ignorant after you have enough people who've shared wisdom with you as long as you can retain it.Anil: That's right.Corey: Thank you so much for taking the time to speak with me.Anil: So, glad to be here.Corey: Anil Dash, CEO of Gletch—or Glitch as he insists on calling it. I'm Cloud Economist Corey Quinn and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice along with an angry comment telling me how your small team at AWS is going to crush Glitch into the dirt just as soon as they find a name that's dumb enough for the service.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.

Screaming in the Cloud
President Biden's Advice in Action with Dan Woods

Screaming in the Cloud

Play Episode Listen Later Dec 28, 2021 39:28


About DanDan is CISO and VP of Cybersecurity for Shipt, a Target subsidiary. He worked previously as a Distinguished Engineer on Target's cloud infrastructure. He served as CTO for Joe Biden's 2020 Presidential campaign. Prior to that Dan worked with the Hillary for America tech team through the Groundwork, and contributed as a founding developer on Spinnaker while at Netflix. Dan is an O'Reilly published author and avid public speaker.  Links: Shipt: https://www.shipt.com/ Twitter: https://twitter.com/danveloper LinkedIn: https://www.linkedin.com/in/danveloper TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: It seems like there is a new security breach every day. Are you confident that an old SSH key, or a shared admin account, isn't going to come back and bite you? If not, check out Teleport. Teleport is the easiest, most secure way to access all of your infrastructure. The open source Teleport Access Plane consolidates everything you need for secure access to your Linux and Windows servers—and I assure you there is no third option there. Kubernetes clusters, databases, and internal applications like AWS Management Console, Yankins, GitLab, Grafana, Jupyter Notebooks, and more. Teleport's unique approach is not only more secure, it also improves developer productivity. To learn more visit: goteleport.com. And not, that is not me telling you to go away, it is: goteleport.com.Corey: Writing ad copy to fit into a 30 second slot is hard, but if anyone can do it the folks at Quali can. Just like their Torque infrastructure automation platform can deliver complex application environments anytime, anywhere, in just seconds instead of hours, days or weeks. Visit Qtorque.io today and learn how you can spin up application environments in about the same amount of time it took you to listen to this ad.Corey: Welcome to Screaming in the Cloud, I'm Corey Quinn. Sometimes I talk to people who are involved in working on the nonprofit slash political side of the world. Other times I talk to folks who are deep in the throes of commercial businesses, and I obviously personally spend more of my time on one of those sides of the world than I do the other. But today's guest is a little bit different, Dan Woods is the CISO and VP of Cybersecurity at Shipt, a division of Target where he's worked for a fair number of years, but took some time off for his side project, the side hustle as the kids call it, as the CTO for the Biden campaign. Dan, thank you for joining me.Dan: Yeah. Thank you, Corey. Happy to be here.Corey: So, you have an interesting track record as far as your career goes, you've been at Target for a long time. You were a distinguished engineer—not to be confused with ‘extinguished engineer,' which is just someone who is finally—the fire has gone out. And from there you went from being a distinguished engineer to a VP slash CISO, which generally looks a lot less engineer-like, and a lot more, at least in my experience, of sitting in a whole lot of executive-level meetings, managing teams, et cetera. Was that, in fact, an individual contributor—or IC—move into a management track, or am I just misunderstanding this because these are commonly overloaded terms in our industry?Dan: Yeah, yeah, no, that's exactly right. So, IC to leadership, two distinct tracks, distinct career paths. It was something that I've spent a number of years thinking about and more or less working toward and making sure that it was the right path for me to go. The interesting thing about the break that I took in the middle of Target when I was CTO for the campaign is that that was a leadership role, right. I led the team. I managed the team.I did performance reviews and all of that kind of managerial stuff, but I also sat down and did a lot of tech. So, it was kind of like a mix of being a senior executive, but also still continuing to be a distinguished engineer. So, then the natural path out of that for me was to make a decision about do I continue to be an individual contributor or do I go into a leadership track? And I felt like for a number of reasons that my interests more aligned with being on the leadership side of the world, and so that's how I've ended up where I am.Corey: And correct me if I'm wrong because generally speaking political campaigns are not usually my target customers given the fact that they're turning the entire AWS environment off in a few months—win or lose—and yeah, that is, in fact, remains the best way to save money on your AWS bill; it's hard for me to beat that. But at that point most of the people you're working with are in large part volunteers I would imagine.So, managing in a traditional sense of, “Well, we're going to have your next quarterly review.” Well, your candidate might not be in the race then, and what we're going to put you on a PIP, and what exactly you're going to stop letting me volunteer here? You're going to dock them pay—you're not paying me for this. It becomes an interesting management challenge I would imagine just because the people you're working with are passionate and volunteering, and a lot of traditional management and career advice doesn't necessarily map one-to-one I would have to assume.Dan: That is the best way that I've heard it described yet. I try to explain this to folks sometimes and it's kind of difficult to get that message across that like there is sort of a base level organization that exists, right. There were full-time employees who were a part of the tech team, really great group of folks especially from very early on willing to join the campaign and be a part of what it was that we were doing.And then there was this whole ecosystem of folks who just wanted to volunteer, folks who wanted to be a part of it but didn't want to leave their 9:00 to 5:00 who wanted to come in. One of the most difficult things about—we rely on volunteers very heavily in the political space, and very grateful for all the folks who step up and volunteer with organizations that they feel passionate about. In fact, one of the best little tidbits of wisdom the President imparted to me at one point, we were having dinner at his house very early on in the campaign, and he said, “The greatest gift that you can give somebody is your time.” And I think that's so incredibly true. So, the folks who volunteer, it's really important, really grateful that they're all there.In particular, how it becomes difficult, is that you need somebody to manage the volunteers, right, who are there. You need somebody to come up with work and check in that work is getting done because while it's great that folks want to volunteer five, ten hours a week, or whatever it is that they can put in, we also have very real things that need to get done, and they need to get done in a timely manner.So, we had a lot of difficulty especially early on in the campaign utilizing the volunteers to the extent that we could because we were such a small and scrappy team and because everybody who was working on the campaign at the time had a lot of responsibilities that they needed to see through on their own. And so getting into this, it's quite literally a full-time job having to sit down and follow up with volunteers and make sure that they have the appropriate amount of work and make sure that we've set up our environment appropriately so that volunteers can come and go and all of that kind of stuff, so yeah.Corey: It's always an interesting joy looking at the swath of architectural decisions and how they came to be. I talked on a previous episode with Jackie Singh, who was, I believe, after your tenure as CISO, she was involved on the InfoSec side of things, and she was curious as to your thought process or rationale with a lot of the initial architectural decisions that she talked about on her episode which I'm sure she didn't intend it this way, but I am going to blatantly miscategorize as, “Justify yourself. What were you thinking?” Usually it takes years for that kind of, “I don't understand what's going on here so I'm playing data center archeologist or cloud spelunker.” This was a very short window. How did decisions get made architecturally as far as what you're going to run things on? It's been disclosed that you were on AWS, for example. Was that a hard decision?Dan: No, not at all. Not at all. We started out the campaign—I in particular I was one of the first employees hired onto the campaign and the idea all along was that we're not going to be clever, right? We're basically just going to develop what needs to be developed. And the idea with that was that a lot of the code that we were going to sit down and write or a lot of the infrastructure that we were going to build was going to be glue, it not AWS Glue, right, ideally, but just glue that would bind data streams together, right?So, data movement, vendor A produces a CSV file for you and it needs to end up in a bucket somewhere. So, somebody needs to write the code to make that happen, or you need to find a sufficient vendor who can make that happen. There's a lot more vendors today believe it or not than there were two years ago that are doing much better in that kind of space, but two years ago we had the constraints of time and money.Our idea was that the code that we were going to write was going to be for those purposes. What it actually turned into is that in other areas of the business—and I will call it a business because we had formalized roadmaps and different departments working on different things—but in other areas of the business where we didn't have enough money to purchase a solution, we had the ability to go and write software.The interesting thing about this group of technologists who came together especially early on in the campaign to build out the tech team most of them came from an enterprise software development background, right? So, we had the know-how of how to build things at scale and how to do continuous delivery and continuous deployment, and how to operate a cloud-native environment, and how to build applications for that world.So, we ended up doing things like writing an API for managing our donor vetting pipeline, right? And that turned into a complex system of Lambda functions and continuous delivery for a variety of different services that facilitated that pipeline. We also built an architecture for our mobile app which there were plenty of companies that wanted to sell us a mobile app and we just couldn't afford it so we ended up writing the mobile app ourselves.So, after some point in time, what we said was we actually have a fairly robust and complex software infrastructure. We have a number of microservices that are doing various things to facilitate the operation of the business, and something that we need to do is we need to spend a little bit of time and make sure that we're building this in a cohesive way, right? And what part of that means was that, for example, we had to take a step back and say, “Okay, we need to have a unified identity service.” We can't have a different identity—or we can't have every single individual service creating its own identity. We need to have—Corey: I really wish you could pass that lesson out on some of the AWS service teams.Dan: [laugh]. Yes, I know. I know. Yeah. So, we went through—Corey: So, there were some questionable choices you made in there, like you started that with the beginning of, “Well, we had no time which is fine and no budget. So, we chose AWS.” It's like, “Oh, that looks like the exact opposite direction of a great decision, given, you know, my view on it.” Stepping past that entirely, you are also dealing with challenges that I don't think map very well to things that exist in the corporate world. For example, you said you had to build a donor vetting pipeline.It's in the corporate world I didn't have it. It's one of those, “Why in the world would I get in the way of people trying to give me money?” And the obvious answer in your case is, federal law, and it turns out that the best outcome generally does not involve serving prison time. So, you have to address these things in ways that don't necessarily have a one-to-one analog in other spaces.Dan: That's true. That's true. Yes, correct to the federal law thing. Our more pressing reason to do this kind of thing was that we made a commitment very early on in the campaign that we wouldn't take money from executives of the gas and oil industry, for example. There were another bunch of other commitments that were made, but it was inconceivable for us to have enough people that could possibly go manually through those filings. So, for us to be able to build an automated system for doing that meant that we were literally saving thousands of human hours and still getting a beneficial result out of it.Corey: And everything you do is subject to intense scrutiny by folks who are willing to make hay out of anything. If it had leaked at the time, I would have absolutely done some ridiculous nonsense thing about, “Ah, clearly looking at this AWS bill. Joe Biden's supports managed NAT gateway data processing pricing.” And it's absolutely not, but that doesn't stop people from making hay about this because headlines are going to be headlines.And do you have to also deal with the interesting aspect—industrial espionage is always kind of a thing, but by and large most companies don't have to worry that effectively half of the population is diametrically opposed to the thing it is that they're trying to do to the point where they might very well try to get insiders there to start leaking things out. Everything you do has to be built with optics in mind, working under tight constraints, and it seems like an almost insurmountable challenge except for the fact where you actually pulled it off.Dan: Yeah. Yeah. Yeah. We kept saying that the tech was not the story, right, and we wanted to do everything within our power to keep the conversation on the candidate and not on emails or AWS bills or any of that kind of stuff. And so we were very intentional about a lot of the decisions that we ended up making with the idea that if the optics are bad, we pull away from the primary mission of what it is that we're trying to do.Corey: So, what was it that qualified you to be the CTO of a—at the time very fledgling and uncertain campaign, given that you were coming from a role where you were a distinguished engineer, which is not nothing, let's be clear, but it's an executive-level of role rather than a hands-on level of role as CTO. And then if we go back in time, you were one of the founding developers of Spinnaker over at Netflix.And I have a lot of thoughts about Netflix technology and a lot of thoughts about Spinnaker as well, and none of those thoughts are, “This seems like a reasonable architecture I should roll out for a presidential campaign.” So, please, don't take this as the insult that probably sounds like, but why were you the CTO that got tapped?Dan: Great question. And I think in some ways, right place, right time. But in other ways probably needs to speak a little bit to the journey of how I've gotten anywhere in my career. So, going back to Netflix, yeah, so I worked in Netflix. I had the opportunity to work with a lot of incredibly bright and talented folks there. One of the people in particular who I met there and became friends with was Corey Bertram who worked on the core SRE team.Corey left Netflix to go off and at the time he was just like, “I'm going to go do a political startup.” The interesting thing about Netflix at the time—this was 2013, so, this was just after the Obama for America '12 campaign. And a bunch of folks from OFA world came and worked at Netflix and a variety of other organizations in the Bay Area. Corey was not one of those people but we were very well-connected with folks in that world, and Corey said he was going off to do a political startup, and so after my non-mutual departure from Netflix, I was talking to Corey and he said, “Hey, why don't you come over and help us figure out how to do continuous delivery over on the political startup.” That political startup turned into the groundwork which turned into essentially the tech platform for the Hillary for America campaign.So, I had the opportunity working for the groundwork to work very closely with the folks in the technology organization at HFA. And that got me more exposure to what that world is and more connections into that space. And the groundwork was run by Corey, but was the CEO or head—I don't even know what he called himself, was Michael Slaby, who was President Obama's CTO in 2008 and had a bigger technical role in the 2012 campaign.And so, for his involvement in HFA '16 meant that he was a person who was very well connected for the 2020 campaign. And when we were out at a political conference in late 2018 and he said, “Hey, I think that Vice President Biden is going to run. Do you have any interest in talking with his team?” And I said, “Yes, absolutely. Please introduce me.”And I had a couple of conversations with Greg Schultz who was the campaign manager and we just hit it off. And it was a really great fit. Greg was an excellent leader. He was a real visionary, exactly the person that President Biden needed. And he brought me in to set up the tech operation and get everything to where we ultimately won the primary and won the election after that.Corey: And then, as all things do, it ended and the question then becomes, “Great, what's next?” And the answer for you was apparently, “Okay, I'm going to go back to Target-ish.” Although now you're the CISO of a Target subsidiary, Shipt and Target's relationship is—again, I imagine I have that correct as far as you are in fact a subsidiary of Target, so it wasn't exactly a new company, but rather a transition into the previous organization you were in a different role.Dan: Yeah, correct. Yeah, it's a different department inside of Target, but my paycheck still come from Target. [laugh].Corey: So, what was it that inspired you to go into the CISO role? Because obviously security is everyone's job, which is what everyone says, which is why we get away with treating it like it's nobody's job because shared responsibilities tend to work out that way.Dan: Yeah.Corey: And you've done an awful lot of stuff that was not historically deeply security-centric although there's always an element passing through it. Now, going into a CISO role as someone without a deep InfoSec background that I'm aware of, what drove that? How did that work?Dan: You know, I think the most correct answer is that security has always been in my blood. I think like most people who started out—Corey: There are medications for that now.Dan: Yeah, [laugh] good. I might need them. [laugh]. I think like most folks who are kind of my era who started seriously getting into software development and computer system administration in the late ‘90s, early thousands, cybersecurity it wasn't called cybersecurity at the time. It wasn't even called InfoSec, right, it was just called, I don't know, dabbling or something. But that was a gateway for getting into Linux system administration, network engineering, so forth and so on.And for a short period of time I became—when I was getting my RHCE certification way back in the day, I became pretty entrenched in network security and that was a really big focus area that I spent a lot of time on and I got whatever the supplemental network security certification from Red Hat was at the time. And then I realized pretty quickly that the world isn't going to need box operators for very long, and this was just before the DevOps revolution had really come around and more and more things were automated.So, we were still doing hand deployments. I was still dropping WAR files onto a file system and restarting Apache. That was our deployment process. And I saw the writing on the wall and I said, “If I don't dedicate myself to becoming first and foremost a software engineer, then I'm not going to have a very good time in technology here.” So, I jumped out of that and I got into software development, and so that's where my software engineering career evolved out of.So, when I was CTO for the campaign, I like to tell people that I was a hundred percent of CTO, I was a hundred percent a CIO, and I was a hundred percent of CISO for the first 514 days of the campaign or whatever it was. So, I was 300 percent doing all of the top-level technology jobs for the campaign, but cybersecurity was without a doubt the one that we would drop everything for every single time.And that was by necessity; we were constantly under attack on the campaign. And a lot of my headspace during that period of time was dedicated to how do we make sure that we're doing things in the most secure way? So, when I left—when I came back into Target and I came back in as a distinguished engineer there were some areas that they were hoping that I could contribute positively and help move a couple of things along.The idea always the whole time was going to be for me to jump into a leadership position. And I got a call one day from Rich Agostino who's the CISO for Target and he said, “Hey, Shipt needs a cybersecurity operation built out and you're looking for a leadership role. Would you be interested in doing this?” And believe it or not, I had missed the world of cybersecurity so much that when the opportunity came up I said, “Yes, absolutely. I'll dive in head first.” And so that was the path for getting there.Corey: This episode is sponsored by our friends at Oracle HeatWave is a new high-performance accelerator for the Oracle MySQL Database Service. Although I insist on calling it “my squirrel.” While MySQL has long been the worlds most popular open source database, shifting from transacting to analytics required way too much overhead and, ya know, work. With HeatWave you can run your OLTP and OLAP, don't ask me to ever say those acronyms again, workloads directly from your MySQL database and eliminate the time consuming data movement and integration work, while also performing 1100X faster than Amazon Aurora, and 2.5X faster than Amazon Redshift, at a third of the cost. My thanks again to Oracle Cloud for sponsoring this ridiculous nonsense.Corey: My take to cybersecurity space is, a little, I think, different than most people's journeys through it. The reason I started a Thursday edition of the Last Week in AWS newsletter is the security happenings in the AWS ecosystem for folks who don't have the word security in their job titles because I used to dabble in that space a fair bit. The problem I found is that is as you move up the ladder to executives that our directors, VPs, and CISOs, the language changes significantly.And it almost becomes a dialect of corporate-speak that I find borderline impenetrable, versus the real world terminology we're talking about when, “Okay, let's make sure that we rotate credentials on a reasonable expected basis where it makes sense,” et cetera et cetera. It almost becomes much more of a box-checking compliance exercise slash layering on as much as you possibly can that for plausible deniability for the inevitable breach that one day hits and instead of actually driving towards better outcomes.And I understand that's a cynical, strange perspective, but I started talking to people about this, and I'm very far from alone in that, which is why people are subscribing to that newsletter and that's the corner of the market I wanted to start speaking to. So, given that you've been an engineer practitioner trying to build things and now a security executive as well, is my assessment of the further higher up you go the entire messaging and purpose change, or is that just someone who's been in the trenches for too long and hasn't been on that side of the world, and I have a certain lack of perspective that would make this all very clear. Which I freely accept, if that's the case.Dan: No, I think that you're right for a lot of organizations. I think that that's a hundred percent true, and it is exactly as you described: a box-checking exercise for a lot of organizations. Something that's important to remember about Target is—Target was the subject of a data breach in 2012, and that was before there were data breaches every single day, right.Now, we look at a data breach and we say that's just going to happen, right, that's the cost of doing business. But back in 2012 it was really a very big story and it was a very big deal, and there was quite a bit of activity in the Target technology world after that breach. So, it reshaped the culture quite literally, new executives were brought in, but there's this whole world of folks inside of Target who have never forgotten that, right, and work day-in and day-out to make sure that we don't have another breach.So, security at Target is a main centrally thought about kind of thing. So, it's very much something that is a part of the way that people operate inside of Target. So, coming over to Shipt, obviously, Shipt is—it is a subsidiary. It is a part of Target, but it doesn't have that long history and hasn't had that same kind of experience. The biggest thing that we really needed at Shipt is first and foremost to get the program established, right. So, I'm three or four months onto the job now and we've tripled the team size. I've been—Corey: And you've stayed out of the headlines, which is basically the biggest and most accurate breach indicator I've found so far.Dan: So far so good. Well, but the thing that we want to do though is to be able to bring that same kind of focus of importance that Target has on cybersecurity into the world of engineering at Shipt. And it's not just a compliance game, and it's not just a thing where we're just trying to say that we have it. We're actually trying to make sure that as we go forward we've got all these best practices from an organization that's been through the bad stuff that we can adopt into our day-to-day and kind of get it done.When we talk about it at an executive level, obviously we're not talking about the penetration tests done by the red team the earlier day, right. We're not calling any of that stuff out in particular. But we do try to summarize it in a way that makes it clear that the thing that we're trying to do is build a security-minded culture and not just check some boxes and make sure that we have the appropriate titles in the appropriate places so that our insurance rates go down, right. We're actually trying to keep people safe.Corey: There's a lot to be said for that. With the Target breach back in—I want to say 2012, was it?Dan: 2012. Yep.Corey: Again, it was a wake-up call and the argument that I've always seen is that everyone is vulnerable—just depends on how much work it's going to take to get there. And for, credit where due, there was a complete rotation in the executive levels which whether that's fair or not, I—people have different opinions on it; my belief has always been you own the responsibility, regardless of who's doing the work.And there's no one as fanatical as a convert, on some level, and you've clearly been doing a lot of things in the right direction. The thing that always surprises me is that when I wind up seeing these surveys in the industry that—what is it? 65% of companies say that they would be vulnerable to a breach, and everybody said, “Oh, we should definitely look at those companies.” My argument is, “Hang on a sec. I want to talk to the 35% who say, ‘oh, we're impenetrable.'” because, spoiler, you are not.No one is. Just the question of how heavy is the lift and how much work is it going to take to get there? I do know that mouthing off in public about how perfect the security of anything is, is the best way to more or less climb to the top of a mountain during a thunderstorm, a hold up a giant metal rod, and curse the name of God. It doesn't lead to positive outcomes, basically ever. In turn, this also leads to companies not talking about security openly.I find that in many cases it is easier for me to get people to talk about their AWS bills than their InfoSec posture. And I do believe, incidentally, those two things are not entirely unrelated, but how do you view it? It was surprisingly easy to get Shipt's CISO to have a conversation with me here on this podcast. It is significantly more challenging in most other companies.Dan: Well, in fairness, you've been asking me for about two-and-a-half years pretty regularly [laugh] to come.Corey: And I always say I will stop bothering you if you want. You said, “No, no. Ask me again in a few months. Ask me again, after the election. Ask me again after—I don't know, like, the one-day delivery thing gets sorted out.” Whatever it happens to be. And that's fine. I follow up religiously, and eventually I can wear people down by being polite yet persistent.Dan: So, persistence on you is actually to credit here. No, I think to your question though, I think that there's a good balance. There's a good balance in being open about what it is that you're trying to do versus over-sharing areas that maybe you're less proficient in, right. So, it wouldn't make a lot of sense for me to come on here and tell you the areas that we need to develop into security. But on the other side of things, I am very happy to come in and talk to you about how our incident response plan is evolving, right, and what our plan looks like for doing all of that kind of stuff.Some of the best security practitioners who I've worked with in the world will tell you that you're not going to prevent a breach from a motivated attacker, and your job as CISO is to make sure that your response is appropriate, right, more so than anything. So, our incident response areas where today we're dedicating quite a bit of effort to build up our proficiency, and that's a very important aspect of the cybersecurity program that we're trying to build here.Corey: And unlike the early days of a campaign, you still have to be ultra-conscious about security, but now you have the luxury of actually being able to hire security staff because it turns out that, “Please come volunteer here,” is not presumably Shipt's hiring pitch.Dan: That's correct. Yeah, exactly. We have a lot of buy-in from the rest of leadership to build out this program. Shipt's history with cybersecurity is one where there were a couple of folks who did a remarkably good job for just being two or three of them for a really long period of time who ran the cybersecurity operation very much was not a part of the engineering culture at Shipt, but there still was coverage.Those folks left earlier in the year, all of them, simultaneously, unfortunately. And that's sort of how the position became open to me in the first place. But it also meant that I was quite literally starting with next to nothing, right. And from that standpoint it made it feel a lot like the early days of the campaign because I was having to build a team from scratch and having to get people motivated to come and work on this thing that had kind of an unknown future roadmap associated with it and all of that kind of stuff.But we've been very privileged to—because we have that leadership support we're able to pay market rates and actually hire qualified and capable and competent engineers and engineering leaders to help build out the aspects of this program that we need. And like I said, we've managed to—we weren't exactly at zero when I walked in the door. So, when I say we were able to quadruple the team, it doesn't mean that we just added four zeros there, [laugh] but we've got a little bit over a dozen people focusing on all areas of security for the business that we can think of. And that's just going to continue to grow. So, it's exciting; it's a challenge. But having the support of the entire organization behind something like this really, really helps a lot.Corey: I know we're running out of time for a lot of the interview, but one more question I want to ask you about is, when you're the CISO for a nationally known politician who is running for the highest office, the risk inherent to getting it wrong is massive. This is one of those mistakes will show indelibly for the rest of, well, one would argue US history, you could arguably say that there will be consequences that go that far out.On the other side of it, once you're done on the campaign you're now the CISO at Shipt. And I am not in any way insinuating that the security of your customers, and your partners, and your data across the board is important. But it does not seem to me from the outside that it has the same, “If we get this wrong there are repercussions that will extend into my grandchildren's time.” How do you find that your ability to care as deeply about this has changed, if it has?Dan: My stress levels are a lot lower I'll say that, but—Corey: You can always spot the veterans on an SRE team because—when I say veterans I mean veterans from the armed forces because, “No one's shooting at me. We can't serve ads right now. I'm really not going to run around and scream like, ‘My hair's on fire,' because this is nothing compared to what stress can look like.” And yeah there's always a worst stressor, but, on some level, it feels like it would be an asset. And again this is not to suggest you don't take security seriously. I want to be very clear on that point.Dan: Yeah, yeah, no. The important challenge of the role is building this out in a way that we have coverage over all the areas that we really need, right, and that is actually the kind of stuff that I enjoy quite a bit. I enjoy starting a program. I enjoy seeing a program come to fruition. I enjoy helping other people build their careers out, and so I have a number of folks who are at earlier at points in their career who I'm very happy that we have them on our team because I can see them grow and I can see them understand and set up what the next thing for them to do is.And so when I look at the day-to-day here, I was motivated on the campaign by that reality of like there is some quite literal life or death stuff that is going to happen here. And that's a really strong presser to make sure that you're doing all the right stuff at the right time. In this case, my motivation is different because I actually enjoy building this kind of stuff out and making sure that we're doing all the right stuff and not having the stress of, like, this could be the end of the world if we get this wrong.Means that I can spend time focusing on making sure that the program is coming together as it should, and getting joy from seeing the program come together is where a lot of that motivation is coming from today. So, it's just different, right? It's a different thing, but at the end of the day it's very rewarding and I'm enjoying it and can see this continuing on for quite some time.Corey: And I look forward to ideally getting you back in another two-and-a-half years after I began badgering you in two hours in order to come back on the show. If—Dan: [laugh].Corey: —people want to hear more about what you're up to, how you view about these things, potentially consider working with you, where can they find you?Dan: Best place although I've not been as active because it has been very busy the last couple of months, but find me on Twitter, @danveloper, find me on LinkedIn. Those—you know, I posted a couple of blog posts about the technology choices that we made on the campaign that I think folks find interesting, and periodically I'll share out my thoughts on Twitter about whatever the most current thing is, Kubernetes or AWS about to go down or something along those lines. So, yeah, that's the best way. And I tweet out all the jobs and post all the jobs that we're hiring for on LinkedIn and all of that kind of stuff. So, usual social channels. Just not Facebook.Corey: Amen to that. And I will of course include links to those things in the [show notes 00:37:29]. Thank you so much for taking the time to speak with me. I appreciate it.Dan: Thank you, Corey.Corey: Dan Woods, CISO and VP of Cybersecurity at Shipt, also formerly of the Biden campaign because wherever he goes he clearly paints a target on his back. I'm Cloud Economist, Corey Quinn and this is Screaming in the Cloud. If you've enjoyed this podcast please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast please leave a five-star review on your podcast platform of choice along with an incoherent rant that is no doubt tied to either politics or the alternate form of politics: Spinnaker.Dan: [laugh].Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.

Screaming in the Cloud
Working the Weather in the Cloud with Jake Hendy

Screaming in the Cloud

Play Episode Listen Later Dec 22, 2021 32:59


About JakeTechnical Lead by day at the Met Office in the UK, leading a team of software developers delivering services for the UK. By night, gamer and fitness instructor, attempting to get a home cinema and gaming setup whilst coralling 3 cats, 2 rabbits, 2 fish tanks, and my wonderful girlfriend.Links: Met Office: https://www.metoffice.gov.uk Twitter: https://twitter.com/jakehendy TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: It seems like there is a new security breach every day. Are you confident that an old SSH key, or a shared admin account, isn't going to come back and bite you? If not, check out Teleport. Teleport is the easiest, most secure way to access all of your infrastructure. The open source Teleport Access Plane consolidates everything you need for secure access to your Linux and Windows servers—and I assure you there is no third option there. Kubernetes clusters, databases, and internal applications like AWS Management Console, Yankins, GitLab, Grafana, Jupyter Notebooks, and more. Teleport's unique approach is not only more secure, it also improves developer productivity. To learn more visit: goteleport.com. And not, that is not me telling you to go away, it is: goteleport.com. Corey: This episode is sponsored in part by our friends at Redis, the company behind the incredibly popular open source database that is not the bind DNS server. If you're tired of managing open source Redis on your own, or you're using one of the vanilla cloud caching services, these folks have you covered with the go to manage Redis service for global caching and primary database capabilities; Redis Enterprise. To learn more and deploy not only a cache but a single operational data platform for one Redis experience, visit redis.com/hero. Thats r-e-d-i-s.com/hero. And my thanks to my friends at Redis for sponsoring my ridiculous non-sense.  Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. It's often said that the sun never sets on the British Empire, but it's often very cloudy and hard to see the sun because many parts of it are dreary and overcast. Here to talk today about how we can predict those things in advance—in theory—is Jake Hendy, Tech Lead at the Met Office. Jake, thanks for joining me.Jake: Hey, Corey, it's lovely to be here. Thanks for inviting me on.Corey: There's a common misconception that its startups in San Francisco or the culture thereof, if you can even elevate it to being a culture above something you'd find in a petri dish, that is where cloud stuff happens, where the computer stuff is done. And I've always liked cutting against that. There are governments that are doing interesting things with Cloud; there are large companies and ‘move fast and break things' is the exact opposite of what you generally want from institutions that date back centuries. What's it like working on Cloud, something that for all intents and purposes didn't exist 20 years ago, in the context of a government office?Jake: As you can imagine, it was a bit of a foray into cloud for us when it first came around. We weren't one of the first people to jump. The Met Office, we've got our own data centers, which we've proudly sit on that contains supercomputers and mainframes as well as a plethora of x86 hardware. So, we didn't move fast at the start, but nowadays, we don't move at breakneck speeds, but we like to take advantage of those managed services. It gets out of the way of managing things for us.Corey: Let's back up a second because I tend to be stereotypically American in many ways. What is the Met Office?Jake: What is the Met Office? The Met Office is the UK's National Meteorological Service. And what does that mean? We do a lot of things though with meteorology, from weather forecasting and climate research from our Hadley Centre—which is world-renowned—down to observations, collections, and partnerships around the world. So, if you've been on a plane over Europe, the Middle East, Africa, over parts of Asia, that plane took off because the Met Office provided a forecast for that plane. There's a whole range of things we can talk about there, if you want Corey, of what the Met Office actually does.Corey: Well, let's ask some of the baseline questions. You think of a weather office in a particular country as, oh okay, it tracks the weather in the area of operations for that particular country. Are you looking at weather on a global basis, on a somewhat local basis, or—as mentioned—since due to a long many-century history it turns out that there are UK Commonwealth territories scattered around the globe, where do you start? Where do you stop?Jake: We don't start and we don't stop. The Met Office is very much a 24/7 operation. So, we've got a 24/7 operation center with staff constantly manning it, doing all sorts of things. So, we've got a defense, we work heavily with our defense colleagues from UK armed forces to NATO partners; we've got aviation, as mentioned; we've got marine shipping from—most of the listeners in the UK will have heard of the shipping forecast at one point or another. And we've got private sector as well, from transport, to energy, supermarkets, and more. We have a very heavy UK focus, for obvious reasons, but our remit goes wide. You can actually go and see some of our model data is actually on Amazon Open Data. We've got MOGREPS, which is our ensemble forecast, as well as global models and UK models, with a 24-hour time lag, but feel free to go and have a play. And you can see the wide variety of data that we produce in just those few models.Corey: Yeah, just pulling up your website now; looking at where I am here in San Francisco, it gives me a detailed hour-by-hour forecast. There are only two problems I see with it. The first is that it's using Celsius units, which I—Jake: [laugh].Corey: —as a matter of policy, don't believe in because in this country, we don't really use things that make sense in measuring context. And also, I don't believe it's a real weather site because it's not absolutely festooned with advertisements for nonsense, which is apparently—I wasn't aware—a thing that you could have on the internet. I thought that showing weather data automatically meant that you had to attempt to cater to the lowest common denominator at all times.Jake: That's an interesting point there. So, the Met Office is owned and operated by Her Majesty's Government. We are a Trading Fund with the Department for Business, Energy and Industrial Strategy. But what does that mean it's a Trading Fund?k it means that we're funded by public money. So, that's called the Public Weather Service.But we also offer a more commercial venture. So, depending on what extensions you've got going on in your browser, there are actually adverts that do run on our website, and we do this to help recover some of the cost. So, the Public Weather Service has to recover some of that. And then lots of things are funded by the Public Weather Service, from observations, to public forecasting. But then there are more those commercial ventures such as the energy markets that have more paid products, and things like that as well. So, maybe not that many adverts, but definitely more usable.Corey: Yeah, I disabled the ad blocker, and I'm reloading it and I'm not seeing any here. Maybe I'm just considered to be such a poor ad targeting prospect at this point that people have just given up in despair. Honestly, people giving up on me in despair is kind of my entire shtick.Jake: We focus heavily on user-centered design, so I was fortunate in their previous team to work in our digital area, consumer digital, which looked after our web and mobile channels. And I can heartily say that there are a lot of changes, had a lot of heavy research into them. Not just internal, getting [unintelligible 00:06:09] and having a look at it, but what does this is actually mean for members of the? Public sending people out doing guerrilla public testing, standing outside Tescos—which is one of our large superstores here—and saying, “Hey, what do you think of this?” And then you'd get a variety of opinions, and then features would be adjusted, tweaked, and so on.Corey: So, you folks have been a relatively early adopter, especially in an institutional context. And by institution, I mean, one of those things that feels like it is as permanent as the stones in a castle, on some level, something that's lasted more than 20 years here in California, what a concept. And part of me wonders, were you one of the first UK government offices to use the cloud, and is that because you do weather and someone was very confused by what Cloud meant?Jake: [laugh]. I think we were possibly one of the first; I couldn't say if we were the first. Over in the UK, we've got a very capable network of government agencies doing some wonderful, and very cloud things. And the Government Digital Service was an initiative set up—uh, I can't remember, and I—unfortunately I can't remember the name of the report that caused its creation, but they had a big hand in doing design and cloud-first deployments. In the Met Office, we didn't take a, “Ah, screw it. Let's jump in,” we took a measured step into the cloud waters.Like I said, we've been running supercomputers since the '50s, and mainframes as well, and x86. I mean, we've been around for 100 years, so we constantly adapt, and engage, and iterate, and improve. But we don't just jump in and take a risk because like you said, we are an institution; we have to provide services for the public. It's not something that you can just ignore. These are services that protect life and property, both at home and abroad.Corey: You have provided a case study historically to AWS, about your use cases of what you use, back in 2014. It was, oh, you're a heavy user of EC2, and looking at the clock, and oh, it's 2014. Surprise. But you've also focused on other services as well. I believe you personally provided a bit of a case study slash story of round your use of Pinpoint of all things, which is a wrapper around SES, their email service, in the hopes of making it a little bit more, I guess, understandable slash fully-featured for contacting people, but in my experience is a great sales device to drive business to its competitors.What's it been like working, I guess, both simultaneously with the tried and true, tested yadda, yadda, yadda, EC2 RDS style stuff, but then looking at what else you're deep into Lambda, and DynamoDB, and SQS sort of stands between both worlds give it was the first service in beta, but it also is a very modern way of thinking about services. How do you contextualize all of that? Because AWS has product strategies, clearly, “Yes.” And they build anything for anyone is more or less what it seems. How do you think about the ecosystem of services that are available and apply it to problems that you're working on?Jake: So, in my personal opinion, I think the Met Office is one of a very small handfuls of companies around the world that could use every Amazon service that's offered, even things like Ground Station. But on my first day in the office, I went and sat at my desk and was talking to my new colleagues, and I looked to the left and he said, “Oh, yeah, that's a satellite dish collecting data from a satellite passing overhead.” So, we very much pick the best tool for the job. So, we have systems which do heavy number crunching, and very intense things, we'll go for EC2.We have systems that store data that needs relationships and all sorts of things. Fine, we'll go RDS. In my space, we have over a billion observations a year coming through the system I lead on SurfaceNet. So, do we need RDS? No. What about if we use something like S3 and Glue and Athena to run queries against this?We're very fortunate that we can pick the best tool for the job, and we pride ourselves on getting the most out of our tools and getting the most value for money. Because like I said, we're funded by the taxpayer; the taxpayer wants value for money, and we are taxpayers ourselves. We don't want to see our money being wasted when we got a hundred size auto-scaling group, when we could do it with Lambda instead.Corey: It's fascinating talking about some of the forward-looking stuff, and oh, serverless and throw everything at Cloud and be all in on cloud. Cloud, cloud, cloud. Cloud is the future. But earlier this year, there was a press release where the Met Office and Microsoft are going to be joining forces to build the world's, and I quote, “Most powerful weather and climate forecasting supercomputer.” The government—your government, to be clear—is investing over a billion pounds in the project.It is slated to be online and running by the middle of next year, 2022, which for a government project as I contextualize them feels like it's underwear-on-outside-the-pants superhero speed. But that, I guess, is what happens when you start looking at these public-private partnerships in some respects. How do you contextualize that? What is the story behind, oh, we're—you're clearly investing heavily in cloud, but you're also building your own custom enormous supercomputer rather than just waiting for AWS to drop one at re:Invent. What is the decision-making process look like? What is the strategy behind it?Jake: Oh. [laugh]. So—I'll have to be careful here—supercomputing is something that we've been doing for a long time, since the '50s, and we've grown with that. When the Met Office moved offices from Bracknell in 2002, 2003, we run two supercomputers for operational resilience, at that point [unintelligible 00:12:06] building in the new building; it was ready, and they were like, “Okay, let's move a supercomputer.” So, it came hurtling down the motorway, plugged in, and congrats, we've now got two supercomputers running again. We're very fortunate—Corey: We had one. It got lonely. We wanted to make it a friend. Yeah, I get it.Jake: Yeah. It's long distance; it works. And the Met Office is actually very good at running projects. We've done many supercomputers over the years, and supercomputing our models, we run some very intense models, and we have more demands. We know we can do better.We know there's the observations in my group we collect, there's the science that's continually improving and iterating and getting better, and our limit isn't poor optimizations or poorly written code. They're scientists running some fantastic code; we have a team who go and optimize these models, and you know, in one release, they may knock down a model runtime by four minutes. And you think, okay, that's four minutes, but for example, if that's four minutes across 400 nodes, all of a sudden you've now got 400 nodes that have then got four minutes more of compute. That could be more research, that could be a different model run. You know, we're very good at running these things, and we're very fortunate with very technically capable to understand the difference between a workload that belongs on AWS, a workload that belongs on a supercomputer.And you know, a supercomputer has many benefits, which the cloud providers… are getting into, you know, we have a high performance clusters on Amazon and Azure, or with, you know, InfiniBand networking. But sometimes you really can't beat a hunking great big ton of metal and super water-cooling, sat in a data center somewhere, backed by—we're very fortunate to have one hundred percent renewable energy for the supercomputer, which is—if you look at any of the power requirements for a supercomputer is phenomenal, so we're throwing that credentials behind it for climate change as well. You can't beat a supercomputer sometimes.Corey: This episode is sponsored by our friends at Oracle HeatWave is a new high-performance accelerator for the Oracle MySQL Database Service. Although I insist on calling it “my squirrel.” While MySQL has long been the worlds most popular open source database, shifting from transacting to analytics required way too much overhead and, ya know, work. With HeatWave you can run your OLTP and OLAP, don't ask me to ever say those acronyms again, workloads directly from your MySQL database and eliminate the time consuming data movement and integration work, while also performing 1100X faster than Amazon Aurora, and 2.5X faster than Amazon Redshift, at a third of the cost. My thanks again to Oracle Cloud for sponsoring this ridiculous nonsense. Corey: I'm somewhat fortunate in the despite living in a world of web apps, these days, my business partner used to work at the Department of Energy at Oak Ridge National Lab, helping with the care and feeding of the supercomputer clusters that they had out there. And you're absolutely right; that matches my understanding with the idea that there are certain workloads you're not going to be able to beat just having this enormous purpose-built cluster sitting there ready to go. Or even if you can, certainly not economically. I have friends who are in the batch side of the world, the HPC side of the world over in the AWS organizations, and they keep—“Hey, look at this. This thing's amazing.”But so much of what they're talking about seems to distill down to, “I have this one-off giant compute task that needs to get done.” Yes, you're right. If I need to calculate the weather one time, then okay, I can make an argument for going with cloud but you're doing this on what appears to be a pretty consistent basis. You're not just assuming—as best I can tell that, “And starting next Wednesday, it will be sunny forever. The end.”Jake: I'm sure many people would love it if we could do weather on-demand.Corey: Oh, yes. [unintelligible 00:15:09] going to reserved instance weather. That would be great. Like, “All right. I'd like to schedule some rain, please.” It really seems like it's one of those areas that is one of the most commonly accepted in science fiction without any real understanding of just what it would take to do something like that. Even understanding and predicting the weather is something that is beyond an awful lot of our current capabilities.Jake: This is exactly it. So, the Met Office is world-renowned for its research capabilities and those really in-depth, very powerful models that we run. So, I mentioned earlier, something called MOGREPS, which is the Met Office's ensemble-based models. And what do we mean by ensembles? You may see in the documentation it's got 18 members.What does that mean? It means that we actually run a simulation 18 times, and we tweak the starting parameters based on these real world inputs. And then you have a number of members that iterate through and supercomputer runs all of them. And we have deterministic models, which have one set of inputs. And you know, it's not just, as you say, one time; these models must run.There are a number of models we do, models on sea state as well, and they've all got to run, so we generally tend to run our supercomputers at top capacity. It's not often you get to go on a supercomputer and there'll be some space for your job to execute right this minute. And there's all the setup as well, so it's not just okay, the supercomputer is ready to go, but there's all the things that go into it, like, those observations, whether it's from the surface, whether it's from satellite data passing overhead, we have our own lightning network, as well. We have many things, like a radar network that we own, and operate. We collaborate with the environment agency for rainfall. And all these things they feed into these models.Okay, now we produce a model, and now it's got to go out. So, it's got to come off the supercomputer, it's got to be processed, maybe the grid that we run the models on needs to be reprojected because different people feed maps in different ways. Then there's got to be cut up because not every customer wants to know what the weather is everywhere. They've got a bit they care about. And of course, these models aren't small; you know, they can be terabytes, so there's also a case of customers might not want to download terabytes; that might cost them a lot. They might only be able to process gigabytes an hour.But then there's other products that we do processing on, so weather models, it might take 40 minutes to over an hour for a model to run. Okay, that's great. You might have missed the first step. Okay, well, we can enrich it with other data that's come in, things like nowcasting, where we do very short runs for the next six-hour forecast. There's a whole number of things that run in the office. And we don't have a choice; they run operationally 24/7, around the clock.I mentioned to you before we started recording, we had an incident of ‘Beast from the East' a number of years back. Some of your listeners may remember this; in the UK, we had a front come in from the east and the UK was blanketed with snow. It was a real severe event. We pretty much kept most of our services running. We worked really hard to make sure that they continued working.And personally I say, perhaps when you go shopping for Black Friday, you might go to a retailer and it's got a queue system up because, you know, it mimics that queue thing when you're outside a store, like in Times Square, and it's raining, be like oh, I might get a deal a minute. I think possibly in the Met Office, we have almost the inverse problem. If the weather's benign, we're still there. People rely on us to go, “Yeah, okay. I can go out and have fun.” When the weather's bad, we don't have a choice. We have to be there because everybody wants us to be there, but we need to be there. It's not a case of this is an optional service.Corey: People often forget that yeah, we are living in a world in which, especially with climate change doing what it's doing, if you get this wrong, people can very easily die. That is not something to take lightly. It's not just about can I go outside and play a pickup game of basketball today?Jake: Exactly. So, you know, operationally, we have something called the National Severe Weather Warning Service, where we issue guidance and alerts across the UK, based on severe weather. And there's a number of different weather types that we issued guidance for. And the severity of that goes from yellow to amber to red. And these are manually generated products, so there's the chief meteorologist who's on shift, and he approves these.And these warnings don't just go out to the members of the public. They go out to Cabinet Office, they go out to first responders, they go out to a number of people who are interested in the weather and have a responsibility. But the other side is that we don't issue a weather warning willy-nilly. It's a measured, calculated decision by our very capable operations team. And once that weather system has passed, the weather story has changed, we'll review it. We go back and we say what could we have done differently?Could the models have predicted this earlier? Could we have new data which would have picked up on this? Some of our next generation products that are in beta, would they have spotted this earlier? There's a lot of service review that continually goes on because like I said, we are the best, and we need to stay the best. People rely on us.Corey: So, here's a question that probably betrays my own ignorance, and that's okay, that's what I'm here to do. When I was a kid, I distinctly remember—first, this is not the era wish the world was black and white; I'm a child of the '80s, let's be clear here, so this is not old-timey nonsense quite as much, but distinctly remember that it was a running gag how unreliable the weather report always was, and it was a bit hit or miss, like, “Well, the paper says it's going to be sunny today, but we're going to pack an umbrella because we know how this works.” It feels, and I could be way off base on this, but it really feels like weather forecasting has gotten significantly more accurate since I was a kid. Is that just nostalgia, and I remember my parents complaining about it, or has there been a qualitative improvement in the accuracy of weather forecasting?Jake: I wish I could tell you all the scientific improvements that we've made, but there's many groups of scientists in the office who I would more than happily shift that responsibility over to, but quite simply, yes. We have a lot of partners we work with around the world—the National Weather Service, DWD in Germany, Meteo France, just to name but a few; there are many—and we all collaborate with data. We all iterate. You know, the American Meteorological Society holds a conference every year, which we attend. And there have been absolutely leaping changes in forecast quality and accuracy over the years.And that's why we continually upgrade our supercomputers. Like I said, yeah, there's research and stuff, but we're pulling in all this science and Meteorology is generally very chaotic systems. We're still discovering many things around how the climate works and how the weather systems work. And we're going to use them to help improve quality of life, early warnings, actually, we can say, oh, in three days time, it's going to be sunny at the beach. Be great if you could know that seven days in advance. It would be great if you knew that 14 days in advance.I mean, we might not do that because at the moment, we might have an idea, but there's also the case of understanding, you know, it's a probability-based decision. And people say, “Oh, it's not going to rain.” But actually, it's a case of, well, we said there's a 20% probability is going to rain. That doesn't mean it's not going to, but it's saying, “Two times out of ten, at this time it's going to rain.” But of course, if you go out 14 days, that's a long lead time, and you know, you talk about chaos theory, and the butterfly moves and flaps its wings, and all of a sudden a [cake 00:22:50] changes color from green to pink or something like that, some other location in the world.These are real systems that have real impacts, so we have to balance out the science of pure numbers, but what do people do with it? And what can people do with it, as well? So, that's why we talk about having timely data as well. People say, “Well, you could run these simulations and all your products take longer to process them and generate them,” but for example, in SurfaceNet, we have five minutes to process an observation once it comes in. We could spend hours fine-tuning that observation to make it perfect, but it needs to be useful.Corey: As you take a look throughout all of the things that AWS is doing—and sure, not all of these are going to necessarily apply directly to empowering the accuracy of weather forecasts, let's be clear here—but you have expressed personal interest in for example, IoT, a bunch of the serverless nonsense we're seeing out there. What excites you the most? What has you the most enthusiastic about what the future the cloud might hold? Because unlike almost everyone else I talk to in this space, you are not selling anything. You don't have a position—that I'm aware of—that oh, yeah, I super want to see this particular thing win the industry because that means you get to buy a boat.You work for the Met Office; you know that in some cases, oh, that boat is not going to have a great time in that part of the world anyway. I don't need one. So, you're a little bit more objective than most people. I have pushing a corporate story. What excites you? Where do you see the future of this industry going in ways that are neat?Jake: Different parts of the office will tell you different things, you know. We worked with Google DeepMind on AI and machine learning. We work with many partners on AI and machine learning, we use it internally, as well. On a personal level, I like quality of life improvements and things that just make my life as both the developer fun and interesting. So, CDK was a big thing.I was a CloudFormation wizard—still hate writing YAML—but the CDK came along and it was [unintelligible 00:24:52] people wouldn't say, but that wasn't, like, know when Lambda launched back in, what, 2013? 2014? No, but it made our lives easier. It meant that actually, we didn't have to worry about, okay, how do we do templating with YAML? Do we have to run some pre-processes or something?It meant that we could invest a little bit of time upfront on CDK and migrating everything over, and then that freed us up to actually doing things that we need for what we call the business or the organization, delivering value, you know? It's great playing with tech but, you know, I need to deliver value. And I think, what was it, in the Google SRE book, they limit the things they do, toiling of manual tasks that don't really contribute anything, they're more like keeping the lights on. Let's get rid of that. Let's focus on delivering value.It's why Lambda is so great. I could patch an EC2, I can automate it, you know, you got AWS Systems Manager Patch Manager, or… whatever its name is, they can go and manage all those patches for you. Why when I can do it in a Lambda and I don't need to worry about it?Corey: So, one last question that I have for you is that you're a tech lead. It's easy for folks to fall into the trap of assuming, “Oh, you're a government. It's like an enterprise only bigger, slower, and way, way, way busier.” How many hundreds of thousands of engineers are working at the Met Office along with you?Jake: So, you can have a look at our public report and you can see the number of staff we have. I think there's about 1800 staff that work at the Met Office. And that includes our account manage, that includes our scientists, that includes HR and legal. And I'd say there's probably less than 300 people who work in technology, as we call it, which is managing our IT estate, managing our Linux estate, managing our storage area networks because, funnily enough, managing petabytes of data is not an easy thing. You know, managing a supercomputer, a mainframe.There really aren't that many people here at the office, but we do so much great stuff. So, as a technical lead, I'm not just a leader of services, but I lead a team of people. I'm responsible for them, for empowering them, and helping them to develop their own careers and their own training. So, it's me and a team of four that look after SurfaceNet. And it's not just SurfaceNet; we've got other systems we look after that SurfaceNet produces data for. Sending messages around the world on the World Meteorological Organization's global telecommunications system. What a mouthful. But you know, these messages go all around the world. And some people might say, “Well, I got a huge team for that.” Well, [unintelligible 00:27:27]. We have other teams that help us—I say, help us—in their own right, they transmit that data. But we're really—I personally wouldn't say we were huge, but boy, do we pack a punch.Corey: Can I just say on a personal note, it's so great to talk to someone who's focusing on building out these environments and solving these problems for a higher purpose slash calling than—and I will get letters for this—than showing ads to people on the internet. I really want to thank you for taking time out of your day to speak with me. If people want to learn more about what you're up to, how you do it, potentially consider maybe joining you if they are eligible to work at the Met Office, where can they find you?Jake: Yeah, so you do have to be a resident in the UK, but www.metoffice.gov.uk is our home on the internet. You can find me on Twitter at @jakehendy, and I could absolutely chew Corey's ear off for many more hours about many of the wonderful services that the Met Office provides. But I can tell he's got something more interesting to do. So, uh [crosstalk 00:28:29]—Corey: Oh, you'd be surprised. It's loads of fun to—no, it's always fun to talk to people who are just in different areas that I don't get to work with very often. It turns out that most of my customers are not focused on telling you what the weather is going to do. And that's fine; it takes all kinds. It's just neat to have this conversation with a different area of the industry. Thank you so much for being so generous with your time. I appreciate it.Jake: Thank you very much for inviting me on. I guess if we get some good feedback, I'll have to come on and I will have to chew your ear off after all.Corey: Don't offer if you're not serious.Jake: Oh, I am.Corey: Jake Hendy, Tech Lead at the Met Office. I'm Cloud Economist Corey Quinn and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice along with a comment yelling at one or both of us for having the temerity to rain on your parade.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.

North Meets South Web Podcast
A Christmas Extravaganza

North Meets South Web Podcast

Play Episode Listen Later Dec 20, 2021 94:35


Jake and Michael are joined by No Plans To Merge's Caleb Porzio and Daniel Coulbourne, as well as TJ Miller (not the actor), and Chris Gmyr to look back at 2021, forward to 2022, and a very one-sided end of year game of Family Feud.This episode is sponsored by Makeable, WorkVivo, and for the first (and last) time, White Cheddar Cheez Its.This episode was streamed live.Show links David Hemphill Andrew Del Prete Caleb Porzio Daniel Coulbourne No Plans To Merge TJ Miller Chris Gmyr Curology Laravel Beyond CRUD

How To Code Well
EP 140 - Code Iteration time is critical to web development success

How To Code Well

Play Episode Listen Later Dec 17, 2021 33:26


Change log We've almost finished the first pass of the documentation for the new PHP course. I have 1 / 2 lessons to go. There are two more passes that I want to do before recording any of the lessons. The next pass will be to create the project from start to finish using the documentation but on a Ubuntu VM. The last pass will be to do the same bu on a Windows VM Last weeks show that was on shopping carts went live earlier this week. I had some internet issues last week so I couldn't do the live stream Your feedback 1) Eric writes in on my hack to check if PHP is working YT short and asks if the next video will be on the PHP 8 match expression and when to use it. I can't promise that it will be the next YT short but I can say that It is on the schedule. 2) Keral writes in on my PHP array sum tutorial and asks how do we sum fields in a database. There is a SUM function in MySQL. It takes one argument and that is the expression that you want to calculate. You can use it to find the sum of a field from several records by suppling the field name as the expression. Then add the reset of the query as usual. For example, To get the sum of a shopping basket you could do something like this; Let's say you had a table of order items with a foreign key linking to an orders table. Each order item record would also have a price field. You could get the sum of the orders price like so: select sum(order_item.price) from order_item where order_item.order_id = 4; Obviously change the table name of order to something else as order is a reserved keyword in MySQL. If you have a comment that you want read out on the show then either write a message on a YouTube video or send me a message on our contact form. Code Iteration time is critical to web development success I want to talk about a blog posted titled 3 lines of code shouldn't take all day https://devtails.xyz/3-lines-of-code-shouldnt-take-all-day by Adam Berg. In the post Adam talks about his time developing video games at Electronic Arts. He mentions that due to the testing tools and development requirements it could take up to a day to write 3 lines of code. This iteration time was greatly reduced when he used 'test beds'. Adam mentions in his article that the test beds focused on particular areas of code which meant the tooling was slimmed down. This allowed him to laser in to the code that needed to be tested and as a result made the code iteration far quicker. He goes on in the post to talk about using unit testing which also made his life a lot easier. These test packages only contained the code which was specific to his teams requirements. Again, this was a slimmed down portion of a game. He mentions that these test packages took less than a second to compile and run. This made him less distracted and he could focus on the task at hand.

Screaming in the Cloud
“Liqui”fying the Database Bottleneck with Robert Reeves

Screaming in the Cloud

Play Episode Listen Later Dec 16, 2021 50:45


About RobertR2 advocates for Liquibase customers and provides technical architecture leadership. Prior to co-founding Datical (now Liquibase), Robert was a Director at the Austin Technology Incubator. Robert co-founded Phurnace Software in 2005. He invented and created the flagship product, Phurnace Deliver, which provides middleware infrastructure management to multiple Fortune 500 companies.Links: Liquibase: https://www.liquibase.com Liquibase Community: https://www.liquibase.org Liquibase AWS Marketplace: https://aws.amazon.com/marketplace/seller-profile?id=7e70900d-dcb2-4ef6-adab-f64590f4a967 Github: https://github.com/liquibase Twitter: https://twitter.com/liquibase TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: It seems like there is a new security breach every day. Are you confident that an old SSH key, or a shared admin account, isn't going to come back and bite you? If not, check out Teleport. Teleport is the easiest, most secure way to access all of your infrastructure. The open source Teleport Access Plane consolidates everything you need for secure access to your Linux and Windows servers—and I assure you there is no third option there. Kubernetes clusters, databases, and internal applications like AWS Management Console, Yankins, GitLab, Grafana, Jupyter Notebooks, and more. Teleport's unique approach is not only more secure, it also improves developer productivity. To learn more visit: goteleport.com. And not, that is not me telling you to go away, it is: goteleport.com. Corey: You know how Git works right?Announcer: Sorta, kinda, not really. Please ask someone else.Corey: That's all of us. Git is how we build things, and Netlify is one of the best ways I've found to build those things quickly for the web. Netlify's Git-based workflows mean you don't have to play slap-and-tickle with integrating arcane nonsense and web hooks, which are themselves about as well understood as Git. Give them a try and see what folks ranging from my fake Twitter for Pets startup, to global Fortune 2000 companies are raving about. If you end up talking to them—because you don't have to; they get why self-service is important—but if you do, be sure to tell them that I sent you and watch all of the blood drain from their faces instantly. You can find them in the AWS marketplace or at www.netlify.com. N-E-T-L-I-F-Y dot com.Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. This is a promoted episode. What does that mean in practice? Well, it means the company who provides the guest has paid to turn this into a discussion that's much more aligned with the company than it is the individual.Sometimes it works, Sometimes it doesn't, but the key part of that story is I get paid. Why am I bringing this up? Because today's guest is someone I met in person at Monktoberfest, which is the RedMonk conference in Portland, Maine, one of the only reasons to go to Maine, speaking as someone who grew up there. And I spoke there, I met my guest today, and eventually it turned into this, proving that I am the envy of developer advocates everywhere because now I can directly tie me attending one conference to making a fixed sum of money, and right now they're all screaming and tearing off their headphones and closing this episode. But for those of you who are sticking around, thank you. My guest today is the CTO and co-founder of Liquibase. Please welcome Robert Reeves. Robert, thank you for joining me, and suffering the slings and arrows I'm about to hurled directly into your arse, as a warning shot.Robert: [laugh]. Man. Thanks for having me. Corey, I've been looking forward to this for a while. I love hanging out with you.Corey: One of the things I love about the Monktoberfest conference, and frankly, anything that RedMonk gets up to is, forget what's on stage, which is uniformly excellent; forget the people at RedMonk who are wonderful and I aspire to do more work with them in different ways; they're great, but the people that they attract are invariably interesting, they are invariably incredibly diverse in terms of not just demographics, but interests and proclivities. It's just a wonderful group of people, and every time I get the opportunity to spend time with those folks I do, and I've never once regretted it because I get to meet people like you. Snark and cynicism about sponsoring this nonsense aside—for which I do thank you—you've been a fascinating person to talk to you because you're better at a lot of the database-facing things than I am, so I shortcut to instead of forming my own opinions, I just skate off of yours in some cases. You're going to get letters now.Robert: Well, look, it's an occupational hazard, right? Releasing software, it's hard so you have to learn these platforms, and part of it includes the database. But I tell you, you're spot on about Monktoberfest. I left that conference so motivated. Really opened my eyes, certainly injecting empathy into what I do on a day-to-day basis, but it spurred me to action.And there's a lot of programs that we've started at Liquibase that the germination for that seed came from Monktoberfest. And certainly, you know, we were bummed out that it's been canceled two years in a row, but we can't wait to get back and sponsor it. No end of love and affection for that team. They're also really smart and right about a hundred percent of the time.Corey: That's the most amazing part is that they have opinions that generally tend to mirror my own—which, you know—Robert: [laugh].Corey: —confirmation bias is awesome, but they almost never get it wrong. And that is one of the impressive things is when I do it, I'm shooting from the hip and I already have an apology half-written and ready to go, whereas when dealing with them, they do research on this and they don't have the ‘I'm a loud, abrasive shitpostter on Twitter' defense to fall back on to defend opinions. And if they do, I've never seen them do it. They're right, and the fact that I am as aligned with them as I am, you'd think that one of us was cribbing from the other. I assure you that's not the case.But every time Steve O'Grady or Rachel Stephens, or Kelly—I forget her last name; my apologies is all Twitter, but she studied medieval history, I remember that—or James Governor writes something, I'm uniformly looking at this and I feel a sense of dismay, been, “Dammit. I should have written this. It's so well written and it makes such a salient point.” I really envy their ability to be so consistently on point.Robert: Well, they're the only analysts we pay money to. So, we vote with our dollars with that one. [laugh].Corey: Yeah. I'm only an analyst when people have analyst budget. Other than that, I'm whatever the hell you describe me. So, let's talk about that thing you're here to show. You know, that little side project thing you found and are the CTO of.I wasn't super familiar with what Liquibase does until I looked into it and then had this—I got to say, it really pissed me off because I'm looking at it, and it's how did I not know that this existed back when the exact problems that you solve are the things I was careening headlong into? I was actively annoyed. You're also an open-source project, which means that you're effectively making all of your money by giving things away and hoping for gratitude to come back on you in the fullness of time, right?Robert: Well, yeah. There's two things there. They're open-source component, but also, where was this when I was struggling with this problem? So, for the folks that don't know, what Liquibase does is automate database schema change. So, if you need to update a database—I don't care what it is—as part of your application deployment, we can help.Instead of writing a ticket or manually executing a SQL script, or generating a bunch of docs in a NoSQL database, you can have Liquibase help you out with that. And so I was at a conference years ago, at the booth, doing my booth thing, and a managing director of a very large bank came to me, like, “Hey, what do you do?” And saw what we did and got angry, started yelling at me. “Where were you three years ago when I was struggling with this problem?” Like, spitting mad. [laugh]. And I was like, “Dude, we just started”—this was a while ago—it was like, “We just started the company two years ago. We got here as soon as we could.”But I struggled with this problem when I was a release manager. And so I've been doing this for years and years and years—I don't even want to talk about how long—getting bits from dev to test to production, and the database was always, always, always the bottleneck, whether it was things didn't run the same in test as they did, eventually in production, environments weren't in sync. It's just really hard. And we've automated so much stuff, we've automated application deployment, lowercase a compiled bits; we're building things with containers, so everything's in that container. It's not a J2EE app anymore—yay—but we haven't done a damn thing for the database.And what this means is that we have a whole part of our industry, all of our database professionals, that are frankly struggling. I always say we don't sell software Liquibase. We sell piano recitals, date nights, happy hours, all the stuff you want to do but you can't because you're stuck dealing with the database. And that's what we do at Liquibase.Corey: Well, you're talking about database people. That's not how I even do it. I would never call myself that, for very good reason because you know, Route 53 remains the only database I use. But the problem I always had was that, “Great. I'm doing a deployment. Oh, I'm going to put out some changes to some web servers. Okay, what's my rollback?” “Well, we have this other commit we can use.” “Oh, we're going to be making a database schema change. What's your rollback strategy,” “Oh, I've updated my resume and made sure that any personal files I had on my work laptop been backed up somewhere else when I immediately leave the company when we can't roll back.” Because there's not really going to be a company anymore at that point.It's one of those everyone sort of holds their breath and winces when it comes to anything that resembles a schema change—or an ALTER TABLE as we used to call it—because that is the mistakes will show territory and you can hope and plan for things in pre-prod environments, but it's always scary. It's always terrifying because production is not like other things. That's why I always call my staging environment ‘theory' because things work in theory but not in production. So, it's how do you avoid the mess of winding up just creating disasters when you're dealing with the reality of your production environments? So, let's back up here. How do you do it? Because it sounds like something people would love to sell me but doesn't exist.Robert: [laugh]. Well, it's real simple. We have a file, we call it the change log. And this is a ledger. So, databases need to be evolved. You can't drop everything and recreate it from scratch, so you have to apply changes sequentially.And so what Liquibase will do is it connects to the database, and it says, “Hey, what version are you?” It looks at the change log, and we'll see, ehh, “There's ten change sets”—that's what components of a change log, we call them change sets—“There's ten change sets in there and the database is telling me that only five had been executed.” “Oh, great. Well, I'll execute these other five.” Or it asks the database, “Hey, how many have been executed?” And it says, “Ten.”And we've got a couple of meta tables that we have in the database, real simple, ANSI SQL compliant, that store the changes that happen to the database. So, if it's a net new database, say you're running a Docker container with the database in it on your local machine, it's empty, you would run Liquibase, and it says, “Oh, hey. It's got that, you know, new database smell. I can run everything.”And so the interesting thing happens when you start pointing it at an environment that you haven't updated in a while. So, dev and test typically are going to have a lot of releases. And so there's going to be little tiny incremental changes, but when it's time to go to production, Liquibase will catch it up. And so we speak SQL to the database, if it's a NoSQL database, we'll speak their API and make the changes requested. And that's it. It's very simple in how it works.The real complex stuff is when we go a couple of inches deeper, when we start doing things like, well, reverse engineering of your database. How can I get a change log of an existing database? Because nobody starts out using Liquibase for a project. You always do it later.Corey: No, no. It's one of those things where when you're doing a project to see if it works, it's one of those, “Great, I'll run a database in some local Docker container or something just to prove that it works.” And, “Todo: fix this later.” And yeah, that todo becomes load-bearing.Robert: [laugh]. That's scary. And so, you know, we can help, like, reverse engineering an entire database schema, no problem. We also have things called quality checks. So sure, you can test your Liquibase change against an empty database and it will tell you if it's syntactically correct—you'll get an error if you need to fix something—but it doesn't enforce things like corporate standards. “Tables start with T underscore.” “Do not create a foreign key unless those columns have an ID already applied.” And that's what our quality checks does. We used to call it rules, but nobody likes rules, so we call it quality checks now.Corey: How do you avoid the trap of enumerating all the bad things you've seen happen because at some point, it feels like that's what leads to process ossification at large companies where, “Oh, we had this bad thing happen once, like, a disk filled up, so now we have a check that makes sure that all the disks are at least 20, empty.” Et cetera. Great. But you keep stacking those you have thousands and thousands and thousands of those, and even a one-line code change then has to pass through so many different tests to validate that this isn't going to cause the failure mode that happened that one time in a unicorn circumstance. How do you avoid the bloat and the creep of stuff like that?Robert: Well, let's look at what we've learned from automated testing. We certainly want more and more tests. Look, DevOp's algorithm is, “All right, we had a problem here.” [laugh]. Or SRE algorithm, I should say. “We had a problem here. What happened? What are we going to change in the future to make sure this doesn't happen?” Typically, that involves a new standard.Now, ossification occurs when a person has to enforce that standard. And what we should do is seek to have automation, have the machine do it for us. Have the humans come up and identify the problem, find a creative way to look for the issue, and then let the machine enforce it. Ossification happens in large organizations when it's people that are responsible, not the machine. The machines are great at running these things over and over again, and they're never hung over, day after Super Bowl Sunday, their kid doesn't get sick, they don't get sick. But we want humans to look at the things that we need that creative energy, that brain power on. And then the rote drudgery, hand that off to the machine.Corey: Drudgery seems like sort of a job description for a lot of us who spend time doing operation stuff.Robert: [laugh].Corey: It's drudgery and it's boring, punctuated by moments of sheer terror. On some level, you're more or less taking some of the adrenaline high of this job away from people. And you know, when it comes to databases, I'm kind of okay with that as it turns out.Robert: Yeah. Oh, yeah, we want no surprises in database-land. And that is why over the past several decades—can I say several decades since 1979?Corey: Oh, you can s—it's many decades, I'm sorry to burst your bubble on that.Robert: [laugh]. Thank you, Corey. Thank you.Corey: Five, if we're being honest. Go ahead.Robert: So, it has evolved over these many decades where change is the enemy of stability. And so we don't want change, and we want to lock these things down. And our database professionals have become changed from sentinels of data into traffic cops and TSA. And as we all know, some things slip through those. Sometimes we speed, sometimes things get snuck through TSA.And so what we need to do is create a system where it's not the people that are in charge of that; that we can set these policies and have our database professionals do more valuable things, instead of that adrenaline rush of, “Oh, my God,” how about we get the rush of solving a problem and saving the company millions of dollars? How about that rush? How about the rush of taking our old, busted on-prem databases and figure out a way to scale these up in the cloud, and also provide quick dev and test environments for our developer and test friends? These are exciting things. These are more fun, I would argue.Corey: You have a list of reference customers on your website that are awesome. In fact, we share a reference customer in the form of Ticketmaster. And I don't think that they will get too upset if I mention that based upon my work with them, at no point was I left with the impression that they played fast and loose with databases. This was something that they take very seriously because for any company that, you know, sells tickets to things you kind of need an authoritative record of who's bought what, or suddenly you don't really have a ticket-selling business anymore. You also reference customers in the form of UPS, which is important; banks in a variety of different places.Yeah, this is stuff that matters. And you support—from the looks of it—every database people can name except for Route 53. You've got RDS, you've got Redshift, you've got Postgres-squeal, you've got Oracle, Snowflake, Google's Cloud Spanner—lest people think that it winds up being just something from a legacy perspective—Cassandra, et cetera, et cetera, et cetera, CockroachDB. I could go on because you have multiple pages of these things, SAP HANA—whatever the hell that's supposed to be—Yugabyte, and so on, and so forth. And it's like, some of these, like, ‘now you're just making up animals' territory.Robert: Well, that goes back to open-source, you know, you were talking about that earlier. There is no way in hell we could have brought out support for all these database platforms without us being open-source. That is where the community aligns their goals and works to a common end. So, I'll give you an example. So, case in point, recently, let me see Yugabyte, CockroachDB, AWS Redshift, and Google Cloud Spanner.So, these are four folks that reached out to us and said, either A) “Hey, we want Liquibase to support our database,” or B) “We want you to improve the support that's already there.” And so we have what we call—which is a super creative name—the Liquibase test harness, which is just genius because it's an automated way of running a whole suite of tests against an arbitrary database. And that helped us partner with these database vendors very quickly and to identify gaps. And so there's certain things that AWS Redshift—certain objects—that AWS Redshift doesn't support, for all the right reasons. Because it's data warehouse.Okay, great. And so we didn't have to run those tests. But there were other tests that we had to run, so we create a new test for them. They actually wrote some of those tests. Our friends at Yugabyte, CockroachDB, Cloud Spanner, they wrote these extensions and they came to us and partnered with us.The only way this works is with open-source, by being open, by being transparent, and aligning what we want out of life. And so what our friends—our database friends—wanted was they wanted more tooling for their platform. We wanted to support their platform. So, by teaming up, we help the most important person, [laugh] the most important person, and that's the customer. That's it. It was not about, “Oh, money,” and all this other stuff. It was, “This makes our customers' lives easier. So, let's do it. Oop, no brainer.”Corey: There's something to be said for making people's lives easier. I do want to talk about that open-source versus commercial divide. If I Google Liquibase—which, you know, I don't know how typing addresses in browsers works anymore because search engines are so fast—I just type in Liquibase. And the first thing it spits me out to is liquibase.org, which is the Community open-source version. And there's a link there to the Pro paid version and whatnot. And I was just scrolling idly through the comparison chart to see, “Oh, so ‘Community' is just code for shitty and you're holding back advanced features.” But it really doesn't look that way. What's the deal here?Robert: Oh, no. So, Liquibase open-source project started in 2006 and Liquibase the company, the commercial entity, started after that, 2012; 2014, first deal. And so, for—Nathan Voxland started this, and Nathan was struggling. He was working at a company, and he had to have his application—of course—you know, early 2000s, J2EE—support SQL Server and Oracle and he was struggling with it. And so he open-sourced it and added more and more databases.Certainly, as open-source databases grew, obviously he added those: MySQL, Postgres. But we're never going to undo that stuff. There's rollback for free in Liquibase, we're not going to be [laugh] we're not going to be jerks and either A) pull features out or, B) even worse, make Stephen O'Grady's life awful by changing the license [laugh] so he has to write about it. He loves writing about open-source license changes. We're Apache 2.0 and so you can do whatever you want with it.And we believe that the things that make sense for a paying customer, which is database-specific objects, that makes sense. But Liquibase Community, the open-source stuff, that is built so you can go to any database. So, if you have a change log that runs against Oracle, it should be able to run against SQL Server, or MySQL, or Postgres, as long as you don't use platform-specific data types and those sorts of things. And so that's what Community is about. Community is about being able to support any database with the same change log. Pro is about helping you get to that next level of DevOps Nirvana, of reaching those four metrics that Dr. Forsgren tells us are really important.Corey: Oh, yes. You can argue with Nicole Forsgren, but then you're wrong. So, why would you ever do that?Robert: Yeah. Yeah. [laugh]. It's just—it's a sucker's bet. Don't do it. There's a reason why she's got a PhD in CS.Corey: She has been a recurring guest on this show, and I only wish she would come back more often. You and I are fun to talk to, don't get me wrong. We want unbridled intellect that is couched in just a scintillating wit, and someone is great to talk to. Sorry, we're both outclassed.Robert: Yeah, you get entertained with us; you learn with her.Corey: Exactly. And you're still entertained while doing it is the best part.Robert: [laugh]. That's the difference between Community and Pro. Look, at the end of the day, if you're an individual developer just trying to solve a problem and get done and away from the computer and go spend time with your friends and family, yeah, go use Liquibase Community. If it's something that you think can improve the rest of the organization by teaming up and taking advantage of the collaboration features? Yes, sure, let us know. We're happy to help.Corey: Now, if people wanted to become an attorney, but law school was too expensive, out of reach, too much time, et cetera, but they did have a Twitter account, very often, they'll find that they can scratch that itch by arguing online about open-source licenses. So, I want to be very clear—because those people are odious when they email me—that you are licensed under the Apache License. That is a bonafide OSI approved open-source license. It is not everyone except big cloud companies, or service providers, which basically are people dancing around—they mean Amazon. So, let's be clear. One, are you worried about Amazon launching a competitive service with a dumb name? And/or have you really been validated as a product if AWS hasn't attempted and failed to launch a competitor?Robert: [laugh]. Well, I mean, we do have a very large corporation that has embedded Liquibase into one of their flagship products, and that is Oracle. They have embedded Liquibase in SQLcl. We're tickled pink because that means that, one, yes, it does validate Liquibase is the right way to do it, but it also means more people are getting help. Now, for Oracle users, if you're just an Oracle shop, great, have fun. We think it's a great solution. But there's not a lot of those.And so we believe that if you have Liquibase, whether it's open-source or the Pro version, then you're going to be able to support all the databases, and I think that's more important than being tied to a single cloud. Also—this is just my opinion and take it for what it's worth—but if Amazon wanted to do this, well, they're not the only game in town. So, somebody else is going to want to do it, too. And, you know, I would argue even with Amazon's backing that Liquibase is a little stronger brand than anything they would come out with.Corey: This episode is sponsored by our friends at Oracle HeatWave is a new high-performance accelerator for the Oracle MySQL Database Service. Although I insist on calling it “my squirrel.” While MySQL has long been the worlds most popular open source database, shifting from transacting to analytics required way too much overhead and, ya know, work. With HeatWave you can run your OLTP and OLAP, don't ask me to ever say those acronyms again, workloads directly from your MySQL database and eliminate the time consuming data movement and integration work, while also performing 1100X faster than Amazon Aurora, and 2.5X faster than Amazon Redshift, at a third of the cost. My thanks again to Oracle Cloud for sponsoring this ridiculous nonsense. Corey: So, I want to call out though, that on some level, they have already competed with you because one of database that you do not support is DynamoDB. Let's ignore the Route 53 stuff because, okay. But the reason behind that, having worked with it myself, is that, “Oh, how do you do a schema change in DynamoDB?” The answer is that you don't because it doesn't do schemas for one—it is schemaless, which is kind of the point of it—as well as oh, you want to change the primary, or the partition, or the sort key index? Great. You need a new table because those things are immutable.So, they've solved this Gordian Knot just like Alexander the Great did by cutting through it. Like, “Oh, how do you wind up doing this?” “You don't do this. The end.” And that is certainly an approach, but there are scenarios where those were first, NoSQL is not a acceptable answer for some workloads.I know Rick [Horahan 00:26:16] is going to yell at me for that as soon as he hears me, but okay. But there are some for which a relational database is kind of a thing, and you need that. So, Dynamo isn't fit for everything. But there are other workloads where, okay, I'm going to just switch over. I'm going to basically dump all the data and add it to a new table. I can't necessarily afford to do that with anything less than maybe, you know, 20 milliseconds of downtime between table one and table two. And they're obnoxious and difficult ways to do it, but for everything else, you do kind of need to make ALTER TABLE changes from time to time as you go through the build and release process.Robert: Yeah. Well, we certainly have plans for DynamoDB support. We are working our way through all the NoSQLs. Started with Mongo, and—Corey: Well, back that out a second then for me because there's something I'm clearly not grasping because it's my understanding, DynamoDB is schemaless. You can put whatever you want into various arbitrary fields. How would Liquibase work with something like that?Robert: Well, that's something I struggled with. I had the same question. Like, “Dude, really, we're a schema change tool. Why would we work with a schemaless database?” And so what happened was a soon-to-be friend of ours in Europe had reached out to me and said, “I built an extension for MongoDB in Liquibase. Can we open-source this, and can y'all take care of the care and feeding of this?” And I said, “Absolutely. What does it do?” [laugh].And so I looked at it and it turns out that it focuses on collections and generating data for test. So, you're right about schemaless because these are just documents and we're not going to go through every single document and change the structure, we're just going to have the application create a new doc and the new format. Maybe there's a conversion log logic built into the app, who knows. But it's the database professionals that have to apply these collections—you know, indices; that's what they call them in Mongo-land: collections. And so being able to apply these across all environments—dev, test, production—and have consistency, that's important.Now, what was really interesting is that this came from MasterCard. So, this engineer had a consulting business and worked for MasterCard. And they had a problem, and they said, “Hey, can you fix this with Liquibase?” And he said, “Sure, no problem.” And he built it.So, that's why if you go to the MongoDB—the liquibase-mongodb repository in our Liquibase org, you'll see that MasterCard has the copyright on all that code. Still Apache 2.0. But for me, that was the validation we needed to start expanding to other things: Dynamo, Couch. And same—Corey: Oh, yeah. For a lot of contributors, there's a contributor license process you can go through, assign copyright. For everything else, there's MasterCard.Robert: Yeah. Well, we don't do that. Look, you know, we certainly have a code of conduct with our community, but we don't have a signing copyright and that kind of stuff. Because that's baked into Apache 2.0. So, why would I want to take somebody's ability to get credit and magical internet points and increase the rep by taking that away? That's just rude.Corey: The problem I keep smacking myself into is just looking at how the entire database space across the board goes, it feels like it's built on lock-in, it's built on it is super finicky to work with, and it generally feels like, okay, great. You take something like Postgres-squeal or whatever it is you want to run your database on, yeah, you could theoretically move it a bunch of other places, but moving databases is really hard. Back when I was at my last, “Real job,” quote-unquote, years ago, we were late to the game; we migrated the entire site from EC2 Classic into a VPC, and the biggest pain in the ass with all of that was the RDS instance. Because we had to quiesce the database so it would stop taking writes; we would then do snapshot it, shut it down, and then restore a new database from that RDS snapshot.How long does it take, at least in those days? That is left as an experiment for the reader. So, we booked a four hour maintenance window under the fear that would not be enough. It completed in 45 minutes. So okay, there's that. Sparked the thing up and everything else was tested and good to go. And yay. Okay.It took a tremendous amount of planning, a tremendous amount of work, and that wasn't moving it very far. It is the only time I've done a late-night deploy, where not a single thing went wrong. Until I was on the way home and the Uber driver sideswiped a city vehicle. So, there we go—Robert: [laugh].Corey: —that's the one. But everything else was flawless on this because we planned these things out. But imagine moving to a different provider. Oh, forget it. Or imagine moving to a different database engine? That's good. Tell another one.Robert: Well, those are the problems that we want our database professionals to solve. We do not want them to be like janitors at an elementary school, cleaning up developer throw-up with sawdust. The issue that you're describing, that's a one time event. This is something that doesn't happen very often. You need hands on the keyboard, you want people there to look for problems.If you can take these database releases away from those folks and automate them safely—you can have safety and speed—then that frees up their time to do these other herculean tasks, these other feats of strength that they're far better at. There is no silver bullet panacea for database issues. All we're trying to do is take about 70% of DBAs time and free it up to do the fun stuff that you described. There are people that really enjoy that, and we want to free up their time so they can do that. Moving to another platform, going from the data center to the cloud, these sorts of things, this is what we want a human on; we don't want them updating a column three times in a row because dev couldn't get it right. Let's just give them the keys and make sure they stay in their lane.Corey: There's something glorious about being able to do that. I wish that there were more commonly appreciated ways of addressing those pains, rather than, “Oh, we're going to sell you something big and enterprise-y and it's going to add a bunch of process and not work out super well for you.” You integrate with existing CI/CD systems reasonably well, as best I can tell because the nice thing about CI/CD—and by nice I mean awful—is that there is no consensus. Every pipeline you see, in a release engineering process inherently becomes this beautiful bespoke unicorn.Robert: Mm-hm. Yeah. And we have to. We have to integrate with whatever CI/CD they have in place. And we do not want customers to just run Liquibase by itself. We want them to integrate it with whatever is driving that application deployment.We're Switzerland when it comes to databases, and CI/CD. And I certainly have my favorite of those, and it's primarily based on who bought me drinks at the last conference, but we cannot go into somebody's house and start rearranging the furniture. That's just rude. If they're deploying the app a certain way, what we tell that customer is, “Hey, we're just going to have that CI/CD tool call Liquibase to update the database. This should be an atomic unit of deployment.” And it should be hidden from the person that pushes that shiny button or the automation that does it.Corey: I wish that one day that you could automate all of the button pushing, but the thing that always annoyed me in release engineering was the, “Oh, and here's where we stop to have a human press the button.” And I get it. That stuff's scary for some folks, but at the same time, this is the nature of reality. So, you're not going to be able to technology your way around people. At least not successfully and not for very long.Robert: It's about trust. You have to earn that database professional's trust because if something goes wrong, blaming Liquibase doesn't go very far. In that company, they're going to want a person [laugh] who has a badge to—with a throat to choke. And so I've seen this pattern over and over again.And this happened at our first customer. Major, major, big, big, big bank, and this was on the consumer side. They were doing their first production push, and they wanted us ready. Not on the call, but ready if there was an issue they needed to escalate and get us to help them out. And so my VP of Engineering and me, we took it. Great. Got VP of engineering and CTO. Right on.And so Kevin and I, we stayed home, stayed sober [laugh], you know—a lot of places to party in Austin; we fought that temptation—and so we stayed and I'm texting with Kevin, back and forth. “Did you get a call?” “No, I didn't get a call.” It was Friday night. Saturday rolls around. Sunday. “Did you get a—what's going on?” [laugh].Monday, we're like, “Hey. Everything, okay? Did you push to the next weekend?” They're like, “Oh, no. We did. It went great. We forgot to tell you.” [laugh]. But here's what happened. The DBAs push the Liquibase ‘make it go' button, and then they said, “Uh-Oh.” And we're like, “What do you mean, uh-oh?” They said, “Well, something went wrong.” “Well, what went wrong?” “Well, it was too fast.” [laugh]. Something—no way. And so they went through the whole thing—Corey: That was my downtime when I supposed to be compiling.Robert: Yeah. So, they went through the whole thing to verify every single change set. Okay, so that was weekend one. And then they go to weekend two, they do it the same thing. All right, all right. Building trust.By week four, they called a meeting with the release team. And they said, “Hey, process change. We're no longer going to be on these calls. You are going to push the Liquibase button. Now, if you want to integrate it with your CI/CD, go right ahead, but that's not my problem.” Dev—or, the release team is tier one; dev is tier two; we—DBAs—are tier three support, but we'll call you because we'll know something went wrong. And to this day, it's all automated.And so you have to earn trust to get people to give that up. Once they have trust and you really—it's based on empathy. You have to understand how terrible [laugh] they are sometimes treated, and to actively take care of them, realize the problems they're struggling with, and when you earn that trust, then and only then will they allow automation. But it's hard, but it's something you got to do.Corey: You mentioned something a minute ago that I want to focus on a little bit more closely, specifically that you're in Austin. Seems like that's a popular choice lately. You've got companies that are relocating their headquarters there, presumably for tax purposes. Oracle's there, Tesla's there. Great. I mean, from my perspective, terrific because it gets a number of notably annoying CEOs out of my backyard. But what's going on? Why is Austin on this meteoric rise and how'd it get there?Robert: Well, a lot of folks—overnight success, 40 years in the making, I guess. But what a lot of people don't realize is that, one, we had a pretty vibrant tech hub prior to all this. It all started with MCC, Microcomputer Consortium, which in the '80s, we were afraid of the Japanese taking over and so we decided to get a bunch of companies together, and Admiral Bobby Inman who was director planted it in Austin. And that's where it started. You certainly have other folks that have a huge impact, obviously, Michael Dell, Austin Ventures, a whole host of folks that have really leaned in on tech in Austin, but it actually started before that.So, there was a time where Willie Nelson was in Nashville and was just fed up with RCA Records. They would not release his albums because he wanted to change his sound. And so he had some nice friends at Atlantic Records that said, “Willie, we got this. Go to New York, use our studio, cut an album, we'll fix it up.” And so he cut an album called Shotgun Willie, famous for having “Whiskey River” which is what he uses to open and close every show.But that album sucked as far as sales. It's a good album, I like it. But it didn't sell except for one place in America: in Austin, Texas. It sold more copies in Austin than anywhere else. And so Willie was like, “I need to go check this out.”And so he shows up in Austin and sees a bunch of rednecks and hippies hanging out together, really geeking out on music. It was a great vibe. And then he calls, you know, Kris, and Waylon, and Merle, and say, “Come on down.” And so what happened here was a bunch of people really wanted to geek out on this new type of country music, outlaw country. And it started a pattern where people just geek out on stuff they really like.So, same thing with Austin film. You got Robert Rodriguez, you got Richard Linklater, and Slackers, his first movie, that's why I moved to Austin. And I got a job at Les Amis—a coffee shop that's closed—because it had three scenes in that. There was a whole scene of people that just really wanted to make different types of films. And we see that with software, we see that with film, we see it with fashion.And it just seems that Austin is the place where if you're really into something, you're going to find somebody here that really wants to get into it with you, whether it's board gaming, D&D, noise punk, whatever. And that's really comforting. I think it's the community that's just welcoming. And I just hope that we can continue that creativity, that sense of community, and that we don't have large corporations that are coming in and just taking from the system. I hope they inject more.I think Oracle's done a really good job; their new headquarters is gorgeous, they've done some really good things with the city, doing a land swap, I think it was forty acres for nine acres. They coughed up forty for nine. And it was nine acres the city wasn't even using. Great. So, I think they're being good citizens. I think Tesla's been pretty cool with building that factory where it is. I hope more come. I hope they catch what is ever in the water and the breakfast tacos in Austin.Corey: [laugh]. I certainly look forward to this pandemic ending; I can come over and find out for myself. I'm looking forward to it. I always enjoyed my time there, I just wish I got to spend more of it.Robert: How many folks from Duckbill Group are in Austin now?Corey: One at the moment. Tim Banks. And the challenge, of course, is that if you look across the board, there really aren't that many places that have more than one employee. For example, our operations person, Megan, is here in San Francisco and so is Jesse DeRose, our manager of cloud economics. But my business partner is in Portland; we have people scattered all over the country.It's kind of fun having a fully-distributed company. We started this way, back when that was easy. And because all right, travel is easy; we'll just go and visit whenever we need to. But there's no central office, which I think is sort of the dangerous part of full remote because then you have this idea of second-class citizens hanging out in one part of the country and then they go out to lunch together and that's where the real decisions get made. And then you get caught up to speed. It definitely fosters a writing culture.Robert: Yeah. When we went to remote work, our lease was up. We just didn't renew. And now we have expanded hiring outside of Austin, we have folks in the Ukraine, Poland, Brazil, more and more coming. We even have folks that are moving out of Austin to places like Minnesota and Virginia, moving back home where their family is located.And that is wonderful. But we are getting together as a company in January. We're also going to, instead of having an office, we're calling it a ‘Liquibase Lounge.' So, there's a number of retail places that didn't survive, and so we're going to take one of those spots and just make a little hangout place so that people can come in. And we also want to open it up for the community as well.But it's very important—and we learned this from our friends at GitLab and their culture. We really studied how they do it, how they've been successful, and it is an awareness of those lunch meetings where the decisions are made. And it is saying, “Nope, this is great we've had this conversation. We need to have this conversation again. Let's bring other people in.” And that's how we're doing at Liquibase, and so far it seems to work.Corey: I'm looking forward to seeing what happens, once this whole pandemic ends, and how things continue to thrive. We're long past due for a startup center that isn't San Francisco. The whole thing is based on the idea of disruption. “Oh, we're disruptive.” “Yes, we're so disruptive, we've taken a job that can be done from literally anywhere with internet access and created a land crunch in eight square miles, located in an earthquake zone.” Genius, simply genius.Robert: It's a shame that we had to have such a tragedy to happen to fix that.Corey: Isn't that the truth?Robert: It really is. But the toothpaste is out of the tube. You ain't putting that back in. But my bet on the next Tech Hub: Kansas City. That town is cool, it has one hundred percent Google Fiber all throughout, great university. Kauffman Fellows, I believe, is based there, so VC folks are trained there. I believe so; I hope I'm not wrong with that. I know Kauffman Foundation is there. But look, there's something happening in that town. And so if you're a buy low, sell high kind of person, come check us out in Austin. I'm not trying to dissuade anybody from moving to Austin; I'm not one of those people. But if the housing prices [laugh] you don't like them, check out Kansas City, and get that two-gig fiber for peanuts. Well, $75 worth of peanuts.Corey: Robert, I want to thank you for taking the time to speak with me so extensively about Liquibase, about how awesome RedMonk is, about Austin and so many other topics. If people want to learn more, where can they find you?Robert: Well, I think the best place to find us right now is in AWS Marketplace. So—Corey: Now, hand on a second. When you say the best place for anything being the AWS Marketplace, I'm naturally a little suspicious. Tell me more.Robert: [laugh]. Well, best is, you know, it's—[laugh].Corey: It is a place that is there and people can find you through it. All right, then.Robert: I have a list. I have a list. But the first one I'm going to mention is AWS Marketplace. And so that's a really easy way, especially if you're taking advantage of the EDP, Enterprise Discount Program. That's helpful. Burn down those dollars, get a discount, et cetera, et cetera. Now, of course, you can go to liquibase.com, download a trial. Or you can find us on Github, github.com/liquibase. Of course, talking smack to us on Twitter is always appreciated.Corey: And we will, of course, include links to that in the [show notes 00:46:37]. Robert Reeves, CTO and co-founder of Liquibase. I'm Cloud Economist Corey Quinn, and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice along with an angry comment complaining about how Liquibase doesn't support your database engine of choice, which will quickly be rendered obsolete by the open-source community.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.

Modernize or Die ® Podcast - CFML News Edition
Modernize or Die® - CFML News for December 14th, 2021 - Episode 128

Modernize or Die ® Podcast - CFML News Edition

Play Episode Listen Later Dec 14, 2021 56:34


2021-12-14 Weekly News - Episode 128Watch the video version on YouTube at https://youtu.be/_GrDec5PVwg Hosts: Gavin Pickin - Senior Developer for Ortus SolutionsDan Card  - Software Developer for Ortus SolutionsThanks to our Sponsor - Ortus SolutionsThe makers of ColdBox, CommandBox, ForgeBox, TestBox and almost every other Box out there. A few ways  to say thanks back to Ortus Solutions: Like and subscribe to our videos on YouTube.  Subscribe to our Podcast on your Podcast Apps and leave us a review Sign up for a free or paid account on CFCasts, which is releasing new content every week Buy Ortus's new Book - 102 ColdBox HMVC Quick Tips and Tricks on GumRoad (http://gum.co/coldbox-tips) Patreon SupportWe have 37 patreons providing 97% of the funding for our Modernize or Die Podcasts via our Patreon site: https://www.patreon.com/ortussolutions. News and EventsNew Host - Dan CardDan introduces himself and gives a quick run down of his CFML experience.Log4j Vulnerability ReportedThere is a critical security vulnerability (CVE-2021-44228 aka Log4Shell) in the java library log4j which is a popular logging library for java applications. It is included in both Adobe ColdFusion and Lucee for example.Putting together some info to help sort this issue out as it pertains to ColdFusion and Lucee users. I'll update this entry as needed.https://www.petefreitag.com/item/923.cfm Adobe's update on the matter (thanks charlie for pointing this out)Blog - https://coldfusion.adobe.com/2021/12/update-log4j-vulnerability/ Update - https://helpx.adobe.com/coldfusion/kb/log4j-vulnerability-coldfusion.html TLDR for AdobeThere is a critical security vulnerability (CVE-2021-44228) in the Log4j, which is a popular logging library for Java-based applications. The vulnerability also impacts Adobe ColdFusion.Adobe is investigating any potential impact and is taking action including updating affected systems to the latest versions of Apache Log4j recommended by the Apache Software Foundation.ColdFusion plans to release a patch (version(s) 2021, 2018) for this log4j vulnerability to customers on 12/17/2021. VERY FAST FOR ADOBE - THEY DONT MOVE FAST USUALLYIn the meantime, we recommend that ColdFusion users apply the following workarounds/mitigations steps, until this patch is released.Lucee is not affected https://dev.lucee.org/t/lucee-is-not-affected-by-the-log4j-jndi-exploit-cve-2021-44228/9331 Charlie's Blog on the matter https://www.carehart.org/blog/2021/12/14/about_the_log4jshell_pandemic https://coldfusion.adobe.com/2021/12/dealing-recent-log4j-vulnerability-adobe-releases-update/ More news links about Log4j https://www.zdnet.com/article/log4j-flaw-attackers-are-making-thousands-of-attempts-to-exploit-this-severe-vulnerability/New CommandBox FeatureAdd the equivalent of the mod_cfml tomcat valve into CommandBox as an Undertow handler to auto-create contexts based on the front-end servers's virtual hosts.Support the same request headers and behavior of mod_cfmlIdeally, this should have drop-in support behind BonCode IIS or Apache's mod_cfml moduleSupport max contexts settingMake this new behavior off (opt-in) by default Support and require shared key for security (Note, the current mod_cfml Tomcat valve does not require the shared key, but we will)https://ortussolutions.atlassian.net/browse/COMMANDBOX-1411 CBSecurity V2.15.0 released

airhacks.fm podcast with adam bien
Serverless Java on AWS

airhacks.fm podcast with adam bien

Play Episode Listen Later Dec 10, 2021 63:31


An airhacks.fm conversation with Mark Sailes (@MarkSailes3) about: the BBC micro computer with a cassette, the PRINT 10, 386, 486 and a Pentium with an internet connection, learning Apache, using Mandrake Linux at university, a first web page - a huge experience, PHP, MySQL and "we don't need transactions", the fantastic phpMyAdmin, using Java, C++ and Python at the university, the great JavaDoc, Eclipse and NetBeans, the great Java collection JavaDoc, migrating from java.util.Vector to java.util.List, working as backend junior Java developer, from junior over senior to team lead, 3% improvement with 97% rewrite, working for AWS, "Essentialism: The Disciplined Pursuit of Less" book, the WebLogic build engineer, pre pooling EJBs, Hey Enterprise EJB Developers Now Is The Time To Go Serverless, Lambda with API Gateway is a transition to Event Driven Architectures, Using AWS Lambda with an Application Load Balancer, cloud native, event driven architectures with AWS Lambda and Java, testable, asynchronous AWS Lambda, the serverless Kafka on AWS, archive and replay with Amazon Event Bridge, fast cold starts with AWS Lambda, milliseconds invocations with AWS Lambda, testing asynchronous AWS Lambda with JUnit, the limitations of mocking, AWS Cloud Development Kit (CDK) and AWS SAM CLI, swapping out Lambdas with SAM, describing AWS infrastructure with CDK, no YAML deployments with CDK, shareable infrastructure with compilable Java code, AWS CDK constructs--reusable cloud pieces Mark Sailes on twitter: @MarkSailes3, Mark's blog: mark-sailes.medium.com

Screaming in the Cloud
Building Distributed Cognition into Your Business with Sam Ramji

Screaming in the Cloud

Play Episode Listen Later Dec 9, 2021 39:56


About SamA 25-year veteran of the Silicon Valley and Seattle technology scenes, Sam Ramji led Kubernetes and DevOps product management for Google Cloud, founded the Cloud Foundry foundation, has helped build two multi-billion dollar markets (API Management at Apigee and Enterprise Service Bus at BEA Systems) and redefined Microsoft's open source and Linux strategy from “extinguish” to “embrace”.He is nerdy about open source, platform economics, middleware, and cloud computing with emphasis on developer experience and enterprise software. He is an advisor to multiple companies including Dell Technologies, Accenture, Observable, Fletch, Orbit, OSS Capital, and the Linux Foundation.Sam received his B.S. in Cognitive Science from UC San Diego, the home of transdisciplinary innovation, in 1994 and is still excited about artificial intelligence, neuroscience, and cognitive psychology.Links: DataStax: https://www.datastax.com Sam Ramji Twitter: https://twitter.com/sramji Open||Source||Data: https://www.datastax.com/resources/podcast/open-source-data Screaming in the Cloud Episode 243 with Craig McLuckie: https://www.lastweekinaws.com/podcast/screaming-in-the-cloud/innovating-in-the-cloud-with-craig-mcluckie/ Screaming in the Cloud Episode 261 with Jason Warner: https://www.lastweekinaws.com/podcast/screaming-in-the-cloud/what-github-can-give-to-microsoft-with-jason-warner/ TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: This episode is sponsored in part by our friends at Redis, the company behind the incredibly popular open source database that is not the bind DNS server. If you're tired of managing open source Redis on your own, or you're using one of the vanilla cloud caching services, these folks have you covered with the go to manage Redis service for global caching and primary database capabilities; Redis Enterprise. Set up a meeting with a Redis expert during re:Invent, and you'll not only learn how you can become a Redis hero, but also have a chance to win some fun and exciting prizes. To learn more and deploy not only a cache but a single operational data platform for one Redis experience, visit redis.com/hero. Thats r-e-d-i-s.com/hero. And my thanks to my friends at Redis for sponsoring my ridiculous non-sense.  Corey: Are you building cloud applications with a distributed team? Check out Teleport, an open source identity-aware access proxy for cloud resources. Teleport provides secure access to anything running somewhere behind NAT: SSH servers, Kubernetes clusters, internal web apps and databases. Teleport gives engineers superpowers! Get access to everything via single sign-on with multi-factor. List and see all SSH servers, kubernetes clusters or databases available to you. Get instant access to them all using tools you already have. Teleport ensures best security practices like role-based access, preventing data exfiltration, providing visibility and ensuring compliance. And best of all, Teleport is open source and a pleasure to use.Download Teleport at https://goteleport.com. That's goteleport.com.Corey: Welcome to Screaming in the Cloud, I'm Cloud Economist Corey Quinn, and recurring effort that this show goes to is to showcase people in their best light. Today's guest has done an awful lot: he led Kubernetes and DevOps Product Management for Google Cloud; he founded the Cloud Foundry Foundation; he set open-source strategy for Microsoft in the naughts; he advises companies including Dell, Accenture, the Linux Foundation; and tying all of that together, it's hard to present a lot of that in a great light because given my own proclivities, that sounds an awful lot like a personal attack. Sam Ramji is the Chief Strategy Officer at DataStax. Sam, thank you for joining me, and it's weird when your resume starts to read like, “Oh, I hate all of these things.”Sam: [laugh]. It's weird, but it's true. And it's the only life I could have lived apparently because here I am. Corey, it's a thrill to meet you. I've been an admirer of your public speaking, and public tweeting, and your writing for a long time.Corey: Well, thank you. The hard part is getting over the voice saying don't do it because it turns out that there's no real other side of public shutting up, which is something that I was never good at anyway, so I figured I'd lean into it. And again, I mean, that the sense of where you have been historically in terms of your career not, “Look what you've done,” which is a subtext that I could be accused of throwing in sometimes.Sam: I used to hear that a lot from my parents, actually.Corey: Oh, yeah. That was my name growing up. But you've done a lot of things, and you've transitioned from notable company making significant impact on the industry, to the next one, to the next one. And you've been in high-flying roles, doing lots of really interesting stuff. What's the common thread between all those things?Sam: I'm an intensely curious person, and the thing that I'm most curious about is distributed cognition. And that might not be obvious from what you see is kind of the… Lego blocks of my career, but I studied cognitive science in college when that was not really something that was super well known. So, I graduated from UC San Diego in '94 doing neuroscience, artificial intelligence, and psychology. And because I just couldn't stop thinking about thinking; I was just fascinated with how it worked.So, then I wanted to build software systems that would help people learn. And then I wanted to build distributed software systems. And then I wanted to learn how to work with people who were thinking about building the distributed software systems. So, you end up kind of going up this curve of, like, complexity about how do we think? How do we think alone? How do we learn to think? How do we think together?And that's the directed path through my software engineering career, into management, into middleware at BEA, into open-source at Microsoft because that's an amazing demonstration of distributed cognition, how, you know, at the time in 2007, I think, Sourceforge had 100,000 open-source projects, which was, like, mind boggling. Some of them even worked together, but all of them represented these groups of people, flung around the world, collaborating on something that was just fundamentally useful, that they were curious about. Kind of did the same thing into APIs because APIs are an even better way to reuse for some cases than having the source code—at Apigee. And kept growing up through that into, how are we building larger-scale thinking systems like Cloud Foundry, which took me into Google and Kubernetes, and then some applications of that in Autodesk and now DataStax. So, I love building companies. I love helping people build companies because I think business is distributed cognition. So, those businesses that build distributed systems, for me, are the most fascinating.Corey: You were basically handed a heck of a challenge as far as, “Well, help set open-source strategy,” back at Microsoft, in the days where that was a punchline. And credit where due, I have to look at Microsoft of today, and it's not a joke, you can have your arguments about them, but again in those days, a lot of us built our entire personality on hating Microsoft. Some folks never quite evolved beyond that, but it's a new ballgame and it's very clear that the Microsoft of yesteryear and the Microsoft of today are not completely congruent. What was it like at that point understanding that as you're working with open-source communities, you're doing that from a place of employment with a company that was widely reviled in the space.Sam: It was not lost on me. The irony, of course, was that—Corey: Well, thank God because otherwise the question where you would have been, “What do you mean they didn't like us?”Sam: [laugh].Corey: Which, on some levels, like, yeah, that's about the level of awareness I would have expected in that era, but contrary to popular opinion, execs at these companies are not generally oblivious.Sam: Yeah, well, if I'd been clever as a creative humorist, I would have given you that answer instead of my serious answer, but for some reason, my role in life is always to be the straight guy. I used to have Slashdot as my homepage, right? I love when I'd see some conspiracy theory about, you know, Bill Gates dressed up as the Borg, taking over the world. My first startup, actually in '97, was crushed by Microsoft. They copied our product, copied the marketing, and bundled it into Office, so I had lots of reasons to dislike Microsoft.But in 2004, I was recruited into their venture capital team, which I couldn't believe. It was really a place that they were like, “Hey, we could do better at helping startups succeed, so we're going to evangelize their success—if they're building with Microsoft technologies—to VCs, to enterprises, we'll help you get your first big enterprise deal.” I was like, “Man, if I had this a few years ago, I might not be working.” So, let's go try to pay it forward.I ended up in open-source by accident. I started going to these conferences on Software as a Service. This is back in 2005 when people were just starting to light up, like, Silicon Valley Forum with, you know, the CEO of Demandware would talk, right? We'd hear all these different ways of building a new business, and they all kept talking about their tech stack was Linux, Apache, MySQL, and PHP. I went to one eight-hour conference, and Microsoft technologies were mentioned for about 12 seconds in two separate chunks. So, six seconds, he was like, “Oh, and also we really like Microsoft SQL Server for our data layer.”Corey: Oh, Microsoft SQL Server was fantastic. And I know that's a weird thing for people to hear me say, just because I've been renowned recently for using Route 53 as the primary data store for everything that I can. But there was nothing quite like that as far as having multiple write nodes, being able to handle sharding effectively. It was expensive, and you would take a bath on the price come audit time, but people were not rolling it out unaware of those things. This was a trade off that they were making.Oracle has a similar story with databases. It's yeah, people love to talk smack about Oracle and its business practices for a variety of excellent reasons, at least in the database space that hasn't quite made it to cloud yet—knock on wood—but people weren't deploying it because they thought Oracle was warm and cuddly as a vendor; they did it because they can tolerate the rest of it because their stuff works.Sam: That's so well said, and people don't give them the credit that's due. Like, when they built hypergrowth in their business, like… they had a great product; it really worked. They made it expensive, and they made a lot of money on it, and I think that was why you saw MySQL so successful and why, if you were looking for a spec that worked, that you could talk through through an open driver like ODBC or JDBC or whatever, you could swap to Microsoft SQL Server. But I walked out of that and came back to the VC team and said, “Microsoft has a huge problem. This is a massive market wave that's coming. We're not doing anything in it. They use a little bit of SQL Server, but there's nothing else in your tech stack that they want, or like, or can afford because they don't know if their businesses are going to succeed or not. And they're going to go out of business trying to figure out how much licensing costs they would pay to you in order to consider using your software. They can't even start there. They have to start with open-source. So, if you're going to deal with SaaS, you're going to have to have open-source, and get it right.”So, I worked with some folks in the industry, wrote a ten-page paper, sent it up to Bill Gates for Think Week. Didn't hear much back. Bought a new strategy to the head of developer platform evangelism, Sanjay Parthasarathy who suggested that the idea of discounting software to zero for startups, with the hope that they would end up doing really well with it in the future as a Software as a Service company; it was dead on arrival. Dumb idea; bring it back; that actually became BizSpark, the most popular program in Microsoft partner history.And then about three months later, I got a call from this guy, Bill Hilf. And he said, “Hey, this is Bill Hilf. I do open-source at Microsoft. I work with Bill Gates. He sent me your paper. I really like it. Would you consider coming up and having conversation with me because I want you to think about running open-source technology strategy for the company.” And at this time I'm, like, 33 or 34. And I'm like, “Who me? You've got to be joking.” And he goes, “Oh, and also, you'll be responsible for doing quarterly deep technical briefings with Bill… Gates.” I was like, “You must be kidding.” And so of course I had to check it out. One thing led to another and all of a sudden, with not a lot of history in the open-source community but coming in it with a strategist's eye and with a technologist's eye, saying, “This is a problem we got to solve. How do we get after this pragmatically?” And the rest is history, as they say.Corey: I have to say that you are the Chief Strategy Officer at DataStax, and I pull up your website quickly here and a lot of what I tell earlier stage companies is effectively more or less what you have already done. You haven't named yourself after the open-source project that underlies the bones of what you have built so you're not going to wind up in the same glorious challenges that, for example, Elastic or MongoDB have in some ways. You have a pricing page that speaks both to the reality of, “It's two in the morning. I'm trying to get something up and running and I want you the hell out of my way. Just give me something that I can work with a reasonable free tier and don't make me talk to a salesperson.” But also, your enterprise tier is, “Click here to talk to a human being,” which is speaking enterprise slash procurement slash, oh, there will be contract negotiation on these things.It's being able to serve different ends of your market depending upon who it is that encounters you without being off-putting to any of those. And it's deceptively challenging for companies to pull off or get right. So clearly, you've learned lessons by doing this. That was the big problem with Microsoft for the longest time. It's, if I want to use some Microsoft stuff, once you were able to download things from the internet, it changed slightly, but even then it was one of those, “What exactly am I committing to here as far as signing up for this? And am I giving them audit rights into my environment? Is the BSA about to come out of nowhere and hit me with a surprise audit and find out that various folks throughout the company have installed this somewhere and now I owe more than the company's worth?” That was always the haunting fear that companies had back then.These days, I like the approach that companies are taking with the SaaS offering: you pay for usage. On some level, I'd prefer it slightly differently in a pay-per-seat model because at least then you can predict the pricing, but no one is getting surprise submarined with this type of thing on an audit basis, and then they owe damages and payment in arrears and someone has them over a barrel. It's just, “Oh. The bill this month was higher than we expected.” I like that model I think the industry does, too.Sam: I think that's super well said. As I used to joke at BEA Systems, nothing says ‘I love you' to a customer like an audit, right? That's kind of a one-time use strategy. If you're going to go audit licenses to get your revenue in place, you might be inducing some churn there. It's a huge fix for the structural problem in pricing that I think package software had, right?When we looked at Microsoft software versus open-source software, and particularly Windows versus Linux, you would have a structure where sales reps were really compensated to sell as much as possible upfront so they could get the best possible commission on what might be used perpetually. But then if you think about it, like, the boxes in a curve, right, if you do that calculus approximation of a smooth curve, a perpetual software license is a huge box and there's an enormous amount of waste in there. And customers figured out so as soon as you can go to a pay-per-use or pay-as-you-go, you start to smooth that curve, and now what you get is what you deserve, right, as opposed to getting filled with way more cost than you expect. So, I think this model is really super well understood now. Kind of the long run the high point of open-source meets, cloud, meets Software as a Service, you look at what companies like MongoDB, and Confluent, and Elastic, and Databricks are doing. And they've really established a very good path through the jungle of how to succeed as a software company. So, it's still difficult to implement, but there are really world-class guides right now.Corey: Moving beyond where Microsoft was back in the naughts, you were then hired as a VP over at Google. And in that era, the fact that you were hired as a VP at Google is fascinating. They preferred to grow those internally, generally from engineering. So, first question, when you were being hired as a VP in the product org, did they make you solve algorithms on a whiteboard to get there?Sam: [laugh]. They did not. I did have somewhat of an advantage [because they 00:13:36] could see me working pretty closely as the CEO of the Cloud Foundry Foundation. I'd worked closely with Craig McLuckie who notably brought Kubernetes to the world along with Joe Beda, and with Eric Brewer, and a number of others.And he was my champion at Google. He was like, “Look, you know, we need him doing Kubernetes. Let's bring Sam in to do that.” So, that was helpful. I also wrote a [laugh] 2000-word strategy document, just to get some thoughts out of my head. And I said, “Hey, if you like this, great. If you don't throw it away.” So, the interviews were actually very much not solving problems in a whiteboard. There were super collaborative, really excellent conversations. It was slow—Corey: Let's be clear, Craig McLuckie's most notable achievement was being a guest on this podcast back in Episode 243. But I'll say that this is a close second.Sam: [laugh]. You're not wrong. And of course now with Heptio and their acquisition by VMware.Corey: Ehh, they're making money beyond the wildest dreams of avarice, that's all well and good, but an invite to this podcast, that's where it's at.Sam: Well, he should really come on again, he can double down and beat everybody. That can be his landmark achievement, a two-timer on Screaming in [the] Cloud.Corey: You were at Google; you were at Microsoft. These are the big titans of their era, in some respect—not to imply that there has beens; they're bigger than ever—but it's also a more crowded field in some ways. I guess completing the trifecta would be Amazon, but you've had the good judgment never to work there, directly of course. Now they're clearly in your market. You're at DataStax, which is among other things, built on Apache Cassandra, and they launched their own Cassandra service named Keyspaces because no one really knows why or how they name things.And of course, looking under the hood at the pricing model, it's pretty clear that it really is just DynamoDB wearing some Groucho Marx classes with a slight upcharge for API level compatibility. Great. So, I don't see it a lot in the real world and that's fine, but I'm curious as to your take on looking at all three of those companies at different eras. There was always the threat in the open-source world that they are going to come in and crush you. You said earlier that Microsoft crushed your first startup.Google is an interesting competitor in some respects; people don't really have that concern about them. And your job as a Chief Strategy Officer at Amazon is taken over by a Post-it Note that simply says ‘yes' on it because there's nothing they're not going to do, or try, and experiment with. So, from your perspective, if you look at the titans, who is it that you see as the largest competitive threat these days, if that's even a thing?Sam: If you think about Sun Tzu and the Art of War, right—a lot of strategy comes from what we've learned from military environments—fighting a symmetric war, right, using the same weapons and the same army against a symmetric opponent, but having 1/100th of the personnel and 1/100th of the money is not a good plan.Corey: “We're going to lose money, going to be outcompeted; we'll make it up in volume. Oh, by the way, we're also slower than they are.”Sam: [laugh]. So, you know, trying to come after AWS, or Microsoft, or Google as an independent software company, pound-for-pound, face-to-face, right, full-frontal assault is psychotic. What you have to do, I think, at this point is to understand that these are each companies that are much like we thought about Linux, and you know, Macintosh, and Windows as operating systems. They're now the operating systems of the planet. So, that creates some economies of scale, some efficiencies for them. And for us. Look at how cheap object storage is now, right? So, there's never been a better time in human history to create a database company because we can take the storage out of the database and hand it over to Amazon, or Google, or Microsoft to handle it with 13 nines of durability on a constantly falling cost basis.So, that's super interesting. So, you have to prosecute the structure of the world as it is, based on where the giants are and where they'll be in the future. Then you have to turn around and say, like, “What can they never sell?”So, Amazon can never sell something that is standalone, right? They're a parts factory and if you buy into the Amazon-first strategy of cloud computing—which we did at Autodesk when I was VP of cloud platform there—everything is a primitive that works inside Amazon, but they're not going to build things that don't work outside of the Amazon primitives. So, your company has to be built on the idea that there's a set of people who value something that is purpose-built for a particular use case that you can start to broaden out, it's really helpful if they would like it to be something that can help them escape a really valuable asset away from the center of gravity that is a cloud. And that's why data is super interesting. Nobody wakes up in the morning and says, “Boy, I had such a great conversation with Oracle over the last 20 years beating me up on licensing. Let me go find a cloud vendor and dump all of my data in that so they can beat me up for the next 20 years.” Nobody says that.Corey: It's the idea of data portability that drives decision-making, which makes people, of course, feel better about not actually moving in anywhere. But the fact that they're not locked in strategically, in a way that requires a full software re-architecture and data model rewrite is compelling. I'm a big believer in convincing people to make decisions that look a lot like that.Sam: Right. And so that's the key, right? So, when I was at Autodesk, we went from our 100 million dollar, you know, committed spend with 19% discount on the big three services to, like—we started realize when we're going to burn through that, we were spending $60 million or so a year on 20% annual growth as the cloud part of the business grew. Thought, “Okay, let's renegotiate. Let's go and do a $250 million deal. I'm sure they'll give us a much better discount than 19%.” Short story is they came back and said, “You know, we're going to take you from an already generous 19% to an outstanding 22%.” We thought, “Wait a minute, we already talked to Intuit. They're getting a 40% discount on a $400 million spend.”So, you know, math is hard, but, like, 40% minus 22% is 18% times $250 million is a lot of money. So, we thought, “What is going on here?” And we realized we just had no credible threat of leaving, and Intuit did because they had built a cross-cloud capable architecture. And we had not. So, now stepping back into the kind of the world that we're living in 2021, if you're an independent software company, especially if you have the unreasonable advantage of being an open-source software company, you have got to be doing your customers good by giving them cross-cloud capability. It could be simply like the Amdahl coffee cup that Amdahl reps used to put as landmines for the IBM reps, later—I can tell you that story if you want—even if it's only a way to save money for your customer by using your software, when it gets up to tens and hundreds of million dollars, that's a really big deal.But they also know that data is super important, so the option value of being able to move if they have to, that they have to be able to pull that stick, instead of saying, “Nice doggy,” we have to be on their side, right? So, there's almost a detente that we have to create now, as cloud vendors, working in a world that's invented and operated by the giants.Corey: This episode is sponsored by our friends at Oracle HeatWave is a new high-performance accelerator for the Oracle MySQL Database Service. Although I insist on calling it “my squirrel.” While MySQL has long been the worlds most popular open source database, shifting from transacting to analytics required way too much overhead and, ya know, work. With HeatWave you can run your OLTP and OLAP, don't ask me to ever say those acronyms again, workloads directly from your MySQL database and eliminate the time consuming data movement and integration work, while also performing 1100X faster than Amazon Aurora, and 2.5X faster than Amazon Redshift, at a third of the cost. My thanks again to Oracle Cloud for sponsoring this ridiculous nonsense.Corey: When we look across the, I guess, the ecosystem as it's currently unfolding, a recurring challenge that I have to the existing incumbent cloud providers is they're great at offering the bricks that you can use to build things, but if I'm starting a company today, I'm not going to look at building it myself out of, “Ooh, I'm going to take a bunch of EC2 instances, or Lambda functions, or popsicles and string and turn it into this thing.” I'm going to want to tie together things that are way higher level. In my own case, now I wind up paying for Retool, which is, effectively, yeah, it runs on some containers somewhere, presumably, I think in Azure, but don't quote me on that. And that's great. Could I build my own thing like that?Absolutely not. I would rather pay someone to tie it together. Same story. Instead of building my own CRM by running some open-source software on an EC2 instance, I wind up paying for Salesforce or Pipedrive or something in that space. And so on, and so forth.And a lot of these companies that I'm doing business with aren't themselves running on top of AWS. But for web hosting, for example; if I look at the reference architecture for a WordPress site, AWS's diagram looks like a punchline. It is incredibly overcomplicated. And I say this as someone who ran large WordPress installations at Media Temple many years ago. Now, I have the good sense to pay WP Engine. And on a monthly basis, I give them money and they make the website work.Sure, under the hood, it's running on top of GCP or AWS somewhere. But I don't have to think about it; I don't have to build this stuff together and think about the backups and the failover strategy and the rest. The website just works. And that is increasingly the direction that business is going; things commoditize over time. And AWS in particular has done a terrible job, in my experience, of differentiating what it is they're doing in the language that their customers speak.They're great at selling things to existing infrastructure engineers, but folks who are building something from scratch aren't usually in that cohort. It's a longer story with time and, “Well, we're great at being able to sell EC2 instances by the gallon.” Great. Are you capable of going to a small doctor's office somewhere in the American Midwest and offering them an end-to-end solution for managing patient data? Of course not. You can offer them a bunch of things they can tie together to something that will suffice if they all happen to be software engineers, but that's not the opportunity.So instead, other companies are building those solutions on top of AWS, capturing the margin. And if there's one thing guaranteed to keep Amazon execs awake at night, it's the idea of someone who isn't them making money somehow somewhere, so I know that's got to rankle them, but they do not speak that language. At all. Longer-term, I only see that as a more and more significant crutch. A long enough timeframe here, we're talking about them becoming the Centurylinks of the world, the tier one backbone provider that everyone uses, but no one really thinks about because they're not a household name.Sam: That is a really thoughtful perspective. I think the diseconomies of scale that you're pointing to start to creep in, right? Because when you have to sell compute units by the gallon, right, you can't care if it's a gallon of milk, [laugh] or a gallon of oil, or you know, a gallon of poison. You just have to keep moving it through. So, the shift that I think they're going to end up having to make pragmatically, and you start to see some signs of it, like, you know, they hired but could not retain Matt [Acey 00:23:48]. He did an amazing job of bringing them to some pragmatic realization that they need to partner with open-source, but more broadly, when I think about Microsoft in the 2000s as they were starting to learn their open-source lessons, we were also being able to pull on Microsoft's deep competency and partners. So, most people didn't do the math on this. I was part of the field governance council so I understood exactly how the Microsoft business worked to the level that I was capable. When they had $65 billion in revenue, they produced $24 billion in profit through an ecosystem that generated $450 billion in revenue. So, for every dollar Microsoft made, it was $8 to partners. It was a fundamentally platform-shaped business, and that was how they're able to get into doctors offices in the Midwest, and kind of fit the curve that you're describing of all of those longtail opportunities that require so much care and that are complex to prosecute. These solved for their diseconomies of scale by having 1.2 million partner companies. So, will Amazon figure that out and will they hire, right, enough people who've done this before from Microsoft to become world-class in partnering, that's kind of an exercise left to the [laugh] reader, right? Where will that go over time? But I don't see another better mathematical model for dealing with the diseconomies of scale you have when you're one of the very largest providers on the planet.Corey: The hardest problem as I look at this is, at some point, you hit a point of scale where smaller things look a lot less interesting. I get that all the time when people say, “Oh, you fix AWS bills, aren't you missing out by not targeting Google bills and Azure bills as well?” And it's, yeah. I'm not VC-backed. It turns out that if I limit the customer base that I can effectively service to only AWS customers, yeah turns out, I'm not going to starve anytime soon. Who knew? I don't need to conquer the world and that feels increasingly antiquated, at least going by the stories everyone loves to tell.Sam: Yeah, it's interesting to see how cloud makes strange bedfellows, right? We started seeing this in, like, 2014, 2015, weird partnerships that you're like, “There's no way this would happen.” But the cloud economics which go back to utilization, rather than what it used to be, which was software lock-in, just changed who people were willing to hang out with. And now you see companies like Databricks going, you know, we do an amazing amount of business, effectively competing with Amazon, selling Spark services on top of predominantly Amazon infrastructure, and everybody seems happy with it. So, there's some hint of a new sensibility of what the future of partnering will be. We used to call it coopetition a long time ago, which is kind of a terrible word, but at least it shows that there's some nuance in you can't compete with everybody because it's just too hard.Corey: I wish there were better ways of articulating these things because it seems from the all the outside world, you have companies like Amazon and Microsoft and Google who go and build out partner networks because they need that external accessibility into various customer profiles that they can't speak to super well themselves, but they're also coming out with things that wind up competing directly or indirectly, with all of those partners at the same time. And I don't get it. I wish that there were smarter ways to do it.Sam: It is hard to even talk about it, right? One of the things that I think we've learned from philosophy is if we don't have a word for it, we can't be intelligent about it. So, there's a missing semantics here for being able to describe the complexity of where are you partnering? Where are you competing? Where are you differentiating? In an ecosystem, which is moving and changing.I tend to look at the tools of game theory for this, which is to look at things as either, you know, nonzero-sum games or zero-sum games. And if it's a nonzero-sum game, which I think are the most interesting ones, can you make it a positive sum game? And who can you play positive-sum games with? An organization as big as Amazon, or as big as Microsoft, or even as big as Google isn't ever completely coherent with itself. So, thinking about this as an independent software company, it doesn't matter if part of one of these hyperscalers has a part of their business that competes with your entire business because your business probably drives utilization of a completely different resource in their company that you can partner within them against them, effectively. Right?For example, Cassandra is an amazingly powerful but demanding workload on Kubernetes. So, there's a lot of Cassandra on EKS. You grow a lot of workload, and EKS business does super well. Does that prevent us from working with Amazon because they have Dynamo or because they have Keyspaces? Absolutely not, right?So, this is when those companies get so big that they are almost their own forest, right, of complexity, you can kind of get in, hang out, do well, and pretty much never see the competitive product, unless you're explicitly looking for it, which I think is a huge danger for us as independent software companies. And I would say this to anybody doing strategy for an organization like this, which is, don't obsess over the tiny part of their business that competes with yours, and do not pay attention to any of the marketing that they put out that looks competitive with what you have. Because if you can't figure out how to make a better product and sell it better to your customers as a single purpose corporation, you have bigger problems.Corey: I want to change gears slightly to something that's probably a fair bit more insulting, but that's okay. We're going to roll with it. That seems to be the theme of this episode. You have been, in effect, a CIO a number of times at different companies. And if we take a look at the typical CIO tenure, industry-wide, it's not long; it approaches the territory from an executive perspective of, “Be sure not to buy green bananas. You might not be here by the time they ripen.” And I'm wondering what it is that drives that and how you make a mark in a relatively short time frame when you're providing inputs and deciding on strategy, and those decisions may not bear fruit for years.Sam: CIO used to—we used say it stood for ‘Career Is Over' because the tenure is so short. I think there's a couple of reasons why it's so short. And I think there's a way I believe you can have impact in a short amount of time. I think the reason that it's been short is because people aren't sure what they want the CIO role to be.Do they want it to be a glorified finance person who's got a lot of data processing experience, but now really has got, you know, maybe even an MBA in finance, but is not focusing on value creation? Do they want it to be somebody who's all-singing, all-dancing Chief Data Officer with a CTO background who did something amazing and solved a really hard problem? The definition of success is difficult. Often CIOs now also have security under them, which is literally a job I would never ever want to have. Do security for a public corporation? Good Lord, that's a way to lose most of your life. You're the only executive other than the CEO that the board wants to hear from. Every sing—Corey: You don't sleep; you wait, in those scenarios. And oh, yeah, people joke about ablative CSOs in those scenarios. Yeah, after SolarWinds, you try and get an ablative intern instead, but those don't work as well. It's a matter of waiting for an inevitability. One of the things I think is misunderstood about management broadly, is that you are delegating work, but not the responsibility. The responsibility rests with you.So, when companies have these statements blaming some third-party contractor, it's no, no, no. I'm dealing with you. You were the one that gave my data to some sketchy randos. It is your responsibility that data has now been compromised. And people don't want to hear that, but it's true.Sam: I think that's absolutely right. So, you have this high risk, medium reward, very fungible job definition, right? If you ask all of the CIO's peers what their job is, they'll probably all tell you something different that represents their wish list. The thing that I learned at Autodesk, I was only there for 15 months, but we established a fundamental transformation of the work of how cloud platform is done at the company that's still in place a couple years later.You have to realize that you're a change agent, right? You're actually being hired to bring in the bulk of all the different biases and experiences you have to solve a problem that is not working, right? So, when I got to Autodesk, they didn't even know what their uptime was. It took three months to teach the team how to measure the uptime. Turned out the uptime was 97.7% for the cloud, for the world's largest engineering software company.That is 200 hours a year of unplanned downtime, right? That is not good. So, a complete overhaul [laugh] was needed. Understanding that as a change agent, your half-life is 12 to 18 months, you have to measure success not on tenure, but on your ability to take good care of the patient, right? It's going to be a lot of pain, you're going to work super hard, you're going to have to build trust with everyone, and then people are still going to hate you at the end. That is something you just have to kind of take on.As a friend of mine, Jason Warner joined Redpoint Ventures recently, he said this when he was the CTO of GitHub: “No one is a villain in their own story.” So, you realize, going into a big organization, people are going to make you a villain, but you still have to do incredibly thoughtful, careful work, that's going to take care of them for a long time to come. And those are the kinds of CIOs that I can relate to very well.Corey: Jason is great. You're name-dropping all the guests we've had. My God, keep going. It's a hard thing to rationalize and wrap heads around. It's one of those areas where you will not be measured during your tenure in the role, in some respects. And, of course, that leads to the cynical perspective as well, where well, someone's not going to be here long and if they say, “Yeah, we're just going to keep being stewards of the change that's already underway,” well, that doesn't look great, so quick, time to do a cloud migration, or a cloud repatriation, or time to roll something else out. A bit of a different story.Sam: One of the biggest challenges is how do you get the hearts and the minds of the people who are in the organization when they are no fools, and their expectation is like, “Hey, this company's been around for decades, and we go through cloud leaders or CIOs, like Wendy's goes through hamburgers.” They could just cloud-wash, right, or change-wash all their language. They could use the new language to describe the old thing because all they have to do is get through the performance review and outwait you. So, there's always going to be a level of defection because it's hard to change; it's hard to think about new things.So, the most important thing is how do you get into people's hearts and minds and enable them to believe that the best thing they could do for their career is to come along with the change? And I think that was what we ended up getting right in the Autodesk cloud transformation. And that requires endless optimism, and there's no room for cynicism because the cynicism is going to creep in around the edges. So, what I found on the job is, you just have to get up every morning and believe everything is possible and transmit that belief to everybody.So, if it seems naive or ingenuous, I think that doesn't matter as long as you can move people's hearts in each conversation towards, like, “Oh, this person cares about me. They care about a good outcome from me. I should listen a little bit more and maybe make a 1% change in what I'm doing.” Because 1% compounded daily for a year, you can actually get something done in the lifetime of a CIO.Corey: And I think that's probably a great place to leave it. If people want to learn more about what you're up to, how you think about these things, how you view the world, where can they find you?Sam: You can find me on Twitter, I'm @sramji, S-R-A-M-J-I, and I have a podcast that I host called Open||Source||Datawhere I invite innovators, data nerds, computational networking nerds to hang out and explain to me, a software programmer, what is the big world of open-source data all about, what's happening with machine learning, and what would it be like if you could put data in a container, just like you could put code in a container, and how might the world change? So, that's Open||Source||Data podcast.Corey: And we'll of course include links to that in the [show notes 00:35:58]. Thanks so much for your time. I appreciate it.Sam: Corey, it's been a privilege. Thank you so much for having me.Corey: Likewise. Sam Ramji, Chief Strategy Officer at DataStax. I'm Cloud Economist Corey Quinn, and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice, along with a comment telling me exactly which item in Sam's background that I made fun of is the place that you work at.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.

North Meets South Web Podcast
Industrial shifts, polymorphism, and single table inheritance

North Meets South Web Podcast

Play Episode Listen Later Dec 6, 2021 51:05


In this episode, Jake and Michael discuss Michael's change in job (again), and when you might choose single table inheritance over polymorphism.This episode is sponsored by Workvivo and Makeable.dk and was streamed live.Show links No Compromises Laravel test generator Parental Pushing polymorphism to the database

Screaming in the Cloud
Handling Time-Series Data with Brian Mullen

Screaming in the Cloud

Play Episode Listen Later Dec 1, 2021 31:40


About BrianBrian is an accomplished dealmaker with experience ranging from developer platforms to mobile services. Before InfluxData, Brian led business development at Twilio. Joining at just thirty-five employees, he built over 150 partnerships globally from the company's infancy through its IPO in 2016. He led the company's international expansion, hiring its first teams in Europe, Asia, and Latin America. Prior to Twilio Brian was VP of Business Development at Clearwire and held management roles at Amp'd Mobile, Kivera, and PlaceWare.Links:InfluxData: https://www.influxdata.com TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: This episode is sponsored in part by my friends at ThinkstCanary. Most companies find out way too late that they've been breached. ThinksCanary changes this and I love how they do it. Deploy canaries and canary tokens in minutes and then forget about them. What's great is the attackers tip their hand by touching them, giving you one alert, when it matters. I use it myself and I only remember this when I get the weekly update with a “we're still here, so you're aware” from them. It's glorious! There is zero admin overhead  to this, there are effectively no false positives unless I do something foolish. Canaries are deployed and loved on all seven continents. You can check out what people are saying at canary.love. And, their Kub config canary token is new and completely free as well. You can do an awful lot without paying them a dime, which is one of the things I love about them. It is useful stuff and not an, “ohh, I wish I had money.” It is speculator! Take a look; that's canary.love because it's genuinely rare to find a security product that people talk about in terms of love. It really is a unique thing to see. Canary.love. Thank you to ThinkstCanary for their support of my ridiculous, ridiculous nonsense.   Corey: Writing ad copy to fit into a 30 second slot is hard, but if anyone can do it the folks at Quali can. Just like their Torque infrastructure automation platform can deliver complex application environments anytime, anywhere, in just seconds instead of hours, days or weeks. Visit Qtorque.io today and learn how you can spin up application environments in about the same amount of time it took you to listen to this ad.Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. This promoted guest episode is brought to us by our friends at InfluxData. And my guest is titled as the Chief Marketing Officer at InfluxData, and I don't even care because his bio has something absolutely fascinating that I want to address instead. Brian Mullen is an accomplished dealmaker is how the bio starts. And so many of us spend time negotiating deals, but so few people describe ourselves in that way. First, Brian, thank you for joining us. And secondly, what's up with that?Brian: [laugh]. Well, thanks, Corey, very excited to be here. And yes, dealmaker; I guess that would be apropos. How did I get into marketing? Well, a lot of my career is spent in business development, and so I think that's where the dealmaker part comes from.Several different roles, including my first role at Influx—when I joined Influx—was in business development and partnerships. And so, prior to coming to Influx, I spent many years building out the business development team at Twilio, growing that up, and we did a lot of deals with carriers, with Cloud partners, with all kinds of different partners; you name it, we worked with them. And then moving into Influx, joined in an BD capacity here and had a couple different roles that eventually evolved to Chief Marketing Officer. But  that's where the dealmaker comes from. I like to do deals, it's always nice to have one on the side   in whatever capacity you're working in, it's nice to have a deal or two working on the side. It kind of keeps you fresh.Corey: It's fun because people think, “Oh, a deal. You're thinking of mergers and acquisitions, and how hard could that be? You just show up with a bag of money and give it to people and then you have a deal closed.” And oh, if only it were that simple. Every client engagement we have on the consulting side has been a negotiation back and forth, and the idea is to ideally get everyone to the point where they're happy, but honestly, if everyone's slightly unhappy but can live with the result, we'll take that too.And as people go through their own careers it's, you're always trying to make a deal in some form: when you try to get a project approved, or you're trying to get resources thrown at something—by which I generally mean money, not people, though people, too—it's something that isn't necessarily clearly understood or discussed very often, despite the fact that half of what I do is negotiating with AWS on behalf of clients for better contractual terms. The thing that I think takes people by surprise the most is that dealmaking is almost never about pounding the table, being angry, and walking out, like you read the world's worst guide to buying a car or something. It's about finding the win for everyone. At least that's the way I've always approached it.Brian: That's a good point. And actually that wording that you described of finding a win for everybody, that's how I always thought about it. I think about it as first of all, you're trying to understand what the other party—and it could be an individual, it could be a company, it could be a group of companies, sometimes—you're trying to understand what their goals are, what their agenda is and see how that matches with your own; sometimes they're opposing, sometimes they're overlapping. And then everyone has to have some perceived win  in a deal. And it's not competitively; it's more like you just have to have value, that is kind of what the win is – having value in that deal.And so that's the way I always approached it. And doing deals, whether you're in BD or sales, or if you're working with vendors and you're in a different functional role, sometimes it's not even commercial, it's just about aligning resources, perhaps. Our deal might be that you and I are both going to put a collective effort into building something or taking something to market. In another scenario might be like, I'm going to pay for this service that you're delivering, or vice versa. Or we're going to go and bring two revenue-generating products together and take them to market. Whatever it might be, it doesn't matter so much what the mechanics are of the deal, but it's usually about aligning those agendas and in having someone get utility, get value on the other side.Corey: I think that people lose sight of the fact as well, that when you're talking about a service provider—and let's be clear, InfluxData has launched a cloud platform that we'll talk about in a minute—this is not the one-off transactional relationship; once the deal is signed, you've got to work with these people. When they host parts of your production infrastructure, whether you want to admit it or not they're your partner more so than they are your vendor. It has to be an ongoing relationship that people are, if they at least aren't thrilled with it, can at least be happy enough to live with, otherwise it just winds up with this growing sense of resentment and it just sort of leads nowhere.Brian: Yeah, there really is no deal moment. Yes, people sign agreements with companies, but that's just the very beginning. Your relationship evolves from there. We're delivering a product, we're delivering this platform that handles time-series data to our customers, and we're asking them to trust us with their product that they're taking out to market. They're asking us to handle their data and to deliver service to them that they're turning into their production applications. And so it's a big responsibility. And so we care about the relationship with our customers to continue that.Corey: So, I first really became aware of time-series data a few years back during a re:Invent keynote when they pre-announced Timestream, which took entirely too long to come to market. Okay, great. So, you're talking about time-series data. Can you explain what that means in simple terms? And I learned over the next eight minutes that they were talking about it, that no, no, they couldn't. I wound up more confused by the end of the announcement than I was at the beginning.So, assuming that I have the same respect for databases as you would expect for someone whose favorite data store is Route 53—because you can misuse it as a beautiful database—what is time-series data and why does it matter in 2021?Brian: Sure, it's a good question. And I was there in that audience as well that day. So, we think of time-series data as really any type of data that's stamped in time, in some way. It could be every hour, every minute, every second, every half second, whatever. But more specifically, it's any type of data that is generated by some source—and that could be a sensor sources within systems or an actual application—and these things change over time, and then therefore, stamped in time in some way.They can come at different frequencies, like I said, from nanoseconds to seconds, or minutes and hours, but the most important thing is that they usually trigger a workflow, trigger some sort of action. And so that's really what our platform is about. It allows people to handle this type of data and then work with it from there in their applications, trigger new workflows, et cetera. Because the historical context of what happens is super important.And when we talk about sources, it could be really many things. It could be in physical spaces, and we have a lot of IoT types of customers and use cases. And those are things like devices and sensors on the factory floor, out in the field, it's on a vehicle. It's even in space, believe it or not. There are customers that are using us on satellites.And then it can also be sources from within software, applications, and infrastructure, things like VMs, and containers, and microservices, all emitting time-series data. And it could be applications like crypto, or financial, or stock market, agricultural type of applications that are themselves as applications emitting data. So, you think about all these sources that are out there from the physical world to the virtual world, and they're all generating time-series data, and our platform is really specially designed to handle that kind of data. And we can get into some details of what exactly that means, but that's really why we're here. That's what time-series is all about.Corey: And this is the inherent challenge I think we're seeing across the entire industry slash ecosystem. I mean, this is airing during re:Invent week, but at the time we are recording this, we have not yet seen the Tuesday keynote that Adam Selipsky will take to the stage, and no doubt, render the stat I'm about to throw at you completely obsolete. But depending on how you count them, there's somewhere between 13 and 15 managed database or database-like services today that AWS offers. And they never turn things off and they're always releasing new things, supposedly on behalf of customers; in practice because someone somewhere wants to get promoted by launching a new service; good for them. Godspeed.If we look into the uncertain future, at some point, someone's job is going to be disambiguating between the 40 different managed database services that AWS offers and picking the one that works. What differentiates time-series from—let's just start with an easy one—something like MySQL or Postgres—or ‘Postgres-squeal' is how I insist on pronouncing that one. Let's stay away from things like Neptune because no one knows what a social graph database is and I assure you, you almost certainly don't need one. Where does something like Influx work in a way that, “Huh. Running this on MySQL is really starting to suck.”Brian: When and why is it time to consider a specialized tool. And in fact, that's actually what we see a lot with our customers is coming to us around that time when a time-series is a problem to solve for them is reaching the point where they really need a specialized tool that's kind of built for that. And so one way to look at that is really just to think about time-series in general as a type of data. It's rapidly rising. It's the fastest growing data category out there right now.And the reason for that is it's being driven by two big macro trends. One is the explosion of all these applications and services running in the cloud. They're expanding horizontally, they're running in more regions, they're in many cases running on multiple clouds, and so it's just getting big—the workloads are getting bigger and bigger. And those are emitting time-series data. And then simultaneously, you have this  growth of all these devices and sensors that are coming online out in the real world: batteries, and temperature gauges, and all kinds of stuff, both new and old, that is coming online, and those sources are generating a lot of time-series data.So typically, we're in a moment now, where a lot of developers are faced with this massive growth of time-series data. And if you think about some data set that you have, that you're putting into some kind of traditional database, now add the component of time as a multiplier by all the data you have. Instead of that one data, that one metric, you're now looking at doing that every one second in perpetuity. And so it's just an order of magnitude more data that you're dealing with. And then you also have this notion of—when you have that magnitude of data, you have fidelity, you're taking a lot of it in at the same time, I mean, very quickly, so you have  batch or stream data coming in at super high volume, and you may need that for a few minutes or a few hours or days, but maybe you don't need it for months and years.And so you'd maybe dropped down to kind of a lower fidelity for the longer-term. But you really have this  toggling back and forth of the high fidelity and low fidelity, all coming at you at pretty high volume. And so typically what happens is, is when the workloads get big enough, the legacy tools, they're just not equipped to do it. And a developer—if they have a small set of time-series they're dealing with, what is the first thing they're going to do? They're going to look around and be like, “Hey, what do I have here? Oh, I've got Mongo over here. I've got Splunk, or I've got this old relational database, I can put it in.”And that's typically what they'll do, and that works fine until it doesn't. And then that's when they come around looking for a specialized tool. So, we really sit in Influx and, frankly, other time-series products really do sit at that point where people are considering a specialized tool just because the workload has gotten such that it requires that.Corey: Yeah. Taking a look at most of the offerings in the space; anything that winds up charging anything more than a very tiny fraction of a penny—from what you're describing—is going to quickly become non-economical, where it's, “Oh, we're going to charge you”—like using S3: every, I think, 1000 writes cost a penny—“Oh, we're just going to use S3 for this.” Well, at some of these data volumes, that means that your request charge on S3 is very quickly going to become the largest single line item in your bill, which is nothing short of impressive in a lot of cases, but it also probably means that you've taken a very specific tool—like an iPad—and tried to use it as something else—like a hammer—and no one's particularly happy with that outcome.Brian: Yeah. First of all, having usage-based pricing is really important. We think about it as allowing people to have the full version of the product without a major commitment, and be using it in test scenarios and then later in the very early production scenarios. But as a principle, it's important for people that just signed up two hours ago using your product are basically using the same full product that the biggest customers that you have are using that are paying many, many thousands or tens of thousands per month. And so the way to do that is to offer usage-based pricing and not force people to commit to something before they're ready to do it.And so there's ways to unlock lower pricing, and we, like a lot of companies, offer annual pricing and we have a sales team that worked with folks to basically draw down their unit costs on the use of the platform once they kind of get comfortable with their workload. So, there's definitely avenues to get lower price, and we're believers in that. And we also want to, from a product development perspective, try to make the product more efficient. And so we basically are trying to drive down the costs through efficiencies in the product: make it run faster, make queries take less time, and also ship products on top of it that require developers to write less code themselves, kind of, do more of the work for them.Corey: One of the things I find particularly compelling about what you've done is it is an open-source project. If I want to go ahead and run some time-series experiments myself, I can spin it up anywhere I want and run it however I see fit. Now, at some point, if I'm doing this for anything more than, “Oh, let's see how I can misuse this today,” I probably want to at least consider letting someone who's better at running these things than I am take it over. And as I'm looking through your customer list, the thing that strikes me is how none of these things are quite like the other. We're talking about companies like Hulu is probably not using it the same way as Capital One is, at least I certainly hope not. You have Texas Instruments; you also have Adobe. And it sort of runs an entire gamut of none of these companies quite look alike; I have to imagine their use cases are also somewhat varied, too.Brian: Yeah, that's right. And we really do see as a platform, and with time-series being the common problem that people are looking to solve, we see this pretty broad set of use cases and customer types. And we have some more traditional customers like the Cisco's and the IBM's of the world, and then some  relatively new folks like Tesla and Hulu and others that are a little bit more recent. But they're all trying to solve the same fundamental problem with time-series, which is “How can I handle it in an efficient way and make use of it meaningfully in my applications and services?”And we were talking earlier about having some sources of time-series data being in, kind of a virtual space, like in infrastructure and software, and then some being in physical space, like in devices and sensors out in the real world. So, we have breadth in that way, too. We have folks who are building big software observability infrastructure solutions on us, and we also have people that are pulling data off of the devices on a solar panel that's sitting on a house in the emerging world, right? So, you have basically these two far ends of the spectrum, but all using this specialized tool to handle the time-series data that they're generating.Corey: It seems to me that for most of these use cases and the way you describe it, it's more about the overall shape of the data when we're talking about time-series more so than it is any particular data point in isolation. Is that accurate, or are there cases where that is very much not the case?Brian: I think that's accurate. What people are mostly trying to understand is context for what's happening. And so it's not necessarily—to your point—not searching for one specific data point or moment, but it's really understanding context for some general state that has changed or some trend that has emerged, whatever that might be, and then making sense of that, and then taking action on that. And taking an action could mean a couple of different things, too. It could be in an observability sense, where somebody in  an operator type of mode where they're looking at dashboards and paying attention to  infrastructure that's running and then need to take some sort of action based on that. It also, in many cases, is automated in some way: it's either some series of automated responses to some state that is reached that is visible in the data, or is actually kicking off some new series of tasks or actions inside of an application based on what is occurring and shown by the time-series data.Corey: You know what doesn't add to your AWS bill? Free developer security from Snyk. Snyk is a frictionless security platform that meets developers where they are, finding and fixing vulnerabilities right from the CLI, IDEs, repos, and pipelines. And Snyk integrates seamlessly with AWS offerings like CodePipeline, EKS, ECR, and oh so much more.Secure with Snyk and save some loot. Learn more at snyk.io/scream. That's S-N-Y-K-dot-I-O/screamCorey: So, we've talked about, you have an open-source product, which is the sort of thing that most people listening to this should have a vague idea of, “Oh, that means I can go on GitHub and download it and start using it, if it's not already in my package manager.” Great. You also have the enterprise offering, which is more or less, I presume, a supported distribution of this—for lack of a better term—that you then wind up providing blessed configurations thereof and helping run support for that—for companies that want to run it on-prem. Is that directionally accurate, or am I grossly mischaracterizing [laugh] what your enterprise offering is?Brian: Directionally accurate, of course. You could have a great job in marketing. I really think you could.Corey: Oh, you know, I would argue, on some level, I probably do. The challenge I have is that I keep conflating marketing with spectacle and that leads down to really unfortunate, weird places. But one additional area, which is relatively recent since the last time I spoke with Paul—one of the cofounders of your company—on this show is InfluxDB Cloud, which is one of those, “Oh, let me see if I look—if I'm right.” And sure enough, yeah, you wind up managing the infrastructure for us and it becomes a pay-per consumption model the way that most cloud service providers do, without the really obnoxious hidden 15 levels of billing dimensions.Brian: Yes, we are trying to bring the transparency back. But yes, you're correct. We have open-source and we have—it's very popular—we have over 500,000-plus instances of that deployed globally today in the community. And that's typically very common for developers to get started using the open-source, easily recognizable, it's been out for a long time, and so many people start the journey there.And then we have InfluxDB Enterprise, which it's actually a clustered version of InfluxDB open-source. So, it allows you to basically handle in an environment that you want to manage yourself, you manage a cluster and scale it out and handle ever-increasing workloads and have things like redundancy and replication, et cetera. But that's really specifically for people who want to deploy and operate the software themselves, which is a good set of people; we have a lot of folks who have done that. But one of the areas that's a little bit more recent is InfluxDB Cloud, which is really, for folks who don't want to have anything to do with the management; they really just want to use it as a service, send their data in—Corey: Yeah, give me an API endpoint, and I want you to worry about the care, and the feeding, and the waking up at two in the morning when a disk starts filling up. Yeah, that is the best kind of problem from my perspective: someone else's.Brian: Exactly. That's our job. And increasingly, we've seen folks gravitate to that. We've got a lot of folks have signed up on this product since it launched in 2019, and it's really increasingly where they begin their journey, maybe not even going to the open-source just going directly to this because it's relatively simple to get started.It's priced based on usage. People pay for three vectors: they have the amount of data in; they have number of queries made against the platform; and then storage, how much data you have and for how long. And depending on the use case, some people keep it around for relatively short time, like a few days or a couple of weeks. Other folks have it for many, many months and potentially years in some places. So, you really have that option.But I would say the three products are really about how you want to run it. Do you care about running the, kind of, underlying infrastructure and managing it or do you just want to hit an endpoint, as you said.Corey: You launched this, I want to say in 2019, which feels about directionally right. And I know it was after Timestream was announced, so I just want to say first, how kind and selfless it was of you to validate AWS's market, which is, you know how they always like to clarify and define what they're doing when they decide to enter every single market anywhere to compete with everyone. It turns out, I don't get the sense that they like it quite [laugh] as much being on the other side of that particular divide, but that's the best kind of problem, too: again, someone else's.Brian: Yeah, I think that's really true.Corey: The challenge that I have is that it seems like a weird direction to go in as a company, though it is clearly based upon a number of press releases you have made about the success and market traction that you found, it feels, on some level, like it is falling into an older version of an open-source trap of assuming that, “Well, we wrote the software therefore we are the best people you could pick to run it.” That was what a lot of companies did; it turns out that AWS has this operational excellence, as they call it, and what the rest of us call burning through people and making them wake up in the middle of the night to fix things before it becomes customer-visible. But from the outside, there's no difference. It seems, however, that you have built something that is clearly resonating, and in a big way, in a way that—I've got to be direct with you—the AWS time-series service that they are offering has not been finding success.Brian: Thank you for saying that, and we feel pretty excited about the success we've had even being in the same market as Amazon. And Amazon does a phenomenal job at running products at scale, and the breadth that they have in their product lineup is pretty impressive, especially when they roll out new stuff at AWS re:Invent every year. But we've been able to find some pretty good success with our approach, and it's based on a couple of things. So, one is being the company that actually develops and still deploys the open-source is really important. People gravitate to that.Our roots as a company are open-source, we've been a part of and fostered this community over many, many years, and there's a certain trust in the direction that we're taking the company. And Paul, our founder who you mentioned, he's been front and center with that community, pretty deeply engaged for many, many years. I think that carries a lot of weight. At least that's the way we think about it. But then as far as commercial products go, we really think about it as going to where our customers are, going to where developers are. And that could mean the language that they prefer, the language of preference for them. And that could [crosstalk 00:22:25]—Corey: Oh, and it's very clear; it seems that most database companies that I talk to—again, without naming names—tend to focus on the top-down sale, but I've never worked in an environment where the database that will be used was dictated by anyone other than the application developers who are the closest to the technical requirements for the workload. I've never understood this model of, “Oh, we're going to talk to the C suite because we believe that they're going to pick a database vendor based upon who has box seats this season.” I've never gotten that and that probably means I'm a terrible enterprise marketer, on some level. But unlike almost every other player in the database space, I've never struggled to understand what the hell your messaging has meant, other than the technical bits that I just don't have quite enough neurons to bang together to create sparks to fully understand. It is very clearly targeted at a builder rather than someone who's more or less spending their entire life in meetings. Which, oh, God, that's me.Brian: [laugh]. Yes, it's very much the case. We are focused on the developer. And that developer is a builder of an application or service that is seeing the light of day, it's going out and being used by their own end-users and end-customers.And so we care about going to where those developers are, and that could mean going and making your product easily used in the language and tool that customer cares about. So, if you're a Python developer, it's important for us to have tools and make it easy for Python developers. We have client libraries for Python, for example. It also means going to the cloud where your customers are. And this is something that differentiates us as well, when you start looking at what the other cloud providers are offering, in that data—like it or not—has gravity. And so somebody that has built their whole stack on AWS and sure they care about using a service that is going to receive their data, and that also being in AWS, but—Corey: It has to live where the customers are, especially with data egress charges being what they are, too.Brian: Exactly.Corey: And data gravity is real. The cloud provider people pick is the one where their data lives because of that particular inflection in the market.Brian: Absolutely true. And so that's great if you're only going after people who are on AWS, but what about Google Cloud and what about Microsoft Azure? There are a lot of developers that are building on those platforms as well, and that's one of the reasons we want to go there as well. So, InfluxDB Cloud is a multi-cloud offering, and it's equal experience and capability and pricing on each of the three major clouds. You can buy directly from us; you can put it on any of your cloud bills in one of those marketplaces, and to us that's like a really, really fundamental point is to bring your product and make it as easy to use on those platforms and in those languages, and in those realms and use cases where people are already working.Corey: I'm a big believer in multi-cloud for the use case you just defined. Because I know I'm going to get letters if I don't say this based upon my public multi-cloud is a dumb default worst practice for most folks—because it is, on a workload-by-workload basis—but you're building a service that has to be close to where your customers are and for that specific thing, yeah, it makes an awful lot of sense for you to have a presence across all the different providers. Now, here's the $64,000 question for you: is the experience as an InfluxDB Cloud customer meaningfully different between different providers?Brian: It's not. We actually pride ourselves on it being the same. Using InfluxDB, you sign up for InfluxDB Cloud, you come in, you set up your account, create your organization, and then you choose which underlying cloud provider you want your account to be provisioned in. And so it actually comes as a secondary choice; it's not something that is gated in the beginning, and that allows us to deliver a uniform experience across the board. And you may in a future use case, maybe somebody wants to have part of what they're building data living in AWS and maybe part of it living in Azure, I mean, that could be a scenario as well.However, typically what we've seen—and you've probably seen this as well—is  most developers are—and organizations—are building mostly on one cloud. I don't see a lot of  multi-cloud in that organization. But we ourselves need to be multi-cloud in order to go to where those people are working. And so that's the distinction. It's for us as a company that delivers product to those people, it's important for us to go where they are, whereas they themselves are not necessarily running on all three cloud products; they're probably running on one platform.Corey: Yeah. On a workload-by-workload basis, that's what generally makes sense. Anytime you have someone who has a particular workload that needs to be in multiple providers, okay, great, you're going to put that out there, but their backend systems, their billing, their marketing, all the rest, is not going to go down that path for a variety of excellent reasons, mostly that it is a colossal pain, and a bunch of, more or less, solving the same problems over and over, rather than the whole point of cloud being to make it someone else's. I want to thank you for taking so much time to speak to me about how you're viewing the evolution of the market, how you're seeing your move into cloud, and how you're effectively targeting folks who can actually care about the implementation details of a database rather than, honestly, suits. If people want to learn more, where can they find you?Brian: They can go to our website; it's the easiest place to go. So, influxdata.com. You can read all about InfluxDB, it's a pretty easy sign up to get underway. So, I recommend that people get their hands dirty with the product. That's the easiest way to understand what it's all about.Corey: And if you do end up doing that, please tell them I sent you because the involuntary flinch whenever people mention my name to vendors is one of my favorite parts of being me. Brian, thank you so much for being so generous with your time. I appreciate it.Brian: Thanks so much for having us on. It was great.Corey: Brian Mullen, Chief Marketing Officer—and dealmaker—at InfluxData. I'm Cloud Economist Corey Quinn, and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice along with a long, angry comment telling me that you work on the Timestream service team, and your product is the best. It's found huge success, but I've just never met any of your customers and I can't because they all live in Canada.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.

What's new in Cloud FinOps?
Episode 11 - September Cloud FinOps News

What's new in Cloud FinOps?

Play Episode Listen Later Nov 29, 2021 28:15


Steve and Frank are back with the latest cloud FinOps news for September! This episode covers the following topics: Amazon Redshift extends Automatic Table Optimization to support Column Compression EncodingAWS Cost Categories introduces Split Charge rules for allocation of shared costsAmazon MSK adds metrics for increased visibility of capacityAmazon Textract Updates: Up to 32% Price Reduction in 8 AWS Regions and Up to 50% Reduction in Asynchronous Job Processing TimesAmazon Elastic File System introduces Intelligent-Tiering to automatically optimize storage costsAmazon S3 Intelligent-Tiering – Improved Cost Optimizations for Short-Lived and Small ObjectsGeneral availability: Azure Files supports storage capacity reservations for premium, hot, and cool tiersPublic preview: Scale-down mode in AKSGeneral availability: Lower capacity reservation tier for Azure Monitor dedicated clusters AWS CodeBuild now supports a small ARM machine type Amazon RDS now supports T4g instances for MySQL, MariaDB, and PostgreSQL databasesNow auto-terminate idle EMR clusters to lower costVisit our website to find out more about cloud FinOps.

Scaling Postgres
Episode 193 Entity-Attribute-Value Design | JSON Subscripting | mysql-fdw Push-down | New Regex

Scaling Postgres

Play Episode Listen Later Nov 28, 2021 9:27


In this episode of Scaling Postgres, we discuss using entity-attribute-value designs, new JSON subscripting capabilities, the mysql-fdw support for aggregate pushdowns and new regex functions in PG 15. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.cybertec-postgresql.com/en/entity-attribute-value-eav-design-in-postgresql-dont-do-it/ https://blog.logrocket.com/whats-new-json-postgresql-v14/ https://www.enterprisedb.com/blog/aggregate-push-down-mysqlfdw https://www.depesz.com/2021/11/26/waiting-for-postgresql-15-add-assorted-new-regexp_xxx-sql-functions/ https://blog.crunchydata.com/blog/using-timescaledb-extension-with-the-pgo-the-postgres-operator https://www.highgo.ca/2021/11/27/how-to-run-a-specific-regression-test/ https://postgresql.life/post/pavel_luzanov/

airhacks.fm podcast with adam bien
Debezium, Server, Engine, UI and the Outbox

airhacks.fm podcast with adam bien

Play Episode Listen Later Nov 28, 2021 67:11


An airhacks.fm conversation with Gunnar Morling (@gunnarmorling) about: debezium as analytics enablement, enriching events with quarkus, ksqlDB and PrestoDB and trino, cloud migrations with Debezium, embedded Debezium Engine, debezium server vs. Kafka Connect, Debezium Server with sink connectors, Apache Pulsar, Redis Streams are supporting Debezium Server, Debezium Server follows the microservice architecture, pluggable offset stores, JDBC offset store is Apache Iceberg connector, DB2, MySQL, PostgreSQL, MongoDB change streams, Cassandra, Vitess, Oracle, Microsoft SQL Server scylladb is cassandra compatible and provides external debezium connector, debezium ui is written in React, incremental snapshots, netflix cdc system, DBLog: A Watermark Based Change-Data-Capture Framework, multi-threaded snapshots, internal data leakage and the Outbox pattern, debezium listens to the outbox pattern, OpenTracing integration and the outbox pattern, sending messages directly to transaction log with PostgreSQL, Quarkus outbox pattern extension, the transaction boundary topic Gunnar Morling on twitter: @gunnarmorling and debezium.io

Screaming in the Cloud
The “Banksgiving” Special with Tim Banks

Screaming in the Cloud

Play Episode Listen Later Nov 25, 2021 34:54


About TimTim's tech career spans over 20 years through various sectors. Tim's initial journey into tech started as a US Marine. Later, he left government contracting for the private sector, working both in large corporate environments and in small startups. While working in the private sector, he honed his skills in systems administration and operations for large Unix-based datastores. Today, Tim leverages his years in operations, DevOps, and Site Reliability Engineering to advise and consult with clients in his current role. Tim is also a father of five children, as well as a competitive Brazilian Jiu-Jitsu practitioner. Currently, he is the reigning American National and 3-time Pan American Brazilian Jiu-Jitsu champion in his division.TranscriptCorey: Hello, and welcome to Screaming in the Cloud with your host, Chief cloud economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: This episode is sponsored in part by our friends at Vultr. Spelled V-U-L-T-R because they're all about helping save money, including on things like, you know, vowels. So, what they do is they are a cloud provider that provides surprisingly high performance cloud compute at a price that—while sure they claim its better than AWS pricing—and when they say that they mean it is less money. Sure, I don't dispute that but what I find interesting is that it's predictable. They tell you in advance on a monthly basis what it's going to going to cost. They have a bunch of advanced networking features. They have nineteen global locations and scale things elastically. Not to be confused with openly, because apparently elastic and open can mean the same thing sometimes. They have had over a million users. Deployments take less that sixty seconds across twelve pre-selected operating systems. Or, if you're one of those nutters like me, you can bring your own ISO and install basically any operating system you want. Starting with pricing as low as $2.50 a month for Vultr cloud compute they have plans for developers and businesses of all sizes, except maybe Amazon, who stubbornly insists on having something to scale all on their own. Try Vultr today for free by visiting: vultr.com/screaming, and you'll receive a $100 in credit. Thats v-u-l-t-r.com slash screaming.Corey: This episode is sponsored in part by something new. Cloud Academy is a training platform built on two primary goals. Having the highest quality content in tech and cloud skills, and building a good community the is rich and full of IT and engineering professionals. You wouldn't think those things go together, but sometimes they do. Its both useful for individuals and large enterprises, but here's what makes it new. I don't use that term lightly. Cloud Academy invites you to showcase just how good your AWS skills are. For the next four weeks you'll have a chance to prove yourself. Compete in four unique lab challenges, where they'll be awarding more than $2000 in cash and prizes. I'm not kidding, first place is a thousand bucks. Pre-register for the first challenge now, one that I picked out myself on Amazon SNS image resizing, by visiting cloudacademy.com/corey. C-O-R-E-Y. That's cloudacademy.com/corey. We're gonna have some fun with this one!Corey: Welcome to Screaming in the Cloud. I am Cloud Economist Corey Quinn joined by Principal Cloud Economist here at The Duckbill Group Tim Banks. Tim, how are you?Tim: I'm doing great, Corey. How about yourself?Corey: I am tickled pink that we are able to record this not for the usual reasons you would expect, but because of the glorious pun in calling this our Banksgiving episode. I have a hard and fast rule of, I don't play pun games or make jokes about people's names because that can be an incredibly offensive thing. “And oh, you're making jokes about my name? I've never heard that one before.” It's not that I can't do it—I play games with language all the time—but it makes people feel crappy. So, when you suggested this out of the blue, it was yes, we're doing it. But I want to be clear, I did not inflict this on you. This is your own choice; arguably a poor one. We're going to find out.Tim: 1000% my idea.Corey: So, this is your show. It's a holiday week. So, what do you want to do with our Banksgiving episode?Tim: I want to give thanks for the folks who don't normally get acknowledged through the year. Like you know, we do a lot of thanking the rock stars, we do a lot of thanking the big names, right, we also do a lot of, you know, some snarky jabs at some folks. Deservingly—not folks, but groups and stuff like that; some folks deserve it, and we won't be giving them thanks—but some orgs and some groups and stuff like that. And I do think with that all said, we should acknowledge and thank the folks that we normally don't get to, folks who've done some great contributions this year, folks who have helped us, helped the industry, and help services that go unsung, I think a great one that you brought up, it's not the engineers, right? It's the people that make sure we get paid. Because I don't work for charity. And I don't know about you, Corey. I haven't seen the books yet, but I'm pretty sure none of us here do and so how do we get paid? Like I don't know.Corey: Oh, sure you have. We had a show on a somewhat simplified P&L during the all hands meeting because, you know, transparency matters. But you're right, those are numbers there and none of that is what we could have charged but didn't because we decided to do more volunteer work for AWS. If we were going to go down that path, we would just be Community Heroes and be done with it.Tim: That's true. But you know, it's like, I do my thing and then, you know, I get a paycheck every now and then. And so, as far as I know, I think most of that happens because of Dan.Corey: Dan is a perfect example. He's been a guest on this show, I don't know it has as aired at the time that this goes out because I don't have to think about that, which is kind of the point. Dan's our CFO and makes sure that a lot of the financial trains keep running on time. But let's also be clear, the fact that I can make predictions about what the business is going to be doing by a metric other than how much cash is in the bank account at this very moment really freed up some opportunity for us. It turned into adult supervision for folks who, when I started this place and then Mike joined, and it was very much not an area that either one of us was super familiar with. Which is odd given what we do here, but we learned quickly.The understanding not just how these things work—which we had an academic understanding of—but why it mattered and how that applies to real life. Finance is one of those great organizations that doesn't get a lot of attention or respect outside of finance itself. Because it's, “Oh, well they just control the money. How hard could it be?” Really, really hard.Tim: It really is. And when we dig into some of these things and some of the math that goes and some of what the concerns are that, you know, a lot of engineers don't really have a good grasp on, and it's eye opening to understand some of the concerns. At least some of the concerns at least from an engineering aspect. And I really don't give much consideration day to day about the things that go on behind the scenes to make sure that I get paid.But you look at this throughout the industry, like, how many of the folks that we work with, how many folks out there doing this great work for the industry, do they know who their payroll person is? Do they know who their accountant team is? Do they know who their CFO or the other people out there that are doing the work and making sure the lights stay on, that people get paid and all the other things that happen, right? You know, people take that for granted. And it's a huge work and those people really don't get the appreciation that I think they deserve. And I think it's about time we did that.Corey: It's often surprising to me how many people that I encounter, once they learn that there are 12 employees here, automatically assume that it's you, me, and maybe occasionally Mike doing all the work, and the other nine people just sort of sit here and clap when I tell a funny joke, and… well, yes, that is, of course, a job duty, but that's not the entire purpose of why people are here.Natalie in marketing is a great example. “Well, Corey, I thought you did the marketing. You go and post on Twitter and that's where business comes from.” Well, kind of. But let's be clear, when I do that, and people go to the website to figure out what the hell I'm talking about.Well, that website has words on it. I didn't put those words on that site. It directs people to contact us forms, and there are automations behind that that make sure they go to the proper place because back before I started this place and I was independent, people would email me asking for help with their bill and I would just never respond to them. It's the baseline adult supervision level of competence that I keep aspiring to. We have a sales team that does fantastic work.And that often is one of those things that'll get engineering hackles up, but they're not out there cold-calling people to bug them about AWS bills. It's when someone reaches out saying we have a problem with our AWS spend, can you help us? The answer is invariably, “Let's talk about that.” It's a consultative discussion about why do you care about the bill, what does success look like, how do you know this will be a success, et cetera, et cetera, et cetera, that make sure that we're aimed at the right part of the problem. That's incredibly challenging work and I am grateful beyond words, I don't have to be involved with the day-in, day-out of any of those things.Tim: I think even beyond just that handling, like, the contracts and the NDAs, and the various assets that have to be exchanged just to get us virtually on site, I've [unintelligible 00:06:46] a couple of these things, I'm glad it's not my job. It is, for me, overwhelmingly difficult for me to really get a grasp and all that kind of stuff. And I am grateful that we do have a staff that does that. You've heard me, you see me, you know, kind of like, sales need to do better, and a lot of times I do but I do want to make sure we are appreciating them for the work that they do to make sure that we have work to do. Their contribution cannot be underestimated.Corey: And I think that's something that we could all be a little more thankful for in the industry. And I see this on Twitter sometimes, and it's probably my least favorite genre of tweet, where someone will wind up screenshotting some naive recruiter outreach to them, and just start basically putting the poor person on blast. I assure you, I occasionally get notices like that. The most recent example of that was, I got an email to my work email address from an associate account exec at AWS asking what projects I have going on, how my work in the cloud is going, and I can talk to them about if I want to help with cost optimization of my AWS spend and the rest. And at first, it's one of those, I could ruin this person's entire month, but I don't want to be that person.And I did a little LinkedIn stalking and it turns out, this looks like this person's first job that they've been in for three months. And I've worked in jobs like that very early in my career; it is a numbers game. When you're trying to reach out to 1000 people a month or whatnot, you aren't sitting there googling what every one of them is, does, et cetera. It's something that I've learned, that is annoying, sure. But I'm in an incredibly privileged position here and dunking on someone who's doing what they are told by an existing sales apparatus and crapping on them is not fair.That is not the same thing as these passive-aggressive [shit-tier 00:08:38] drip campaigns of, “I feel like I'm starting to stalk you.” Then don't send the message, jackhole. It's about empathy and not crapping on people who are trying to find their own path in this ridiculous industry.Tim: I think you brought up recruiters, and, you know, we here at The Duckbill Group are currently recruiting for a senior cloud economist and we don't actually have a recruiter on staff. So, we're going through various ways to find this work and it has really made me appreciate the work that recruiters in the past that I've worked with have done. Some of the ones out there are doing really fantastic work, especially sourcing good candidates, vetting good candidates, making sure that the job descriptions are inclusive, making sure that the whole recruitment process is as smooth as it can be. And it can't always be. Having to deal with all the spinning plates of getting interviews with folks who have production workloads, it is pretty impressive to me to see how a lot of these folks get—pull it off and it just seems so smooth. Again, like having to actually wade through some of this stuff, it's given me a true appreciation for the work that good recruiters do.Corey: We don't have automated systems that disqualify folks based on keyword matches—I've never been a fan of that—but we do get applicants that are completely unsuitable. We've had a few come in that are actual economists who clearly did not read the job description; they're spraying their resume everywhere. And the answer is you smile, you decline it and you move on. That is the price you pay of attempting to hire people. You don't put them on blast, you don't go and yell at an entire ecosystem of people because looking for jobs sucks. It's hard work.Back when I was in my employee days, I worked harder finding new jobs than I often did in the jobs themselves. This may be related to why I get fired as much, but I had to be good at finding new work. I am, for better or worse, in a situation where I don't have to do that anymore because once again, we have people here who do the various moving parts. Plus, let's be clear here, if I'm out there interviewing at other companies for jobs, I feel like that sends a message to you and the rest of the team that isn't terrific.Tim: We might bring that up. [laugh].Corey: “Why are you interviewing for a job over there?” It's like, “Because they have free doughnuts in the office. Later, jackholes.” It—I don't think that is necessarily the culture we're building here.Tim: No, no, it's not. Specially—you know, we're more of a cinnamon roll culture anyways.Corey: No. In my case, it's one of those, “Corey, why are you interviewing for a job at AWS?” And the answer is, “Oh, it's going to be an amazing shitpost. Just wait and watch.”Tim: [laugh]. Now, speaking of AWS, I have to absolutely shout out to Emily Freeman over there who has done some fantastic work this year. It's great when you see a person get matched up with the right environment with the right team in the right role, and Emily has just been hitting out of the park ever since he got there, so I'm super, super happy to see her there.Corey: Every time I get to collaborate with her on something, I come away from the experience even more impressed. It's one of those phenomenal collaborations. I just—I love working with her. She's human, she's empathetic, she gets it. She remains, as of this recording, the only person who has ever given a talk that I have heard on ML Ops, and come away with a better impression of that space and thinking maybe it's not complete nonsense.And that is not just because it's Emily, so I—because—I'm predisposed to believe her, though I am, it's because of how she frames it, how she views these things, and let's be clear, the content that she says. And that in turn makes me question my preconceptions on this, and that is why she has that I will listen and pay attention when she speaks. So yeah, if Emily's going to try and make a point, there's always going to be something behind it. Her authenticity is unimpeachable.Tim: Absolutely. I do take my hat's off to everyone who's been doing DevRel and evangelism and those type of roles during pandemics. And we just, you know, as the past few months, I've started back to in-person events. But the folks who've been out there finding new way to do those jobs, finding a way to [crosstalk 00:12:50]—Corey: Oh, staff at re:Invent next week. Oh, my God.Tim: Yeah. Those folks, I don't know how they're being rewarded for their work, but I can assure you, they probably need to be [unintelligible 00:12:57] better than they are. So, if you are staff at re:Invent, and you see Corey and I, next week when we're there—if you're listening to this in time—we would love to shake your hand, elbow bump you, whatever it is you're comfortable with, and laud you for the work you're doing. Because it is not easy work under the best of circumstances, and we are certainly not under the best of circumstances.Corey: I also want to call out specific thanks to a group that might take some people aback. But that group is AWS marketing, which given how much grief I give them seems like an odd thing for me to say, but let's be clear, I don't have any giant companies whose ability to continue as a going concern is dependent upon my keeping systems up and running. AWS does. They have to market and tell stories to everyone because that is generally who their customers are: they round to everyone. And an awful lot of those companies have unofficial mottos of, “That's not funny.” I'm amazed that they can say anything at all, given how incredibly varied their customer base is, I could get away with saying whatever I want solely because I just don't care. They have to care.Tim: They do. And it's not only that they have to care, they're in a difficult situation. It's like, you know, they—every company that sizes is, you know, they are image conscious, and they have things that say what like, “Look, this is the deal. This is the scenario. This is how it went down, but you can still maintain your faith and confidence in us.” And people do when AWS services, they have problems, if anything comes out like that, it does make the news and the reason it doesn't make the news is because it is so rare. And when they can remind us of that in a very effective way, like, I appreciate that. You know, people say if anything happens to S3, everybody knows because everyone depends on it and that's for good reason.Corey: And let's not forget that I run The Duckbill Group. You know, the company we work for. I have the Last Week in AWS newsletter and blog. I have my aggressive shitposting Twitter feed. I host the AWS Morning Brief podcast, and I host this Screaming in the Cloud. And it's challenging for me to figure out how to message all of those things because when people ask what you do, they don't want to hear a litany that goes on for 25 seconds, they want a sentence.I feel like I've spread in too many directions and I want to narrow that down. And where do I drive people to and that was a bit of a marketing challenge that Natalie in our marketing department really cut through super well. Now, pretend I work in AWS. The way that I check this based upon a public list of parameters they stub into Systems Manager Parameter Store, there are right now 291 services that they offer. That is well beyond any one person's ability to keep in their head. I can talk incredibly convincingly now about AWS services that don't exist and people who work in AWS on messaging, marketing, engineering, et cetera, will not call me out on it because who can provably say that ‘AWS Strangle Pony' isn't a real service.Tim: I do want to call out the DevOps—shout out I should say, the DevOps term community for AWS Infinidash because that was just so well done, and AWS took that with just the right amount of tongue in cheek, and a wink and a nod and let us have our fun. And that was a good time. It was a great exercise in improv.Corey: That was Joe Nash out of Twilio who just absolutely nailed it with his tweet, “I am convinced that a small and dedicated group of Twitter devs could tweet hot takes about a completely made up AWS product—I don't know AWS Infinidash or something—and it would appear as a requirement on job specs within a week.” And he was right.Tim: [laugh]. Speaking of Twitter, I want to shout out Twitter as a company or whoever does a product management over there for Twitter Spaces. I remember when Twitter Spaces first came out, everyone was dubious of its effect, of it's impact. They were calling it, you know, a Periscope clone or whatever it was, and there was a lot of sneering and snarking at it. But Twitter Spaces has become very, very effective in having good conversations in the group and the community of folks that have just open questions, and then to speak to folks that they probably wouldn't only get to speak to about this questions and get answers, and have really helpful, uplifting and difficult conversations that you wouldn't otherwise really have a medium for. And I'm super, super happy that whoever that product manager was, hats off to you, my friend.Corey: One group you're never going to hear me say a negative word about is AWS support. Also, their training and certification group. I know that are technically different orgs, but it often doesn't feel that way. Their job is basically impossible. They have to teach people—even on the support side, you're still teaching people—how to use all of these different varied services in different ways, and you have to do it in the face of what can only really be described as abuse from a number of folks on Twitter.When someone is having trouble with an AWS service, they can turn into shitheads, I've got to be honest with you. And berating the poor schmuck who has to handle the AWS support Twitter feed, or answer your insulting ticket or whatnot, they are not empowered to actually fix the underlying problem with a service. They are effectively a traffic router to get the message to someone who can, in a format that is understood internally. And I want to be very clear that if you insult people who are in customer service roles and blame them for it, you're just being a jerk.Tim: No, it really is because I'm pretty sure a significant amount of your listeners and people initially started off working in tech support, or customer service, or help desk or something like that, and you really do become the dumping ground for the customers' frustrations because you are the only person they get to talk to. And you have to not only take that, but you have to try and do the emotional labor behind soothing them as well as fixing the actual problem. And it's really, really difficult. I feel like the people who have that in their background are some of the best consultants, some of the best DevRel folks, and the best at talking to people because they're used to being able to get some technical details out of folks who may not be very technical, who may be under emotional distress, and certainly in high stress situations. So yeah, AWS support, really anybody who has support, especially paid support—phone or chat otherwise—hats off again. That is a service that is thankless, it is a service that is almost always underpaid, and is almost always under appreciated.Corey: This episode is sponsored by our friends at Oracle HeatWave is a new high-performance accelerator for the Oracle MySQL Database Service. Although I insist on calling it “my squirrel.” While MySQL has long been the worlds most popular open source database, shifting from transacting to analytics required way too much overhead and, ya know, work. With HeatWave you can run your OLTP and OLAP, don't ask me to ever say those acronyms again, workloads directly from your MySQL database and eliminate the time consuming data movement and integration work, while also performing 1100X faster than Amazon Aurora, and 2.5X faster than Amazon Redshift, at a third of the cost. My thanks again to Oracle Cloud for sponsoring this ridiculous nonsense.Corey: I'll take another team that's similar to that respect: Commerce Platform. That is the team that runs all of AWS billing. And you would be surprised that I'm thanking them, but no, it's not the cynical approach of, “Thanks for making it so complicated so I could have a business.” No, I would love it if it were so simple that I had to go find something else to do because the problem was that easy for customers to solve. That is the ideal and I hope, sincerely, that we can get there.But everything that happens in AWS has to be metered and understood as far as who has done what, and charge people appropriately for it. It is also generally invisible; people don't understand anything approaching the scale of that, and what makes it worst of all, is that if suddenly what they were doing broke and customers weren't built for their usage, not a single one of them would complain about it because, “All right, I'll take it.” It's a thankless job that is incredibly key and central to making the cloud work at all, but it's a hard job.Tim: It really is. And is a lot of black magic and voodoo to really try and understand how this thing works. There's no simple way to explain it. I imagine if they were going to give you the index overview of how it works with a 10,000 feet, that alone would be, like, a 300 page document. It is a gigantic moving beast.And it is one of those things where scale will show all the flaws. And no one has scale I think like AWS does. So, the folks that have to work and maintain that are just really, again, they're under appreciated for all that they do. I also think that—you know, you talk about the same thing in other orgs, as we talked about the folks that handle the billing and stuff like that, but you mentioned AWS, and I was thinking the other day how it's really awesome that I've got my AWS driver. I have the same, like, group of three or four folks that do all my deliveries for AWS.And they have been inundated over this past year-and-a-half with more and more and more stuff. And yet, I've still managed—my stuff is always put down nicely on my doorstep. It's never thrown, it's not damaged. I'm not saying it's never been damaged, but it's not damaged, like, maybe FedEx I've [laugh] had or some other delivery services where it's just, kind of, carelessly done. They still maintain efficiency, they maintain professionalism [unintelligible 00:21:45] talking to folks.What they've had to do at their scale and at that the amount of stuff they've had to do for deliveries over this past year-and-a-half has just been incredible. So, I want to extend it also to, like, the folks who are working in the distribution centers. Like, a lot of us here talk about AWS as if that's Amazon, but in essence, it is those folks that are working those more thankless and invisible jobs in the warehouses and fulfillment centers, under really bad conditions sometimes, who's still plug away at it. I'm glad that Amazon is at least saying they're making efforts to improve the conditions there and improve the pay there, things like that, but those folks have enabled a lot of us to work during this pandemic with a lot of conveniences that they themselves would never be able to enjoy.Corey: Yeah. It's bad for society, but I'm glad it exists, obviously. The thing is, I would love it if things showed up a little more slowly if it meant that people could be treated humanely along the process. That said, I don't have any conception of what it takes to run a company with 1.2 million people.I have learned that as you start managing groups and managing managers of groups, it's counterintuitive, but so much of what you do is no longer you doing the actual work. It is solely through influence and delegation. You own all of the responsibility but no direct put-finger-on-problem capability of contributing to the fix. It takes time at that scale, which is why I think one of the dumbest series of questions from, again, another group that deserves a fair bit of credit which is journalists because this stuff is hard, but a naive question I hear a lot is, “Well, okay. It's been 100 days. What has Adam Selipsky slash Andy Jassy changed completely about the company?”It's, yeah, it's a $1.6 trillion company. They are not going to suddenly grab the steering wheel and yank. It's going to take years for shifts that they do to start manifesting in serious ways that are externally visible. That is how big companies work. You don't want to see a complete change in direction from large blue chip companies that run things. Like, again, everyone's production infrastructure. You want it to be predictable, you want it to be boring, and you want shifts to be gradual course corrections, not vast swings.Tim: I mean, Amazon is a company with a population of a medium to medium-large sized city and a market cap of the GDP of several countries. So, it is not a plucky startup; it is not this small little tech company. It is a vast enterprise that's distributed all over the world with a lot of folks doing a lot of different jobs. You cannot, as you said, steer that ship quickly.Corey: I grew up in Maine and Amazon has roughly the same number employees as live in Maine. It is hard to contextualize how all of that works. There are people who work there that even now don't always know who Andy Jassy is. Okay, fine, but I'm not talking about don't know him on site or whatever. I'm saying they do not recognize the name. That's a very big company.Tim: “Andy who?”Corey: Exactly. “Oh, is that the guy that Corey makes fun of all the time?” Like, there we go. That's what I tend to live for.Tim: I thought that was Werner.Corey: It's sort of every one, though I want to be clear, I make it a very key point. I do not make fun of people personally because it—even if they're crap, which I do not believe to be the case in any of the names we've mentioned so far, they have friends and family who love and care about them. You don't want someone to go on the internet and Google their parent's name or something, and then just see people crapping all over. That's got to hurt. Let people be people. And, on some level, when you become the CEO of a company of that scale, you're stepping out of reality and into the pages of legend slash history, at some point. 200 years from now, people will read about you in history books, that's a wild concept.Tim: It is I think you mentioned something important that we would be remiss—especially Duckbill Group—to mention is that we're very thankful for our families, partners, et cetera, for putting up with us, pets, everybody. As part of our jobs, we invite strangers from the internet into our homes virtually to see behind us what is going on, and for those of us that have kids, that involves a lot of patience on their part, a lot of patients on our partners' parts, and other folks that are doing those kind of nurturing roles. You know, our pets who want to play with us are sitting there and not able to. It has not been easy for all of us, even though we're a remote company, but to work under these conditions that we have been over the past year-and-a-half. And I think that goes for a lot of the folks in industry where now all of a sudden, you've been occupying a room in the house or space in the house for some 18-plus months, where before you're always at work or something like that. And that's been a hell of an adjustment. And so we talk about that for us folks that are here pontificating on podcasts, or banging out code, but the adjustments and the things our families have had to go through and do to tolerate us being there cannot be overstated how important that is.Corey: Anyone else that's on your list of people to thank? And this is the problem because you're always going to forget people. I mean, the podcast production crew: the folks that turn our ramblings into a podcast, the editing, the transcription, all of it; the folks that HumblePod are just amazing. The fact that I don't have to worry about any of this stuff as if by magic, means that you're sort of insulated from it. But it's amazing to watch that happen.Tim: You know, honestly, I super want to thank just all the folks that take the time to interact with us. We do this job and Corey shitposts, and I shitpost and we talk, but we really do this and rely on the folks that do take the time to DM us, or tweet us, or mention us in the thread, or reach out in any way to ask us questions, or have a discussion with us on something we said, those folks encourage us, they keep us accountable, and they give us opportunities to learn to be better. And so I'm grateful for that. It would be—this role, this job, the thing we do where we're viewable and seen by the public would be a lot less pleasant if it wasn't for y'all. So, it's too many to name, but I do appreciate you.Corey: Well, thank you, I do my best. I find this stuff to be so boring if you couldn't have fun with it. And so many people can't have fun with it, so it feels like I found a cheat code for making enterprise software solutions interesting. Which even saying that out loud sounds like I'm shitposting. But here we are.Tim: Here we are. And of course, my thanks to you, Corey, for reaching out to me one day and saying, “Hey, what are you doing? Would you want to come interview with us at The Duckbill Group?”Corey: And it was great because, like, “Well, I did leave AWS within the last 18 months, so there might be a non-compete issue.” Like, “Oh, please, I hope so. Oh, please, oh, please, oh, please. I would love to pick that fight publicly.” But sadly, no one is quite foolish enough to take me up on it.Don't worry. That's enough of a sappy episode, I think. I am convinced that our next encounter on this podcast will be our usual aggressive self. But every once in a while it's nice to break the act and express honest and heartfelt appreciation. I'm really looking forward to next week with all of the various announcements that are coming out.I know people have worked extremely hard on them, and I want them to know that despite the fact that I will be making fun of everything that they have done, there's a tremendous amount of respect that goes into it. The fact that I can make fun of the stuff that you've done without any fear that I'm punching down somehow because, you know it is at least above a baseline level of good speaks volumes. There are providers I absolutely do not have that confidence towards them.Tim: [laugh]. Yeah, AWS, as the enterprise level service provider is an easy target for a lot of stuff. The people that work there are not. They do great work. They've got amazing people in all kinds of roles there. And they're often unseen for the stuff they do. So yeah, for all the folks who have contributed to what we're going to partake in at re:Invent—and it's a lot and I understand from having worked there, the pressure that's put on you for this—I'm super stoked about it and I'm grateful.Corey: Same here. If I didn't like this company, I would not have devoted years to making fun of it. Because that requires a diagnosis, not a newsletter, podcast, or shitposting Twitter feed. Tim, thank you so much for, I guess, giving me the impetus and, of course, the amazing name of the show to wind up just saying thank you, which I think is something that we could all stand to do just a little bit more of.Tim: My pleasure, Corey. I'm glad we could run with this. I'm, as always, happy to be on Screaming in the Cloud with you. I think now I get a vest and a sleeve. Is that how that works now?Corey: Exactly. Once you get on five episodes, then you end up getting the dinner jacket, just, like, hosting SNL. Same story. More on that to come in the new year. Thanks, Tim. I appreciate it.Tim: Thank you, Corey.Corey: Tim Banks, principal cloud economist here at The Duckbill Group. I am, of course, Corey Quinn, and thank you for listening.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.

The History of Computing
An Abridged History of Free And Open Source Software

The History of Computing

Play Episode Listen Later Nov 24, 2021 22:34


In the previous episodes, we looked at the rise of patents and software and their impact on the nascent computer industry. But a copyright is a right. And that right can be given to others in whole or in part. We have all benefited from software where the right to copy was waved and it's shaped the computing industry as much, if not more, than proprietary software. The term Free and Open Source Software (FOSS for short) is a blanket term to describe software that's free and/or whose source code is distributed for varying degrees of tinkeration. It's a movement and a choice. Programmers can commercialize our software. But we can also distribute it free of copy protections. And there are about as many licenses as there are opinions about what is unique, types of software, underlying components, etc. But given that many choose to commercialize their work products, how did a movement arise that specifically didn't? The early computers were custom-built to perform various tasks. Then computers and software were bought as a bundle and organizations could edit the source code. But as operating systems and languages evolved and businesses wanted their own custom logic, a cottage industry for software started to emerge. We see this in every industry - as an innovation becomes more mainstream, the expectations and needs of customers progress at an accelerated rate. That evolution took about 20 years to happen following World War II and by 1969, the software industry had evolved to the point that IBM faced antitrust charges for bundling software with hardware. And after that, the world of software would never be the same. The knock-on effect was that in the 1970s, Bell Labs pushed away from MULTICS and developed Unix, which AT&T then gave away as compiled code to researchers. And so proprietary software was a growing industry, which AT&T began charging for commercial licenses as the bushy hair and sideburns of the 70s were traded for the yuppy culture of the 80s. In the meantime, software had become copyrightable due to the findings of CONTU and the codifying of the Copyright Act of 1976. Bill Gates sent his infamous “Open Letter to Hobbyists” in 1976 as well, defending the right to charge for software in an exploding hobbyist market. And then Apple v Franklin led to the ability to copyright compiled code in 1983. There was a growing divide between those who'd been accustomed to being able to copy software freely and edit source code and those who in an up-market sense just needed supported software that worked - and were willing to pay for it, seeing the benefits that automation was having on the capabilities to scale an organization. And yet there were plenty who considered copyright software immoral. One of the best remembered is Richard Stallman, or RMS for short. Steven Levy described Stallman as “The Last of the True Hackers” in his epic book “Hackers: Heroes of the Computer Revolution.” In the book, he describes the MIT Stallman joined where there weren't passwords and we didn't yet pay for software and then goes through the emergence of the LISP language and the divide that formed between Richard Greenblatt, who wanted to keep The Hacker Ethic alive and those who wanted to commercialize LISP. The Hacker Ethic was born from the young MIT students who freely shared information and ideas with one another and help push forward computing in an era they thought was purer in a way, as though it hadn't yet been commercialized. The schism saw the death of the hacker culture and two projects came out of Stallman's technical work: emacs, which is a text editor that is still included freely in most modern Unix variants and the GNU project. Here's the thing, MIT was sitting on patents for things like core memory and thrived in part due to the commercialization or weaponization of the technology they were producing. The industry was maturing and since the days when kings granted patents, maturing technology would be commercialized using that system. And so Stallman's nostalgia gave us the GNU project, born from an idea that the industry moved faster in the days when information was freely shared and that knowledge was meant to be set free. For example, he wanted the source code for a printer driver so he could fix it and was told it was protected by an NDAQ and so couldn't have it. A couple of years later he announced GNU, a recursive acronym for GNU's Not Unix. The next year he built a compiler called GCC and the next year released the GNU Manifesto, launching the Free Software Foundation, often considered the charter of the free and open source software movement. Over the next few years as he worked on GNU, he found emacs had a license, GCC had a license, and the rising tide of free software was all distributed with unique licenses. And so the GNU General Public License was born in 1989 - allowing organizations and individuals to copy, distribute, and modify software covered under the license but with a small change, that if someone modified the source, they had to release that with any binaries they distributed as well. The University of California, Berkley had benefited from a lot of research grants over the years and many of their works could be put into the public domain. They had brought Unix in from Bell Labs in the 70s and Sun cofounder and Java author Bill Joy worked under professor Fabry, who brought Unix in. After working on a Pascal compiler that Unix coauthor Ken Thompson left for Berkeley, Joy and others started working on what would become BSD, not exactly a clone of Unix but with interchangeable parts. They bolted on the OSI model to get networking and through the 80s as Joy left for Sun and DEC got ahold of that source code there were variants and derivatives like FreeBSD, NetBSD, Darwin, and others. The licensing was pretty permissive and simple to understand: Copyright (c) . All rights reserved. Redistribution and use in source and binary forms are permitted provided that the above copyright notice and this paragraph are duplicated in all such forms and that any documentation, advertising materials, and other materials related to such distribution and use acknowledge that the software was developed by the . The name of the may not be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED ``AS IS AND WITHOUT ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. By 1990 the Board of Regents at Berkley accepted a four clause BSD license that spawned a class of licenses. While it's matured into other formats like a 0 clause license it's one of my favorites as it is truest to the FOSS cause. And the 90s gave us the Apache License, from the Apache Group, loosely based on the BSD License and then in 2004 leaning away from that with the release of the Apache License 2 that was more compatible with the GPL license. Given the modding nature of Apache they didn't require derivative works to also be open sourced but did require leaving the license in place for unmodified parts of the original work. GNU never really caught on as an OS in the mainstream, although a collection of tools did. The main reason the OS didn't go far is probably because Linus Torvalds started releasing prototypes of his Linux operating system in 1991. Torvalds used The GNU General Public License v2, or GPLv2 to license his kernel, having been inspired by a talk given by Stallman. GPL 2 had been released in 1991 and something else was happening as we turned into the 1990s: the Internet. Suddenly the software projects being worked on weren't just distributed on paper tape or floppy disks; they could be downloaded. The rise of Linux and Apache coincided and so many a web server and site ran that LAMP stack with MySQL and PHP added in there. All open source in varying flavors of what open source was at the time. And collaboration in the industry was at an all-time high. We got the rise of teams of developers who would edit and contribute to projects. One of these was a tool for another aspect of the Internet, email. It was called popclient, Here Eric S Raymond, or ESR for short, picked it up and renamed it to fetchmail, releasing it as an open source project. Raymond presented on his work at the Linux Congress in 1997, expanded that work into an essay and then the essay into “The Cathedral and the Bazaar” where bazaar is meant to be like an open market. That inspired many to open source their own works, including the Netscape team, which resulted in Mozilla and so Firefox - and another book called “Freeing the Source: The Story of Mozilla” from O'Reilly. By then, Tim O'Reilly was a huge proponent of this free or source code available type of software as it was known. And companies like VA Linux were growing fast. And many wanted to congeal around some common themes. So in 1998, Christine Peterson came up with the term “open source” in a meeting with Raymond, Todd Anderson, Larry Augustin, Sam Ockman, and Jon “Maddog” Hall, author of the first book I read on Linux. Free software it may or may not be but open source as a term quickly proliferated throughout the lands. By 1998 there was this funny little company called Tivo that was doing a public beta of a little box with a Linux kernel running on it that bootstrapped a pretty GUI to record TV shows on a hard drive on the box and play them back. You remember when we had to wait for a TV show, right? Or back when some super-fancy VCRs could record a show at a specific time to VHS (but mostly failed for one reason or another)? Well, Tivo meant to fix that. We did an episode on them a couple of years ago but we skipped the term Tivoization and the impact they had on GPL. As the 90s came to a close, VA Linux and Red Hat went through great IPOs, bringing about an era where open source could mean big business. And true to the cause, they shared enough stock with Linus Torvalds to make him a millionaire as well. And IBM pumped a billion dollars into open source, with Sun moving to open source openoffice.org. Now, what really happened there might be that by then Microsoft had become too big for anyone to effectively compete with and so they all tried to pivot around to find a niche, but it still benefited the world and open source in general. By Y2K there was a rapidly growing number of vendors out there putting Linux kernels onto embedded devices. TiVo happened to be one of the most visible. Some in the Linux community felt like they were being taken advantage of because suddenly you had a vendor making changes to the kernel but their changes only worked on their hardware and they blocked users from modifying the software. So The Free Software Foundation updated GPL, bundling in some other minor changes and we got the GNU General Public License (Version 3) in 2006. There was a lot more in GPL 3, given that so many organizations were involved in open source software by then. Here, the full license text and original copyright notice had to be included along with a statement of significant changes and making source code available with binaries. And commercial Unix variants struggled with SGI going bankrupt in 2006 and use of AIX and HP-UX Many of these open source projects flourished because of version control systems and the web. SourceForge was created by VA Software in 1999 and is a free service that can be used to host open source projects. Concurrent Versions System, or CVS had been written by Dick Grune back in 1986 and quickly became a popular way to have multiple developers work on projects, merging diffs of code repositories. That gave way to git in the hearts of many a programmer after Linus Torvalds wrote a new versioning system called git in 2005. GitHub came along in 2008 and was bought by Microsoft in 2018 for 2018. Seeing a need for people to ask questions about coding, Stack Overflow was created by Jeff Atwood and Joel Spolsky in 2008. Now, we could trade projects on one of the versioning tools, get help with projects or find smaller snippets of sample code on Stack Overflow, or even Google random things (and often find answers on Stack Overflow). And so social coding became a large part of many a programmers day. As did dependency management, given how many tools are used to compile a modern web app or app. I often wonder how much of the code in many of our favorite tools is actually original. Another thought is that in an industry dominated by white males, it's no surprise that we often gloss over previous contributions. It was actually Grace Hopper's A-2 compiler that was the first software that was released freely with source for all the world to adapt. Sure, you needed a UNIVAC to run it, and so it might fall into the mainframe era and with the emergence of minicomputers we got Digital Equipment's DECUS for sharing software, leading in part to the PDP-inspired need for source that Stallman was so adamant about. General Motors developed SHARE Operating System for the IBM 701 and made it available through the IBM user group called SHARE. The ARPAnet was free if you could get to it. TeX from Donald Knuth was free. The BASIC distribution from Dartmouth was academic and yet Microsoft sold it for up to $100,000 a license (see Commodore ). So it's no surprise that people avoided paying upstarts like Microsoft for their software or that it took until the late 70s to get copyright legislation and common law. But Hopper's contributions were kinda' like open source v1, the work from RMS to Linux was kinda' like open source v2, and once the term was coined and we got the rise of a name and more social coding platforms from SourceForge to git, we moved into a third version of the FOSS movement. Today, some tools are free, some are open source, some are free as in beer (as you find in many a gist), some are proprietary. All are valid. Today there are also about as many licenses as there are programmers putting software out there. And here's the thing, they're all valid. You see, every creator has the right to restrict the ability to copy their software. After all, it's their intellectual property. Anyone who chooses to charge for their software is well within their rights. Anyone choosing to eschew commercialization also has that right. And every derivative in between. I wouldn't judge anyone based on any model those choose. Just as those who distribute proprietary software shouldn't be judged for retaining their rights to do so. Why not just post things we want to make free? Patents, copyrights, and trademarks are all a part of intellectual property - but as developers of tools we also need to limit our liability as we're probably not out there buying large errors and omissions insurance policies for every script or project we make freely available. Also, we might want to limit the abuse of our marks. For example, Linus Torvalds monitors the use of the Linux mark through the Linux Mark Institute. Apparently some William Dell Croce Jr tried to register the Linux trademark in 1995 and Torvalds had to sue to get it back. He provides use of the mark using a free and perpetual global sublicense. Given that his wife won the Finnish karate championship six times I wouldn't be messing with his trademarks. Thank you to all the creators out there. Thank you for your contributions. And thank you for tuning in to this episode of the History of Computing Podcast. Have a great day.

Les Cast Codeurs Podcast
LCC 267 - Lagom efface sa dette technique

Les Cast Codeurs Podcast

Play Episode Listen Later Nov 15, 2021 76:33


Antonio et Emmanuel discutent Microsoft et Java, cryostat, Java 17, Micronaut, Quarkus, Play framework, Lagom, Amazon, CORS, CSS (si si), Hibernate Reactive, AtomicJar, canary, amplification algorithmique. Enregistré le 12 novembre 2021 Téléchargement de l'épisode LesCastCodeurs-Episode–267.mp3 News Langages Blog sur les extraits de code dans les JavaDocs (18 Oct 2021) C'est plus agréable à utiliser que les balises pre, pas besoin d'escaping (pour < et >), l'espace à gauche est normalisé On peut mettre en valeur certaines portion, ou remplacer par une expression régulière certains bouts Et on peut également externaliser d'où vient l'extrait de code, au lieu de le mettre dans la JavaDoc, on peut référencer une région de son vrai code. Donc au moins, on est sûr que c'est du code valide et qui compile évidemment Gunnar explique comment reprendre le code provenant de nos classes de test, pour le faire apparaître dans les JavaDocs, créant ainsi une vraie documentation “exécutable” Compress class space (27 Mars 2019) compressed object ou class pointer sur 64buts en 32 bits vis adresse relative due adresse relative, la Klass structure dans le metaspace doit être mémoire contiguë et pré allouée initialement (risque de non reallocation si mémoire libre non contiguë ) Donc le classpart et le non class part séparés dans le meta space. Klass is 32G max et contiguë et la klass part est appelé compressed class space Par défaut 1G mais configurable jusqu'à 3G. C'est virtual mémoire, juste une réservation. 1K pas classe environ donc 1000000 de classes max Que quand on utilise compressed oops Que pour Java heap size de 32G max Cryostat 2.0 (18 Oct 2021) Fournit une API sécurisée pour profiler et monitored les applis Java dans les containers avec Java Flight Recorder Cryostat peut récupérer stoquer et analyser les enregistrements flight recorder de containers Ensuite consommé par graphana ou l.appli JDK Mission Control desktop Fichier reste local au container par défaut donc pas pratique Connection via JMX directe pas pratique ni secure par défaut Cryostat récupère les recording via HTTPS A un opérateur kubernetes Etc Microsoft augmente ses investissements dans Java. (4 Nov 2021) Microsoft rejoints le JCP Travaille sur VSCode for Java avec Red Hat Est OK avec le LTS passant à 2 ans et va aider à supporter ces releases plus fréquentes Librairies Micronaut 3.1 (11 Oct 2021) support d'applications utilisant JDK 17 améliorations d'injections de dependances (repeatable scopes, primitive beans, etc) les classes générées sont plus petites et amélioration de consommation mémoire sous GraalVM routes HTTP par regexp random port binding (pour les conflits de tests) Changement certificats TLs via refresh sans arreter le serveur Kotlin coroutine supportées dans micronaut data extension de la couverture de support JPA (e.g. attribute converter) support des informers Kubernetes via le Kubernetes SDK integration Oracle Coherence sortie du mode preview Quarkus 2.4 (27 Oct 2021) Hibernate Reactive 1.0.0.Final Introducing Kafka Streams DevUI (c'est cool pour développer ca et savoir ce qui se passe Support continuous testing for multi module projects Support AWT image resize via new AWT extension Lightbend lâche Play Framework (20 Oct 2021) lightbend construit sur Scala, akka, et play framework C'est le moment de la 2.0 je crois Mais avec le cloud, ils veulent se focaliser sur les systèmes distribués Akka Open Source et Akka Serverless (leur PaaS) Laisse Play à la,communité et lightbend arrête d'investir dedans Dans une orga séparée Besoin de sponsors et de contributeurs Question: ils n'avaient pas déjà arrêté Scala? Lightbend déveste de Lagom aussi (27 Oct 2021) Lagom effacé par akka Platform'et Akka Serverless Trop de contraintes limitantes dans le framework Mais si client de Lightbend, supporté sur Lagom mais sans nouvelle fonctionnalité Infrastructure Installer et utiliser podman-machine sur macOS (19 Oct 2021) La virtualisation s'appuie sur qemu et met en place une VM dans laquelle les pods tournent. Podman Machine pour installer une VM linux avec les outils fonctionne aussi sous linux pour ceux qu ne supportent pas podman ou pour sandboxer fonctionne sous M1 homebrew pour l'installation comme docker machine avant en gros il y a aussi une belle présentation de Devoxx France Cloud Amazon déclaré la guerre à Microsoft en utilisant les arguments “Proprietaire” (28 Oct 2021) Aurora a un font qui parler protocole SQL server (Babelfish pour Aurora PostgreSQL). Et convertit les T-SQL Open source the t sql vers Postgres (debug). Sous license ASL Pas tout open sourcé encore Web CORS expliqué (12 Oct 2021) inclue images d'autres sites, c'est l'origine les cookies, credeitials etc etaient envoyés yahoo mail pouvait filer les credentials des utilisateurs une iFrame pouvait lire le contenu d'une autre iFrame (Netscape met en place le Cross-Frame Scripting) Access-Control-Allow-Origin: * est ok si pas de données privées Rendre une page HTML brute jolie en 100 caractères de CSS (16 Oct 2021) basique mais expliqué ligne par ligne E.g. 60–80 caractères pour la lecture Et 100 bytes de plus pour améliorer Data elasticsearch 8.0 will require java 17 (3 Nov 2021) definitely easier for something standalone than a library or anything that needs to share the JDK with all its apps PR GitHub Hibernate Reactive 1.0.0, ça vaut le coup ? (27 Oct 2021) PostgreSQL, MySQL, MariaDB, Db2, SQL Server, and CockroachDB bases de donnés désignées pour des interactions classiques Donc les constructions haut niveau ont tendances à être limitées par le protocole sous-jacent ce qui ne se voyait pas ou peu en JDBC utiliser HR si votre appli est déjà réactive au cœur (e.g. RESTEasy reactive dans Quarkus ou une appli Vert.x) Compareperfs acec techempower mais avec angle latence à un volume donné et et pas throughout max 20 requêtes d'affilée 20k request/s -> 35k sous 10ms de latence. C'est la valeur relative qui est intéressante Une requête et du processing pour rendre au client, peu de différence Toruhghput tend à être meilleur Amélioration de réactive sur un an Un vidéo cast sur le sujet Outillage AtomicJar se lance dans une offre Cloud (04 Nov 2021) les containers de test containers ne tournent plus en local Mais dans le cloud de AtomicJar A plus de spores source qu'une machine locale typique (2 cores et 8GB ram pour la docker machine) peut utiliser la machine quand les tests tournent Pour CI limitées vs containers ou les cloud IDE pour pas trop dépenser Pas de problème avec M1 Un petit binaire à installer (eg via curl) TestContainers et Quarkus: TestContainer Cloud fonctionne avec Dev Service (les containers lancés et configurés automatiquement) Encore en cours de développement (beta privée et on peut demander invitation) Méthodologies Canary releases ou avoir des testeurs (04 Nov 2021) canary release est une release en prod mais sur un petit sous ensemble des utilisateurs Peut aider a voir si une nouvelle fonctionnalité intéresse les utilisateurs avant de commiter sur le long terme Toujours option du retour arrière Donc peut on réduire les tests internes ? Risque de réputation ou abandon utilisateur (acquisition et rétentions sont chères) Test automatisés compréhensifs permettent le risque de canary Test exploratoires pour compléter les tests automatiques Loi, société et organisation Le droit à decompiler pour corriger des erreurs confirmé légal (21 Oct 2021) arrêt du 6 octobre 2021 Pour corriger une erreur affectant le fonctionnement y compris via la désactivation d'une fonction affectant le bon fonctionnement de l'application Influence de l'amplificartion algorithmique sur le contenu politique (21 Octo 2021) les recommendations algorithmiques amplifient-elle le contenu politique ? dans le cas des timeline organisées algorithmiquement et pas reverse chronologique Est-ce que ça varie entre partis politiques ou groups politiques Des sources de nouvelles plus amplifiées que d'autre Les élus sont plus amplifiés que le contenu politique général Pas d'amplification particulière d.individus ces d'autres au sein du même parti ???? La,droite tend à avoir une amplification plus importante que la gauche Les sources de nouvelles orientées à droite sont aussi plus amplifiées que celles de gauche La méthodologie est détaillée sur par exemple ce qu'est un journal de droite Pourquoi c'est amplifié différemment est une question plus difficile à répondre Amplification n'est pas mauvaise par défaut mais elle l'est si elle amène à un traitement préférentiel du à l'algorithme (vs comment les gens interagissent sur la plateforme) Le PDF de l'étude intégrale Conférences DevFest Lille le 19 novembre 2021 Devoxx France du 20 au 22 avril 2021 SunnyTech les 30 juin et 1er juillet 2022 à Montpellier Nous contacter Soutenez Les Cast Codeurs sur Patreon https://www.patreon.com/LesCastCodeurs Faire un crowdcast ou une crowdquestion Contactez-nous via twitter https://twitter.com/lescastcodeurs sur le groupe Google https://groups.google.com/group/lescastcodeurs ou sur le site web https://lescastcodeurs.com/

Screaming in the Cloud
The Future of Google Cloud with Richard Seroter

Screaming in the Cloud

Play Episode Listen Later Nov 11, 2021 40:47


About RichardHe's also an instructor at Pluralsight, a frequent public speaker, and the author of multiple books on software design and development. Richard maintains a regularly updated blog (seroter.com) on topics of architecture and solution design and can be found on Twitter as @rseroter. Links: Twitter: https://twitter.com/rseroter LinkedIn: https://www.linkedin.com/in/seroter Seroter.com: https://seroter.com TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: This episode is sponsored in part by our friends at Vultr. Spelled V-U-L-T-R because they're all about helping save money, including on things like, you know, vowels. So, what they do is they are a cloud provider that provides surprisingly high performance cloud compute at a price that—while sure they claim its better than AWS pricing—and when they say that they mean it is less money. Sure, I don't dispute that but what I find interesting is that it's predictable. They tell you in advance on a monthly basis what it's going to going to cost. They have a bunch of advanced networking features. They have nineteen global locations and scale things elastically. Not to be confused with openly, because apparently elastic and open can mean the same thing sometimes. They have had over a million users. Deployments take less that sixty seconds across twelve pre-selected operating systems. Or, if you're one of those nutters like me, you can bring your own ISO and install basically any operating system you want. Starting with pricing as low as $2.50 a month for Vultr cloud compute they have plans for developers and businesses of all sizes, except maybe Amazon, who stubbornly insists on having something to scale all on their own. Try Vultr today for free by visiting: vultr.com/screaming, and you'll receive a $100 in credit. Thats v-u-l-t-r.com slash screaming.Corey: You know how git works right?Announcer: Sorta, kinda, not really Please ask someone else!Corey: Thats all of us. Git is how we build things, and Netlify is one of the best way I've found to build those things quickly for the web. Netlify's git based workflows mean you don't have to play slap and tickle with integrating arcane non-sense and web hooks, which are themselves about as well understood as git. Give them a try and see what folks ranging from my fake Twitter for pets startup, to global fortune 2000 companies are raving about. If you end up talking to them, because you don't have to, they get why self service is important—but if you do, be sure to tell them that I sent you and watch all of the blood drain from their faces instantly. You can find them in the AWS marketplace or at www.netlify.com. N-E-T-L-I-F-Y.comCorey: Welcome to Screaming in the Cloud. I'm Corey Quinn. Once upon a time back in the days of VH1, which was like MTV except it played music videos, would have a show that was, “Where are they now?” Looking at former celebrities. I will not use the term washed up because that's going to be insulting to my guest.Richard Seroter is a returning guest here on Screaming in the Cloud. We spoke to him a year ago when he was brand new in his role at Google as director of outbound product management. At that point, he basically had stars in his eyes and was aspirational around everything he wanted to achieve. And now it's a year later and he has clearly failed because it's Google. So, outbound products are clearly the things that they are going to be deprecating, and in the past year, I am unaware of a single Google Cloud product that has been outright deprecated. Richard, thank you for joining me, and what do you have to say for yourself?Richard: Yeah, “Where are they now?” I feel like I'm the Leif Garrett of cloud here, joining you. So yes, I'm still here, I'm still alive. A little grayer after twelve months in, but happy to be here chatting cloud, chatting whatever else with you.Corey: I joke a little bit about, “Oh, Google winds up killing things.” And let's be clear, your consumer division which, you know, Google is prone to that. And understanding a company's org chart is a challenge. A year or two ago, I was of the opinion that I didn't need to know anything about Google Cloud because it would probably be deprecated before I really had to know about it. My opinion has evolved considerably based upon a number of things I'm seeing from Google.Let's be clear here, I'm not saying this to shine you on or anything like that; it's instead that I've seen some interesting things coming out of Google that I consider to be the right moves. One example of that is publicly signing multiple ten-year deals with very large, serious institutions like Deutsche Bank, and others. Okay, you don't generally sign contracts with companies of that scale and intend not to live up to them. You're hiring Forrest Brazeal as your head of content for Google Cloud, which is not something you should do lightly, and not something that is a short-term play in any respect. And the customer experience has continued to improve; Google Cloud products have not gotten worse, and I'm seeing in my own customer conversations that discussions about Google Cloud have become significantly less dismissive than they were over the past year. Please go ahead and claim credit for all of that.Richard: Yeah. I mean, the changes a year ago when I joined. So, Thomas Kurian has made a huge impact on some of that. You saw us launch the enterprise APIs thing a while back, which was, “Hey, here's, for the most part, every one of our products that has a fixed API. We're not going to deprecate it without a year's notice, whatever it is. We're not going to make certain types of changes.” Maybe that feels like, “Well, you should have had that before.” All right, all we can do is improve things moving forward. So, I think that was a good change.Corey: Oh, I agree. I think that was a great thing to do. You had something like 80-some-odd percent coverage of Google Cloud services, and great, that's going to only increase with time, I can imagine. But I got a little pushback from a few Googlers for not being more congratulatory towards them for doing this, and look, it's a great thing. Don't get me wrong, but you don't exactly get a whole lot of bonus points and kudos and positive press coverage—not that I'm press—for doing the thing you should have been doing [laugh] all along.It's, “This is great. This is necessary.” And it demonstrates a clear awareness that there was—rightly or wrongly—a perception issue around the platform's longevity and that you've gone significantly out of your way to wind up addressing that in ways that go far beyond just yelling at people on Twitter they don't understand the true philosophy of Google Cloud, which is the right thing to do.Richard: Yeah, I mean, as you mentioned, look, the consumer side is very experimental in a lot of cases. I still mourn Google Reader. Like, those things don't matter—Corey: As do we all.Richard: Of course. So, I get that. Google Cloud—and of course we have the same cultural thing, but at the same time, there's a lifecycle management that's different in Google Cloud. We do not deprecate products that much. You know, enterprises make decade-long bets. I can't be swap—changing databases or just turning off messaging things. Instead, we're building a core set of things and making them better.So, I like the fact that we have a pretty stable portfolio that keeps getting a little bit bigger. Not crazy bigger; I like that we're not just throwing everything out there saying, “Rock on.” We have some opinions. But I think that's been a positive trend, customers seem to like that we're making these long-term bets. We're not going anywhere for a long time and our earnings quarter after quarter shows it—boy, this will actually be a profitable business pretty soon.Corey: Oh, yeah. People love to make hay, and by people, I stretch the term slightly and talk about, “Investment analysts say that Google Cloud is terrible because at your last annual report you're losing something like $5 billion a year on Google Cloud.” And everyone looked at me strangely, when I said, “No, this is terrific. What that means is that they're investing in the platform.” Because let's be clear, folks at Google tend to be intelligent, by and large, or at least intelligent enough that they're not going to start selling cloud services for less than it costs to run them.So yeah, it is clearly an investment in the platform and growth of it. The only way it should be turning a profit at this point is if there's no more room to invest that money back into growing the platform, given your market position. I think that's a terrific thing, and I'm not worried at all about it losing money. I don't think anyone should be.Richard: Yeah, I mean, strategically, look, this doesn't have to be the same type of moneymaker that even some other clouds have to be to their portfolio. Look, this is an important part, but you look at those ten-year deals that we've been signing: when you look at Univision, that's a YouTube partnership; you look at Ford that had to do with Android Auto; you look at these others, this is where us being also a consumer and enterprise SaaS company is interesting because this isn't just who's cranking out the best IaaS. I mean, that can be boring stuff over time. It's like, who's actually doing the stuff that maybe makes a traditional company more interesting because they partner on some of those SaaS services. So, those are the sorts of deals and those sorts of arrangements where cloud needs to be awesome, and successful, and make money, doesn't need to be the biggest revenue generator for Google.Corey: So, when we first started talking, you were newly minted as a director of outbound product management. And now, you are not the only one, there are apparently 60 of you there, and I'm no closer to understanding what the role encompasses. What is your remit? Where do you start? Where do you stop?Richard: Yeah, that's a good question. So, there's outbound product management teams, mostly associated with the portfolio area. So network, storage, AI, analytics, database, compute, application modernization-y sort of stuff—which is what I cover—containers, dev tools, serverless. Basically, I am helping make sure the market understands the product and the product understands the market. And not to be totally glib, but a lot of that is, we are amplification.I'm amplifying product out to market, analysts, field people, partners: “Do you understand this thing? Can I help you put this in context?” But then really importantly, I'm trying to help make sure we're also amplifying the market back to our product teams. You're getting real customer feedback: “Do you know what that analyst thinks? Have you heard what happened in the competitive space?”And so sometimes companies seem to miss that, and PMs poke their head up when I'm about to plan a product or I'm about to launch a product because I need some feedback. But keeping that constant pulse on the market, on customers, on what's going on, I think that can be a secret weapon. I'm not sure everybody does that.Corey: Spending as much time as I do on bills, admittedly AWS bills, but this is a pattern that tends to unfold across every provider I've seen. The keynotes are chock-full of awesome managed service announcements, things that are effectively turnkey at further up the stack levels, but the bills invariably look a lot more like, yeah, we spend a bit of money on that and then we run 10,000 virtual instances in a particular environment and we just treat it like it's an extension of our data center. And that's not exciting; that's not fun, quote-unquote, but it's absolutely what customers are doing and I'm not going to sit here and tell them that they're wrong for doing it. That is the hallmark of a terrible consultant of, “I don't understand why you're doing what you're doing, so it must be foolish.” How about you stop and gain some context into why customers do the things that they do?Richard: No, I send around a goofy newsletter every week to a thousand or two people, just on things I'm learning from the field, from customers, trying to make sure we're just thinking bigger. A couple of weeks ago, I wrote an idea about modernization is awesome, and I love when people upgrade their software. By the way, most people migration is a heck of a lot easier than if I can just get this into your cloud, yeah love that; that's not the most interesting thing, to move VMs around, but most people in their budget, don't have time to rewrite every Java app to go. Everybody's not changing .NET framework to .NET core.Like, who do I think everybody is? No, I just need to try to get some incremental value first. Yes, then hopefully I'll swap out my self-managed SQL database for a Spanner or a managed service. Of course, I want all of that, but this idea that I can turn my line of business loan processing app into a thousand functions overnight is goofy. So, how are we instead thinking more pragmatically about migration, and then modernizing some of it? But even that sort of mindset, look, Google thinks about innovation modernization first. So, also just trying to help us take a step back and go, “Gosh, what is the normal path? Well, it's a lot of migration first, some modernization, and then there's some steady-state work there.”Corey: One of the things that surprised me the most about Google Cloud in the market, across the board, has been the enthusiastic uptake for enterprise workloads. And by enterprise workloads, I'm talking about things like SAP HANA is doing a whole bunch of deployments there; we're talking Big Iron-style enterprise-y things that, let's be honest, countervene most of the philosophy that Google has always held and espoused publicly, at least on conference stages, about how software should be built. And I thought that would cut against them and make it very difficult for you folks to gain headway in that market and I could not have been more wrong. I'm talking to large enterprises who are enthusiastically talking about Google Cloud. I've got a level with you, compared to a year or two ago, I don't recognize the place.Richard: Mmm. I mean, some of that, honestly, in the conversations I have, and whatever I do a handful of customer calls every week, I think folks still want something familiar, but you're looking for maybe a further step on some of it. And that means, like, yes, is everybody going to offer VMs? Yeah, of course. Is everyone going to have MySQL? Obviously.But if I'm an enterprise and I'm doing these generational bets, can I cheat a little bit, and maybe if I partner with a more of an innovation partner versus maybe just the easy next step, am I buying some more relevance for the long-term? So, am I getting into environment that has some really cool native zero-trust stuff? Am I getting into environment with global backend services and I'm not just stitching together a bunch of regional stuff? How can I cheat by using a more innovation vendor versus just lifting and shifting to what feels like hosted software in another cloud? I'm seeing more of that because these migrations are tough; nobody should be just randomly switching clouds. That's insane.So, can I make, maybe, one of these big bets with somebody who feels like they might actually even improve my business as a whole because I can work with Google Pay and improve how I do mobile payments, or I could do something here with Android? Or, heck, all my developers are using Angular and Flutter; aren't I going to get some benefit from working with Google? So, we're seeing that, kind of, add-on effect of, “Maybe this is a place not just to host my VMs, but to take a generational leap.”Corey: And I think that you're positioning yourselves in a way to do it. Again, talk about things that you wouldn't have expected to come out of Google of all places, but your console experience has been first-rate and has been for a while. The developer experience is awesome; I don't need to learn the intricacies of 12 different services for what I'm trying to do just in order to get something basic up and running. I can stop all the random little billing things in my experimental project with a single click, which that admittedly has a confirm, which you kind of want. But it lets you reason about these things.It lets you get started building something, and there's a consistency and cohesiveness to the console that, again, I am not a graphic designer, by any stretch of the imagination. My most commonly used user interface is a green-screen shell prompt, and then I'm using Vim to wind up writing something horrifying, ideally in Python, but more often in YAML. And that has been my experience, but just clicking around the console, it's clear that there was significant thought put into the design, the user experience, and the way of approaching folks who are starting to look very different, from a user persona perspective.Richard: I can—I mean, I love our user research team; they're actually fun to hang out with and watch what they do, but you have to remember, Google as a company, I don't know, cloud is the first thing we had to sell. Did have to sell Gmail. I remember 15 years ago, people were waiting for invites. And who buys Maps or who buys YouTube? For the most part, we've had to build things that were naturally interesting and easy-to-use because otherwise, you would just switch to anything else because everything was free.So, some of that does infuse Google Cloud, “Let's just make this really easy to use. And let's just make sure that, maybe, you don't hate yourself when you're done jumping into a shell from the middle of the console.” It's like, that should be really easy to do—or upgrade a database, or make changes to things. So, I think some of the things we've learned from the consumer good side, have made their way to how we think of UX and design because maybe this stuff shouldn't be terrible.Corey: There's a trope going around, where I wound up talking about the next million cloud customers. And I'm going to have to write a sequel to it because it turns out that I've made a fundamental error, in that I've accepted the narrative that all of the large cloud vendors are pushing, to the point where I heard from so many folks I just accepted it unthinkingly and uncritically, and that's not what I should be doing. And we'll get to what I was wrong about in a minute, but the thinking goes that the next big growth area is large enterprises, specifically around corporate IT. And those are folks who are used to managing things in a GUI environment—which is fine—and clicking around in web apps. Now, it's easy to sit here on our high horse and say, “Oh, you should learn to write code,” or YAML, which is basically code. Cool.As an individual, I agree, someone should because as soon as they do that, they are now able to go out and take that skill to a more lucrative role. The company then has to backfill someone into the role that they just got promoted out of, and the company still has that dependency. And you cannot succeed in that market with a philosophy of, “Oh, you built something in the console. Now, throw it away and do it right.” Because that is maddening to that user persona. Rightfully so.I'm not that user persona and I find it maddening when I have to keep tripping over that particular thing. How did that come to be, from your perspective? First, do you think that is where the next million cloud customers come from? And have I adequately captured that user persona, or am I completely often the weeds somewhere?Richard: I mean, I shared your post internally when that one came out because that resonated with me of how we were thinking about it. Again, it's easy to think about the cloud-native operators, it's Spotify doing something amazing, or this team at Twitter doing something, or whatever. And it's not even to be disparaging. Like, look, I spent five years in enterprise IT and I was surrounded by operators who had to run dozen different systems; they weren't dedicated to just this thing or that. So, what are the tools that make my life easy?A lot of software just comes with UIs for quick install and upgrades, and how does that logic translate to this cloud world? I think that stuff does matter. How are you meeting these people a little better where they are? I think the hard part that we will always have in every cloud provider is—I think you've said this in different forums, but how do I not sometimes rub the data center on my cloud or vice versa? I also don't want to change the experience so much where I degrade it over the long term, I've actually somehow done something worse.So, can I meet those people where they are? Can we pull some of those experiences in, but not accidentally do something that kind of messes up the cloud experience? I mean, that's a fine line to walk. Does that make sense to you? Do you see where there's a… I don't know, you could accidentally cater to a certain audience too much, and change the experience for the worse?Corey: Yes, and no. My philosophy on it is that you have to meet customers where they are, but only to a point. At some point, what they're asking for becomes actively harmful or disadvantageous to wind up providing for them. “I want you to run my data center for me,” is on some level what some cloud environments look like, and I'm not going to sit here and tell people they're inherently wrong for that. Their big reason for moving to the cloud was because they keep screwing up replacing failed hard drives in their data center, so we're going to put it in the cloud.Is it more expensive that way? Well, sure in terms of actual cash outlay, it almost certainly is, but they're also not going down every month when a drive fails, so once the value of that? It's a capability story. That becomes interesting to me, and I think that trying to sit here in isolation, and say that, “Oh, this application is not how we would build it at Google.” And it's, “Yeah, you're Google. They are insert an entire universe of different industries that look nothing whatsoever like Google.” The constraints are different, the resources are different, and—Richard: Sure.Corey: —their approach to problem-solving are different. When you built out Google, and even when you're building out Google Cloud, look at some of the oldest craftiest stuff you have in your entire all of Google environment, and then remember that there are companies out there that are hundreds of years old. It's a different order of magnitude as far as era, as far as understanding of what's in the environment, and that's okay. It's a very broad and very diverse world.Richard: Yeah. I mean, that's, again, why I've been thinking more about migration than even some of the modernization piece. Should you bring your network architecture from on-prem to the cloud? I mean, I think most cases, no. But I understand sometimes that edge firewall, internal trust model you had on-prem, okay, trying to replicate that.So, yeah, like you say, I want to meet people where they are. Can we at least find some strategic leverage points to upgrade aspects of things as you get to a cloud, to save you from yourself in some places because all of a sudden, you have ten regions and you only had one data center before. So, many more rooms for mistakes. Where are the right guardrails? We're probably more opinionated than others at Google Cloud.I don't really apologize for that completely, but I understand. I mean, I think we've loosened up a lot more than maybe people [laugh] would have thought a few years ago, from being hyper-opinionated on how you run software.Corey: I will actually push back a bit on the idea that you should not replicate your on-premises data center in your cloud environment. Sure, are there more optimal ways to do it that are arguably more secure? Absolutely. But a common failure mode in moving from data center to cloud is, “All right, we're going to start embracing this entirely new cloud networking paradigm.” And it is confusing, and your team that knows how the data center network works really well are suddenly in way over their heads, and they're inadvertently exposing things they don't intend to or causing issues.The hard part is always people, not technology. So, when I glance at an environment and see things like that, perfect example, are there more optimal ways to do it? Oh, from a technology perspective, absolutely. How many engineers are working on that? What's their skill set? What's their position on all this? What else are they working on? Because you're never going to find a team of folks who are world-class experts in every cloud? It doesn't work that way.Richard: No doubt. No doubt, you're right. There's areas where we have to at least have something that's going to look similar, let you replicate aspects of it. I think it's—it'll just be interesting to watch, and I have enough conversations with customers who do ask, “Hey, where are the places we should make certain changes as we evolve?” And maybe they are tactical, and they're not going to be the big strategic redesign their entire thing. But it is good to see people not just trying to shovel everything from one place to the next.Corey: This episode is sponsored in part by something new. Cloud Academy is a training platform built on two primary goals. Having the highest quality content in tech and cloud skills, and building a good community the is rich and full of IT and engineering professionals. You wouldn't think those things go together, but sometimes they do. Its both useful for individuals and large enterprises, but here's what makes it new. I don't use that term lightly. Cloud Academy invites you to showcase just how good your AWS skills are. For the next four weeks you'll have a chance to prove yourself. Compete in four unique lab challenges, where they'll be awarding more than $2000 in cash and prizes. I'm not kidding, first place is a thousand bucks. Pre-register for the first challenge now, one that I picked out myself on Amazon SNS image resizing, by visiting cloudacademy.com/corey. C-O-R-E-Y. That's cloudacademy.com/corey. We're gonna have some fun with this one!Corey: Now, to follow up on what I was saying earlier, what I think I've gotten wrong by accepting the industry talking points on is that the next million cloud customers are big enterprises moving from data centers into the cloud. There's money there, don't get me wrong, but there is a larger opportunity in empowering the creation of companies in your environment. And this is what certain large competitors of yours get very wrong, where it's we're going to launch a whole bunch of different services that you get to build yourself from popsicle sticks. Great. That is not useful.But companies that are trying to do interesting things, or people who want to found companies to do interesting things, want something that looks a lot more turnkey. If you are going to be building cloud offerings, that for example, are terrific building blocks for SaaS companies, then it behooves you to do actual investments, rather than just a generic credit offer, into spurring the creation of those types of companies. If you want to build a company that does payroll systems, in a SaaS, cloud way, “Partner with us. Do it here. We will give you a bunch of credits. We will introduce you to your first ten prospective customers.”And effectively actually invest in a company success, as opposed to pitch-deck invest, which is, “Yeah, we'll give you some discounting and some credits, and that's our quote-unquote, ‘investment.'” actually be there with them as a partner. And that's going to take years for folks to wrap their heads around, but I feel like that is the opportunity that is significantly larger, even than the embedded existing IT space because rather than fighting each other for slices of the pie, I'm much more interested in expanding that pie overall. One of my favorite questions to get asked because I think it is so profoundly missing the point is, “Do you think it's possible for Google to go from number three to number two,” or whatever the number happens to be at some point, and my honest, considered answer is, “Who gives a shit?” Because number three, or number five, or number twelve—it doesn't matter to me—is still how many hundreds of billions of dollars in the fullness of time. Let's be real for a minute here; the total addressable market is expanding faster than any cloud or clouds are going to be able to capture all of.Richard: Yeah. Hey, look, whoever who'll be more profitable solving user problems, I really don't care about the final revenue number. I can be the number one cloud tomorrow by making Google Cloud free. What's the point? That's not a sustainable business. So, if you're just going for who can deploy the most VCPUs or who can deploy the most whatever, there's ways to game that. I want to make sure we are just uniquely solving problems better than anybody else.Corey: Sorry, forgive me. I just sort of zoned out for a second there because I'm just so taken aback and shocked by the idea of someone working at a large cloud provider who expresses a philosophy that isn't lying awake at night fretting over the possibility of someone who isn't them as making money somewhere.Richard: [laugh]. I mean, your idea there, it'll be interesting to watch, kind of, the maker's approach of are you enabling that next round of startups, the next round of people who want to take—I mean, honestly, I like the things we're doing building block-wise, even with our AI: we're not just handing you a vision API, we're giving you a loan processing AI that can process certain types of docs, that more packaged version of AI. Same with healthcare, same with whatever. I can imagine certain startups or a company idea going, “Hey, maybe I could disrupt or serve a new market.”I always love what Square did. They've disrupted emerging markets, small merchants here in North America, wherever, where I didn't need a big expensive point of sale system. You just gave me the nice, right building blocks to disrupt and run my business. Maybe Google Cloud can continue to provide better building blocks, but I do like your idea of actually investment zones, getting part of this. Maybe the next million users are founders and it's not just getting into some of these companies with, frankly, 10, 20, 30,000 people in IT.I think there's still plenty of room in these big enterprises to unlock many more of those companies, much more of their business. But to your point, there's a giant market here that we're not all grabbing yet. For crying out loud, there's tons of opportunity out here. This is not zero-sum.Corey: Take it a step further beyond that, and today, if you have someone who's enterprising, early on in their career, maybe they just got out of school, maybe they have just left their job and are ready to snap, or they have some severance money that they want to throw into something. Great. What do they want to do if they have an idea for a company? Well today, that answer looks a lot like, well, time to go to a boot camp and learn to code for six months so you can build a badly done MVP well enough to get off the ground and get some outside investment, and then go from there. Well, what if we cut that part out entirely?What if there were building blocks of I don't need to know or care that there's a database behind it, or what a database looks like. Picture Visual Basic in a web browser for building apps, and just take this bit of information I give you and store it and give it back to me later. Sure, you're going to have some significant challenges in the architecture or something like that as it goes from this thing that I'm talking about as an MVP to something planet-scale—like a Spotify for example—but that's not most businesses, and that's okay. Get out of the way and let people innovate and iterate on what it is they're doing more rapidly, and make it more accessible to teach people. That becomes huge; that gets the infrastructure bits that cloud providers excel at out of the way, and all it really takes is packaging those things into a golden path of what a given company of a particular profile should be doing, if—unless they have reason to deviate from it—and instead of having this giant paradox of choice issue, it's, “Oh, okay, I'll drag-drop, build things accordingly.”And under the hood, it's doing all the configuration of services and that's great. But suddenly, you've made being a founder of a software company—fundamentally—accessible to people who are not themselves software engineers. And I know that's anathema to some people, and I don't even slightly care because I am done with gatekeeping.Richard: Yeah. No, it's exciting if that can pull off. I mean, it's not the years ago where, how much capital was required to find the rack and do all sorts of things with tech, and hire some developers. And it's an amazing time to be software creators, now. The more we can enable that—yeah, I'm along for that journey, sign me up.Corey: I'm looking forward to seeing how it winds up shaking out. So, I want to talk a little bit about the paradox of choice problem that I just mentioned. If you take a look at the various compute services that every cloud provider offers, there are an awful lot of different choices as far as what you can run. There's the VM model, there's containers—if you're in AWS, you have 17 ways to run those—and you wind up—any of the serverless function story, and other things here and there, and managed services, I mean and honestly, Google has a lot of them, nowhere near as many as you do failed messaging products, but still, an awful lot of compute options. How do customers decide?What is the decision criteria that you see? Because the worst answer you can give someone who doesn't really know what they're doing is, “It depends,” because people don't know how to make that decision. It's, “What factors should I consider then, while making that decision?” And the answer has to be something somewhat authoritative because otherwise, they're going to go on the internet and get yelled at by everyone because no one is ever going to agree on this, except that everyone else is wrong.Richard: Mm-hm. Yeah, I mean, on one hand, look, I like that we intentionally have fewer choices than others because I don't think you need 17 ways to run a container. I think that's excessive. I think more than five is probably excessive because as a customer, what is the trade-off? Now, I would argue first off, I don't care if you have a lot of options as a vendor, but boy, the backends of those better be consistent.Meaning if I have a CI/CD tool in my portfolio and it only writes to two of them, shame on me. Then I should make sure that at least CI/CD, identity management, log management, monitoring, arguably your compute runtime should be a late-binding choice. And maybe that's blasphemous because somebody says, “I want to start up front knowing it's a function,” or, “I want to start it's a VM.” How about, as a developer, I couldn't care less. How about I just build cool software and maybe even at deploy time, I say, “This better fits in running in Kubernetes.” “This is better in a virtual machine.”And my cost of changing that later is meaningless because, hey, if it is in the container, I can switch it between three or four different runtimes, the identity management the same, it logs the exact same way, I can deploy CI/CD the same way. So, first off, if those things aren't the same, then the vendor is messing up. So, the customer shouldn't have to pay the cost of that. And then there gets to be other actual criteria. Look, I think you are looking at the workload itself, the team who makes it, and the strategy to figure out the runtime.It's easy for us. Google Compute Engine for VMs, containers go in GKE, managed services that need some containers, there are some apps around them, are Cloud Functions and Cloud Run. Like, it's fairly straightforward and it's going to be an OR situation—or an AND situation not an OR, which is great. But we're at least saying the premium way to run containers in Google Cloud for systems is GKE. There you go. If you do have a bunch of managed services in your architecture and you're stitching them together, then you want more serverless things like Cloud Run and Cloud Functions. And if you want to just really move some existing workload, GCE is your best choice. I like that that's fairly straightforward. There's still going to be some it depends, but it feels better than nine ways to run Kubernetes engines.Corey: I'm sure we'll see them in the fullness of time.Richard: [laugh].Corey: So, talk about Anthos a bit. That was a thing that was announced a while back and it was extraordinarily unclear what it was. And then I looked at the pricing and it was $10,000 a month with a one-year minimum commitment, and is like, “Oh, it's not for me. That's why I don't get it.” And I haven't really looked back at it since. But it is something else now. It almost feels like a wrapper brand, in some respects. How's it going? [unintelligible 00:29:26]?Richard: Yeah. Consumption, we'll talk more upcoming months on some of the adoption, but we're finally getting the hockey stick, which always comes delayed with platforms because nobody adopts platforms quickly. They buy the platform and a year later they start to actually build new development, migrate the things they have. So, we're starting to see the sort of growth. But back to your first point. And I even think I poorly tried to explain it a year ago with you. Basically, look, Anthos is the ability to manage fleets of GKE clusters, wherever they are. I don't care if they're on-prem, I don't care if they're in Google Cloud, I don't care if they're Amazon. We have one customer who only uses Anthos on AWS. Awesome, rock on.So, how do I put GKE clusters everywhere, but then do fleet management because look, some people are doing an app per cluster. They don't want to jam 50 apps in the cluster from different teams because they don't like the idea that this app requires root access; now you can screw around with mine. Or, you didn't update; that broke the cluster. I don't want any of that. So, you're going to see companies more, doing even app per cluster, app per developer per cluster.So, now I have a fleet problem. How do I keep it in sync? How do I make sure policy is consistent? Those sorts of things. So, Anthos is kind of solving the fleet management challenge and replacing people's first-gen app platform.Seeing a lot of those use cases, “Hey, we're retiring our first version of Docker Enterprise, Mesos, Cloud Foundry, even OpenShift,” saying, “All right, now's the time for our next version of our app platform. How about GKE, plus Cloud Run on top of it, plus other stuff?” Sounds good. So, going well is a, sort of—as you mentioned, there's a brand story here, mainly because we've also done two things that probably matter to you. A, we changed the price a lot.No minimum commit, remarkably at 20% of the cost it was when we launched, on purpose because we've gotten better at this. So, much cheaper, no minimum commit, pay as you go. Be on-premises, on bare metal with GKE. Pay by the hour, I don't care; sounds great. So, you can do that sort of stuff.But then more importantly, if you're a GKE customer and you just want config management, service mesh, things like that, now you can buy all of those independently as well. And Anthos is really the brand for fleet management of GKE. And if you're on Google Cloud only, it adds value. If you're off Google Cloud, if you're multi-cloud, I don't care. But I want to manage fleets of compute clusters and create them. We're going to keep doubling down on that.Corey: The big problem historically for understanding a lot of the adoption paradigm of Kubernetes has been that it was, to some extent, a reimagining of how Google ran and built software internally. And I thought at the time, the idea was—from a cynical perspective—that, “All right, well, your crappy apps don't run well on Google-style infrastructure so we're going to teach the entire world how to write software the way that we do.” And then you end up with people running their blog on top of Kubernetes, where it's one of those, like, the first blog post is, like, “How I spent the last 18 months building Kubernetes.” And, okay, that is certainly a philosophy and an approach, but it's almost approaching Windows 95 launch level of hype, where people who didn't own computers were buying copies of it, on some level. And I see the term come up in conversations in places where it absolutely has no place being brought up. “How do I run a Kubernetes cluster inside of my laptop?” And, “It's what you got going on in there, buddy?”Richard: [laugh].Corey: “What do you think you're trying to do here because you just said something that means something that I think is radically different to me than it is to you.” And again, I'm not here to judge other people's workflows; they're all terrible, except for mine, which is an opinion held by everyone about their own workflow. But understanding where people are, figuring out how to get there, how to meet customers where they are and empower them. And despite how heavily Google has been into the Kubernetes universe since its inception, you're very welcoming to companies—and loud-mouth individuals on Twitter—who have no use for Kubernetes. And working through various products you offer, I don't ever feel like a second-class citizen. There's really something impressive about that, of not letting the hype dictate the product and marketing decisions of it.Richard: Yeah, look, I think I tweeted it recently, I think the future of software is managed services with containers in the gap, for the most part. Whereas—if you can use managed services, please do. Use them wherever you can. And if you have to sling some code, maybe put it in a really portable thing that's really easy to run in lots of places. So, I think that's smart.But for us, look, I think we have the best container workflow from dev tools, and build tools, and artifact registries, and runtimes, but plenty of people are running containers, and you shouldn't be running Kubernetes all over the place. That makes sense for the workload, I think it's better than a VM at the retail edge. Can I run a small cluster, instead of a weird point-of-sale Windows app? Maybe. Maybe it makes sense to have a lightweight Kubernetes cluster there for consistency purposes.So, for me, I think it's a great medium for a subset of software. Google Cloud is going to take whatever you got, which is great. I think containers are great, but at the same time, I'm happily going to let you deploy a function that responds to you adding a storage item to a bucket, where at the same time give you a SaaS service that replaces the need for any code. All of those are terrific. So yeah, we love Kubernetes. We think it's great. We're going to be the best version to run it. But that's not going to be your whole universe.Corey: No, and I would argue it absolutely shouldn't be.Richard: [laugh]. Right. Agreed. Now again, for some companies, it's a great replacement for this giant fleet of VMs that all runs at eight percent utilization. Can I stick this into a bunch of high-density clusters? Absolutely you should. You're going to save an absolute fortune doing that and probably pick up some resilience and functionality benefits.But to your point, “Do I want to run a WordPress site in there?” I don't know, probably not. “Do I need to run my own MySQL?” I'd prefer you not do that. So, in a lot of cases, don't use it unless you have to. That should go for all compute nowadays. Use managed services.Corey: I'm a big believer in going down that approach just because it is so much easier than trying to build it yourself from popsicle sticks because you theoretically might have to move it someday in the future, even though you're not.Richard: [laugh]. Right.Corey: And it lets me feel better about a thing that isn't going to be used by anything that I'm doing in the near future. I just don't pretend to get it.Richard: No, I don't install a general purpose electric charger in my garage for any electric car I may get in the future; I charge for the one I have now. I just want it to work for my car; I don't want to plan for some mythical future. So yeah, premature optimization over architecture, or death in IT, especially nowadays where speed matters, don't waste your time building something that can run in nine clouds.Corey: Richard, I want to thank you for coming on again a year later to suffer my slings, arrows, and other various implements of misfortune. If people want to learn more about what you're doing, how you're doing it, possibly to pull a Forrest Brazeal and go work with you, where can they find you?Richard: Yeah, we're a fun place to work. So, you can find me on Twitter at @rseroter—R-S-E-R-O-T-E-R—hang out on LinkedIn, annoy me on my blog seroter.com as I try to at least explore our tech from time to time and mess around with it. But this is a fun place to work. There's a lot of good stuff going on here, and if you work somewhere else, too, we can still be friends.Corey: Thank you so much for your time today. Richard Seroter, director of outbound product management at Google. I'm Cloud Economist Corey Quinn and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice along with an angry comment into which you have somehow managed to shove a running container.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.

North Meets South Web Podcast
Authoring Statamic content with Jesse Leite

North Meets South Web Podcast

Play Episode Listen Later Nov 8, 2021 63:31


In this episode, Jake and Michael are joined by Statamic's Jesse Leite to discuss some of the various content authoring strategies available in the flat-file content management system, Statamic.This episode is sponsored by Makeable.dk and Workvivo and was streamed live.Show links Jesse Leite Statamic

Screaming in the Cloud
The Mayor of Wholesome Twitter with Mark Thompson

Screaming in the Cloud

Play Episode Listen Later Oct 28, 2021 41:18


About MarkMark loves to teach and code.He is an award winning university instructor and engineer. He comes with a passion for creating meaningful learning experiences. With over a decade of developing solutions across the tech stack, speaking at conferences and mentoring developers he is excited to continue to make an impact in tech. Lately, Mark has been spending time as a Developer Relations Engineer on the Angular Team.Links:Twitter: https://twitter.com/marktechson TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: This episode is sponsored in part by our friends at Vultr. Spelled V-U-L-T-R because they're all about helping save money, including on things like, you know, vowels. So, what they do is they are a cloud provider that provides surprisingly high performance cloud compute at a price that—while sure they claim its better than AWS pricing—and when they say that they mean it is less money. Sure, I don't dispute that but what I find interesting is that it's predictable. They tell you in advance on a monthly basis what it's going to going to cost. They have a bunch of advanced networking features. They have nineteen global locations and scale things elastically. Not to be confused with openly, because apparently elastic and open can mean the same thing sometimes. They have had over a million users. Deployments take less that sixty seconds across twelve pre-selected operating systems. Or, if you're one of those nutters like me, you can bring your own ISO and install basically any operating system you want. Starting with pricing as low as $2.50 a month for Vultr cloud compute they have plans for developers and businesses of all sizes, except maybe Amazon, who stubbornly insists on having something to scale all on their own. Try Vultr today for free by visiting: vultr.com/screaming, and you'll receive a $100 in credit. Thats v-u-l-t-r.com slash screaming.Corey: This episode is sponsored in part by something new. Cloud Academy is a training platform built on two primary goals. Having the highest quality content in tech and cloud skills, and building a good community the is rich and full of IT and engineering professionals. You wouldn't think those things go together, but sometimes they do. Its both useful for individuals and large enterprises, but here's what makes it new. I don't use that term lightly. Cloud Academy invites you to showcase just how good your AWS skills are. For the next four weeks you'll have a chance to prove yourself. Compete in four unique lab challenges, where they'll be awarding more than $2000 in cash and prizes. I'm not kidding, first place is a thousand bucks. Pre-register for the first challenge now, one that I picked out myself on Amazon SNS image resizing, by visiting cloudacademy.com/corey. C-O-R-E-Y. That's cloudacademy.com/corey. We're gonna have some fun with this one!Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. Anyone who has the misfortune to follow me on Twitter is fairly well aware that I am many things: I'm loud, obnoxious, but snarky is most commonly the term applied to me. I've often wondered, what does the exact opposite of someone who is unrelentingly negative about things in cloud look like? I'm here to answer that question is lightness and happiness and friendliness on Twitter, personified. His Twitter name is @marktechson. My guest today is Mark Thompson, developer relations engineer at Google. Mark, thank you for joining me.Mark: Oh, I'm so happy to be here. I really appreciate you inviting me. Thanks.Corey: Oh, by all means. I'm glad we're doing these recordings remotely because I strongly suspect, just based upon the joy and the happiness and the uplifting aspects of what it is that you espouse online that if we ever shook hands, we'd explode as we mutually annihilate each other like matter and antimatter combining.Mark: Feels right. [laugh].Corey: So, let's start with the day job; seems like the easy direction to go in. You're a developer relations engineer. Now, I've heard of developer advocates, I've heard of the DevRel term, a lot of them get very upset when I refer to them as ‘devrelopers', but that's the game that we play with language. What is the developer relations engineer?Mark: So, I describe my job this way: I like to help external communities with our products. I work on the Angular team, so I like to help our external communities but then I also like to work with our internal team to help improve our product. So, I see it as helping as a platform, as a developer relations engineer. But the engineer part is, I think, is important here because, at Google, we still do coding and we still write things; I'm going to contribute to the Angular platform itself versus just only giving talks or only writing blog posts to creating content, they still want us to do things like solve problems with the platform as well.Corey: So, this is where my complete and abject lack of understanding of the JavaScript ecosystem enters the conversation. Let's be clear here, first let me check my assumptions. Angular is a JavaScript framework, correct?Mark: Technically a TypeScript framework, but you could say JavaScript.Corey: Cool. Okay, again, this is not me setting you up for a joke or anything like that. I try to keep my snark to Twitter, not podcast because that tends to turn an awful lot into me berating people, which I try to reserve for those who really have earned it; they generally have the word chief somewhere in their job title. So, I'm familiar with sort of an evolution of the startups that I worked at where Backbone was all the rage, followed by, “Oh, you should never use Backbone. You should be using Angular instead.”And then I sort of—like, that was the big argument the last time I worked in an environment like that. And then I see things like View and React and several other things. At some point, it seems like, pick a random name out of the air; if it's not going to be a framework, it's going to be a Pokemon. What is the distinguishing characteristic or characteristics of Angular?Mark: I like to describe Angular to people is that the value-add is going to be some really incredible developer ergonomics. And when I say that I'm thinking about the tooling. So, we put a lot of work into making sure that the tooling is really strong for developers, where you can jump in, you can get started and be productive. Then I think about scale, and how your application runs at scale, and how it works at scale for your teams. So, scale becomes a big part of the story that I tell, as well, for Angular.Corey: You spend an awful lot of time telling stories about Angular. I'm assuming most of them are true because people don't usually knowingly last very long in this industry when they just get up on stage and tell lies, other than, “This is how we do it in our company,” which is the aspirational conference-ware that we all wish we ran. You're also, according to your bio, which of course, is always in the [show notes 00:04:16], you're an award-winning university instructor. Now, award-winning—great. For someone who struggled mightily in academia, I don't know much about that world. What is it that you teach? How does being a university instructor work? I imagine it's not like most other jobs where you wind up showing up, solving algorithms on a whiteboard, and they say, “Great, can you start tomorrow?”Mark: Sure. So, when I was teaching at university, what I was teaching was mostly coding bootcamps. So, some universities have coding bootcamps that they run themselves. And so I was a part of some instructional teams that work in the university. And that's how I won the Teaching Excellence Award. So, the award that I won actually was the Distinguished Teaching Excellence Award, based on my performance at work when I was teaching at university.Corey: I want to be clear here, it's almost enough to make someone question whether you really were involved there because the first university, according to your background that you worked on was Northwestern, but then it was through the Harvard Extension School, and I was under the impression that doing anything involving Harvard was the exact opposite of an NDA, where you're contractually bound to mention that, “Oh, I was involved with Harvard in the following way,” at least three times at any given conversation. Can you tell I spent a lot of time dealing with Harvard grads?Mark: [laugh]. Yeah, Harvard is weird like that, where people who've worked there or gone there, it comes up as a first thing. But I'll tell the story about it if someone asks me, but I just like to talk about univer—that's why I say ‘university,' right? I don't say, “Oh, I won an award at Northwestern.” I just say, “University award-winning instructor.”The reason I say even the ‘award-winning', that part is important for credibility, specifically. It's like, hey, if I said I'm going to teach you something, I want you to know that you're in really good hands, and that I'm really going to do my best to help you. That's why I mention that a lot.Corey: I'll take that even one step further, and please don't take this as in any way me casting aspersions on some of your colleagues, but very often working at Google has felt an awful lot like that in some respects. I've never seen you do it. You've never had to establish your bona fides in a conversation that I've seen by saying, “Well, at Google this is how we do it.” Because that's a logical fallacy of appeal to authority in many respects. Yeah, I'm sure you do a lot of things at Google at a multinational trillion-dollar company that if I'm founding a four-person startup called Twitter for Pets might not necessarily be the same constraints that I'm faced with.I'm keenly appreciative folks who recognize that distinction and don't try and turn it into something else. We see it with founders, too, “Oh, we're a small scrappy startup and our founders used to work at Google.” And it's, “Hmm, I'm wondering if the corporate culture at a small startup might be slightly different these days.” I get it. It does resonate and it carries weight. I just wonder if that's one of those unexamined things that maybe it's time to dive into a bit more.Mark: Hmm. So, what's funny about that is—so people will ask me, what do I do? And it really depends on context. And I'll usually say, “Oh, I work for a company on the West Coast,” or, “For a tech company on the West Coast.” I'll just say that first.Because what I really want to do is turn the conversation back to the person I'm talking to, so here's where that unrelenting positivity kind of comes in because I'm looking at ways, how can I help boost you up? So first, I want to hear more about you. So, I'll kind of like—I won't shrink myself, but I'll just be kind of vague about things so I could hear more about you so we're not focused on me. In this case, I guess we are because I'm the guest, but in a normal conversation, that's what I would try to do.Corey: So, we've talked about JavaScript a little bit. We've talked about university a smidgen. Now, let me complete the trifecta of things that I know absolutely nothing about, specifically positivity on Twitter. You have been described to me as the mayor of wholesome Twitter. What is that about?Mark: All right, so let me be really upfront about this. This is not about toxic positivity. We got to get that out in the open first, before I say anything else because I think that people can hear that and start to immediately think, “Oh, this guy is just, you know, toxic positivity where no matter what's happening, he's going to be happy.” That is not the same thing. That is not the same thing at all.So, here's what I think is really interesting. Online, and as you know, as a person on Twitter, there's so many people out there doing damage and saying hurtful things. And I'm not talking about responding to someone who's being hurtful by being hurtful. I mean the people who are constantly harassing women online, or our non-binary friends, people who are constantly calling into question somebody's credibility because of, oh, they went to a coding bootcamp or they came from self-taught. All these types of ways to be really just harmful on Twitter.I wanted to start adding some other perspective of the positivity side of just being focused on value-add in our interactions. Can I craft this narrative, this world, where when we meet, we're both better off because of it, right? You feel good, I feel good, and we had a really good time. If we meet and you're having a bad time, at least you know that I care about you. I didn't fix you. I didn't, like, remove the issue, but you know that somebody cares about you. So, that's what I think wholesome positivity comes into play is because I want to be that force online. Because we already have plenty of the other side.Corey: It's easy for folks who are casual observers of my Twitter nonsense to figure, “Oh, he's snarky and he's being clever and witty and making fun of big companies”—which I do–And they tend to shorthand that sometimes to, “Oh, great. He's going to start dunking on people, too.” And I try mightily to avoid that it's punch up, never down.Mark: Mm-hm.Corey: I understand there's a school of thought that you should never be punching at all, which I get. I'm broken in many ways that apparently are entertaining, so we're going to roll with that. But the thing that incenses me the most—on Twitter in my case—is when I'll have something that I'll put out there that's ideally funny or engaging and people like it and it spreads beyond my circle, and then you just have the worst people on the internet see that and figure, “Oh, that's snarky and incisive. Ah, I'm like that too. This is my people.”I assure you, I am not your people when that is your approach to life. Get out of here. And curating the people who follow and engage with you on Twitter can be a full-time job. But oh man, if I wind up retweeting someone, and that act brings someone who's basically a jackwagon into the conversation, it's no. No-no-no.I'm not on Twitter to actively make things worse unless you're in charge of cloud pricing, in which case yes, I am very much there to make your day worse. But it's, “Be the change you want to see in the world,” and lifting people up is always more interesting to me than tearing people down.Mark: A thousand percent. So, here's what I want to say about that is, I think, punching up is fine. I don't like to moderate other people's behavior either, though. So, if you'd like punching up, I think it'd be funny. I laugh at jokes that people make.Now, is it what I'll do? Probably not because I haven't figured out a good way for me to do it that still goes along my core values. But I will call out stuff. Like if there's a big company that's doing something that's pretty messed up, I feel comfortable calling things out. Or when drama happens and people are attacking someone, I have no problem with just be like, “Listen, this person is a stand-up person.”Putting myself kind of like… just kind of on the front line with that other person. Hey, look, this person is being attacked right now. That person is stand-up, so if you got a problem them, you got a problem with me. That's not the same thing as being negative, though. That's not the same thing as punching down or harming people.And I think that's where—like I say, people kind of get that part confused when they think that being kind to people is a sign of weakness, which is—it takes more strength for me to be kind to people who may or may not deserve it, by societal standards. That I'll try to understand you, even though you've been a jerk right now.Corey: Twitter excels at fomenting outrage, and it does it by distancing us from being able to easily remember there's a person on the other side of these things. It is ways you're going to yell at someone, even my business partner in a text message. Whenever we start having conversations that get a little heated—which it happens; business partnership is like a marriage—it's oh, I should pick up the phone and call him rather than sending things that stick around forever, that don't reflect the context of the time, and five years later when I see it, I feel ashamed." I'm not here to advocate for other people doing things on Twitter the way that I do because what I do is clever, but the failure mode of clever in my case is being a complete jerk, and I've made that mistake a lot when I was learning to do it when my audience was much smaller, and I hurt people. And whenever I discovered that that is what happened, I went out of my way, and still do, to apologize profusely.I've gotten relatively good at having to do less of those apologies on an ongoing basis, but very often people see what I'm doing and try to imitate what they're seeing; it just comes off as mean. And that's not acceptable. That's not something that I want to see more of in the world. So, those are my failure modes. I have to imagine the only real failure mode that you would encounter with positivity is inadvertently lifting someone up who turns out to be a trash goblin.Mark: [laugh]. That and I think coming off as insincere. Because if someone is always positive or a majority of the time, positive, if I say something to you, and you don't know me that actually mean it, sincerity is incredibly hard to get over text. So, if I congratulate you on your job, you might be like, “Oh, he's just saying that for attention for himself because now he's being the nice guy again.” But sincerity is really, really hard to convey, so that's one of the failure modes is like I said, being sincere.And then lifting up people who don't deserve to be lifted up, yeah, that's happened before where I've engaged with people or shared some of their stuff in an effort to boost them, and find out, like you said, legit trash goblin, like, their home address is under a bridge because they're a troll. Like, real bad stuff. And then you have back off of that endorsement that you didn't know. And people will DM you, like, “Hey, I see that you follow this person. That person is a really bad person. Look at what they're saying right now.” I'm like, “Well, damn, I didn't know it was bad like that.”Corey: I've had that on the podcast, too, where I'll have a conversation with someone and then a year or so later, they'll wind up doing something horrifying, or something comes to light and the rest, and occasionally people will ask, “So, why did you have that person on this show?” It's yeah, it turns out that when we're having a conversation, that somehow didn't come up because as I'm getting background on people and understanding who they are and what they're about in the intake questionnaire, there is not a separate field for, “Are you terrible to women?” Maybe there should be, but that's something that it's—you don't see it. And that makes it easy to think that it's not there until you start listening more than you speak, and start hearing other people's stories about it. This is the challenge.As much as I aspire at times to be more positive and lift folks up, this is the challenge of social media as it stands now. I had a tweet the other day about a service that AWS had released with the comment that this is fantastic and the team that built it should be proud. And yeah, that got a bit of engagement. People liked it. I'm sure it was passed around internally, “Yay, the jerk liked something.” Fine.A month ago, they launched a different service, and my comment was just distilled down to, “This is molten garbage.” And that went around the tech internet three times. When you're positive, it's one of those, “Oh, great. Yeah, that's awesome.” Whereas when I savage things, it's, “Hey, he's doing it again. Come and look at the bodies.” Effectively the rubbernecking thing. “There's been a terrible accident, let's go gawk at it.”Mark: Right.Corey: And I don't quite know what to do with that because it leads to the mistaken and lopsided impression that I only ever hate things and I don't think that a lot of stuff is done well. And that's very much not the case. It doesn't restrict itself to AWS either. I'm increasingly impressed by a lot of what I'm seeing out of Google Cloud. You want to talk about objectivity, I feel the same way about Oracle Cloud.Dunking on Oracle was a sport for me for a long time, but a lot of what they're doing on a technical and on a customer-approach basis in the cloud group is notable. I like it. I've been saying that for a couple of years. And I'm gratified the response from the audience seems to at least be that no one's calling me a shill. They're saying, “Oh, if you say it, it's got to be true.” It's, “Yes. Finally, I have a reputation for authenticity.” Which is great, but that's the reason I do a lot of the stuff that I do.Mark: That is a tough place to be in. So, Twitter itself is an anomaly in terms of what's going to get engagement and what isn't. Sometimes I'll tweet something that at least I think is super clever, and I'm like, “Oh, yeah. This is meaningful, sincere, clever, positive. This is about to go bananas.” And then it'll go nowhere.And then I'll tweet that I was feeling a depression coming on and that'll get a lot of engagement. Now, I'm not saying that's a bad thing. It's just, it's never what I think. I thought that the depression tweet was not going to go anywhere. I thought that one was going to be like, kind of fade into the ether, and then that is the one that gets all the engagement.And then the one about something great that I want to share, or lifting somebody else up, or celebrating somebody that doesn't go anywhere. So, it's just really hard to predict what people are going to really engage with and what's going to ring true for them.Corey: Oh, I never have any idea of how jokes are going to land on Twitter. And in the before times, I had the same type of challenge with jokes in conference talks, where there's a joke that I'll put in there that I think is going to go super well, and the audience just sits there and stares. That's okay. My jokes are for me, but after the third time trying it with different audiences and no one laughs, okay, I should keep it to myself, then. Other times just a random throwaway comment, and I find it quoted in the newspaper almost. And it's, “Oh, okay.”Mark: [laugh].Corey: You can never tell what's going to hit and what isn't.Mark: Can we talk about that though? Like—Corey: Oh, sure.Mark: Conference talking?Corey: Oh, my God, no.Mark: Conference speaking, and just how, like—I remember one time I was keynoting—well I was emceeing and I had the opening monologue. And so [crosstalk 00:17:45]—Corey: We call that a keynote. It's fine. It is—I absolutely upgrade it because people know what you're talking about when you say, “I keynoted the thing.” Do it. Own it.Mark: Yeah.Corey: It's yours.Corey: So, I was emcee and then I did the keynote. And so during the keynote rehearsals—and this is for all the academia, right, so all these different university deans, et cetera. So, in the practice, I'm telling this joke, and it is landing, everybody's laughing, blah, blah, blah. And then I get in there, and it was crickets. And in that moment, you want to panic because you're like, “Holy crap, what do I do because I was expecting to be able to ride the wave of the laughter into my next segment,” and now it's dead silent. And then just that ability to have to be quick on your feet and not let it slow you down is just really hard.Corey: This episode is sponsored by our friends at Oracle HeatWave is a new high-performance accelerator for the Oracle MySQL Database Service. Although I insist on calling it “my squirrel.” While MySQL has long been the worlds most popular open source database, shifting from transacting to analytics required way too much overhead and, ya know, work. With HeatWave you can run your OLTP and OLAP, don't ask me to ever say those acronyms again, workloads directly from your MySQL database and eliminate the time consuming data movement and integration work, while also performing 1100X faster than Amazon Aurora, and 2.5X faster than Amazon Redshift, at a third of the cost. My thanks again to Oracle Cloud for sponsoring this ridiculous nonsense.Corey: It's a challenge. It turns out that there are a number of skills that are aligned but are not the same when it comes to conference talks, and I think that is something that is not super well understood. There's the idea of, “I can get on stage in front of a bunch of people with a few loose talking points, and just riff,” that sort of an improv approach. There's the idea of, “Oh, I can get on stage with prepared slides and have presenter notes and have a whole direction and theme of what I'm doing,” that's something else entirely. But now we're doing video and the energy is completely different.I've presented live on video, I've done pre-recorded video, but in either case, you're effectively talking to the camera and there is no crowd feedback. So, especially if you'd lean on jokes like I tend to, you can't do a cheesy laugh track as an insert, other than maybe once as its own joke. You have to make sure that you can resonate and engage with folks, but there are no subtle cues from the audience like half the front row getting up and walking out. You have to figure out what it is that resonates, what it is that doesn't, why people should care. And of course, distinguishing and differentiating between this video that you're watching now and the last five Zoom meetings that you've been on that look an awful lot the same; why should you care about this talk?Mark: The hardest thing to do. I think speaking remotely became such a big challenge. So, over time it became a little easier because I found some of the value in it, but it was still much harder because of all the things that you said. What became easier was that I didn't have to go to a place. That was easier.So, I could take three different conference talks in a day for three different organizations. So, that was easier. But what was harder, just like you said, not being able to have that energy of the crowd to know when you're on point because you look for that person in the audience who's nodding in agreement, or the person who's shaking their head furiously, like, “Oh, this is all wrong.” So, you might need to clarify or slow down or—you lose all your cues, and that's just really, really hard. And I really don't like doing video pre-recorded talks because those take more energy for me than they do the even live virtual because I have to edit it and I have to make sure that take was right because I can't say, “Oh, excuse me. Well, I meant to say this.”And I guess I could leave that in there, but I'm too much of a—I love public speaking, so I put so much pressure on myself to be the best version of myself at every opportunity when I'm doing public speaking. And I think that's what makes it hard.Corey: Oh, yeah. Then you add podcasts into the mix, like this one, and it changes the entire approach. If I stumble over my words in the middle of a sentence that I've done a couple of times already, on this very show, I will stop and repeat myself because it's easier to just cut that out in post, and it sounds much more natural. They'll take out ums, ahs, stutters, and the rest. Live, you have to respond to that very differently, but pre-recorded video has something of the same problem because, okay, the audio you can cut super easily.With video, you have to sort of a smear, and it's obvious when people know what they're looking at. And, “Wait, what was that? That was odd. They blew a take.” You can cheat, which is what I tend to do, and oh, I wind up doing a bunch of slides in some of my talks because every slide transition is an excuse to cut because suddenly for a split second I'm not on the camera and we can do all kinds of fun things.But it's all these little things, and part of the problem, too, with the pandemic was, we suddenly had to learn how to be A/V folks when previously we had the good fortune slash good sense to work with people who are specialist experts in this space. Now it's, “Well, I guess I am the best boy grip today,” whate—I'm learning what that means [laugh] as we—Mark: That's right.Corey: —continue onward. Ugh. I never signed up for this, but it's the thing that happens to you instead of what you plan on. I think that's called life.Mark: Feels right. Feels right, yeah. It's just one of those things. And I'm looking forward to the time after this, when we do get back to in-person talks, and we do get to do some things. So, I have a lot of hot takes around speaking. So, I came up in Toastmasters. Are you familiar with Toastmasters at all?Corey: I very much am.Mark: Oh, yeah. Okay, so I came up in Toastmasters, and for people at home who don't know, it's kind of like a meetup where you go and you actually practice public speaking, based on these props, et cetera. For me, I learned to do things like not say ‘um' and ‘ah' on stage because there's someone in the room counting every time you do it, and then when you get that review at the end when they give you your feedback, they'll call that out. Or when you say ‘like you know,' or too many ‘and so', all these little—I think the word is disfluencies that you use that people say make you sound more natural, those are things that were coached out with me for public speaking. I just don't do those things anymore, and I feel like there are ways for you not to do it.And I tweeted that before, that you shouldn't say ‘um' and ‘ah' and have someone tell me, “Oh, no, they're a natural part of language.” And then, “It's not natural and it could freak people out.” And I was like, “Okay. I mean, you have your opinion about that.” Like, that's fine, but it's just a hot take that I had about speaking.I think that you should do lots of things when you speak. The rate that you walk back and forth, or should you be static? How much should be on your slides? People put a lot of stuff on slides, I'm like, “I don't want to read your slides. I'd rather listen to you use your slides.” I mean, I can go on and on. We should have another podcast called, “Hey, Mark talks about public speaking,” because that is one of my jams. That and supporting people who come from different paths. Those two things, I can go on for hours about.Corey: And they're aligned in a lot of respects. I agree with you on the public speaking. Focusing on the things that make you a better speaker are not that hard in most cases, but it's being aware of what you're doing. I thought I was a pretty good speaker when I had a coach for a little while, and she would stand there, “Give just the first minute of your talk.” And she's there and writing down notes; I get a minute in and it's like, “Okay, I can't wait to see what she doesn't like once I get started.” She's like, “Nope. I have plenty. That will cover us for the next six weeks.” Like, “O…kay? I guess she doesn't know what she's doing.”Spoiler she did, in fact, know what she was doing and was very good at it and my talks are better for it as a result. But it comes down to practicing. I didn't have a thing like Toastmasters when I was learning to speak to other folks. I just did it by getting it wrong a lot of times. I would speak to small groups repeatedly, and I'd get better at it in time.And I would put time-bound on it because people would sit there and listen to me talk and then the elevator would arrive at our floor and they could escape and okay, they don't listen to me publicly speaking anymore, but you find time to practice in front of other folks. I am kidding, to be clear. Don't harass strangers with public speaking talks. That was in fact a joke. I know there's at least one person in the audience who's going to hear that and take notes and think, “Ah, I'm going to do that because he said it's a good idea.” This is the challenge with being a quote-unquote, “Role model” sometimes. My role model approach is to give people guidance by providing a horrible warning of what not to do.Mark: [laugh].Corey: You've gone the other direction and that's kind of awesome. So, one of the recurring themes of this show has been, where does the next generation come from? Where do we find the next generation of engineer, of person working in cloud in various ways? Because the paths that a lot of us walked who've been in this space for a decade or more have been closed. And standing here, it sounds an awful lot like, “Oh, go in and apply for jobs with a firm handshake and a printed copy of your resume and ask to see the manager and you'll have a job before dark.”Yeah, what worked for us doesn't work for people entering the workforce today, and there have to be different paths. Bootcamps are often the subject of, I think, a deserved level of scrutiny because quality differs wildly, and from the outside if you don't know the space, a well-respected bootcamp that knows exactly what it's doing and has established long-term relationships with a number of admirable hiring entities in the space and grifter who threw together a website look identical. It's a hard problem to solve. How do you view teaching the next generation and getting them into this space, assuming that that isn't something that is morally reprehensible? And some days, I wonder if exposing this industry to folks who are new to it isn't a problem.Mark: No, good question. So, I think in general—so I am pro bootcamp. I am pro self-taught. I was not always. And that's because of personal insecurity. Let's dive into that a little bit.So, I've been writing code since I was probably around 14 because I was lucky enough to go to a high school to had a computer science program on the south side of Chicago, one school. And then when I say I was lucky, I was really lucky because the school that I went to wasn't a high resource school; I didn't go to a private school. I went to a public school that just happened that one of the professors from IIT, also worked on staff a few days a week at my school, and we could take programming classes with this guy. Total luck. And so I get into computer science that way, take AP Computer Science in high school—which is, like, the pre-college level—then I go into undergrad, then I go into grad school for computer science.So, like, as traditional of a path that you can get. So, in my mind, it was all about my sweat equity that I had put in that disqualified everybody else. So, Corey, if you come from a bootcamp, you haven't spent the time that I spent learning to code; you haven't sweat, you haven't had to bleed, you haven't tried to write a two's complement algorithm on top of your other five classes for that semester. You haven't done it, definitely you don't deserve to be here. So, that was so much of my attitude, until—until—I got the opportunity to have my mind completely blown when I got asked to teach.Because when I got to asked to teach, I thought, “Yeah, I'm going to have my way of going in there and I'm going to show them how to do it right. This is my chance to correct these coding bootcampers and show them how it goes.” And then I find these people who were born for this life. So, some of us are natural talents, some of us are people who can just acquire the talent later. And both are totally valid.But I met this one student. She was a math teacher for years in Chicago Public Schools. She's like, “I want a career change.” Comes to the program that I taught at Northwestern, does so freaking well that she ends up getting a job at Airbnb. Now, if you have to make her go back four years at university, is that window still open for her? Maybe not.Then I meet this other woman, she was a paralegal for ten years. Ten years as a paralegal was the best engineer in the program when I taught, she was the best developer we had. Before the bootcamp was over, she had already gotten the job offer. She was meant for this. You see what I'm saying?So, that's why I'm so excited because it's like, I have all these stories of people who are meant for this. I taught, and I met people that changed the way I even saw the rest of the world. I had some non-binary trans students; I didn't even know what pronouns were. I had no idea that people didn't go by he/him, she/her. And then I had to learn about they and them and still teach you code without misgendering you at the same time, right because you're in a classroom and you're rapid-fire, all right, you—you know, how about this person? How about that person? And so you have to like, it's hard to take—Corey: Yeah, I can understand async, await, and JavaScript, but somehow understanding that not everyone has the pronouns that you are accustomed to using for people who look certain ways is a bridge too far for you to wrap your head around. Right. We can always improve, we can always change. It's just—at least when I screw up async, await, I don't make people feel less than. I just make—Mark: Totally.Corey: —users feel that, “Wow, this guy has no idea how to code.” You're right, I don't.Mark: Yeah, so as I'm on my soapbox, I'll just say this. I think coding bootcamps and self-taught programs where you can go online, I think this is where the door is the widest open for people to enter the industry because there is no requirement of a degree behind this. I just think that has just really opened the door for a lot of people to do things that is life-changing. So, when you meet somebody who's only making—because we're all engineers and we do all this stuff, we make a lot of money. And we're all comfortable. When you meet somebody where they go from 40,000 to 80,000, that is not the same story for—as it is for us.Corey: Exactly. And there's an entire school of thought out there that, “Oh, you should do this for the love because it is who you are, it is who you were meant to be.” And for some people, that's right, and I celebrate and cherish those folks. And there are other folks for whom, “I got into tech because of the money.” And you know what?I celebrate and cherish those folks because that is not inherently wrong. It says nothing negative about you whatsoever to want to improve your quality of life and wanting to support your family in varying ways. I have zero shade to throw at either one of those people. And when it comes to which of those two people do I want to hire, I have no preference in either direction because both are valid and both have directions that they can think in that the other one may not necessarily see for a variety of reasons. It's fine.Mark: I wanted to be an engineering manager. You know why? Not because I loved leadership; because I wanted more money.Corey: Yes.Mark: So, I've been in the industry for quite a long time. I'm a little bit on the older side of the story, right? I'm a little bit older. You know, for me, before we got ‘staff' and ‘principal' and all this kind of stuff, it was senior software engineer and then you topped out in terms of your earning potential. But if you wanted more, you became a manager, director, et cetera.So, that's why I wanted to be a manager for a while; I wanted more money, so why is my choice to be a manager more valuable than those people who want to make more money by coming into engineering or software development? I don't think it is.Corey: So, we've talked about positivity, we've talked about dealing with unpleasant people, we've talked about technology, and then, of course, we've talked about getting up on soapboxes. Let's tie all of that together for one last topic. What is your position on open-source in cloud?Mark: I think open-source software allows us to do a lot of incredible things. And I know that's a very light, fluffy, politically correct answer, but it is true, right? So, we get to take advantage of the brains of so many different people, all the ideas and contributions of so many different people so that we can do incredible things. And I think cloud really makes the world more accessible in general because—so when I used to do websites, I had to have a physical server that I would have to, like, try to talk to my ISP to be able to host things. And so, there was a lot of barriers to entry to do things that way.Now, with cloud and open-source, I could literally pick up a tool and deploy some software to the cloud. And the tool could you open-source so I can actually see what's happening and I could pick up other tools to help build out my vision for whatever I'm creating. So, I think open-source just gives a lot of opportunity.Corey: Oh, my stars, yes. It's even far more so than when I entered the field, and even back then there were challenges. One of the most democratizing aspects of cloud is that you can work with the same technologies that giant companies are using. When I entered the workforce, it's, “Wow, you're really good with Apache, but it seems like you don't really know a whole lot about the world of enterprise storage. What's going on with that?”And the honest answer was, “Well, it turns out that on my laptop, I can compile Apache super easily, but I'm finding it hard, given that I'm new to the workforce, to afford a $300,000 SAN in my garage, so maybe we can wind up figuring out that there are other ways to do it.” That doesn't happen today. Now, you can spin something up in the cloud, use it for a little bit. You're done, turn it off, and then never again have to worry about it except over in AWS land where you get charged 22 cents a month in perpetuity for some godforsaken reason you can't be bothered to track down and certainly no one can understand because, you know, cloud billing.Mark: [laugh].Corey: But if that's the tax versus the SAN tax, I'll take it.Mark: So, what I think is really interesting what cloud does, I like the word democratization because I think about going back to—just as a lateral reference to the bootcamp thing—I couldn't get my parents to see my software when I was in college when I made stuff because it was on my laptop. But when I was teaching these bootcamp students, they all deployed to Heroku. So, in their first couple of months, the cloud was allowing them to do something super cool that was not possible in the early days when I was coming up, learning how to code. And so they could deploy to Heroku, they could use GitHub Pages, you know like, open-source still coming into play. They can use all these tools and it's available to them, and I still think to me that is mind-blowing that I would have to bring my physical laptop or desktop home and say, “Mom, look at this terminal window that's doing this algorithm that I just did,” versus what these new people can do with the cloud. It's like, “Oh, yeah, I want to build a website. I want to publish it today. Publish right now.” Like, during our conversation, we both could have probably spent up a Hello World in the cloud with very little.Corey: Well, you could have. I could have done it in some horrifying way by using my favorite database: DNS. But that's a separate problem.Mark: [laugh]. Yeah, but I go to Firebase deploy and create a quick app real quick; Firebase deploy. Boom, I'm in the cloud. And I just think that the power behind that is just outstanding.Corey: If I had to pick a single cloud provider for someone new to the field to work with, it would be Google Cloud, and it's not particularly close. Just because the developer experience for someone who has not spent ten years marinating in cloud is worlds apart from what you're going to see in almost every other provider. I take it back, it is close. Neck-and-neck in different ways is also DigitalOcean, just because it explains things; their documentation is amazing and it lets people get started. My challenge with DigitalOcean is that it's not thought of, commonly, as a tier-one cloud provider in a lot of different directions, so the utility of learning how that platform works for someone who's planning to be in the industry for a while might potentially not get them as far.But again, there's no wrong answer. Whatever interests you, whenever you have to work on, do it. The obvious question of, “What technology should I learn,” it's, “Well, the ones that the companies you know are working with,” [laugh] so you can, ideally, turn it into something that throws off money, rather than doing it in your spare time for the love of it and not reaping any rewards from it.Mark: Yeah. If people ask me what should they use it to build something? And I think about what they want to do. And I also will say, “What will get you to ship the fastest? How can you ship?”Because that's what's really important for most people because people don't finish things. You know, as an engineer, how many side projects you probably have in the closet that never saw the light of day because you never shipped. I always say to people, “Well, what's going to get you to ship?” If it's View, use View and pair that with DigitalOcean, if that's going to get you to ship, right? Or use Angular plus Google Cloud Platform if that's going to get you to ship.Use what's going to get you to ship because—if it's just your project you're trying to run on. Now, if it's a company asking me, that's a consulting question which is a different answer. We do a much more in-detail analysis.Corey: I want to thank you so much for taking the time to speak with me about, honestly, a very wide-ranging group of topics. If people want to learn more about who you are, how you think, what you're up to, where can they find you?Mark: You can always find me spreading the love, being positive, hanging out. Look, if you want to feel better about yourself, come find me on Twitter at @marktechson—M-A-R-K-T-E-C-H-S-O-N. I'm out there waiting for you, so just come on and have a good time.Corey: And we will, of course, throw links to that in the [show notes 00:36:45]. Thank you so much for your time today.Mark: Oh, it's been a pleasure. Thanks for having me.Corey: Mark Thompson, developer relations engineer at Google. I'm Cloud Economist Corey Quinn and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice along with an angry, deranged comment that you spent several weeks rehearsing in the elevator.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.

Modernize or Die ® Podcast - CFML News Edition
Modernize or Die® - CFML News for October 27th, 2021 - Episode 123

Modernize or Die ® Podcast - CFML News Edition

Play Episode Listen Later Oct 27, 2021 65:44


2021-10-27 Weekly News - Episode 123Watch the video version on YouTube at https://www.youtube.com/watch?v=dLQhiLcHpH0 Hosts: Brad Wood - Senior Developer for Ortus SolutionsGavin Pickin - Senior Developer for Ortus SolutionsThanks to our Sponsor - Ortus SolutionsThe makers of ColdBox, CommandBox, ForgeBox, TestBox and almost every other Box out there. A few ways  to say thanks back to Ortus Solutions: Like and subscribe to our videos on YouTube.  Sign up for a free or paid account on CFCasts, which is releasing new content every week Buy Ortus's new Book - 102 ColdBox HMVC Quick Tips and Tricks on GumRoad (http://gum.co/coldbox-tips) Patreon SupportWe have 37 patreons providing 93% of the funding for our Modernize or Die Podcasts via our Patreon site: https://www.patreon.com/ortussolutions. Now offering Annual Memberships, pay for the year and save 10% - great for businesses.News and EventsPreside Version 10.16.0 is outSee our release and upgrade notes/video:Video: https://t.co/OZo8qRURWe Release Notes: https://t.co/bSt8vA9OT3 Documentation: https://t.co/k3P3rHff6k Online CF Meetup - Using LaunchDarkly for feature flag management in CF applications, w/ Brad WoodThursday, October 28, 2021 at 9:00 AM to 10:00 AM PDTFeature flags are a system of enabling certain functionality in your app based on test groups, cross-cutting segments of users, and your internal release processes. Feature flags can be updated on the fly at any time by any user and don't require deploying new code to your servers. LaunchDarkly is a system that helps you manage your feature flags and how they respond to the users of your site. It offers detailed tracking of each user, each flag, and a robust set of rules for determining which users see which features. In this session, we'll see an overview of how to use the new LaunchDarkly SDK which can be used in ColdFusion applications. Demos will include both ColdBox apps and non-ColdBox legacy apps.https://www.meetup.com/coldfusionmeetup/events/281577538/ Adobe 1 Day Workshop - Adobe ColdFusion Workshop with Damien BruyndonckxWed, November 10, 202109:00 - 17:00 CEST EUROPEANJoin the Adobe ColdFusion Workshop to learn how you and your agency can leverage ColdFusion to create amazing web content. This one-day training will cover all facets of Adobe ColdFusion that developers need to build applications that can run across multiple cloud providers or on-premise.https://coldfusion-workshop.meetus.adobeevents.com/ ICYMI - Into the Box 2021 - Videos are now availableVideos are now available on CFCasts!https://cfcasts.com/series/into-the-box-2021Free for subscribers; Free for ITB 2021 attendees; available as a one-time purchase for $199.If you bought a ticket to Into the Box 2021 and have not received a coupon for access to the videos on CFCasts, please contact us from the CFCasts support page. https://cfcasts.com/supportICYMI - Ortus Webinar for October - Gavin Pickin - Building Quick APIs - the extended versionIn this session we will use ColdBox's built in REST BaseHandler, and with CBSecurity and Quick ORM we will set up a secure API using fluent query language - and you'll see how quick Quick development can be!https://www.ortussolutions.com/events/webinarsRecording will be posted to CFCasts soonHacktoberfest 2021Support open source throughout October!Hacktoberfest encourages participation in the open source community, which grows bigger every year. Complete the 2021 challenge and earn a limited edition T-shirt.GIVING TO OPEN SOURCEOpen-source projects keep the internet humming—but they can't do it without resources. Donate and support their awesome work.TREES NOT TEESRather than receive t-shirts as swag, you can choose to have a tree planted in your name and help make Hacktoberfest 2021 more carbon neutral.To win a reward, you must sign up on the Hacktoberfest site and make four pull requests on any repositories classified with the 'hacktoberfest 'topic on GitHub or GitLab by October 31. If an Ortus Solutions repo that you want to contribute to is not marked with the `hacktoberfest` topic, please let us know so we can fix it.https://hacktoberfest.digitalocean.com/ CFCasts Content Updateshttps://www.cfcasts.com Just ReleasedUp and Running with Quick Testing with Quick Step 11 Exercise Coming this week Up and Running with Quick Building Quick APIs Send your suggestions at https://cfcasts.com/supportConferences and TrainingMicrosoft IgniteNovember 2–4, 2021 Opportunity awaits, with dedicated content spotlighting Microsoft Business Applications and Microsoft Security.https://myignite.microsoft.com/homeDeploy by Digital OceanTHE VIRTUAL CONFERENCE FOR GLOBAL DEVELOPMENT TEAMSNovember 16-17, 2021 https://deploy.digitalocean.com/homeAWS re:InventNOV. 29 – DEC. 3, 2021 | LAS VEGAS, NVCELEBRATING 10 YEARS OF RE:INVENTVirtual: FreeIn Person: $1799https://reinvent.awsevents.com/ Postgres BuildOnline - FreeNov 30-Dev 1 2021https://www.postgresbuild.com/ ITB Latam 2021December 2-3, 2021Into the Box LATAM is back and better than ever! Our virtual conference will include speakers from El Salvador and all over the world, who'll present on the latest web and mobile technologies in Latin America.Registration is completely free so don't miss out!https://latam.intothebox.org/ Adobe ColdFusion Summit 2021December 7th and 8th - VirtualSpeakers are finalized and some Speakers and some session descriptions are now on the siteRegister for Free - https://cfsummit.vconfex.com/site/adobe-cold-fusion-summit-2021/1290Blog - https://coldfusion.adobe.com/2021/09/adobe-coldfusion-summit-2021-registrations-open/ Tweet from Mark Takata OK! I can finally let you all know that for the @Adobe @coldfusion #CFSummit2021 keynote we will be featuring @ashleymcnamara! Her talk will focus on the history & future of DevRel how we got here & where we're going.cfsummit.vconfex.com to register!#CFML #DevRel #conferencehttps://twitter.com/MarkTakata/status/1449063259072438277 https://twitter.com/MarkTakata jConf.devNow a free virtual eventDecember 9th starting at 8:30 am CDT/2:30 pm UTC.https://2021.jconf.dev/?mc_cid=b62adc151d&mc_eid=8293d6fdb0 More conferencesNeed more conferences, this site has a huge list of conferences for almost any language/community.https://confs.tech/Blogs, Tweets and Videos of the WeekBlog - Ben Nadel - Reading Environment (ENV) Variables From The Server Scope In Lucee CFML 5.3.7.47This is a pro-tip that I originally picked up from Julian Halliwell a few years ago. However, I sometimes talk to people who don't realize that this is possible. So, I wanted to try and amplify Julian's post. In Lucee CFML, you can read environment (ENV) variables directly out of the server scope. They are just automatically there - no dipping into the Java layer or dealing with the java.lang.System class. Lucee CFML brings these values to the surface for easy consumption.https://www.bennadel.com/blog/4140-reading-environment-env-variables-from-the-server-scope-in-lucee-cfml-5-3-7-47.htm Blog - Ben Nadel - Making SQL Queries More Flexible With LIKE In MySQL 5.7.32 And Lucee CFML 5.3.7.47While you might stand-up something like Elasticsearch, Lucene, or Solr in order to provide robust and flexible text-based searches in your ColdFusion application, your relational database is more than capable of performing (surprisingly fast) pattern matching on TEXT and VARCHAR fields using the LIKE operator. This is especially true if the SQL query in question is already being limited based on an indexed value. At InVision, I often use the LIKE operator to allow for light-weight text-based searches. And, as of late, I've been massaging the inputs in order to make the matches even more flexible, allowing for some slightly fuzzy matching in Lucee CFML 5.3.7.47.https://www.bennadel.com/blog/4137-making-sql-queries-more-flexible-with-like-in-mysql-5-7-32-and-lucee-cfml-5-3-7-47.htm Blog - Ben Nadel - Creating A Group-Based Incrementing Value In MySQL 5.7.32 And Lucee CFML 5.3.7.47In the past few weeks, I've been learning a lot about how I can leverage SERIALIZABLE transactions in MySQL, the scope of said transactions, and some hidden gotchas around locking empty rows. As a means to lock (no pun intended) some of that information in my head-meat, I thought it would be a fun code kata to create a Jira-inspired ticketing system in Lucee CFML 5.3.7.47 that uses an application-defined, group-based incrementing value in MySQL 5.7.32.https://www.bennadel.com/blog/4135-creating-a-group-based-incrementing-value-in-mysql-5-7-32-and-lucee-cfml-5-3-7-47.htm Blog - Ben Nadel - Creating A Group-Based Incrementing Value Using LAST_INSERT_ID() In MySQL 5.7.32 And Lucee CFML 5.3.7.47Yesterday, I took inspiration from Jira's ticketing system and explored the idea of creating a group-based incrementing value in MySQL. In my approach, I used a SERIALIZABLE transaction to safely "update and read" a shared sequence value across parallel threads. In response to that post, my InVision co-worker - Michael Dropps - suggested that I look at using LAST_INSERT_ID(expr) to achieve the same outcome with less transaction isolation. I had never seen the LAST_INSERT_ID() function used with an expression argument before. So, I wanted to revisit yesterday's post using this technique.https://www.bennadel.com/blog/4136-creating-a-group-based-incrementing-value-using-last-insert-id-in-mysql-5-7-32-and-lucee-cfml-5-3-7-47.htm Blog / Documentation - Zac Spitszer - Building and testing Lucee extensions documentationI have written up a detailed guide on how to Build and Test Lucee Extensions, using Lucee Script Runner and Apache Ant.It's a little bit complicated to setup, but I have developed a toolchain, which once set up, makes the entire process really dead simple.https://dev.lucee.org/t/building-and-testing-lucee-extensions-documentation/9053 Tweet - Mark Takata - Adobe - The CF Summit 2021 Keynote announcementOK! I can finally let you all know that for the @Adobe @coldfusion #CFSummit2021 keynote we will be featuring @ashleymcnamara! Her talk will focus on the history & future of DevRel how we got here & where we're going.cfsummit.vconfex.com to register!#CFML #DevRel #conferencehttps://twitter.com/MarkTakata/status/1449063259072438277 https://twitter.com/MarkTakata Tweet - Ben Nadel - Monolith DeploysIt's 10:50 AM.I work in a monolithic #Lucee #CFML codebase.And, I just started my 3rd deployment of the day.It's amazing how much work you can get done when you stop worrying about what other people think of your technology choices.

North Meets South Web Podcast
Programming in pairs, statically generated PHP, and how even does Statamic work?

North Meets South Web Podcast

Play Episode Listen Later Oct 24, 2021 42:09


In this episode, Jake and Michael discuss a number of tools used for pair programming, approaches to statically generating sites in PHP, and show that despite Statamic existing in the Laravel community for many years, they have absolutely no idea how it actually works.This episode is sponsored by Makeable.dk and Workvivo and was streamed live.Show links Sublive StreamYard Around Tuple CSS Tricks Chris Coyier Shop Talk Show Statamic Pagely WP Engine Statamic 3 Guide

Screaming in the Cloud
Navigating the Morass of the Internet with Chloe Condon

Screaming in the Cloud

Play Episode Listen Later Oct 21, 2021 42:32


About ChloeChloe is a Bay Area based Cloud Advocate for Microsoft. Previously, she worked at Sentry.io where she created the award winning Sentry Scouts program (a camp themed meet-up ft. patches, s'mores, giant squirrel costumes, and hot chocolate), and was featured in the Grace Hopper Conference 2018 gallery featuring 15 influential women in STEM by AnitaB.org. Her projects and work with Azure have ranged from fake boyfriend alerts to Mario Kart 'astrology', and have been featured in VICE, The New York Times, as well as SmashMouth's Twitter account. Chloe holds a BA in Drama from San Francisco State University and is a graduate of Hackbright Academy. She prides herself on being a non-traditional background engineer, and is likely one of the only engineers who has played an ogre, crayon, and the back-end of a cow on a professional stage. She hopes to bring more artists into tech, and more engineers into the arts.Links: Twitter: https://twitter.com/ChloeCondon Instagram: https://www.instagram.com/gitforked/ YouTube: https://www.youtube.com/c/ChloeCondonVideos TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: This episode is sponsored in part by our friends at Vultr. Spelled V-U-L-T-R because they're all about helping save money, including on things like, you know, vowels. So, what they do is they are a cloud provider that provides surprisingly high performance cloud compute at a price that—while sure they claim its better than AWS pricing—and when they say that they mean it is less money. Sure, I don't dispute that but what I find interesting is that it's predictable. They tell you in advance on a monthly basis what it's going to going to cost. They have a bunch of advanced networking features. They have nineteen global locations and scale things elastically. Not to be confused with openly, because apparently elastic and open can mean the same thing sometimes. They have had over a million users. Deployments take less that sixty seconds across twelve pre-selected operating systems. Or, if you're one of those nutters like me, you can bring your own ISO and install basically any operating system you want. Starting with pricing as low as $2.50 a month for Vultr cloud compute they have plans for developers and businesses of all sizes, except maybe Amazon, who stubbornly insists on having something to scale all on their own. Try Vultr today for free by visiting: vultr.com/screaming, and you'll receive a $100 in credit. Thats v-u-l-t-r.com slash screaming.Corey: This episode is sponsored in part by Honeycomb. When production is running slow, it's hard to know where problems originate: is it your application code, users, or the underlying systems? I've got five bucks on DNS, personally. Why scroll through endless dashboards, while dealing with alert floods, going from tool to tool to tool that you employ, guessing at which puzzle pieces matter? Context switching and tool sprawl are slowly killing both your team and your business. You should care more about one of those than the other, which one is up to you. Drop the separate pillars and enter a world of getting one unified understanding of the one thing driving your business: production. With Honeycomb, you guess less and know more. Try it for free at Honeycomb.io/screaminginthecloud. Observability, it's more than just hipster monitoring.Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. Somehow in the years this show has been running, I've only had Chloe Condon on once. In that time, she's over for dinner at my house way more frequently than that, but somehow the stars never align to get us together in front of microphones and have a conversation. First, welcome back to the show, Chloe. You're a senior cloud advocate at Microsoft on the Next Generation Experiences Team. It is great to have you here.Chloe: I'm back, baby. I'm so excited. This is one of my favorite shows to listen to, and it feels great to be a repeat guest, a friend of the pod. [laugh].Corey: Oh, yes indeed. So, something-something cloud, something-something Microsoft, something-something Azure, I don't particularly care, in light of what it is you have going on that you have just clued me in on, and we're going to talk about that to start. You're launching something new called Master Creep Theatre and I have a whole bunch of questions. First and foremost, is it theater or theatre? How is that spelled? Which—the E and the R, what direction does that go in?Chloe: Ohh, I feel like it's going to be the R-E because that makes it very fancy and almost British, you know?Corey: Oh, yes. And the Harlequin mask direction it goes in, that entire aesthetic, I love it. Please tell me what it is. I want to know the story of how it came to be, the sheer joy I get from playing games with language alone guarantee I'm going to listen to whatever this is, but please tell me more.Chloe: Oh, my goodness. Okay, so this is one of those creative projects that's been on my back burner forever where I'm like, someday when I have time, I'm going to put all my time [laugh] and energy into this. So, this originally stemmed from—if you don't follow me on Twitter, oftentimes when I'm not tweeting about '90s nostalgia, or Clippy puns, or Microsoft silly throwback things to Windows 95, I get a lot of weird DMs. On every app, not just Twitter. On Instagram, Twitter, LinkedIn, oh my gosh, what else is there?Corey: And I don't want to be clear here just to make this absolutely crystal clear, “Hey, Chloe, do you want to come back on Screaming in the Cloud again?” Is not one of those weird DMs to which you're referring?Chloe: No, that is a good DM. So, people always ask me, “Why don't you just close your DMs?” Because a lot of high profile people on the internet just won't even have their DMs open.Corey: Oh, I understand that, but I'm the same boat. I would have a lot less nonsense, but at the same time, I want—at least in my case—I want people to be able to reach out to me because the only reason I am what I am is that a bunch of people who had no reason to do it did favors for me—Chloe: Yes.Corey: —and I can't ever repay it, I can only ever pay it forward and that is the cost of doing favors. If I can help someone, I will, and that's hard to do with, “My DMs are closed so hunt down my email address and send me an email,” and I'm bad at email.Chloe: Right. I'm terrible at email as well, and I'm also terrible at DMs [laugh]. So, I think a lot of folks don't understand the volume at which I get messages, which if you're a good friend of mine, if you're someone like Corey or a dear friend like Emily, I will tell you, “Hey, if you actually need to get ahold of me, text me.” And text me a couple times because I probably see it and then I have ADHD, so I won't immediately respond. I think I respond in my head but I don't.But I get anywhere from, I would say, ohh, like, 30 on a low day to 100 on a day where I have a viral tweet about getting into tech with a non-traditional background or something like that. And these DMs that I get are really lovely messages like, “Thank you for the work you do,” or, “I decided to do a cute manicure because the [laugh] manicure you posted,” too, “How do I get into tech? How do I get a job at Microsoft?” All kinds of things. It runs the gamut between, “Where's your shirt from?” Where—[laugh]—“What's your mother's maiden name?”But a lot of the messages that I get—and if you're a woman on the internet with any sort of presence, you know how there's that, like—what's it called in Twitter—the Other Messages feature that's like, “Here's the people you know. Here's the people”—the message requests. For the longest time were just, “Hey,” “Hi,” “Hey dear,” “Hi pretty,” “Hi ma'am,” “Hello,” “Love you,” just really weird stuff. And of course, everyone gets these; these are bots or scammers or whatever they may be—or just creeps, like weird—and always the bio—not always but I [laugh] would say, like, these accounts range from either obviously a bot where it's a million different numbers, an account that says, “Father, husband, lover of Jesus Christ and God.” Which is so [laugh] ironic… I'm like, “Why are you in my DMs?”Corey: A man of God, which is why I'm in your DMs being creepy.Chloe: Exactly. Or—Corey: Just like Christ might have.Chloe: And you would be shocked, Corey, at how many. The thing that I love to say is Twitter is not a dating site. Neither is LinkedIn. Neither is Instagram. I post about my boyfriend all the time, who you've met, and we adore Ty Smith, but I've never received any unsolicited images, knock on wood, but I'm always getting these very bait-y messages like, “Hey, beautiful. I want to take you out.” And you would be shocked at how many of these people are doing it from their professional business account. [laugh]. Like, works at AWS, works at Google; it's like, oh my God. [laugh].Corey: You get this under your name, right? It ties back to it. Meanwhile—again, this is one of those invisible areas of privilege that folks who look like me don't have to deal with. My DM graveyard is usually things like random bot accounts, always starting with, “Hi,” or, “Hey.” If you want to guarantee I never respond to you, that is what you say. I just delete those out of hand because I don't notice or care. It is either a bot, or a scam, or someone who can't articulate what they're actually trying to get from me—Chloe: Exactly.Corey: —and I don't have the time for it. Make your request upfront. Don't ask to ask; just ask.Chloe: I think it's important to note, also, that I get a lot of… different kinds of these messages and they try to respond to everyone. I cannot. If I responded to everybody's messages that I got, I just wouldn't have any time to do my job. But the thing that I always say to people—you know, and managers have told me in the past, my boyfriend has encouraged me to do this, is when people say things like, “Close your DMs,” or, “Just ignore them,” I want to have the same experience that everybody else has on the internet. Now, it's going to be a little different, of course, because I look and act and sound like I do, and of course, podcasts are historically a visual medium, so I'm a five-foot-two, white, bright orange-haired girl; I'm a very quirky individual.Corey: Yes, if you look up ‘quirky,' you're right there under the dictionary definition. And every time—like, when we were first hanging out and you mentioned, “Oh yeah, I used to be in theater.” And it's like, “You know, you didn't even have to tell me that, on some level.” Which is not intended to be an insult. It's just theater folks are a bit of a type, and you are more or less the archetype of what a theatre person is, at least to my frame of reference.Chloe: And not only that, but I did musicals, so you can't see the jazz hands now, but–yeah, my degree is in drama. I come from that space and I just, you know, whenever people say, “Just ignore it,” or, “Close your DMs,” I'm like, I want people to be able to reach out to me; I want to be able to message one-on-one with Corey and whoever, when—as needed, and—Corey: Why should I close my DMs?Chloe: Yeah.Corey: They're the ones who suck. Yeah.Chloe: [laugh]. But over the years, to give people a little bit of context, I've been working in tech a long time—I've been working professionally in the DevRel space for about five or six years now—but I've worked in tech a long time, I worked as a recruiter, an office admin, executive assistant, like, I did all of the other areas of tech, but it wasn't until I got a presence on Twitter—which I've only been on Twitter for I think five years; I haven't been on there that long, actively. And to give some context on that, Twitter is not a social media platform used in the theater space. We just use Instagram and Facebook, really, back in the day, I'm not on Facebook at all these days. So, when I discovered Twitter was cool—and I should also mention my boyfriend, Ty, was working at Twitter at the time and I was like, “Twitter's stupid. Who would go on this—[laugh] who uses this app?”Fast-forward to now, I'm like—Ty's like, “Can you please get off Twitter?” But yeah, I think I've just been saving these screenshots over the last five or so years from everything from my LinkedIn, from all the crazy stuff that I dealt with when people thought I was a Bitcoin influencer to people being creepy. One of the highlights that I recently found when I was going back and trying to find these for this series that I'm doing is there was a guy from Australia, DMed me something like, “Hey, beautiful,” or, “Hey, sexy,” something like that. And I called him out. And I started doing this thing where I would post it on Twitter.I would usually hide their image with a clown emoji or something to make it anonymous, or not to call them out, but in this one I didn't, and this guy was defending himself in the comments, and to me in my DM's saying, “Oh, actually, this was a social experiment and I have all the screenshots of this,” right? So, imagine if you will—so I have conversations ranging from things like that where it's like, “Actually I messaged a bunch of people about that because I'm doing a social experiment on how people respond to, ‘Hey beautiful. I'd love to take you out some time in Silicon Valley.'” just the weirdest stuff right? So, me being the professional performer that I am, was like, these are hilarious.And I kept thinking to myself, anytime I would get these messages, I was like, “Does this work?” If you just go up to someone and say, “Hey”—do people meet this way? And of course, you get people on Twitter who when you tweet something like that, they're like, “Actually, I met my boyfriend in Twitter DMs,” or like, “I met my boyfriend because he slid into my DMs on Instagram,” or whatever. But that's not me. I have a boyfriend. I'm not interested. This is not the time or the place.So, it's been one of those things on the back burner for three or four years that I've just always been saving these images to a folder, thinking, “Okay, when I have the time when I have the space, the creative energy and the bandwidth to do this,” and thankfully for everyone I do now, I'm going to do dramatic readings of these DMs with other people in tech, and show—not even just to make fun of these people, but just to show, like, how would this work? What do you expect the [laugh] outcome to be? So Corey, for example, if you were to come on, like, here's a great example. A year ago—this is 2018; we're in 2021 right now—this guy messaged me in December of 2018, and was like, “Hey,” and then was like, “I would love to be your friend.” And I was like, “Nope,” and I responded, “Nope, nope, nope, nope.” There's a thread of this on Twitter. And then randomly, three weeks ago, just sent me this video to the tune of Enrique Iglesias' “Rhythm Divine” of just images of himself. [laugh]. So like, this comedy [crosstalk 00:10:45]—Corey: Was at least wearing pants?Chloe: He is wearing pants. It's very confusing. It's a picture—a lot of group photos, so I didn't know who he was. But in my mind because, you know, I'm an engineer, I'm trying to think through the end-user experience. I'm like, “What was your plan here?”With all these people I'm like, “So, your plan is just to slide into my DMs and woo me with ‘Hey'?” [laugh]. So, I think it'll be really fun to not only just show and call out this behavior but also take submissions from other people in the industry, even beyond tech, really, because I know anytime I tweet an example of this, I get 20 different women going, “Oh, my gosh, you get these weird messages, too?” And I really want to show, like, A, to men how often this happens because like you said, I think a lot of men say, “Just ignore it.” Or, “I don't get anything like that. You must be asking for it.”And I'm like, “No. This comes to me. These people find us and me and whoever else out there gets these messages,” and I'm just really ready to have a laugh at their expense because I've been laughing for years. [laugh].Corey: Back when I was a teenager, I was working in some fast food style job, and one of my co-workers saw customer, walked over to her, and said, “You're beautiful.” And she smiled and blushed. He leaned in and kissed her.Chloe: Ugh.Corey: And I'm sitting there going what on earth? And my other co-worker leaned over and is like, “You do know that's his girlfriend, right?” And I have to feel like, on some level, that is what happened to an awful lot of these broken men out on the internet, only they didn't have a co-worker to lean over and say, “Yeah, they actually know each other.” Which is why we see all this [unintelligible 00:12:16] behavior of yelling at people on the street as they walk past, or from a passing car. Because they saw someone do a stunt like that once and thought, “If it worked for them, it could work for me. It only has to work once.”And they're trying to turn this into a one day telling the grandkids how they met their grandmother. And, “Yeah, I yelled at her from a construction site, and it was love at first ‘Hey, baby.'” That is what I feel is what's going on. I have never understood it. I look back at my dating history in my early 20s, I look back now I'm like, “Ohh, I was not a great person,” but compared to these stories, I was a goddamn prince.Chloe: Yeah.Corey: It's awful.Chloe: It's really wild. And actually, I have a very vivid memory, this was right bef—uh, not right before the pandemic, but probably in 2019. I was speaking on a lot of conferences and events, and I was at this event in San Jose, and there were not a lot of women there. And somehow this other lovely woman—I can't remember her name right now—found me afterwards, and we were talking and she said, “Oh, my God. I had—this is such a weird event, right?”And I was like, “Yeah, it is kind of a weird vibe here.” And she said, “Ugh, so the weirdest thing happened to me. This guy”—it was her first tech conference ever, first of all, so you know—or I think it was her first tech conference in the Bay Area—and she was like, “Yeah, this guy came to my booth. I've been working this booth over here for this startup that I work at, and he told me he wanted to talk business. And then I ended up meeting him, stupidly, in my hotel lobby bar, and it's a date. Like, this guy is taking me out on a date all of a sudden,” and she was like, “And it took me about two minutes to just to be like, you know what? This is inappropriate. I thought this is going to be a business meeting. I want to go.”And then she shows me her hands, Corey, and she has a wedding ring. And she goes, “I'm not married. I have bought five or six different types of rings on Wish App”—or wish.com, which if you've never purchased from Wish before, it's very, kind of, low priced jewelry and toys and stuff of that nature. And she said, “I have a different wedding ring for every occasion. I've got my beach fake wedding ring. I've got my, we-got-married-with-a-bunch-of-mason-jars-in-the-woods fake wedding ring.”And she said she started wearing these because when she did, she got less creepy guys coming up to her at these events. And I think it's important to note, also, I'm not putting it out there at all that I'm interested in men. If anything, you know, I've been [laugh] with my boyfriend for six years never putting out these signals, and time and time again, when I would travel, I was very, very careful about sharing my location because oftentimes I would be on stage giving a keynote and getting messages while I delivered a technical keynote saying, “I'd love to take you out to dinner later. How long are you in town?” Just really weird, yucky, nasty stuff that—you know, and everyone's like, “You should be flattered.”And I'm like, “No. You don't have to deal with this. It's not like a bunch of women are wolf-whistling you during your keynote and asking what your boob size is.” But that's happening to me, and that's an extra layer that a lot of folks in this industry don't talk about but is happening and it adds up. And as my boyfriend loves to remind me, he's like, “I mean, you could stop tweeting at any time,” which I'm not going to do. But the more followers you get, the more inbound you get. So—Corey: Right. And the hell of it is, it's not a great answer because it's closing off paths of opportunity. Twitter has—Chloe: Absolutely.Corey: —introduced me to clients, introduced me to friends, introduced me to certainly an awful lot of podcast guests, and it informs and shapes a lot of the opinions that I hold on these things. And this is an example of what people mean when they talk about privilege. Where, yeah, “Look at Corey”—I've heard someone say once, and, “Nothing was handed to him.” And you're right, to be clear, I did not—like, no one handed me a microphone and said, “We're going to give you a podcast, now.” I had to build this myself.But let's be clear, I had no headwinds of working against me while I did it. There's the, you still have to do things, but you don't have an entire cacophony of shit heels telling you that you're not good enough in a variety of different ways, to subtly reinforcing your only value is the way that you look. There isn't this whole, whenever you get something wrong and it's a, “Oh, well, that's okay. We all get things wrong.” It's not the, “Girls suck at computers,” trope that we see so often.There's a litany of things that are either supportive that work in my favor, or are absent working against me that is privilege that is invisible until you start looking around and seeing it, and then it becomes impossible not to. I know I've talked about this before on the show, but no one listens to everything and I just want to subtly reinforce that if you're one of those folks who will say things like, “Oh, privilege isn't real,” or, “You can have bigotry against white people, too.” I want to be clear, we are not the same. You are not on my side on any of this, and to be very direct, I don't really care what you have to say.Chloe: Yeah. And I mean, this even comes into play in office culture and dynamics as well because I am always the squeaky wheel in the room on these kind of things, but a great example that I'll give is I know several women in this industry who have had issues when they used to travel for conferences of being stalked, people showing up at their hotel rooms, just really inappropriate stuff, and for that reason, a lot of folks—including myself—wouldn't pick the conference event—like, typically they'll be like, “This is the hotel everyone's staying at.” I would very intentionally stay at a different hotel because I didn't want people knowing where I was staying. But I started to notice once a friend of mine, who had an issue with this [unintelligible 00:17:26], I really like to be private about where I'm staying, and sometimes if you're working at a startup or larger company, they'll say, “Hey, everyone put in this Excel spreadsheet or this Google Doc where everyone's staying and how to contact them, and all this stuff.” And I think it's really important to be mindful of these things.I always say to my friends—I'm not going out too much these days because it's a pandemic—and I've done Twitter threads on this before where I never post my location; you will never see me. I got rid of Swarm a couple [laugh] years ago because people started showing up where I was. I posted photos before, you know, “Hey, at the lake right now.” And people have shown up. Dinners, people have recognized me when I've been out.So, I have an espresso machine right over here that my lovely boyfriend got me for my birthday, and someone commented, “Oh, we're just going to act like we don't see someone's reflection in the”—like, people Zoom in on images. I've read stories from cosplayers online who, they look into the reflection of a woman's glasses and can figure out where they are. So, I think there's this whole level. I'm constantly on alert, especially as a woman in tech. And I have friends here in the Bay Area, who have tweeted a photo at a barbecue, and then someone was like, “Hey, I live in the neighborhood, and I recognize the tree.”First of all, don't do that. Don't ever do that. Even if you think you're a nice, unassuming guy or girl or whatever, don't ever [laugh] do that. But I very intentionally—people get really confused, my friends specifically. They're like, “Wait a second, you're in Hawaii right now? I thought you were in Hawaii three weeks ago.” And I'm like, “I was. I don't want anyone even knowing what island or continent I'm on.”And that's something that I think about a lot. When I post photo—I never post any photos from my window. I don't want people knowing what my view is. People have figured out what neighborhood I live in based on, like, “I know where that graffiti is.” I'm very strategic about all this stuff, and I think there's a lot of stuff that I want to share that I don't share because of privacy issues and concerns about my safety. And also want to say and this is in my thread on online safety as well is, don't call out people's locations if you do recognize the image because then you're doxxing them to everyone like, “Oh”—Corey: I've had a few people do that in response to pictures I've posted before on a house, like, “Oh, I can look at this and see this other thing and then intuit where you are.” And first, I don't have that sense of heightened awareness on this because I still have this perception of myself as no one cares enough to bother, and on the other side, by calling that out in public. It's like, you do not present yourself well at all. In fact, you make yourself look an awful lot like the people that we're warned about. And I just don't get that.I have some of these concerns, especially as my audience has grown, and let's be very clear here, I antagonize trillion-dollar companies for a living. So, first if someone's going to have me killed, they can find where I am. That's pretty easy. It turns out that having me whacked is not even a rounding error on most of these companies' budgets, unfortunately. But also I don't have that level of, I guess, deranged superfan. Yet.But it happens in the fullness of time, as people's audiences continue to grow. It just seems an awful lot like it happens at much lower audience scale for folks who don't look like me. I want to be clear, this is not a request for anyone listening to this, to try and become that person for me, you will get hosed, at minimum. And yes, we press charges here.Chloe: AWSfan89, sliding into your DMs right after this. Yeah, it's also just like—I mean, I don't want to necessarily call out what company this was at, but personally, I've been in situations where I've thrown an event, like a meetup, and I'm like, “Hey, everyone. I'm going to be doing ‘Intro to blah, blah, blah' at this time, at this place.” And three or four guys would show up, none of them with computers. It was a freaking workshop on how to do or deploy something, or work with an API.And when I said, “Great, so why'd you guys come to this session today?” And maybe two have iPads, one just has a notepad, they're like, “Oh, I just wanted to meet you from Twitter.” And it's like, okay, that's a little disrespectful to me because I am taking time out to do this workshop on a very technical thing that I thought people were coming here to learn. And this isn't the Q&A. This is not your meet-and-greet opportunity to meet Chloe Condon, and I don't know why you would, like, I put so much of my life online [laugh] anyway.But yeah, it's very unsettling, and it's happened to me enough. Guys have shown up to my events and given me gifts. I mean, I'm always down for a free shirt or something, but it's one of those things that I'm constantly aware of and I hate that I have to be constantly aware of, but at the end of the day, my safety is the number one priority, and I don't want to get murdered. And I've tweeted this out before, our friend Emily, who's similarly a lady on the internet, who works with my boyfriend Ty over at Uber, we have this joke that's not a joke, where we say, “Hey if I'm murdered, this is who it was.” And we'll just send each other screenshots of creepy things that people either tag us in, or give us feedback on, or people asking what size shirt we are. Just, wiki feed stuff, just really some of the yucky of the yuck out there.And I do think that unless you have a partner, or a family member, or someone close enough to you to let you know about these things—because I don't talk about these things a lot other than my close friends, and maybe calling out a weirdo here and there in public, but I don't share the really yucky stuff. I don't share the people who are asking what neighborhood I live in. I'm not sharing the people who are tagging me, like, [unintelligible 00:22:33], really tagging me in some nasty TikToks, along with some other women out there. There are some really bad actors in this community and it is to the point where Emily and I will be like, “Hey, when you inevitably have to solve my murder, here's the [laugh] five prime suspects.” And that sucks. That's [unintelligible 00:22:48] joke; that isn't a joke, right? I suspect I will either die in an elevator accident or one of my stalkers will find me. [laugh].Corey: It's easy for folks to think, oh, well, this is a Chloe problem because she's loud, she's visible, she's quirky, she's different than most folks, and she brings it all on herself, and this is provably not true. Because if you talk to, effectively, any woman in the world in-depth about this, they all have stories that look awfully similar to this. And let me forestall some of the awful responses I know I'm going to get. And, “Well, none of the women I know have had experiences like this,” let me be very clear, they absolutely have, but for one reason or another, they either don't see the need, or don't see the value, or don't feel safe talking to you about it.Chloe: Yeah, absolutely. And I feel a lot of privilege, I'm very lucky that my boyfriend is a staff engineer at Uber, and I have lots of friends in high places at some of these companies like Reddit that work with safety and security and stuff, but oftentimes, a lot of the stories or insights or even just anecdotes that I will give people on their products are invaluable insights to a lot of these security and safety teams. Like, who amongst us, you know, [laugh] has used a feature and been like, “Wait a second. This is really, really bad, and I don't want to tweet about this because I don't want people to know that they can abuse this feature to stalk or harass or whatever that may be,” but I think a lot about the people who don't have the platform that I have because I have 50k-something followers on Twitter, I have a pretty big online following in general, and I have the platform that I do working at Microsoft, and I can tweet and scream and be loud as I can about this. But I think about the folks who don't have my audience, the people who are constantly getting harassed and bombarded, and I get these DMs all the time from women who say, “Thank you so much for doing a thread on this,” or, “Thank you for talking about this,” because people don't believe them.They're just like, “Oh, just ignore it,” or just, “Oh, it's just one weirdo in his basement, like, in his mom's basement.” And I'm like, “Yeah, but imagine that but times 40 in a week, and think about how that would make you rethink your place and your position in tech and even outside of tech.” Let's think of the people who don't know how this technology works. If you're on Instagram at all, you may notice that literally not only every post, but every Instagram story that has the word COVID in it, has the word vaccine, has anything, and they must be using some sort of cognitive scanning type thing or scanning the images themselves because this is a feature that basically says, hey, this post mentioned COVID in some way. I think if you even use the word mask, it alerts this.And while this is a great feature because we all want accurate information coming out about the pandemic, I'm like, “Wait a minute. So, you're telling me this whole time you could have been doing this for all the weird things that I get into my DMs, and people post?” And, like, it just shows you, yes, this is a global pandemic. Yes, this is something that affects everyone. Yes, it's important we get information out about this, but we can be using these features in much [laugh] more impactful ways that protects people's safety, that protects people's ability to feel safe on a platform.And I think the biggest one for me, and I make a lot of bots; I make a lot of Twitter bots and chatbots, and I've done entire series on this about ethical bot creation, but it's so easy—and I know this firsthand—to make a Twitter account. You can have more than one number, you can do with different emails. And with Instagram, they have this really lovely new feature that if you block someone, it instantly says, “You just blocked so and so. Would you like to block any other future accounts they make?” I mean, seems simple enough, right?Like, anything related—maybe they're doing it by email, or phone number, or maybe it's by IP, but like, that's not being done on a lot of these platforms, and it should be. I think someone mentioned in one of my threads on safety recently that Peloton doesn't have a block user feature. [laugh]. They're probably like, “Well, who's going to harass someone on Peloton?” It would happen to me. If I had a Peloton, [laugh] I assure you someone would find a way to harass me on there.So, I always tell people, if you're working at a company and you're not thinking about safety and harassment tools, you probably don't have anybody LGBTQ+ women, non-binary on your team, first of all, and you need to be thinking about these things, and you need to be making them a priority because if users can interact in some way, they will stalk, harass, they will find some way to misuse it. It seems like one of those weird edge cases where it's like, “Oh, we don't need to put a test in for that feature because no one's ever going to submit, like, just 25 emojis.” But it's the same thing with safety. You're like, who would harass someone on an app about bubblegum? One of my followers were. [laugh].Corey: This episode is sponsored by our friends at Oracle HeatWave is a new high-performance accelerator for the Oracle MySQL Database Service. Although I insist on calling it “my squirrel.” While MySQL has long been the worlds most popular open source database, shifting from transacting to analytics required way too much overhead and, ya know, work. With HeatWave you can run your OLTP and OLAP, don't ask me to ever say those acronyms again, workloads directly from your MySQL database and eliminate the time consuming data movement and integration work, while also performing 1100X faster than Amazon Aurora, and 2.5X faster than Amazon Redshift, at a third of the cost. My thanks again to Oracle Cloud for sponsoring this ridiculous nonsense.Corey: The biggest question that doesn't get asked that needs to be in almost every case is, “Okay. We're building a thing, and it's awesome. And I know it's hard to think like this, but pivot around. Theoretically, what could a jerk do with it?”Chloe: Yes.Corey: When you're designing it, it's all right, how do you account for people that are complete jerks?Chloe: Absolutely.Corey: Even the cloud providers, all of them, when the whole Parler thing hit, everyone's like, “Oh, Amazon is censoring people for freedom of speech.” No, they're actually not. What they're doing is enforcing their terms of service, the same terms of service that every provider that is not trash has. It is not a problem that one company decided they didn't want hate speech on their platform. It was all the companies decided that, except for some very fringe elements. And that's the sort of thing you have to figure out is, it's easy in theory to figure out, oh, anything goes; freedom of speech. Great, well, some forms of speech violate federal law.Chloe: Right.Corey: So, what do you do then? Where do you draw the line? And it's always nuanced and it's always tricky, and the worst people are the folks that love to rules-lawyer around these things. It gets worse than that where these are the same people that will then sit there and make bad faith arguments all the time. And lawyers have a saying that hard cases make bad law.When you have these very nuanced thing, and, “Well, we can't just do it off the cuff. We have to build a policy around this.” This is the problem with most corporate policies across the board. It's like, you don't need a policy that says you're not allowed to harass your colleagues with a stick. What you need to do is fire the jackwagon that made you think you might need a policy that said that.But at scale, that becomes a super-hard thing to do when every enforcement action appears to be bespoke. Because there are elements on the gray areas and the margins where reasonable people can disagree. And that is what sets the policy and that's where the precedent hits, and then you have these giant loopholes where people can basically be given free rein to be the worst humanity has to offer to some of the most vulnerable members of our society.Chloe: And I used to give this talk, I gave it at DockerCon one year and I gave it a couple other places, that was literally called “Diversity is not Equal to Stock Images of Hands.” And the reason I say this is if you Google image search ‘diversity' it's like all of those clip arts of, like, Rainbow hands, things that you would see at Kaiser Permanente where it's like, “We're all in this together,” like, the pandemic, it's all just hands on hands, hands as a Earth, hands as trees, hands as different colors. And people get really annoyed with people like me who are like, “Let's shut up about diversity. Let's just hire who's best for the role.” Here's the thing.My favorite example of this—RIP—is Fleets—remember Fleets? [laugh]—on Twitter, so if they had one gay man in the room for that marketing, engineering—anything—decision, one of them I know would have piped up and said, “Hey, did you know ‘fleets' is a commonly used term for douching enima in the gay community?” Now, I know that because I watch a lot of Ru Paul's Drag Race, and I have worked with the gay community quite a bit in my time in theater. But this is what I mean about making sure. My friend Becca who works in security at safety and things, as well as Andy Tuba over at Reddit, I have a lot of conversations with my friend Becca Rosenthal about this, and that, not to quote Hamilton, but if I must, “We need people in the room where it happens.”So, if you don't have these people in the room if you're a white man being like, “How will our products be abused?” Your guesses may be a little bit accurate but it was probably best to, at minimum, get some test case people in there from different genders, races, backgrounds, like, oh my goodness, get people in that room because what I tend to see is building safety tools, building even product features, or naming things, or designing things that could either be offensive, misused, whatever. So, when people have these arguments about like, “Diversity doesn't matter. We're hiring the best people.” I'm like, “Yeah, but your product's going to be better, and more inclusive, and represent the people who use it at the end of the day because not everybody is you.”And great examples of this include so many apps out there that exists that have one work location, one home location. How many people in the world have more than one job? That's such a privileged view for us, as people in tech, that we can afford to just have one job. Or divorced parents or whatever that may be, for home location, and thinking through these edge cases and thinking through ways that your product can support everyone, if anything, by making your staff or the people that you work with more diverse, you're going to be opening up your product to a much bigger marketable audience. So, I think people will look at me and be like, “Oh, Chloe's a social justice warrior, she's this feminist whatever,” but truly, I'm here saying, “You're missing out on money, dude.” It would behoove you to do this at the end of the day because your users aren't just a copy-paste of some dude in a Patagonia jacket with big headphones on. [laugh]. There are people beyond one demographic using your products and applications.Corey: A consistent drag against Clubhouse since its inception was that it's not an accessible app for a variety of reasons that were—Chloe: It's not an Android. [laugh].Corey: Well, even ignoring the platform stuff, which I get—technical reasons, et cetera, yadda, yadda, great—there is no captioning option. And a lot of their abuse stuff in the early days was horrific, where you would get notifications that a lot of people had this person blocked, but… that's not a helpful dynamic. “Did you talk to anyone? No, of course not. You Hacker News'ed it from first principles and thought this might be a good direction to go in.” This stuff is hard.People specialize in this stuff, and I've always been an advocate of when you're not sure what to do in an area, pay an expert for advice. All these stories about how people reach out to, “Their black friend”—and yes, it's a singular person in many cases—and their black friend gets very tired of doing all the unpaid emotional labor of all of this stuff. Suddenly, it's not that at all if you reach out to someone who is an expert in this and pay them for their expertise. I don't sit here complaining that my clients pay me to solve AWS billing problems. In fact, I actively encourage that behavior. Same model.There are businesses that specialize in this, they know the area, they know the risks, they know the ins and outs of this, and consults with these folks are not break the bank expensive compared to building the damn thing in the first place.Chloe: And here's a great example that literally drove me bananas a couple weeks ago. So, I don't know if you've participated in Twitter Spaces before, but I've done a couple of my first ones recently. Have you done one yet—Corey: Oh yes—Chloe: —Corey?Corey: —extensively. I love that. And again, that's a better answer for me than Clubhouse because I already have the Twitter audience. I don't have to build one from scratch on another platform.Chloe: So, I learned something really fascinating through my boyfriend. And remember, I mentioned earlier, my boyfriend is a staff engineer at Uber. He's been coding since he's been out of the womb, much more experienced than me. And I like to think a lot about, this is accessible to me but how is this accessible to a non-technical person? So, Ty finished up the Twitter Space that he did and he wanted to export the file.Now currently, as the time of this podcast is being recorded, the process to export a Twitter Spaces audio file is a nightmare. And remember, staff engineer at Uber. He had to export his entire Twitter profile, navigate through a file structure that wasn't clearly marked, find the recording out of the multiple Spaces that he had hosted—and I don't think you get these for ones that you've participated in, only ones that you've hosted—download the file, but the file was not a normal WAV file or anything; he had to download an open-source converter to play the file. And in total, it took him about an hour to just get that file for the purposes of having that recording. Now, where my mind goes to is what about some woman who runs a nonprofit in the middle of, you know, Sacramento, and she does a community Twitter Spaces about her flower shop and she wants a recording of that.What's she going to do, hire some third-party? And she wouldn't even know where to go; before I was in tech, I certainly would have just given up and been like, “Well, this is a nightmare. What do I do with this GitHub repo of information?” But these are the kinds of problems that you need to think about. And I think a lot of us and folks who listen to this show probably build APIs or developer tools, but a lot of us do work on products that muggles, non-technical people, work on.And I see these issues happen constantly. I come from this space of being an admin, being someone who wasn't quote-unquote, “A techie,” and a lot of products are just not being thought through from the perspective—like, there would be so much value gained if just one person came in and tested your product who wasn't you. So yeah, there's all of these things that I think we have a very privileged view of, as technical folks, that we don't realize are huge. Not even just barrier to entry; you should just be able to download—and maybe this is a feature that's coming down the pipeline soon, who knows, but the fact that in order for someone to get a recording of their Twitter Spaces is like a multi-hour process for a very, very senior engineer, that's the problem. I'm not really sure how we solve this.I think we just call it out when we see it and try to help different companies make change, which of course, myself and my boyfriend did. We reached out to people at Twitter, and we're like, “This is really difficult and it shouldn't be.” But I have that privilege. I know people at these companies; most people do not.Corey: And in some cases, even when you do, it doesn't move the needle as much as you might wish that it would.Chloe: If it did, I wouldn't be getting DMs anymore from creeps right? [laugh].Corey: Right. Chloe, thank you so much for coming back and talk to me about your latest project. If people want to pay attention to it and see what you're up to. Where can they go? Where can they find you? Where can they learn more? And where can they pointedly not audition to be featured on one of the episodes of Master Creep Theatre?Chloe: [laugh]. So, that's the one caveat, right? I have to kind of close submissions of my own DMs now because now people are just going to be trolling me and sending me weird stuff. You can find me on Twitter—my name—at @chloecondon, C-H-L-O-E-C-O-N-D-O-N. I am on Instagram as @getforked, G-I-T-F-O-R-K-E-D. That's a Good Placepun if you're non-technical; it is an engineering pun if you are. And yeah, I've been doing a lot of fun series with Microsoft Reactor, lots of how to get a career in tech stuff for students, building a lot of really fun AI/ML stuff on there. So, come say hi on one of my many platforms. YouTube, too. That's probably where—Master Creep Theatre is going to be, on YouTube, so definitely follow me on YouTube. And yeah.Corey: And we will, of course, put links to that in the [show notes 00:37:57]. Chloe, thank you so much for taking the time to speak with me. I really appreciate it, as always.Chloe: Thank you. I'll be back for episode three soon, I'm sure. [laugh].Corey: Let's not make it another couple of years until then. Chloe Condon, senior cloud advocate at Microsoft on the Next Generation Experiences Team, also chlo-host of the Master Creep Theatre podcast. I'm Cloud Economist Corey Quinn, and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice along with a comment saying simply, “Hey.”Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.

IGeometry
the cost of long-running database transaction (postgres/mysql)

IGeometry

Play Episode Listen Later Oct 21, 2021 9:25


The cost of a long-running update transaction that eventually failed in Postgres (or any other database for that matter. In Postgres, any DML transaction touching a row creates a new version of that row. if the row is referenced in indexes, those need to be updated with the new tuple id as well. There are exceptions with optimization such as heap only tuples (HOT) where the index doesn't need to be updated but that doesn't always happens. If the transaction rolls back, then the new row versions created by this transaction (millions in my case) are now invalid and should NOT be read by any new transaction. You have two solutions to address this, do you clean all dead rows eagerly on transaction rollback? Or do you do it lazily as a post process? Postgres does the lazy approach, a command called vacuum which is called periodically Postgres attempts to remove those dead rows and free up space in the page. Whats the harm of leaving those dead rows in? Its not really correctness issues at all, in fact transactions know not to read those dead rows by checking the state of the transaction that created them. This is however expensive, the check to see of the transaction that created this row is committed or rolled-back. Also the fact that those dead rows live in disk pages with alive rows makes an IO not efficient as the database has to filter out dead rows. For example, a page may have contained 1000 rows, but only 1 live row and 999 dead rows, the database will make that IO but only will get a single row of it. Repeat that and you end up making more IOs. More IOs = slower performance. Other databases do the eager approach and won't let you even start the database before rolling back is successfully complete, using undo logs. Which one is right and which one is wrong? Here is the fun part! Nothing is wrong or right, its all decisions that we engineers make. Its all fundamentals. Its up to you to understand and pick. Anything can work. You can make anything work if you know what you are dealing with. If you want to learn about the fundamentals of databases and demystify it check out my udemy course https://database.husseinnasser.com --- Send in a voice message: https://anchor.fm/hnasr/message Support this podcast: https://anchor.fm/hnasr/support

Syntax - Tasty Web Development Treats
Potluck — Coding for Kids × MongoDB Hosting × NoMoreFoo × Best Cities for Dev Jobs × GraphQL Resolvers × Package Security × Prototypes and Portfolios × More!

Syntax - Tasty Web Development Treats

Play Episode Listen Later Oct 20, 2021 59:48


It's another Potluck! In this episode, Scott and Wes answer your questions about privacy policies, coding for kids, MongaDB hosting, cloud backups, system design, #NoMoreFoo, and much more! Prismic - Sponsor Prismic is a Headless CMS that makes it easy to build website pages as a set of components. Break pages into sections of components using React, Vue, or whatever you like. Make corresponding Slices in Prismic. Start building pages dynamically in minutes. Get started at prismic.io/syntax. Sentry - Sponsor If you want to know what's happening with your code, track errors and monitor performance with Sentry. Sentry's Application Monitoring platform helps developers see performance issues, fix errors faster, and optimize their code health. Cut your time on error resolution from hours to minutes. It works with any language and integrates with dozens of other services. Syntax listeners new to Sentry can get two months for free by visiting Sentry.io and using the coupon code TASTYTREAT during sign up. Cloudinary - Sponsor Cloudinary is the best way to manage images and videos in the cloud. Edit and transform for any use case, from performance to personalization, using Cloudinary's APIs, SDKs, widgets, and integrations. Show Notes 04:49 - Ben Lamers: Heyo Scott and Wes! I am building a web app currently with my brother, and I was wondering when we get to launch it how do you go about correctly writing/adding Terms of Use and Privacy Policy. I'm assuming this may be quite different depending on the platform so maybe general resources or tips for this. Thanks! 06:45 - Fumbles O'Brian: Do you have any recommendations for teaching young children how to code? I have a 5-year-old niece in kindergarten who is absolutely fascinated watching me work, and I'd like to start teaching her basic concepts when she's able to read/write better. For example, she loves watching me make UI changes in React, it blows her mind that changing letters on one screen changes what a website looks like. 11:01 - Kenny: Gentlemen! Love this show and the content you put out. It keeps me occupied during my 5 and 6 mile runs. Thank you both for working so hard to keep it active, I know it takes a lot of work. I'm curious what you think about hosting your own MongoDB server? I'm relatively new to Mongo but want to start working with it for smaller projects. I've used MySQL for a decade, hosted online with shared hosting. Worked well for my relational db needs. Should I host my own Mongo when I'm ready for production, or pay the reasonable costs for something like Linode or maybe even Atlas? I have experience in Linux (enough to get by) and have my own virtualization cluster that I can spin up a server in seconds, along with an enterprise level firewall for managing traffic to and from. I actually just spun up a docker server this week and have a Mongo container running on it, though it's not accessible outside my network. This is purely for my development environments. Despite the firewall, my concern is security. Is it worth paying for a trusted solution like Linode, or should I put a little time in locking down my own Mongo container for my own use? Thank you both! Keep up the great work. 14:42 - Mike: Not a question but more of a rant… It's 2021, almost 2022, can we all stop using ‘foo' and ‘bar' and ‘baz' when teaching a programming concept? I applaud both of you because I don't recall seeing any of your content ever using such atrocious terms, however, I'm sad to see other prominent educators in the web development community use these terms from time to time. I feel like there are so many better examples that we could use to explain a concept and the use of ‘foo' is just confusing to beginners. That's all, just wanted to get that off my chest. Thanks for a wonderful podcast! #nomorefoo 18:53 - Amir: Hey Wes and Scott, thank you for your awesome podcast. What are the best cities in Canada and USA to get (more quantity, highest-paying) developer jobs? 23:44 - LW: Hi guys, I am finally starting to get into GraphQL and I don't get it. Specifically I am working to convert an existing REST API to GraphQL. This seems really tough and there is not much guidance out there on how to do it. The main part I am unsure of is how to write resolvers. If I use the existing query then GraphQL just seems like an over-engineered filter method. If I write an individual resolver for each column in the table - that's gonna be 100s of resolvers and super annoying to write. Have either of you ever moved something from REST to GraphQL? And, if so, how did you handle this? 27:57 - Dan: How does someone learn and actually practice using these system design topics like load balancing, caching, and database sharding. I have never had the need to use some of these things in my day-to-day work, but recently been interviewing and in the system design portion of the interview I feel a little lost. I've read about these topics and watched videos but haven't really seen how to implement these things. Any good resource recommendations? 31:57 - Matt: How do you know if you can trust an NPM package, from an unknown developer, that does not have many GitHub stars and has relatively few downloads? (The repo that made me ask this question is https://github.com/Wondermarin/react-color-palette). NPM audit automatically runs when you install a package, do any of you ever use additional security checks? 38:32 - Yosef: Hi I'm a beginner front-end developer and I heard you saying that being able to copy prototypes is a valuable skill, so I found some Figma free template and I copied them, the question is can I put them in my portfolio or deploy them? 40:00 - Nick: Hey dudes! I picked up a freelance project to make a brochure-style website and found myself having trouble to decide on what tools to pick for this site. I wanted to ask you and get your take, what tools/tech would you use to build a brochure site? By this, I mean the site should have mainly company information that is ideally editable by the stakeholders and has a contact form. Thanks! 44:22 - Casey: Hi Scooter and Wild Wes! Why do I feel so dirty when I'm forced to use negative values in CSS? 45:45 - Gnommer: Do you use some cloud sync service to backup your directory with projects? I mean OneDrive, Dropbox etc. I tried to use it alongside with Git, and it just messed my files so badly. On the other side I feel very uncomfortable without any backup apart from Github. BTW, according to last Potluck: polish ‘ł/Ł' is pronounced like ‘w' in ‘what a sick podcast you have'. Best from Poland ;) Links https://www.ryzerobotics.com/tello https://www.mongodb.com/cloud/atlas https://snyk.io/ https://deno.land/ https://kit.svelte.dev/ https://astro.build/ https://www.gatsbyjs.com/ https://www.dropbox.com/ https://www.backblaze.com/ https://www.synology.com/ https://support.apple.com/en-us/HT201250 ××× SIIIIICK ××× PIIIICKS ××× Scott: The Way Down Wes: Wooster Shortcut Shameless Plugs Scott: Modern GraphQL with Prisma - Sign up for the year and save 25%! Wes: All Courses - Use the coupon code ‘Syntax' for $10 off! Tweet us your tasty treats! Scott's Instagram LevelUpTutorials Instagram Wes' Instagram Wes' Twitter Wes' Facebook Scott's Twitter Make sure to include @SyntaxFM in your tweets

Screaming in the Cloud
Keeping the Cloudwatch with Ewere Diagboya

Screaming in the Cloud

Play Episode Listen Later Oct 14, 2021 32:21


About EwereCloud, DevOps Engineer, Blogger and AuthorLinks: Infrastructure Monitoring with Amazon CloudWatch: https://www.amazon.com/Infrastructure-Monitoring-Amazon-CloudWatch-infrastructure-ebook/dp/B08YS2PYKJ LinkedIn: https://www.linkedin.com/in/ewere/ Twitter: https://twitter.com/nimboya Medium: https://medium.com/@nimboya My Cloud Series: https://mycloudseries.com TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: This episode is sponsored in part by Honeycomb. When production is running slow, it's hard to know where problems originate: is it your application code, users, or the underlying systems? I've got five bucks on DNS, personally. Why scroll through endless dashboards, while dealing with alert floods, going from tool to tool to tool that you employ, guessing at which puzzle pieces matter? Context switching and tool sprawl are slowly killing both your team and your business. You should care more about one of those than the other, which one is up to you. Drop the separate pillars and enter a world of getting one unified understanding of the one thing driving your business: production. With Honeycomb, you guess less and know more. Try it for free at Honeycomb.io/screaminginthecloud. Observability, it's more than just hipster monitoring.Corey: This episode is sponsored in part by Liquibase. If you're anything like me, you've screwed up the database part of a deployment so severely that you've been banned from touching every anything that remotely sounds like SQL, at at least three different companies. We've mostly got code deployments solved for, but when it comes to databases we basically rely on desperate hope, with a roll back plan of keeping our resumes up to date. It doesn't have to be that way. Meet Liquibase. It is both an open source project and a commercial offering. Liquibase lets you track, modify, and automate database schema changes across almost any database, with guardrails to ensure you'll still have a company left after you deploy the change. No matter where your database lives, Liquibase can help you solve your database deployment issues. Check them out today at liquibase.com. Offer does not apply to Route 53.Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. I periodically make observations that monitoring cloud resources has changed somewhat since I first got started in the world of monitoring. My experience goes back to the original Call of Duty. That's right: Nagios.When you set instances up, it would theoretically tell you when they were unreachable or certain thresholds didn't work. It was janky but it kind of worked, and that was sort of the best we have. The world has progressed as cloud has become more complicated, as technologies have become more sophisticated, and here today to talk about this is the first AWS Hero from Africa and author of a brand new book, Ewere Diagboya. Thank you for joining me.Ewere: Thanks for the opportunity.Corey: So, you recently published a book on CloudWatch. To my understanding, it is the first such book that goes in-depth with not just how to wind up using it, but how to contextualize it as well. How did it come to be, I guess is my first question?Ewere: Yes, thanks a lot, Corey. The name of the book is Infrastructure Monitoring with Amazon CloudWatch, and the book came to be from the concept of looking at the ecosystem of AWS cloud computing and we saw that a lot of the things around cloud—I mostly talked about—most of