Podcasts about NewSQL

Bienvenue dans notre épisode de podcast consacré aux bases de données en 2023 ! Rejoignez-nous pour découvrir les dernières tendances dans le monde des bases de données, de SQL à NoSQL, en passant par les bases de données distribuées et les nouvelles générations de bases de données comme NewSQL. Retrouvez toutes les notes et les liens de l'épisode sur cette page : https://double-slash.dev/podcasts/newdb/

Welcome to another engaging and informative episode of Data Gurus! Sima is delighted to have Raj Bains, the CEO and Founder at Prophecy, joining her for today's show! Raj started Prophecy five years ago and decided to build one big product to get data ready for analytics. He talks to Sima about his business and product and where he fits within the industry. Raj's background Raj started his professional career in the early 2000s. He started with graphics, worked as a developer, got into power tools, and worked at Microsoft. After that, he joined a team at NVIDIA to build CUDA, which now gets used for Bitcoin mining. Then he moved into the data space and shifted from engineering into marketing and product management. While selling big data platforms and managing Apache Hive for the Hadoop Company at Hortonworks, he saw data users struggling to be productive with outdated tools. He solved that problem by building power tools to make it easy to get data ready for analytics very quickly. He has focused on doing that since then. Prophecy Raj started Prophecy in 2017 to build a visual tooling layer to get data ready for analytics. Data scientists, data analysts, and data engineers can use the tool to avoid having to do unnecessary work. Standardization There is a big problem with standardization within large organizations when it comes to building data pipelines. The problem with unstandardized codes  In the past, people within the industry used to write scripts. Then they started using standardized visual tools. After that, they moved to the cloud, but nobody wanted to get locked into that tool, so they got rid of the visual development tools and started using codes. The codes within companies are unstandardized, however, so everyone's code looks different. That has led to many different problems. A solution Prophecy's clients do visual drag-and-drop development, and Raj and his team write high-quality code for them. That has opened people up to returning to visual development and allowed much more standardization within the industry. Building a solution Raj and his team started working with a few big credit card companies and banks. Then they spent two or three years building their product. They still have a year or two of building ahead of them, but they have reached a point where they can solve the entire problem for companies using their product. An enterprise standard They have created an enterprise standard for all lines of business, where data analysts, data engineers, and everyone else within a company use the same tool and speak the same language. Data pipelines Prophecy makes it quick and easy for companies to build data pipelines. Data pipelines are essential for analytics because they provide the necessary information for asking intelligent questions. The data has to be high-quality, timely, and in the right shape to answer questions quickly. The future Raj believes that they will solve the issues with data management within the next three to five years. Bio:   Raj is the founder & CEO of Prophecy. Previously, Raj led project management of Apache Hive at Hortonworks through their IPO. He also headed product management and marketing for a NewSQL database startup. Raj continues to actively develop compiler and database technologies in his quest to create data tools “that don't suck.” His engineering roles include developing a NewSQL database, building CUDA at NVIDIA as a founding engineer, and as a compiler engineer working on Microsoft Visual Studio.   Links: Email me your thoughts! Sima@Infinity-2.com LinkedIn Twitter Infinity-2.com   Connect with Raj Raj Bains on LinkedIn Raj Bains on Twitter

Watch this episode on YouTube: https://youtu.be/GZVugQeJ_XY This Week's Guest is Data Tools Guru, Raj Bains Raj is the founder & CEO of Prophecy. Previously, Raj led project management of Apache Hive at Hortonworks through their IPO. He also headed product management and marketing for a NewSQL database startup. Raj continues to actively develop compiler and database […]

Today on That Tech Pod, Laura and Gabi talk with Raj Bains. Raj is the founder & CEO of Prophecy. Previously, Raj led project management of Apache Hive at Hortonworks through their IPO. He also headed product management and marketing for a NewSQL database startup. Raj continues to actively develop compiler and database technologies in his quest to create data tools “that don't suck.” His engineering roles include developing a NewSQL database, building CUDA at NVIDIA as a founding engineer, and as a compiler engineer working on Microsoft Visual Studio.

In this episode, Warner is joined by Daniel Holt, VP of Solutions Engineering at Cockroach Labs to discuss distributed databases, RDBMS vs NoSQL vs NewSQL and most importantly, all about Cockroach Db!

About AlexAlex holds a Ph.D. in Computer Science and Engineering from UC San Diego, and has spent over a decade building high-performance, robust data management and processing systems. As an early member of a couple fast-growing startups, he's had the opportunity to wear a lot of different hats, serving at various times as an individual contributor, tech lead, manager, and executive. Prior to joining the Duckbill Group, Alex spent a few years as a freelance data engineering consultant, helping his clients build, manage and maintain their data infrastructure. He lives in Los Angeles, CA.Links: Twitter: https://twitter.com/alexras/ Personal page: https://alexras.info Old Consulting website with blog: https://bitsondisk.com TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: The company 0x4447 builds products to increase standardization and security in AWS organizations. They do this with automated pipelines that use well-structured projects to create secure, easy-to-maintain and fail-tolerant solutions, one of which is their VPN product built on top of the popular OpenVPN project which has no license restrictions; you are only limited by the network card in the instance. To learn more visit: snark.cloud/deployandgoCorey: Today's episode is brought to you in part by our friends at MinIO the high-performance Kubernetes native object store that's built for the multi-cloud, creating a consistent data storage layer for your public cloud instances, your private cloud instances, and even your edge instances, depending upon what the heck you're defining those as, which depends probably on where you work. It's getting that unified is one of the greatest challenges facing developers and architects today. It requires S3 compatibility, enterprise-grade security and resiliency, the speed to run any workload, and the footprint to run anywhere, and that's exactly what MinIO offers. With superb read speeds in excess of 360 gigs and 100 megabyte binary that doesn't eat all the data you've gotten on the system, it's exactly what you've been looking for. Check it out today at min.io/download, and see for yourself. That's min.io/download, and be sure to tell them that I sent you. Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. I'm the chief cloud economist at The Duckbill Group, which people are generally aware of. Today, I'm joined by our most recent principal cloud economist, Alex Rasmussen. Alex, thank you for joining me today, it is a pleasure to talk to you, as if we aren't talking to each other constantly, now that you work here.Alex: Thanks, Corey. It's great being here.Corey: So, I followed a more, I'd say traditional path for a cloud economist, but given that I basically had to invent the job myself, the more common path because imagine that you start building a role from scratch and the people you wind up looking for initially look a lot like you. And that is grumpy sysadmin, historically, turned into something, kind of begrudgingly, that looks like an SRE, which I still maintain are the same thing, but it is imperative people not email me about that. Yes, I know, you work at Google. But instead, what I found during my tenure as a sysadmin, is that I was working with certain things an awful lot, like web servers, and other things almost never, like databases and data warehouses. Because if you screw up a web server, we all have a good laugh, the site's down for a couple of minutes, life goes on, you have a shame trophy on your desk if that's your corporate culture, things continue.Mess up the data severely enough, and you don't have a company anymore. So, I was always told to keep my aura away from the expensive spendy things that power a company. You are sort of the first of a cloud economist subtype that doesn't resemble that. Before you worked here, you were effectively an independent consultant working on data engineering. Before that, you had a couple of jobs, but you had gotten a PhD in computer science, which means, first, you are probably one of the people in this world most qualified to pass some crappy job interview of solving a sorting algorithm on a whiteboard, but how did you get here from where you were?Alex: Great question. So, I like to joke that I kind of went to school until somebody told me that I had to stop. And I took that and went and started—or didn't start, but I was an early engineer at a startup and then was an executive at another early-stage one, and did a little bit of everything. And went freelance, did that for a couple of years, and worked with all kinds of different companies—vast majority of those being startups—helping them with data infrastructure problems. I've done a little bit of everything throughout my career.I've been, you know, IC, manager, manager, manager, IT guy, everything in between. I think on the data side of things, it just sort of happened, to be honest with you, it kind of started with the stuff that I did for my dissertation and parlayed that into a job back when the big data wave was starting to kind of truly crest. And I've been working on data infrastructure, basically my entire career. So, it wasn't necessarily something that was intentional. I've just been kind of taking the opportunity that makes the most sense for me it kind of every juncture. And my career path has been a little bit strange, both by academic and industrial standards. But I like where I'm at and I gained something really valuable from each of those experiences. So.Corey: It's been an interesting area of I won't say weakness here, but it's definitely been a bit of a challenge when we look at an AWS environment and even talking about a typical AWS customer without thinking of any of them in particular, I can already tell you a few things are likely to be true. For example, the number one most expensive line item in their bill is going to be EC2, and compute is the thing that powers it. Now, maybe that is they're running a bunch of instances the old-fashioned way. Maybe they're running Kubernetes but that's how it shows up. There's a lot of things that could be, and we look at what rounds that out.Now, the next item down should almost certainly not be data transfer and if so we should have a conversation, but data in one form or another is very often going to be number two. And that can mean a bunch of different things, historically. It could mean, “Oh, you have a whole bunch of stuff in S3. Let's talk about access patterns. Let's talk about lifecycle policies. Let's talk about making sure the really important stuff is backed up somewhere. Maybe you want to spend more on that particular aspect of it.”If it's on EBS volumes, that's interesting and definitely worth looking into and trying to understand the context of what's going on. Periodically we'll see a whole bunch of additional charges that speak to some of that EC2 charge in the form of EMR, AWS's Elastic MapReduce, which charges a per-hour instance charge, but also charges you for the instances that are running under the hood and under the EC2 line item. So, there's a lot of data lifecycle stuff, there's a lot of data ecosystem stories, that historically we've consulted out with experts in that particular space. And that's great, but we were starting to have to drag those people in on more and more engagements as we saw them. And we realized that was really something we had to build out as a core competency for ourselves.And we started out not intending to hire for someone with that specialty, but the more we talked to you, the more it became clear that this was a very real and very growing need that we and our customers have. How closely it is what you're doing now as far as AWS bill analysis and data pattern deep-dive align with what you were doing as a freelance consultant in the space?Alex: A lot more than you might expect. You know, I think that increasingly, what you're seeing now is that a company's core differentiator is its data, right, how much of it they have, what they do with it. And so, you know, to your point, I think when you look at any company's cloud spend, it's going to be pretty heavy on the data side in terms of, like, where have you put it? What are you doing to process it? Where is it going once it's been processed? And then how is that—Corey: And data transfer is a very important first word in that two-word sequence.Alex: Oh, sure is. And so I think that, like, in a lot of ways, the way that a customer's cloud architecture looks and the way that their bill looks kind of as a consequence of that is kind of a reification in a way of the way that the data flows from one place to another and what's done with it at each step along the way. I think what complicates this is that companies that have been around for a little while have lived through this kind of very amorphous, kind of, polyglot way that we're approaching data. You know, back when I was first getting started in the big data days, it was MapReduce, MapReduce, MapReduce, right? And we quickly [crosstalk 00:07:29]—Corey: Oh, yes. The MapReduce white paper out of Google, a beautiful April Fool's Day prank that the folks at Yahoo fell for hook, line, and sinker. They wrote Hadoop, and now we're all stuck with that pattern. Great gag, they really should have clarified they were kidding. Here we are.Alex: Exactly. So—Corey: I mostly kid.Alex: No, for sure. But I think especially when it comes to data, we tend to over-index on what the large companies do and then quickly realize that we've made a mistake and correct backwards, right? So, there was this big push toward MapReduce for everything until people realize that it was just a pain in the neck to operate and to build. And so then we moved into Spark, so kind of up-leveled a little bit. And then there was this kind of explosion of NoSQL and NewSQL databases that hit the market.And MongoDB inexplicably won that war and now we're kind of in this world where everything is cloud data warehouse, right? And now we're trying to wrestle with, like, is it actually a good idea to put everything in one warehouse and have SQL be the lingua franca on top of it? But it's all changing so rapidly. And when you come into a customer that's been around for 10 or 15 years, and has, you know, been in the cloud for a substantial—Corey: Yeah, one of those ancient customers. That is—Alex: I know, right?Corey: —basically old enough to almost get a driver's license? Oh, yeah.Alex: Right. It's one of those things where it's like, “Ah, yes, in startup years, you're, like, a hundred years old,” right? But still, you know, I think you see this, kind of—I wouldn't call it a graveyard of failed experiments, right, but it's a collection of, like, “Well, we tried this, and it kind of worked and we're keeping it around because the cost of moving this stuff around—the kind of data gravity, so to speak—is high enough that we're not going to bother transitioning it over.” But then you get into this situation where you have to bend over backwards to integrate anything with anything else. And we're still kind of in the early days of fixing that.Corey: And the AWS bill pattern that we see all the time across the board of those experiments were not successful and do not need to exist, but there's no context into that. The person that set them up left five years ago, the jobs are still running on time. What's happening with them? Well, we could stop them and see who screams, but very often, that's not the right answer either.Alex: And I think there's also something to note there, too, which is like, getting rid of data is very scary, right? I mean, if you resize a Kubernetes cluster from 15 nodes to 10, nobody's going to look at you sideways. But if you go, “Hey, we're just going to drop these tables.” The immediate reaction that you get, particularly from your data science team more often than not is, “Oh, God, what if we need that?” And so the conversation never really happens, and that causes this kind of snowball of data debt that persists in some cases for many, many years.Corey: Yeah, in some cases, what I found has been successful on those big unknown questions is don't delete the data, but restrict access to it for a few weeks and see what happens. Look into it a bit and make sure that it's not like, “Oh, cool. We just did for a month, and now we don't need that data. Let's get rid of it.” And then another month goes by it's like, “So, time to report quarterly earnings. Where's the data?”Oh, dear, that's not going to go well, for anyone. And understanding what's happening, the idea of cloning a petabyte of data so you can run an experiment on it. And okay, turns out the experiment wasn't needed. Do we still need to keep all of that?Alex: Yeah.Corey: The underlying platform advancements have been helpful toward this as well, a petabyte of data now in Glacier Deep Archive cost the princely sum of a thousand bucks a month, which is pretty close to the idea of why would I ever delete data ever again? I can get it back within a day if I need it, so let's just put it there instead.Alex: Right. You know, funny story. When I was in graduate school, we were dealing with, you know, 100 terabyte datasets on the regular that we had to generate every time because we only had 200 terabytes of raw storage. [laugh]. And this was before cloud was yet mature enough that we could get the kind of performance numbers that we wanted off of it.And we would end up having to delete the input data to make room for the output data. [laugh]. And thankfully, we don't need to do that anymore. But there are a lot of, kind of, anti-patterns that arise from that too, right? If data is easy to keep around forever, it stays around forever.And if it's easy to, let's say, run a SQL command against your Snowflake instance that scans 20 terabytes of data, you're just going to do it, and the exposure of that to you is so minimal that you can end up causing a whole bunch of problems for yourself by the fact that you don't have to deal with stuff at that low-level of abstraction anymore.Corey: It's always fun watching how this stuff manifests—because I'm dipping a toe into it from time to time—the easy, naive answer that we could give every customer but we don't is, “Huh. So, you have a whole bunch of EMR stuff? Well, you know, if you migrate that into something else, you'll save a whole bunch of money on that.” With no regard for the 500 jobs that run against that EMR cluster on a consistent basis that form is a key part of business process. “Yeah, if you could just do the entire flow of how data is operated with throughout your entire business that would be swell because you can save tens of thousands of dollars a month on that.” Yeah, how about we don't suggest things that are just absolute buffoonery.Alex: Well, and it's like, you know, you hit on a good point. Like, one of my least favorite words in the English language is the word ‘just.' And you know, I spent a few years as a freelance data consultant, and you know, a lot of what I would hear sometimes from customers is, “Well, why don't we ‘just' deprecate X?”Corey: “Why don't we just—” “I'm going to stop you there because there is no ‘just.'”Alex: Exactly.Corey: There's always context that we cannot have as outsiders.Alex: Precisely. Precisely. And digging into that really is—it's the fun part of the job, but it's also the hard part of the job.Corey: Before we created The Duckbill Group, which was really when I took Mike Julian on as business partner and CEO and formed the entity, I had something in common with you; I was freelancing for a couple of years beforehand. Now, I know why I wound up deciding, all right, we're going to turn this into a company, but what was it that I guess made you decide to, you know, freelancing is all well and good, but it's time to get something that looks a lot more like a quote-unquote, “Traditional job.”Alex: So, I think, on one level, I went freelance because I wasn't exactly sure what I wanted to do next. And I knew what I was good at. I knew what I had a lot of experience at, and I thought, “Well, I can just go out and kind of find a bunch of people that are willing to hire me to do what I'm good at doing, and then maybe eventually I'll find one of them that I like enough that I'll go and work for them. Or maybe I'll come up with some kind of a business model that I can repeat enough times that I don't have to worry that I wake up tomorrow and all of my clients are gone and then I have to go live in a van down by the river.”And I think when I heard about the opening at The Duckbill Group, I had been thinking for a little while about well, this has been going fine for a long time, but effectively what I've been doing is I've been you know, a staff-level data engineer for hire. And do I want to do something more than that, you know? Do I want to do something more comp—perhaps more sophisticated or more complex than that? And I rapidly came to the conclusion that in order to do that, I would have to have sales and marketing, and I would have to, you know, spend a lot of my time bringing in business. And that's just not something that I have really any experience in or I'm any good at.And, you know, I also recognize that, you know, I'm a relatively small fish in a relatively large pond, and if I wanted to get the kind of like, large scale people, the like the big, you know, Fortune 1000 company kind of customers, they may not pay attention to somebody like me. And so I think that ultimately, what I saw with The Duckbill Group was, number one, a group of people that were strongly aligned to the way that I wanted to keep doing this sort of work, right? Cultural alignment was really strong, good people, but also, you know, you folks have a thing that you figured out, and that puts you 10 to 15 steps ahead of where I was. And I was kind of staring down the barrel that, I'm like, am I going to have to take six months not doing client work so that I can figure out how to make this business sustain? And, you know, I think that ultimately, like, I just looked at it, and I said, this just makes sense to me, like, as a next step. And so here we all are.Corey: This episode is sponsored by our friends at Oracle Cloud. Counting the pennies, but still dreaming of deploying apps instead of “Hello, World” demos? Allow me to introduce you to Oracle's Always Free tier. It provides over 20 free services and infrastructure, networking, databases, observability, management, and security. And—let me be clear here—it's actually free. There's no surprise billing until you intentionally and proactively upgrade your account. This means you can provision a virtual machine instance or spin up an autonomous database that manages itself, all while gaining the networking, load balancing, and storage resources that somehow never quite make it into most free tiers needed to support the application that you want to build. With Always Free, you can do things like run small-scale applications or do proof-of-concept testing without spending a dime. You know that I always like to put asterisks next to the word free? This is actually free, no asterisk. Start now. Visit snark.cloud/oci-free that's snark.cloud/oci-free.Corey: It's always fun seeing how people perceive what we've done from the outside. Like, “Oh, yeah, you just stumbled right onto the thing that works, and you've just been going, like, gangbusters ever since.” Then you come aboard, it's like, “Here, look at this pile of things that didn't pan out over here.” And it's, you get to see how the sausage is made in a way that we talk about from time to time externally, but surprisingly, most of our marketing efforts aren't really focused on, “And here's this other time we screwed up as well.” And we're honest about it, but it's not sort of the thing that we promote as the core message of what we do and who we are.A question I like to ask people during job interviews, and I definitely asked you this, and I'll ask you now, which is going to probably throw some folks for a loop because who talks to their current employees like this? But what's next for you? When it comes time for you to leave the Duckbill Group, what do you want to do after this job?Alex: That's a great question. So, I mean, as we've mentioned before, you know, my career trajectory has been very weird and circuitous. And, you know, I would be lying to you if I said that I had absolute certainty about what the rest of that looks like. I've learned a few things about myself in the course of my career, such as it is. In my kind of warm, gooey center, I build stuff. Like, that is what gives me joy, it is what makes me excited to wake up in the morning.I love looking at big, complicated things, breaking them down into pieces, and figuring out how to make the pieces work in a way that makes sense. And, you know, I've spent a long time in the data ecosystem. I don't know, necessarily, if that's something that I'm going to do forever. I'm not necessarily pigeonholing myself into that part of the space just yet, but as long as I get to kind of wake up in the morning, and say, “I'm going to go and build things and it's not going to actively make the world any worse,” I'm happy with that. And so that's really—you know, might go back to freelancing, might go and join another group, another company, big small, who knows. I'm kind of leaving that up to the winds of destiny, so to speak.Corey: One thing that I have found incredi—sorry. Let me just address that first. Like that—Alex: Sure.Corey: —is the right way to think about it. My belief has always been that you don't necessarily have, like, the ten-year plan, or the five-year plan or whatever it is because that's where you're going to go so much as it gives you direction and forces you to keep moving so you don't wind up sitting in the same place for five years with one year of experience repeated five times. It helps you remember the bigger picture. Because I've always despised this fiction that we see in job interviews where average tenure in our industry is 18 to 36 months, give or take, but somehow during the interviews, we all talk like this is now your forever job, and after 25 years, you'll retire. And yeah, let's be a little more realistic than that.My question is always what is next and how can we align in a way that helps you get to what's coming? That's the purpose behind the question, and that's—the only way to make that not just a drippingly insincere question is to mean it and to continue to focus on it from time to time of, great. What are you learning what's next? Now, at the time of this recording, you've been here, I believe three weeks if I'm not mistaken?Alex: I've—this is week two for me at time of recording.Corey: Excellent. Yes, my grasp of time is sort of hazy at the best of times. I have a—I do a lot of things.Alex: For sure.Corey: But yeah, it has been an eye-opening experience for me, not because, “Oh, wow, we have an employee.” Yeah, we've done that a few times before. But rather because of your background, you are asking different questions than we typically get during onboarding. I had a blog post go out recently—or will be by the time this airs—about a question that you asked about, “Wow, onboarding into our internal account structure for AWS is way more polished than I've ever seen it before. Is that something you built in-house? What is that?”And great. Oh, terrific, I'd forgotten that this is kind of a novel thing. No. What we're using is AWS's SSO offering, which is such a well-built, polished product that I can only assume that it's under NDA because Amazonians don't talk about it ever. But it's great.It has a couple of annoyances, but beyond that, it's something that I'm a big fan of, but I'd forgotten how transformative that is, compared to the usual approach of all right, here's your username, here's a password you're going to have to change, here are your IAM credentials to store on disk forever. It's the ability to look at what we're doing through the eyes of someone who is clearly deep into the technical weeds, but not as exposed to all of the minutiae of the 300-some-odd AWS services is really a refreshing thing for all of us, just because it helps us realize what it's like to see some of this stuff for the first time, as well as gives me content ideas because if it's new to you, I promise you are not the only person who's seeing it that way. And if you don't really understand something well enough to explain it, I would argue you don't really understand the thing, so it forces me to get more awareness around exactly how different facets work. It's been an absolutely fantastic experience so far, from my perspective.Alex: Thank you. Right back at you. I mean, spending so many years working with startups, my kind of level of expected sophistication is, “I'm going to write your password on the back of a napkin. I have fifteen other things to do. Go figure it out.” And so you know, it's always nice to see—particularly players like AWS that are such 800-pound gorillas—going in and trying to uplevel that experience in a way that feels like—because I mean, like, look, AWS could keep us with the, “Here's a CSV with your username and password. Good luck, have fun.” And you know, they would still make—Corey: And they're going to have to because so much automation is built around that—Alex: Oh yeah—Corey: In so many places.Alex: —so much.Corey: It's always net-additive, they never turn anything off, which is increasingly an operational burden.Alex: Yeah, absolutely. Absolutely. But yeah, it's nice to see them up-level this in a way that feels like they're paying attention to their customers' pain. And that's always nice to see.Corey: So, we met a few years ago—in the before times—at a mixer that we wound up throwing—slash meetup. It was in Southern California for some AWS event or another. You've been aware of who we are and what we do for a while now, so I'm very curious to know—and the joy of having these conversations is that I don't actually know what the answer is going to be, so this may never see the light of day if it goes to weird—Alex: [laugh].Corey: —in the wrong direction, but—no I'm kidding. What has been, I guess, the biggest points of dissonance or surprises based upon your perception of who we are and what we do externally, versus joining and seeing how the sausage is made?Alex: You know, I think the first thing is—um, well, how to put this. I think that a lot of what I was expecting, given how much work you all do and how big—well, ‘you all;' we do—and how big the list of clients is and how it gets bigger every day, I was expecting this to be, like, this very hyper put together, like, every little detail has been figured out kind of engagement where I would have to figure out how you all do this. And coming in and realizing that a lot of it is just having a lot of in-depth knowledge born from experience of a bunch of stuff inside of this ecosystem, and then the rest of it is kind of free jazz, is kind of encouraging. Because as someone that was you know, as a freelancer, right, who do you see, right? You see people who have big public presences or people who are giant firms, right?On the GCP side, SADA Systems is a great example. They're another local company for me here in Los Angeles, and—Corey: Oh, yes. [unintelligible 00:24:48] Miles has been a recurring guest on the show.Alex: Yeah. And he's great. And, like, they have this enormous company that's got, like, all these different specializations and they're basically kind of like the middleman for GCP on a lot of things. And, like, you see that, and then you kind of see the individual people that are like, “Yeah, you know, I'm not really going to tell you that I only have two clients and that if both of them go away, I'm screwed, but, like, I only have two clients, and if both of them go away, I'm screwed.” And so, you know, I think honestly seeing that, like, what you've built so far and what I hope to help you continue to build is, you know, you've got just enough structure around the thing so that it makes sense, and the rest of it, you're kind of admitting that no plan ever survives contact with the client, right, and that everybody's going to be different than that everybody's problems are going to be different.And that you can't just go in and say, “Here's a dashboard, here's a calculator, have fun, give me my money,” right? Because that feels like—in optimization spaces of any kind, be that cloud, or data or whatever, there's this, kind of, push toward, how do I automate myself out of a job, and the realization that you can't for something like this, and that ultimately, like, you're just going to have to go with what you know, is something that I kind of had a suspicion was the case, but this really made it clear to me that, like, oh, this is actually a reasonable way of going about this.Corey: We thought otherwise at one point. We thought that this was something could be easily addressed their software. We launched our DuckTools SaaS platform in beta and two months later, did the—our incredible journey has come to an end, and took it off of a public offering. Because it doesn't lend itself to solving these problems in software in any reasonable way. I am ever more convinced over time that the idea of being able to solve cloud cost optimization with software at VC-scale is a red herring.And yeah, it just isn't going to work because it's one size fits some. Our customers are, by definition, exceptional in many respects, and understanding the context behind why things are the way that they are mean that we can only go so far with process because then it becomes a let's have a conversation and let's be human. Otherwise, we try to overly codify the process, and congratulations, we just now look like really crappy software, but expensive because it's all people doing it. It doesn't work that way. We have tools internally that help smooth over a lot of those edges, but by and large, people who are capable of performing at especially at the principal level for a cloud economics role, inherently are going to find themselves stifled by too much process because they need to have the freedom to dig into the areas that are relevant to the customer.It's why we can't recraft all of our statements of work in ways that tend to shy away from explicitly defined deliverables. Because we deliver an outcome, but it's going to depend entirely, in most cases, up on what we discover along the way. Maybe a full-on report isn't the best way of presenting the data in the way that we see it. Maybe it's a small proof of concept script or something like that. Maybe it's, I don't know, an interpretive dance in front of the company's board.Alex: [laugh]. Right.Corey: I'm open to exploring opportunities. But it comes down to what is right for the customer. There's a reason we only ever charge a fixed fee for these things, and it's because at that point, great, we're giving you the advice that we'd implement ourselves. We have no partnerships with any vendor in the space just to avoid bias or the perception of same. It's important that we are the authoritative source around these things.Honestly, the thing that surprised me the most about all this is how true to that vision we've stayed as we've as we flushed out what works, what doesn't. And we can distantly fail to go out of business every month. I am ecstatic about that. I expected this to wind up cratering into a mountain four months after I went freelance. Not yet.Alex: Well, I mean, I think there's another aspect of this too, right? Because I've spent a lot of my career working inside of venture capital-backed companies. And there's a lot of positive things to be said about having ready access to that kind of cash, but it does something to your business the second you take it. And I've been in a couple of situations where, like, once you actually have that big bucket of money, the incentive is grow, right? Hire more people get more customers, go, go, go, go, go.And sometimes what you'll find is that you'll spend the time and the money on an initiative and it's clearly not working. And you just kind of have to keep doubling down because now you've got customers that are using this thing and now you have to maintain it, and before you know it, you've got this albatross hanging around your neck. And like one of the things that I really respect about the way that Duckbill Group is is handling this by not taking outside cash is, like, it frees you up to make these kinds of bets, and then two months later say, “Well, that didn't work,” and try something else. And you know, that's very difficult to do once you have to go and convince someone with, you know, money flowing out of their ears, that that's the right thing to do.Corey: We have to be intentional about what we're doing. One of the benefits of bringing you aboard is that one, it does improve our capacity for handling more engagements at the same time, but it also improves the quality of the engagements that we are delivering. Instead of basically doing a round-robin assignment policy we can—Alex: Right.Corey: —we consult with each other; we talk about specific areas in which we have specific expertise. You get dragged into a lot of data portions of existing engagements, and the rest of us get pulled into other areas in which you might not be as strong. For example, “What are all of these ridiculous services? I can't make heads or tails have the ridiculous naming side of it.” Surprise, that's not a you problem.It comes down to being able to work collaboratively and let each other shine in a way that doesn't mean we load people up with work. We're very strict about having a 40-hour or less work week, just because we're not rushing for an exit. We want to enjoy our time working, we want to enjoy what we're doing, and then we want to go home and don't think about work until it's time to come back and think about these things. Like, it's a lifestyle company, but that lifestyle doesn't need to be run, run, run, run, run all the time, and it doesn't need to be something that people barely tolerate.Alex: Yeah. And I think that, you know, especially coming from being an army of one in a lot of engagements, it is really refreshing to be able to—see because, you know, I'm fortunate enough, I have friends in the industry that I can go and say like, “I have no idea how to make heads or tails of X.” And you know, I can get help that way, but ultimately, like, the only other outlet that I have here is the customer and they're not bringing me in if they have those answers readily to hand. And so being able to bounce stuff off of other people inside of an organization like this has been really refreshing.Corey: One of the things I've appreciated about your tenure here so far is the questions that you ask are pitched at the perfect level, by which I mean, it is never something you could answer with a three-second visit to Google, but it's also not something that you've spent three days spinning your wheels on trying to understand. You do a bit of digging; it's a little unclear, especially since there are multiple paths to go down, and then you flag it for clarification. And there's really so much to be said for that. Really, when we're looking for markers of seniority in the interview process, it's admitting you don't know something, but then also talking about how you would go about getting the answer. And it's—because no one has all this stuff in their head. I spend a disturbing amount of time looking at search engines and trying to reformulate queries and to get answers that make sense.I don't have the entirety of AWS shoved into my head. Yet. I'm sure there's something at re:Invent that's going to be scary and horrifying that will claim to do it and basically have a poor user interface, but all right. When that comes, we'll reevaluate then because this industry is always changing.Alex: For sure. For sure. And I think it's, it's worth pointing out that, like, one of the things that having done this for a long time gives you is this kind of scaffolding in your head that you can hang things over. We're like, you don't need to have every single AWS service memorized, but if you've got that scaffold in your head going, “Oh, like, this thing sounds like it hangs over this part of the mental scaffold, and I've seen other things that do that, so I wonder if it does this and this and this,” right? And that's a lot of it, honestly.Because especially, like, when I was solely in the data space, there's a new data wareho—or a new, like, data catalog system coming out every other week. You know, there are a thousand different things that claim to do MLOps, right? And whenever, like, someone comes to me and says, “Do you have experience with such and such?” And the answer was usually, “Well if you hum a few bars, I can fake it.” And, you know, that tends to help a great deal.Corey: Yeah. “No, but I'll find out and get back to you,” the right answer. Making it up and being wrong is the best way to get rejected from an environment. That's not just consulting; that's employment, too. If 95% of the time, you give the right answer, but that one time and 20 you're going to just make it up, well, I have to validate the other 19 because I never know when someone's faking it or not. There's that level of earned trust that's important.Alex: Well, yeah. And you're being brought in to be the expert in the room. That doesn't necessarily mean that you are the all-seeing, all-knowing oracle of knowledge but, like, if you say a thing, people are just going to believe you. And so, you know, it's beholden on you—Corey: If not, we have a different problem.Alex: Well, yeah, exactly. Hopefully, right? But yeah, I mean, it's beholden on you to be honest with your customer at a certain point, I think.Corey: I really want to thank you for taking the time out of your day to got with me about this. And I would love to have you back on in a couple of months once you're fully up to speed and spinning at the proper RPMs and see what's happened then. I—Alex: Thank you. I'd—Corey: —really appreciate—Alex: —love to.Corey: —your time where's the best place for people to learn more about you if they haven't heard your name before?Alex: Well, let's see. I am @alexras on Twitter, A-L-E-X-R-A-S. My personal website is alexras.info.I've done some writing on data stuff, including a pretty big collection of blog posts on the data side of the AWS ecosystem that are still on my consulting page, bitsondisk.com. Other than that—I mean, yeah, Twitter is probably the best place to find me, so if you want to talk more about any weird, nerd data stuff, then please feel free to reach out there.Corey: And links to that will, of course, be in the [show notes 00:35:57]. Thanks again for your time. I really appreciate it.Alex: Thank you. It's been a pleasure.Corey: Alex Rasmussen, principal cloud economist here at The Duckbill Group. I am Corey Quinn, cloud economist to the stars, and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice along with an angry, insulting comment that you then submit to three other podcast platforms just to make sure you have a backup copy of that particular piece of data.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.

The open-source model has been a big catalyst for technology businesses across the world. While it is clear how they fit into other business models, it is still unclear what the business model behind an open-source business is. In this episode, we aim to answer that and more. We invited Neil Han, Head of APAC and EMEA for PingCAP, the first open-source unicorn in China, which raised 340million USD in total. Neil is a hardcore software person and a well-connected professional in the software industry across the globe with great operations and management skillset, looking forward to working with a fast-growing company and great leadership team to grow together. PingCAP raises $270M in Nov. 2020 to develop core technologies and advance the global expansion of its offerings! PingCAP is founded by the team that built TiDB, a world-leading open-source, cloud-native, distributed SQL/NewSQL database for elastic scale and real-time analytics, which is compatible with MySQL and enables companies to painlessly scale their business while keeping the underlying infrastructure simple and serve as a one-stop solution for all online transactions and analysis. PingCAP is the most valued NewSQL company on the planet!EPISODE NOTESThe open-source model has been a big catalyst for technology businesses across the world. While it is clear how they fit into other business models, it is still unclear what the business model behind an open-source business is. In this episode, we aim to answer that and more. We invited Neil Han, Head of APAC and EMEA for PingCAP, the first open-source unicorn in China, which raised 340million USD in total. Neil is a hardcore software person and a well-connected professional in the software industry across the globe with great operations and management skillset, looking forward to working with a fast-growing company and great leadership team to grow together.Show Notes:[1:26] Introduction to Neil Han[2:39] PingCap's value proposition[5:14] Companies that best leverage PingCap's services[8:05] The business model behind an open source business[12:08] Where the community contributors come from[15:05] Where the customers come from[17:29] Experience of selling the service to Chinese customers[2200] Customers as contributors to the product: A case of Square[26:38] The scope of growth for open-source software in APAC[30:16] Connect with NeilMany thanks to our guests Neil Han; host Oscar Ramos; producers Eva Shi and Sagar Chaudhary; editor David; organizer Chinaccelerator; and sponsor People Squared. Be sure to check out our website www.chinaccelerator.comShare, subscribe, review, enjoy!To join our listener group on WeChat, please add SOSV Helper (WeChat ID: sosvhero) and ask for the group invitation.To subscribe to our newsletter, please visit www.asiastartuppulse.comFollow us on LinkedIn: https://www.linkedin.com/company/asia-startup-pulseEmail us: asp-team@asiastartuppulse.com

Liquan Pei, Senior Software Engineer, Database Kernel at PingCAP sat down with the HOSS Matt Yonkovit to talk about PingCAP, Hybrid Transactional and Analytical Processing ( HTAP), NewSQL, and getting deeper into TiDB.  PingCAP is founded by the team that built TiDB, a world leading Open-Source distributed NewSQL database, for globally scalable HTAP which is compatible with MySQL, and enables companies to painlessly scale their business while keeping the underlying infrastructure simple.

The open-source model has been a big catalyst for technology businesses across the world. While it is clear how they fit into other business models, it is still unclear what the business model behind an open-source business is. In this episode, we aim to answer that and more. We invited Neil Han, Head of APAC and EMEA for PingCAP, the first open-source unicorn in China, which raised 340million USD in total. Neil is a hardcore software person and a well-connected professional in the software industry across the globe with great operations and management skillset, looking forward to working with a fast-growing company and great leadership team to grow together. Show Notes:[1:26] Introduction to Neil Han[2:39] PingCap's value proposition[5:14] Companies that best leverage PingCap's services[8:05] The business model behind an open source business[12:08] Where the community contributors come from[15:05] Where the customers come from[17:29] Experience of selling the service to Chinese customers[2200] Customers as contributors to the product: A case of Square[26:38] The scope of growth for open-source software in APAC[30:16] Connect with Neil Many thanks to our guests  Neil Han; host Oscar Ramos; producers Eva Shi and  Sagar Chaudhary; editor David; organizer Chinaccelerator; and sponsor People Squared. Be sure to check out our website www.chinaccelerator.comShare, subscribe, review, enjoy!To join our listener group on WeChat, please add SOSV Helper (WeChat ID: sosvhero) and ask for the group  invitation.To subscribe to our newsletter, please visit  www.asiastartuppulse.comFollow us on LinkedIn:  https://www.linkedin.com/company/asia-startup-pulseEmail us: asp-team@asiastartuppulse.com

話したネタ ゼロから作る時系列データベースエンジン 時系列データとは何か？ RDBで時系列データを扱う場合の課題とは？ 時系列データの特徴とは？ イミュータブルなデータとは？ influxDB Timescale DB VictoriaMetrics M3DB 時系列DBにおけるカーディナリティの高さとは？ tstorage なぜ時系列DBを自分で実装したのか？ ali gosivy tstorageの設計概要は？ パーティショニングのメリットとは？ Write Amplificatonとは？ Bloom Filter LSM Treeとは？ 34. NewSQLとは w/ tzkb メモリパーティションの特徴とは？ 時系列データをソート済みにする工夫 QuestDB パーティションをフラッシュするタイミングは？ Write Ahead Log データ量を削減する工夫は？ Gorilla: A Fast, Scalable, In-Memory Time Series Database タイムスタンプとデータを分けて符号化する delta encoding と delta-of-delta encoding データ側はXORで符号化する tstorageのdisadvantageは？ tstorageの今後の開発方針 YAGNI原則 【メディア事業部】サーバーサイドエンジニア（基盤） 宣伝 fukabori.fm の個人スポンサー募集中

No episódio de hoje, conversamos com o professor Fábio Coutinho sobre o Grupo de Pesquisa em Sistema Distribuídos (GSD) e algumas de suas linhas de pesquisas, como soluções NewSQL para banco de dados distribuídos, aplicações IoT, sistemas móveis e embarcados, entre outros. Instagram: @lacomp.ufal Host: Rodrigo Santos (@rosantostkd) Cohosts: Lucas A. Lisboa (@lucasa.lisboa) / Leonardo Leite (@leomvader) Editor: Paulo Victor (@paulov59)

Show Notes(1:56) Jim went over his education at Trinity College Dublin in the late 90s/early 2000s, where he got early exposure to academic research in distributed systems.(4:26) Jim discussed his research focused on dynamic software architecture, particularly the K-Component model that enables individual components to adapt to a changing environment.(5:37) Jim explained his research on collaborative reinforcement learning that enables groups of reinforcement learning agents to solve online optimization problems in dynamic systems.(9:03) Jim recalled his time as a Senior Consultant for MySQL.(9:52) Jim shared the initiatives at the RISE Research Institute of Sweden, in which he has been a researcher since 2007.(13:16) Jim dissected his peer-to-peer systems research at RISE, including theoretical results for search algorithm and walk topology.(15:30) Jim went over challenges building peer-to-peer live streaming systems at RISE, such as GradientTV and Glive.(18:18) Jim provided an overview of research activities at the Division of Software and Computer Systems at the School of Electrical Engineering and Computer Science at KTH Royal Institute of Technology.(19:04) Jim has taught courses on Distributed Systems and Deep Learning on Big Data at KTH Royal Institute of Technology.(22:20) Jim unpacked his O’Reilly article in 2017 called “Distributed TensorFlow,” which includes the deep learning hierarchy of scale.(29:47) Jim discussed the development of HopsFS, a next-generation distribution of the Hadoop Distributed File System (HDFS) that replaces its single-node in-memory metadata service with a distributed metadata service built on a NewSQL database.(34:17) Jim rationalized the intention to commercialize HopsFS and built Hopsworks, an user-friendly data science platform for Hops.(36:56) Jim explored the relative benefits of public research money and VC-funded money.(41:48) Jim unpacked the key ideas in his post “Feature Store: The Missing Data Layer in ML Pipelines.”(47:31) Jim dissected the critical design that enables the Hopsworks feature store to refactor a monolithic end-to-end ML pipeline into separate feature engineering and model training pipelines.(52:49) Jim explained why data warehouses are insufficient for machine learning pipelines and why a feature store is needed instead.(57:59) Jim discussed prioritizing the product roadmap for the Hopswork platform.(01:00:25) Jim hinted at what’s on the 2021 roadmap for Hopswork.(01:03:22) Jim recalled the challenges of getting early customers for Hopsworks.(01:04:30) Jim intuited the differences and similarities between being a professor and being a founder.(01:07:00) Jim discussed worrying trends in the European Tech ecosystem and the role that Logical Clocks will play in the long run.(01:13:37) Closing segment.Jim’s Contact InfoLogical ClocksTwitterLinkedInGoogle ScholarMediumACM ProfileGitHubMentioned ContentResearch Papers“The K-Component Architecture Meta-Model for Self-Adaptive Software” (2001)“Dynamic Software Evolution and The K-Component Model” (2001)“Using feedback in collaborative reinforcement learning to adaptively optimize MANET routing” (2005)“Building Autonomic Systems Using Collaborative Reinforcement Learning” (2006)“Improving ICE Service Selection in a P2P System using the Gradient Topology” (2007)“gradienTv: Market-Based P2P Live Media Streaming on the Gradient Overlay” (2010)“GLive: The Gradient Overlay as a Market Maker for Mesh-Based P2P Live Streaming” (2011)“HopsFS: Scaling Hierarchical File System Metadata Using NewSQL Databases” (2016)“Scaling HDFS to More Than 1 Million Operations Per Second with HopsFS” (2017)“Hopsworks: Improving User Experience and Development on Hadoop with Scalable, Strongly Consistent Metadata” (2017)“Implicit Provenance for Machine Learning Artifacts” (2020)“Time Travel and Provenance for Machine Learning Pipelines” (2020)“Maggy: Scalable Asynchronous Parallel Hyperparameter Search” (2020)Articles“Distributed TensorFlow” (2017)“Reflections on AWS’s S3 Architectural Flaws” (2017)“Meet Michelangelo: Uber’s Machine Learning Platform” (2017)“Feature Store: The Missing Data Layer in ML Pipelines” (2018)“What Is Wrong With European Tech Companies?” (2019)“ROI of Feature Stores” (2020)“MLOps With A Feature Store” (2020)“ML Engineer Guide: Feature Store vs. Data Warehouse” (2020)“Unifying Single-Host and Distributed Machine Learning with Maggy” (2020)“How We Secure Your Data With Hopsworks” (2020)“One Function Is All You Need For ML Experiments” (2020)“Hopsworks: World’s Only Cloud-Native Feature Store, now available on AWS and Azure” (2020)“Hopsworks 2.0: The Next Generation Platform for Data-Intensive AI with a Feature Store” (2020)“Hopsworks Feature Store API 2.0, a new paradigm” (2020)“Swedish startup Logical Clocks takes a crack at scaling MySQL backend for live recommendations” (2021)ProjectsApache Hudi (by Uber)Delta Lake (by Databricks)Apache Iceberg (by Netflix)MLflow (by Databricks)Apache Flink (by The Apache Foundation)PeopleLeslie Lamport (The Father of Distributed Computing)Jeff Dean (Creator of MapReduce and TensorFlow, Lead of Google AI)Richard Sutton (The Father of Reinforcement Learning — who wrote “The Bitter Lesson”)Programming BooksC++ Programming Languages books (by Scott Meyers)“Effective Java” (by Joshua Bloch)“Programming Erlang” (by Joe Armstrong)“Concepts, Techniques, and Models of Computer Programming” (by Peter Van Roy and Seif Haridi)

Show Notes(1:56) Jim went over his education at Trinity College Dublin in the late 90s/early 2000s, where he got early exposure to academic research in distributed systems.(4:26) Jim discussed his research focused on dynamic software architecture, particularly the K-Component model that enables individual components to adapt to a changing environment.(5:37) Jim explained his research on collaborative reinforcement learning that enables groups of reinforcement learning agents to solve online optimization problems in dynamic systems.(9:03) Jim recalled his time as a Senior Consultant for MySQL.(9:52) Jim shared the initiatives at the RISE Research Institute of Sweden, in which he has been a researcher since 2007.(13:16) Jim dissected his peer-to-peer systems research at RISE, including theoretical results for search algorithm and walk topology.(15:30) Jim went over challenges building peer-to-peer live streaming systems at RISE, such as GradientTV and Glive.(18:18) Jim provided an overview of research activities at the Division of Software and Computer Systems at the School of Electrical Engineering and Computer Science at KTH Royal Institute of Technology.(19:04) Jim has taught courses on Distributed Systems and Deep Learning on Big Data at KTH Royal Institute of Technology.(22:20) Jim unpacked his O’Reilly article in 2017 called “Distributed TensorFlow,” which includes the deep learning hierarchy of scale.(29:47) Jim discussed the development of HopsFS, a next-generation distribution of the Hadoop Distributed File System (HDFS) that replaces its single-node in-memory metadata service with a distributed metadata service built on a NewSQL database.(34:17) Jim rationalized the intention to commercialize HopsFS and built Hopsworks, an user-friendly data science platform for Hops.(36:56) Jim explored the relative benefits of public research money and VC-funded money.(41:48) Jim unpacked the key ideas in his post “Feature Store: The Missing Data Layer in ML Pipelines.”(47:31) Jim dissected the critical design that enables the Hopsworks feature store to refactor a monolithic end-to-end ML pipeline into separate feature engineering and model training pipelines.(52:49) Jim explained why data warehouses are insufficient for machine learning pipelines and why a feature store is needed instead.(57:59) Jim discussed prioritizing the product roadmap for the Hopswork platform.(01:00:25) Jim hinted at what’s on the 2021 roadmap for Hopswork.(01:03:22) Jim recalled the challenges of getting early customers for Hopsworks.(01:04:30) Jim intuited the differences and similarities between being a professor and being a founder.(01:07:00) Jim discussed worrying trends in the European Tech ecosystem and the role that Logical Clocks will play in the long run.(01:13:37) Closing segment.Jim’s Contact InfoLogical ClocksTwitterLinkedInGoogle ScholarMediumACM ProfileGitHubMentioned ContentResearch Papers“The K-Component Architecture Meta-Model for Self-Adaptive Software” (2001)“Dynamic Software Evolution and The K-Component Model” (2001)“Using feedback in collaborative reinforcement learning to adaptively optimize MANET routing” (2005)“Building Autonomic Systems Using Collaborative Reinforcement Learning” (2006)“Improving ICE Service Selection in a P2P System using the Gradient Topology” (2007)“gradienTv: Market-Based P2P Live Media Streaming on the Gradient Overlay” (2010)“GLive: The Gradient Overlay as a Market Maker for Mesh-Based P2P Live Streaming” (2011)“HopsFS: Scaling Hierarchical File System Metadata Using NewSQL Databases” (2016)“Scaling HDFS to More Than 1 Million Operations Per Second with HopsFS” (2017)“Hopsworks: Improving User Experience and Development on Hadoop with Scalable, Strongly Consistent Metadata” (2017)“Implicit Provenance for Machine Learning Artifacts” (2020)“Time Travel and Provenance for Machine Learning Pipelines” (2020)“Maggy: Scalable Asynchronous Parallel Hyperparameter Search” (2020)Articles“Distributed TensorFlow” (2017)“Reflections on AWS’s S3 Architectural Flaws” (2017)“Meet Michelangelo: Uber’s Machine Learning Platform” (2017)“Feature Store: The Missing Data Layer in ML Pipelines” (2018)“What Is Wrong With European Tech Companies?” (2019)“ROI of Feature Stores” (2020)“MLOps With A Feature Store” (2020)“ML Engineer Guide: Feature Store vs. Data Warehouse” (2020)“Unifying Single-Host and Distributed Machine Learning with Maggy” (2020)“How We Secure Your Data With Hopsworks” (2020)“One Function Is All You Need For ML Experiments” (2020)“Hopsworks: World’s Only Cloud-Native Feature Store, now available on AWS and Azure” (2020)“Hopsworks 2.0: The Next Generation Platform for Data-Intensive AI with a Feature Store” (2020)“Hopsworks Feature Store API 2.0, a new paradigm” (2020)“Swedish startup Logical Clocks takes a crack at scaling MySQL backend for live recommendations” (2021)ProjectsApache Hudi (by Uber)Delta Lake (by Databricks)Apache Iceberg (by Netflix)MLflow (by Databricks)Apache Flink (by The Apache Foundation)PeopleLeslie Lamport (The Father of Distributed Computing)Jeff Dean (Creator of MapReduce and TensorFlow, Lead of Google AI)Richard Sutton (The Father of Reinforcement Learning — who wrote “The Bitter Lesson”)Programming BooksC++ Programming Languages books (by Scott Meyers)“Effective Java” (by Joshua Bloch)“Programming Erlang” (by Joe Armstrong)“Concepts, Techniques, and Models of Computer Programming” (by Peter Van Roy and Seif Haridi)

話したネタ 2000年初頭のデータストアは何が主流だったのか？ OLTPとDWH データベースから見るとReadのスケールアウトは難しくない Web系で難しいのはWriteのスケールアウト RDBのReadのスケールアウト方法とは？ Web + RDB + Cache のアーキテクチャの辛い点は？ UniverseとMultiverse Oracle Exadata RDBにおける全文検索 NewSQLとは何か？ NoSQLとは何を指すか？ トランザクション処理はなぜ難しいのか？ マルチマスタの難しさ Google Cloud Spannerについて 金の弾丸 YugabyteDB/CockroachDB/TiDB YugabyteDBの特徴は？ PostgreSQL互換とMySQL互換という売り NewSQLの技術要素は？ NewSQLのレプリケーションはどうやるか？ Raftとは？ DBにおけるShardingとは何か？ Partioningとは何か？ RDBのデータ構造は何を利用しているか？ B+TreeのRead/Writeはどうやるか？ B+Treeの計算量は？ NewSQLのデータ構造は？ LSM Tree(Log Structured Merge Tree)とは？ B+Treeのメリット・デメリット LSM Treeのメリット・デメリット DBに難しいのは古いバージョンのデータを取るとき MVCC(Multi Version Concurrency Control)とは？ LSM Treeで古いデータをどうやって探すのか？ Bloom Filter Facebook製のRocksDB 分散トランザクションをどう実現するのか？ DBにおける分離レベルとは？ Read Commited/Repeatable Read/Serializable SpannerのExternal Consistency AWS Auroraの裏側の作りは？ OracleのRAC(Real Application Cluster)とは？ 令和時代のアプリケーション開発者のデータストア選定について MySQLとPostgrSQLの使い分けは？ どうやってDBについて学習するか？ CAPの定理をあえて使う必要はない Database Internals 輪読会

Это отличный выпуск про базы данных в целом и про некоторые детали мира БД. Если вы интересуетесь базами данных, но совсем глубоко в них не погружены – этот выпуск для вас. В гостях: Олонцев Сергей – эксперт по разработке ПО в Лаборатории Касперского, разработчик Big Data, специалист по базам данных, DBA по MS SQL, развивает сообщество MS SQL в Москве, спикер конференций и митапов. Про что: Кто такие администраторы баз данных (DBA) и за что им платят от 170000 руб. в Москве?Типы баз данныхACID, кластеризация, шардированиеБегло про (гео-)распределённые базы данных, cloud native, NewSQLХранилища и озёра данных (data warehouse, data lake) Скачать файл подкаста Добавить RSS в подкаст-плеер. Подкаст доступен в iTunes. Скачать все выпуски подкаста вы можете с яндекс-диска. Блог Сергея Олонцева: – olontsev.ru – sergeyolontsev.com Ссылки к выпуску: – Пирамида Маслоу-DBA: – Типы пользователей и администраторов БД: – Как правильно: sql или sequel? – MSDN – Olla Hallengren – Blitz – SQLSkills Blogs – SQL Performance – Microsoft WhitepapersUrl podcast:https://dts.podtrac.com/redirect.mp3/https://fs.linkmeup.ru/podcasts/sysadmin/linkmeup_sysadmins-V021(2020-04).mp3

Это отличный выпуск про базы данных в целом и про некоторые детали мира БД. Если вы интересуетесь базами данных, но совсем глубоко в них не погружены – этот выпуск для вас. В гостях: Олонцев Сергей – эксперт по разработке ПО в Лаборатории Касперского, разработчик Big Data, специалист по базам данных, DBA по MS SQL, развивает сообщество MS SQL в Москве, спикер конференций и митапов. Про что: Кто такие администраторы баз данных (DBA) и за что им платят от 170000 руб. в Москве?Типы баз данныхACID, кластеризация, шардированиеБегло про (гео-)распределённые базы данных, cloud native, NewSQLХранилища и озёра данных (data warehouse, data lake) Скачать файл подкаста Добавить RSS в подкаст-плеер. Подкаст доступен в iTunes. Скачать все выпуски подкаста вы можете с яндекс-диска. Блог Сергея Олонцева: – olontsev.ru – sergeyolontsev.com Ссылки к выпуску: – Пирамида Маслоу-DBA: – Типы пользователей и администраторов БД: – Как правильно: sql или sequel? – MSDN – Olla Hallengren – Blitz – SQLSkills Blogs – SQL Performance – Microsoft WhitepapersUrl podcast:https://dts.podtrac.com/redirect.mp3/https://fs.linkmeup.ru/podcasts/sysadmin/linkmeup_sysadmins-V021(2020-04).mp3

Это отличный выпуск про базы данных в целом и про некоторые детали мира БД. Если вы интересуетесь базами данных, но совсем глубоко в них не погружены – этот выпуск для вас. В гостях: Олонцев Сергей – эксперт по разработке ПО в Лаборатории Касперского, разработчик Big Data, специалист по базам данных, DBA по MS SQL, развивает сообщество MS SQL в Москве, спикер конференций и митапов. Про что: Кто такие администраторы баз данных (DBA) и за что им платят от 170000 руб. в Москве?Типы баз данныхACID, кластеризация, шардированиеБегло про (гео-)распределённые базы данных, cloud native, NewSQLХранилища и озёра данных (data warehouse, data lake) Скачать файл подкаста Добавить RSS в подкаст-плеер. Подкаст доступен в iTunes. Скачать все выпуски подкаста вы можете с яндекс-диска. Блог Сергея Олонцева: – olontsev.ru – sergeyolontsev.com Ссылки к выпуску: – Пирамида Маслоу-DBA: – Типы пользователей и администраторов БД: – Как правильно: sql или sequel? – MSDN – Olla Hallengren – Blitz – SQLSkills Blogs – SQL Performance – Microsoft Whitepapers

“A chaque use-case sa base NoSQL”Pour l'épisode 37, je recevais Vincent Heuschling, Fondateur de Affini-Tech et animateur du podcast Big Data Hebdo. Vincent vient nous raconter son parcours dans les bases de données et surtout nous expliquer le théorème CAP et les raisons de l’arrivée du NoSQL. Nous passons en revue avec lui les différents paradigmes (majeurs) du NoSQL et dans quels cas ils peuvent s’utiliser. Nous parlons enfin du NewSQL et tentons de répondre à cette question qui nous taraude tous: le SQL est-il mort ? On débriefe de l'épisode avec Emmanuel Demey qui sera dans l'épisode #45Ecouter l'épisode #37 en entier : #37 - Le SQL est mort, vive le SQL - Vincent Heuschling

This is the first episode of the Computing Podcast. There's massive innovation happening in all layers of the modern application. Starting from cloud substrates all the way to the edges like your browser or an IoT device. Today we begin by talking about the fourth wave of distributed systems. Going from NoSQL to NewSQL. Follow us on Twitter @dosco @strlen Links Time, Clocks, and the Ordering of Events in a Distributed System Paxos Consensus Algorithm Raft Consensus Algorithm Alex's blog post on non-Newtonian universe of distributed systems Amazon Aurora Design Considerations Amazon Aurora: Parallel Query

“A chaque use-case sa base NoSQL”Le SQL et ses bases Oracle/MySQL sont la base de tout dev. On y est tous passé. On en fait même encore beaucoup. Pourtant on entends parler de tant de nouvelles technos. Depuis plusieurs années le NoSQL a le vent en poupe et semble une promesse à tous les manquements d’une bonne vieille base SQL. Pourtant quand 2 technos robustes occupent le terrain du SQL, comment choisir parmi toutes les options de NoSQL ? Qu’est ce que le NewSQL ? Le SQL est-il mort ?Ce nouvel épisode d’IFTTD - If This Then Dev, présenté par Bruno Soulez et produit par CosaVostra, se penche sur cette techno si ancienne et pourtant si fiable: le SQL. Pourquoi le NoSQL est-il nécessaire, et à quel problème toutes ces technos tentent de répondre ? Le NewSQL est-il l’espoir de tout unifier ? Le D.E.V. de la semaine est Vincent Heuschling, Fondateur de Affini-Tech et animateur du podcast Big Data Hebdo. Vincent vient nous raconter son parcours dans les bases de données et surtout nous expliquer le théorème CAP et les raisons de l’arrivée du NoSQL. Nous passons en revue avec lui les différents paradigmes (majeurs) du NoSQL et dans quels cas ils peuvent s’utiliser. Nous parlons enfin du NewSQL et tentons de répondre à cette question qui nous taraude tous: le SQL est-il mort ? Liens évoqués pendant l’émissionLe podcast de Vincent: https://www.bigdatahebdo.com/Retrouvez tous nos épisodes sur notre site https://ifttd.io/listes-des-episodes/Continuons la discussion@vhe74 (https://twiter.com/vhe74)@ifthisthendev (https://twitter.com/ifthisthendev)@bibear (https://twitter.com/bibear)Discord (https://discord.gg/FpEFYZM)Facebook (https://www.facebook.com/ifthisthendev/)LinkedIn (https://www.linkedin.com/company/if-this-then-dev/)

Un repaso sobre las principales categorías de base de datos. Respuesta en Quora sobre este tema: https://qr.ae/pNvL9W

Стабильная, как Go 1, тройка Generic Talks снова на связи. К нам зашел Александр Морозов, напомнить о GopherCon Russia 2020 и поговорить о таймерах. В Телеграме помимо канала есть и чат, там можно все обсудить https://t.me/generictalks 00:00:00 - Гость Александр и его карьерный путь за руку с Go к C++ 00:06:52 - Как Гугл делает так, чтобы инженеры придерживались одного стиля написания кода. Анонс 2х докладов на GopherCon Russia от Александра и Елены Морозовы. 00:08:25 - Говорим про таймеры в Go и вообще) Связь таймеров со scheduler. Структуры данных для хранения таймеров плюсы и минусы разных подходов. 00:31:05 - Менеджмент сложности в языке и runtime. Гибкость и скорость экспериментов в разных языках. 00:50:43 - Статья про то, как какой-то человек потратил целую жизнь зря из-за того, что начал работать с Go вместо Rust. 01:17:09 - Spanner, и n00b intro в newSQL от Богдана :)

One of the biggest challenges for any business trying to grow and reach customers globally is how to scale their data storage. FaunaDB is a cloud native database built by the engineers behind Twitter's infrastructure and designed to serve the needs of modern systems. Evan Weaver is the co-founder and CEO of Fauna and in this episode he explains the unique capabilities of Fauna, compares the consensus and transaction algorithm to that used in other NewSQL systems, and describes the ways that it allows for new application design patterns. One of the unique aspects of Fauna that is worth drawing attention to is the first class support for temporality that simplifies querying of historical states of the data. It is definitely worth a good look for anyone building a platform that needs a simple to manage data layer that will scale with your business.

Сегодня мы в гостях у самого высоконагруженного сервиса, написанного на java, в России и странах СНГ. Ежемесячная аудитория этой соцсети 71 млн уникальных пользователей. Сегодня Loftblog в гостях у Одноклассников! Часть 1. Курс «Adroid-разработка: продвинутый уровень»: http://loftschool.com/course/adv-android/?utm_source=youtube&utm_medium=video&utm_campaign=devshow99&utm_content=link&utm_term=devshow99 Полезные ссылки от ведущего backend-разработчика Вадима Цесько: 1. Лекция про “Actor Model” (ссылки в слайдах): https://incubos.org/posts/2014/11/21/actor-model/ 2. Доклад “Потоковая обработка данных на Actor Model” (применение в Яндексе): https://addconf.ru/ru/talk/12823 3. Доклад “Фремворк Akka и его использование в Яндексе”: https://incubos.org/posts/2014/04/18/jpoint/ 4. Доклад “Потоковая обработка событий” (развитие темы): https://incubos.org/posts/2017/02/05/streaming-matching/ 5. Ссылки на бесплатные статьи ACM к предыдущим докладам: https://incubos.org/biography/ 6. Официальный сайт языка Scala: https://www.scala-lang.org/ 7. Официальный сайт фреймворка Akka: https://akka.io/ 8. Официальный сайт Apache Cassandra: http://cassandra.apache.org 9. Хранилище NewSQL от Одноклассников (Cassandra с транзакциями): https://habr.com/ru/company/odnoklassniki/blog/417593/ 10. Хранилище One Blob Storage от Одноклассников: http://profyclub.ru/docs/174 11. Хранилище One Cold Storage от Одноклассников: https://2017.jokerconf.com/2017/talks/4fasygftomyekgaqaaiauo/ 12. Все доклады Одноклассников в блоге на Хабр: https://habr.com/ru/company/odnoklassniki/blog/ 13. Библиотека one-nio от Одноклассников: https://github.com/odnoklassniki/one-nio 14. Java профайлер от Одноклассников: https://github.com/jvm-profiling-tools/async-profiler 15. Проекты Одноклассников с открытым исходным кодом: https://github.com/odnoklassniki 16. Технополис для студентов Политеха: https://polis.mail.ru 17. Слайды и видео лекций из курса Highload от инженеров Одноклассников: https://habr.com/ru/company/odnoklassniki/blog/437858/ 18. Группа с записями всех лекций Технополис в Одноклассниках: https://ok.ru/technopolis 19. Все образовательные лекции проекта Технострим Mail.ru: https://www.youtube.com/channel/UCmqEpAsQMcsYaeef4qgECvQ 20. Альтернативный Computer Science Center от JetBrains и Яндекс: https://compscicenter.ru/ Школа онлайн-образования: https://loftschool.com/ Telegram Loftblog: https://t-do.ru/loftblog Telegram IT-обучение: https://t-do.ru/it_loft Slack: http://slack.loftblog.ru/ Сайт: http://loftblog.ru/ Instagram: https://www.instagram.com/loftblog/ Группа вконтакте: http://vk.com/loftblog Facebook: http://www.facebook.com/loftblog Twitter: http://twitter.com/loft_blog

Hvordan får man ideer og inspiration til at finde nye database teknologier som kan anvendes i kundernes projekter? Jakob giver et eksempel på dette i forhold til NewSQL teknologien, repræsenteret ved CocroachDB. I Kents Corner fortæller Kent om Hit-Girl. Den amerikanske tegneserie er en udløber af Kick-ass, som både findes som tegneserie og film. Den meget voldelige serie har for nyligt fået en dansk tegner, og hun skal fylde store sko ud efter et par af branchens helt store navne.

Dans cet épisode, Emmanuel et Guillaume vous parlent de JDK 11, 12, 9, de GraalVM, de Kotlin, de Java et Jakarta EE, de serverless, de multi cloud, de consistance de données, de Linux, de l’Europe, de Bercy. Mais pas que ! Enregistré le 25 septembre 2018 Téléchargement de l’épisode LesCastCodeurs-Episode–196.mp3 News Langages Comprendre Java 9 et plus, on recommence: l’agenda proposé pour Java 12 Le train de release, fonctionalités vs securité Vive le Java libre! Des Java Champions Java 11 sort aujourd’hui Exemple d’utilisation de HttpClient de Java 11 (avec GSON pour marshalling JSON) JShell en profondeur sur InfoQ Concise method bodies Le podcast sur la circoncision Java reflection, but much faster, from OptaPlanner The Graal Frenzy par Julien Ponge Les co-routines en Kotlin et structured concurrency Voir aussi Java project Loom Runtimes Mettez à jour vos Jackson à la version 2.9.7 La suite de test de Java EE est open source La roadmap d’Eclipse Glassfish Oracle annonce Helidon Infrastructure LogDevice de Facebook, un homologue de Apache Kafka LogDevice vs Kafka Cloud Séries sur Spring Cloud sur GCP par Josh Long Serverless best practices Multi cloud is a trap NewSQL échouent dans leurs garanties et j’accuse Spanner Web Axa supprime son appli mobile: voilà pourquoi ? CORS un article explicatif Outillage Qui a la plus grande sur GitHub en Open Source Méthodologies Le désenchantement dans le développement logiciel Sécurité Protéger l’identité contre silhouette Faille de sécurité dans la distribution alpine utilisée dans les images docker Loi, société et organisation Linux se dote d’un code de conduite L’union Européenne adopte la directive sur le droit d’auteur: La directive de l’UE sur le droit d’auteur à l’heure du numérique est adoptée Droit d’auteur : préparer la défaite en célébrant la victoire du court-termisme Contre analyse de l’impact des articles 11 et 13 sur l’internet libre Ouverture des « sources » du simulateur économique de Bercy Outils de l’épisode Pouvoir faire un checkout de la pull request GitHub git config --global --add remote.origin.fetch "+refs/pull/*/head:refs/remotes/origin/pr/*" git fetch origin # And then git checkout pr/123 Rubrique débutant Les changements d’états dans Git Conférences Paris Web les 4, 5 et 6 octobre 2018. DevFest Nantes les 18 et 19 octobre 2018 - sold out. Jenkins World Europe du 22 au 25 octobre 2018 à Nice - (utilisez le code JWAHERITIER pour obtenir 20% de réduction). VoxxedDays Microservices du 29 au 31 octobre 2018. DevFest Toulouse le 8 novembre 2018. Devoxx Belgique du 12 au 16 novembre 2018 - sold out. Bdx.io le 9 novembre 2018 - sold out. Codeurs en Seine le 22 novembre 2018. Snowcamp du 23 au 26 janvier 2019. Nous contacter Soutenez Les Cast Codeurs sur Patreon https://www.patreon.com/LesCastCodeurs Faire un crowdcast ou une crowdquestion Contactez-nous via twitter https://twitter.com/lescastcodeurs sur le groupe Google https://groups.google.com/group/lescastcodeurs ou sur le site web https://lescastcodeurs.com/

Новости Проблемы с конфигурацией How And Why Swiftype Moved From EC2 To Real Hardware Triton: Docker and the “best of all worlds” Tiny Docker Operating Systems Citadel Exploring the cloud infrastructure landscape for container deployment Terraform 0.4 ChefConf 2015 Яблоко купило фундаментальную базу данных Доклад Ивана Глушкова про NewSQL базы данных FirebirdSQL ChefServer 12.0.7 зарелизился, про Policyfile Роутер от Facebook, и модульный свитч от Facebook Deploy Chef on Google Compute Engine with a Click Митап про Ансибл Конференция RootConf Рассылка DevOpsRu Slack чатик hangops Русское сообщество hangops

Recording Venue: Skype Guest: Michael Stonebraker Dr. Michael Stonebraker, one of the leading researchers and technology entrepreneurs in the database space, joins Robert for a discussion of database architecture and the emerging NewSQL family of databases. Dr. Stonebraker opens with his take on how the database market is segmented around a small number of use […]