Podcast appearances and mentions of pete yeah

In this episode of Building Texas Business, I sit down with Pete Mora, founder of Fajita Pete's, to talk about his journey from running a full-service restaurant to building a scalable, off-premise food concept focused on delivery and catering. Pete shares how starting small and keeping the menu focused allowed him to maintain quality and simplify operations. He explains that by limiting the menu and designing the kitchen accordingly, they were able to keep costs low while serving large groups efficiently. His approach helped transition from dine-in service to a streamlined catering and delivery model. We also discuss the importance of hiring well and establishing effective systems. Pete admits he learned the hard way about managing people and the value of setting expectations early. As the company grew, building a culture based on respect, structure, and direct communication became essential to maintaining consistency across locations. Finally, Pete reflects on what it takes to be an entrepreneur. He emphasizes being prepared for challenges, not romanticizing the journey, and staying committed to the long haul. His advice is to set small, achievable goals and surround yourself with people who complement your weaknesses. It's not about doing everything yourself, but about building a structure that supports the growth of your business and your team. SHOW HIGHLIGHTS I delve into the entrepreneurial journey of Pete Mora, founder of Fajita Pete's, exploring his evolution from waiting tables in college to establishing a successful restaurant chain. We discuss Pete's strategic business model, which emphasizes a small footprint and limited menu to enhance operational efficiency and specialize in off-premise services like delivery and catering. Pete shares insights on effective team management, highlighting the importance of hiring the right people, setting clear expectations, and fostering a supportive work environment. The episode explores how Pete's innovative approach and adaptation of technology, particularly during the COVID-19 pandemic, contributed to the growth and success of Fajita Pete's. I examine the significance of maintaining quality and consistency across locations, with a focus on centralizing production processes and collaborating with partners who possess operational expertise. We discuss the advantages of being based in Houston, a competitive environment that offers access to resources and professionals, and how this has influenced the scalability of the business. Pete provides practical advice for aspiring entrepreneurs, emphasizing the value of setting achievable goals and understanding that success can also come from significant contributions within a company. LINKSShow Notes Previous Episodes About BoyarMiller About Fajita Pete's GUESTS Pete MoraAbout Pete TRANSCRIPT (AI transcript provided as supporting material and may contain errors) INTRO Welcome to the Building Texas Business Podcast. Interviews with thought leaders and organizational visionaries from across industry. Join us as we talk about the latest trends, challenges and growth opportunities to take your business to the next level. The Building Texas Business Podcast is brought to you by Boyer Miller, providing counsel beyond expectations. Find out how we can make a meaningful difference to your business at BoyerMillercom and by your podcast team, where having your own podcast is as easy as being a guest on ours. Discover more at yourpodcastteam Now. Here's your host, chris Hanslick. Chris: All right, pete, welcome to Building Texas Business. Thanks for taking time to come on the show. I've been a big fan of your food for years, so really excited about this opportunity. Pete: Well, thank you. I know you told me that the babies grew up eating some of it, so those are always my favorite stories when we get to see multi-generational type of business. We've done some kids from kinder through college now. That's how old we are, that's how long you've been around, I love that I hope my daughters listen. Chris: They don't listen to all these. I think I know that to be true, but they're going to listen to this one. Hayden and Holland grew up on Fajita Pete's. For sure, Fajita Pete's, tell us what that is what you're known for, fajita Pete's. Pete: We're a very small footprint restaurant. 98% of what we do is off-premise. That's our thing. It's either pickup delivery. Catering is a huge part of what we do. By catering, we specialize in corporate catering. Okay, so office lunches, like here at your firm, people call us up 30 people, 11, 30, 14th floor, 29, 25, richmond, you know and then we just show up. They, they like the ease of it. We do three types of deliveries. We do with the warmers, with the servers if you need them, or just drop off and get out of your way which is the most? popular one, but we specialize in a very limited menu so that way we can feed the hundreds of people we do every morning on catering, and then at night we turn into a delivery company, kind of like a pizza company. You call us up, feed us for four. Boom, we show up at your door with the food. Chris: And I can attest that you show up fast. Pete: Yep, yeah, that's from those small menus, so we specialize. There's very few things I'm good at, so I try to do as little as possible so I can be good at those things. Chris: Yeah, keep it simple, stupid, exactly, I love it. So I would think keeping the menu narrow like that helps you with controlling food costs and inventory and managing kind of the cost side of your business. Pete: Yeah, business-wise it's good on a lot of levels. The fixed costs, like you said, the rent. So the menu decides your kitchen layout. So, it decides your equipment package. It decides how many feet I mean you can get very granular with these things. It decides how many feet you need between the equipment, how you lay everything. Our menu specifically allows you to get high ticket items in the small area you know fajitas and group type of events. So we got a high ticket average for the restaurant industry and that's because our items are sold for families or for big groups like this. So it allows you to crank out a lot of food in a short amount of time and allows you to keep it fresh, and it's also a menu that transports well you can compartmentalize everything and it arrives the way you want it to it makes sense. Chris: So let's go back to the beginning. What was the inspiration for you to start this in the first place? At the beginning. Pete: Well, I waited tables to school. I went to. I'm from, naturally from columbia, south america, when we moved here. I grew up inief. I went through high school here, I went to U of H, got into the entrepreneurship program at U of H and that was a program that taught me how to really think about what I can do and scale. Having waited tables, I said well, restaurants really allow you to find out very quickly if you're going to make it or not. And my mom always told me we didn't bring you here to fill out applications, we brought you here to do something. Chris: Wow, I love that yeah. Pete: So with that mentality I was blessed enough to get out of the entrepreneurship program and run into the Meyer family. You know Meyer Land, meyer Park, sure, and we were able to get a very good deal on a restaurant that had been closed for a few years. Mr Meyer was very gracious, allowed me to go into his office. I learned a lot through that negotiation and we started a full service restaurant right out of college, 60 tables in Meyer Park, and that really taught us a lot. I mean typical immigrant story. Right, we made the tables in our garage. You know, walk through the snow in the Houston summers, all that stuff Uphill both ways. Chris: Yeah, exactly. Pete: So that experience was very good, and I always mentioned the full service restaurant because it led to what ended up being Fajita. Chris: Pizza yeah, it's part of your journey, right yeah? Pete: Taught me. The first year was just survival. Second year was figuring out. Okay, now that we kind of know what we're doing because you never know you kind of know what you're doing. Now how do we add revenue? And that's how catering became a topic. Chris: Okay. Pete: It got to over the six years we were there, where we grew the catering business to where it was doing more than our lunch business, and so when it was time to renew a lease, it just didn't really make sense to have the 6,300 square feet we had. Right, we took a leap of faith and went to a 1,200 square foot kitchen in West University and, thank God, it worked out. Yeah, you know, and that's kind of, we were able to keep what we liked from the restaurant business and do it in a way that allowed me to keep doing it, because maybe if I was still in full service I wouldn't be in the business anymore. It was just a very you know, starting at 23, and it was just very hard to get it going yeah uh, but it taught all the lessons I wouldn't learn. I mean, you learn, you know how they say. You learn more the first six months out of college than you do. Chris: The four years in right, yeah. So what was the name of that first restaurant? Pete: Poblano's Mexican Grill. Chris: Okay. Pete: It was in Westmire Park. Now they tore down that area. There's a Coles there. It's a nice development. Chris: Yeah. Pete: I'm very grateful to the people in that area. They were very nice to us. We still deliver to them a lot. You know, we have people that come by the kids. Like I mentioned earlier, we've been able to feed kids from since they're little to college, and when they're in town they come and say hi. So so it's been a very good community based business from the beginning, and I think that's what allowed us to really get the roots down. That allowed us to, you know, withstand all the problems that come with a new business. Chris: Yeah. So let's talk a little bit about those early days. You just said, and kind of alluded to it, those first six months out of college. You feel like you learned a lot more than you did in the four years in college. Anyone in any business starting out on their own there's a lot of headwinds. There's lessons learned. What were some of the things that the challenges you faced that you might share with the listeners to say now you know I went through this, you're likely to go through it too. Here's what I did to help and maybe something you experienced that someone might learn from your experience, to avoid Anything that you share from that perspective. Pete: I think at the beginning, with the restaurant business, you're in the people business right, and the biggest issue that you don't get practice at is managing your employees, your team, your coworkers. I think that was a big eye-opener because maybe people you run into people that don't have the work ethic you have, so you have to learn and I was, you know, young and my mom always told me start when you're young and dumb, because if you're smart enough, you won't do it. If you were smart enough, you wouldn't do this. Chris: Well, there's plenty of people that have come on the podcast that have said you know, yeah, you're never really ready, you just got to take the leap. It's like having a child. Pete: Yeah, you're never ready, it just comes, comes and now you have to figure it out. So, dealing with a crew setting up, the importance of setting up systems from the beginning and not willy-nilly as you go, which I did at the beginning, that's a problem. I said well, we'll figure it out. Chris: Follow your heart. Pete: I think during those trying times at the beginning, it was that developing systems, figuring out how to be a better leader to the people, and also all that while controlling cost, and what really helped me through it because businesses fail from being undercapitalized or mismanaged and what helped me through it is that we were always able to live very below our means to really withstand those pressures that came at the beginning. Chris: Okay, yeah. So yeah, it's good insight. You talked about the challenges of you know work. You're managing a team. There's challenges in building the team right Because you got to get it right and you're going to run across times where you make bad hires. What are some of the things that you've learned over the years? To try to? You know perfect that either the interview process or whatever to make sure you're hiring the right team members that can kind of buy in and get aligned behind your mission and your beliefs. Pete: I think it's hiring. When you hire out of necessity, you're already at a disadvantage. You just need bodies, right. I think, hiring at the right, getting the right person even if you don't have a spot for them, and making a spot for them Somebody that you think can be good for your team. 85 percent of the employee issues are solved at the hiring table. Yeah, you know, you need to get better at picking people who represent, and now people when they're applying, they talk very well, right, so you've got to get used to looking through the through the BS. That's the hard part yeah, and, and we're in the industry where we're talkers right, the end is the restaurant industry and also understanding and being compassionate that most people that are in the restaurant industry didn't that wasn't their goal originally. We get a lot of people that end up in this industry okay, so you have to be understanding of the path they've been through yeah, I didn't think about that yeah, and so so you have to allow them a growth path within your company, allow them a system that shows them respect and treats them like people, because I hear horror stories about how people get treated other places or and it's just the industry word, sure. So I think having that empathy and looking at the same time, looking through the BS and making sure the person you're hiring has somewhat the same values and goals, or at least providing a structure to where it's very clear what you expect from them, I think that's the only fair way to hire somebody Right, because at the beginning I was hiring people and I didn't have a structure, so that's unfair to them because I figured out, like I'm doing current. Chris: Right, right, right, you're living and breathing it. Yeah, someone you hire may not. Pete: No, they need a job yeah, and so learning that that immaturity I had at the beginning, that was key as well and it was unfair to the employee. So really creating systems becomes a way to be more fair to your employees. Set clear expectations, trackable expectations and achievable expectations. I think that's key to them. Chris: So you have this restaurant right, full service for six years or so, and you make a decision that's pretty significant to kind of really change up your entire business model. What was it that kind of you know led you to get the I don't know confidence or the ability to take that risk Like? Pete: everything else. Necessity Necessity Right, because you're signing an extension of a lease and it feels like a plea deal. That's not a good that's a good sign that you shouldn't sign that extension. You know Right, good sign that you shouldn't sign that extension, you know so. Then, luckily, some of my um. Greg lewis is a gentleman who found the spot for me at west you and I'm always very grateful to him. He was just a customer of mine. He said, hey, there's a spot. If you really think about doing it, there's a spot that's available, you should check it out and so I'm very grateful to him. Always he does a lot of real estate here in houston and um, it was a leap of faith in that. I knew that in 10 years I didn't want to be doing what I was currently doing. So I always think if you don't want to do it in 10 years, then don't do it today, because what are you doing? So I said I like this part of the industry and it was also more scalable. The model I was creating was more scalable and coming out of the entrepreneurship program, I mean, you go there to scale businesses. They teach you a lot of things. So I said this is more scalable. With my small brain, it was easier to fill in the spreadsheet and really explain to myself and prove to myself that it was going to be more profitable in the long run. Chris: It's interesting. I wrote a note to come back to this because when you were talking about full service to what you're doing now, it struck me that what you're doing now is way more scalable than a full service restaurant. It's tough and there's plenty of great examples in town that are still very successful restaurants, but those full service restaurants seem to have a. They gain up, the ones that sustain, develop a personality, typically around that founder and entrepreneur right who's there, and we have lots of great examples. I frequent them a lot but they're tough to scale. Maybe they go to two locations, but it's like it's hard to get beyond that personality of that person and what you've done is is may created something that still has great quality food, has your stamp on it, but doesn't require you to be at all the locations for it to be successful and I think that comes from the, the low operating cost, the simplified menu we spoke about earlier, all those things you know. Pete: I do think that the full service there's always. There's great restaurants here. Like I said, I was born in colombia. I learned how to eat mexican food at ninfas yeah you know, and, and there's a lot of amazing restaurants here in houston. People always ask me well, what do you think about this place? They're great. There's a dude, we're blessed in houston there's so much good food and it's just to me. I tell them it's just an honor to be even mentioned. Like they say oh well, I like your food. I compare it to this restaurant. I compare that. You know that's crazy yeah now it's to think of how it's scaled and how how people know we're actually starting to go from the. What the heck is a fajita pizza? Chris: oh, there's a fajita pizza right, which is a weird turn yeah, you know, I don't know how to tell a different mind. Yeah, yeah, so so you, yeah, so you start this new concept, you start to it. Let's talk through what were some of the challenges you faced in scaling the business. Pete: I think it was educating the market. Right, it was really letting people know because at that time in 2008,. You either got pizza delivered or Chinese and that's it. Yeah, so really educating the market that we do delivery. They're like, oh, okay, so for like 150 people or no, no, no, we do. And you would tell the customers would come in, it's a little room like this. They would come in multiple times and then until while they were there, they saw drivers going. They go oh, you deliver to houses. And it was just an education time because Uber wasn't really big. No, third-party platforms, so that was a very difficult thing. I also put the tortilla lady right behind the counter so I could see her hand-rolling tortillas, because also, when you go to smaller locations, people assume it's a lesser product. Right, it's an inferior product. So I said, well, let's put the lady right here so they know, because they're used to going to the restaurant and seeing the ladies make tortillas the good ones, you know and so they were like, oh, I get it, y'all make your food. It was just a big education process, educating the markets always was the hardest thing at the beginning. Chris: Talking about the tortillas, your quality of food is outstanding, my opinion. Thank you. I think other people agree. One of the challenges for a restaurant, I would think, in the scale, is maintaining the quality and the consistency. Is maintaining the quality and the consistency what? And you? You talked earlier about putting systems in place, but what? What are you doing to make sure that quality and consistency is there, because you're now all across the state of texas? Pete: yep. So it was very difficult to begin. Like I said, we we used to trim in house marinating house. It's a. I mean, we can get into the weeds on processes and procedures, but you would always maybe go to another store where they added sugar instead of salt to the marinade. So you're like, oh, we can't do that. Very early on I realized that I needed more system, more structure, and one of my goals was to grow. But I knew I couldn't do it the way I was doing it. Then we got to the point where we could marinate our stuff centrally and distribute it to our food distributor who takes it to the stores. So that was one thing that saved some concern, because that's the biggest heartache is the worst thing you can hear is I don't like that one as much as that one. Chris: Right, it's just different, right? Yeah, that hurts yeah. Pete: So it's developing relationships with manufacturers that can actually do your recipe the right way and deliver it to the customers. Chris: Sorry, yeah, go ahead. You've been talking a lot. Advert: Hello friends, this is Chris Hanslick, your Building Texas business host. Did you know that Boyer Miller, the producer of this podcast, is a business law firm that works with entrepreneurs, corporations and business leaders? Our team of attorneys serve as strategic partners to businesses by providing legal guidance to organizations of all sizes. Get to know the firm at boyermillercom. And thanks for listening to the show. So let's talk a little bit about innovation. What are some of the things that you feel like you have done, or maybe doing that are somewhat innovative, to help your business grow? Pete: Well, you go back to getting a good team around you, right? So when I started to grow, I ran into one of my partners, joey Aguia, who had a lot of operational experience in the industry. He was a franchisee for multiple concepts and, because I don't want people to think this all came out of my brain, I had a lot of help along the way and I would stop you because that is such a great point. Chris: Rarely has anyone just done it on their own right. You surround yourself and maybe we'll talk a little bit about that in a second, but I think it's a great point we should pause on, it's knowing what you're not good at. Yeah. Pete: And luckily. Chris: I'm so bad at so many things that it was just finding people all over the place that could help me. Well, I doubt that part but I bet you know. Pete: So innovation so you were talking a little bit about All right. So innovation was going to these manufacturers and our recipe is citrus-based. So technology really caught up to what we do in that not that we do anything special, but the way we do it was very hard for technology to be able to put it in a stable way that could perform the way we expected. So along the way we got with further processors who marinated our product and that was a three-year process to really get it right. So that's innovation on their side right. For us it was our technology, our apps, our ability that helped us survive through COVID. Actually Our ability to do curbside, to be quick and responsive, the way we package our things, the way we set up our kitchen, which took a couple iterations, and to really limit steps and increase customers service through speed and convenience and, like I tell people, the three C's you know convenience, a consistency and quality dad joke, like my daughter says that joke and and so. So those three things really helped us to speed up our service and match. So the problem there becomes matching your service through technology and through your processes on the front end. So really investing into that at an early point helped us keep a little bit ahead of the curve along the way. Okay, so those type of innovations. Chris: That's great. Head a little bit ahead of the curve along the way, okay. So those type of that's great. So, as I mentioned, started in houston and one spot in west university. Uh, you've now expanded throughout the state. What are some of the advantages that you've experienced or see in being a texas-based business and maybe it'll kind of spread across the state? I? Pete: think that that well, I came to Houston. I don't you know, being here, you don't realize how big it is and how competitive it is, and, as I think being in that competitive environment really helped us, you know you gotta. I mean, you can't sell fajitas 23 years in Houston and not be good. Chris: Right. Pete: There's 10,000 restaurants that do a great job in Houston, so I think that was one of the things Having access to big providers, distribution, the ports, being close for our produce. You know being close to the border, so I think that being a hub allows you to control costs, develop business and really grow strategically. Having so many people here law firms, business professionals to help you along the way, you have everything you need in Houston, In Texas, you know, by professionals to help you along the way. You have everything you need in Houston In Texas by default, so I think that was really an advantage. Although it's hard to make it, I think you have the tools you need if you're lucky enough to run into them early. Chris: Okay, that's good. So, when you think about the restaurant industry, what are some of the headwinds that you feel like your company or the industry may be facing today, or that you see around the corner that you're trying? Pete: to. Right now. I think there's a crunch. People are spending a little bit less, right? I think we all see it at the grocery stores, we see it everywhere. So a headwind is how do you translate your value proposition to the guests? We're a community-based business, so we tell people teachers, preachers and coaches so that is part of the value proposition you have as a brand. But people still want to. I mean, it's all about the meat and potatoes, right? They want to get a good value for their food. So the headwinds are figuring out how to provide value, control cost and still deliver the quality that the customer deserves. Gotcha. Chris: What was one of the? When you think about struggles or lessons learned, what was one of the maybe failures or mistakes you feel like you made along the way in those early days that you overcame, but it taught you a lesson that you'll never forget. Pete: I think it was really like we mentioned in the beginning hiring problems. At the beginning it was a lack of maturity. It was a lack of having systems, like we mentioned. I think, maybe underestimating what I did in that well, if I did, anybody can do it and maybe making bad decisions on people I could put in key situations, and those probably came because my systems weren't good enough also. So I think really underestimating what you do as an entrepreneur is a problem that we all have. Underpricing your services is a problem we all have. And trying to compete with my full service restaurant, trying to be everything to everybody, having 70 items on the menu that was the biggest lesson. I said no, there's too much waste. It's too hard to be good at everything else. Let's just be good at one thing and replicate that. So I think those lessons really got us to where we are in learning how to trust other people in areas that you're weak at. Chris: I said well, going back to being the best at one thing, it's almost like the GE mindset, right? Yeah, let's be. If you can't be the best, let's not do it. Pete: Yeah, I got any color you want, as long as it's black. Yeah, you know so that's the way you, that's the only way I could figure out how to do it. Chris: Yeah, the kind of going back to the people and I agree with your emphasis on so much of it is in the interview process, hiring process. You usually learn that the hard way, right. And then you get to what do you do when you realize you made a mistake? And what have you kind of learned over the years about when you realize you've made a mistake, how to handle that and how to move? Pete: on. I think you have to be fair to the person. You have to let them go. You have to get rid of the people, right, inform them of why it's probably not a good fit. Most of the times, people just fire themselves. But in this situation where you make the mistake, you just have to be fair. Let them know what their strengths and weaknesses are, because I would want somebody to do it to my son. Sure, let him know where he messed up so he can be better next time. Say, hey, it ain't going to be here, but if you get better, you'll do better there. So having those clear conversations those are tough conversations to have, sure, they are, but they're important because they're part of leadership is not when you're with the person you know. Good leadership extends to when they're gone from you and in the lessons, like a lot of the leaders I've come across, I say the things they say still 30 years later because they're good leaders. So they don't have to be next to you to be a leader. So we, as people who are in charge of businesses, we have to understand that the leadership style we need needs to be forward thinking, right. Chris: But it's a great mindset to share. So let's talk a little bit about leadership then. I always like to ask people how would you describe your leadership style? How do you think that's evolved over the last? Pete: almost 20 years. I think it's like people, I'm very relaxed leadership style. I'm really not very confrontational. I need to get better at those things. But I take everybody's opinion. It's very disseminated the way we make decisions. It's never hey, make the call, pete, right, no, no, we talk about it. I involve everybody. People that maybe even ask why are you asking my opinion? Why do I? Whatever, I think it's very important to always take the opinion of people actually in the front lines, and that's the reason I like having my store, my original store, always because I'm making decisions for a store in Chicago, well, I need to feel the pain of that decision, I need to feel the weight of it, and if I wouldn't do it at my store, why would I do it at theirs? So it keeps you honest. You have to be with the team. I guess is the question for leadership. Chris: Sounds like kind of that servant mindset. Pete: Yes. Chris: I'm not going to ask you to do something I wouldn't do myself Exactly. Pete: And then, once decisions are made, expect everyone to know, get behind it, move forward. Yep, exactly, I think that's important part of especially the industry we're in, because people you, you deal with hourlies, salaries. Now with the franchising corporate staff is a different employee than at the store level, but everybody has a, a trigger. You can. You can touch people in different ways, but you have to take the time to know what motivates them. So you have to to be in it. Chris: Got to be in it. So we've kind of been talking around this, but let's go right into culture, right? So all you're talking about hiring the right people, telling them and being clear about expectations, giving them motivation and incentive to do that job. You know, how would you describe the culture that you believe you have at Fajita Pete's? And then what are you doing to make sure that, as you scale the business beyond that one location, that it is resonating in those other? Pete: locations. We need to do a better job of culture building. You always need to be doing a better job of culture building. I think it's by example, like I told about the leadership side. So we try to communicate directly with the teams. If there's issues, you communicate directly with the frontline people that don't expect a call from you, but it lets them know hey, somebody's watching, and not only when there's issues. When there's good things happening, you need to communicate that to them, because it's like putting deposits in a bank account right, every interaction you have with somebody is that you're putting a deposit or you're taking a withdrawal and if you go and only withdraw, they're not going to listen to you. There's nothing there. So you have to build that up. So those interactions are part of what we do as a culture. We need to incorporate probably more team meetings, as we do. We did at the beginning and now, as the company grew, we have multi-unit franchisees who kind of have a good vibe going on good culture within them. But I do think getting back to more scheduled team events is going to be good. We just had one a few months ago. So those things that allow people to see that you're in it with them. You're going through the same issues it's not just you and you have that support system. I think those things are critical. Chris: I like that and I think the team building right Creating opportunity for them to connect and build relationships so that the team within the team has a connection, feels like they've got each other's back will end up being a better product for the customer Because they're taking pride in what they're doing and want to help each other be successful. Pete: You have to believe your own BS. You know what I mean. Chris: If not, because part of the service that. Pete: You have to believe your own BS. You know what I mean. If not, because part of the service that we try to teach is genuine service. So you have to have a genuine. Like I tell people, don't learn my spiel, but learn in your words. Say the same things in your words, because I don't want it to sound rehearsed True. But, it's a tough thing, as we grow so fast, to really you go through six months at a time in the blink of an eye. So it's something that we have to be more intentional on and keep building that Right. Chris: There's so many things going on right. That's a good point as an entrepreneur, just how you juggle and manage because you're trying to keep the business going, trying to grow the business, you're trying to maintain relationships up, down and all around. But you got to. I think it comes back to systems and processes. Right To say we're going to have quarterly team meetings or whatever they're going to be, so that there's a scheduled cadence to doing the things that help create the connectivity. Pete: Yep, that's part of growth. I think it's just taking stuff off your plate, putting in the right, because on top of all the business stuff there's also life. Yeah, you have a family, yeah, exactly I know so you think about. Chris: you were a great education at U of H and the entrepreneurial program. So what? What type of advice, being where you are now looking back, for someone out there that maybe is thinking about starting on their own, like you did years ago? What are some of the one, two, three things you might say as advice to say, if you're thinking about starting your own business, regardless of the industry? Pete: What's some something you might say as advice to say if you're thinking about starting your own business, regardless of the industry, what's something you could pass on to them? I think you have to have. You have to go in it with open eyes and understand that it's going to be. You can have the best intentions and you can do everything fault, but it's always your problem. So I think that's having you have to know that going in and thank God, now there's industry. You hear, hey, I started a business, sold it in five months for X amount. That's awesome, but that's not everybody. So be ready to. If you're not planning on keeping it, don't start it. Be ready for that. So I buy a couple of little commercial real estate properties. Are you going to flip it? If I'm not willing to keep it for 30 years, I'm not going to buy it today. Now will I flip it? Probably. But you have to go in with the mentality If you're not going to keep it, don't start it. And, like the good things, set yourself small goals, because you always have the big picture as the entrepreneur, as the founder, sure, but set small goals and celebrate the wins, right, you know. So I, I think it, once you achieve those smaller goals, those milestones. You really need to celebrate them for you, because we have, I tell people, entrepreneurship is almost a disease, right, right, and you don't recommend it to everybody. It's not for everybody. 100 so, and there's nothing wrong with not being an entrepreneur now. Now there's a culture that everybody needs to be their own boss and side hustles, and not for everybody. There's nothing. There's entrepreneurship that they teach us in the entrepreneurship program also how to grow within the company Interesting, how to add value to your boss. Chris: I hadn't heard of that. That's great yeah. Pete: So that's a very important part, because not everybody, it's not for everybody. Chris: It's not, and that's a great point, right. Everyone thinks, and because of those, the the things that get the headlines right someone starts a company, a year later, they sold it to whatever it sounds easy, we're making it. Pete: You know the media romanticism about it yeah, but it's not for everyone. Chris: I mean, we're all different, we all have different strengths and and we said this earlier, you know in the podcast, in this episode it's not easy, it's not for everybody. So just because you can't do it, that make you a bad person. You have, you could have a very significant role within a company, even if you haven't found it uh, I would think that's a great thing on your resume is failure right, right that that that I mean. Pete: that's how we all learn, absolutely, you know. And another another thing is people who have done things. If you can move one rock out of the path of somebody coming behind, then that's also our responsibility. Right Now, will they listen or not? Because I remember 23,. You knew everything, you know, yeah, so, but you have to be available to those people as well available to those people as well. Chris: Okay, so you said you moved from Columbia. Yes, sir, what's your favorite thing about Houston or Texas? Is there an event you like to go to every year, or sporting event or cultural event? Pete: About Houston is that there's a lot of live events. So I like live stuff, I like sports, I love the Astros. Go to the Texas games, rocket games, I like that. If you wake up at, if you you're one in the morning and you want to eat food from any country in the world, you pick it. There's a restaurant here in houston for it. Yeah, I like that. There's a good international community. I mean, there's bad things, always the traffic and everything, but that comes with it, right. So I like the opportunity that it's there, because then you can learn from different things, you can apply what you learn in other areas. So that's what I like about it Just the broadness of it, the availability of whatever you can think of and the access to different people that come from maybe not a similar background, but we all have mothers, grandmas, fathers, brothers, sisters. So you end up realizing that the world is not as big as you think and we're not as different as we think. Chris: Very true, very true. So I'm 90-something episodes into this podcast. I've asked every guest this question at the end. When I ask you even though it just seems a little bit odd, but I ask my guests, what do you prefer? Tex-mex or barbecue? You with the fajita restaurant? It seems like it's not a fair question. Pete: It's not a fair question, because my favorite type of food is chinese. Okay, so no. But houston has a great uh text mix scene and and I think it's just awesome when people come here you've seen the interviews they come in for barbecue text mix, but there's so much more in between, sure, you know? I think it's. It's such a beautiful, it's a good city. Chris: It's a great city to be a foodie. Yeah, because it's also where it's such a beautiful part of town. Pete: It's a great city to be a foodie. Yeah, because it's also where it's at on the map right. You can access different produce products that match other cuisines, so then you end up getting more authentic food, like you would if you went there Now. It's not equal apples to apples, but it's a very good representation of whatever that culture is trying to portray. Chris: So last question you've mentioned is it's it's not being an easy being an entrepreneur, run your business. So what do you do to kind of for yourself, to kind of recharge, relax? Pete: Spend a lot of time in the afternoons with the family. The funny thing is cooking, which we do all day. But you know, go home, clip a branch of the rosemary plant, make some steaks for the family, spend time, go to live events. I think recharging that social battery. Being in crowds, I like that environment. That's part kind of the restaurant kind of gives you that. But you know, going to different live events, concerts and sporting events is really kind of the way to charge up and it allows you to be in the moment and, even though you never clock out as an entrepreneur, it allows you to kind of disengage for just enough. Chris: Yeah, you know until you're driving out. Nearly right, yeah, so well, pete, this has been great really your stories amazing and inspiring. Like I said, I've been a big fan of the food for four years before you, so getting to meet you has been a real pleasure. Pete: Thank you, thanks for coming on. Thank you, brother. Chris: Appreciate it.T Special Guest: Pete Mora.

Owning a property management company can be expensive, risky, and stressful. Property management business owners often surround themselves with the wrong team members. Today, property management growth experts Jason and Sarah Hull sit down with Pete Neubig with VPM to talk about building effective and efficient property management teams. You'll Learn [01:58] Having a business in “Chaos Mode” [09:02] The importance of core values [14:45] How VAs help your business thrive [23:18] Accountability, KPIs, and training [30:06] Creating company culture with VAs [37:07] Getting the right people in the right roles [41:30] VAs for property management companies Tweetables “When you're in high growth, you seem to be in chaos mode, and when you're in chaos mode, you don't make any money.” “When you're not proactive in your business and you're reactive, you're losing trust and churn goes up.” “If you don't have your org structure correct, it doesn't matter how many whistles and bells you have.” “I think every business owner needs to build the business around themselves.” Resources DoorGrow and Scale Mastermind DoorGrow Academy DoorGrow on YouTube DoorGrowClub DoorGrowLive TalkRoute Referral Link Transcript [00:00:00] Pete: If you don't have your org structure correct, it doesn't matter how many whistles and bells you have. If your org structure is not correct, It all goes to hell in a handbasket.  [00:00:09] Jason: All right. Welcome DoorGrow property managers to the DoorGrow show. If you are a property management entrepreneur that wants to add doors, make a difference, increase revenue, help others impact lives, and you are interested in growing in business and life, and you're open to doing things a bit differently, then you are a DoorGrow property manager. [00:00:28] DoorGrow property managers love the opportunities, daily variety, unique challenges, and freedom that property management brings. Many in real estate think you're crazy for doing it. You think they're crazy for not, because you realize that property management is the ultimate, high trust gateway to real estate deals, relationships, and residual income. At DoorGrow, we are on a mission to transform property management, business owners, and their businesses. We want to transform the industry, eliminate the BS, build awareness, change perception, expand the market, and help the best property management entrepreneurs win. [00:01:03] I'm your host, property management expert, Jason Hull, the founder and CEO of DoorGrow, along with Sarah Hull, my wife. Co-owner of DoorGrow and the COO of DoorGrow. Now let's get into the show.  [00:01:16] So our guest today, we've got Pete Neubig back on the show with VPM Solutions. Welcome Pete.  [00:01:23] Pete: Welcome Jason, sarah. Thanks for having me.  [00:01:25] Jason: Yeah, good to have you. So now Pete, you were an operator of a property management company.  [00:01:31] Pete: That's correct.  [00:01:32] Jason: With Steve Rosenberg and you really helped to dial in the operations there and build that up. And now you're helping people do this in their property management business with your VA company. So we're going to be chatting about today is the number one way to increase productivity and profitability, so this should be interesting. So Pete, what is the number one way to increase productivity and profitability? Let's get into the subject.  [00:01:57] Pete: Sure. So before I jump right in, I'll talk about just a little brief history of Empire Industries, which was the company that we owned. So, we came from the investor side, Steve and I, we partnered up, we owned about 31 homes. Bought too many, didn't know how to manage it. [00:02:12] We love the idea of buying the deal. We hated the idea of managing it. So we went out looking for management firms and then realized we felt we could build a better mousetrap, which we ended up doing. Our original vision, I know you talk a lot about vision in your coaching, our original vision was we were going to own 500 homes and manage them ourselves, and within a year, that vision went to crap and we ended up managing 60 homes and I owned 37 of them. I'm like, "Steve, how are we managing these other homes?" And we were third-party managing all of a sudden. Because he felt that everybody needed help. And so we started third-party managing. So that's how we got into it, and we ended up building a better mousetrap and we created a third-party management firm and we took it from those 31 doors that we had all class D minus stuff, which is a whole other podcast. And I think you've actually listened to one of yours recently about something like that. So we ended up taking it to about 980 single-family homes and nothing more than four units in Texas, single families, one to four units and we went to three markets. We were in Houston, Dallas, and Fort Worth. And what happened was with us, our vision was no longer aligned. Steve wanted to take the property manager firm national. I wanted to literally just stay in Houston and get like 1500 homes. And so that fractured the partnership to the point where we decided to sell the business. Long story short, I couldn't afford to buy him out and he didn't want to buy me out. So we ended up selling to My Management, took a job with them for a couple of years, and realized I was no longer employable and that's when I started VPM Solutions. So that's the short version of it, but we were in chaos mode for many years at Empire. [00:03:43] When you're in high growth, I don't know if you've seen this with your clients, but we were in high growth and when you're in high growth, you seem to be in chaos mode. And when you're in chaos mode, you don't make any money. We didn't anyway. And so what we had found was our number one challenge was payroll costs. So the number one challenge that I've seen, and I've talked to a lot of people across the country, your number one challenge is either growth or payroll costs. The interesting thing about property management because it's a service-based industry and because it's so service-based that you almost have to stress your team out to make money. [00:04:16] Right? So you're on this kind of seesaw where I don't really have that many doors, but I need the people. But so the salary cost is so high that there's no money for me. As I grow the doors. Right. Now I don't hire anybody, but now I'm making money, but my team is now completely stressed out. They work in weekends, they work at nights, they're taking phone calls. They don't give the great customer service. And so payroll costs, what we saw was with us, our payroll costs are about 56%. Which is really high. A business should be around 30 to 36% is what I was taught by my business coach. I don't know if you've seen anything different in the service space, but that's what I've heard. So I had to figure out how to get my payroll cost down from 56% all the way down to about 30%. And I'll tell you how I did it with virtual assistants, so I'll let the cat out of the bag, right? We got it down to about 34%. So from 56 to 34%, and every percentage that you save in payroll costs is a dollar in your pocket. [00:05:11] But then you'd be like, "Well, Pete, if you have less people or, you have less payroll, typically you have less people. And if you have less people, your team is stressed," and I get all that. But let's talk a little bit about what happens when you have a stressed team. Okay. So when you have a stressed team, the little things go out the window, right? [00:05:27] All of a sudden, you're not making those calls to get those online reviews. All of a sudden, you're not making the calls and your communication goes downhill. And when a landlord owner or an owner client calls you to find out what's going on with the problem, whether it's maintenance, lease, you know, lease renewal, whatever it is, they feel like they're managing you. [00:05:43] So when you're not proactive in your business and you're reactive, you're losing trust and churn goes up. At Empire, our churn was around 34%, which is insane, right? The average churn in the business, my understanding is like 18 to 20%. Right.  [00:06:00] Jason: And that's his annual churn.  [00:06:02] Pete: Yeah. So it's high, right? [00:06:03] 34%. And I can tell you that the majority of it was people were unhappy with our service. Yeah. Right. So it wasn't good churn, right? Because you have good, neutral, bad, however you want to define it. We had mainly bad churn. People weren't selling houses and like, "all right, we're out of here, we sold." No, they were taking them because they were not getting the love, the communication really from us. So by having these payroll costs so high, I couldn't afford it. I couldn't afford people. So what happened, especially after 2020 with that pandemic is that the cost of hiring people got incredibly high, right? [00:06:34] So I call them low-level, low enjoyment jobs. Let's take a maintenance coordinator, for example, right? That's the number one job that is posted on VPM solutions today. Is the maintenance coordinator. So that's the first thing people look for typically. Well, a maintenance coordinator in Houston, Texas, back in 2018, 2019, was about a $35,000 a year job. Well, after 2020, people that want to do a job, they want like about $50-55,000, right? And the company just can't absorb that. They can't afford to hire people. On top of that, the type of people that we were getting were GEDs or high school, diplomas, no longer college-educated people wanted that job. Most of those people have challenges in their life and they bring them into your business. So, this all came to a head. I had a lady named Sharon, and Sharon was my front office coordinator. This is back in a day when we had these things called offices and office space. [00:07:22] Jason: Yeah.  [00:07:23] Pete: So I remember those days. In 2019 and before so people would walk into our office, drop off, rant or whatever. Right. And Sharon was this, she was like this angry lady. And I'm like this tells you what my hiring process was back then it was not very good. And some of the things that you teach, I'm like, man, I wish I would have known that back in 18, 17 and 19. So she's the wrong person. She was the wrong person and she was the wrong fit. But in my mind, I'm like, "Well, she's mean." I'm like, "She'd be great for a maintenance coordinator, right? She can tell people no all the time." So I decided instead of firing her, I decided to promote her, right. Which was a terrible mistake. So I promote Sharon to maintenance coordinator. Now, unfortunately for Sharon, she was my maintenance coordinator. I was actually managing properties back then at the time. And so just for that, she probably should've got some hazard pay. So I get that. I'm not the easiest guy to work for, especially when I'm managing properties. So Sharon comes and within one week, Now I gave Sharon a raise, so I moved her from front office to the maintenance coordinator. She was making about $35, I gave her like $ 40,000. [00:08:20] She's making what I think is decent money. That's not great money. I get that, but it was good money at the time. Within one week, she comes to my office. She tells me she needs more money. I'm already just scraping by as the business. Just scraping by, single-digit profit margin. So that's when I realized that I could eliminate her position. I can hire three people that are overseas for the same cost as one Sharon. But here's the big difference. Those three people, they're obviously bilingual, right? And here I'm in Houston and Dallas and Fort Worth at the time, Spanish is like, a lot of our tenants, about a third of them didn't really speak English. A lot of our vendors, Spanish was their first language. So I can get bilingual people, I can get college-educated people, I can get people that are ready or knowing that they want to work from home. And here's the most important thing though. I can get people that were not just a J-O-B to them, but a career and they were excited about the opportunity to work with us and for us. And so the attitude and all of a sudden I can find people that align with our core values.  [00:09:18] Jason: Yeah. That's significant to be able to find people that align with your core values. Yup.  [00:09:22] Pete: A hundred percent. But now I have three people doing the work. So now what happened is I had a little hesitation from my property managers, right? Because property managers are designed to be taskers. Right. So I had to take my property managers and I had to lift them up. And we actually changed the name. We said, you're no longer considered a property manager. You're a client relations specialist. Or an asset manager. I like asset manager better, but that was one of the fights I lost with Rosenberg. If anybody knows Steve, he's 6'4 full of muscles. So we arm wrestled and I lost on that one. We call them client relations specialists.  [00:09:55] Jason: But you wanted to call them what? Asset managers?  [00:09:58] Pete: Asset managers. I think an asset manager just has a little bit more cachet. And if you really think about it, right? How many clients do you have, like you're listening, that call you up and tell you how to manage their property, even though you're the expert? I felt the property manager, I call them gophers. I felt the property manager, they had to take these calls from these owners all the time and say, "Hey, go to my property, make sure the water in the pool is being filled up. Go to my property. Gas man's going to come there. I want to know about this $12 expense." meaningless and small conversations. You would never have those conversations with the guy managing your money, right? Imagine calling your Smith Barney guy and say, "I don't like the way you made this trade. Like you should make this trade different." no, you just let the guy do his thing. So how do you let us do our thing? Well, words are powerful and property manager to me has lost its luster. And it just reminds me of a gopher.  [00:10:45] Jason: I think also the phrase property manager in the property management space has become like saying " miscellaneous role" and that like it doesn't have meaning a lot of times there runs into this a lot with coaching our clients. [00:10:58] Sarah: Like, "what does your property manager do?" And they're like, "they pretty much do everything." "Okay..."  [00:11:02] Pete: And that's a problem And the reason why they do everything is because they can't afford more people because the margin is so slim. Right, so we got to the point where our property managers got elevated, made them client relations specialists. And what does that mean? It means that they had to learn a new skill. They had to manage by reports. They had to manage people because now all of the low level property management tasks were being done by my team in the virtual assistant world. And when I mean everything, but by the time Empire was done now, granted, we're almost a thousand units. But at that point we can hire some people. Everybody had one hat, which was a beautiful thing because now you can have your job description really set. You can have your KPIs really set. You can have your DISC profile really set. And you know who to hire. [00:11:43] And they have one or two numbers and they end up doing a much better job than the manager who's doing all of it. So over the course of your growth, you have to change your infrastructure, right? You go from portfolio to hybrid, hybrid to departmental to pod and all that good stuff. I got to departmental, we never got to pod and then we sold. That was probably going to be the next move for us. [00:12:05] If you don't have your org structure correct, it doesn't matter how many whistles and bells you have. I could have property meld and I can have Zapier and I can have lead simple. I can have all these things. I can have a bunch of VAs, but if your org structure is not correct, It all goes to hell in a handbasket, just even quicker, right? [00:12:22] Cause now you have all this stuff happening even faster and it just gets crazier. And so with us, what we did is we had the structure, right? So now the managers, they're not taking those first phone calls. what was happening, Jason, is that when people would call, right? An owner client would call, my manager would pick up the phone. And as they're talking to this person, they're literally online and doing 14 tasks, responding to 18 emails. And people can hear that, they can see that and they can feel it over the phone. And so what do they do? Well, you don't really have enough time for me, I'm going to go take my property elsewhere. Or if you mess up, you know what, not only do you not have time for me, you mess up, right? So now what we do is we have everything happening on a low level.  [00:13:01] My managers told me, and I've talked to other managers since, my managers told me that maintenance took 80% of their time, right? And so I've heard that time and time again. So that was the first thing. So everybody always asks " okay, if I do hire a virtual assistant what's the first thing I should hire?" And the answer is, it depends for me. I knew my churn rate was directly related to the way we handle me.. I knew it. I didn't have to have a consultant come in and tell me that, right? [00:13:27] I just getting beat up every day by it. So I ended up hiring I was going to hire one remote team member, I ended up hiring four, right? And I trained them, figuring that somebody is going to drop off, but I wanted to train them all together. Now I did the training. Training is like literally the most tedious thing ever. And nobody wants to train. Everybody wants to hire somebody that they know exactly how to do it and they know exactly how to do it your way. It doesn't work that way. You have to take one step back to two steps forward. What people don't realize is the time you spend training your people, you get back in perpetuity forever. Because if you train your people correctly and you have good core values and you have a great culture, they ain't going to leave, right? People are so worried. I'm going to transfer, isn't going to leave. Yeah. If you're running a crappy company. Right. If you're running a crappy company and yeah, I'd be freaking worried too. [00:14:11] Right. Yeah. Make sure you're running a great company. You train the people. And then here's the great thing. As people moved on, whether they moved on and got another job or they moved on because I promoted them, guess who did the training for the next batch? My team did the training for the next batch. By the way, my churn rate for my remote team was way less than my churn rate in my US team. Right. Right. Incredibly different.  [00:14:32] Jason: Churn rate of retaining clients, of team members?  [00:14:36] Pete: Team members. Retaining team members. Churn rate of clients and you have churn rate of team members, right?  [00:14:39] Jason: Yeah. Their loyalty is just a lot stronger because they don't get these kinds of opportunities as often.  [00:14:45] Pete: Correct. Correct. So once my maintenance team was on board, now my manager, I literally saved with the narrow minds 80%, but here's the funny thing, right? So as I'm training. I had a director of operations. Her name was Margo and I still talk to Margo today. I love Margo. She would come to my office every day for 90 days. She came to my office with her cup of coffee every morning and said, "I don't think these VAs are going to work. I don't think these virtual assistants are going to work." Okay. Because when I was training right now, I did the training, not Margo. [00:15:12] I was training them, but when I was training them. What we had to do is every work order had to go to the property manager, then to the virtual assistant, then the virtual assistant would talk to the resident, the owner, bring it back to the property manager because they were getting, they were training, right? So they had to learn what to do in each situation, which caused my property managers more time, right? So that 80 percent went to 90 percent or even a hundred percent or 110. Now they're working extra hours. So they hated it. On day 91 I don't know if I'm allowed to say this, but I shit you not, day 91, she comes into my office and she has our same cup of coffee and I'm getting ready to listen to the spiel and she goes, " do we have maintenance anymore?" [00:15:47] Yeah. And I laughed and it took 90 days, but I got it. Yeah. The point where, so all of the work orders were being done by the remote team that nothing was getting escalated anymore. Only very little things right? So my managers do say, what do they do? Well, they take on all the escalations. Now imagine. What brain power, right? My team in United States, they were the ones that were the experts, right? So, but imagine if they only are dealing with high level escalations, not all the other little, because how many times did I have all these little things get done, but then we missed the big thing. [00:16:18] And then all of a sudden what happens is I call them taps, two by fours, and mack trucks, right? A tap is basically a maintenance request. That's going unanswered for, let's call it 15 days. Okay. That's a tap. The two by four is now the resident bypass you calls the owner. Now the owner knows that it hadn't happened or the resident blasts you on social. And then the mack truck is the lawsuit that gets across your desk, the tech, the report the complaint to the the real estate commission. Right. Or you're just getting, or you lose a client, right? Yeah. Those are two of those. So my team was so busy that they were missing the taps that they were becoming two by fours. And these are called fires. All right. And all we're doing is trying to deal with this fire. And then of course, every once in a while you get a mack truck, right? And it's what the heck? So now that my managers are not doing the day to day stuff, they're able to be proactive. So they're looking at reports. They're literally looking for taps. And now they're solving those taps. What that means is now the owner clients not calling you to find out what's going on. You're calling them, you're reaching out to them, you're letting them know, or you're taking care of it before they even, before it even becomes an issue. [00:17:18] And so by, by having your high dollar people that are licensed and they have experience by allowing them to not do the low level, low enjoyment stuff they actually became not only do they take all the escalation, but they actually became internal salespeople. All of a sudden, and this is stuff that we didn't anticipate, all of a sudden, though, like my company's name was empire property management in realty. That 90 percent of my customers had no idea that we could buy and sell homes for them. We're called realty and no idea. But once I got my property managers to be client relations specialist, guess what's happening. All of a sudden people are going to buy houses and they're buying them through us. [00:17:54] All of a sudden people want to sell. They want to sell through us. So all of a sudden our revenue goes up, right? Then all of a sudden they're like, who do investors hang out with? They hang out with other investors, right? You're the, you're like the five most, what is it? It's the old saying that you hear you're the average of the five people you hang out the most. All of a sudden they're getting, we're getting referrals. We never got referrals. So now we're getting a bunch of sales. We're getting a bunch of referrals. We're getting people to buy stuff where the agent, right? And when you're the agent, you get, you build that, that relationship. [00:18:18] And so now all of a sudden our churn rate dips down to, I think it was 22 percent from 34%. Right. So the interesting thing is I told you when I first started, right, I went from 54 percent to 34 percent in payroll costs. My payroll actually stayed the same. It was the churn rate that went down, increased my revenue. [00:18:36] It was the other clients, right? And retaining people and getting more clients. That's what, that's where the difference was. And now my managers. We're incredibly happy. They're no longer working nights and weekends. They're no longer stressed. Right now. And so now they are they're having the best life ever. [00:18:53] And my VA team, my remote team, they're making more money than they've ever made before. And it was easy to, and then they all had KPIs and they were all like. People want to inherently do a good job. They do. Right. And so, but they don't know how to do a good job unless you tell them what that looks like. And that's the job description. And they want to report card and that's KPIs. And my team down there, we had them in Mexico cause they're the Spanish speaking. But what happened was again, another thing that we didn't realize was not only the team do the work, they hit the KPIs, they exceeded the KPIs and we create a bonus structure around the KPIs. [00:19:26] So if you hit the KPI, you got a firm handshake. Thank you. Right. But if you exceeded the KPI, you got a bonus. And if you were part of a team, everybody in team added the KPI or you didn't get the bonus. And what I like about with the virtual team is the bonus was a hundred dollars a month. If you hit a certain level, you got a hundred dollars for us wasn't a lot of money, a hundred dollars to somebody in the US. Like literally would get mad at me. That's a little, that's too little of a bonus. It doesn't even fill up my car. Right. And they throw it at you. Somebody in the Philippines or Mexico or Costa Rica it's an extra couple of days of work per month. [00:19:58] So they were really appreciative of that, of the opportunity to make more money. What happened was everybody started exceeding their KPIs to the point where I couldn't make the KPI any more difficult. Like it just is what it is. And they were just doing it. And then here's the magic. [00:20:11] What happened next? was they ended up updating or changing the process. So my deal as the business owner was, I am the policy maker, I make the policy, but you own the process. And when somebody comes in and says, "Hey, I changed the process." And I use this example a lot. I had Jessica who was running all my lease renewals. [00:20:30] So we had about a thousand units and I have one person doing all lease renewals, inspections and lease renewals. Our policy was that you could not do a lease renewal unless an inspection was done, an annual inspection was complete. And we used to start the process 60 days out. Jessica moved it to 90 days out. And when I was talking to them, I'm like, Jessica, I'm just curious what made you, and I don't, I try not to ask why questions because why questions put people blame, excuse, denial below the line and they get defensive. I asked, what made you decide to move it from 60 to 90 days? And she goes, "well, with 90 days, I can do X and Y. Like I can get to the owners faster. I know if the, if the residents do it" and she laid it all out. I'm like, amazing. She was doing a better job than I could have done because that's what her core focus was. Yeah. She was just on that. So then what people will say to me is Pete. [00:21:13] Okay. Well, how do you know she's just not doing the lease renewals and not the inspections because she wants to hit our number. Right. That's the first question I get all the time. And I say, "well, we hire people based on our core values. And one of our core values was integrity. And so if you hire people with integrity, they're not going to do the loop around." [00:21:30] I was able to run reports very quickly that determined all the lease renewals and if they had an inspection done so I've been reporting it. It was very simple to, to make sure that I was, I hold them accountable. Yeah, that's another core value of that we had is we hope people get, we run our business by numbers. [00:21:48] We hold people accountable. And so that's so, so because we did all of that, we were able to solve our challenge of no profit or single digit profit margin to, double digit and eventually get to about that 20 percent profit margin, even though we, even while we were still investing in a lot of money, growing the business. [00:22:07] Jason: Yeah, so we've, I love all the stuff you've been talking about. I think we've had some phenomenal results getting clients to improve their profit margin. And we've got clients easily getting up to 40%. Sarah ran her business over 60 and I think the three biggest profit levers are building a really solid process system, a really solid people system, and a really solid planning system and the planning system we call DoorGrow OS. [00:22:33] But that was really where we started to motivate the team to think in terms of outcomes and get them to think more strategically like business owners. And so that strategic work is what moves businesses forward. That's where they're innovating. That's where they're improving a process and so those kind of goals, if we give a team member an outcome and we say, "figure out however you can best do this, within our values with integrity. Figure out a better way," then I'm not concerned about micromanaging them. I we're less involved in managing the team. They're now managing themselves because they're trying to achieve the outcome. And a lot of team members in a lot of business don't even have job descriptions. So they don't even know what outcomes they're expecting. [00:23:15] Pete: If you're not sure what they're supposed to do. How do they know what they're supposed to do?  [00:23:18] Jason: Right. And if you ask anyone listening to this, if you ask your team members. This would be a curious and interesting experience for you or experiment. Ask your team members, "what are the outcomes that you think are most important for your role?" and compare that with what you think they are. I think you might be surprised. These should be agreed upon and defined, right? That should be in the job descriptions. Pete, I really appreciate all your transparency and sharing, because a lot of times everybody wants to, especially with like coaches in the industry, I see a lot of people coaching mentoring, but you don't get to see how the sausage is made and you don't really hear the challenges they have, but they might be really charismatic. They might really be good at speaking, but there's a lot of stuff going on behind the scenes. And then what a lot of coaches in the industry do is they try to get people to build the business the way they did, which may not even be working. And so I think what's important, I think every business owner needs to build the business around themselves. It needs to be built to allow them the maximum level of fulfillment and freedom and contribution and support in their own business and that fifth reason of safety and certainty. [00:24:25] And that means every business is going to be unique because every business owner is unique. If you started a property management business right now, it would be run very differently than some others, because you're very operationally minded and you would build your team very differently than somebody that's very visionary sales oriented, right? [00:24:41] And I think it's important to get the right team built around you. And a lot of times I think the foundational challenges, a lot of business owners aren't clear on themselves. And then they start building a team and they're miserable. They have an entire team and they're still miserable. They've built the wrong team. [00:24:55] Pete: Well, I think every new business owner does that, right? They don't feel like they deserve good people. So they self sabotage sometimes. Right.  [00:25:04] Jason: They don't believe the good people are out there. A lot of times they just don't even believe there's good people. They're like, "Oh, everybody's terrible." so guess what they attract? Right. And what's surprising the truth is just like you had mentioned, when you find good people, they will exceed you doing that role. Especially if it's one of your minus signs, it's not one of the hats you enjoy wearing, they will be better at it than you, if they enjoy doing it. A hundred percent. And that's super humbling for these early stage entrepreneurs, because they think they're the best at everything initially.  [00:25:33] Pete: There's two thoughts, right? So when you hire somebody, there's the whole abdication of it. And then there's a delegation and then there's the micromanagement. And so, what I find is that when people hire people in the United States, they abdicate a lot of times when they hire people that are remote, they want to micromanage for whatever reason, even though they've invested a lot more money in the person in the United States. Right. And then there's people that just, they just abdicate regardless. [00:25:58] And what I mean by abdication is, I'm a property manager. I'm doing a whole bunch of stuff myself. I hire an assistant and I just throw up on them and say, here's all the things that I'm doing. Go ahead and do it. There's no direction. There's no accountability. There's no management. [00:26:11] Right. And then they get excited. " Oh yeah, I'm a great delegator." No, you're an abdicator. You're not a delegator because you're not giving them the tools and the guidance that's needed. And then what happens is the VA or the person leaves and " well, I don't understand. I can't find any good people, so I'm just going to keep doing it myself." the first thing is when you hire somebody, you have to understand, you just can't just abdicate. You have to make time for them, especially in the first couple of months, right? They're learning you and your culture. At the end of the day, if you are the sole operator and the business owner each one of us have core values, right? We have our personal core values. Most of those are going to be embedded into the company that we built. They should be anyway. You shouldn't change your core values for your company. If I'm full of integrity, I'm not going to create a company that's not, that doesn't have a lot of integrity, right? [00:26:55] So these people are going to learn by you training them or your team training them, right? Core values always get pushed down. If you're listening to this and you do not have core values in your company, you have core values in your company they're just not yours. The team created core values. They push them up and they may or may not be the ones that you want. Right. But when you hire somebody, it's important that you spend a lot of time with them to train them properly so that they understand what they're doing. What I have found is that most jobs can be trained within two to three weeks. Especially if you're wearing one hat. The more, what I call decision points or if then else's, and the biggest one that I've found is in maintenance coordination has a lot of decision points. What if it's over the threshold? What if it's a home warranty? What if it's an emergency? What if it's cosmetic? [00:27:39] Right? You go on and on. That's why it took me 90 days. Because we had to go through every one of those scenarios and I had to train on. And it's just a little bit more in depth. My least renewable person, I was able to train her in two to three weeks. And you're right. And so by the training and by creating the KPIs and then by having a weekly meeting with structure. [00:27:57] Right. So nothing gets me more fired up than having a meeting, just to have a meeting. And then we sit there and we sit there for an hour and I literally just wasted not just my time, but everybody else's time all because we don't have any structure. So I'm a big fan of EOS. I'm sure that you have something that's very similar to a meeting structure. [00:28:15] Jason: We call it DoorGrow OS.  [00:28:16] Pete: DoorGrow OS. So DoorGrow OS. So if you're not part of DoorGrow, join DoorGrow and get on the OS. That's like number one, right? Because if you just get your meetings in order, you will see an increase in productivity just like that. So by the way, the maintenance team that I built, they always reported to me, even when I sold, until the day I sold the company. I just had a soft spot for them. I like maintenance. I know I'm weird that way, but I really did. And so they reported to me. My other team, I had other supervisors. I actually had supervisors in Mexico that were managing the other team members in Mexico. And that supervisor report to somebody in the U. S. or to report directly to me. But I still had my weekly meeting with my team every week. And we had our OS and one of the questions I asked every week, there's two questions that were always number one was always. "What can I do as the business owner to make your job easier?" I think there's a, I think there's a sphere, a circle, right? My job is to take care of my team. My team's job is to take care of the client. The client's job is to take care of the business and the business job is to take care of me. That's the circle right? So no the client is not always right. And let's do what we have to do to make sure that if we did mess up, we want to make it right. And I get all that. But how can I make my team's job easier? And that could be, I need to go talk to Sandy in accounting because she's not doing something or it means, "Hey, can you create this report for me?" I need a whatever it is. What can I do? Then the last question I asked on every meeting was what is your stress level on a scale of one to 10? And this was really important because it does two things. Number one, if somebody is a 10 plus for three weeks in a row, they are ready to punch out. Yeah. No one wants to work in a stressful environment for more than if we can see that Hey, it's summer, we're a little short staffed, you're going to be stressed for next, six to eight weeks, but there's a, but we're going to do X, Y, and Z to get out of it, I get it and people will handle stress for a short period of time. [00:30:05] The second thing is, believe it or not, sometimes people are stressed out and has nothing to do with you or your company. I know we all think it's about us and our company, but personal stuff. So one time I actually. And so if anybody's 10 plus and I want to talk to them, I do it off the meet. Like we have a one on one say, "Hey, stay on everybody else. Get off the meeting, whatever." Yeah. And I had this one lady 10 plus and I said, "Hey you're usually a two what's going on. My brother got hit by a car right now." What this does is everybody's always asking me how how can I, how can I bring my team, my remote team into our culture. This is a great way, right? Because at the end of the day, just like you, you want to give time to your owner clients and you want to build relationships, you want to build relationships with your remote team. And so by, by taking an interest in them as human beings. [00:30:52] Right. It doesn't mean you have to give them, I'm not going to, I didn't fly down and give them a whole bunch of money. I just listened and I cared that her brother was doing okay. I would ask, and it was just an emotional human thing. My team, if your team, if your remote team know that you actually do care about them. So if your remote team knows that you care about them, they're not going to leave you for a 50 cents more or a dollar more an hour. They're just not. Because most of the time, if you're paying them a fair wage. They are making more than enough money to cover their, what I call their nut, just to cover their living expenses. So they're not going to leave because the grass isn't always greener and they are freaking happy. [00:31:28] If you make your team happy by asking them, how can I help? How can I make your job easier? And letting them know that you care about them as people. That's the, that's like a number three question I get, right? Number one is how do I train them? Number two is where do I find them? [00:31:41] Number three is how do I make a part of the team? This is how you make a part of the team, right? By, by advocating and just throwing a bunch of throwing a bunch of stuff on them and letting them go. That's not how you do it. And by micromanaging, I'm saying, I want to see all the screenshots. I want you to write down everything you did from this time to this time. [00:31:57] And if you take a 15 minute break, I need you to punch out and punch in. Right. You said it earlier. You manage by results. That's what I do. Do I care if you put 40 hours a weekend? I really don't. I'll pay you for 40. But if you get if if you're available and I need you, right. So I have managed on availability first, it had to be available. [00:32:16] So we have policy. We use Slack. If I Slack you, you Slack me back within 30 minutes. If I email you, you email me back within four hours. If we have a meeting, you're on video and you're in your home office. None of this Starbucks crap, none of this on the beach crap, like you're in your home office, you're working, right? [00:32:30] So availability is number one. Then number two is KPIs. Are you meeting or exceeding your KPIs? Number three, and if I have the right KPIs, I can just look and if it's green, I know that position is doing well. And then number three is escalations. Am I getting calls from our clients or from internal members of the company saying that you're not, that you can't, that you're not doing your job or you're not getting back to them or whatever. [00:32:53] Those are the three things I need to know. I don't need to know that you're moving your mouse every 30 seconds. I could care less on that. If I got those three things, I know, and again, I know I have the right people because I hired them based on my core values or the company's core values. [00:33:06] Jason: Yeah, totally. We do a lot of the similar things at DoorGrow. Like one of my mentors would say, cadence is culture. And I really believe that the cadence of your meetings creates the culture. It really does. And this is where you're able to set the culture with your team. And we ask questions like, where are you stuck? How can we support you? We do caught being awesome. We, and I think what team members really want more than money, a lot of entrepreneurs, we like money, right? We don't hate money. And so we assume mistakenly that's the highest priority for all of our team members. Well, I'll just give them bonuses or I'll pay them more. The reality is most team members. With the exception of maybe entrepreneurs and salespeople, most everybody else on the planet would prefer once their basic needs are met, financially would prefer to be recognized rather than get a bonus. And so creating the right cadence and creating a system like DoorGrow OS allows the team to be seen and recognized for their accomplishments strategically and moving the business forward. [00:34:03] And that prioritizes that we find that if you can get those three systems in place. The planning system, that's DoorGrow OS here at DoorGrow. The people system, we've got DoorGrow Hiring, Applicant Tracking System, etc. And the process system, we've got DoorGrow Flow and some other stuff. If you have these three systems in place, these are three of the biggest profit levers you can get in place. [00:34:23] And a lot of times people try to skip those three and jump right into profitability and micromanage through just more severe actions, more severe KPIs, and trying to control more. Thinking they can squeeze more blood from the stone when if they did these three profit levers, we've got clients that are hitting amazing profit margins. [00:34:42] They don't even have KPIs because they don't even need them because they trust their team members so much and their team members are really great culture fits and really motivated. And so focus on those three profit levers first, and you're going to make a lot more money. And really what happens is you get three times the output from good team members. [00:34:59] Easily and they can be anywhere. And what's, what I love about being able to have a remote team, we've got team members all over the place. Some of the U S Canada, Mexico, one's in London now, Philippines. I'm able to hire the best. I'm able to hire the best, no matter where they are. And I'm able to also for certain roles, get, make sure it's really affordable for the business. [00:35:20] And so we're not, I'm not too particular about where they're at or what they're doing. It just needs to be a price point that we can afford. And I need a really good outcome. And if we can get that, then that's the ideal. And it's easier for me to run things remotely than if everybody were interrupting me coming into my office all day long, it's a lot quieter. [00:35:42] And I feel like everybody's able to get more done, but we're able to create that connection in our daily huddles. We check in with everybody, ask where they're stuck. We do one on ones like you were talking about. All these things to figure out where everybody are at. The one thing that we do that I think is really impactful is we have our team members do time studies, not as a punitive measure, as a way to support them and figure out how to get them additional support and help. [00:36:05] And this is where we figure out which, what are their plus and minus signs. So Adam, who's been on my team for almost, I think almost a decade now. Yeah. I'm like nine years. And he started as a content writer and he's done multiple time studies and every time he gets really honest with me, he's these are the things I don't enjoy doing anymore. [00:36:21] I'm not enjoying doing all this writing. I'm, what do you enjoy? I enjoy interacting with the clients. He now manages our entire department for websites, branding, all this. He's got a whole team under him. Whereas nobody initially would have thought, Hey, Adam is a manager, but he by default naturally became one because we just got him the support he needed. [00:36:40] And so he's been, he, and that's how we've been able to retain Adam. And the cool thing about retaining team members is they're like wine. They get better with time. Better and better. And so Adam knows lots of ins and outs in the business. He's super adaptable and versatile, and we're able to use them for billing related stuff and website stuff. [00:36:58] And there's so many things over time that he's developed and absorbed and learned. He can run significant pieces of the business for me if necessary.  [00:37:07] Pete: Well, I'll give you a funny story because, here I am teaching and telling you, oh, here's how hire people. Right? So when I first started VPM Leon, who is our onboarding guy now came over and he was with me at mind and he was with me at empire. [00:37:20] So I've known Leon and I knew he had our core values, right? And so we're like, maybe eight months in and I go to one of my business partners and I go, "Hey man, I don't think Leon's working out," and he's like, "really?" he did the, I called the Mongolian reversal, right? Because he basically takes my words and he puts them right back at me. [00:37:34] He goes "let me ask you what's his job description?" And it's crickets. So I'm like, "yeah, he don't really have a job description." He's " what's his KPIs?" I'm like, "yeah, we haven't really got to that." So he's like, "how much have you trained him?" And I'm like, "all right, enough." [00:37:45] Basically, Leon was the right guy. I just didn't know what he's supposed to do. So how did he know what he's supposed to do? So then I got serious about the job description. And then what we realized is Leon was running about two hats, maybe three hats. It's really like he, he was good at one of them. [00:38:01] So we ended up hiring another guy, Angelino, and gave that hat away. And now Leon just runs and now he is. Thriving and exceeding all of the metrics that we put in his place. And he's the happiest he's ever been. And even though, this stuff, sometimes you have to continuously, make sure that you're doing it. [00:38:20] Jason: Oh yeah. We had a conversation last night about a team member that we realized they weren't doing some things right. And Sarah put it back in my face. She's well, did you train them on this? And I was like, No, I didn't. I made a mistake in training. I thought they would understand it in my superficial explanation. [00:38:38] And yeah,  [00:38:39] Pete: it's shortcuts, right? Those three things that you put out there, the hiring and the process, it sounds so easy, right? But we know it's tedious. And there's, that's a, that's the reason. Why most entrepreneurs who are most of 'em, are visionaries, right? A lot of guys start business with visionaries. [00:38:53] They're not in the details. They don't like doing that. It's not natural, right? I need an integrator. They need a, they need an integrator. I'm guessing Sarah's the integrator. I'm the integrator. I'm guessing you're the visionary, right? So they need an integrator to, to literally do that stuff and you get, like I said you, when you do it you get it back in perpetuity, like it just, once the system is complete, it's just tweaking. It's not rebuilding, once, and and but a lot of visionaries, they skip that part because they don't like that part. [00:39:18] Yeah. I agree. It's from hire a consultant or hire the, hire somebody that, that likes that stuff.  [00:39:23] Jason: Yeah.  [00:39:24] Sarah: And I love that you just keep like, thank you for continuously driving home the point. Like you have to train people. You have to. And a lot of times what we see is we see doesn't matter your location. [00:39:34] Doesn't matter your size. Doesn't even matter what industry you're in. People hire out of pain, which makes sense, but they're in so much pain that they're like, Oh, they think as soon as they hire somebody, they're like, Oh, like I'm, it's solved. It's not solved yet because you haven't trained them.  [00:39:48] It's still your problem until they are properly trained. And it does take time. So for a period of time, when you hire somebody, your life is going to get worse. You're going to be taking on more if you want them to do a good job That is what has to happen because if you hire somebody and you're like, "here just have it like baptism by fire figure it out go ahead and do it." [00:40:09] It's not going to work out. You're going to be frustrated They're going to be frustrated and it looks bad for both people and then you guys are both frustrated at each other and you're like Why are they not working out? And this person is like I didn't even get training. I don't like you're mad at me all the time. [00:40:22] And I just I don't even know what to do, but you didn't tell me what to do. Help me.  [00:40:25] Pete: I'm not going to hire people because I just, there's no good people out there. Right. It's just, when I was telling you that story about training the maintenance team, I was trained about two hours a day on the maintenance, which is a little too much, probably an hour and a half is probably the maximum we can take. [00:40:37] But I was doing two hours. That didn't mean that my 10 hour day. was still a 10 hour day. It became a 12 hour day because I still had 10 hours of work. I had to do, I just took on more, two hours of training. And a lot of times they ask more, a lot of times it's even more than that because as you're training, what I have found, and maybe you guys see the same thing is as I'm training, I actually learned a lot more about my processes and about my company, and then I realized, oh. [00:41:00] There's no policy here. Oh, there's no field for that. Oh, that's just in my head. However, I feel that day I'm going to, I'm going to judge on that. And so I, there was a lot of work that I ended up having to do as I'm creating the, to training, oh man, this process is not exactly at all what I thought. [00:41:16] Jason: Yeah. Yeah. Cool. Well, Pete, this has been an awesome conversation. We appreciate you coming on the show. Why don't you tell everybody just a little bit about VPM solutions. Do a quick plug and how they can reach out and connect with you.  [00:41:30] Pete: Yeah. So, thanks for that. So VPM solutions is an online platform that connects property management companies with remote team members. [00:41:38] It's a direct hire, so they don't work for VPM. They work directly for you. You negotiate the hourly rate. There is no upfront cost and there's no fee to use the site. So it's all free for the company managers. The way VPM makes money is the virtual assistant. Pays 10%. So when they apply to a job, they have a breakdown of this is how much hourly rate that I'm applying for. [00:41:59] It is how much that BPM charges a platform fee. And this is how much that I'll get. We also have about 20, I think 23 free training. So, there's training on the site from fair housing to marketing, social media, to pro we have a flagship property manager, one on one courses. It's about nine 12, 12 courses, nine hours of content. [00:42:20] Wow. And it's there just to teach folks the basics of property management. No, you're not going to hire them and they're going to be able to run and be a property manager for you, but they're going to know the ins and outs of the verbiage of just the life cycle, like high level stuff. But it's our attempt to get people trained up so that when you, so that when you get them, they're not like that, at least they're crawling. [00:42:44] Right. Yeah. They have a little bit of deal, a little bit of information. And then we also have we also have some free resources that are on the side as well. Like we have I think we have 50 job descriptions with this profiles that we assume, assume assumptive this profiles. [00:43:00] We also have like org charts, like what you should, or chart should be as you grow your business. And then we also have just a list of all the vendors and resources and all the different Facebook groups and all of the conferences that are out there for profit management. [00:43:13] Matter of fact, you're actually on that site by the way, as a vendor. So, yeah. So. That's what we do. And then we also offer what we call the white glove service. It's a free service that helps you go through the hiring process. Because we, what we realized early on, it's a do it yourself platform, but what we realize is most people don't have a hiring process and no idea what to do. [00:43:34] So we guide them. Now your team your clients probably have a good hiring process, but we'll offer, like we'll offer that free white glove service to them as well, if they want to come in and just. Need a little bit of help. What should they ask before they interview? There's some red tape. [00:43:47] Like we say, you get a disc profile, and then the, we have these courses that they take, they get certifications, you can search based on those certifications. So it's really the only platform literally built for property management.  [00:43:57] Jason: Love it. Yeah. Very cool. We'll check it out. So everybody make sure you check out Pete Neubig's VPM solutions. [00:44:04] Take a look at that. And Pete, thanks for being on the show today. It's good conversation.  [00:44:08] Pete: Yeah. Thanks guys. Thanks Jason. Thanks Sarah. Appreciate you.  [00:44:11] Jason: All right. So if you are a property management entrepreneur, you're wanting to grow or add more doors or you're struggling with dealing with your team, reach out to us at DoorGrow. [00:44:19] We can help you with this. We do this all the time. We would love to support you. We have clients that are easily going from, we can help you scale anywhere from zero to a thousand plus, and anybody can do this in the next three to five years. We would love to support you, help you scale your business and help you save collapse a lot of time and not have to go through a make. [00:44:37] So many mistakes in your business. And so until next time to our mutual growth. Bye everyone. [00:44:42] you just listened to the #DoorGrowShow. We are building a community of the savviest property management entrepreneurs on the planet in the DoorGrowClub. Join your fellow DoorGrow Hackers at doorgrowclub.com. Listen, everyone is doing the same stuff. SEO, PPC, pay-per-lead content, social direct mail, and they still struggle to grow!  [00:45:09] At DoorGrow, we solve your biggest challenge: getting deals and growing your business. Find out more at doorgrow.com. Find any show notes or links from today's episode on our blog doorgrow.com, and to get notified of future events and news subscribe to our newsletter at doorgrow.com/subscribe. Until next time, take what you learn and start DoorGrow Hacking your business and your life.

Episode One Hundred and Fifty-Four: Life Hack (2017)Who Am I (2014)Billy: Boy, hacker movies always make this stuff way more exciting.Pete: Yeah, with the dramatic closeups and booming techno scores, real hacking is just... typing.Billy: Throw me a bone dude, just turn to me dramatically and say "we're in".Pete: We're in!-- The Venture Bros., "The Curse of the Haunted Problem"Logo design by: https://www.fiverr.com/ideahitsIntro voiceover by: https://www.youtube.com/kevincrockerheyjameswatchthis@gmail.com

About PetePete is currently the Head of Growth And Community for AppMap, the open source dynamic runtime code analyzer. Pete also works with early stage startups, helping them navigate the complex world of early stage new product development.Pete also fully acknowledges his profile pic is slightly out of date, but has been too lazy to update it to reflect current hair growth trends.Links:AppMap: https://appmap.io/ TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: If you asked me to rank which cloud provider has the best developer experience, I'd be hard-pressed to choose a platform that isn't Google Cloud. Their developer experience is unparalleled and, in the early stages of building something great, that translates directly into velocity. Try it yourself with the Google for Startups Cloud Program over at cloud.google.com/startup. It'll give you up to $100k a year for each of the first two years in Google Cloud credits for companies that range from bootstrapped all the way on up to Series A. Go build something, and then tell me about it. My thanks to Google Cloud for sponsoring this ridiculous podcast.Corey: Cloud native just means you've got more components or microservices than anyone (even a mythical 10x engineer) can keep track of. With OpsLevel, you can build a catalog in minutes and forget needing that mythical 10x engineer. Now, you'll have a 10x service catalog to accompany your 10x service count. Visit OpsLevel.com to learn how easy it is to build and manage your service catalog. Connect to your git provider and you're off to the races with service import, repo ownership, tech docs, and more. Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn and this is probably my favorite recurring episode slash tradition, every year. I drag Pete Cheslock on who talks with me about his experience at re:Invent. Last year, Pete, you didn't go. The year before, none of us went because it was all virtual, but it feels like we're finally getting back into the swing of things. How are you, Pete?Pete: I am doing great. It is always a pleasure. It was amazing to see other humans in person at a industry event. As weird as it sounds to say that, you know, it was great to be in Vegas [laugh], it was mostly great, just because there were other humans there too that I wanted to see.Corey: Because this is going to confuse folks who haven't been following our various adventures, these days, you are the Head of Growth and Community at AppMap. But you and I have been talking for years and you did a stint working at The Duckbill Group here with us as a cloud economist. Ah, I miss those days. It was fun working with you and being able to bother you every day as opposed to just on special occasions like this.Pete: Yeah, I know. I got to slide into your Slack DMs in addition, and then when I didn't get a response, I would slide into your Twitter DMs. It worked out perfectly. So yeah, it's been a wild ride. I mean, I took an interlude from my startup journey by continually working at tech startups.And yeah, I got to join onboard the Duckbill and have, you know, a really wonderful time cutting bills and diving into all of the amazing parts of people's Amazon usage. But I am also equally broken in my brain, and continually said to myself, “Maybe I'll do another startup.” [laugh].Corey: Right. And it turns out that we're not a startup. Everyone likes to think we are. It's like, oh, okay—like Amazon, for example, has us historically in their startup division as far as how they—the buckets as they put different accounts into. And if you look at us through that lens, it's yeah, we're a specific kind of startups, specifically a failing startup—or failed—because to us growth is maybe we'll hire one or two people next year, as opposed to, “Oh, yeah, we're going to TEDx this place.” No, yeah, we're building a lifestyle business very much by design.Pete: I'd be very curious how many account managers actually Duckbill has kind of churned through because usually, you get to keep your account manager if you're growing at a pretty incredible clip. And it's kind of a bellwether for, like, how fast are we—are we growing so fast that we have kept our account manager for multiple years?Corey: Your timing is apt. We're a six-year-old company and I just met our fourth account manager last week.Pete: [laugh].Corey: No it's, honestly, what happens with AWS account managers is the only way you get to keep them is if your spend trajectory on AWS matches their career trajectory inside of AWS. Because if you outpace them, they'll give you to someone that they view as being more senior, whereas if they outpace you, they're going to stop dealing with the small accounts and move on to the bigger ones. Honestly, at this point, I've mostly stopped dealing with my account managers. I had one that was just spectacular. It was sad to see him get promoted; good for him.But I get tired of trying to explain the lunacy that is me to someone on the AWS side every year. It just doesn't make sense because my accounts are super weird and when they try and suggest the usual things that work for 99.995% of AWS customers and things they care about, it falls to custard when it comes to me specifically. And that's not on them; it's because I'm weird and broken.Pete: I'm remembering now one of the best account managers that I ever worked with at a startup, years and years ago. She was with us for a couple of years, pretty solidly. And then, you know, because careers are long and jobs are short, when I was at The Duckbill Group again, doing work, turns out she was the account manager on this other thing, you know? Which, like, looking at the company she was account manager for was like 500x [laugh] my previous company, so I was like, “Oh, yeah. You're clearly moving up in the world because my company did not 500x.” So, sometimes you got to chase the ones who are.Corey: So, let's talk about re:Invent. This felt like the first re:Invent post-pandemic. And let's be clear, I wound up getting Covid by the end, so I don't recommend that to everyone. But let's be clear, this was not a re:Invent were anyone officially accepted that Covid existed. I was one of the only people wearing masks to most of the events I was at. Great load of good that did me.But it was big. It was the usual sixty-some-odd-thousand people that had been in previous years, as opposed to the hard cap of 30 or so that they had last year so it felt smaller and more accessible. Nope. Right back to bizarre numbers of people. But fewer sponsors than most years, so it felt like their budget was severely constrained. And they were trying to have not as many sponsors, but still an overwhelming crush of attendees. It felt odd, but definitely very large scale.Pete: Yeah, I can echo that a hundred percent. I'm sure we've talked about this in previous ones, but I've had the pleasure—well, I don't know, some might call it not a pleasure, but it's been a pleasure to watch re:Invent grow over so many years. I went to the first re:Invent. A company I was at actually sponsored it. And remembering back to that first re:Invent, it was kind of quaint by comparison.There were 4000 people at the first re:Invent, which again, it's a big conference, especially when a lot of the conferences that I think I was really attending at the time were like, you know, 600, 1000, maybe tops. To go to a 4000-person event in Vegas especially, it's again, in the same Expo Hall it's been since that first one, it still felt big. But every person stayed in the Venetian. Pretty much everyone was in the same hotel, all of the attendees that year. All the talks were there.There was, you know, a lot [laugh]—I mean, a lot less of everything that was there. And so, watching it grow over time, not only as a sponsor because I've actually been—kind of worked re:Invent as a, like, a booth person for many of these years for multiple different sponsors and had to coordinate that aspect of it, but then also a couple of times just being more, like, attendee, right, just someone who could go and kind of consume the content. This year was more on the side of being more of an attendee where I got to just kind of experience the Expo Hall. You know, I actually spent a lot of time in the Expo Hall because a big part of why I was there was—Corey: To get t-shirts.Pete: Yeah, we'll get to—I was running low on not only t-shirts but socks. My socks were really worse for wear the last few years. I had to, like, re-up that, right [laugh]?Corey: Yeah, you look around. It's like, “Well, none of you people have, like, logoed pants? What's the deal here? Like, I have to actually buy those myself. I don't—I'm not here to spend money.”Pete: Yeah, I know. So. And so yeah, this year, it felt—it was like Covid wasn't a thing. It wasn't in anyone's mind. Just walking around—Vegas in general, obviously, it's kind of in its own little bubble, but, you know, I've been to other events this year that were much more controlled and had a lot more cautious attendees and this was definitely not like that at all. It felt very much, like the last one I was at. The last one I was at was 2019 and it was a big huge event with probably 50,000-plus people. And this one felt like to me at least, attendee-wise, it definitely felt bigger than that one in a lot of ways.Corey: I think that when all is said and done, it was a good event, but it wasn't necessarily what a lot of folks were expecting. What was your take on the content and how the week played out?Pete: Yeah, so I do, in many ways, kind of miss [laugh] the event of yore that was a little bit more of a targeted, focused event. And I understand that it will never be that kind of event anymore. Maybe they start splitting it off to be, you know, there's—just felt much more like a builder event in previous years. The content in the keynotes, you know, the big keynotes and things like that would be far more, these big, iterative improvements to the cloud. That's something that always felt kind of amazing to see. I mean, for years and years, it was like, “Who's ready for another re:Invent price drop?” Right? It was all about, like, what's the next big price drop going to be?Corey: Was it though because I never was approaching with an eye toward, “Oh, great. What are they going to cut prices on now?” That feels like the least interesting things that ever came out of re:Invent, at least for me. It's, what are they doing architecturally that lets me save money, yes. Or at least do something interesting architecturally, great. I didn't see Lambda when it first came out, for example, as a cost opportunity, although, of course, it became one. I saw it as this is a neat capability that I'm looking forward to exploring.Pete: Yeah, and I think that's what was really cool about some of those early ones is these, like, big things would get released. Like, Lambda was a big thing that got released. There was just these larger types of services coming out. And I think it's one of your quotes, right? Like, there's a service for everyone, but every service isn't for everyone.Corey: Yeah.Pete: And I feel like, you know, again, years ago, looking back, it felt like more of the services were more geared towards the operational, the early adopters of Amazon, a lot of those services was for those people. And it makes sense. They got to spread out further, they've got to have kind of a wider reach to grow into all of these different areas. And so, when they come out with things that, yeah, to me, I'm like, “This is ridiculous feature. Who would ever use this?” Like, there's probably a dozen other people at different companies that are obscenely excited because they're at some enterprise that has been ignored for years and now finally they're getting the exact tooling that they need, right?Corey: That made sense for a long time. I think that now, the idea that we're going to go and see an Andy Jassy-era style feature drop of, “Here's five new databases and a whole new compute platform and 17,000 more ways to run containers,” is not necessarily what is good for the platform, certainly not good for customers. I think that we're seeing an era of consolidation where, okay, you have all these services to do a thing. How do I pick which one to use? How do I get onto a golden path that I can also deviate from without having to rebuild everything? That's where customers seem to be. And it feels like AWS has been relatively slow to acknowledge or embrace that to me.Pete: Yeah, a lot of the services, you know, are services they're probably building for just their own internal purposes, as well. You know, I know, they are for a while very motivated to get off anything Oracle-related, so they started building these services that would help migrate, you know, away from Oracle because they were trying to do it themselves. But also, it's like, there's still—I mean, I talk with friends of mine who have worked at Amazon for many years and I'm always fascinated by how excited they are still to be there because they're operating at a scale that just doesn't exist anywhere else, right? It's like, they're off on their lone island that go into work somewhere else is almost going backwards because you've already solved problems at this lower level of scale. That's obviously not what you want to be doing anymore.And at the scale that they're at for some of these services, even like the core services, the small improvements they're making, they seem so simple and basic, like a tiny EBS improvement, you're like, “Ugh, that's so boring.” But at their level of scale for, like, something like an EBS, like one of those top five services, the impact of that tiny little change is probably even so amazingly impactful. Like it's just so huge [laugh], you know, inside that scope of the business that is just—that's what—if you really start pulling the thread, you're like, “Wow, actually, that is a massive improvement.” It just doesn't feel that way because it's just oh, it's just this tiny little thing [laugh]. It's like, just almost—it's too simple. It's too simple to be complex, except at massive scale.Corey: Exactly. The big problem I ran into is, I should have noticed it last year, but it was Adam Selipsky's first re:Invent and I didn't want to draw too many conclusions on it, but now we have enough dots to make a line—specifically two—where he is not going to do the Andy Jassy thing of getting on stage and reading off of a giant 200 item list of new feature and service announcements, which in AWS parlance, are invariably the same thing, and they wind up rolling all of that out. And me planning my content schedule for re:Quinnvent around that was a mistake. I had to cancel a last-minute rebuttal to his keynote because there was so little there of any substance that all would have been a half-hour-long personal attack and I try not to do that.Pete: Mmm. Yeah, the online discussion, I feel like, around the keynote was really, like, lackluster. It was yeah, like you said, very devoid of… not value; it's not really the right word, but just substance and heft to it. And maybe, look, we were just blessed with many, many years of these dense, really, really dense, full keynotes that were yeah, just massive feature drops, where here's a thing and here's a thing, and it was almost that, like, Apple-esque style kind of keynote where it was like, we're just going to bombard you with so many amazing things that kind of is in a cohesive storyline. I think that's the thing that they were always very good about in the past was having a cohesive story to tell about all of these crazy features.All of these features that they were just coming out with at this incredible velocity, they could weave the story around it. And you felt like, yeah, keynote was whatever hour, two hours long, but it would go by—it always felt like it would go by quickly because they were just they had down kind of really tight messaging and kept your attention the whole way through because you were kind of like, “Well, what's next? There's always—there's more. There's got to be more.” And there would be, right? There would be that payoff.Corey: I'm glad that they recognized that what got them here won't get them there, but I do wish that they had done a better job of signaling that to us in more effective ways. Does that make any sense?Pete: Yeah, that's an interesting… it's kind of an interesting thought exercise. I mean, you kind of mentioned before earlier, before we started recording, the CMO job is still available, it's still open [laugh] at AWS. So, if this was a good way to attract a top-tier CMO, I'd almost feel like if you were that person to come in and be like, “Hey, this did not work. Here are the following reasons and here's what you need to do to improve it.” Like, you might have a pretty solid shot of landing that role [laugh].Corey: Yeah, I'm not trying to make people feel intentionally bad over it. This stuff is very hard, particularly at scale. The problem I had with his keynote was not in fact that he was a bad speaker, far from it. He was good a year ago, he's clearly put work and energy into becoming better over the past year. From a technical analysis of how is Adam Selipsky as a public speaker, straight A's as far as I'm concerned, and I spent a lot of time focusing on this stuff myself as a professional speaker myself. I have no problems with how he wound up delivering any of the content. My problem was with the content itself. It feels like he was let down by the content team.Pete: Yeah, it definitely felt not as dense or as rich as we had come to expect in previous years. I don't think it was that the content didn't exist. It's not like they didn't build just as much, if not way, way more than they have in previous years. It just seemed to just not be part of the talk.I don't know. I always kind of wonder, too, is this just an audience thing? Which is, like, maybe I'm just not the audience for his talk, right? Was there someone else in that Expo Hall, someone else watching the stream, that was just kept on the edge of their seat hearing these stories? I don't know. I'm really kind of curious. Like, you know, are we only representing this one slice of the pie, basically?Corey: I think part of the problem is that re:Invent has grown so big, that it doesn't know what it wants to be anymore. Is it a sales event? By the size of the Expo Hall, yeah, it kind of is. Is it a partner expo where they talk about how they're going to milk various companies? Possibly. There's certainly one of those going on.There was an analyst summit that I attended for a number of days during re:Invent this year. They have a whole separate thing for press. The community has always been a thriving component of re:Invent, at least for me, and seeing those folks is always terrific. Is it supposed to be where they dump a whole bunch of features and roadmap information? Is it time for them to wind up having executive meetings with their customers? It tries to be all of those things and as a result, at this scale it feels like it is failing to be any of them, at least in an effective, cohesive way.Pete: Yeah, and you really nailed each of the personas, like, of a re:Invent attendee. I've talked to many people who are considering going to re:Invent, and they're, “I don't really know if I want to go, but I really want to go to some sessions, and I really want to do this.” And I always have to kind of push back and say, “Look, if you're only going there to attend talks,” like, just don't bother, right? As everyone knows, the talks are all recorded, you can watch them later. I did have conversations with some engineer, principal engineer level software folks that were there and the prevailing consensus from chatting with those folks, kind of anecdotally, is that, like, they had actually a lot of struggles even getting into some of these sessions, which for anyone who has been to re:Invent in the last, I don't know, four or five years, like, it's still a challenge, right?There's—you got to register for a lot of these talks way far in advance, there'll be a standby list, there'll be a standby line. It's a lot of a lot. And so, there's not usually a ton of value there. And so, I always try to say, like, “If you're going to re:Invent your, kind of, main purpose to go would be more for networking,” or just you're going because of the human interaction that you hope to get out of it, right, the high bandwidth conversations that are really hard to do in other areas. And I think you've nailed a bunch of those, right? Like, an analyst briefing is really efficient if you can get all the analysts in a room versus doing one-off analyst meetings.Meeting with big enterprises and hearing their thoughts and feelings and needs and requirements, you can get a lot of those conversations. And especially, too, if, like, talking to an enterprise and they got a dozen people all spread over the world, well you can get them all in one room, like, that's pretty amazing in this world. And then on the sales side, I feel like granted, I spent most of my time in the Expo Hall, but that was probably the area that I think you said earlier which I really picked up on, which was the balance between sponsors and attendees felt out of whack. Like it felt like there were way more attendees than the sponsors that I would have kind of expect to see.Now, there were a lot of sponsors on that Expo Hall and it took days. I mean, I was on the Expo Hall for days walking around and chatting with different companies and people. But one of the things that I saw that I have never seen before was a number of sponsorship booths, right—and these are booths that are, like, prebuilt, ten by ten-foot size or the smaller ones—that were blanks. They were like, you know, like, in a low-quality car where you have blank buttons that, like, if you paid more you get that feature. Walking around, there was a nonzero number of just straight-up empty booth, blank booths around which, I don't know, like, that felt kind of telling. Like, did they not sell all their sponsorships? Has that ever happened? I don't even know. But this was I felt like the first I've had—Corey: Or did companies buy the sponsorship and then realize that it was so expensive to go on top of it, throwing bad money after good might not have made sense. Because again, when people—Pete: Right.Corey: —brought out these sponsorships, in many cases, it was in the very early days of the growing recession we've seen now. And they may have been more optimistic, their circumstances may certainly have changed. I do know that pricing for re:Invent sponsorships was lower this past year than it had been in previous years. In 2019, for example, they had two Expo Halls, one at the ARIA Quad and the other at the Venetian. They had just one this year, which made less running around on my part, but still.Pete: Yeah, I do remember that, that they had so many sponsors. What I would say about the sponsors that there's two parts of this that were actually interesting. One, you're definitely right. As someone who has sponsored re:Invent before and has had to navigate that world, you are likely going to commit to the sponsorship as early as June, you know, could even be earlier than June depending on how big of a thing that you're doing. But it's early. It's usually in the summertime that you're—if you haven't made a decision by the summertime, like, you could actually not get a booth, right?And this was, I remember, the last one that I had sponsored was maybe 2018, 2019. And, like, you don't want those last few booths. Like, they put you in the back and not a good way. But going there, were a lot of—I did notice a lot of booths that had pretty massive layoffs who still had the booths, you know, and again, large booths, large companies, which again, same thing. I kind of am like, “Wow, like, how many employees did that booth cost you, right?”Because like [laugh], some of these booths are hundreds of thousands of dollars to sponsor. And then the other thing that I actually noticed, too, which I was honestly a little surprised by, with the exception of the Datadog booth; I love my friends at Datadog, they have the most amazingly aggressive booth BDRs who are always just, they'll get you if you're, like, hovering near them. And there's always someone to talk to over there. Like, they staff it really, really well. But there were some other booths that I was actually really interested in talking to some of the people to learn about their technology, that I actually waited to talk to someone. Like, I waited for someone to talk to, and then finally I'm like, “You know what? I'm going to come back.”And then I came back and waited again. So, it's like, how many of these sponsors obviously spent a lot of money to go there, then months later, they start looking at the people that they have to support this, they've already had some layoffs and probably sent a much smaller audience there to actually, like, operate the booth.Corey: This episode is sponsored in part by Honeycomb. When production is running slow, it's hard to know where problems originate. Is it your application code, users, or the underlying systems? I've got five bucks on DNS, personally. Why scroll through endless dashboards while dealing with alert floods, going from tool to tool to tool that you employ, guessing at which puzzle pieces matter? Context switching and tool sprawl are slowly killing both your team and your business. You should care more about one of those than the other; which one is up to you. Drop the separate pillars and enter a world of getting one unified understanding of the one thing driving your business: production. With Honeycomb, you guess less and know more. Try it for free at honeycomb.io/screaminginthecloud. Observability: it's more than just hipster monitoring.Corey: One bright light that I did enjoy and I always enjoy, though I'm not certain how actionable it is in the direct sense, was Peter DeSantis' Monday Night Live keynote. It was great. I mean, the one criticism I had of it—on Twitter at the time, before that thing melted and exploded—was that it was a bit mislabeled because it really should have been what it turned into midway through of surprise computer science lecture with Professor DeSantis. And I was totally there for it. But it was fun just watching some of the deep magic that makes this all work be presented in a way that most of us normally don't get to see.Pete: Well, this was the first year they did not do their Midnight Madness over-the-top kind of thing. And I also I don't recall that I saw them doing one of the other things I feel like is at night is they're, like, giant wing-eating competition. Am I wrong? Did they do that this year and I just missed hearing about it?Corey: They did not. Turns out that competitive Gluttony is not as compelling as it once was. But they also canceled their Midnight Madness event a month or two before re:Invent itself. What was super weird to me was that there was no event—community or otherwise—that sprung forth to seize that mantle. So, you had a whole bunch of people who were used to going for several hours that night to a big event with nothing to do.And at 9 p.m. they started just dumping a whole bunch of service releases in their blog and RSS feeds and the rest, and it just felt very weird and discordant. Like, do they think that we have nothing better to do than sit here and read through this on a Sunday night where we would have otherwise been at a party? Well yeah, in my case, I'm super sad and of course, I had nothing better to do that night. But most people had things going on.Pete: Yeah. Yeah, exactly. I think also, if you—maybe it's a little bit better now but I don't know when you have to buy that many chicken wings in advance, but with supply chains being what they are and the cost of chicken wings, I mean, not that I track the cost of chicken wings, except I absolutely do every time I go to Costco, they're up substantially. So, that was probably a contributing factor to the wing-eating contest: supply chain pain and suffering. But yeah, it's really interesting that just even in what some of the sponsors kind of were doing this year over previous years, I doubt they did this in 2021—but maybe, I don't know—but definitely not in 2019, something that I don't recall to this level was the sponsors essentially booking out entire restaurants near the venue every single day of the conference.And so again, if you were at this event like we were, and you at the end of the thing, were just like, I just want to sit and I've got a handful of friends, I want to sit and, like, have a drink, and just, like, chat and catch up and hear how the day went and everything else, finding a place to actually go to do that was very, very hard to do. And the thing that I noticed was—again, seemed like it was new this year; I don't recall it in 2019 to this level is, there were a lot of the big sponsors that had just booked a whole restaurant, breakfast, lunch, and dinner, like, from open to close, fully booked it, which was honestly, brilliant.Corey: Oh, yeah. If you bring 200, 300 people to an event, you've got to feed him somehow. And, “Hey, can we just rent out your restaurant for the entirety of this week?” Is not out of the question compared to what you'd even spend just reimbursing that sea of people to go and eat somewhere else.Pete: Exactly. The reason—I'm approaching this from, like, a business perspective—if I had a large group of enterprise salespeople and they need a place to book meetings, well, it's super compelling if I'm being courted by one of these salespeople and they're like, “Hey, come and have breakfast. Come and grab a coffee.” You know, and there's a place where you can sit down and quietly enjoy that meal or coffee while having a sale. Like, I'll have that sales conversation and I'm going to be way more motivated to show up to it because you're telling me it's like, this is where we're going to meet.Versus some of my friends were trying to, like, coordinate a lunch or a coffee and it's like, do we want to go to the Starbucks that has 500 people in line or do we want to walk four hotels, you know, down the street to find a bar that has video poker that no one will be sitting at and that we can just sit down and talk, right? It kind of felt like those were your two options.Corey: One thing that MongoDB did is rented out the Sugarcane restaurant. And they did this a couple of years in a row and they wound up effectively making it available to community leaders, heroes, and whatnot, for meetings or just a place to sit down and catch your breath. And I think that was a brilliant approach. You've gone to the trouble of setting this thing up for meetings for your execs and whatnot. Why not broaden it out?You can't necessarily do it for everyone, for obvious reasons, but it was nice to just reach out to folks in your orbit and say, “Yeah, this is something available to you.” I thought that was genius. And I—Pete: Oh yeah.Corey: —wish I thought of doing something like that. Let's be clear. I also wish I had rent-out-Sugarcane-for-a-week budget. But you know—Pete: [laugh].Corey: —we take what we can get.Pete: Yeah. That'll be a slight increase to the Spite Budget to support that move.Corey: Just a skosh, yeah.Pete: Yeah, the MongoDB, they were one that I do remember had done it similarly. I don't know if they had done it, kind of, full-time before, but a friend of mine work there, had invited me over and said, “Hey, like, come by, let's grab a drink. You know, we've got this hotel, you know, this restaurant kind of booked out.” And that was back in 2019. Really enjoyed it.And yeah, I noticed it was like, you know, basically, they had this area available, again, a place to sit down, to open your laptop, to respond to some emails, making it available to community people should have been a no-brainer to, really, all of these other sponsors that may have times of less kind of attendance, right? So obviously, at any of the big meals, maybe that's when you can't make it available for all the people you want to, but there's going to be off hours in between times that making that available and offering that up generates a supreme amount of goodwill, you know, in the community because you know, you're just looking for a place to sit out and drink some water [laugh].Corey: Yeah, that was one challenge that I saw across the board. There were very few places to just sit and work on something. And I'm not talking a lounge everywhere around every corner was needed necessarily or even advisable. No, the problem I've got was that I just wanted to sit down for two or three minutes and just type up an email quickly, or a tweet or something, and nope, you're walking and moving the whole time.Pete: Yeah. Now honestly, this would be a—this was a big missed opportunity for the Amazon event planning folks. There was a lot of unnecessary space usage that I understand why they had it. Here's an area you could play Foursquare, here's an area that had seesaws that you could sit on. Like, just, I don't know, kitschy stuff like that, and it was kind of off to the side or whatever.Those areas honestly, like, we're kind of off to the side, they were a little bit quieter. Would have been a great spot to just, like, load up some chairs and couches and little coffee tables and just having places that people could sit down because what ended up happening—and I'm sure you saw it just like I did—is that any hallway that had somewhere that you could lean your back against had a line of people just sitting there on their laptops because again, a lot of us are at this event, but we're also have jobs that we're working at, too, and at some point during the day, you need to check in, you need to check some stuff out. It felt like a lack of that kind of casual space that you can just relax in. And when you add on top of all the restaurants nearby being essentially fully booked, it really, really leaves you hanging for any sort of area to sit and relax and just check a thing or talk to a person or anything like that.Corey: Yeah, I can't wait to see what lessons get learned from this and how it was a mapping to next year, across the board. Like, I have a laundry list of things that I'm going to do differently at re:Invent next year. I do every year. And sometimes it works out; sometimes it really doesn't. And it's a constant process of improvement.I mean, one of the key breakthroughs for me was when I finally internalized the idea that, yeah, this isn't going to be like most jobs where I get fired in the next six months, where when I'm planning to go to re:Invent this is not the last re:Invent I will be at in my current capacity, doing what I do professionally. And that was no small thing. Where oh, yeah. So, I'm already making plans, not just for next re:Invent, but laying the groundwork for the re:Invent after that.Pete: Yeah, I mean, that's smart way to do it. And especially, too, when you don't consider yourself an analyst, even though you obviously are an analyst. Maybe you do consider yourself an analyst, but you're [laugh] more, you know, you're also the analyst who will go and actually use the product and start being like, “Why does this work the way it does?” But you're kind of a little bit the re:Invent target audience in a lot of ways, right? You're kind of equal parts on the analyst expert and user as well. It's like you kind of touch in a bunch of those areas.But yeah, I mean, I would say the one part that I definitely enjoyed was the nature walk that you did. And just seeing the amount of people that also enjoyed that and came by, it was kind of surreal to watch you in, like, full safari garb, basically meandering through the Expo Hall with this, like, trail of, like, backpacks [laugh] following you around. It was a lot of fun. And, you know, it's like stuff like that, where people are looking for interesting takes on, kind of, the state of something that is its own organism. Like, the Expo Hall is kind of its own thing that is outside of the re:Invent control. It's kind of whatever is made up by the people who are actually sponsoring it.Corey: Yeah, it was neat to see it play out. I'm curious to see how it winds up continuing to evolve in future years. Like right now, the Nature Walk is a blast, but it was basically at the top and I had something like 50 people following me around at one point. And that is too big for the Expo Hall. And I'm not there to cause a problem for AWS. Truly, I'm not. So, I need to find ways to embrace that in ways that don't kill the mojo or the energy but also don't create problems for, you know, the company whose backup I am perched upon, yelling more or less ridiculous things.Pete: [laugh]. I think it was particularly interested in how many people I'd be walking by and every once a while I would see, like, a friend of mine, someone actually working one of the booths and just be like, “What's going on here?” Like, I know one of my friends even said, “Yeah, like, nothing draws a crowd like a crowd.” And you can almost see more people [laugh] just, like, connecting themselves onto this safari train moving their way through. Yeah, it's a sight to see, that's for sure [laugh].Corey: Yeah, I'll miss aspects of this. Again, nothing can ever stay the same, on some level. You've got to wind up continuing to evolve and grow or you wind up more or less just frozen in place[ and nothing great ever happens for you.Pete: Yeah, I mean, again, Expo Hall has gone through these different iterations, and I—you know, when it does come to the event, as I kind of think back, I probably have spent most of my time actually in the Expo Hall, usually just related to the fact that, like, when you're a sponsor, like, you're just—that's where you're at. For better or worse, you're going to be in there. And especially if you're a sponsor, you want to check out what other sponsors are doing because you want to get ideas around things that you might want to try in later years. I mentioned Datadog before because Datadog to this day continues to have the best-designed booth ever, right? Like when it comes to a product that is highly demoable, I've been myself as a sponsor, it has always been a struggle to have a very effective demo setup.And I actually remember, kind of, recommending to a startup that I was at years ago, I'm doing a demo setup that was very, very similar to how Datadog did it because it was brilliant, where you have this, like, octagon around a main area of tables, and having double-sided demo stations. A lot more people are doing this now, but again, as I walked by I was again reminded just how effective that setup is because not only do you have people that just they don't want to talk, they just want to look, and they can kind of safely stand there and look, but you also have enough people staffing the booth for conversations that for, like me who actually might want to ask for questions, I don't have to wait and I can get an answer and be taken care of right away, versus some other booths. This year, one of the areas that I actually really enjoyed—and I don't even know the details of, like, how it all came about—but it looked like some sort of like Builder's Expo. I don't know if you remember walking by there, but there was a whole area of different people who had these little IoT or various powered things. One of them was, like, a marble sorting thing that was set up with a bunch of AWS services. I think there was like the Simple Beer Service V… four or five at this point. I had one of those iterations.It was some sort of mixture between Amazon software services that were powering these, like, physical things that you can interact with. But what was interesting is like, I have no idea, like, how it was set up, and who—I'm assuming it was Amazon specific—but each of these little booths were like chocked up with information about who they were and what they built, which gave it a feel of, like, this was like a last-minute builder event thing. It didn't feel like it was a highly produced thing. It had a much more casual feel to it, which honestly got me more interested to spend time there and check out the different booths.Corey: It was really nice to be able to go and I feel like you got to see all of the booths and whatnot. I know in previous years, it feels like you go looking for specific companies and you never find them. And you thought, “Oh”—Pete: [laugh].Corey: —“They must not have been here.” You find out after the fact, oh no, just you were looking in the wrong direction because there was so much to see.Pete: There were definitely still a couple of those. I had a list of a handful of booths I wanted to stop by, either to say hi to someone I knew who was going to be there or just to chat with them in general, there was a couple that I had to do a couple loops to really track them down. But yeah, I mean, it didn't feel as overly huge as a previous one, or as previous ones. I don't know, maybe it was like the way they designed it, the layout was maybe a little bit more efficient so that you could do loops through, like, an outer loop and an inner loop and actually see everything, or if it just was they just didn't have enough sponsors to truly fill it out and maybe that's why it felt like it was a little bit more approachable.I mean, it was still massive. I mean, it was still completely over the top, and loud and shiny lights and flashing things and millions of people. But it is kind of funny that, like, if you do enough of these, you can start to say, “Oh well, I don't know, it's still felt… a little bit less [laugh] for some reason.”Corey: Yeah, just a smidgen. Yeah. Pete, it is always a pleasure to get your take on re:Invent and see what you saw that I didn't and vice versa. And same time next year, same place?Pete: Yeah. I mean, like I said, one of my favorite parts of re:Invent is, you know, we always try to schedule, like, an end-of-event breakfast when we're both just supremely exhausted. Most of us don't even have a voice by the end. But just being able to, like, catch up and do our quick little recap and then obviously to be able to get on a podcast and talk about it is always a lot of fun. And yeah, thanks again for having me. This is—it's always, it's always a blast.Corey: It really is. Pete Cheslock, Head of Growth and Community at AppMap. I'm Cloud Economist Corey Quinn and this is Screaming in the Cloud. If you enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice and then put something insulting about me in the next keynote because you probably work on that content team.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.

Welcome to Freedom Friday w/guest Dr. Steve Turley, join us as we discuss the attempted cancel culture attack on Steve's new Documentary "The Rise of the American Patriot.  How a Remnant church stepped up and saved the premier and why its important to stand with others in times of censorship.https://www.turleytalks.com/https://www.facebook.com/turleytalks/https://twitter.com/DrTurleyTalkshttps://www.instagram.com/turleytalks/https://rumble.com/c/DrSteveTurleyhttps://insidersclub.turleytalks.com/welcome  Welcome to freedom Fridays. This is Dr. Steve Turley. And he is a well, you're an author, you're a movie maker, you have a podcast, you've got a YouTube channel, if you've if you've been on YouTube, I mean, which I haven't you see this guy, and I love. I love your demeanor, your care, your kind of your, your style of commentary, because it's very, it's funny, it's light hearted, you know, because we're looking at some dark subjects. And you bring such a good, just uplifting and entertaining way of looking at some of these things. So I appreciate you coming on the program. Steve  6:17  Oh, thanks, Gary. It's it's, it's my honor, we were just talking earlier, you know, you are in a bluer area and a very, very red state. So I'm in a very, very blue state. And so I guess on the little red dot and that blue state. So we have, we see we see comparable challenges in our own backyards. And I think we can encourage each other a lot through it. Gary Duncan  6:41  Yeah, thank you. Let's talk about your new document that you just came out on the 15th, I believe. And I'll read a little bit about some pushback you got as soon as it came out before it came out. Your your documentary is called the return of the American patriot. Because you're the page you're the professor, patriot, right? Patriot, Professor, Steve  7:00  Patriot professor, that's Gary Duncan  7:02  you, I was thinking, if I had you as a professor, when I was in college, I probably would have stayed awake during history class. Because I mean, your the way you bring about the news and and things that are happening in our culture and in the church and things like that, is it just keeps you it keeps you focused, but entertained enough to to not walk away really ticked off. You know what I'm saying? And you bring a great perspective to it. So talk a little bit about the documentary you've got out, and you're kind of some of the things you've run against, you know, producing Oh, Steve  7:41  yeah, well, so this movie really tries to present that our 20 minute documentary, that kind of hopeful optimism that Ronald Reagan gave to us any great movement is going to have to be optimistic at its very core also ends up eating itself and dies or just look at woke leftism, and just the resentment that killing their movement. Yeah, this returning the American patriot is actually a it's a, it's a documentary on the rise of the Pennsylvania Magga movement. It really is the story of how ordinary Americans who never before involved in politics rose up in mass and mobilized to successfully take on unconstitutional COVID mandates, election integrity issues, woke school boards all across their state. It's a very inspirational story of the people effectively pushing back against the permanent political class. And you would think that anyone who openly supported democracy would be interested in a film like this, you know, it's as democratic as it gets. But little did we know that the very drama we captured on film would actually play itself out in real time for the premiere we had. We had scheduled a live premiere on July 16th, at a local IMAX theater in Lancaster, Pennsylvania called Penn cinemas and they had a 400 seat capacity. We opened up the tickets and we sold out literally in hours. We sold out in 24 hours, those 400 tickets. And then we learned just days before the premiere that a group of woke activists called Stand Up Lancaster cry bullied the movie theater to cancel our premiere. Remember, these are people who actually believe you can work try to wrap your mind around this. These are people actually believe that censorship is a form of free speech. They literally believe that right? They defend big tech and all sorts of censoring us because they say that's their right to their own freedom of speech. That's their Gary Duncan  9:58  free speech. Ah, that's their nobody else's. Steve  10:02  Exactly. Exactly. Exactly. Well, sad. Right? Exactly. Again, it's a square circle. It's just, it's beyond absurd. They cry bullied this movie theater to canceled premiere and unfortunately the owner of the theater was a coward and he caved and canceled our contract just days before the scheduled viewing. So there we were without a venue for tickets sold. No venue. So we went to another venue. And it was the Wyndham hotel in Lancaster. And they gave us a contract to rent out their theater venue there. They had hosted the republican party a few months prior to that the state Republican Party, so we thought we were pretty safe with them. And they there was they had an 800 seat capacity. So we doubled our ticket sales again. So it was really like, Oh, this is great. This is awesome. And then the same thing happened. They were they were they weren't just cried bullied, I was told there was even threats of mob violence if they didn't shut us down. And and so there we were two venues canceled. We were being mocked and ridiculed by the local Lancaster paper, which is a total left wing rag. It's the most Whoa, it's called LNP. It is pathetic, their board of editors actually came out and defended these practices. So again, now we have a medium major media outlet local media outlet defending censorship. It's absolutely astonishing. And keep in mind, Gary, keep in mind for a movie no one had ever even seen. This, this was this was the most dangerous movie, no one saw. I mean, literally no one. I hadn't even sat through the whole thing by this time, right? They were getting my staff was gonna surprise me with the whole edited version. So there we were no venue. And that is and this is why I'm so honored to talk with you. That is when the pastors of Christ Community Church and Camphill godly men stepped in, they have a 1200 seat auditorium, replete with a full movie theater quality sound system, massive movie theater size screen, and they offered it to us and their 1200 seat auditorium. We ended up selling all 1200 tickets. Okay, so we went from 400 to 1200. Talk about three fold increase, right? Yeah, God Gary Duncan  12:28  had better plans for you than you thought it and Steve  12:31  that there's no way we could have planned this. We would never have planned our brains don't think that way. Let's plan for a 1200 seat, you know, Premier with Doug Mastriano there and all that sort of stuff, kind of stuff who's running for governor there? They offered to us and the moment these leftists heard about that, they started threatening the church. Okay. Again, this tells you who these people really are. They started threatening the church they were going to contact the IRS which they did. Again, there was an the same media outlets did the exact same thing. You know, violating separation between church and state bringing Doug Mastriani was campaigning for Governor there in person and blah, blah, blah, violating the Johnson Amendment all this nonsense. And and so when it ended up having all of this you know, proverbial dung hitting the fan. Even the lawyer of the church told the pastors you need to drop this, right, because lawyers are risk averse. That's what do you need to drop this? We're getting, we don't want the IRS breathing down our neck and so forth. Those two pastors stood firm. They told their lawyer take a hike. We're standing for liberty and truth. And, and, and they hosted us. We came in 1200 People Doug mastriano, huge premiere, it was absolutely amazing. Electric standing ovation at the end. And in the end, Gary, in the end, seven protesters showed up they weren't even allowed on to the vicinity. They had to hang out on the street across from the church seven protesters with their little arts and crafts, you know, signs document separation between church and state. I think even one guy said I worship Satan something ridiculous right? And, and just to show you that God, God does have a very wonderful sense of humor for his children. Gary, it was raining. So they had to stand out there in the rain, with their masks on looking repulsively ridiculous as people who love faith, family and freedom were all gathered together in an astonishing fellowship. It was absolutely beautiful. You know, Doug master on game got up gave a very great It's just and, and beautiful talk. And it was an amazing testimony to what patriots can do when we all stick together. Gary Duncan  15:09  Wow, that's awesome. As you were talking about that I was getting this picture of this little, tiny weeny little mouse and this huge elephant. And, and it gives me encouragement because I've really focused a lot on where's the church, and I could see how the left and the small 1% or half a percent of a population controls the whole country and my church. And so we really that's encouraging to hear. And and it's who's the church again, is you need to you need to give that name out again, because the people that hosted you and those pastors because they really that's a there and a thing goes for that. Yeah, Steve  15:51  absolutely. Christ Community Church in Camp Hill, Pennsylvania, Camp Hill, two words, Pennsylvania, Gary Duncan  15:58  they if you're looking for a church in that in that city, then this is the one to go to, because they that's what you that, to me, that's Ephesians six, you know, when you stand and they're willing to stand against all odds, and that's, that's awesome. Steve  16:13  And just just to drive this home, the pastor when he got up to give the prayer before the whole event, invited everyone of course, if you're looking for churches to come here, and by the way, next week, we're showing another movie 2000 mules. So these guys, are they these guys are the real deal. Gary Duncan  16:32  They are the remnant church. Steve  16:34  They are so bold, it's beautiful. Gary Duncan  16:38  That that is great. What would you so that was the one way churches and the community to you know, leadership in church could get involved? Get your get your documentary and have it hosted at their church? I mean, are you are you pursuing that at all or looking? Steve  17:00  Oh, yeah, no, we've had we had the documentary going around all over the place now at this point. So it just last Thursday. It went live live streaming. So now you can actually stream it live. If you go to the return of the American patriot.com You can get your own copy. And, and absolutely, I think we even have a situation. We have a protocol from where you can you can show it in a mass viewing. Okay. Oh, absolutely. Yeah. Gary Duncan  17:31  And I'll put the links in the show notes and all that for a site and all that. What's tell us a little bit about some of the stories or the things that are in the movie are a couple of them just kind of thesis. Steve  17:43  Yeah, well so what it centers on so Pennsylvania is interesting because it is probably the single most Rhino infested state in the nation a lot of people don't know that so when we think of like you know, Republicans in name only write your Neo cons just just PETE Yeah their note well I prefer Diablo Democrat and all but label or you know if they're part of the unit party the Lindsey Graham's the Mitt Romney's the middle Lincoln project, right to Lincoln Lane candy project. Exactly. Liz Cheney. They're there. They're part of the permanent political class. It's radically secularized, radically globalist that hates our culture's customs traditions, hate family, faith, family and freedom. Those things get in the way of their their globalist projects and so forth Gary Duncan  18:34  are wolves in sheep's clothing. Steve  18:36  They really are because they because they campaign is patriots but then they Governor governor's permanent political class members. And when we tend to think of states like that, I mean up until recently, we tend to think of some a place like Arizona where you know, John McCain had such an inordinate effect effects and influence well that's gone now Carrie Lake and Blake masters and Mark Finch him have all crushed that it's now Maga country and Carrie Lake is going to win by the way she's definitely on track and I think like masters gonna have no mark Finch up to he's almost 10 over his opponent. So the these people he's going for Secretary of State so the, you know, you tend to think that these people these rhinos are in places like Arizona or even most recently, like Utah with Mitt Romney and their crazy Governor given his pronouns out and all this sort of nonsense. Okay, keep in mind these are Republicans, but but Pennsylvania is actually the worst Pennsylvania the Republican Party is no different, literally than the Democratic Party when Doug Mastriano one who's a dear Christian brother, amaze me. He's He's literally now the poster boy for Christian nationalism, as they call it today. The big boogeyman we could talk about that, which is a lot of fun, you know, but nevertheless, he, when he got the nomination was almost 50% of the vote with a Guys in that race for the Republican nomination, the top five Republicans in the Senate, his suppose it colleagues, he's a state senator, they all turned around and endorsed the Democrat Josh Shapiro. And keep in mind, Gary, Josh Shapiro is all for unfettered abortion. Right. He's all for, you know transgenderism is all for CRT in our, in our classrooms. I mean, this is full blown woke nonsense and Republicans are endorsing it. This movie is all about moms and dads and grandparents and people who've never been involved at Amish Pennsylvania dodge, all rising up together, mobilizing and organizing and taking back the Republican Party with Doug Mastriano has nomination being sort of the crown jewel of this project, taking back the Republican Party away from these rhinos, these Diablos and giving it back to the people so that the values of faith, family and freedom become the values of the party they want all they want. In the end. It's a process, the technical processes known as re territorialization. Yeah, it's a fancy schmancy word. But what it just simply means what we're seeing today, and this is what I think Christian nationalism actually is. It's if globalism, de territorial losses of globalism, dis embeds and dislodge his political life away from the local and to the trans local to this managerial class that oversees the entire political and economic complex, if globalism, D territory alized his political involvement, what the what the collapse of globalism is basically is a re territorial loss. And it's bringing politics back to the people back to the local back to the county and the community. And what you're seeing in Pennsylvania in particular, but you're seeing it all over Pennsylvania is a microcosm, what you're seeing is these communities, all organizing all across their counties to create a single party, that is once again, upholding the values, interests, concerns of those who love faith, family and freedom. That's all they want. They just want their leaders to represent their values rather than despise them. And that's the movie takes you through the journey of how they did that through the COVID mandate and sanity through the CRT and sanity and through the election. You know, shenanigans in 2020. Gary Duncan  22:43  With would this be sound like a great movie that I need to show in our Davidson County Republican Party group to see how it's done? Because, you know, as we talked earlier, I was I'm in a red state, you know, super majority of Republicans in the state. But in my county, which is the state capital of the Davidson County, which were the state capitalism. It's full blown blue, communist. I mean, those school board is full on communist liberal. I mean, they voted down the Nash, the Republican National Committee come into town for the convention. Yeah, we're, we're, we're going to be paying for people to go across our state employees to go across the state lines to get abortions. I mean, it's just bizarre. And, you know, how do how does a small well, not a small county, but the main county in the state had, how do we fight against this? I mean, were those small, local areas, were they blue? Or were they more red, and people just came together? I know, it's a groundswell of grassroots effort. Steve  23:54  It is so Pennsylvania's can be a little different, because Pennsylvania, so if anything, like we're talking about earlier that the the Republican Party is a bit complacent in Tennessee from from what I'm hearing, or to say, whereas whereas in Pennsylvania, that complacency, characterized the last three decades, and and now there's kind of a reawakening going on. And, yeah, it applying it to your particular locality would be interesting that that's going to involve, I think, some some, you know, creative inventiveness on your part. How do you awaken the population, your own locality, one thing that seems to be doing it and this there's a section in this movie that touches on it, one thing that seems to be doing it is wokeness wokeness is freaking out, even the left. That's something we've got some studies on that now. So we're finding that anything woke will actually tend to split the left. So think of people like Bill Maher, or peers Morgan or a Dave Rubin, either even even even a Jordan Peterson would have said he would have been considered center left. Five years ago, you would have considered himself center left five years ago. They, they they abhor wokeness every bit as much as any conservative wokeness is, is a pourraient to most people. And so the more we push culture wars, and this is what Mastriano is doing. It's what Glenn Younkin did. So well so ingeniously in, in Virginia, back in the 2021 election, where he's pushing CRT CRT soon, and he made Terry McAuliffe actually defend teachers, unions, school boards, and CRT and that split the last half of Macola Fs constituents went back, I don't want that. Whereas the other half were ravenously, eating it up like zombies, you know, eating up a body or something like that. So that's one side of it. But the other side of it is as it's splitting and laugh, woke issues, unite the right woke issue for and again, for lack of better terms left and right, right. But it unites the right unites the Republicans. In other words, if you ask Democrats, do you support this work issue? 50% Say yes. 50% say no, you ask Republicans to support this work is you 100% basically say, No. So Republicans are more likely to come out and vote against a woke issue than Democrats are to come out and vote for it. So pushing the culture wars, from the vantage point of the woke left exposing the woke left, that seems to have a very powerful animating capacity. Gary Duncan  26:48  So how going back to churches again, because that's my thing is Yeah, is because they've got the biggest voice, because they've got people in front of them. And what you just said there is pushing the culture because the church should be the one that changes culture, not the culture, change the church, of course, and we need those pastors and those leaders that will take that very thing and push that narrative of the culture in a biblical way that educates and motivates the people sitting in the pews. And, and I'm not seeing that I'm starting to see a lot around the country happen. And I'm seeing one or two or three maybe churches here in town that do that. But it's not, I don't see it as and that's why I like you, because you're very positive. I'm usually a positive guy. But after 2020, I just went downhill. Yeah, positiveness. Because it saw assault. 2020 is is the dividing point within the church of woke and realism. There because we're in a spiritual war, we're in our spiritual war that I don't think people really get. It's a biblical revelational. In times spiritual war, we've always been in war. But this one takes a different to me, this one takes on a different connotation, because what we're doing to the children, what we're allowing to be done to the children in the name of not offending other people, you know, with masks and all the stuff and then the wokeness in the schools. And I've been to several school board meetings, and I think I've yet to see a pastor stand up, and shame and, and, and preach to the school boards, right about what they're doing. I've seen regular people. And I think that it has to actually some a lot of the movement is coming from the people sitting in the pews that are sick and tired. Yes, what's going on? And there's nothing in the pulpit that says, This is how we deal with transgenderism. This is how we deal with because they don't want to get yelled at. They don't want to leave the church because we had a church split. Pastor left, but he's a very, I'm telling you, Steve burger. I mean, if you ever heard him, right, he's on fire about what we should be doing. Ryan. Steve  29:13  It is it is. It is a a very chilling testimony, that the person who has done more for Christianity and has just recently delivered, perhaps the single best message to the church is from a Canadian psychologist who doesn't even go to church. Gary Duncan  29:35  Jordan Peterson? Oh, yes. Okay. Okay. Right. Steve  29:38  Right. I mean, that's it. That's a testimony to either Well, I should say it's a judgment. I mean, that's, I mean, if you think about how, what he has been able to say, I mean, I don't know if you saw his message to Christian churches. I like to Well, yeah, it's very good. It's very good. I've got on my channel. I did a little commentary. on it, but it was absolutely brilliant. Woke I mean, he he made he didn't he didn't mince words, woke ism is a crippled religion. It is an it is a it is a pernicious violent ideology that wants to erase the church. And so the only way the church is going to push back against woke ism is by not being woke. But being the opposite. And and you cannot be more opposite woke than to speak into the hearts and minds of the men of your congregation. You've got to speak to the men. And you've got to let men all over the nation know that if they want to be men. And if they want a place where they're allowed to be men without being disparaged. The only place is the Christian church. That's when you see revival. When you see men come because you know the old saying if you if you when children and your evangelistic efforts if you win children, you win children, okay? If you if you if you win, wives, you know, you win wives, but if you win, husbands, you win the husbands, you win the wives and you win the children, there's a right there's, there's an order to which God created the world a creational order. And woke ism is just throwing it all into utter chaos, as did Satan and Genesis chapter three, the serpent, turn the world upside down, right? It's supposed to be God, man, woman animals, and we're just not in that, that kind of order. And Satan that the serpent turns around and makes it animal woman man and God's not even there. Right. So that's the great inversion that we've seen. So woke ism is very, it's just, it's just in line with that. So what we entered understand what's really going on big picture seems to me is that for the last 100 years or so that's those were when the seeds were being sold sown, but it really came to fore in the 1940s, as I understand it. Before 1940, the Supreme Court saw religion as a public good, as did all of our founding fathers. They all believed in what's called an accommodationist conception of religion. And the accommodations, conceptual religion is church and state work together in partnership for the betterment of human society, to create a republic of virtue of free men, because they knew the founding fathers knew that the only way we could be free, is if we were self governing, but the only way we could be self governing is if we had if we we tapped into a virtue tradition of some kind. And for them, of course, 98% of them that's going to be Christian and formed fruits of the Spirit, you name it, right Sermon on the Mountain, like 10 commandments, but the only way you can really tap into a faith tradition genuinely, is if you're free. Right? So and the only way you're ultimately free, is if you're if you're cultivating some kind of virtue, but the only way you're cultivating virtue is by tapping into some kind of faith. And the only way you're tapping to real true faith is through freedom which grows virtue which God has faith with God is free. That's called the Golden Triangle of freedom. And so they understood the church is indispensable to a free people. You have to have a sacred vision of the good to which we can all aspire, in order for us to be a people living in Liberty walking in Liberty, right? The Galatians passage barks a lot. For liberty, you have been set free. After 1940 For whatever reason, it's hard to pinpoint why but obviously, it seems to be something in the legal the law schools in the universities, the Supreme Court started instituting a separation test doctrine between church and state. So while the accommodations doctrine always made a clear distinction between church and state, the state's not the church, the church, not the state, Christ's humanity and divinity, right. They're not commingled in the lie. By Gary Duncan  34:24  grant. That's why they came to America, one of the reasons they got away from England, Steve  34:28  because it was a state church. Exactly. Right. Exactly. Right. So so the church and the state are different, and yet they work together. And that's what made our experiments so powerful, so amazing, because freedom is what holds it all together, in that sense. And so then, after 1940, the Supreme Court started instituting more of a separations for you and we know that because they started quoting from a letter that Thomas Jefferson wrote rode to the Danbury, Baptists, Danbury, Connecticut, I'm not born not too far away from there, where he used this phrase against, not even in the Constitution, Jefferson didn't even write the constitution is not even in the Constitution, you know the phrase, the separation between church and state. And before 1940. That phrase was only used as I understand about a handful of times in Supreme Court decisions and deliberations in the light after 1940. It's been used 1000s of times. So something happened there. And what that did in effect is it drove the church from the public square, and consigned it solely into the private sphere of life care. You see it in our urban planning, you know, think of your New England Commonwealth, what was the most prominent building on the town green, the steeple, the church, absolutes, the church, you go into the medieval towns of Europe, the most prominent building that you see in the center of this beautiful medieval city, it's the church, right, and the castle is right across from often in the shadow of the church. So in our urban planning, we can see the the role the place that the church played in a flourishing human society today in modern urban planning, where's the church? Gary Duncan  36:25  Not it's beside the coffee shop, Steve  36:28  you got it. That's it, you got it. It's in the place of consumption, and consumerism. So you've got your, you've got your pizza hut, you know, you've got your dry cleaners, and you've got first Methodist, you've got it, it's push, it's pushed into the periphery, the private sphere of life. And here's how we tie it all together, Gary, there is no way that the church can proclaim truth, social truth, cultural truth, in that position, any more than Pizza Hut can. That's what happened to us. We got privatize, we were talking about it. Earlier in the break. You said it perfectly belonged to the church today, in most people's minds, even inside the church, thank God for the Holy Spirit, converting our hearts. But even inside the church, our social conditions have trained us to believe that being part of the church is being part of a club. It's being part of you might as well be part of weightwatchers right or what our yoga club or whatever it is, it has no objective moral truth to its proclamation anymore. So it's now ridiculed and laughed at and dismissed, and the like, what pastors are going to have to rediscover. And by the way, the Maga movement is right there to help them with this. What they're going to have to rediscover is a voice that can speak publicly. Again, this is not personal, private truth. Christ is truth, the law galls holds the universe together. And when we proclaim God's truth, that's true for everyone, regardless of whether you believe it or not, because that's public and a private, I get it personal, private, Subjective Truth. That's your deal. That's my deal. You like pistachio? I like chocolate. No problem, I got it. But when you're dealing with public, social life, truth is objective. It applies to all not just some, it's objective. It's not just subjective, it's obligatory. It's not just optional. Now, of course, we're free to respond. But that freedom of response is predicated on the objectivity of that truth. And that's what our pastors have to rediscover, when they rediscover it. They were discovered this that we're on fire. Gary Duncan  38:50  Yeah, I think, like you're saying, we've lost the identity or am I'm really about waking up the leadership, because the leaders are what leads the people, not the people lead the leader, but that's kind of where it's going. But we've lost our identity and our authority in our power. I mean, I'm reading the Bible, and it's saying, you know, these signs shall follow those who believe you will cast out demons you will heal the sick, right? speak in new tongues, and I'm looking around I'm going well, when's the last time I cast out a demon? Or anybody's cast out a demon and healed the sick and it's like, that's our heritage. God gave us through the Holy Spirit these abilities when he chooses to do those things. And if if leadership is not telling us and helping us, it's because that's the purpose. The purpose of these gifts are to edify the church so we can go out and do these things. And it just it just and that's why 2020 blew me away so bad is because the deception that came about And it's like, who are we anymore? And no wonder young people in the Millennials don't want to get involved in a church unless it's happy clappy and coffee and smoke machines and fog machines and all that kind of stuff. You know, it's a country club again. So and the one thing I'm, and here's my negative, so help me out here. I think when we get to the when we do wake up the left, and the evil that's behind the globalists and all the things, they're not going to let it go too long. I mean, you're talking about, it's getting time to where we're going to sacrifice more than just our good name, our lives, our livelihood, there's people out there now doing that and praise God to those that are standing and fighting. But I think that what the the leadership of the church needs to really get ready, is to gird up, because I see the coming age of the church not being happy clappy, but it's gonna be persecuted beyond belief where we're at coming. You see what they did in 2020? What's What are they willing to do if they're willing to kill children? And they're willing to euthanize the older and they're willing to, to propagate a bio weapon across the whole world? What are they willing to do? Ya know, it's scary when the church does stand up, what are they willing to do and we got to be ready for it. We got to know how to fight back. And that's my whole thing. We're not fighting. We've raised a bunch of in the last 50 years, we've raised a bunch of chocolate soldiers. You're in the first moment of any heat, we melt like, like little statues of a bunny rabbit. Steve  41:42  But that's what privatized faith does it right. It has it has no backbone, you know, it's like, you know, you renounce pistachio ice cream, I'm gonna punch you okay, I pronounced that. Right. That's that. That's what privatize faith does? It is it is. There's no I mean, when Alexander Solzhenitsyn came over to the United States, and gave his Harvard address in 1979, that amazing address called the world split apart. One of the first things he did it, I mean, everybody thought he was gonna go rah, rah West, he actually said no, Soviet Union is pretty horrible, pretty terrible. But the West is just a secular, it's just as atheistic and you're gonna go in the same direction, you're just you're doing it with four car garages, you know, you make it a little bit more tolerable. And he said, the one thing he noticed about the West, the principal characteristic that caught his eyes, we've lost courage. And in the classical world, the Four Virtues, wisdom, moderation, justice, and courage of those four, and that, you know, corresponds to the four elements of the cosmos. Right, that's all held together by the law girl. So we've got our four gospels, right, that all held together by Christ, the logos themselves, and then we have the additional faith, hope and love virtues and all that sort of stuff. It's all part of this amazing world, this identity would belong to 2000 years old, and even stretching before that with these, these pagan traditions that end up getting transfigured and, and by Christ himself. I mean, Solzhenitsyn pointed out, you know, of those Four Virtues, all the ancients recognize courage was the most important one, because without courage, you don't have the fortitude to defend the other three. Without courage, you can't defend, in this case, faith, hope and love. You can't defend it. You can't defend faith, family and freedom requires. It requires courage. So I think, look, I think the good news in this is, again, you look around the world and what's happened. You had 70 years of Soviet communism ravaged land. That was the jewel of Byzantium. Okay, so you're talking Czarist Russia, 60,000, ornate, gorgeous churches. Some of the biog some of the historical biographies you'll read, I mean, they weren't incredibly Pieta stick people, extraordinarily so Bolsheviks come in, and literally ravaged it so that by the end of that 70 year period, interesting, interesting number of by the way, at the end of that 70 year period, there were only 2000 functioning churches left they were gone, that it was going through 1000 monasteries during the Czar's period. Not a single monastery was an operation when the when the Soviet Union fell. And by the way, keep in mind, keep in mind, the very day the Soviet Union officially fell the most atheistic regime on the planet. Of course, it was December 25. It was Christmas Day, right. So a new birth happened because now here we are 30 years later. And by the way, this is this is true for much of eastern Europe as a whole 30 years later, a Russia is now approaching upwards of five 50,000 churches, they are on track by the year 2050, to be in full restoration of that czarist orthodox glory and the book by John Burgess, a theological historian at Pittsburgh seminary, where he went out to study the role of Christianity in Russia right now, what happened in Russia, in effect was communism got replaced by orthodoxy. So the vacuum that communism left was filled by going back to their identity by going back to their civilization, their cultures, a customs, and traditions. So for right or wrong, all that sort of stuff, you're not getting it all that what we have to understand is, that was after 70 years of the single most incessant atheistic regime on the planet that was literally killing millions upon millions upon millions of people. And here we are 30 years after it's collapsed. And the church is on fire. They're like never before there was a 2012 study done by the journal for the scientific study of religion. And they they had a marker of religious revivals, they had about seven different gauges for determining the level of religious revival. And they concluded there's no way around it. Christianity is on fire. So is Soviet Union falls and 1992. Only about 3019 91 Sorry, 30% of the population because themselves Christian, today. 70% does, it's hot. It's cool to be a Christian, and we're like we were talking about earlier today, what they were able to do is they were able to rediscover it and re weave Christianity into their culture into their, into their life. We they could do that with with, you know, their Russian Orthodox resources. And imagine what we can do with our evangelical resources. The way we can reawaken the church and weave it into every aspect of life. So yeah, it's gonna be dark. No question. Yeah, there's a Friday upon us. But you know what day follows Friday. Darkness of Friday is always followed by Sunday. Always. That's the Christian gospel all ways guaranteed. And when we have that faith, when we have that confidence, we cannot be intimidated. And when we cannot be intimidated, we like an axe, we begin turning the world upside down. Gary Duncan  47:40  Oh, great. I feel better already. No, that's good. That's good. I know you gotta run. One last question. We're gonna take it. See real quick. I'll edit this out. Sure. Okay. And you may have already answered but so where do you see us going from here in our culture? I know you're involved with reawakening tour. You've been involved with that. So with the church, do you? Where do you Well, let me ask you this. Where do you see England going? Now that we've got this new monarchy? That Well, Queen Elizabeth, she died? Okay. Sure. Sure. 70 years in reign? That's an interesting historical marker right there. Steve  48:35  Absolutely. Yeah. Yeah. Which? Yeah. Yeah, yeah. Cuz in many ways, she really was the last monarch of Christendom, of real genuine Christian emerge, she was, she was anointed, she was literally anointed with the Holy oil as Queen back in her and throne, their coronation 1953 It was the same coronation ceremony that goes all the way back to the 10th century to their very first king. And, and she did she I think she she held that position brilliantly, as only she could is sort of the last, you know, Elizabeth and member of the Elizabethan era, as it were, where we go from here is very difficult to see because England has again like we do they have a choice and but England? Well, no, it's going to be the same to be the same. In many respects. The choice is, are you going to continue down this futile road of flippant leftist skepticism and doubt and secularism? Or are you going to rediscover in many ways, like Her funeral was a call to do? And this by the way goes for Anglican clergy, who were probably even worse than most lay people on this woke nonsense stuff. I know I went to school with them. in Durham University, are they going to re embrace truth as understood in 2000 years of the Christian tradition unbroken? Or are they going to go the way of secular leftist liberalism? If they go back to truth that is going to be the most unifying, powerful, socially revivifying choice they could possibly make. If they go down the road of secular flippant you know, liberalism, the UK will dismember you can write that down. The UK will dismember it will fall and we're talking was we're talking first all of its abroad territories, you know, in the, in the, in the Caribbean area and so forth and Pacific and it's, it's going to dismember there and then you'll start to see the Scottish referendums come out, you're gonna start seeing United Ireland movements come out like never before, you're gonna start seeing Welsh nationalism come out, like you're gonna see English nationalism come out like never before we start breaking apart what it'll break apart and and it'll start tribal laws, and there's no way and I think we're going that's inevitably what's gonna happen with us. Because that's, we're already there. The 35 nations have been added to the world map since 1991. Since the fall of the Soviet Union, we're already there. Write a text that today is more popular than it's ever been, right, the Texas independence movement. I mean, it doesn't matter what we're looking at. Everywhere we look whether it's the breakaway republics and Donetsk and Lugansk, in the, in the Donbass region in Ukraine, whether it's Transnistria Transnistria, is a breakaway Republic from Moldova that wants to hook up back with with Russia, or Burundi and Rwanda or Slovakia and the Czech Republic, the two Sudan's, you name it everywhere we look, the world is breaking up. And the question is, what can hold it together? What holds the nation together and you said, you use the word culture, and the church being at the center of culture, we have to remember what the word is at the heart of the word culture. And that's the Latin cult cult us. We're not talking about people knocking on your doors, giving you tracks and I had to lead us to worship or something like that. Right? Right. We're talking the old old word cultist, meaning worship, the place of worship, your culture always comes out of, it flows out of the font of what you worship. And if we are going to worship, if we're gonna go back to worship God, in the way that has sustained Western civilization now, for 2000 years, we're gonna have another 2000 years, no problem, if we're gonna go and embrace this brave new world, things will break up. There's no way the brave new world of secular liberalism, gloves, and so forth is going to shatter, it's going to collapse. And we're going to break up into all of our own different regional loyalties or, or ideological loyalties or religious worlds, whatever it is, or increasing like BLM and so forth. Racial loyalties, we're going to try belies breakup and there's nothing that will stop that apart from returning to our Christian faith, the Christian faith alone will hold the UK together, the Christian faith alone will hold the United States together, I am very optimistic about the United States. I have to be honest, I'm not particularly optimistic with the UK. Gary Duncan  53:49  So the church is the glue that holds it all together, our Christian faith, and so that if we can regain our identity, which I think that's kind of the grassroots the move that that's coming along is regaining our roots in our belief, and in America, the church, and what are our power and authority is in the culture, I think you're right, if we can regain that back and take it back from the darkness, we do still have some time we are the salt and the law. And so ultimately, that's, that's our job is to, to, to push back evil, and to you know, continue to have freedom in this country. So the return of the American patriot, your documentary, that's a good place to start to getting encouraged and I'm gonna get a hold of that and watch it and get encouraged to to just stand and fight and like Ephesians six says to stand and that's what we're doing and I appreciate you very much for what you're doing and, and all the work that you're doing through YouTube and just getting the word out because every little bit counts. You know, and you're doing a big part. So I want to encourage the little guys out there as well. You know if you've got a voice to stand and fight against this, do it. But Dr. Steve Turley appreciate your time very much. And thank you so much for, for what you're doing in our culture. Steve  55:20  Thank you, Garrett. God bless you. God bless everything you're doing God bless Tennessee in that little blue pot a little blue patcher in the Nashville area. Little God bless and you are you're doing God's work and calling the church his people and particularly the leaders to embrace who we've been called to be. We are We are more than conquerors through Him who called us Yeah, Gary Duncan  55:44  very good. Thank you so much. You appreciate God bless. Appreciate it very much. It was pleasure meet you. Steve  55:51  Oh, right back at you, man. Yeah, right back. atcha Yeah, I saw I saw the probe. Transcribed by https://otter.ai 

About PetePete does many startup things at Allma. Links: Last Tweet in AWS: https://lasttweetinaws.com Twitter: https://twitter.com/petecheslock LinkedIn: https://www.linkedin.com/in/petecheslock/ TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: This episode is sponsored in part byLaunchDarkly. Take a look at what it takes to get your code into production. I'm going to just guess that it's awful because it's always awful. No one loves their deployment process. What if launching new features didn't require you to do a full-on code and possibly infrastructure deploy? What if you could test on a small subset of users and then roll it back immediately if results aren't what you expect? LaunchDarkly does exactly this. To learn more, visitlaunchdarkly.com and tell them Corey sent you, and watch for the wince.Corey: This episode is sponsored in part by our friends at Redis, the company behind the incredibly popular open source database that is not the bind DNS server. If you're tired of managing open source Redis on your own, or you're using one of the vanilla cloud caching services, these folks have you covered with the go to manage Redis service for global caching and primary database capabilities; Redis Enterprise. To learn more and deploy not only a cache but a single operational data platform for one Redis experience, visit redis.com/hero. Thats r-e-d-i-s.com/hero. And my thanks to my friends at Redis for sponsoring my ridiculous non-sense.  Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. I am joined—as is tradition, for a post re:Invent wrap up, a month or so later, once everything is time to settle—by my friend and yours, Pete Cheslock. Pete, how are you?Pete: Hi, I'm doing fantastic. New year; new me. That's what I'm going with.Corey: That's the problem. I keep hoping for that, but every time I turn around, it's still me. And you know, honestly, I wouldn't wish that on anyone.Pete: Exactly. [laugh]. I wouldn't wish you on me either. But somehow I keep coming back for this.Corey: So, in two-thousand twenty—or twenty-twenty, as the children say—re:Invent was fully virtual. And that felt weird. Then re:Invent 2021 was a hybrid event which, let's be serious here, is not really those things. They had a crappy online thing and then a differently crappy thing in person. But it didn't feel real to me because you weren't there.That is part of the re:Invent tradition. There's a midnight madness thing, there's a keynote where they announce a bunch of nonsense, and then Pete and I go and have brunch on the last day of re:Invent and decompress, and more or less talk smack about everything that crosses our minds. And you weren't there this year. I had to backfill you with Tim Banks. You know, the person that I backfield you with here at The Duckbill Group as a principal cloud economist.Pete: You know, you got a great upgrade in hot takes, I feel like, with Tim.Corey: And other ways, too, but it's rude of me to say that to you directly. So yeah, his hot takes are spectacular. He was going to be doing this with me, except you cannot mess with tradition. You really can't.Pete: Yeah. I'm trying to think how many—is this third year? It's at least three.Corey: Third or fourth.Pete: Yeah, it's at least three. Yeah, it was, I don't want to say I was sad to not be there because, with everything going on, it's still weird out there. But I am always—I'm just that weird person who actually likes re:Invent, but not for I feel like the reasons people think. Again, I'm such an extroverted-type person, that it's so great to have this, like, serendipity to re:Invent. The people that you run into and the conversations that you have, and prior—like in 2019, I think was a great example because that was the last one I had gone to—you know, having so many conversations so quickly because everyone is there, right? It's like this magnet that attracts technologists, and venture capital, and product builders, and all this other stuff. And it's all compressed into, like, you know, that five-day span, I think is the biggest part that makes so great.Corey: The fear in people's eyes when they see me. And it was fun; I had a pair of masks with me. One of them was a standard mask, and no one recognizes anyone because, masks, and the other was a printout of my ridiculous face, which was horrifyingly uncanny, but also made it very easy for people to identify me. And depending upon how social I was feeling, I would wear one or the other, and it worked flawlessly. That was worth doing. They really managed to thread the needle, as well, before Omicron hit, but after the horrors of last year. So, [unintelligible 00:03:00]—Pete: It really—Corey: —if it were going on right now, it would not be going on right now.Pete: Yeah. I talk about really—yeah—really just hitting it timing-wise. Like, not that they could have planned for any of this, but like, as things were kind of not too crazy and before they got all crazy again, it feels like wow, like, you know, they really couldn't have done the event at any other time. And it's like, purely due to luck. I mean, absolute one hundred percent.Corey: That's the amazing power of frugality. Because the reason is then is it's the week after Thanksgiving every year when everything is dirt cheap. And, you know, if there's one thing that I one-point-seve—sorry, their stock's in the toilet—a $1.6 trillion company is very concerned about, it is saving money at every opportunity.Pete: Well, the one thing that was most curious about—so I was at the first re:Invent in-what—2012 I think it was, and there was—it was quaint, right?—there was 4000 people there, I want to say. It was in the thousands of people. Now granted, still a big conference, but it was in the Sands Convention Center. It was in that giant room, the same number of people, were you know, people's booths were like tables, like, eight-by-ten tables, right? [laugh].It had almost a DevOpsDays feel to it. And I was kind of curious if this one had any of those feelings. Like, did it evoke it being more quaint and personable, or was it just as soulless as it probably has been in recent years?Corey: This was fairly soulless because they reduced the footprint of the event. They dropped from two expo halls down to one, they cut the number of venues, but they still had what felt like 20,000 people or something there. It was still crowded, it was still packed. And I've done some diligent follow-ups afterwards, and there have been very few cases of Covid that came out of it. I quarantined for a week in a hotel, so I don't come back and kill my young kids for the wrong reasons.And that went—that was sort of like the worst part of it on some level, where it's like great. Now I could sit alone at a hotel and do some catch-up and all the rest, but all right I'd kind of like to go home. I'm not used to being on the road that much.Pete: Yeah, I think we're all a little bit out of practice. You know, I haven't been on a plane in years. I mean, the travel I've done more recently has been in my car from point A to point B. Like, direct, you know, thing. Actually, a good friend of mine who's not in technology at all had to travel for business, and, you know, he also has young kids who are under five, so he when he got back, he actually hid in a room in their house and quarantine himself in the room. But they—I thought, this is kind of funny—they never told the kids he was home. Because they knew that like—Corey: So, they just thought the house was haunted?Pete: [laugh].Corey: Like, “Don't go in the west wing,” sort of level of nonsense. That is kind of amazing.Pete: Honestly, like, we were hanging out with the family because they're our neighbors. And it was like, “Oh, yeah, like, he's in the guest room right now.” Kids have no idea. [laugh]. I'm like, “Oh, my God.” I'm like, I can't even imagine. Yeah.Corey: So, let's talk a little bit about the releases of re:Invent. And I'm going to lead up with something that may seem uncharitable, but I don't think it necessarily is. There weren't the usual torrent of new releases for ridiculous nonsense in the same way that there have been previously. There was no, this service talks to satellites in space. I mean, sure, there was some IoT stuff to manage fleets of cars, and giant piles of robots, and cool, I don't have those particular problems; I'm trying to run a website over here.So okay, great. There were enhancements to a number of different services that were in many cases appreciated, in other cases, irrelevant. Werner said in his keynote, that it was about focusing on primitives this year. And, “Why do we have so many services? It's because you asked for it… as customers.”Pete: [laugh]. Yeah, you asked for it.Corey: What have you been asking for, Pete? Because I know what I've been asking for and it wasn't that. [laugh].Pete: It's amazing to see a company continually say yes to everything, and somehow, despite their best efforts, be successful at doing it. No other company could do that. Imagine any other software technology business out there that just builds everything the customers ask for. Like from a product management business standpoint, that is, like, rule 101 is, “Listen to your customers, but don't say yes to everything.” Like, you can't do everything.Corey: Most companies can't navigate the transition between offering the same software in the Cloud and on a customer facility. So, it's like, “Ooh, an on-prem version, I don't know, that almost broke the company the last time we tried it.” Whereas you have Amazon whose product strategy is, “Yes,” being able to put together a whole bunch of things. I also will challenge the assertion that it's the primitives that customers want. They don't want to build a data center out of popsicle sticks themselves. They want to get something that solves a problem.And this has been a long-term realization for me. I used to work at Media Temple as a senior systems engineer running WordPress at extremely large scale. My websites now run on WordPress, and I have the good sense to pay WP Engine to handle it for me, instead of doing it myself because it's not the most productive use of my time. I want things higher up the stack. I assure you I pay more to WP Engine than it would cost me to run these things myself from an infrastructure point of view, but not in terms of my time.What I see sometimes as the worst of all worlds is that AWS is trying to charge for that value-added pricing without adding the value that goes along with it because you still got to build a lot of this stuff yourself. It's still a very janky experience, you're reduced to googling random blog posts to figure out how this thing is supposed to work, and the best documentation comes from externally. Whereas with a company that's built around offering solutions like this, great. In the fullness of time, I really suspect that if this doesn't change, their customers are going to just be those people who build solutions out of these things. And let those companies capture the up-the-stack margin. Which I have no problem with. But they do because Amazon is a company that lies awake at night actively worrying that someone, somewhere, who isn't them might possibly be making money somehow.Pete: I think MongoDB is a perfect example of—like, look at their stock price over the last whatever, years. Like, they, I feel like everyone called for the death of MongoDB every time Amazon came out with their new things, yet, they're still a multi-billion dollar company because I can just—give me an API endpoint and you scale the database. There's is—Corey: Look at all the high-profile hires that Mongo was making out of AWS, and I can't shake the feeling they're sitting there going, “Yeah, who's losing important things out of production now?” It's, everyone is exodus-ing there. I did one of those ridiculous graphics of the naming all the people that went over there, and in—with the hurricane evacuation traffic picture, and there's one car going the other way that I just labeled with, “Re:Invent sponsorship check,” because yeah, they have a top tier sponsorship and it was great. I've got to say I've been pretty down on MongoDB for a while, for a variety of excellent reasons based upon, more or less, how they treated customers who were in pain. And I'd mostly written it off.I don't do that anymore. Not because I inherently believe the technology has changed, though I'm told it has, but by the number of people who I deeply respect who are going over there and telling me, no, no, this is good. Congratulations. I have often said you cannot buy authenticity, and I don't think that they are, but the people who are working there, I do not believe that these people are, “Yeah, well, you bought my opinion. You can buy their attention, not their opinion.” If someone changes their opinion, based upon where they work, I kind of question everything they're telling me is, like, “Oh, you're just here to sell something you don't believe in? Welcome aboard.”Pete: Right. Yeah, there's an interview question I like to ask, which is, “What's something that you used to believe in very strongly that you've more recently changed your mind on?” And out of politeness because usually throws people back a little bit, and they're like, “Oh, wow. Like, let me think about that.” And I'm like, “Okay, while you think about that I want to give you mine.”Which is in the past, my strongly held belief was we had to run everything ourselves. “You own your availability,” was the line. “No, I'm not buying Datadog. I can build my own metric stack just fine, thank you very much.” Like, “No, I'm not going to use these outsourced load balancers or databases because I need to own my availability.”And what I realized is that all of those decisions lead to actually delivering and focusing on things that were not the core product. And so now, like, I've really flipped 180, that, if any—anything that you're building that does not directly relate to the core product, i.e. How your business makes money, should one hundred percent be outsourced to an expert that is better than you. Mongo knows how to run Mongo better than you.Corey: “What does your company do?” “Oh, we handle expense reports.” “Oh, what are you working on this month?” “I'm building a load balancer.” It's like that doesn't add the value. Don't do that.Pete: Right. Exactly. And so it's so interesting, I think, to hear Werner say that, you know, we're just building primitives, and you asked for this. And I think that concept maybe would work years ago, when you had a lot of builders who needed tools, but I don't think we have any, like, we don't have as many builders as before. Like, I think we have people who need more complete solutions. And that's probably why all these businesses are being super successful against Amazon.Corey: I'm wondering if it comes down to a cloud economic story, specifically that my cloud bill is always going to be variable and it's difficult to predict, whereas if I just use EC2 instances, and I build load balancers or whatnot, myself, well, yeah, it's a lot more work, but I can predict accurately what my staff compensation costs are more effectively, that I can predict what a CapEx charge would be or what the AWS bill is going to be. I'm wondering if that might in some way shape it?Pete: Well, I feel like the how people get better in managing their costs, right, you'll eventually move to a world where, like, “Yep, okay, first, we turned off waste,” right? Like, step one is waste. Step two is, like, understanding your spend better to optimize but, like, step three, like, the galaxy brain meme of Amazon cost stuff is all, like, unit economics stuff, where trying to better understand the actual cost deliver an actual feature. And yeah, I think that actually gets really hard when you give—kind of spread your product across, like, a slew of services that have varying levels of costs, varying levels of tagging, so you can attribute it. Like, it's really hard. Honestly, it's pretty easy if I have 1000 EC2 servers with very specific tags, I can very easily figure out what it costs to deliver product. But if I have—Corey: Yeah, if I have Corey build it, I know what Corey is going to cost, and I know how many servers he's going to use. Great, if I have Pete it, Pete's good at things, it'll cut that server bill in half because he actually knows how to wind up being efficient with things. Okay, great. You can start calculating things out that way. I don't think that's an intentional choice that companies are making, but I feel like that might be a natural outgrowth of it.Pete: Yeah. And there's still I think a lot of the, like, old school mentality of, like, the, “Not invented here,” the, “We have to own our availability.” You can still own your availability by using these other vendors. And honestly, it's really heartening to see so many companies realize that and realize that I don't need to get everything from Amazon. And honestly, like, in some things, like I look at a cloud Amazon bill, and I think to myself, it would be easier if you just did everything from Amazon versus having these ten other vendors, but those ten other vendors are going to be a lot better at running the product that they build, right, that as a service, then you probably will be running it yourself. Or even Amazon's, like, you know, interpretation of that product.Corey: A few other things that came out that I thought were interesting, at least the direction they're going in. The changes to S3 intelligent tiering are great, with instant retrieval on Glacier. I feel like that honestly was—they talk a good story, but I feel like that was competitive response to Google offering the same thing. That smacks of a large company with its use case saying, “You got two choices here.” And they're like, “Well, okay. Crap. We're going to build it then.”Or alternately, they're looking at the changes that they're making to intelligent tiering, they're now shifting that to being the default that as far as recommendations go. There are a couple of drawbacks to it, but not many, and it's getting easier now to not have the mental overhead of trying to figure out exactly what your lifecycle policies are. Yeah, there are some corner cases where, okay, if I adjust this just so, then I could save 10% on that monitoring fee or whatnot. Yeah, but look how much work that's going to take you to curate and make sure that you're not doing something silly. That feels like it is such an in the margins issue. It's like, “How much data you're storing?” “Four exabytes.” Okay, yeah. You probably want some people doing exactly that, but that's not most of us.Pete: Right. Well, there's absolutely savings to be had. Like, if I had an exabyte of data on S3—which there are a lot of people who have that level of data—then it would make sense for me to have an engineering team whose sole purpose is purely an optimizing our data lifecycle for that data. Until a point, right? Until you've optimized the 80%, basically. You optimize the first 80, that's probably, air-quote, “Easy.” The last 20 is going to be incredibly hard, maybe you never even do that.But at lower levels of scale, I don't think the economics actually work out to have a team managing your data lifecycle of S3. But the fact that now AWS can largely do it for you in the background—now, there's so many things you have to think about and, like, you know, understand even what your data is there because, like, not all data is the same. And since S3 is basically like a big giant database you can query, you got to really think about some of that stuff. But honestly, what I—I don't know if—I have no idea if this is even be worked on, but what I would love to see—you know, hashtag #AWSwishlist—is, now we have countless tiers of EBS volumes, EBS volumes that can be dynamically modified without touching, you know, the physical host. Meaning with an API call, you can change from the gp2 to gp3, or io whatever, right?Corey: Or back again if it doesn't pan out.Pete: Or back again, right? And so for companies with large amounts of spend, you know, economics makes sense that you should have a team that is analyzing your volumes usage and modifying that daily, right? Like, you could modify that daily, and I don't know if there's anyone out there that's actually doing it at that level. And they probably should. Like, if you got millions of dollars in EBS, like, there's legit savings that you're probably leaving on the table without doing that. But that's what I'm waiting for Amazon to do for me, right? I want intelligent tiering for EBS because if you're telling me I can API call and you'll move my data and make that better, make that [crosstalk 00:17:46] better [crosstalk 00:17:47]—Corey: Yeah it could be like their auto-scaling for DynamoDB, for example. Gives you the capacity you need 20 minutes after you needed it. But fine, whatever because if I can schedule stuff like that, great, I know what time of day, the runs are going to kick off that beat up the disks. I know when end-of-month reporting fires off. I know what my usage pattern is going to be, by and large.Yeah, part of the problem too, is that I look at this stuff, and I get excited about it with the intelligent tiering… at The Duckbill Group we've got a few hundred S3 buckets lurking around. I'm thinking, “All right, I've got to go through and do some changes on this and implement all of that.” Our S3 bill's something like 50 bucks a month or something ridiculous like that. It's a no, that really isn't a thing. Like, I have a screenshot bucket that I have an app installed—I think called Dropshare—that hooks up to anytime I drag—I hit a shortcut, I drag with the mouse to select whatever I want and boom, it's up there and the URL is not copied to my clipboard, I can paste that wherever I want.And I'm thinking like, yeah, there's no cleanup on that. There's no lifecycle policy that's turning into anything. I should really go back and age some of it out and do the rest and start doing some lifecycle management. It—I've been using this thing for years and I think it's now a whopping, what, 20 cents a month for that bucket. It's—I just don't—Pete: [laugh].Corey: —I just don't care, other than voice in the back of my mind, “That's an unbounded growth problem.” Cool. When it hits 20 bucks a month, then I'll consider it. But until then I just don't. It does not matter.Pete: Yeah, I think yeah, scale changes everything. Start adding some zeros and percentages turned into meaningful numbers. And honestly, back on the EBS thing, the one thing that really changed my perspective of EBS, in general, is—especially coming from the early days, right? One terabyte volume, it was a hard drive in a thing. It was a virtual LUN on a SAN somewhere, probably.Nowadays, and even, like, many years after those original EBS volumes, like all the limits you get in EBS, those are actually artificial limits, right? If you're like, “My EBS volume is too slow,” it's not because, like, the hard drive it's on is too slow. That's an artificial limit that is likely put in place due to your volume choice. And so, like, once you realize that in your head, then your concept of how you store data on EBS should change dramatically.Corey: Oh, AWS had a blog post recently talking about, like, with io2 and the limits and everything, and there was architecture thinking, okay. “So, let's say this is insufficient and the quarter-million IOPS a second that you're able to get is not there.” And I'm sitting there thinking, “That is just ludicrous data volume and data interactivity model.” And it's one of those, like, I'm sitting here trying to think about, like, I haven't had to deal with a problem like that decade, just because it's, “Huh. Turns out getting these one thing that's super fast is kind of expensive.” If you paralyze it out, that's usually the right answer, and that's how the internet is mostly evolved. But there are use cases for which that doesn't work, and I'm excited to see it. I don't want to pay for it in my view, but it's nice to see it.Pete: Yeah, it's kind of fun to go into the Amazon calculator and price out one of the, like, io2 volumes and, like, maxed out. It's like, I don't know, like $50,000 a month or a hun—like, it's some just absolutely absurd number. But the beauty of it is that if you needed that value for an hour to run some intensive data processing task, you can have it for an hour and then just kill it when you're done, right? Like, that is what is most impressive.Corey: I copied 130 gigs of data to an EFS volume, which was—[unintelligible 00:21:05] EFS has gone from “This is a piece of junk,” to one of my favorite services. It really is, just because of its utility and different ways of doing things. I didn't have the foresight, just use a second EFS volume for this. So, I was unzipping a whole bunch of small files onto it. Great.It took a long time for me to go through it. All right, now that I'm done with that I want to clean all this up. My answer was to ultimately spin up a compute node and wind up running a whole bunch of—like, 400, simultaneous rm-rf on that long thing. And it was just, like, this feels foolish and dumb, but here we are. And I'm looking at the stats on it because the instance was—all right, at that point, the load average [on the instance 00:21:41] was like 200, or something like that, and the EFS volume was like, “Ohh, wow, you're really churning on this. I'm now at, like, 5% of the limit.” Like, okay, great. It turns out I'm really bad at computers.Pete: Yeah, well, that's really the trick is, like, yeah, sure, you can have a quarter-million IOPS per second, but, like, what's going to break before you even hit that limit? Probably many other things.Corey: Oh, yeah. Like, feels like on some level if something gets to that point, it a misconfiguration somewhere. But honestly, that's the thing I find weirdest about the world in which we live is that at a small-scale—if I have a bill in my $5 a month shitposting account, great. If I screw something up and cost myself a couple hundred bucks in misconfiguration it's going to stand out. At large scale, it doesn't matter if—you're spending $50 million a year or $500 million a year on AWS and someone leaks your creds, and someone spins up a whole bunch of Bitcoin miners somewhere else, you're going to see that on your bill until they're mining basically all the Bitcoin. It just gets lost in the background.Pete: I'm waiting for those—I'm actually waiting for the next level of them to get smarter because maybe you have, like, an aggressive tagging system and you're monitoring for untagged instances, but the move here would be, first get the creds and query for, like, the most used tags and start applying those tags to your Bitcoin mining instances. My God, it'll take—Corey: Just clone a bunch of tags. Congratulations, you now have a second BI Elasticsearch cluster that you're running yourself. Good work.Pete: Yeah. Yeah, that people won't find that until someone comes along after the fact that. Like, “Why do we have two have these things?” And you're like—[laugh].Corey: “Must be a DR thing.”Pete: It's maxed-out CPU. Yeah, exactly.Corey: [laugh].Pete: Oh, the terrible ideas—please, please, hackers don't take are terrible ideas.Corey: I had a, kind of, whole thing I did on Twitter years ago, talking about how I would wind up using the AWS Marketplace for an embezzlement scheme. Namely, I would just wind up spinning up something that had, like, a five-cent an hour charge or whatnot on just, like, basically rebadge the CentOS Community AMI or whatnot. Great. And then write a blog post, not attached to me, that explains how to do a thing that I'm going to be doing in production in a week or two anyway. Like, “How to build an auto-scaling group,” and reference that AMI.Then if it ever comes out, like, “Wow, why are we having all these marketplace charges on this?” “I just followed the blog post like it said here.” And it's like, “Oh, okay. You're a dumbass. The end.”That's the way to do it. A month goes by and suddenly it came out that someone had done something similarly. They wound up rebadging these community things on the marketplace and charging big money for it, and I'm sitting there going like that was a joke. It wasn't a how-to. But yeah, every time I make these jokes, I worry someone's going to do it.Pete: “Welcome to large-scale fraud with Corey Quinn.”Corey: Oh, yeah, it's fraud at scale is really the important thing here.Corey: This episode is sponsored by our friends at Oracle HeatWave is a new high-performance accelerator for the Oracle MySQL Database Service. Although I insist on calling it “my squirrel.” While MySQL has long been the worlds most popular open source database, shifting from transacting to analytics required way too much overhead and, ya know, work. With HeatWave you can run your OLTP and OLAP, don't ask me to ever say those acronyms again, workloads directly from your MySQL database and eliminate the time consuming data movement and integration work, while also performing 1100X faster than Amazon Aurora, and 2.5X faster than Amazon Redshift, at a third of the cost. My thanks again to Oracle Cloud for sponsoring this ridiculous nonsense.Corey: I still remember a year ago now at re:Invent 2021 was it, or was it 2020? Whatever they came out with, I want to say it wasn't gp3, or maybe it was, regardless, there was a new EBS volume type that came out that you were playing with to see how it worked and you experimented with it—Pete: Oh, yes.Corey: —and the next morning, you looked at the—I checked Slack and you're like well, my experiments yesterday cost us $5,000. And at first, like, the—my response is instructive on this because, first, it was, “Oh, my God. What's going to happen now?” And it's like, first, hang on a second.First off, that seems suspect but assume it's real. I assumed it was real at the outset. It's “Oh, right. This is not my personal $5-a-month toybox account. We are a company; we can absolutely pay that.” Because it's like, I could absolutely reach out, call it a favor. “I made a mistake, and I need a favor on the bill, please,” to AWS.And I would never live it down, let's be clear. For a $7,000 mistake, I would almost certainly eat it. As opposed to having to prostrate myself like that in front of Amazon. I'm like, no, no, no. I want one of those like—if it's like, “Okay, you're going to, like, set back the company roadmap by six months if you have to pay this. Do you want to do it?” Like, [groans] “Fine, I'll eat some crow.”But okay. And then followed immediately by, wow, if Pete of all people can mess this up, customers are going to be doomed here. We should figure out what happened. And I'm doing the math. Like, Pete, “What did you actually do?” And you're sitting there and you're saying, “Well, I had like a 20 gig volume that I did this.” And I'm doing the numbers, and it's like—Pete: Something's wrong.Corey: “How sure are you when you say ‘gigabyte,' that you were—that actually means what you think it did? Like, were you off by a lot? Like, did you mean exabytes?” Like, what's the deal here?Pete: Like, multiple factors.Corey: Yeah. How much—“How many IOPS did you give that thing, buddy?” And it turned out what happened was that when they launched this, they had mispriced it in the system by a factor of a million. So, it was fun. I think by the end of it, all of your experimentation was somewhere between five to seven cents. Which—Pete: Yeah. It was a—Corey: Which is why you don't work here anymore because no one cost me seven cents of money to give to Amazon—Pete: How dare you?Corey: —on my watch. Get out.Pete: How dare you, sir?Corey: Exactly.Pete: Yeah, that [laugh] was amazing to see, as someone who has done—definitely maid screw-ups that have cost real money—you know, S3 list requests are always a fun one at scale—but that one was supremely fun to see the—Corey: That was a scary one because another one they'd done previously was they had messed up Lightsail pricing, where people would log in, and, like, “Okay, so what is my Lightsail instance going to cost?” And I swear to you, this is true, it was saying—this was back in 2017 or so—the answer was, like, “$4.3 billion.” Because when you see that you just start laughing because you know it's a mistake. You know, that they're not going to actually demand that you spend $4.3 billion for a single instance—unless it's running SAP—and great.It's just, it's a laugh. It's clearly a mispriced, and it's clearly a bug that's going to get—it's going to get fixed. I just spun up this new EBS volume that no one fully understands yet and it cost me thousands of dollars. That's the sort of thing that no, no, I could actually see that happening. There are instances now that cost something like 100 bucks an hour or whatnot to run. I can see spinning up the wrong thing by mistake and getting bitten by it. There's a bunch of fun configuration mistakes you can make that will, “Hee, hee, hee. Why can I see that bill spike from orbit?” And that's the scary thing.Pete: Well, it's the original CI and CD problem of the per-hour billing, right? That was super common of, like, yeah, like, an i3, you know, 16XL server is pretty cheap per hour, but if you're charged per hour and you spin up a bunch for five minutes. Like, it—you will be shocked [laugh] by what you see there. So—Corey: Yeah. Mistakes will show. And I get it. It's also people as individuals are very different psychologically than companies are. With companies it's one of those, “Great we're optimizing to bring in more revenue and we don't really care about saving money at all costs.”Whereas people generally have something that looks a lot like a fixed income in the form of a salary or whatnot, so it's it is easier for us to cut spend than it is for us to go out and make more money. Like, I don't want to get a second job, or pitch my boss on stuff, and yeah. So, all and all, routing out the rest of what happened at re:Invent, they—this is the problem is that they have a bunch of minor things like SageMaker Inference Recommender. Yeah, I don't care. Anything—Pete: [laugh].Corey: —[crosstalk 00:28:47] SageMaker I mostly tend to ignore, for safety. I did like the way they described Amplify Studio because they made it sound like a WYSIWYG drag and drop, build a React app. It's not it. It basically—you can do that in Figma and then it can hook it up to some things in some cases. It's not what I want it to be, which is Honeycode, except good. But we'll get there some year. Maybe.Pete: There's a lot of stuff that was—you know, it's the classic, like, preview, which sure, like, from a product standpoint, it's great. You know, they have a level of scale where they can say, “Here's this thing we're building,” which could be just a twinkle in a product managers, call it preview, and get thousands of people who would be happy to test it out and give you feedback, and it's a, it's great that you have that capability. But I often look at so much stuff and, like, that's really cool, but, like, can I, can I have it now? Right? Like—or you can't even get into the preview plan, even though, like, you have that specific problem. And it's largely just because either, like, your scale isn't big enough, or you don't have a good enough relationship with your account manager, or I don't know, countless other reasons.Corey: The thing that really throws me, too, is the pre-announcements that come a year or so in advance, like, the Outpost smaller ones are finally available, but it feels like when they do too many pre-announcements or no big marquee service announcements, as much as they talk about, “We're getting back to fundamentals,” no, you have a bunch of teams that blew the deadline. That's really what it is; let's not call it anything else. Another one that I think is causing trouble for folks—I'm fortunate in that I don't do much work with Oracle databases, or Microsoft SQL databases—but they extended RDS Custom to Microsoft SQL at the [unintelligible 00:30:27] SQL server at re:Invent this year, which means this comes down to things I actually use, we're going to have a problem because historically, the lesson has always been if I want to run my own databases and tweak everything, I do it on top of an EC2 instance. If I want to managed database, relational database service, great, I use RDS. RDS Custom basically gives you root into the RDS instance. Which means among other things, yes, you can now use RDS to run containers.But it lets you do a lot of things that are right in between. So, how do you position this? When should I use RDS Custom? Can you give me an easy answer to that question? And they used a lot of words to say, no, they cannot. It's basically completely blowing apart the messaging and positioning of both of those services in some unfortunate ways. We'll learn as we go.Pete: Yeah. Honestly, it's like why, like, why would I use this? Or how would I use this? And this is I think, fundamentally, what's hard when you just say yes to everything. It's like, they in many cases, I don't think, like, I don't want to say they don't understand why they're doing this, but if it's not like there's a visionary who's like, this fits into this multi-year roadmap.That roadmap is largely—if that roadmap is largely generated by the customers asking for it, then it's not like, oh, we're building towards this Northstar of RDS being whatever. You might say that, but your roadmap's probably getting moved all over the place because, you know, this company that pays you a billion dollars a year is saying, “I would give you $2 billion a year for all of my Oracle databases, but I need this specific thing.” I can't imagine a scenario that they would say, “Oh, well, we're building towards this Northstar, and that's not on the way there.” Right? They'd be like, “New Northstar. Another billion dollars, please.”Corey: Yep. Probably the worst release of re:Invent, from my perspective, is RUM, Real User Monitoring, for CloudWatch. And I, to be clear, I wrote a shitposting Twitter threading client called Last Tweet in AWS. Go to lasttweetinaws.com. You can all use it. It's free; I just built this for my own purposes. And I've instrumented it with RUM. Now, Real User Monitoring is something that a lot of monitoring vendors use, and also CloudWatch now. And what that is, is it embeds a listener into the JavaScript that runs on client load, and it winds up looking at what's going on loading times, et cetera, so you can see when users are unhappy. I have no problem with this. Other than that, you know, liking users? What's up with that?Pete: Crazy.Corey: But then, okay, now, what this does is unlike every other RUM tool out there, which charges per session, meaning I am going to be… doing a web page load, it charges per data item, which includes HTTP errors, or JavaScript errors, et cetera. Which means that if you have a high transaction volume site and suddenly your CDN takes a nap like Fastly did for an hour last year, suddenly your bill is stratospheric for this because errors abound and cascade, and you can have thousands of errors on a single page load for these things, and it is going to be visible from orbit, at least with a per session basis thing, when you start to go viral, you understand that, “Okay, this is probably going to cost me some more on these things, and oops, I guess I should write less compelling content.” Fine. This is one of those one misconfiguration away and you are wailing and gnashing teeth. Now, this is a new service. I believe that they will waive these surprise bills in the event that things like that happen. But it's going to take a while and you're going to be worrying the whole time if you've rolled this out naively. So it's—Pete: Well and—Corey: —I just don't like the pricing.Pete: —how many people will actively avoid that service, right? And honestly, choose a competitor because the competitor could be—the competitor could be five times more expensive, right, on face value, but it's the certainty of it. It's the uncertainty of what Amazon will charge you. Like, no one wants a surprise bill. “Well, a vendor is saying that they'll give us this contract for $10,000. I'm going to pay $10,000, even though RUM might be a fraction of that price.”It's honestly, a lot of these, like, product analytics tools and monitoring tools, you'll often see they price be a, like, you know, MAU, Monthly Active User, you know, or some sort of user-based pricing, like, the number of people coming to your site. You know, and I feel like at least then, if you are trying to optimize for lots of users on your site, and more users means more revenue, then you know, if your spend is going up, but your revenue is also going up, that's a win-win. But if it's like someone—you know, your third-party vendor dies and you're spewing out errors, or someone, you know, upgraded something and it spews out errors. That no one would normally see; that's the thing. Like, unless you're popping open that JavaScript console, you're not seeing any of those errors, yet somehow it's like directly impacting your bottom line? Like that doesn't feel [crosstalk 00:35:06].Corey: Well, there is something vaguely Machiavellian about that. Like, “How do I get my developers to care about errors on consoles?” Like, how about we make it extortionately expensive for them not to. It's, “Oh, all right, then. Here we go.”Pete: And then talk about now you're in a scenario where you're working on things that don't directly impact the product. You're basically just sweeping up the floor and then trying to remove errors that maybe don't actually affect it and they're not actually an error.Corey: Yeah. I really do wonder what the right answer is going to be. We'll find out. Again, we live, we learn. But it's also, how long does it take a service that has bad pricing at launch, or an unfortunate story around it to outrun that reputation?People are still scared of Glacier because of its original restore pricing, which was non-deterministic for any sensible human being, and in some cases lead to I'm used to spending 20 to 30 bucks a month on this. Why was I just charged two grand?Pete: Right.Corey: Scare people like that, they don't come back.Pete: I'm trying to actually remember which service it is that basically gave you an estimate, right? Like, turn it on for a month, and it would give you an estimate of how much this was going to cost you when billing started.Corey: It was either Detective or GuardDuty.Pete: Yeah, it was—yeah, that's exactly right. It was one of those two. And honestly, that was unbelievably refreshing to see. You know, like, listen, you have the data, Amazon. You know what this is going to cost me, so when I, like, don't make me spend all this time to go and figure out the cost. If you have all this data already, just tell me, right?And if I look at it and go, “Yeah, wow. Like, turning this on in my environment is going to cost me X dollars. Like, yeah, that's a trade-off I want to make, I'll spend that.” But you know, with some of the—and that—a little bit of a worry on some of the intelligent tiering on S3 is that the recommendation is likely going to be everything goes to intelligent tiering first, right? It's the gp3 story. Put everything on gp3, then move it to the proper volume, move it to an sc or an st or an io. Like, gp3 is where you start. And I wonder if that's going to be [crosstalk 00:37:08].Corey: Except I went through a wizard yesterday to launch an EC2 instance and its default on the free tier gp2.Pete: Yeah. Interesting.Corey: Which does not thrill me. I also still don't understand for the life of me why in some regions, the free tier is a t2 instance, when t3 is available.Pete: They're uh… my guess is that they've got some free t—they got a bunch of t2s lying around. [laugh].Corey: Well, one of the most notable announcements at re:Invent that most people didn't pay attention to is their ability now to run legacy instance types on top of Nitro, which really speaks to what's going on behind the scenes of we can get rid of all that old hardware and emulate the old m1 on modern equipment. So, because—you can still have that legacy, ancient instance, but now you're going—now we're able to wind up greening our data centers, which is part of their big sustainability push, with their ‘Sustainability Pillar' for the well-architected framework. They're talking more about what the green choices in cloud are. Which is super handy, not just because of the economic impact because we could use this pretty directly to reverse engineer their various margins on a per-service or per-offering basis. Which I'm not sure they're aware of yet, but oh, they're going to be.And that really winds up being a win for the planet, obviously, but also something that is—that I guess puts a little bit of choice on customers. The challenge I've got is, with my serverless stuff that I build out, if I spend—the Google search I make to figure out what the most economic, most sustainable way to do that is, is going to have a bigger carbon impact on the app itself. That seems to be something that is important at scale, but if you're not at scale, it's one of those, don't worry about it. Because let's face it, the cloud providers—all of them—are going to have a better sustainability story than you are running this in your own data centers, or on a Raspberry Pi that's always plugged into the wall.Pete: Yeah, I mean, you got to remember, Amazon builds their own power plants to power their data centers. Like, that's the level they play, right? There, their economies of scale are so entirely—they're so entirely different than anything that you could possibly even imagine. So, it's something that, like, I'm sure people will want to choose for. But, you know, if I would honestly say, like, if we really cared about our computing costs and the carbon footprint of it, I would love to actually know the carbon footprint of all of the JavaScript trackers that when I go to various news sites, and it loads, you know, the whatever thousands of trackers and tracking the all over, like, what is the carbon impact of some of those choices that I actually could control, like, as a either a consumer or business person?Corey: I really hope that it turns into something that makes a meaningful difference, and it's not just greenwashing. But we'll see. In the fullness of time, we're going to figure that out. Oh, they're also launching some mainframe stuff. They—like that's great.Pete: Yeah, those are still a thing.Corey: I don't deal with a lot of customers that are doing things with that in any meaningful sense. There is no AWS/400, so all right.Pete: [laugh]. Yeah, I think honestly, like, I did talk to a friend of mine who's in a big old enterprise and has a mainframe, and they're actually replacing their mainframe with Lambda. Like they're peeling off—which is, like, a great move—taking the monolith, right, and peeling off the individual components of what it can do into these discrete Lambda functions. Which I thought was really fascinating. Again, it's a five-year-long journey to do something like that. And not everyone wants to wait five years, especially if their support's about to run out for that giant box in the, you know, giant warehouse.Corey: The thing that I also noticed—and this is probably the—I guess, one of the—talk about swing and a miss on pricing—they have a—what is it?—there's a VPC IP Address Manager, which tracks the the IP addresses assigned to your VPCs that are allocated versus not, and it's 20 cents a month per IP address. It's like, “Okay. So, you're competing against a Google Sheet or an Excel spreadsheet”—which is what people are using for these things now—“Only you're making it extortionately expensive?”Pete: What kind of value does that provide for 20—I mean, like, again—Corey: I think Infoblox or someone like that offers it where they become more cost-effective as soon as you hit 500 IP addresses. And it's just—like, this is what I'm talking about. I know it does not cost AWS that kind of money to store an IP address. You can store that in a Route 53 TXT record for less money, for God's sake. And that's one of those, like, “Ah, we could extract some value pricing here.”Like, I don't know if it's a good product or not. Given its pricing, I don't give a shit because it's going to be too expensive for anything beyond trivial usage. So, it's a swing and a miss from that perspective. It's just, looking at that, I laugh, and I don't look at it again.Pete: See I feel—Corey: I'm not usually price sensitive. I want to be clear on that. It's just, that is just Looney Tunes, clown shoes pricing.Pete: Yeah. It's honestly, like, in many cases, I think the thing that I have seen, you know, in the past few years is, in many cases, it can honestly feel like Amazon is nickel-and-diming their customers in so many ways. You know, the explosion of making it easy to create multiple Amazon accounts has a direct impact to waste in the cloud because there's a lot of stuff you have to have her account. And the more accounts you have, those costs grow exponentially as you have these different places. Like, you kind of lose out on the economies of scale when you have a smaller number of accounts.And yeah, it's hard to optimize for that. Like, if you're trying to reduce your spend, it's challenging to say, “Well, by making a change here, we'll save, you know, $10,000 in this account.” “That doesn't seem like a lot when we're spending millions.” “Well, hold on a second. You'll save $10,000 per account, and you have 500 accounts,” or, “You have 1000 accounts,” or something like that.Or almost cost avoidance of this cost is growing unbounded in all of your accounts. It's tiny right now. So, like, now would be the time you want to do something with it. But like, again, for a lot of companies that have adopted the practice of endless Amazon accounts, they've almost gone, like, it's the classic, like, you know, I've got 8000 GitHub repositories for my source code. Like, that feels just as bad as having one GitHub repository for your repo. I don't know what the balance is there, but anytime these different types of services come out, it feels like, “Oh, wow. Like, I'm going to get nickeled and dimed for it.”Corey: This ties into the re:Post launch, which is a rebranding of their forums, where, okay, great, it was a little crufty and it need modernize, but it still ties your identity to an IAM account, or the root email address for an Amazon account, which is great. This is completely worthless because as soon as I change jobs, I lose my identity, my history, the rest, on this forum. I'm not using it. It shows that there's a lack of awareness that everyone is going to have multiple accounts with which they interact, and that people are going to deal with the platform longer than any individual account will. It's just a continual swing and a miss on things like that.And it gets back to the billing question of, “Okay. When I spin up an account, do I want them to just continue billing me—because don't turn this off; this is important—or do I want there to be a hard boundary where if you're about to charge me, turn it off. Turn off the thing that's about to cost me money.” And people hem and haw like this is an insurmountable problem, but I think the way to solve it is, let me specify that intent when I provision the account. Where it's, “This is a production account for a bank. I really don't want you turning it off.” Versus, “I'm a student learner who thinks that a Managed NAT Gateway might be a good thing. Yeah, I want you to turn off my demo Hello World app that will teach me what's going on, rather than surprising me with a five-figure bill at the end of the month.”Pete: Yeah. It shouldn't be that hard. I mean, but again, I guess everything's hard at scale.Corey: Oh, yeah. Oh yeah.Pete: But still, I feel like every time I log into Cost Explorer and I look at—and this is years it's still not fixed. Not that it's even possible to fix—but on the first day of the month, you look at Cost Explorer, and look at what Amazon is estimating your monthly bill is going to be. It's like because of your, you know—Corey: Your support fees, and your RI purchases, and savings plans purchases.Pete: [laugh]. All those things happened, right? First of the month, and it's like, yeah, “Your bill's going to be $800,000 this year.” And it's like, “Shouldn't be, like, $1,000?” Like, you know, it's the little things like that, that always—Corey: The one-off charges, like, “Oh, your Route 53 zone,” and all the stuff that gets charged on a monthly cadence, which fine, whatever. I mean, I'm okay with it, but it's also the, like, be careful when that happen—I feel like there's a way to make that user experience less jarring.Pete: Yeah because that problem—I mean, in my scenario, companies that I've worked at, there's been multiple times that a non-technical person will look at that data and go into immediate freakout mode, right? And that's never something that you want to have happen because now that's just adding a lot of stress and anxiety into a company that is—with inaccurate data. Like, the data—like, the answer you're giving someone is just wrong. Perhaps you shouldn't even give it to them if it's that wrong. [laugh].Corey: Yeah, I'm looking forward to seeing what happens this coming year. We're already seeing promising stuff. They—give people a timeline on how long in advance these things record—late last night, AWS released a new console experience. When you log into the AWS console now, there's a new beta thing. And I gave it some grief on Twitter because I'm still me, but like the direction it's going. It lets you customize your view with widgets and whatnot.And until they start selling widgets on marketplace or having sponsored widgets, you can't remove I like it, which is no guarantee at some point. But it shows things like, I can move the cost stuff, I can move the outage stuff up around, I can have the things that are going on in my account—but who I am means I can shift this around. If I'm a finance manager, cool. I can remove all the stuff that's like, “Hey, you want to get started spinning up an EC2 instance?” “Absolutely not. Do I want to get told, like, how to get certified? Probably not. Do I want to know what the current bill is and whether—and my list of favorites that I've pinned, whatever services there? Yeah, absolutely do.” This is starting to get there.Pete: Yeah, I wonder if it really is a way to start almost hedging on organizations having a wider group of people accessing AWS. I mean, in previous companies, I absolutely gave access to the console for tools like QuickSight, for tools like Athena, for the DataBrew stuff, the Glue DataBrew. Giving, you know, non-technical people access to be able to do these, like, you know, UI ETL tasks, you know, a wider group of a company is getting access into Amazon. So, I think anything that Amazon does to improve that experience for, you know, the non-SREs, like the people who would traditionally log in, like, that is an investment definitely worth making.Corey: “Well, what could non-engineering types possibly be doing in the AWS console?” “I don't know, jackhole, maybe paying the bill? Just a thought here.” It's the, there are people who look at these things from a variety of different places, and you have such sprawl in the AWS world that there are different personas by a landslide. If I'm building Twitter for Pets, you probably don't want to be pitching your mainframe migration services to me the same way that you would if I were a 200-year-old insurance company.Pete: Yeah, exactly. And the number of those products are going to grow, the number of personas are going to grow, and, yeah, they'll have to do something that they want to actually, you know, maintain that experience so that every person can have, kind of, the experience that they want, and not be distracted, you know? “Oh, what's this? Let me go test this out.” And it's like, you know, one-time charge for $10,000 because, like, that's how it's charged. You know, that's not an experience that people like.Corey: No. They really don't. Pete, I want to thank you for spending the time to chat with me again, as is our tradition. I'm hoping we can do it in person this year, when we go at the end of 2022, to re:Invent again. Or that no one goes in person. But this hybrid nonsense is for the birds.Pete: Yeah. I very much would love to get back to another one, and yeah, like, I think there could be an interesting kind of merging here of our annual re:Invent recap slash live brunch, you know, stream you know, hot takes after a long week. [laugh].Corey: Oh, yeah. The real way that you know that it's a good joke is when one of us says something, the other one sprays scrambled eggs out of their nose. Yeah, that's the way to do it.Pete: Exactly. Exactly.Corey: Pete, thank you so much. If people want to learn more about what you're up to—hopefully, you know, come back. We miss you, but you're unaffiliated, you're a startup advisor. Where can people find you to learn more, if they for some unforgivable reason don't know who or what a Pete Cheslock is?Pete: Yeah. I think the easiest place to find me is always on Twitter. I'm just at @petecheslock. My DMs are always open and I'm always down to expand my network and chat with folks.And yeah, right, now, I'm just, as I jokingly say, professionally unaffiliated. I do some startup advisory work and have been largely just kind of—honestly checking out the state of the economy. Like, there's a lot of really interesting companies out there, and some interesting problems to solve. And, you know, trying to spend some of my time learning more about what companies are up to nowadays. So yeah, if you got some interesting problems, you know, you can follow my Twitter or go to LinkedIn if you want some great, you know, business hot takes about, you know, shitposting basically.Corey: Same thing. Pete, thanks so much for joining me, I appreciate it.Pete: Thanks for having me.Corey: Pete Cheslock, startup advisor, professionally unaffiliated, and recurring re:Invent analyst pal of mine. I'm Cloud Economist Corey Quinn and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice along with an angry comment calling me a jackass because do I know how long it took you personally to price CloudWatch RUM?Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.

What Doesn't Work II PETE: Here's one for you...acupuncture may not be helpful for improving  functional ambulation, spasticity and activities of daily living for the lower extremity. Again we're still talking about the lower extremity. Here's one that may not surprise you...Neuro Aid may not be beneficial for improving stroke severity DEB: Huh! That's interesting. It made it in here, into the EBRSR PETE: Yeah. How about this one? Stimulants may not be beneficial for improving motor function. Do you remember when stimulants were all the rage for people with brain injury because it was thought that they would somehow get better if they had...what's the big one again? DEB: Ritalin... PETE: Adderall DEB: oh Adderall yeah... Ritalin's the one that I seem to remember somebody talking about a lot at the hospital PETE: For brain injury recovery? DEB: I think so because it was a stroke center PETE: How about this one...have you ever heard of wholebody vibration? DEB: Yeah! PETE: Yeah? Wholebody vibration may not be beneficial for improving balance, functional ambulation, and muscle strength. How ‘bout them apples? DEB: How ‘bout them apples? PETE: Yeah... DEB: Is that going to come up again in what does work? PETE: That's a good question because I know for acupuncture there are some things where it does work DEB: Yeah...I do too. PETE: You can have that. You can have stuff that doesn't work for one thing but where it works for another. Yeah, absolutely DEB: Maybe we should've done a comparison what does this intervention work for, what doesn't it work for? PETE: That is the third one...so we have what doesn't work, what does work and what's the comparison for things that do work and don't work and when do they work and not work? This is a lot of work! EPISODE SUMMARY: In this episode of NOGGINS & NEURONS: Stroke and TBI Recovery Simplified, Pete and Deb talk about more of What Doesn't Work. We discovered: There's mixed evidence around bilateral training How clinical reasoning works and why it's important in the rehab process The literature is mixed about music therapy, telerehabilitation, arm/shoulder robotics for arm recovery Neuro Aid engages in shady practices and it may not improve spasticity Research supports that recovery does NOT occur from proximal to distal – it is indeed a myth! Yet again the importance of bringing meaning into recovery Acupuncture, Botox, exoskeleton systems, whole body vibration and stimulants may not be effective for lower extremity recovery It's important to look closely at evidence available for treating spatial neglect That hyperbaric oxygen therapy isn't effective for stroke recovery Pete and Deb infuse clinical reasoning and stories into this interesting conversation about stroke and brain injury interventions that don't work. People who have experienced stroke or brain injury are smart. They are also in a vulnerable space and may be susceptible to being taken advantage of. This why all of us, survivors, caregivers and healthcare providers must do our research to ensure they receive high quality interventions that work. As always, we want to hear your top takeaways!  LINKS TO ARTICLES, BOOKS AND OTHER IMPORTANT INFORMATION: Bennet, M., Weibel, S., Wasiak, J., Schnabel, A., French, C., & Kranke, P. (2014). Hyperbaric oxygen therapy for acute ischemic stroke. Cochrane Review Bordoloi, K., & Deka, R. Scientific Reconciliation of the Concepts and Principles of Rood Approach. International Journal of Health Sciences and Research EBRSR Chapter 10: Upper Extremity Motor Interventions Stroke Engine: AOTA & Choosing Wisely Neuroaid: Partial Truth and Petty Theft Stroke and Hyperbaric Oxygen Therapy Stem Cell Treatment Available in China I Stem Cell Treatment Available in China II Questions and Comments about the podcast: NogginsAndNeurons@gmail.com DONATE TO NOGGINS & NEURONS: Donate to Noggins And Neurons and get an Allstar Pete Trading Card Using your PayPal app:   RESOURCES: Pete's blog, book, Stronger After Stroke, and talks. Blog Book: Stronger After Stroke, 3rd edition Pete's talk for the American College of Rehabilitation Medicine. Deb's OT Resources: Deb's OT resources The OT's Guide to Mirror Therapy Tri-Fold Mirror (US address only) Occupational Therapy Intervention: Scavenger Hunt Visual Scanning for Adults REQUEST TO BE A GUEST ON NOGGINS & NEURONS. If you're passionate about stroke recovery and have information or a story you believe will help others, we'd love help you share it on the show. Complete the guest request form below and let's see if we're a good fit!   Guest Request Form MUSIC: “Soft Inspiration” by Scott Holmes/Scott Holmes Music/scottholmesmusic.com ✨Google Podcasts  ✨iTunes    ✨Spotify

TranscriptCorey: This episode is sponsored in part byLaunchDarkly. Take a look at what it takes to get your code into production. I’m going to just guess that it’s awful because it’s always awful. No one loves their deployment process. What if launching new features didn’t require you to do a full-on code and possibly infrastructure deploy? What if you could test on a small subset of users and then roll it back immediately if results aren’t what you expect? LaunchDarkly does exactly this. To learn more, visitlaunchdarkly.com and tell them Corey sent you, and watch for the wince.Jesse: Today, on a very special episode of AWS Morning Brief: Fridays From the Field, we say our goodbyes to Pete Cheslock.Amy: Oh, no. Did the ops bus finally get him?Jesse: No. Wait, what? What? No. No, he’s not—Amy: You know, the ops bus, the one that takes out all of the ops people, which is why you need data recovery plans.Jesse: [laugh]. I mean, I have plans for other reasons, but no. No, Pete, Pete’s not dead. He’s just—I mean, he’s dead to me, but he’s just not going to be here anymore.Amy: Only on the inside.Jesse: Welcome to AWS Morning Brief: Fridays From the Field. I’m Jesse DeRose.Amy: I’m Amy Arumbulo Negrette.Pete: I am Pete Cheslock. I’m here for one last, beautiful, glorious time.Jesse: I feel like this is going to be like Breakfast Club but in the data center server room.Pete: Yeah. A little bit. I think so. We will all sit cross-legged on the floor in a circle, share our thoughts and feelings. And maybe some sushi. There were sushi in that movie. And that was, like, really advanced back then in the ’80s.Jesse: Yeah, I like that. So Pete, you want to give us a little bit of background about why you will be moving on from this podcast?Pete: Moving on to a whole new world. Yes. Sadly, I am not dead. The ops bus did not get me, and I was not eaten by my smoker, my meat smoker.Jesse: [laugh]. Although at this point, it’s probably overdue.Pete: You know, the odds of all three of those are pretty high out, to be really perfectly honest, given this pandemic and everything else going on in this world.Amy: Isn’t that how it works? You eventually become the smoked meat.Pete: Yeah, yeah.Jesse: [laugh].Pete: All the time. You know, you are what you eat. And if you eat junk and whatnot—so I eat smoked meats, eventually, I’m just going to become, you know, smoked meats, I guess. But no, I am moving on from The Duckbill Group. Just bittersweet is the best word I can come up with. Very sad, but also very excited.I’m moving on to a new role at a new company that was just kind of an opportunity that I couldn’t pass up. And I’m really excited for something new, but really sad because I don’t get to work with two of my three favorite cloud economists, Jesse, and Amy. Yeah, Corey is one, too, and yes, it’s fun to work with him. But it’s also fun to rag on him a little bit as well.Jesse: I’m pretty sure you still have the opportunity to rag on him no matter where you go.Pete: Yeah, that’s true. I mean, we’re Twitter connected. So, I can just slide into his DMs as needed. Yeah.Amy: And really, what else is Twitter for—Pete: Exactly.Jesse: [laugh].Amy: —than roasting former coworkers and bosses?Pete: Yeah, I expect a constant stream of Twitter DMs every time you find something, some little fun nugget that I’ve left behind.Jesse: I feel like that’s appropriate. So today, Pete, I have two questions for you now that you will be moving on from Duckbill Group, moving on from this podcast, I want to know, looking back at your time here working with Duckbill Group, what did you learn? What are the things that surprised you, that you didn’t expect? And what would you say to somebody who wanted to start working in this space, maybe start a career in cloud economics on their own?Pete: Yeah, so this kind of feels like an exit interview a little bit.Jesse: [laugh]. And a very public exit interview at that. So, make sure that we bleep all the swear words.Pete: I think it’s in Duckbill fashion to do a public—a very public-facing exit interview, right? That is Duckbill in a nutshell.Jesse: I think the only thing more public is if Corey asks you to hold the exit interview on Twitter.Amy: Exactly.Pete: [laugh]. I mean, we might have to do that, now. I like that idea. Yeah, so I think those are great questions, and I love the opportunity to talk about it. Because Duckbill is a fantastic company, and coming into Duckbill last year was totally by luck.Not really—no, not—luck is maybe not the right word. But I had been doing some consulting on my own, and the pandemic and some other forces caused a bunch of my consulting work to dry up really quickly. And I was sitting at home and I’m like, “Wow, I should get a real job.” And I saw a tweet from Mike on Twitter that was like, “Oh, we’re growing The Duckbill Group.” And Mike and Corey and I have known each other for such a long time.We’ve always said it’d be great to work together at some point in the future, but it’s so hard [laugh] to do. You know, to kind of work with your friends, and timing, and circumstance, and schedule, and everything else. And so when I saw that, I was like, wow, like that might be a lot of fun working with that crew. And I’ve got a lot of experience in AWS and I’ve—my title at one of my previous companies was Captain COGS—for Cost Of Goods Sold—because I was so diligent with the Amazon bill. So, it’s kind of one of those things where I felt like I could be useful and helpful to the organization, and talking with Mike and Corey, it just made a ton of sense.And so, it was a lot of fun to come on board. So, but then once you’re kind of in, and you start doing this type of work—and you know, Amy and Jesse, you’ve both experienced this—I think no matter how much knowledge you have of Amazon, very, very quickly, you realize that you actually don’t know as much as you really think you did, right?Jesse: Yeah.Pete: Because it’s so—there’s just so much.Amy: And it changes once every five minutes.Pete: [laugh].Jesse: Oh, yeah.Amy: Literally if you—well, just keep an eye on that changelog, you can watch your day get ruined as time goes on.Jesse: [laugh].Pete: [laugh]. It’s—yeah, it’s a real-time day ruining. And that’s the new. It’s like Amazon Kinesis: It’s all real-time.Jesse: [laugh].Pete: Yeah, it’s so true. And I think the reason behind it is, you know, one of the first things I kind of realized is that when you are working inside of a business and you’re trying to understand, like, an Amazon service, you don’t necessarily go that deep because you’ve got a real job and other stuff to do. And when you’re finally, like—let’s say you’re in Cost Explorer; this is actually my favorite one because learning this took us a while. The documents aren’t very good. But in Cost Explorer, there’s a dropdown box that can show you your charges in different ways: unblended view, blended view, amortized view—if I’m saying that word really incorrectly—net-amortized view, net-unblended view. Like, what do all these mean?Most people just are like, unblended, move on with their lives. But at some point, you kind of need to know and answer that question, and then understand the impact, and all those things, and spending more hours than I care to count trying to correlate the bill and Cost Explorer to look the same. Something that simple, why is that so hard? You know, it’s things like that.Amy: Why is that so hard? I do not understand it. It is exhausting. [laugh].Jesse: It drives me absolutely crazy, and it’s something that in previous roles, you could just say, “Well, this isn’t my responsibility, so I’m not going to worry about it.” But now we’ve got clients who are asking us these questions because it is our responsibility and we do need to worry about it.Pete: Yeah, exactly. So, I think that’s just, kind of, one example. Now, there was a ton that I learned. I mean, just in how discounts might be applied when you look at charges in an account whether if you have an enterprise discount program, or private pricing in some way. I think one of my favorite ones—and this is actually something that catches a lot of people up—is especially in Cost Explorer, there’s kind of two ways that you can view a charge.So, let’s say you’re looking at S3, and you are trying to find your usage by the usage type. Like, I want to compare standard storage to maybe data transfer or something like that. And you go and group by usage type, and they’ll show you, “Hey, for your S3 for this month or day or whatever, you’ll have some spend associated storage and data transfer,” and you’re like, “That’s neat.” And then you say to yourself, “Now, I want to look at it by API.” And maybe you’ll see, wow, there’s a ton of spend associated with GETs or PUTs.And you’ll think that that is actually a request charge. And it’s totally not. It’s like, when you group by API, it’s the API that started the charge, not the charge itself. So, you could have a PUT that started the charge, but the charge itself is actually storage. It’s the little things like that, where you might glance at it in your account and go, “Oh, okay.” But then when you actually need to get down to the per penny on spend and share it with a client, you go even further down the rabbit hole.Jesse: Because why would all of the billing information across different sources be accurate?Amy: And also, why would things be named the same between the bill, and Cost Explorer, and the curve? Having those names be the same, that would just make it too easy, and just streamline the process too much, and be too logical. No, let’s work for it. We have to work for it. It’s a pillar of excellence; we have to work for it.Pete: [laugh]. Exactly. So yeah, I think it’s those types of things that you just start seeing the edge cases. But because of, kind of, the work we do, we keep going. We’re not just, “Oh, wow. Haha, silly Amazon.”But then we keep diving in deeper and deeper to figure out the why. And the reason for that really just comes down to the fact that we’ll need to communicate that in some effective way to the client to get them to understand it. And actually, that kind of leads me to the other thing that I think is probably the most important skill of being a cloud economist, of being in finops, is your ability to write long-form writing, being able to write clear, concise information explaining why the spend is what it is, explaining all of these edge cases, all these interesting parts of cloud cost management, and being able to write that down in such a way that anyone could read it; like a CFO could understand how the charges are happening, just like a head of engineering, who has maybe more impact to the spend.Jesse: Being able to communicate, the differences between different AWS services, between different billing modes, to different audiences is so critical to the work that we do because we’re ultimately going to be working with different people with different backgrounds at every single client that we work with. So, we need to be able to speak the language of different audiences.Amy: And it’s really different, how different C Suites, different departments, their goals are going to be different, too, because they have requirements that they have to fulfill. Finance is very concerned about the literal cost of things, while engineering is—they understand that their architecture comes at a price, and so long as they have the budget for it, they’re cool with it. And you just have to align what those goals are, and have that translate as like, into the document as, “They built it this way for this reason, which was fine at that stage. But as you grow, you need to make sure that it also fulfills these other external expectations.”Corey: Let’s be honest—the past year has been a nightmare for cloud financial management. The pandemic forced us to move workloads to the cloud sooner than anticipated, and we all know what that means—surprises on the cloud bill and headaches for anyone trying to figure out what caused them. The CloudLIVE 2021 virtual conference is your chance to connect with FinOps and cloud financial management practitioners and get a behind-the-scenes look into proven strategies that have helped organizations like yours adapt to the realities of the past year. Hosted by CloudHealth by VMware on May 20th, the CloudLIVE 2021 conference will be 100% virtual and 100% free to attend, so you have no excuses for missing out on this opportunity to connect with the cloud management community. Visit cloudlive.com/corey to learn more and save your virtual seat today. That’s cloud-l-i-v-e.com/corey to register.Pete: Yeah, that’s exactly right. I mean, it’s just—and can you imagine, you have some knowledge you want to share around something as complex as the Amazon bill. I mean we ask for a PDF of your bill when you start working with Duckbill Group. That could be hundreds of pages long, and you’re trying to distill that down into something that, really, anyone can understand. It’s a true superpower to be able to write long-form content like that really well.And I never used to like writing. I was never—never really enjoyed it that much and over the last year, that muscle that you’re working out, now, the ability to write many, many pages around this type of content, just it comes so much more easily. So, I think that’s another big aspect, right? The more you work on it, obviously the easier it gets.Jesse: I don’t know about you, but now that I have focused more on flexing that writing and communication muscle, I’ve noticed it more in both everyone that I work with day-to-day with Duckbill Group and also in my daily life, just watching how people communicate with each other, and how effectively people communicate with each other; it’s both amazing and nerve-wracking all at the same time.Pete: [laugh]. I know. And even—not to say that whenever we sit down to write our reports that we give to our clients, we don’t go through the wave of emotions between the back and forth of, like, “I don’t know what to write,” and then, “Oh, I know of a lot of stuff to write. Let me just get something down.” And then you can’t stop writing. It’s just—it’s this emotional roller coaster that I feel like no matter how many times we need to write a lot of detailed information down, everyone always goes through.Amy: And we really do have a highly collaborative process here, too, where we’re all in the same document, writing, and the person who owns any given report will always have the same stage at the end when all of the sections are filled out, where they go to one of the other people on the team and go, “Every word I put down is absolute garbage. Please help me trim it down, take it out. I don’t even care anymore. Just look at it and tell me that I wrote down words that are in some kind of human language.” [laugh].Jesse: [laugh].Pete: [laugh]. Oh, the plight of the writer. It’s, like, the imposter syndrome that affects the writer. It’s like, “Okay. I wrote a bunch of stuff. I think it’s terrible.” And then you sleep on it, you come back the next day, and you’re like, “Actually, this is pretty good.” [laugh].Amy: I explained concepts. It was fine. I didn’t use a single comma for three pages, but it’s probably fine. [laugh].Jesse: [laugh].Pete: You can take one of mine. Usually, all of my draft documents are commas and M-dashes, just all over the place. Yeah, so I think that’s honestly a big superpower. And I think the last two things that—this is actually something that I’ve looked for in people that I’ve wanted to work with, and people I was hiring, and I see it here as well as these, kind of, two concepts of intellectual curiosity and aptitude to learn, where if you have a base knowledge around Amazon and you have those other attributes—that curiosity and truly enjoying learning—you can accelerate your ability to understand this so incredibly quickly because there’s such a wealth of information out there, and there’s so many documents, there’s so much stuff. It just requires someone who really cares enough to dive in and really want to understand.That’s something that I think we’ve seen here is that the folks who are most successful are just—they want to know why, and they’re not satisfied until they can explain it in a simple way to someone else. That’s the key, right? The attribute of a true expert is someone who can explain something very difficult in a simple way. And I think that’s something that would be critical if you were joining Duckbill, if you were building your own finops or cloud finance team, it is so complex. It’s the intersection of technical architecture and cost, and it touches almost the entire business. So, I think those are some other attributes that I think are just incredibly helpful.Jesse: We’re also usually not entirely satisfied until we’ve either opened a support case with AWS, responded to one of their feedback icons in the AWS documentation—the public AWS documentation—or trolled somebody on Twitter saying, “Shame on you, AWS, for writing documentation that doesn’t make sense.”Amy: It’ll be fine. Someone in your mentions will go, “Did you check the region?” And you would have, and then it’ll still be wrong.Jesse: [laugh].Amy: And it’ll be fine. [laugh]. Eventually, we’ll fix it.Pete: That one—Jesse: Too soon.Pete: —that one still hurts, when we—oh, I’m just like, “Why do the numbers not line up?” And then someone was like—Amy: It's a thing I check for, even if it’s like, “It’s a global resource.” I don’t care. Just tell me. Just tell me it’s fine. [laugh].Pete: “Are you in the right region?” Like—“Dammit, no, I’m not. Oh.” [laugh]. Yeah, that happens to the best of us.Amy: I did, unfortunately, burn so many hours, I think it was last week trying to find out where someone had put their resources. It’s like, “Oh, not us-west-2. It’s us-west-1. Of course.” [laugh].Jesse: So, annoying. Well, I would just like to say, Pete, it has been a joy and a pleasure working with you, it has been a joy and a pleasure complaining about AWS with you, on this podcast, so thank you for your time. That sounded really… really, really standoffish. I didn’t mean it quite as bad as it came off there. [laugh].Pete: Well, you know, I think we need to thank Corey for having a child and thus needing to offload some of his podcast duties over to us, and then the fact that we just never gave him the podcast back, and we just took it over.Jesse: Well, if you’ve got questions that you’d like us to answer, you can go to lastweekinaws.com/QA. And if you’ve enjoyed this podcast, please go to lastweekinaws.com/review and give it a five-star review on your podcast platform of choice, whereas if you hated this podcast, please go to lastweekinaws.com/review, give it a five-star rating on your podcast platform of choice and tell us what qualities you’re looking for when building out your cloud finance team.Pete: Thanks for coming in.Announcer: This has been a HumblePod production. Stay humble.

Links: Cloud FinOps: https://www.amazon.com/Cloud-FinOps-Collaborative-Real-Time-Management/dp/1492054623 FinOps Foundation: https://www.Finops.org/ AWS cost management blog: https://aws.amazon.com/blogs/aws-cost-management/ Mastering AWS Cost Optimization: https://www.amazon.com/Mastering-AWS-Cost-Optimization-operational/dp/965572803X TranscriptCorey: This episode is sponsored in part by LaunchDarkly. Take a look at what it takes to get your code into production. I’m going to just guess that it’s awful because it’s always awful. No one loves their deployment process. What if launching new features didn’t require you to do a full-on code and possibly infrastructure deploy? What if you could test on a small subset of users and then roll it back immediately if results aren’t what you expect? LaunchDarkly does exactly this. To learn more, visit launchdarkly.com and tell them Corey sent you, and watch for the wince.Pete: Hello, and welcome to the AWS Morning Brief: Fridays From the Field. I am Pete Cheslock.Jesse: I’m Jesse DeRose.Pete: Wow, we’re back again. And guess what? We have even more questions. I am… I am… I don’t even know. I have so many emotions right now that are conflicting between a pandemic and non-pandemic that I just—I’m just so happy. I’m just so happy that you listen, all of you out there, all you wonderful humans out there are listening. But more importantly, you are going into lastweekinaws.com/QA and you’re sending us some really great questions.Jesse: Yeah.Pete: And we’re going to answer some more questions today. We’re having so much fun with this, that we’re just going to keep the good times rolling. So, if you also want to keep these good times rolling, send us your questions, and we’ll just—yeah, we’ll just roll with it. Right, Jesse?Jesse: Absolutely. We’re happy to answer more questions on air, happy to let you pick our brains.Pete: All right. Well, we got a couple more questions. Let’s kick it off, Jesse.Jesse: Yeah. So, the first question today is from Barry. Thank you, Barry. “New friend of the pod here.” Always happy to have friends of the pod. Although I do feel like that starts to get, like, Children of the Corn, kind of. I think we started that, and I also am excited about it, and also upset with myself for starting that.Pete: That’s all right. Friend of the pod. Friend of the pod.Jesse: “New friend of the pod here. I work in strategic sourcing and procurement and I was curious if there are any ways that you recommend to get up to speed with managing cloud spend. This is usually closely monitored by finance or different groups in product, but I can see a significant potential value for a sourcing professional to help, also.” And that’s from Barry, thank you, Barry.Pete: Well, I’m struggling not to laugh. “This is usually closely monitored by finance or different groups in product.”Jesse: Yeah…Pete: But I mean, let’s be honest, it’s not monitored by anyone. It’s just running up a meter in a taxi going 100 miles an hour.Jesse: Yeah, that’s the hardest part. I want everybody to be involved in the cloud cost management practice, but there’s that same idea of if it’s everyone’s responsibility, it’s no one’s responsibility. And so this usually ends up at a point where you’ve got the CFO walking over to the head of engineering saying, “Why did the spend go up?” And that’s never a good conversation to have.Pete: No, never a good one. Well, Barry because you’re a friend of the pod, we will answer this question for you. And honestly, I think it’s a great question, which is, we actually have been working with a lot of larger enterprises and these enterprises still have their classic sourcing and procurement teams. That’s not an expertise that is going away anytime soon, but like most teams within the company that are adopting cloud, it’s obviously going to evolve as people are moving away from, kind of, capital intensive purchases and into, honestly, more complex, multi-year OpEx style purchases, with cloud services and all the different vendors that come with it. It’s going to just get a lot harder.I mean, it’s probably already a lot harder for those types of teams. And so there’s a bunch of places I think that you can go that can help level up your skills around cloud spend. And I would say the first place that I personally got to dive in a little bit more—I mean, my history has been using Amazon cloud and being a person who cared about how much my company spent on it, but when you—joining Duckbill, you need to dive into other areas around the FinOps world. And the book, the O’Reilly book, Cloud FinOps is actually a really great resource.Yeah, I think it’s really well written and there’s a lot of great chapters within there that you can kind of pick and choose based on what you’re most interested in learning about. If you’re trying to learn more about unit economics, or you’re trying to learn more about how to monitor and track things like that, it’s a great book to dive into, and becomes a really great reference that you can leverage as you’re trying to level up this expertise within yourself or your team.Jesse: It’s a really, really great resource. The other thing to think about is any kind of collaborative social spaces where you can be with like-minded individuals who also care about cloud costs. Now, there’s a number of meetups that exist under the FinOps title that may be worth looking into. Obviously, we’re recording this during the pandemic so I don’t recommend doing those in person. But as you are able to, there may be opportunities for in-person meetups and smaller local groups focusing on cloud cost management strategies together. But also check out the FinOps Foundation. They have a Slack space that I would love to tell you more about, but unfortunately, we’re not allowed to join. So—Pete: Yep.Jesse: —I can’t really say more about it than that. I would hope that you’re allowed to join, but they have some strict guidelines. So, I mean, the worst that can happen is they say no; it’s definitely worth signing up.Pete: Yeah, and they have to us. [laugh].Jesse: Yeah.Pete: I think when you get into the FinOps Foundation, you should angrily say that we should have more FinOps experts in here like the great Jesse DeRose should be a member of this one because right now, he’s just framed his rejection notice from there, and—Jesse: Oh, yeah.Pete: —while it looks beautiful on the wall, while I’m on a Zoom with him, I want more for you, Jesse.Jesse: I want more for me, too. I’m not going to lie.Pete: So, I don’t know this might sound a little ridiculous that I’m going to say something nice about AWS, but they have a fantastic cost management blog. This is a really fantastic resource, really incredible resource, with a lot more content more recently. They seem to be doing some great work on the recruiting side and bringing on some real fantastic experts around cost management.I mean, just recently within the past few months they talk about unit economics: How to select a unit metric that might support your business, talking more about unit metrics in practice. They start at the basics, too. I mean, obviously, we deal a lot in unit economics and unit metrics; they will start you off with something very basic and say, “Well, what even is this thing?” And talk to you more about cost reporting using AWS organizations for some of this. It’s a really fantastic resource.It’s all free, too, which is—it’s weird to say that something from AWS is free. So, anytime that you can find a free resource from Amazon, I say, highly recommend it. But there are a lot of blogs on the AWS site, but again, the Cost Management Blog, great resource. I read it religiously; I think what they’re writing is some of, really, the best content on the blog in general.Jesse: There’s one other book that I want to recommend called Mastering AWS Cost Optimization and we’ll throw links to all these in the [show notes 00:07:30], but I, unfortunately, have not read this book yet, so I can’t give strong recommendations for it, but it is very similar in style and vein to the Cloud FinOps book that we just mentioned, so might be another great resource to pick up to give you some spot learning of different components of the cloud cost management workflow and style.Pete: Awesome. Yeah, definitely agree. I’d love to see, again, more content out here. There’s a lot of stuff that exists. And even A Cloud Guru has come up with cost management training sessions.Again, we’d like to see more and more of this. I’d love to see more of this come from Amazon. I’d love to see—you know, they have a certification path in all these different areas; I’d love to see more of that in the cost management world because I think it’s going to become more complex, and having that knowledge, there is so much knowledge, it’s spread so far across AWS, helping more people get up to speed on it will be just critical for businesses who want to better understand what their spend is doing. So, really great question, Barry, friend of the pod. We should get some pins for that, right? Friend of the pod pins?Jesse: Oh, yeah.Pete: And yeah, really great question. Really appreciate you sending it and hopefully that helps you. And if not, guess what? You can go to lastweekinaws.com/QA, and just ask us a follow-up question, Barry. Because you’re a friend of the pod. So, we’ll hopefully hear from you again soon.Jesse: Thanks, Barry.Pete: Thanks.Announcer: If your mean time to WTF for a security alert is more than a minute, it’s time to look at Lacework. Lacework will help you get your security act together for everything from compliance service configurations to container app relationships, all without the need for PhDs in AWS to write the rules. If you’re building a secure business on AWS with compliance requirements, you don’t really have time to choose between antivirus or firewall companies to help you secure your stack. That’s why Lacework is built from the ground up for the cloud: Low effort, high visibility, and detection. To learn more, visit lacework.com.Pete: All right, we have one more question. Jesse, what is it?Jesse: “All right, most tech execs I speak with have already chosen a destination hyperscaler of choice. They ask me to take them there. I can either print out a map they can follow, procedural style, or I can be their Uber driver. I could be declarative. I prefer the latter for flexibility reasons, but having said that, where does one actually start?Do you start with Infrastructure as a Service and some RDS to rid them of that pesky expensive Oracle bill? Do we start with a greenfield? I mean, having a massive legacy footprint, it takes a while to move things over, and integrating becomes a costly affair. There’s definitely a chicken and egg scenario here. How do I ultimately find the best path forward?” That question is from Marsellus Wallace? Thank you, Marsellus.Pete: Great question. And I’m not just saying that. I guess I have a question. Or at least, maybe we have different answers based on what this really looks like. Is this a legacy data center migration?The solution here is basically lift-and-shift. Do it quickly. And most importantly, don’t forget to refactor and clean up after you shut down your old data center. Don’t leave old technical debt behind. And, yeah, you’re going to spend a lot, you’re going to look at your bill and go, “Holy hell, what just happened here?”But it’s not going to stay that way. That’s probably—if you do it right—the highest your bill is going to be because lift-and-shift means basically just moving compute from one location to another. And if you’re—as we spoken about probably a million times, Jesse and I, if you just run everything on EC2 like a data center, it’s the most expensive way to do the cloud stuff. So, you’re going to then refactor and bring in ephemerality and tiering of data and all those fun things that we talk about. Now, is this a hybrid cloud world?That’s a little bit different because that means you’re not technically going to get rid of, maybe, physical locations or physical data centers, so where do you start? It’s my personal opinion—and Jesse has his own opinion, too, and guess what it’s our podcast and we’re going to tell it like it is.Jesse: [laugh].Pete: [laugh]. You know, my belief is, starting with storage is honestly a great way to get into cloud. Specifically S3. Maybe even your corporate file systems, using a tool like FSX. It’s honestly why many businesses start their cloud journey, by moving corporate email and file systems into the cloud.I mean, as a former Microsoft Exchange administrator, I am thoroughly happy that you don’t have to manage that, really, anymore and you can push that in the cloud. So, I think storage is honestly a great way to get started within there: Get S3 going, move your file systems in there, move your email in there if you haven’t yet. That’s a really great way to do it. Now, the next one that I would move probably just as aggressively into and, Marsellus, you mentioned it: RDS, right? “Should we move into RDS, get rid of expensive Oracle bills?”Yeah, anytime you can pay ol’ Uncle Larry less money is better in my mindset. Databases are, again, another really great way of getting into AWS. They work so well, RDS is just such a great service, but don’t forget about DMS, the database migration service. This is the most underrated cloud service that Amazon has in there, it will help you migrate your workloads into RDS, into Amazon Aurora. But one thing I do want to call out before you start migrating data in there, talk to your account manager—you have one even if you don’t think you have one—before starting anything, and have them help you identify if there are any current programs that exist to help you migrate that data in.Again, Amazon will incentivize you to do it, they will provide you credits, like map credits or other investment credits, maybe even professional services that can help you migrate this data from an on-premise Oracle into AWS, I think you will be very pleasantly surprised with how aggressive that they can be to help you get into there. The last thing that I would say is another great thing to move in our data projects. So, let’s say you want to do a greenfield one, greenfield type of project into Amazon, data projects are a really great way to move in there. I’m talking things like EMR, Databricks, Qubole, you get to take advantage of Spot Fleets with EMR, but also Databricks and Qubole can manage Spot infrastructure and really take advantage of cloud ephemerality. So if, like I said, you started by pushing all your data into S3, you’re already halfway there on a really solid data engineering project, and now you get to leverage a lot of these other ancillary services like Glue, Glue DataBrew, Athena, Redshift.I mean, once the data is in S3, you have a lot of flexibility. So, that’s my personal opinion on where to get started there. But Jesse, I know you always have a different take on these, so where do you think that they should start?Jesse: Yeah, I think all of the recommendations you just made are really, really great options. I always like to look at this from the perspective of the theory side or the strategy side. What ultimately do these tech execs want to accomplish? Is it getting out of data centers? Is it better cost visibility?Is it optimizing spend? Is it better opportunity to move fast, get new R&D things that you can’t get in a data center? What do these tech execs ultimately want to accomplish? And ask them. Start by asking them.Prioritize the work that they want to accomplish first, and work with teams to change their behaviors to accomplish their goals. One of the biggest themes that we see in the space moving from data centers into cloud providers or even just growing within a given cloud provider is cost visibility. Do teams know why their spend is what it is? Do they know why it went up or down month-over-month? Can they tell you the influences and the drivers that cause their spend to go up or down?Can they specifically call out which teams or product usage increased or decreased, and what ultimately led to your spending changing? Make sure that every team has an architecture diagram and they can explain how they use AWS, how data moves from one service to another, both within their product and to other products. Because there’s definitely going to be sharp edges with data transfer between accounts. We’ve seen this happen to a number of clients before; I’ve gotten bit by this bullet. So, talk to your teams, or talk to your tech executives and have those tech executives talk to their teams to understand what do they ultimately want to accomplish?Can they tie all of what they’re trying to accomplish back to business metrics? Maybe a spike in user logins generated more usage? If you’re a photo storage company, did a world event prompt a lot of users to upload photos prompting higher storage costs? Are you able to pull out these specific insights? That’s ultimately the big question here. Can you boil it down to a business KPI that changed, that ultimately impacted your AWS spend?Pete: I think this is a scenario of where you get started. Why not both? Just maybe do both of these things that we’re saying.Jesse: Yeah.Pete: And honestly, I think you’ll end up in a pretty great place. So, let us know how that works out, Marsellus, and thank you for the question. Again, you also can send us your questions, and we will maybe answer these on a future episode; lastweekinaws.com/QA, drop a question in there, put your name, or not or a fake name, or even a joke. That’s fine, too. I don’t know what the text limit is on the name, Jesse. Can you put a joke there? I don’t know. You know what? Test that out for us. It’s not slash QA for nothing. So, give that a little QA, or a question and answer and [unintelligible 00:17:29]. All right. Well, thanks, Jesse, for helping me out answering more questions.Jesse: Thanks, everybody for the awesome questions.Pete: If you enjoyed this podcast, please go to lastweekinaws.com/review, give it a five-star review on your podcast platform of choice, whereas if you hated this podcast, please go to lastweekinaws.com/review and give it a five-star rating on your podcast platform of choice and tell us, what would be the last thing that you would move to AWS? It’s QuickSight, isn’t it?Jesse: [laugh].Pete: Thanks, everyone. Bye-bye.Announcer: This has been a HumblePod production. Stay humble.

Links:Unconventional Guide: https://www.duckbillgroup.com/resources/unconventional-guide-to-aws-cost-management/ TranscriptCorey: This episode is sponsored in part by LaunchDarkly. Take a look at what it takes to get your code into production. I’m going to just guess that it’s awful because it’s always awful. No one loves their deployment process. What if launching new features didn’t require you to do a full-on code and possibly infrastructure deploy? What if you could test on a small subset of users and then roll it back immediately if results aren’t what you expect? LaunchDarkly does exactly this. To learn more, visit launchdarkly.com and tell them Corey sent you, and watch for the wince.Pete: Hello, and welcome to the AWS Morning Brief: Fridays From the Field. We’re back again, my name is Pete Cheslock.Jesse: I’m Jesse DeRose. So, happy to be back in the studio after our whirlwind tour of the Unconventional Guide that I feel like we’ve been on for roughly as long as the pandemic’s been going on at this point; probably a little bit less. But lots of really great content there that we were happy to talk about, and I’m happy to be moving on to some other topics.Pete: Yeah, absolutely. And the topics, we actually get to move on to some of our favorite topics, which are answering your questions. And it turns out, Jesse, there’s more than two people that listen to us. There’s a lot of you; there are dozens of you out there, and we love it.Jesse: You like me. You really like me.Pete: So, great. So, great to see. We’ve been getting tons of fantastic questions, a few of which we’re going to answer right now. You can also have your question answered by going over to the lastweekinaws.com/QA and enter in your question there. You can enter in your name, or you can leave it blank, or you could just put something funny there. Anything works. We’re happy to dive in deeper on any particular topic, again, whether it’s about this recent Unconventional Guide series or just something you’re curious about in your day-to-day in your cost management life.Jesse: Today’s questions are really great because they ultimately get at the practical side of all of our recommendations. Because I feel like every single time I subscribe to one of those self-help books or blogs and I read all these really great short, sweet tidbits, I think to myself, “This is perfect. I’ll go apply this to everything in my life.” But then doing the actual work part is so much harder. Where do you even start with that first step once you’ve got the big picture grand idea? So, today we’ve got some really, really great questions, focusing on the best ways to get started on your cloud cost management journey. So, let’s start off with these questions.First question is, “Could you cover some practical approaches to applying some of your Cost Management Guide? A lot of your suggestions sound simple on paper, but in practice, they become quite complicated.” So, true. Absolutely, absolutely a concern. “I’ve had some success pulling in a small group of subject matter experts together for short periods of time focusing on low risk, easy things to do. How have you approached actually doing this? What meetings do you set up? What do you take for notes? How do you document your savings? How do you find new opportunities?” That’s from Brian O. Brian O., That’s a really, really great question.The other one that I want to add to this: “We’re a big AWS shop, and I’ve spent some time inside the AWS beast in the past, and I still struggle with multi-account multi-region data transfer in general, but specifically analyzing cost and usage. There are examples specifically like if data transfer out goes up $25,000 last month, how do you attribute that? How do you know where to apply that? How do you know what ultimately prompted that spend? Love how you work through these types of challenges. What is relatively easy at a single account level gets exponentially more complex with every account and region we function in.” So, true. And that’s from Todd. Thank you, Todd. In both cases, absolutely true.There’s this really great idea of we can give you the really short and sweet things to think about, but taking those first steps for practically applying these ideas is tough, and it needs to scale over time. And not every practice does.Pete: Yeah, these are great questions. I, kind of, am remembering that meme that was around for a while, which was, how to draw an owl. “First, draw two circles, and then, you know, you draw the rest of the owl.”Jesse: Yeah.Pete: And honestly, oftentimes, some of the stuff even that we say, Jesse, feels that way, and it doesn’t intend to come across that way. It’s just, we could bore you all on a multi-hour long recording of some of these topics. I mean, we do this with our clients, and our clients pay for this pleasure [laugh] for us to put them to sleep with our soft tones of the cloud cost management world. But I think the reality is that it is complex and there are probably unlikely to be quick wins in a lot of these places. One thing that we found is honestly, monitoring, visibility, I think all the cool kids are calling it observability now—Jesse: [laugh].Pete: —you know, I can’t believe I’m going to say this, but CloudWatch is actually probably one of the best cloud cost reduction tools that exist out there. There are so many services within AWS that you’re probably using today, that by default, report data to CloudWatch. And those statistics are potentially a huge place to identify resources that are over-provisioned and underused, idle resources, things like that. I can’t tell you how many times that I will go into a client account, and one of the first places I go to is—after Cost Explorer—is probably CloudWatch. So, monitoring spend and monitoring what’s happening there is kind of a great way to get started on that cloud cost idea because you’re getting charged for everything that happens, so knowing what’s happening, and knowing how it’s changing over time is a great way to start understanding and reducing it.Jesse: Yeah. And I think AWS is probably also using some of those CloudWatch metrics in their optimization recommendations that they make within their own optimization tooling. And it’s probably just not clearly defined or clearly outlined for AWS customers to be able to use the same metrics. So, I feel like if my Compute Optimizer could quickly load or link to a graph that showed me low CPU utilization across a number of instances, that’s a really handy way for me to start using more of CloudWatch’s metrics.Pete: Yeah, I think Compute Optimizer is honestly, criminally underused out there. I don’t know why. Then honestly, one of the other complaints is like, “Well, you can’t get memory statistics unless you have a CloudWatch Agent.” Yes. So honestly, install the CloudWatch agent; have it report up, the, like, one or two memory metrics that Compute Optimizer needs to make a recommendation and the cost will more than pay for itself.And now you can even output those statistics to S3 and do some fun programmatic stuff with it. Put those outputs in front of the engineers that own those resources and be like, “Hey, yo. This thing says, change your i3.24xl. Could you move it to something a little bit more useful, like a t3.small?”Jesse: And these are just some practical applications for some of the specific metrics we’re talking about, but this is a practice that you might want to turn into a process, you might want to turn into an ongoing amount of work. And in a lot of cases, we’ve seen this start as one engineer who’s really interested in understanding AWS, really passionate about the bill, maybe isn’t in a leadership or management role so maybe they don’t have a direct business requirement to optimize their spend, but they’re really, really interested in this work and they grow into a role where they are taking on more and more of this work. And that’s not scalable; that engineer is going to get burned out very, very quickly if they have a day-to-day role that is focused in development and doing all of this optimization work, cost cloud management work on the side. We generally recommend at least one dedicated individual who starts building these dashboards, starts looking at some of these metrics, starts these conversations with teams, and ultimately grow that into a full team.Pete: Right. I think that’s the biggest thing that we’re seeing in the industry is actual cloud finance teams coming into existence at companies. It’s such a critical role and it’s sad to see when people are like, “Arg, spend is out of control. We’re doubling year over year on spend and no one really seems to know why.” And honestly, it’s because no one cares about it. You don’t have any ownership on it. And, you know, we see it a lot, right? It’s like, “Well, everyone owns the Amazon bill.” That’s code for, “No one owns the Amazon bill.”Jesse: Yeah.Pete: But these cloud finance teams, and even the term cloud economist, as silly as it is, it’s centered in reality, which is we create financial models to understand spend and we dive into those numbers to make the usage makes sense to folks like CFOs inside of companies. Yeah, there’s a couple of ways that we have seen some of this done at scale. In one case using kind of active monitoring, and actively monitoring the spend based on really granular budgets, and reporting it as such. So, maybe you’re breaking these budgets out to be product-specific, or team specific, or business unit, or things like that, and then basically reaching out to these engineering teams. Because you are actively monitoring the spend on a recurring basis, you can reach out to those teams, when their spend goes over a given threshold that you’ve put in place, or when you, maybe, find some optimization opportunities.You’re probably thinking to yourself, “Wow, I don’t have the time for that.” Yeah, but you need to create the time or you need to create the team for this. The companies that we work with who have a dedicated team around this are the ones that do the best. In some cases, we’ve seen having a Dedicated Cloud finance team causes the bill to actually decrease over time, which, you’re thinking to yourself, “Wow. An Amazon bill that goes down? We so rarely see that.”Even for us, our clients come to us, we help them find optimizations. They’ll make those optimizations, but then they replace that spend with other investments. Usually, it’s new projects and new spend. But actually seeing the bill go down because of a dedicated effort of a team is still, again, amazing to see. The other side is we’ve seen maybe more of a passive monitoring or something around the background of things where you have a cloud platform team that provides abstractions and guardrails to the user.So, you’re not really trying to actively stand in the way of users and what they’re able to do and reaching out to them in an ongoing way, but you’re abstracting away the kind of complexity of the cloud and letting them basically live in a safe space that you are controlling for them. And that’s another way that you can kind of build in some of this cloud financial knowledge where teams can get that visibility into what they’re spending and know, is this too high? Is it going out of a boundary? Is there a number that I need to keep inside of? I think these are important things that level of visibility around cost and that team’s actual charges get people to start thinking, “Well, hold on a second. We’re above budget.” Even though maybe it’s not a real budget, “We’re above a spend by 20%. We need to bring that down.” And you give them the tools they need and the dashboards to effect that change on their own.Jesse: This idea of passive monitoring is really all about making the right thing the easy thing to do. If you, as a member of the cloud platform team or as a member of leadership who cares about cloud spend, wants to make sure that teams are managing their spend in some capacity, maybe not actively directly, but at some level make sure that there are these guide rails in place that keep them within the boundaries of what they’re ultimately able to do, or what you ultimately want them to work on. This makes it a lot easier for them to not spin up an i3 instance that they don’t ultimately need; it makes it a lot easier for them to not deploy resources that are missing tags. Put as many guardrails in place that keep the teams independently able to work within the space that they are building, and developing, and functioning, but ultimately gives them the opportunity to continue being independent and really thrive within whatever work they’re doing.Pete: Yeah, the next thing that we recommend to everyone. And actually, we recommend it before even engaging with The Duckbill Group, you’ll get an onboarding document of things to do, and the thing we always recommend is turn on the Cost and Usage Report. If you’re listening to this and you’re like, “What’s the Cost and Usage Report?” Well, boy are you going to have a fun learning because it is a highly granular usage report of everything that you’ve ever done within Amazon, and it’s extremely powerful. The downside is that it can be hard to navigate; it takes a little time to learn.But go turn it on; the cost is minimal; it’s the cost of storing this data in S3. Preferably when you turn this on, turn it on into Parquet format because it’ll allow you to query it with tools like Athena, or Tableau, or Looker or—God forbid—SageMaker. And this tool, this Cost and Usage Report, lets you dive in at an extremely granular level, down to the resource visibility—per hour, per resource visibility. And it’s something you have to enable, but again, highly recommend it to enable that resource level usage. Because now you can go and find out, well, for SageMaker I’m seeing a growth in spend.Well, which resource is it within SageMaker? You can break that down really granularly. So, Cost and Usage Report is another place that, again, if you’re not using this today, if you don’t have at least a SageMaker dashboard, which costs basically nothing—a couple of dollars a month—pointed at your Cost and Usage Report, you’re missing out on some really great ways to understand the changes in spend over time.Announcer: If your mean time to WTF for a security alert is more than a minute, it’s time to look at Lacework. Lacework will help you get your security act together for everything from compliance service configurations to container app relationships, all without the need for PhDs in AWS to write the rules. If you’re building a secure business on AWS with compliance requirements, you don’t really have time to choose between antivirus or firewall companies to help you secure your stack. That’s why Lacework is built from the ground up for the Cloud: low effort, high visibility, and detection. To learn more, visit lacework.com.Jesse: Another couple of really great options are the AWS Cost Anomaly Detection service and AWS Budgets. Both are free, which is absolutely fantastic. I highly recommend checking them out. AWS Cost Anomaly Detection, once enabled, will actually look for anomalies in your spend across different AWS services, across different cost attribution tags, across different cost categories. There’s a lot of opportunities here for you to see anomalous spend and act on it.This can be shared with teams as soon as the anomaly occurs, through Slack notification or an email, or maybe you get email notifications on a weekly basis, or a monthly basis, or some kind of recurring basis, for all of the anomalies that you saw within a given time period. We recorded an episode about Cost Anomaly Detection a while back; highly recommend checking that episode out. It’s got a lot of really great features and recommendations for getting started.The other one I mentioned is AWS Budgets. Again, if you’re not really sure where to start, try creating some budgets for your teams. Maybe look at the last six months of spend for each team, maybe look at spend across different tags, or team units, or business units, whatever makes the most sense for the way that your organization is set up, and create some budgets for those groups. These budgets could be for specific AWS services if you are a single team running within a single AWS account, it could be as complex as multiple business units across multiple different accounts across different parts of the organization. There’s lots of great opportunities here for you to start to better understand your spend, get better visibility into your cloud spend.Pete: Yeah, absolutely. I think all of those are great tools that can really help you. And, Jesse, I know we’ve talked about this before. Even just monitoring your tagging, not like, “Oh, are we tagging 50% of our resources?” But you want to monitor for your untagged by spend. So, if 95% of your spend is tagged, you’re crushing it. That’s amazing. But that may only be 50% of your things.So, I guess, care less about how many of your resources are tagged—because some of them just can’t be tagged, or are tagged in a painful way—but focus more on the money aspect of it. And that will lead you into the ability to start creating some governance strategies. And that term governance, it just—Jesse: Oof.Pete: —makes me feel gross. Yeah. Oh god, terrible word. But the [laugh] sad state of the world is, that’s what most companies we talk to need; they just don’t have it. When the companies that we talk to who are like, “Our spend is going up, and we’re not sure why.” Or, “How do we get our engineers to care about cost savings?” And things like that. You know, having a governance strategy, a way to react to those changes in spend in a, hopefully, automated way, is critical to helping control that spend.Jesse: This really gets to the heart of why is cloud cost management important? It could be important for different reasons for different parts of the organization. Account structure, tagging, all of these different things can be important for different parts of the organization for different reasons. And that’s fine. The important thing is to socialize those reasons why to all the different parts of the company so that everybody understands what’s at stake.Everybody understands how they can collaborate and create these best practices together. This really dives into the idea of behaviors and systems. I know it sounds a little bit not within the vein of engineering work, and finance, and cloud cost management, but what kind of behaviors do you ultimately want to see within your teams? What kind of actions do you want to see your engineers taking? Do you want them to start thinking about cost in all of their architecture discussions?Do you want them to review the budgets that you’ve created for them every month? Every week? During stand-up meetings? What kind of things do you ultimately want to see them doing on a regular basis that maybe they aren’t doing right now, that maybe would ultimately help the company succeed with all of this cloud cost management work that you’re creating? And again, going back to the idea of making the right thing the easy thing to do, how can you improve the existing technical systems that you have within your organization to make the right thing the easy thing to do?How can you change your CI/CD pipelines? How can you change the tools that you’re using for cost visibility, like Looker, or Tableau, or SageMaker, or something else, such that teams can quickly and easily self-service the information that they need to make their decisions to go about their days, go about their work more easily?Pete: So, Jesse, you’re saying that it’s a mixture of software and culture? Kind of sounds like DevOps a little bit, doesn’t it? [laugh].Jesse: Yeah. Yeah, it kind of does.Pete: Yeah, it kind of does. So, you know, I think all of that is to say, it’s hard work, it’s not going to come easy, but how would we get started? Like, when we enter into an engagement with one of our clients, we’re coming in from total outsiders and we’re trying to navigate through a company with complex communication structures, and maybe teams that are entrenched in different ways. How do we get started? Well, we dive in; we start with big numbers, right?What are your top ten places your money goes, just by service? I’ll answer it for you. It’s probably EC2, S3, RDS, and then dealer’s choice for the last ones, maybe data transfer, maybe Lambda, if you’re really weird. And if Lambda is in your top five, you should absolutely give us a call because that should not be the case. [laugh].But start with those big numbers, understand where the money is going. But then go to the next level in. Okay, within EC2, where is your spend going? Or the dastardly EC2 ‘other cost’ category; okay’s the money going? Is it in regional data transfer, which is also what’s called cross-AZ data transfer? Is it in your NAT gateways? Why?That’s the next question. Why is the spend high in that area? You may not be able to understand because it may not be tagged—we find that a lot—but start asking questions. And that’s what we do: We start reaching out to technical folks within the company. We’d say, “Hey, we see you’ve got a high amount of usage on EMR, but they’re all clusters that are running 24/7. They’re not scaling up and down as the jobs are happening. Who knows more about EMR?” And we just start asking questions. And we’re asking them, “Well, are you doing anything on the cost optimization side? Have you tried to do anything cost optimization-wise to reduce it, and you haven’t been able to? How does this infrastructure scale? Does it scale linearly with the number of users? Does it scale in a different way? Who are the consumers?”And then you kind of even go another level down to see, do you find anything that just looks odd? I saw on one account for a client we were working with, VPC costs were just extremely high, much higher than I’ve ever seen before. What was interesting is that the cost was not data-transfer related; it was the pure number of endpoints that they had created that that cost far outweighed any other costs to data transfer; there was just a piece of technical debt that they were aware of, but the structure of their multi-accounts, they just couldn’t do anything about it. But again, you’re looking for things like that. And you know that you are doing a good job if, essentially, you can get to the end of this process—which could take months and it could take years depending on your scale—is if you can answer this question that if your customers, or users, or consumers of the applications on your cloud service if they increased by 200%, 500%, 1,000%. What would happen to your cloud spend? How would it change? That’s the end game you’re trying to get to. That’s the unit economics, the unit economic model and forecasting, and now you’re a superhero because now you can answer that question that not a lot of people are able to answer with their cloud usage.Jesse: I also want to add that, as you’re asking questions, you’re going to find teams that specifically will tell you, “We created this infrastructure in this way because security told us to,” or, “Because our business requirements say that we have an SLA that means we need to keep data for this amount of time at this level of availability.” And that’s totally fine. That doesn’t mean that you need to necessarily change those requirements. But now you might have a dollar amount for those business decisions. Now, you might ultimately be able to say, okay, our product SLA may say that we need to keep data for 90 days, but keeping data for 90 days, that business decision is costing us hundreds of thousands of dollars every month because of the sheer volume of data that we now have to keep. Is that something that we ultimately are okay with? And are we okay spending that much money every month to keep this business decision, or do we need to revisit that business decision? And that’s only something that you and your teams can decide for yourselves.Pete: Awesome. These are great questions. You could also send us a question lastweekinaws.com/QA. We would love to spend some time diving into it and just helping you out and helping you get through your day. That’s what we’re here for.If you enjoyed this podcast, please go to lastweekinaws.com/review and give it a five-star review on your podcast platform of choice, whereas if you hated this podcast, please go to lastweekinaws.com/review. Give it a five-star review on your podcast platform of choice, and tell us what is your favorite EC2 instance to turn off for your engineers.Jesse: [laugh].Pete: Thanks, everyone.Announcer: This has been a HumblePod production. Stay humble.

Links: Unconventional Guide to AWS Cost Management: https://www.duckbillgroup.com/resources/unconventional-guide-to-aws-cost-management/ Migrate from Oracle to Amazon Aurora: https://aws.amazon.com/getting-started/hands-on/migrate-oracle-to-amazon-aurora/ TranscriptCorey: This episode is sponsored in part by LaunchDarkly. Take a look at what it takes to get your code into production. I’m going to just guess that it’s awful because it’s always awful. No one loves their deployment process. What if launching new features didn’t require you to do a full-on code and possibly infrastructure deploy? What if you could test on a small subset of users and then roll it back immediately if results aren’t what you expect? LaunchDarkly does exactly this. To learn more, visit launchdarkly.com and tell them Corey sent you, and watch for the wince.Pete: Hello, and welcome to the AWS Morning Brief: Fridays From the Field. I am Pete Cheslock.Jesse: I’m Jesse DeRose.Pete: We’re coming at you again with some more listener questions from the Unconventional Guide to AWS Cost Management. I’m excited. People are listening to us, Jesse.Jesse: This is fantastic. I’m really excited that we have one fan. I’ve always wanted one fan.Pete: Well, two fans now. Maybe even more because we keep getting questions. And you can also be one of our Friends of the Pod by going to lastweekinaws.com/QA. And you can give us some feedback, you can give us a question and, like, will totally answer it because we like Friends of the Pod.Jesse: We may or may not enter you into a raffle to get a Members Only jacket that’s branded with ‘Friends with the Pod.’Pete: We should get some pins made, maybe.Jesse: Ohh…Pete: I think that's a good idea.Jesse: Yeah.Pete: So, what are we answering today, or attempting to answer for our listener, Jesse?Jesse: So today, we’ve got a really great question from [Godwin 00:01:20]. Thank you, Godwin, Godwin writes, “I truly believe that the system that I support is, like, a data hoarder. We do a lot of data ingestion, we recently did a lift-and-shift of the system to AWS, we use an Oracle database. The question is, how do I segregate the data and start thinking about moving it out of traditional relational databases and into other types of databases? Presently, our method is all types of data goes into a quote-unquote, ‘all-purpose database,’ and the database is growing quite fast. Where should I get started?”Pete: Well, I just want to commend you for a lift-and-shift into Amazon. That’s a Herculean feat, no matter what you’re lifting and shifting over. Hopefully, you have maybe started to decommission those original data centers and you don’t just have more data in twice as many locations.Jesse: [laugh]. But I also want to call out well done for thinking about not just the lift-and-shift, but the next step. I feel like that’s the thing that a lot of people forget about. They think about the lift-and-shift, and then they go, “Awesome. We’re hybrid. We’re in AWS, now. We’re in our data center. We’re good. Case closed.” And they forget that there’s a lot more work to do to modernize all those workloads in AWS, once you’ve lifted and shifted. And this is part of that conversation.Pete: Yeah, that’s a really good point because I know we’ve talked about this in the past, the lift-and-shift shot clock: when you don’t start migrating, start modernizing those applications to take advantage of things that are more cloud-native, the technical debt is really going to start piling up, and the folks that are going to manage that are going to get more burnt out, and it really is going to end poorly. So, the fact you’re starting to think about this now is a great thing. Also, what is available to you now that you’re on AWS is huge compared to a traditional data center.Jesse: Yeah.Pete: And that’s not just talking about the—I don’t even know if I’ve ever counted how many different databases exist on Amazon. I mean, they have a database for, at this point, every type of data. I mean, is there a type of data that they’re going to create, just so that they can create a database to put it into?Jesse: Wouldn’t surprise me at this point.Pete: They’ll find a way [laugh] to come up with that charge on your bill. But when it comes to Oracle, specifically Oracle databases, there’s obviously a big problem in not only the cost of the engine, running the database on a RDS or something to that effect, but you have licensing costs that are added into it as well. Maybe you have a bring-your-own-license or maybe you’re just using the off-the-shelf, but the off-the-shelf, kind of, ‘retail on-demand pricing’ RDS—I’m using air quotes for all these things, but you can’t see that—they will just have the licensing costs baked in as well. So, you’re paying for it—kind of—either way.Jesse: And I think this is something also to think about that we’ll dive into in a minute, but one of the things that a lot of people forget about when they move into AWS says that you’re not just paying for data sitting on a piece of hardware in a data center that’s depreciating, now. You’re paying for storage, you’re paying for I/O costs, you’re paying for data transfer, to Pete’s point, you’re also paying for some of the license as well, potentially. So, there’s lots of different costs associated with keeping an Oracle Database running in AWS. So, that’s actually probably the best place to start thinking about this next step about where to get started. Think about the usage patterns of your data.And this may be something that you need to involve engineering, maybe involve product for if they’re part of these conversations for storage of your product or your feature sets. Think about what are the usage patterns of your data?Pete: Yeah, exactly. Now, you may say to yourself, “Well, we’re on Oracle”—and I’m sure people listening are like, “Well, that’s your problem. You should just move off of Oracle.” And since you can’t go back in time and undo that decision—and the reality is, it probably was a good decision at the time. There’s a lot of businesses, including Amazon, who ran all of their systems on Oracle.And then migrated off of them. Understanding the usage patterns, what type of data is going into Oracle, I think is a big one. Because if you can understand the access patterns of the types of data that are going in, that can help you start peeling off where that data should go. Now, let’s say you’re just pushing all new data created. And we don’t even know what your data is, so we’re going to take some wild assumptions here on what you could possibly do—but more so just giving you homework, really—thinking about the type of data going in, right?If you’re just—“I’m pushing all of my data into this database because someday we might need to query it.” That’s actually a situation where you really want to start thinking of leveraging more of a data warehouse-style approach to it, where you have a large amount of data being created, you don’t know if you’re going to need to query it in the future, but you might want to glean some value out of that. Using S3, which is now available to you outside of your data center world, is going to be super valuable to just very cheaply shove data into S3, to be able to go back in later time. And then you can use things like Athena to ad hoc query that data, or leverage a lot of the ingestion services that exist to suck that data into other databases. But thinking about what’s being created, when it is going into places is a big first step to start understanding, well, how quickly does this data need to come back?Can the query be measured in many seconds? Can it be done ad hoc, like in Athena? Does it need to be measured in milliseconds? What’s the replication that needs to happen? Is this very valuable data that we need to have multiple backups on?Is it queried more than it’s created? Maybe you need to have multiple replica reader databases that are there. So, all these types of things of really understanding just what’s there to begin with, and it’s probably going to be in talking to a lot of engineering teams.Jesse: Yeah, you can think about this project in the same way that you might move from a monolith to a microservice architecture. So, if you’re moving from a monolith to a microservice architecture, you might start peeling away pieces of the monolith, one at a time. Pieces that can easily be turned into microservices that stand on their own within the cloud, even if they’re running on the same underlying infrastructure as the monolith itself within AWS. And then, as you can pull those pieces away, then start thinking about does this need to be in a relational database? Does this need to have the same amount of uptime and availability as the resources that are sitting in my Oracle Database right now?All those things that Pete just mentioned, start thinking about all of those components to figure out where best to pull off the individual components of data, and ultimately put them in different places within AWS. And to be clear, there’s lots of great guides on the internet that talk about moving from your Oracle database into, gosh, just about any database of choice. AWS even has specific instructions for this, and we’ll throw a link in the [show notes 00:09:02].They really, really want you to move this data to RDS Aurora. They go through painstaking detail to talk about using the AWS schema conversion tool to convert your schema over; they talk about the AWS database migration service to migrate the data over, and then they talk about performing post-migration activities such as running SQL queries for validating the object types, object count, things like that. I think that a lot of folks actually don’t know that the database migration service exists, and it’s something worth calling out as a really powerful tool.Pete: Yeah, the Amazon DMS service is honestly I think, a super-underrated service that people just don’t know about. It has the ability to replicate data from both on-premises databases to Amazon databases but also databases already running on Amazon. You could replicate from a database running on EC2 into Aurora. You could replicate that into S3—you know, replicate data into S3 that way, bringing things into sync—replicate that data into S3, and then maybe use it for other purposes. It can replicate data from DocumentDB into other sources.So, they’re clearly doing a big investment in there. And to Jesse’s point, yeah, Amazon really wants this data. So, talk to your account manager as you’re testing out some of these services. Do a small proof of concept, maybe, to see how well it works, if you can understand the queries, or you can point your application over at an Aurora database with some of this data migrated in; that’s a great way to understand how well this could work for your organization. But as Jesse mentioned, they do want that data in Aurora.So, if it turns out that you’re looking at your—you know, migrate some data in there, and it’s starting to work, and you’re kind of getting a feel for the engineering effort to migrate there, stop. Talk to your account manager before you spend any more money on Aurora because it’s very likely that they can put together a program—if a program doesn’t already exist—to incentivize you to move that data over; they can give you subject matter expertise; they can provide you credits to help you migrate that data over. Don’t feel like you have to do this on your own. You have an account team; you should definitely reach out to them, and they will provide you a lot of help to get that data in there. They’ve done it for many of their other clients, and they’re happy to do it for you because they know that, long term, when you move that data to Aurora, it’s going to be very sticky in Aurora.You’re probably not going to move off of there. It’s a long game for them; that’s how they play it. So, check out those services; that could be a really great way to help you get rid of your Oracle addiction.Jesse: Yeah, and if you’re able to, as we talked about earlier, if you’re able to identify workloads that don’t need to run in a relational database, or don’t need to run in, maybe, a database at all, for that matter, stick that data in S3. Call it a day. Put them on lifecycle management policies or different storage tiers, and use Athena for ad hoc queries, or maybe Redshift if you’re doing more data warehouse-style tasks. But if that data doesn’t need to live in a relational database, there are many cheaper options for that data.Pete: Exactly. But one last point I will make is don’t shove it into MongoDB just because you want to have schema-less, or—Jesse: Please.Pete: —think about what you’re going to use it for, think about what the data access patterns because there is a right place for your data. Don’t just jump into no-SQL just ‘cause because you’ll probably end up with a bigger problem. In the long run.Corey: If your mean time to WTF for a security alert is more than a minute, it's time to look at Lacework. Lacework will help you get your security act together for everything from compliance service configurations to container app relationships, all without the need for PhDs in AWS to write the rules. If you're building a secure business on AWS with compliance requirements, you don't really have time to choose between antivirus or firewall companies to help you secure your stack. That's why Lacework is built from the ground up for the Cloud: low effort, high visibility and detection. To learn more, visit lacework.com.Pete: So Jesse, I’m looking at our list of questions. And it turns out, we have another question.Jesse: Ohh.Pete: Two questions came in.Jesse: You like me, you really like me!Pete: It’s so great. Again, you can also send us a question, lastweekinaws.com/QA. You can go there, drop in a question and feel free to put your name. Or not; you can be anonymous, it’s totally fine. We’ll happily answer your question either way. So Jesse, who is our next question from? What is this one about?Jesse: This one’s from [Joseph 00:13:19]. They write in, “Hey, folks. Love the show. Longtime listener, first-time caller.” Thank you. “I would love to know how people manage their costs in AWS Batch. Jobs themselves can’t be tagged for cost allocation, which makes things a bit complicated.” Lord Almighty, yes, it does. “How best should I see if the jobs are right-sized? Are they over-provisioned in terms of memory or compute? What’s the best way to see if EC2 is my better choice, versus Fargate, versus other options? How can I tell if the batch-managed cluster itself is under-utilized?”Pete: Oof. This is a loaded question with a lot of variables.Jesse: Yeah. And so we’re going to break it down because there’s definitely a couple questions here. But I want to start off with what AWS Batch is, just really quick to make sure everybody’s on the same page here. AWS Batch, effectively, is a managed service in AWS that schedules it and runs your batch computing jobs on top of AWS compute resources. Effectively, it does a lot of the heavy lifting configuration for you so you can just focus on analyzing the results of those queries.Pete: Yeah, exactly. And Batch supports a really wide variety of tooling that can operate this, and that’s why it’s hard for us to give, specifically, how you might optimize this, but I think some of the optimizations actually mirror a lot of the optimizations we’ve done with optimizing EMR clusters and things of that nature, where you’re running these distributed jobs. And you want to make sure that if you’re running straight off of EC2 instances, then you want to make sure that they are essentially maxed out. If the CPU is anything less than 100% for an on-demand instance, then there’s wasted, or there’s opportunity for improvement. And so making sure that your jobs are sized appropriately and balancing out memory and CPU so that, effectively, you’re using all of the memory and all of the CPU, that’s a real basic first step.But honestly, a lot of folks kind of miss out on that. They just kind of run a job and go off and do their own thing. They never really go back and look at those graphs. You can go to CloudWatch, they’re all going to be there for you.Jesse: Yeah. And to this point, there’s always an opportunity to make these workloads more ephemeral. If you have the opportunity to make it more ephemeral, please, please, please, please, absolutely do so. Unless your batch job needs to run 24/7. We’ve seen that in a few cases where they have, essentially, clusters that are running 24/7, but they’re not actually utilized regularly; the workloads are only scheduled for a short amount of time.So, if you don’t need those batch jobs running 24/7, please, by all means, move to more ephemeral resources, like Fargate. Fargate on Spot, Spot Instances in general, or even Lambda, which AWS Batch now supports as well.Pete: Yeah, it has some step function support, which is pretty interesting. Yeah, this is a great opportunity to aggressively—aggressively—leverage Spots, if you’re not currently today. The reality is that check out Fargate on Spot if you don’t need, like, a custom operating system, you don’t need a custom EBS volume size. If you do, then EC2 on Spot is probably the best option that you really have. But really do not want to be running anything on on-demand instances. Even on-demand instances with a really good savings plan, you’re still leaving money on the table because Spot Instances are going to be a lot cheaper than even the best savings plan that’s out there.Jesse: And I think that’s a good point, too, Pete, which is if you do need to run these workloads on-demand, 24/7, think about if you can get away with using Spot Instances. If you can’t get away with using Spot Instances, at least purchase a savings plan if you don’t do anything else. If you take nothing else away from this, at least make sure that you have some kind of savings plan in place for these resources so that you’re not paying on-demand costs 24/7. But in most cases, you can likely make them more ephemeral, which is going to save you a lot more money in the long run.Pete: Yeah, exactly. That’s the name of the game. I mean, when we talk to folks on Amazon, the more ephemeral you can make your application—the more you can have it handle interruption—the less expensive it will be to operate. And that goes from everywhere from Spot Instances and how they’re priced, right? If you just get a normal Spot Instance, it will have a really aggressive discount on it if you need zero time in advance before interruption.So, if that instance can just go in at any second, then you’ll get the best discount on that Spot Instance. But if your app needs a little time, or runs for a defined period of time—let’s say your app runs for one hour—you can get a defined duration Spot of one hour, you’ll get a great discount still and you’ll only pay for however long you use it, but you will get that resource for one whole hour, and then you’ll lose it. If that’s still too aggressive, there’s configurable options up to six hours. Again, less discount, but more stability in that resource. So, that’s the trade-off you make when you move over to Spot Instances.Jesse: So, I also want to make sure that we get to the second part of this question, which is about attributing cost to your AWS Batch workloads. According to the AWS Batch documentation, you can tag AWS Batch compute environments, jobs, job definitions, and job queues, but you can’t propagate those tags to the underlying resources that actually run those jobs. Which to me, kind of just defeats the point.Pete: Yeah. [sigh]. Hashtag AWS wishlist here. You know, again, continuing to expand out tagging support for things that don’t support it. I know we’ve seen kind of weird inconsistencies, and just even, like, tagging ECS jobs and where you have to tag them for they’re to apply.So, I know it’s a hard problem, but obviously, it’s something that should be continually worked out on because, yeah, if you’re trying to attribute these costs, you’re left with the only option to run them in separate Amazon accounts, which solves this problem, but again, depending on your organization, could increase just the management overhead of those. But that is the ultimate way. I mean, that is the one way to ensure 100% of costs are encapsulated to a service is to have them run in a dedicated account. The downside being is that if you have a series of different jobs running across a different, maybe, business units, then obviously that’s going to break down super quick.Jesse: Yeah, and it’s also worth calling out that if there’s any batch jobs that need to send data to different places—maybe the batch job belongs to product A, but it needs to send data to product B—there’s going to be some amount of data transfer either across regionally or across accounts in order to share that data, depending on how your organization, how your products are set up. So, keep in mind that there are potentially some minor charges that may appear with this, but ultimately, if you’re talking about the best ways to really attribute costs for your AWS Batch workloads, linked accounts is the way to go.Pete: Yeah. If you need attribution down to the penny—some of our clients absolutely do. For invoicing purposes, they need attribution for business unit down to the penny. And if you’re an organization that needs that, then the only way to get that, effectively, is segmented accounts. So, keep that in mind.Again, until Amazon comes out with the ability to get a little bit more flexible tagging, but also, too, feel free to yell at your account manager—I mean, ask them nicely. They are people, too. But, you know, let them know that you want this. Amazon builds what the customers want, and if you don’t tell them that you want it, they’re not going to prioritize it. I’m not saying if you tell them, you’re going to get it in a couple of months, but you’re never going to get it if you don’t say anything. So, definitely let people know when there’s something that doesn’t work the way you expect it to.Jesse: Absolutely.Pete: Awesome. Wow. Two questions. I feel it’s like Christmas. Except—Jesse: [laugh].Pete: —it’s Christmas in almost springtime. It’s great. Well, again, you, too, can join us by being a Friend of the Pod, which Jesse really loves that one for some reason. [laugh].Jesse: Yeah. Don’t know why, but it’s going to be stuck in my brain.Pete: Exactly. You too can be a Friend of the Pod by going to lastweekinaws.com/QA and you can send us a question. We would love to spend some time in a future episode, answering them for you.If you’ve enjoyed this podcast, please go to lastweekinaws.com/review. Give it a five-star review on your podcast platform of choice, whereas if you hated this podcast, please go to lastweekinaws.com/review and give it a five-star rating on your podcast platform of choice and tell us why you want to be a Friend of the Pod. Thank you.Announcer: This has been a HumblePod production. Stay humble. 

Links:Unconventional Guide to AWS Cost Management:https://www.duckbillgroup.com/resources/unconventional-guide-to-aws-cost-management/ TranscriptCorey: This episode is sponsored in part by LaunchDarkly. Take a look at what it takes to get your code into production. I’m going to just guess that it’s awful because it’s always awful. No one loves their deployment process. What if launching new features didn’t require you to do a full-on code and possibly infrastructure deploy? What if you could test on a small subset of users and then roll it back immediately if results aren’t what you expect? LaunchDarkly does exactly this. To learn more, visit launchdarkly.com and tell them Corey sent you, and watch for the wince.Pete: Hello, and welcome to the AWS Morning Brief: Fridays From the Field. I am Pete Cheslock.Jesse: I’m Jesse DeRose.Pete: We’re back again. And we’re here. We made it, Jesse.Jesse: I was worried. This was a journey. Thank you, everybody, for coming on this journey with us.Pete: It was quite an experience going through the Unconventional Guide to AWS Cost Savings. We’ve made it. I just can’t believe we’re here.Jesse: Yeah.Pete: So, what are we talking about today for the culmination of our magnum opus of cost savings optimizations?Jesse: This is a fun one. And I know I keep saying that this is my favorite about everyone, but I have to admit that this one, this topic today probably is my absolute favorite. This one I get really nerdy over. Today, we’re talking about how to predict your future and make your CFO happy. No—spoiler alert—there are not any crystal balls involved in this one. There’s no stock market conversations.This is talking about how you can use all of the different things that we’ve talked about throughout the course of this Unconventional Guide to really bring it all together into a couple ideas that will help you better understand your cloud costs, and really better understand your business, I think.Pete: Yeah. All of the things we talked about really lead up to this one, which is the clients of ours that are the most mature, who are incredibly optimized in their Amazon usage, are the ones who have adopted a majority of these specific items. They all lead to this last one, that ability to predict your future usage based on something that’s happening internally, or if a salesperson comes to you and says, “Hey, we’re about to close this deal, but I need to discount our service.” People are going to start wanting to know well, what is the cheapest that you could sell your service for and still have a positive gross margin?Jesse: Yeah. So, if you’ve done a lot of the things that we’ve talked about in the last couple episodes—I apologize, I know homework’s not the best for a podcast—but if you’ve had the opportunity to work on some of those things, you should have a ton of valuable insights into your spend. We’re talking about tagging, and showback models in particular, maybe even a chargeback model. But you can ultimately use all of this data to better understand what is your forecasted spend is going to look like with a new potential customer coming onto the platform? Or if you get into the topic that we’re going to talk about today, which is mostly unit economics, you can really understand how much can I discount my service and still make a profit, like Pete mentioned?Pete: Yeah, I mean, imagine there’s a global pandemic that happens, and it causes your usage to spike by 500% within the course of a month. How did your spend change? Do you know where it changed? And did it change in ways that you were expecting it to? Like, my databases grew by a lot, and this other thing didn’t grow by very much.Like, that would be expected. But also another thing that—a question that we actually like to ask a lot of our clients, if your sales just doubled overnight, okay would your spend change? Where are the places that are most expensive to operate your service? And again, this is kind of generic. I’ve worked in a lot of SaaS services, so I always think of sales, but just think of whether you’re using the cloud for a SaaS service that you provide and sell, like, B2C, things like that, or B2B, you still have users.They might be internal users. Well, what if your users doubled overnight? What if half the company was using your internal service and now the whole company is? How does that change your usage?Jesse: And it’s also important to think about not just your AWS usage, but all of the other services that you use that support your overall business model: things like monitoring and observability tools, logging vendors, maybe third-party sim tools. All of these are affecting your overall total infrastructure cost and are all part of this conversation. So, it’s really important to start thinking about those architecture diagrams. Remember, when we said, way, way back at the beginning of this conversation, to overlay costs on top of your architecture diagram, understanding that, understanding what parts of your product or what parts of your architecture are the most expensive will really help you understand what’s going to change?Pete: Yeah, let’s say you’ve got a six-figure bill to Datadog or one of the big log management vendors out there, but inside of that bill, is that all just evenly spread across the whole business? What if your log vendor was—the entire spend was all by one service that some developer left the debug logging enabled for? You know, you’d want a way of understanding that maybe that spend was concentrated in maybe a non-production aspect of your account. Because then again, that wouldn’t grow, right? That wouldn’t affect your growth in your sales the same way as if maybe all of your services were equally sending logs of a certain volume over.So, all of those extra services, they all add up, and we see it more and more, as more of our clients start adopting more than just Amazon services: they might be adopting a Snowflake, they might be adopting third-party services running databases running in other services, or EMR type workloads that are not on EMR, and they’re running on Qubole or things like that. There’s just a lot of these services that more and more people are consuming from that fall outside of just the AWS invoice.Jesse: And this also gets back to not just architecture diagrams, but also tagging and showback models, cost visibility, really understanding where your spend is going. And this is fantastic to understand where your spend is going, but finance is probably going to want something a little bit more than this. It’s not just about how much are we spending, or where are we spending it, and maybe it’s not even a finance question. Maybe this is a sales conversation, assuming that you’re a SaaS company. Maybe this is, as Pete mentioned before, “Hey, we want to understand where can we provide discounts? What services can we ultimately discount to negotiate getting new customers on the platform?”Pete: So, Jesse, we hear a lot of these terms a lot, and I’d love a ‘explain like I am a five-year-old’ version of it, but we hear a chargeback. And we hear showback, and honestly, I’ve never worked at those massive companies where you might implement these things, but can you give us just a real quick—for all the listeners out there, when we say showback, what does that mean? And when we say chargeback, what does that mean?Jesse: So, a showback model essentially takes all of your cloud costs, all of your total infrastructure spend, your AWS spend, all the third-party spend, and it shows every team, every product, every microservice, maybe, depending—or maybe even business unit, depending on how your organization is split up—it shows each one of those units, how much they are actually spending, how much they’re actually using these different cloud vendors. So, this is where tagging comes in super handy because if you’ve tagged all of your taggable resources, and properly attributed all of your cloud costs with tagging and linked accounts, you have a very clear idea of who’s spending what. You know very clearly, maybe 70 to 80% of your total infrastructure spend is related to one particular product because all the cost is attributed to one particular product. And maybe that’s something you didn’t know before. Maybe now you know okay, maybe that product needs to be a little bit more expensive so that we can make sure that we are making money off of it, or profiting off of it, whereas other services can be discounted because they’re not as expensive.Whereas in a chargeback model, you are ultimately not just showing each of these teams, hey, here’s how much you spent on AWS and Datadog usage and all these other vendors every month, you’re actually charging them for that usage. You’re actually pulling their cloud costs from their budget.Pete: Yeah. They might actually have a budget of money. It’s all—if you want to really explain like I’m five, it’d be like, I give my child their—they get $1 for all of the tasks that they do throughout the week, I don’t actually give them the money because I usually have to subtract out their, like, Roblox spend of the week [laugh] and things like that. It’s all virtual, but at the end of the day, you know, we’re kind of virtually giving this business unit some money, and then, kind of, virtually charging them for their services within.Jesse: Yeah. And this is mature. We don’t see a lot of companies doing this. This is hard because you have to take other steps first to get here. And so this is why we harp so much on cost attribution through tagging and through linked accounts.This is why we harp so much on cost visibility and overlaying those cloud costs on your architecture diagrams to understand all of this data to lead to this point, which is understanding, where, how much is my primary product actually costing us? How much is my secondary product actually costing us? Or maybe how much is this business unit costing us in terms of total infrastructure spend?Pete: Yeah, I mean, I can kind of share my history with this at previous companies is that, again, eventually someone in the financial department is going to say, “What was our cost for Amazon?” They specifically will want to know the production cost because that figures into a term called gross margin, which you often hear at SaaS businesses. Gross margin is basically you take all the revenue that came in and you subtract away what it took to support that revenue. And mostly, that is just the Amazon bill and these other vendors, perhaps, and you end up with a percentage. And hopefully, it’s a positive percentage.It means you’re theoretically making money at a gross, I mean, obviously, before you pay salaries and all those other items, but that being kind of beside the point for now, that number, you’re probably going to get asked for. So, you wouldn’t want to give like your straight Amazon bill, like, “Oh, well, we spent $100,000 last month,” because some of that spend was probably in research and development; it was probably in a development account or a QA account. You really just want your product spent. So, at a previous company, the first step we took was break out our spend via production and development, just two criteria. Now, for us because we started with just a handful of accounts—this was before a lot of accounts were more prevalent, before organizations—before it was easy to handle a lot of accounts—we had a Prod and a Dev. Super easy. Look at Prod, look at Dev. There’s the two bills.But then as time went on, we needed to get more granular. We were running some development workloads, testing out new databases at scale in kind of a hidden dark deployed mode, in production. Well, we want to subtract that spend from there. And that requires tagging. I mean, that’s why we really harped on tagging for a couple of episodes because tagging is the only way you’re going to be able to do that.Now, we see more often a lot of our clients do maybe an account per product, or account per business unit. Those are, again, really effective ways to corral your spend to make it really easy to break it out and add it up. It’s really just trying to break it down to the most reasonable spend unit possible that you can then play around with and adjust. Mostly to go back to your CFO when they asked you, “Hey, I need to know this specific answer.” You’ve got it hopefully somewhat available.Jesse: Okay, so this is where we’re going to start talking about unit economics. And hopefully, your eyes will not glaze over. I want to make sure that—this is important, this is really actually beneficial. It’s not just a specific economic thing that you learned back in Econ 101. This is actually going to be useful and beneficial for your organization.So, unit economics describes your product in terms of revenues and costs in relation to a unit KPI—that’s where the ‘unit’ term comes from in ‘unit economics’—that tracks closely with customer demand. So, that’s a really gross definition, I know, and I apologize.Pete: You know, and we can even extend that a little bit further and give some good examples. Like, maybe if you are a website that provides eLearning services, your unit might be the number of daily active users or thousands of daily active users, right, could be a thing. That could be a unit that you’re selling. I actually worked at a SaaS company where we sold a piece of software that would run per server, and we broke our unit down to the servers—the things that we sold—down to that level.Jesse: Yeah, if you are in the airline industry, for example, your unit would probably be every passenger. How many passengers are you able to sell tickets to on every plane? What do those costs look like?Corey: If your mean time to WTF for a security alert is more than a minute, it's time to look at Lacework. Lacework will help you get your security act together for everything from compliance service configurations to container app relationships, all without the need for PhDs in AWS to write the rules. If you're building a secure business on AWS with compliance requirements, you don't really have time to choose between antivirus or firewall companies to help you secure your stack. That's why Lacework is built from the ground up for the Cloud: low effort, high visibility and detection. To learn more, visit lacework.com.Pete: And you don’t need just, like, one unit. Maybe you have one unit for your whole platform like the whole gross production spend breaks down into one specific unit, you could do that. But you could also have units at a per-service level because maybe it’s like you’re processing a lot of documents. I worked for an email archiving company, forever ago, and our unit was the amount of emails that were indexed and archived so we could figure out, we might have one customer who just didn’t generate a lot of emails, but they had tons of users. Well, one of our units was the volume of emails that we were indexing and archiving for that customer, whereas on the flip side, if maybe our spend was driven more by user count, and not document count, maybe that’s what we want our unit to be, is per user.Jesse: Yeah. It’s really important to call out that you might have a single easy-to-define unit; you might have a more complex relationship that’s weighted with a couple different factors of different components of the architecture. But ultimately, your unit KPI and how you break out your costs to support your customers will be unique to your overall business.Pete: Exactly. And this is where you’re only going to find this answer out with a lot of conversations, internally. It could come to you pretty easily, you know, just based on how your business is. But I think for a lot of folks using Amazon, especially if you’re just in a specific business unit inside of a broader business, it could be a little bit more challenging to figure out. But what you’re really trying to do is just figure out, when X changes, our spend changes, and we spend more or we spend less. Try to solve for X. That’s really what you’re trying to do.Jesse: Okay. So, now we’ve covered the unit KPI part of this conversation. Awesome. So, we’re done, Pete, right? We just take our AWS bill and then—Pete: Yeah.Jesse: —divide it by the unit and we’re done.Pete: So, easy. I’ve got my unit. I’ve got my bill. I got an iPhone that can do a calculator. Good to go.Jesse: [laugh]. Good. We’re done. Well, wait. What about if I have multiple AWS accounts? Wait, what if I have multiple different products?Pete: Yeah, that’s… I mean, I kind of calculator. I mean, I might be here all day, but…Jesse: [laugh]. I’ve got a whiteboard. We’ve got some time.Pete: Yeah, we got time. That’s a great point, though. Again, what if you do have things that are just spread all over the place? What if you’ve got two different products, two different services inside the same account? Because of course, you would. That’s a super normal thing. I’m not even saying that sarcastically. That’s a super normal thing.Jesse: Absolutely.Pete: Well, how do you handle this? How do you handle shared services?Jesse: Yeah. I mean I—Pete: We could go on for too long on that one, but these are questions you really want to start asking.Jesse: Yeah. And remember that you’re potentially going to have different unit KPIs for different products, for different business units. That’s fine. That’s expected. But make sure that you are measuring appropriately for each of those.The incoming costs, the incoming revenues, and costs for each of these isn’t going to change. That’s coming from your tagged usage and your linked accounts, but maybe the unit that you’re dividing that spend by is going to change, and that’s fine. This is where a spreadsheet comes in super, super handy. I love my Excel spreadsheets for this. Very, very easy to just bring in all of the bill data across different accounts, and really clearly attribute this spend is for the service, or the spend is for this product, or the spend is for this business unit, and then divide that by the unit that we have in question to get your actual unit KPI, to get your unit economic metric.Pete: Yeah, and this is where the superpower comes in. Once you have this number, now you can better understand and make product-level decisions. Again, whether you’re a SaaS product with a product you’re selling to external customers or building an internal tool, your product is the thing that the internal users consume. Your product decisions can now be driven by this. I mean, I have recollections of conversations with product teams, where they would talk about certain services internally, how they wanted to expand and do all the stuff with it.And I said, “Well, right now that one service represents one-third of our total spend, right? Our gross margin, that is one-third. But we looked at the users, and it’s only being used by one percent.” When you have these big numbers and saying, “Wow, the company spends a third of their money on something one percent of people use,” then maybe that’s not the place we want to be investing product decisions into. Maybe it is, but you don’t know enough to have that conversation unless you have this data.Jesse: Absolutely. I think there’s one other small caveat that we haven’t touched on that I do want to call out, and this comes back to your conversation about tagging. We have noticed that a lot of teams want to tag to a certain extent, and then start building their showback models immediately, which is great that you’ve got investment, you’ve got energy, you really want to get to that showback model, get to that chargeback model, that unit economic model space. But if your usage or cloud usage is not thoroughly tagged and accurately tagged, your resulting data is not going to be accurate either. So, we think about this in terms of a cost margin or a cost error.So, for example, if your production spend or your production usage is only 60% tagged, that means you’ve got 40% error in that data that’s coming in; your cloud spend for production has 40% error margin, which is huge.Pete: Yeah, exactly. Track your untagged spend, as well as your tagged spend. I mean, make sure you have a story for the things that are not tagged. That includes things like data transfer and things that maybe are not as taggable within AWS. That’s an important aspect of this that you’ll want to make sure you’re at least not forgetting about.Even if you can’t tag it, you don’t have a solution for it, make sure it’s in the back of your head that this is maybe not as accurate of a forecast because we’re just taking data transfer and dividing it by product versus actually looking at which product uses the most to transfer.Jesse: Yeah, and this is a tough concept, so don’t feel bad if you listen to this episode, again, don’t feel bad if you go download the Unconventional Guide from the Duckbill Group website—we’ll have the link in the [show notes 00:20:12]—this one is a tough concept because it brings in a lot of other moving parts to ultimately get at this one unifying really, really important idea. This is one that we see a lot of clients and potential clients struggle with. So, if you’re taking some time to understand this concept, you’re not alone.Pete: Exactly. This is the goal of all of the previous work, and this is something that you would measure in just a multi-year commitment in most businesses. And the larger the business is, the longer that work is going to take because it’s hard, there are a lot of moving pieces, and so many things need to be done in advance of all of this. And again, realize you’re not doing this work in a vacuum. There’s things that are moving and shifting as it’s all happening. So, don’t beat yourself up if you’re looking at this and thinking to yourself, “This is just a huge task. I’m never going to get this done.” It’s just not something that’s going to happen overnight.All right, well, hey, if you’ve enjoyed this podcast, if you’ve enjoyed this series, please go to lastweekinaws.com/reviewand give it a five-star rating on your podcast platform of choice, whereas if you hated this podcast, please go to lastweekinaws.com/review give it a five-star rating, but then tell us what’s the next series you want to have if you didn’t like this one.Also, don’t forget, you can give us your feedback and any questions that we’ll be continuing to answer in future episodes lastweekinaws.com/QA. You don’t need to put your name, can be totally anonymous. Give us your question. We’d love to dive into some of those topics.And finally, you can download our Unconventional Guide, the whole PDF of everything we’ve talked about at the Duckbill Group website. We’ll include that link in our [show notes 00:21:51] and you can head over there. Thanks again.Announcer: This has been a HumblePod production. Stay humble.

In this episode of The IoT Unicorn Podcast, Remco Ploeg of Altius discusses the challenges and opportunities of creating carbon neutral homes powered by Azure. Download Transcript Here Episode: 00:00 Pete:  Welcome to the IoT unicorn podcast. This is Pete Bernard from Microsoft. And this podcast is for anyone interested in the long-term technology trends in the IoT space and the journey from here to there. So let's get started. Thank you, Remco, I appreciate your time. Thanks for joining us here. So you're actually based in the Netherlands, and I'm here in Bellevue, Washington, and through the magic of... I'm actually using the Squadcast platform right now to connect and record this, but... Welcome to the IoT Unicorn.   00:16 Remco: Thank you, Pete, for having me.   00:18 Pete: Great, so you're based in the Netherlands, and I've been there a bunch of times. I used to go there actually, when I would go to Barcelona for NWC, there was always like a flight at the crack of dawn from Barcelona, and I would transfer in the Netherlands to get back to Seattle. And then one year, I kinda got smart and I said, you know, I'm gonna go to Amsterdam the night before, get a good night's sleep, and then I'll take the 10 AM Direct to Seattle. So I've spent a bunch of those kind of layover nights in Amsterdam, so it's an awesome, awesome place, but... Are you from there originally?   00:52 Remco: Yes, I'm from Rotterdam, so that's the other big city in the Netherlands so that's south of Amsterdam.   00:58 Pete: I see.   01:00 Remco: And I was a lot I think also on the same plane as yourself, so I had to do a lot... To Seattle with the direct flight in the morning. So... And also coming back with the flight early in the morning in Amsterdam again.   01:12 Pete: Yeah, yeah.   01:14 Remco: So I'm based at the moment in Utrecht, in the middle of the country.   01:18 Pete: Okay. So you've been at Altius for about a year or two, a couple of years?   01:25 Remco: Yeah, a little bit, a little bit more than a year. And the beginning of this year, there was an acquisition of Altius by Avanade...   01:34 Pete: Yes.   01:34 Remco: So, I'm joining formally Avanade from the first of January, the coming year 2021. But already, I think for 6 months, I'm working side by side with my Avanade colleagues.   01:45 Pete: And so I know you've been... I know another thing, I did a little research is you were... Altius was named Microsoft's AI Partner of the Year, so that's a big deal. So tell me more about that. What is Altius in AI? What is the... Do you consider Altius an AI company or more of an IoT company that's using AI or how would you describe it?   02:08 Remco: Yeah, so if you look at Altius, we are at the moment, with 400 people in the UK, Netherlands, and India. We've got a full focus on data and AI, so that's also our focus. So IoT is more or less, no side-job but we saw that...   02:21 Pete: A means to an end.   02:23 Remco: A means to an end, exactly. And then started when I joined Altius so that's one and a half years ago with also combining AI with IoT, 'cause I think that that's a great combination that we have there.   02:36 Pete: Yeah, for sure. A lot of times that we've had folks from Qualcomm and other... More telecom-related, I think we had BT on here recently, and it was like the 5G plus AI plus IoT or pick your network that certainly becomes kind of a game-changer for what you can do with a little bit of data, over a large number of sensors or a lot of data... [02:58] ____.   02:58 Remco: I think, already at the moment, even without 5G, 5G is of course already rolling out, we can already do a lot with IoT.   03:11 Pete: Yeah, so actually interesting on that topic. You talk about IoT, so how much do you think with AI and IoT are you seeing on the Cloud versus the Edge, and how much experience are you getting now, are you seeing in a more of an emergence of Edge AI in addition to the cloud AI or what are you seeing there?   03:31 Remco: Yeah, that's a good question. I don't know, 10 years ago, we were moving everything to the clouds, but now we see some of our clouds going back again, so I do a lot of projects around connected buildings. I think that's a great example with Edge computing, is the amount of sensors, especially in new buildings and smart buildings, it's so enormous that moving every data point to the cloud, it's sometimes technical, not possible, and the second, it's too expensive.   04:00 Pete: Right.   04:01 Remco: So we see there are movement back putting Edge devices in the building itself again, and also doing AI on the Edge device itself because of course what you don't want to do in a building is controlling lights in the cloud, for example, you want to control locally, if the internet connection is out there you want to still put on your lights on. And since a couple of years, we are also doing AI on the Edge and AI on the cloud, of course already a little bit longer, and we are controlling, for example, Edge Tech Systems with AI depending on the expected usage of the building and certain conditions. We put certain settings into the building when running those... Yeah, AI is now on Edge device that help us and our clients a lot to control those devices.   04:54 Pete: Yeah, also I had understood talking to some other customers too about Edge AI, especially in AI Vision, it's something where you want to actually process locally, just from a privacy perspective too. I mean there's a transport, obviously, you can't keep streams of data going up into Azure and doing live video analytics, I guess you could and some people do, but for a lot of maybe smaller implementations or other implementations, you wanna kinda do things locally, act locally and then keep the data on-prem basically, right?   05:24 Remco: Yeah, exactly. So we felt also again the smart, to smart build solutions, things like security with cameras, where we can analyze the data off the camera, to see, okay, is somebody trying to breach into the building or do something else. And we all use Edge AI for it.   05:42 Pete: Right, yeah, yeah, that's fantastic. Tell me a little bit, I heard about some of the carbon-neutral housing efforts that you were doing, so tell me more about that. That sounds intriguing.   05:54 Remco: Yeah. So for one of our clients in the Netherlands, it's a company called TBI, and it's a local company with around six or seven [06:02] ____, and one of their main goals is to be the most sustainable builder in the Netherlands, and for that they are building carbon-neutral houses. So that means that the houses are totally carbon-neutral from a building perspective, but also from a usage perspective. So the people that are going to live in that home, on normal conditions, don't need to pay any energy bill every year, so they are really zero... We call it zero-net houses in the Netherlands but carbon-neutral is a better naming for that.   06:35 Pete: So, they're generating power on-site as well, so they're generating their own power...   06:39 Remco: Exactly.   06:39 Pete: And then also all the building is smart enough and efficient enough where it's only consuming the power that's generated on-site. Is that basically?   06:47 Remco: Exactly, so they put solar panels on it to be extracted data from the solar panels. We've got all kind of meters in the home, smart meters, smart edge meters, all kind of temperature, humidity, CO2 sensors in the home. We extract all the data into the clouds, to do the analytics and to prove that the house is also carbon neutral.   07:07 Pete: And so do those exist? [chuckle] It seems like... That's a tough one, isn't it? I guess it depends on the size of your house and what you're doing in your house, but is that feasible?   07:17 Remco: Yeah. So in the Netherlands, we don't have big houses like in the US.   07:20 Pete: Right, right. No McMansions.   07:23 Remco: So it's more, yeah, so it's more [07:26] ____ houses. And yeah, we have around, also normal houses, and I think 120, 130 meters, square meters, so that's not huge.   07:36 Pete: No, no. The square footage of the house also limits maybe the amount of energy you can create, right? You've got limited space for solar panels.   07:41 Remco: Exactly, exactly. Yeah, yeah. Yeah, exactly. Yeah, yeah, yeah. And one of the other things that is of course very important is, of course, the energy with the solar panels. Second of all, of course, if in the winter, for example, if the homeowner put the window open, the whole day, that house will never be carbon neutral because heating will...   08:05 Pete: Yeah.   08:05 Remco: Go the whole day and all night.   08:07 Pete: Can't solve for bad behaviour.   08:08 Pete: And that's why we need to...   [laughter]   08:10 Remco: Exactly, yeah. So we also try to help the homeowner to get insight in that usage and that energy.   08:18 Pete: I see.   08:18 Remco: To reduce energy.   08:20 Pete: Is the power generated primarily through solar or is there any kind of wind or other geothermal or...   08:25 Remco: No, at the moment, in the houses, it's really solar panels and of course the extra energy that they bring into the home is from green energy. So in the Netherlands, most of the homes use green energy.   08:37 Pete: Ah, see, I see.   08:39 Remco: Or other solar panels...   08:39 Pete: So they're getting a power feed from the grid, from the government grid, that's green energy, that's probably wind-powered, right? And then they're augmenting that with local solar, and then the energy they're consuming...   08:50 Remco: Exactly.   08:50 Pete: Is kind of net neutral, I guess.   08:53 Remco: Net neutral, yeah. Exactly.   08:55 Pete: Fantastic.   08:56 Remco: Yeah. Because in the winter, of course, you don't have enough sun. We don't have enough sun in the Netherlands.   09:00 Pete: Right. Same here, I mean you know...   09:01 Remco: At the end of winter [09:02] ____.   09:02 Pete: Yeah, Northwest, we don't have... There's no sun here.   [laughter]   09:06 Remco: No. A lot of rain.   09:08 Pete: Interesting, wow. So has that solution been deployed then? I mean, your solution with...   09:13 Remco: Yeah. We have now around, I think 700 houses deployed.   09:18 Pete: Wow, fantastic.   09:19 Remco: And depending of the speed, of course, of building extra houses will be added every month. Something like that.   09:25 Pete: So basically what you're doing is you're instrumenting. 'Cause I know in the US and maybe in the Netherlands too, there are ways to instrument your patch panel, your incoming electrical panel, to look at loads on a per circuit basis. And then what you're doing is then you're doing that kind of analysis, you're doing the HVAC kind of heating cooling analysis. Are you doing anything about the appliances themselves in the house? Are there appliances that are kind of determining being smart about their energy usage or is that all just happening sort of asynchronously from the rest of the system?   09:58 Remco: Yeah, so we put some, let's call them Smart Energy power adaptors in the home... To see, okay, the refrigerator, how much energy is that costing, etcetera, etcetera. So we're also getting that data. In the future, there are plans to also put in a small panel in the home to give direct feedback to the... To the homeowner itself. So at the moment, it's more or less... It's more information behind for the homeowners, so they can watch later on how the house is performing. TBI can see it directly, but the homeowner sees it later. So they want to bring that information already directly in the home, so the homeowner can react on it directly.   10:44 Pete: Right, right. So you're measuring current draw from different outlets, for lack of a better term, right? It's... Giving them a heads-up and...   10:52 Remco: Yeah.   10:53 Pete: But in the future, then hopefully these things that are plugged into the wall will get smarter and smarter about... Everyone needs to sort of get a message to sort of go into a low power mode because somebody's running the dryer maybe... I don't know. Like is there intra-appliance communication going on here, or is it just sort of every appliance for themselves?   11:14 Remco: Yeah, we see that already with load balancing. So in the Netherlands, we've got a big amount of electric cars that we are charging at home. And so we've got a lot of solutions also in this case with load balancing. So depending on the usage of the home, the car will load more or less energy. So I expect much more solutions also in the more smaller devices that can do more or less the load balancing and talking to each other what's happening.   11:47 Pete: Yeah, exactly. I think that's kind of key. We actually just installed a level two charger here in the US, 'cause we have a Chevy Bolt, our newest car. It's great, and you know the 110-volt charge here, that doesn't really... That only works for emergencies. It takes like an hour to get five miles of charge, so the level two we had installed and run off a 40-amp breaker in the panel and... Yeah, I could see that the inter sort of communication between devices, that will be sort of maybe the next step.   12:16 Remco: Yeah, we see that already here sometimes with more [12:19] ____ systems that try to connect all the kind of devices to each other. But that's still, in my opinion, more for tech guys like us, because it doesn't work always, and... You need some pack of expertise.   12:33 Pete: And a bunch of logos on the box, it's supposed to work with the other thing, it never does. So, yeah, been there. [laughter]   12:38 Remco: Exactly, yeah. It never does.   12:39 Pete: So what are you using Azure for in this case? I'm curious. What are your... There's obvious things, I can name them, but I'm just curious. How are you leveraging Azure in this particular solution?   12:50 Remco: Yeah. So in the start, we've built our own device, our hardware partner did that in the home. And the device is managed by the Azure IT app, where we get the data from the device into that. But we can also control the device now. So it's...   13:05 Pete: And is that sort of a monitoring device that's kind of like a power monitoring?   13:09 Remco: No, it's more, let's call it a gateway. I think that's the best naming convention for this device. So it's really the center of the device in the home. It'll extract the data from the solar panels, from the heating, from...   13:22 Pete: Okay. Right, right, right.   13:23 Remco: Etcetera, etcetera. And the data is directly feed into the systems. Of course, we've got also some devices that we cannot connect in the home itself, and we extract the data from APIs of those suppliers.   13:36 Pete: Right, right.   13:37 Remco: Those are more or less the two options for data ingestion into the platform.   13:40 Pete: Okay.   13:43 Remco: We're using Azure Digital Twins version two to make a replication of the home itself, so we get data from Outerdesk. Outerdesk is a piece of software where they design the homes with, the data from the Outerdesk we import into Outerdesk... Into the... Sorry, Azure Digital Twins. And we combine that data with the sensor data in the home. And that combination reflects into a digital twin of every home of TBI.   14:10 Pete: Wow. That's cool.   14:10 Remco: And next the data flow into the digital twin, we analyze the data with applications like Timeshare with Insights, where we can do simple Timeshare risk analytics. And of course, this data is all time-based data, solar data, panel data, with consumption and data, and energy data, etcetera, etcetera, so that they can do the fast analytics by themselves. And the other one is we use Azure Stream Analytics, where we can analyze the data for anomaly detection. So we know, for example, one of the biggest dealings with TBIS, it's a really simple one but it's water pressure. So in older homes in the Netherlands, we've got gas boilers, or sometimes electric boilers, and they need a certain water pressure in them. If you don't have enough water pressure, you cannot shower, and you don't have heat. And it's an easy solution because you just put extra water into the boiler system and it works.   15:15 Pete: Right.   15:16 Remco: But yeah, if you are in the morning, and you want to go to the shower and it doesn't work, most people will call, in distress, TBI... And say, "Okay, my boiler doesn't work." So then the mechanic will go to the house and fix the problem, and it gets quite expensive.   15:30 Pete: Right, they're very expensive. So you have... So there's sensors for water pressure in the pipes? Is that...   15:36 Remco: Yeah, no, it's a sensor for water pressure in the boiler.   15:41 Pete: Down in the boiler. Okay.   15:44 Remco: So it's indirect in the pipe of course. And that data we get in, and we see a certain pattern that it's declining every time, and then, of course, we can... Or call the homeowner at first, so they're now calling the homeowner, and ask them "Can you fill it by yourself?" And if not, they will come to you and fill it for you of course. But yeah, it's more... Let's say proactive maintenance. Instead of predictive maintenance, it's more proactive.   16:07 Pete: Right.   16:09 Remco: So that's one of the options they are doing. We're using Azure Machine Learning, also how to calculate optimized boiler temperature, to reduce energy in your boiler systems you can set a certain boiler temperature. And of course, how high the boiler temperature, or how more energy you will consume and you need to find per home the optimized boiler temperature, so we use machine learning for that, Azure Machine Learning. And of course, we use Power BI to present all the data to the stakeholders of TBI.   16:44 Pete: Wow, that's cool. So you're getting your money's worth then, on Azure. [chuckle]   16:48 Remco: Yeah, sorry, yeah.   16:50 Pete: That's cool. Are you doing any Edge AI, speaking of Edge AI, on the gateway itself, or is it really more of a data collector sender?   16:58 Remco: Yeah, so at the moment, it's really in Data Collector and of course we can send commons back. Based on this platform, we are also building out now also for the same customer a connected buildings platform, so same architecture but different use case, and therefore we use Edge devices of course.   17:16 Pete: Yeah, I can imagine. Actually, I had someone from RXR realty on the show about a month ago, I don't know if you've heard that one, but that was interesting 'cause they're focused more on the commercial... They're one of the largest real estate companies in New York City. And so they focus on commercial real estate, and in fact, they're using Azure for a lot of work, safe at work scenarios around social distancing, and mask-wearing, and occupancy, and other things. So I can imagine once you move into a commercial space, there's obviously the energy usage and the efficiency, which you guys are focusing on here for the personal home, but then there's all these other scenarios, and when you get into smart buildings, obviously that's kind of a whole lot more complex.   18:01 Remco: Yep. Yeah, and the other issue where, of course, we checked if we could do something with Edge devices, especially from a machinery perspective in the future that can be in a good solution, but if you look at the moment for the pricing model, between Edge device and then [18:18] ____ device, it's more or less almost 50% cheaper to put you and then, yeah, [18:23] ____ more stupid device in the home.   18:24 Pete: Yeah, no, that's true. That's true.   18:27 Remco: Yeah, and it's getting better also. On the simple devices, you can also already do some simple machine learning or smart analytics stuff, as a Microsoft... They put also a lot of energy, and with Edge [18:43] ____, for example, that can do really simple machine learning on a really simple device... You have a lot of CPU power.   18:52 Pete: So if your had your wish list of, "I really wish this technology existed to help me with these solutions, and it doesn't exist yet," is there anything that's kind of top of mind for you that you could snap your fingers and say, "Kinda wish we had that."   19:05 Remco: That is a really good question. Yeah, so for this use case is more or less for the device in the home. So as TBI, to get a bit of hardware power, maybe build some... Piece of hardware. And it's of course school, and I like that, but if you... As a construction company, do you want also to be in a hardware builder of those devices in the home? So I'm looking forward also, if you look at Microsoft, and what they are doing with things like Surface Laptops, etcetera, will they come ever with a cheap, really good more and less Edge or [19:47] ____ Autos device, Microsoft branded working really good... That's one of the things that...   19:53 Pete: I see. So sort of a Microsoft Edge AI sort of platform or something that...   20:01 Remco: And of course, yeah, there are some... Yeah, so there are, of course, already some development kits for that... With the fusion Kit and, etcetera, but that is more or less for the, yeah... For playing around with AI... It's really cool device. But from a production perspective, you need of course something else.   20:18 Pete: Interesting. Yeah, no, that would be good. That would be good. I think there's a lot of... Just a lot of work ahead of us in terms of... When you talked about Edge AI and just a lot of the things that Altius is doing is certainly on the cutting edge. You said you have about 700 homes, so obviously lots more to go with that. Do you know of any other overseas... I'm wondering if there's any equivalent sort of efforts going on in the US. I know that there's a lot around efficiency, but not necessarily around marrying efficiency with the kind of intelligence... I don't know if there's anything else.   20:52 Remco: No, we see in Europe a good interest now in this solution. We also try to more resell this solution. They say it's Avanade's. And of course, you can use the same concept in a building. I think in the US also, to save energy in a big building, you can save a lot of money.   21:14 Pete: Sure, sure.   21:15 Remco: I think these guys can also help with that.   21:20 Pete: Yeah, yeah, for sure. Fantastic. How's the pandemic been affecting your business over the past... I guess... Year, almost, God forbid.   21:30 Remco: Yeah. I have to say, in the Netherlands, it's quite good. So they are still building houses. There's a big need for houses in the Netherlands, especially cheaper houses. So we see a lot of attention, and I think if you look at the pandemic and data and AI, I think companies need more and more data and AI also during pandemic and also afterwards. From that perspective, I see a good future around this topic. If you look at IoT, a lot of companies are now investing... Okay, how can I do the same without the people, and IoT, of course, can help with that, with more automatic... Think about connect the factories.   22:16 Pete: I think actually it's interesting...   22:18 Remco: [22:19] ____ etcetera, etcetera.   22:19 Pete: One of the things I've heard in a constant theme is the pandemic, obviously, it's been pretty horrific, but the... We are getting a lot more data-savvy as a population, we're learning to understand data, and the importance of data and data can mean life and death in many cases, so just the sort of data savviness of the population, it's a positive thing and like you said, I think people are trying to just now use technology to be safer, to be more efficient, remote and automated. That's kind of fast-forwarded a lot of investment in technology over the past nine months or so. And I guess part of it is doing more with less, in many cases. We're trying to be more efficient and more effective. I think once we can get the pandemic under control, we get the... Climate change comes back into the front page, as you would say, and so the work that you guys are doing in terms of really being smart with energy, energy is such a finite resource and... Although I guess it's infinite, if you consider like the sun and the solar system, but anyway...   23:28 Remco: And the wind.   23:29 Pete: And the wind. I guess it's infinite. No. But that's kind of the next thing is we need to get smart and take some of the technology investments we've made and really apply them into our everyday life and... Yeah, yeah, 20, 30 years from now, this will just be commonplace, not having this kind of smarts in your home, in your building and not having a renewable energy, not being carbon-neutral itself will probably seem very odd a decade from now.   23:58 Remco: If you look at the IoT... IoT is already... They're frightened about... 20 or 25 years...   24:03 Pete: Yes, yes. We had that discussion the other day. I've been involved in it for that long. We used to call it embedded systems, and now it's called IoT. Now, but actually now they're gonna change it, it's gonna be called Edge, so Edge computing is now the cool... Even cooler than IoT. [chuckle] Get ready for another name change, but... Yeah, no, it's fascinating. Well, it's great, I really appreciate you taking the time and explaining what you guys are doing. Any kind of closing thoughts or other things you wanna communicate out to the audience around where this stuff is heading.   24:40 Remco: Yeah, I think what you just mentioned, if you look at sustainability of the epidemic, I think that should be one of the main topics for us in the world, and I think IoT can really help with that, creating that achievement with sustainability. And of course, in your home, it's all small what you're doing, but if everybody's doing it, it's really big for the world.   25:03 Pete: Right, right.   25:05 Remco: So we should use these kind of technologies in our homes, and our buildings. That will really help a lot saving energy and reducing a lot of, yeah, bad air, in the air, in the world.   25:18 Remco: Yeah, you know, I hear you. I think there's a phrase, I think it's like, "I can't solve the problems of the world by myself but I can solve the problems that are here locally, on the ground that I stand". And I guess if everyone is doing that and you kind of using technology in the right way to be sustainable, then it does add up. That's gonna be an important one. Perfect. Great, well, Remco, I really appreciate the time. It's been nice meeting you and I appreciate all the support of the Microsoft community, and hopefully our paths will physically cross at some point, maybe some future Mobile World Congress, I don't know, Barcelona or Netherlands or something in the future.   25:58 Remco: Or maybe on the airport of Amsterdam, man.   26:00 Pete: Yeah, I'm sure. I'm sure we've actually probably passed each other in the airport at some point.   26:03 Remco: Exactly, yeah.   26:04 Pete: Cool, alright. Appreciate...   26:05 Remco: Nice to meet you, Pete.   26:06 Pete: Thanks, Remco. Take care.   26:07 Remco: Thank you.   26:07 Pete: Bye-bye.

In this episode of the IoT Unicorn Podcast, Dr. David Rhew, Chief Medical Officer & VP of Healthcare, Worldwide Commercial Business, from Microsoft, shares his experience navigating through today's pandemic and the digital transformation of healthcare. Download the Transcript Here 00:00 Pete: Okay, well, we're here with Dr. David Rhew from Microsoft. David, really appreciate your time. We have a lot of things to talk about today, we're gonna try to squeeze it into the allotted time period, but thanks again for joining us. 00:14 Dr. David Rhew: Thanks, Pete, it's a real pleasure to be here. 00:16 Pete: Good, good. Yeah, and full transparency, something happened in the first conversation I had with David where it didn't record properly, so we're actually going through this one again. So it should be nice and well-practiced. So live and die by Teams, I guess. But anyway, David, so we, as I mentioned, chatted a couple times now, and you're actually fairly new to Microsoft. I think before we get into a lot of really interesting topics I think listeners wanna hear about around digital transformation of healthcare and what's going on with COVID-19 and Microsoft, maybe you can give us a little run-up to how did you end up here at Microsoft, and you've been here almost exactly a year now, so you can give us a little bit of background on yourself and your journey to Microsoft. 01:07 DR: Sure. Well, first of all, I'm a physician, I'm a healthcare researcher, and also a technologist. And really the combination of those three have evolved rather organically throughout my career. It's been remarkable how those three have converged to allow us to be able to start thinking about how healthcare can be used to improve health outcomes, or I should say how technology can be used to improve health outcomes, and really excited to be a part of that program here at Microsoft as we start launching technologies, predominantly cloud-based solutions with artificial intelligence to drive that. 01:45 DR: My story, I guess, begins when I was in college. I was thinking quite a bit about different types of ways that I could help people, and I guess my initial thought was helping people would probably be best served if I went to medical school, so did a curriculum, a pre-medical curriculum. And as part of that program, I think I gained a lot of the basic skills needed to be a doctor, but one of the things that I did also was I was curious about other types of activities and other types of skills. Technology was always a fascination. This was around... In the 1980s, and video games were pretty popular then. These are the arcade video games, not the ones that we typically use... 02:28 Pete: Yeah, the good ones. 02:29 DR: The classics, the Space Invaders, the Pac-Man, Mario, and I was fascinated by that. I just felt, what an incredible way for us to be able to start thinking about how we can not only spend our time, but also how we could use technology to create new experiences. And I started doing a lot of programming, in fact, I became a computer science major as well as a cellular and molecular biologist. And then I went to med school, and in med school there's not a whole lot of opportunity to use computers apart from a word processor, so I felt in many ways that that part of my career journey was put on hold. And it was on hold for a while because what I ended up doing is after I graduated from medical school, I went down a path of exploration in healthcare, specifically looking at ways that we could reduce variation and improve access to care and improve the overall quality of care. And it was done predominantly through what we referred to as guidelines. Turns out that if you were to go to a doctor in... Probably your local doctor, and you were to go to maybe survey the same type of... Ask another doctor across the country or even the globe how they would treat the same type of condition, you'll get a lot of different responses. 03:54 DR: And in fact, when they've done analyses, they've found that variation in care is pretty dramatic, even for things that have been proven to be beneficial. And what we learned in some of the investigations that I was a part of and others have been actively looking at is that a lot of that has to do with just the fact that we don't have mechanisms to remind clinicians to provide that right information at the right time. And I started building basically programs that would provide that right information at the right time. It was very manual. In many cases we had nurses and other clinicians run around the hospital, identifying patients, giving pieces of paper to doctors, saying, "Oh, by the way, your patient fulfils low-risk criteria, you could switch him from an IV antibiotic to an oral antibiotic and send them home," when traditionally, they might have stayed for another few days or maybe even longer. And we ran these programs, we found that it was highly effective. And not only did it reduce the length of stay and reduce total cost, but when we followed up with these patients, they actually did quite well and they were quite satisfied, so... The less time you can spend in a hospital's always good. 05:05 Pete: Yeah, for sure. 05:06 DR: People were having a good time just finding that, hey, you know what, this is something that provided value to just the patient, the health system. And that really got me thinking about how do I start scaling this, because you can't have a person run around the hospital with every piece of information. It really has to be automated. So working with some colleagues, we put together some software that ultimately became a company, and this company got acquired by Cerner, which is a large healthcare information technology company, and next thing I know, I'm in the middle of implementing EHRs across the country and even the globe. And so seeing patients half the time and working with technology was my life work for quite a bit of the early part of my career. And I learned a lot, I learned a lot about how technology is an enabler. It really helps us to be able to achieve some of the goals. But it was really predominantly focused on the inpatient and in-clinic experience. And so I started thinking about, what about outside of the hospital? Could we actually start engaging patients and family members in a more effective manner than simply just sending them a text reminder or giving them access to the patient portal? 06:30 DR: And so what we started thinking about as an industry was this whole category called digital health and connected care. And there were many consumer companies that were looking to get into the space, Apple would be a good one, but also Samsung. And I had an opportunity to talk to folks at Samsung, shared a bit about what I was interested, in terms of where I wanted to take technology, and they shared a similar vision, so it was kind of a great match, and they asked me to be their chief medical officer. For six years I was Chief Medical Officer at Samsung. And during those time periods of working at Cerner and the electronic health world and also working at Samsung I kinda got a chance to experience both ends of the spectrum in terms of what clinicians experience and what patients do. And that bridge was something that I was looking to find a way to make it more seamless, more ubiquitous, which really brought me to Microsoft because of the fact that with its enterprise cloud infrastructure ability for us to be able to have those communications, data communicating also now more freely within the electronic health records space into using HO7 fire standards into a common platform. We could do a lot more than what we were currently doing, and that really is the opportunity for all of us to start thinking about how technology can help us achieve some of the outcomes. 07:51 Pete: Yeah, it's fascinating. I think it's... You can imagine healthcare is one of the great data science challenges we have, there's such a massive amount of information and knowledge base, and like you were saying, the knowledge and the way people are treated and the treatment plans kind of vary, and having access to all of the knowledge collectively and having all the data analyzed. I'm a big Fitbit Versa fan myself, and the measure itself is a pretty key part of my regimen. And especially in people that are not doing too well being able to have all of that data accessible from edge devices and being able to basically get to the right outcomes and treatment plans is pretty, pretty critical stuff. You can't think of too many more purpose-driven business outcomes than that. So that's amazing. It must be... And you're... I know when I joined Microsoft, the first two years, people said it takes two years to find the bathroom at Microsoft, 'cause it... There's so much going on. 08:54 Pete: So you're a year into it, and of course, and we'll talk about the obvious elephant in the room here, you've been in the middle of helping us steer through this pandemic, probably halfway through your first year. So that must have been quite a challenge to sort of come on board and then sort of this all happened, right? Can you give us a little insight as to what was that like, how did that sort of ramp up for you? 09:22 DR: Yeah when I joined Microsoft, I guess there are two chapters or two parts to my time at Microsoft. The first six months were essentially spent working very closely with our partners, our clients, implementing the technologies, the cloud-based technologies to help them achieve some of their business goals, and then when COVID hit and it really started in... For us in January, I know December was probably one of the first times we started hearing about this in China, but we have colleagues, we have Microsoft folks that work in China, and we were very concerned about their health and what was going on. And then when it continued to spread throughout the country and then to other parts of South East Asia, we realized that this was something that was gonna require a pretty coordinated effort within Microsoft around this. Turns out that my background as a physician is in infectious disease. I was actually an AIDS physician during the time, during the AIDS epidemic, and I was seeing patients that were fairly young, healthy individuals that would deteriorate in a rather short time period, and then succumb to the illness. Today there are so many parallels, you're seeing this affect so many young, as well as older individuals, we're seeing a need for public health and also a need for us to be able to accelerate the time to research to vaccine and treatments. 11:00 DR: We never developed a vaccine for AIDS, but we did come up with a treatment that in many ways has allowed us to be able to better control that. And so with that as sort of a backdrop, I was asked to serve as the international coordinator for Microsoft's COVID-19 response, and that was an incredible opportunity to understand really all the different groups within Microsoft that touch the different countries that interface with the governments and the non-government agencies, and what we as a large technology company can do to lean in. That involves providing software cloud services, AI skills and resources, and in many ways, that was our first response to how we could address the pandemic. When the pandemic hit, or I should say when the infections started appearing in the States, particularly in the Kirkland area, near Seattle, that really hit home for many of us in the Microsoft family, because that's where our offices are, our main headquarters are. So yeah. You remember very well. 12:08 DR: And that, during that time, we were recognizing that we needed to do more than just simply provide our technologies, we needed to innovate, we needed to solve some of these problems. So in working with organizations like Providence Health System, we customized our chatbots and made them COVID-19 specific screening and triage tools. We tied it directly to portals that could allow for virtual care assessments, and then tied it into lab testing, we built out mechanisms to provide food services and other types of critical supplies to people that were quarantined at home. We started investigating how we could work through collaboratives to better enable exchange of data and promote the development of a variety of different types of solutions, or I should say ways for us to be able to procure critical supplies such as personal protective equipment. 13:05 DR: And so that process was an extraordinary time. We partnered with companies like GE Healthcare to create virtual ICUs to enable multiple patients to be managed by a single trained clinician. And we started spending a lot of time thinking about treatment. We were thinking about how AI and a variety of other tools can be used to help accelerate the pace of discovery both from a scientific R&D perspective as well as clinical trial recruitment. And so, this has been an incredible journey for us. 13:36 Pete: Yeah, I think, I was looking at the recent Inspire conference and some of the talks there going on and Satya was saying that we've sort of advanced... In two months, we sort of advanced about two years worth of innovation in the time of great crisis. Throughout history, it's sort of a potential real accelerant for a lot of historic inevitables, so we were on the certain trajectory. A separate topic, but online learning and other things, that was sort of a thing that was nascent to be experimented with, and then all of a sudden, it was sort of like, we're all gonna do it now at the same time. 14:14 DR: Yup. 14:14 Pete: Same with a lot of the, obviously, remote collaboration that we're doing. So I can imagine in the healthcare space, you had been working for a long time on the whole digital transformation of healthcare. And now because of the pandemic, we've had to really accelerate a lot of that stuff and really bring to bear a lot of the technologies we were kind of trying out and really sort of making them much more mainstream a lot more quickly. And I know before this particular chat, I was mentioning to you about where... We've been involved recently in how do we take some of the techniques we've done for retail in terms of supply chain management in a typical store, and how do we apply that to healthcare facilities so they can understand their supply of PPE and other things and how can we automate that using a lot of the edge AI as well as cloud capabilities that we would have in a typical grocery store. 15:09 Pete: And so, we're seeing all of this stuff just sort of happen, and obviously, because of the pandemic, there's obviously an underlying urgency that we need to cooperate and innovate as fast as possible. So that must be... I can imagine, I always ask people when they say like, "You come in the morning, you have a cup of coffee. And then what happens?" I can imagine in your job, you have a cup of coffee and then there's probably about 2000 emails in your inbox. 15:38 DR: A lot of times these emails are things that have a direct relevance to what we're seeing and living today. So for instance, much of what I've been focusing on recently have been things that just return to work and return to school. These are topics that we know are of high importance to many individuals, technology can play an important role. But in the setting of a pandemic, almost everything has to be done with health and public safety and mechanisms that will allow us to be able to suppress and/or make sure that the infection doesn't get out of control. So there's just an interesting... I guess, a collision course between how healthcare has now touched every single vertical, whether it's retail like you're describing, whether it's manufacturing. 16:28 Pete: Yeah, hospitality. 16:29 DR: And now education, hospitality. And it's been, I think, a learning experience for all of us 'cause we're now starting to realize that this pandemic isn't gonna go away simply by providing some of the existing technology. We're gonna have to sort of out-think it, build the strategies to get faster delivery of or the maturation of our R&D so we learn what works. A great example would be convalescent plasma. We knew that this work for other types of conditions, but to enable this to be something that we could use more widely, there are two factors, and it all boiled down to one, really. We need more convalescent plasma. We need more donors. More donors for the studies and more donors for the actual... I guess, the convalescent plasma that could potentially be delivered and transfused into patients. And that has gotten us refocused on what can technology do to help clinical trial recruitment or donor recruitment. 17:39 DR: It's funny, because I don't think that in the past, if I would have thought, what is it that will accelerate the research and the ability, that I would have landed on that being sort of one of the critical pieces, but it is. And that's one of the things that we're starting to recognize that sometimes, we're surprised in what is actually the critical piece. 18:00 Pete: Yeah. And one of the other interesting by-products, I think, of this, and I know when we sit down at dinner every night with my teenage kids, we talk about the news of the day and this is inevitably is a topic. And at least one of the things I think that's been, I guess, positive out of this is we're not only innovating like crazy to out-think this, as you said, but we're also becoming a lot more educated around data science. And people are now able to talk about numbers and analyze data and talk about our values and really be a lot more analytical in understanding data. And I think that's just good, that's just good for everything moving forward. And again, sort of accelerating that trend where now, everyone has to become pretty fluent in understanding statistics and data and be able to talk about it in a rational way, regardless of whether you're a high school student or a technology professional. So I think it'll be fascinating to see down the road how much of the accelerations stick, some of the new habits and practices and skills that we're building and things that we're doing together stick as more permanent. So yeah, it's fascinating. 19:18 DR: Yeah, absolutely. 19:19 Pete: I saw Bill Gates on CNN last night. It's always great to see Bill G., as we like to call him around here, talking about vaccines, hot topic, and I think it's going to be... It's gonna become kind of the next chapter of the story at some point as we get into that phase. And there's a couple of factors there, obviously, there is the development and the new techniques to develop vaccines that are being pioneered right now. Obviously then the logistics of manufacture and distribution, right? Which is gonna be interesting. And I think the last time we talked, when we didn't record, but it was fascinating, 'cause you were talking about the paradox of supply and demand with vaccines, right? Like how to make sure we make enough. Make sure there's enough demand to take the vaccine, but also make sure we have enough supply to get it out. 20:14 DR: Well, one of the strange things that we've realized is that, and it ties into what your earlier statement around how we're becoming far more educated, but at the same time, we're also recognizing that not everyone believes the facts. And because of that, education and our ability for us to engage people to help them understand their concerns and to be able to then create greater awareness programs, adoption programs is so critical. So with regards to vaccines, it's very possible that we may have folks that need it that will refuse it. And what we wanna do is we wanna get ahead of that, we know that there's certain groups that this would be of highest importance, these are age groups, demographics such as ethnicity, comorbidities, those are individuals for whom this should be in all likelihood, be prioritized first. Just given the fact that they're the ones who actually probably will need it the most to prevent the biggest... To have the biggest impact, which is death. At the same time they may be the ones least likely to respond to it. 21:24 DR: So we're kind of like in this... It's always a double-edged sword. We're kinda recognizing who needs it, but they may not want it, they may not respond to it, we may need to actually do a second booster, how do we actually do proper and fair allocation of this? Many of these things, hopefully, it'll be a problem that we can address soon because to have a vaccine would be so important, but again, with every step along the way, we're realizing that there's some challenges. 21:56 Pete: Yeah. Yeah. I recall actually, when I was very young, we had the swine flu vaccines and we had to... I don't know how old I was, but I do remember going with my family and a big, big crowd, I don't know, it was big gymnasiums or something like that, and there was a huge long line. And we all lined up for our swine flu vaccines in the arm. And yeah, I mean, there's obviously a double-edged sword of the information distribution, which is fantastic, and everyone has the opportunity to be informed; at the same time with social media and other things, people have the opportunity to be misinformed, and so there's a lot of challenges out there. I'd heard some statistic, something like the seasonal flu vaccine only has about an uptake of about in the 40% range. Even though it's pretty well established that that is a really good way to prevent the flu and we've all had the flu. It's pretty nasty. 22:48 Pete: And for some people it can be deadly. So I think that'll be another thing is like how do we use technology to help people, like you said, kind of understand what's gonna be healthy for them and also help them feel comfortable taking that step forward to invest in their health, which ultimately is all of our healths, it's the kind of the fact that we all need to protect each other from this virus and getting people educated on that. So that'll be sort of the next wave and like you said, it can't come soon enough, I guess, as we see this thing unfolding. 23:25 DR: Absolutely, yeah. I think that's the one key lesson learned from this pandemic, is that this is not anything that one individual or one organization can solve, it's gonna require a coordinated community effort around this to both protect us as well as to get through it. 23:42 Pete: Yeah. Yeah. For sure. Definitely. Well, we always say IoT is a team sport, but in the case of the COVID-19, it kinda takes it to a whole other level. Right? 23:51 DR: Right. Absolutely. 23:53 Pete: Fantastic. Well, David, I don't wanna take up too much of your time, like I said, this is the second time we've had this conversation. So but I really appreciate... Any kind of final thoughts or things that our listeners should be aware of in terms of what you're working on here at Microsoft or what they should be doing to help themselves? 24:12 DR: Well, one of the things that I'm most proud about is that Microsoft is taking a very... I'd say a very important position of their role and responsibility in the community and the world. We look at us as responsible corporate citizens, we have to do what we can lean in to help address the COVID-19 crisis. Innovate as quickly as possible through partnerships, but also address other issues that we face today. This could be everything from racial injustice, societal issues, such as... And also environmental issues. And what we have found is that these are all interconnected, where healthcare used to be about just simply a medical condition and treating it, what we realized that the most significant factors in many cases, have to do with what we'll refer to as social determinants of health. Your income, your education, where you live, the foods that you eat, and your ability to afford those foods. The people that you are socializing with or not socializing with, these are all... It's an interconnected world, and healthcare is becoming interconnected in so many different ways, so as we think holistically about how we improve one's health and well-being, it'll probably touch on things that we never even envisioned in the past. 25:39 Pete: Yeah. Yeah. That's fascinating. You're right, it is a holistic approach that we need to take and we are much more connected probably than we ever imagined. So good. Good stuff. Well, David, again, thank you so much for your time, appreciate it. And I'll see you around. 25:55 DR: Thanks, Pete. 25:56 Pete: Okay, thanks.

Jeff:Welcome to the VIP Home podcast where we talk about all things homeowners need to know. Today we're speaking with Pete and Megan from Powerhouse, which is a TV show produced with Alliant Energy. Welcome to you both.Caroline:Tell us a little bit about Powerhouse and how it started back in 1996. How did you guys get started?Pete:Well, thank you, Caroline and Jeff. It's great for Megan and myself to be with you guys today to talk about this. We're going to be coming up on 25 years for Powerhouse and Alliant Energy started this back in 1996, because it wanted really to educate its utility customers about the importance of energy conservation and safety, but also to help us save energy dollars. They started the Powerhouse program looking at energy efficiency. It's a 30 minute program that airs in six markets across the Alliant Energy service territory in the upper Midwest on Saturdays and Sundays. Megan and I are very fortunate to be hosting it for the past 24 years.Megan:I have a theater background. Pete is in broadcasting, so we kind of have different background experiences. The cool thing is, is that Pete and I have known each other almost all our lives. We grew up across the street from each other.Caroline:Wow. That is so awesome. No wonder why you guys work so well together.Pete:The number one question we get asked, because again, we know each other so well and play off of each other. They do think we're married. We're married, but not to each other. We're learning and that's the great thing about Powerhouse is that we're sharing the insight that we see in terms of helping customers be comfortable in their homes, be knowledgeable and save dollars and save energy and be more efficient when it comes to energy in the home.Megan:The interesting angle that Alliant has chosen to take it is that Pete and I are kind of speaking for the consumer. We're speaking for the customer. We ask the questions that our audience would want to hear. We don't present as though we know everything. We've learned a lot over the years through this process.Caroline:At VIP HomeLink, our goal is to make the homeowner's lives easier. Although we're homeowners, we don't know everything. That's why [inaudible 00:02:05] wonderful guests like yourselves is so wonderful for us and our bran and to just share knowledge of home ownership with everyone. We like to look into homeowner horror stories, sort of those stories that no one really wants to talk about, but maybe a few years down the road, you get a good laugh out of it or you're frightened to even think of that it might happen again. Do you guys have a story like that, that you could share with us?Pete:One of our first years here at the house that I live in, one of our segments was blowing in insulation up in the attic. Oftentimes, Megan and I, we have the do it yourself projects, and I was helping with our expert to blow in insulation up in my attic. I'm maybe not the most agile or gifted in terms of home projects and I stepped off one of the joists in the attic and put my foot through the ceiling in my house as I was blowing in insulation. I did get the insulation and my attic is much better insulated, but I had to repair the ceiling in one of the bedrooms because I was a goof and slipped off the joists and put my foot through the ceiling.Jeff:You pulled the Chevy Chase from Christmas vacation where he's up there [crosstalk 00:03:10] himself and he just....Pete:Absolutely. Yeah. I did that. That's a bit of my own horror story that I did on one of our shoots.Megan:The crew and I have gotten a lot of mileage out of that through the years. There's a, sometimes they put together a blooper reel and that's kind of fun. Maybe this is just a horror story to me, but they were trying to kind of figure out what the show was going to be and looking back, Pete and I did some things that I can't believe we did. Pete was in the shower for one episode. I was in a hot tub in a bathing suit for one episode.Jeff:Got to get those ratings. I mean, it's [crosstalk 00:03:50].Megan:Oh my gosh. To me, that's a horror story. Then there was another shoot that I remember where they shipped us off to Wisconsin for a catalog shoot. We had all these products that were for sale in helping you be more energy efficient. One section was all about grilling. Well, they do these things so far in advance. I think it was February in Wisconsin and Pete and I are wearing shorts and T-shirts and trying not to breathe so you couldn't see our breath. We were freezing. That was a horror story too.Caroline:I used to intern at a magazine. We did the reverse as well so we were doing Christmas in July and everyone was in [inaudible 00:04:30]. I remember all the models being like, "This is horrible." That is a horror story in my opinion because I [inaudible 00:04:37] serve chilli. No. No. Too much.Jeff:I actually do have a horror story. We bought a condo in Hoboken, fourth floor walkup. I decided, hey, I'm just going to replace the switch. How hard can that be? I watched YouTube videos. I thought I can do this. How hard can electrical work be really watch the video. I did it. I brought my wife in for the big reveal and I turned it on and then you just see like go up the wall and just like burn all the way up. I was like, "Oh my God." Yeah. From that point on, I was not allowed to do any DIY, especially electrical work. We had to do another chandelier in the dining room. I got my very smart, downstairs neighbor who had an engineering degree. He came in and he wired it all. I was just like, "Oh God." Horror stories abound when it comes to electrical work, I can only imagine.Caroline:I feel like that was such a good segue to start talking about how somebody could start their home energy efficiency journey.Pete:We talk about insulation and over the many years of Powerhouse, doing a home energy assessment, we've had professionals that come in and do it. You can also go around your own house and do an assessment. The whole thing about is, is keeping in the wintertime, keeping the warm air in your house and in the summertime, keeping the cool air in your house. Again, not allowing vice versa. Insulation is certainly a great place to start. Attic insulation is a great way to check and make sure you do have enough insulation above you because the warm air rises. If you don't have enough insulation in your attic, that's the first place. We always say, when you do that assessment, start there. Then your walls, making sure that you have enough insulation in the walls, which may be a little bit more difficult.Pete:Again, on Powerhouse, we have a the do it yourself projects, but we also, we'll bring in the experts and the true professionals. We are not the experts. We're sharing the insight and the knowledge. That's the fun part for us. I've learned so much about taking care of our home, being comfortable, energy efficiency. It's the insulation in the walls, making sure that you have enough there even around gaskets, your plugins, making sure that those, it really starts with the insulation in your attic, your walls and in the floors.Jeff:Installation, that sounds like a professional job. What about DIY? Are there simpler things that I can do that are just easy? I run to the hardware store and do myself.Pete:Yeah. You can spend maybe 50 to 75 to a hundred dollars caulking, weatherstripping, going around and checking windows because windows obviously are the biggest source where if you've got gaps in the framing, get a caulking tube and a caulking gun and seal around the windows, weather stripping on the floors. During the wintertime, keep the warm air in from getting outside. Those are very easy things that you can do yourself. We talk about that on Powerhouse. It seems like that's one of those shows every year that we get into, as we get into the winter season.Megan:Led light bulbs. That's newer technology, and those can make such a difference. We always encourage homeowners. It's a little bit more of an investment, but it really pays itself off. What you want to do is you want to take those few lights that you use most frequently, or that are hard to reach, those pain in the neck lights and replace those because they do last much, much longer.Jeff:The technology has gotten a lot better over the past decade I'd say. The one thing about led lights in the beginning was just the look of them. They had this like hospital quality to them, very I'm in an office and it starts flicking of that florescent. Nowadays, I mean, I replaced a lot of our lights, not all of them, but a lot of them with the Phillips hue so it's all smart connected. You can adjust the very, my wife has a very specific setting that she likes the kitchen on and it's called Mrs. White or whatever. It's very customizable and the quality of the light I feel has just completely changed over the years.Pete:The LEDs, the upfront costs are much more than the old incandescent bulbs, but LEDs and the thing that we, Megan and I, have stress on Powerhouse is again, how much longer they last in terms of years for a bulb. We often talk about putting LED bulbs, just start replacing the ones that you have to replace more often that are easy access. As you said, the design has gotten much more friendlier. The lighting's much better. LED bulbs, 95% of the energy goes into the lighting with LED bulbs, which is certainly much different than the old incandescent. Only 5% of a LED is going to energy usage. That's the big thing. It's longer lasting and much, much more energy efficient with LED bulbs.Jeff:We moved into the house and we had all ... all the ceilings were kind of the cheaper ... I don't know if they were LEDs, but they were a certain type of white that, like I said, it was like a hospital flicking on all the lights. I don't know if I'm light sensitive, but it was just like, they had to go. We put them all in a box and gave it back to the builder who redid the house and then made the investment in the smart LED bulbs. Because not only is it customizable, but you can put them on timers, you can reduce your energy consumption that way and make sure at the end of the day, they turn off all the lights and all the lights are off. You don't leave the one in the basement on by accident. I thought that was a really a plus, but downside, they are expensive.Caroline:Do lights on dimmers count, like if you dim your light, does that count as energy efficient? I mean, my fiance definitely has some bulbs that are the LED, but not all, but we have every one on a dimmer.Megan:My guess would be that it would be using less energy.Pete:I would think. The one thing that you do have to make sure that when you buy LEDs, that they do allow for dimming. I mean, you go to your hardware store, you do have to make sure that they are built for dimmers.Caroline:Can we touch a little bit on appliances and energy efficient appliances and how that affects your overall efficiency?Megan:When it comes to your appliances, there are a handful that you really need to maintain well and use appropriately because they use the bulk of your energy. That would be your refrigerator, your dishwasher, and your washer and dryer. Of course, maintenance is important on all three of those things. With your refrigerator, you want to keep the temperature at 40 degrees and the freezer at zero, that's going to be an energy efficient path and still get the job done. Also, I'm going to sound like your mother for a second, but you don't want to leave the doors open. Know what you want out of the refrigerator before you go in there. It takes a lot more energy then to re cool the unit if you stand there with the doors open. Don't forget about the vent and the coils down below the refrigerator. Every year, check that out and clean that out with your vacuum cleaner.Megan:Or if you have a pet, you'll want to do it more than once a year. Check that because that keeps the flow running efficiently. You also don't want to overcrowd your refrigerator. They're designed to basically be full, not jam packed because then the circulation doesn't work. When it comes to the dishwasher, there are a couple of things you can do. You can use the eco settings that are built right into the dishwashers that we buy today and only run it when it's full. Some people are in the habit of doing it every night, whether there are four cups in there or it's loaded and do wait until it's full because you're going to use not only less energy, but you're going to use less water. Another tip is to put it on the air-dry setting. That's going to save you energy too and rinse your dishes off.Megan:Even though the new dishwashers can sense how dirty your dishes are, but do yourself a favor and rinse them off because you're going to get a cleaner wash and it's going to require less energy to do that. To the point that I made about the dishwasher, you also want to take into account when it comes to your washer and dryer. Do full loads. Doing a couple shirts at a time is not going to save you any energy at all. Wait until you have an appropriate load and also use cold water. It'll get your clothes just as clean. Make sure that you clean out not only your lint catch, but also your dryer vent too. That can cause big problems. If you have anyone who is ill in your house, of course, you want to make sure that you put it on this sterilize setting.Pete:In normal times, we talk about washing with cold water, but if you do know that you've got some sick people in your home, in that instance, we do recommend washing with hot water during that time. Megan mentioned cleaning that lint filter in your dryer, but also the duct coming out of it. One of our viewers reached out to us after one of the shows we did and said that he got out his vac and cleaned up the vent coming out of his dryer. It was like a new dryer he said, because it actually dried the clothes much more efficiently. One of the common things that we talk about with your furnaces is changing that filter vent once a month. Most people don't. It's about making sure your furnace is running efficiently and if you've got a dirty filter, it's not going to run efficiently and you're going to use more energy and you're going to use more dollars to run that furnace.Pete:That's one of those things that I always stress. Keep track on your phone, hey it's time the first of the month to change my furnace filter. The other thing that we also recommend is having a service plan. Have your heating and cooling system checked once a year. That's a well worth it 50 to a hundred dollars service call, but to make sure that your furnace, that your air conditioner is running efficiently, again, we'll save over the years, save energy dollars for you and your home.Jeff:It's interesting you say that. VIP HomeLink is an app, and it's basically for people who they have a home, they want to stay on top of these things, but we all know it's easier said than done. You say, "Oh, I'm going to do this." Then life gets in the way. The app is there to actually send you reminders. You put the information in on exactly what your HVAC system is. You can log in your what filter you need, and it'll send you notifications every time you need to change the filter, every time you need it serviced. It'll actually keep you on point there, because like I said, it's one thing to say, "I'm going to do this." It's another thing to actually do it.Megan:A lot of times we tell our viewers, mark a day or think of a day on your calendar and do that monthly. Now, your app sounds luxurious, and I'll certainly look into that, but in a simple way, if you just think the first of the month, I'm going to check my filter.Pete:Probably the one thing that we've done over the years on Powerhouse is if people haven't, it's a programmable thermostat. I know on some of your other podcasts, you've had a smart home, the programmable thermostat. Maybe it's a Nest that you can control off your phone, which is so nice that you can set back at night, turn down that thermostat and save, turn it down 10 degrees. Over a year time and 10 hours, I think you can save something like 10% on your heating bill. There's really savings and again, that's what, here on Powerhouse, we're about. Saving energy, but also helping our customers and our consumers save energy dollars. Megan and I have put in more programmable thermostats over the years. That's another small investment. They can be as simple as maybe a 25 to a $50 investment. Again, you can go up and get Nest and have everything programmed off your iPhone, which is wonderful. That's significant savings in energy and energy dollars.Caroline:I know Ruth is a very smart home tech kind of guy.Jeff:I like the gadgets. When we moved in, that was the first thing I did was let's get the Nest thermostats. Let's get all the security stuff. Let's get all hooked up and you put eco mode on. It kind of just keeps it at a comfortable ... It's not blowing air on you, but Nest will remind me even when to change my HVAC filter, which I didn't know when I bought it. I mean, it's an amazing device.Pete:It really is. Again, I think that's one of the other things here on Powerhouse that Megan and I are very fortunate is to learn about the new technology all around our house. We've done shows on things in terms of reminders with a smart home, to change the filter, to setback your water heater. Some people might be away for a while and they can set back that. It is amazing the technology, the changes that we've seen over the past 24 years of Powerhouse. It all comes back again to helping that homeowner be more comfortable saving energy and saving energy dollars.Megan:Right now it's, we're approaching summer and here in the Midwest, it's really, really hot. That's another thing. If you can program that with your phone, program your air conditioning. You don't have to have it run all day with a programmable thermostat. You can set that to kick in before you get home from work, before you get back from vacation. One of the things that you can do to maintain your air conditioner is to clean it off. We had an episode where I got to do that, and I had no idea that I could do that myself. Of course, I had a professional walk me through it. You just go outside and you have your air conditioner, make sure that it's turned off at the electrical panel.Megan:Then you take your garden hose and you spray it all the debris off the outside and inside the compressor, spray the fins on the inside carefully, starting at the top down so you don't force any debris in there. That's something that you can do that's really, really simple. Another thing is to make sure that you've got airflow around your air conditioning compressor. A lot of people have landscaping and things like that because they want to hide it. Well make sure you keep it trimmed away so you have about a foot, at least a foot around there for circulation.Jeff:I thought I read that it's not a good idea to put a cover on an air conditioner because it can create mold problems or mildew or something like that. Dumb air conditioners, you can go and you can cover it with like an air conditioner cover, which they sell. I assume there's a need and a reason for that, but then I also read you don't want to cover it.Pete:You just got to make sure again, debris and everything's clear. To me that would be the benefit of having it covered, but then you also have to check to see if there's anything that may have crawled up or may have gotten on the fans. Just be aware of that.Jeff:Some resident chipmunks in there.Pete:Sure.Jeff:I did have a dryer vent [inaudible 00:18:54]. I think somebody moved the dryer and it popped out of the wall, but it's so big that we couldn't see behind it. You don't want to scratch the floors and all that. We only knew something was wrong because we'd run the dryer and then it would get really humid and the hallway started fogging up and it turned out that somebody must've moved it and it popped out off the wall. All the wet steam was basically not going where it should. It's just going into the room and we have a small room. That was an issue. We called the professional.Jeff:They came over and charged me four or five hundred bucks to really ... I mean, I was like, "Oh my God, is that really necessary? Can I not do this myself?" He had a whole thing, it goes all the way up to the roof, I guess. He had to put this thing together and clean the whole thing because I guess dryer vents are a big source of fires. There's something like 15,000 fires a year, the NFPA says are caused by dryer vents. It's a real problem. You got to stay up on top of it or else you can put your family at risk.Pete:I think that's a good point. Megan was mentioning cleaning the coils on your refrigerator. I mean, you don't need to do that, but once a year, but again, making sure that that vent is connected properly. That it is again, blowing that hot air and getting it out of your home. Making sure that the vent is clear there and then go on outside where that vent actually vents to the outside, making sure that's clean out there. I just last weekend went and checked mine. I wanted to make sure that I didn't have any issue, but you're right. A dryer vent fires, I hate to say, can be more common than you think, but a little bit of maintenance can help save from a fire. Also, just the overall, again, the efficiency of your dryer operating.Megan:Speaking of vents, I'm going to reverse the season, but if you have a gas fireplace, they vent outside. I know it's important to make sure that they are covered and that when you're not using it, the six months that you're not utilizing it, things can get in there. Animals can get in there. Debris, all of that kind of stuff so in reverse, that's something that you want to make sure that you check out and have a professional look at. Always. We always recommend a seasonal tune up on those appliances, on your air conditioner, as well as your fireplaces and your furnace. That's another vent issue, right?Jeff:I have a gas fireplace. That is definitely now on the top of my list because you know what? I was outside and I was trying to find the dryer vent exhaust, and the guy said it was on the roof. Then there's another vent. I was like, "What is this?" It's the gas fireplace vent. Now I know.Caroline:My parents had a horror story. They were using, I guess their fireplace once. I don't know. [inaudible 00:21:45] not really sure. Then one day there was this squirrel in the bedroom because I guess the fireplace vent wasn't closed all the way. It was just start of fall into winter. There was just a squirrel in the house. If that doesn't count as a horror story, I don't know what does.Jeff:A squirrel in the house counts as a horror story. I've had that in that apartment that I told you about in Hoboken. I had a squirrel. I was home and I walked into one room and my dogs are there and they just start going bananas. I go in the other room and the squirrel had come in the bathroom and walked into the bedroom and the dogs discovered that. It was just going around in a circle, just like on the ceiling. I literally just closed the door and then freaked out. Then I was like, "Okay, I got to do this. I'm not going to trap an animal right now at this point in time." I just opened the window and left. Then I came back like four hours later, just hoping it was gone. Thank God, it was gone, but not after peeing all over my head.Megan:I think I'd rather have a squirrel than bats.Pete:On Powerhouse, again, we've had episodes again, where chimney sweeps, the importance of making sure your chimney is clean and again, having a professional come out and clean your chimney. We've had episodes where they talked about the chimney sweep, talked about different animals that have been found and maybe have been dead there. That also takes me over to cleaning your ducts, your duct work in your house, occasionally is a good thing to do. If you have a lot of pets, it is worthwhile, but again, make sure your duct work is cleaned. It's about efficiency and making sure again, your appliances are operating efficiently. Just like we know with your car, you have an oil change and a checkup with your car. You need to have a checkup of your house system to making sure that it's operating properly and efficiently. Cleaning your ducts, D-U-C-T-S, and making sure that again, that it's functioning efficiently.Caroline:These are amazing tips. We were talking about outside home efficiency so can we just touch a little bit more kind of beyond the AC unit or whatever the real term is for that, and just kind of touch base on some other outdoor efficiencies that would help along your journey?Megan:Well, I think with landscaping, this goes back to the air conditioner, energy.gov says that you can save up to 50% of your energy if you shelter your air conditioner. Shade it with a bush, a tree, of course, distanced. If you think about it, we run more efficiently when we've got a little bit of shade when the heat is pouring down on us. Your air conditioner is no different. Call a professional, make sure that everything is operating smoothly and I will just reiterate what Pete said. The system of your house is designed to be efficient and all your appliances are designed to work well, but we have to do our part and take care of them as well.Pete:Plant that tree to shade your house, your overall house can make a difference. Just a little bit of shade on your house can cool the house so that the sun isn't beating down on it. That's another opportunity. Again, we've done that numerous times on Powerhouse. Come out and plant a tree and again, obviously again, think safety when you're going to plant a tree. Call to make sure you're not digging into a power line. Look up and make sure you're now also not going up into power lines above that might be up there. We always stress safety on any, do it yourself projects on Powerhouse.Caroline:People like Jeff Ruth here might take things into their own hands when they should be calling a pro.Jeff:Felled enough trees in my day that I know to call a pro.Megan:One thing that can make a real difference in your energy usage is how you plug in all of the things around your house. You think about all of the cell phone chargers that we have on the small end, but then we also have our home office. We have all kinds of little appliances, toasters, things like that, that we don't use all the time, but they are always drawing power. We call that phantom power. That phantom energy can really add up, up to 10% of your utility bill. That's huge. I have a prop for you. This is a smart strip. What this has is different plugs here that will remain on if necessary, like say your wifi router. You don't want that to go off. There's a designated place for those items, but for the things, video game systems, things like that, that you don't use all of the time, your DVD player, things like that you can put in here. Then it actually will sense when you're not using it and shut those things off.Caroline:That is an amazing product.Jeff:Yeah. We're going to definitely recommend because I need one. I'm going to find one, we're going to recommend it to the listeners and give some links out there so you can find the right one for you.Pete:With your ceiling fans during the summer, they are pushing the cool air down. You want to make sure it's spinning the right way to push down. In the winter months you want to pull the warmer air up. Remember when you're out of a room, I remind my family to turn off ceiling fans if you're not in the room.Jeff:Is that because money doesn't grow on trees?Caroline:You mentioned the right way. There's clearly a wrong way. Is the right way for cool air counterclockwise or clockwise or [crosstalk 00:27:25]?Pete:Well, again, making sure you can feel it when you turn it on. Is it pushing down? I mean, you can feel it pushing down. Make sure it's spinning that way. I'll let the listeners check themselves, okay?Caroline:Perfect. Perfect. Your website, discusses home energy assessments. What would that entail and how does a homeowner know that they're ready for such an assessment?Megan:On the Alliant Energy website, we offer an energy assessment and you enter all of your specific information in there, and it's really basically a checkup for your home. It offers recommendations of ways that you can improve and it's something that once you make those improvements, you can then watch your utility bill and see how things change after you do that. I would say every five years, you can revisit it too. If you've done any other home improvements, you can plug those in. It just kind of keeps things up to date. It's not unlike your app actually. You can keep all of that in one place as well.Jeff:We have a lot of things in common. I'm definitely excited to check out more about the Alliant Energy assessment. Can you tell us a little more about where to find that and how to, is it for just people in the service area or can anybody go and get tips there?Pete:Most utilities, again, I think around the country are offering that. I would always say to your listeners to check with your utilities for what kind of services they provide in terms of a home energy assessment. I think most energy companies today are trying to be good citizens of our Earth and are looking at ways to make sure your home is energy efficient and offering that. I would say check that. One of the things that we stress on Powerhouse is energy star rated appliances. When you're going out to look for new appliances, make sure they have that energy star on the product.Pete:Maybe you've got that second old sort of a beverage refrigerator or beer refrigerator that maybe is 20 years old, but it sure keeps those beverages cold. That's not always the best use of that old refrigerator because they really burn through energy. A fridge that's about 15, 18 years old, it might be time to look at a new refrigerator because they are so much more energy efficient here today than just 15, 20 years ago. You can save again, a lot of energy dollars that you're paying to keep those beverages cold.Megan:Another service that Alliant energy offers is a refrigerator recycling program. To Pete's point, you want to check with your utility company and see what services they offer, because you might be surprised.Caroline:Thank you for all these tips. I mean, I'm so enlightened. Knowledge is power they say.Megan:People are sheltering in place and their home a lot more. Their utility bills are creeping up because of that. With the use of the home office, kids playing video games, all kinds of things, homeschooling, using your computer more than you might have before. People are cooking a lot more. There are ways that you can use your kitchen a little bit more efficiently, and that is to scale down your appliances, especially as we're approaching summer. Grill outside. It's a common sense thing. Use your crock pot, a slow cooker, toaster oven, they use so much less energy than torquing up your oven.Megan:Also, you can scale down how you use your stove top as well. You want to make sure that you use your cookware appropriately by using the right pan size so you're not wasting energy by extra heat coming up. Also, put lids on things. That's going to speed up your cooking and it's going to use less energy too.Caroline:I know my fiance doesn't quite get the toaster oven versus the oven. I'm really trying to help him out there. It also got so warm that the toaster oven is, it seems a bit quicker, but it also doesn't make the whole place like it is outside.Jeff:I think one of the best wedding gifts that I ever got was the Breville toaster oven. It's like-Caroline:I just said that, for our wedding.Jeff:It's great. I mean, I use it every day.Megan:It's fast, quick, easy. You can watch it happen and it doesn't heat up the kitchen.Pete:Brilliant. Again, appreciate being able to, Caroline and Jeff, to talk with you guys and share tips with your listeners. As we like to say, always as we finish an episode of Powerhouse, with these tips and ideas and saving energy, we can make your house a power house.Megan:A powerhouse. Visit our website, Powerhousetv.com. There are loads of tips on there.Caroline:Thank you so much, Megan and Pete. It was so lovely to have you both. We hope that we can partner again soon. Our missions really align in that we really have a lot in common. We'd love to have you guys back on the show one day.Pete:Thank you.Megan:We would love it. Thank you.Jeff:Thanks guys. Have a good one.Caroline:Have a great day.Megan:Thanks. You too.Jeff:Bye.

If you work in the healthcare industry you have a huge responsibility when it comes to managing sensitive patient information, whether you're a big software vendor or a single physiotherapist, everyone needs to follow the same rules, and there are some pretty serious consequences for not doing it properly.  Do you know what your obligations are and if you're doing a good job? Check this episode out to find out!   Who is Anna Johnston Anna Johnston is one of Australia's most respected experts in privacy law and practice. She has qualifications in law, public policy and management, and 26 years' experience in legal, policy and research roles.  Anna has a breadth of perspectives and a wealth of experience to dealing with privacy and data governance issues. She is the former Deputy Privacy Commissioner for NSW, so she knows the regulator's perspective and since 2004 is the Director for consulting firm “Salinger Privacy”. Anna has been called upon to provide expert testimony before various Parliamentary inquiries and the Productivity Commission, spoken at numerous conferences, and is regularly asked to comment on privacy issues in the media.   Anna holds a first class honours degree in Law, a Masters of Public Policy with honours, a Graduate Certificate in Management, a Graduate Diploma of Legal Practice, and a Bachelor of Arts, plus a number of other relevant and well regarded certificates and industry associations.  In this Episode you'll learn 2:08 - About Salinger Privacy 4:55 - Privacy Concerns in Data (with a focus on health tech) 8:15 - All about, privacy reviews, data flows, data governance, and privacy design 14:28 -AI - How does it fit ethically, legally and is policy keeping up with innovation  16:40 - AI - GDPR, challenges for AI with diagnostic decisions   20:10 - AI - Transparency, Accountability and Consent 26:00 - Legal Obligations with Data Privacy Key TakeAways When it comes to privacy law in Australia, the same laws and consequences apply to everyone dealing with healthcare information - whether they are a big institution of a single doctor. While Data Privacy breaches do happen they are often the result of lack of education and or the best intentions in mind, not so much because of malicious intent Often AI is trained on data that was collected not for the intention of training the machine, so the concept of informed consent is a tricky one  The simple “tick this box to agree” actually isn't enough and more emphasis needs to be put on clearly communicating clearly with the person who's data is being collected The expectations of patients data privacy holds the health and medical industries to the highest levels of scrutiny meaning that breaches are to be reported to the Price  Commissioners office and the patients whose privacy has been breached   Links Anna Johnston Twitter - @SalingerPrivacy  Anna Johnston LinkedIn - https://www.linkedin.com/in/anna-johnston-ba188410a/ Notifiable Data Breaches Scheme - https://www.oaic.gov.au/ndb  GDPR - https://www.oaic.gov.au/privacy/guidance-and-advice/australian-entities-and-the-eu-general-data-protection-regulation/  MSIA - https://msia.com.au/  Salinger Privacy - https://www.salingerprivacy.com.au/  My Health Record (Formerly PCEHR) - https://www.myhealthrecord.gov.au/  NDIS - https://www.ndis.gov.au/  National Health and Medical Research Council - https://www.nhmrc.gov.au/    Transcript [00:00:00] Pete: With me today is Anna Johnson. Anna is one of Australia's most respected experts in Privacy Law and practice. She has qualifications in law, public policy and management and 26 years experience in legal policy and research roles. Anna has a breadth of perspectives and a wealth of experience in dealing with privacy and data governance issues. She's the former deputy privacy commissioner for New South Wales.So she really knows regulatory perspective well, and since 2004 is the director for consulting firm Salinger Privacy Anna holds a first-class honours degree in law, a masters of public policy and honors a graduate certificate in management a graduate diploma of legal practice Anna Bachelor of Arts plus a number of other relevant and well-regarded certificates and Industry associations, Anna no longer practices as a solicitor so I am allowed to tell the occasional lawyer joke apparently which is great because that's what I'll probably do Anna thanks so much for joining. [00:01:06] Anna: Thanks Peter great to be here. [00:01:07] Pete: I think we came across each other because you were doing some stuff with MSIA a before the Medical Software Industry Association. [00:01:15] Anna: Yes, I presented at their annual conference recently and then also ran a workshop about privacy by Design so for anyone in that space of Designing health-related technology how to understand the kind of the skills and strategies that will help you build privacy compliance into the design upfront rather than trying to retrofit later.  [00:01:39] Pete: Love to get into more of that detail a bit later on in the conversation too. So, you know you're well well primed for the health Tech space and it's kind of cool to have someone on the show that you know is involved in many different Industries. You're not a vendor you're another player in this kind of big space in an area that's super important these days in our area of Health Tech being data privacy and security and whatnot.So I'm super excited about this conversation. So tell me a little bit more about Salinger Privacy what you guys do and where your clients operate? [00:02:13] Anna: Sure. So well basically we do all things privacy, so we do consulting, training, and we offer resources and one of the things I love about working in the Privacy space is It's just a fascinating intersection between law ethics and Technology. There's you know, there's always something new. There's always a new technology you coming around the corner that we have to get our heads around and help our clients manage that intersection between their Legal obligations ethics customer expectations and then you know what the technology can and what the technology should be allowed to do so, we work across as I said Consulting, training, and resources and we are an Australian business, we've got clients across Australia occasionally we dip our toe into the waters of New Zealand as well. But our clients come from their quite the mix. So, quite a lot of government clients but also businesses from the big end of town, to the nonprofits and also the small and very much Tech startup space.So we have clients everywhere from the kind of you know top ASX companies down to you know, one person's got a great new tech idea with working out of their spare bedroom at the moment kind of space. [00:03:28] Pete: Nice as to how much of it do you reckon is in that Health space? [00:03:34] Anna: Yeah health is really common as probably the second biggest sector after government. Although of course, you know often government is also in the health sector. So sometimes our clients will be the health service provider. So someone directly in that Health Service provision space and they just want to make sure they're dotting their I's crossing their T's in the way that they're collecting and using their patients data, but more typically where, not so much that direct service provision, but all the organizations that use and collect and hold and store health information. So sometimes that's insurance companies for example, sometimes it's governments working in public policy organizations getting into the data analytics space so focusing particularly on you know health and disability data for example, and then there's been some really big-ticket kind of projects we've worked on. So we worked on the Privacy impact assessment on the original design for My Health Record, back when it was originally called the Personally Controlled Electronic Health Record the original setup of the National Disability Insurance Scheme. So we've been involved in privacy impact assessments very early on in those very very big-ticket government projects which touch on health and disability data in particular. [00:04:55] Pete: So in health in particular then what are some of the biggest privacy concerns you see today that the pop-up. [00:05:02] Anna: So what I think is quite interesting about the health sector and it makes it different to other sectors is the health sector is a standout but in a bad way, unfortunately, so the health sector consistently tops the list of sectors reporting notifiable data breaches in Australia.So and when we talk about a notifiable data breach we're talking about when personal information has either been lost. Subject to unauthorized access or subject to an unauthorized disclosure [00:05:33] Pete: because it was those relatively recently wasn't it that want kind of recently that was something change that meant that companies needed to be more transparent with that kind of thing. [00:05:43] Anna: Yeah absolutely so the law was changed in February 2018. To make notification of it. So if you have this kind of data breach and if it's likely to result in serious harm to one or more individuals. It's now the law in Australia that you need to notify both price commissioner's office and those affected individuals, so your patient. [00:06:03] Pete: It's not just big companies or small companies. [00:06:05] Anna: So in the health sector at covers any health service provider regardless of their size. So you might be a one-person physiotherapy business, you know or an independent Locum you uncovered by the federal privacy act. So regardless of your size all Health Service Providers are covered. Outside the health sector, there is an exemption for small businesses. But that exemption does not apply to health service providers. So the health sector is already called out for I guess expectations of a high level of privacy protection for businesses no matter their size in the health sector just because of you know, patient's expectations. And so I think one of the things that makes the health sector different is patient expectations, so it's not that the type of privacy risks or privacy issues are different for health technology, for example, technology design as for any other type of Technology design, but the difference is that patients expectations about the protection of their Health Data are much higher. There's just this sort of intuitive if it's my health information. It must be kept absolutely private, but also the consequences of privacy breaches tend to be higher when you're talking about health information compared with say, you know, The Accidental disclosure of someone's credit card details. Yeah, there are some financial risks. But those risks can be resolved, you know relatively straightforward way. I don't want to minimize those risks, but it's quite a different story in terms of the repercussions individuals can face if their health information is disclosed without Authority. So that might be it could be discrimination embarrassment implications for their employment implications for insurance and all the rest. That's what makes the challenges for people working in technology into the health sector and technology so much higher not that as I said, not that. The nature of the Privacy risks themselves are terribly different. It's just that the expectations are higher and the consequences are worse if you have a data breach. [00:08:17] Pete: So you mentioned that you guys do privacy reviews. What is a privacy review exactly? [00:08:24] Anna: So we did two different kinds so one is called a privacy impact assessment and the other is generally called a privacy audit or a privacy compliance review and the difference really is where you're at in the design process for what we're reviewing. So if you are at the design stage of a new project new technology project, for example, we get in at the design stage and do what's called a privacy impact assessment. If you want us to review something that's already up and running. So your business as usual. We basically call that a privacy audit but regardless of which one of those we doing. We ask the same kind of questions and regardless of whether its the design of the software. It might be the design of a business process. It might be the design of a paper form. It doesn't have to be, you know, a high-tech project to need this kind of review. So regardless of the nature of the project we tend to ask the same questions so you know can and should we collect this data can and should we use it for this particular purpose who can we disclose it to? How do we keep it safe? So when we look at a new project, for example, we look at two broad things one is data flows and the other is data governance. So when what I describe as data flows what we're looking at is. What personal information is being collected? How is it going to be used? Who will it be disclosed to so those three points collection use and disclosure and for each of those we then ask is this going to be appropriate meaning is it going to be lawful? So is it going to comply with the Privacy principles that govern collection use and disclosure but not just is it going to be lawful? Is it going to meet your customers? You know your patients expectations. Is it going to be proportionate to a legitimate business need and is there critically, is there a more privacy-protective way you can achieve that business objective? Yeah, so always trying to come up with you know, helping our clients come up with the most privacy-protective design of a technology of a form of whatever it is but in a way that still achieves the businesses objectives. So once we've settled those questions about authorizing the data flows and making sure that there are lawful and appropriate then we look at data governance. So we usually start with looking at transparency. So have you communicated clearly to your customers about those data flows? You know how their personal information is going to be collected used and disclosed so that they actually understand what's going to happen. You know, I talk about the no-surprises rule no one likes to be surprised what's going to happen with their data and if they have if they're going to have choices is there a really easy way for them to manage those choices? You know, is it as straightforward as a swipe left or right on the app to say yes or no to something and one thing that's really important is in terms of transparency is for organizations to separate out what we see is three different things but often bundled together. So those three things are your privacy policy a collection notice explaining. At the point of collection what it is you're doing with the person's information and a consent mechanism if you're going to rely on consent, so those three things serve three quite different purposes, but especially online. The design practices often companies will jumble the three all together into one long legalistic confusing document and then they make users just tick agree [00:11:55] Pete: Tick a box and you can and you can click the link. Click the link to go read it that you it's not down the bottom. [00:12:02] Anna: Yeah, and we know no one ever reads it, I don't even read them. So we so in terms of data governance. We look importantly transparency. And then finally we look at other data governance questions, like have your staff being trained. Do you have a clear pathway for managing any requests you get for patients to access their data or correct it do you have a clear pathway for managing privacy complaints. Do you have a data breach response plan in place to your staff know what to do in the event of a data breach, so. All of those things that of data flows and data governance form part of whether we're doing a privacy impact assessment of a new project or a privacy audit of an existing business process and again, whether its software or something else, we look at both data flows and data governance as part of our privacy review. [00:12:53] Pete: And if I think about it from my experience. Often, you know, if I'm thinking as a health Tech vendor not many of them go out with any kind of massive intention on I don't know to steal patients information or doing something cynical with the data, but I've seen in the past two, it's not about the intention of what they're going to do with it, but it's almost the perception of what's going to happen or so having that kind of review or someone outside of the business to do that sounds like a pretty sensible thing to do. [00:13:23] Anna: Yeah, absolutely and certainly my experience having worked in you know, in a regulatory role in the primes Commissioner's Office the vast majority of privacy complaints and the vast majority of privacy breaches and data breaches are not coming from a point of malicious conduct or deliberately people doing the wrong thing. It's accidents and it's oversights and its people simply not understanding what their obligations are. Understanding that there are alternative ways to design things. So absolutely. Yeah. I very very rarely see privacy breaches arising from deliberate misconduct. Yeah. It's much more coming from a place of ignorance and sometimes people trying to do the right thing, you know trying to be helpful in trying to help the clients but accidentally doing the wrong thing. [00:14:20] Pete: Yeah, that can happen in health care too. Can you just send this across to me? I really need it because of this particular situation or something. Yeah. [00:14:27] Anna: Yeah. Absolutely. [00:14:28] Pete: It seems to be the right thing to do. It's a balance. So I'm thinking about that In our world AI artificial intelligence that's a big point of discussion regarding privacy for me anyway at the moment. How well do you think policies keeping up with the rate of pace of innovation in Australia more broadly as AI is really Innovative space and there are other things going on too, how's policy keeping up. [00:14:50] Anna: I think there's a constant challenge whether it's AI or any other kind of new technology. There's always this challenge of Law and policy keeping up. The first point I'd make is that privacy laws are designed deliberately. They're drafted deliberately to be technology-neutral and format neutral. So the idea is that they shouldn't actually be always playing catch-up. We've tried to anticipate in the drafting of our privacy laws technologies that haven't even been thought of yet and our starting point with those laws is Broad framed general kind of principles and it's all about respecting humans autonomy and dignity. So sort of one answer is the law is keeping up because it's it was already anticipating new technologies and that those new technologies should be being managed Under the Umbrella of existing laws and policies. But at the same time obviously the law is constantly being challenged in terms of how workable it is in practice and certainly with artificial intelligence the ethical and legal implications are something that not just in Australia but governments around the world are grappling with right at the moment. So there are projects trying to come up with legal and ethical frameworks to cover AI here in Australia the federal department of innovation and industries been working on something there are projects in the EU there are projects in the US There's a lot of activity going on at the moment and lots of those projects around the world are focusing on things like the fairness of AI as well as transparency. So in particular in Europe some of your listeners. May have heard of the GDPR are already. So that's a privacy law in Europe that was recently reformed the general data protection regulation and one of the reforms that was introduced is what you might call a right to algorithmic transparency. So that means that's kind of the laws way of trying to ensure that algorithms developed from AI from machine learning and from AI will be fair and accountable in terms of the impact of decision making that is made or decisions made based on those algorithms. So there's kind of a right to human review of computers decisions and there are rights to ask companies to pause or stop the processing and we would call that using or disclosing someone's personal information in order to ask for an explanation of well you know, how is this algorithm? Working so why you know, why was I denied health insurance or why is it why my premiums going up and my next-door neighbors are going down for example. [00:17:49] Pete: and its even more like as we're moving to space where artificial intelligence is assisting the process of Diagnostics and looks at an image and says this patient has cancer or not. You know that having that in a black box is not you know, and then just you know, let's ask the computer and wait to see what. Is it so much ambiguity there? [00:18:12] Anna: Yeah, absolutely. And in a legal sense, I think courts will increasingly struggle with this as well. If someone is challenging a decision, so it might not be the you know, the diagnosis but maybe it's the health insurers decision based on the diagnosis. You know, we're going to pay your claim or we're not going to pay your claim or whatever it is. You know based on some kind of calculation of risk of that disease developing for example, or you know, if the algorithm can't be explained to a court if it can't be explained to a judge. How is anyone going to be able to determine whether that algorithm was working in a fair and accurate way so one of the really critical privacy principles is it's called the data quality principle or the accuracy principle and it says that each of us has the right to ensure that only accurate relevant up-to-date complete not misleading information is used in decision-making about us and that obviously. Becomes more critical, you know the rubber hits the road where the decision is going to impact us negatively. So the decision is going to be you don't get the insurance or we don't pay your claim. You don't get the job. You don't get access to housing you don't get access to credit for example, and so if you've got decisions made in a black box and no one can explain how they're made because. Yeah, there was some machine learning going on in the AI system came up with its own algorithm. How can anyone test how can a court test whether or not that decision making and the data on which it was based was, you know accurate Fair relevant up-to-date Etc. So that's certainly one big challenge for AI that the sort of the transparency and the accountability for it and I think the other Big Challenge or the other area where AI poses a challenge in terms of compliance with Privacy Law is the lawfulness of the data flows in the first place. So, you know, it's when I was talking about when we do a privacy review we're looking at the data flows meaning what personal information is collected, how it's used who it's disclosed to and in the world of AI your ability to lawfully collect use or disclose data. It's extremely hard to rely on consent as your lawful mechanism consent isn't is by no means not this by no means the only lawful mechanism. There are lots of ways under the Privacy principles that allow companies and governments to collect use and disclose personal information. But quite often consent is what organizations try to rely on but in AI it's really challenging. So if you think about do you do example AI is being used to diagnose some health conditions? Yeah, much of the data. Used in the first place to train the machine learning that will create the AI will create the algorithm that training data what we call a training data will have been collected for some other purpose. So it will have been years worth of data collecting about real hospitals being treated in real patients being treated in real hospitals. That and that becomes the training data set for the machine learning. So it's fairly likely that the patients in the past were not asked to consent but that time to sometime in the future use of their data for this quite different purpose. [00:21:56] Pete: That's something that wasn't even thought of at the time. [00:21:59] Anna: So it's not just about treating you at some point in the future a machine will use your data to train another machine to recognize patterns in data, so but even now if we started to ask patients for their consent, you know as well as us treating you in hospital today. Do you consent to your information being used for AI development in the future? How could a patient today possibly give informed consent? Because the whole point of machine learning and AI is to kind of throw all the data in the mix and just see what pops out it's not a kind of if you like old-fashioned kind of you know, he's a research by hypothesis. This is the question we're asking here's exactly how we're going to conduct the experiment. Yes. So it's not like a clinical trial whereas a patient. I know what my disease is. I'm being offered a new kind of medicine. I've been warned about the possible side effects, and I've had the chance to say yes or no AI and machine learning at based on quite different kinds of research practices, which don't usually involve. That kind of one-on-one sit-down discussion with an individual. It's based on very very large data sets to create those training data sets. It's based on historical data. And typically you don't go back and you don't have the ability to go back and ask for everyone's consent. It's very difficult to rely on patient consent as the lawful basis for health information to be collected used or disclosed for AI purposes. As I said, it's not the only possibility but quite often companies work on the assumption that consent is going to be their legal mechanism and it turns out not to be.Kind of the pragmatic solution for them, but I don't think that that's something that's particularly. Well understood yet. [00:23:52] Pete: What is the solution then like if consent isn't it? Like how does a company doing AI in health or any area I guess operate? [00:24:01] Anna: so there are other legal mechanisms and one of them is and it depends, you know, which Privacy Law you're talking about which jurisdiction you're in but there's usually some kind of research exemption and that usually, again it differs kind of from state to state and federal and Country to country but the research exemptions usually have some role for human research Ethics Committee which gets to weigh up the ethical considerations. Think about where the public interest lies and that committee usually has the power to waive the requirement for consent. There is this kind of structured way to work through thinking about those issues and the National Health and Medical Research Council has guidelines on you know how to set up a human research Ethics Committee and what a properly constituted committee looks like and all of the factors that they need to, you know, there are guidelines about how they need to reach their kind of decision making, so it's not as simple as simply you know, those the tick a box mandatory terms and conditions. That's not going to constitute a valid consent in Privacy Law. So that's just not the right legal mechanism in the most in the majority of cases for artificial intelligence kind of development.  [00:25:25] Pete: Wow so much complexity to factor in and you can going through even just the tip of the iceberg of all of that you can see a lot of work underneath it and questions and kind of vagueness that kind of speak to the reasons why the rate of innovation moves so much faster than other areas that are important like Policy. That's really interesting. Hey look so moving on what should Australian health tech software vendors be most concerned about when developing a solution today then. [00:26:01] Anna: I think first of all make sure you're thinking about both your legal obligations and your customers expectations, you know, the law is by the law. I'm talking about the Privacy principles built into Privacy Law the law tries to codify your basic ethical obligations, but it really sets the minimum kind of standard and often your customers expectations will set a higher standard than just legal compliance. So legal compliance is obviously necessary, but it really should just be considered the minimum Baseline not the entire set of things that you need to think about. I mentioned before the role of consent is in reality quite fraught so if you are relying on your patients consent to do something with their health information you absolutely need to make sure that that consent is actually going to be valid under privacy law you know, it will hold up to scrutiny. So you can't under Privacy Law. You can't say that you're relying on a patient's consent if they actually had no choice to say. No, it has to be voluntary. It has to be informed it has to be specific So it can't be included in mandatory terms and conditions, for example, an opt-out model is not consent. For example, so as I said consent is not the only legal mechanism. There are plenty of other mechanisms. But if that's the one you're relying on you need to be really careful to get that right and another thing is to make sure that your technology has been designed with privacy in mind. So we talked about this concept of privacy by design which is all about baking your privacy controls into the design of systems from. The beginning rather than trying to you know retrofit them in later and I think well what I find usually is a lot of effort goes into the cybersecurity side of things, you know, keeping out the external Bad actors and that's obviously incredibly important but our particular kind of expertise and our skill set is focused more on the internal actors so when you are whether you're designing tech your configuring it implementing it you need to think about your customers but also about your staff or your trusted users your trusted insiders. So making sure that Tech is designed so that its staff or other authorized users only see the absolute minimum amount of personal information about your customers or your patients that they really need to do their job, you know, the legislation says that you have to do this a lot of people come back and say oh, we've got a code of conduct for our employees. We make the more sign it so that's okay. The law says that that is not enough and you know case law comes basically the law that comes from Court decisions and tribunal decisions backs that up that just having you know, letting all staff see all patient records but saying oh, but they signed a code of conduct that's not going to be enough You won't be complying with your privacy legal obligations if that's all you're doing. So you need the same things like role-based access controls, but there's a whole bunch of other privacy controls that can be built into Tech design and it will depend on the kind of product you're or service that you're designing But depending on what it is you're doing, you know, if you're if you've got a data analytics project and using a data warehouse, for example, we would look at filtering out certain data fields. And then we'd look at masking other data fields from the view of particular user groups. If you think about something like an E-health record system, you would limit the search functionality to prevent misuse, you know, the kind of scenario we're usually looking at is, you know, could a staff member look up health information about their partner or their ex-partner or their next-door neighbour so you might put in a test that users need to pass before they can even access customer records. For example, rather than just enabling any user to do a global search against any customer or patient name, so there's plenty of different things you can do. So we use 8 privacy design strategies to help guide Our advice to our clients when we're reviewing technology design software design and sometimes the solution lies in the design of their technology itself, but quite often it's outside the technology so the solution might be or a mix of you know, staff training policies and procedures back to that transparency issue. So how you communicate with your customers. There are lots of different angles we can come from when we're trying to mitigate privacy risks. [00:30:50] Pete: Wow there's a lot to cover I'm sure there are many people listening and thinking this probably a few things that could be applied in their business in the healthcare space, whether it's they're providing the service or the software that sits behind it. I think it's evident that it that it's something that's important to everyone from that single physio, you mentioned right through to the big organizations have got a lot more structure and process to handle this stuff and even they get it wrong a lot too. So having a dedicated focus in that like you guys is particularly interesting so Thank you for sharing your thoughts and insights on that particular topic. [00:31:30] Anna: Great. Thanks for having me on the show.  

Panel Kevin Harwood (twitter github blog) Jaim Zuber (twitter Sharp Five Software) Ben Scheirman (twitter github blog NSSreencast) Andrew Madsen (twitter github blog) Pete Hodgson (twitter github blog) Charles Max Wood (twitter github Teach Me To Code Rails Ramp Up) Discussion 02:44 - Does iOS7's NSURLSession obviate the need for AFNetworking? 03:20 - SSL Pinning Charles Multiple Certificates 08:09 - Reachability 10:24 - Is AFNetworking 2.0 based of NSURLConnection? AFHTTPRequestOperationManager AFHTTPSessionManager 11:52 - Serialization 12:18 - Session Manager NSURLSessionTask NSURLSessionDataTask 15:59 - Using AFNetworking Upgrading 18:11 - AFNetworking and iOS7 20:46 - Prefetching 22:00 - Contributors 22:37 - The three20 Library Category Methods BlocksKit 30:53 - Managing a Large iOS Open-Source Library Mattt Thompson @mattt Mutual Mobile 34:00 - Submitting a Feature to Mattt Picks Macintosh Software Business (Yahoo Group) (Andrew) Low -- Christmas (Jaim) Awful Recruiters (Ben) backup (Ben) Three Africans Coffee (Ben) The Mute Button in Gmail (Pete) P2 Magazine (Pete) Chasin' Freshies: a fresh hop IPA from Deschutes (Pete) The Hobbit: The Desolation of Smaug (Chuck) AFHARchiver (Kevin) Bamboo (Kevin) Next Week Streaming with Chris Adamson Transcript PETE: I actually don't [unintelligible] that much. BEN: But you are British. You have to. PETE: Yeah. I'm a traitor to my nation. I also  don't watch football that much.  And that's why I use ‘football', not ‘soccer'. CHUCK: Hey everybody and welcome to episode 33 of the iPhreaks Show. This week on our panel, we have Jaim Zuber. JAIM: Hello from Minneapolis. CHUCK: Ben Scheirman. Andrew Madsen. ANDREW: Hi from Salt Lake City. CHUCK: Pete Hodgson. PETE: Hello from my pajamas. CHUCK: I'm Charles Max Wood from devchat.tv, with a real quick announcement: if you are interested in learning Ruby on Rails, my Rails Ramp Up course; if you buy it at the beginning of the year… actually, I´ll give you a few days. If you buy it by January 4th, you can get 30% off. You can get that on railsrampup.com We also have a special guess, and that's Kevin Harwood. KEVIN: Hey guys, from Austin, Texas. CHUCK: Is it snowing in Austin? KEVIN: It's actually 79 degrees right now. I think the high, it gets up 75 today. So it's a nice day here in Austin. ANDREW: That sounds nice. JAIM: Not bad. So you are an Auburn guy? KEVIN: I am. It was a pretty good weekend. Me and Tim Cook had a lot to cheer for on Saturday. JAIM: I can sense the glow all the way through the internet. KEVIN: I haven't stopped grinning since Saturday evening. CHUCK: [Laughs] JAIM: Yeah, that Auburn virus really infected my timeline. Really, the only person on my timeline that was tweeting anything other than football was John Siracusa and he was talking about TVs or something. PETE: I totally tune out whenever time it is that people tweet about this. I think it's like Sundays or Mondays or something. I get quite annoyed with Twitter and I just stopped using because I don't know, they are talking about touchdowns and basket hoops or something. I don't know. It's all very confusing to me. KEVIN: I'm actually hoping Twitter releases some statistic like they do, like a super bowl halftime show or something and see if we can see an impact from that game and see the usage spike on Twitter. PETE: Someone should do some sentiment analysis on Twitter, where they like to find out… that would be really cool actually to map like… JAIM: Didn't Apple buy a company that does that? PETE: Really? JAIM: Yeah, for like 200 million. What was it called, Topsy? Isn't that what they did? KEVIN: Yeah, I think so. PETE: You are telling me I just came up a 200 million dollar idea? [Laughter] I'm not going to tell you guys my other ideas.

Panel Jason Felice (twitter github blog Maitria) Pete Hodgson (twitter github blog) Charles Max Wood (twitter github Teach Me To Code Rails Ramp Up) Discussion 01:08 - Jason Felice Introduction 02:03 - “Expressive Code” 04:35 - iOS Programming in Vim ibtool Xcode 06:39 - Why Vim? Muscle Memory The Tool Chain Polar Puzzles 11:30 - Build Process xctool 12:23 - Testing Kiwi The Objective Vimmer 16:50 - Vim Tactics Code Completion clang_complete Omnicomplete 21:06 - Navigation 23:11 - AppCode 25:22 - Toolchain Wishlist Scheme Lisp Clojure RubyMotion 31:51 - Building and Installing Vim Picks No Mouse Fridays! (Pete) AppCode (Pete) Practical Vim: Edit Text at the Speed of Thought by Drew Neil (Pete) VimCasts (Pete) vimium (Pete) VimGolf (Chuck) Emacs (Chuck) Jason Felice: Getting Started with iOS Development With Vim (Jason) lambdanative (Jason) Nathan Sorenson: Clojure to Native via Scheme (Jason) Next Week Scalable Cloud Apps with Aaron Douglas Transcript PETE: My boy is teething at the moment, and we got up today at 5 o'clock in the morning. That's nice. CHUCK: [Laughs] CHUCK: Hey everybody and welcome to Episode 20 of iPhreaks! This week on our panel, we have Pete Hodgson. PETE: Good morning from San Francisco! CHUCK: I'm Charles Max Wood from DevChat.tv. We have a special guest, and that's Jason…is it Felice? JASON: Yeah, it's Felice, that's right. Hello from Detroit this time! CHUCK: I do that just to make Pete laugh. PETE: [Laughs] Yeah, it makes me laugh, I freaked out. [Laughs] Oh, I was going to say like you did the episode number, and I was like thinking, “Should I remind him? Because I don't remind him, I'd keep him say anything.” [Laughs] CHUCK: No, I was seriously actually thinking, “I should ask Jason just to make sure I know how to say his last name,” and I was like, “Nope! I'm going to do this just for Pete.” JASON: [Laughs] PETE: Oh, man. Well, I'm touched. Thank you very much. CHUCK: So Jason, since you're new to the show, do you want to introduce yourself really quickly? JASON: Sure! That's probably the hardest question. I have been doing mobile development for, I don't know, it's like 3 years or something like that, but it really depends on what you count. I've done some of embedded development before that, and some dust wholesome stuff back in the day, something like '90 something. And mostly, I've been on the next person, which is why I found first to home with iPhone stuff, and I've done some Android. I think the bigger focus that I have is on creating expressive beautiful code and helping people to do that because I think a lot of people just haven't been exposed to really expressive code especially when we're talking mobile environment. PETE: What do you mean by expressive code? This is a good sidebar conversation we can talk about. I'm interested particularly in expressive code in Objective C [laughs]. It's an interesting language for that kind of stuff I think. JASON: It is, yes. And that brings up an interesting question, which is like ‘how much of the language you have to know before code is expressive?' Because everybody that I have actually worked with are brought into Objective C into them, like for first couple of weeks, group of programmers who weren't familiar with it. Like my first 2 weeks, they're just like, “I don't even see code. It just looks like ASCII. I'm not even sure what's going on either.” CHUCK: [Laughs] JASON: That's the response that I've got. And it's interesting because after a week or 2, they're like, “Oh, okay! Got it!” I think it has to do with the [unclear] versus the indentation, the way that it uses it being different from most of the languages. PETE: Yeah. JASON: But I've seen Kiwi who makes nice expressive tests for the most part that's sort of the DSL approach.

Panel Ben Scheirman (twitter github blog NSSreencast) Rod Schmidt (twitter github infiniteNIL) Pete Hodgson (twitter github blog) Charles Max Wood (twitter github Teach Me To Code Rails Ramp Up) Discussion 01:22 - Launching a UIWebView and pointing it to a remote URL Autoplay Streaming over 3G or LTE 03:01 - HTTP Live Streaming AVPlayer MPMoviePlayerController MPMoviePlayerViewController Microsoft Silverlight AV Foundation 11:24 - AVPlayer Asynchronous Key Loading Protocol AVURLAsset Learning Core Audio: A Hands-On Guide to Audio Programming for Mac and iOS by Chris Adamson Key-Value Observing (KVO) Deli Radio AVAudioPlayer 19:42 - Use Cases System Sound Audio Categories Playback Control AVQueuePlayer 32:21 - Core Audio Learning Core Audio: A Hands-On Guide to Audio Programming for Mac and iOS by Chris Adamson Adding effects to audio and video AV Audio Mix Echo 38:51 - Interruption 42:04 - Network Connections Network Link Conditioner in Lion - Matt Gemmell 44:07 - .MP3, .CAF, .AIFF, .AAC 45:32 - Transcoding Zencoder M3U Picks Audacity (Rod) Customers (Rod) The Little Redis Book by Karl Seguin (Ben) MMDrawerController (Ben) MacBuildServer (Ben) OpenEmu (Ben) Reveal App (Pete) Snap CI (Pete) Buildozer (Pete) ThinkGeek (Pete) Commit (Chuck) Candy Crush Saga (Chuck) Mini Golf MatchUp (Chuck) Portal (Chuck) Next Week Web Apps & HTML5 vs Native Apps Transcript ROD: I'd get my Dad a Darth Vader helmet...because he's my father. BEN: Yeah, I got it. [laughter] [This show is sponsored by The Pragmatic Studio. The Pragmatic Studio has been teaching iOS development since November of 2008. They have a 4-day hands-on course where you'll learn all the tools, APIs, and techniques to build iOS Apps with confidence and understand how all the pieces work together. They have two courses coming up: the first one is in July, from the 22nd - 25th, in Western Virginia, and you can get early registration up through June 21st; you can also sign up for their August course, and that's August 26th - 29th in Denver, Colorado, and you can get early registration through July 26th. If you want a private course for teams of 5 developers or more, you can also sign up on their website at pragmaticstudio.com.] CHUCK: Hey everybody and welcome to Episode 10 of iPhreaks! That's right, we're on the double digits now! This week on our panel, we have Ben Scheirman. BEN: Hello from NSScreencast.com! CHUCK: Rod Schmidt. ROD: Hello from Salt Lake! CHUCK: Pete Hodgson. PETE: Hello from thepete.net! [Ben laughs] CHUCK: And I'm Charles Max Wood from DevChat.tv! This week we are going to be talking about "Audio and Video" in your apps. BEN: So this is where you just launch a UIWebView and point it to remote URL and then you're done? PETE: I did that once. CHUCK: All the games that I play, I have to turn the sound off on them. PETE: I actually did do that once, Ben. BEN: Yes, it's the quick and easy way to do it. PETE: Yup, it was surprisingly good. I discovered, we're going to jump straight into rearcane pit of noise, but didn't let you do "Autoplay" on video; Apple doesn't want you to do that. Can you still not do that if you're using native video? BEN: You can do whatever you want with the native stuff. PETE: Okay. So for the web one, you can't. But this -- BEN: I think it's just kind of the Safari limitation... PETE: Yeah [chuckles]. CHUCK: Every browser should do that. That rise me asked, too. PETE: I think they say it's a battery issue more than anything else like they don't want you firing up the radio to download like 50 maybe, to start offering conserve concept. BEN: Yeah, they have gotten a little bit more strict on the rules for that, and I don't remember the exact numbers off the top of my head.

Panel Saul Mora (twitter github NSBrief) Rod Schmidt (twitter github infiniteNIL) Pete Hodgson (twitter github blog) Ben Scheirman (twitter github blog NSSreencast) Charles Max Wood (twitter github Teach Me To Code Rails Ramp Up) Discussion 01:22 - Cora Data Core Data Programming Guide SQLite fmdb Transient Entities and Core Data | Cocoa Is My Girlfriend 07:50 - Stores and Contexts NSManagedObject Persistent Store Coordinator Core Data Editor Creating a CoreData Model in Code | Cocoanetics 21:17 - Faulting and Fetching NSFetchRequest The Law of Leaky Abstractions -com.apple.CoreData.SQLDebug 1 Base 2 CouchDB MongoDB TokyoCabinet 27:48 - Is Core Data the right tool for the job? Brent Simmons: On switching away from Core Data 29:46 - Managed Object Context MagicalRecord Core Data and Threads, Without the Headache | Cocoa Is My Girlfriend Core Data: Data Storage and Management for iOS, OS X, and iCloud by Marcus S. Zarra 38:22 - Importing Data 40:08 - Predicates Predicate Programming Guide Picks mogenerator (Ben) PonyDebugger (Ben) xctool (Pete) Neo4j (Pete) AeroPress (Pete) TICoreDataSync (Rod) Countly (Rod) Heil PR-40 Dynamic Studio Recording Microphone (Chuck) Roland R-05 Studio WAVE/MP3 Recorder (Chuck) iOS Programming: The Big Nerd Ranch Guide by Aaron Hillegass (Chuck) NSBrief (Saul) Wasabi Sync (Saul) Sip (Saul) Star Trek II: The Wrath of Khan (Saul) Next Week WWDC Transcript SAUL: I like your style. CHUCK: Hey everybody and welcome to Episode 6 of iPhreaks! This week on our panel we have, Rod Schmidt ROD: Hello from Salt Lake City! CHUCK: We also have Pete Hodgson. PETE: Good morning from San Francisco! CHUCK: We also have Ben Scheirman. BEN: Hello from Houston, Texas! CHUCK: I'm Charles Max Wood from DevChat.tv. And we have a special guest this week, that is Saul Mora! SAUL: Hello from Denver! CHUCK: Denver? I thought you said Fort Collins? Is that not the same thing? SAUL: [laughs] No, that's where the beer is. Okay [laughs] CHUCK: Oh, I see. SAUL: Right. But yeah, that wouldn't be so bad to go and get some beer now. CHUCK: If you go and get too much beer, is it a one-way trip [inaudible]? SAUL: [chuckles] Yeah, well, I have to take some guest with me. CHUCK: Oh, here you go. SAUL: But no...yeah, that's where the New Belgium Brewery is, so I take guests over there quite often. So for anybody comes and visit me in Denver, definitely head on up there. CHUCK: Well I don't drink alcohol, but I'm going to be in Denver this weekend. SAUL: Cool! CHUCK: Maybe I'll come and shake your hand, buy you lunch, or something. SAUL: Yeah! Just let me know! CHUCK: Awesome! ROD: You get to be the driver. [laughter] SAUL: There you go! CHUCK: I don't know what my wife would say about that. [laughter] SAUL: Oh, there are plenty of breweries out here to visit. So, we can visit them anywhere. CHUCK: Awesome! Well this week, we're going to be talking about CoreData. Or, do you call it Core Data? SAUL: [laughs] I thought that was an English thing; Pete might know. PETE: I'll refer to it as Core Data! [laughter] PETE: It's the French pronunciation. I still say Data; it's one of the few English things that I still say in the English way [inaudible]. SAUL: So do you say Beta or Beta? PETE: Oh, that's a good (question). I think I say Beta now just because it's like a -- I was going to say just because it's a software thing, but Beta was a software thing. So, I don't know. SAUL: We have gotten to him! Great! [Chuck laughs] PETE: Yeah. My covers are blown. [laughter] PETE: Actually, I'm native Texan. CHUCK: Yeah, you've seen that? Now you only sound cool when you're talking about things other than computers. PETE: Yup. CHUCK: Alright. So CoreData,

Panel Josh Abernathy (twitter github blog) Rod Schmidt (twitter github infiniteNIL) Ben Scheirman (twitter github blog NSSreencast) Pete Hodgson (twitter github blog) Charles Max Wood (twitter github Teach Me To Code Rails Ramp Up) Discussion 01:25 - Josh Abernathy Introduction GitHub GitHub for Mac GitHub Issues App GitHub Jobs App Lua Corona SDK 03:48 - Differences between writing an app for Mac and writing an app for iOS AppKit UIKit Chameleon twui 05:37 - Model View Controller Model View ViewModel Knockout.js 013 JSJ Knockout.js with Steven Sanderson (JavaScript Jabber) 11:51 - Testing specta expecta OCMock 15:04 - NSTableView Cocoa Programming for Mac OS X (4th Edition) by Aaron Hillegass 17:28 - iOS vs Mac The Rude Awakening for iOS Devs: Josh Abernathy 22:05 - Memory Management 002 iPhreaks Show - Memory Management Garbage Collection ARC 24:32 - Binding 27:23 - Fixing AppKit 32:09 - APIs 33:18 - App Store Sandboxing 36:34 - Resources Cocoa Controls Tweetbot Twitter Mac App The Hit List Things Cocoa Programming for Mac OS X (4th Edition) by Aaron Hillegass 40:47 - Sharing Code Xamarin cheddar-mac cheddar-ios ReactiveCocoa Picks MOO (Ben) Kaleidoscope (Ben) Briefs (Ben) clojurem (Rod) Oblivion Soundtrack (Rod) CloudApp (Pete) MindNode (Pete) LimeChat: IRC Client for Mac (Pete) People are not resources - The Philosophical Developer (Pete) Downton Abbey (Chuck) Downton Abbey at 54 Below - Season 4, Episode 1 Sneak Peek (Chuck) GitHub (Chuck) Daring Fireball Linked List: Using Quartz Composer to Recreate Facebook Home (Josh) Next Week Xcode Transcript BEN: Have you seen that app "Little Inferno" by the guys who created World of Goo? PETE: Mm-mm BEN: It's a great game. It's on MacHeist right now; I think that's still going on. So if you -- CHUCK: Oh, I saw that! BEN: Anyway, so my son is 3 and he's really adept at using the iPad, but he's never really used the computer before so like the whole mouse thing is totally foreign to him. But, he was watching me play this game and he gave it a shot. He's actually learning the click and drag stuff, which is pretty awesome. PETE: Awesome. BEN: I guess the downside is just learning to burn things... [laughter] CHUCK: Nice! BEN: It's just kind of the point of the game. So...I don't know [laughs]. PETE: Yeah. It's a tradeoff, right? BEN: Yes. PETE: Dragging, clicking, burning... CHUCK: Hey everybody and welcome to Episode 4 of iPhreaks! This week on our show we have, Rod Schimdt. ROD: Hello, hello! CHUCK: Ben Scheirman. BEN: Hello from Houston! CHUCK: Pete Hodgson. PETE: Hello from [inaudible], San Francisco! CHUCK: I'm Charles Max Wood from devchat.tv. This week we have a special guest, and that is Josh Abernathy. Did I say that right? JOSH: Yeah! Yeah, you said it right. CHUCK: If that's more than 2 syllables, I'm going to screw it up. JOSH: [laughs] People always get turn off because it's long, but it's just like it looks. CHUCK: Oh, I see. So, do you want to introduce yourself really quickly? JOSH: Yeah! I'm Josh Abernathy. I work at GitHub on the GitHub for Mac App, and various other side things. And yeah, I've been doing Mac and iOS stuff for quite a while now. So hopefully, I'll have something interesting to say about the topic. CHUCK: So is there a GitHub app for iOS? JOSH: We have a couple different iOS Apps. There's an Issues App and there's a Jobs App, neither of them are particularly well-maintained at the moment. So, we kind of try to pretend we don't have any iOS Apps. CHUCK: I see. PETE: I actually tried to use the Issues App the other day... [Josh laughs] PETE: And then I went and look...Is it open source? Is it available kind of the code -- JOSH: No...

Panel Pete Hodgson (twitter github blog) Ben Scheirman (twitter github blog NSSreencast) Rod Schmidt (twitter github infiniteNIL) Charles Max Wood (twitter github Teach Me To Code Rails Ramp Up) Discussion 03:00 - Why don't people test in iOS? 06:12 - Testing Definitions and the Mechanics of Testing Instruments User Guide TestPyramid 13:00 - Why do we test our code? Why is it worth it? Feedback Communication 19:28 - Practicing TDD (Test-driven development) UI View Controllers 25:33 - Unit Testing is hard 28:01 - Tools Send Testing Kit aka OCUnit Application and Logic Tests Running objective-c tests from the command line (with color) (Ben's Script based on Eloy Durán's) Continuous Integration Code Coverage Git Hooks gh-unit Kiwi Cedar RubyMotion Writing Tests for RubyMotion Apps Bacon 42:00 - Frank Selenium Zucchini appium bwoken KIF Calabash Jenkins Writing iOS acceptance tests using Kiwi: Pete Hodgson PublicAutomation Picks TextExpander (Ben) Alfred (Ben) Jenkins (Ben) Oban Scotch (Ben) NSScreencast: Automated Testing with Kiwi (Pete) Rock Climbing (Pete) Scarlett Red (Pete) Test-Driven iOS Development by Graham J. Lee (Rod) 42 (Rod) Test iOS Apps with UI Automation: Bug Hunting Made Easy by Jonathan Penn (Pete) Backbone.js (Chuck) LaunchBar (Chuck) Next Week iOS/Mac & differences with Josh Abernathy Transcript PETE: So what are we talking about this Tuesday morning? CHUCK: I'm not sure, but I think we should write a test for it first. CHUCK: Hey everybody and welcome to Episode 3 of iPhreaks! This week on our panel we have, Pete Hodgson. PETE: Hello from Butte Lake! [Ben laughs] CHUCK: Ben Scheirman...Butte Lake... BEN: Very well done. Very well done. [Ben laughs] CHUCK: Ben Scheirman. BEN: Hello from Houston! CHUCK: We also have, Rod Schimdt. ROD: Hello from Salt Lake City! CHUCK: Sorry, Rod. I was looking at Pete's picture and I was like "No, I already said Pete". [laughter] BEN: Yeah, for those who didn't get the joke we were looking at the transcription from last episode, or from episode 1. And -- CHUCK: Did that get fixed? PETE: It got fixed, yeah. BEN: Okay. So originally, Pete said he's from Berkeley and it came through as Butte Lake, which I thought was hilarious. PETE: I was pretty -- I was looking for the transcript -- it's pretty hilarious how much my accent has closed issues. For whatever personal machine is doing that transcription is definitely challenged by my accent. CHUCK: We're really sorry to the transcriptionist. PETE: Yeah. [laughter] CHUCK: We will pick our panelist more carefully next time. PETE: Oh! It's my fault, huh? [laughter] CHUCK: Anyway...And you can tell I had to ask if it got fixed because I just asked Mandy to do it and assume it's done. PETE: Yeah. No, she fixed it. She fixed it very very quickly. CHUCK: Yeah. PETE: And I'm used to that. I'm living in a [inaudible] when you found out like you have automated voice systems. They often don't work with British accent so I have to put on like a stupid American accent when I'm...operator! [laughter] PETE: Reservations. [laughter] CHUCK: It's funny, too, because a lot of times on those automated systems, they have somebody with a British accent or a fake British accent like speaking. BEN: Yeah. PETE: Yeah. But they didn't understand British. Siri didn't understand British for a very long time because you couldn't get like, if you lived in the US, you couldn't get American, so you couldn't get British Siri to work with like American information. So if I wanted to actually know about anything about America like where I live, I'd have to use the American version Siri, but she couldn't understand my pronunciation. [Ben laughs] CHUCK: So is the British Siri more polite?