Mobycast

In this episode, we cover the following topics: We discuss the features and limitations of serving files directly from S3. We then talk about how CloudFront can address many of S3's limitations. In particular, CloudFront is performant, inexpensive and allows us to use custom CNAMEs with TLS encryption. How to create a secure CloudFront distribution for files hosted in S3. What is OAI (Origin Access Identity), why we need it and how to set it up. We show how you can configure your CloudFront distribution to use TLS and redirect HTTP to HTTPS. We finish up by discussing "byte-range requests" and how to enable them for our image hosting solution. Detailed Show NotesWant the complete episode outline with detailed notes? Sign up here: https://mobycast.fm/show-notes/End SongBeauty in Rhythm by Roy EnglandMore InfoFor a full transcription of this episode, please visit the episode webpage.We'd love to hear from you! You can reach us at: Web: https://mobycast.fm Voicemail: 844-818-0993 Email: ask@mobycast.fm Twitter: https://twitter.com/hashtag/mobycast Reddit: https://reddit.com/r/mobycast

In this episode, we cover the following topics: A common feature for web apps is image upload. And we all know the "best practices" for how to build this feature. But getting it right can be tricky. We start off by discussing the problem space, and what we want to solve. A key goal is to have a solution that is massively scalable while being cost-effective. We outline the general architecture of the solution, with separate techniques for handling image uploading and downloading. We then dive deep into how to handle image uploading, highlighting various techniques for controlling access over who can perform uploads. Two common techniques for securing uploads when using AWS are presigned URLs and presigned POSTs. We discuss how each works and when to use one over the other. We finish up by putting everything together and detailing the steps involved with uploading an image. Detailed Show NotesWant the complete episode outline with detailed notes? Sign up here: https://mobycast.fm/show-notes/Support Mobycasthttps://glow.fm/mobycastEnd SongLazy Sunday by Roy EnglandMore InfoFor a full transcription of this episode, please visit the episode webpage.We'd love to hear from you! You can reach us at: Web: https://mobycast.fm Voicemail: 844-818-0993 Email: ask@mobycast.fm Twitter: https://twitter.com/hashtag/mobycast Reddit: https://reddit.com/r/mobycast

Show DetailsJon Christensen and Chris Hickman of Kelsus and Rich Staats of Secret Stache conclude their series on the birth of NoSQL and DynamoDB. They compare the NoSQL database, Leviathan, created by Chris's startup in the late 1990s to today's DynamoDB. A lot of things haven't changed, even though technology has evolved. It's cyclical. There are patterns and problems that continue to dominate.  Some of the highlights of the show include: Reason for Creation of NoSQL Database: How to scale database with Internet-scale applications to have a virtual pool of infinite storage that can be scaled out Main Architecture Components of Leviathan: API client Update distributor (UD) Base server (storage node) Shepherd (housekeeping management system)   Additional core components included smart IP and storage abstraction layer (SAL) Leviathan mostly used C code and minimal Java code to support users Big difference between DynamoDB and Leviathan is request router and partition metadata system living on the server vs. living on the edge Leviathan was a closed system with an instance for every network or data center; not designed to run as a software as a service, like DynamoDB Leviathan was strongly consistent, unlike DynamoDB's eventually consistent model Definition and Different Types of Transactions Shepherd was used to identify and address consistency, synchronous, and timing issues  Rather than using a file system, Leviathan used relational databases  Links and ResourcesDynamoDBMicrosoft SQLOracle DBAWS IoT GreengrassKelsusSecret Stache Media Quotes:“We had the same kind of problems that DynamoDB had - how do you scale your database dealing with Internet-scale applications and have this virtual pool of infinite storage that can be scaled out.” Chris Hickman “This system and this technology went through many iterations.” Chris Hickman “You can't have a 100% consistent state across everything. It's just impossible. How do you do the right thing?” Chris Hickman “The big difference between DynamoDB and Leviathan...is the request router and partition metadata system living on the server vs. living out at the edge.” Jon Christen 

Show DetailsWhat's under the hood of Amazon's DynamoDB? Jon Christensen and Chris Hickman of Kelsus continue their discussion on DynamoDB, specifically about it's architecture and components. They utilize a presentation from re:Invent titled, Amazon DynamoDB Under the Hood: How we built a hyper-scale database.  Some of the highlights of the show include: Partition keys and global secondary indexes determine how data is partitioned across a storage node; allows you to scale out, instead of up Understand how a database is built to make architecture/component definitions less abstract DynamoDB has four components:           1.   Request Router: Frontline service that receives and handles requests          2.   Storage Node: Services responsible for persisting and retrieving data          3.   Partition Metadata System: Keeps track of where data is located          4.   Auto Admin: Handles housekeeping aspects to manage system What level of uptime availability do you want to guarantee? Replication: Strongly Consistent vs. Eventually Consistent Walkthrough of Workflow: What happens when, what does it mean when… DynamoDB architecture and components are designed to improve speed and scalability Split Partitions: Longer times that your database is up and the more data you put into it, the more likely you're going to get a hot partition or partitions that are too big  Links and ResourcesDynamoDBre:InventAmazon DynamoDB Under the Hood: How we built a hyper-scale databasePaxos AlgorithmAmazon S3Amazon Relational Database Service (RDS)MongoDBJSONKelsusSecret Stache MediaQuotes:“Keep in mind that data is partitioned across storage node, and that's a key feature of being able to scale out, as opposed to scaling up.” Jon Christensen“Amazon was opening up the kimono...how DynamoDB has been architected and constructed and how it works.” Chris Hickman“Managed Service - they get to decide how it's architected...because they also have to keep it up and live up to their SLA.” Chris Hickman“The longer the time that your database is up and the more data you put into it, the more likely that you're going to get a hot partition or partitions are just going to get too big.” Chris Hickman

Show DetailsJon Christensen and Chris Hickman of Kelsus and Rich Staats of Secret Stache continue their discussion on the birth of NoSQL and DynamoDB. They examine DynamoDB's architecture and popularity as a solution for Internet-scale databases. Some of the highlights of the show include: Challenges, evolution, and reasons associated with Internet-scale data DynamoDB has been around a long time, but people are finally using it DynamoDB and MongoDB are document or key value stores that offer scalability and event-driven programming to reduce complexity Techniques for keeping NoSQL database's replicated data in sync Importance of indexes to understand query patterns DynamoDB's Table Concept: Collection of documents/key value items; must have partition key to uniquely identify items in table and determine data distribution Sort keys create indexes (i.e. global/local secondary index) to support items within partitioning  Query a DynamoDB database based on what's being stored and using keys; conduct analysis on queries to determine their effectiveness Links and ResourcesAWSre:InventDynamoDBNoSQLMongoDBGrouponJSONPostgreSQLKelsusSecret Stache MediaQuotes:“Kind of what drove this evolution from SQL to NoSQL - realizing that the constraints were now different, the economics of the resources that were being used.” Chris Hickman“People are realizing that Dynamo is not an ugly stepchild.” Jon Christensen“Event-driven programming...it's very popular, and it's going to become even more popular.” Chris HickmanEnd SongBenirrás Nights by Roy England ft. Dovetracks

Show DetailsJon Christensen and Rich Staats learn about Chris Hickman's first venture-backed startup (circa 1998) and its goal to build a database for Internet-scale applications. His story highlights what software is all about – history repeating itself because technology/software is meant to solve problems via new tools, techniques, and bigger challenges at bigger scales.Some of the highlights of the show include: Why Chris left Microsoft and how much it cost him; yet, he has no regrets Chris's concept addressed how to build a scalable database layer; how to partition, chart, and cluster; and how to make it highly available and a completely scale-out architecture Chris couldn't use the code he had created for it while at Microsoft; but from that, he  learned what he wouldn't do again Chris let the file system be the database at Microsoft, and the project was named, Internet File Store (IFS); it used backend code and was similar to S3 Chris named his startup Viathan; had to do copyright, trademark, and domain name searches Data for the Microsoft project could be stored in files/XML documents; Viathan took a different approach and used relational databases instead of a file system Companies experienced problems at the beginning of the Internet; rest of ecosystem wasn't developed and there weren't enough people needing Internet solutions yet Viathan went through several iterations that led to patents being issued and being considered as Prior art Viathan's technology couldn't just be plugged in and turned on, applications had to be modified – a tough sell Chris did groundbreaking work for what would become DynamoDB Links and ResourcesAWSDynamoDBAWS re:Invent 2018 – Keynote with Werner Vogelsre:InventDeepRacerJSONMoby DickMongoDB Acid CompliancePrior ArtKelsusSecret Stache Media

Jon's Castos Brief Description

Chris Hickman and Jon Christensen of Kelsus and Rich Staats from Secret Stache offer a history lesson on the unique challenges of data at “Internet scale” that gave birth to NoSQL and DynamoDB. How did AWS get to where it is with DynamoDB? And, what is AWS doing now? Some of the highlights of the show include: Werner's Worst day at Amazon: Database system crashes during Super Saver Shipping Amazon strives to prevent problems that it knows will happen again by realizing relational database management systems aren't built/designed for the Internet/Cloud Internet: Scale up vs. scale out via databases or servers; statefulness of databases prevents easy scalability Need sharding and partitioning of data to have clusters that can be scaled up individually Amazon's Aha Moment: Realization that 90% of data accessed was simplistic, rather than relational; same thing happened at Microsoft - recall the Internet Tidal Wave memo? Challenge of building applications using CGI bin when Internet was brand new Solution: Build your own Internet database; optimize for scalability  LinksAWSre:InventDynamoDBNoSQLAWS re:Invent 2018 - Keynote with Andy JassyAWS re:Invent 2018 - Keynote with Werner VogelsOracle DatabaseBill Gates' Internet Tidal WaveCGI BinKelsusSecret Stache MediaEnd SongWhisper in a Dream by UskmatuMore InfoWe'd love to hear from you! You can reach us at: Web: https://mobycast.fm Voicemail: 844-818-0993 Email: ask@mobycast.fm Twitter: https://twitter.com/hashtag/mobycast Reddit: https://reddit.com/r/mobycast

Original Show Notes:At the recent Gluecon event, a popular topic centered around how to prevent Cloud Lock-in. Chris Hickman and Jon Christensen of Kelsus and Rich Staats from Secret Stache discuss why you your time is better spent focusing on one cloud provider. If/when Cloud Lock-in becomes an issue, you will have the resources to deal with it.Some of the highlights of the show include: AWS Fargate is ‘serverless ECS'. You don't need to manage your own cluster nodes. This sounds great, but we've found the overhead of managing your own cluster to be minimal. Fargate is more expensive than ECS, and you have greater control if you manage your own cluster. Cloud lock-in was a huge concern among people at Gluecon 2018. People from large companies talked about ‘being burned' in the past with vendor lock-in. The likely risks are (1) price gouging and (2) vendors going out of business. Cloud allows people to deploy faster and more cheaply than running their own hardware, as long as you don't have huge scale. Few businesses get large enough to need their own data center on-prem to save money. Small and startup companies often start off in the Cloud. Big companies often have their own data centers and they are now migrating to the Cloud. AWS does allow you to run their software in your own data center, but this ties you to AWS. There is huge complication and risk to architecting a system to run in multiple cloud environments, and it almost certainly wouldn't run optimally in all clouds. We think the risk of AWS hiking prices drastically, or going out of business, is essentially zero. If you were building a microservice-based multi-cloud system, some of the difficulties include: Which cloud hosts the database? How do I spread my services across 2 clouds? What about latency between cloud providers networks? How do I maintain security? How do I staff people who are experts at operating in both clouds? It's clear that lock-in is a real fear for many companies, regardless of our opinion that it shouldn't be such a concern. Jon thinks the fear of lock-in may drive cloud providers toward standardization; Chris thinks AWS doesn't have a compelling reason to standardize since they're the industry leader. Our advice: as a small or medium size company, don't worry about cloud lock in. If you get big enough that it's really a concern, we recommend building abstractions for the provider-specific parts of your system, and having a backup of your system ready to run in a 2nd cloud provider, but don't try to run them concurrently. Links and Resources Kelsus Secret Stache Media AWS Fargate re:Invent Gluecon Kubernetes

Start with 39. The Birth of NoSQL and DynamoDB – Part 1.If you like that one, finish the five part series. If you still want more? Ask me for advice. I'll tell you what are the next best ones at jon@mobycast.fm.

In this episode, we cover the following topics: Developing a system for automatically updating containers when secrets are updated is a two-part solution. First, we need to be notified when secrets are updated. Then, we need to trigger an action to update the ECS service. CloudWatch Events can be used to receive notifications when secrets are updated. We explain CloudWatch Events and its primary components: events, rules and targets. Event patterns are used to filter for the specific events that the rule cares about. We discuss how to write event patterns and the rules of matching events. The event data structure will be different for each type of emitter. We detail a handy tip for determining the event structure of an emitter. We discuss EventBridge and how it relates to CloudWatch Events. We explain how to create CloudWatch Event rules for capturing update events emitted by both Systems Manager Parameter Store and AWS Secrets Manager. AWS Lambda can be leveraged as a trigger of CloudWatch Events. We explain how to develop a Lambda function that invokes the ECS API to recycle all containers. We finish up by showing how this works for a common use case: using the automatic credential rotation feature of AWS Secrets Manager with a containerized app running on ECS that connects to a RDS database. Detailed Show NotesWant the complete episode outline with detailed notes? Sign up here: https://mobycast.fm/show-notes/Support Mobycasthttps://glow.fm/mobycastEnd SongNight Sea Journey by Derek RussoMore InfoFor a full transcription of this episode, please visit the episode webpage.We'd love to hear from you! You can reach us at: Web: https://mobycast.fm Voicemail: 844-818-0993 Email: ask@mobycast.fm Twitter: https://twitter.com/hashtag/mobycast Reddit: https://reddit.com/r/mobycast

In this episode, we cover the following topics: In this new series, we are discussing database consistency models explained in three acts. This episode is "Act III: Eventual consistency saves the web (circa early 2000s)". We explain eventual consistency and the motivation behind the philosophy. The BASE acronym stands for three key properties of a distributed system that utilizes eventual consistency. We define and explain these BASE attributes: Basically available Soft state Eventual consistency We share the story of Werner Vogel's keynote at re:Invent 2018, where he outlined the reasons why DynamoDB was created. In particular, DynamoDB allows for an eventual consistency data model. Interestingly, the DynamoDB story closely parallels what happened when Chris was at Microsoft. It just happened at least 6 years earlier. We then wrap up everything we have learned about ACID, CAP, and BASE by providing some guidelines on when to choose ACID vs. BASE systems. Detailed Show NotesWant the complete episode outline with detailed notes? Sign up here: https://mobycast.fm/show-notes/Support Mobycasthttps://glow.fm/mobycastEnd SongWhisper In A Dream (Feathericci Remix) by UskmatuMore InfoFor a full transcription of this episode, please visit the episode webpage.We'd love to hear from you! You can reach us at: Web: https://mobycast.fm Voicemail: 844-818-0993 Email: ask@mobycast.fm Twitter: https://twitter.com/hashtag/mobycast Reddit: https://reddit.com/r/mobycast

In this episode, we cover the following topics: In this new series, we are discussing database consistency models explained in three acts. This episode is "Act II: The arrival of the Internet creates new challenges (circa 1998)". Problems with building large scale-out systems led to the "discovery" of the CAP theorem (by Eric Brewer of Inktomi). We explain what the CAP theorem postulates and break it down in understandable terms. The three properties of the CAP theorem are consistency, availability and partition tolerance. What exactly is meant by "partition tolerance"? A key implication of the CAP theorem is that must choose your priorities. As a system scales, it cannot be both available and consistent. We discuss Physalia, a technology developed by AWS for making Elastic Block Service (EBS) more resilient. The design of Physalia was inspired by the principles of the CAP theorem. We then take a personal story detour that is (mostly) related to the CAP theorem. It turns out, Eric Brewer and Chris share a common experience during the first Internet bubble. Detailed Show NotesWant the complete episode outline with detailed notes? Sign up here: https://mobycast.fm/show-notes/Support Mobycasthttps://glow.fm/mobycastEnd SongDisruption by Miquel SallaMore InfoFor a full transcription of this episode, please visit the episode webpage.We'd love to hear from you! You can reach us at: Web: https://mobycast.fm Voicemail: 844-818-0993 Email: ask@mobycast.fm Twitter: https://twitter.com/hashtag/mobycast Reddit: https://reddit.com/r/mobycast

In this episode, we cover the following topics: In this new series, we are discussing database consistency models explained in three acts. This episode is "Act I: Transaction processing (circa 1973)". We start with the motivation behind talking about database soup - why are ACID, CAP, and BASE important to understand? We define transaction processing and its origins. What exactly is a "transaction"? Transactions are governed by ACID semantics. We define and explain the four characteristics of the ACID acronym:: Atomicity Consistency Isolation Durability The computer scientist, Jim Gray, came up with the idea of ACID semantics in the late 1970's. We discuss some of the history behind this, along with a bizarre and tragic ending to his story. We also share a personal story about another important player in transaction processing, Phil Bernstein. Detailed Show NotesWant the complete episode outline with detailed notes? Sign up here: https://mobycast.fm/show-notes/Support Mobycasthttps://glow.fm/mobycastEnd SongTalcum by Lost LakeMore InfoFor a full transcription of this episode, please visit the episode webpage.We'd love to hear from you! You can reach us at: Web: https://mobycast.fm Voicemail: 844-818-0993 Email: ask@mobycast.fm Twitter: https://twitter.com/hashtag/mobycast Reddit: https://reddit.com/r/mobycast

Oh by the way, buy girl scout cookies from my daughter here! GIRL SCOUT COOKIEEEEES!In this episode, we cover the following topics: Technology is changing at an increasing rate, with a constant stream of new things to learn. We discuss how innovation has changed the rules of the game. "Life moves pretty fast. If you don't stop and look around once in a while, you could miss it." - Ferris Bueller Chris recounts a personal story that emphasizes the importance of continual learning and growth. During preparation for a previous Mobycast mini-series, he relies on a trusted solution pattern for accessing private subnets. But during final fact checking, he discovers entirely new options that may be better/easier solutions. We break down some important lessons learned from this experience. Why do we think the following Stephen King quote is sage advice for all of us? "Kill your darlings, kill your darlings, even when it breaks your egocentric little scribbler's heart, kill your darlings." We then turn our attention to discussing what is your most important skill. What is it? How can you cultivate it? Why does this matter? Detailed Show NotesWant the complete episode outline with detailed notes? Sign up here: https://mobycast.fm/show-notes/Support Mobycasthttps://glow.fm/mobycastEnd SongDisco Pigs (Rave Mix) by Roy EnglandMore InfoFor a full transcription of this episode, please visit the episode webpage.We'd love to hear from you! You can reach us at: Web: https://mobycast.fm Voicemail: 844-818-0993 Email: ask@mobycast.fm Twitter: https://twitter.com/hashtag/mobycast Reddit: https://reddit.com/r/mobycast

In this episode, we cover the following topics: We continue our discussion of microVMs with a look at Kata Containers. Kata Containers formed by the merger of two projects: Intel Clear Containers and Hyper runV. How does Kata Containers integrate with existing container tooling? How mature are Kata Containers - are they ready for production? We then take a look at unikernels, which take a dramatically different approach to solving the problem of providing high security with blazing performance. The benefits of unikernels along with a comparison on how they differ from containers. We discuss some of the most popular unikernel implementations, including OSv and MirageOS. Does the future point to a deathmatch between containers and unikernels, or will there be a need for both approaches to cloud-native apps? DETAILED SHOW NOTESWant the complete episode outline with detailed notes? Sign up here: https://mobycast.fm/show-notes/SUPPORT MOBYCASThttps://glow.fm/mobycastEND SONGPalm of Your Hand by BlynkwthMORE INFOFor a full transcription of this episode, please visit the episode webpage.We'd love to hear from you! You can reach us at: Web: https://mobycast.fm Voicemail: 844-818-0993 Email: ask@mobycast.fm Twitter: https://twitter.com/hashtag/mobycast Reddit: https://reddit.com/r/mobycast

In this episode, we cover the following topics: We revisit a misunderstanding from last week's show to find out exactly what the Firecracker team means when they list "Single VM per Firecracker process" as a security benefit. We discuss what's next on the Firecracker product roadmap, with particular emphasis on support for snapshot/restore. We learn how AWS uses Firecracker in production today with AWS Lambda. AWS is currently working on updating Fargate to use Firecracker. We look at why they are doing this and the design details of updating Fargate to use Firecracker. We finish by looking at how you can use Firecracker for your own containers, by incorporating Firecracker-aware tooling into your container infrastructure. Specifically, we look at firecracker-containerd and Weave Ignite. DETAILED SHOW NOTESWant the complete episode outline with detailed notes? Sign up here: https://mobycast.fm/show-notes/SUPPORT MOBYCASThttps://glow.fm/mobycastEND SONGThing Is by Public AddressMORE INFOFor a full transcription of this episode, please visit the episode webpage.We'd love to hear from you! You can reach us at: Web: https://mobycast.fm Voicemail: 844-818-0993 Email: ask@mobycast.fm Twitter: https://twitter.com/hashtag/mobycast Reddit: https://reddit.com/r/mobycast

In this episode, we cover the following topics: We review virtual machines (full virtualization) and their benefits and tradeoffs. We then revisit containers (OS-level virtualization) and briefly recap how they use OS kernel features to enable virtualization. Containers provide great performance and resource efficiency, but at the cost of losing strong isolation. Can we have the performance and efficiency benefits of containers but with the strong isolation of VMs? There are some promising technologies that aim to combine the best of both VM and container worlds: microVMs, unikernels and container sandboxes. What are microVMs? What are unikernels? What are container sandboxes? AWS Firecracker is one of the most talked about microVMs. We discuss what it is, and the key benefits of using Firecracker. DETAILED SHOW NOTESWant the complete episode outline with detailed notes? Sign up here: https://mobycast.fm/show-notes/SUPPORT MOBYCASThttps://glow.fm/mobycastEND SONGSmooth Modulator by aMIGAaMIGOMORE INFOFor a full transcription of this episode, please visit the episode webpage.We'd love to hear from you! You can reach us at: Web: https://mobycast.fm Voicemail: 844-818-0993 Email: ask@mobycast.fm Twitter: https://twitter.com/hashtag/mobycast Reddit: https://reddit.com/r/mobycast

In this episode, we cover the following topics: AWS offers not one, but two, managed services for secrets management. Systems Manager Parameter Store and AWS Secrets Manager have similar functionality, making it sometimes confusing to know which to use. We compare and contrast the two services to help guide your choice. The three types of sensitive data injection supported by Elastic Container Service (ECS). Understanding when sensitive data is injected into the container and how to handle updates to secrets (such as credential rotation). The required configuration changes and IAM permissions you need to enable ECS integration with Parameter Store and Secrets Manager. A walkthrough of the specific steps you need to take to update your ECS application to support secrets integration. Detailed Show NotesWant the complete episode outline with detailed notes? Sign up here: https://mobycast.fm/show-notes/Support Mobycasthttps://glow.fm/mobycastEnd SongStraddling by Derek RussoMore InfoFor a full transcription of this episode, please visit the episode webpage.We'd love to hear from you! You can reach us at: Web: https://mobycast.fm Voicemail: 844-818-0993 Email: ask@mobycast.fm Twitter: https://twitter.com/hashtag/mobycast Reddit: https://reddit.com/r/mobycast

Support Mobycast-> https://glow.fm/mobycast

Support Mobycast-> https://glow.fm/mobycast

Support Mobycast-> https://glow.fm/mobycast

Support Mobycast-> https://glow.fm/mobycast

Support Mobycasthttps://glow.fm/mobycastIn this episode, we cover the following topics: Before we get started, a CAVEAT. There are other (potentially BETTER) ways of accessing resources on private subnets.  We'll talk about these (such as AWS Client VPN or AWS Systems Manager Session Manager) in future episodes.  But a great choice (with the most flexibility/power) remains our current choice: a third-party software-only VPN solution.  There are many options for third-party software VPNs, both commercial and open source. Some of the options we considered include:  SoftEther  Openswan  OpenVPN (* our choice)  Discussion of the different flavors and pricing models for OpenVPN Access Server. Step-by-step walkthrough of installing OpenVPN Access Server via the AWS Marketplace. Including how to setup TLS for your VPN server.  We detail the process of how to create private subnets within a VPC.  Create new subnets to be used as private subnets, keeping in mind a multi-AZ design.  Routing table considerations.  Setting up a NAT gateway to forward Internet traffic for private subnets.  Some pro tips to keep in mind when building out your cloud network.  CIDR block considerations (the "Goldilocks" approach to sizing).  Did you know that NAT gateways are SPOFs? We discuss how to improve availability.  Links VPC with Public and Private Subnets (NAT) Software VPN OpenVPN SoftEther Openswan Amazon Web Services EC2 BYOL appliance quick start guide AWS Certificate Manager ZeroSSL End SongTachyon, by Roy EnglandFor a full transcription of this episode, please visit the episode webpage.We'd love to hear from you! You can reach us at: Web: https://mobycast.fm Voicemail: 844-818-0993 Email: ask@mobycast.fm Twitter: https://twitter.com/hashtag/mobycast Reddit: https://reddit.com/r/mobycast  

Support Mobycasthttps://glow.fm/mobycastShow DetailsIn this episode, we cover the following topics: Subnet 101 Public subnetsUsed for public facing resources which allow inbound connections from the public Internet Private subnets What are they? Used for resources that should not be exposed to open Internet Do not allow direct access from open Internet Require use of network address translation (NAT) for egress-only Internet access Why use private subnets? Protect your cloud servers from script kiddies Limit exposure Security groups and routing tables allow resources on public subnets to communicate with private subnets NAT (network address translation) deep dive What is NAT? Remaps one IP address space into anotherDone by modifying network address information in IP header of packets while in transit across routing device Tool to deal with IPv4 address exhaustionOnly need single public IP address for NAT, which hides entire private network behind it Note: Actual role of NAT device is both address translation and port address translation How does it work? IP header consists of: Source IP Source port Destination IP Destination port Routing device modifies IP address in packets Outgoing packets (from private-to-public)Source IP and port changed to NAT valuesI.e. packets appear to originate from NAT (instead of private IP itself) Incoming packets (public-to-private)Dest IP and port changes to private values For TCP/UDPNAT keeps in memory table that maps traffic to private IPs Table includes each active connection (particularly the destination address and port) When reply comes back to router, uses table to determine private IP that reply should be forwarded to Port numbers are changed so combination of IP and port on returned packet can be unambiguously mapped to corresponding private destination Note: conversation to open Internet has to originate in private network!This is because initial message establishes required information in translation table How can a single computer have both public and private IP addresses?A quick primer on IP addresses and network interface cards MAC (media access control) address Physical address Unique ID assigned to NIC IP addressLogical address Network switches maintain Address Resolution Protocol (ARP) tables that map IP addresses to MAC addressesARP table used to know which MAC address to attach to packet Single NIC can have multiple IP addresses Alas, private subnets are less convenient than public subnets. Instances on private subnet won't be publicly accessible, they can only be accessed from inside the network. This leads to the problem of how to connect to an instance on a private subnet from a remote location?Three broad categories of solutions: Direct Connect Dedicated network connection over private lines straight into AWS backbone Requires network equipment on customer side Cons: Requires dedicated hardware Expensive Applicable only when you have an on-prem location that needs to be physically connected to VPC Bastion host (jump host) Public-facing server running SSH daemonOnce connected to bastion host, users can then ssh to machines on private subnet Typically have a single instance on public subnetMinimizes surface area to be protected Cons: Adds an extra layer of indirection ssh key management is more complicated SPOF Security risk of protecting the bastion host VPN (virtual private network) Many different options, ranging in cost and equipment requirements For both connecting on-prem location, as well as general remote user access VPN Available options Managed VPN Managed IPsec VPN connection over existing internet Quick and usually simple method for making secure connection to VPC Can be used as redundant link for Direct Connect Supports static routes or BGP peering/routing How to setup: Designate an appliance to act as your customer gateway (usually the on-prem router) Create VPN connection in AWS and download config file for your customer gateway Configure customer gateway with config file VPN CloudHub Connect locations in hub and spoke manner using Virtual Private Gateway Allows remote locations to communicate with each other via the hub (Virtual Private Gateway in AWS) Each remote location uses Site-to-Site VPN connection to connect to hub Reuses existing internet connection Supports BGP routes to direct traffice.g. use MPLS first then CloudHub VPN as backup How to setup:Assign multiple Customer Gateways to a Virtual Private Gateway, each with their own BGP ASN and unique IP ranges Third-party software VPN You provide your own VPN endpoint/software Use this option if you must manage both ends of VPN connection How to setup:Install VPN software via Marketplace appliance or on EC2 instance TIL... AWS has increased the options Managed VPN is now known as "AWS Site-to-Site VPN" New option: "AWS Client VPN" Fully managed, highly available software-only VPN Supports OpenVPN-based clients We'll discuss "AWS Client VPN" in-depth in a future episode Our choice for this episode: let's setup a third-party software VPN solutionRationale: Not too much $$$ Pretty sophisticated solution that's easy to manage Links VPC with Public and Private Subnets (NAT) Network-to-Amazon VPC Connectivity Options Network address translation RFC 1631 - The IP Network Address Translator (NAT) Multiple IP Addresses AWS VPN Introducing AWS Client VPN to Securely Access AWS and On-Premises Resources End SongZero Gravity by Roy EnglandFor a full transcription of this episode, please visit the episode webpage.We'd love to hear from you! You can reach us at: Web: https://mobycast.fm Voicemail: 844-818-0993 Email: ask@mobycast.fm Twitter: https://twitter.com/hashtag/mobycast Reddit: https://reddit.com/r/mobycast

Support Mobycasthttps://glow.fm/mobycastIn this episode, we cover the following topics: AWS re:Invent general overview December 2nd thru December 6th 2,500+ sessions, spread over 6 venues, spanning 2.5 miles of the Las Vegas Strip Discuss the 4 primary types of content and the pros/cons of eachSessions, chalk talks, workshops and builders sessions Our general observations of themes to expect this yearHint: Kubernetes is hot We point out some of the sessions we are particularly looking forward to re:Invent is not all work, you get to play too re:play Other parties and where to find them Our predictions for new product/service announcements at re:Invent 2019 We also talk about Apple's recent launch of a new MacBook Pro (goodbye butterfly keyboard!) Links AWS re:Invent AWS re:Invent Twitter feed AWS re:Invent 2019 - How to re:Invent YouTube channel Startup Central is Headed to re:Invent! A Cloud Guru Guide to re:Invent Linux Academy Guide to re:Invent Las Vegas Monorail Unofficial - re:Invent Parties Twitter feed Unofficial - List of AWS re:Invent Conference and Vendor Parties Apple launches 16-inch MacBook Pro with 6 speakers and 'Magic Keyboard' End SongFlowerchild (Lost Lake Remix) by Owen Ni For a full transcription of this episode, please visit the episode webpage.We'd love to hear from you! You can reach us at: Web: https://mobycast.fm Voicemail: 844-818-0993 Email: ask@mobycast.fm Twitter: https://twitter.com/hashtag/mobycast Reddit: https://reddit.com/r/mobycast

Support Mobycasthttps://glow.fm/mobycastShow DetailsIn this episode, we cover the following topics: Container networking ECS networking mode Configures the Docker networking mode to use for the containers in the taskSpecified as part of the task definition Valid values: noneContainers do not have external connectivity and port mappings can't be specified in the container definition bridge Utilizes Docker's built-in virtual network which runs inside each container instance Containers on an instance are connected to each other using the docker0 bridge Containers use this bridge to communicate with endpoints outside of the instance using primary ENI of instance they are running on Containers share networking properties of the primary ENI, including the firewall rules and IP addressing Containers are addressed by combination of IP address of primary ENI and host port to which they are mapped Cons: You cannot address these containers with the IP address allocated by DockerIt comes from pool of locally scoped addresses You cannot enforce finely grained network ACLs and firewall rules host Bypass Docker's built-in virtual network and maps container ports directly to the EC2's NIC directly You can't run multiple instantiations of the same task on a single container instance when port mappings are used awsvpc Each task is allocated its own ENI and IP addressMultiple applications (including multiple copies of same app) can run on same port number without conflict You must specify a NetworkConfiguration when you create a service or run a task with the task definition Default networking mode is bridge host and awsvpc network modes offer the highest networking performance They use the Amazon EC2 network stack instead of the virtualized network stack provided by the bridge mode Cannot take advantage of dynamic host port mappings Exposed container ports are mapped directly... host: to corresponding host port awsvpc: to attached elastic network interface port Task networking (aka awsvpc mode networking) Benefits Each task has its own attached ENIWith primary private IP address and internal DNS hostname Simplifies container networkingNo host port specified Container port is what is used by task ENI Container ports must be unique in a single task definition Gives more control over how tasks communicate With other tasks Containers share a network namespace Communicate with each other over localhost interfacee.g. curl 127.0.0.1:8080 With other services in VPC Note: containers that belong to the same task can communicate over the localhost interface Take advantage of VPC Flow Logs Better security through use of security groupsYou can assign different security groups to each task, which gives you more fine-grained security Limitations The number of ENIs that can be attached to EC2 instances is fairly smallE.g. c5.large EC2 may have up to 3 ENIs attached to it 1 primary, and 2 for task networking Therefore, you can only host 2 tasks using awsvpc mode networking on a c5.large However, you can increase ENI density using "VPC trunking" VPC trunking Allows for overcoming ENI density limits Multiplexes data over shared communication link How it works Two ENIs are attached to the instance Primary ENI Trunk ENINote that enabling trunking consumes an additional IP address per instance Your account, IAM user, or role must opt in to the awsvpcTrunking account setting Benefits Up to 5x-17x more ENIs per instance E.g. with trunking, c5.large goes from 3 to 12 ENIs1 primary, 1 trunk, and 10 for task networking Migrating a container from EC2 to Fargate IAM roles Roles created automatically by ECS Amazon ECS service-linked IAM role, AWSServiceRoleForECSGives permission to attach ENI to instance Task Execution IAM Role (ecsTaskExecutionRole)Needed for: Pulling images from ECR Pushing logs to CloudWatch Create a task-based IAM role Required because we don't have an ecsInstanceRole anymore Create a IAM policy that gives minimal privileges needed by task Remember two categories of policies: AWS Managed Customer Managed We are going to create a new customer managed policy that contains only the permissions our app needsKMS Decrypt, S3 GETs from specific bucket IAM -> Policies -> Create Policy -> JSONSee IAM Policy example below Create role based on "Elastic Container Service Task" service role This service role gives permission to ECS to use STS to assume role (sts:AssumeRole) and perform actions on its behalf IAM -> Roles -> Create Role "Select type of trusted entity": AWS Service Choose "Elastic Container Service", and then "Elastic Container Service Task" use case Next, then attach IAM policy we created to the role and save Task definition file changes Task-level parameters Add FARGATE for requiredCompatibilities Use awsvpc as the network mode Specify cpu and memory limits at the task level Specify Task Execution IAM Role (executionRoleARN)Allows task to pull images from ECR and send logs to CloudWatch Logs Specify task-based IAM role (taskDefinitionArn)Needed to give task permissions to perform AWS API calls (such as S3 reads) Container-level parametersOnly specify containerPort (do not specify hostPort) See Task Definition example below Create ECS service Choose cluster Specify networking VPC, subnets Create a security group for this task Security group is attached to the ENI Allow inbound port 80 traffic Auto-assign public IP Attach to existing application load balancer Specify production listener (port/protocol) Create a new target group When creating target group, you specify "target type" Instance IP Lambda function For awsvpc mode (and by default, Fargate), you must use the IP target type Specify path pattern for ALB listener, health check pathNote: you cannot specify host-based routing through the consoleYou can update that after creating the service through the ALB console Update security groups Security group for ALBAllow outbound port 80 to the security group we attached to our ENI Security group for RDSAllow inbound port 3306 from the security group for our ENI Create Route 53 recordALIAS pointing to our ALB Log integration with SumoLogic Update task to send logs to stdout/stderrDo not log to file Configure containers for CloudWatch logging ("awslogs" driver) Create Lambda function that subscribes to CloudWatch Log Group Lambda function converts from CloudWatch format to Sumo, then POSTs data to Sumo HTTP Source DeadLetterQueue is recommended to handle/retry failed sends to Sumo Links Introducing Cloud Native Networking for Amazon ECS Containers Task Networking in AWS Fargate Under the Hood: Task Networking for Amazon ECS Task Networking with the awsvpc Network Mode ECS Task Definition - Network Mode How many tasks using the awsvpc network mode can be launched Optimizing Amazon ECS task density using awsvpc network mode Migrating Your Amazon ECS Containers to AWS Fargate Sumo Logic - Collect Logs from AWS Fargate End SongDrifter - Roy EnglandFor a full transcription of this episode, please visit the episode webpage.We'd love to hear from you! You can reach us at: Web: https://mobycast.fm Voicemail: 844-818-0993 Email: ask@mobycast.fm Twitter: https://twitter.com/hashtag/mobycast Reddit: https://reddit.com/r/mobycast

Support Mobycasthttps://glow.fm/mobycastLinks Techcrunch article Mirantis Docker End SongLa Place by IwaFor a full transcription of this episode, please visit the episode webpage.We'd love to hear from you! You can reach us at: Web: https://mobycast.fm Voicemail: 844-818-0993 Email: ask@mobycast.fm Twitter: https://twitter.com/hashtag/mobycast Reddit: https://reddit.com/r/mobycast

Support Mobycasthttps://glow.fm/mobycastIn this episode, we cover the following topics:Identity and access management for ECS Primary roles ECS Container Instance IAM Role ecsInstanceRole IAM policy and role required by ECS agent to make ECS API calls on your behalf ECS Service Scheduler IAM Role ecsServiceRole ECS service scheduler makes calls to EC2 and ELB APIs on your behalfRegister/deregister container instances with load balancers ECS Task Execution IAM Role ecsTaskExecutionRole Also used by ECS agent to make AWS API calls on your behalf Typical use cases Your task uses Fargate and is... pulling a container image from Amazon ECR uses the awslogs log driver Your tasks uses either Fargate or EC2 launch type and... pulls images from private registry the task definition is referencing sensitive data using Secrets Manager or Parameter Store Secondary roles ECS Service Auto Scaling IAM Role ecsAutoscaleRole Used by Application Auto Scaling service to describe CloudWatch alarms and registered servicesUpdates ECS services's desired count CloudWatch Events IAM Role ecsEventsRole Required role when you have ECS scheduled tasks Interacts with CloudWatch Events rules and targets This IAM policy and role gives CloudWatch permissions to run ECS tasks on your behalf ECS CodeDeploy IAM Role ecsCodeDeployRole Required when doing blue/green deployments (powered by CodeDeploy) Best practice: Using task-based IAM roles IAM role for Amazon ECS tasks Allows you to specify an IAM role that can be used by the containers in a task IAM role for task is specified using the taskRoleArn setting in task definition Prefer more granular task-based IAM roles over instance roles Each specific task definition or service should have its own role Benefits of task-based IAM roles Least privilege By specifying access at the task-level (instead of at the instance-level), we can have fine-grained control Only give the minimum required permissions for the tasks to operate Credential isolationContainer can only use credentials assigned to it Auditability Access and event logging available via CloudTrail CloudTrail logs show taskArn Creating a task-based IAM role First create IAM policy that specifies the minimal permissions needed by your containersOr use an existing managed policy Next create an IAM role for your taskCreate role based on Amazon Elastic Container Service Task Role service role Then attach your IAM policy to the task role Example: Container needs to make S3 calls Create a new IAM role for the task, and attach the "AmazonS3ReadOnlyAccess" policy to the role Then use the role ARN in task definition Links Amazon Elastic Container Service AWS Fargate - Product Page ECS Fargate - Developer Guide IAM Roles for Tasks End SongBeauty in Rhythm (Fredy Grogan Remix) - Roy EnglandFor a full transcription of this episode, please visit the episode webpage.We'd love to hear from you! You can reach us at: Web: https://mobycast.fm Voicemail: 844-818-0993 Email: ask@mobycast.fm Twitter: https://twitter.com/hashtag/mobycast Reddit: https://reddit.com/r/mobycast

Support Mobycasthttps://glow.fm/mobycastIn this episode, we cover the following topics: Amazon Elastic Container Service (ECS) basics Orchestration system for containers Well integrated with all the other Amazon services – More bang for your buck ECS components Cluster Logical grouping of tasks or services For EC2 launch type, set of EC2 instances that are defined and managed by: Launch Configuration Auto Scale Group Service Allows you to run and maintain a specified number of instances of a task definition simultaneously For long-running applications Task Defines a collection of containers that you want to run together Specifies resource quotas needed to run (e.g. memory, CPU, disk volumes) Simple deployment with ECSBuild image, publish image, create task definition revision, update ECS service Running containers Three methods Create a long running taskECS service, service scheduler, integration with ELB Run a single task Create a scheduled task We are going to focus on the most typical use case - ECS servicesYou have to choose a launch typeEC2 or Fargate Fargate Announced at re:Invent 2017Generally available since 2018 What is it?Allows you to run containers without having to manage servers or clusters of EC2s Don't need to choose server types, decide when to scale your clusters, or optimize cluster packing You get complete control over task placement within your own VPCBut underlying infrastructure is managed by Fargate Benefits No clusters to manage Seamless scaling Only pay for when you are running tasks Ideal for batch jobs, cron jobs and other on-and-off workloads Running cluster of instances constantly incurs costs, but Fargate stops billing when containers stop Specifics Each Fargate task has its own isolation boundaryIt does not share the underlying kernel, CPU resources, memory resources, or ENI Leverages Firecracker microVM Increases efficiency (e.g. approximately 50% price cut for Fargate in January 2019 due to Firecracker) Tasks must be launched into a cluster Cluster is logical infrastructure and permissions boundary for isolating groups of tasks Clusters support running both EC2 and Fargate launch types (mix-n-match) Fargate tasks require awsvpc network mode Provides each task with an ENI You must specify one or more subnets You must specify one or more security groups Decide on whether to assign public IP address to ENI If on public subnet, you must assign public IP to pull images If on private subnet, just requires NAT gateway You must specify CPU and memory at the task levelYou can also optionally specify CPU and memory at container level Only supports the following log drivers awslogsSends log information to CloudWatch Logs splunk Pricing Based on amount of CPU and memory used Charged by the second, with minimum charge of 1 minute Example costs for running a blog server 24x7 Note: costs for us-west-2 region Fargate, 0.25 VCPU, 0.5GB memory per vCPU per hour: $0.04048 per GB per hour: $0.004445 Memory = $1.60 (30 days * 24 hours * 0.5 GB * 0.004445) CPU = $7.29 (30 days * 24 hours * 0.25VCPU * 0.04048) Total = $8.89 / month t2.micro, 1 VCPU, 1GB memory per hour: $0.0116 Total = $8.35 (30 days * 24 hours * 0.0116) t3.nano, 2 VCPU, 0.5GB memory per hour: $0.0052 Total = $3.74 (30 days * 24 hours * 0.0052) Links Amazon Elastic Container Service ECS Fargate AWS Fargate Price Reduction – Up to 50% End SongERRE - LamictalFor a full transcription of this episode, please visit the episode webpage.We'd love to hear from you! You can reach us at: Web: https://mobycast.fm Voicemail: 844-818-0993 Email: ask@mobycast.fm Twitter: https://twitter.com/hashtag/mobycast Reddit: https://reddit.com/r/mobycast

Support Mobycasthttps://glow.fm/mobycastIn this episode, we cover the following topics: Container runtimes  Responsible for:  Setting up namespaces and cgroups for containers  Running commands inside those namespaces and cgroups  Types of runtimes  Low-level Handles tasks related to containers such as:  Creating a container  Attaching a process to an existing container  High-level  Handles "high level" tasks such as:  Image creation  Image management  Defers container tasks to "low level" runtime  Container standards Open Container Initiative (OCI)  Established in June 2015 by Docker and others  Contains two specifications:  Runtime Specification (runtime-spec) runc is an implementation of OCI runtime-spec  Image Specification (image-spec)  Runc  Low-level container runtime  CLI tool for spawning and running containers according to the OCI specification  Container runtime originally developed as part of Docker Later lifted out as separate open source project  Containerd  High-level container runtime  Provides abstraction layer for the syscalls and OS-specific functionality required to run container  Platforms can build on top of abstraction layer without having to drop down to kernel  Much nicer to work with abstractions (Container, Task, Snapshot) than to manage calls to clone() and mount()  Available as daemon for Linux and Windows  Designed to be embedded into a larger system (like Docker) Used by container platforms like Docker and Kubernetes  Under the hood, containerd uses runc  Only deals with core container management  Push/pull functionality  Image management  Container lifecycle APIs Create, execute and manage containers and their tasks  Snapshot management  Out of scope Networking  Very complicated, much more platform specific than abstracting Linux calls  Instead, containerd exposes events so consumers can subscribe to events they care about E.g. hook events to add interfaces to network namespace  Exposes functionality via GRPC API Listens on Linux socket  Runtime platforms  Docker  Uses containerd, plus shim  Shim allows for daemon-less containers e.g. You can upgrade Docker daemon without restarting containers  rkt  Application container engine, part of CoreOS (RedHat)  rkt has no centralized "init" daemon Instead it launches containers directly from client commands  Was part of CNCF, but archived in August 2019  Reasons cited by CNCF:  "end user adoption has severely declined"  "project activity and the number of contributors has also steadily declined over time"  containerd and CRI-O are now the CNCF container runtime projects  CRI-O  Implementation of the Kubernetes CRI (Container Runtime Interface) to enable using OCI runtimes  Lightweight alternative to using Docker, Moby or rkt as the runtime for k8s  Allows k8s to use any OCI-compliant runtime as the container runtime for running pods  Supports runc and Kata Containers  Revisit pseudo code example of creating a container Steps:  Create root filesystem for container Spin up busybox in Docker container, and then export filesystem  Run "launcher" process that sets up "child" namespace  Launcher process forks new child process (now under new namespaces) Child process then forks new process for container  chroot (to our root filesystem)  mount any other FS  set cgroups (e.g. apply CPU constraints)  Links Open Container Initiative - OCI runc containerd What is containerd? CoreOS rkt rkt vs Docker End SongMiquel Salla - All is Coming Back (Focalist Remix) For a full transcription of this episode, please visit the episode webpage.We'd love to hear from you! You can reach us at: Web: https://mobycast.fm Voicemail: 844-818-0993 Email: ask@mobycast.fm Twitter: https://twitter.com/hashtag/mobycast Support Mobycast:https://glow.fm/mobycast

In this episode, we cover the following topics: Operating-system-level virtualization = containers Allows the resources of a computer to be partitioned via the kernelAll containers share single kernel with each other AND the host system Depend on their host OS to do all the communication and interaction with the physical machineContainers don't need a hypervisor; they run directly within the host machine's kernel Containers are using the underlying operational system resources and drivers This is why you cannot run different OSes on the same host systemi.e. Windows containers can run on Windows only, and Linux Containers can run on Linux only What we think of different OSes (RHEL, CentOS, SUSE, Debian, Ubuntu) are not really different...They are all same core OS (Linux), they just differ in apps/files Based on the virtualization, isolation, and resource management mechanisms provided by the Linux kernel namespaces cgroups Container history FreeBSD Jails (2000) BSD userland software that runs on top of the chroot(2) system callchroot is used to change the root directory of a set of processes Processes created in the chrooted environment cannot access files or resources outside of it Jails virtualize access to the file system, the set of users, and the networking subsystem A jail is characterized by four elements: Directory subtree: the starting point from which a jail is enteredOnce inside the jail, a process is not permitted to escape outside of this subtree Hostname IP address Command: the path name of an executable to run inside the jail Configured via jail.conf file LXC containers (2008)Userspace interface for the Linux kernel features to contain processes, including: Kernel namespaces (ipc, uts, mount, pid, network and user) Apparmor and SELinux profiles Seccomp policies Chroots (using pivot_root) Kernel capabilities CGroups (control groups) Docker containers (2014) Early versions of Docker used LXC as the container runtime LXC was made optional in v0.9 (March 2014) Replaced by libcontainer) libcontainer became the core of runC LXC was dropped in v1.10 (February 2016) Container technology Containers are just processes. So what makes them special? Namespaces Restrict what you can SEE Virtualize system resources, like the file system or networking Makes it appear to processes within the namespace that they have their own isolated instance of resource Changes to the global resource only visible to processes that are members of the namespace Processes inherit from parent Linux provides the following namespaces: IPC (interprocess communications)CLONE_NEWIPC: Isolates System V IPC, POSIX message queues NetworkCLONE_NEWNET: Isolates network devices, stacks, ports, etc MountCLONE_NEWNS: Isolates mount points PIDCLONE_NEWPID: Isolates process IDs UserCLONE_NEWUSER: Isolates user and group IDs UTS (Unix Timesharing System)CLONE_NEWUTS: Isolates hostname and NIS domain name CgroupCLONE_NEWCGROUP: Isolates cgroup root directory Syscall interfaceSystem call is the fundamental interface between an app and the Linux kerneli.e. Linux kernel calls to create/enter namespaces for processes Control groups (cgroups) Restrict what you can DO Limits an application (container) to a specific set of resources like CPU and memory Allow containers to share available hardware resources and optionally enforce limits and constraints Creating, modifying, using cgroups is done through the cgroup virtual filesystem Processes inherit from parent Can be reassigned to different cgroups Memory CPU / CPU cores Devices I/O Processes Using cgroups To see mounted cgroups:mount | grep cgroup To create a new cgroup:mkdir /sys/fs/cgroup/cpu/chris To set "cpu.shares" to 512:echo 512 > /sys/fs/cgroup/cpu/chris/cpu.shares Now add a process to this cgroup:echo > /sys/fs/cgroup/cpu/chris/cgroup.procs Pseudo code: Creating a containerSteps: Create root filesystem for containerSpin up busybox in Docker container, and then export filesystem Run "launcher" process that sets up "child" namespace Launcher process forks new child process (now under new namespaces)Child process then forks new process for container chroot (to our root filesystem) mount any other FS set cgroups (e.g. apply CPU constraints) Links FreeBSD Jails Linux Container Project - LXC, LXD, LXCFS namespaces - overview of Linux namespaces cgroups kernel documentation What Have Namespaces Done For You Lately? - YouTube video End SongBettie Black & Sophia - Something BeautifulFor a full transcription of this episode, please visit the episode webpage.We'd love to hear from you! You can reach us at: Web: https://mobycast.fm Voicemail: 844-818-0993 Email: ask@mobycast.fm Twitter: https://twitter.com/hashtag/mobycast

Sponsors Circle CI Episode on CI/CD with Circle CI Show DetailsIn this episode, we cover the following topics: Hypervisor implementations  Hyper-V  Type 1 hypervisor from Microsoft  Architecture  Implements isolation of virtual machines in terms of a partition Partition is logical unit of isolation in which each guest OS executes  Parent partition  Virtualization software runs in parent partition and has direct access to hardware Requires supported version of Windows Server  There must be at least one parent partition  Parent partition creates child partitions which host the guest OSes Done via Hyper-V "hypercall" API  Parent partitions run a Virtualization Service Provider (VSP) which connects to the VMBus Handles device access requests from child partition  Child partition  Does not have direct access to hardware Has virtual view of processor and runs in Guest Virtual Address (not necessarily the entire virtual address space)  Hypervisor handles interrupts to processor, and redirects to respective partition  Any request to the virtual devices is redirected via the VMBus to the devices in the parent partition  VMBus Logical channel which enables inter-partition communication  KVM (Kernel-based Virtual Machine)  Virtualization module in Linux kernel  Turns Linux kernel into hypervisor  Available in mainline Linux since 2007  Can run multiple VMs running unmodified Linux or Windows images  Leverages hardware virtualization Via CPU virtualization extensions (Intel VT or AMD-V)  But also provides paravirtualization support for Linux/FreeBSD/NetBSD/Windows using VirtIO API  Architecture  Kernel component Consists of:  Loadable kernel module, kvm.ko, that provides the core virtualization infrastructure  Processor specific module, kvm-intel.ko or kvm-amd.ko  Userspace component QEMU (Quick Emulator)  Userland program that does hardware emulation  Used by KVM for I/O emulations  AWS hypervisor choices & history  AWS uses custom hardware for faster EC2 VM performance  Original EC2 technology ran highly customized version of Xen hypervisor  VMs can run using either paravirtualization (PV) or hardware virtual machine (HVM)  HVM guests are fully virtualized VMs on top of hypervisor are not aware they are sharing with other VMs  Memory allocated to guest OSes is scrubbed by hypervisor when it is de-allocated  Only AWS admins have access to hypervisors  AWS found that Xen has many limitations that impede their growth Engineers improved performance by moving parts of software stack to purpose-built hardware components  C3 instance family (2013) Debut of custom chips in Amazon EC2 Custom network interface for faster bandwidth and throughput  C4 instance family (2015) Offload network virtualization to custom hardware with ASIC optimized for storage services  C5 instance family (2017) Project Nitro  Traditional hypervisors do everything Protect the physical hardware and bios, virtualize the CPU, storage, networking, management tasks  Nitro breaks apart those functions, offloading to dedicated hardware and software  Replace Xen with a highly optimized KVM hypervisor tightly coupled with an ASIC  Very fast VMs approaching performance of bare metal server  Amazon EC2 – Bare metal instances (2017) Use Project Nitro  Links Xen Project Kernel Virtual Machine QEMU Mastering KVM Virtualization Hyper-V AWS Nitro System AWS re:Invent 2018: Powering Next-Gen EC2 Instances: Deep Dive into the Nitro System AWS re:Invent 2017: C5 Instances and the Evolution of Amazon EC2 Virtualization End SongFax - StagesFor a full transcription of this episode, please visit the episode webpage.We'd love to hear from you! You can reach us at: Web: https://mobycast.fm Voicemail: 844-818-0993 Email: ask@mobycast.fm Twitter: https://twitter.com/hashtag/mobycast 

Sponsor Circle CI Episode on CI/CD with Circle CI Show DetailsIn this episode, we cover the following topics: VMs vs containers - why revisit?Originally talked about this in episode 1 Got most of it right, but some inconsistencies/holes Let's revisit to fill in the gaps, and dive a whole LOT deeper this time around Types of virtualization Full virtualization ("virtual machines") Simulates enough hardware to allow an unmodified "guest" OS to be run in isolation Resources of computer are partitioned via hypervisor Examples:VMWare, Parallels, VirtualBox, Hyper-V Operating-system-level virtualization ("containers") Resources of computer are partitioned via the kernel"Guest" OSes share same running instance of OS as the host system Based on the virtualization, isolation, and resource management mechanisms provided by the Linux kernelnamespaces and cgroups Examples:Docker, LXC, FreeBSD jails Hypervisors Also known as a Virtual Machine Manager (VMM) Creates and runs virtual machines It is a process that separates OS and apps from underlying physical hardware Multiple VMs share virtualized hardware resources When you create a new VM, the following happens: Hypervisor allocates memory and CPU space for VMs exclusive use Complete OS is installed onto the VM The VM's OS communicates with the hypervisor to perform tasks Host OS is able to see all physical hardware, whereas guest OS (VM) can only see hardware to which hypervisor has granted access Two types of hypervisors Type 1 (also called "native" or "bare metal" hypervisors) Run directly on the host's hardware to control the hardware and manage the guest VMsruns in ring 0 Are an OS themselves (simple OS on top of which you run VMs)the physical machine the hypervisor is running on serves only for virtualization purposesExceptions: Hyper-V, KVM ExamplesXen, Microsoft Hyper-V, VMware ESX/ESXi Type 2 (also called "hosted" hypervisors) Run on conventional OS, just like other apps Guest OS runs as a process on the host Hypervisor separates the guest OS from the host OS ExamplesVirtualBox, Parallels Protection levels (rings) x86 family of CPUs provide a range of protection levels also known as rings Ring 0 has the highest level privilege (kernel/supervisor) Ring 3 lowest level (applications) Hypervisor occupies ring 0 of CPU Kernels for any guest operating systems running on the system must run in less privileged CPU rings But most OS kernels are written explicitly to run in ring 0 Techniques to deal with this: Full virtualization hypervisor provides CPU emulation to handle ring 0 operations made by unmodified guest OS kernels emulation process requires both time and system resourcesinferior performance Paravirtualization Technique in which hypervisor provides an API and the OS of the guest VM calls that API Requires guest OS to be modified (to make API calls)Replace any privileged operations that will only run in ring 0 of the CPU with calls to the hypervisor ("hypercalls") Allows tasks to run in host OS (instead of in guest OS where performance would be worse) Hardware virtualization Requires a CPU with hardware virtualization extensions, such as Intel VT or AMD-VIntel virtualization (VT-x) Virtual Machine Extensions Adds ten new instructions VMPTRLD, VMPTRST, VMCLEAR, VMREAD, VMWRITE, VMCALL, VMLAUNCH, VMRESUME, VMXOFF, and VMXON. These instructions permit entering and exiting a virtual execution mode where the guest OS perceives itself as running with full privilege (ring 0), but the host OS remains protected. Reduces/eliminates any OS modifications in guest OS Provides an additional privilege mode above ring 0 in which the hypervisor can operateessentially leaving ring 0 available for unmodified guest OSes Better performance than paravirtualization Links Virtual machine Hypervisor What is a hypervisor? What Is A Hypervisor? Types Of Hypervisors 1 & 2 End SongTime for Trees - Sad Livin in the (New York) City - (David Last Remix)For a full transcription of this episode, please visit the episode webpage.We'd love to hear from you! You can reach us at: Web: https://mobycast.fm Voicemail: 844-818-0993 Email: ask@mobycast.fm Twitter: https://twitter.com/hashtag/mobycast

Sponsor Circle CI Episode on CI/CD with Circle CI Show DetailsIn this episode, we cover the following topics: Pillars in depth Performance Efficiency "Ability to use resources efficiently to meet system requirements and maintain that efficiency as demand changes and technology evolves" Design principles Easy to try new advanced technologies (by letting AWS manage them, instead of standing them up yourself) Go global in minutes Use serverless architectures Experiment more often Mechanical sympathy (use the technology approach that aligns best to what you are trying to achieve) Key service: CloudWatch Focus areas SelectionServices: EC2, EBS, RDS, DynamoDB, Auto Scaling, S3, VPC, Route53, DirectConnect ReviewServices: AWS Blog, AWS What's New MonitoringServices: CloudWatch, Lambda, Kinesis, SQS TradeoffsServices: CloudFront, ElastiCache, Snowball, RDS (read replicas) Best practices SelectionChoose appropriate resource typesCompute, storage, database, networking Trade OffsProximity and caching Cost Optimization "Ability to run systems to deliver business value at the lowest price point" Design principles Adopt a consumption model (only pay for what you use) Measure overall efficiency Stop spending money on data center operations Analyze and attribute expenditures Use managed services to reduce TCO Key service: AWS Cost Explorer (with cost allocation tags) Focus areas Expenditure awarenessServices: Cost Explorer, AWS Budgets, CloudWatch, SNS Cost-effective resourcesServices: Reserved Instances, Spot Instances, Cost Explorer Matching supply and demandServices: Auto Scaling Optimizing over timeServices: AWS Blog, AWS What's New, Trusted Advisor Key pointsUse Trusted Advisor to find ways to save $$$ The Well-Architected Review Centered around the question "Are you well architected?" The Well-Architected review provides a consistent approach to review workload against current AWS best practices and gives advice on how to architect for the cloud Benefits of the review Build and deploy faster Lower or mitigate risks Make informed decisions Learn AWS best practices The AWS Well-Architected Tool Cloud-based service available from the AWS console Provides consistent process for you to review and measure your architecture using the AWS Well-Architected Framework Helps you: Learn Measure Improve Improvement plan Based on identified high and medium risk topics Canned list of suggested action items to address each risk topic MilestonesMakes a read-only snapshot of completed questions and answers Best practices Save milestone after initially completing workload review Then, whenever you make large changes to your workload architecture, perform a subsequent review and save as a new milestone Links AWS Well-Architected AWS Well-Architected Framework - Online/HTML versionincludes drill down pages for each review question, with recommended action items to address that issue AWS Well-Architected Tool Enhanced Networking Amazon EBS-optimized instance VPC Endpoint Amazon S3 Transfer Acceleration AWS Billing and Cost Management Whitepapers AWS Well-Architected Framework Operational Excellence Pillar Security Pillar Reliability Pillar Performance-Efficiency Pillar Cost Optimization Pillar End Song:The Shadow Gallery by Roy EnglandFor a full transcription of this episode, please visit the episode webpage.We'd love to hear from you! You can reach us at: Web: https://mobycast.fm Voicemail: 844-818-0993 Email: ask@mobycast.fm Twitter: https://twitter.com/hashtag/mobycast Reddit: https://reddit.com/r/mobycast

In this episode, we cover the following topics: Pillars in depth Security "Ability to protect information, systems, and assets while delivering business value through risk assessments and mitigation strategies" Design principles Implement strong identity foundation Enable traceability Security at all layers Automate security best practices Protect data in transit and at rest Keep people away from data Prepare for security events Key service: AWS IAM Focus areas Identity and access managementServices: IAM, AWS Organizations, MFA Detective controlsServices: CloudTrail, CloudWatch, AWS Config, GuardDuty Infrastructure protectionServices: VPC, Shield, WAF Data protectionServices: KMS, ELB (encryption), Macie (detect sensitive data) Incident responseServices: IAM, CloudFormation Best practices Identity and access managementAWS Cognito Act as broker between login providers Securely access any AWS service from mobile device Data protection Encrypt Encryption at rest Encryption in transit Encrypted backups Versioning Storage resiliency Detailed logging Incident responseEmploy strategy of templated "clean rooms" Create new trusted environment to conduct investigation Use CloudFormation to easily create the "clean room" environment Reliability "Ability to recover from failures, dynamically acquire resources to meet demand and mitigate disruptions such as network issues" Design principles Test recovery procedures Auto recover from failures Scale horizontally to increase availability Stop guessing capacity Manage change with automation Key service: CloudWatch Focus areas FoundationsServices: IAM, VPC, Trusted Advisor (visibility into service limits), Shield (protect from DDoS) Change managementServices: CloudTrail, AWS Config, CloudWatch, Auto Scaling Failure managementServices: CloudFormation, S3, Glacier, KMS Best practices Foundations Take into account physical and service limits High availability No single points of failure (SPOF) Multi-AZ design Load balancing Auto scaling Redundant connectivity Software resilience Failure management Backup and disaster recoveryRPO, RTO Inject failures to test resiliency Key points Plan network topology Manage your AWS service and rate limits Monitor your system Automate responses to demand Backup In the next episode, we'll cover the remaining 2 pillars and discuss how to perform a Well-Architected Review. Links AWS Well-Architected AWS Well-Architected Framework - Online/HTML versionincludes drill down pages for each review question, with recommended action items to address that issue AWS re:Invent 2018: How AWS Minimizes the Blast Radius of Failures - ARC338 Shuffle Sharding: Massive and Magical Fault Isolation Whitepapers AWS Well-Architected Framework Operational Excellence Pillar Security Pillar Reliability Pillar Performance-Efficiency Pillar Cost Optimization Pillar End song:The Runner (David Last Remix) - FaxFor a full transcription of this episode, please visit the episode webpage.We'd love to hear from you! You can reach us at: Web: https://mobycast.fm Voicemail: 844-818-0993 Email: ask@mobycast.fm Twitter: https://twitter.com/hashtag/mobycast

In this episode, we cover the following topics: AWS Well-Architected Framework Provides consistent approach to evaluating systems against cloud best practices Helps advise changes necessary to make specific architecture align with best practices Comprised of 3 components: Design Principles Pillars Operational Excellence Security Reliability Performance Efficiency Cost Optimization Questions General design principlesCloud-native has changed everything. In cloud, you can: Stop guessing capacity needs Test at scale Automate all the things to make experimentation easier Allow for evolutionary architectures (you are never stuck with a particular technology) Drive architectures using data (allows you to make fact based decisions on how to improve your workload) Improve through game days Pillars in depthOperational Excellence "Ability to run and monitor systems to deliver business value and to continuously improve supporting processes and procedures" Design principles Perform operations as code Annotate documentation Make frequent, small, reversible changes Refine operations procedures frequently Anticipate failure Learn from all operational failures Key service: CloudFormation Focus areas PrepareServices: AWS Config, AWS Config Rules OperateServices: CloudWatch, X-Ray, CloudTrail, VPC Flow Logs EvolveServices: Elasticsearch (for searching log data to gain insights), CloudWatch Insights Best practices Prepare Implement telemetry for: Application Workload User activity Dependencies Implement transaction traceability Operate Any event for which you raise an alert should have associated runbookRunbook defines triggers for escalations Users should be notified when system is impacted Communicate status through dashboardsProvide dashboards to communicate the current operating status of the business and provide metrics of interest EvolveFeedback loops Identify areas for improvement Gauge impact of changes to the system (i.e. did it make an improvement?) Perform operations metrics reviewsRetrospective analysis of operations metricsUse these reviews to identify opportunities for improvement, potential courses of action, and share lessons learned Key points Runbooks, playbooks Document environments Make small changes through automation Monitor workload with business metrics Exercise your response to failures Have well-defined escalation management In future episodes, we'll cover the remaining 4 pillars Links AWS Well-Architected Framework - Online/HTML versionincludes drill down pages for each review question, with recommended action items to address that issue Are You Well-Architected? AWS re:Invent 2016 Keynote: Werner VogelsSee: 25:45 through 31:25 Runbooks Playbooks AWS Service Health Dashboard AWS Personal Health Dashboard Whitepapers AWS Well-Architected Framework Operational Excellence Pillar Security Pillar Reliability Pillar Performance-Efficiency Pillar Cost Optimization Pillar End Song:30 Days & 30 Nights by Fortune FinderFor a full transcription of this episode, please visit the episode webpage.We'd love to hear from you! You can reach us at: Web: https://mobycast.fm Voicemail: 844-818-0993 Email: ask@mobycast.fm Twitter: https://twitter.com/hashtag/mobycast Reddit: https://reddit.com/r/mobycast

In episode 77 of Mobycast, Jon and Chris walk through the Twelve-Factor App framework. Welcome to Mobycast, a weekly conversation about cloud-native development, AWS, and building distributed systems.

The Twelve-Factor App methodology  Drafted by developers at Heroku based upon their observations of what made good apps  First presented by Adam Wiggins circa 2011 (then published in 2012)  The Factors  1 - Codebase: one codebase tracked in revision control, many deploys  2 - Dependencies: explicitly declare and isolate dependencies  3 - Config: strict separation of config from code  4 - Backing services: foster loose coupling by treating backing services as attached resources  5 - Build, release, run: strictly separate build and run stages  6 - Processes: processes are stateless and share-nothing  7 - Port binding: export services via port binding  8 - Concurrency: scale out via the process model  9 - Disposability: processes are disposable, they can be started or stopped at a moment's notice  10 - Dev/prod parity: Keep development, staging, and production as similar as possible  11 - Logs: treat logs as event streams, don't manage log files  12 - Admin processes: admin and utility code ships with app code to avoid synchronization issues  What's Missing? 7 years since first being published, what changes should be made to make it more relevant for today?  Some have argued for adding 3 additional factors:  Telemetry  Security  "API First"-philosophy  For a full transcription of this episode, please visit the episode webpage.End song:Flowerchild (Roy England Remix) by Owen Ni - Make MistakesWe'd love to hear from you! You can reach us at: Web: https://mobycast.fm Voicemail: 844-818-0993 Email: ask@mobycast.fm Twitter: https://twitter.com/hashtag/mobycast Reddit: https://reddit.com/r/mobycast

In Episode 76 of Mobycast, Jon and Chris finish our series on encryption by digging into AWS’ encryption services. Welcome to Mobycast, a weekly conversation about cloud-native development, AWS, and building distributed systems.

In Episode 75 of Mobycast, we continue with part three of our series on encryption. In particular, we discuss end-to-end encryption in practice. Welcome to Mobycast, a weekly conversation about cloud-native development, AWS, and building distributed systems.

In Episode 75 of Mobycast, we continue with part three of our series on encryption. In particular, we discuss end-to-end encryption in practice. Welcome to Mobycast, a weekly conversation about cloud-native development, AWS, and building distributed systems.

In episode 74 of Mobycast, we continue with part two of our series on encryption. In particular, we'll discuss Transport Layer Security in practice. Welcome to Mobycast, a weekly conversation about cloud-native development, AWS, and building distributed systems.

In episode 74 of Mobycast, we continue with part two of our series on encryption. In particular, we'll discuss Transport Layer Security in practice. Welcome to Mobycast, a weekly conversation about cloud-native development, AWS, and building distributed systems.

In episode 73 of Mobycast, we start a new series on encryption and dive into the essentials. Welcome to Mobycast, a weekly conversation about cloud-native development, AWS, and building distributed systems.

In episode 73 of Mobycast, we start a new series on encryption and dive into the essentials. Welcome to Mobycast, a weekly conversation about cloud-native development, AWS, and building distributed systems.

In Episode 72 of Mobycast, we dive into part two of our discussion on growing high performing remote and international engineering teams. Welcome to Mobycast, a weekly conversation about cloud-native development, AWS, and building distributed systems.

In Episode 72 of Mobycast, we dive into part two of our discussion on growing high performing remote and international engineering teams. Welcome to Mobycast, a weekly conversation about cloud-native development, AWS, and building distributed systems.

In episode 71 of Mobycast, Jon and Chris discuss lessons learned while working with remote and international engineering teams. Welcome to Mobycast, a weekly conversation about cloud-native development, AWS, and building distributed systems.

In episode 71 of Mobycast, Jon and Chris discuss lessons learned while working with remote and international engineering teams. Welcome to Mobycast, a weekly conversation about cloud-native development, AWS, and building distributed systems.

In Episode 70 of Mobycast, we wrap up our bootcamp on Microservices with the discussion on Micro Frontends. Welcome to Mobycast, a weekly conversation about cloud-native development, AWS, and building distributed systems.