POPULARITY
Global experiential agency vStream has collaborated with AMD, the high-performance and adaptive computing leader, to launch in Dublin the first European AMD Product Experience Centre. This groundbreaking interactive and immersive centre is designed to showcase the transformative power of the latest innovations from AMD in real-world applications across AI, cloud computing, gaming, and more by using immersive storytelling, interactive displays, and cutting-edge augmented reality experiences. The new centre was opened by Ruth Cotter, Senior Vice-President for Marketing, Communications and Human Resources at AMD, together with Andrew Jenkinson and Niall O'Driscoll, co-founders of vStream, and Brendan Farley, Managing Director Ireland and Corporate Vice President of Wireless Engineering at AMD. Andrew Jenkinson, Co-Founder of vStream, commented: "We are incredibly proud to have worked with AMD on this project. The AMD Product Experience Centre is a testament to our shared vision of innovation, bringing advanced technology to life in a way that is both engaging and educational. Our expertise in immersive storytelling has allowed us to create a space where visitors can truly interact with and understand the game-changing technologies from AMD." Niall O'Driscoll, Co-Founder of vStream, added: "At vStream, we believe that technology should be experienced, not just explained. The AMD Product Experience Centre encapsulates that philosophy by turning complex semiconductor innovations into a tangible, interactive journey. We are excited to see visitors explore and engage with incredible technological advancements from AMD." Ruth Cotter, SVP Marketing, Communications and Human Resources at AMD, said: "We are delighted to open AMD's inaugural Product Experience Centre that is strategically aligned to our R&D and engineering presence in Ireland and is expected to support further collaboration and breakthrough innovations with our customers and partners. Our collaboration with vStream enables us to showcase how AMD's cutting-edge processors and adaptive computing technologies are shaping the future across a range of industries through a unique immersive experience." A Journey Through Innovation Designed and built by vStream, the award-winning experiential agency, the AMD Product Experience Centre integrates multiple interactive elements to engage visitors in a dynamic and memorable way: A Personal Welcome from AMD CEO Dr. Lisa Su - Upon entering the space, visitors are greeted by AMD's CEO via a large switchable glass screen, setting the stage for an inspiring exploration of AMD technology. Interactive Display Cases - Through an intuitive gamified experience, visitors can see how AMD processors power the world around us, with motion graphics, animations, and videos revealing physical products inside the futuristic transparent LED display cases. The Intelligent Table - An innovative tactile experience where users place AMD chips - AMD Versal, AMD Ryzen, AMD EPYC, and AMD Instinct - on the table surface to unlock interactive content that explores their impact across industries. Augmented Reality Experiences - A deep dive into how AMD is driving AI advancements in PCs with AMD Ryzen processors and in supercomputers with the latest generations of AMD EPYC processors and AMD Instinct accelerators, offering visitors a first-hand look at the future of AI-powered computing. Built on vStream's award-winning work in Augmented Reality with brands including Pfizer and McLaren F1, this experience allows the user to interact with digital assets, while still seeing their surroundings and interacting with other guests - a true example of the power of spatial computing. A Seamless Blend of Innovation and Experience The AMD Experience Centre was meticulously crafted by vStream, from concept to production and installation, ensuring a seamless fusion of technology and storytelling. With a fully user-friendly management system, AMD representatives can demonstr...
See the latest innovations in silicon design from AMD with new system-on-a-chip high bandwidth memory breakthroughs with up to 7 terabytes of memory bandwidth in a single virtual machine - and how it's possible to get more than 8x speed-ups without sacrificing compatibility from the previous generation to HBv5. These use AMD EPYC™ 9004 Processors with AMD 3D V-Cache™ Technology. And find out how Microsoft's own silicon including custom ARM-based Cobalt CPUs and Maia AI accelerators for performance and power efficiency. Mark Russinovich, Azure CTO, Deputy CISO, Technical Fellow, and Microsoft Mechanics lead contributor, shows how with workloads spanning Databricks, Siemens, Snowflake, or Microsoft Teams, Azure provides the tools to improve efficiency and performance in your datacenter at hyperscale. ► QUICK LINKS: 00:00 - 7TB memory bandwidth in a single VM 00:51 - Efficiency and optimization 02:33 - Choose the right hardware for workloads 04:52 - Microsoft Cobalt CPUs and Maia AI accelerators 06:14 - Hardware innovation for diverse workloads 07:53 - Speedups with HBv5 VMs 09:04 - Compatibility moving from HBv4 to HBv5 11:29 - Future of HPC 12:01 - Wrap up ► Link References Check out https://aka.ms/AzureHPC For more about HBv5 go to https://aka.ms/AzureHBv5 ► Unfamiliar with Microsoft Mechanics? As Microsoft's official video series for IT, you can watch and share valuable content and demos of current and upcoming tech from the people who build it at Microsoft. • Subscribe to our YouTube: https://www.youtube.com/c/MicrosoftMechanicsSeries • Talk with other IT Pros, join us on the Microsoft Tech Community: https://techcommunity.microsoft.com/t5/microsoft-mechanics-blog/bg-p/MicrosoftMechanicsBlog • Watch or listen from anywhere, subscribe to our podcast: https://microsoftmechanics.libsyn.com/podcast ► Keep getting this insider knowledge, join us on social: • Follow us on Twitter: https://twitter.com/MSFTMechanics • Share knowledge on LinkedIn: https://www.linkedin.com/company/microsoft-mechanics/ • Enjoy us on Instagram: https://www.instagram.com/msftmechanics/ • Loosen up with us on TikTok: https://www.tiktok.com/@msftmechanics
Aujourd'hui, voici pourquoi Lisa Su, la PDG d'AMD, vient d'être nommée PDG de l'année 2024 par le magazine Time.Il faut dire qu'au milieu des annonces autour de la bataille de l'IA entre Qualcomm, Nvidia et Intel, AMD apparaît moins dans les titres de presse. Pourtant, la PDG de ce spécialiste de l'infrastructure vient d'être honorée.Alors pourquoi ce titre si prestigieux ?AMD a centuplé sa capitalisation boursièreEt bien pour commencer AMD a centuplé sa capitalisation boursière sous la direction de Lisa Su. En 2014, quand elle prend les commandes de ce géant des semi-conducteurs, l'entreprise traverse une période difficile.Mais grâce à une stratégie audacieuse et une vision claire, AMD est passé en 10 ans d'une capitalisation boursière de 2 milliards de dollars à plus de 200 milliards aujourd'hui.Un des plus grands tournants initié par la femme d'affaires a été le développement et la commercialisation de la gamme de processeurs AMD EPYC. Ce sont ces puces qui ont permis à AMD de devenir un acteur majeur dans les secteurs des serveurs informatiques et des centres de données. Sous le règne de Lisa Su, la part de marché de l'entreprise dans ce secteur est passée de 1 % à près de 34 %. De quoi damer le pion à l'éternel rival Intel, qui vient lui de perdre son PDG, mis de force à la retraite.Et les processeurs EPYC équipent aujourd'hui certains des superordinateurs les plus rapides et les plus économes en énergie au monde.La reine de l'innovationLe second point, c'est que Lisa Su est aussi une experte de l'innovation. Sous son leadership, AMD a investi massivement en recherche et développement, avec un montant de près de 6 milliards de dollars rien qu'en 2023.Ces investissements permettent à AMD de proposer désormais des solutions d'infrastructure pour l'intelligence artificielle, un domaine clé pour l'avenir.À titre d'exemple, AMD a récemment racheté Silo AI, un laboratoire d'IA en Europe, et ZT Systems, un fournisseur d'infrastructure spécialisé pour les géants du cloud.La plus grosse acquisition du secteur, c'est ellePour couronner le tout, Lisa Su a aussi marqué l'histoire des semi-conducteurs en réussissant la plus grosse acquisition jamais réalisée dans ce secteur. Il s'agit de celle de Xilinx, spécialiste de l'informatique adaptative.Une opération qui a fait d'AMD un leader incontournable du secteur.Née à Taïwan et diplômée du prestigieux Massachusetts Institute of Technology, Lisa Su a toujours été une pionnière. Elle a commencé sa carrière chez IBM et pilote aujourd'hui une entreprise à la pointe de l'innovation.Le ZD Tech est sur toutes les plateformes de podcast ! Abonnez-vous !Hébergé par Ausha. Visitez ausha.co/politique-de-confidentialite pour plus d'informations.
We cover linux for better Mac gaming, Thunderbird's Android beta, and NVIDIA's Wayland progress. There's Open Razer, the release of Ubuntu 24.10, and more news about Gnome foundation. And don't forget the AMD Epyc Turin server CPU release with impressive AVX-512 support. For tips, we have "look" for doing dictionary lookups, yum and dnf tricks, pv for monitoring or slowing a pipe, and bless for a useful hex editor GUI. See the show notes at https://bit.ly/4876oxh and until next time! Host: Jonathan Bennett Co-Hosts: Rob Campbell, Jeff Massie, and David Ruggles Want access to the video version and exclusive features? Become a member of Club TWiT today! https://twit.tv/clubtwit Club TWiT members can discuss this episode and leave feedback in the Club TWiT Discord.
AMD is sinds 2016 bezig met een stevige opmars. Anno 2024 is het bedrijf succesvoller dan ooit, het produceert steeds meer verschillende chips. De chipfabrikant kent weinig tegenslagen en elke nieuwe serie is weer wat succesvoller dan de vorige.AMD produceert op dit moment verschillende soorten chips. Het heeft Ryzen-processors voor desktops en laptops waarmee het langzaam terrein wint. Daarnaast heeft het de AMD Epyc chips voor het datacenter. Deze doen het erg goed, omdat deze chips niet alleen krachtiger, maar ook veel energiezuiniger zijn en minder koeling vereisen. Steeds meer datacenters kiezen om die reden voor AMD. Het operationeel houden van een AMD-stack is simpelweg goedkoper dan Intel, vanwege de stroom- en koelingkosten. Daarnaast is AMD ook nog steeds competitief op de GPU-markt. In eerste instantie vooral op de consumentenmarkt met de Radeon-serie. De kennis die het daar heeft opgedaan heeft AMD echter ook een goede start geboden in de markt voor AI-chips. Nvidia is weliswaar de absolute marktleider in GPU's voor AI-doeleinden, maar AMD doet het op ruime afstand ook zeker niet slecht. De AI-chips van AMD verkopen als warme broodjes en zorgen voor flink wat extra omzet.AMD heeft op dit moment twee grote concurrenten: Nvidia voor de AI-chips en de GPU's en Intel voor de CPU's. AMD heeft Intel op het CPU-vlak inmiddels ruimschoots afgetroefd, op de GPU's en AI-chips is er nog ruimte voor verbetering. In die markt zet AMD echter ook stappen.In deze aflevering van Techzine Talks analyseren we de chips van AMD, de strategie van het bedrijf, de verhoudingen tot de concurrentie en uiteraard de toekomst.
En el actual panorama tecnológico, donde las tecnologías críticas son cada vez más demandadas, Dell Technologies y AMD anticipan que la Inteligencia Artificial (IA) se consolida como impulsor de la innovación en 2024. Un aspecto clave en este avance es el rol de los ecosistemas abiertos. Este jueves 25 de enero, conscientes de esta tendencia, Dell y AMD han comenzado a capacitar a sus canales en colaboración con su principal socio en Bolivia DMC SA. El objetivo es claro: proporcionar a los clientes soluciones sencillas, seguras y a medida, capaces de abordar desafíos complejos mediante el uso de IA.
Ep#172 AWS re:invent 2023 AMD EPYC Recap: Do More With Less with Mike Thompson
Ep#172 AWS re:invent 2023 AMD EPYC Recap: Do More With Less with Mike Thompson
This week Noah and Steve discuss picking out a vHost and considerations for deploying it into production. -- During The Show -- 02:00 Types of AI Amount of compute required is astronomical Foundational model vs tweaking 05:55 Kid Friendly distro? - Chris Endless OS (https://www.endlessos.org/) What age to give kids a computer Why give a kid a computer Why Endless OS OpenDNS Filtering 14:13 Serial Connection To Proxmox VMs - Michael Client Setting Host Setting Enable the serial console Proxmox Wiki (https://pve.proxmox.com/wiki/Serial_Terminal) 17:15 pfSense blocking active connections - Bradly Stateful firewalls don't break active connections/sessions 21:00 News Wire EXT4 Corruption Bug - LWN (https://lwn.net/Articles/954285/) Gnome 45.2 - Gnome (https://discourse.gnome.org/t/gnome-45-2-released/18358) Libreoffice 7.6.4 - Libreoffice (https://www.libreoffice.org/download/release-notes/) Jellyfin Android TV App - Jellyfin (https://jellyfin.org/posts/androidtv-v0.16.0/) Jellyfin Roku App - Jellyfin (https://jellyfin.org/posts/roku-200) Debian 12.4 - Debian (https://www.debian.org/News/2023/20231210) Alpine Linux 3.19 - Alpine Linux (https://wiki.alpinelinux.org/wiki/Release_Notes_for_Alpine_3.19.0) Linux 6.8 Dropping Old Graphics Drivers - Phoronix (https://www.phoronix.com/news/Linux-6.8-No-More-UMS-ioctls) NSA & ESF Recommended Practices - NSA (https://www.nsa.gov/Press-Room/Press-Releases-Statements/Press-Release-View/Article/3613105/nsa-and-esf-partners-release-recommended-practices-for-managing-open-source-sof/) OpenZeppelin Vulnerability - Bleeping Computer (https://www.bleepingcomputer.com/news/security/multiple-nft-collections-at-risk-by-flaw-in-open-source-library/) Bluetooth Authentication Bypass - Silicon Angle (https://siliconangle.com/2023/12/07/critical-bluetooth-security-flaw-discovered-google-apple-linux-devices/) Krasue RAT - The Hacker News (https://thehackernews.com/2023/12/new-stealthy-krasue-linux-trojan.html) Automatic LLM AI Jail Break - Robust Intelligence (https://www.robustintelligence.com/blog-posts/using-ai-to-automatically-jailbreak-gpt-4-and-other-llms-in-under-a-minute) EU AI Act - Reuters (https://www.reuters.com/technology/eus-ai-act-could-exclude-open-source-models-regulation-2023-12-07/) Purple Llama - Info World (https://www.infoworld.com/article/3711284/meta-releases-open-source-tools-for-ai-safety.html) Apple Open Sources AI Tools - The Stack (https://www.thestack.technology/apple-quietly-open-sources-key-ai-tools/) Systemd 255 - The Verge (https://www.theverge.com/2023/12/7/23992512/linux-blue-screen-of-death-bsod-systemd-update) - Phoronix (https://www.phoronix.com/news/systemd-255) 24:00 Beeper Mini First impression, really cool but will only work till Apple notices Android users clearly want modern features 3 days after release, it all came to a halt Apple's FUD statement Beeper mini enabled security for non Apple users Apple's response reduces security and privacy Apple's response protects the iMessage lock-in effect Issue with other "encrypted apps" Focus of Beeper Beeper cloud uses its own cloud server Give beeper mini a review Beeper blog post (https://blog.beeper.com/p/beeper-mini-is-back) 37:45 vHost Hardware What is a vHost What does Steve consider network drives RAM CPU Lots of compute nodes vs a few large nodes Stage 1 - is it viable $1k-50k quotes Started with 2 vdevs with 3 drives Stay under 85% Stage 2 Scale up DELL EMC POWEREDGE R7425 8 BAY LFF SERVER 2x AMD EPYC 7451 H330 3 PCI RISER RPS DELL PowerEdge R6525 1U Server 2 x AMD EPYC 7542 2.9Ghz CPU 256 GB No HDD Can save a lot buying used Local vs Central storage Data centralized qcow2 on vHost 2 vdevs 2 disks per vdev Dell EMC KTN-STL3 drive shelf 15 disks in 2U Requires LSI SAS9200-8e NetApp DS4246 24 disks in 4U Requires LSI SAS9200-8e QSFP SFF-8436 Mini SAS SFF-8088 Cable Don't store Nextcloud data on OS qcow2 disk There will always be a single point of failure Change ZFS settings based on data being stored Easiest way to get a vHost up and running KVM vs "appliance OS" Bridging vs MAC vTap RAM is likely your biggest constraint Ubuntu libvirt doc (https://ubuntu.com/server/docs/virtualization-libvirt) -- The Extra Credit Section -- For links to the articles and material referenced in this week's episode check out this week's page from our podcast dashboard! This Episode's Podcast Dashboard (http://podcast.asknoahshow.com/367) Phone Systems for Ask Noah provided by Voxtelesys (http://www.voxtelesys.com/asknoah) Join us in our dedicated chatroom #GeekLab:linuxdelta.com on Matrix (https://element.linuxdelta.com/#/room/#geeklab:linuxdelta.com) -- Stay In Touch -- Find all the resources for this show on the Ask Noah Dashboard Ask Noah Dashboard (http://www.asknoahshow.com) Need more help than a radio show can offer? Altispeed provides commercial IT services and they're excited to offer you a great deal for listening to the Ask Noah Show. Call today and ask about the discount for listeners of the Ask Noah Show! Altispeed Technologies (http://www.altispeed.com/) Contact Noah live [at] asknoahshow.com -- Twitter -- Noah - Kernellinux (https://twitter.com/kernellinux) Ask Noah Show (https://twitter.com/asknoahshow) Altispeed Technologies (https://twitter.com/altispeed)
AMD kehren endlich mit neuen Threadripper (Pro) zurück in den HEDT-Markt: Bis zu 96-Zen-4-Kerne in einem Paket (Pro) und verdammt viele PCIe-Lanes. Intels 14. Generation für den Consumer-Bereich nicht nur im Vergleich langweilig, sondern einfach generell. Einer der belanglosesten Releases seit langem; eigentlich nur 200 MHz mehr, außer beim 14700K(F): Der bringt vier E-Cores mehr als der 13700K(F). Spannendes passiert in China mit dem Prozessor FTC870 von Phytium, der laut Spec-CPU2017-Benchmark (fragwürdiger Benchmark) mit AMD Epyc 7443 (Zen 3) konkurrieren möchte. Und Qualcomm möchte zusammen mit Google einen SoC auf Basis von RISC-V zur Serienreife bringen und so eine Alternative zu Arm haben. Nach dem Nvidia-Debakel und dem Börsengang von Arm sehr begrüßenswert. Wir haben diesmal viel zu Autos (yay): BMW und Mercedes schielen neidisch auf Tesla oder Nio und möchten auch Direktvertrieb, um die Preise zu kontrollieren und Händler zu umgehen. Alles zum Wohle des Kunden natürlich. Und dann kommt da Citroën mit dem neuen "Kleinwagen" Ë-C3. Preislich ganz interessant für ein E-Auto, aber "Kleinwagen"? Naja. Mike hat sich das "The Bloodline" im Early Access angesehen, ein Indie-Sandbox-RPG mit Anleihen an Elder Scrolls, Mount & Blade und Survival Games. Interessante Ansätze, viel Potential, aber noch sehr rumpelig. Das braucht noch viel Zeit und viele Patches. Zum Schluss haben wir leuchtende Petunien und Spinnen-Geschichten. Viel Spaß mit Folge 175! Sprecher: Meep, Mohammed Ali Dad, Michael KisterProduktion: Michael KisterTitelbild: Mohammed Ali DadBildquellen:Aufnahmedatum: 20.10.2023 Besucht unsim Discord https://discord.gg/SneNarVCBMauf Twitter https://twitter.com/technikquatschauf Bluesky https://bsky.app/profile/technikquatsch.bsky.socialauf Youtube https://www.youtube.com/@technikquatsch 00:00:00 Eigenpromo zum Switch-2-Special mit Mats, Unholy (Wars)https://technikquatsch.de/special-die-switch-2-technikquatsch-fantasiert-feat-special-guest-mats/, auch auf Youtube: https://www.youtube.com/watch?v=oKEslNJz5HsWitchery - Unholy Wars https://www.youtube.com/watch?v=zt9j9mgNSug 00:11:23 AMD Threadripper 7000 (Pro)https://www.anandtech.com/show/21092/amd-unveils-ryzen-threadripper-7000-family-zen-4-for-workstations-and-hedthttps://www.computerbase.de/2023-10/amd-ryzen-threadripper-pro-7000-mit-64-zen-4-kernen-als-hedt-cpu-auch-fuer-den-desktop/ 00:18:56 Intel 14. Generation: 14600K(F), 14700K(F), 14900K(F)https://www.computerbase.de/2023-10/intel-core-i9-14900k-i7-14700k-i5-14600k-test/ 00:24:34 CPU-Hersteller Phytium aus China möchte mit Zen 3 Epyc konkurrierenhttps://www.golem.de/news/phytium-ftc870-chinesische-cpu-konkurriert-mit-amd-zen-3-2310-178657.html 00:29:07 Qualcomm stellt SoC auf RISC-V-Basis für WearOS vorhttps://www.computerbase.de/2023-10/qualcomm-und-google-smartwatch-soc-mit-risc-v-cpu-fuer-wear-os-geplant/https://www.qualcomm.com/news/releases/2023/10/qualcomm-to-bring-risc-v-based-wearable-platform-to--wear-os-by- 00:36:26 BMW und Mercedes wollen auf Direktvertrieb setzenhttps://www.handelsblatt.com/unternehmen/industrie/autoindustrie-bmw-plant-direktvertrieb-und-will-die-preise-kontrollieren-/29432448.html 01:00:01 der neue "Kleinwagen" Citroën Ë-C3https://www.golem.de/news/citroen-e-c3-von-den-schwierigkeiten-einen-e-kleinwagen-zu-bauen-2310-178661.html 01:20:01 "The Bloodline" Early Access Preview: interessantes Sandbox RPG mit Potential, das noch reifen musshttps://store.steampowered.com/app/1159290/The_Bloodline/ 01:32:56 "glow in the dark" Petunien in den USA zum Verkauf zugelassenhttps://www.heise.de/news/Gruenes-Licht-Zulassung-fuer-den-Verkauf-von-leuchtenden-Petunien-9337678.html 01:41:40 CW: Spinnen 01:48:06 Terrifier auf Amazon Prime als geschnittene FSK 18https://www.amazon.de/Terrifier-David-Howard-Thornton/dp/B07KRF9XYKhttps://www.schnittberichte.com/schnittbericht.php?ID=71543 01:50:45 Ende
Welcome episode 223 of The CloudPod Podcast! It's a full house - Justin, Matt, Ryan, and Jonathan are all here this week to discuss all the cloud news you need. This week, cost optimization is the big one, with a deep dive on the newest AWS blog. Additionally, we've got updates to BigQuery, Google's Health Service, managed services for Prometheus, and more. Titles we almost went with this week:
Cisco Acquires Accedian and SamKnows https://twitter.com/willtowntech/status/1674446283858227201?s=46&t=W46jiq63kgn_RPDlW_rmJQ July Reminder: Don't Eat Dirty Data https://www.ibm.com/products/watsonx-governance AMD Epyc in Storage https://www.forbes.com/sites/moorinsights/2023/06/23/research-note-oracle-launches-exadata-x10m/?sh=26db92f56a6e https://www.globenewswire.com/news-release/2023/06/21/2692088/0/en/AMD-EPYC-Embedded-Series-Processors-Power-New-HPE-Alletra-Storage-MP-Solution.html What a Mean AI Bot Can Do https://arxiv.org/pdf/2305.15324.pdf HPE Discover Recap https://www.forbes.com/sites/patrickmoorhead/2023/06/26/hpe-further-highlights-greenlake-on-prem-cloud-through-at-discover-event/ https://www.forbes.com/sites/moorinsights/2023/06/22/hpe-extends-its-enterprise-naas-leadership-at-discover-2023/ https://twitter.com/MattKimball_MIS/status/1671289647769739264 https://twitter.com/MattKimball_MIS/status/1671218196236619776 https://twitter.com/MattKimball_MIS/status/1671295050670817280 https://twitter.com/willtowntech/status/1671874959356882944?s=46&t=W46jiq63kgn_RPDlW_rmJQ IBM's 1st Quantum Data Center in Europe https://newsroom.ibm.com/2023-06-06-IBM-to-Build-its-First-European-Quantum-Data-Center-to-Serve-Expanding-Ecosystem Infrastructure is not Commoditized https://www.forbes.com/sites/moorinsights/2023/06/30/analyst-quick-take-oracle-database-on-arm/?sh=6c3d50e26139 https://aibusiness.com/verticals/dell-hpe-lenovo-and-nvidia-tout-ai-hardware-solutions-to-enterprises- IonQ's Swiss Quantum Data Center https://investors.ionq.com/news/news-details/2023/IonQ-and-QuantumBasel-Partner-to-Achieve-Future-Quantum-Advantages-With-Deployment-of-Two-Generations-of-IonQ-Quantum-Systems-in-Europe/default.aspx Disclaimer: This show is for information and entertainment purposes only. While we will discuss publicly traded companies on this show. The contents of this show should not be taken as investment advice.
In this episode of What's Next, Tony Bartlett — Dell Technologies Director of Data Centre Compute for the SADC Region — meets with Aki Anastasiou to discuss Dell Technologies' new PowerEdge servers. Bartlett has worked at Dell Technologies for over 16 years and has a comprehensive knowledge of the brand's solutions. Prior to joining Dell Technologies, Bartlett worked at Tiger Brands, where he held positions such as Group Technology Architect and Divisional IT Manager. Across his nearly 30 years of industry experience, Bartlett has cultivated a deep passion for the data centre sector and how it is constantly changing to keep up with new demands. This has led to him being an active member of the local technology community, frequently taking part in industry events and spearheading thought leadership initiatives. In this episode of What's Next, Tony Bartlett discusses why Dell Technologies chose to power its new PowerEdge servers with 4th-generation AMD EPYC processors.
This is a recap of the top 10 posts on Hacker News on .This podcast was generated by Wondercraft: https://www.wondercraft.ai/?utm_source=hackernews_recap Please ping at team AT wondercraft.ai with feedback.(00:37): Try: run a command and inspect its effects before changing your live systemOriginal post: https://news.ycombinator.com/item?id=36461102&utm_source=wondercraft_ai(02:10): Why doesn't TypeScript properly type Object.keys?Original post: https://news.ycombinator.com/item?id=36457557&utm_source=wondercraft_ai(03:26): $900k Median Package for Engineers at OpenAIOriginal post: https://news.ycombinator.com/item?id=36460082&utm_source=wondercraft_ai(04:44): US vendor accused of violating GDPR by reputation-scoring EU citizensOriginal post: https://news.ycombinator.com/item?id=36460243&utm_source=wondercraft_ai(06:12): Every Mastodon user has an RSS feedOriginal post: https://news.ycombinator.com/item?id=36461701&utm_source=wondercraft_ai(07:42): Take action, Protect end-to-end encryptionOriginal post: https://news.ycombinator.com/item?id=36459055&utm_source=wondercraft_ai(09:13): Common bugs in writing (2021)Original post: https://news.ycombinator.com/item?id=36457051&utm_source=wondercraft_ai(10:36): Building a new ZX Spectrum from all new partsOriginal post: https://news.ycombinator.com/item?id=36458849&utm_source=wondercraft_ai(11:56): North Korean market prices suggest serious food shortagesOriginal post: https://news.ycombinator.com/item?id=36459517&utm_source=wondercraft_ai(13:19): AMD EPYC 97x4 “Bergamo” CPUs: 128 Zen 4c CPU Cores for Servers, Shipping NowOriginal post: https://news.ycombinator.com/item?id=36458678&utm_source=wondercraft_aiThis is a third-party project, independent from HN and YC. Text and audio generated using AI, by wondercraft.ai. Create your own studio quality podcast with text as the only input in seconds at app.wondercraft.ai. Issues or feedback? We'd love to hear from you: team@wondercraft.ai
It was another week, and that means more pcper podcast goodness. Josh had his very, very old headset and a bad laptop sound card on the road with him, Kent was back again, with a fab case review and we talked about a lot of other tech and newsy stuff. We even had Clippy.00:00 Prologue and Intro02:38 Burger of the Week04:26 RTX 4060 (non-Ti) release date moves up09:40 AMD's EPYC Bargamo CPUs14:07 AMD ups their AI game (and more AI discussion)24:31 Microsoft offering official Surface parts27:31 Josh interrupts to talk about AMD EPYC some more28:57 Are larger cards the key to more VRAM? (satire) 30:15 Moving 12VHPWR to the back of the GPU32:42 Kent uses a 4090 as wall art34:00 Just in time, PCI-SIG is working on PCI Express 7.037:11 Mandatory Arc coverage40:32 Desktop GPU sales lowest in decades (and much rambling)49:27 Clippy shame (and Kent's LGR story)53:07 Security Corner1:03:57 Gaming Quick Hits1:09:39 Fractal Terra corrections1:12:39 Thermaltake Ceres 500 TG ARGB case review1:26:35 Picks of the Week1:42:43 Outro ★ Support this podcast on Patreon ★
En busca de garantizar eficiencia, disponibilidad y resiliencia en las redes, en su última propuesta, AMD EPYC ofreció soluciones innovadoras junto con protección de datos y rendimiento óptimo, utilizando múltiples redes en convivencia, expansión del ecosistema de dispositivos, seguridad avanzada y conectividad sostenible.
Data centers are crucial to the operations of businesses and individuals in today's world. The hardware in these installations is responsible for many aspects of everyday life and global commerce. But like every device, each piece of technology in a DC needs resources to build, run, and often, cool.To commemorate Earth Month, in this episode of the Tech Means Business podcast, we talk to two major players in the data center equipment space, AMD and Dell Technologies. Can these two global players change the ways that data centers operate so they run cooler, faster, and greener while ensuring the best performance and ROI?In the APJ region, the race for cutting-edge technology is particularly strong: HPC (high-performance computing), AI (artificial intelligence), and ML (machine learning) are increasingly becoming table stakes in competitive markets. Listen in to hear how Dell PowerEdge servers, powered by the latest generations of AMD's microprocessors are producing tangible business results while reducing carbon emissions.Today's guests are Peter Chambers from AMD and Chris Kelly of Dell Technologies.To learn more about the green features and specifications of the next-generation Dell PowerEdge servers:https://www.dell.com/en-sg/dt/corporate/newsroom/announcements/detailpage.press-releases~usa~2022~11~20221110-next-generation-dell-poweredge-servers-dramatically-improve-performance-for-more-sustainable-data-centers.htm#/filter-on/Country:en-sgAMD's ecological stance and contributions to lower power consumption are here:https://www.amd.com/en/corporate-responsibility/environmental-sustainabilitySign up for a bespoke workshop with Dell Technologies to plan your business goals and help accelerate your digital future:https://www.dell.com/en-sg/dt/what-we-do/customer-engagement-programs/customer-solution-centers.htmMore about Dell Technologies World 2023https://www.dell.com/en-sg/dt/events/delltechnologiesworld/2023/index.htmChris Kelly, Senior Vice President, Data Center Solutions at Dell Technologies, Asia Pacific and Japan, can be found on LinkedIn:https://www.linkedin.com/in/chris-kelly-7a0b825/AMD's Peter Chambers, Managing Director – Sales, Asia Pacific & Japan, is here:https://www.linkedin.com/in/peter-chambers-59bb819/And liquid-cooled Joe Green is here:https://www.linkedin.com/in/josephedwardgreen/
Destacados por reducir la huella de carbono, los procesadores de cuarta generación AMD EPYC, fueron presentados por la Dra. Lisa Su, Presidente y Directora Ejecutiva de AMD durante el evento “Together We Advance Data Centers”.
Der 96-Kern-Prozessor Epyc 9004 übertrumpft sämtliche Intel-Xeons und bringt Neuerungen wie CXL. Eine Einordnung im Audio-Podcast Bit-Rauschen, Folge 2022/24.
Welcome to Episode 127 Announcements Patreon Update name_pending197 Jeremy Arinomi Andrew Tatro Bruce Robert David S0l3mn LiNuXsys666 Mark The Mentor Marc Julius Andi J Charles American Cancer Society Funds are up to $224.48 Patrons: Get your Iron Sysadmin Merch at Teespring! https://teespring.com/stores/ironsysadmin Support the Iron Sysadmin Podcast AND try out Riverside.fm by using this link: https://riverside.fm/?utm_campaign=campaign_1&utm_medium=affiliate&utm_source=rewardful&via=ironsysadmin DC610 pubcrawl coming up 11/19! Chat [Nate] From ebay, It's an HP Proliant It's like a decade old, but has 24 xeon cores, and 96gb of memory. I paid under $180 including shipping. https://labgopher.com I bought a new old server… 3d printer is acting up a bit. Started a new hacky project on the jeep. Or, at least started planning it and getting parts together anyway. Getting a donkey (must be rideable) [XenoPhage] Want a “server” board… 64G-128G max, start w/ 32G .. Maybe dual CPU? Thinking maybe the AMD Epyc? 6+ SATA https://store.steampowered.com/steamdeckdock Starting to think about building a new server Got a new toy for the Steam Deck… Honey-do list++ News https://www.theregister.com/2022/10/12/drone-roof-attack/ https://www.youtube.com/watch?v=Cdk4Zw2oYdc https://arstechnica.com/gaming/2022/10/carmack-wants-a-250-vr-headset-to-counterpoint-the-1499-quest-pro/ https://www.youtube.com/watch?v=ouq5yyzSiAw https://www.wired.com/story/us-chip-sanctions-kneecap-chinas-tech-industry/ https://www.theverge.com/2022/10/13/23402171/firefox-relay-phone-masking-service-privacy-security https://cybernews.com/privacy/dji-drone-tracking-data-exposed-in-us/ Watch us live on the 2nd and 4th Thursday of every month! Subscribe and hit the bell! https://www.youtube.com/IronSysadminPodcast OR https://twitch.tv/IronSysadminPodcast Discord Community: https://discord.gg/wmxvQ4c2H6 Matrix Space: https://matrix.to/#/#IronSysadmin:trixie.undrground.org Find us on Twitter, and Facebook! https://www.facebook.com/ironsysadmin https://www.twitter.com/ironsysadmin Subscribe wherever you find podcasts! And don't forget about our patreon! https://patreon.com/ironsysadmin Intro and Outro music credit: Tri Tachyon, Digital MK 2http://freemusicarchive.org/music/Tri-Tachyon/
Salve, jovens! No programa de hoje, vamos falar sobre como podcasts sobre crimes reais estão conseguindo mudar os rumos de algumas decisões judiciais ao redor do mundo. Ainda teremos NASA e Hideo Kojima juntos, o Instagram querendo acabar com os nudes não solicitados, a alta cúpula da Amazon enquadrada e a onda de demissões chegando à Meta. Publi: Conheça os processadores de servidor x86 de melhor desempenho do mercado e eleve sua produtividade nos negócios com AMD EPYC™.
Full Description / Show Notes Corey and Linda talk about Tiktok and the online developer community (1:18) Linda talks about what prompted her to want to work at AWS (5:29) Linda discusses navigating the change from just being part of the developer community to being an employee of AWS (10:37) Linda talks about moving AWS more in the direction of short form content, and Corey and Linda talk about the Tiktok algorithm (15:56) Linda talks about the potential struggle of going from short form to long form content (25:21) About LindaLinda Vivah is a Site Reliability Engineer for a major media organization in NYC, a tech content creator, an AWS community builder member, a part-time wedding singer, and the founder of a STEM jewelry shop called Coding Crystals. At the time of this recording she was about to join AWS in her current position as a Developer Advocate.Linda had an untraditional journey into tech. She was a Philosophy major in college and began her career in journalism. In 2015, she quit her tv job to attend The Flatiron School, a full stack web development immersive program in NYC. She worked as a full-stack developer building web applications for 5 years before shifting into SRE to work on the cloud end internally.Throughout the years, she's created tech content on platforms like TikTok & Instagram and believes that sometimes the best way to learn is to teach.Links Referenced:lindavivah.com: https://lindavivah.com TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: This episode is sponsored in part by Honeycomb. When production is running slow, it's hard to know where problems originate. Is it your application code, users, or the underlying systems? I've got five bucks on DNS, personally. Why scroll through endless dashboards while dealing with alert floods, going from tool to tool to tool that you employ, guessing at which puzzle pieces matter? Context switching and tool sprawl are slowly killing both your team and your business. You should care more about one of those than the other; which one is up to you. Drop the separate pillars and enter a world of getting one unified understanding of the one thing driving your business: production. With Honeycomb, you guess less and know more. Try it for free at honeycomb.io/screaminginthecloud. Observability: it's more than just hipster monitoring.Corey: Let's face it, on-call firefighting at 2am is stressful! So there's good news and there's bad news. The bad news is that you probably can't prevent incidents from happening, but the good news is that incident.io makes incidents less stressful and a lot more valuable. incident.io is a Slack-native incident management platform that allows you to automate incident processes, focus on fixing the issues and learn from incident insights to improve site reliability and fix your vulnerabilities. Try incident.io, recover faster and sleep more.Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. We talk a lot about how people go about getting into this ridiculous industry of ours, and I've talked a little bit about how I go about finding interesting and varied guests to show up and help me indulge my ongoing love affair on this show with the sound of my own voice. Today, we're going to be able to address both of those because today I'm speaking to Linda Haviv, who, as of this recording, has accepted a job as a Developer Advocate at AWS, but has not started. Linda, welcome to the show.Linda: Thank you so much for having me, Corey. Happy to be here.Corey: So, you and I have been talking for a while and there's been a lot of interesting things I learned along the way. You were one of the first people I encountered when I joined the TikToks, as all the kids do these days, and was trying to figure out is there a community of folks who use AWS. Which really boils down to, “So, where are these people that are sad all the time?” Well, it turns out, they're on TikTok, so there we go. We found my people.And that was great. And we started talking, and it turns out that we were both in the AWS community builder program. And we've developed a bit of a rapport. We talk about different things. And then, I guess, weird stuff started happening, in the context of you were—you're doing very well at building an audience for yourself on TikTok.I tried it, and it was—my sense of humor sometimes works, sometimes doesn't. I've had challenges in finding any reasonable way to monetize it because a 30-second video doesn't really give nuance for a full ad read, for example. And you've been looking at it from the perspective of a content creator looking to build the audience slash platform is step one, and then, eh, step two, you'll sort of figure out aspects of monetization later. Which, honestly, is a way easier way to do it in hindsight, but, yeah, the things that we learn. Now, that you're going to AWS, first, you planning to still be on the TikToks and whatnot?Linda: Absolutely. So, I really look at TikTok as a funnel. I don't think it's the main place, you're going to get that deep-dive content but I think it's a great way, especially for things that excite you or get you into understanding it, especially beginner-type audience, I think there's a lot of untapped market of people looking to into tech, or technologists that aren't in the cloud. I mean, even when I worked—I worked as a web developer and then kind of learned more about the cloud, and I started out as a front-end developer and shifted into, like, SRE and infrastructure, so even for people within tech, you can have a huge tech community which there is on TikTok, with a younger community—but not all of them really understand the cloud necessarily, depending on their job function. So, I think it's a great way to kind of expose people to that.For me, my exposure came from community. I met somebody at a meetup who was working in cloud, and it wasn't even on the job that I really started getting into cloud because many times in corporations, you might be working on a specific team and you're not really encountering other ends, and it seems kind of like a mystery. Although it shouldn't seem like magic, many times when you're doing certain job functions—especially the DevOps—could end up feeling like magic. So, [laugh] for the good and the bad. So sometimes, if you're not working on that end, you really sometimes take it for granted.And so, for me, I actually—meetups were the way I got exposed to that end. And then I brought it back into my work and shifted internally and did certifications and started, even, lunch-and-learns where I work to get more people in their learning journey together within the company, and you know, help us as we're migrating to the cloud, as we're building on the cloud. Which, of course, we have many more roles down the road. I did it for a few years and saw the shift. But I worked at a media company for many years and now shifting to AWS, and so I've seen that happen on different ends.Not—oh, I wasn't the one doing the migration because I was on the other end of that time, but now for the last two years, I was working on [laugh] the infrastructure end, and so it's really fascinating. And many people actually—until now I feel like—that will work on maybe the web and mobile and don't always know as much about the cloud. I think it's a great way to funnel things in a quick manner. I think also society is getting used to short videos, and our attention span is very low, and I think for—Corey: No argument here.Linda: —[crosstalk 00:04:39] spending so mu—yeah, and we're spending so much time on these platforms, we might as well, you know, learn something. And I think it depends what content. Some things work well, some things doesn't. As with anything content creation, you kind of have to do trial and error, but I do find the audience to be a bit different on TikTok versus Twitter versus Instagram versus YouTube. Which is interesting how it's going to play out on YouTube, too, which is a whole ‘nother topic conversation.Corey: Well, it's odd to me watching your path. It's almost the exact opposite of mine where I started off on the back-end, grumpy sysadmin world and, “Oh, why would I ever need to learn JavaScript?” “Well, genius, because as the world progresses, guess what? That's right. The entire world becomes JavaScript. Welcome.”And it took me a long time to come around to that. You started with the front-end world and then basically approached from the exact opposite end. Let's be clear, back in my day, mine was the common path. These days, yours is very much the common path.Linda: Yeah.Corey: I also want to highlight that all of those transitions and careers that you spoke about, you were at the same company for nine years, which in tech is closer to 30. So, I have to ask, what was it that inspired you, after nine years, to decide, “I'm going to go work somewhere else. But not just anywhere; I'm going to AWS.” Because normally people don't almost institutionalized lifers past a certain point.Linda: [laugh].Corey: Like, “Oh, you'll be there till you retire or die.” Whereas seeing significant career change after that long in one place, even if you've moved around internally and experienced a lot of different roles, is not common at all what sparked that?Linda: Yeah. Yeah, no, it's such a good question. I always think about that, too, especially as I was reflecting because I'm, you know, in the midst of this transition, and I've gotten a lot of reflecting over the last two weeks [laugh], or more. But I think the main thing for me is, I always, wherever I was—and this kind of something that—I'm very proactive when it comes to trying to transition. I think, even when I was—right, I held many roles in the same company; I used to work in TV production and actually left for three months to go to a coding boot camp and then came back on the other end, but I understood the product in a different way.So, for that time period, it was really interesting to work on the other end. But, you know, as I kind of—every time I wanted to progress further, I always made a move that was actually new and put me in an uncomfortable place, even within the same company. And I'm at the point now that I'm in my career, I felt like this next step really needs to be, you know, at AWS. It's not, like, the natural progression for me. I worked alongside—on the client end—with AWS and have seen so many projects come through and how much our own workloads have changed.And it's just been an incredible journey, also dealing with accounts team. On that end, I've worked alongside them, so for me, it was kind of a natural progression. I was very passionate about cloud computing at AWS and I kind of wanted to take it to that next place, and I felt like—also, dealing with the community as part of my job is a dream part to me because I was always doing that on the side on social media. So, it wasn't part of my day-to-day job. I was working as an SRE and an infrastructure engineer, so I didn't get to do that as part of my day-to-day.I was making videos at 2 a.m. and, you know, kind of trying to, like, do—you know, interact with the community like that. And I think—I come from a performing background, the people background, I was singing since I was four years old. I always go to—I was a wedding singer, so I go into a room and I love making people happy or giving value. And I think, like, education has a huge part of that. And in a way, like making that content and—Corey: You got to get people's attention—Linda: Yeah.Corey: —you can't teach them a damn thing.Linda: Right. Exactly. So, it's kind of a mix of everything. It's like that performance, the love of learning. You know, between you and I, like, I wanted to be a lawyer before I thought I was going to—before I went to tech.I thought I was going to be a lawyer purely because I loved the concept of going to law school. I never took time to think about the law part, like, being the lawyer part. I always thought, “Oh, school.” I'm a student at heart. I always call myself a professional student. I really think that's part of what you need to be in this world, in this tech industry, and I think for me, that's what keeps my fire going.I love to experiment, to learn, to build. And there's something very fulfilling about building products. If you take a step back, like, you're kind of—you know, for me that part, every time I look back at that, that always is what kind of keeps me going. When I was doing front-end, it felt a lot more like I was doing smaller things than when I was doing infrastructure, so I felt like that was another reason why I shifted. I love doing the front-end, but I felt like I was spending two days on an Internet Explorer bug and it just drove me—[laugh] it just made it feel unfulfilling versus spending two days on, you know, trying to understand why, you know, something doesn't run the infrastructure or, like, there's—you know, it's failing blindly, you know? Stuff like that. Like, I don't know, for me that felt more fulfilling because the problem was more macro. But I think I needed both. I have a love for both, but I definitely prefer being back-end. So. [laugh]. Well, I'm saying that now but—[laugh].Corey: This might be a weakness on my part where I'm basically projecting onto others, and this is—I might be completely wrong on this, but I tend to take a bit of a bifurcated view of community. I mean, community is part of the reason that I know the things I know and how I got to this place that I am, so use that as a cautionary tale if you want. But when I talk to someone like you at this moment, where you're in the community, I'm in the community, and I'm talking to you about a problem I'm having and we're working on ways to potentially solve that or how to think about that. I view us as basically commiserating on these things, whereas as soon as you start on day one—and yes, it's always day one—at AWS and this becomes your day job and you work there, on some level, for me, there's a bit shift that happens and a switch gets flipped in my head where, oh, you actually work at this company. That means you're the problem.And I'm not saying that in a way of being antagonistic. Please, if you're watching or listening to this, do not antagonize the developer advocates. They have a very hard job understanding all this so they can explain that to the rest of us. But how do you wind up planning to navigate, or I guess your views on, I guess, handling the shift between, “One of the customers like the rest of us,” to, as I say, “Part of the problem,” for lack of a better term.Linda: Or, like, work because you kind of get the—you know. I love this question and it's something I've been pondering a lot on because I think the messaging will need to be a little different [coming from me 00:10:44] in the sense of, there needs to be—just in anything, you have to kind of create trust. And to create trust, you have to be vulnerable and authentic. And I think I, for example, utilize a lot of things outside of just the AWS cloud topic to do that now, even, when I—you know, kind of building it without saying where I work or anything like that, going into this role and it being my job, it's going to be different kind of challenge as far as the messaging, but I think it still holds true that part, that just developing trust and authenticity, I might have to do more of that, you know? I might have to really share more of that part, share other things to really—because it's more like people come, it doesn't matter how much somet—how many times you explain it, many times, they will see your title and they will judge you for it, and they don't know what happened before. Every TikTok, for example, you have to act like it's a new person watching. There is no series, you know? Like, yes, there's a series but, like, sometimes you can make that but it's not really the way TikTok functions or a short-form video functions. So, you kind of have to think this is my first time—Corey: It works really terribly when you're trying to break it out that way on TikTok.Linda: [laugh]. Yeah.Corey: Right. Here's part 17 of my 80-TikTok-video saga. And it's, “Could you just turn this into a blog post or put this on YouTube or something? I don't have four hours to spend learning how all this stuff works in your world.”Linda: Yeah. And you know, I think repeating certain things, too, is really important. So, they say you have to repeat something eight times for people to see it or [laugh] something like that. I learned that in media [crosstalk 00:12:13]—Corey: In a row, or—yeah. [laugh].Linda: I mean, the truth is that when you, kind of like, do a TikTok maybe, like, there's something you could also say or clarify because I think there's going to be—and I'm going to have to—there's going to be a lot of trial and error for me; I don't know if I have answers—but my plan is going into it very much testing that kind of introduction, or, like, clarifying what that role is. Because the truth is, the role is advocating on behalf of the community and really helping that community, so making sure that—you don't have to say it as far as a definition maybe, but, like, making sure that comes across when you create a video. And I think that's going to be really important for me, and more important than the prior even creating content going forward. So, I think that's one thing that I definitely feel like is key.As well as creating more raw interaction. So, it depends on the platform, too. Instagram, for example, is much more community—how do I put this? Instagram is much more easy to navigate as far as reaching the same community because you have something, like, called Instagram Stories, right? So, on Instagram Stories, you're bringing those stories, mostly the same people that follow you. You're able to build that trust through those stories.On TikTok, they just released Stories. I haven't really tried them much and I don't play with it a lot, but I think that's something I will utilize because those are the people that are already follow you, meaning they have seen a piece of content. So, I think addressing it differently and knowing who's watching what and trying to kind of put yourself in their shoes when you're trying to, you know, teach something, it's important for you to have that trust with them. And I think—key to everything—being raw and authentic. I think people see through that. I would hope they do.And I think, uh, [laugh] that's what I'm going to be trying to do. I'm just going to be really myself and real, and try to help people and I hope that comes through because that's—I'm passionate about getting more people into the cloud and getting them educated. And I feel like it's something that could also allow you to build anything, just from anywhere on your computer, brings people together, the world is getting smaller, really. And just being able to meet people through that and there's just a way to also change your life. And people really could change their life.I changed my life, I think, going into tech and I'm in the United States and I, you know—I'm in New York, you know, but I feel like so many people in the States and outside of the States, you know, all over the world, you know, have access to this, and it's powerful to be able to build something and contribute and be a part of the future of technology, which AWS is.Corey: I feel like, in three years or whatever it is that you leave AWS in the far future, we're going to basically pull this video up and MST3k came together. It's like, “Remember how naive you were talking about these things?” And I'm mostly kidding, but let's be serious. You are presumably going to be focusing on the idea of short-form content. That is—Linda: Yeah.Corey: What your bread-and-butter of audience-building has been around, and that is something that is new for AWS.Linda: Yeah.Corey: And I'm always curious as to how companies and their cultures continue to evolve. I can only imagine there's a lot of support structure in place for that. I personally remember giving a talk at an AWS event and I had my slides reviewed by their legal team, as they always do, and I had a slide that they were looking at very closely where I was listing out the top five AWS services that are bullshit. And they don't really have a framework for that, so instead, they did their typical thing of, “Okay, we need to make sure that each of those services starts with the appropriate AWS or Amazon naming convention and are they capitalized properly?” Because they have a framework for working on those things.I'm really curious as to how the AWS culture and way of bringing messaging to where people are is going to be forced to evolve now that they, like it or not, are going to be having significantly increased presence on TikTok and other short-form platforms.Linda: I mean, it's really going to be interesting to see how this plays out. There's so much content that's put out, but sometimes it's just not reaching the right audience, so making sure that funnel exists to the right people is important and reaching those audiences. So, I think even YouTube Shorts, for example. Many people in tech use YouTube to search a question.They do not care about the intro, sometimes. It depends what kind of following, it depends if [in gaming 00:16:30], but if you're coming and you're building something, it's like a Stack Overflow sometimes. You want to know the answer to your question. Now, YouTube Shorts is a great solution to that because many times people want the shortest possible answer. Now, of course, if it's a tutorial on how to build something, and it warrants ten minutes, that's great.Even ten minutes is considered, now, Shorts because TikTok now has ten-minute videos, but I think TikTok is now searchable in the way YouTube is, and I think let's say YouTube Shorts is short-form, but very different type of short-form than TikTok is. TikTok, hooks matter. YouTube answers to your questions, especially in chat. I wouldn't say everything in YouTube is like that; depends on the niche. But I think even within short-form, there's going to be a different strategy regarding that.So, kind of like having that mix. I guess, depending on platform and audience, that's there. Again, trial and error, but we'll see how this plays out and how this will evolve. Corey: This episode is sponsored in part by our friends at Vultr. Optimized cloud compute plans have landed at Vultr to deliver lightning-fast processing power, courtesy of third-gen AMD EPYC processors without the IO or hardware limitations of a traditional multi-tenant cloud server. Starting at just 28 bucks a month, users can deploy general-purpose, CPU, memory, or storage optimized cloud instances in more than 20 locations across five continents. Without looking, I know that once again, Antarctica has gotten the short end of the stick. Launch your Vultr optimized compute instance in 60 seconds or less on your choice of included operating systems, or bring your own. It's time to ditch convoluted and unpredictable giant tech company billing practices and say goodbye to noisy neighbors and egregious egress forever. Vultr delivers the power of the cloud with none of the bloat. Screaming in the Cloud listeners can try Vultr for free today with a $150 in credit when they visit getvultr.com/screaming. That's G-E-T-V-U-L-T-R dot com slash screaming. My thanks to them for sponsoring this ridiculous podcast.Corey: I feel like there are two possible outcomes here. One is that AWS—Linda: Yeah.Corey: Nails this pivot into short-form content, and the other is that all your TikTok videos start becoming ten minutes long, which they now support, welcome to my TED Talk. It's awful, and then you wind up basically being video equivalent for all of your content, of recipes when you search them on the internet where first they circle the point to death 18 times with, “Back when I was a small child growing up in the hinterlands, we wound—my grandmother would always make the following stew after she killed the bison with here bare hands. Why did grandma kill a bison? We don't know.” And it just leads down this path so they can get, like, long enough content or they can have longer and longer articles to display more ads.And then finally at the end, it's like ingredient one: butter. Ingredient two, there is no ingredient two. Okay. That explains why it's delicious. Awesome. But I don't like having people prolong it. It's just, give me the answer I'm looking for.Linda: Yeah.Corey: Get to the point. Tell me the story. And—Linda: And this is—Corey: —I'm really hoping that is not the direction your content goes in. Which I don't think it would, but that is the horrifying thing and if for some chance I'm right, I will look like Nostradamus when we do that MST3k episode.Linda: No, no. I mean, I really am—I always personally—even when I was creating content these last few years and testing different things, I'm really a fan of the shortest way possible because I don't have the patience to watch long videos. And maybe it's because I'm a New Yorker that can't sit down from the life of me—apart from when I code of course—but, you know, I don't like wasting time, I'm always on the go, I'm with my coffee, I'm like—that's the kind of style I prefer to bring in videos in the sense of, like, people have no time. [laugh]. You know?The amount of content we're consuming is just, uh, bonkers. So, I don't think our mind is really a built for consuming [laugh] this much content every time you open your phone, or every time you look, you know, online. It's definitely something that is challenging in a whole different way. But I think where my content—if it's ten minutes, it better be because I can't shorten it. That's my thing. So, you can hold me accountable to that because—Corey: Yeah, I want ten minutes of—Linda: I'm not a—Corey: Content, not three minutes of content in a ten-minute bag.Linda: Exactly. Exactly. So, if it's a ten-minute video, it would have been in one hour that I cut down, like, meaning a tutorial, a very much technical types of content. I think things that are that long, especially in tech, would be something like, on that end—unless, of course, you know, I'm not talking about, like, longer videos on YouTube which are panels or that kind of thing. I'm talking more like if I'm doing something on TikTok specifically.TikTok also cares about your watch time, so if people aren't interested in it, it's not going to do well, it doesn't matter how many followers you have. Which is what I do like about the way TikTok functions as opposed to, let's say, Instagram. Instagram is more like it gives it to your following—and this is the current state, I don't know if it always evolves—but the current state is, Instagram Reels kind of functions in a way where it goes first to the people that follow you, but, like, in a way that's more amplified than TikTok. TikTox tests people that follows you, but if it's not a good video, it won't do well. And honestly, they're many good videos videos that don't go viral. I'm not talking about that.Sometimes it's also the topic and the niche and the sound and the title. I mean, there's so many people who take a topic and do it in three different ways and one of them goes viral. I mean, there's so many factors that play into it and it's hard to really, like, always, you know, kind of reverse engineer but I do think that with TikTok, things won't do well, more likely if it's not a good piece of content as opposed to—or, like, too long, right? Not—I shouldn't say not good a good piece of content—it's too long.Corey: The TikTok algorithm is inscrutable to me. TikTok is firmly convinced, based upon what it shows me, that I am apparently a lesbian. Which okay, fine. Awesome. Whatever. I'm also—it keeps showing me ads for ADHD stuff, and it was like, “Wow, like, how did it know that?” Followed by, “Oh, right. I'm on TikTok. Nevermind.”And I will say at one point, it recommended someone to me who, looking at the profile picture, she's my nanny. And it's, I have a strong policy of not, you know, stalking my household employees on social media. We are not Facebook friends, we are not—in a bunch of different areas. Like, how on earth would they have figured this out? I'm filling the corkboard with conspiracy and twine followed by, “Wait a minute. We probably both connect from the same WiFi network, which looks like the same IP address and it probably doesn't require a giant data science team to put two and two together on those things.” So, it was great. I was all set to do the tinfoil hat conspiracy, but no, no, that's just very basic correlation 101.Linda: And also, this is why I don't enable contacts on TikTok. You know, how it says, “Oh, connect your contacts?”Corey: Oh, I never do that. Like, “Can we look at your contacts?”Linda: Never.Corey: “No.” “Can we look at all of your photos?” “Absolutely not.” “Can we track you across apps?” “Why would anyone say yes to this? You're going to do it anyway, but I'll say no.” Yeah.Linda: Got to give the least privilege. [laugh]. Definitely not—Corey: Oh absolutely.Linda: Yeah. I think they also help [crosstalk 00:22:40]—Corey: But when I'm looking at—the monetization problem is always a challenge on things like this, too, because when I'm—my guilty TikTok scrolling pleasures hit, it's basically late at night, I just want to see—I want something to want to wind down and decompress. And I'm not about ready to watch, “Hey, would you like to migrate your enterprise database to this other thing?” It's, I… no. There's a reason that the ads that seem to be everywhere and doing well are aimed at the mass market, they're generally impulse buys, like, “Hey, do you want to set that thing over there on fire, but you're not close enough to get the job done? But this flame thrower today. Done.”And great, like, that is something everyone can enjoy, but these nuanced database products and anything else is B2B SaaS style stuff, it feels like it's a very tough sell and no one has quite cracked that nut, yet.Linda: Yeah, and I think the key there—this is, I'm guessing based on, like, what I want to try out a lot—is the hook and the way you're presenting it has to be very product-focused in the sense that it needs to be very relatable. Even if you don't know anything about tech, you need to be—like, for example, in the architecture page on AWS, there's a video about the Emirates going to Mars mission. Space is a very interesting topic, right? I think, a hook, like, “Do want to see how, like, how this is bu—” like, it's all, like, freely available to see exactly [laugh] how this was built. Like, it might—in the right wording, of course—it might be interesting to someone who's looking for fun-fact-style content.Now, is it really addressing the people that are building everyday? Not really always, depends who's on there and the mass market there. But I feel like going on the product and the things that are mass-market, and then working backwards to the tech part of it, even if they learn something and then want to learn more, that's really where I see TikTok. I don't think every platform would be, maybe, like this, but that's where I see getting people: kind of inviting them in to learn more, but making it cool and fun. It's very important, but it feels cool and fun. [laugh]. So.Because you're right, you're scrolling at 2 a.m. who wants to start seeing that. Like, it's all about how you teach. The content is there, the content has—you know, that's my thing. It's like, the content is there. You don't need to—it's yes, there's the part where things are always evolving and you need to keep track of that; that's whole ‘nother type thing which you do very well, right?And then there's a part where, like, the content that already exists, which part is evergreen? Meaning, which part is, like, something that could be re—also is not timely as far as update, for example, well-architected framework. Yes, it evolves all the time, you always have new pillars, but the guide, the story, that is an evergreen in some sense because that guide doesn't, you know, that whole concept isn't going anywhere. So, you know, why should someone care about that?Corey: Right. How to turn on two-factor authentication for your AWS account.Linda: Right.Corey: That's evergreen. That's the sort of thing that—and this is the problem, I think, AWS has had for a long time where they're talking about new features, new enhancements, new releases. But you look what people are actually doing and so much of it is just the same stuff again and again because yeah, that is how most of the cloud works. It turns out that three-quarters of company's production infrastructures tends to run on EC2 more frequently than it tends to run on IoT Greengrass. Imagine that.So, there's this idea of continuing to focus on these things. Now, one of my predictions is that you're going to have a lot of fun with this and on some level, it's going to really work for you. In others, it's going to be hilariously—well, its shortcomings might be predictable. I can just picture now you're at re:Invent; you have a breakout talk and terrific. And you've successfully gotten your talk down to one minute and then you're sitting there with—Linda: [laugh].Corey: —the remainder of maybe 59. Like, oh, right. Yeah. Turns out not everything is short-form. Are you predicting any—Linda: Yep.Corey: Problems going from short-form to long-form in those instances?Linda: I think it needs to go hand-in-hand, to be honest. I think when you're creating any short-form content, you have—you know, maybe something short is actually sometimes in some ways, right, harder because you really have to make sure, especially in a technical standpoint, leaving things out is sometimes—leaves, like, a blind spot. And so, making sure you're kind of—whatever you're educating, you kind of, to be clear, “Here's where you learn more. Here's how I'm going to answer this next question for you: go here.” Now, in a longer-form content, you would cover all that.So, there's always that longevity. I think even when I write a script, and there's many scripts I'm still [laugh] I've had many ideas until now I've been doing this still at 2 a.m. so of course, there's many that didn't, you know, get released, but those are the things that are more time consuming to create because you're taking something that's an hour-long, and trying to make sure you're pulling out the things that are most—that are hook-style, that invite people in, that are accurate, okay, that really give you—explain to you clearly where are the blind spots that I'm not explaining on this video are. So, “XYZ here is, like, the high level, but by the way, there's, like, this and this.” And in a long-form, you kind of have to know the long-form version of it to make the short-form, in some ways, depending on what—you're doing because you're funneling them to somewhere. That's my thing. Because I don't think there should be [crosstalk 00:27:36]—Corey: This is the curse of Twitter, on some level. It's, “Well, you forgot about this corner case.” “Yeah, I had 280 characters to get into.” Like, the whole point of short-form content—which I do consider Twitter to be—is a glimpse and a hook, and get people interested enough to go somewhere and learn more.For something like AWS, this makes a lot of sense. When you highlight a capability or something interesting, it's something relevant, whereas on the other side of it, where it's this, “Oh, great. Now, here's an 8000-word blog post on how I did this thing.” Yeah, I'm going to get relatively fewer amounts of traffic through that giant thing, but the people who are they're going to be frickin' invested because that's going to be a slog.Linda: Exactly.Corey: “And now my eight-hour video on how exactly I built this thing with TypeScript.” Badly—Linda: Exactly.Corey: —as it turns out because I'm a bad programmer.Linda: [laugh]. No, you're not. I love your shit-posting. It's great.Corey: Challenge accepted.Linda: [laugh]. I love what you just mentioned because I think you're hitting the nail on the head when it comes to the quality content that's niche focus, like, there needs to be a good healthy mix. I think always doing that, like, mass-market type video, it doesn't give you, also, the credibility you need. So, doing those more niche things that might not be relevant to everybody, but here and there, are part of that is really key for your own knowledge and for, like, the com—you know, as far as, like, helping someone specific. Because it's almost like—right, when you're selling a service and you're using social media, right, not everybody's going to buy your service. It doesn't matter what business you're in right? The deep-divers are going to be the people that pay up. It's just a numbers game, right? The more people you, kind of, address from there, you'll find—Corey: It's called a funnel for a reason.Linda: Right. Exactly.Corey: Free content, paid content. Almost anyone will follow me on Twitter; fewer than will sign up for a newsletter; fewer will listen to a podcast; fewer will watch a video, and almost none of them will buy a consulting engagement. But ‘almost' and ‘actually none of them,' it turns out is a very different world.Linda: Exactly. [laugh]. So FYI, I think there's—Corey: And that's fine. That's the way it works.Linda: That's the way it works. And I think there needs to be that niche content that might not be, like, the most viral thing, but viral doesn't mean quality, you know? It doesn't. There's many things that play into what viral is, but it's important to have the quality content for the people that need that content, and finding those people, you know, it's easier when you have that kind of mass engagement. Like, who knows? I'm a student. I told you; I'm a professional student. I'm still [laugh] learning every day.Corey: Working with AWS almost makes it a requirement. I wish you luck—Linda: Yeah.Corey: —in the new gig and I also want to thank you for taking time out of your day to speak with me about how you got to this point. And we're all very eager to see where you go from here.Linda: Thank you so much, Corey, for having me. I'm a huge fan, I love your content, I'm an avid reader of your newsletter and I am looking forward to very much being in touch and on the Twitterverse and beyond. So. [laugh].Corey: If people want to learn more about what you're up to, and other assorted nonsense, where's the best place they can go to find you?Linda: So, the best place they could go is lindavivah.com. I have all my different social handles listed on there as well a little bit about me, and I hope to connect with you. So, definitely go to lindavivah.com.Corey: And that link will, of course, be in the [show notes 00:30:39]. Thank you so much for taking the time to speak with me. I really appreciate it.Linda: Thank you, Corey. Have a wonderful rest of the day.Corey: Linda Haviv, AWS Developer Advocate, very soon now anyway. I'm Cloud Economist Corey Quinn and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice, smash the like and subscribe buttons, and of course, leave an angry comment that you have broken down into 40 serialized TikTok videos.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.
据财联社报道,从正邦科技处获悉,公司将在保证生产经营安全的前提下,与债权人积极协商沟通,后续逐步进行款项兑付工作。正邦科技6月8日晚间发布公告称,因流动资金紧张,公司及子公司近期出现5.42亿商票逾期未兑付的情形。据财联社报道,从多名相关人士处获悉,京东旗下社区团购平台京喜拼拼进一步收缩业务线,可能仅保留北京、郑州两地业务。目前山东济南、河南安阳、湖北等地已解散团长群。京喜拼拼此前3月已经历过一轮撤城,从20多个省份缩减至北京、山东、河南、湖北四省市。据界面报道,针对蔚来高管否认与AMD合作,AMD中国官微6月8日晚间回应称,蔚来采购了用于HPC研发的服务器,该批服务器使用的是基于“Zen 3”架构的AMD EPYC处理器。36氪获悉,有投资者在互动平台提问:有传言说宁德时代今年半年报可能会确认一季度发生的十多亿期货投资损失,导致二季度业绩的非常差,请公司解释下怎么回事。对此,宁德时代表示:公司的套保业务以现货为基础,期货端的浮动损益有相应的现货予以对冲,对公司业绩影响较小,上述情况不属实。据“上海迪士尼度假区发布”消息,星愿公园、迪士尼世界商店及蓝天大道将于今天起恢复运营,上海迪士尼乐园、迪士尼小镇及度假区的两座主题酒店继续保持暂时关闭,重新开放时间待定。据财联社报道,诺基亚CEO佩卡·伦德马克(Pekka Lundmark)日前接受采访时表示,新一代高速通信“6G”或将在2030年实用化。作为6G的具体使用案例,伦德马克提出看法称,“将利用传感器从人体获取各种数据并加以利用,医疗将明显改善。人类不再是用户,将成为互联网的一部分”。据新浪科技报道,天风国际证券分析师郭明錤表示,高通将推出代号为Hamoa的芯片与苹果Apple Silicon芯片对标,对比苹果M2采用台积电5nmN5P工艺,该芯片采用4nm工艺,预计2023年第三季度量产。不过在向苹果发起挑战前,高通必须说服PC厂商使用高通芯片而放弃X86芯片。
About AlyssaAlyssa Miller, Business Information Security Officer (BISO) for S&P Global, is the global executive leader for cyber security across the Ratings division, connecting corporate security objectives to business initiatives. She blends a unique mix of technical expertise and executive presence to bridge the gap that can often form between security practitioners and business leaders. Her goal is to change how security professionals of all levels work with our non-security partners throughout the business.A life-long hacker, Alyssa has a passion for technology and security. She bought her first computer herself at age 12 and quickly learned techniques for hacking modem communications and software. Her serendipitous career journey began as a software developer which enabled her to pivot into security roles. Beginning as a penetration tester, her last 16 years have seen her grow as a security leader with experience across a variety of organizations. She regularly advocates for improved security practices and shares her research with business leaders and industry audiences through her international public speaking engagements, online content, and other media appearances.Links Referenced: Cybersecurity Career Guide: https://alyssa.link/book A-L-Y-S-S-A dot link—L-I-N-K slash book: https://alyssa.link/book Twitter: https://twitter.com/AlyssaM_InfoSec alyssasec.com: https://alyssasec.com TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: This episode is sponsored in part by our friends at Vultr. Optimized cloud compute plans have landed at Vultr to deliver lightning-fast processing power, courtesy of third-gen AMD EPYC processors without the IO or hardware limitations of a traditional multi-tenant cloud server. Starting at just 28 bucks a month, users can deploy general-purpose, CPU, memory, or storage optimized cloud instances in more than 20 locations across five continents. Without looking, I know that once again, Antarctica has gotten the short end of the stick. Launch your Vultr optimized compute instance in 60 seconds or less on your choice of included operating systems, or bring your own. It's time to ditch convoluted and unpredictable giant tech company billing practices and say goodbye to noisy neighbors and egregious egress forever. Vultr delivers the power of the cloud with none of the bloat. Screaming in the Cloud listeners can try Vultr for free today with a $150 in credit when they visit getvultr.com/screaming. That's G-E-T-V-U-L-T-R dot com slash screaming. My thanks to them for sponsoring this ridiculous podcast.Corey: This episode is sponsored in part by Honeycomb. When production is running slow, it's hard to know where problems originate. Is it your application code, users, or the underlying systems? I've got five bucks on DNS, personally. Why scroll through endless dashboards while dealing with alert floods, going from tool to tool to tool that you employ, guessing at which puzzle pieces matter? Context switching and tool sprawl are slowly killing both your team and your business. You should care more about one of those than the other; which one is up to you. Drop the separate pillars and enter a world of getting one unified understanding of the one thing driving your business: production. With Honeycomb, you guess less and know more. Try it for free at honeycomb.io/screaminginthecloud. Observability: it's more than just hipster monitoring.Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. One of the problems that many folks experience in the course of their career, regardless of what direction they're in, is the curse of high expectations. And there's no escaping for that. Think about CISOs for example, the C-I-S-O, the Chief Information Security Officer.It's generally a C-level role. Well, what's better than a C in the academic world? That's right, a B. My guest today is breaking that mold. Alyssa Miller is the BISO—B-I-S-O—at S&P Global. Alyssa, thank you for joining me to suffer my slings and arrows—Alyssa: [laugh].Corey: —as we go through a conversation that is certain to be no less ridiculous than it has begun to be already.Alyssa: I mean, I'm good with ridiculous, but thanks for having me on. This is awesome. I'm really excited to be here.Corey: Great. What the heck's BISO?Alyssa: [laugh]. I never get that question. So, this is—Corey: “No one's ever asked me that before.” [crosstalk 00:03:38]—Alyssa: Right?Corey: —the same thing as, “Do you know you're really tall?” “No, you're kidding.” Same type of story. But I wasn't clear. That means I'm really the only person left wondering.Alyssa: Exactly. I mean, I wrote a whole blog on it the day I got the job, right? So, Business Information Security Officer, Basically what it means is I am like the CISO but for my division, the Ratings Division at S&P Global. So, I lead our cyber security efforts within that division, work closely with our information security teams, our corporate IT teams, whatever, but I don't report to them; I report into the business line.I'm in the divisional CTO's org structure. And so, I'm the one bridging that gap between that business side where hey, we make all the money and that corporate InfoSec side where hey, we're trying to protect all the things, and there's usually that little bit of a gap where they don't always connect. That's me building the bridge across that.Corey: Someone who speaks both security and business is honestly in a bit of rare supply these days. I mean, when I started my Thursday newsletter podcast nonsense Last Week in AWS: Security, the problem I kept smacking into was everything I saw was on one side of that divide or the other. There was the folks who have the word security in their job title, and there tends to be this hidden language of corporate speak. It's a dialect I don't fully understand. And then you have the community side of actual security practitioners who are doing amazing work, but also have a cultural problem that more or less distills down to being an awful lot of shitheads in them there waters.And I wanted something that was neither of those and also wasn't vendor captured, which is why I decided to start storytelling in that space. But increasingly, I'm seeing that there's a significant problem with people who are able to contextualize security in the context of business. Because if you're secure enough, you can stop all work from ever happening, whereas if you're pure business side and only care about feature velocity and the rest, like, “Well, what happens if we get breached?” It's, “Oh, don't worry, I have my resume up to date.” Not the most reassuring answer to give people. You have to be able to figure out where that line lies. And it seems like that figuring out where that line is, is more or less your entire stock-in-trade.Alyssa: Oh absolutely, yeah. I mean, I can remember my earliest days as a developer, my cynical attitude towards security myself was, you know, their Utopia would be an impenetrable room full of servers that have no connections to anything, right? Like that would be wildly secure, yet completely useless. And so yeah, then I got into security and now I was one of them. And, you know, it's one of those things, you sit in, say a board meeting sometime and you listen to a CISO, a typical CISO talk to the board, and they just don't get it.Like, there's so much, “Hey, we're implementing this technology and we're doing this thing, and here's our vulnerability counts, and here's how many are overdue.” And none of that means anything. I mean, I actually had a board member ask me once, “What is a CISO?” I kid you not. Like, that's where they're at.Like, so don't tell them what you're doing, but tell them why connected back to, like, “Hey, the business needs this and this, and in order to do it, we've got to make sure it's secure, so we're going to implement these couple of things. And here's the roadmap of how we get from where we are right now to where we need to be so they can launch that new service or product,” or whatever the hell it is that they're going to do.Corey: It feels like security is right up there with accounting, in the sense of fields of endeavor where you don't want someone with too much personality involved. Because if the CISO's sitting there talking to the board, it's like, “So, what do you do here, exactly?” And the answer is the honest, “Hey, remember last month how we were in The New York Times for that giant data breach?” And they do a split take, “No, no, I don't.” “Exactly. You're welcome.” On some level, it is kind of honest, but it also does not instill confidence when you're that cavalier with the description of what it is you do here.Alyssa: Oh there's—Corey: At least there's some corners. I prefer—Alyssa: —there's so much—Corey: —places where that goes over well, but that's me.Alyssa: Yeah. But there's so much of that too, right? Like, here's the one I love. “Well, you know, it's not if you get breached, it's when. Oh, by the way, give me millions and millions of dollars, so I can make sure we don't get breached.”But wait, you just told me we're going to get breached no matter what we do. [laugh]. We do that in security. Like, and then you wonder why they don't give you funding for the initiative. Like, “Hello?” You know?And that's the thing that gets me it's like, can we just sit back and understand, like, how do you message to these people? Yeah I mean, you bring up the accounting thing; the funny thing is, at least all of them understand some level of accounting because most of them have MBAs and business degrees where they had to do some accounting. They didn't go through cyber security in their MBA program.So, one of my favorite questions on Twitter once was somebody asked me, you know, if I want to get into cyber security leadership, what is the one thing that I should focus on or what skills should I study? I said, “Go study MBA concepts.” Like, forget all the cyber security stuff. You probably have plenty of that technolog—go understand what they learn in MBA programs. And if you can start to speak that language, that's going to pay dividends for bridging that gap.Corey: So, you don't look like the traditional slovenly computer geek showing up at those meetings who does not know how to sound as if they belong in the room. Like, it's unfair, on some level, and I used to have bitter angst about that. Like, “Why should how I dress matter how people perceive me?” Yeah, in an absolute sense you're absolutely right, however, I can talk about the way the world is or the way I wish it were and there has to be a bit of a divide there.Alyssa: Oh, for sure. Yeah. I mean, you can't deny that you have to be prepared for the audience you're walking into. Now, I work in big conservative financial services on Wall Street. You know, and I had this conversation with a prominent member of our community when I started the job.I'm like, “Boy, I guess I can't really put stickers on my laptop. I'm going to have to get, you know, a protector or something to put stickers on.” Because the last thing I want to do is go into a boardroom with my laptop and whip out a bunch of hacker stickers on the backside of my laptop. Like, in a lot of spaces that will work, but you can't really do that when you're, you know, at, you know, the executive level and you're in a conservative, financial [unintelligible 00:10:16]. It just, I would love to say they should deal with that, I should be able to have pink hair, and you know, face tattoos and everything else, but the reality is, yeah, I can do all that, but these are still human beings who are going to react to that.And it's the same when talking about cyber security, then. Like, I have to understand as a security practitioner that all they know about cyber security is it's big and scary. It's the thing that keeps them up at night. I've had board members tell me exactly that. And so, how do I make it a little less scary, or at least get them to have some confidence in me that I'll, like, carry the shield in front of them and protect them. Like, that's my job. That's why I'm there.Corey: When I was starting my consultancy five years ago, I was trying to make a choice between something in the security cloud direction or the cost cloud direction. And one of the things that absolutely tipped the balance for me was the fact that the AWS bill is very much a business-hours-only problem. No one calls me at two in the morning screaming their head off. Usually. But there's a lot of alignment between those two directions in that you can spend all your time and energy fixing security issues and/or reducing the bill, but past a certain point, knock it off and go do the thing that your company is actually there to do.And you want to be responsible to a point on those things, but you don't want it to be the end-all-be-all because the logical outcome of all of that, if you keep going, is your company runs out of money and dies because you're not going to either cost optimize or security optimize your business to its next milestone. And weighing those things is challenging. Now, too many people hear that and think, “See, I don't have to worry about those things at all.” It's, “Oh, you will sooner or later. I promise.”Alyssa: So, here's the fallacy in that. There is this assumption that everything we do in security is going to hamper the business in some way and so we have to temper that, right? Like, you're not wrong. And we talked about before, right? You know, security in a traditional sense, like, we could do all of the puristic things and end up just, like, screeching the world to a halt.But the reality is, we can do security in a way that actually grows the business, that actually creates revenue, or I should say enables the creation of revenue in that, you know, we can empower the business to do more things and to be more innovative by how we approach security in the organization. And that's the big thing that we miss in security is, like, look, yes, we will always be a quote-unquote, “Cost center,” right? I mean, we in security don't—unless you work for a security organization—we're not getting revenue attributed to us, we're not creating revenue. But we are enabling those people who can if we approach it right.Corey: Well, the Red Team might if they go a little off-script, but that's neither here nor there.Alyssa: I—yeah, I mean, I've had that question. “Like, couldn't we just sell resell our Red Team services?” No. No. That's not our core [crosstalk 00:13:14]Corey: Oh, I was going the other direction. Like, oh, we're just going to start extorting other businesses because we got bored this week. I'm kidding. I'm kidding. Please don't do an investigation, any law enforcement—Alyssa: I was going to say, I think my [crosstalk 00:13:22]—Corey: —folks that happen to be listening to this.Alyssa: [crosstalk 00:13:24] is calling me right now. They're want to know what I'm [laugh] talking about. But no—Corey: They have some inquiries they would like you to assist them with and they're not really asking.Alyssa: Yeah, yeah, they're good at that. No, I love them, though. They're great. [laugh]. But no, seriously, like, I mean, we always think about it that way because—and then we wonder why do we have the reputation of, you know, the Department of No.Well, because we kind of look at it that way ourselves; we don't really look at, like how can we be a part of the answer? Like, when we look at, like, DevSecOps, for instance. Okay, I want to bring security into my pipeline. So, what do we say? “Oh, shared responsibility. That's a DevOps thing.” So, that means security is everybody's responsibility. Full stop.Corey: Right. It's a—Alyssa: Well—Corey: And there, I agree with you wholeheartedly. Cost is—Alyssa: But—Corey: —aligned with this. It has to be easier to do it the right way than to just go off half-baked and do it yourself off the blessed path. And that—Alyssa: So there—Corey: —means there's that you cannot make it harder to do the right thing; you have to make it easier because you will not win against human psychology. Depending on someone when they're done with an experiment to manually go in and turn things off. It will not happen. And my argument has been that security and cost are aligned constantly because the best way to secure something and save money on at the same time is to turn that shit off. You wouldn't think it would be that simple, but yet here we are.Alyssa: But see, here's the thing. This is what kills me. It's so arrogant of security people to look at it and say that right? Because shared responsibility means shared. Okay, that means we have responsibilities we're going to share. Everybody is responsible for security, yes.Our developers have responsibilities now that we have to take a share in as well, which is get that shit to production fast. Period. That is their goal. How fast can I pop user stories off the backlog and get them to deployment? My SRE is on the ops side. They're, like, “We just got to keep that stuff running. That's all we that's our primary focus.”So, the whole point of DevOps and DevSecOps was everybody's responsible for every part of that, so if I'm bringing security into that message, I, as security, have to be responsible for site's stability; I, in security, have to be responsible for efficient deployment and the speed of that pipeline. And that's the part that we miss.Corey: This episode is sponsored in parts by our friend EnterpriseDB. EnterpriseDB has been powering enterprise applications with PostgreSQL for 15 years. And now EnterpriseDB has you covered wherever you deploy PostgreSQL on-premises, private cloud, and they just announced a fully-managed service on AWS and Azure called BigAnimal, all one word. Don't leave managing your database to your cloud vendor because they're too busy launching another half-dozen managed databases to focus on any one of them that they didn't build themselves. Instead, work with the experts over at EnterpriseDB. They can save you time and money, they can even help you migrate legacy applications—including Oracle—to the cloud. To learn more, try BigAnimal for free. Go to biganimal.com/snark, and tell them Corey sent you.Corey: I think you might be the first person I've ever spoken to that has that particular take on the shared responsibility model. Normally, when I hear it, it's on stage from an AWS employee doing a 45-minute song-and-dance about what the secured responsibility model is, and generally, that is interpreted as, “If you get breached, it's your fault, not ours.”Alyssa: [laugh].Corey: Now, you can't necessarily say it that directly to someone who has just suffered a security incident, which is why it takes 45 minutes and slides and diagrams and excel sheets and the rest. But that is what it fundamentally distills down to, and then you wind up pointing out security things that they've had that [unintelligible 00:17:11] security researchers have pointed out and they are very tight-lipped about those things. And it's, “Oh, it's not that you're otherworldly good at security; it's that you're great at getting people to shut up.” You know, not me, for whatever reason because I'm noisy and obnoxious, but most people who actually care about not getting fired from their jobs, generally don't want to go out there making big cloud companies look bad. Meanwhile, that's kind of my entire brand.Alyssa: I mean, it's all about lines of liability, right?Corey: Oh yeah.Alyssa: I mean, where am I liable, where am I not? And yeah, well, if I tell you you're responsible for security on all these things, and I can point to any part of that was part of the breach, well, hey, then it's out of my hands. I'm not liable. I did what I said I would; you didn't secure your stuff. Yeah, it's—and I mean, and some of that is to be fair.Like, I mean, okay, I'm going to host my stuff on your computer—the whole cloud is just somebody else's computer model is still ultimately true—but, yeah, I mean, I'm expecting you to provide me a stable and secure environment and then I'm going to deploy stuff on it, and you are expecting me to deploy things that are stable and secure as well. And so, when they say shared model or shared responsibility model, but it—really if you listen to that message, it's the exact opposite. They're telling you why it's a separate responsibility model. Here's our responsibilities; here's yours. Boom. It's not about shared; it's about separated.Corey: One of the most formative, I guess, contributors to my worldview was 13 years ago, I went on a date and met someone lovely. We got married. We've been together ever since, and she's an attorney. And it is been life-changing to understand a lot of that perspective, where it turns out when you're dealing with legal, they are not—and everyone says, “Oh, and the lawyers insisted on these things.”No, they didn't. A lawyer's entire role in a company is to identify risk, and then it is up to the business to make a decision around what is acceptable and what is not. If your lawyers ever insist on something, what that actually means in my experience is, you have said something profoundly ignorant that is one of those, like—that is—they're doing the legal equivalent of slapping the gun out of the toddler's hand of, “No, you cannot go and tweet that because you'll go to prison,” level of ridiculous nonsense where it is, “That will violate the law.” Everything else is different shades of the same answer: it depends. Here's what to consider.Alyssa: Yes.Corey: And then you choose—and the business chooses its own direction. So, when you have companies doing what appeared to be ridiculous things, like Oracle, for example, loves to begin every keynote with a disclaimer about how nothing they're about to say is true, the lawyers didn't insist on that—though they are the world's largest law firm, Kirkland Ellison. But instead, it's this entire story of given the risk and everything that we know about how we say things onstage and people gunning for us, yeah, we are going to [unintelligible 00:20:16] this disclaimer first. Most other tech companies do not do that exact thing, which I've got to say when you're sitting in the audience ready to see the new hotness that's about to get rolled out and it starts with a disclaimer, that is more or less corporate-speak for, “You are about to hear some bullshit,” in my experience.Alyssa: [laugh]. Yes. I mean and that's the thing, like, [clear throat], you know, we do deride legal teams a lot. And you know, I can find you plenty of security people who hate the fact that when you're breached, who's the first call you make? Well, it's your legal team.Why? Because they're the ones who are going to do everything in their power to limit the amount that you can get sued on the back-end for anything that got exposed, that you know, didn't meet service levels, whatever the heck else. And that all starts with legal privilege.Corey: They're reporting responsibilities. Guess who keeps up on what those regulatory requirements are? Spoiler, it's probably not you, whoever's listening to this, unless you're an attorney because that is their entire job.Alyssa: Yes, exactly. And, you know, work in a highly regulated environment—like mine—and you realize just how critical that is. Like, how do I know—I mean, there are times there's this whole discussion of how do you determine if something is a material impact or not? I don't want to be the one making that, and I'm glad I don't have to make that decision. Like, I'll tell you all the information, but yes, you lawyers, you compliance people, I want you to make the decision of if it's a material impact or not because as much as I understand about the business, y'all know way more about that stuff than I do.I can't say. I can only say, “Look, this is what it impacted. This is the data that was impacted. These are the potential exposures that occurred here. Please take that information now and figure out what that means, and is there any materiality to that that now we have to report that to the street.”Corey: Right, right. You can take my guesses on this or you can get it take an attorney's. I am a loud, confident-sounding white guy. Attorneys are regulated professionals who carry malpractice insurance. If they give wrong advice that is wrong enough in these scenarios, they can be sanctioned for it; they can lose their license to practice law.And there are challenges with the legal profession and how much of a gatekeeper the Bar Association is and the rest, but this is what it is [done 00:22:49] for itself. That is a regulated industry where they have continuing education requirements they need to certify in a test that certain things are true when they say it, whereas it turns out that I don't usually get people even following up on a tweet that didn't come true very often. There's a different level of scrutiny, there's a different level of professional bar it raises to, and it turns out that if you're going to be legally held to account for things you say, yeah, turns out a lot of your answers to are going to be flavors of, “It depends.”Alyssa: [laugh].Corey: Imagine that.Alyssa: Don't we do that all the time? I mean, “How critical is this?” “Well, you know, it depends on what kind of data, it depends on who the attacker is. It depends.” Yeah, I mean, that's our favorite word because no one wants to commit to an absolute, and nor should we, I mean, if we're speaking in hyperbole and absolutes, boy, we're doing all the things wrong in cyber.We got to understand, like, hey, there is nuance here. That's how you run—no business runs on absolutes and hyperbole. Well, maybe marketing sometimes, but that's a whole other story.Corey: Depends on if it's done well or terribly.Alyssa: [laugh]. Right. Exactly. “Hey, you can be unhackable. You can be breached-proof.” Oh, God.Corey: Like, what's your market strategy? We're going to paint a big freaking target in the front of the building. Like, I still don't know how Target the company was ever surprised by a data breach that they had when they have a frickin' bullseye as their logo.Alyssa: “Come get us.”Corey: It's, like, talk about poking the bear. But there we are.Alyssa: [unintelligible 00:24:21] no. I mean, hey, [unintelligible 00:24:23] like that was so long ago.Corey: It still casts a shadow.Alyssa: I know.Corey: People point to that as a great example of, like, “Well, what's going to happen if we get breached?” It's like, well look at Target because they wound up—like, their stock price a year later was above where it had been before and it seemed to have no lasting impact. Yeah, but they effectively replaced all of the execs, so you know, let's have some self-interest going on here by named officers of the company. It's, “Yeah, the company will be fine. Would you like to still be here what it is?”Alyssa: And how many lawsuits do you think happened that you never heard about because they got settled before they were filed?Corey: Oh, yes. There's a whole world of that.Alyssa: That's what's really interesting when people talk about, like, the cost of breach and stuff, it's like, we don't even know. We can't know because there is so much of that. I mean, think about it, any organization that gets breached, the first thing they're trying to do is keep as much of it out of the news as they can, and that includes the lawsuits. And so, you know, it's like, all right, well, “Hey, let's settle this before you ever file.”Okay, good. No one will ever know about that. That will never show up anywhere. It is going to show up on a balance sheet anywhere, right? I mean, it's there, but it's buried in big categories of lots of other things, and how are you ever going to track that back without, you know, like, a full-on audit of all of their accounting for that year? Yeah, it's—so I always kind of laugh when people start talking about that and they want to know, what's the average cost of a breach. I'm like, “There's no way to measure that. There is none.”Corey: It's not cheap, and the reputational damage gets annoying. I still give companies grief for these things all the time because it's—again, the breach is often about information of mine that I did not consciously choose to give to you and the, “Oh, I'm going to blame a third-party process.” No, no, you can outsource work, but not responsibility. You can't share that one.Alyssa: Ah, third-party diligence, uh, that seems to be a thing. You know, I think we're supposed to make sure our third parties are trustworthy and doing the right things too, right? I mean, it's—Corey: Best example I ever saw that was an article in the Wall Street Journal about the Pokemon company where they didn't name the vendor, but they said they declined to do business with them in part based upon their lax security policy around S3 buckets. That is the first and so far only time I have had an S3 Bucket Responsibility Award engraved and sent to their security director. Usually, it's the ignoble prize of the S3 Bucket Negligence Award, and there are oh so many of those.Alyssa: Oh, and it's hard, right? Because you're standing—I mean, I'm in that position a lot, right? You know, you're looking at a vendor and you've got the business saying, “God, we want to use this vendor. All their product is great.” And I'm sitting there saying, but, “Oh, my God, look at what they're doing. It's a mess. It's horrible. How do I how do we get around this?”And that's where, you know, you just have to kind of—I wish I could say no more, but at the end of the day, I know what that does. That just—okay, well, we'll go file an exception and we'll use it anyway. So, maybe instead, we sit and work on how to do this, or maybe there is an alternative vendor, but let's sort it out together. So yeah, I mean, I do applaud them. Like that's great to, like, be able to look at a vendor and say, “No, we ain't touching you because what you're doing over there is nuts.” And I think we're learning more and more how important that is, with a lot of the supply chain attacks.Corey: Actually, I'm worried about having emailed you, you're going to leak my email address when your inbox inevitably gets popped. Come on. It's awful stuff.Alyssa: Yeah, exactly. So, I mean, it's we there's—but like everything, it's a balance again, right? Like, how can we keep that business going and also make sure that their vendors—so that's where it just comes down to, like, okay, let's talk contracts now. So, now we're back to legal.Corey: We are. And if you talk to a lawyer and say, “I'm thinking about going to law school,” the answer is always the same. “No… don't do it.” Making it clear that is apparently a terrible life and professional decision, which of course, brings us to your most recent terrible life and professional decision. As we record this, we are reportedly weeks away from you having a physical copy in your hands of a book.And the segue there is because no one wants to write a book. Everyone wants to have written a book, but apparently—unless you start doing dodgy things and ghost-writing and exploiting people in the rest—one is a necessary prerequisite for the other. So, you've written a book. Tell me about it.Alyssa: Oof, well, first of all, spot on. I mean, I think there are people who really do, like, enjoy the act of writing a book—Corey: Oh, I don't have the attention span to write a tweet. People say, “Oh, you should write a book, Corey,” which I think is code for them saying, “You should shut up and go away for 18 months.” Like, yeah, I wish.Alyssa: Writing a book has been the most eye-opening experience of my life. And yeah, I'm not a hundred percent sure it's one I'll ever—I've joked with people already, like, I'll probably—if I ever want another book, I'll probably hire a ghostwriter. But no, I do have a book coming out: Cybersecurity Career Guide. You know, I looked at this cyber skills gap, blah, blah, blah, blah, blah, we hear about it, 4 million jobs are going to be left open.Whatever, great. Well, then how come none of these college grads can get hired? Why is there this glut of people who are trying to start careers in cyber security and we can't get them in?Corey: We don't have six months to train you, so we're going to spend nine months trying to fill the role with someone experienced?Alyssa: Exactly. So, 2020 I did a bunch of research into that because I'm like, I got to figure this out. Like, this is bizarre. How is this disconnect happening? I did some surveys. I did some interviews. I did some open-source research. Ended up doing a TED Talk based off of that—or TEDx Talk based off of that—and ultimately that led into this book. And so yeah, I mean, I just heard from the publisher yesterday, in fact that we're, like, in that last stage before they kick it out to the printers, and then it's like three weeks and I should have physical copies in my hands.Corey: I will be getting one when it finally comes out. I have an almost, I believe, perfect track record of having bought every book that a guest on this show has written.Alyssa: Well, I appreciate that.Corey: Although, God help me if I ever have someone, like, “So, what have you done?” “I've written 80 books.” Like, “Well, thank you, Stephen King. I'm about to go to have a big—you're going to see this number of the company revenue from orbit at this point with that many.” But yeah, it's impressive having written a book. It's—Alyssa: I mean, for me, it's the reward is already because there are a lot of people have—so my publisher does really cool thing they call it early acc—or electronic access program, and where there are people who bought the book almost a year ago now—which is kind of, I feel bad about that, but that's as much my publisher as it is me—but where they bought it a year ago and they've been able to read the draft copy of the book as I've been finishing the book. And I'm already hearing from them, like, you know, I'm hearing from people who really found some value from it and who, you know, have been recommending it other people who are trying to start careers and whatever. And it's like, that's where the reward is, right?Like, it was, it's hell writing a book. It was ten times worse during Covid. You know, my publisher even confirmed that for me that, like, look, yeah, you know, authors around the globe are having problems right now because this is not a good environment conducive to writing. But, yeah, I mean, it's rewarding to know that, like, all right, there's going to be this thing out there, that, you know, these pages that I wrote that are helping people get started in their careers, that are helping bring to light some of the real challenges of how we hire in cyber security and in tech in general. And so, that's the thing that's going to make it worthwhile. And so yeah, I'm super excited that it's looking like we're mere weeks now from this thing being shipped to people who have bought it.Corey: So, now it's racing, whether this gets published before the book does. So, we'll see. There is a bit of a production lag here because, you know, we have to make me look pretty and that takes a tremendous amount of effort.Alyssa: Oh, stop. Come on now. But it will be interesting to see. Like, that would actually be really cool if they came out at about the same time. Like, you know, I'm just saying.Corey: Yeah. We'll see how it goes. Where's the best place for people to find you if they want to learn more?Alyssa: About the book or in general?Corey: Both.Alyssa: So—Corey: Links will of course be in the [show notes 00:32:49]. Let's not kid ourselves here.Alyssa: The book is real easy. Go to Alyssa—A-L-Y-S-S-A, back here behind me for those of you seeing the video. Um—I can't point the right direction. There we go. That one. A-L-Y-S-S-A dot link—L-I-N-K slash book. It's that simple. It'll take you right to Manning's site, you can get in.Still in that early access program, so if you bought it today, you would still be able to start reading the draft versions of it. If you want to know more about me, honestly, the easiest way is to find me on Twitter. You can hear all the ridiculousness of flight school and barbecue and some security topics, too, once in a while. But at @alyssam_infosec. Or if you want to check out the website where I blog, every rare occasion, it's alyssasec.com.Corey: And all of that will be in the [show notes 00:33:41]. Thank you—Alyssa: There's a lot. [laugh].Corey: I'm looking forward to seeing it, too. Thank you so much for taking the time to deal with my nonsense today. I really appreciate it.Alyssa: Oh, that was nonsense? Are you kidding me? This was a great discussion. I really appreciate it.Corey: As have I. Thanks again for your time. It is always great to talk to people smarter than I am—which is, let's be clear, most people—Alyssa Miller, BISO at S&P Global. I'm Cloud Economist Corey Quinn, and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice—or smash the like and subscribe button if this is on the YouTubes—whereas if you've hated the podcast, same thing, five-star review, platform of choice, smash both of the buttons, but also leave an angry comment, either on the YouTube video or on the podcast platform, saying that this was a waste of your time and what you didn't like about it because you don't need to read Alyssa's book; you're going to get a job the tried and true way, by printing out a copy of your resume and leaving it on the hiring manager's pillow in their home.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.
Bullish News For AMD Stock and Nvidia Stock Price Investors. Jose Najarro looks at AMD Stock News about being the top supercomputer with AMD EPYC, and Nvidia Stock Price After recent NVDA Stock News on NVDA Grace Hopper Superchip. Time to buy AMD Stock Price or NVDA Stock Pricehttps://www.fool.com/jose*A portion of this video is sponsored by The Motley Fool. Visit https://fool.com/josenajarro to get access to my special offer. The Motley Fool Stock Advisor returns are 557% as of 3/31/2021 and measured against the S&P 500 returns of 122% as of 3/31/2021. Past performance is not an indicator of future results. All investing involves a risk of loss. Individual investment results may vary, not all Motley Fool Stock Advisor picks have performed as well.*I have a position in $TSM $NVDA $AMD $MSFTNewsLetterhttps://www.fool.com/josenajarroDISCORD GROUP!! https://discord.gg/wbp2Z9STwitter: https://twitter.com/_JoseNajarroDISCLAIMER: I am not a financial advisor. All content provided on this channel, and my other social media channels/videos/podcasts/posts, is for entertainment purposes only and reflects my personal opinions. Please do your own research and talk with a financial advisor before making any investing decisions.
About JamesJames has been part of AWS for over 15 years. During that time he's led software engineering for Amazon EC2 and more recently leads the AWS Commerce Platform group that runs some of the largest systems in the world, handling volumes of data and request rates that would make your eyes water. And AWS customers trust us to be right all the time so there's no room for error.Links Referenced:Email: jamesg@amazon.comTranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: This episode is sponsored in part by our friends at Vultr. Optimized cloud compute plans have landed at Vultr to deliver lightning-fast processing power, courtesy of third-gen AMD EPYC processors without the IO or hardware limitations of a traditional multi-tenant cloud server. Starting at just 28 bucks a month, users can deploy general-purpose, CPU, memory, or storage optimized cloud instances in more than 20 locations across five continents. Without looking, I know that once again, Antarctica has gotten the short end of the stick. Launch your Vultr optimized compute instance in 60 seconds or less on your choice of included operating systems, or bring your own. It's time to ditch convoluted and unpredictable giant tech company billing practices and say goodbye to noisy neighbors and egregious egress forever. Vultr delivers the power of the cloud with none of the bloat. “Screaming in the Cloud” listeners can try Vultr for free today with a $150 in credit when they visit getvultr.com/screaming. That's G-E-T-V-U-L-T-R dot com slash screaming. My thanks to them for sponsoring this ridiculous podcast.Corey: Finding skilled DevOps engineers is a pain in the neck! And if you need to deploy a secure and compliant application to AWS, forgettaboutit! But that's where DuploCloud can help. Their comprehensive no-code/low-code software platform guarantees a secure and compliant infrastructure in as little as two weeks, while automating the full DevSecOps lifestyle. Get started with DevOps-as-a-Service from DuploCloud so that your cloud configurations are done right the first time. Tell them I sent you and your first two months are free. To learn more visit: snark.cloud/duplo. Thats's snark.cloud/D-U-P-L-O-C-L-O-U-D. Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. And I've been angling to get someone from a particular department at AWS on this show for nearly its entire run. If you were to find yourself in an Amazon building and wander through the various dungeons and boiler rooms and subterranean basements—I presume; I haven't seen nearly as many of you inside of those buildings as people might think—you pass interesting departments labeled things like ‘Spline Reticulation,' or whatnot. And then you come to a very particular group called Commerce Platform.Now, I'm not generally one to tell other people's stories for them. My guest today is James Greenfield, the VP of Commerce Platform at AWS. James, thank you for joining me and suffering the slings and arrows I will no doubt be hurling at you.James: Thanks for having me. I'm looking forward to it.Corey: So, let's start at the very beginning—because I guarantee you, you're going to do a better job of giving the chapter and verse answer than I would from a background mired deeply in snark—what is Commerce Platform? It sounds almost like it's the retail website that sells socks, books, and underpants.James: So, Commerce Platform actually spans a bunch of different things. And so, I'm going to try not to bore you with a laundry list of all of the things that we do—it's a much longer list than most people assume even internal to AWS—at its core, Commerce Platform owns all of the infrastructure and processes and software that takes the fact that you've been running an EC2 instance, or you're storing an object in S3 for some period of time, and turns it into a number at the end of the month. That is what you asked for that service and then proceeds to try to give you as many ways to pay us as easily as possible. There are a few other bits in there that are maybe less obvious. One is we're also responsible for protecting the platform and our customers from fraudulent activity. And then we're also responsible for helping collect all of the data that we need for internal reporting to support some of the back-ends services that a business needs to do things like revenue recognition and general financial reporting.Corey: One of the interesting aspects about the billing system is just how deeply it permeates everything that happens within AWS. I frequently say that when it comes to cloud, cost and architecture are foundationally and fundamentally the same exact thing. If your entire service goes down, a few interesting things happen. One, I don't believe a single customer is going to complain other than maybe a few accountants here and there because the books aren't reconciling, but also you've removed a whole bunch of constraints around why things are the way that they are. Like, what is the most efficient way to run this workload?Well, if all the computers suddenly become free, I don't really care about efficiency, so much is, “Oh, hey. There's a fly, what do I have as a flyswatter? That's right, I'm going to drop a building on it.” And those constraints breed almost everything. I've said, for example, that S3 has infinite storage because it does.They can add drives faster than we're able to fill them—at least historically; they added some more replication services—but they're going to be able to buy hard drives faster than the rest of us are going to be able to stretch our budgets. If that constraint of the budget falls away, all bets are really off, and more or less, we're talking about the destruction of the cloud as a viable business entity. No pressure or anything.James: [laugh].Corey: You're also a recent transplant into AWS billing as a whole, Commerce Platform in general. You spent 15 years at the company, the vast majority of that over an EC2. So, either it was you've been exiled to a basically digital Siberia or it was one of those, “Okay, keeping all the EC2 servers up, this is easy. I don't see what people stress about.” And they say, “Oh, ho ho, try this instead.” How did you find yourself migrating over to the Commerce Platform?James: That's actually one I've had a lot from folks that I've worked with. You're right, I spent the first 15 or so years of my career at AWS in EC2, responsible for various things over there. And when the leadership role in Commerce Platform opened up, the timing was fortuitous, and part of it, I was in the process of relocating my family. We moved to Vancouver in the middle of last year. And we had an opening in the role and started talking about, potentially, me stepping into that role.The reason that I took it—there's a few reasons, but the primary reason is that if I look back over my career, I've kind of naturally gravitated towards owning things where people only really remember that they exist when they're not working. And for some reason, you know, I enjoy the opportunity to try to keep those kinds of services ticking over to the point where people don't notice them. And so, Commerce Platform lands squarely in that space. I've always been attracted to opportunities to have an impact, and it's hard to imagine having much more of an impact than in the Commerce Platform space. It underpins everything, as you said earlier.Every single one of our customers depends on the service, whether they think about it or realize it. Every single service that we offer to customers depends on us. And so, that really is the sort of nexus within AWS. And I'm a platform guy, I've always been a platform guy. I like the force multiplier nature of platforms, and so Commerce Platform, you know, as I kind of thought through all of those elements, really was a great opportunity to step in.And I think there's something to be said for, I've been a customer of Commerce Platform internally for a long time. And so, a chance to cross over and be on the other side of that was something that I didn't want to pass up. And so, you know, I'm digging in, and learning quickly, ramping up. By no means an expert, very dependent on a very smart, talented, committed group of people within the team. That's kind of the long and short of how and why.Corey: Let's say that I am taking on the role of an AWS product team, for the sake of argument. I know, keep the cringe down for a second, as far as oh, God, the wince is just inevitable when the idea of me working there ever comes up to anyone. But I have an idea for a service—obviously, it runs containers, and maybe it does some other things as well—going from idea to six-pager to MVP to barely better than MVP day-one launch, and at some point, various things happen to that service. It gets staff with a team, objectives and a roadmap get built, a P&L and budget, and a pricing model and the rest. One the last thing that happens, apparently, is someone picks the worst name off of a list of candidates, slaps it on the product, and ships it off there.At what point does the billing system and figuring out the pricing dimensions for a given service tend to factor in? Is that a last-minute story? Is that almost from the beginning? Where along that journey does, “Oh, by the way, we're building this thing. Maybe we should figure out, I don't know, how to make money from it.” Factor into the conversation?James: There are two parts to that answer. Pretty early on as we're trying to define what that service is going to look like, we're already typically thinking about what are the dimensions that we might charge along. The actual pricing discussions typically happen fairly late, but identifying those dimensions and, sort of, the right way to present it to customers happens pretty early on. The thing that doesn't happen early enough is actually pulling the Commerce Platform team in. but it is something that we're going to work this year to try to get a little bit more in front of.Corey: Have you found historically that you have a pretty good idea of how a service is going to be priced, everything is mostly thought through, a service goes to either private preview or you're discussing about a launch, and then more or less, I don't know, someone like me crops up with a, “Hey, yeah, let's disregard 90% of what the service does because I see a way to misuse the remaining 10% of it as a database.” And you run some mental math and realize, “Huh. We're suddenly giving, like, eight petabytes of storage per customer away for free. Maybe we should guard against that because otherwise, it's rife with misuse.” It used to be that I could find interesting ways to sneak through the cracks of various services—usually in pursuit of a laugh—those are getting relatively hard to come by and invariably a lot more trouble than they're worth. Is that just better comprehensive diligence internally, is that learning from customers, or am I just bad at this?James: No, I mean, what you're describing is almost a variant of the Defender's Dilemma. They are way more ways to abuse something than you can imagine, and so defending against that is pretty challenging. And it's important because, you know, if you turn the economics of something upside down, then it just becomes harder for us to offer it to customers who want to use it legitimately. I would say 90% of that improvement is us learning. We make plenty of mistakes, but I think, you know, one of the things that I've always been impressed by over my time here is how intentional we are trying to learn from those mistakes.And so, I think that's what you're seeing there. And then we try very hard to listen to customers, talk to folks like you, because one of the best ways to tackle anything it smells of the Defender's Dilemma is to harness that collective creativity of a large number of smart people because you really are trying to cover as much ground as possible.Corey: There was a fun joke going around a while back of what is the most expensive environment you can get running on a free tier account before someone from AWS steps in, and I think I got it to something like half a billion dollars in the first month. Now, I haven't actually tested this for reasons that mostly have to do with being relatively poor compared to, you know, being able to buy Guam. And understanding as well the fraud protections built into something like AWS are largely built around defending against getting service usage for free that in some way, shape or form, benefits the attacker. The easy example of that would be mining cryptocurrency, which is just super-economic as long as you use someone else's AWS account to do it. Whereas a lot of my vectors are, “Yeah, ignore all of that. How do I just make the bill artificially high? What can I do to misuse data transfer? And passing a single gigabyte through, how much can I make that per gigabyte cost be?” And, “Oh, circular replication and the Lambda invokes itself pattern,” and basically every bad architectural decision you can possibly make only this time, it's intentional.And that shines some really interesting light on it. And I have to give credit where due, a lot of that didn't come from just me sitting here being sick and twisted nearly so much as it did having seen examples of that type of misconfiguration—by mistake—in a variety of customer accounts, most confidently my own because it turns out that the way I learn things is by screwing them up first.James: Yeah, you've touched on a couple of different things in there. So, you know, maybe the first one is, I typically try to draw a line between fraud and abuse. And fraud is essentially trying to spend somebody else's money to get something for free. And we spent a lot of time trying to shut that down, and we're getting really good at catching it. And then abuse is either intentional or unintentional. There's intentional abuse: You find a chink in our armor and you try to take advantage of it.But much more commonly is unintentional abuse. It's not really abuse, you know. Abuse has very negative connotations, but it's unintentionally setting something up so that you run up a much larger bill than you intended. And we have a number of different internal efforts, and we're working on a bunch more this year, to try to catch those early on because one of my personal goals is to minimize the frequency with which we surprise customers. And the least favorite kind of surprise for customers is a [laugh] large bill. And so, what you're talking about there is, in a sufficiently complex system, there's always going to be weaknesses and ways to get yourself tied up in knots.We're trying both at the service team level, but also within my teams to try to find ways to make it as hard as possible to accidentally do that to yourself and then catch when you do so that we can stop it. And even more on the intentional abuse side of things, if somebody's found a way to do something that's problematic for our services, then you know, that's pretty much on us. But we will often reach out and engage with whoever's doing and try to understand what they're trying to do and why. Because often, somebody's trying to do something legitimate, they've got a problem to solve, they found a creative way to solve it, and it may put strain on the service because it's just not something we designed for, and so we'll try to work with them to use that to feed into either new services, or find a better place for that workload, or just bolster what they're using. And maybe that's something that eventually becomes a fully-fledged feature that we offer the customers. We're always open to learning from our customers. They have found far more creative ways to get really cool things done with our services than we've ever imagined. And that's true today.Corey: I mean, most of my service criticisms come down to the fact that you have more-or-less built a very late model, high performing iPad, and I'm out there complaining about, “What a shitty hammer this thing is, it barely works at all, and then it breaks in my hand. What gives?” I would also challenge something you said a minute ago that the worst day for some customers is to get a giant surprise bill, but [unintelligible 00:13:53] to that is, yeah, but, on some level, that kind of only money; you do have levers on your side to fix those issues. A worse scenario is you have a customer that exhibits fraud-like behavior, they're suddenly using far more resources than they ever did before, so let's go ahead and turn them off or throttle them significantly, and you call them up to tell them you saved them some money, and, “Our Superbowl ad ran. What exactly do you think you're doing?” Because they don't get a second bite at that kind of Apple.So, there's a parallel on both sides of this. And those are just two examples. The world is full of nuances, and at the scale that you folks operate at. The one-in-a-million events happen multiple times a second, the corner cases become common cases, and I'm surprised—to be direct—how little I see you folks dropping the ball.James: Credit to all of the teams. I think our secret sauce, if anything, really does come down to our people. Like, a huge amount of what you see as hopefully relatively consistent, good execution comes down to people behind the scenes making sure. You know, like, some of it is software that we built and made sure it's robust and tested to scale, but there's always an element of people behind the scenes, when you hit those edge cases or something doesn't quite go the way that you planned, making sure that things run smoothly. And that, if anything, is something that I'm immensely proud of and is kind of amazing to watch from the inside.Corey: And, on some level, it's the small errors that are the bigger concern than the big ones. Back a couple years ago, when they announced GP3 volumes at re:Invent, well, great, well spin up a test volume and kick the tires on it for an hour. And I think it was 80 or 100 gigs or whatnot, and the next day in the bill, it showed up as about $5,000. And it was, “Okay, that's not great. Not great at all.” And it turned out that it was a mispricing error by I think a factor of a million.And okay, at least it stood out. But there are scenarios where we were prepared to pay it because, oops, you got one over on us. Good job. That's never been the mindset I've gotten about AWS's philosophy for pricing. The better example that I love because no one took it seriously, was a few years before that when there was a LightSail bug in the billing system, and it made the papers because people suddenly found that for their LightSail instance, they were getting predicted bills of $4 billion.And the way I see it, you really only had to make that work once and then you've made your numbers for the year, so why not? Someone's going to pay for it, probably. But that was such out-of-the-world numbers that no one saw that and ever thought it was anything other than a bug. It's the small pernicious things that creep in. Because the billing system is vast; I had no idea when I started working with AWS bills just how complicated it really was.James: Yeah, I remember both of those, and there's something in there that you touched on that I think is really important. That's something that I realized pretty early on at Amazon, and it's why customer obsession is our flagship leadership principle. It's not because it's love and butterflies and unicorns; customer obsession is key to us because that's how you build a long-term sustainable business is your customers depend on you. And it drives how we think about everything that we do. And in the billing space, small errors, even if there are small errors in the customer's favor, slowly erode that trust.So, we take any kind of error really seriously and we try to figure out how we can make sure that it doesn't happen again. We don't always get that right. As you said, we've built an enormous, super-complex business to growing really quickly, and really quick growth like that always acts as kind of a multiplier on top of complexity. And on the pricing points, we're managing millions of pricing points at the moment.And our tools that we use internally, there's always room for improvement. It's a huge area of focus for us. We're in the beginning of looking at applying things like formal methods to make sure that we can make very hard guarantees about the correctness of some of those. But at the end of the day, people are plugging numbers in and you need as many belts and braces as possible to make sure that you don't make mistakes there.Corey: One of the things that struck me by surprise when I first started getting deep into this space was the fact that the finalized bill was—what does it mean to have this be ‘finalized?' It can hit the Cost and Usage Report in an S3 bucket and it can change retroactively after the month closed periodically. And that's when I started to have an inkling of a few things: Not just the sheer scale and complexity inherent to something like the billing system that touches everything, but the sheer data retention stories where you clearly have to be able to go back and reconstruct a bill from the raw data years ago. And I know what the output of all of those things are in the form of Cost and Usage Reports and the billing data from our client accounts—which is the single largest expense in all of our AWS accounts; we spent thousands and thousands and thousands of dollars a year just on storing all of that data, let alone the processing piece of it—the sheer scale is staggering. I used to wonder why does it take you a day to record me using something to it's showing up in the bill? And the more I learned the more it became a how can you do that in only a day?James: Yes, the scale is actually mind-boggling. I'm pretty sure that the core of our billing system is—I'm reasonably confident it's the largest or one of the largest data processing systems on the planet. I remember pretty early on when I joined Commerce Platform and was still starting to wrap my head around some of these things, Googling the definition of quadrillion because we measured the number of metering events, which is how we record usage in services, on a daily basis in the quadrillions, which is a billion billions. So, it's just an absolutely staggering number. And so, the scale here is just out of this world.That's saying something because it's not like other services across AWS are small in their own right. But I'm still reasonably sure that being one of a handful of services that is kind of at the nexus of AWS and kind of deals with the aggregate of AWS's scale, this is probably one of the biggest systems on the planet. And that shows up in all sorts of places. You start with that input, just the sheer volume of metering events, but that has to produce as an output pretty fine-grained line item detailed information, which ultimately rolls up into the total that a customer will see in their bill. But we have a number of different systems further down the pipeline that try to do things like analyze your usage, make sensible recommendations, look for opportunities to improve your efficiency, give you the ability to slice and dice your data and allocate it out to different parts of your business in whatever way it makes sense for your business. And so, those systems have to deal with anywhere from millions to billions to recently, we were talking about trillions of data points themselves. And so, I was tangentially aware of some of the scale of this, but being in the thick of it having joined the team really just does underscore just how vast the systems are.Corey: I think it's, on some level, more than a little unfortunate that that story isn't being more widely told, more frequently. Because when Commerce Platform has job postings that are available on the website, you read it and it's very vague. It doesn't tend to give hard numbers about a lot of these things, and people who don't play in these waters can easily be forgiven for thinking the way that you folks do your job is you fire up one of those 24 terabyte of RAM instances that—you know, those monstrous things that you folks offer—and what do you do next? Well, Microsoft Excel. We have a special high memory version that we've done some horse-trading with our friends over at Microsoft for.It's, yeah, you're several steps beyond that, at this point. It's a challenging problem that every one of your customers has to deal with, on some level, as well. But we're only dealing with the output of a lot of the processing that you folks are doing first.James: You're exactly right. And a big focus for some of my teams is figuring out how to help customers deal with that output. Because even if you're talking about couple of orders of magnitude reduction, you're still talking about very large numbers there. So, to help customers make sense of that, we have a range of tools that exist, we're investing in.There's another dimension of complexity in the space that I think is one that's also very easy to miss. And I think of it as arbitrary complexity. And it's arbitrary because some of the rules that we have to box within here are driven by legislative changes. As you operate more and more countries around the world, you want to make sure that we're tax compliant, that we help our customers be tax compliant. Those rules evolve pretty rapidly, and Country A may sit next to Country B, but that doesn't mean that they're talking to one another. They've all got their own ideas. They're trying to accomplish r—00:22:47Corey: A company is picking up and relocating from India to Germany. How do we—James: Exactly.Corey: —change that on the AWS side and the rest? And it's, “Hoo boy, have you considered burning it all down and filing an insurance claim to start over?” And, like, there's a lot of complexity buried underneath that that just doesn't rise to the notice of 99% of your customers.James: And the fact that it doesn't rise to the notice is something that we strive for. Like, these shouldn't be things that customers have to worry about. Because it really is about clearing away the things that, as far as possible, you don't want to have to spend time thinking about so that you can focus on the thing that your business does that differentiates you. It's getting rid of that undifferentiated heavy lifting. And there's a ton of that in this space, and if you're blissfully unaware of it, then hopefully that means that we're doing our job.Corey: What I'm, I think, the most surprised about, and I have been for a long time. And please don't take this as an insult to various other folks—engineers, the rest, not just in other parts of AWS but throughout the other industry—but talking to the people who work within Commerce Platform has always been just a fantastic experience. The caliber of people that you have managed to attract and largely retain—we don't own people, they do matriculate out eventually—but the caliber of people that you've retained on your teams has just been out of this world. And at first, I wondered, why are these awesome people working on something as boring and prosaic as billing? And then I started learning a little bit more as I went, and, “Oh, wow. How did they learn all the stuff that they have to hold in their head in tension at once to be able to build things like this?” It's incredibly inspiring just watching the caliber of the people that you've been able to bring in.James: I've been really, really excited joining this team, as I've gotten other folks on the team because there's some super-smart people here. But what's really jumped out to me is how committed the team is. This is, for the most part, a team that has been in the space for many years. Many of them have—we talk about boomerangs, folks who live AWS, go spend some time somewhere else and come back and there's a surprisingly high proportion of folks in Commerce Platform who have spent time somewhere else and then come back because they enjoy the space, they find that challenging, folks are attracted to the ability to have an impact because it is so foundational. But yeah, there's a super-committed core to this team. And I really enjoy working with teams where you've got that because then you really can take the long view and build something great. And I think we have tons of opportunities to do that here.Corey: It sounds ridiculous, but I've reached out to team members before to explain two-cent variances in my bill, and never once have I been confronted with a, “It's two cents. What do you care?” They understand the requirement that these things be accurate, not just, “Eh, take our word for it.” And also, frankly, they understand that two cents on a $20 bill looks a little different on a $20 million bill. So yeah, let us figure out if this is systemic or something I have managed to break.It turns out the Cost and Usage Report processing systems don't love it when there's a cost allocation tag whose name contains an emoji. Who knew? It's the little things in life that just have this fun way of breaking when you least expect it.James: They're also a surprisingly interesting problem. So like, it turns out something as simple as rounding numbers consistently across a distributed system at this scale, is a non-trivial problem. And if you don't, then you do get small seventh or eighth decimal place differences that add up to something that then shows up as a two-cent difference somewhere. And so, there's some really, really interesting problems in the space. And I think the team often takes these kinds of things as a personal challenge. It should be correct, and it's not, so we should go make sure it is correct. The interesting problems abound here, but at the end of the day, it's the kind of thing that any engineering team wants to go and make sure it's correct because they know that it can be.Corey: This episode is sponsored in parts by our friend EnterpriseDB. EnterpriseDB has been powering enterprise applications with PostgreSQL for 15 years. And now EnterpriseDB has you covered wherever you deploy PostgreSQL on premises, private cloud, and they just announced a fully managed service on AWS and Azure called BigAnimal, all one word. Don't leave managing your database to your cloud vendor because they're too busy launching another half dozen manage databases to focus on any one of them that they didn't build themselves. Instead, work with the experts over at EnterpriseDB. They can save you time and money, they can even help you migrate legacy applications, including Oracle, to the cloud.To learn more, try BigAnimal for free. Go to biganimal.com/snark, and tell them Corey sent you.Corey: On the one hand, I love people who just round and estimate—we all do that, let's be clear; I sit there and I back-of-the-envelope everything first. But then I look at some of your pricing pages and I count the digits after the zeros. Like, you're talking about trillionths of a dollar on some of your pricing points. And you add it up in the course of a given hour and it's like, oh, it's $250 a month, most months. And it's you work backwards to way more decimal places of precision than is required, sometimes.I'm also a personal fan of the bill that counts, for example, number of Route 53 zones. Great. And it counts them to four decimal places of precision. Like, I don't even know what half of it Route 53 zone is at this point, let alone something to, like, ah the 1,000th of the zone is going to cause this. It's all an artifact of what the underlying systems are.Can you by any chance shed a little light on what the evolution of those systems has been over a period of time? I have to imagine that anything you built in the early days, 16 years ago or so from the time of this recording when S3 launched to general availability, you probably didn't have to worry about this scope and scale of what you do, now. In fact, I suspect if you tried to funnel this volume through S3 back then, the whole thing would have collapsed under its own weight. What's evolved over the time that you had the billing system there? Because changes come slowly to your environment. And frankly, I appreciate that as a customer. I don't like surprising people in finance.James: Yeah, you're totally right. So, I joined the EC2 team as an engineer myself, some 16 years ago, and the very first thing that I did was our billing integration. And so, my relationship with the Commerce Platform organization—what was the billing team way back when—it goes back over my entire career at AWS. And at the time, the billing team was similar, you know, [unintelligible 00:28:34] eight people. And that was everything. There was none of the scale and complexity; it was all one system.And much like many of our biggest, oldest services—EC2 is very similar, S3 is as well—there's been significant growth over the last decade-and-a-half. A lot of that growth has been rapid, and rapid growth presents its own challenges. And you live with decisions that you make early on that you didn't realize were significant decisions that have pretty deep implications 15 years later. We're still working through some of those; they present their own challenges. Evolving an existing system to keep up with the growth of business and a customer base that's as varied and complex as ours is always challenging.And also harder but I also think more fun than a clean sheet redo at this point. Like, that's a great thought exercise for, well, if we got to do this again today, what would we do now that we've learned so much over the last 15 years? But there's this—I find it personally fascinating challenge with evolving a live system where it's like, “No, no, like, things exist, so how do we go from there to where we want to be next?”Corey: Turn the billing system off for 18 months, rebuild—James: Yeah. [laugh].Corey: The whole thing from first principles. Light it up. I'm sure you'd have a much better billing system, and also not a company left anymore.James: [laugh]. Exactly, exactly. I've always enjoyed that challenge. You know, even prior to AWS, my previous careers have involved similar kinds of constraints where you've got a live system, or you've got an existing—in the one case, it was an existing SDK that was deployed to tens of thousands of customers around the world, and so backwards compatibility was something that I spent the first five years of my career thinking about it way more detail than I think most people do. And it's a very similar mindset. And I enjoy that challenge. I enjoy that: How do I evolve from here to there without breaking customers along the way?And that's something that we take pretty seriously across AWS. I think SimpleDB is the poster child for we never turn things off. But that applies equally to the services that are maybe less visible to customers, and billing is definitely one of them. Like, we don't get to switch stuff off. We don't get to throw things away and start again. It's this constant state of evolution.Corey: So, let's say that I were to find a way to route data through a series of two Managed NAT Gateways and then egress to internet, and the sheer density of the expense of that traffic tears a hole in the fabric of space-time, it goes back 15 years ago, and you can make a single change to how the billing system was built. What would it be? What pisses you off the most about the current constraints that you have to work within or around?James: I think one of the biggest challenges we've got, actually, is the concept of an account. Because an account means half-a-dozen different things. And way back, when it seemed like a great idea, you just needed an account; an account was your customer, and it was the same thing as the boundary that you put all your resources inside. And of course, it's the same thing that you're going to roll all of your usage up and issue a bill against. And that has been one of the areas that's seen the most evolution and probably still has a pretty long way to go.And what's interesting about that is, that's probably something we could have seen coming because we watched the retail business go through, kind of, the same evolution because they started with, well, a customer is a customer is a customer and had to evolve to support the concept of sellers and partners. And then users are different than customers, and you want to log in and that's a different thing. So, we saw that kind of bifurcation of a single entity into a wide range of different related but separate entities, and I think if we'd looked at that, you know, thought out 15 years, then yeah, we could probably have learned something from that. But at the same time, when AWS first kicked off, we had wild ambitions for it, but there was no guarantee that it was going to be the monster that it is today. So, I'm always a little bit reluctant to—like, it's a great thought exercise, but it's easy to end up second-guessing a pretty successful 15 years, so I'm always a little bit careful to walk that line. But I think account is one of the things that we would probably go back and think about a little bit more.Corey: I want to be very clear with this next question that it is intentionally setting up a question I suspect you get a lot. It does not mirror my own thinking on the matter even slightly, but I get a version of it myself all the time. “AWS bills, that sounds boring as hell. Why would you choose to work on such a thing?” Now, I have a laundry list of answers to that aren't nearly as interesting as I suspect yours are going to be. What makes working on this problem space interesting to you?James: There's a bunch of different things. So, first and foremost, the scale that we're talking about here is absolutely mind-blowing. And for any engineer who wants to get stuck into problems that deal with mind-blowingly large volumes of data, incredibly rich dimensions, problems where, honestly, applying techniques like statistical reasoning or machine learning is really the only way to chip away at it, that exists in spades in the space. It's not always immediately obvious, and I think from the outside, it's easy to assume this is actually pretty simple. So, the scale is a huge part of that.Corey: “Oh, petabytes. How quaint.”James: [laugh]. Exactly. Exactly I mean, it's mind-blowing every time I see some of the numbers in various parts of the Commerce Platform space. I talked about quadrillions earlier. Trillions is a pretty common unit of measure.The complexity that I talked about earlier, that's a result of external environments is another one. So, imposed by external entities, whether it's a government or a tax authority somewhere, or a business requirement from customers, or ourselves. I enjoy those as well. Those are different kinds of challenge. They really keep you on your toes.I enjoy thinking of them as an engineering problem, like, how do I get in front of them? And that's something we spend a lot of time doing in Commerce Platform. And when we get it right, customers are just unaware of it. And then the third one is, I personally am always attracted to the opportunity to have an impact. And this is a space where we get to hopefully positively impact every single customer every day. And that, to me is pretty fulfilling.Those are kind of the three standout reasons why I think this is actually a super-exciting space. And I think it's often an underestimated space. I think once folks join the team and sort of start to dig in, I've never heard anybody after they've joined, telling me that what they're doing is boring. Challenging, yes. Is frustrating, sometimes. Hard, absolutely, but boring never comes up.Corey: There's almost no service, other than IAM, that I can think of that impacts every customer simultaneously. And it's easy for me to sit in the cheap seats and say, “Oh, you should change this,” or, “You should change that.” But every change you have is so massive in scale that it's going to break a whole bunch of companies' automations around the bill processing in different ways. You have an entire category of user persona who is used to clicking a certain button in this certain place in the console to generate the report every month, and if that button moves or changes color, or has a different font, suddenly that renders their documentation invalid, and they're scrambling because it's not their core competency—nor should it be—and every change you make is so constricted, just based upon all the different concerns that you've got to be juggling with. How do you get anything done at all? I find that to be one of the most impressive aspects about your organization, bar none.James: Yeah, I'm not going to lie and say that it isn't a challenge, but a lot of it comes down to the talent that we have on the team. We have a super-motivated, super-smart, super-engaged team, and we spend a lot of time figuring out how to make sure that we can keep moving, keep up with the business, keep up with a world that's getting more complicated [laugh] with every passing day. So, you've kind of hit on one of the core challenges there, which is, how do we keep up with all of those different dimensions that are demanding an increasing amount of engineering and new support and new investment from us, while we keep those customers happy?And I think you touched on something else a little bit indirectly there, which is, a lot of our customers are actually pretty technical across AWS. The customers that Commerce Platform supports, are often the least technical of our customers, and so often need the most help understanding why things are the way they are, where the constraints are.Corey: “A big bill from Amazon. How many books did you people buy last month?”—James: [laugh]. Exactly.Corey: —is still very much level of understanding in some cases. And it's not because they're dumb; far from it. It's just, imagine that some people view there as being more to life than understanding the nuances and intricacies of cloud computing. How dare they?James: Exactly. Who would have thought?Corey: So, as you look now over all of your domain, such as it is, what sucks the most? What are you looking to fix as far as impactful changes that the rest of the world might experience? Because I'm not going to accept one of those questions like, “Oh, yeah, on the back-end, we have this storage subsystem for a tertiary thing that just annoys me because it wakes us up once in a whi”—no, no, I want something customer-facing. What's the painful thing you're looking at fixing next?James: I don't like surprising customers. And free tier is, sort of, one of those buckets of surprises, but there are others. Another one that's pretty squarely in my sights is, whether we like it or not, customer accounts get compromised. Usually, it's a password got reused somewhere or was accidentally committed into a GitHub repository somewhere.And we have pretty established, pretty effective mechanisms for finding all of those, we'll scan for passwords and credentials, and alert customers to those, and help them correct that pretty quickly. We're also actually pretty good at detecting when an account does start to do something that suggests that it's been compromised. Usually, the first thing that a compromised account starts to do is cryptocurrency mining. We're pretty quick to catch those; we catch those within a matter of hours, much faster most days.What we haven't really cracked and where I'm focused at the moment is getting back to the customer in a way that's effective. And by that I mean specifically, we detect an account compromised super-quickly, we reach out automatically. And so, you know, a customer has got some kind of contact from us usually within a couple of hours. It's not having the effect that we need it to. Customers are still being surprised a month later by a large bill. And so, we're digging into how much of that is because they never saw the contact, they didn't know what to do with the contact.Corey: It got buried with all the other, “Hey, we saw you spun up an S3 bucket. Have you heard of what S3 is?” Again, that's all valuable, but you have 300-some-odd services. If you start doing that for every service, you're going to hit mail sending limits for Gmail.James: Exactly. It's not just enough that we detect those and notify customers; we have to reduce the size of the surprise. It's one thing to spend 100 bucks a month on average, and then suddenly find that your spend has jumped $250 because you reused the password somewhere and somebody got ahold of it and it's cryptocurrency-mining your account. It's a whole different ballgame to spend 100 bucks a month and then at the end of the month discover that your bill is suddenly $2,000 or $20,000. And so, that's something that I really wanted to make some progress on this year. Corey: I've really enjoyed our conversation. If people want to learn more about how you view these things, how you're approaching some of these problems, or potentially are just the right kind of warped to consider joining up, where's the best place for them to go?James: They should drop me an email at jamesg@amazon.com. That is the most direct way to get hold of me, and I promise I will get back to you. I try to stay on top of my email as much as possible. But that will come straight to me, and I'm always happy to talk to folks about the space, talk to folks about opportunities in this team, opportunities across AWS, or just hear what's not working, make sure that it's something that we're aware of and looking at.Corey: Throughout Amazon, but particularly within Commerce Platform, I've always appreciated the response of, whenever I report something, no matter how ridiculous it is—and I assure you there's an awful lot of ridiculousness in my bug reports—the response has always been the same: “Tell me more. Help me understand what it is you're trying to achieve—even if it is ridiculous—so we can look at this and see what is actually going on.” Every Amazonian team has been great about that or you're not at Amazon very long, but you folks have taken that to an otherworldly level. I just want to thank you for doing that.James: I appreciate you for calling that out. We try, you know, we really do. We take listening to our customers very seriously because, at the end of the day, that's what makes us better, and that's how we make sure we're in it for the long haul.Corey: Thanks once again for being so generous with your time. I really appreciate it.James: Yeah, thanks for having me on. I've enjoyed it.Corey: James Greenfield, VP of Commerce Platform at AWS. I'm Cloud Economist Corey Quinn, and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice along with an angry comment—possibly on YouTube as well—about how you aren't actually giving this five-stars at all; you have taken three trillions of a star off of the rating.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.
About AmyAmy Tobey has worked in tech for more than 20 years at companies of every size, working with everything from kernel code to user interfaces. These days she spends her time building an innovative Site Reliability Engineering program at Equinix, where she is a principal engineer. When she's not working, she can be found with her nose in a book, watching anime with her son, making noise with electronics, or doing yoga poses in the sun.Links Referenced: Equinix Metal: https://metal.equinix.com Personal Twitter: https://twitter.com/MissAmyTobey Personal Blog: https://tobert.github.io/ TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: This episode is sponsored in part by our friends at Vultr. Optimized cloud compute plans have landed at Vultr to deliver lightning-fast processing power, courtesy of third-gen AMD EPYC processors without the IO or hardware limitations of a traditional multi-tenant cloud server. Starting at just 28 bucks a month, users can deploy general-purpose, CPU, memory, or storage optimized cloud instances in more than 20 locations across five continents. Without looking, I know that once again, Antarctica has gotten the short end of the stick. Launch your Vultr optimized compute instance in 60 seconds or less on your choice of included operating systems, or bring your own. It's time to ditch convoluted and unpredictable giant tech company billing practices and say goodbye to noisy neighbors and egregious egress forever. Vultr delivers the power of the cloud with none of the bloat. “Screaming in the Cloud” listeners can try Vultr for free today with a $150 in credit when they visit getvultr.com/screaming. That's G-E-T-V-U-L-T-R dot com slash screaming. My thanks to them for sponsoring this ridiculous podcast.Corey: Finding skilled DevOps engineers is a pain in the neck! And if you need to deploy a secure and compliant application to AWS, forgettaboutit! But that's where DuploCloud can help. Their comprehensive no-code/low-code software platform guarantees a secure and compliant infrastructure in as little as two weeks, while automating the full DevSecOps lifestyle. Get started with DevOps-as-a-Service from DuploCloud so that your cloud configurations are done right the first time. Tell them I sent you and your first two months are free. To learn more visit: snark.cloud/duplo. Thats's snark.cloud/D-U-P-L-O-C-L-O-U-D.Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. Every once in a while I catch up with someone that it feels like I've known for ages, and I realize somehow I have never been able to line up getting them on this show as a guest. Today is just one of those days. And my guest is Amy Tobey who has been someone I've been talking to for ages, even in the before-times, if you can remember such a thing. Today, she's a Senior Principal Engineer at Equinix. Amy, thank you for finally giving in to my endless wheedling.Amy: Thanks for having me. You mentioned the before-times. Like, I remember it was, like, right before the pandemic we had beers in San Francisco wasn't it? There was Ian there—Corey: Yeah, I—Amy: —and a couple other people. It was a really great time. And then—Corey: I vaguely remember beer. Yeah. And then—Amy: And then the world ended.Corey: Oh, my God. Yes. It's still March of 2020, right?Amy: As far as I know. Like, I haven't checked in a couple years.Corey: So, you do an awful lot. And it's always a difficult question to ask someone, so can you encapsulate your entire existence in a paragraph? It's—Amy: [sigh].Corey: —awful, so I'd like to give a bit more structure to it. Let's start with the introduction: You are a Senior Principal Engineer. We know it's high level because of all the adjectives that get put in there, and none of those adjectives are ‘associate' or ‘beginner' or ‘junior,' or all the other diminutives that companies like to play games with to justify paying people less. And you're at Equinix, which is a company that is a bit unlike most of the, shall we say, traditional cloud providers. What do you do over there and both as a company, as a person?Amy: So, as a company Equinix, what most people know about is that we have a whole bunch of data centers all over the world. I think we have the most of any company. And what we do is we lease out space in that data center, and then we have a number of other products that people don't know as well, which one is Equinix Metal, which is what I specifically work on, where we rent you bare-metal servers. None of that fancy stuff that you get any other clouds on top of it, there's things you can get that are… partner things that you can add-on, like, you know, storage and other things like that, but we just deliver you bare-metal servers with really great networking. So, what I work on is the reliability of that whole system. All of the things that go into provisioning the servers, making them come up, making sure that they get delivered to the server, make sure the API works right, all of that stuff.Corey: So, you're on the Equinix cloud side of the world more so than you are on the building data centers by the sweat of your brow, as they say?Amy: Correct. Yeah, yeah. Software side.Corey: Excellent. I spent some time in data centers in the early part of my career before cloud ate that. That was sort of cotemporaneous with the discovery that I'm the hardware destruction bunny, and I should go to great pains to keep my aura from anything expensive and important, like, you know, the SAN. So—Amy: Right, yeah.Corey: Companies moving out of data centers, and me getting out was a great thing.Amy: But the thing about SANs though, is, like, it might not be you. They're just kind of cursed from the start, right? They just always were kind of fussy and easy to break.Corey: Oh, yeah. I used to think—and I kid you not—that I had a limited upside to my career in tech because I sometimes got sloppy and I was fairly slow at crimping ethernet cables.Amy: [laugh].Corey: That is very similar to growing up in third grade when it became apparent that I was going to have problems in my career because my handwriting was sloppy. Yeah, it turns out the future doesn't look like we predicted it would.Amy: Oh, gosh. Are we going to talk about, like, neurological development now or… [laugh] okay, that's a thing I struggle with, too right, is I started typing as soon as they would let—in fact, before they would let me. I remember in high school, I had teachers who would grade me down for typing a paper out. They want me to handwrite it and I would go, “Cool. Go ahead and take a grade off because if I handwrite it, you're going to take two grades off my handwriting, so I'm cool with this deal.”Corey: Yeah, it was pretty easy early on. I don't know when the actual shift was, but it became more and more apparent that more and more things are moving towards a world where you could type. And I was almost five when I started working on that stuff, and that really wound up changing a lot of aspects of how I started seeing things. One thing I think you're probably fairly well known for is incidents. I want to be clear when I say that you are not the root cause as—“So, why are things broken?” “It's Amy again. What's she gotten into this time?” Great.Amy: [laugh]. But it does happen, but not all the time.Corey: Exa—it's a learning experience.Amy: Right.Corey: You've also been deeply involved with SREcon and a number of—a lot of aspects of what I will term—and please don't yell at me for this—SRE culture—Amy: Yeah.Corey: Which is sometimes a challenging thing to wind up describing or putting a definition around. The one that I've always been somewhat partial to is, “SRE is DevOps, except you worked at Google for a while.” I don't know how necessarily accurate that is, but it does rile people up.Amy: Yeah, it does. Dave Stanke actually did a really great talk at SREcon San Francisco just a couple weeks ago, about the DORA report. And the new DORA report, they split SRE out into its own function and kind of is pushing against that old model, which actually comes from Liz Fong-Jones—I think it's from her, or older—about, like, class SRE implements DevOps, which is kind of this idea that, like, SREs make DevOps happen. Things have evolved, right, since then. Things have evolved since Google released those books, and we're all just figured out what works and what doesn't a little bit.And so, it's not that we're implementing DevOps so much. In fact, it's that ops stuff that kind of holds us back from the really high impact work that SREs, I think, should be doing, that aren't just, like, fixing the problems, the symptoms down at the bottom layer, right? Like what we did as sysadmins 20 years ago. You know, we'd go and a lot of people are SREs that came out of the sysadmin world and still think in that mode, where it's like, “Well, I set up the systems, and when things break, I go and I fix them.” And, “Why did the developers keep writing crappy code? Why do I have to always getting up in the middle of the night because this thing crashed?”And it turns out that the work we need to do to make things more reliable, there's a ceiling to how far away the platform can take us, right? Like, we can have the best platform in the world with redundancy, and, you know, nine-way replicated data storage and all this crazy stuff, and still if we put crappy software on top, it's going to be unreliable. So, how do we make less crappy software? And for most of my career, people would be, like, “Well, you should test it.” And so, we started doing that, and we still have crappy software, so what's going on here? We still have incidents.So, we write more tests, and we still have incidents. We had a QA group, we still have incidents. We send the developers to training, and we still have incidents. So like, what is the thing we need to do to make things more reliable? And it turns out, most of it is culture work.Corey: My perspective on this stems from being a grumpy old sysadmin. And at some point, I started calling myself a systems engineer or DevOps or production engineer, or SRE. It was all from my point of view, the same job, but you know, if you call yourself a sysadmin, you're just asking for a 40% pay cut off the top.Amy: [laugh].Corey: But I still tended to view the world through that lens. I tended to be very good at Linux systems internals, for example, understanding system calls and the rest, but increasingly, as the DevOps wave or SRE wave, or Google-isation of the internet wound up being more and more of a thing, I found myself increasingly in job interviews, where, “Great, now, can you go wind up implementing a sorting algorithm on the whiteboard?” “What on earth? No.” Like, my lingua franca is shitty Bash, and no one tends to write that without a bunch of tab completions and quick checking with manpages—die.net or whatnot—on the fly as you go down that path.And it was awful, and I felt… like my skill set was increasingly eroding. And it wasn't honestly until I started this place where I really got into writing a fair bit of code to do different things because it felt like an orthogonal skill set, but the fullness of time, it seems like it's not. And it's a reskilling. And it made me wonder, does this mean that the areas of technology that I focused on early in my career, was that all a waste? And the answer is not really. Sometimes, sure, in that I don't spend nearly as much time worrying about inodes—for example—as I once did. But every once in a while, I'll run into something and I looked like a wizard from the future, but instead, I'm a wizard from the past.Amy: Yeah, I find that a lot in my work, now. Sometimes things I did 20 years ago, come back, and it's like, oh, yeah, I remember I did all that threading work in 2002 in Perl, and I learned everything the very, very, very hard way. And then, you know, this January, did some threading work to fix some stability issues, and all of it came flooding back, right? Just that the experiences really, more than the code or the learning or the text and stuff; more just the, like, this feels like threads [BLEEP]-ery. Is a diagnostic thing that sometimes we have to say.And then people are like, “Can you prove it?” And I'm like, “Not really,” because it's literally thread [BLEEP]-ery. Like, the definition of it is that there's weird stuff happening that we can't figure out why it's happening. There's something acting in the system that isn't synchronized, that isn't connected to other things, that's happening out of order from what we expect, and if we had a clear signal, we would just fix it, but we don't. We just have, like, weird stuff happening over here and then over there and over there and over there.And, like, that tells me there's just something happening at that layer and then have to go and dig into that right, and like, just basically charge through. My colleagues are like, “Well, maybe you should look at this, and go look at the database,” the things that they're used to looking at and that their experiences inform, whereas then I bring that ancient toiling through the threading mines experiences back and go, “Oh, yeah. So, let's go find where this is happening, where people are doing dangerous things with threads, and see if we can spot something.” But that came from that experience.Corey: And there's so much that just repeats itself. And history rhymes. The challenge is that, do you have 20 years of experience, or do you have one year of experience repeated 20 times? And as the tide rises, doing the same task by hand, it really is just a matter of time before your full-time job winds up being something a piece of software does. An easy example is, “Oh, what's your job?” “I manually place containers onto specific hosts.” “Well, I've got news for you, and you're not going to like it at all.”Amy: Yeah, yeah. I think that we share a little bit. I'm allergic to repeated work. I don't know if allergic is the right word, but you know, if I sit and I do something once, fine. Like, I'll just crank it out, you know, it's this form, or it's a datafile I got to write and I'll—fine I'll type it in and do the manual labor.The second time, the difficulty goes up by ten, right? Like, just mentally, just to do it, be like, I've already done this once. Doing it again is anathema to everything that I am. And then sometimes I'll get through it, but after that, like, writing a program is so much easier because it's like exponential, almost, growth in difficulty. You know, the third time I have to do the same thing that's like just typing the same stuff—like, look over here, read this thing and type it over here—I'm out; I can't do it. You know, I got to find a way to automate. And I don't know, maybe normal people aren't driven to live this way, but it's kept me from getting stuck in those spots, too.Corey: It was weird because I spent a lot of time as a consultant going from place to place and it led to some weird changes. For example, “Oh, thank God, I don't have to think about that whole messaging queue thing.” Sure enough, next engagement, it's message queue time. Fantastic. I found that repeating myself drove me nuts, but you also have to be very sensitive not to wind up, you know, stealing IP from the people that you're working with.Amy: Right.Corey: But what I loved about the sysadmin side of the world is that the vast majority of stuff that I've taken with me, lives in my shell config. And what I mean by that is I'm not—there's nothing in there is proprietary, but when you have a weird problem with trying to figure out the best way to figure out which Ruby process is stealing all the CPU, great, turns out that you can chain seven or eight different shell commands together through a bunch of pipes. I don't want to remember that forever. So, that's the sort of thing I would wind up committing as I learned it. I don't remember what company I picked that up at, but it was one of those things that was super helpful.I have a sarcastic—it's a one-liner, except no sane editor setting is going to show it in any less than three—of a whole bunch of Perl, piped into du, piped into the rest, that tells you one of the largest consumers of files in a given part of the system. And it rates them with stars and it winds up doing some neat stuff. I would never sit down and reinvent something like that today, but the fact that it's there means that I can do all kinds of neat tricks when I need to. It's making sure that as you move through your career, on some level, you're picking up skills that are repeatable and applicable beyond one company.Amy: Skills and tooling—Corey: Yeah.Amy: —right? Like, you just described the tool. Another SREcon talk was John Allspaw and Dr. Richard Cook talking about above the line; below the line. And they started with these metaphors about tools, right, showing all the different kinds of hammers.And if you're a blacksmith, a lot of times you craft specialized hammers for very specific jobs. And that's one of the properties of a tool that they were trying to get people to think about, right, is that tools get crafted to the job. And what you just described as a bespoke tool that you had created on the fly, that kind of floated under the radar of intellectual property. [laugh].So, let's not tell the security or IP people right? Like, because there's probably billions and billions of dollars of technically, like, made-up IP value—I'm doing air quotes with my fingers—you know, that's just basically people's shell profiles. And my God, the Emacs automation that people have done. If you've ever really seen somebody who's amazing at Emacs and is 10, 20, 30, maybe 40 years of experience encoded in their emacs settings, it's a wonder to behold. Like, I look at it and I go, “Man, I wish I could do that.”It's like listening to a really great guitar player and be like, “Wow, I wish I could play like them.” You see them just flying through stuff. But all that IP in there is both that person's collection of wisdom and experience and working with that code, but also encodes that stuff like you described, right? It's just all these little systems tricks and little fiddly commands and things we don't want to remember and so we encode them into our toolset.Corey: Oh, yeah. Anything I wound up taking, I always would share it with people internally, too. I'd mention, “Yeah, I'm keeping this in my shell files.” Because I disclosed it, which solves a lot of the problem. And also, none of it was even close to proprietary or anything like that. I'm sorry, but the way that you wind up figuring out how much of a disk is being eaten up and where in a more pleasing way, is not a competitive advantage. It just isn't.Amy: It isn't to you or me, but, you know, back in the beginning of our careers, people thought it was worth money and should be proprietary. You know, like, oh, that disk-checking script as a competitive advantage for our company because there are only a few of us doing this work. Like, it was actually being able to, like, manage your—[laugh] actually manage your servers was a competitive advantage. Now, it's kind of commodity.Corey: Let's also be clear that the world has moved on. I wound up buying a DaisyDisk a while back for Mac, which I love. It is a fantastic, pretty effective, “Where's all the stuff on your disk going?” And it does a scan and you can drive and collect things and delete them when trying to clean things out. I was using it the other day, so it's top of mind at the moment.But it's way more polished than that crappy Perl three-liner. And I see both sides, truly I do. The trick also, for those wondering [unintelligible 00:15:45], like, “Where is the line?” It's super easy. Disclose it, what you're doing, in those scenarios in the event someone is no because they believe that finding the right man page section for something is somehow proprietary.Great. When you go home that evening in a completely separate environment, build it yourself from scratch to solve the problem, reimplement it and save that. And you're done. There are lots of ways to do this. Don't steal from your employer, but your employer employs you; they don't own you and the way that you think about these problems.Every person I've met who has had a career that's longer than 20 minutes has a giant doc somewhere on some system of all of the scripts that they wound up putting together, all of the one-liners, the notes on, “Next time you see this, this is the thing to check.”Amy: Yeah, the cheat sheet or the notebook with all the little commands, or again the Emacs config, sometimes for some people, or shell profiles. Yeah.Corey: Here's the awk one-liner that I put that automatically spits out from an Apache log file what—the httpd log file that just tells me what are the most frequent talkers, and what are the—Amy: You should probably let go of that one. You know, like, I think that one's lifetime is kind of past, Corey. Maybe you—Corey: I just have to get it working with Nginx, and we're good to go.Amy: Oh, yeah, there you go. [laugh].Corey: Or S3 access logs. Perish the thought. But yeah, like, what are the five most high-volume talkers, and what are those relative to each other? Huh, that one thing seems super crappy and it's coming from Russia. But that's—hmm, one starts to wonder; maybe it's time to dig back in.So, one of the things that I have found is that a lot of the people talking about SRE seem to have descended from an ivory tower somewhere. And they're talking about how some of the best-in-class companies out there, renowned for their technical cultures—at least externally—are doing these things. But there's a lot more folks who are not there. And honestly, I consider myself one of those people who is not there. I was a competent engineer, but never a terrific one.And looking at the way this was described, I often came away thinking, “Okay, it was the purpose of this conference talk just to reinforce how smart people are, and how I'm not,” and/or, “There are the 18 cultural changes you need to make to your company, and then you can do something kind of like we were just talking about on stage.” It feels like there's a combination of problems here. One is making this stuff more accessible to folks who are not themselves in those environments, and two, how to drive cultural change as an individual contributor if that's even possible. And I'm going to go out on a limb and guess you have thoughts on both aspects of that, and probably some more hit me, please.Amy: So, the ivory tower, right. Let's just be straight up, like, the ivory tower is Google. I mean, that's where it started. And we get it from the other large companies that, you know, want to do conference talks about what this stuff means and what it does. What I've kind of come around to in the last couple of years is that those talks don't really reach the vast majority of engineers, they don't really apply to a large swath of the enterprise especially, which is, like, where a lot of the—the bulk of our industry sits, right? We spend a lot of time talking about the darlings out here on the West Coast in high tech culture and startups and so on.But, like, we were talking about before we started the show, right, like, the interior of even just America, is filled with all these, like, insurance and banks and all of these companies that are cranking out tons of code and servers and stuff, and they're trying to figure out the same problems. But they're structured in companies where their tech arm is still, in most cases, considered a cost center, often is bundled under finance, for—that's a whole show of itself about that historical blunder. And so, the tech culture is tend to be very, very different from what we experience in—what do we call it anymore? Like, I don't even want to say West Coast anymore because we've gone remote, but, like, high tech culture we'll say. And so, like, thinking about how to make SRE and all this stuff more accessible comes down to, like, thinking about who those engineers are that are sitting at the computers, writing all the code that runs our banks, all the code that makes sure that—I'm trying to think of examples that are more enterprise-y right?Or shoot buying clothes online. You go to Macy's for example. They have a whole bunch of servers that run their online store and stuff. They have internal IT-ish people who keep all this stuff running and write that code and probably integrating open-source stuff much like we all do. But when you go to try to put in a reliability program that's based on the current SRE models, like SLOs; you put in SLOs and you start doing, like, this incident management program that's, like, you know, you have a form you fill out after every incident, and then you [unintelligible 00:20:25] retros.And it turns out that those things are very high-level skills, skills and capabilities in an organization. And so, when you have this kind of IT mindset or the enterprise mindset, bringing the culture together to make those things work often doesn't happen. Because, you know, they'll go with the prescriptive model and say, like, okay, we're going to implement SLOs, we're going to start measuring SLIs on all of the services, and we're going to hold you accountable for meeting those targets. If you just do that, right, you're just doing more gatekeeping and policing of your tech environment. My bet is, reliability almost never improves in those cases.And that's been my experience, too, and why I get charged up about this is, if you just go slam in these practices, people end up miserable, the practices then become tarnished because people experienced the worst version of them. And then—Corey: And with the remote explosion as well, it turns out that changing jobs basically means their company sends you a different Mac, and the next Monday, you wind up signing into a different Slack team.Amy: Yeah, so the culture really matters, right? You can't cover it over with foosball tables and great lunch. You actually have to deliver tools that developers want to use and you have to deliver a software engineering culture that brings out the best in developers instead of demanding the best from developers. I think that's a fundamental business shift that's kind of happening. If I'm putting on my wizard hat and looking into the future and dreaming about what might change in the world, right, is that there's kind of a change in how we do leadership and how we do business that's shifting more towards that model where we look at what people are capable of and we trust in our people, and we get more out of them, the knowledge work model.If we want more knowledge work, we need people to be happy and to feel engaged in their community. And suddenly we start to see these kind of generational, bigger-pie kind of things start to happen. But how do we get there? It's not SLOs. It maybe it's a little bit starting with incidents. That's where I've had the most success, and you asked me about that. So, getting practical, incident management is probably—Corey: Right. Well, as I see it, the problem with SLOs across the board is it feels like it's a very insular community so far, and communicating it to engineers seems to be the focus of where the community has been, but from my understanding of it, you absolutely need buy-in at significantly high executive levels, to at the very least by you air cover while you're doing these things and making these changes, but also to help drive that cultural shift. None of this is something I have the slightest clue how to do, let's be very clear. If I knew how to change a company's culture, I'd have a different job.Amy: Yeah. [laugh]. The biggest omission in the Google SRE books was [Ers 00:22:58]. There was a guy at Google named Ers who owns availability for Google, and when anything is, like, in dispute and bubbles up the management team, it goes to Ers, and he says, “Thou shalt…” right? Makes the call. And that's why it works, right?Like, it's not just that one person, but that system of management where the whole leadership team—there's a large, very well-funded team with a lot of power in the organization that can drive availability, and they can say, this is how you're going to do metrics for your service, and this is the system that you're in. And it's kind of, yeah, sure it works for them because they have all the organizational support in place. What I was saying to my team just the other day—because we're in the middle of our SLO rollout—is that really, I think an SLO program isn't [clear throat] about the engineers at all until late in the game. At the beginning of the game, it's really about getting the leadership team on board to say, “Hey, we want to put in SLIs and SLOs to start to understand the functioning of our software system.” But if they don't have that curiosity in the first place, that desire to understand how well their teams are doing, how healthy their teams are, don't do it. It's not going to work. It's just going to make everyone miserable.Corey: It feels like it's one of those difficult to sell problems as well, in that it requires some tooling changes, absolutely. It requires cultural change and buy-in and whatnot, but in order for that to happen, there has to be a painful problem that a company recognizes and is willing to pay to make go away. The problem with stuff like this is that once you pay, there's a lot of extra work that goes on top of it as well, that does not have a perception—rightly or wrongly—of contributing to feature velocity, of hitting the next milestone. It's, “Really? So, we're going to be spending how much money to make engineers happier? They should get paid an awful lot and they're still complaining and never seem happy. Why do I care if they're happy other than the pure mercenary perspective of otherwise they'll quit?” I'm not saying that it's not worth pursuing; it's not a worthy goal. I am saying that it becomes a very difficult thing to wind up selling as a product.Amy: Well, as a product for sure, right? Because—[sigh] gosh, I have friends in the space who work on these tools. And I want to be careful.Corey: Of course. Nothing but love for all of those people, let's be very clear.Amy: But a lot of them, you know, they're pulling metrics from existing monitoring systems, they are doing some interesting math on them, but what you get at the end is a nice service catalog and dashboard, which are things we've been trying to land as products in this industry for as long as I can remember, and—Corey: “We've got it this time, though. This time we'll crack the nut.” Yeah. Get off the island, Gilligan.Amy: And then the other, like, risky thing, right, is the other part that makes me uncomfortable about SLOs, and why I will often tell folks that I talk to out in the industry that are asking me about this, like, one-on-one, “Should I do it here?” And it's like, you can bring the tool in, and if you have a management team that's just looking to have metrics to drive productivity, instead of you know, trying to drive better knowledge work, what you get is just a fancier version of more Taylorism, right, which is basically scientific management, this idea that we can, like, drive workers to maximum efficiency by measuring random things about them and driving those numbers. It turns out, that doesn't really work very well, even in industrial scale, it just happened to work because, you know, we have a bloody enough society that we pushed people into it. But the reality is, if you implement SLOs badly, you get more really bad Taylorism that's bad for you developers. And my suspicion is that you will get worse availability out of it than you would if you just didn't do it at all.Corey: This episode is sponsored by our friends at Revelo. Revelo is the Spanish word of the day, and its spelled R-E-V-E-L-O. It means “I reveal.” Now, have you tried to hire an engineer lately? I assure you it is significantly harder than it sounds. One of the things that Revelo has recognized is something I've been talking about for a while, specifically that while talent is evenly distributed, opportunity is absolutely not. They're exposing a new talent pool to, basically, those of us without a presence in Latin America via their platform. It's the largest tech talent marketplace in Latin America with over a million engineers in their network, which includes—but isn't limited to—talent in Mexico, Costa Rica, Brazil, and Argentina. Now, not only do they wind up spreading all of their talent on English ability, as well as you know, their engineering skills, but they go significantly beyond that. Some of the folks on their platform are hands down the most talented engineers that I've ever spoken to. Let's also not forget that Latin America has high time zone overlap with what we have here in the United States, so you can hire full-time remote engineers who share most of the workday as your team. It's an end-to-end talent service, so you can find and hire engineers in Central and South America without having to worry about, frankly, the colossal pain of cross-border payroll and benefits and compliance because Revelo handles all of it. If you're hiring engineers, check out revelo.io/screaming to get 20% off your first three months. That's R-E-V-E-L-O dot I-O slash screaming.Corey: That is part of the problem is, in some cases, to drive some of these improvements, you have to go backwards to move forwards. And it's one of those, “Great, so we spent all this effort and money in the rest of now things are worse?” No, not necessarily, but suddenly are aware of things that were slipping through the cracks previously.Amy: Yeah. Yeah.Corey: Like, the most realistic thing about first The Phoenix Project and then The Unicorn Project, both by Gene Kim, has been the fact that companies have these problems and actively cared enough to change it. In my experience, that feels a little on the rare side.Amy: Yeah, and I think that's actually the key, right? It's for the culture change, and for, like, if you really looking to be, like, do I want to work at this company? Am I investing my myself in here? Is look at the leadership team and be, like, do these people actually give a crap? Are they looking just to punt another number down the road?That's the real question, right? Like, the technology and stuff, at the point where I'm at in my career, I just don't care that much anymore. [laugh]. Just… fine, use Kubernetes, use Postgres, [unintelligible 00:27:30], I don't care. I just don't. Like, Oracle, I might have to ask, you know, go to finance and be like, “Hey, can we spend 20 million for a database?” But like, nobody really asks for that anymore, so. [laugh].Corey: As one does. I will say that I mostly agree with you, but a technology that I found myself getting excited about, given the time of the recording on this is… fun, I spent a bit of time yesterday—from when we're recording this—teaching myself just enough Go to wind up being together a binary that I needed to do something actively ridiculous for my camera here. And I found myself coming away deeply impressed by a lot of things about it, how prescriptive it was for one, how self-contained for another. And after spending far too many years of my life writing shitty Perl, and shitty Bash, and worse Python, et cetera, et cetera, the prescriptiveness was great. The fact that it wound up giving me something I could just run, I could cross-compile for anything I need to run it on, and it just worked. It's been a while since I found a technology that got me this interested in exploring further.Amy: Go is great for that. You mentioned one of my two favorite features of Go. One is usually when a program compiles—at least the way I code in Go—it usually works. I've been working with Go since about 0.9, like, just a little bit before it was released as 1.0, and that's what I've noticed over the years of working with it is that most of the time, if you have a pretty good data structure design and you get the code to compile, usually it's going to work, unless you're doing weird stuff.The other thing I really love about Go and that maybe you'll discover over time is the malleability of it. And the reason why I think about that more than probably most folks is that I work on other people's code most of the time. And maybe this is something that you probably run into with your business, too, right, where you're working on other people's infrastructure. And the way that we encode business rules and things in the languages, in our programming language or our config syntax and stuff has a huge impact on folks like us and how quickly we can come into a situation, assess, figure out what's going on, figure out where things are laid out, and start making changes with confidence.Corey: Forget other people for a minute they're looking at what I built out three or four years ago here, myself, like, I look at past me, it's like, “What was that rat bastard thinking? This is awful.” And it's—forget other people's code; hell is your own code, on some level, too, once it's slipped out of the mental stack and you have to re-explore it and, “Oh, well thank God I defensively wound up not including any comments whatsoever explaining what the living hell this thing was.” It's terrible. But you're right, the other people's shell scripts are finicky and odd.I started poking around for help when I got stuck on something, by looking at GitHub, and a few bit of searching here and there. Even these large, complex, well-used projects started making sense to me in a way that I very rarely find. It's, “What the hell is that thing?” is my most common refrain when I'm looking at other people's code, and Go for whatever reason avoids that, I think because it is so prescriptive about formatting, about how things should be done, about the vision that it has. Maybe I'm romanticizing it and I'll hate it and a week from now, and I want to go back and remove this recording, but.Amy: The size of the language helps a lot.Corey: Yeah.Amy: But probably my favorite. It's more of a convention, which actually funny the way I'm going to talk about this because the two languages I work on the most right now are Ruby and Go. And I don't feel like two languages could really be more different.Syntax-wise, they share some things, but really, like, the mental models are so very, very different. Ruby is all the way in on object-oriented programming, and, like, the actual real kind of object-oriented with messaging and stuff, and, like, the whole language kind of springs from that. And it kind of requires you to understand all of these concepts very deeply to be effective in large programs. So, what I find is, when I approach Ruby codebase, I have to load all this crap into my head and remember, “Okay, so yeah, there's this convention, when you do this kind of thing in Ruby”—or especially Ruby on Rails is even worse because they go deep into convention over configuration. But what that's code for is, this code is accessible to people who have a lot of free cognitive capacity to load all this convention into their heads and keep it in their heads so that the code looks pretty, right?And so, that's the trade-off as you said, okay, my developers have to be these people with all these spare brain cycles to understand, like, why I would put the code here in this place versus this place? And all these, like, things that are in the code, like, very compact, dense concepts. And then you go to something like Go, which is, like, “Nah, we're not going to do Lambdas. Nah”—[laugh]—“We're not doing all this fancy stuff.” So, everything is there on the page.This drives some people crazy, right, is that there's all this boilerplate, boilerplate, boilerplate. But the reality is, I can read most Go files from top to the bottom and understand what the hell it's doing, whereas I can go sometimes look at, like, a Ruby thing, or sometimes Python and e—Perl is just [unintelligible 00:32:19] all the time, right, it's there's so much indirection. And it just be, like, “What the [BLEEP] is going on? This is so dense. I'm going to have to sit down and write it out in longhand so I can understand what the developer was even doing here.” And—Corey: Well, that's why I got the Mac Studio; for when I'm not doing A/V stuff with it, that means that I'll have one core that I can use for, you know, front-end processing and the rest, and the other 19 cores can be put to work failing to build Nokogiri in Ruby yet again.Amy: [laugh].Corey: I remember the travails of working with Ruby, and the problem—I have similar problems with Python, specifically in that—I don't know if I'm special like this—it feels like it's a SRE DevOps style of working, but I am grabbing random crap off a GitHub constantly and running it, like, small scripts other people have built. And let's be clear, I run them on my test AWS account that has nothing important because I'm not a fool that I read most of it before I run it, but I also—it wants a different version of Python every single time. It wants a whole bunch of other things, too. And okay, so I use ASDF as my version manager for these things, which for whatever reason, does not work for the way that I think about this ergonomically. Okay, great.And I wind up with detritus scattered throughout my system. It's, “Hey, can you make this reproducible on my machine?” “Almost certainly not, but thank you for asking.” It's like ‘Step 17: Master the Wolf' level of instructions.Amy: And I think Docker generally… papers over the worst of it, right, is when we built all this stuff in the aughts, you know, [CPAN 00:33:45]—Corey: Dev containers and VS Code are very nice.Amy: Yeah, yeah. You know, like, we had CPAN back in the day, I was doing chroots, I think in, like, '04 or '05, you know, to solve this problem, right, which is basically I just—screw it; I will compile an entire distro into a directory with a Perl and all of its dependencies so that I can isolate it from the other things I want to run on this machine and not screw up and not have these interactions. And I think that's kind of what you're talking about is, like, the old model, when we deployed servers, there was one of us sitting there and then we'd log into the server and be like, I'm going to install the Perl. You know, I'll compile it into, like, [/app/perl 558 00:34:21] whatever, and then I'll CPAN all this stuff in, and I'll give it over to the developer, tell them to set their shebang to that and everything just works. And now we're in a mode where it's like, okay, you got to set up a thousand of those. “Okay, well, I'll make a tarball.” [laugh]. But it's still like we had to just—Corey: DevOps, but [unintelligible 00:34:37] dev closer to ops. You're interrelating all the time. Yeah, then Docker comes along, and add dev is, like, “Well, here's the container. Good luck, asshole.” And it feels like it's been cast into your yard to worry about.Amy: Yeah, well, I mean, that's just kind of business, or just—Corey: Yeah. Yeah.Amy: I'm not sure if it's business or capitalism or something like that, but just the idea that, you know, if I can hand off the shitty work to some other poor schlub, why wouldn't I? I mean, that's most folks, right? Like, just be like, “Well”—Corey: Which is fair.Amy: —“I got it working. Like, my part is done, I did what I was supposed to do.” And now there's a lot of folks out there, that's how they work, right? “I hit done. I'm done. I shipped it. Sure. It's an old [unintelligible 00:35:16] Ubuntu. Sure, there's a bunch of shell scripts that rip through things. Sure”—you know, like, I've worked on repos where there's hundreds of things that need to be addressed.Corey: And passing to someone else is fine. I'm thrilled to do it. Where I run into problems with it is where people assume that well, my part was the hard part and anything you schlubs do is easy. I don't—Amy: Well, that's the underclass. Yeah. That's—Corey: Forget engineering for a second; I throw things to the people over in the finance group here at The Duckbill Group because those people are wizards at solving for this thing. And it's—Amy: Well, that's how we want to do things.Corey: Yeah, specialization works.Amy: But we have this—it's probably more cultural. I don't want to pick, like, capitalism to beat on because this is really, like, human cultural thing, and it's not even really particularly Western. Is the idea that, like, “If I have an underclass, why would I give a shit what their experience is?” And this is why I say, like, ops teams, like, get out of here because most ops teams, the extant ops teams are still called ops, and a lot of them have been renamed SRE—but they still do the same job—are an underclass. And I don't mean that those people are below us. People are treated as an underclass, and they shouldn't be. Absolutely not.Corey: Yes.Amy: Because the idea is that, like, well, I'm a fancy person who writes code at my ivory tower, and then it all flows down, and those people, just faceless people, do the deployment stuff that's beneath me. That attitude is the most toxic thing, I think, in tech orgs to address. Like, if you're trying to be like, “Well, our liability is bad, we have security problems, people won't fix their code.” And go look around and you will find people that are treated as an underclass that are given codes thrown over the wall at them and then they just have to toil through and make it work. I've worked on that a number of times in my career.And I think just like saying, underclass, right, or caste system, is what I found is the most effective way to get people actually thinking about what the hell is going on here. Because most people are just, like, “Well, that's just the way things are. It's just how we've always done it. The developers write to code, then give it to the sysadmins. The sysadmins deploy the code. Isn't that how it always works?”Corey: You'd really like to hope, wouldn't you?Amy: [laugh]. Not me. [laugh].Corey: Again, the way I see it is, in theory—in theory—sysadmins, ops, or that should not exist. People should theoretically be able to write code as developers that just works, the end. And write it correct the first time and never have to change it again. Yeah. There's a reason that I always like to call staging environments in places I work ‘theory' because it works in theory, but not in production, and that is fundamentally the—like, that entire job role is the difference between theory and practice.Amy: Yeah, yeah. Well, I think that's the problem with it. We're already so disconnected from the physical world, right? Like, you and I right now are talking over multiple strands of glass and digital transcodings and things right now, right? Like, we are detached from the physical reality.You mentioned earlier working in data centers, right? The thing I miss about it is, like, the physicality of it. Like, actually, like, I held a server in my arms and put it in the rack and slid it into the rails. I plugged into power myself; I pushed the power button myself. There's a server there. I physically touched it.Developers who don't work in production, we talked about empathy and stuff, but really, I think the big problem is when they work out in their idea space and just writing code, they write the unit tests, if we're very lucky, they'll write a functional test, and then they hand that wad off to some poor ops group. They're detached from the reality of operations. It's not even about accountability; it's about experience. The ability to see all of the weird crap we deal with, right? You know, like, “Well, we pushed the code to that server, but there were three bit flips, so we had to do it again. And then the other server, the disk failed. And on the other server…” You know? [laugh].It's just, there's all this weird crap that happens, these systems are so complex that they're always doing something weird. And if you're a developer that just spends all day in your IDE, you don't get to see that. And I can't really be mad at those folks, as individuals, for not understanding our world. I figure out how to help them, and the best thing we've come up with so far is, like, well, we start giving this—some responsibility in a production environment so that they can learn that. People do that, again, is another one that can be done wrong, where it turns into kind of a forced empathy.I actually really hate that mode, where it's like, “We're forcing all the developers online whether they like it or not. On-call whether they like it or not because they have to learn this.” And it's like, you know, maybe slow your roll a little buddy because the stuff is actually hard to learn. Again, minimizing how hard ops work is. “Oh, we'll just put the developers on it. They'll figure it out, right? They're software engineers. They're probably smarter than you sysadmins.” Is the unstated thing when we do that, right? When we throw them in the pit and be like, “Yeah, they'll get it.” [laugh].Corey: And that was my problem [unintelligible 00:39:49] the interview stuff. It was in the write code on a whiteboard. It's, “Look, I understood how the system fundamentally worked under the hood.” Being able to power my way through to get to an outcome even in language I don't know, was sort of part and parcel of the job. But this idea of doing it in artificially constrained environment, in a language I'm not super familiar with, off the top of my head, it took me years to get to a point of being able to do it with a Bash script because who ever starts with an empty editor and starts getting to work in a lot of these scenarios? Especially in an ops role where we're not building something from scratch.Amy: That's the interesting thing, right? In the majority of tech work today—maybe 20 years ago, we did it more because we were literally building the internet we have today. But today, most of the engineers out there working—most of us working stiffs—are working on stuff that already exists. We're making small incremental changes, which is great that's what we're doing. And we're dealing with old code.Corey: We're gluing APIs together, and that's fine. Ugh. I really want to thank you for taking so much time to talk to me about how you see all these things. If people want to learn more about what you're up to, where's the best place to find you?Amy: I'm on Twitter every once in a while as @MissAmyTobey, M-I-S-S-A-M-Y-T-O-B-E-Y. I have a blog I don't write on enough. And there's a couple things on the Equinix Metal blog that I've written, so if you're looking for that. Otherwise, mainly Twitter.Corey: And those links will of course be in the [show notes 00:41:08]. Thank you so much for your time. I appreciate it.Amy: I had fun. Thank you.Corey: As did I. Amy Tobey, Senior Principal Engineer at Equinix. I'm Cloud Economist Corey Quinn, and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, or on the YouTubes, smash the like and subscribe buttons, as the kids say. Whereas if you've hated this episode, same thing, five-star review all the platforms, smash the buttons, but also include an angry comment telling me that you're about to wind up subpoenaing a copy of my shell script because you're convinced that your intellectual property and secrets are buried within.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.
About YoavYoav is a security veteran recognized on Microsoft Security Response Center's Most Valuable Research List (BlackHat 2019). Prior to joining Orca Security, he was a Unit 8200 researcher and team leader, a chief architect at Hyperwise Security, and a security architect at Check Point Software Technologies. Yoav enjoys hunting for Linux and Windows vulnerabilities in his spare time.Links Referenced: Orca Security: https://orca.security Twitter: https://twitter.com/yoavalon TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: This episode is sponsored in part by our friends at Vultr. Optimized cloud compute plans have landed at Vultr to deliver lightning fast processing power, courtesy of third gen AMD EPYC processors without the IO, or hardware limitations, of a traditional multi-tenant cloud server. Starting at just 28 bucks a month, users can deploy general purpose, CPU, memory, or storage optimized cloud instances in more than 20 locations across five continents. Without looking, I know that once again, Antarctica has gotten the short end of the stick. Launch your Vultr optimized compute instance in 60 seconds or less on your choice of included operating systems, or bring your own. It's time to ditch convoluted and unpredictable giant tech company billing practices, and say goodbye to noisy neighbors and egregious egress forever. Vultr delivers the power of the cloud with none of the bloat. "Screaming in the Cloud" listeners can try Vultr for free today with a $150 in credit when they visit getvultr.com/screaming. That's G E T V U L T R.com/screaming. My thanks to them for sponsoring this ridiculous podcast.Corey: Finding skilled DevOps engineers is a pain in the neck! And if you need to deploy a secure and compliant application to AWS, forgettaboutit! But that's where DuploCloud can help. Their comprehensive no-code/low-code software platform guarantees a secure and compliant infrastructure in as little as two weeks, while automating the full DevSecOps lifestyle. Get started with DevOps-as-a-Service from DuploCloud so that your cloud configurations are done right the first time. Tell them I sent you and your first two months are free. To learn more visit: snark.cloud/duplocloud. Thats's snark.cloud/D-U-P-L-O-C-L-O-U-D. Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. Periodically, I would say that I enjoy dealing with cloud platform security issues, except I really don't. It's sort of forced upon me to deal with much like a dead dog is cast into their neighbor's yard for someone else to have to worry about. Well, invariably, it seems like it's my yard.And I'm only on the periphery of these things. Someone who's much more in the trenches in the wide world of cloud security is joining me today. Yoav Alon is the CTO at Orca Security. Yoav, thank you for taking the time to join me today and suffer the slings and arrows I'll no doubt be hurling your way.Yoav: Thank you, Corey, for having me. I've been a longtime listener, and it's an honor to be here.Corey: I still am periodically surprised that anyone listens to these things. Because it's unlike a newsletter where everyone will hit reply and give me a piece of their mind. People generally don't wind up sending me letters about things that they hear on the podcast, so whenever I talk to somebody listens to it as, “Oh. Oh, right, I did turn the microphone on. Awesome.” So, it's always just a little on the surreal side.But we're not here to talk necessarily about podcasting, or the modern version of an AM radio show. Let's start at the very beginning. What is Orca Security, and why would folks potentially care about what it is you do?Yoav: So, Orca Security is a cloud security company, and our vision is very simple. Given a customer's cloud environment, we want to detect all the risks in it and implement mechanisms to prevent it from occurring. And while it sounds trivial, before Orca, it wasn't really possible. You will have to install multiple tools and aggregate them and do a lot of manual work, and it was messy. And we wanted to change that, so we had, like, three guiding principles.We call it seamless, so I want to detect all the risks in your environment without friction, which is our speak for fighting with your peers. We also want to detect everything so you don't have to install, like, a tool for each issue: A tool for vulnerabilities, a tool for misconfigurations, and for sensitive data, IAM roles, and such. And we put a very high priority on context, which means telling you what's important, what's not. So, for example, S3 bucket open to the internet is important if it has sensitive data, not if it's a, I don't know, static website.Corey: Exactly. I have a few that I'd like to get screamed at in my AWS account, like, “This is an open S3 bucket and it's terrible.” I look at it the name is assets.lastweekinaws.com. Gee, I wonder if that's something that's designed to be a static hosted website.Increasingly, I've been slapping CloudFront in front of those things just to make the broken warning light go away. I feel like it's an underhanded way of driving CloudFront adoption some days, but not may not be the most charitable interpretation thereof. Orca has been top-of-mind for a lot of folks in the security community lately because let's be clear here, dealing with security problems in cloud providers from a vendor perspective is an increasingly crowded—and clouded—space. Just because there's so much—there's investment pouring into it, everyone has a slightly different take on the problem, and it becomes somewhat challenging to stand out from the pack. You didn't really stand out from the pack so much as leaped to the front of it and more or less have become the de facto name in a very short period of time, specifically—at least from my world—when you wound up having some very interesting announcements about vulnerabilities within AWS itself. You will almost certainly do a better job of relating the story, so please, what did you folks find?Yoav: So, back in September of 2021, two of my researchers, Yanir Tsarimi and Tzah Pahima, each one of them within a relatively short span of time from each other, found a vulnerability in AWS. Tzah found a vulnerability in CloudFormation which we named BreakingFormation and Yanir found a vulnerability in AWS Glue, which we named SuperGlue. We're not the best copywriters, but anyway—Corey: No naming things is hard. Ask any Amazonian.Yoav: Yes. [laugh]. So, I'll start with BreakingFormation which caught the eyes of many. It was an XXE SSRF, which is jargon to say that we were able to read files and execute HTTP requests and read potentially sensitive data from CloudFormation servers. This one was mitigated within 26 hours by AWS, so—Corey: That was mitigated globally.Yoav: Yes, globally, which I've never seen such quick turnaround anywhere. It was an amazing security feat to see.Corey: Particularly in light of the fact that AWS does a lot of things very right when it comes to, you know, designing cloud infrastructure. Imagine that, they've had 15 years of experience and basically built the idea of cloud, in some respects, at the scale that hyperscalers operate at. And one of their core tenets has always been that there's a hard separation between regions. There are remarkably few global services, and those are treated with the utmost of care and delicacy. To the point where when something like that breaks as an issue that spans more than one region, it is headline-making news in many cases.So it's, they almost never wind up deploying things to all regions at the same time. That can be irksome when we're talking about things like I want a feature that solves a problem that I have, and I have to wait months for it to hit a region that I have resources living within, but for security, stuff like this, I am surprised that going from, “This is the problem,” to, “It has been mitigated,” took place within 26 hours. I know it sounds like a long time to folks who are not deep in the space, but that is superhero speed.Yoav: A small correction, it's 26 hours for, like, the main regions. And it took three to four days to propagate to all regions. But still, it's speed of lighting in for security space.Corey: When this came out, I was speaking to a number of journalists on background about trying to wrap their head around this, and they said that, “Oh yeah, and security is always, like, the top priority for AWS, second only to uptime and reliability.” And… and I understand the perception, but I disagree with it in the sense of the nightmare scenario—that every time I mention to a security person watching the blood drain from their face is awesome—but the idea that take IAM, which as Werner said in his keynote, processes—was it 500 million or was it 500 billion requests a second, some ludicrous number—imagine fails open where everything suddenly becomes permitted. I have to imagine in that scenario, they would physically rip the power cables out of the data centers in order to stop things from going out. And that is the right move. Fortunately, I am extremely optimistic that will remain a hypothetical because that is nightmare fuel right there.But Amazon says that security is job zero. And my cynical interpretation is that well, it wasn't, but they forgot security, decided to bolt it on to the end, like everyone else does, and they just didn't want to renumber all their slides, so instead of making it point one, they just put another slide in front of it and called the job zero. I'm sure that isn't how it worked, but for those of us who procrastinate and building slide decks for talks, it has a certain resonance to it. That was one issue. The other seemed a little bit more pernicious focusing on Glue, which is their ETL-as-a-Service… service. One of them I suppose. Tell me more about it.Yoav: So, one of the things that we found when we found the BreakingFormation when we reported the vulnerability, it led us to do a quick Google search, which led us back to the Glue service. It had references to Glue, and we started looking around it. And what we were able to do with the vulnerability is given a specific feature in Glue, which we don't disclose at the moment, we were able to effectively take control over the account which hosts the Glue service in us-east-1. And having this control allowed us to essentially be able to impersonate the Glue service. So, every role in AWS that has a trust to the Glue service, we were able to effectively assume a role into it in any account in AWS. So, this was more critical a vulnerability in its effect.Corey: I think on some level, the game of security has changed because for a lot of us who basically don't have much in the way of sensitive data living in AWS—and let's be clear, I take confidentiality extremely seriously. Our clients on the consulting side view their AWS bills themselves as extremely confidential information that Amazon stuffs into a PDF and emails every month. But still. If there's going to be a leak, we absolutely do not want it to come from us, and that is something that we take extraordinarily seriously. But compared to other jobs I've had in the past, no one will die if that information gets out.It is not the sort of thing that is going to ruin people's lives, which is very often something that can happen in some data breaches. But in my world, one of the bad cases of a breach of someone getting access to my account is they could spin up a bunch of containers on the 17 different services that AWS offers that can run containers and mine cryptocurrency with it. And the damage to me then becomes a surprise bill. Okay, great. I can live with that.Something that's a lot scarier to a lot of companies with, you know, serious problems is, yep, fine, cost us money, whatever, but our access to our data is the one thing that is going to absolutely be the thing that cannot happen. So, from that perspective alone, something like Glue being able to do that is a lot more terrifying than subverting CloudFormation and being able to spin up additional resources or potentially take resources down. Is that how you folks see it too, or is—I'm sure there's nuance I'm missing.Yoav: So yeah, the access to data is top-of-mind for everyone. It's a bit scary to think about it. I have to mention, again, the quick turnaround time for AWS, which almost immediately issued a patch. It was a very fast one and they mitigated, again, the issue completely within days. About your comment about data.Data is king these days, there is nothing like data, and it has all the properties of everything that we care about. It's expensive to store, it's expensive to move, and it's very expensive if it leaks. So, I think a lot of people were more alarmed about the Glue vulnerability than the CloudFormation vulnerability. And they're right in doing so.Corey: I do want to call out that AWS did a lot of things right in this area. Their security posture is very clearly built around defense-in-depth. The fact that they were able to disclose—after some prodding—that they checked the CloudTrail logs for the service itself, dating back to the time the service launched, and verified that there had never been an exploit of this, that is phenomenal, as opposed to the usual milquetoast statements that companies have. We have no evidence of it, which can mean that we did the same thing and we looked through all the logs in it's great, but it can also mean that, “Oh, yeah, we probably should have logs, shouldn't we? But let's take a backlog item for that.” And that's just terrifying on some level.It becomes a clear example—a shining beacon for some of us in some cases—of doing things right from that perspective. There are other sides to it, though. As a customer, it was frustrating in the extreme to—and I mean, no offense by this—to learn about this from you rather than from the provider themselves. They wound up putting up a security notification many hours after your blog post went up, which I would also just like to point out—and we spoke about it at the time and it was a pure coincidence—but there was something that was just chef's-kiss perfect about you announcing this on Andy Jassy's birthday. That was just very well done.Yoav: So, we didn't know about Andy's birthday. And it was—Corey: Well, I see only one of us has a company calendar with notable executive birthdays splattered all over it.Yoav: Yes. And it was also published around the time that AWS CISO was announced, which was also a coincidence because the date was chosen a lot of time in advance. So, we genuinely didn't know.Corey: Communicating around these things is always challenging because on the one hand, I can absolutely understand the cloud providers' position on this. We had a vulnerability disclosed to us. We did our diligence and our research because we do an awful lot of things correctly and everyone is going to have vulnerabilities, let's be serious here. I'm not sitting here shaking my fist, angry at AWS's security model. It works, and I am very much a fan of what they do.And I can definitely understand then, going through all of that there was no customer impact, they've proven it. What value is there to them telling anyone about it, I get that. Conversely, you're a security company attempting to stand out in a very crowded market, and it is very clear that announcing things like this demonstrates a familiarity with cloud that goes beyond the common. I radically changed my position on how I thought about Orca based upon these discoveries. It went from, “Orca who,” other than the fact that you folks have sponsored various publications in the past—thanks for that—but okay, a security company. Great to, “Oh, that's Orca. We should absolutely talk to them about a thing that we're seeing.” It has been transformative for what I perceive to be your public reputation in the cloud security space.So, those two things are at odds: The cloud provider doesn't want to talk about anything and the security company absolutely wants to demonstrate a conversational fluency with what is going on in the world of cloud. And that feels like it's got to be a very delicate balancing act to wind up coming up with answers that satisfy all parties.Yoav: So, I just want to underline something. We don't do what we do in order to make a marketing stand. It's a byproduct of our work, but it's not the goal. For the Orca Security Research Pod, which it's the team at Orca which does this kind of research, our mission statement is to make cloud security better for everyone. Not just Orca customers; for everyone.And you get to hear about the more shiny things like big headline vulnerabilities, but we also have very sensible blog posts explaining how to do things, how to configure things and give you more in-depth understanding into security features that the cloud providers themselves provide, which are great, and advance the state of the cloud security. I would say that having a cloud vulnerability is sort of one of those things, which makes me happy to be a cloud customer. On the one side, we had a very big vulnerability with very big impact, and the ability to access a lot of customers' data is conceptually terrifying. The flip side is that everything was mitigated by the cloud providers in warp speed compared to everything else we've seen in all other elements of security. And you get to sleep better knowing that it happened—so no platform is infallible—but still the cloud provider do work for you, and you'll get a lot of added value from that.Corey: You've made a few points when this first came out, and I want to address them. The first is, when I reached out to you with a, “Wow, great work.” You effectively instantly came back with, “Oh, it wasn't me. It was members of my team.” So, let's start there. Who was it that found these things? I'm a huge believer giving people credit for the things that they do.The joy of being in a leadership position is if the company screws up, yeah, you take responsibility for that, whether the company does something great, yeah, you want to pass praise onto the people who actually—please don't take this the wrong way—did the work. And not that leadership is not work, it absolutely is, but it's a different kind of work.Yoav: So, I am a security researcher, and I am very mindful for the effort and skill it requires to find vulnerabilities and actually do a full circle on them. And the first thing I'll mention is Tzah Pahima, which found the BreakingFormation vulnerability and the vulnerability in CloudFormation, and Yanir Tsarimi, which found the AutoWarp vulnerability, which is the Azure vulnerability that we have not mentioned, and the Glue vulnerability, dubbed SuperGlue. Both of them are phenomenal researcher, world-class, and I'm very honored to work with them every day. It's one of my joys.Corey: Couchbase Capella Database-as-a-Service is flexible, full-featured and fully managed with built in access via key-value, SQL, and full-text search. Flexible JSON documents aligned to your applications and workloads. Build faster with blazing fast in-memory performance and automated replication and scaling while reducing cost. Capella has the best price performance of any fully managed document database. Visit couchbase.com/screaminginthecloud to try Capella today for free and be up and running in three minutes with no credit card required. Couchbase Capella: make your data sing.Corey: It's very clear that you have built an extraordinary team for people who are able to focus on vulnerability research. Which, on some level, is very interesting because you are not branded as it were as a vulnerability research company. This is not something that is your core competency; it's not a thing that you wind up selling directly that I'm aware of. You are selling a security platform offering. So, on the one hand, it makes perfect sense that you would have a division internally that works on this, but it's also very noteworthy, I think, that is not the core description of what it is that you do.It is a means by which you get to the outcome you deliver for customers, not the thing that you are selling directly to them. I just find that an interesting nuance.Yoav: Yes, it is. And I would elaborate and say that research informs the product, and the product informs research. And we get to have this fun dance where we learn new things by doing research. We [unintelligible 00:18:08] the product, and we use the customers to teach us things that we didn't know. So, it's one of those happy synergies.Corey: I want to also highlight a second thing that you have mentioned and been very, I guess, on message about since news of this stuff first broke. And because it's easy to look at this and sensationalize aspects of it, where, “See? The cloud providers security model is terrible. You shouldn't use them. Back to data centers we go.” Is basically the line taken by an awful lot of folks trying to sell data center things.That is not particularly helpful for the way that the world is going. And you've said, “Yeah, you should absolutely continue to be in cloud. Do not disrupt your cloud plan as a result.” And let's be clear, none of the rest of us are going to find and mitigate these things with anything near the rigor or rapidity that the cloud providers can and do demonstrate.Yoav: I totally agree. And I would say that the AWS security folks are doing a phenomenal job. I can name a few, but they're all great. And I think that the cloud is by far a much safer alternative than on-prem. I've never seen issues in my on-prem environment which were critical and fixed in such a high velocity and such a massive scale.And you always get the incremental improvements of someone really thinking about all the ins and outs of how to do security, how to do security in the cloud, how to make it faster, more reliable, without a business interruptions. It's just phenomenal to see and phenomenal to witness how far we've come in such a relatively short time as an industry.Corey: AWS in particular, has a reputation for being very good at security. I would argue that, from my perspective, Google is almost certainly slightly better at their security approach than AWS is, but to be clear, both of them are significantly further along the path than I am going to be. So great, fantastic. You also have found something interesting over in the world of Azure, and that honestly feels like a different class of vulnerability. To my understanding, the Azure vulnerability that you recently found was you could get credential material for other customers simply by asking for it on a random high port. Which is one of those—I'm almost positive I'm misunderstanding something here. I hope. Please?Yoav: I'm not sure you're misunderstanding. So, I would just emphasize that the vulnerability again, was found by Yanir Tsarimi. And what he found was, he used a service called Azure Automation which enables you essentially to run a Python script on various events and schedules. And he opened the python script and he tried different ports. And one of the high ports he found, essentially gave him his credentials. And he said, “Oh, wait. That's a really odd port for an HTTP server. Let's try, I don't know, a few ports on either way.” And he started getting credentials from other customers. Which was very surprising to us.Corey: That is understating it by a couple orders of magnitude. Yes, like, “Huh. That seems sub-optimal,” is sort of like the corporate messaging approved thing. At the time you discover that—I'm certain it was a three-minute-long blistering string of profanity in no fewer than four languages.Yoav: I said to him that this is, like, a dishonorable bug because he worked very little to find it. So it was, from start to finish, the entire research took less than two hours, which, in my mind, is not enough for this kind of vulnerability. You have to work a lot harder to get it. So.Corey: Yeah, exactly. My perception is that when there are security issues that I have stumbled over—for example, I gave a talk at re:Invent about it in the before times, one of them was an overly broad permission in a managed IAM policy for SageMaker. Okay, great. That was something that obviously was not good, but it also was more of a privilege escalation style of approach. It wasn't, “Oh, by the way, here's the keys to everything.”That is the type of vulnerability I have come to expect, by and large, from cloud providers. We're just going to give you access credentials for other customers is one of those areas that… it bugs me on a visceral level, not because I'm necessarily exposed personally, but because it more or less shores up so many of the arguments that I have spent the last eight years having with folks are like, “Oh, you can't go to cloud. Your data should live on your own stuff. It's more secure that way.” And we were finally it feels like starting to turn a cultural corner on these things.And then something like that happens, and it—almost have those naysayers become vindicated for it. And it's… it almost feels, on some level, and I don't mean to be overly unkind on this, but it's like, you are absolutely going to be in a better security position with the cloud providers. Except to Azure. And perhaps that is unfair, but it seems like Azure's level of security rigor is nowhere near that of the other two. Is that generally how you're seeing things?Yoav: I would say that they have seen more security issues than most other cloud providers. And they also have a very strong culture of report things to us, and we're very streamlined into patching those and giving credit where credit's due. And they give out bounties, which is an incentives for more research to happen on those platforms. So, I wouldn't say this categorically, but I would say that the optics are not very good. Generally, the cloud providers are much safer than on-prem because you only hear very seldom on security issues in the cloud.You hear literally every other day on issues happening to on-prem environments all over the place. And people just say they expect it to be this way. Most of the time, it's not even a headline. Like, “Company X affected with cryptocurrency or whatever.” It happens every single day, and multiple times a day, breaches which are massively bigger. And people who don't want to be in the cloud will find every reason not to be the cloud. Let us have fun.Corey: One of the interesting parts about this is that so many breaches that are on-prem are just never discovered because no one knows what the heck's running in an environment. And the breaches that we hear about are just the ones that someone had at least enough wherewithal to find out that, “Huh. That shouldn't be the way that it is. Let's dig deeper.” And that's a bad day for everyone. I mean, no one enjoys those conversations and those moments.And let's be clear, I am surprisingly optimistic about the future of Azure Security. It's like, “All right, you have a magic wand. What would you do to fix it?” It's, “Well, I'd probably, you know, hire Charlie Bell and get out of his way,” is not a bad answer as far as how these things go. But it takes time to reform a culture, to wind up building in security as a foundational principle. It's not something you can slap on after the fact.And perhaps this is unfair. But Microsoft has 30 years of history now of getting the world accustomed to oh, yeah, just periodically, terrible vulnerabilities are going to be discovered in your desktop software. And every once a month on Tuesdays, we're going to roll out a whole bunch of patches, and here you go. Make sure you turn on security updates, yadda, yadda, yadda. That doesn't fly in the cloud. It's like, “Oh, yeah, here's this month's list of security problems on your cloud provider.” That's one of those things that, like, the record-scratch, freeze-frame moment of wait, what are we doing here, exactly?Yoav: So, I would say that they also have a very long history of making those turnarounds. Bill Gates famously did his speech where security comes first, and they have done a very, very long journey and turn around the company from doing things a lot quicker and a lot safer. It doesn't mean they're perfect; everyone will have bugs, and Azure will have more people finding bugs into it in the near future, but security is a journey, and they've not started from zero. They're doing a lot of work. I would say it's going to take time.Corey: The last topic I want to explore a little bit is—and again, please don't take this as anyway being insulting or disparaging to your company, but I am actively annoyed that you exist. By which I mean that if I go into my AWS account, and I want to configure it to be secure. Great. It's not a matter of turning on the security service, it's turning on the dozen or so security services that then round up to something like GuardDuty that then, in turn, rounds up to something like Security Hub. And you look at not only the sheer number of these services and the level of complexity inherent to them, but then the bill comes in and you do some quick math and realize that getting breached would have been less expensive than what you're spending on all of these things.And somehow—the fact that it's complex, I understand; computers are like that. The fact that there is—[audio break 00:27:03] a great messaging story that's cohesive around this, I come to accept that because it's AWS; talking is not their strong suit. Basically declining to comment is. But the thing that galls me is that they are selling these services and not inexpensively either, so it almost feels, on some level like, shouldn't this on some of the built into the offerings that you folks are giving us?And don't get me wrong, I'm glad that you exist because bringing order to a lot of that chaos is incredibly important. But I can't shake the feeling that this should be a foundational part of any cloud offering. I'm guessing you might have a slightly different opinion than mine. I don't think you show up at the office every morning, “I hate that we exist.”Yoav: No. And I'll add a bit of context and nuance. So, for every other company than cloud providers, we expect them to be very good at most things, but not exceptional at everything. I'll give the Redshift example. Redshift is a pretty good offering, but Snowflake is a much better offering for a much wider range of—Corey: And there's a reason we're about to become Snowflake customers ourselves.Yoav: So, yeah. And there are a few other examples of that. A security company, a company that is focused solely on your security will be much better suited to help you, in a lot of cases more than the platform. And we work actively with AWS, Azure, and GCP requesting new features, helping us find places where we can shed more light and be more proactive. And we help to advance the conversation and make it a lot more actionable and improve from year to year. It's one of those collaborations. I think the cloud providers can do anything, but they can't do everything. And they do a very good job at security; it doesn't mean they're perfect.Corey: As you folks are doing an excellent job of demonstrating. Again, I'm glad you folks exist; I'm very glad that you are publishing the research that you are. It's doing a lot to bring a lot I guess a lot of the undue credit that I was giving AWS for years of, “No, no, it's not that they don't have vulnerabilities like everyone else does. It just that they don't ever talk about them.” And they're operationalizing of security response is phenomenal to watch.It's one of those things where I think you've succeeded and what you said earlier that you were looking to achieve, which is elevating the state of cloud security for everyone, not just Orca customers.Yoav: Thank you.Corey: Thank you. I really appreciate your taking the time out of your day to speak with me. If people want to learn more, where's the best place they can go to do that?Yoav: So, we have our website at orca.security. And you can reach me out on Twitter. My handle is at @yoavalon, which is @-Y-O-A-V-A-L-O-N.Corey: And we will of course put links to that in the [show notes 00:29:44]. Thanks so much for your time. I appreciate it.Yoav: Thank you, Corey.Corey: Yoav Alon, Chief Technology Officer at Orca Security. I'm Cloud Economist Corey Quinn, and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, or of course on YouTube, smash the like and subscribe buttons because that's what they do on that platform. Whereas if you've hated this podcast, please do the exact same thing, five-star review, smash the like and subscribe buttons on YouTube, but also leave an angry comment that includes a link that is both suspicious and frightening, and when we click on it, suddenly our phones will all begin mining cryptocurrency.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.
About CaseyCasey spends his days leveraging AWS to help organizations improve the speed at which they deliver software. With a background in software development, he has spent the past 20 years architecting, building, and supporting software systems for organizations ranging from startups to Fortune 500 enterprises.Links Referenced: “17 Ways to Run Containers in AWS”: https://www.lastweekinaws.com/blog/the-17-ways-to-run-containers-on-aws/ “17 More Ways to Run Containers on AWS”: https://www.lastweekinaws.com/blog/17-more-ways-to-run-containers-on-aws/ kubernetestheeasyway.com: https://kubernetestheeasyway.com snark.cloud/quinntainers: https://snark.cloud/quinntainers ECS Chargeback: https://github.com/gaggle-net/ecs-chargeback twitter.com/nektos: https://twitter.com/nektos TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: This episode is sponsored by our friends at Revelo. Revelo is the Spanish word of the day, and its spelled R-E-V-E-L-O. It means “I reveal.” Now, have you tried to hire an engineer lately? I assure you it is significantly harder than it sounds. One of the things that Revelo has recognized is something I've been talking about for a while, specifically that while talent is evenly distributed, opportunity is absolutely not. They're exposing a new talent pool to, basically, those of us without a presence in Latin America via their platform. It's the largest tech talent marketplace in Latin America with over a million engineers in their network, which includes—but isn't limited to—talent in Mexico, Costa Rica, Brazil, and Argentina. Now, not only do they wind up spreading all of their talent on English ability, as well as you know, their engineering skills, but they go significantly beyond that. Some of the folks on their platform are hands down the most talented engineers that I've ever spoken to. Let's also not forget that Latin America has high time zone overlap with what we have here in the United States, so you can hire full-time remote engineers who share most of the workday as your team. It's an end-to-end talent service, so you can find and hire engineers in Central and South America without having to worry about, frankly, the colossal pain of cross-border payroll and benefits and compliance because Revelo handles all of it. If you're hiring engineers, check out revelo.io/screaming to get 20% off your first three months. That's R-E-V-E-L-O dot I-O slash screaming.Corey: Couchbase Capella Database-as-a-Service is flexible, full-featured and fully managed with built in access via key-value, SQL, and full-text search. Flexible JSON documents aligned to your applications and workloads. Build faster with blazing fast in-memory performance and automated replication and scaling while reducing cost. Capella has the best price performance of any fully managed document database. Visit couchbase.com/screaminginthecloud to try Capella today for free and be up and running in three minutes with no credit card required. Couchbase Capella: make your data sing.Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. My guest today is someone that I had the pleasure of meeting at re:Invent last year, but we'll get to that story in a minute. Casey Lee is the CTO with a company called Gaggle, which is—as they frame it—saving lives. Now, that seems to be a relatively common position that an awful lot of different tech companies take. “We're saving lives here.” It's, “You show banner ads and some of them are attack platforms for JavaScript malware. Let's be serious here.” Casey, thank you for joining me, and what makes the statement that Gaggle saves lives not patently ridiculous?Casey: Sure. Thanks, Corey. Thanks for having me on the show. So Gaggle, we're ed-tech company. We sell software to school districts, and school districts use our software to help protect their students while the students use the school-issued Google or Microsoft accounts.So, we're looking for signs of bullying, harassment, self-harm, and potentially suicide from K-12 students while they're using these platforms. They will take the thoughts, concerns, emotions they're struggling with and write them in their school-issued accounts. We detect that and then we notify the school districts, and they get the students the help they need before they can do any permanent damage to themselves. We protect about 6 million students throughout the US. We ingest a lot of content.Last school year, over 6 billion files, about the equal number of emails ingested. We're looking for concerning content and then we have humans review the stuff that our machine learning algorithms detect and flag. About 40 million items had to go in front of humans last year, resulted in about 20,000 what we call PSSes. These are Possible Student Situations where students are talking about harming themselves or harming others. And that resulted in what we like to track as lives saved. 1400 incidents last school year where a student was dealing with suicide ideation, they were planning to take their own lives. We detect that and get them help within minutes before they can act on that. That's what Gaggle has been doing. We're using tech, solving tech problems, and also saving lives as we do it.Corey: It's easy to lob a criticism at some of the things you're alluding to, the idea of oh, you're using machine learning on student data for young kids, yadda, yadda, yadda. Look at the outcome, look at the privacy controls you have in place, and look at the outcomes you're driving to. Now, I don't necessarily trust the number of school administrations not to become heavy-handed and overbearing with it, but let's be clear, that's not the intent. That is not what the success stories you have alluded to. I've got to say I'm a fan, so thanks for doing what you're doing. I don't say that very often to people who work in tech companies.Casey: Cool. Thanks, Corey.Corey: But let's rewind a bit because you and I had passed like ships in the night on Twitter for a while, but last year at re:Invent something odd happened. First, my business partner procrastinated at getting his ticket—that's not the odd part; he does that a lot—but then suddenly ticket sales slammed shut and none were to be had anywhere. You reached out with a, “Hey, I have a spare ticket because someone can't go. Let me get it to you.” And I said, “Terrific. Let me pay you for the ticket and take you to dinner.”You said, “Yes on the dinner, but I'd rather you just look at my AWS bill and don't worry about the cost of the ticket.” “All right,” said I. I know a deal when I see one. We grabbed dinner at the Venetian. I said, “Bust out your laptop.” And you said, “Oh, I was kidding.” And I said, “Great. I wasn't. Bust it out.”And you went from laughing to taking notes in about the usual time that happens when I start looking at these things. But how was your recollection of that? I always tend to romanticize some of these things. Like, “And then everyone's restaurant just turned, stopped, and clapped the entire time.” Maybe that part didn't happen.Casey: Everything was right up until the clapping part. That was a really cool experience. I appreciate you walking through that with me. Yeah, we've got lots of opportunity to save on our AWS bill here at Gaggle, and in that little bit of time that we had together, I think I walked away with no more than a dozen ideas for where to shave some costs. The most obvious one, the first thing that you keyed in on, is we had RIs coming due that weren't really well-optimized and you steered me towards savings plans. We put that in place and we're able to apply those savings plans not just to our EC2 instances but also to our serverless spend as well.So, that was a very worthwhile and cost-effective dinner for us. The thing that was most surprising though, Corey, was your approach. Your approach to how to review our bill was not what I thought at all.Corey: Well, what did you expect my approach was going to be? Because this always is of interest to me. Like, do you expect me to, like, whip a portable machine learning rig out of my backpack full of GPUs or something?Casey: I didn't know if you had, like, some secret tool you were going to hit, or if nothing else, I thought you were going to go for the Cost Explorer. I spend a lot of time in Cost Explorer, that's my go-to tool, and you wanted nothing to do with Cost Exp—I think I was actually pulling up Cost Explorer for you and you said, “I'm not interested. Take me to the bills.” So, we went right to the billing dashboard, you started opening up the invoices, and I thought to myself, “I don't remember the last time I looked at an AWS invoice.” I just, it's noise; it's not something that I pay attention to.And I learned something, that you get a real quick view of both the cost and the usage. And that's what you were keyed in on, right? And you were looking at things relative to each other. “Okay, I have no idea about Gaggle or what they do, but normally, for a company that's spending x amount of dollars in EC2, why is your data transfer cost the way it is? Is that high or low?” So, you're looking for kind of relative numbers, but it was really cool watching you slice and dice that bill through the dashboard there.Corey: There are a few things I tie together there. Part of it is that this is sort of a surprising thing that people don't think about but start with big numbers first, rather than going alphabetically because I don't really care about your $6 Alexa for Business spend. I care a bit more about the $6 million, or whatever it happens to be at EC2—I'm pulling numbers completely out of the ether, let's be clear; I don't recall what the exact magnitude of your bill is and it's not relevant to the conversation.And then you see that and it's like, “Huh. Okay, you're spending $6 million on EC2. Why are you spending 400 bucks on S3? Seems to me that those two should be a little closer aligned. What's the deal here? Oh, God, you're using eight petabytes of EBS volumes. Oh, dear.”And just, it tends to lead to interesting stuff. Break it down by region, service, and use case—or usage type, rather—is what shows up on those exploded bills, and that's where I tend to start. It also is one of the easiest things to wind up having someone throw into a PDF and email my way if I'm not doing it in a restaurant with, you know, people clapping standing around.Casey: [laugh]. Right.Corey: I also want to highlight that you've been using AWS for a long time. You're a Container Hero; you are not bad at understanding the nuances and depths of AWS, so I take praise from you around this stuff as valuing it very highly. This stuff is not intuitive, it is deeply nuanced, and you have a business outcome you are working towards that invariably is not oriented day in day out around, “How do I get these services for less money than I'm currently paying?” But that is how I see the world and I tend to live in a very different space just based on the nature of what I do. It's sort of a case study and the advantage of specialization. But I know remarkably little about containers, which is how we wound up reconnecting about a week or so before we did this recording.Casey: Yeah. I saw your tweet; you were trying to run some workload—container workload—and I could hear the frustration on the other end of Twitter when you were shaking your fist at—Corey: I should not tweet angrily, and I did in this case. And, eh, every time I do I regret it. But it played well with the people, so that does help. I believe my exact comment was, “‘me: I've got this container. Run it, please.' ‘Google Cloud: Run. You got it, boss.' AWS has 17 ways to run containers and they all suck.”And that's painting with an overly broad brush, let's be clear, but that was at the tail end of two or three days of work trying to solve a very specific, very common, business problem, that I was just beating my head off of a wall again and again and again. And it took less than half an hour from start to finish with Google Cloud Run and I didn't have to think about it anymore. And it's one of those moments where you look at this and realize that the future is here, we just don't see it in certain ways. And you took exception to this. So please, let's dive in because 280 characters of text after half a bottle of wine is not the best context to have a nuanced discussion that leaves friendships intact the following morning.Casey: Nice. Well, I just want to make sure I understand the use case first because I was trying to read between the lines on what you needed, but let me take a guess. My guess is you got your source code in GitHub, you have a Docker file, and you want to be able to take that repo from GitHub and just have it continuously deployed somewhere in Run. And you don't want to have headaches with it; you just want to push more changes up to GitHub, Docker Build runs and updates some service somewhere. Am I right so far?Corey: Ish, but think a little further up the stack. It was in service of this show. So, this show, as people who are listening to this are probably aware by this point, periodically has sponsors, which we love: We thank them for participating in the ongoing support of this show, which empowers conversations like this. Sometimes a sponsor will come to us with, “Oh, and here's the URL we want to give people.” And it's, “First, you misspelled your company name from the common English word; there are three sublevels within the domain, and then you have a complex UTM tagging tracking co—yeah, you realize people are driving to work when they're listening to this?”So, I've built a while back a link shortener, snark.cloud because is it the shortest thing in the world? Not really, but it's easily understandable when I say that, and people hear it for what it is. And that's been running for a long time as an S3 bucket with full of redirects, behind CloudFront. So, I wind up adding a zero-byte object with a redirect parameter on it, and it just works.Now, the challenge that I have here as a business is that I am increasingly prolific these days. So, anything that I am not directly required to be doing, I probably shouldn't necessarily be the one to do it. And care and feeding of those redirect links is a prime example of this. So, I went hunting, and the things that I was looking for were, obviously, do the redirect. Now, if you pull up GitHub, there are hundreds of solutions here.There are AWS blog posts. One that I really liked and almost got working was Eric Johnson's three-part blog post on how to do it serverlessly, with API Gateway, and DynamoDB, no Lambdas required. I really liked aspects of what that was, but it was complex, I kept smacking into weird challenges as I went, and front end is just baffling to me. Because I needed a front end app for people to be able to use here; I need to be able to secure that because it turns out that if you just have a, anyone who stumbles across the URL can redirect things to other places, well, you've just empowered a whole bunch of spam email, and you're going to find that service abused, and everyone starts blocking it, and then you have trouble. Nothing lasts the first encounter with jerks.And I was getting more and more frustrated, and then I found something by a Twitter engineer on GitHub, with a few creative search terms, who used to work at Google Cloud. And what it uses as a client is it doesn't build any kind of custom web app. Instead, as a database, it uses not S3 objects, not Route 53—the ideal database—but a Google sheet, which sounds ridiculous, but every business user here knows how to use that.Casey: Sure.Corey: And it looks for the two columns. The first one is the slug after the snark.cloud, and the second is the long URL. And it has a TTL of five seconds on cache, so make a change to that spreadsheet, five seconds later, it's live. Everyone gets it, I don't have to build anything new, I just put it somewhere around the relevant people can access it, I gave him a tutorial and a giant warning on it, and everyone gets that. And it just works well. It was, “Click here to deploy. Follow the steps.”And the documentation was a little, eh, okay, I had to undo it once and redo it again. Getting the domain registered was getting—ported over took a bit of time, and there were some weird SSL errors as the certificates were set up, but once all of that was done, it just worked. And I tested the heck out of it, and cold starts are relatively low, and the entire thing fits within the free tier. And it is reminiscent of the magic that I first saw when I started working with some of the cloud providers services, years ago. It's been a long time since I had that level of delight with something, especially after three days of frustration. It's one of the, “This is a great service. Why are people not shouting about this from the rooftops?” That was my perspective. And I put it out on Twitter and oh, Lord, did I get comments. What was your take on it?Casey: Well, so my take was, when you're evaluating a platform to use for running your applications, how fast it can get you to Hello World is not necessarily the best way to go. I just assumed you're wrong. I assumed of the 17 ways AWS has to run containers, Corey just doesn't understand. And so I went after it. And I said, “Okay, let me see if I can find a way that solves his use case, as I understand it, through a quick tweet.”And so I tried to App Runner; I saw that App Runner does not meet your needs because you have to somehow get your Docker image pushed up to a repo. App Runner can take an image that's already been pushed up and deployed for you or it can build from source but neither of those were the way I understood your use case.Corey: Having used App Runner before via the Copilot CLI, it is the closest as best I can tell to achieving what I want. But also let's be clear that I don't believe there's a free tier; there needs to be a load balancer in front of it, so you're starting with 15 bucks a month for this thing. Which is not the end of the world. Had I known at the beginning that all of this was going to be there, I would have just signed up for a bit.ly account and called it good. But here we are.Casey: Yeah. I tried Copilot. Copilot is a great developer experience, but it also is just pulling together tons of—I mean just trying to do a Copilot service deploy, VPCs are being created and tons IAM roles are being created, code pipelines, there's just so much going on. I was like 20 minutes into it, and I said, “Yeah, this is not fitting the bill for what Corey was looking for.” Plus, it doesn't solve my the way I understood your use case, which is you don't want to worry about builds, you just want to push code and have new Docker images get built for you.Corey: Well, honestly, let's be clear here, once it's up and running, I don't want to ever have to touch the silly thing again.Casey: Right.Corey: And that's so far has been the case, after I forked the repo and made a couple of changes to it that I wanted to see. One of them was to render the entire thing case insensitive because I get that one wrong a lot, and the other is I wanted to change the permanent 301 redirect to a temporary 302 redirect because occasionally, sponsors will want to change where it goes in the fullness of time. And that is just fine, but I want to be able to support that and not have to deal with old cached data. So, getting that up and running was a bit of a challenge. But the way that it worked, was following the instructions in the GitHub repo.The developer environment had spun up in the Google's Cloud Shell was just spectacular. It prompted me for a few things and it told me step by step what to do. This is the sort of thing I could have given a basically non-technical user, and they would have had success with it.Casey: So, I tried it as well. I said, “Well, okay, if I'm going to respond to Corey here and challenge him on this, I need to try Cloud Run.” I had no experience with Cloud Run. I had a small example repo that loosely mapped what I understood you were trying to do. Within five minutes, I had Cloud Run working.And I was surprised anytime I pushed a new change, within 45 seconds the change was built and deployed. So, here's my conclusion, Corey. Google Cloud Run is great for your use case, and AWS doesn't have the perfect answer. But here's my challenge to you. I think that you just proved why there's 17 different ways to run containers on AWS, is because there's that many different types of users that have different needs and you just happen to be number 18 that hasn't gotten the right attention yet from AWS.Corey: Well, let's be clear, like, my gag about 17 ways to run containers on AWS was largely a joke, and it went around the internet three times. So, I wrote a list of them on the blog post of “17 Ways to Run Containers in AWS” and people liked it. And then a few months later, I wrote “17 More Ways to Run Containers on AWS” listing 17 additional services that all run containers.And my favorite email that I think I've ever received in feedback was from a salty AWS employee, saying that one of them didn't really count because of some esoteric reason. And it turns out that when I'm trying to make a point of you have a sarcastic number of ways to run containers, pointing out that well, one of them isn't quite valid, doesn't really shatter the argument, let's be very clear here. So, I appreciate the feedback, I always do. And it's partially snark, but there is an element of truth to it in that customers don't want to run containers, by and large. That is what they do in service of a business goal.And they want their application to run which is in turn to serve as the business goal that continues to abstract out into, “Remain a going concern via the current position the company stakes out.” In your case, it is saving lives; in my case, it is fixing horrifying AWS bills and making fun of Amazon at the same time, and in most other places, there are somewhat more prosaic answers to that. But containers are simply an implementation detail, to some extent—to my way of thinking—of getting to that point. An important one [unintelligible 00:18:20], let's be clear, I was very anti-container for a long time. I wrote a talk, “Heresy in the Church of Docker” that then was accepted at ContainerCon. It's like, “Oh, boy, I'm not going to leave here alive.”And the honest answer is many years later, that Kubernetes solves almost all the criticisms that I had with the downside of well, first, you have to learn Kubernetes, and that continues to be mind-bogglingly complex from where I sit. There's a reason that I've registered kubernetestheeasyway.com and repointed it to ECS, Amazon's container service that is not requiring you to cosplay as a cloud provider yourself. But even ECS has a number of challenges to it, I want to be very clear here. There are no silver bullets in this.And you're completely correct in that I have a large, complex environment, and the application is nuanced, and I'm willing to invest a few weeks in setting up the baseline underlying infrastructure on AWS with some of these services, ideally not all of them at once because that's something a lunatic would do, but getting them up and running. The other side of it, though, is that if I am trying to evaluate a cloud provider's handling of containers and how this stuff works, the reason that everyone starts with a Hello World-style example is that it delivers ideally, the meantime to dopamine. There's a reason that Hello World doesn't have 18 different dependencies across a bunch of different databases and message queues and all the other complicated parts of running a modern application. Because you just want to see how it works out of the gate. And if getting that baseline empty container that just returns the string ‘Hello World' is that complicated and requires that much work, my takeaway is not that this user experience is going to get better once I'd make the application itself more complicated.So, I find that off-putting. My approach has always been find something that I can get the easy, minimum viable thing up and running on, and then as I expand know that you'll be there to catch me as my needs intensify and become ever more complex. But if I can't get the baseline thing up and running, I'm unlikely to be super enthused about continuing to beat my head against the wall like, “Well, I'll just make it more complex. That'll solve the problem.” Because it often does not. That's my position.Casey: Yeah, I agree that dopamine hit is valuable in getting attached to want to invest into whatever tech stack you're using. The challenge is your second part of that. Your second part is will it grow with me and scale with me and support the complex edge cases that I have? And the problem I've seen is a lot of organizations will start with something that's very easy to get started with and then quickly outgrow it, and then come up with all sorts of weird Rube Goldberg-type solutions. Because they jumped all in before seeing—I've got kind of an example of that.I'm happy to announce that there's now 18 ways to run containers on AWS. Because in your use case, in the spirit of AWS customer obsession, I hear your use case, I've created an open-source project that I want to share called Quinntainers—Corey: Oh, no.Casey: —and it solves—yes. Quinntainers is live and is ready for the world. So, now we've got 18 ways to run containers. And if you have Corey's use case of, “Hey, here's my container. Run it for me,” now we've got a one command that you can run to get things going for you. I can share a link for you and you could check it out. This is a [unintelligible 00:21:38]—Corey: Oh, we're putting that in the [show notes 00:21:37], for sure. In fact, if you go to snark.cloud/quinntainers, you'll find it.Casey: You'll find it. There you go. The idea here was this: There is a real use case that you had, and I looked at AWS does not have an out-of-the-box simple solution for you. I agree with that. And Google Cloud Run does.Well, the answer would have been from AWS, “Well, then here, we need to make that solution.” And so that's what this was, was a way to demonstrate that it is a solvable problem. AWS has all the right primitives, just that use case hadn't been covered. So, how does Quinntainers work? Real straightforward: It's a command-line—it's an NPM tool.You just run a [MPX 00:22:17] Quinntainer, it sets up a GitHub action role in your AWS account, it then creates a GitHub action workflow in your repo, and then uses the Quinntainer GitHub action—reusable action—that creates the image for you; every time you push to the branch, pushes it up to ECR, and then automatically pushes up that new version of the image to App Runner for you. So, now it's using App Runner under the covers, but it's providing that nice developer experience that you are getting out of Cloud Run. Look, is container really the right way to go with running containers? No, I'm not making that point at all. But the point is it is a—Corey: It might very well be.Casey: Well, if you want to show a good Hello World experience, Quinntainer's the best because within 30 seconds, your app is now set up to continuously deliver containers into AWS for your very specific use case. The problem is, it's not going to grow for you. I mean that it was something I did over the weekend just for fun; it's not something that would ever be worthy of hitching up a real production workload to. So, the point there is, you can build frameworks and tools that are very good at getting that initial dopamine hit, but then are not going to be there for you unnecessarily as you mature and get more complex.Corey: And yet, I've tilted a couple of times at the windmill of integrating GitHub actions in anything remotely resembling a programmatic way with AWS services, as far as instance roles go. Are you using permanent credentials for this as stored secrets or are you doing the [OICD 00:23:50][00:23:50] handoff?Casey: OIDC. So, what happens is the tool creates the IAM role for you with the trust policy on GitHub's OIDC provider, sets all that up for you in your account, locks it down so that just your repo and your main branch is able to push or is able to assume the role, the role is set up just to allow deployments to App Runner and ECR repository. And then that's it. At that point, it's out of your way. And you're just git push, and couple minutes later, your updates are now running an App Runner for you.Corey: This episode is sponsored in part by our friends at Vultr. Optimized cloud compute plans have landed at Vultr to deliver lightning fast processing power, courtesy of third gen AMD EPYC processors without the IO, or hardware limitations, of a traditional multi-tenant cloud server. Starting at just 28 bucks a month, users can deploy general purpose, CPU, memory, or storage optimized cloud instances in more than 20 locations across five continents. Without looking, I know that once again, Antarctica has gotten the short end of the stick. Launch your Vultr optimized compute instance in 60 seconds or less on your choice of included operating systems, or bring your own. It's time to ditch convoluted and unpredictable giant tech company billing practices, and say goodbye to noisy neighbors and egregious egress forever.Vultr delivers the power of the cloud with none of the bloat. "Screaming in the Cloud" listeners can try Vultr for free today with a $150 in credit when they visit getvultr.com/screaming. That's G E T V U L T R.com/screaming. My thanks to them for sponsoring this ridiculous podcast.Corey: Don't undersell what you've just built. This is something that—is this what I would use for a large-scale production deployment, obviously not, but it has streamlined and made incredibly accessible things that previously have been very complex for folks to get up and running. One of the most disturbing themes behind some of the feedback I got was, at one point I said, “Well, have you tried running a Docker container on Lambda?” Because now it supports containers as a packaging format. And I said no because I spent a few weeks getting Lambda up and running back when it first came out and I've basically been copying and pasting what I got working ever since the way most of us do.And response is, “Oh, that explains a lot.” With the implication being that I'm just a fool. Maybe, but let's be clear, I am never the only person in the room who doesn't know how to do something; I'm just loud about what I don't know. And the failure mode of a bad user experience is that a customer feels dumb. And that's not okay because this stuff is complicated, and when a user has a bad time, it's a bug.I learned that in 2012. From Jordan Sissel the creator of LogStash. He has been an inspiration to me for the last ten years. And that's something I try to live by that if a user has a bad time, something needs to get fixed. Maybe it's the tool itself, maybe it's the documentation, maybe it's the way that GitHub repo's readme is structured in a way that just makes it accessible.Because I am not a trailblazer in most things, nor do I intend to be. I'm not the world's best engineer by a landslide. Just look at my code and you'd argue the fact that I'm an engineer at all. But if it's bad and it works, how bad is it? Is sort of the other side of it.So, my problem is that there needs to be a couple of things. Ignore for a second the aspect of making it the right answer to get something out of the door. The fact that I want to take this container and just run it, and you and I both reach for App Runner as the default AWS service that does this because I've been swimming in the AWS waters a while and you're a frickin AWS Container Hero, where it is expected that you know what most of these things do. For someone who shows up on the containers webpage—which by the way lists, I believe 15 ways to run containers on mobile and 19 ways to run containers on non-mobile, which is just fascinating in its own right—and it's overwhelming, it's confusing, and it's not something that makes it is abundantly clear what the golden path is. First, get it up and working, get it running, then you can add nuance and flavor and the rest, and I think that's something that's gotten overlooked in our mad rush to pretend that we're all Google engineers, circa 2012.Casey: Mmm. I think people get stressed out when they tried to run containers in AWS because they think, “What is that golden path?” You said golden path. And my advice to people is there is no golden path. And the great thing about AWS is they do continue to invest in the solutions they come up with. I'm still bitter about Google Reader.Corey: As am I.Casey: Yeah. I built so much time getting my perfect set of RSS feeds and then I had to find somewhere else to—with AWS, the different offerings that are available for running containers, those are there intentionally, it's not by accident. They're there to solve specific problems, so the trick is finding what works best for you and don't feel like one is better than the other is going to get more attention than others. And they each have different use cases.And I approach it this way. I've seen a couple of different people do some great flowcharts—I think Forrest did one, Vlad did one—on ways to make the decision on how to run your containers. And I break it down to three questions. I ask people first of all, where are you going to run these workloads? If someone says, “It has to be in the data center,” okay, cool, then ECS Anywhere or EKS Anywhere and we'll figure out if Kubernetes is needed.If they need specific requirements, so if they say, “No, we can run in the cloud, but we need privileged mode for containers,” or, “We need EBS volumes,” or, “We want really small container sizes,” like, less than a quarter-VCP or less than half a gig of RAM—or if you have custom log requirements, Fargate is not going to work for you, so you're going to run on EC2. Otherwise, run it on Fargate. But that's the first question. Figure out where are you going to run your containers. That leads to the second question: What's your control plane?But those are different, sort of related but different questions. And I only see six options there. That's App Runner for your control plane, LightSail for your control plane, Rosa if you're invested in OpenShift already, EKS either if you have Momentum and Kubernetes or you have a bunch of engineers that have a bunch of experience with Kubernetes—if you don't have either, don't choose it—or ECS. The last option Elastic Beanstalk, but let's leave that as a—if you're not currently invested in Elastic Beanstalk don't start today. But I look at those as okay, so I—first question, where am I going to run my containers? Second question, what do I want to use for my control plane? And there's different pros and cons of each of those.And then the third question, how do I want to manage them? What tools do I want to use for managing deployment? All those other tools like Copilot or App2Container or Proton, those aren't my control plane; those aren't where I run my containers; that's how I manage, deploy, and orchestrate all the different containers. So, I look at it as those three questions. But I don't know, what do you think of that, Corey?Corey: I think you're onto something. I think that is a terrific way of exploring that question. I would argue that setting up a framework like that—one or very similar—is what the AWS containers page should be, just coming from the perspective of what is the neophyte customer experience. On some level, you almost need a slide of have choose your level of experience ranging from, “What's a container?” To, “I named my kid Kubernetes because I make terrible life decisions,” and anywhere in between.Casey: Sure. Yeah, well, and I think that really dictates the control plane level. So, for example, LightSail, where does LightSail fit? To me, the value of LightSail is the simplicity. I'm looking at a monthly pricing: Seven bucks a month for a container.I don't know how [unintelligible 00:30:23] works, but I can think in terms of monthly pricing. And it's tailored towards a console user, someone just wants to click in, point to an image. That's a very specific user, there's thousands of customers that are very happy with that experience, and they use it. App Runner presents that scale to zero. That's one of the big selling points I see with App Runner. Likewise, with Google Cloud Run. I've got that scale to zero. I can't do that with ECS, or EKS, or any of the other platforms. So, if you've got something that has a ton of idle time, I'd really be looking at those. I would argue that I think I did the math, Google Cloud Run is about 30% more expensive than App Runner.Corey: Yeah, if you disregard the free tier, I think that's have it—running persistently at all times throughout the month, the drop-out cold starts would cost something like 40 some odd bucks a month or something like that. Don't quote me on it. Again and to be clear, I wound up doing this very congratulatory and complimentary tweet about them on I think it was Thursday, and then they immediately apparently took one look at this and said, “Holy shit. Corey's saying nice things about us. What do we do? What do we do?” Panic.And the next morning, they raised prices on a bunch of cloud offerings. Whew, that'll fix it. Like—Casey: [laugh].Corey: Di-, did you miss the direction you're going on here? No, that's the exact opposite of what you should be doing. But here we are. Interestingly enough, to tie our two conversation threads together, when I look at an AWS bill, unless you're using Fargate, I can't tell whether you're using Kubernetes or not because EKS is a small charge. And almost every case for the control plane, or Fargate under it.Everything else just manifests as EC2 spend. From the perspective of the cloud provider. If you're running a Kubernetes cluster, it is a single-tenant application that can have some very funky behaviors like cross-AZ chatter back and fourth because there's no internal mechanism to say talk to the free thing, rather than the two cents a gigabyte thing. It winds up spinning up and down in a bunch of different ways, and the behavior patterns, because of how placement works are not necessarily deterministic, depending upon workload. And that becomes something that people find odd when, “Okay, we look at our bill for a week, what can you say?”“Well, first question. Are you running Kubernetes at all?” And they're like, “Who invited these clowns?” Understand, we're not prying into your workloads for a variety of excellent legal and contractual reasons, here. We are looking at how they behave, and for specific workloads, once we have a conversation engineering team, yeah, we're going to dive in, but it is not at all intuitive from the outside to make any determination whether you're running containers, or whether you're running VMs that you just haven't done anything with in 20 years, or what exactly is going on. And that's just an artifact of the billing system.Casey: We ran into this challenge in Gaggle. We don't use EKS, we use ECS, but we have some shared clusters, lots of EC2 spend, hard to figure out which team is creating the services that's running that up. We actually ended up creating a tool—we open-sourced it—ECS Chargeback, and what it does is it looks at the CPU memory reservations for each task definition, and then prorates the overall charge of the ECS cluster, and then creates metrics in Datadog to give us a breakdown of cost per ECS service. And it also measures what we like to refer to as waste, right? Because if you're reserving four gigs of memory, but your utilization never goes over two gigs, we're paying for that reservation, but you're underutilizing.So, we're able to also show which services have the highest degree of waste, not just utilization, so it helps us go after it. But this is a hard problem. I'd be curious, how do you approach these shared ECS resources and slicing and dicing those bills?Corey: Everyone has a different approach, too. This there is no unifiable, correct answer. A previous show guest, Peter Hamilton, over at Remind had done something very similar, open-sourced a bunch of these things. Understanding what your spend is important on this, and it comes down to getting at the actual business concern because in some cases, effectively dead reckoning is enough. You take a look at the cluster that is really hard to attribute because it's a shared service. Great. It is 5% of your bill.First pass, why don't we just agree that it is a third for Service A, two-thirds for Service B, and we'll call it mostly good at that point? That can be enough in a lot of cases. With scale [laugh] you're just sort of hand-waving over many millions of dollars a year there. How about we get into some more depth? And then you start instrumenting and reporting to something, be it CloudWatch, be a Datadog, be it something else, and understanding what the use case is.In some cases, customers have broken apart shared clusters for that specific reason. I don't think that's necessarily the best approach from an engineering perspective, but again, this is not purely an engineering decision. It comes down to serving the business need. And if you're taking up partial credits on that cluster, for a tax credit for R&D for example, you want that position to be extraordinarily defensible, and spending a few extra dollars to ensure that it is the right business decision. I mean, again, we're pure advisory; we advise customers on what we would do in their position, but people often mistake that to be we're going to go for the lowest possible price—bad idea, or that we're going to wind up doing this from a purely engineering-centric point of view.It's, be aware of that in almost every case, with some very notable weird exceptions, the AWS Bill costs significantly less than the payroll expense that you have of people working on the AWS environment in various ways. People are more expensive, so the idea of, well, you can save a whole bunch of engineering effort by spending a bit more on your cloud, yeah, let's go ahead and do that.Casey: Yeah, good point.Corey: The real mark of someone who's senior enough is their answer to almost any question is, “It depends.” And I feel I've fallen into that trap as well. Much as I'd love to sit here and say, “Oh, it's really simple. You do X, Y, and Z.” Yeah… honestly, my answer, the simple answer, is I think that we orchestrate a cyber-bullying campaign against AWS through the AWS wishlist hashtag, we get people to harass their account managers with repeated requests for, “Hey, could you go ahead and [dip 00:36:19] that thing in—they give that a plus-one for me, whatever internal system you're using?”Just because this is a problem we're seeing more and more. Given that it's an unbounded growth problem, we're going to see it more and more for the foreseeable future. So, I wish I had a better answer for you, but yeah, that's stuff's super hard is honest, but it's also not the most useful answer for most of us.Casey: I'd love feedback from anyone from you or your team on that tool that we created. I can share link after the fact. ECS Chargeback is what we call it.Corey: Excellent. I will follow up with you separately on that. That is always worth diving into. I'm curious to see new and exciting approaches to this. Just be aware that we have an obnoxious talent sometimes for seeing these things and, “Well, what about”—and asking about some weird corner edge case that either invalidates the entire thing, or you're like, “Who on earth would ever have a problem like that?” And the answer is always, “The next customer.”Casey: Yeah.Corey: For a bounded problem space of the AWS bill. Every time I think I've seen it all, I just have to talk to one more customer.Casey: Mmm. Cool.Corey: In fact, the way that we approached your teardown in the restaurant is how we launched our first pass approach. Because there's value in something like that is different than the value of a six to eight-week-long, deep-dive engagement to every nook and cranny. And—Casey: Yeah, for sure. It was valuable to us.Corey: Yeah, having someone come in to just spend a day with your team, diving into it up one side and down the other, it seems like a weird thing, like, “How much good could you possibly do in a day?” And the answer in some cases is—we had a Honeycomb saying that in a couple of days of something like this, we wound up blowing 10% off their entire operating budget for the company, it led to an increased valuation, Liz Fong-Jones says that—on multiple occasions—that the company would not be what it was without our efforts on their bill, which is just incredibly gratifying to hear. It's easy to get lost in the idea of well, it's the AWS bill. It's just making big companies spend a little bit less to another big company. And that's not exactly, you know, saving the lives of K through 12 students here.Casey: It's opening up opportunities.Corey: Yeah. It's about optimizing for the win for everyone. Because now AWS gets a lot more money from Honeycomb than they would if Honeycomb had not continued on their trajectory. It's, you can charge customers a lot right now, or you can charge them a little bit over time and grow with them in a partnership context. I've always opted for the second model rather than the first.Casey: Right on.Corey: But here we are. I want to thank you for taking so much time out of well, several days now to argue with me on Twitter, which is always appreciated, particularly when it's, you know, constructive—thanks for that—Casey: Yeah.Corey: For helping me get my business partner to re:Invent, although then he got me that horrible puzzle of 1000 pieces for the Cloud-Native Computing Foundation landscape and now I don't ever want to see him again—so you know, that happens—and of course, spending the time to write Quinntainers, which is going to be at snark.cloud/quinntainers as soon as we're done with this recording. Then I'm going to kick the tires and send some pull requests.Casey: Right on. Yeah, thanks for having me. I appreciate you starting the conversation. I would just conclude with I think that yes, there are a lot of ways to run containers in AWS; don't let it stress you out. They're there for intention, they're there by design. Understand them.I would also encourage people to go a little deeper, especially if you got a significantly large workload. You got to get your hands dirty. As a matter of fact, there's a hands-on lab that a company called Liatrio does. They call it their Night Lab; it's a one-day free, hands-on, you run legacy monolithic job applications on Kubernetes, gives you first-hand experience on how to—gets all the way up into observability and doing things like Canary deployments. It's a great, great lab.But you got to do something like that to really get your hands dirty and understand how these things work. So, don't sweat it; there's not one right way. There's a way that will probably work best for each user, and just take the time and understand the ways to make sure you're applying the one that's going to give you the most runway for your workload.Corey: I will definitely dig into that myself. But I think you're right, I think you have nailed a point that is, again, a nuanced one and challenging to put in a rage tweet. But the services don't exist in a vacuum. They're not there because, despite the joke, someone wants to get promoted. It's because there are customer needs that are going on that, and this is another way of meeting those needs.I think there could be better guidance, but I also understand that there are a lot of nuanced perspectives here and that… hell is someone else's workflow—Casey: [laugh].Corey: —and there's always value in broadening your perspective a bit on those things. If people want to learn more about you and how you see the world, where's the best place to find you?Casey: Probably on Twitter: twitter.com/nektos, N-E-K-T-O-S.Corey: That might be the first time Twitter has been described as a best place for anything. But—Casey: [laugh].Corey: Thank you once again, for your time. It is always appreciated.Casey: Thanks, Corey.Corey: Casey Lee, CTO at Gaggle and AWS Container Hero. And apparently writing code in anger to invalidate my points, which is always appreciated. Please do more of that, folks. I'm Cloud Economist Corey Quinn, and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, or the YouTube comments, which is always a great place to go reading, whereas if you've hated this podcast, please leave a five-star review in the usual places and an angry comment telling me that I'm completely wrong, and then launching your own open-source tool to point out exactly what I've gotten wrong this time.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.
About ScottCloud security historian.Developed flaws.cloud, CloudMapper, and Parliament.Founding team for fwd:cloudsecLinks: Block: https://block.xyz/ Twitter: https://twitter.com/0xdabbad00 TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: This episode is sponsored in part by our friends at Vultr. Optimized cloud compute plans have landed at Vultr to deliver lightning fast processing power, courtesy of third gen AMD EPYC processors without the IO, or hardware limitations, of a traditional multi-tenant cloud server. Starting at just 28 bucks a month, users can deploy general purpose, CPU, memory, or storage optimized cloud instances in more than 20 locations across five continents. Without looking, I know that once again, Antarctica has gotten the short end of the stick. Launch your Vultr optimized compute instance in 60 seconds or less on your choice of included operating systems, or bring your own. It's time to ditch convoluted and unpredictable giant tech company billing practices, and say goodbye to noisy neighbors and egregious egress forever. Vultr delivers the power of the cloud with none of the bloat. "Screaming in the Cloud" listeners can try Vultr for free today with a $150 in credit when they visit getvultr.com/screaming. That's G E T V U L T R.com/screaming. My thanks to them for sponsoring this ridiculous podcast.Corey: Couchbase Capella Database-as-a-Service is flexible, full-featured and fully managed with built in access via key-value, SQL, and full-text search. Flexible JSON documents aligned to your applications and workloads. Build faster with blazing fast in-memory performance and automated replication and scaling while reducing cost. Capella has the best price performance of any fully managed document database. Visit couchbase.com/screaminginthecloud to try Capella today for free and be up and running in three minutes with no credit card required. Couchbase Capella: make your data sing.Corey: Welcome to Screaming in the Cloud, I'm Corey Quinn. I am joined by a returning guest with a bit of a different job. Scott Piper was formerly an independent security researcher—basically the independent security researcher in the AWS space—but now he's a Principal Engineer over at Block. Scott, welcome back.Scott: Thanks for having me, again, Corey.Corey: So, you've taken a corporate job, and when that happened, I have to confess, I was slightly discouraged because oh, now it's going to be like one of those stories of when someone you know goes to work at Apple because no one knows anyone at Apple; we just used to know people who went there and then we kind of lost touch because it's a very insular thing. Not the Block slash Square slash whatever they're calling themselves this week has that reputation. But InfoSec is always a very nuanced space and companies that have large footprints and, you know, handle financial transaction processing generally don't encourage loud voices that attract attention around anything that isn't directly aligned with the core mission of the company. But you're still as public and prolific as ever. Was that a difficult balance for you to strike?Scott: So, when I was considering employment options, that was something that I made clear to any companies that I was talking to, that this is something that probably will and should continue because a lot of my value to these companies is because I'm able to have discussions, able to impact change because of that public persona. So yeah, so I think that it was something that they were aware of, and a risk that they took. [laugh]. But yeah, it's been useful.Corey: This is the sort of conversation I would have expected to have with, “Yeah, things seem to be continuing the same, and I haven't rocked any boats, yet and they haven't fired me, knock on wood.” Except that recently you've launched yet something else that I am personally a fan of. Now, before we get into the specifics of what it is you're up to these days, I should call out that since your last appearance on this show, I have really leaned into the Thursday newsletter podcast duo of Last Week in AWS: Security Edition. Rounding up what happened the previous week—yes, it was the previous week, and it comes out on Thursdays—because, you know, timing and publication, things are hard, computers, you know how it is—aimed at a target audience that is very much not you: People who have to care about security, but are not immersed in the space. It's a, “All right, what now? What do I have to pay attention to?”Because there's a lot of noise in this space, there's a lot of vendor-captured stuff out there. There's very little that is for people who work in security but don't have the word security anywhere near their job title. And I have to confess that one of my easy shortcuts is, “Oh, it's a pretty thin issue this week,” which is not inherently a bad thing, let's be clear, it's not yay, the three things you need to care about in security then eight more of filler; that's not what we're about. But I always want to make sure I didn't miss something meaningful, and one of my default publication steps is, “What's Scott been tweeting about this week?” Just to make sure that I didn't miss something that I really should be talking about.And every single time I pull up your Twitter feed, I find myself learning something, whether it's a new concept, or whether it is a nuance on an existing thing I was already aware of. So first, thank you for all the work that you do as a member of the community, despite having a, “Regular corporate job,” quote-unquote, you're still very present. It's appreciated.Scott: Thank you. Yeah. And I mean, that newsletter is great for people that don't want to be spending multiple hours per day trolling through Twitter and reading that. So, it provides, also, something great for the community to not have to spend all that time on Twitter like I do [laugh], unfortunately.Corey: It also strives—sort of—to be something approaching an upbeat position of not quite as cynical and sarcastic as the Monday issue. I try to be not just this is the thing that happened, but go a little bit into and this is why it matters. This is how to think about it. This thing that Amazon put out is nonsense, however, here's the kernel hidden within it that might lead to something, such as thinking about how you do sign-on, or how to think about protecting MFA devices, or stuff like that you normally care about a lot right after you really should have cared about it but didn't at all. So, it's just the idea of aiming in a slightly different audience.Scott: Yeah definitely. And it provides value that it does, it takes some delay so that you can read what everybody has written, how they've responded to the different news outtakes, you're not just including the hot takes. For example, as of this morning, there's a certain incident with an authentication provider, and it's not really clear if there was actually a breach or not. And so it's valuable to take a moment to understand what happened, get all the voices to have expressed their points, so you can summarize those issues.Corey: An internal term that we've used to describe the position here is that I am prolific but I also have things to do as a part of my job that do not involve sitting there hitting refresh on Twitter like mad all the time. The idea is to have the best take not the first take—Scott: Exactly.Corey: And if that means that I lose a bunch of eyeballs and early ad impressions in the middle of the night and whatnot, well, great. I don't sell ad impressions anyway, so what does it matter? It winds up lending itself to a more thoughtful analysis of figuring out, in the sober light of day, is this a nothing-burger or is this enormous? With that SSO issue that you're alluding to—[cough] Okta—sorry, something caught in my throat there—very clearly, something is going on, but if I had written next week's newsletter last night while it was still very unclear, it would have been a very different tone than the one that I would have written this morning after their public statement, and even still a certainly different tone that it would take a couple of days once more information is almost certain to come to light. And that is something that is, I think, underappreciated in certainly on Twitter, where an old tweet—there's nothing worse than an old tweet unless you're using it to drag someone for something—that, “Well, we have different perspectives on that nowadays. It's not 2018 anymore.” Right. Okay, cool.Scott: Yep. [laugh].Corey: But something that you've done has been a bit of a pivot lately. Historically, you have been right there in my sweet spot of needling cloud providers for their transgressions in various ways. Cool, right there with you. We could co-author a book on the subject. But lately, you've started a community list of [IMSDv2 00:07:04] abuses.Now, first, we should talk about what IMSDv2 is. It's the name that it clearly came from Amazon because that's a name only a cloud provider bad at naming things could possibly love. What is it?Scott: So, it's the Instance Metadata Service, Version Two. If there's a version two, you can imagine there was a version one at some point. And the version two—Corey: And there's a version two because Amazon prod—the first one was terrible, but they don't turn anything off, ever, so this is the way and the light and the future; we're going to leave that old thing around until your great-grandchild dies of old age.Scott: Exactly, yeah. So, when EC2s first came out, and IAM roles first came out, you wanted to give your EC2s the ability to use AWS privileges, so this is how those EC2s are getting access to their credentials that they can use. And the way in which this was originally done was there's this magic IP address, this 169.254.169.254 IP address, which is very important for security on AWS because if anything can access that magic IP address from an EC2 instance, you can steal their credentials of that EC2, and therefore basically become that EC2 instance, in terms of what it can do in the AWS environment.And so in 2019, there was a large breach of Capital One that was related to this. And so as a result of that—I think that AWS probably had this new version, probably, in the works for a while, but I think that motivated their faster release of this new version, and so IMDSv2 changed how you would obtain these credentials. So, you basically—instead of making a single GET request to this IP address, now you had to make multiple requests, they were now PUT request instead of a GET request, there was a challenge and response, there's the hop limit. So, there's all these various things that are going to make it harder and basically mitigate a lot of the different types of vulnerabilities that previously would be used in order to obtain these credentials. The problem, though, is that IMDSv1 still exists on EC2s, unless you as a customer are enforcing IMDSv2.And so, in order to do this in a large environment, it's difficult—theoretically, it's a simple thing; all you should have to do is update your SDK and now you're able to make use of the latest version. And if you're using any version of the SDK that was released in the past over two years, you already should be using IMDSv2 there, but you have to enforce it. And so that's where the problem is. And what was most problematic to me is now that I work for a company, we have run into the problem that there are some vendor solutions that we use that weren't allowing us to enforce IMDSv2 across all of our different accounts. And this is something I've heard from a number of other customers as well.And so I decided to create this list with vendors that I've had to deal with, vendors that other customers have had to deal with, in order to basically try and solve this problem once and for all. It's been multiple years now and a lot of these vendors, unfortunately, were also security vendors. And so that makes the conversation a little bit easier, to basically put them on this wall-of-shame and say, “You're a security vendor and you're not allowing your customers to enforce best practices of security.”Corey: I want to call on a couple of things around that. Originally the metadata service was used for a number of other things—still is—beyond credentials. It is not the credential service as envisioned by a lot of folks. The way that—also we'll find those credentials empty until there's an EC2 instance role, and those credentials will both be scoped what that instance does and automatically rotated in the fullness of time so they're not long-lived credentials that once you have them, they will last forever. This is, of course, a best practice and something you should be leveraging, but scope those credentials down, or you wind up with one of the ways that was chained together in the Capital One breach a few years ago.It's also worth noting that service would have been more useful earlier in time with a few functions. For example, you can use the metadata service to retrieve the instance tags about the EC2 instance. When I requested it in 2015, it was not possible. But they had released it in January of this year, 2022, long after we have all come up with workarounds for this, where we could have used that to set the hostname internally on the system, if you're looking for something basic and easy. It would have been something then you could have used to automatically self-register with DNS without having to jump through a whole bunch of hoops to do it manually.And you look at this, and it's wow, that's a whole lot of crappy tooling I can just throw into the trash heap of history you don't need anymore. But the IMSDv2, you're right, makes it a lot harder, there has to be a conversation, not just something you can sort of bankshot something off of to get access to it. And it's a terrific mitigation. What I've liked about your list of more or less shaming companies for doing this is, on the one hand, you have companies who take themselves off of the list as soon as it's up there. It's, “Oh, we love when people talk about us. Wait, what's that? They're saying something unkind? On the internet?” And they'll fix it, which honestly is better than I expected.And then every once in a while you'll see something that's horrifying of, “Oh, yeah, we're not vulnerable to that at all because we tell you to create permanent long-lived credentials, store them on disk and we'll use those instead.” And it's… that is, like, guaranteeing that no one is going to break down your door by making your walls out of tissue paper. Don't do that. Like, that has gone so far around the band that has come back around again. So, hopefully that got fixed.Scott: And I think you pointed out a couple of things I want to talk about with this is that, one, it has actually been very successful in terms of getting large vendors to make changes. Currently, of the seven vendors that have ever been listed there, are three of them have already made fixes and have been removed from the list. And the list has only been up for about a month. And so, in terms of getting enterprise solution vendors to make changes within, like, just a few weeks is very surprising to me. And these are things that people have been asking for for years now, and so it had motivated them a lot there.And the other thing that I want to point out is people have looked at the success that it's had and considered maybe we should make wall-of-shame lists, for all the things that we want. And I want to point out that there are some things about this problem, the IMDSv2 specifically, that make it work for having this wall-of-shame list like this. One of them is that not supporting or not allowing customers to enforce IMDSv2 is basically always bad. There is not a use case where you can make a claim—Corey: There is no nuance where that, in this case, is the thing to do, like having an open S3 bucket: There are use cases where that is very much something you want to do, but it's the uncommon case.Scott: Exactly. That I think is an important thing. Another thing is it's not just putting up a list, you know, like that is what people are seeing publicly, but behind the scenes, there's a lot of other things that are happening. One, I am communicating with various customers, customers that are reporting this issue to me, in order to try to better understand what's happening there, so that I can then relay that information to the company. So, I'm not just putting up the list; I'm also, behind the scenes, having conversations with these different companies to try to get timelines from them, to try to make sure that they are aware of the problem, they are aware that they're on this list, how to get off the list. So, there's that conversation happening.There's also the conversation that I'm happening with AWS in order to make various requests that AWS improve this for customers, to make this easier. And this is something that is public on that repo. I have my list of requests to AWS so that people can relay that to their own TAMs at AWS to basically say these are things we want as well. And so this includes things like, “I want an AWS account to have the ability to default to always be enforcing IMDSv2.” You know, so as an example, when you create an EC2 through the web console—which people can say, oh, you should always be using Infrastructure as Code; the reality is many folks are using the web console to create EC2s to do other changes.And when you create an EC2 in the web console, by default, it's going to allow IMDSv1 still. And so my request to AWS is, you should allow me to just default enforce IMDSv2. Also, the web console does not give you visibility into which EC2s are enforcing it and which ones are not. And also, you do not have the ability in the web console to enforce it. You cannot click on an EC2 and say, “Please enforce it now.”So, it's all these various, like, minor changes that I'm requesting AWS to do.Corey: It has to be done at instance creation time.Scott: Exactly. And so there is an API that you can make in order to change it afterwards, but that's only an API so you have to use the CLI or some other mechanism; you can't do it in the web console. But the other thing that I'm requesting AWS do is if security is a priority for AWS and they have all these other partners that are security companies, that they should be requiring their partners to also be enforcing this in their various products. So, if a partner is basically not allowing your AWS customers to enforce security best practices, then perhaps that partnership should be revoked in some way. And so that's a more aggressive thing that I'm asking AWS to do, but I think is reasonable.Corey: I'd also like them to get all of their own first-party services to support this, too.Scott: That's true as well. So, AWS is currently on the list. And so, they have one service, Data Pipelines, which if you are an AWS customer and you are using that service, you are not going to be able to enforce IMDSv2 in your environment. So, AWS themselves, unfortunately, is not allowing customers to enforce this. And then AWS themselves in their own production servers, we have seen indications that they do not enforce IMDSv2 on their own production servers.So, the best practice that they are telling customers to follow, they unfortunately are not following it themselves. And so the way in which we saw this was Orca is a security company that ended up finding this issue with AWS—and there's a lot of questions in terms of what all exactly they found—but they had this post that they called “Breaking Formation” in which they were somehow able to find—basically exploit to some degree—and again, it's unclear exactly what they were able to exploit here—but they were able to exploit AWS production servers that are responsible for the CloudFormation service. And in their blog post, they had a screenshot which showed that those production servers are not enforcing IMDSv2. And so AWS themselves is struggling with this as well, as are many customers. So, it's something that, you know, I put together this list of requests in hopes that AWS can make it easier for not only customers but also themselves to be able to enforce it.Corey: There are a lot of different things that we wish companies did differently, particularly if that company is AWS. Why is this the particular windmill that you've decided to tilt at given—let's say—it's not exactly slim pickins out there as far as changes that we wish companies would make? Obviously, you mentioned at one point, there is no drawback to enabling this, but a lot could be said for other aspects as well. Why is this one so important?Scott: So, in part, I personally have some, I guess, history with this [laugh], basically, IMDSv2, and so we can discuss this. This is back when Capital One had their breach in 2019, there was this Senator, Senator Ron Wyden, who sent this email over to AWS, to Steve Schmidt, who was the CISO at the time there and still is the CISO, and he basically—Corey: Now, he's head of security for all of Amazon.Scott: Yeah, yeah.Corey: CJ is now the AWS CISO. And he has the good sense to hide.Scott: Yeah. [laugh]. So, at the time, this Senator Ron Wyden had send over this email—and obviously it's not Senator Ron Wyden himself, you know, it's one of his, like, technical people on staff that is able to give him this information—and he sends this email to AWS saying, “Hey, this metadata service played a role in this very significant breach. Why hasn't this been fixed?” And Steve Schmidt responded, and because it's communications between a senator, I guess it has to become public.So, Steve Schmidt responds, saying that, “Hey, we never knew that this was an issue before,” is essentially what he responds with. And that irked me because I had reported this to AWS previously, as had many other people. So, there was a conference presentation by this guy Andrés Riancho at BlackHat, I believe in 2014, and he had presented previously in 2013, so it was a known issue; it had been around for a while. But I took the time to actually report it to AWS Security. So, I went through the correct channel of making sure that AWS was aware of a security concern, as a security researcher—so reporting it through that correct channel there—and provided Senator Ron Wyden with all this information.And so, then he then requested that the FTC begin a federal investigation into AWS, related to basically not following the best practices that security researchers have recommended. So, that was, kind of like, my early, I guess, involvement with this issue. So, it's something that I've been interested in for a while to make sure that this is resolved completely at some point.Corey: This episode is sponsored by our friends at Oracle Cloud. Counting the pennies, but still dreaming of deploying apps instead of “Hello, World” demos? Allow me to introduce you to Oracle's Always Free tier. It provides over 20 free services and infrastructure, networking, databases, observability, management, and security. And—let me be clear here—it's actually free. There's no surprise billing until you intentionally and proactively upgrade your account. This means you can provision a virtual machine instance or spin up an autonomous database that manages itself, all while gaining the networking, load balancing, and storage resources that somehow never quite make it into most free tiers needed to support the application that you want to build. With Always Free, you can do things like run small-scale applications or do proof-of-concept testing without spending a dime. You know that I always like to put asterisks next to the word free? This is actually free, no asterisk. Start now. Visit snark.cloud/oci-free that's snark.cloud/oci-free.Corey: It's always fun watching where people come from, as far as the security problems that they call out. There was, I believe in the cloud security forum Slack, a thread of recently about what security issues are top-of-mind and that should be fixed as a baseline expectation. In fact, let me dig it out because that is one of those things that I think is well worth having the conversation properly on this.Good examples of risky, insecure defaults in AWS. And people are talking about IMDSv1, and they're talking about all kinds of other in-depth things, and my contribution to it was, “If I go and I spin up an AWS account, until I go out of my way, I'm operating as root in that account. That seems bad.” And a few responses to that were oh, the basically facepalming, “Oh, of course.” I wish that there were an easy way to get AWS SSO as the default because it is the right answer for so many different things. It solves so many painful problems that otherwise you're going to wind up stuck with.And this stuff is hard and confusing; when people are starting out with this for the first time, they're not approaching this from, “All right, how do I be extremely secure?” They want to get some work done. For fun a year ago, I spun up a test account—unattached to any organization—and because account aliases are globally unique, I somehow came up with the account ‘shitposting' because that's pretty much what I use it for. The actual reason I wanted that was I wanted something completely unattached from any other account that I could easily take screenshots from at any point, and the worst case scenario is okay, I've exposed some credential of my own in an account that has no privileged access to anything; I just have to apologize for all the Bitcoin mining now. And honestly, I think AWS would love that marketing campaign; they'd see my face on a billboard looking horrified. It'll be great.But I turned on every security service as I went because, of course, security is the most important thing. And there were so many to turn on, and the bill was approaching 50 bucks a month for an empty account. And it's. It starts to feel a little weird and more than a little wrong.Scott: [laugh]. Yeah, my personal concern in terms of default security features is really that problem of the cost controls, I think that that still is a big issue that AWS does not have cost controls such that when a student wants to try and use AWS for the very first time and somehow they spin up large EC2 instance, or they just you know, end up creating an access key and that access key gets leaked and somehow their account gets compromised and used for Bitcoin mining, now they're stuck with that large AWS bill. For a student who has no budget, is in debt, and now is suddenly being, you know, hit with multiple thousands of dollars on their bill, that I think is very problematic, and that is something that I wish AWS would change as a default is basically, if you are creating AWS account for the very first time, have some type of—I don't know how this would look, but maybe just be able to say, like, I don't ever want this AWS account to spend more than $100 per month, and I'm okay if you end up destroying all my data in the account because I have no money and money is more important to me than whatever data I may store in here.Corey: Make an answer to that question mandatory, just as putting a credit card in is mandatory. Because there are two extremes here. It's more or less the same problem of AWS not knowing who its customers are beyond an AWS account, but there's a spectrum somewhere between I'm a student who wants to learn how the cloud works, and my approach to security is very much the same. Don't let randos spin up resources in my account, and I don't ever want to be charged. If that means you turn off my “Hello World” blog post, okay, great.On the other end, it's this is Netflix. And this is our, you know, eight-millionth account that we're spending up to do a thing and what do you mean you're applying service quotas to it? I thought we had an understanding?—everything is a service quota, let's be clear—Scott: Yep.Corey: —or a company that's about to run a Superbowl ad. Yeah, there's going to be a lot of traffic there. Don't touch it. Just make it work. We don't care what it costs.Understanding where you fall on the cost perspective—as well as a security point of view of, “We're a bank, which means forget security best practices, we have compliance obligations that cannot be altered in this account and here's what they are.” There has to be a way that is easy and approachable for people to wind up moving that slider to whatever position best represents them. Because there are accounts where I never want to be charged a thing. And that's an important thing because—and I've been talking about this for a while because I'm convinced it's a matter of time—that poor kid who wound up trading on margin at Robinhood, woke up saw that he was seven-hundred-and-some-odd grand in debt and killed himself. When it all settled out, I think he turned something like a $30,000 profit when all was said and done, which just serves to make it worse.I can see a scenario in which that happens, and part of the contributors to it are that we used to see that the surprise bill for compromised accounts was 10, 15, 20 grand. Now, they're 70 to 90 because there are more regions, more services to run containers—because of course there are—and the payoff is such that the people exploiting this have gotten very practiced and very operationalized at spinning up those resources quickly, and they cost a lot very quickly. I mean, the third use case that they're not aiming at yet is people like me, where it's, oh, you have a free account that sandboxed; I want to get the high score on the free tier because all their fraud is attuned to you making money. With me, it's nope, just going to run up the store to embarrass Amazon. That's not a common exploit vector, but I'm very much here.Scott: [laugh]. Yep. And that also is the thing though: The Denial of Wallet attack is also a concern on AWS, as well, where you've written a blog post about this, how if you are able to make use of data transfer in different ways, you can run up very high multi-million dollar bills in people's AWS accounts and even AWS's own protections and defenses against trying to look for cost spikes and things like that is delayed by multiple hours. And so you can still end up spending a lot of money in people's accounts, or one thing that's wild is an S3 object locking; that feature, the whole purpose behind it is to ensure data can never be deleted. It exists for various compliance reasons, so even AWS themselves cannot delete certain data.So, if an attacker is able to abuse that functionality in somebody's account, they can end up locking data such that for the next 100 years, it can never be deleted and you're going to have to pay for that for the next 100 years inside your account. The only way of not paying for that anymore is to move everything that you have in an AWS account to a new account, and then ask AWS to delete that account, which is not going to be reasonable under most circumstances.Corey: Yeah, alternatively, it's one of those scenarios where well, the only other option is to start physically ripping hard drives out of racks in a bunch of different data centers. It's wild to me. It's such an attack surface that honestly I believe for the longest time that AWS Security is otherworldly good. And as we start seeing from these breaches, no, what really is otherworldly good is their ability to apply pressure to people not to go public with things they discover that they then wind up keeping quiet because once this whole Orca stuff came out, we started digging, and Aidan Steele found some stuff where you could just get unfiltered, raw outputs of CloudTrail events by setting up a couple of rules in weird ways.And that was a giant problem, and it was never disclosed publicly. I don't know if any of my events were impacted; I can't trust that they would have told me if they were. And for the first time, I'm looking at things like confidential computing, which are designed around well, what if you don't trust your cloud provider? Historically, I guess I was naive because my approach was, “Well, then you shouldn't be using the cloud.” Now it's, “Well, that's actually kind of a good point.”Because it's not that I don't trust my cloud provider to necessarily do what they're telling me. I just don't trust them to tell me what they're doing. And that's part of it. The, “Well, we found an issue, but you can't prove we had an issue, so we're going to say nothing.” And when it comes to light—because it always does—it erodes trust in a big way. And trust is everything in cloud.Scott: Yeah. And so with some of the breaches that have come out, I created another GitHub repo to start tracking all the different security incidents that I could find for the three cloud providers, Azure, GCP, and AWS. And so on there, I started listing not only some of the blog posts from security companies that had been able to exploit vulnerabilities in the cloud providers, but also just anything else that I felt was a security mistake in some way. And so there's a number of things I tried to avoid on there. Like, I tried to avoid listing something that's kind of like a business decision, for example, services that get released that don't have CloudTrail support. That's a security concern to me, but that's kind of a business decision that they decided to release a service before it supported all that functionality.So, I tried to start listing off all those different things in order to also keep track of you know, is there a security provider that's worse than the others? Are there any type of common patterns that I can see? And so I tried to look through some of those different things. And that's been interesting because also I really only focus on AWS, and so I haven't really known what all has been happening with GCP and Azure. And that was interesting because there's been two issues that have happened on AWS where the exact same issue happened on the other cloud providers. And so that tells me, that's concerning to me because that tells me tht—Corey: Because those are not discovered at the same time let's be clear.Scott: Yeah. These were, like, over a year apart. And so basically, somebody had found something on GCP, and then a year-plus later, somebody else found the exact same issue on AWS. And then similarly, there was an issue with Azure and then a year-plus later, same issue on AWS. And that's concerning because that tells me that AWS may not be monitoring what are the security issues that are impacting other cloud providers, and therefore checking whether or not they happen to themselves?That's something that you would expect a mature security team to be doing is to be monitoring what are public incidents that are happening to my competitors, and am I impacted similarly? Or what can I do to try and identify those issues, fix them, make sure they never happen? All those types of steps in terms of security maturity. And that's something that then I'm a little concerned of that we've seen those issues happen before. There's also, on AWS specifically, they have had a number of issues related to their IAM-managed policies that keep cropping up.And so they have had a number of incidents where they were releasing policies that shouldn't have been released in some way. And that's concerning that showed that they don't really have a change management process that you would expect. Usually, you would expect a company to be having GitHub PRs and approval processes and things like that, in order to make sure that there's a second set of eyes on something before it gets released.Corey: Particularly things of this level of sensitivity. This is not—like, I was making fun of them a day or two ago for having broken the copyright footer and not updating them since 2020 because instead of the ‘copyright' symbol, they used an ‘at' symbol. Minor stuff, but like that's fun to needle people about, but it doesn't actually matter for anything.Scott: Yeah.Corey: Security matters and mistakes show.Scott: Yeah. And so there had been some examples where they released a policy that was called, like, ‘cheese puffs something' and it's like, okay, that's clearly, like, an internal service of some sort. But I'd called them out and, like, I'd sent an email to AWS Security being like, “Hey, you need to make sure that you have change management processes on your IAM policies because one day you're going to do something that is bad.” And one day they did. They made a change to the read-only access policy, and that basically—they removed every single privilege, somebody had ended up, you know, internally, removed every single privileges to the read-only access policy and replaced it with a whole bunch of write privileges for, I think, the Cassandra service.And so, that was like, clearly they've made a mistake that they should have made sure they were correcting because you know, they had these previous incidents. Another kind of similar one was in December, there was a support policy where they had added S3 GetObject to that policy, and that was concerning in terms of have they just given all of their support employees access to everybody's content in their S3 buckets? And so AWS made some statements saying that there were other controls in place there so it wouldn't have been possible. But it's those types of things that [crosstalk 00:33:17]—Corey: Originally, those statements were made on Twitter, let's be clear here.Scott: Yes. Yeah. [laugh].Corey: And I feel like there's a—while I deeply appreciate how accessible a lot of their senior people are, I cannot point the executive leadership team at a client to some tweets that someone made. That is not a public statement of record that works on this.Scott: Exactly.Corey: They're learning. We'll get there sooner or later, I presume. I want to thank you for taking the time to speak with me, as always, I'll throw links to these repos into the [show notes 00:33:46], but if they want to know more what you have to say, where's the best place to find you?Scott: So, my Twitter, which, unfortunately, is a handle written in hex, but it's—‘dabbadoo' is how you would pronounce it, but it's probably easiest to see a link for it. So, that's probably the main place to look for me.Corey: That's why my old Twitter handle was my amateur radio callsign. I don't use that one anymore. It's just easier. And I think that's the right answer. Besides, given what you do, it's easy enough if people want your attention. They screw up badly enough, you'll come to them.Scott: Yep. [laugh].Corey: Scott, I really appreciate your time. Thanks again.Scott: Thank you.Corey: Scott Piper, Principal Engineer at Block and, more or less, roving security troubadour for lack of a better term. I'm Cloud Economist Corey Quinn, and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice or a comment on the YouTubes saying that this episode is completely invalid because you wind up using the old version of the metadata service and you've never had a problem. That you know of.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.
About MollyMolly White is a software engineer and team lead. She's also a longtime Wikipedia editor and advocate for free and open knowledge, and has more recently become an outspoken critic of cryptocurrencies and web3 more broadly.Links: web3isgoinggreat.com: https://web3isgoinggreat.com lasttweetinaws.com: https://lasttweetinaws.com mollywhite.net: https://mollywhite.net @molly0xFFF: https://twitter.com/molly0xFFF @web3isgreat: https://twitter.com/web3isgreat ponzl: http://ponzl.com TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: This episode is sponsored in part by our friends at Vultr. Optimized cloud compute plans have landed at Vultr to deliver lightning fast processing power, courtesy of third gen AMD EPYC processors without the IO, or hardware limitations, of a traditional multi-tenant cloud server. Starting at just 28 bucks a month, users can deploy general purpose, CPU, memory, or storage optimized cloud instances in more than 20 locations across five continents. Without looking, I know that once again, Antarctica has gotten the short end of the stick. Launch your Vultr optimized compute instance in 60 seconds or less on your choice of included operating systems, or bring your own. It's time to ditch convoluted and unpredictable giant tech company billing practices, and say goodbye to noisy neighbors and egregious egress forever. Vultr delivers the power of the cloud with none of the bloat. "Screaming in the Cloud" listeners can try Vultr for free today with a $150 in credit when they visit getvultr.com/screaming. That's G E T V U L T R.com/screaming. My thanks to them for sponsoring this ridiculous podcast.Corey: This episode is sponsored by our friends at Revelo. Revelo is the Spanish word of the day, and its spelled R-E-V-E-L-O. It means “I reveal.” Now, have you tried to hire an engineer lately? I assure you it is significantly harder than it sounds. One of the things that Revelo has recognized is something I've been talking about for a while, specifically that while talent is evenly distributed, opportunity is absolutely not. They're exposing a new talent pool to, basically, those of us without a presence in Latin America via their platform. It's the largest tech talent marketplace in Latin America with over a million engineers in their network, which includes—but isn't limited to—talent in Mexico, Costa Rica, Brazil, and Argentina. Now, not only do they wind up spreading all of their talent on English ability, as well as you know, their engineering skills, but they go significantly beyond that. Some of the folks on their platform are hands down the most talented engineers that I've ever spoken to. Let's also not forget that Latin America has high time zone overlap with what we have here in the United States, so you can hire full-time remote engineers who share most of the workday as your team. It's an end-to-end talent service, so you can find and hire engineers in Central and South America without having to worry about, frankly, the colossal pain of cross-border payroll and benefits and compliance because Revelo handles all of it. If you're hiring engineers, check out revelo.io/screaming to get 20% off your first three months. That's R-E-V-E-L-O dot I-O slash screaming.Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. For a while now I have resisted the siren song of doing an episode covering the wide world of Web 3. So, if you're deep into that space, you can rejoice because it's finally time to change that. Now, the other side of that, for at least some of you, is that my guest today is Molly White, who's a software engineer. But more notable as of recent days for running a collection of interesting stories coming out of the world of Web 3, at web3isgoinggreat.com. Molly, thank you for joining me.Molly: Thanks for having me.Corey: So, by day, you're a software engineer, which means you're already predisposed to writing things that humans find very difficult to understand. And now you're—in your spare time apparently—writing about Web 3, which is a topic that humans find very difficult to understand. For some reason, you have a flair for telling stories about this basically impenetrable to outsiders space in a way that makes it look, first off simpler to understand, and secondly—let's be clear—patently ridiculous. How did you find your way into this part of the world?Molly: Well, [sigh] I think as a software engineer, it's a little hard to avoid the Web 3 thing. You hear about it from your colleagues or the people on tech Twitter, or, you know, you see it in the news. And it's—Corey: Or people behind you at Starbucks who won't stop talking, et cetera.Molly: Yeah, they sneak right up on you. And so people, you know, when you hear about something that's supposed to be the future of the web, you know, if you're a web, software engineer, I think it's sort of natural to try to figure out, “Oh, what's this thing? You know, I need to learn more about this.” And that's sort of how I got into it. You know, I tell the story about how I've known about cryptocurrencies for a long time—you know, Bitcoin has been around for a while now—and I was just extremely uninterested [laugh] in them for a very long time.But with this sort of rebrand, recently, I've sort of been forced, I think, to pay a little more attention to it since it seems to be one of those things that people have to engage with whether they want to or not. Or at least people hope that is what the Web 3 thing will become. So.Corey: I come from a background of being a grumpy old Unix administrator, and I've been around long enough to see the inevitable cycle where this shiny, exciting new technology of today is the legacy garbage I have to support in production tomorrow. And this breeds more than a little bit of cynicism, where whenever someone says, “We have this new thing,” it's, “All right. Let me get out the checklist. What happens when jerks get involved? How is it going to break? How am I going to hate this thing? How is it going to completely ruin my week?”And people building technologies—this is probably no surprise—don't generally like questions like that. And I get it because they're trying to do something creative and build something that solves a problem that sometimes they can—they're the only ones who can define. But also tends to be this sort of love with the technology where, “I see nothing wrong with the technology I've built whatsoever.” It's, “Yeah, you probably wouldn't.”And that's okay because that doesn't bound itself to cryptocurrency or blockchain stuff—Molly: Yeah, absolutely.Corey: It runs the gamut from databases to messaging protocols to someone's sketched-out version of an iPhone they wish that someone would build, et cetera, et cetera. No technology is perfect. There's an ancient place on the internet I used to hang out that had the motto of, “All hardware sucks, all software sucks.” Varying degrees and to different levels, but they all suck. And they're not wrong.I love aspects of the Web 3 community: Their optimism, for example, is something that I find inspiring; their ability to stay on message is also incredibly—honestly, it's admirable. I just wish the message were slightly different. What is your take on how all of this stuff is—I guess, not just the technology itself, but the hype train that accompanies it?Molly: Yeah. Well, first of all, I agree with a lot of what you just said. I think optimists have a great role to play in the software world, and I think cynics also do, and I sort of wish there was a little more balance in this particular technology. I also agree with you the community and sort of the people who—a lot of the people who are trying to get involved in this stuff, I really admire and I think are really passionate, and, like, really smart and, you know, have the right motivations. But it's a little frustrating sometimes to see that the optimism can turn into a very aggressive, sort of almost protectiveness around the technology where they are almost unwilling to, you know, examine whether or not there might be flaws behind the product that they're hoping to build.And that's where I get really worried because I think in order to build software responsibly, you need to be open to the skepticism and the criticism and the questions, and it overwhelmingly has felt like the sort of Web 3 community has not been, which I find really worrisome.Corey: Spare me from the cascade of, “Do your own research,” whenever you say something negative. Like, it seems to be a pervasive ail of our society, where it's, “You're just going to believe what people tell you?” “What, you mean, legitimate experts who've been looking at”—Molly: [laugh]. Yeah, the experts?Corey: —“The space for decades?” Yeah, it's like, what research am I going to do on YouTube in 20 minutes that is going to outweigh that? It's not do your own research; it's carefully curate the bias of the media your consuming until you come around to my worldview, and that's not the same thing as research.Molly: I agree. Yeah. And it's weird how we see that same thing cropping up in, like, Covid-19 conspiracies and QAnon, and then it's like, “And also crypto.” Okay, that's a little weird.Corey: It's very odd watching the rise of this. Blockchain is an interesting technology, absolutely, and this recent extension into non-fungible tokens, or NFTs, I—the first time I saw it was relatively recently and it very quickly became impossible to avoid, if for no other reason than I keep getting tagged by brand new empty Twitter accounts doing replies of, “Tag three people to wind up getting airdropped on this,” and I sure do love the fact that Twitter can find no way whatsoever to stop this from happening. Lovely. And it's, “Okay, looking at this, what is this?” Like, “Look at how much money these things sell for.”And I have extensive background in finance, so I can spot it from a mile away, like, “Oh, yeah, that. That's a money-laundering scam.” Like, wait, that doesn't seem fair. Like, you tell me everyone involved in NFTs is a money launderer? Oh, absolutely not; that's a terrible money laundering scam.You need to have people who are not money launderers, otherwise, the entire thing gets shut down and everyone gets arrested. You need to have people who are not themselves criminals, basically interspersed and the dominant party in this, so the rest of us can money launder. And it's like there's nothing new under the sun, and the idea that regulators are somehow complete naive fools does not usually pay dividends. And people have a long time to reflect on this in federal prison.Molly: Right. Yeah. And I think we've been seeing this sort of trailing regulation coming in a little bit. You know, there's a lag between when someone does something blatantly criminal and then when the, you know, the US Attorney's Office announcements come out a year or two later saying that, “Oh, and we just charged this person with”—you know–“Fraud or whatever it is.” I sort of, every once in a while, I read some of those announcements from, you know, the Department of Justice or the various other groups and, you know, they'll describe what someone has done in the crypto space or, you know, financial fraud, and it's like, whoo, boy that looks similar to a lot of these projects that are just launching now. I wonder if anyone's getting a little bit uncomfortable reading these. [laugh].Corey: This thing that I understand as well that I am, I am a cynic and I have been basically down on emerging technologies a lot, which means I've been wrong an awful lot. In 2006 2007, I thought virtualization was a very niche thing that was only going to be suitable for a couple of weird workloads because, like, how many underutilized computers could there really be in the world? Yeah, I was wrong. Then I said that cloud was absolutely not going to take on, and through about 2012, I was very anti-cloud because, “Oh, you're going to trust your stuff on someone else's computer and give your uptime to them and your—their security over to them. It'll never catch on.” Yeah, I was wrong there, too.I thought containers were ridiculous. And they are in some ways, but they're also the way the world works. I'm actually very bullish on serverless, which means it's not going to succeed in the market because I'm invariably—Molly: Oh, no. [laugh].Corey: —wrong in this. Exactly. But people are saying, “Well, what makes Web 3 or any of these blockchain technologies any different than all the other things that I was wrong about?” And my feeling around this is that at least I could understand what those other technologies, what the problem they were setting out to solve was. It continues to shift depending upon the narrative line that people are pushing on this.And I also remember the response I got every case previously, which was, “You'll see it sooner or later. It'll be fine.” There was never this urgency baked into it of, “You have to get in now, or you're going to lose out and be poor forever.” And I was extraordinarily gullible growing up, so when I see this, it's like, okay, when you're trying to pressure me into doing something, it's because you're deriving some benefit if I do. And I'm very cynical these days, perhaps unfairly so.When you grow up being constantly you may have to fall for pranks and whatnot because you have no sense of guile, then great. You sort of have an overreaction the other direction. It's mostly served me well, but I look at this and okay, ignoring the entire bubble of that ecosystem—and I'm hoping you have an answer for this—what is the real-world problem that I, as an individual or as a business, have that Web 3 solves for me?Molly: Well, it's been great for ransomware. So if—Corey: Oh, yes.Molly: —you're doing that—[laugh]. Yeah, no, I have a very similar feeling to this around, you know—Corey: Ransomware is ridic—there's already ways to do that. It's like, we're going to take your data and we're not going to give it back to you unless you pay us enormous piles of money. Yeah, that's called cloud egress charges. It's been done, and it's a lot less computationally intensive. I'm mostly kidding, but not entirely.And yeah, yeah, it's so much easier now to wind up extorting money from people through this thing. Yeah, I. Don't find that often to be a feature, and, frankly, people who do I don't really want them within a thousand miles of me.Molly: Right. Yeah. And I mean, I think a lot of the problems—you'll even see people saying this, you know, with very thin veils of legitimacy, but a lot of people are basically saying, “Well, we want to do something that we can't do with traditional money because it's regulated.” [laugh]. And so seeing that, it's like, “Really?” You're like, you know, the regulations—Corey: What are you trying to do over there, buddy? And yes, I admit, there are certain things that I find obnoxious about the way that in, you know, non-crypto society that we deal with money and the challenges we have with it. Some of the fees attached to things, some of the way that it takes, “Wow, we can send messages at the speed of thought in real-time, but it still takes how long for a payment to clear through these systems?” I get it; there are reasons they are the things are the way that they are. But it all mostly works, to be clear.Are there opportunities for improvement? Absolutely. Do I think that the way to do that is to basically come up with an entirely new form of money? I—maybe if you're starting from scratch, but I kind of have a hard time accepting that it's going to work that way for everyone.Molly: Right. And I also think there's sort of this pervasive issue with a lot of the projects in Web 3, where they are actually trying to solve very real problems, very serious problems, and you know, the fact that there's, you know, unfairness in the banking system, or that there's fees, or that, you know, there are people who are making an enormous amount of money off of people just trying to send small amounts of money, like, I get that, and I get that you might want to solve those problems. But overwhelmingly, it seems like there's sort of this opinion of like, okay, so we have this bad thing now. We have this different thing here, so this different thing has to be better than this bad thing. And it's like, no, no, no, wait, hang on. It's possible to, like, replace a bad thing with something that's worse, and I think we need to consider that what we're trying to do here looks a lot like that.And so you know, people are talking about, you know, “Oh, well bank the unbanked.” They don't have access to banking and so we'll fix that with blockchains. And it's like, no, I think what we'll do actually with the blockchain is we'll probably end up scamming the unbanked because this place is totally unregulated and regulations actually protect people a lot of the time. You know, so that I think that's really worrisome, the sort of just idea that we have something different and so it's better.Corey: The thing that always catches my eye when people talk about this: “Oh, it's the new form of money. It's going to solve all of the social injustice problems.” Okay, maybe I stumbled upon this secret hidden community of altruists that are out there, but generally speaking, looking at the broad sweep of human behavior, you can make a few observations, and one of them is that the rich generally do not desire company. And the idea of, oh, this is going to magically fix systemic inequality, I don't know that that's necessarily true.Molly: Right.Corey: And a lot of the pr—say what you will about problems with the existing financial regulations that are out there if I screw up, and I accidentally wind up doing a wire transfer of rent or for buying a car to the wrong account, there are established ways that gets reversed, and between large institutions, it's basically a phone call, a letter, and it gets done within a day or so. Whereas with crypto, it's [sings] doesn't it suck to be you? Di di. And that just becomes… well, is there any recourse? None. That doesn't strike me as a feature, to be honest, that strikes me as a bug.Molly: Right. And I was actually—it is interesting. I was recently rereading the Bitcoin white paper because it's one of those things that people are constantly like, “Well, read the Bitcoin white paper and you'll totally understand it all.” And it's interesting how in the Bitcoin white paper, they talk about how this new system will prevent fraud, but if you look at what they're talking about as fraud, they're talking about people illegitimately reversing transactions. So like, you know, take the example you buy something on Amazon, you receive whatever item it is, and then you do a chargeback. And you then you have your item, and you haven't paid for it.That is, like, the one thing that this, you know, person who came up with Bitcoin is describing as fraud. And that's, like, the one thing that is hoping to be prevented. And it's like, I kind of get the idea that, like, [laugh] at some point, you know, someone's scammed, Satoshi in this way, and it's just, like, this is what came from it. But it's such an odd perception that is, like, the only kind of fraud and, like, that is always a bad thing to be able to reverse a transaction. I find that really fascinating because it's just like, that's actually a really good thing a lot of the time.Corey: This episode is sponsored by our friends at Oracle Cloud. Counting the pennies, but still dreaming of deploying apps instead of “Hello, World” demos? Allow me to introduce you to Oracle's Always Free tier. It provides over 20 free services and infrastructure, networking, databases, observability, management, and security. And—let me be clear here—it's actually free. There's no surprise billing until you intentionally and proactively upgrade your account. This means you can provision a virtual machine instance or spin up an autonomous database that manages itself, all while gaining the networking, load balancing, and storage resources that somehow never quite make it into most free tiers needed to support the application that you want to build. With Always Free, you can do things like run small-scale applications or do proof-of-concept testing without spending a dime. You know that I always like to put asterisks next to the word free? This is actually free, no asterisk. Start now. Visit snark.cloud/oci-free that's snark.cloud/oci-free.Corey: But I will defend the Web 3 community, which I know is somewhat surprising because again, my feelings on this stuff are nuanced. But—Molly: Sure.Corey: Everyone says they have this massive problem with InfoSec and the rest, and I don't believe that that is necessarily true. I do not believe that the people writing the code that powers these blockschain—or however the pluralize is improperly—are somehow much worse developers than everyone else. But the incentives are radically different because if I screw up on some of my Lambda functions, great, you can get access to I don't know the API tokens for my lasttweetinaws.com Twitter client. Okay, great. Now, you can spam Twitter. It's not that interesting to people and it's not considered high value.Whereas yeah, if I wind up breaking through this little thing, I can wind up getting, what, $200 million? Yeah, suddenly, it's probably worth spending significant time on security reviews. So, I do think that folks are being a little unfairly maligned there just because the way that they're approaching this it does not match the rigor that is take—and care that is taken to systems that in the fiat finance world—as they love to call it—that wind up [unintelligible 00:20:10] money, there's oversight, there is planning, there is testing, there are entire teams of people doing nothing other than InfoSec review, rather than, “Well, it's on GitHub; my job is done.”Molly: Yeah. Yeah, I've heard people refer to it as self-paying bug bounties before where the bounty is, you know, the money that you can pull out of these exchanges or whatever project you actually, you know, are able to exploit. And I think that's very accurate. And I think you're right, you know, I think that there's nothing that—I mean, there, I'm sure they are particularly incompetent developers in Web 3 as there are particularly incompetent developers in any sector, but I do think that you're right, that it's just an enormous incentive to find any small bug. And I think also part of it is that a lot of the concepts that people are working with are extremely difficult to, sort of, wrap your mind around.You know, this is a little bit of a tangent, but a lot of the attacks, we just saw three attacks in one day that all relied on something called a flash loan exploit. And trying to wrap my head around what a flash loan is just like, it doesn't jibe with, like, current financial systems and so it's really hard to, sort of, comprehend, and I think it's probably hard for developers to code against because it's just a very different way of thinking about loans. You know, like a flash loan is basically a loan that you take out the loan, and you pay it back in one transaction, which, in real life has no purpose, right? There's no reason you would go to a bank, borrow $10,000, and then immediately give them those $10,000 back. But you know, [crosstalk 00:21:47]—Corey: Financial equivalent of a managed NAT gateway that winds up just transferring for every cent that's passed through it. I've seen stuff historically, before they fixed bugs like this, in credit card reward systems, where basically you can just cycle spend through and it doesn't do anything other than suddenly starts cranking your point balance into the stratosphere, so you could save up your poi—frequent flier miles to go to space or whatnot.Molly: Yeah, exactly. Right. And I think, you know, that's the, sort of, same idea here. You know, people use these flash loans for all sorts of weird, you know, yield farming and just sort of bananas stuff. And, you know, I think trying to code against a lot of stuff, you have to really understand those things very well, and not necessarily just be a good developer, but also understand the economics behind it and the incentives that people are, you know, chasing. And that's tough. [laugh].Corey: I will say that you are far from alone in criticizing crypto, but I've patterned a lot of my own cynicism and trepidation around the space after the way that you engage with it. And what I mean by that is not that I build hilarious websites about these things that chronicle its shortcomings, but rather that you don't personalize it, you don't take the step that so many folks do and say, “Oh, this person is now going to work at a crypto company, therefore, they're a sellout. Therefore, they're out to scam people. Therefore, they're just the devil incarnate.”And it's no, I don't believe that either. I'm curious to hear their reasons for it. They don't owe me an explanation and I'm certainly not going to harass them on Twitter about these things, but the idea that someone is somehow now working for a company that engages in this stuff, and therefore they are now to be written off as a human being is something that I just find distasteful in the extreme. And I've never once seen you cross that line.Molly: Yeah, I also really disagree with that. Which, you know, may be controversial to some of my fellow skeptics, but I think we can agree to disagree on that. I don't think that it is, you know—I think that people have very good reasons for going to work for companies that I don't necessarily personally agree with, you know? And I think there are a lot of examples of people who work for companies in spaces that are, you know, questionable. A lot of the big social networks have done things that are pretty horrifying when, you know, look at the recent exposés around Facebook or, you know, all those things.You know, there are people who work for defense companies, which I don't necessarily agree with, you know, those kinds of things. And I think everyone has to do their own, sort of, personal math around what makes sense for them, where their ethics lie. You know, a lot of these companies I will say pay a lot of money, and I can't necessarily fault someone for needing to pay the bills, right, even if it means working for a company that I think is maybe not the best. But I think there's—Corey: Yeah, I used to give people who worked with Facebook tremendous amounts of crap. I don't do that anymore. I was wrong. I'm not going to personally harangue people for where they work. You never know someone's individual situation. I—Molly: Exactly.Corey: I'm not apologizing for the company; I want to do no business with them, but I will no longer be going after people individually because they work there. Because until you walk a mile in someone's shoes, you don't know what's going on there.Molly: Right. And I also think there's just not much point to it, right? Like, if we want to hold Facebook to account, for example, I don't think going after some software engineer or customer support rep or whatever is going to make any difference, other than making their life particularly unpleasant. And, you know, that I think applies to the Web 3 crypto space as well. You know, I will absolutely dunk on someone who I think is, you know, malicious and scammy and taking advantage of people, and I will say the same things about companies that are doing that, but I do think that there are very well-intentioned people who are working in this space for a ton of different reasons.It's, you know, personal curiosity; some people just aren't convinced yet that—you know, some people think this could do a lot of good and that they, you know, should engage in the space in good faith and, you know, go work for these companies, and try to make sure that the companies are pushing towards good. You know, I don't personally think that there's much that can be done there; I think that's a tough angle, but I respect people for trying. And I think there's also a huge amount of just, uh—I think a lot of the hate or the vitriol that has been targeted at these people who are going to work for crypto companies is very selective, in some ways, you know? You see a woman going to work for a crypto company or person of color going to work for a crypto company and their replies look markedly different from the white guy who goes to work for a crypto company. It's all, “Congratulations,” and, “Oh, he's going to be so rich,” and all that kind of stuff, and there's not so much you know, hand-wringing of hands, whether or not they are a scammer or all that kind of thing. It's like, you're only allowed to, you know, go and get that bag or whatever if you're a white guy; everyone else is held to a different standard.Corey: For people who look like me, the bars on the floor, let's be very clear here. It's, “Good for you. Go after it. Go and get it.” And there is a systemic problem, on some level, that I think that we have not really grappled with as a society, which is that even well-paid software engineers still feel the pressure that in order to be prosperous and guarantee financial security for you and your family, you now also to be a part-time trader in various ways, and invest, which very often is misused in place of what it actually is, which is speculation or gambling.Molly: Gambling. Yeah. [laugh].Corey: That is the way to prosperity. Because we have survivorship biases; no one likes to trumpet their failures. It's the same problem we see with tech conferences: People get up and talk about, “This is the thing we built and it's awesome.” And you talk to people who work there. It's like, “Yeah, I don't recall that project going anywhere near that well.”And, yeah, it we all tell these aspirational, heroic stories of what we've done, and we trumpet the things we're proud of. And it just, it isn't sustainable. It isn't something that I think we've spent a lot of time on. And this is software engineers were talking about. Remember, once growing up, at least, there was the idea that you could—this was this wild, subversive idea that you could be a schoolteacher in a city and actually live in the city in which you taught. Now, that is basically a fantasy. And we see that across the board. That's not great for anything.Molly: Yeah. And I actually blame, you know, economic circumstance for a lot of the crypto hype. You know, there are a lot of people who are in tough spots right now, you know? The pandemic has certainly had a huge impact on some people, especially people working in, you know, service jobs and things like that. And so people are in, you know, pretty dire straits as a result of that.There's also enormous student loans, the housing market is bonkers, you know? There's so many things that are really making people suffer financially. And so then when crypto comes along, and people start talking about 60,000% APY and all this, you know, you're going to triple your money, you're going to buy this bored ape NFT at $100 and it's going to be $500,000 next year. People fall for that because it's enormously appealing, right? And I think there's a lot of blame to be placed on the media for, sort of, buying into a lot of that.There's been a lot of very credulous reporting, I would say, on some of the people who claim to make a ton of money off of these things. And so people, you know, that, when they see, you know, CNBC, for example, will highlight these, you know, people who were just scraping by, they were going paycheck to paycheck, they put $50 into a project and now they're millionaires, you know? And people see that because there's no point for CNBC to talk about the person who invested their, you know, their rent payment into a crypto project and then couldn't pay rent because they lost it all, you know? Or the person who took out margin loans and is now in debt to these various companies that are lending people money to gamble on crypto. Those are not the stories that make the headlines and so people get a very skewed view of how many people are actually making a ton of money in this space and how many people are actually losing a lot of money in this space. And I put a lot of blame on various media companies for that.Corey: Well, take Twitter as an example. Yeah, I would classify them in many respects as a media company. Imagine for a second that if every time someone tweeted something about AWS, like, “Well, I got surprised at my AWS bill,” or, “Huh, I'm having some trouble with AWS Lambda.” Suddenly, 15 bots all replied and quote tweets and the rest saying, “Ah, this person helped me out. Talk to them,” or fake accounts with a, “Here's our support forum. Please fill this out.” It goes to a Google Doc.It seems like the easiest thing in the world to automatically wind up detecting and blocking—just because it is clearly keyword triggered, it is very obvious when it happens, and somehow, it just keeps persisting. It makes you wonder, on some level—like, counts as engagement and users, so it makes the numbers that we report on earnings go up, so I guess we're going to keep doing it. It just feels like, on some level, Twitter has empowered a lot of this in a way that most normal places would not.Which, of course, brings us to the other project you've been involved with for a very long time: Wikipedia. Now, it seems like a weird thing to say, “Oh, yes. You've been an editor on Wikipedia.” Yeah, so is basically everyone at some point because it turns out, it's a couple of clicks away. Your something a little more than that, but I don't pretend to understand the Wikipedia [instructor 00:31:31]. Tell me about that.Molly: Yeah. Yeah. So, I'm a Wikipedia editor. I'm a prolific, I guess, Wikipedia editor, you might say. But I've been actively editing Wikipedia for over a decade now. I'm also a member of the, sort of, administrative group on that project, and I've also served a couple of terms on what's called the arbitration committee, which helps mediate disputes among community members. So yeah, I'm pretty involved. [laugh].Corey: How much of your involvement in that community has bled over into your, frankly, amazing coverage of Web 3?Molly: An enormous amount. [laugh]. I think you can very—I think, if you look at the entries on Web 3 is Going Great, you can kind of see the Wikipedia voice in them. It's a little hard for me to escape that sort of style of writing because I've just been doing it for so long, and it's the majority of the writing I do. So, you definitely see that a lot.And, you know, I've had a couple of people say things, you know, like, you know, “How do you cover stuff in such a, you know, detached way?” And it's like, “Oh, well, I write encyclopedias in my spare time.” [laugh]. There's obviously a lot more sarcasm and sort of personal bias in the Web 3 is Going Great project, which is why I started it because I can't do that on Wikipedia, and I won't do that on Wikipedia. But that's where a lot of it comes from, is that sort of that instinct, I think, that you develop as a Wikipedia editor to sort of research and chronicle and record and share what you're seeing. It's hard to escape.Corey: I do want to call attention, though, to other long-form writing that you do because, “Wikipedia, who wrote this?” Well, the answer is always lots of people. But if you go to mollywhite.net and look at your long-form writing, it's pretty easy to understand who wrote this.It's not nearly as clinical and encyclopedaic as you might expect, from your description just now. It's very approachable, very engaging, writing that reflects on topics in a way that only long-form can and Twitter generally cannot. And it's great, you could read this and not realize that you're deeply involved in the Wikipedia part of it, right up until the point you get to the end. And then you see the extensive list of references at the bottom of the page because apparently footnotes and citation is a habit that you can't get away from there, but it's—Molly: I can't help myself. [laugh].Corey: Nowhere near as dry and clinical as you're implying.Molly: Yeah, that's true. I do take more of an essay approach in my long-form writing. One thing I've really loved about Web 3 is Going just Great is that you sort of don't need to necessarily know, like, what's a blockchain, and what's an NFT, and what's, you know, distributed, you know, database or whatever, before you start reading it. It's sort of approachable, you can read one or two entries, and then you can go do whatever else and you don't have to do this, sort of, deep dive. But it also lacks, I think, that ability to go a little deeper into some of the problems or some of the really huge issues I see with the underlying technologies because it's, you know, it's very much of one hit, and then you move on to the next thing.So, I've started blogging a little bit on the side, I guess, to sort of go into a little more depth on some of my concerns, just both as, like, a technologist but also just as someone who's been on the internet for a long time and who's been a member of communities. You know, the Wikimedia community is very similar to a lot of the communities that you're seeing crop up in these Web 3 projects, especially to do with the DAOs. And so, I sort of have over a decade of experience in it community like that, and I'm watching a lot of these new DAOs, you know, who say they're coming up with this brand new model, you know, they've invented this new form of governance. And I'm watching them, it's like, “Oh, you're about to step on that same rake we stepped on 15 years ago.”And that concerns me a lot, especially because you know, the Wikimedia community, there's harm that can be done by—for sure, and it has happened, but there's not really financial harm that happens with a Wikipedia editor, you know? You're not buying to engage with the Wikimedia community. I certainly hope not because you're being scammed. But with these DAOs, you know, you're paying to engage with a community that is not taking lessons that it could be taking from Wikimedia, from co-ops, from mutual organizations, you know? They could be looking at history a little bit more, I think.Corey: Tech does this across the board. It's, “We're in San Francisco. We're going to reinvent and disrupt industry x.” Okay, fine, great. Maybe it works. Maybe it doesn't. Godspeed. “And while we're at it, we're going to reinvent other things, too, that we think the world gets wrong, like, how to interview people.” And the common thing on Twitter is no one knows how to interview engineers properly; it can't be solved.And yes, yes it can. There were multi-decade studies conducted in places like GM, Coca-Cola, et cetera, on how to lead to positive outcomes while interviewing and what to do, and whenever you bring that up, Twitter gets very angry about that because, “No, that's different. That's a different time and a different era, and the world works differently, now.” And, “Great, okay, keep disrupting things, but you can save a lot of time by having a conversation or two with people who've walked that road before? You don't have to go it alone.”Molly: Right. Yeah, I think that's a huge thing. Kelsey Hightower has done a lot of conversations around that, around how—he's doing incredible work talking about, you know, blockchains, and crypto and stuff—and he's talked a lot about how it looks like a lot of these projects are sort of reliving history around—you know, he has a very technological approach to it, so he talks about, you know, the sort of security things that are not being considered and the, you know, the various infrastructure sides of things that are just sort of being reinvented without any sort of consideration to the past lessons. I think that's just a very classic—you're right, it's a very classic, like, Silicon Valley way of doing things. There's sort of the running joke about how people reinvent buses every couple of years.You know, Uber is like, we're going to make a service where a bunch of people can all get in a car together and drive someplace. It's like, “Oh, right, yeah. A public bus system.” We have those. I think there's very similar comparisons to draw in Web 3.Corey: There absolutely are. And I want to thank you for being so generous with your time. If people want to learn more, where can they find you?Molly: You can find me on Twitter. I am both @molly0xFFF, and also @web3isgreat on Twitter. And then there's my website, web3isgoinggreat.com, and my other website, mollywhite.net. I'll be at all of those.Corey: Yes. And for fun, I wound up pointing a domain over to you, over to your site as well, ponzl—P-O-N-Z-E-L dot com. It's like P-O-N-Z-I except the I is an L because the crypto people can never seem to quite take the L. But there you have it. It's not a Ponzi scheme; it's something different.Molly: It's a Ponzl scheme.Corey: Thank you so much for being generous with your time. I appreciate it.Molly: Thank you for having me.Corey: Molly White, Web 3 chronicler of our time, and software engineer. I'm Cloud Economist Corey Quinn, and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice along with an angry comment telling me that no, crypto is not reinventing a bus because a bus can only run over someone once.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.
About JohnnyJohnny was born in Cleveland, OH and graduated from the University of Toledo with a Bachelor's in Computer Science Engineering. He began his career as a software engineer focused on embedded device protocols and systems engineering. Eventually he realized that Program Management worked better with the grain of his brain, so he took his career in that direction.In 2019, he was hired by Google Cloud to serve as a Communications Lead on their incident management teams. Most recently, he joined Waymo in November 2021 as a Technical Program Manager, acting as an anti-entropy agent for the self-driving car company's offboard infrastructure teams.Outside his day job, Johnny enjoys mountain biking, playing piano and trumpet, personal finance, coaching, and studying complex systems. He currently lives in Sunnyvale, CA with his wife Emily, and is expecting their first child in April 2022! Links: Original Twitter thread: https://twitter.com/QuinnyPig/status/1436129343399346184 Personal website: https://jmpod.com LinkedIn: https://www.linkedin.com/in/jmpod Twitter: https://twitter.com/gratitudeisfree/ Instagram: https://www.instagram.com/gratitudeisfree/ TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: Couchbase Capella Database-as-a-Service is flexible, full-featured and fully managed with built in access via key-value, SQL, and full-text search. Flexible JSON documents aligned to your applications and workloads. Build faster with blazing fast in-memory performance and automated replication and scaling while reducing cost. Capella has the best price performance of any fully managed document database. Visit couchbase.com/screaminginthecloud to try Capella today for free and be up and running in three minutes with no credit card required. Couchbase Capella: make your data sing.Corey: This episode is sponsored in part by LaunchDarkly. Take a look at what it takes to get your code into production. I'm going to just guess that it's awful because it's always awful. No one loves their deployment process. What if launching new features didn't require you to do a full-on code and possibly infrastructure deploy? What if you could test on a small subset of users and then roll it back immediately if results aren't what you expect? LaunchDarkly does exactly this. To learn more, visit launchdarkly.com and tell them Corey sent you, and watch for the wince.Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. Every once in a while I get feedback from people who I've encountered who are impacted in various ways. Most of it is feedback delivered of the kind you might expect, like, “Unsubscribe me from this newsletter,” or, “Block,” or sometimes bricks thrown through my window. But occasionally, I get some truly horrifying feedback, and far and away one of the most horrifying things I can ever be told is, “So, I was reading one of your tweet threads and it changed the course of my career.”It's like, “Oh, dear,” because nothing good is going to happen after something like that. It's, “Yeah, they were going to name something terrible here at AWS, so I ran over my boss in the parking lot,” is sort of what I'm expecting to hear. But I got that exact feedback about life-changing tweet threads from today's guest. We'll get into what that tweet thread was a little bit, but let's first let the other person talk for a minute. Johnny Podhradsky is a technical program manager at Waymo. Specifically, of Offboard Infrastructure. Johnny, thanks for suffering through a long, painful introduction, as well as, more or less, the slings and arrows that invariably come with being on the show.Johnny: Thanks, Corey. I'm grateful to be here.Corey: So, first things first. I always like to find out what people actually do for a living that is usually a source of entertainment, if nothing else. You are a technical program manager—or TPM as they say in tech companies—of Offboard Infrastructure. I'm assuming because Waymo, is at least theoretically, a self-driving car company, ‘offboard' means things that are not on the vehicle themselves.Johnny: That's exactly right. Yeah.Corey: Fantastic. Now, ask the dumb question because I'm still not sure I have an answer after however many years in this industry. What does a technical program manager do?Johnny: [laugh]. I get that question a lot. Often people try to distinguish between what's a technical program manager do versus what does a product manager do.Corey: Or a project manager, too, because there's a lot of different ways it can express itself, and I'm a PM, and it's, “Oh, wonderful. That's like four different acronyms I can disambiguate into and I'm probably going to get it wrong.”Johnny: And to make it even more confusing, it varies company by company. So, just focus in on specifically what I do as a technical program manager, I'm an anti-entropy agent, right? I make sure things stay on track, specifically embedded into technical teams. So, I have a degree in engineering; I'm able to speak fluently about technology. And the entire idea, the entire purpose of my existence is to make sure that things don't fall apart. So, I'm keeping track of people and resources; I'm keeping track of overall timelines; risks and mitigations for programs that are ongoing, whether they're small with just a few people or cross-org, cross-functional teams; serving as an unblocker and making sure that all the dependencies that exist between the various tasks in the teams are addressed ahead of time so that we know what needs to be done when.Corey: It's one of those useful almost glue functions, it feels like that is, “Well, what have you actually built? Point at the thing you've constructed yourself from your hands on your keyboard?” And it's hard to do and it's very nebulous, when you're not directly able to point to a website, for example. “Yeah, you see that button in the corner? I made that button.” Great.Like, that's the visceral thing that people can wrap their heads around. Project and program management feels to me like one of those areas that, in theory, you don't need those people to be a part of building anything, but in practice you very much do. Another example of this—from my own history, of course—is operations because in theory, you just have developers write code correctly the first time and then they leave it where it is and it never needs to be updated again, and there's no reason to have operations folks. Yeah. As they say, the difference between theory and practice is that in theory, there is none.Johnny: I'll buy that. Yeah, when it comes to actual, I mean, digital, but physical deliverables and things that you can show that you've done, there are standards that you can have with documentation, like Gantt charts and risk registers and all that sort of thing, but it is very much a glue role. It is very much a gentle nudge to get things done. And it really revolves around the transparency and making sure that the people who are invested in the success of whatever it is that you're doing program-wise are aware of what's going on as far ahead of time as possible. That's why I like to consider it sort of an anti-entropy role because things will just naturally go off the rails if no one is there to help guide them.I mean, that doesn't happen in every situation, of course, but having someone dedicated to the role of making sure that things are moving according to a good rhythm is a critical role. And it just so happens that that is sort of the way the grain of my brain works and I discovered that throughout the course of my career.Corey: So, let's get back to the reason you originally reached out to me. I think that is always an interesting topic to explore because whenever someone says, “Wow, your tweet really helped me with my career,” I get worried. Because as I said before, I am one of the absolute best in the world at getting myself fired from jobs, so when it comes to being a good employee, mostly my value is as a counter-example of advice I'll give [unintelligible 00:05:49] job interviews. For example, when they say something condescending and rude, insult them right back because A, it's funny, and that plays well on Twitter. And B, interviews are always two-way streets, and if they're going to treat you like crap, you don't want to work there anyway, so you may as well have some fun with it. But a lot of what I say doesn't really lend itself to the kind of outcomes that lead to happy employment scenarios. So, I've got to ask, what the hell did I say?Johnny: Yeah, it was kind of serendipitous. I'm in a number of Slack communities, one of them being the Cleveland Tech Slack—if you're in Cleveland or around Cleveland, I highly recommend it—and someone just randomly posted this thread right in the middle of me interviewing at Waymo. So, previously before Waymo, I was at Google, and I loved my job. I loved the team that I was on, I loved the—I mean, I was still very much in the honeymoon phase of Silicon Valley. I had moved to Silicon Valley from Cleveland in 2019 with my then fiance.And so I was just, you know, bright-eyed and bushy-tailed, and everything was just incredible to me; why would I ever consider leaving this? So, I had an interview at Waymo and I ended up getting an offer and I just didn't know whether I should take it. Because I loved where I was at and I really enjoyed the opportunities, so it was just, you know, ten out of ten. One of the things that I was thinking about then was, you know, I kept thinking back to our first team dinner where our teammates were sharing their stories of their careers. And my mentor, Ted, had mentioned how he had worked on the iPhone at Apple and was in the same room with Steve Jobs.And me being a Cleveland boy, just it sounded like, “Whoa.” My eyes got really big like dinner plates. And it's just like, “I'm sitting at a table with people who have done these things with these people.” And I was wondering, like, what did that mean for my career? And so where did I want to take my career and have those kinds of stories? So fast-forwarding, you know, I was interviewing at Waymo; I ended up getting the offer. And I was just on the fence; I couldn't decide if that was the way I wanted to go, if I really wanted to leave my amazing job at Google.Corey: What was holding you back on that? Was it a sense of well you want to be disloyal to the existing team? You were thriving in the role you're in? Was it the risk of well, I don't know how I'll do in a different company solving different problems? What was it that was holding you back?Johnny: It was all of those. When you do an apples-to-apples comparison, you don't really know what you're getting into when you're going to a new company, and that's part of why your thread was so critical in making my decision. Just to say exactly what you said in the tweet, “So, an anonymous Twitter person DM'ed me this morning with a scenario. Quote, ‘I work at a large cloud company that makes inscrutable naming decisions, and I have an offer elsewhere for 35% more. Should I take it?'” to which you said, “Oh, good heavens, yes. A thread.”What followed is a number of questions that you asked exactly like you just asked now and your short answers to them. And they were just so on point and so quick, and it was so serendipitous for me to see that because this ended up being the tipping point that made me decide that, yes, this is the direction that I want to go. And you know, I'm—let's see, I started in November, so five months into the role. It was more than I ever expected; it's harder than I ever expected, but I'm growing so much, I'm getting a ton of eustress, if you're familiar with that concept of the positive stress that makes your muscles grow. And just wanted to give back to you and in thanks and gratitude for being that tipping point. And that thread definitely led me down this path, so thank you for that.Corey: It's interesting because so far as of this recording, there are no two podcast episodes that came out of that thread because, to be clear, this was the thread-summary of a half-hour conversation I had with the person who messaged me about whether or not she should take the role. Because her manager had gone to bat for her to give her a raise and… yeah, she wanted to be loyal and show thanks for that. Which I get, but the counterpoint to that is okay, you turn down the offer out of loyalty. Great. A month goes by.Now, your manager tells you that he or she is leaving to go work at a different company. Well, that opportunity is gone. Now, what? When it comes to career management, you can't love a company because the company can't ever love you back. And I got some pushback on that from Brian Hall, the VP of Product Marketing at Google Cloud—something about Google seems to be inspiring feedback on this one—because he spent something like 20 years at Microsoft and learned how to work within an organization, and then transfer jobs a couple of times to Amazon, they tried to non-compete lawsuit him on the way out—because, I don't know, his PowerPoints were just that amazing or something, or they're never going to replace his ability to name services badly—who knows why.But he took the other position on this. And I'm not saying that my way is always right, it is provably not, as a self-described terrible employee, but it really is interesting that that's the thing that resonated the most. I take a very mercenary approach to my career and I'm not convinced that's at all the best way, but when someone dangles a significant opportunity in front of you, I always take the view that it's better to explore and learn something about yourself if it appeals and the rest of the stars tend to align. And there's a certain reluctance to go out and try new things, but it's not like you're leaving your family. It's not like you're selling out people who've come to depend on you.Employment is fundamentally a business transaction and the company is never going to be able to have any sort of feeling for you, so you shouldn't necessarily have this sense of loyalty, and oh, it'd be it would leave the team in the lurch if I left. That is the company's problem to deal with. No one is irreplaceable.Johnny: Yeah, and a lot of times when you were talking there, you talked about ‘the company, the company,' but really, it's the people that you're working with that—and that was really what was weighing on me the most. I found myself in the same position. I had just recently gotten promoted. You know, my manager, and my team had gone to bat for me a lot, and so it's hard for me to walk away. But it was ultimately the strong relationships that I had built with the team and my managers over time that allowed me to make this step because as a program manager, I'm always thinking that anything I work on needs to survive multiple generations of stakeholders.So, everything that I do on a day-to-day basis has a breadcrumb trail, so that, hey, if I were to get hit by a bus tomorrow, someone with minimal amount of effort, can pick that up and move forward. And I've actually built that mindset into my entire career. Walking away from a role, you know, it'll always leave a gap, it'll always be challenging for the people and the teams around you, especially if you, you know, have a great affection for them, but by setting myself up to exit and still being there, since you know, Waymo is within the Alphabet companies and I can still talk with my old team, it wasn't like I was completely leaving; I was kind of still there if I needed to be, if they needed help or needed to find something. But I can definitely see what how that would be challenging moving to a totally different company. But yeah, it's really important that if you're thinking about exiting, you have a good exit plan. And I'm all about planning as a program manager, and that just helped kind of grease the wheels a little bit.Corey: I want to call it my own bias. You're right, I use the term team and company interchangeably because that's been my entire career. I, right now, have 12 employees here at The Duckbill Group and it is indistinguishable for me to make any meaningful distinction between team and company. Personally, I'm also not allowed to leave the company, given that I own it, and it looks really bad to the rest of the team if I decide, yeah, I'm going to go do something else now. People don't like playing games with their future.You're on the exact opposite end of a very wide spectrum. It's not that Google slash Alphabet is a big company, but you went from working on cloud computing to self-driving cars and you didn't leave the company, you're still at the same place as far as the benefits, the tenure, the organization, the name on the paycheck in all likelihood, and a bunch of other niceties as well. It almost presents is looking a little bit more like a transfer than it does leaving for a brand new job slash company.Johnny: It definitely was a soft landing to go from Google to Waymo. There were a lot of risks—again, talking about risks and mitigations—that I was concerned about that we're just kind of alleviated by the fact that okay, you can keep your same health care plan and various other things. So, that made it a soft landing for me. But yeah, it really was just making sure that the thing that I was working on at Google was able to be carried forward by the team and the people that I really enjoyed working with. So.Corey: As you went through all of this, you said that you were in Ohio before you wound up taking the job at Google—Johnny: Yeah, Cleveland [crosstalk 00:14:22].Corey: —and one of the best parts about Ohio [unintelligible 00:14:22] family and spending time there is you get to leave at some point. And—Johnny: [laugh].Corey: There was a large part of that of, great. I felt the same way growing up in Maine, let's be very clear here, where when I came to California, it was going to this storied place out of legend. And that was wild. And once your worldview expands, it feels very hard to go back again. At least for me.It took me years to really internalize that if this particular job or this particular path didn't work out, my failure mode—if you want to call it that—was not and then I return to Maine with my tail between my legs and go back to the relatively dead end retail fast food job that I was working before, comparatively. No. It's like, you go in a different direction; you apply the skill set; you have the stamp of validation on you. I mean, you have something working for you that I never did, which is the legitimacy of a household name on your resume. Whereas you look at mine, it's just basically a collection of, “Who are they again?” And, “You make that company up?”Which, fine, whatever. There's a bias in tech—particularly—towards big company names because that's a stamp of approval. You've already got that. The world is very much your oyster when it comes to solving the type of problem that you've been aimed at. I'm used to thinking about this from a almost purely technical point of view.It's like I'm here to write some javascript—badly—and I can write bad JavaScript for you or I can write bad JavaScript for that company across the street, and everyone knows what it is that they're going to get from you: Technical debt. Whereas when you're a technical program manager, that is something that you said varies from between company to company. And you hear founders talking about, “Oh yeah, our first engineering hire, we're going to bring in a VP of engineering; we're going to bring in a whole bunch of engineers; it's going to be great.” You very rarely hear people talk about how excited they are like, “Oh yeah, employee number three is going to be a technical program manager, and we're going to just blow the doors off of folks.” Which haven't been through the growth process myself, yeah, we really should have had a technical program manager analog far sooner; it would have helped us blow the doors off of competition. And great, the things we learn, but only in hindsight.Articulating the value of what a software engineer does is relatively straightforward, even for folks who aren't great salespeople for their own work. Being a TPM inherently requires, on some level, a verification that your understanding and the person that you're talking to are communicating about the same thing. Like, if you wind up having to solve code on a whiteboard, maybe that is part of your conception of it—I mean, you work at Google, probably—but for most companies, it's yeah, my ability to write shitty JavaScript is not the determining factor of success in a TPM role. How do you go about even broaching that conversation?Johnny: So, part of the way that program managers can be successful is through anticipating what's coming next and understanding not only the patterns that were implanted over time, but also thinking ahead. And this actually kind of takes me back to why I learned program management in the first place. Pretty early in my life, I started feeling a great deal of anxiety, especially thinking towards future situations, or, you know, even in the present moment. I mean, we've all been through it right? Right before the big test, you're feeling anxious; maybe talking to your crush—or before you talk to your crush—you're feeling this anticipatory anxiety; in hindsight replaying that interview that you just went through.For me, I was kind of like, constantly stuck in this future-state mode about being anxious about what's coming next, and that combined with ADHD—which is something that I also have—is kind of a wicked combination. And we can talk about that separately, but once I started understanding what program management did and how program management allowed businesses to keep things on track, I realized that there was a parallel into my own life there. The skill of program management actually became my defense against the crippling anxiety that I felt anticipating future events. And it's really become kind of the primary lens by which I understand and synthesize the world around me. And I know that sounds kind of weird, but with ADHD, I have a tendency to either being total diffuse mode and just working on nothing in particular, and letting my attention take me, or being in hyperfocus mode. And when you're hyper-focused and anxious, it can be a deadly combination, right?So, what I learned was taking that hyperfocus and taking that idea of program management and figuring out what it takes to get from here to there. I'm a strong believer in go as far as you can see, and when you get there, you'll see further. And this skill of program management kind of becomes the stepwise function by which I get to that later point, very much like you were saying with coming to Waymo: You never know what you're going to get until you get there. Well, now I see further and in hindsight, it was the right decision. So, the concept of program management is bringing structure, is bringing order, is bringing hierarchy to the chaos and uncertainty that we all naturally navigate in whatever we're doing and trying to transmute that into some kind of transparent order and rhythm, not only for my own benefit to reduce my overall anxiety, but also for the benefit of everyone else who's interested in what's going on. Does that answer your question?Corey: No, it absolutely does. Dealing with ADHD has been sort of what I've been struggling with my entire life. I was lucky and got diagnosed very early, but I always thought it was an aspect of business, but in many respects, it's not just about owning a business; it's about any aspect of your career, where the hardest thing you're ever going to have to do, on some level, is learn to understand and handle your own psychology where there are so many aspects of how things happening can impact us internally. I can't control what event happens next, of people yelling at me on Twitter, or I get a cease and desist from Amazon after they finally realized five years in, “You're not nearly as funny as we thought you were. Stop it.”Great. I can deal with those things, but the question is how I'm going to handle what happens in that type of eventuality? It's, am I going to spiral into a bitter depression? Am I going to laugh it off and keep going on things that are clearly working? Am I going to do something else? And so much of it comes from—at least in my experience—the ability to think through what's going on in a somewhat dispassionate way, and not internalize all of it to a point where you freeze. It's way easier said than done, I want to be very clear on this.Johnny: That's absolutely right. Stepping back, seeing the forest for the trees. I've recently become fascinated with systems thinking. You know, I'm in Silicon Valley, so I might as well start looking into a complex adaptive systems—Corey: Oh, no.Johnny: —[crosstalk 00:21:09] buzzword. We don't have to go down that thread because I'm very much an amateur when it comes to it, but what it does is it forces you to look at the connections between the components rather than the reductionism approach of let's look at this component, let's look at this component… instead, it forces you to step back and see the system as a whole. And so when you're responding to you just got a cease and desist, you know, of course you're going to feel depression, of course you're going to feel anxiety, and understanding all those as part of the system of experiencing that situation, it lets you kind of step back and say, okay, it's normal to be feeling this, it's normal to be feeling that. How can I harness these and structure my approach so that I can get to some further point where I not only know what I can do, and what options are available to me, but I have a clear path forward and strategy for how I want to approach this.Corey: How long have you been in your career at this point?Johnny: So, I graduated college in 2009. And I worked at my first company for about ten years from 2005, so I guess you could say 17 years, plus or minus, if you don't count internships.Corey: Looking back, it's easy to look at where we are at any given point in our career and feel that, oh, well, here's where I started, and here's where I am now, and here are the steps I took along the way where there's a sense of plodding inevitability to it. But there never is because when you're in the moment, in the eternal now that we live in, it's there are millions of things you could do next. If you were to be able to go back to your to talk to yourself at the beginning of your career, what would you do differently? What advice would you give yourself that would have really helped out early on?Johnny: You know, I think the thing that gave me the most leverage in my career was—as I move forward—is seeking out communities of like-minded, positive people. On the surface, that sounds a little shallow; of course, you would want to seek out communities, but what I've observed is that the self-organizing communities that pop up around technologies, or ideas, or roles, their communities of people who want to help you succeed. And I think, you know, one of the ways I reached out to you and was able to contact you was through one of these communities, right? So, you know, I talked a little bit the Cleveland Tech Slack earlier; most people aren't familiar with what mediums are even available. There's Discord, there's forums, there's Slack, there's probably other areas that I'm not aware of, where you can find people who will help you find that next step in your career.Actually [laugh] I got my first taste of community in online video games, so—Corey: Oh no.Johnny: —playing World of Warcraft back in 2003, you know you would have a guild—I was, gosh, how old was I in 2003, basically, early-20s and, you know, you'd have a guild of 40 people trying to coordinate all over one single voice chat server. And there was various groups and subdivisions, and so that was almost a project management exercise in itself. That's where I first learned project management. By the way, I have a sneaking suspicion that the roles that we play and that we are have an affinity for in video games mirror the roles that were best suited to play in life. So, I find myself playing a support class in League of Legends or a priest in World of Warcraft or Lord of the Rings Online. I'm always that support person, the glue that helps keep things moving. And surprise, that's exactly what I do for my career. And it works perfectly. So.Corey: The accountant I keep playing gets eaten by goblins constantly, but, you know—Johnny: [laugh].Corey: —that's the joy that I suppose.Johnny: So, pretty early on, I developed this skill of creating friendships, and those friendships, in turn opened me up to these new communities. So, if I were to give one piece of advice to my early self, it would be to put more emphasis on finding and seeking out the communities that consists of people who are interested in the things that you're interested in, but also are willing to help you get to where you want to go. How do you succeed? Well, you find someone who is doing what you want and you talk to them. About it and you figure out how to get to where you're at from where you're at.And maybe they can't help you, maybe they can help you but, you know, we have a unique ability to crowdsource our questions, whether it's on Reddit, whether it's on Slack or Discord, and just say, “Hey, I'm thinking about this thing. Does anyone have any thoughts?” You're immediately—you know, if you ask the question correctly—given five or six different opinions, and then you can kind of meld and understand, okay, here are the options. Again, going back to what we were saying about how do you even decide what the next steps are? You can crowdsource that now, and so the one piece of advice that I would give is to seek out communities of like-minded positive people.Corey: This episode is sponsored in part by our friends at Vultr. Optimized cloud compute plans have landed at Vultr to deliver lightning fast processing power, courtesy of third gen AMD EPYC processors without the IO, or hardware limitations, of a traditional multi-tenant cloud server. Starting at just 28 bucks a month, users can deploy general purpose, CPU, memory, or storage optimized cloud instances in more than 20 locations across five continents. Without looking, I know that once again, Antarctica has gotten the short end of the stick. Launch your Vultr optimized compute instance in 60 seconds or less on your choice of included operating systems, or bring your own. It's time to ditch convoluted and unpredictable giant tech company billing practices, and say goodbye to noisy neighbors and egregious egress forever. Vultr delivers the power of the cloud with none of the bloat. "Screaming in the Cloud" listeners can try Vultr for free today with a $150 in credit when they visit getvultr.com/morning. That's G E T V U L T R.com/morning. My thanks to them for sponsoring this ridiculous podcast.Corey: And I think the positivity is important. There's a lot as particularly in tech, that breeds a certain cynicism that breeds a contempt almost. And Lord knows, I'm not one to judge; I revel in a lot of that when it comes to making fun of companies' ridiculous marketing and some of the nonsense we have to deal with, but it has to be tempered. You can't do what some of the communities I started out with did. IRC, learn how to configure Debian or FreeBSD, where it was generally, “Oh, great, someone else joined? Let's see what this dumbass wants.”It doesn't work that way. It's like just waiting for someone to ask a question so you can sink the knives in is not helpful. Punch up, not down. And making people feel welcomed and valued, even if they don't understand the local behavioral norms quite yet is super important. I'm increasingly discovering, as I suspect you are as well, that I'm older than I thought were when I talk to folks who are just starting their careers about here's how to manage a career, here's how to think about this, I am veering dangerously close to giving actively harmful advice, if I'm not extraordinarily careful because the path that I walked is very much closed.It is a different world; there are different paths; there's a different societal understanding of technology and its place in the world. There's a—what worked for me does absolutely not work the same way for folks who aren't wildly over-represented. And I increasingly have to back off lest I wind up giving the, I guess, career Boomer advice style of irrelevant and actively harmful stuff. How are you thinking about that?Johnny: So, I guess that kind of gets into the underpinnings of what I think it takes to be successful, right, and how do you find success in any aspect of your career? And—Corey: And what is success?Johnny: It differs for every person—yeah, what is success? And we were talking just before the show about how every person experiences not only what is success, but what does success mean and what do you believe the key is differently. For me—and this is pretty on—brand with where I am in my career and what I do—is I think the key to success is preparation. And it really ties into finding those communities and asking those questions, right?There's three key aspects to it, right? First is understanding how you learn. Everyone learns differently, and so knowing how you learn—and you know, college and school is kind of meant to kind of eke that out; it's how best do you learn? How best can you succeed with these tasks that we give you, study for this test, learn these concepts? If you can understand how you learn, that's the first step in preparing correctly, right, building your personal knowledge systems around that, taking notes, ordered hierarchy, structured thinking, that sort of thing.Knowledge management is a good field, if you ever have some time to figure out what you want to do with your external hard drive of your whiteboard like I have back behind me here. The second aspect is just mastering how to seek out information, right? So, how do you prepare? Well, you have to understand how to seek out information. You mentioned, you know, positive communities versus potentially cynical or toxic communities. Their opinions are still very valid.They might be jaded and they might provide a cynical opinion, but you still need to encompass that within the spectrum of your understanding of the world, right, because they have something that happened to them, or they have some experience that still is very valid from their perspective. So, seeking out information, understanding the people and the tools at your disposal, the communities that you can go to knowing how to discern the signal from the noise. And again, that's really where your thread that really helped me—because you nailed a bunch of the questions that I just wasn't entirely sure on in that Twitter thread, and when I went through that, it hit some of the major points that I was just uncertain on, and you just gave very clear, albeit, you know, somewhat tongue in cheek cynical advice, to say like, don't worry about the company, worry about yourself. And that really was helping me get to that next step.And then lastly, how do you prepare? And this is the one I always struggle with. It's calibrating your confidence barometer. What does that even mean? How can you calibrate your own barometer of your confidence? It's a knowingness; it's knowing what to expect.And so for example, when I was getting into Google, I had no idea what to expect in terms of the interviews. So, what's the first thing I do? I go out and I ask a bunch of people, people who know people who are at Google people who are at Google, what do I expect? What should I prepare for? What communities should I join? What books should I read? What YouTube videos should I watch?I ended up finding a book called Cracking the PM Interview by Gayle—I think her name is Laakmann McDowell. There's a Cracking the Coding Interview as well. That ended up being, like, exactly what I needed, and going through that cover-to-cover got me into Google, amongst other things, and talking with the community. So, calibrating your confidence parameter, that knowingness of, I know that I'm ready enough for this. There will always be things that catch you by surprise, but knowing that you're ready and having that preparation and that internal knowingness not only increases your confidence, but it also increases your ability to operate improvisationally when you're in the moment.And in fact, that's exactly what I went through for this podcast. I have a little document in front of me where I just jotted my notes down last night, I was thinking through, what do I want to cover? What do I want to say? How can I respond to the questions that he's going to ask me? He might ask me, you know, a curveball, but I have some thoughts that are structured, I'm prepared for this so that no matter what happens, I'll be okay. And again, that really gets down to that essence of philosophy of program management that I have. No matter what happens, I'll be okay; no matter what happens, we'll be okay. And believing in that and having a level of knowingness—[laugh].Corey: I am not a planner at all. For me, my confidence comes from the fact that I can't predict what's going to happen so I don't even try. Instead, what I do is I focus on preparing myself to be effectively dynamic enough that whatever curveball comes my way, I can twist myself in a knot and catch it, which drives people to distraction when they're trying to plan a panel that I'm going to be on. “Okay, so we're going to ask this, what's your answer going to be?” I have absolutely no idea until I find the words coming out of my mouth.And if I try and do a rehearsal, I'll make completely different points, and that really bothers folks. It's, I don't know; I'm not here to read a script. I'm here to tell stories, which is great for, you know, improv panel activity and challenging if you're trying to get a software project off the ground. So, you know, there are different strengths that call us in different ways.Johnny: Exactly. I mean, the flip side of preparation is improvisation. And you know, I spent ten years as a jazz musician playing trumpet in a swing band back in Cleveland before I moved out here. And that really helped me understand how to think improvisationally, right? They give you the chords, the underlying structure by which you can operate, and then you can kind of choose your own path through there.And sometimes it's good, sometimes it's bad, you learn over time, you come up with libraries of ideas to pull out of your head at any given time. So, there is an aspect of preparation to improvisation. And I think if you, I would encourage you to think about it more; I bet you do more planning than you think you do; maybe you just don't call it that.Corey: No, I have people for that now.Johnny: [laugh]. “I have people for that.”Corey: I am very deliberately offloading that. Honestly, that was part of the challenge I had psychologically of running my own place. If I were just a little better at following a list or planning things in advance, all these people around me wouldn't have to do all this extra work to clean up my mess. Instead, it's okay, let it go. Just let it go and instead, focus on the thing that I can do this differentiated. That was my path. I don't know how well it works for others, and again, I'm swimming in privilege when I say it.One last topic I want to get into, I think it might be part of the reason that you and I are talking so much about the future, the next generation, and the rest is we're recording this on March 9th. I don't know the date this is going to air, but there's a decent chance that will be after April 22nd, where you and your wife Emily are expecting your first child. So congratulations, even though I'm a little early. I definitely want to get that in there.Johnny: Thank you.Corey: Have you found that since you realized you were expecting a child—with an arrival date, which is generally more accurate than most Amazon order dates—that you find yourself thinking a lot more about the future and how you're going to wind up encapsulating some of the lessons you picked up along the way for, I guess, the next generation of your family?Johnny: Yeah. I mean, everyone who finds himself in this situation, finds himself somewhere between panic and bliss, right? There's some balance that I have to find there. And fortunately, my wife Emily, and I have a very strong rapport when it comes to how I think and how she thinks, and so we're able to—you know, our emotional intelligence is very high; we talk about that sort of thing a lot. And we try to plan for the future as best we can, knowing that things will go off the rails as soon as you know, what's the old saying about the best laid plans and how, you know, every plan is—Corey: Man plans and God laughs.Johnny: Yeah, or goes awry as soon as the first shot is fired, et cetera. Thinking more than five years out is still pretty challenging for me, but thinking within the first five years, we can already sketch out some plans. I already have some ideas of where we want to go and what we want to do and how we want this new child, this being, to experience the world and how we want to impart the things and the wisdom that we've learned and experiences and skills that we've developed—Emily and I—to this new child, realizing that I have no idea what's coming and I have no idea what to expect because I just really haven't had much exposure to babies or children at all in my life, so I'm just kind of rolling the dice here and trusting that it'll all work out really well. And again, going back to communities, the communities that I'm in, there are parenting channels, there are friends and family that I can talk to. So, I have everything that I need in terms of knowledge.Now, I just need to go through the experience, right? So, I'm definitely thinking a lot about the future. In fact, I've got a—I don't know if you can see it here—quarterly plan for my life up here on the wall that I [unintelligible 00:35:33]. It's just something that I can glance at every so often, and there it is, right, there: ‘Q1 2022: Kid.'Corey: How long has that ‘Q1 2022: Kid' been on the board? Like oh, since 2014? Like that is remarkably good planning.Johnny: Mid-2021.Corey: Okay, fair enough.Johnny: No joking: Mid-2021.Corey: [laugh].Johnny: Yeah, just even having that up there and writing a sticky note and slapping it on there for, like, a hey, here's what I think, some of them fall off, some of them don't fall off, but I'll tell you what, more than more often than not, it actually ends up working and happening and being realized, no matter what it is. Because just having it there and glancing at it every so often is that repetition, it keeps it on my mind. It's like, hey, I should probably think about that. The next thing you know, it's done. And then I can take it off and put it in my binder of accomplishments.Corey: I am about five years ahead of you on that particular path that you're on because five years ago, I was expecting my first child. And I don't want to spoil the surprise entirely, but I will Nostradamus this prediction here, five years from now, when you go back and listen to or watch this episode and listen to yourself talk about how you're planning to parent and your hopes and your dreams, you are going to, in a fit of rage, attempt to build a time machine to travel back to what is now the present day for us, in order to slap yourself unconscious for how naive you are being [laugh] because that is—I'm hearing my words coming out of your mouth in a bunch of different ways, and oh my God, I was—it's the common parent story you all these hopes and dreams and aspirations for kids and then they hand you a tiny little baby and suddenly it becomes viscerally real in a different way where, “It's going to be a little while until I can teach you to do a job interview, isn't it?” And other things start wind up happening to, like—Johnny: [laugh]. Right.Corey: —what do I do? I've never held a baby before. How do I not drop it and kill it? And later in time they learn to talk. They talk an awful lot, and then it's like, how do I give them a bath without drowning them in the process? Not because I'm bad at it, but just because I'm at my wit's end because I haven't slept in three days.Parenting is one of the hardest things you'll ever do and everyone has opinions on it. And it's gratifying to know that the world continues to go on even in these after-times where things have gotten fairly dark. It's nice to see that flash of optimism and remember walking down at myself. It's exciting times for you. Congratulations.Johnny: Yeah. Thank you. It's a beautiful thing. And I'm self-aware and I have a knowingness of my naivete, right? And that's part of the fun.And the whole idea of it is an explorative journey. I have no idea what to expect, but I have a good support system; my wife is incredible. She has an early childhood education degree, so that's going to be really useful. Yeah. And so kind of going back to that concept of preparation.And I don't feel a lot of anxiety about it because I am feeling like I have the knowledge, the community, the friends, the family in place so that no matter what happens, I'll be able to maneuver through it. And I can ask, and I can get help. Yeah, so that's where my head is at with that. [laugh].Corey: We'll be checking back in once you're up to your elbows and diapers and I assure you, you'll be lucky if it stops your elbows.Johnny: [laugh].Corey: I really want to thank you for taking the time to talk to me about your own journey and, I guess, a variety of different things; hard to encapsulate it all at once. If people want to learn more or chat with you, where's the best place to find you?Johnny: Yeah, thanks for asking. So, I have a website jmpod.com, JM Pod. My middle name is Michael. So, John Michael Podhradsky. jmpod.com. That links to my blog, there's links to LinkedIn, Twitter, Instagram. I'm most active on Instagram.I'm always looking to connect with and just chat with new people, people who want a new perspective, people who are interesting or want to share their stories with me. Coaching is something that I thought of doing in the long-term. It's not on the plate right now because I'm focused on my current career, but that's something that I'm very interested in doing, so you know, happy to field that questions or if anyone wants to reach out and hey, what communities can I look for or where should I be looking for communities, I'm happy to help with that as well.Corey: I will, of course, put a link to that in the [show notes 00:39:39]. Thanks again for your time. I really appreciate it.Johnny: Yeah, this was a fantastic experience. It's the first podcast I've done, I'm hoping it went well, and I really appreciate that you even asked me to do this. It was a surprise. My eyes went like dinner plates when you said, “Hey, why don't you come join me?” And I said, “Absolutely. That sounds like a fantastic idea.” So, thank you again, Corey. I really appreciate spending time with you and looking forward to doing it again sometime in the future. With a baby in the background, screaming. [laugh].Corey: Oh, yes. They do eventually sleep; you won't believe it for the first three months, but they do eventually pass out. Johnny Podhradsky, technical program manager of Offboard Infrastructure at Waymo. I'm Cloud Economist Corey Quinn and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice along with an angry comment telling me exactly which tweet of mine you followed for advice and it did not in fact help your career one iota.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.
On The Cloud Pod this week, Peter finally gets to share his top announcements of 2021. Plus, Google increases security with Siemplify, Azure updates Defender, and AWS comes into the new year with a lot of changes. A big thanks to this week's sponsors: Foghorn Consulting, which provides full-stack cloud solutions with a focus on strategy, planning, and execution for enterprises seeking to take advantage of the transformative capabilities of AWS, Google Cloud, and Azure. This week's highlights
美國CES即將在週三(5日)開展,技嘉(2376)宣布以線上數位的形式,展出近年極力推廣的「智慧生活圈」中的各種關鍵技術及智慧物聯網應用。 期間,技嘉將透過官網平台「INDUSTRY」介紹各種可帶動產業進行數位轉型的解決方案。
At the start of 2022, we bring you a series of episodes that pull together the five most interesting predictions we found in multiple areas in tech. Today we look at high-performance computing, in which the processing power of a billion-billion calculations per second is close to reality. As to quantum computing, real-world problem solving is still far away, experts say. 1. High-performance computing on the cloud will go mainstream High-Performance Computing (HPC) in the cloud has reached the mainstream, according to a report by Market Watch, which projects that the market for cloud HPC will rise from $6.9 billion in 2020 to $146 billion by 2027. The major factors driving the growth of the cloud HPC market are - complex applications management, the emergence of the big data market, & the adoption of the pay-as-you-go model. IBM, Microsoft, Google, Dell, Amazon Web Services (AWS), Penguin Computing, Sabalcore Computing, Adaptive Computing, Gompute, & Univa Corporation are among the companies leading the market. 2. And so HPC-as-a-service will find traction Many vendors have moved from selling equipment to providing HPCaaS, & its rise is linked to the emergence of the cloud as an HPC solution, according to Verdict. The trend towards HPCaaS is, therefore, benefitting cloud players such as Amazon Web Services (AWS), Google, & Alibaba although traditional HPC vendors are also offering HPCaaS. HPCaaS can be a compelling option for end-users as it puts intense data processing & workloads that require high-performance within reach of companies that lack the necessary capital to hire skilled staff & invest in hardware. HPCaaS brings HPC capabilities to those companies that cannot afford to develop HPC knowledge & infrastructure in-house. 3. Exascale HPC will arrive The high-performance computing (HPC) industry for a decade has been planning for the arrival of exascale systems—supercomputers that can process at least one exaflop or a quintillion (a billion billion) calculations per second. After years of planning, innovations & missed deadlines, the world is ready to fully embrace exascale computing, according to The New Stack. In the US, the first of three planned exascale systems—Frontier, which will be powered by AMD Epyc processors & Radeon Instinct MI200 GPUs—is being assembled at the Oak Ridge National Laboratory & is expected to deliver a performance of 1.5 exaflops. On the heels of that will come Aurora, which will run on Intel's new 4th Generation Xeon Scalable Sapphire Rapids CPUs & Xe-HPC Ponte Vecchio GPUs. It's expected to be completed later in 2022 at the Argonne National Lab. 4. Quantum computing will continue baby steps As to quantum computing, the technology is steadily improving, but it will likely continue to boast more media coverage than practical applications in 2022, experts at consultancy Deloitte predict. Fewer than a dozen companies worldwide will be using QCs as part of their day-to-day operations & only for a limited number of use cases, mainly around optimisation problems. The 2022 revenues for QC hardware, software, & QC-as-a-service will likely be less than $500 million. 5. Investments in QC will likely remain strong Investor interest will likely continue to be strong, according to Deloitte. VCs invested more than $1 billion into the sector in 2021, & one company even went public with a multibillion-dollar valuation. Further, investment in quantum by governments, including China, India, Japan, Germany, Netherlands, Canada, & the US, will likely bring the total to more than $5 billion for the year, Deloitte estimates.
On The Cloud Pod this week, the team finds out whose re:Invent 2021 crystal ball was most accurate. Also Graviton3 is announced, and Adam Selipsky gives his first re:Invent keynote. A big thanks to this week's sponsors: Foghorn Consulting, which provides full-stack cloud solutions with a focus on strategy, planning and execution for enterprises seeking to take advantage of the transformative capabilities of AWS, Google Cloud and Azure. JumpCloud, which offers a complete platform for identity, access, and device management — no matter where your users and devices are located. This week's highlights
[本集節目由 Supermicro 美超微電腦贊助播出] Super Micro Computer, Inc. 為企業級運算、儲存、網路解決方案和綠色運算技術等領域的全球領導者。 Supermicro A+ 系統與組合式架構為第 3 代 AMD EPYC™ 處理器進行了優化,使得效能功耗比及性價比達到更出色的境界。 歡迎共同與 Supermicro 參加 Healthcare+ Expo,將於 2021 年 12 月 2 日至 5 日以絕佳性能、高可擴充性和高效率,為醫療保健領域的各種關鍵任務,提供最優化的解決方案。 了解更多: https://learn-more.supermicro.com/healthcareexpo-2021 ----------------------- 特別公告:收到許多聽眾回饋一次聽完一集 Podcast 的體驗較好,本集開始恢復每週日上架一集,感謝大家繼續支持我們的節目!
之前科技酷宅分享過許多智慧型手機、電腦設備、消費性產品、各種科技小物等等,其實這些背後往往都需要有一個強效的伺服器來儲存、雲端運算、進行數據分析。而由於影視特效動畫、虛擬實境愈來愈精緻的需求下,也都需要如此高效能的伺服器或工作站來支持。 這集節目,我們邀請到 Supermicro 美超微電腦股份有限公司的兩位產品經理 Jade 與 Sylvia 來與大家分享 Supermicro 強大的跨伺服器、儲存、網路的解決方案,特別是它可依客戶需求彈性組合的「模組化構建式伺服器解決方案 」(Building Block Solutions)。 本集節目由 Supermicro 美商美超微贊助播出 Super Micro Computer, Inc. 為企業級運算、儲存、網路解決方案和綠色運算技術等領域的全球領導者。 Supermicro A+ 系統與組合式架構為第3代 AMD EPYC™ 處理器進行了優化,使得效能功耗比及性價比達到更出色的境界。 了解更多: https://www.supermicro.com/zh_tw/products/aplus…
Obviously the Steam Deck falls victim to the supply chain problems, AMD EPYC announcements and how big they're getting in the datacenter, Cisco switch vulnerabilities, review of the Lexar DDR4 3600, why Intel wanted Centaur, and beQuiet! intros some RGB fans. Plus all the "more". You're here for that.Another week is in the books, much more in a pure stream of consciousness that was originally streamed live, and has now been edited for human consumption. Full list of subjects in the time stamps list below.Timestamps0:00 Intro01:19 Burger of the Week03:30 Steam Deck Delayed06:15 AMD's EPYC Data Center Event26:41 Podcast Sponsor: Comet Backup27:58 Why Intel Wanted Centaur33:04 Alder Lake Has AVX-512 Support?34:42 Some Cisco Switch Vulnerabilities37:33 Podcast Sponsor: VPLS 39:01 be quiet! Intros Company's First ARGB Fans41:38 The Stylish Raijintek PAN SLIM Case46:36 Lexar Hades RGB DDR4 3600 RAM51:19 Picks of the Week★ Support this podcast on Patreon ★
Esta semana en Pixeles, te contamos todo sobre el Metaverso y el cambio de nombre de Facebook. Una entrevista con Juan Francisco Aguilar, General Manager Dell Technologies Mexico y Luis Gerardo Garcia General Manager AMD MX, nos hablan de cómo la Pymes pueden lograr una transformación digital con Servidores Dell EMC PowerEdge y tecnología AMD EPYC. El Model 3 de Tesla es el auto ¡más vendido de Europa! te decimos porqué y Halo infinite ahora sí parece será el mejor juego de fin de año. ¡Entra al mundo de los Geeks!
0:00 Exploiting the Giving Tree 0:07 What'sa Meta with you 1:43 Intel XeSS + Aurora 2:48 AMD EPYC: 256 cores 3:37 Wealthfront 4:18 QUICK BITS 4:25 Google Fi E2E call encryption 4:49 DJI Ronin 4D 5:13 Microsoft finds Shrootless macOS flaw 5:47 Playstation PC 6:20 US Army wants 300kW laser News Sources: https://lmg.gg/LdfI8
隨著美國旅遊復甦,出門代步的需求也快速回溫,但身為共享汽車叫車服務平台的 Uber 和 Lyft 卻在困境裡掙扎,利用 Uber 和 Lyft 叫車的乘客也發現最近的乘車價錢之高,等待時間之久,這一切的原因都在於目前司機供給嚴重不足,據統計,目前司機至少還缺少 40%,而即便 Uber 和 Lyft 對司機提出了各種補助方式和教育訓練,司機們仍然不願意回鍋。同時,以餐廳和生鮮雜貨外送的叫車平台 DoorDash 卻蒸蒸日上,在上星期市值一度超越 Uber,本集就讓我們一起來聊聊叫車平台的近況和未來吧! 本集節目由 Supermicro 贊助播出 Super Micro Computer, Inc. 是領先的高性能、高效率伺服器技術創新企業。適用於資料中心、雲端計算、企業IT、Hadoop/大數據、高性能計算和崁入式系統的先進伺服器 Building Block Solution® 的全球首要供應商,Supermicro 致力於透過其「We Keep IT Green®」計畫來保護環境,並且向客戶提供市面上最節能、最環保的解決方案。 Supermicro A+ 系統與組合式架構為第3代 AMD EPYC™ 處理器進行了優化,使得效能功耗比及性價比達到更出色的境界。 Supermicro 全新 H12世代A+ 產品系列搭載 AMD EPYC 7003 系列處理器,已經準備好用最具需求的工作負載,為現代資料中心設立全新標準。A+ 伺服器無可挑剔的性能表現與成本效益比,透過可配置性、密集的系統,能夠從小型至大型規模供應更快且更好的結果。 ➡️ https://www.supermicro.com/zh_tw/products/aplus?utm_source=podcast&utm_medium=episode_description&utm_campaign=CTA&utm_id=just_kidding_tech_soundon 矽谷輕鬆談傳送門 ➡️ https://linktr.ee/jktech #Uber #Lyft #DynamicPricing #SurgePricing #UberEats #DoorDash #Instacart #FoodDelivery #GroceryDelivery #Podcast #JustKiddingTech #矽谷輕鬆談
Twitch Mythen. diZee, spiel doch mal EFT. Der Spielemarkt ist kaputt. Sounds und Kameras. Jim's Heißluftfritteuse. diZee und der AMD EPYC. Encoding PC?! NDI und SDI. Talk über den umbau von thisEguy's neuem Studio. Smarthome. Netzwerk und Strom. Werkzeug und Handwerk. Umzüge. Twitch Sub-Payout.
Gigabyte (2376) increased its server layout and advanced to MWC 2021, and its new edge computing server products seized the opportunity for telecom operators to deploy 5G and 6G pre-planning. Optimistic about the increasing demand for infrastructure such as data centers and cloud storage and high-speed computing, Gigabyte has set a goal of double-digit annual growth and gross profit margin increase for the server business this year. However, the supply chain shortage of materials and other impacts became severe in the second quarter, and the legal person also paid close attention to whether Gigabyte's server shipment momentum would be pressured after the second quarter. The development of the smart life circle of the next technology generation is accelerating. Gigabyte will focus on the high-speed computing (HPC) data center and server business this year. At the one-month COMPUTEX 2021 online exhibition, it will release its data center. After the terminal computing and personal computer life applications, to the 5G to the edge smart IoT deployment products, we will focus on edge computing server products with 5G as the core from the edge computing architecture, and even for 6G. In addition to AMD EPYC servers based on the x86 architecture and Intel Xeon expandable processors, GIGABYTE also launched the ARM64 architecture-based Ampere Altra server, which is fully deployed in the edge computing server product line, targeting telecom operators And enterprises are rushing into the 5G technology application layout market. GIGABYTE has previously released an optimistic outlook for the server business this year, especially for its server material stocking strategy. Since the second half of last year, it has been preparing materials two quarters in advance. Therefore, it is happy to see the shipment of the server business in the first half of this year. The momentum is intact, and it is expected that the double-digit annual growth rate will continue throughout the year. 股票發發發,「台股大行情」開始用LINE@了! 我們將會提供許多好康資訊給大家,請透過下方連結將我們加入好友。 一. https://line.me/R/ti/p/%40gcy7397x Line搜尋 @jqa3557y 二. Telegram,理財STB https://t.me/stbstock11 三. YT 理財教學頻道 https://www.youtube.com/watch?v=t-K-9Um96MQ The legal person believes that with sufficient spare parts and components, GIGABYTE's server business is expected to achieve annual growth of more than 20% throughout the year. However, its growth in data center customers is becoming conservative, and the main growth momentum is expected to come from Fermenting business opportunities at the edge 純網路銀行一切服務都透過網路完成,可以節省掉實體銀行的店租、ATM 、行員、水電費、維護費之類的成本,進而提供更優惠的存款及貸款利率,促進金融產業邁向4.0 Telegram,理財STB https://t.me/stbstock11 技嘉(2376)加碼伺服器布局、前進MWC 2021,旗下邊緣運算伺服器新品搶進電信運營商部署5G及6G前置規畫商機。看好資料中心、雲端存儲等基礎建置及高速運算需求增溫,技嘉對今年伺服器業務設下雙位數年增、毛利率提升的目標。惟供應鏈料件短缺等影響在第二季轉為嚴峻,法人亦密切關注技嘉在第二季後伺服器出貨動能會否受壓。 下一科技世代的智慧生活圈發展正加速起步,技嘉今年將高速運算(HPC)資料中心暨伺服器業務作為營運重點之一,於為期一個月的COMPUTEX 2021線上展中,端出旗下從資料中心之終端運算、個人電腦的生活應用,到5G至邊緣的智慧物聯網部署產品後,再聚焦從邊緣運算架構以5G為核心、甚至為6G進行前置布局的邊緣運算伺服器產品。 除了基於x86架構的超微AMD EPYC、與英特爾Intel Xeon可擴充處理器之伺服器外,技嘉也推出基於ARM64架構的Ampere Altra伺服器,全面部署在邊緣運算的伺服器產品線,鎖定電信運營商及企業在5G技術應用布局市場搶進。 技嘉先前對於今年度的伺服器業務釋出樂觀展望,尤針對伺服器料件備貨策略,內部自去年下半年起即以提前兩個季度備料因應,因此樂看今年上半年伺服器業務的出貨動能無虞,並估全年續以雙位數年增成長可期。 法人則認為,在零組件備料充足的情況下,技嘉伺服器業務全年將可望實現兩成以上的年成長,不過對其在資料中心客戶的成長性轉趨保守,預期主要增長動能多來自於邊緣運算商機發酵 Powered by Firstory Hosting
Leading global tech analysts Patrick Moorhead (Moor Insights & Strategy) and Daniel Newman (Futurum Research) are front and center on The Six Five analyzing the tech industry's biggest news each and every week and also conducting interviews with tech industry "insiders" on a regular basis. The Six Five represents six (6) handpicked topics that will be covered for five (5) minutes each. Welcome to this week's edition of “The 6-5.” I'm Patrick Moorhead with Moor Insights & Strategy, co-host, joined by Daniel Newman with Futurum Research. On this week's show we will be talking: Windows 11 https://twitter.com/PatrickMoorhead/status/1408077679115608079?s=20 Salesforce and AWS Tie-up https://www.salesforce.com/news/press-releases/2021/06/23/salesforce-aws-partnership-expansion/ https://press.aboutamazon.com/news-releases/news-release-details/aws-and-salesforce-announce-expansive-partnership-unify HPE Discover https://www.forbes.com/sites/patrickmoorhead/2021/06/24/hpe-discover-2021the-company-makes-its-customer-case-during-the-age-of-insight/ https://www.forbes.com/sites/moorinsights/2021/06/22/hpes-project-aurora-is-a-big-deal--here-is-what-you-need-to-know/ https://www.hpe.com/psnow/doc/a00115571enw https://futurumresearch.com/research-notes/hpe-announces-new-vertically-optimized-greenlake-cloud-services-at-hpe-discover/ https://futurumresearch.com/research-notes/hpe-acquires-determined-ai-to-accelerate-ml-training-capabilities/ Oracle Support Rewards https://twitter.com/PatrickMoorhead/status/1407729984555388929?s=20 https://futurumresearch.com/research-notes/oracle-support-rewards-entices-customer-acceleration-of-cloud-journey/ Google Cloud goes big on AMD Epyc https://twitter.com/PatrickMoorhead/status/1405666066534109190?s=20 Amazon Invests in Plus https://twitter.com/PatrickMoorhead/status/1407456413102886954?s=20 Disclaimer: This show is for information and entertainment purposes only. While we will discuss publicly traded companies on this show. The contents of this show should not be taken as investment advice.
An anonymous engineer who specializes in Azure & SAP discusses the server-centric design of the XBOX Series X, Intel Ice Lake Xeon, AMD EPYC, and other subjects related to gaming and servers. [SPONSOR: https://www.cdkoffers.com/] 20% software discount code: brokensilicon Windows 10 Pro OEM key: https://bit.ly/2vfKucI Office 2016: https://bit.ly/3aBenUX Office 2019: https://bit.ly/2GcGdJn Windows 10 pro OEM +Office 2019 package: https://bit.ly/2Orz0Jx 0:00 Anonymous Guest Introduction 7:33 Guest's Background in PC Gaming 16:10 XBOX Series X APU was Designed for Servers AND Console 26:10 Will Microsoft put the XBOX Series S APU in Surface Devices? 32:25 Console vs PC Gaming 42:14 Why isn't full Windows 10 on XBOX?! 46:56 Windows Software, Intel Alder Lake Scheduling, Lakefield 53:17 How Cloud Server Services Work, Azure for Multiplayer Games 1:12:20 Azure vs AWS & Luna 1:18:06 Intel Xeon vs AMD EPYC 1:36:23 Is it worth it to host your own private cloud? 1:38:09 CXL, OMI, GenZ, Optane 1:48:46 Disaggregating Servers, ARM, Apple M1 2:03:32 Mid-Generation XBOX & PlayStation upgrades, MGS Remake https://www.theverge.com/2020/6/18/21295326/microsoft-project-xcloud-xbox-series-x-servers-hardware-2021 https://www.phoronix.com/forums/forum/hardware/processors-memory/1253015-linux-5-13-to-add-perf-support-for-intel-alder-lake https://venturebeat.com/2019/07/18/intel-and-sap-broaden-datacenter-tech-partnership/ https://youtu.be/bqkGNtsZejc
✘ Werbung: https://www.Whisky.de/shop/ Von der Computergemeinde fast nicht bemerkt, vollzieht sich gerade eine disruptive Unterbrechung in der CPU-Entwicklung. Jeder aus der Informationstechnologie kennt das empirische Moore'sche Gesetz, das die Verdopplung der #Prozessorleistung alle 2 Jahre beschreibt. Doch dieses Gesetz nach #Moore ist nicht umfassend. Es gibt Bereiche, die schneller laufen als andere. Und das führt in naher Zukunft zur Disruption, weil der #Energieverbrauch der Prozessoren um mehr als eine Zehnerpotenz abnimmt. Doku Faichild ► https://youtu.be/DCbRZGDV-ws Doko Intel ► https://youtu.be/JH2nXMv6yZI Ray Kurzweil ► https://youtu.be/5Ma33-ibjHs Chromebooks ► https://youtu.be/nor5BOJUY8M
Cette semaine : GTA Online adopte un patch, Prime Gaming, Ghost 4.0, TikTok passion tracking, Bandcamp Live, LADbible TV - série "Minutes With", Rollin' de Brave Girls, Jay-Jay Johanson - Rorschach Test, AMD Epyc Milan, Intel toujours : les pubs "Don't Switch", Qualcomm aussi en panne de chips, et le monde de la pub vs. Apple. Lisez plutôt Torréfaction #170 : GTA Online adopte un patch, Ghost 4.0, LADTV, AMD Epyc Milan, les douleurs d’Intel, le monde du tracking vs. Apple avec sa vraie mise en page sur Geekzone. Pensez à vos rétines.
Addison Snell, Dan Olds, and Tiffany Trader discuss AMD's launch of Epyc "Milan."
It's a strange time for infrastructure companies as we start seeing the first full-quarter earnings in this COVID-19 world. Hewlett Packard Enterprise & Lenovo are first up, and we're learning from them how to think about this market. At the same time, AMD's EPYC server CPU continues to gain traction, with new wins in HCI and Deep Learning. Join hosts Matt Kimball and Steve McDowell, both senior analysts at Moor Insights & Strategy, as they digest it all. 01:08 Attending Virtual Conferences 05:50 AMD EPYC & HCI and Edge Computiong 08:50 AMD EPYC & Nvidia's New A100 13:16 We don't talk about HPE Earnings 21:35 Lenovo continues to show strength 30:00 The End.
LLDB Threading support now ready, Multiple IPSec VPN tunnels with FreeBSD, Netflix Optimized FreeBSD's Network Stack More Than Doubled AMD EPYC Performance, happy eyeballs with unwind(8), AWS got FreeBSD ARM 12, OpenSSH U2F/FIDO support, and more. Headlines LLDB Threading support now ready for mainline (https://blog.netbsd.org/tnf/entry/lldb_threading_support_now_ready) Upstream describes LLDB as a next generation, high-performance debugger. It is built on top of LLVM/Clang toolchain, and features great integration with it. At the moment, it primarily supports debugging C, C++ and ObjC code, and there is interest in extending it to more languages. In February, I have started working on LLDB, as contracted by the NetBSD Foundation. So far I've been working on reenabling continuous integration, squashing bugs, improving NetBSD core file support, extending NetBSD's ptrace interface to cover more register types and fix compat32 issues and fixing watchpoint support. Then, I've started working on improving thread support which is taking longer than expected. You can read more about that in my September 2019 report. So far the number of issues uncovered while enabling proper threading support has stopped me from merging the work-in-progress patches. However, I've finally reached the point where I believe that the current work can be merged and the remaining problems can be resolved afterwards. More on that and other LLVM-related events happening during the last month in this report. Multiple IPSec VPN tunnels with FreeBSD (https://blog.socruel.nu/text-only/how-to-multiple-ipsec-vpn-tunnels-on-freebsd.txt) The FreeBSD handbook describes an IPSec VPN tunnel between 2 FreeBSD hosts (see https://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/ipsec.html) But it is also possible to have multiple, 2 or more, IPSec VPN tunnels created and running on a FreeBSD host. How to implement and configure this is described below. The requirements is to have 3 locations (A, B and C) connected with IPSec VPN tunnels using FreeBSD (11.3-RELEASE). Each location has 1 IPSec VPN host running FreeBSD (VPN host A, B and C). VPN host A has 2 IPSec VPN tunnels: 1 to location B (VPN host B) and 1 to location C (VPN host C). News Roundup Netflix Optimized FreeBSD's Network Stack More Than Doubled AMD EPYC Performance (https://www.phoronix.com/scan.php?page=news_item&px=Netflix-NUMA-FreeBSD-Optimized) Drew Gallatin of Netflix presented at the recent EuroBSDcon 2019 conference in Norway on the company's network stack optimizations to FreeBSD. Netflix was working on being able to deliver 200Gb/s network performance for video streaming out of Intel Xeon and AMD EPYC servers, to which they are now at 190Gb/s+ and in the process that doubled the potential of EPYC Naples/Rome servers and also very hefty upgrades too for Intel. Netflix has long been known to be using FreeBSD in their data centers particularly where network performance is concerned. But in wanting to deliver 200Gb/s throughput from individual servers led them to making NUMA optimizations to the FreeBSD network stack. Allocating NUMA local memory for kernel TLS crypto buffers and for backing files sent via sentfile were among their optimizations. Changes to network connection handling and dealing with incoming connections to Nginx were also made. For those just wanting the end result, Netflix's NUMA optimizations to FreeBSD resulted in their Intel Xeon servers going from 105Gb/s to 191Gb/s while the NUMA fabric utilization dropped from 40% to 13%. unwind(8); "happy eyeballs" (https://marc.info/?l=openbsd-tech&m=157475113130337&w=2) In case you are wondering why happy eyeballs: It's a variation on this: https://en.wikipedia.org/wiki/Happy_Eyeballs unwind has a concept of a best nameserver type. It considers a configured DoT nameserver to be better than doing it's own recursive resolving. Recursive resolving is considered to be better than asking the dhcp provided nameservers. This diff sorts the nameserver types by quality, as above (validation, resolving, dead...), and as a tie breaker it adds the median of the round trip time of previous queries into the mix. One other interesting thing about this is that it gets us past captive portals without a check URL, that's why this diff is so huge, it rips out all the captive portal stuff (please apply with patch -E): 17 files changed, 385 insertions(+), 1683 deletions(-) Please test this. I'm particularly interested in reports from people who move between networks and need to get past captive portals. Amazon now has FreeBSD ARM 12 (https://aws.amazon.com/marketplace/pp/B081NF7BY7) Product Overview FreeBSD is an operating system used to power servers, desktops, and embedded systems. Derived from BSD, the version of UNIX developed at the University of California, Berkeley, FreeBSD has been continually developed by a large community for more than 30 years. FreeBSD's networking, security, storage, and monitoring features, including the pf firewall, the Capsicum and CloudABI capability frameworks, the ZFS filesystem, and the DTrace dynamic tracing framework, make FreeBSD the platform of choice for many of the busiest web sites and most pervasive embedded networking and storage systems. OpenSSH U2F/FIDO support in base (https://www.undeadly.org/cgi?action=article;sid=20191115064850) I just committed all the dependencies for OpenSSH security key (U2F) support to base and tweaked OpenSSH to use them directly. This means there will be no additional configuration hoops to jump through to use U2F/FIDO2 security keys. Hardware backed keys can be generated using "ssh-keygen -t ecdsa-sk" (or "ed25519-sk" if your token supports it). Many tokens require to be touched/tapped to confirm this step. You'll get a public/private keypair back as usual, except in this case, the private key file does not contain a highly-sensitive private key but instead holds a "key handle" that is used by the security key to derive the real private key at signing time. So, stealing a copy of the private key file without also stealing your security key (or access to it) should not give the attacker anything. Once you have generated a key, you can use it normally - i.e. add it to an agent, copy it to your destination's authorized_keys files (assuming they are running -current too), etc. At authentication time, you will be prompted to tap your security key to confirm the signature operation - this makes theft-of-access attacks against security keys more difficult too. Please test this thoroughly - it's a big change that we want to have stable before the next release. Beastie Bits DragonFly - git: virtio - Fix LUN scan issue w/ Google Cloud (http://lists.dragonflybsd.org/pipermail/commits/2019-November/719945.html) Really fast Markov chains in ~20 lines of sh, grep, cut and awk (https://0x0f0f0f.github.io/posts/2019/11/really-fast-markov-chains-in-~20-lines-of-sh-grep-cut-and-awk/) FreeBSD Journal Sept/Oct 2019 (https://www.freebsdfoundation.org/past-issues/security-3/) Michael Dexter is raising money for Bhyve development (https://twitter.com/michaeldexter/status/1201231729228308480) syscall call-from verification (https://marc.info/?l=openbsd-tech&m=157488907117170) FreeBSD Forums Howto Section (https://forums.freebsd.org/forums/howtos-and-faqs-moderated.39/) Feedback/Questions Jeroen - Feedback (http://dpaste.com/0PK1EG2#wrap) Savo - pfsense ports (http://dpaste.com/0PZ03B7#wrap) Tin - I want to learn C (http://dpaste.com/2GVNCYB#wrap) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag.
With the ever-increasing need for efficiency and better price performance, a balanced data center infrastructure is a crucial component of today's cloud solutions. Organizations seeking to maximize the value of their cloud and on-premises infrastructures now have a choice when selecting their compute and GPU platforms. In this session, a representative from AMD discusses how AMD EPYC processors and AMD GPUs can help you optimize your workload performance and security while providing significant cost savings. This presentation is brought to you by AMD, an APN Partner.
Епизод 113 на Nerds2Nerds записан на 01.09.2019 Част 1 – обзор на новините Директен линк към част 1 (mp3) – НАПЛийкс – Libra – The International 2019 – Излезе Strength of the Sword ULTIMATE – AMD Epyc и други…
This week Greg , Nick A, Mike, and Thomas cover a lot of ground; must talk about all the things. This week we talk about: MikroTik CHR perf issues with AMD Epyc 30+ Cisco unauthenticated RCEs for various Cisco equipt. Cisco IOS-XE critical (10/10 CVSS) auth vuln Kubernetes DoS vulns Webmin unauthenticated RCE vuln(More)…
It's CPU release season and we get excited about AMD's new line of server chips. Plus our take on AMD's approach to memory encryption, and our struggle to make sense of Intel's Comet Lake line. Also, a few Windows worms you should know about, the end of the road for EV certs, and an embarrassing new Bluetooth attack.
Leading global tech analysts Patrick Moorhead (Moor Insights & Strategy) and Daniel Newman (Futurum Research) are front and center on The Six Five analyzing the tech industry's biggest news each and every week and also conducting interviews with tech industry "insiders" on a regular basis. The Six Five represents six (6) handpicked topics that will be covered for five (5) minutes each. Welcome to this week’s edition of “The 6-5.” I’m Patrick Moorhead with Moor Insights & Strategy, co-host, joined by Daniel Newman with Futurum Research. On this week’s show we will be talking: AMD EPYC launch https://www.forbes.com/sites/patrickmoorhead/2019/08/07/amd-delivers-big-at-2nd-generation-epyc-datacenter-launch/ https://www.forbes.com/sites/moorinsights/2019/08/07/all-roads-lead-to-rome-for-enterprise-it/ Huawei HarmonyOS https://www.wired.com/story/huawei-harmonyos-no-android-replacement/ https://futurumresearch.com/huawei-prepares-harmonyos-to-launch-if-china-us-strains-escalate/ Samsung Note10 https://www.forbes.com/sites/moorinsights/2019/08/07/samsungs-galaxy-note-becomes-its-own-line-with-the-note10/ Canalys numbers https://futurumresearch.com/samsung-note-10-series-devices-are-gorgeous-but-will-it-sell/ Cisco Earnings https://investor.cisco.com/news/news-details/2019/Cisco-Reports-Fourth-Quarter-And-Fiscal-Year-2019-Earnings/default.aspx https://futurumresearch.com/cisco-posts-solid-4th-quarter-but-waves-caution-flag-why-investors-shouldnt-worry/ HPE Mapr acquisition https://www.hpe.com/us/en/newsroom/press-release/2019/08/hpe-advances-its-intelligent-data-platform-with-acquisition-of-mapr-business-assets.html https://futurumresearch.com/hpe-acquires-mapr-furthering-commitment-to-ai-and-analytics/ NVIDIA conversational records https://techcrunch.com/2019/08/13/nvidia-breaks-records-in-training-and-inference-for-real-time-conversational-ai/ Companies Mentioned: AMD Intel Huawei Google Samsung Xiaomi LG Apple Cisco HPE MapR NVIDIA Alright...let’s get started. Disclaimer: This show is for information and entertainment purposes only. While we will discuss publicly traded companies on this show. The contents of this show should not be taken as investment advice.
On the show today, we speak with Developer Advocate and fellow Googler, Sherol Chen about machine learning and AI. Jon Foust and Aja Hammerly learn about the history and impact of AI and ML on technology and gaming. What does it mean to be human? What can machines do better than humans, and what can humans do better than machines? These are the large questions that we aim to solve in order to understand and use AI. Sherol goes on to explain the types of deep learning machines can achieve, from neural networks to decision trees. Sherol also went into depth about the potential social impact of AI as it assists doctors parsing through medical records and plans agricultural endeavors to maximize food production and safety. Sherol also elaborates on the ethical responsibilities we must realize when developing AI projects. For developers looking to build a new AI project, Sherol outlines the pros and cons of using existing tools like Cloud Speech-to-Text, AutoML and AutoML Tables. Sherol Chen Sherol advocates for Machine Learning for Google Cloud, and works in Research at Google Brain for Machine Learning in Music and Creativity for the Magenta team. She’s taught Artificial Intelligence at Stanford and around the world in six different countries. Her PhD work is in Computer Science, researching storytelling and Artificial Intelligence at the Expressive Intelligence Studio. Cool things of the week AMD EPYC processors come to Google—and to Google Cloud blog Kaggle Petfinder Dataset site Streaming data from Cloud Storage into BigQuery using Cloud Functions blog App Engine Standard Ruby site Thagomizer blog Interview AutoML Tables site AutoML Tables Promo Video video Can Machines Think? article AI Impact Challenge site NeurIPS site ICLR site ICML site Machine Learning Crash Course site TensorFlow site Project Magenta site Cloud Speech-to-Text site Cloud AutoML site Sherol’s Blog blog Question of the week You mentioned that you can run App Engine + Rails, how do you handle migrations? Where can you find us next? Jon will be at PAX Dev and PAX West, the internal game summit at Google in Sunnyvale, and taking some personal time to travel to Montreal. Aja will be hanging around at home, on the internet, and at Seattle.rb. Sound Effect Attribution “Coins 1.wav” by ProjectsU012 of Freesound.org “Wedding Bells.wav” by Maurice_J_K of Freesound.org “Small Group Laugh.wav” by Tim.Kahn of Freesound.org
Calling All Platforms Tech - Tech news for fans of Apple, Google and Microsoft
Samsung: 1:32 - Galaxy Note 10. - Microsoft partners with Samsung. - Galaxy Watch Active2. - Play Galaxy Link. General Tech: 32:20 - Nubia Z20. - Fossil Gen 5 smartwatches. - Huawei Harmony OS. - Black Hat USA 2019: 45:14 - Intel Earnings call: 1:00:45 - Intel Xeon 9200. - AMD Epyc 7002 line. Gaming: 1:19:55 - Third party board partners for 5700/xt. - Overwatch League update. - Ninja partners with Mixer. - Gears 5. - Ghostbusters remastered. www.patreon.com/callingallplatforms T-Shirts! Contact: podcast@callingallplatforms.com Social: Facebook Twitter YouTube Apple Podcasts Google Podcasts Android
Ridge Wallet: Save 10% at Ridge Wallet with offer code LTTJULY at https://www.ridgewallet.com/LTT Honey: Honey automatically applies the best coupon codes to save you money at different online checkouts, try it now at https://www.joinhoney.com/linus Squarespace: Visit https://www.squarespace.com/WAN and use offer code WAN for 10% off Buy an LTT shirt, hoodie, hat, and even our own insulated water bottle at https://lmg.gg/wanlttstore Timestamps: (Courtesy of Ruwaidi) 1:55 Cloudflare terminated 8chan 16:40 Linus take on 8chan takedown 18:58 Userbench controversy 28:20 Sponsor 30:27 Sponsor 31:02 Sponsor 32:00 Actual main topic of the WAN Show 33:00 Linus calling out MKBHD for a challenge 37:00 Linus admitting he has a plan 38:45 AMD EPYC 7002 Rome delivers a knockout 42:00 AMD stock analysis? 45:25 AMD EPYC technobabble 51:40 Speaking of ....oops..not on sale yet 52:05 Dr Su going to IBM rumor 55:20 LTT Store merchandise update 1:10:50 Apple is locking iPhone battery repair 1:18:20 Straw Poll results - Have you had your battery swapped in your phone more than once? 1:19:40 Viewer's comments 1:24:50 Linus will be on vacation next week
Jennifer Huffstetler, VP and GM for Data Center Product Management at Intel, joins Chip Chat for a deep dive into the capabilities of a new class of processors: future Intel® Xeon® Scalable processors codenamed Cascade Lake advanced performance. Architected to deliver performance leadership across the widest range of demanding workloads[1], future Intel Xeon Scalable processors codenamed Cascade Lake advanced performance deliver unprecedented memory bandwidth[2] with more memory channels than any other CPU. These processors are expected to offer superior performance (results estimated based on pre-production hardware) in comparison to AMD EPYC on many demanding applications including: • Physics -- MILC up to 1.5X [quantum chromodynamics] [3] • Weather – WRF up to 1.6X [weather research and forecasting model] [4] • Manufacturing – OpenFOAM up to 1.6X [open source CFD] [5] • Life/material sciences – NAMD (APOA1) up to 2.1X [Nanoscale Molecular Dynamics] [6] • Energy – YASK (ISO3DFD) up to 3.1X [stencil benchmark] [7] For more information, please follow Jennifer on Twitter at https://twitter.com/jenhuffstetler and visit https://intel.com/hpc. Software and workloads used in performance tests may have been optimized for performance only on Intel microprocessors. Performance tests, such as SYSmark and MobileMark, are measured using specific computer systems, components, software, operations and functions. Any change to any of those factors may cause the results to vary. You should consult other information and performance tests to assist you in fully evaluating your contemplated purchases, including the performance of that product when combined with other products. For more information go to www.intel.com/benchmarks. Performance results are based on testing or projections as of 6/2017 to 11/7/2018 and may not reflect all publicly available security updates. See configuration disclosure in https://intel.ly/2VUvY2I for details. No product can be absolutely secure. [1] Performance leadership across the widest array of demanding workloads based on https://intel.ly/2VUvY2I. [2] Native DDR memory bandwidth. [3] Future Intel Xeon Scalable processors codenamed Cascade Lake advanced performance provide up to 1.5x MILC performance in comparison to AMD EPYC 7601 (2S configuration), for details see https://intel.ly/2VUvY2I. [4] Future Intel Xeon Scalable processors codenamed Cascade Lake advanced performance provide up to 1.6x WRF performance in comparison to AMD EPYC 7601 (2S configuration), for details see https://intel.ly/2VUvY2I. [5] Future Intel Xeon Scalable processors codenamed Cascade Lake advanced performance provide up to 1.6x OpenFOAM performance in comparison to AMD EPYC 7601 (2S configuration), for details see https://intel.ly/2VUvY2I. Data collected with OpenFOAM® Foundation v5.0. This offering is not approved or endorsed by OpenCFDLimited, producer and distributor of the OpenFOAM software via www.openfoam.com, and owner of the OPENFOAM® and OpenCFD® trademarks. [6] Future Intel Xeon Scalable processors codenamed Cascade Lake advanced performance provide up to 2.1x NAMD (APOA1) performance in comparison to AMD EPYC 7601 (2S configuration), for details see https://intel.ly/2VUvY2I. [7] Future Intel Xeon Scalable processors codenamed Cascade Lake advanced performance provide up to 3.1x YASK (ISO3DFD) performance in comparison to AMD EPYC 7601 (2S configuration), for details see https://intel.ly/2VUvY2I.
Project Trident 18.12 released, Spotifyd on NetBSD, OPNsense 18.7.10 is available, Ultra EPYC AMD Powered Sun Ultra 24 Workstation, OpenRsync, LLD porting to NetBSD, and more. ##Headlines ###AsiaBSDCon 2019 Call for Papers You have until Jan 30th to submit Full paper requirement is relaxed a bit this year (this year ONLY!) due to the short submission window. You don’t need all 10-12 pages, but it is still preferred. Send a message to secretary@asiabsdcon.org with your proposal. Could be either for a talk or a tutorial. Two days of tutorials/devsummit and two days of conference during Sakura season in Tokyo, Japan The conference is also looking for sponsors If accepted, flight and hotel is paid for by the conference ###Project Trident 18.12 Released Twitter account if you want to keep up on project news Screenshots Project Trident Community Telegram Channel DistroWatch Page LinuxActionNews Review RoboNuggie’s in depth review ###Building Spotifyd on NetBSD These are the steps I went through to build and run Spotifyd (this commit at the time of writing) on NetBSD AMD64. It’s a Spotify Connect client so it means I still need to control Spotify from another device (typically my phone), but the audio is played through my desktop… which is where my speakers and headphones are plugged in - it means I don’t have to unplug stuff and re-plug into my phone, work laptop, etc. This is 100% a “good enough for now solution” for me; I have had a quick play with the Go based microcontroller from spotcontrol and that allows a completely NetBSD only experience (although it is just an example application so doesn’t provide many features - great as a basis to build on though). ##News Roundup ###OPNsense 18.7.10 released 2019 means 19.1 is almost here. In the meantime accept this small incremental update with goodies such as Suricata 4.1, custom passwords for P12 certificate export as well as fresh fixes in the FreeBSD base. A lot of cleanups went into this update to make sure there will be a smooth transition to 19.1-RC for you early birds. We expect RC1 in 1-2 weeks and the final 19.1 on January 29. ###Introducing the Ultra EPYC AMD Powered Sun Ultra 24 Workstation A few weeks ago, I got an itch to build a workstation with AMD EPYC. There are a few constraints. First, I needed a higher-clock part. Second, I knew the whole build would be focused more on being an ultra high-end workstation rather than simply utilizing gaming components. With that, I decided it was time to hit on a bit of nostalgia for our readers. Mainly, I wanted to do an homage to Sun Microsystems. Sun made the server gear that the industry ran on for years, and as a fun fact, if you go behind the 1 Hacker Way sign at Facebook’s campus, they left the Sun Microsystems logo. Seeing that made me wonder if we could do an ultimate AMD EPYC build in a Sun Microsystems workstation. ###OpenRsync This is a clean-room implementation of rsync with a BSD (ISC) license. It is designed to be compatible with a modern rsync (3.1.3 is used for testing). It currently compiles and runs only on OpenBSD. This project is still very new and very fast-moving. It’s not ready for wide-spread testing. Or even narrow-spread beyond getting all of the bits to work. It’s not ready for strong attention. Or really any attention but by careful programming. Many have asked about portability. We’re just not there yet, folks. But don’t worry, the system is easily portable. The hard part for porters is matching OpenBSD’s pledge and unveil. ###The first report on LLD porting LLD is the link editor (linker) component of Clang toolchain. Its main advantage over GNU ld is much lower memory footprint, and linking speed. It is of specific interest to me since currently 8 GiB of memory are insufficient to link LLVM statically (which is the upstream default). The first goal of LLD porting is to ensure that LLD can produce working NetBSD executables, and be used to build LLVM itself. Then, it is desirable to look into trying to build additional NetBSD components, and eventually into replacing /usr/bin/ld entirely with lld. In this report, I would like to shortly summarize the issues I have found so far trying to use LLD on NetBSD. ###Ring in the new It’s the second week of 2019 already, which means I’m curious what Nate is going to do with his series This week in usability … reset the numbering from week 1? That series is a great read, to keep up with all the little things that change in KDE source each week — aside from the release notes. For the big ticket items of KDE on FreeBSD, you should read this blog instead. In ports this week (mostly KDE, some unrelated): KDE Plasma has been updated to the latest release, 5.14.5. KDE Applications 18.12.1 were released today, so we’re right on top of them. Marble was fixed for FreeBSD-running-on-Power9. Musescore caught up on 18 months of releases. Phonon updated to 4.10.1, along with its backends. And in development, Qt WebEngine 5.12 has been prepared in the incongruously-named plasma-5.13 branch in Area51; that does contain all the latest bits described above, as well. ##Beastie Bits NomadBSD 1.2-RC1 Released ZFS - The First Enterprise Blockchain Powersaving with DragonFly laptop NetBSD reaches 100% reproducable builds Potential Bhyve Web Interface? LibGDX proof of concept on OpenBSD - Video LiteCLI is a user-friendly CommandLine client for SQLite database In honor of Donald Knuth’s 81 birthday Stanford uploaded 111 lectures on Youtube Portland BSD Pizza Night - 2018-01-31 19:00 - Sweet Heart Pizza Stockholm BSD February meetup Polish BSD User Group: Jan 25 18:15 - 21:00 AsiaBSDcon 2019 CfP ##Feedback/Questions Greg - VLANs and jails Tara - ZFS on removable disks Casey - Interview with Kirk McKusick Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Followup Linus Tech Tips - Linus Swaps Hard Drive Actuator LLVM in Webkit update Webkit B3 JIT Compiler LLVM Wikipedia - History Initialism CES Alienware Area 51 M The Verge - Report The Verge - Video interview with Alienware Product Lead Intel Ark - Core i9-9900K LGA 1151 LG Rollable TV - The Verge video Intel Keynote Anandtech coverage Ian Cutress tweet about 28 core high-clock Xeon pricing Gsync on freesync The Verge coverage Linus Tech Tips video Microsoft 896 core servers Global Kernel locks in APFS Iain’s tweet about Google Bart MN-Core Pentagram designs for Graphcore Intel Nervana AWS re:Invent AWS Ground Station ARM on EC2 AWS Announcement The Register coverage AMD EPYC on EC2 Intel Cascade Lake Xeon - Pincount news! Sub $1000 IBM Power9 motherboard Adversarial Examples that Fool both Human and Computer Vision Fast and accurate object detection in high resolution 4K and 8K video using GPUs
SCP client vulnerabilities, BSDs vs Linux benchmarks on a Tyan EPYC Server, fame for the Unix inventors, Die IPv4, GhostBSD 18.12 released, Unix in pictures, and more. ##Headlines ###scp client multiple vulnerabilities Overview SCP clients from multiple vendors are susceptible to a malicious scp server performing unauthorized changes to target directory and/or client output manipulation. Description Many scp clients fail to verify if the objects returned by the scp server match those it asked for. This issue dates back to 1983 and rcp, on which scp is based. A separate flaw in the client allows the target directory attributes to be changed arbitrarily. Finally, two vulnerabilities in clients may allow server to spoof the client output. Impact Malicious scp server can write arbitrary files to scp target directory, change the target directory permissions and to spoof the client output. Details The discovered vulnerabilities, described in more detail below, enables the attack described here in brief. The attacker controlled server or Man-in-the-Middle(*) attack drops .bash_aliases file to victim’s home directory when the victim performs scp operation from the server. The transfer of extra files is hidden by sending ANSI control sequences via stderr. For example: user@local:~$ scp user@remote:readme.txt . readme.txt 100% 494 1.6KB/s 00:00 user@local:~$ Once the victim launches a new shell, the malicious commands in .bash_aliases get executed. *) Man-in-the-Middle attack does require the victim to accept the wrong host fingerprint. ###FreeBSD 12.0 vs. DragonFlyBSD 5.4 vs. TrueOS 18.12 vs. Linux On A Tyan EPYC Server Last month when running FreeBSD 12.0 benchmarks on a 2P EPYC server I wasn’t able to run any side-by-side benchmarks with the new DragonFlyBSD 5.4 as this BSD was crashing during the boot process on that board. But fortunately on another AMD EPYC server available, the EPYC 1P TYAN Transport SX TN70A-B8026, DragonFlyBSD 5.4.1 runs fine. So for this first round of BSD benchmarking in 2019 are tests of FreeBSD 11.2, FreeBSD 12.0, DragonFlyBSD 5.4.1, the new TrueOS 18.12, and a few Linux distributions (CentOS 7, Ubuntu 18.04.1 LTS, and Clear Linux) on this EPYC 7601 server in a variety of workloads. DragonFlyBSD 5.4.1 ran fine on this Tyan server and could boot fine unlike the issue encountered on the Dell PowerEdge R7425 for this particular BSD. But on the Tyan server, DragonFlyBSD 5.2.2 wouldn’t boot so only this latest DragonFlyBSD release series was used as part of the comparison. A summary of the operating systems tested for this EPYC 7601 OS benchmark comparison included: DragonFlyBSD 5.4.1 - The latest release of Matthew Dillon’s operating system while using the HAMMER2 file-system and GCC 8.1 compiler that is now the default system compiler for this BSD. FreeBSD 11.2 - The previous stable release of FreeBSD. Installed with a ZFS file-system. FreeBSD 12.0 - The latest stable release of FreeBSD and installed with its ZFS option. TrueOS 18.12 - The latest release of the iX systems’ FreeBSD derivative. TrueOS 18.12 is based on FreeBSD 13.0-CURRENT and uses ZFS by default and was using the Clang 7.0.1 compiler compared to Clang 6.0.1 on FreeBSD 12.0. CentOS Linux 7 - The latest EL7 operating system performance. Ubuntu 18.04.1 LTS - The latest Ubuntu Long Term Support release. Clear Linux 27120 - The latest rolling release as of testing out of Intel’s Open-Source Technology Center. Clear Linux often reflects as close to the gold standard for performance as possible with its insanely tuned software stack for offering optimal performance on x86_64 performance for generally showing best what the hardware is capable of. Throughout all of this testing, the Tyan 2U server was kept to its same configuration of an AMD EPYC 7601 (32 cores / 64 threads) at stock speeds, 8 x 16GB DDR4-2666 ECC memory, and 280GB Intel Optane 900p SSD benchmarks. ##News Roundup National Inventors Hall of Fame honors creators of Unix Dennis Ritchie (Posthumous) and Ken Thompson: UNIX Operating System Thompson and Ritchie’s creation of the UNIX operating system and the C programming language were pivotal developments in the progress of computer science. Today, 50 years after its beginnings, UNIX and UNIX-like systems continue to run machinery from supercomputers to smartphones. The UNIX operating system remains the basis of much of the world’s computing infrastructure, and C language – written to simplify the development of UNIX – is one of the most widely used languages today. ###Die IPV4, Die Imagine, it is 2019. Easy, ha? Imagine, it is 2019 and you want to turn off IPv4. Like, off off. Really off. Not disabling IPv6, but disabling IPv4. Two steps back You might be coming here wondering, why would anybody want to do what we are asking to be done. Well, it is dead simple: We are running data centers (like Data Center Light) with a lot of IPv6 only equipment. There simply is no need for IPv4. So why would we want to have it enabled? Also, here at ungleich, we defined 2019 as the year to move away from IPv4. The challenge Do you like puzzles? Competitions? Challenges? Hacking? Well. If ANY of this is of your interest, here is a real challenge for you: We offer a 100 CHF (roughly 100 USD) for anyone who can give us a detailed description of how to turn IPv4 completely off in an operating system and allowing it to communicate with IPv6 only. This should obviously include a tiny proof that your operating system is really unable to use IPv4 at all. Just flushing IPv4 addresses and keeping the IPv4 stack loaded, does not count. ###GhostBSD 18.12 released GhostBSD 18.12 is an updated iso of GhostBSD 18.10 with some little changes to the live DVD/USB and with updated packages. What has changed since 18.10 removed default call of kernel modules for AMD and Intel replaced octopkg by software-station added back gop hacks to the live system added ghostbsd-drivers and ghostbsd-utils we updated the packages to the latest build ###And Now for a laugh : #unixinpictures ##Beastie Bits We are now closer to the Y2038 bug than the Y2K bug OpenBSD Enterprise use AT&T Unix Books Process title and missing memory space The History of a Security Hole unbound-adblock: The ultimate network adblocker! FreeBSD’s name/value pairs library Pid Rollover Booting OpenBSD kernels in EFI mode with QEMU OpenBSD CVS commit: Make mincore lie BSDCan 2019 CfP ending January 19 - Submit! OpenZFS User Conference - April 18-19 FreeBSD Journal is a free publication now ##Feedback/Questions Chris - Boot environments and SSDs Jonathan - Bytes issued during a zpool scrub Bostjan - ZFS Record Size and my mistakes Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
PC Perspective Podcast #488 - 02/22/18 Join us this week for AMD Ryzen performance reviews, Qualcomm news, and more! You can subscribe to us through iTunes and you can still access it directly through the RSS page HERE. The URL for the podcast is: http://pcper.com/podcast - Share with your friends! iTunes - Subscribe to the podcast directly through the iTunes Store (audio only) Video version on iTunes Google Play - Subscribe to our audio podcast directly through Google Play! RSS - Subscribe through your regular RSS reader (audio only) Video version RSS feed MP3 - Direct download link to the MP3 file Hosts: Ryan Shrout, Jeremy Hellstrom, Josh Walrath, Allyn Malventano Peanut Gallery: Alex Lustenberg, Ken Addison Program length: 1:20:48 Podcast topics of discussion: Join our spam list to get notified when we go live! Patreon PCPer Mailbag #31 - 2/16/2018 Week in Review: 0:04:50 Primochill Vue Coolant Review 0:11:30 AMD Ryzen 5 2400G Memory Speed Performance Analysis 0:20:45 BitFenix Whisper M 850W Power Supply Review 0:24:20 Cherry G80-3494 MX Board Silent Review - A Mechanical Keyboard for the Workplace 0:26:15 Logitech MX Master 2S: For Creatives and Professionals News items of interest: 0:29:05 Qualcomm signs major carriers and retailers for Always Connected PC launch 0:34:00 Windows 10 on ARM Details 0:38:25 Raven Ridge Delidded: der8auer Posts AMD Ryzen 5 2400G Before and After Video 0:41:15 AMD EPYC "Rome" Rumors: 7nm, 64 Cores, 2 Designs 0:46:10 ARM Introduces Kigen OS for Cellular IoT 0:54:20 Qualcomm Introduces TrueWireless Stereo Plus and Broadcast Audio 0:56:45 AMD goes after $15B embedded space with two new embedded processors 1:05:15 Qualcomm Announces Snapdragon 845 Mobile VR Reference Platform Picks of the Week: 1:08:00 Ryan: Noctua Thermal Compound 1:12:30 Allyn: UltraVNC 1:16:55 Jeremy: Tomorrow is the Apocalypse - here 1:18:10 Josh: My poor wife http://pcper.com/podcast http://twitter.com/ryanshrout and http://twitter.com/pcper Closing/outro https://i.imgur.com/9uGq2mv.png
1、华为下月27日发布P20主打三摄 2、LG重启门认赔 3、AMD下代EPYC将有64核心 4、苹果关闭iOS降级通道 5、小米6被下架疑似7要来 6、安卓9命名或为派类食物
PC Perspective Podcast #455 - 06/22/17 Join us for talk about Intel Skylake-X, AMD EPYC 7000 series, IBM 5nm, 802.11ad, and more! You can subscribe to us through iTunes and you can still access it directly through the RSS page HERE. The URL for the podcast is: http://pcper.com/podcast - Share with your friends! iTunes - Subscribe to the podcast directly through the iTunes Store (audio only) Video version on iTunes Google Play - Subscribe to our audio podcast directly through Google Play! RSS - Subscribe through your regular RSS reader (audio only) Video version RSS feed MP3 - Direct download link to the MP3 file Hosts: Ryan Shrout, Jeremy Hellstrom, Josh Walrath, Allyn Malventano Peanut Gallery: Alex Lustenberg, Ken Addison Program length: 1:36:49 Podcast topics of discussion: Join our spam list to get notified when we go live! Patreon Donate to the PC Perspective Mining Pool! A NiceHash How-to Week in Review: 0:06:50 The Intel Core i9-7900X 10-core Skylake-X Processor Review 0:32:30 AMD EPYC 7000 Series Data Center Processor Launch - Gunning for Xeon 0:55:10 IBM Announces 5nm Breakthrough with Silicon Nanosheet Technology 1:01:00 802.11ad (WiGig) Tested with ASUS Prime X299-Deluxe and Netgear Nighthawk X10 News items of interest: 1:13:55 iFixit: Microsoft Surface Laptop Gets a Zero on Repairability https://new.nicehash.com/miner/1HHhVWPRpCUst9bDYtLstMdD7o5SzANk1W Hardware/Software Picks of the Week Ryan: MS Word on iPad Jeremy: The Palette is an interesting input device Josh: Now one of the best overall price/perf NVME Allyn: When you need that second GPU a little further away http://pcper.com/podcast http://twitter.com/ryanshrout and http://twitter.com/pcper Closing/outro Subscribe to the PC Perspective YouTube Channel for more videos, reviews and podcasts!!
Join us for talk about Intel Skylake-X, AMD EPYC 7000 series, IBM 5nm, 802.11ad, and more!