Podcasts about YAML

What can GitHub Copilot do for SysAdmins in 2025? Richard talks to Jessica Deen from GitHub about her experiences using Copilot for her work. Jessica talks about Copilot being the first stop for most tasks - describing the task to Copilot helps you think through the problem, and often the tool can generate code or information to get that task done fast. Today's GitHub Copilot can handle everything from explaining existing code to writing something new, debugging a problem, or even writing documentation!LinksGitHub CopilotChanging the AI Model for Copilot ChatVisual Studio Code InsidersAzure ExtensionsGitHub SparkLaunch DarklyRecorded March 13, 2025

Last time we learned how to install Ruby, install Bundler, install Gems, and build a very simple website using Jekyll as our static site generator into GitHub. In this installment of our Jekyll miniseries, Bart explains Jekyll's build process which is mostly automated by how you name things and the content of the files you create (like adding YAML front matter.) Then we spend some quality time bemoaning how the Jekyll developers reuse the word "assets" to mean two different things. Bart avoids some of the associated confusion by creating some naming conventions of our own. We get to do a worked example where we learn a little bit about Pages in Jekyll and do a few things the hard way that we'll redo the easy way in the coming installments. If you're following along realtime, note that we won't be recording for 6 weeks because of some birthdays and Allison's trip to Japan.

Way back in September of 2022, Bart finished off the Webpack miniseries by leaving it as an exercise for the student to deploy their web apps to GitHub Pages. Bart closes that circle in this installment while teaching us how to use GitHub Actions. We learn about workflows, jobs, steps, events, and runners. Bart includes great tables in the shownotes of the terminology, so we now have a handy reference guide for making our own YAML files to run GitHub actions. You can find Bart's fabulous tutorial shownotes at pbs.bartificer.net. Read an unedited, auto-generated transcript with chapter marks: PBS_2025_02_15 Join our Slack at podfeet.com/slack and check out the Programming By Stealth channel under #pbs. Support Bart by going to lets-talk.ie and pushing one of the big blue support buttons. Referral Links: Parallels Toolbox - 3 months free for you and me Learn through MacSparky Field Guides - 15% off for you and me Backblaze - One free month for me and you Eufy - $40 for me if you spend $200. Sadly nothing in it for you. PIA VPN - One month added to Paid Accounts for both of us CleanShot X - Earns me $25%, sorry nothing in it for you but my gratitude

Arnaud et Emmanuel discutent des nouvelles de ce mois. On y parle intégrité de JVM, fetch size de JDBC, MCP, de prompt engineering, de DeepSeek bien sûr mais aussi de Maven 4 et des proxy de répository Maven. Et d'autres choses encore, bonne lecture. Enregistré le 7 février 2025 Téléchargement de l'épisode LesCastCodeurs-Episode-322.mp3 ou en vidéo sur YouTube. News Langages Les evolutions de la JVM pour augmenter l'intégrité https://inside.java/2025/01/03/evolving-default-integrity/ un article sur les raisons pour lesquelles les editeurs de frameworks et les utilisateurs s'arrachent les cheveux et vont continuer garantir l'integrite du code et des données en enlevant des APIs existantes historiquemnt agents dynamiques, setAccessible, Unsafe, JNI Article expliques les risques percus par les mainteneurs de la JVM Franchement c'est un peu leg sur les causes l'article, auto propagande JavaScript Temporal, enfin une API propre et moderne pour gérer les dates en JS https://developer.mozilla.org/en-US/blog/javascript-temporal-is-coming/ JavaScript Temporal est un nouvel objet conçu pour remplacer l'objet Date, qui présente des défauts. Il résout des problèmes tels que le manque de prise en charge des fuseaux horaires et la mutabilité. Temporal introduit des concepts tels que les instants, les heures civiles et les durées. Il fournit des classes pour gérer diverses représentations de date/heure, y compris celles qui tiennent compte du fuseau horaire et celles qui n'en tiennent pas compte. Temporal simplifie l'utilisation de différents calendriers (par exemple, chinois, hébreu). Il comprend des méthodes pour les comparaisons, les conversions et le formatage des dates et des heures. La prise en charge par les navigateurs est expérimentale, Firefox Nightly ayant l'implémentation la plus aboutie. Un polyfill est disponible pour essayer Temporal dans n'importe quel navigateur. Librairies Un article sur les fetch size du JDBC et les impacts sur vos applications https://in.relation.to/2025/01/24/jdbc-fetch-size/ qui connait la valeur fetch size par default de son driver? en fonction de vos use cases, ca peut etre devastateur exemple d'une appli qui retourne 12 lignes et un fetch size de oracle a 10, 2 a/r pour rien et si c'est 50 lignres retournées la base de donnée est le facteur limitant, pas Java donc monter sont fetch size est avantageux, on utilise la memoire de Java pour eviter la latence Quarkus annouce les MCP servers project pour collecter les servier MCP en Java https://quarkus.io/blog/introducing-mcp-servers/ MCP d'Anthropic introspecteur de bases JDBC lecteur de filke system Dessine en Java FX demarrables facilement avec jbang et testes avec claude desktop, goose et mcp-cli permet d'utliser le pouvoir des librarires Java de votre IA d'ailleurs Spring a la version 0.6 de leur support MCP https://spring.io/blog/2025/01/23/spring-ai-mcp-0 Infrastructure Apache Flink sur Kibernetes https://www.decodable.co/blog/get-running-with-apache-flink-on-kubernetes-2 un article tres complet ejn deux parties sur l'installation de Flink sur Kubernetes installation, setup mais aussi le checkpointing, la HA, l'observablité Data et Intelligence Artificielle 10 techniques de prompt engineering https://medium.com/google-cloud/10-prompt-engineering-techniques-every-beginner-should-know-bf6c195916c7 Si vous voulez aller plus loin, l'article référence un très bon livre blanc sur le prompt engineering https://www.kaggle.com/whitepaper-prompt-engineering Les techniques évoquées : Zero-Shot Prompting: On demande directement à l'IA de répondre à une question sans lui fournir d'exemple préalable. C'est comme si on posait une question à une personne sans lui donner de contexte. Few-Shot Prompting: On donne à l'IA un ou plusieurs exemples de la tâche qu'on souhaite qu'elle accomplisse. C'est comme montrer à quelqu'un comment faire quelque chose avant de lui demander de le faire. System Prompting: On définit le contexte général et le but de la tâche pour l'IA. C'est comme donner à l'IA des instructions générales sur ce qu'elle doit faire. Role Prompting: On attribue un rôle spécifique à l'IA (enseignant, journaliste, etc.). C'est comme demander à quelqu'un de jouer un rôle spécifique. Contextual Prompting: On fournit des informations supplémentaires ou un contexte pour la tâche. C'est comme donner à quelqu'un toutes les informations nécessaires pour répondre à une question. Step-Back Prompting: On pose d'abord une question générale, puis on utilise la réponse pour poser une question plus spécifique. C'est comme poser une question ouverte avant de poser une question plus fermée. Chain-of-Thought Prompting: On demande à l'IA de montrer étape par étape comment elle arrive à sa conclusion. C'est comme demander à quelqu'un d'expliquer son raisonnement. Self-Consistency Prompting: On pose plusieurs fois la même question à l'IA et on compare les réponses pour trouver la plus cohérente. C'est comme vérifier une réponse en la posant sous différentes formes. Tree-of-Thoughts Prompting: On permet à l'IA d'explorer plusieurs chemins de raisonnement en même temps. C'est comme considérer toutes les options possibles avant de prendre une décision. ReAct Prompting: On permet à l'IA d'interagir avec des outils externes pour résoudre des problèmes complexes. C'est comme donner à quelqu'un les outils nécessaires pour résoudre un problème. Les patterns GenAI the thoughtworks https://martinfowler.com/articles/gen-ai-patterns/ tres introductif et pre RAG le direct prompt qui est un appel direct au LLM: limitations de connaissance et de controle de l'experience eval: evaluer la sortie d'un LLM avec plusieurs techniques mais fondamentalement une fonction qui prend la demande, la reponse et donc un score numerique evaluation via un LLM (le meme ou un autre), ou evaluation humaine tourner les evaluations a partir de la chaine de build amis aussi en live vu que les LLMs puvent evoluer. Decrit les embedding notament d'image amis aussi de texte avec la notion de contexte DeepSeek et la fin de la domination de NVidia https://youtubetranscriptoptimizer.com/blog/05_the_short_case_for_nvda un article sur les raisons pour lesquelles NVIDIA va se faire cahllengert sur ses marges 90% de marge quand meme parce que les plus gros GPU et CUDA qui est proprio mais des approches ardware alternatives existent qui sont plus efficientes (TPU et gros waffle) Google, MS et d'autres construisent leurs GPU alternatifs CUDA devient de moins en moins le linga franca avec l'investissement sur des langages intermediares alternatifs par Apple, Google OpenAI etc L'article parle de DeepSkeek qui est venu mettre une baffe dans le monde des LLMs Ils ont construit un competiteur a gpt4o et o1 avec 5M de dollars et des capacites de raisonnements impressionnant la cles c'etait beaucoup de trick d'optimisation mais le plus gros est d'avoir des poids de neurores sur 8 bits vs 32 pour les autres. et donc de quatizer au fil de l'eau et au moment de l'entrainement beaucoup de reinforcemnt learning innovatifs aussi et des Mixture of Expert donc ~50x moins chers que OpenAI Donc plus besoin de GPU qui on des tonnes de vRAM ah et DeepSeek est open source un article de semianalytics change un peu le narratif le papier de DeepSkeek en dit long via ses omissions par ensemple les 6M c'est juste l'inference en GPU, pas les couts de recherches et divers trials et erreurs en comparaison Claude Sonnet a coute 10M en infererence DeepSeek a beaucoup de CPU pre ban et ceratins post bans evalués a 5 Milliards en investissement. leurs avancées et leur ouverture reste extremement interessante Une intro à Apache Iceberg http://blog.ippon.fr/2025/01/17/la-revolution-des-donnees-lavenement-des-lakehouses-avec-apache-iceberg/ issue des limites du data lake. non structuré et des Data Warehouses aux limites en diversite de données et de volume entrent les lakehouse Et particulierement Apache Iceberg issue de Netflix gestion de schema mais flexible notion de copy en write vs merge on read en fonction de besoins garantie atomicite, coherence, isoliation et durabilite notion de time travel et rollback partitions cachées (qui abstraient la partition et ses transfos) et evolution de partitions compatbile avec les moteurs de calcul comme spark, trino, flink etc explique la structure des metadonnées et des données Guillaume s'amuse à générer des histoires courtes de Science-Fiction en programmant des Agents IA avec LangChain4j et aussi avec des workflows https://glaforge.dev/posts/2025/01/27/an-ai-agent-to-generate-short-scifi-stories/ https://glaforge.dev/posts/2025/01/31/a-genai-agent-with-a-real-workflow/ Création d'un générateur automatisé de nouvelles de science-fiction à l'aide de Gemini et Imagen en Java, LangChain4j, sur Google Cloud. Le système génère chaque nuit des histoires, complétées par des illustrations créées par le modèle Imagen 3, et les publie sur un site Web. Une étape d'auto-réflexion utilise Gemini pour sélectionner la meilleure image pour chaque chapitre. L'agent utilise un workflow explicite, drivé par le code Java, où les étapes sont prédéfinies dans le code, plutôt que de s'appuyer sur une planification basée sur LLM. Le code est disponible sur GitHub et l'application est déployée sur Google Cloud. L'article oppose les agents de workflow explicites aux agents autonomes, en soulignant les compromis de chaque approche. Car parfois, les Agent IA autonomes qui gèrent leur propre planning hallucinent un peu trop et n'établissent pas un plan correctement, ou ne le suive pas comme il faut, voire hallucine des “function call”. Le projet utilise Cloud Build, le Cloud Run jobs, Cloud Scheduler, Firestore comme base de données, et Firebase pour le déploiement et l'automatisation du frontend. Dans le deuxième article, L'approche est différente, Guillaume utilise un outil de Workflow, plutôt que de diriger le planning avec du code Java. L'approche impérative utilise du code Java explicite pour orchestrer le workflow, offrant ainsi un contrôle et une parallélisation précis. L'approche déclarative utilise un fichier YAML pour définir le workflow, en spécifiant les étapes, les entrées, les sorties et l'ordre d'exécution. Le workflow comprend les étapes permettant de générer une histoire avec Gemini 2, de créer une invite d'image, de générer des images avec Imagen 3 et d'enregistrer le résultat dans Cloud Firestore (base de donnée NoSQL). Les principaux avantages de l'approche impérative sont un contrôle précis, une parallélisation explicite et des outils de programmation familiers. Les principaux avantages de l'approche déclarative sont des définitions de workflow peut-être plus faciles à comprendre (même si c'est un YAML, berk !) la visualisation, l'évolutivité et une maintenance simplifiée (on peut juste changer le YAML dans la console, comme au bon vieux temps du PHP en prod). Les inconvénients de l'approche impérative incluent le besoin de connaissances en programmation, les défis potentiels en matière de maintenance et la gestion des conteneurs. Les inconvénients de l'approche déclarative incluent une création YAML pénible, un contrôle de parallélisation limité, l'absence d'émulateur local et un débogage moins intuitif. Le choix entre les approches dépend des exigences du projet, la déclarative étant adaptée aux workflows plus simples. L'article conclut que la planification déclarative peut aider les agents IA à rester concentrés et prévisibles. Outillage Vulnérabilité des proxy Maven https://github.blog/security/vulnerability-research/attacks-on-maven-proxy-repositories/ Quelque soit le langage, la techno, il est hautement conseillé de mettre en place des gestionnaires de repositories en tant que proxy pour mieux contrôler les dépendances qui contribuent à la création de vos produits Michael Stepankin de l'équipe GitHub Security Lab a cherché a savoir si ces derniers ne sont pas aussi sources de vulnérabilité en étudiant quelques CVEs sur des produits comme JFrog Artifactory, Sonatype Nexus, et Reposilite Certaines failles viennent de la UI des produits qui permettent d'afficher les artifacts (ex: mettez un JS dans un fichier POM) et même de naviguer dedans (ex: voir le contenu d'un jar / zip et on exploite l'API pour lire, voir modifier des fichiers du serveur en dehors des archives) Les artifacts peuvent aussi être compromis en jouant sur les paramètres propriétaires des URLs ou en jouant sur le nomage avec les encodings. Bref, rien n'est simple ni niveau. Tout système rajoute de la compléxité et il est important de les tenir à mettre à jour. Il faut surveiller activement sa chaine de distribution via différents moyens et ne pas tout miser sur le repository manager. L'auteur a fait une présentation sur le sujet : https://www.youtube.com/watch?v=0Z_QXtk0Z54 Apache Maven 4… Bientôt, c'est promis …. qu'est ce qu'il y aura dedans ? https://gnodet.github.io/maven4-presentation/ Et aussi https://github.com/Bukama/MavenStuff/blob/main/Maven4/whatsnewinmaven4.md Apache Maven 4 Doucement mais surement …. c'est le principe d'un projet Maven 4.0.0-rc-2 est dispo (Dec 2024). Maven a plus de 20 ans et est largement utilisé dans l'écosystème Java. La compatibilité ascendante a toujours été une priorité, mais elle a limité la flexibilité. Maven 4 introduit des changements significatifs, notamment un nouveau schéma de construction et des améliorations du code. Changements du POM Séparation du Build-POM et du Consumer-POM : Build-POM : Contient des informations propres à la construction (ex. plugins, configurations). Consumer-POM : Contient uniquement les informations nécessaires aux consommateurs d'artefacts (ex. dépendances). Nouveau Modèle Version 4.1.0 : Utilisé uniquement pour le Build-POM, alors que le Consumer-POM reste en 4.0.0 pour la compatibilité. Introduit de nouveaux éléments et en marque certains comme obsolètes. Modules renommés en sous-projets : “Modules” devient “Sous-projets” pour éviter la confusion avec les Modules Java. L'élément remplace (qui reste pris en charge). Nouveau type de packaging : “bom” (Bill of Materials) : Différencie les POMs parents et les BOMs de gestion des dépendances. Prend en charge les exclusions et les imports basés sur les classifiers. Déclaration explicite du répertoire racine : permet de définir explicitement le répertoire racine du projet. Élimine toute ambiguïté sur la localisation des racines de projet. Nouvelles variables de répertoire : ${project.rootDirectory}, ${session.topDirectory} et ${session.rootDirectory} pour une meilleure gestion des chemins. Remplace les anciennes solutions non officielles et variables internes obsolètes. Prise en charge de syntaxes alternatives pour le POM Introduction de ModelParser SPI permettant des syntaxes alternatives pour le POM. Apache Maven Hocon Extension est un exemple précoce de cette fonctionnalité. Améliorations pour les sous-projets Versioning automatique des parents Il n'est plus nécessaire de définir la version des parents dans chaque sous-projet. Fonctionne avec le modèle de version 4.1.0 et s'étend aux dépendances internes au projet. Support complet des variables compatibles CI Le Flatten Maven Plugin n'est plus requis. Prend en charge les variables comme ${revision} pour le versioning. Peut être défini via maven.config ou la ligne de commande (mvn verify -Drevision=4.0.1). Améliorations et corrections du Reactor Correction de bug : Gestion améliorée de --also-make lors de la reprise des builds. Nouvelle option --resume (-r) pour redémarrer à partir du dernier sous-projet en échec. Les sous-projets déjà construits avec succès sont ignorés lors de la reprise. Constructions sensibles aux sous-dossiers : Possibilité d'exécuter des outils sur des sous-projets sélectionnés uniquement. Recommandation : Utiliser mvn verify plutôt que mvn clean install. Autres Améliorations Timestamps cohérents pour tous les sous-projets dans les archives packagées. Déploiement amélioré : Le déploiement ne se produit que si tous les sous-projets sont construits avec succès. Changements de workflow, cycle de vie et exécution Java 17 requis pour exécuter Maven Java 17 est le JDK minimum requis pour exécuter Maven 4. Les anciennes versions de Java peuvent toujours être ciblées pour la compilation via Maven Toolchains. Java 17 a été préféré à Java 21 en raison d'un support à long terme plus étendu. Mise à jour des plugins et maintenance des applications Suppression des fonctionnalités obsolètes (ex. Plexus Containers, expressions ${pom.}). Mise à jour du Super POM, modifiant les versions par défaut des plugins. Les builds peuvent se comporter différemment ; définissez des versions fixes des plugins pour éviter les changements inattendus. Maven 4 affiche un avertissement si des versions par défaut sont utilisées. Nouveau paramètre “Fail on Severity” Le build peut échouer si des messages de log atteignent un niveau de gravité spécifique (ex. WARN). Utilisable via --fail-on-severity WARN ou -fos WARN. Maven Shell (mvnsh) Chaque exécution de mvn nécessitait auparavant un redémarrage complet de Java/Maven. Maven 4 introduit Maven Shell (mvnsh), qui maintient un processus Maven résident unique ouvert pour plusieurs commandes. Améliore la performance et réduit les temps de build. Alternative : Utilisez Maven Daemon (mvnd), qui gère un pool de processus Maven résidents. Architecture Un article sur les feature flags avec Unleash https://feeds.feedblitz.com//911939960/0/baeldungImplement-Feature-Flags-in-Java-With-Unleash Pour A/B testing et des cycles de développements plus rapides pour « tester en prod » Montre comment tourner sous docker unleash Et ajouter la librairie a du code java pour tester un feature flag Sécurité Keycloak 26.1 https://www.keycloak.org/2025/01/keycloak-2610-released.html detection des noeuds via la proble base de donnée aulieu echange reseau virtual threads pour infinispan et jgroups opentelemetry tracing supporté et plein de fonctionalités de sécurité Loi, société et organisation Les grands morceaux du coût et revenus d'une conférence. Ici http://bdx.io|bdx.io https://bsky.app/profile/ameliebenoit33.bsky.social/post/3lgzslhedzk2a 44% le billet 52% les sponsors 38% loc du lieu 29% traiteur et café 12% standiste 5% frais speaker (donc pas tous) Ask Me Anything Julien de Provin: J'aime beaucoup le mode “continuous testing” de Quarkus, et je me demandais s'il existait une alternative en dehors de Quarkus, ou à défaut, des ressources sur son fonctionnement ? J'aimerais beaucoup avoir un outil agnostique utilisable sur les projets non-Quarkus sur lesquels j'intervient, quitte à y metttre un peu d'huile de coude (ou de phalange pour le coup). https://github.com/infinitest/infinitest/ Conférences La liste des conférences provenant de Developers Conferences Agenda/List par Aurélie Vache et contributeurs : 6-7 février 2025 : Touraine Tech - Tours (France) 21 février 2025 : LyonJS 100 - Lyon (France) 28 février 2025 : Paris TS La Conf - Paris (France) 6 mars 2025 : DevCon #24 : 100% IA - Paris (France) 13 mars 2025 : Oracle CloudWorld Tour Paris - Paris (France) 14 mars 2025 : Rust In Paris 2025 - Paris (France) 19-21 mars 2025 : React Paris - Paris (France) 20 mars 2025 : PGDay Paris - Paris (France) 20-21 mars 2025 : Agile Niort - Niort (France) 25 mars 2025 : ParisTestConf - Paris (France) 26-29 mars 2025 : JChateau Unconference 2025 - Cour-Cheverny (France) 27-28 mars 2025 : SymfonyLive Paris 2025 - Paris (France) 28 mars 2025 : DataDays - Lille (France) 28-29 mars 2025 : Agile Games France 2025 - Lille (France) 3 avril 2025 : DotJS - Paris (France) 3 avril 2025 : SoCraTes Rennes 2025 - Rennes (France) 4 avril 2025 : Flutter Connection 2025 - Paris (France) 4 avril 2025 : aMP Orléans 04-04-2025 - Orléans (France) 10-11 avril 2025 : Android Makers - Montrouge (France) 10-12 avril 2025 : Devoxx Greece - Athens (Greece) 16-18 avril 2025 : Devoxx France - Paris (France) 23-25 avril 2025 : MODERN ENDPOINT MANAGEMENT EMEA SUMMIT 2025 - Paris (France) 24 avril 2025 : IA Data Day 2025 - Strasbourg (France) 29-30 avril 2025 : MixIT - Lyon (France) 7-9 mai 2025 : Devoxx UK - London (UK) 15 mai 2025 : Cloud Toulouse - Toulouse (France) 16 mai 2025 : AFUP Day 2025 Lille - Lille (France) 16 mai 2025 : AFUP Day 2025 Lyon - Lyon (France) 16 mai 2025 : AFUP Day 2025 Poitiers - Poitiers (France) 24 mai 2025 : Polycloud - Montpellier (France) 24 mai 2025 : NG Baguette Conf 2025 - Nantes (France) 5-6 juin 2025 : AlpesCraft - Grenoble (France) 5-6 juin 2025 : Devquest 2025 - Niort (France) 10-11 juin 2025 : Modern Workplace Conference Paris 2025 - Paris (France) 11-13 juin 2025 : Devoxx Poland - Krakow (Poland) 12-13 juin 2025 : Agile Tour Toulouse - Toulouse (France) 12-13 juin 2025 : DevLille - Lille (France) 13 juin 2025 : Tech F'Est 2025 - Nancy (France) 17 juin 2025 : Mobilis In Mobile - Nantes (France) 24 juin 2025 : WAX 2025 - Aix-en-Provence (France) 25-26 juin 2025 : Agi'Lille 2025 - Lille (France) 25-27 juin 2025 : BreizhCamp 2025 - Rennes (France) 26-27 juin 2025 : Sunny Tech - Montpellier (France) 1-4 juillet 2025 : Open edX Conference - 2025 - Palaiseau (France) 7-9 juillet 2025 : Riviera DEV 2025 - Sophia Antipolis (France) 18-19 septembre 2025 : API Platform Conference - Lille (France) & Online 2-3 octobre 2025 : Volcamp - Clermont-Ferrand (France) 6-10 octobre 2025 : Devoxx Belgium - Antwerp (Belgium) 9-10 octobre 2025 : Forum PHP 2025 - Marne-la-Vallée (France) 16-17 octobre 2025 : DevFest Nantes - Nantes (France) 4-7 novembre 2025 : NewCrafts 2025 - Paris (France) 6 novembre 2025 : dotAI 2025 - Paris (France) 7 novembre 2025 : BDX I/O - Bordeaux (France) 12-14 novembre 2025 : Devoxx Morocco - Marrakech (Morocco) 28-31 janvier 2026 : SnowCamp 2026 - Grenoble (France) 23-25 avril 2026 : Devoxx Greece - Athens (Greece) 17 juin 2026 : Devoxx Poland - Krakow (Poland) Nous contacter Pour réagir à cet épisode, venez discuter sur le groupe Google https://groups.google.com/group/lescastcodeurs Contactez-nous via X/twitter https://twitter.com/lescastcodeurs ou Bluesky https://bsky.app/profile/lescastcodeurs.com Faire un crowdcast ou une crowdquestion Soutenez Les Cast Codeurs sur Patreon https://www.patreon.com/LesCastCodeurs Tous les épisodes et toutes les infos sur https://lescastcodeurs.com/

 Il y a tellement de façons et de supports disponibles pour installer et démarrer la domotique Home Assistant que cela sème la confusion chez de nombreux utilisateurs dès le choix du support. Et que dire ensuite quand il est question de choisir le protocole et les différentes étapes nécessaires à une bonne expérience domotique.Dans cet épisode, je vous explique les différentes étapes à suivre pour bien démarrer l'aventure. De la sélection du support selon votre profil en passant par le choix du protocole, les modules et extensions à installer dès le départ, le YAML, les automatisations, la sécurité ... On passe en revue l'ensemble des bonnes pratiques pour faire de votre expérience domotique un incroyable voyage connecté.Comme toujours, pas d'épisode de Domotique Chronique sans de la lecture annexe pour approfondir les connaissances sur domo-blog.fr Retrouvez ci-dessous les articles cités en référence dans ce podcast :La domotique Home Assistant pour les Nuls - Découvrez toutes les étapes et bonnes pratiques nécessaires pour une aventure domotique parfaite et un démarrage optimal avec Home Assistant.Vous aimez cet épisode?Laissez-nous une évaluation et un avis sur Apple Podcasts ⭐⭐⭐⭐⭐Soutenez Domotique Chronique en m'offrant un café sur Ko-Fi ❤️ 

Natural Language vs Deterministic Interfaces for LLMsKey PointsNatural language interfaces for LLMs are powerful but can be problematic for software engineering and automationBenefits of natural language:Flexible input handlingAccessible to non-technical usersWorks well for casual text manipulation tasksChallenges with natural language:Lacks deterministic behavior needed for automationDifficult to express complex logicResults can vary with slight prompt changesNot ideal for command-line tools or batch processingProposed Solution: YAML-Based InterfaceYAML offers advantages as an LLM interface:Structured key-value formatHuman-readable like Python dictionariesCan be linted and validatedEnables unit testing and fuzz testingUsed widely in build systems (e.g., Amazon CodeBuild)Implementation SuggestionsCreate directories of YAML-formatted promptsBuild prompt templates with defined sectionsRun validation and tests for deterministic behaviorConsider using with local LLMs (Ollama, Rust Candle, etc.)Apply software engineering best practicesConclusionMoving from natural language to YAML-structured prompts could improve determinism and reliability when using LLMs for automation and software engineering tasks.

Title: The Case for Makefiles in Modern DevelopmentKey Points:Makefiles provide consistency between development and production environmentsPrimary benefit is abstracting complex commands into simple, uniform recipesParticularly valuable for CI/CD pipelines and cross-language projectsMakefiles solve real-world production problems through command abstractionCommon commands like make install and make lint work consistently across environmentsMain Arguments:While modern build tools (like Cargo for Rust) are powerful, Makefiles still serve an important role in production environmentsMakefiles prevent subtle bugs caused by environment-specific command variationsThey're especially useful when projects combine multiple languages/tools (Rust, XML, YAML, JavaScript, SQL)Linux ubiquity means Make is reliably available on most serversBalanced Perspective:Not advocating Makefiles for all scenariosAcknowledges limitations of older toolsEmphasizes choosing tools based on specific project needsDraws parallel to other standard Unix tools (Vim, Bash) - limitations balanced by ubiquityKey Takeaway: Makefiles remain valuable for production-first development, particularly in enterprise environments with complex CI/CD requirements, despite newer alternatives.Context: Discussion focuses on practical software engineering decisions, emphasizing the importance of considering production environment needs over local development preferences.

For the Season 13 finale, Elixir Wizards Dan and Charles are joined by Spin42 Engineers Marc Lainez, Thibault Poncelet, and Loïc Vigneron to discuss their work retrofitting a 2007 VW Polo and creating an Open Vehicle Control System (OVCS). Using Elixir, Nerves, and Raspberry Pis, the team is reimagining vehicle technology to extend the lifespan of older cars and reduce waste—all while making the process approachable and open source. The Spin42 team shares the technical details behind OVCS and how they use Elixir and Nerves to interact with the CAN bus and build a Vehicle Management System (VMS) to coordinate various vehicle components. They dive into the challenges of reverse engineering CAN messages, designing a distributed architecture with Elixir processes, and ensuring safety with fail-safe modes and emergency shutoffs. Beyond the technical, the team discusses their motivation for the project—upgrading older vehicles with modern features to keep them on the road, building an open-source platform to share their findings with others, and above all-- to just have fun. They explore potential applications for OVCS in boats, construction equipment, and other vehicles, while reflecting on the hurdles of certifying the system for road use. If you've ever wondered how Elixir and Nerves can drive innovation beyond software, this episode is packed with insights into automotive computing, hardware development, and the collaborative potential of open-source projects. Topics Discussed in this Episode: Retrofitting a 2007 VW Polo with electric engines and modern tech Building an open-source Vehicle Control System (OVCS) using Elixir and Nerves Leveraging Elixir to interact with the CAN bus and parse proprietary messages Designing a Vehicle Management System (VMS) to coordinate vehicle components Developing custom hardware for CAN communication Creating a YAML-based DSL for CAN message and frame descriptions Building a distributed architecture using Elixir processes Ensuring safety with fail-safe modes and emergency shutoffs Using Flutter and Nerves to build a custom infotainment system Exploring autonomous driving features with a ROS2 bridge Developing remote control functionality with a Mavlink transmitter Testing OVCS features at scale with a Traxxas RC car (OVCS Mini) Challenges of certifying OVCS for road use and meeting regulatory requirements Encouraging community contributions to expand OVCS functionality Balancing open-source projects with contract work to sustain development The fun and fulfillment of experimenting with Elixir beyond traditional applications Links mentioned: https://www.spin42.com/ https://nerves-project.org/ Quadcopter https://github.com/Spin42/elicopter https://github.com/linux-can/can-utils https://docs.kernel.org/networking/can.html https://github.com/open-vehicle-control-system/cantastic https://github.com/commaai/opendbc https://en.wikipedia.org/wiki/CANbus#CANFD https://comma.ai/ https://en.wikipedia.org/wiki/CANFD https://webkit.org/wpe/ https://docs.nvidia.com/jetson/archives/r35.4.1/DeveloperGuide/text/SD/WindowingSystems/WestonWayland.html https://buildroot.org/ https://vuejs.org/ https://flutter.dev/ https://github.com/smartrent/elixirflutterembedder https://www.raspberrypi.com/products/raspberry-pi-5/ The Rabbit Pickup https://www.hemmings.com/stories/value-guide-1980-83-volkswagen-pickup https://www.expresslrs.org/software/mavlink https://industrial-training-master.readthedocs.io/en/melodic/source/session7/ROS1-ROS2-bridge.html https://github.com/ros2/rcl https://github.com/open-vehicle-control-system/traxxas Contact Marc, Thibault, and Loïc: info@spin42.com Special Guests: Loïc Vigneron, Marc Lainez, and Thibault Poncelet.

In this episode of Maintainable, Robby speaks with Gulcin Yildirim Jelinek, a Staff Database Engineer at Xata. Joining from Prague, Czech Republic, Gulcin discusses her experiences working with legacy databases, the evolution of Postgres, and her passion for building accessible tech communities.Gulcin shares practical insights into modern database management, including the rise of automation tools like YAML and Pgroll, as well as how extensions like PgVector are unlocking new possibilities for Postgres users. Her work with the Prague PostgreSQL Meetup and Diva Conference highlights her dedication to fostering inclusive and welcoming tech communities.Episode Highlights[00:05:32] What Makes Databases Maintainable? Gulcin reflects on documentation, onboarding, and usability.[00:15:10] From Legacy to Modern: Challenges with legacy systems in hospitals and banks and the transition to Postgres.[00:22:18] PgVector and Vector Search: Introducing Postgres extensions to enable vector-based queries.[00:28:12] Scaling Automation with YAML: How YAML transformed database management and DevOps workflows.[00:33:00] Fostering Community and Accessibility: Gulcin's work with Postgres Europe and Diva Conference.[00:36:15] Mythology with a Twist: Book recommendations featuring Circe and Elektra.Key TakeawaysDocumentation Matters: A well-documented system ensures effective onboarding for both developers and end-users.Automation is Key: Tools like YAML and Pgroll streamline database operations, minimizing downtime and manual intervention.Inclusivity in Tech: Conferences and communities should prioritize accessibility, from catering to translation services.Vector Databases in Postgres: PgVector is making Postgres a viable option for AI-driven workloads, eliminating the need for separate systems.Resources MentionedXata BlogPgrollPrague PostgreSQL MeetupDiva: Dive into AI ConferenceKadin Yazilimci (Women Developers of Turkey)Circe by Madeline MillerElektra by Jennifer SaintConnect with GulcinLinkedInPrague PostgreSQL MeetupDiva ConferenceBook Recommendations:CircleElektra:Links:Kadin Yazilimci (Women Developers of Turkey)Diva: Dive into AI ConferencePrague PostgreSQL MeetupOn XXata BlogPgrollThanks to Our Sponsor!Turn hours of debugging into just minutes! AppSignal is a performance monitoring and error-tracking tool designed for Ruby, Elixir, Python, Node.js, Javascript, and other frameworks.It offers six powerful features with one simple interface, providing developers with real-time insights into the performance and health of web applications.Keep your coding cool and error-free, one line at a time! Use the code maintainable to get a 10% discount for your first year. Check them out! Subscribe to Maintainable on:Apple PodcastsSpotifyOr search "Maintainable" wherever you stream your podcasts.Keep up to date with the Maintainable Podcast by joining the newsletter.

In this New Stack Makers, Codiac aims to simplify app deployment on Kubernetes by offering a unified interface that minimizes complexity. Traditionally, Kubernetes is powerful but challenging for teams due to its intricate configurations and extensive manual coding. Co-founded by Ben Ghazi and Mark Freydl, Codiac provides engineers with infrastructure on demand, container management, and advanced software development life cycle (SDLC) tools, making Kubernetes more accessible.Codiac's interface streamlines continuous integration and deployment (CI/CD), reducing deployment steps to a single line of code within CI/CD pipelines. Developers can easily deploy, manage containers, and configure applications without mastering Kubernetes' esoteric syntax. Codiac also offers features like "cabinets" to organize assets across multi-cloud environments and enables repeatable processes through snapshots, making cluster management smoother.For experienced engineers, Codiac alleviates the burden of manually managing YAML files and configuring multiple services. With ephemeral clusters and repeatable snapshots, Codiac supports scalable, reproducible development workflows, giving engineers a practical way to manage applications and infrastructure seamlessly across complex Kubernetes environments.Learn more from The New Stack about deploying applications on Kubernetes:Kubernetes Needs to Take a Lesson from Portainer on Ease-of-Use Three Common Kubernetes Challenges and How to Solve Them Join our community of newsletter subscribers to stay on top of the news and at the top of your game. 

In this Mob Mentality Show episode, Chris Lucian and Austin Chadwick dive into the complexities of modern CI/CD (Continuous Integration / Continuous Delivery) pipeline code and IaC (Infrastructure as Code), exploring why these critical components of software delivery often exhibit the same problematic attributes as classic Legacy Code. Drawing inspiration from Michael Feathers' seminal book *Working Effectively with Legacy Code*, they analyze the paradox of cutting-edge DevOps practices turning into technical debt almost as soon as they're written. ### Episode Highlights: - **CI/CD Pipeline Code and Legacy Code Parallels**: Why does so much CI/CD and IaC code resemble legacy code? Despite being crucial for continuous delivery and automation, CI/CD pipelines can become fragile, difficult to change, and filled with technical debt if not handled carefully. Austin and Chris discuss why this phenomenon is so common and what makes the codebases for CI/CD pipelines especially prone to these issues.    - **“Edit and Pray” vs. TDD Confidence**: Do your CI/CD changes feel like a roll of the dice? Chris and Austin compare how the lack of test-driven development (TDD) practices in CI/CD code leads to “edit and pray” scenarios. They discuss the confidence that TDD brings to traditional application development and how applying similar principles could reduce fragility in CI/CD code. - **The Pitfalls of YAML in IaC**: Is the problem inherent to YAML? The hosts explore whether the complexity of YAML syntax and configurations is the root cause of the brittleness often found in IaC. They provide real-world examples of IaC configurations that suffer from high cyclomatic complexity—making them feel more like full-blown applications rather than simple configuration files. - **Fear of Change in CI/CD and IaC**: Why are developers often afraid to modify CI/CD pipeline code or IaC? Chris and Austin highlight the psychological aspects of fragile infrastructure—where fear of unintended consequences and lack of fast feedback loops result in slower iterations and more bugs. They explore why these codebases are often re-written from scratch instead of extended and safely enhanced. - **Reducing Fragility through Experiments**: The episode features a recent experiment where CI/CD pipeline code was developed in Python using TDD and separation of concerns. This case study reveals the pros and cons of less YAML and a shift towards more code-based "configurations." Could this approach be a solution to reducing brittleness in IaC and pipelines? - **A World Without Brittle Pipelines?**: Imagine a world without fragile pipelines and brittle configuration files. Chris and Austin discuss strategies to move towards more resilient infrastructure and how teams can focus on improving feedback loops, reducing complexity, and enabling safer, faster CI/CD iterations. Join Chris and Austin as they explore these and other crucial topics that are impacting DevOps teams around the world. Whether you're struggling with high bug rates in your pipelines, slow feedback loops, or simply want to better understand how to manage the complexity of modern infrastructure, this episode is for you! Video and Show Notes: https://youtu.be/3Cs-j055b9g 

Deadlock turns cheaters into frogs! California changes how always-online games are sold, Valve invests in Arch Linux, and organizing your Genshin spreadsheet with YAML.

#ittools es la herramienta definitiva que necesitas ya seas #desarrollador o #administrador de sistemas. Un lugar donde tener todas las imprescindibles No he podido resistirme, pero seguro que cuando conozcas esta herramienta, entenderás a que me refiero, e incluso, sabrás disculparme. Hoy te traigo la herramienta única, una herramienta para gobernarlas a todas. En fin… Seas un desarrollador o un administrador de sistemas, esta herramienta es lo que andabas buscando. ¿Cuantas veces te ha sucedido que necesitabas generar un UUID, o convertir una fecha a un formato determinado, o convertir un color de formato, convertir de JSON a YAML, comparar dos JSON, o…?. Podía seguir así, y te ibas a aburrir antes de que yo terminara. Lo cierto, es que existe una herramienta que te permite tener todo esto y mucho más, en un único lugar. Si llama IT Tools, y puedes o consultarla directamente allí o auto alojarla en tu propio servidor. Más información, enlaces y notas en https://atareao.es/podcast/621

SteveO and Frank sat down with Mel, a PwC engineer, to talk about what it's like to be an engineer and the hurdles they face. They dug into her daily routine - from stand-up meetings to ticket management and making time to focus. They also explored how random meetings can kill productivity and the importance of debugging. They touched on cool topics like pair programming, mob programming, and using tools like Stack Overflow and Git branches. This conversation is all about giving you a peek into an engineer's daily life and challenges. Mel shared her experience working with a bunch of tools and systems, including VDI, email, chat, Kubernetes, OpenLens, Terraform, Angular, React, Node, YAML, JSON, and Python. She explained how her team shares info about their system through tickets and updates in JIRA. Then, they talked about the struggles of saving money and how it's often overlooked in engineering roles. Mel and the hosts agreed that finance and engineering teams need to communicate better and work together. They also stressed the importance of prioritizing and getting feedback.

Thank you to the folks at Sustain (https://sustainoss.org/) for providing the hosting account for CHAOSSCast! CHAOSScast – Episode 89 In this episode of CHAOSSCast, host Alice Sowerby is joined by panelists Dawn Foster, Elizabeth Barron, and Harmony Elendu, to discuss the importance and complexities of organizational participation in open source projects. They delve into how the CHAOSS Practitioner Guides help users make sense of CHAOSS metrics and provide insights on improving open source project health. The conversation covers the significance of diverse organizational participation, the role of community managers, and the challenges of accurately measuring organizational contributions. Additionally, they highlight tools like Augur and Grimoire Lab that aid in data collection and analysis as well as offer practical advice for both individuals and companies to foster healthier, more inclusive open source communities. Press download now to hear more! [00:01:50] Dawn gives an overview of the Practitioner Guides that aim to help users navigate the extensive metrics from the CHAOSS Project. They focus on single topics to improve open source project health by making data more accessible. [00:03:19] Why is the Practitioner Guide so important? Dawn explains that organizational participation impacts open source projects' evolution and Elizabeth makes a point that measuring organizational influence is complex and often not straightforward. [00:05:40] There's a discussion on the key factors in organizational participation. Dawn talks about ideal projects having diverse contributors from various organizations, and Elizabeth talks about leadership roles within the project, such as technical steering committee positions that are crucial. [00:09:49] We learn about the problems that people discover and the solutions to resolve the problems as Dawn explains how they've broken down the make improvements section into two different categories. Elizabeth explains how leadership roles within the project, such as technical steering committee positions, are crucial. [00:12:37] Alice mentions if a project is not crucial for an organization's control, donating to a foundation, like CNCF, can be beneficial for wider adoption and support. Dawn adds that donating a project to a foundation can help if the goal is community growth and projects should not be donated merely for marketing purposes. [00:14:24] There's a conversation on how community managers can help facilitate transparency and encourage open contributions rather that keeping discussions and decisions private within the dominant organization, and they can ensure that onboarding experiences, documentation, and community building are prioritized. [00:17:24] Harmony talks about the impact of organizational dominance when one organization dominates a project, it can reduce the project's openness and health and increasing opportunities for external contributions can enhance diversity and inclusion. [00:19:20] The conversation shifts to challenges in measuring metrics and Elizabeth explains cleaning and maintaining accurate data on contributors is challenging due to issues with email addresses and personal vs. organizational contributions. Dawn tells us about the CNCF using YAML or JSON files to track organizational affiliations and update contributor data, but it requires manual effort to keep this information accurate. [00:23:57 ] Alice brings up limitations of metrics since metrics alone don't provide a complete picture, and Elizabeth shares how metrics should be used alongside insights from practitioners and combining metrics with intuition and observing the project directly provides a better overview. [00:25:22] Dawn mentions to accurately assess project leadership and contributions, it's important to talk to people involved since not all leadership roles and contributions are reflected in metrics or governance documents. [00:26:29] Elizabeth inquires if any projects require contributors to fill out profiles indicating if they are contributing personally or on behalf of their company. Dawn explains that she hasn't seen projects require contribution profiles as a prerequisite, and tools like Augur and Grimoire Lab, with its Sorting Hat feature, help manage and clean organizational data. Value Adds (Picks) of the week: [00:29:13] Dawn's pick is going on vacation to visit her family. [00:29:38] Elizabeth's pick is making mead with her grown son. [00:30:27] Harmony's pick is reaching out to old friends to make you smile. [00:31:22] Alice's pick is Dot, an AI companion app. Panelists: Alice Sowerby Dawn Foster Elizabeth Barron Harmony Elendu Links: CHAOSS (https://chaoss.community/) CHAOSS Project X/Twitter (https://twitter.com/chaossproj?lang=en) CHAOSScast Podcast (https://podcast.chaoss.community/) podcast@chaoss.community (mailto:podcast@chaoss.community) Alice Sowerby Website (https://www.rosmarin.co.uk/) Dawn Foster X/Twitter (https://twitter.com/geekygirldawn?lang=en) Elizabeth Barron X/Twitter (https://twitter.com/elizabethn) Harmony Elendu X/Twitter (https://x.com/ogaharmony) Harmony Elendu Substack (https://substack.com/@harmonyelendu) Harmony Elendu LinkedIn (https://www.linkedin.com/in/harmonyelendu/) CHAOSScast Podcast-Episode 85- Introducing CHAOSS Practitioner Guides: #1 Responsiveness (https://podcast.chaoss.community/85) CHAOSScast Podcast-Episode 88-Practitioner Guides: #2 Contributor Sustainability (https://podcast.chaoss.community/88) CHAOSS- Practitioner Guide: Organizational Participation (https://chaoss.community/practitioner-guide-organizational-participation/) Augur (https://github.com/chaoss/augur) Grimoire Lab: Sorting Hat (https://github.com/chaoss/grimoirelab-sortinghat) The Elder Scrolls: The Official Cookbook by Chelsea Monroe-Cassel (https://www.amazon.com/dp/1683833988) Dot (https://new.computer/)

In this second (and final) installment about YAML, Bart teaches us who to write multi-line strings and how not to write multi-line strings. He teaches us about String Blocks which is a bit head-bendy but allows you to write human-readable strings and also tell YAML what to do with empty lines and white space. After that slightly heavy lift, we learn about how to write much simpler-looking Sequences and Mappings than the way we learned in our introduction to YAML in PBS 168. It's really nifty how you can write them in compact, sensible forms, and even easily combine separate YAML documents into the same sequence or mapping. Finally we learn how to use the `yq` language to query JSON, CSV, and XML files using a language that uses `jq` syntax so you'll feel right at home. Read an unedited, auto-generated transcript with chapter marks: PBS_2024_07_06

An airhacks.fm conversation with Jonathan Schneider (@jon_k_schneider) about: OpenRewrite as an open-source tool for code transformation using lossless semantic trees (LSTs), recipes as programs that manipulate the LST, YAML configuration for defining recipes, dry run and in-place code modification options, separation of open-source and commercial aspects of the project, Moderne as a SaaS platform for large-scale code analysis and transformation, visualization features in Moderne including dependency usage violin charts, impact analysis capabilities, organizational structure in Moderne for managing large codebases, integration of OpenRewrite in various IDEs and tools including Amazon Q Code Transformer, IntelliJ IDEA, and Visual Studio Code, the business model of open-source and commercial offerings, the genesis of OpenRewrite from Gradle Lint in 2015-2016, recent momentum in adoption, Jonathan's background with micrometer project, discussion about IDEs including Visual Studio Code and IntelliJ IDEA, potential future topics including Micrometer and Spinnaker Jonathan Schneider on twitter: @jon_k_schneider

In Programming By Stealth, we've completed our series on the jq language and now Bart Busschots brings us a two-part miniseries about the YAML data format. He takes us through the history of data formats we've "enjoyed" such as fixed-width text files, Comma Separated Value files, through to JSON and XML. All of them had their place in history but also had their downsides. YAML promises to be human-readable (yay) and computer-readable (also yay.) Once we're bought into how YAML is the data format of our dreams, Bart explains that there are only two kinds of data, scalar,s and collections, and that collections can be sequences or mapping and all of these data types go into a document. Luckily this is all of the jargon we'll have to learn and there are useful synonyms from other languages (e.g. sequences are really just arrays). I found this lesson enjoyable and not too hard on my little brain so I suspect you'll enjoy it as much as I did. You can find Bart's fabulous tutorial shownotes at pbs.bartificer.net. Read an unedited, auto-generated transcript with chapter marks: CCATP_2024_06_22

In Programming By Stealth, we've completed our series on the jq language and now Bart Busschots brings us a two-part miniseries about the YAML data format. He takes us through the history of data formats we've "enjoyed" such as fixed-width text files, Comma Separated Value files, through to JSON and XML. All of them had their place in history but also had their downsides. YAML promises to be human-readable (yay) and computer-readable (also yay.) Once we're bought into how YAML is the data format of our dreams, Bart explains that there are only two kinds of data, scalar,s and collections, and that collections can be sequences or mapping and all of these data types go into a document. Luckily this is all of the jargon we'll have to learn and there are useful synonyms from other languages (e.g. sequences are really just arrays). I found this lesson enjoyable and not too hard on my little brain so I suspect you'll enjoy it as much as I did. You can find Bart's fabulous tutorial shownotes at pbs.bartificer.net. Read an unedited, auto-generated transcript with chapter marks: CCATP_2024_06_22

In this episode, Lois Houston and Nikita Abraham, along with senior OCI instructor Mahendra Mehra, dive into the fundamentals of Kubernetes. They talk about how Kubernetes tackles challenges in deploying and managing microservices, and enhances software performance, flexibility, and availability.   OCI Container Engine for Kubernetes Specialist: https://mylearn.oracle.com/ou/course/oci-container-engine-for-kubernetes-specialist/134971/210836   Oracle University Learning Community: https://education.oracle.com/ou-community   LinkedIn: https://www.linkedin.com/showcase/oracle-university/   X (formerly Twitter): https://twitter.com/Oracle_Edu   Special thanks to Arijit Ghosh, David Wright, Radhika Banka, and the OU Studio Team for helping us create this episode.   --------------------------------------------------------   Episode Transcript:   00:00 Welcome to the Oracle University Podcast, the first stop on your cloud journey. During this series of informative podcasts, we'll bring you foundational training on the most popular Oracle technologies. Let's get started! 00:26 Lois: Hello and welcome to another episode of the Oracle University Podcast. I'm Lois Houston, Director of Innovation Programs with Oracle University, and with me is Nikita Abraham, Principal Technical Editor. Nikita: Hi everyone! We've spent the last two episodes getting familiar with containerization and the Oracle Cloud Infrastructure Registry. Today, it's going to be all about Kubernetes. So if you've heard of Kubernetes but you don't know what it is, or you've been playing with Docker and containers and want to know how to take it to the next level, you'll want to stay with us. Lois: That's right, Niki. We'll be chatting with Mahendra Mehra, a senior OCI instructor with Oracle University, about the challenges in containerized applications within a complex business setup and how Kubernetes facilitates container orchestration and improves its effectiveness, resulting in better software performance, flexibility, and availability. 01:20 Nikita: Hi Mahendra. To start, can you tell us when you would use Kubernetes?  Mahendra: While deploying and managing microservices in a distributed environment, you may run into issues such as failures or container crashes. Issues such as scheduling containers to specific machines depending upon the configuration. You also might face issues while upgrading or rolling back the applications which you have containerized. Scaling up or scaling down containers across a set of machines can be troublesome.  01:50 Lois: And this is where Kubernetes helps automate the entire process?  Mahendra: Kubernetes is a portable, extensible, open source platform for managing containerized workloads and services that facilitates both declarative configuration and automation.  You can think of a Kubernetes as you would a conductor for an orchestra. Similar to how a conductor would say how many violins are needed, which one play first, and how loud they should play, Kubernetes would say, how many webserver front-end containers or back-end database containers are needed, what they serve, and how many resources are to be dedicated to each one. 02:27 Nikita: That's so cool! So, how does Kubernetes work?  Mahendra: In Kubernetes, there is a master node, and there are multiple worker nodes. Each worker node can handle multiple pods. Pods are just a bunch of containers clustered together as a working unit. If a worker node goes down, Kubernetes starts new pods on the functioning worker node. 02:47 Lois: So, the benefits of Kubernetes are… Mahendra: Kubernetes can containerize applications of any scale without any downtime. Kubernetes can self-heal containerized applications, making them resilient to unexpected failures.  Kubernetes can autoscale containerized applications as for the workload and ensure optimal utilization of cloud resources. Kubernetes also greatly simplifies the process of deployment operations. With Kubernetes, however complex an operation is, it could be performed reliably by executing a couple of commands at the most. 03:19 Nikita: That's great. Mahendra, can you tell us a bit about the architecture and main components of Kubernetes? Mahendra: The Kubernetes cluster has two main components. One is the control plane, and one is the data plane. The control plane hosts the components used to manage the Kubernetes cluster. And the data plane basically hosts all the worker nodes that can be virtual machines or physical machines. These worker nodes basically host pods which run one or more containers. The containers running within these pods are making use of Docker images, which are managed within the image registry. In case of OCI, it is the container registry. 03:54 Lois: Mahendra, you mentioned nodes and pods. What are nodes? Mahendra: It is the smallest unit of computing hardware within the Kubernetes. Its work is to encapsulate one or more applications as containers. A node is a worker machine that has a container runtime environment within it. 04:10 Lois: And pods? Mahendra: A pod is a basic object of Kubernetes, and it is in charge of encapsulating containers, storage resources, and network IPs. One pod represents one instance of an application within Kubernetes. And these pods are launched in a Kubernetes cluster, which is composed of nodes. This means that a pod runs on a node but can easily be instantiated on another node. 04:32 Nikita: Can you run multiple containers within a pod? Mahendra: A pod can even contain more than one container if these containers are relatively tightly coupled. Pod is usually meant to run one application container inside of it, but you can run multiple containers inside one pod. Usually, it is only the case if you have one main application container and a helper container or some sidecar containers that has to run inside of that pod. Every pod is assigned a unique private IP address, using which the pods can communicate with one another. Pods are meant to be ephemeral, which means they die easily. And if they do, upon re-creation, they are assigned a new private IP address. In fact, Kubernetes can scale a number of these pods to adapt for the incoming traffic, consequently creating or deleting pods on demand. Kubernetes guarantees the availability of pods and replicas specified, but not the liveliness of each individual pod. This means that other pods that need to communicate with this application or component cannot rely on the underlying individual pod's IP address. 05:35 Lois: So, how does Kubernetes manage traffic to this indecisive number of pods with changing IP addresses? Mahendra: This is where another component of Kubernetes called services comes in as a solution. A service gets allocated a virtual IP address and lives until explicitly destroyed. Requests to the services get redirected to the appropriate pods, thus the services of a stable endpoint used for inter-component or application communication. And the best part here is that the lifecycle of service and the pods are not connected. So even if the pod dies, the service and the IP address will stay, so you don't have to change their endpoints anymore. 06:13 Nikita: What types of services can you create with Kubernetes? Mahendra: There are two types of services that you can create. The external service is created to allow external users to connect the containerized applications within the pod. Internal services can also be created that restrict the communication within the cluster. Services can be exposed in different ways by specifying a particular type. 06:33 Nikita: And how do you define these services? Mahendra: There are three types in which you can define services. The first one is the ClusterIP, which is the default service type that exposes services on an internal IP within the cluster. This type makes the service only reachable from within the cluster. You can specify the type of service as NodePort. NodePort basically exposes the service on the same port of each selected node in the cluster using a network address translation and makes the service accessible from the outside of the cluster using the node IP and the NodePort combination. This is basically a superset of ClusterIP. You can also go for a LoadBalancer type, which basically creates an external load balancer in the current cloud. OCI supports LoadBalancer types. It also assigns a fixed external IP to the service. And the LoadBalancer type is a superset of NodePort. 07:25 Lois: There's another component called ingress, right? When do you used that? Mahendra: An ingress is used when we have multiple services on our cluster, and we want the user requests routed to the services based on their pod, and also, if you want to talk to your application with a secure protocol and a domain name. Unlike NodePort or LoadBalancer, ingress is not actually a type of service. Instead, it is an entry point that sits in front of the multiple services within the cluster. It can be defined as a collection of routing rules that govern how external users access services running inside a Kubernetes cluster. Ingress is most useful if you want to expose multiple services under the same IP address, and these services all use the same Layer 7 protocol, typically HTTP. 08:10 Lois: Mahendra, what about deployments in Kubernetes?  Mahendra: A deployment is an object in Kubernetes that lets you manage a set of identical pods. Without a deployment, you will need to create, update, and delete a bunch of pods manually. With the deployment, you declare a single object in a YAML file, and the object is responsible for creating the pods, making sure they stay up-to-date and ensuring there are enough of them running. You can also easily autoscale your applications using a Kubernetes deployment. In a nutshell, the Kubernetes deployment object lets you deploy a replica set of your pods, update the pods and the replica sets. It also allows you to roll back to your previous deployment versions. It helps you scale a deployment. It also lets you pause or continue a deployment. 08:59 Do you want to stay ahead of the curve in the ever-evolving AI landscape? Look no further than our brand-new OCI Generative AI Professional course and certification. For a limited time only, we're offering both the course and certification for free! So, don't miss out on this exclusive opportunity to get certified on Generative AI at no cost. Act fast because this offer is valid only until July 31, 2024. Visit https://education.oracle.com/genai to get started. That's https://education.oracle.com/genai. 09:37 Nikita: Welcome back! We were talking about how useful a Kubernetes deployment is in scaling operations. Mahendra, how do pods communicate with each other?  Mahendra: Pods communicate with each other using a service. For example, my application has a database endpoint. Let's say it's a MySQL service that it uses to communicate with the database. But where do you configure this database URL or endpoints? Usually, you would do it in the application properties file or as some kind of an external environment variable. But usually, it's inside the build image of the application. So for example, if the endpoint of the service or the service name, in this case, changes to something else, you would have to adjust the URL in the application. And this will cause you to rebuild the entire application with a new version, and you will have to push it to the repository. You'll then have to pull that new image into your pod and restart the whole thing. For a small change like database URL, this is a bit tedious. So for that purpose, Kubernetes has a component called ConfigMap. ConfigMap is a Kubernetes object that maintains a key value store that can easily be used by other Kubernetes objects, such as pods, deployments, and services. Thus, you can define a ConfigMap composed of all the specific variables for your environment. In Kubernetes, now you just need to connect your pod to the ConfigMap, and the pod will read all the new changes that you have specified within the ConfigMap, which means you don't have to go on to build a new image every time a configuration changes. 11:07 Lois: So then, I'm just wondering, if we have a ConfigMap to manage all the environment variables and URLs, should we be passing our username and password in the same file? Mahendra: The answer is no. Password or other credentials within a ConfigMap in a plain text format would be insecure, even though it's an external configuration. So for this purpose, Kubernetes has another component called secret. Kubernetes secrets are secure objects which store sensitive data, such as passwords, OAuth tokens, and SSH keys with the encryption within your cluster. Using secrets gives you more flexibility in a pod lifecycle definition and control over how sensitive data is used. It reduces the risk of exposing the data to unauthorized users. 11:50 Nikita: So, you're saying that the secret is just like ConfigMap or is there a difference? Mahendra: Secret is just like ConfigMap, but the difference is that it is used to store secret data credentials, for example, database username and passwords, and it's stored in the base64 encoded format. The kubelet service stores this secret into a temporary file system. 12:11 Lois: Mahendra, how does data storage work within Kubernetes? Mahendra: So let's say we have this database pod that our application uses, and it has some data or generates some data. What happens when the database container or the pod gets restarted? Ideally, the data would be gone, and that's problematic and inconvenient, obviously, because you want your database data or log data to be persisted reliably for long term. To achieve this, Kubernetes has a solution called volumes. A Kubernetes volume basically is a directory that contains data accessible to containers in a given pod within the Kubernetes platform. Volumes provide a plug-in mechanism to connect ephemeral containers with persistent data stores elsewhere. The data within a volume will outlast the containers running within the pod. Containers can shut down and restart because they are ephemeral units. Data remains saved in the volume even if a container crashes because a container crash is not enough to cut off a pod from a node. 13:10 Nikita: Another main component of Kubernetes is a StatefulSet, right? What can you tell us about it?  Mahendra: Stateful applications are applications that store data and keep tracking it. All databases such as MySQL, Oracle, and PostgreSQL are examples of Stateful applications. In a modern web application, we see stateless applications connecting with Stateful application to serve the user request. For example, a Node.js application is a stateless application that receives new data on each request from the user. This application is then connected with a Stateful application, such as MySQL database, to process the data. MySQL stores the data and keeps updating the database on the user's request.  Now, assume you deployed a MySQL database in the Kubernetes cluster and scaled this to another replica, and a frontend application wants to access the MySQL cluster to read and write data. The read request will be forwarded to both these pods. However, the write request will only be forwarded to the first primary pod. And the data will be synchronized with other pods. You can achieve this by using the StatefulSets. Deleting or scaling down a StatefulSet will not delete the volumes associated with the Stateful applications. This gives you your data safety. If you delete the MySQL pod or if the MySQL pod restarts, you can have access to the data in the same volume.  So overall, a StatefulSet is a good fit for those applications that require unique network identifiers; stable persistent storage; ordered, graceful deployment and scaling; as well as ordered, automatic rolling updates. 14:43 Lois: Before we wrap up, I want to ask you about the features of Kubernetes. I'm sure there are countless, but can you tell us the most important ones? Mahendra: Health checks are used to check the container's readiness and liveness status. Readiness probes are intended to let Kubernetes know if the app is ready to serve the traffic.  Networking plays a significant role in container orchestration to isolate independent containers, connect coupled containers, and provide access to containers from the external clients. Service discovery allows containers to discover other containers and establish connections to them. Load balancing is a dedicated service that knows which replicas are running and provides an endpoint that is exposed to the clients. Logging allows us to oversee the application behavior.  The rolling update allows you to update a deployed containerized application with minimal downtime using different update scenarios. The typical way to update such an application is to provide new images for its containers. Containers, in a production environment, can grow from few to many in no time. Kubernetes makes managing multiple containers an easy task. And lastly, resource usage monitoring-- resources such as CPU and RAM must be monitored within the Kubernetes environment. Kubernetes resource usage looks at the amount of resources that are utilized by a container or port within the Kubernetes environment. It is very important to keep an eye on the resource usage of the pods and containers as more usage translates to more cost. 16:18 Nikita: I think we can wind up our episode with that. Thank you, Mahendra, for joining us today. Kubernetes sure can be challenging to work with, but we covered a lot of ground in this episode.  Lois: That's right, Niki! If you want to learn more about the rich features Kubernetes offers, visit mylearn.oracle.com and search for the OCI Container Engine for Kubernetes Specialist course. Remember, all the training is free, so you can dive right in! Join us next week when we'll take a look at the fundamentals of Oracle Cloud Infrastructure Container Engine for Kubernetes. Until then, Lois Houston… Nikita: And Nikita Abraham, signing off! 16:57 That's all for this episode of the Oracle University Podcast. If you enjoyed listening, please click Subscribe to get all the latest episodes. We'd also love it if you would take a moment to rate and review us on your podcast app. See you again on the next episode of the Oracle University Podcast.

On this episode of The Cybersecurity Defenders Podcast, we talk API security with Jeremy Snyder, Founder and CEO at FireTail.io.FireTail.io is a pioneering company specializing in end-to-end API security. With APIs being the number one attack surface and a significant threat to data privacy and security, Jeremy and his team are at the forefront of protecting sensitive information in an increasingly interconnected world.Jeremy brings a wealth of experience in cloud, cybersecurity, and data domains, coupled with a strong background in M&A, international business, business development, strategy, and operations. Fluent in five languages and having lived in five different countries, he offers a unique global perspective on cybersecurity challenges and innovations.FireTail.io's data breach tracker.vacuum - The world's fastest OpenAPI & Swagger linter.Nuclei - Fast and customisable vulnerability scanner based on simple YAML based DSL.

#266: The tech industry has witnessed an explosion of new configuration languages that are reshaping the way we manage data structures and generate configuration files. From traditional formats like JSON and YAML to newer languages like Pkl, CUE, and KCL, the landscape of configuration languages is constantly evolving to meet the demands of modern software development. In this episode, Darin and Viktor talk about where these languages fit in to our projects and how they can help make our lives easier if we are willing to put in the work.   Today's sponsor: Save 25% on your first Barbaro Mojo order using the code DevOps25 https://barbaromojo.com/discount/DevOps25   Pkl https://pkl-lang.org/   CUE https://cuelang.org/   KCL https://www.kcl-lang.io/   YouTube channel: https://youtube.com/devopsparadox   Review the podcast on Apple Podcasts: https://www.devopsparadox.com/review-podcast/   Slack: https://www.devopsparadox.com/slack/   Connect with us at: https://www.devopsparadox.com/contact/  

Today we are talking about web design and development, from a group of people with one thing in common… We love Drupal. This is episode #452 Starshot & Experience Builder. For show notes visit: www.talkingDrupal.com/452 Topics What is Starshot What is Experience builder How will Starshot build on Drupal Core Will Experience builder be added to Core Listener thejimbirch: When will people hear about their pledge Listener brook_heaton: Will experience builder be compatible with layout builder Will Experience builder allow people to style content Listener Matthieu Scarset Who is Starshot trying to compete with Listener Andy Blum Does the DA or other major hosting companies plan to set up cheap, easy hosted Drupal Listener Ryan Szarma Who does this initiative serve in the business community How can people get involved Resources Drupal Starshot Experience Builder Guests Lauri Eskola - lauriii Hosts Nic Laflin - nLighteneddevelopment.com nicxvan John Picozzi - epam.com johnpicozzi Matthew Grasmick - grasmash MOTW Correspondent Martin Anderson-Clutz - mandclu.com mandclu Brief description: Have you ever wanted to have your modules create content when they're installed? There's a module for that. Module name/project name: Default Content Brief history How old: created in Oct 2015 by prolific contributor Lee Rowlands (larowlan) though the most recent releases are by Sascha Grossenbacher (Berdir), also a maintainer of many popular Drupal modules Versions available: 2.0.0-alpha2, which works with Drupal 9 and 10 Maintainership Security coverage: opted in, but needs a stable release Test coverage Documentation Number of open issues: 105 open issues, 29 of which are bugs against the current branch Usage stats: Almost 20,000 sites Module features and usage Provides a way for modules to include default content, in the same way that many modules already include default configuration The module exports content as YAML files, and your module can specify the content that should be exported by listing the UUIDs in the info.yml file It also provides a number of drush commands, to export a single entity, to export an entity and all of its dependencies, or to bulk export all of the content referenced in a module's .info.yml file There is also a companion project to export default content using an action within a view, which also makes me think it could probably be automated with something like ECA if you needed that Exported content should be kept in a content directory in your module, where it will imported during install on any site that has the default_content module installed I thought this would be a good module to cover today because Drupal core's recipe system also includes support for default content, so when you install a recipe it will similarly import any YAML-encoded content in the recipe. In fact, I used this module for the first time exporting taxonomy terms I wanted a recipe to create as default values for a taxonomy it creates. Since Recipes will be a big part of Starshot, I expect default_content to be getting a lot of use in the coming months

Wenn du glaubst, dass du YAML kennst …„YAML Ain't Markup Language“ (ursprünglich „Yet Another Markup Language“) kennen viele nur als Sprache für Konfigurationsdateien. Laut dem Gründer von YAML ist das Format aber nicht dafür gedacht. Und überhaupt nutzen sehr viele Tools nur einen Bruchteil der Fähigkeiten von YAML. Welche das sind, hat uns Tina Müller erklärt. Tina ist u.a. Contributorin zur YAML Spezifikation und gibt uns mal einen Einblick in das Serialisierungs-Format.Wir sprechen über darüber, welches Problem YAML lösen wollte, wie es in der Realität genutzt wird, wie YAML selbst sowie die YAML-Parser in verschiedenen Sprachen weiterentwickelt werden, über die Flaws von YAML, wie zB. das Norway Problem oder die Billion Laughs Attacke und schauen mal welche Features nicht so bekannt sind, wie YAML tags, aliases oder YAMLScript.**** Tue was Gutes mit deiner alten Hardware – Computertruhe.Du hast noch alte Computer-Hardware rumfliegen? Anstatt diese zu verschrotten oder für 20€ bei Kleinanzeigen zu verkaufen - Tue was gutes und Spende diese an die ComputertruheDie Computertruhe ist eine gemeinnützige Organisation, die Computer für bedürftige Menschen bereitstellt.Alle Infos unter https://computertruhe.de/********Das schnelle Feedback zur Episode:

This week, we discuss Kubecon EU, Nvidia's hyper growth, having 55 direct reports and the Worldwide Container Infrastructure Forecast. Plus, is “hello” a proper slack message? Watch the YouTube Live Recording of Episode (https://www.youtube.com/watch?v=b-SnxTaHhL4) 459 (https://www.youtube.com/watch?v=b-SnxTaHhL4) Runner-up Titles "Hey. Got a sec? Want to run something by you.” You don't want to scare you coworkers Eating bugs off your coworkers “Hi” has become a trigger word Rehabilitate the “Hi” 55 Direct Reports Everyone worked at one company, and that one company didn't want to do the work for everyone. The YAML hand off market Rundown Is Hello a proper Slack message? (https://twitter.com/adamhjk/status/1770411476022354075) Please Don't Say Just Hello In Chat (https://www.nohello.com/2013/01/please-dont-say-just-hello-in-chat.html) Kubecon “Designing for Success: UX Principles for Internal Developer Platforms,” (https://www.youtube.com/watch?v=6rqe5Yc13-A) "Boosting Developer Platform Teams with Product Thinking," (https://www.youtube.com/watch?v=Z_KCOcoliLI) “Sometimes, Lipstick Is Exactly What a Pig Needs!” (https://www.youtube.com/watch?v=VhloarnpxVo) OpenCost Introduces Carbon Costs (https://www.opencost.io/blog/carbon-costs) What if the CNCF was private equity? (https://www.thecloudcast.net/2024/03/what-if-cncf-was-private-equity.html) Nvidia Nvidia shares pop on Q4 earnings, generative AI "hits tipping point" (https://www.axios.com/2024/02/21/nvidia-nvda-earnings-q4-stock-price?utm_source=newsletter&utm_medium=email&utm_campaign=newsletter_axiosprorata&stream=top) NVIDIA CEO, Jensen Huang - has 55 direct reports (https://www.threads.net/@1393985902/post/C4dsKakP97W) Video that tracks the trajectories of Intel and NVIDIA (https://www.threads.net/@briansolis/post/C4lZBcKsdtC/?xmt=AQGzp25jpaWtHUqgBJaBCIJrv22Ag5Q0bPRX3Q6xRh_S4Q) Nvidia's latest AI chip will cost more than $30,000, CEO says (https://www.cnbc.com/2024/03/19/nvidias-blackwell-ai-chip-will-cost-more-than-30000-ceo-says.html) Worldwide Container Infrastructure Software Forecast, 2023–2027 (https://www.idc.com/getdoc.jsp?containerId=US49244823&pageType=PRINTFRIENDLY) Relevant to your Interests Leadership Is A Hell Of A Drug (https://ludic.mataroa.blog/blog/leadership-is-a-hell-of-a-drug/) Key OpenAI Executive Played a Pivotal Role in Sam Altman's Ouster (https://www.nytimes.com/2024/03/07/technology/openai-executives-role-in-sam-altman-ouster.html?smid=nytcore-ios-share&referringSource=articleShare) Who Still Works From Home? (https://www.nytimes.com/interactive/2024/03/08/business/economy/remote-work-home.html) A new TikTok ban gains steam (https://www.platformer.news/tik-tok-ban-bill-2024-bytedance-biden/?ref=platformer-newsletter) Sam Altman reinstated to OpenAI board after investigation clears him of wrongdoing (https://venturebeat.com/security/sam-altman-reinstated-to-openai-board-after-investigation-clears-him-of-wrongdoing/) More companies getting rid of free tiers: (https://x.com/planetscale/status/1765438197981708684?s=46&t=zgzybiDdIcGuQ_7WuoOX0A) 49% of founders say they're considering quitting their startup this year (https://sifted.eu/articles/founder-mental-health-2024) The WiFi at Google's new Bay View office hasn't been working properly for months: report (https://www.businessinsider.com/googles-swanky-new-bay-view-office-suffers-bad-wifi-2024-3) Moon Mission Could Redefine Computing in Deep Space (https://www.eetimes.com/data-centers-could-soon-break-lunar-ground/) Doctors Are Using the Apple Vision Pro During Surgery (https://gizmodo.com/doctors-are-using-the-apple-vision-pro-during-surgery-1851329884) Apple Buys Canadian AI Startup as It Races to Add Features (https://www.bloomberg.com/news/articles/2024-03-14/apple-aapl-buys-canadian-ai-startup-darwinai-as-part-of-race-to-add-features?utm_medium=email&utm_source=newsletter&utm_term=240314&utm_campaign=author_19842959&sref=9hGJlFio) Python with braces. Because python is awesome, but whitespace is awful. (https://github.com/mathialo/bython) Europe's AI Act demands extensive "logs" of users (https://www.thestack.technology/bias-biometrics-and-black-boxes-europes-ai-act-what-you-need-to-know/) How The Cloud Is A Trap (https://schedule.sxsw.com/2024/events/PP1144808) Amazon Web Services CEO Adam Selipsky says more than 10,000 organizations are using Bedrock (https://www.axios.com/2024/03/12/aws-ceo-ai-bedrock-amazon-anthropic) Measuring Developer Productivity via Humans (https://martinfowler.com/articles/measuring-developer-productivity-humans.html) Snowflake Stock: Melting Faster Than An Ice Cube (https://seekingalpha.com/article/4678674-snowflake-melting-faster-than-an-ice-cube-snow-stock) Games Are Coming to LinkedIn (https://www.pcmag.com/news/games-are-coming-to-linkedin) Dell Says Remote Employees Won't Be Eligible for Promotions: Report (https://gizmodo.com/dell-remote-employees-eligible-promotions-1851347699) European Cloud Group Calls for Regulatory Scrutiny Over Broadcom's VMware Overhaul (https://www.wsj.com/articles/european-cloud-group-calls-for-regulatory-scrutiny-over-broadcoms-vmware-overhaul-28b7c6ed?st=6n4vd93zeqr9d0o&reflink=article_email_share) Analogpunk, or, Tools, Shoes and Misery (https://schedule.sxsw.com/2024/events/PP1145788) Platform Engineering Day Europe 2024 (https://www.youtube.com/playlist?list=PLj6h78yzYM2Me-TpMQFvCphDu_xm71ed_) Redis Adopts Dual Source-Available Licensing (https://redis.com/blog/redis-adopts-dual-source-available-licensing/) Apple Is in Talks to Let Google Gemini Power iPhone AI Features (https://www.bloomberg.com/news/articles/2024-03-18/apple-in-talks-to-license-google-gemini-for-iphone-ios-18-generative-ai-tools) The MacBook Air gets an M3 upgrade (https://www.theverge.com/2024/3/4/24089999/apple-macbook-air-m3-announced-13-15-inch) Walmart sells a Mac (https://www.threads.net/@parkerortolani/post/C4iaGaFuKS8/?xmt=AQGzjqrbQ8qCsg4UUGYIc8LbOh2c9MoMdzn7sXSwOehXkA) Apple Plans AirPods Overhaul With New Low- and High-End Models, USB-C Headphones (https://www.bloomberg.com/news/articles/2023-10-25/apple-airpods-plans-4th-generation-low-end-3rd-generation-pro-and-usb-c-max) AWS follows Google in announcing unrestricted free data transfers to other cloud providers (https://techcrunch.com/2024/03/05/amazon-follows-google-in-announcing-free-data-transfers-out-of-aws/) Free data transfer out to internet when moving out of AWS | Amazon Web Services (https://aws.amazon.com/blogs/aws/free-data-transfer-out-to-internet-when-moving-out-of-aws/) Buyout Firm Vista Equity Explores Options Including Sale for LogicMonitor (https://www.bloomberg.com/news/articles/2024-03-13/buyout-firm-vista-equity-explores-options-including-sale-for-logicmonitor) Nonsense Airlines Are Coming for Your Carry-Ons (https://www.wsj.com/lifestyle/travel/flights-carry-on-bags-personal-items-3bcd3c2c?st=nx8npa3s7g8tm7f&reflink=article_copyURL_share) Clocks Change (https://www.youtube.com/watch?v=k4EUTMPuvHo) Costco CFO ‘voice' looks back on 40 years, $1.50 hot dogs and leadership (https://www.cfodive.com/news/costco-cfo-voice-40-years-150-hot-dogs-Richard-Galanti/709622/) Star Wars: Millennium Falcon 50p coin unveiled by Royal Mint (https://www.bbc.com/news/uk-wales-68594916) Delta's CEO says controversial Sky Lounge changes reflect the airline's status as premium brand (https://www.fastcompany.com/91060105/deltas-ceo-controversial-sky-lounge-changes-airlines-status-premium-brand) 3D Printed Full-Size Macintosh - The Brewintosh (https://www.youtube.com/watch?v=7N9oz4Ylzm4) Formula 1 chief appalled to find team using Excel to manage 20,000 car parts (https://arstechnica.com/cars/2024/03/formula-1-chief-appalled-to-find-team-using-excel-to-manage-20000-car-parts/) Listener Feedback Chris tell us the Owala Water Bottle is on sale. (https://a.co/d/30B4wA1) Conferences Tanzu (Re)defined online (https://www.youtube.com/watch?v=vDvWDyd98hA), April 3rd, Coté Speaking. Tanzu (Re)defined (https://www.fig-street.com/041124-tanzu-redefined/?utm_source=cote&utm_campaign=devrel&utm_medium=newsletter&utm_content=newsletterUpcoming), April 11th, Coté speaking, Palo Alto. TEQNation (https://conference.teqnation.com), May 22nd, 2024, Utrecht, Coté speaking. NDC Oslo (https://substack.com/redirect/8de3819c-db2b-47c8-bd7a-f0a40103de9e?j=eyJ1IjoiMmQ0byJ9.QKaKsDzwnXK5ipYhX0mLOvRP3vpk_3o2b5dd3FXmAkw), Coté speaking (https://substack.com/redirect/41e821af-36ba-4dbb-993c-20755d5f040a?j=eyJ1IjoiMmQ0byJ9.QKaKsDzwnXK5ipYhX0mLOvRP3vpk_3o2b5dd3FXmAkw), June 12th. DevOpsDays Amsterdam (https://devopsdays.org/events/2024-amsterdam/welcome/), June 19 to 21, 2024, Coté speaking. DevOpsDays Birmingham, August 19–21, 2024 (https://devopsdays.org/events/2024-birmingham-al/welcome/). Open Source Summit North America (https://events.linuxfoundation.org/open-source-summit-north-america/), Seattle April 16-18. Matt's speaking SDT news & hype Join us in Slack (http://www.softwaredefinedtalk.com/slack). Get a SDT Sticker! Send your postal address to stickers@softwaredefinedtalk.com (mailto:stickers@softwaredefinedtalk.com) and we will send you free laptop stickers! Follow us: Twitch (https://www.twitch.tv/sdtpodcast), Twitter (https://twitter.com/softwaredeftalk), Instagram (https://www.instagram.com/softwaredefinedtalk/), Mastodon (https://hachyderm.io/@softwaredefinedtalk), BlueSky (https://bsky.app/profile/softwaredefinedtalk.com), LinkedIn (https://www.linkedin.com/company/software-defined-talk/), TikTok (https://www.tiktok.com/@softwaredefinedtalk), Threads (https://www.threads.net/@softwaredefinedtalk) and YouTube (https://www.youtube.com/channel/UCi3OJPV6h9tp-hbsGBLGsDQ/featured). Use the code SDT to get $20 off Coté's book, Digital WTF (https://leanpub.com/digitalwtf/c/sdt), so $5 total. Become a sponsor of Software Defined Talk (https://www.softwaredefinedtalk.com/ads)! Recommendations Brandon: The E-Gates Modality Will Now Be Implemented at Cancun Airport - Cancun Airport (https://www.cancuniairport.com/the-e-gates-modality-will-now-be-implemented-at-cancun-airport/) Global Entry better then ever (https://globalfinder-usonline.com/glofinderus/?utm_term=global%20entry&utm_campaign=5C1B-2023-04-07&gad_source=1&gclid=CjwKCAjwte-vBhBFEiwAQSv_xZTL8mD-XjuwoT_Kqr6-YHaCUiyCITM5HugRhsRNLqm_50l3sSIJZxoC-jsQAvD_BwE) Mobile Passport Control (MPC) (https://www.cbp.gov/travel/us-citizens/mobile-passport-control) available in Austin Airport Matt: Duck Duck Go (https://duckduckgo.com) Coté: MacBook Pro (https://www.apple.com/shop/buy-mac/macbook-pro/14-inch-m3-max) Photo Credits Header (https://unsplash.com/s/photos/Hello) Artwork (https://unsplash.com/s/photos/Matrix-math)

Cet épisode discute du retour d'experience Java de Netflix, de jQuery, de gouvernance open source, d'Elon Musk, de Kubernetes, de Mistral (gagnant?), d'attaque des LLMs, de developpement de carrière et de Trouble du Déficit de l'Attention avec ou sans Hyperactivité. Enregistré le 15 mars 2024 Téléchargement de l'épisode LesCastCodeurs-Episode-308.mp3 News Les cast codeurs veulent essayer quelque chose de nouveau et le sondage montre que vous aussi. On lance donc une section Ask Me Anything, posez nous une question sur https://lescastcodeurs.com/ama et nous prendrons certaines questions pour donner notre réponse. Lancez-vous, on pense que cette section pourra être sympa :) Langages Retour d'experience de Netflix sur ZGC https://netflixtechblog.com/bending-pause-times-to-your-will-with-generational-zgc-256629c9386b moins de tail latency ce qui veut dire moins de charge sur le systeme (moins de retry) aussi plus facile de trouver les vrais probleme de latence (plus cachés dans les latences de GC) et sans consommation superieure de CPU pour les memes perfs malgré les barrieres differentes de ZGC pas de tuning explicit de leur part (enfin presque) meme si les pointeurs ne sont pas compresses, l'efficacite du GC compense Librairies Sortie de Spock 2.4-M2 https://spockframework.org/spock/docs/2.4-M2/release_notes.html Support de plusieurs librairies de mocking Meilleur support dans les IDEs Et plein d'autres petites améliorations jQuery 4 est sorti ! jQuery est de retour ! https://www.infoq.com/news/2024/03/jquery-4-beta-release-note/ On parle régulièrement du dernier framework JavaScript à la mode, mais jQuery est toujours là Première release majeure depuis 8 ans Suppression de plein de features qui étaient deprecated et maintenant fournie souvent par défaut par les moteurs JavaScript des navigateurs jQuery continue d'être téléchargé de plus en plus au fil du temps, mais peut-être parce qu'il bénéficie du succès des projets qui l'utilisent comme Cypress, WordPress ou Drupal) Quarkus sort sa deuxieme LTS https://quarkus.io/blog/quarkus-3-8-released/ explique les changements importants depuis la LTS 3.2 Infrastructure Linkerd ou plutôt la boîte derrière va faire payer pour accéder aux builds stable du projet. Cela crée des conversations au sein de la CNCF https://www.techtarget.com/searchitoperations/news/366571035/Linkerd-paywall-prompts-online-debate-CNCF-TOC-review deploy envoy, c'est plus dur Buyoant est le principal contributeur derriere Linkerd et ils ont edcider de mettre les distributions stables derriere un paywall pour les societes de plus de 50 employés ($2000 par cluster) les gens se trouve floués par aider au succces et ensuite de trouver piégé La license reste ASL mais la version stable est derriere un paywall, comme red hat enterprise linuix recemment un autre exemple de projet open source qui vire commercial questionne la gouvernance open source, la CNCF va inestiguer et peut etre durcir ces criteres de graduiation Weavework (FLux) a fermé ces dernieres semaines aussi Cloudflare a reecrit un proxy HTTP en rust https://blog.cloudflare.com/how-we-built-pingora-the-proxy-that-connects-cloudflare-to-the-internet/ ils ont utilise NGinx pendant longtemps mais le single worker modele ne permetait pas ceratins optims et ils ont des besolins specifiques bref ils ont reecrit en rust, multi threaded et avec work stealing et ils sont content Le guide du “hater” sur Kubernetes https://paulbutler.org/2024/the-haters-guide-to-kubernetes/ L'auteur se plaint régulièrement de Kubernetes pour sa grande complexité mais reconnait que c'est quand même un grand morceau de technologie A utiliser surtout quand on a besoin de : Exécuter plusieurs processus/serveurs/tâches planifiées. Les exécuter de manière redondante et répartir la charge entre eux. Les configurer, ainsi que les relations entre eux, sous forme de code. L'auteur liste ensuite les fonctionnalités qu'il utilise, qu'il fait attention quand il les utilise, et celles qu'il préfère éviter Utilise : deployments, services, cron jobs, config maps et secrets Attention : stateful set, persistent volume et RBAC Evite : le YAML à la main, les opérateurs et resources customs, Helm, tout ce qui est mesh, les resources ingress, essayer de répliquer la stack K8S complète localement sur sa machine Data et Intelligence Artificielle Mistral AI et Microsoft font un accord sur le modele le plus puissant de Mistral et certains ne sont pas content https://www.latribune.fr/technos-medias/informatique/l-alliance-entre-mistral-et-micr[…]usion-de-l-independance-technologique-europeenne-991558.html Mistral avancait son approche open source mais son modele le plus puissant ne l'est pas ils ont un partenariat exclusif avec Microsoft pour le distribuer Et MS rentre dans le capital Au revoir l'independance de l'IA européenne Au revoir les modeles open source larges cela va a l'encontre du loby et de son positinnement aupres de la commission europeenne ca fait grincer des dents a bruxelles qui avait alléger les contraintes sur les modeles fondamentaux a la demande de Mistral qui menacait de de voir s'allier avec MS si ce n'était pas le cas. Mistral était un fer de lance des modeles open sources pour eviter les biais ils en garderont masi pas les modeles specialisés ou optimisés cela reste une bonne decisione conomique pour Mistral Infinispan 15 est sorti https://infinispan.org/blog/2024/03/13/infinispan-15 JDK 17 Redis Hot Replacement donnant: multi thread, clustering, replication cross site, diff stores de persistence en disk, avoir des caches differentes en namespace différentes avec des règles appliquées à chaque cas d'usage Recherche Vectorielle et stockage des embeddings Integration avec Langchain (Python), Langchain4j, et Quarkus Langchain Améliorations du search, replication cross site, la console, tracing, l'Operateur Kubernetes … Support du Protobuf 3 avec la release de Protostream 5 et meilleur API Outillage Ne pas signer ses commits cryptographiquement ? https://blog.glyph.im/2024/01/unsigned-commits.html L'article cite comme seul avantage d'avoir le petit badge vert sur Github indiquant “vérifié” Responsabilité future inconnue et potentiellement illimitée pour les conséquences de l'exécution du code dans un commit que vous avez signé. Renforcement implicite de GitHub en tant qu'autorité de confiance centralisée dans le monde de l'open source. Introduction de problèmes de fiabilité inconnus dans l'infrastructure qui repose sur les signatures de commit. Une violation temporaire de vos identifiants GitHub entraîne désormais des conséquences potentiellement permanentes si quelqu'un parvient à y introduire une nouvelle clé de confiance. Nouveau type de surcharge de processus continu : les clés de signature de commit deviennent une nouvelle infrastructure permanente à gérer, avec de nouvelles questions comme « que faire des clés expirées », « à quelle fréquence dois-je les renouveler », etc. on peut empecher de pousser des commits non signés Sécurité Des modèles avec des backdoors uploadés sur hugging faces non détecté. https://arstechnica.com/security/2024/03/hugging-face-the-github-of-ai-hosted-code-that-backdoored-user-devices/ par les chercheurs de JFrog Une centaine détectés dont 10 malicieux Des tests de chercheurs mais un faisant un reverse ssh S'appuye sur le format de serialisation pickle en python. Populaire mais connu comme dangereux Une première side attack channel sur les LLMs https://arstechnica.com/security/2024/03/hackers-can-read-private-ai-assistant-chats-even-though-theyre-encrypted/ cela s'appuie sur la taille des packets chiffrés envoyés et leur timing pour détecter la longueur des tokens Ensuite un LLM spécialisé reconstruit la suite de mots la plus probable vu la longueur C'est du à l'UX qui envoie les tokens au fil de l'eau Ć'est facilement corrigeable en rendant les paquets de taille fixe et en ajoutant du hasard de délai d'envoie. Mais c'est rigolo comment les LLMs peuvent amplifier les side channel attacks Architecture Netflix et Java https://www.infoq.com/presentations/netflix-java/ Netflix est un java shop La “stack NEtflix” connue du public a beaucoup evolué Pleins de microservices Gen1: groovy en gateway front end for backend , RxJava et Histrix Gen2: GraphQL et GraphQL federé ; plus de reactif sand dans la gateway Java 17 : 2800 apps java utilisent Azul JDK avait du Java 8 sur du guice et app custom utilisent G1, Java 17 = -20% CPU et Shenandoah pour la gateway Zuul Plans pour Java 21 (ZGC, virtual threads) apres speculatif Ils ont standardisé sur Spring Boot il n'y a pas si longtemps Un long article sur les microservices https://mcorbin.fr/posts/2024-02-12-microservice/ encore un me direz vous oui amis si vous etre en pleine crise existentielle avec votre equipe c'est du mon materiel il va sur les points importants comme synchrone vs asynchrone, les patterns de communication, la copie de données, comment tester le “monotithe” ou plutot comment ne pas le faire etc c'est un peu long mais ca recadre bien Méthodologies Opinion: est-ce qu'on peut devenir dev à partir de 40 ans https://www.codemotion.com/magazine/dev-life/can-you-become-a-programmer-after-40/?utm_source=ActiveCampaign&utm_medium=email&utm_content=5+Frontend+Trends+we+Didn+t+See+Coming+in+2024&utm_campaign=NL_EN_240215+%28Copy%29&vgo_ee=sFCRn4bbw8NuvJwlxj4PgXiVS4eICnA1ZPdkH4DGKyhNNwh6NQ%3D%3D%3Au3g96%2Fz3Uf7kZHAF7tezy9Y0ZJ6paAsE programmeur de CSS a 40 ans, je sais pas :stuck_out_tongue_winking_eye: l'auteur regrette les pubs pour devenir ev a 40 ansd facilement developpeur c'est beaucoup de connaissance et de travail et doit etre un choix, pas un choix pas default ou facile ils decrit certains biais comme un 20 ans sans experience est plus pardonné qu'à 40, le temps a y consacré est différent etc compensé par des actes de motivation (GitHub, participation open source, meetups etc) mais le temps d'apprendre de ces erreurs n'as pas vraiment de court circuit bref une fonrmation c'est bien mais aps suffisant Navigate your own way https://www.infoq.com/presentations/lessons-opportunities-carrier/ IBMer for 21 years. I'm a Java champion Réfléchir à sa carrière en mode time box, chercher sa promotion ? Tu peux décider ton chemin Momentum pandémie ça lui a fait bcp réfléchir sur sa vie et où elle était. Moment où elle quittait IBM pour aller vers RH (cœur se brise) Essentiels pour prendre ton propre chemin Se connaître soit même, reconnaître les différences avec les autres connaître tes valeurs: c'est quoi important pour toi, c'est quoi qui te motive, ce qui te démotive. Écrire des mots Se fixer des objectifs avec l'aide des autres Repusher ses limites, sur des sujets dont tu penses que ce n'est pas possible pour toi Participe activement, entoure toi bien Un talk très personnel et inspirant Un article sur le trouble TDAH chez le développeur adulte https://rlemaitre.com/fr/posts/2023/11/hacker-le-tdah-strat%C3%A9gies-pour-le-d%C3%A9veloppeur-moderne/ Diagnostiqué à 44 ans Schéma d'inattention et d'hyperactivite/implusivite qui interfere avec le fonctionnement Affecte le fonctionnement social scolaire ou professionnel Non diagnistiqué: burn out anxiété ou depression Souvent non diagnostiqué jusqu'à ce que se propres enfants soient diagnostiqués Mais cela amène du positif: hyperfocus, resolution creative de problèmes, adaptation rapide aux changements qui sont du pain béni Le négatif c'est la gestion du temps, organisation, instabilité Discute ensuite les phénomènes dans le cerveau Et donne des techniques et des pièges à éviter Vous avez sûrement des collègues TDAH ou l'êtes-vous meme Conférences La liste des conférences provenant de Developers Conferences Agenda/List par Aurélie Vache et contributeurs : 14-15 mars 2024 : pgDayParis - Paris (France) 17-18 mars 2024 : Cloud Native Rejekts EU 2024 - Paris (France) 19 mars 2024 : AppDeveloperCon - Paris (France) 19 mars 2024 : ArgoCon - Paris (France) 19 mars 2024 : BackstageCon - Paris (France) 19 mars 2024 : Cilium + eBPF Day - Paris (France) 19 mars 2024 : Cloud Native AI Day Europe - Paris (France) 19 mars 2024 : Cloud Native StartupFest Europe - Paris (France) 19 mars 2024 : Cloud Native Wasm Day Europe - Paris (France) 19 mars 2024 : Data on Kubernetes Day - Paris (France) 19 mars 2024 : Istio Day Europe - Paris (France) 19 mars 2024 : Kubeflow Summit Europe - Paris (France) 19 mars 2024 : Kubernetes on Edge Day Europe - Paris (France) 19 mars 2024 : Multi-Tenancy Con - Paris (France) 19 mars 2024 : Observabiity Day Europe - Paris (France) 19 mars 2024 : OpenTofu Day Europe - Paris (France) 19 mars 2024 : Platform Engineering Day - Paris (France) 19 mars 2024 : ThanosCon Europe - Paris (France) 19 mars 2024 : PaaS Forward by OVHcloud | Rancher by SUSE - Paris (France) 19-21 mars 2024 : CloudNativeHacks - Paris (France) 19-21 mars 2024 : IT & Cybersecurity Meetings - Paris (France) 19-22 mars 2024 : KubeCon + CloudNativeCon Europe 2024 - Paris (France) 21 mars 2024 : IA & Data Day Strasbourg - Strasbourg (France) 22-23 mars 2024 : Agile Games France - Valence (France) 26-28 mars 2024 : Forum INCYBER Europe - Lille (France) 27 mars 2024 : La Conf Data | IA - Paris (France) 28-29 mars 2024 : SymfonyLive Paris 2024 - Paris (France) 28-30 mars 2024 : DrupalCamp Roazhon - Rennes (France) 4 avril 2024 : SoCraTes Rennes 2024 - Rennes (France) 4-6 avril 2024 : Toulouse Hacking Convention - Toulouse (France) 8 avril 2024 : Lyon Craft - Lyon (France) 9 avril 2024 : Unconf HackYourJob - Lyon (France) 11 avril 2024 : CI/CDay - Paris (France) 17-19 avril 2024 : Devoxx France - Paris (France) 18-20 avril 2024 : Devoxx Greece - Athens (Greece) 22 avril 2024 : React Connection 2024 - Paris (France) 23 avril 2024 : React Native Connection 2024 - Paris (France) 25-26 avril 2024 : MiXiT - Lyon (France) 25-26 avril 2024 : Android Makers - Paris (France) 3-4 mai 2024 : Faiseuses Du Web 3 - Dinan (France) 8-10 mai 2024 : Devoxx UK - London (UK) 16-17 mai 2024 : Newcrafts Paris - Paris (France) 22-25 mai 2024 : Viva Tech - Paris (France) 24 mai 2024 : AFUP Day Nancy - Nancy (France) 24 mai 2024 : AFUP Day Poitiers - Poitiers (France) 24 mai 2024 : AFUP Day Lille - Lille (France) 24 mai 2024 : AFUP Day Lyon - Lyon (France) 28-29 mai 2024 : Symfony Live Paris - Paris (France) 1 juin 2024 : PolyCloud - Montpellier (France) 6-7 juin 2024 : DevFest Lille - Lille (France) 6-7 juin 2024 : Alpes Craft - Grenoble (France) 7 juin 2024 : Fork it! Community - Rouen (France) 11-12 juin 2024 : OW2con - Paris (France) 12-14 juin 2024 : Rencontres R - Vannes (France) 13-14 juin 2024 : Agile Tour Toulouse - Toulouse (France) 14 juin 2024 : DevQuest - Niort (France) 18 juin 2024 : Tech & Wine 2024 - Lyon (France) 19-20 juin 2024 : AI_dev: Open Source GenAI & ML Summit Europe - Paris (France) 19-21 juin 2024 : Devoxx Poland - Krakow (Poland) 27 juin 2024 : DotJS - Paris (France) 27-28 juin 2024 : Agi Lille - Lille (France) 4-5 juillet 2024 : Sunny Tech - Montpellier (France) 8-10 juillet 2024 : Riviera DEV - Sophia Antipolis (France) 6 septembre 2024 : JUG Summer Camp - La Rochelle (France) 19-20 septembre 2024 : API Platform Conference - Lille (France) & Online 2-4 octobre 2024 : Devoxx Morocco - Marrakech (Morocco) 7-11 octobre 2024 : Devoxx Belgium - Antwerp (Belgium) 10 octobre 2024 : Cloud Nord - Lille (France) 10-11 octobre 2024 : Volcamp - Clermont-Ferrand (France) 10-11 octobre 2024 : Forum PHP - Marne-la-Vallée (France) 16 octobre 2024 : DotPy - Paris (France) 17-18 octobre 2024 : DevFest Nantes - Nantes (France) 17-18 octobre 2024 : DotAI - Paris (France) 6 novembre 2024 : Master Dev De France - Paris (France) 7 novembre 2024 : DevFest Toulouse - Toulouse (France) 8 novembre 2024 : BDX I/O - Bordeaux (France) Nous contacter Pour réagir à cet épisode, venez discuter sur le groupe Google https://groups.google.com/group/lescastcodeurs Contactez-nous via twitter https://twitter.com/lescastcodeurs Faire un crowdcast ou une crowdquestion Soutenez Les Cast Codeurs sur Patreon https://www.patreon.com/LesCastCodeurs Tous les épisodes et toutes les infos sur https://lescastcodeurs.com/

In Episode 116 of the XrmToolCast, Daryl and Scott dive into Power Apps Tools with guests Leonardo and Rudimar. They explore the community-driven "clipboard" packed with YAML snippets, SVG-to-FX converters, Power Fx functions, icons, and more—all aimed at boosting canvas app development. Some of the highlights: Capybaras Backstory of Power Apps Tools When your tools become your weekend projects Power of open source software Earliest show hzomework ever assigned Scott wants to be made into a Capybara Daryl discovers Electric vehicles Leonardo's Info: LinkedIn: https://www.linkedin.com/in/leonardorrusso YouTube: https://www.youtube.com/@Leo_Russo Rudimar's Info: LinkedIn: https://www.linkedin.com/in/rudimar YouTube: https://www.youtube.com/@powerrudy Links: Power Apps Tools: https://powerappstools.fly.dev Capybara: https://en.wikipedia.org/wiki/Capybara Got questions? Have your own tool you'd like to share? Have a suggestion for a future episode, or like a shout-out? Contact Daryl and Scott at cast@xrmtoolbox.com. Follow us on LinkedIn and @XrmToolCast for updates on future episodes. Do you want to see us too? Subscribe to our YouTube channel to view the last episodes. Don't forget to rate and leave a review for this show at Podchaser. Your hosts: Daryl LaBar: https://www.linkedin.com/in/daryllabar | @ddlabar  Scott Durow: https://www.linkedin.com/in/scottdurow | @ScottDurow Editor: Linn Zaw Win: https://www.linkedin.com/in/linnzawwin  | @LinnZawWin Music: https://www.purple-planet.com

This week, we discuss whether or not Kubernetes is boring, Winglang's attempt to simply cloud deployments and Linkerd status as a graduated CNCF project. Plus, a few thoughts on frogs… Watch the YouTube Live Recording of Episode (https://www.youtube.com/watch?v=rXssLonmkEw) 457 (https://www.youtube.com/watch?v=rXssLonmkEw) Runner-up Titles Blame the seagulls Speaking of lizards in our houses Burying people under B-trees Compiler for the Cloud I'm tired of speaking French We need a big pie If we stop shooting each other we can sit down and eat some pie. The Jacob Principle Has there every been a config file format love affair? Rundown Kubernetes Predictions Were Wrong (https://thenewstack.io/kubernetes-predictions-were-wrong/) Wing Programming Language for the Cloud (https://www.winglang.io/) #1262 Health of Linkerd project (https://github.com/cncf/toc/issues/1262) Craig Box X Thread (https://twitter.com/craigbox/status/1760370351828320539) Adam Jacob's X Thread (https://twitter.com/adamhjk/status/1761051900215275760) Five Cloud News Trends for February (https://www.thecloudcast.net/2024/03/five-cloud-news-trends-for-february.html) Relevant to your Interests X adds support for passkeys on iOS after removing SMS 2FA support last year (https://techcrunch.com/2024/01/23/x-adds-support-for-passkeys-on-ios-after-removing-sms-2fa-support-last-year/?guccounter=1&guce_referrer=aHR0cHM6Ly93d3cuZ29vZ2xlLmNvbS8&guce_referrer_sig=AQAAABw_QgA37nl432nhb6Gn3krzLFMb5OvmaZr6thvov5iGJ8UrvQec-jpGjenN-KUP-cSFa4MRW9DXVXJ3_u87Z0-zfC2mR708qqm34sAhBT-mAcL5pP8L04T54Mqn-xnUCDNXYBJFr2Y4oHXFjkIWgTU_iwJ4vqK52MC4hYtaAC9W) CACM Is Now Open Access (https://cacm.acm.org/news/cacm-is-now-open-access-2/) Now Apple says it won't disable iPhone web apps in the EU (https://www.theverge.com/2024/3/1/24087666/apple-disable-iphone-web-apps-eu-reversal) A Few Jelly Beans and a World of Disappointment at Willy Wonka Event (https://www.nytimes.com/2024/02/27/world/europe/willy-wonka-experience-glasgow.html) Elon Musk sues OpenAI and Sam Altman over 'betrayal' of nonprofit AI mission | TechCrunch (https://techcrunch.com/2024/03/01/elon-musk-openai-sam-altman-court/) Meta says it's deleting all Oculus accounts at the end of the month (https://www.theverge.com/2024/3/1/24087855/meta-delete-oculus-accounts) Snowflake CEO Steps Down From Post Richer Than Tim Cook or Satya Nadella (https://www.bloomberg.com/news/articles/2024-03-04/snowflake-ceo-frank-slootman-steps-aside-richer-than-tim-cook-or-satya-nadella) RISC-V launch (https://www.scaleway.com/en/news/scaleway-launches-its-risc-v-servers-in-the-cloud-a-world-first-and-a-firm-commitment-to-technological-independence/) Amazon goes nuclear, acquires atomic datacenter for $650M (https://www.theregister.com/2024/03/04/amazon_acquires_cumulus_nuclear_datacenter/) Red Sea cables have been damaged, disrupting internet traffic (https://www.cnn.com/2024/03/04/business/red-sea-cables-cut-internet/index.html) It's Time to Give Up on Email (https://www.theatlantic.com/technology/archive/2024/03/email-nightmare-just-give-up/677615/) Nonsense When a funeral is clickbait (https://www.theverge.com/24065145/ai-obituary-spam-generative-clickbait) Conferences SCaLE 21x/DevOpsDays LA, March 14th (https://www.socallinuxexpo.org/scale/21x)– (https://www.socallinuxexpo.org/scale/21x)17th, 2024 (https://www.socallinuxexpo.org/scale/21x) — Coté speaking (https://www.socallinuxexpo.org/scale/21x/presentations/we-fear-change), sponsorship slots available. KubeCon EU Paris, March 19 (https://events.linuxfoundation.org/kubecon-cloudnativecon-europe/)– (https://events.linuxfoundation.org/kubecon-cloudnativecon-europe/)22 (https://events.linuxfoundation.org/kubecon-cloudnativecon-europe/) — Coté on the wait list for the platform side conference. Get 20% off with the discount code KCEU24VMWBC20. DevOpsDays Birmingham, April 17–18, 2024 (https://talks.devopsdays.org/devopsdays-birmingham-al-2024/cfp) Exe (https://ismg.events/roundtable-event/dallas-robust-security-java-applications/?utm_source=cote&utm_campaign=devrel&utm_medium=newsletter&utm_content=newsletterUpcoming)cutive dinner in Dallas that Coté's hosting on March 13st, 2024 (https://ismg.events/roundtable-event/dallas-robust-security-java-applications/?utm_source=cote&utm_campaign=devrel&utm_medium=newsletter&utm_content=newsletterUpcoming). If you're an “executive” who might want to buy stuff from Tanzu to get better at your apps, than register. There is also a Tanzu exec event coming up in the next few months, email Coté (mailto:cote@broadcom.com) if you want to hear more about it. Tanzu (Re)defined (https://www.fig-street.com/041124-tanzu-redefined/?utm_source=cote&utm_campaign=devrel&utm_medium=newsletter&utm_content=newsletterUpcoming), April 11th, Palo Alto. SDT news & hype Join us in Slack (http://www.softwaredefinedtalk.com/slack). Get a SDT Sticker! Send your postal address to stickers@softwaredefinedtalk.com (mailto:stickers@softwaredefinedtalk.com) and we will send you free laptop stickers! Follow us: Twitch (https://www.twitch.tv/sdtpodcast), Twitter (https://twitter.com/softwaredeftalk), Instagram (https://www.instagram.com/softwaredefinedtalk/), Mastodon (https://hachyderm.io/@softwaredefinedtalk), BlueSky (https://bsky.app/profile/softwaredefinedtalk.com), LinkedIn (https://www.linkedin.com/company/software-defined-talk/), TikTok (https://www.tiktok.com/@softwaredefinedtalk), Threads (https://www.threads.net/@softwaredefinedtalk) and YouTube (https://www.youtube.com/channel/UCi3OJPV6h9tp-hbsGBLGsDQ/featured). Use the code SDT to get $20 off Coté's book, Digital WTF (https://leanpub.com/digitalwtf/c/sdt), so $5 total. Become a sponsor of Software Defined Talk (https://www.softwaredefinedtalk.com/ads)! Recommendations Brandon: Dune Part 2 (https://www.rottentomatoes.com/m/dune_part_two) Matt: Our Band Could Be Your Life (https://amzn.to/49G2Ulg) Conan O'Brien in Australia (https://conanclassic.com/australia/conan-becomes-a-bondi-beach-lifeguard) Coté: Bruce Sterlings new-ish blog (https://toshareproject.it/artmakerblog/), especially the color scheme. Photo Credits Header (https://unsplash.com/photos/shallow-focus-photography-of-brown-frog-hHDMQqP4jPU) Gemini

This week we're joined by Herrington Darkholme, the creator of AST Grep. AST Grep is a code search tool that uses the abstract syntax tree (AST) of your code to find patterns. We talk about the genesis of AST Grep, the efficiency of AST Grep in code searching, the challenge of expressing complex patterns, the versatility of YAML for rule expression, testing and evolving rules with AST Grep, and expanding AST Grep with SDKs and VS Code integration. You should definitely check out AST Grep if you're looking for a powerful code search tool! Episode sponsored By CodeCrafters (https://codecrafters.io/devtoolsfm) 40% Discount!Episode sponsored By RunMe (https://runme.dev) Become a paid subscriber our patreon, spotify, or apple podcasts for the full episode. https://www.patreon.com/devtoolsfm https://podcasters.spotify.com/pod/show/devtoolsfm/subscribe https://podcasts.apple.com/us/podcast/devtools-fm/id1566647758 https://www.youtube.com/@devtoolsfm/membership

We're writing this one day after the monster release of OpenAI's Sora and Gemini 1.5. We covered this on ‘s ThursdAI space, so head over there for our takes.IRL: We're ONE WEEK away from Latent Space: Final Frontiers, the second edition and anniversary of our first ever Latent Space event! Also: join us on June 25-27 for the biggest AI Engineer conference of the year!Online: All three Discord clubs are thriving. Join us every Wednesday/Friday!Almost 12 years ago, while working at Spotify, Erik Bernhardsson built one of the first open source vector databases, Annoy, based on ANN search. He also built Luigi, one of the predecessors to Airflow, which helps data teams orchestrate and execute data-intensive and long-running jobs. Surprisingly, he didn't start yet another vector database company, but instead in 2021 founded Modal, the “high-performance cloud for developers”. In 2022 they opened doors to developers after their seed round, and in 2023 announced their GA with a $16m Series A.More importantly, they have won fans among both household names like Ramp, Scale AI, Substack, and Cohere, and newer startups like (upcoming guest!) Suno.ai and individual hackers (Modal was the top tool of choice in the Vercel AI Accelerator):We've covered the nuances of GPU workloads, and how we need new developer tooling and runtimes for them (see our episodes with Chris Lattner of Modular and George Hotz of tiny to start). In this episode, we run through the major limitations of the actual infrastructure behind the clouds that run these models, and how Erik envisions the “postmodern data stack”. In his 2021 blog post “Software infrastructure 2.0: a wishlist”, Erik had “Truly serverless” as one of his points:* The word cluster is an anachronism to an end-user in the cloud! I'm already running things in the cloud where there's elastic resources available at any time. Why do I have to think about the underlying pool of resources? Just maintain it for me.* I don't ever want to provision anything in advance of load.* I don't want to pay for idle resources. Just let me pay for whatever resources I'm actually using.* Serverless doesn't mean it's a burstable VM that saves its instance state to disk during periods of idle.Swyx called this Self Provisioning Runtimes back in the day. Modal doesn't put you in YAML hell, preferring to colocate infra provisioning right next to the code that utilizes it, so you can just add GPU (and disk, and retries…):After 3 years, we finally have a big market push for this: running inference on generative models is going to be the killer app for serverless, for a few reasons:* AI models are stateless: even in conversational interfaces, each message generation is a fully-contained request to the LLM. There's no knowledge that is stored in the model itself between messages, which means that tear down / spin up of resources doesn't create any headaches with maintaining state.* Token-based pricing is better aligned with serverless infrastructure than fixed monthly costs of traditional software.* GPU scarcity makes it really expensive to have reserved instances that are available to you 24/7. It's much more convenient to build with a serverless-like infrastructure.In the episode we covered a lot more topics like maximizing GPU utilization, why Oracle Cloud rocks, and how Erik has never owned a TV in his life. Enjoy!Show Notes* Modal* ErikBot* Erik's Blog* Software Infra 2.0 Wishlist* Luigi* Annoy* Hetzner* CoreWeave* Cloudflare FaaS* Poolside AI* Modular Inference EngineChapters* [00:00:00] Introductions* [00:02:00] Erik's OSS work at Spotify: Annoy and Luigi* [00:06:22] Starting Modal* [00:07:54] Vision for a "postmodern data stack"* [00:10:43] Solving container cold start problems* [00:12:57] Designing Modal's Python SDK* [00:15:18] Self-Revisioning Runtime* [00:19:14] Truly Serverless Infrastructure* [00:20:52] Beyond model inference* [00:22:09] Tricks to maximize GPU utilization* [00:26:27] Differences in AI and data science workloads* [00:28:08] Modal vs Replicate vs Modular and lessons from Heroku's "graduation problem"* [00:34:12] Creating Erik's clone "ErikBot"* [00:37:43] Enabling massive parallelism across thousands of GPUs* [00:39:45] The Modal Sandbox for agents* [00:43:51] Thoughts on the AI Inference War* [00:49:18] Erik's best tweets* [00:51:57] Why buying hardware is a waste of money* [00:54:18] Erik's competitive programming backgrounds* [00:59:02] Why does Sweden have the best Counter Strike players?* [00:59:53] Never owning a car or TV* [01:00:21] Advice for infrastructure startupsTranscriptAlessio [00:00:00]: Hey everyone, welcome to the Latent Space podcast. This is Alessio, partner and CTO-in-Residence at Decibel Partners, and I'm joined by my co-host Swyx, founder of Smol AI.Swyx [00:00:14]: Hey, and today we have in the studio Erik Bernhardsson from Modal. Welcome.Erik [00:00:19]: Hi. It's awesome being here.Swyx [00:00:20]: Yeah. Awesome seeing you in person. I've seen you online for a number of years as you were building on Modal and I think you're just making a San Francisco trip just to see people here, right? I've been to like two Modal events in San Francisco here.Erik [00:00:34]: Yeah, that's right. We're based in New York, so I figured sometimes I have to come out to capital of AI and make a presence.Swyx [00:00:40]: What do you think is the pros and cons of building in New York?Erik [00:00:45]: I mean, I never built anything elsewhere. I lived in New York the last 12 years. I love the city. Obviously, there's a lot more stuff going on here and there's a lot more customers and that's why I'm out here. I do feel like for me, where I am in life, I'm a very boring person. I kind of work hard and then I go home and hang out with my kids. I don't have time to go to events and meetups and stuff anyway. In that sense, New York is kind of nice. I walk to work every morning. It's like five minutes away from my apartment. It's very time efficient in that sense. Yeah.Swyx [00:01:10]: Yeah. It's also a good life. So we'll do a brief bio and then we'll talk about anything else that people should know about you. Actually, I was surprised to find out you're from Sweden. You went to college in KTH and your master's was in implementing a scalable music recommender system. Yeah.Erik [00:01:27]: I had no idea. Yeah. So I actually studied physics, but I grew up coding and I did a lot of programming competition and then as I was thinking about graduating, I got in touch with an obscure music streaming startup called Spotify, which was then like 30 people. And for some reason, I convinced them, why don't I just come and write a master's thesis with you and I'll do some cool collaborative filtering, despite not knowing anything about collaborative filtering really. But no one knew anything back then. So I spent six months at Spotify basically building a prototype of a music recommendation system and then turned that into a master's thesis. And then later when I graduated, I joined Spotify full time.Swyx [00:02:00]: So that was the start of your data career. You also wrote a couple of popular open source tooling while you were there. Is that correct?Erik [00:02:09]: No, that's right. I mean, I was at Spotify for seven years, so this is a long stint. And Spotify was a wild place early on and I mean, data space is also a wild place. I mean, it was like Hadoop cluster in the like foosball room on the floor. It was a lot of crude, like very basic infrastructure and I didn't know anything about it. And like I was hired to kind of figure out data stuff. And I started hacking on a recommendation system and then, you know, got sidetracked in a bunch of other stuff. I fixed a bunch of reporting things and set up A-B testing and started doing like business analytics and later got back to music recommendation system. And a lot of the infrastructure didn't really exist. Like there was like Hadoop back then, which is kind of bad and I don't miss it. But I spent a lot of time with that. As a part of that, I ended up building a workflow engine called Luigi, which is like briefly like somewhat like widely ended up being used by a bunch of companies. Sort of like, you know, kind of like Airflow, but like before Airflow. I think it did some things better, some things worse. I also built a vector database called Annoy, which is like for a while, it was actually quite widely used. In 2012, so it was like way before like all this like vector database stuff ended up happening. And funny enough, I was actually obsessed with like vectors back then. Like I was like, this is going to be huge. Like just give it like a few years. I didn't know it was going to take like nine years and then there's going to suddenly be like 20 startups doing vector databases in one year. So it did happen. In that sense, I was right. I'm glad I didn't start a startup in the vector database space. I would have started way too early. But yeah, that was, yeah, it was a fun seven years as part of it. It was a great culture, a great company.Swyx [00:03:32]: Yeah. Just to take a quick tangent on this vector database thing, because we probably won't revisit it but like, has anything architecturally changed in the last nine years?Erik [00:03:41]: I'm actually not following it like super closely. I think, you know, some of the best algorithms are still the same as like hierarchical navigable small world.Swyx [00:03:51]: Yeah. HNSW.Erik [00:03:52]: Exactly. I think now there's like product quantization, there's like some other stuff that I haven't really followed super closely. I mean, obviously, like back then it was like, you know, it's always like very simple. It's like a C++ library with Python bindings and you could mmap big files and into memory and like they had some lookups. I used like this kind of recursive, like hyperspace splitting strategy, which is not that good, but it sort of was good enough at that time. But I think a lot of like HNSW is still like what people generally use. Now of course, like databases are much better in the sense like to support like inserts and updates and stuff like that. I know I never supported that. Yeah, it's sort of exciting to finally see like vector databases becoming a thing.Swyx [00:04:30]: Yeah. Yeah. And then maybe one takeaway on most interesting lesson from Daniel Ek?Erik [00:04:36]: I mean, I think Daniel Ek, you know, he started Spotify very young. Like he was like 25, something like that. And that was like a good lesson. But like he, in a way, like I think he was a very good leader. Like there was never anything like, no scandals or like no, he wasn't very eccentric at all. It was just kind of like very like level headed, like just like ran the company very well, like never made any like obvious mistakes or I think it was like a few bets that maybe like in hindsight were like a little, you know, like took us, you know, too far in one direction or another. But overall, I mean, I think he was a great CEO, like definitely, you know, up there, like generational CEO, at least for like Swedish startups.Swyx [00:05:09]: Yeah, yeah, for sure. Okay, we should probably move to make our way towards Modal. So then you spent six years as CTO of Better. You were an early engineer and then you scaled up to like 300 engineers.Erik [00:05:21]: I joined as a CTO when there was like no tech team. And yeah, that was a wild chapter in my life. Like the company did very well for a while. And then like during the pandemic, yeah, it was kind of a weird story, but yeah, it kind of collapsed.Swyx [00:05:32]: Yeah, laid off people poorly.Erik [00:05:34]: Yeah, yeah. It was like a bunch of stories. Yeah. I mean, the company like grew from like 10 people when I joined at 10,000, now it's back to a thousand. But yeah, they actually went public a few months ago, kind of crazy. They're still around, like, you know, they're still, you know, doing stuff. So yeah, very kind of interesting six years of my life for non-technical reasons, like I managed like three, four hundred, but yeah, like learning a lot of that, like recruiting. I spent all my time recruiting and stuff like that. And so managing at scale, it's like nice, like now in a way, like when I'm building my own startup. It's actually something I like, don't feel nervous about at all. Like I've managed a scale, like I feel like I can do it again. It's like very different things that I'm nervous about as a startup founder. But yeah, I started Modal three years ago after sort of, after leaving Better, I took a little bit of time off during the pandemic and, but yeah, pretty quickly I was like, I got to build something. I just want to, you know. Yeah. And then yeah, Modal took form in my head, took shape.Swyx [00:06:22]: And as far as I understand, and maybe we can sort of trade off questions. So the quick history is started Modal in 2021, got your seed with Sarah from Amplify in 2022. You just announced your Series A with Redpoint. That's right. And that brings us up to mostly today. Yeah. Most people, I think, were expecting you to build for the data space.Erik: But it is the data space.Swyx:: When I think of data space, I come from like, you know, Snowflake, BigQuery, you know, Fivetran, Nearby, that kind of stuff. And what Modal became is more general purpose than that. Yeah.Erik [00:06:53]: Yeah. I don't know. It was like fun. I actually ran into like Edo Liberty, the CEO of Pinecone, like a few weeks ago. And he was like, I was so afraid you were building a vector database. No, I started Modal because, you know, like in a way, like I work with data, like throughout my most of my career, like every different part of the stack, right? Like I thought everything like business analytics to like deep learning, you know, like building, you know, training neural networks, the scale, like everything in between. And so one of the thoughts, like, and one of the observations I had when I started Modal or like why I started was like, I just wanted to make, build better tools for data teams. And like very, like sort of abstract thing, but like, I find that the data stack is, you know, full of like point solutions that don't integrate well. And still, when you look at like data teams today, you know, like every startup ends up building their own internal Kubernetes wrapper or whatever. And you know, all the different data engineers and machine learning engineers end up kind of struggling with the same things. So I started thinking about like, how do I build a new data stack, which is kind of a megalomaniac project, like, because you kind of want to like throw out everything and start over.Swyx [00:07:54]: It's almost a modern data stack.Erik [00:07:55]: Yeah, like a postmodern data stack. And so I started thinking about that. And a lot of it came from like, like more focused on like the human side of like, how do I make data teams more productive? And like, what is the technology tools that they need? And like, you know, drew out a lot of charts of like, how the data stack looks, you know, what are different components. And it shows actually very interesting, like workflow scheduling, because it kind of sits in like a nice sort of, you know, it's like a hub in the graph of like data products. But it was kind of hard to like, kind of do that in a vacuum, and also to monetize it to some extent. I got very interested in like the layers below at some point. And like, at the end of the day, like most people have code to have to run somewhere. So I think about like, okay, well, how do you make that nice? Like how do you make that? And in particular, like the thing I always like thought about, like developer productivity is like, I think the best way to measure developer productivity is like in terms of the feedback loops, like how quickly when you iterate, like when you write code, like how quickly can you get feedback. And at the innermost loop, it's like writing code and then running it. And like, as soon as you start working with the cloud, like it's like takes minutes suddenly, because you have to build a Docker container and push it to the cloud and like run it, you know. So that was like the initial focus for me was like, I just want to solve that problem. Like I want to, you know, build something less, you run things in the cloud and like retain the sort of, you know, the joy of productivity as when you're running things locally. And in particular, I was quite focused on data teams, because I think they had a couple unique needs that wasn't well served by the infrastructure at that time, or like still is in like, in particular, like Kubernetes, I feel like it's like kind of worked okay for back end teams, but not so well for data teams. And very quickly, I got sucked into like a very deep like rabbit hole of like...Swyx [00:09:24]: Not well for data teams because of burstiness. Yeah, for sure.Erik [00:09:26]: So like burstiness is like one thing, right? Like, you know, like you often have this like fan out, you want to like apply some function over very large data sets. Another thing tends to be like hardware requirements, like you need like GPUs and like, I've seen this in many companies, like you go, you know, data scientists go to a platform team and they're like, can we add GPUs to the Kubernetes? And they're like, no, like, that's, you know, complex, and we're not gonna, so like just getting GPU access. And then like, I mean, I also like data code, like frankly, or like machine learning code like tends to be like, super annoying in terms of like environments, like you end up having like a lot of like custom, like containers and like environment conflicts. And like, it's very hard to set up like a unified container that like can serve like a data scientist, because like, there's always like packages that break. And so I think there's a lot of different reasons why the technology wasn't well suited for back end. And I think the attitude at that time is often like, you know, like you had friction between the data team and the platform team, like, well, it works for the back end stuff, you know, why don't you just like, you know, make it work. But like, I actually felt like data teams, you know, or at this point now, like there's so much, so many people working with data, and like they, to some extent, like deserve their own tools and their own tool chains, and like optimizing for that is not something people have done. So that's, that's sort of like very abstract philosophical reason why I started Model. And then, and then I got sucked into this like rabbit hole of like container cold start and, you know, like whatever, Linux, page cache, you know, file system optimizations.Swyx [00:10:43]: Yeah, tell people, I think the first time I met you, I think you told me some numbers, but I don't remember, like, what are the main achievements that you were unhappy with the status quo? And then you built your own container stack?Erik [00:10:52]: Yeah, I mean, like, in particular, it was like, in order to have that loop, right? You want to be able to start, like take code on your laptop, whatever, and like run in the cloud very quickly, and like running in custom containers, and maybe like spin up like 100 containers, 1000, you know, things like that. And so container cold start was the initial like, from like a developer productivity point of view, it was like, really, what I was focusing on is, I want to take code, I want to stick it in container, I want to execute in the cloud, and like, you know, make it feel like fast. And when you look at like, how Docker works, for instance, like Docker, you have this like, fairly convoluted, like very resource inefficient way, they, you know, you build a container, you upload the whole container, and then you download it, and you run it. And Kubernetes is also like, not very fast at like starting containers. So like, I started kind of like, you know, going a layer deeper, like Docker is actually like, you know, there's like a couple of different primitives, but like a lower level primitive is run C, which is like a container runner. And I was like, what if I just take the container runner, like run C, and I point it to like my own root file system, and then I built like my own virtual file system that exposes files over a network instead. And that was like the sort of very crude version of model, it's like now I can actually start containers very quickly, because it turns out like when you start a Docker container, like, first of all, like most Docker images are like several gigabytes, and like 99% of that is never going to be consumed, like there's a bunch of like, you know, like timezone information for like Uzbekistan, like no one's going to read it. And then there's a very high overlap between the files are going to be read, there's going to be like lib torch or whatever, like it's going to be read. So you can also cache it very well. So that was like the first sort of stuff we started working on was like, let's build this like container file system. And you know, coupled with like, you know, just using run C directly. And that actually enabled us to like, get to this point of like, you write code, and then you can launch it in the cloud within like a second or two, like something like that. And you know, there's been many optimizations since then, but that was sort of starting point.Alessio [00:12:33]: Can we talk about the developer experience as well, I think one of the magic things about Modal is at the very basic layers, like a Python function decorator, it's just like stub and whatnot. But then you also have a way to define a full container, what were kind of the design decisions that went into it? Where did you start? How easy did you want it to be? And then maybe how much complexity did you then add on to make sure that every use case fit?Erik [00:12:57]: I mean, Modal, I almost feel like it's like almost like two products kind of glued together. Like there's like the low level like container runtime, like file system, all that stuff like in Rust. And then there's like the Python SDK, right? Like how do you express applications? And I think, I mean, Swix, like I think your blog was like the self-provisioning runtime was like, to me, always like to sort of, for me, like an eye-opening thing. It's like, so I didn't think about like...Swyx [00:13:15]: You wrote your post four months before me. Yeah? The software 2.0, Infra 2.0. Yeah.Erik [00:13:19]: Well, I don't know, like convergence of minds. I guess we were like both thinking. Maybe you put, I think, better words than like, you know, maybe something I was like thinking about for a long time. Yeah.Swyx [00:13:29]: And I can tell you how I was thinking about it on my end, but I want to hear you say it.Erik [00:13:32]: Yeah, yeah, I would love to. So to me, like what I always wanted to build was like, I don't know, like, I don't know if you use like Pulumi. Like Pulumi is like nice, like in the sense, like it's like Pulumi is like you describe infrastructure in code, right? And to me, that was like so nice. Like finally I can like, you know, put a for loop that creates S3 buckets or whatever. And I think like Modal sort of goes one step further in the sense that like, what if you also put the app code inside the infrastructure code and like glue it all together and then like you only have one single place that defines everything and it's all programmable. You don't have any config files. Like Modal has like zero config. There's no config. It's all code. And so that was like the goal that I wanted, like part of that. And then the other part was like, I often find that so much of like my time was spent on like the plumbing between containers. And so my thing was like, well, if I just build this like Python SDK and make it possible to like bridge like different containers, just like a function call, like, and I can say, oh, this function runs in this container and this other function runs in this container and I can just call it just like a normal function, then, you know, I can build these applications that may span a lot of different environments. Maybe they fan out, start other containers, but it's all just like inside Python. You just like have this beautiful kind of nice like DSL almost for like, you know, how to control infrastructure in the cloud. So that was sort of like how we ended up with the Python SDK as it is, which is still evolving all the time, by the way. We keep changing syntax quite a lot because I think it's still somewhat exploratory, but we're starting to converge on something that feels like reasonably good now.Swyx [00:14:54]: Yeah. And along the way you, with this expressiveness, you enabled the ability to, for example, attach a GPU to a function. Totally.Erik [00:15:02]: Yeah. It's like you just like say, you know, on the function decorator, you're like GPU equals, you know, A100 and then or like GPU equals, you know, A10 or T4 or something like that. And then you get that GPU and like, you know, you just run the code and it runs like you don't have to, you know, go through hoops to, you know, start an EC2 instance or whatever.Swyx [00:15:18]: Yeah. So it's all code. Yeah. So one of the reasons I wrote Self-Revisioning Runtimes was I was working at AWS and we had AWS CDK, which is kind of like, you know, the Amazon basics blew me. Yeah, totally. And then, and then like it creates, it compiles the cloud formation. Yeah. And then on the other side, you have to like get all the config stuff and then put it into your application code and make sure that they line up. So then you're writing code to define your infrastructure, then you're writing code to define your application. And I was just like, this is like obvious that it's going to converge, right? Yeah, totally.Erik [00:15:48]: But isn't there like, it might be wrong, but like, was it like SAM or Chalice or one of those? Like, isn't that like an AWS thing that where actually they kind of did that? I feel like there's like one.Swyx [00:15:57]: SAM. Yeah. Still very clunky. It's not, not as elegant as modal.Erik [00:16:03]: I love AWS for like the stuff it's built, you know, like historically in order for me to like, you know, what it enables me to build, but like AWS is always like struggle with developer experience.Swyx [00:16:11]: I mean, they have to not break things.Erik [00:16:15]: Yeah. Yeah. And totally. And they have to build products for a very wide range of use cases. And I think that's hard.Swyx [00:16:21]: Yeah. Yeah. So it's, it's easier to design for. Yeah. So anyway, I was, I was pretty convinced that this, this would happen. I wrote, wrote that thing. And then, you know, I imagine my surprise that you guys had it on your landing page at some point. I think, I think Akshad was just like, just throw that in there.Erik [00:16:34]: Did you trademark it?Swyx [00:16:35]: No, I didn't. But I definitely got sent a few pitch decks with my post on there and it was like really interesting. This is my first time like kind of putting a name to a phenomenon. And I think this is a useful skill for people to just communicate what they're trying to do.Erik [00:16:48]: Yeah. No, I think it's a beautiful concept.Swyx [00:16:50]: Yeah. Yeah. Yeah. But I mean, obviously you implemented it. What became more clear in your explanation today is that actually you're not that tied to Python.Erik [00:16:57]: No. I mean, I, I think that all the like lower level stuff is, you know, just running containers and like scheduling things and, you know, serving container data and stuff. So like one of the benefits of data teams is obviously like they're all like using Python, right? And so that made it a lot easier. I think, you know, if we had focused on other workloads, like, you know, for various reasons, we've like been kind of like half thinking about like CI or like things like that. But like, in a way that's like harder because like you also, then you have to be like, you know, multiple SDKs, whereas, you know, focusing on data teams, you can only, you know, Python like covers like 95% of all teams. That made it a lot easier. But like, I mean, like definitely like in the future, we're going to have others support, like supporting other languages. JavaScript for sure is the obvious next language. But you know, who knows, like, you know, Rust, Go, R, whatever, PHP, Haskell, I don't know.Swyx [00:17:42]: You know, I think for me, I actually am a person who like kind of liked the idea of programming language advancements being improvements in developer experience. But all I saw out of the academic sort of PLT type people is just type level improvements. And I always think like, for me, like one of the core reasons for self-provisioning runtimes and then why I like Modal is like, this is actually a productivity increase, right? Like, it's a language level thing, you know, you managed to stick it on top of an existing language, but it is your own language, a DSL on top of Python. And so language level increase on the order of like automatic memory management. You know, you could sort of make that analogy that like, maybe you lose some level of control, but most of the time you're okay with whatever Modal gives you. And like, that's fine. Yeah.Erik [00:18:26]: Yeah. Yeah. I mean, that's how I look at about it too. Like, you know, you look at developer productivity over the last number of decades, like, you know, it's come in like small increments of like, you know, dynamic typing or like is like one thing because not suddenly like for a lot of use cases, you don't need to care about type systems or better compiler technology or like, you know, the cloud or like, you know, relational databases. And, you know, I think, you know, you look at like that, you know, history, it's a steadily, you know, it's like, you know, you look at the developers have been getting like probably 10X more productive every decade for the last four decades or something that was kind of crazy. Like on an exponential scale, we're talking about 10X or is there a 10,000X like, you know, improvement in developer productivity. What we can build today, you know, is arguably like, you know, a fraction of the cost of what it took to build it in the eighties. Maybe it wasn't even possible in the eighties. So that to me, like, that's like so fascinating. I think it's going to keep going for the next few decades. Yeah.Alessio [00:19:14]: Yeah. Another big thing in the infra 2.0 wishlist was truly serverless infrastructure. The other on your landing page, you called them native cloud functions, something like that. I think the issue I've seen with serverless has always been people really wanted it to be stateful, even though stateless was much easier to do. And I think now with AI, most model inference is like stateless, you know, outside of the context. So that's kind of made it a lot easier to just put a model, like an AI model on model to run. How do you think about how that changes how people think about infrastructure too? Yeah.Erik [00:19:48]: I mean, I think model is definitely going in the direction of like doing more stateful things and working with data and like high IO use cases. I do think one like massive serendipitous thing that happened like halfway, you know, a year and a half into like the, you know, building model was like Gen AI started exploding and the IO pattern of Gen AI is like fits the serverless model like so well, because it's like, you know, you send this tiny piece of information, like a prompt, right, or something like that. And then like you have this GPU that does like trillions of flops, and then it sends back like a tiny piece of information, right. And that turns out to be something like, you know, if you can get serverless working with GPU, that just like works really well, right. So I think from that point of view, like serverless always to me felt like a little bit of like a solution looking for a problem. I don't actually like don't think like backend is like the problem that needs to serve it or like not as much. But I look at data and in particular, like things like Gen AI, like model inference, like it's like clearly a good fit. So I think that is, you know, to a large extent explains like why we saw, you know, the initial sort of like killer app for model being model inference, which actually wasn't like necessarily what we're focused on. But that's where we've seen like by far the most usage. Yeah.Swyx [00:20:52]: And this was before you started offering like fine tuning of language models, it was mostly stable diffusion. Yeah.Erik [00:20:59]: Yeah. I mean, like model, like I always built it to be a very general purpose compute platform, like something where you can run everything. And I used to call model like a better Kubernetes for data team for a long time. What we realized was like, yeah, that's like, you know, a year and a half in, like we barely had any users or any revenue. And like we were like, well, maybe we should look at like some use case, trying to think of use case. And that was around the same time stable diffusion came out. And the beauty of model is like you can run almost anything on model, right? Like model inference turned out to be like the place where we found initially, well, like clearly this has like 10x like better agronomics than anything else. But we're also like, you know, going back to my original vision, like we're thinking a lot about, you know, now, okay, now we do inference really well. Like what about training? What about fine tuning? What about, you know, end-to-end lifecycle deployment? What about data pre-processing? What about, you know, I don't know, real-time streaming? What about, you know, large data munging, like there's just data observability. I think there's so many things, like kind of going back to what I said about like redefining the data stack, like starting with the foundation of compute. Like one of the exciting things about model is like we've sort of, you know, we've been working on that for three years and it's maturing, but like this is so many things you can do like with just like a better compute primitive and also go up to stack and like do all this other stuff on top of it.Alessio [00:22:09]: How do you think about or rather like I would love to learn more about the underlying infrastructure and like how you make that happen because with fine tuning and training, it's a static memory. Like you exactly know what you're going to load in memory one and it's kind of like a set amount of compute versus inference, just like data is like very bursty. How do you make batches work with a serverless developer experience? You know, like what are like some fun technical challenge you solve to make sure you get max utilization on these GPUs? What we hear from people is like, we have GPUs, but we can really only get like, you know, 30, 40, 50% maybe utilization. What's some of the fun stuff you're working on to get a higher number there?Erik [00:22:48]: Yeah, I think on the inference side, like that's where we like, you know, like from a cost perspective, like utilization perspective, we've seen, you know, like very good numbers and in particular, like it's our ability to start containers and stop containers very quickly. And that means that we can auto scale extremely fast and scale down very quickly, which means like we can always adjust the sort of capacity, the number of GPUs running to the exact traffic volume. And so in many cases, like that actually leads to a sort of interesting thing where like we obviously run our things on like the public cloud, like AWS GCP, we run on Oracle, but in many cases, like users who do inference on those platforms or those clouds, even though we charge a slightly higher price per GPU hour, a lot of users like moving their large scale inference use cases to model, they end up saving a lot of money because we only charge for like with the time the GPU is actually running. And that's a hard problem, right? Like, you know, if you have to constantly adjust the number of machines, if you have to start containers, stop containers, like that's a very hard problem. Starting containers quickly is a very difficult thing. I mentioned we had to build our own file system for this. We also, you know, built our own container scheduler for that. We've implemented recently CPU memory checkpointing so we can take running containers and snapshot the entire CPU, like including registers and everything, and restore it from that point, which means we can restore it from an initialized state. We're looking at GPU checkpointing next, it's like a very interesting thing. So I think with inference stuff, that's where serverless really shines because you can drive, you know, you can push the frontier of latency versus utilization quite substantially, you know, which either ends up being a latency advantage or a cost advantage or both, right? On training, it's probably arguably like less of an advantage doing serverless, frankly, because you know, you can just like spin up a bunch of machines and try to satisfy, like, you know, train as much as you can on each machine. For that area, like we've seen, like, you know, arguably like less usage, like for modal, but there are always like some interesting use case. Like we do have a couple of customers, like RAM, for instance, like they do fine tuning with modal and they basically like one of the patterns they have is like very bursty type fine tuning where they fine tune 100 models in parallel. And that's like a separate thing that modal does really well, right? Like you can, we can start up 100 containers very quickly, run a fine tuning training job on each one of them for that only runs for, I don't know, 10, 20 minutes. And then, you know, you can do hyper parameter tuning in that sense, like just pick the best model and things like that. So there are like interesting training. I think when you get to like training, like very large foundational models, that's a use case we don't support super well, because that's very high IO, you know, you need to have like infinite band and all these things. And those are things we haven't supported yet and might take a while to get to that. So that's like probably like an area where like we're relatively weak in. Yeah.Alessio [00:25:12]: Have you cared at all about lower level model optimization? There's other cloud providers that do custom kernels to get better performance or are you just given that you're not just an AI compute company? Yeah.Erik [00:25:24]: I mean, I think like we want to support like a generic, like general workloads in a sense that like we want users to give us a container essentially or a code or code. And then we want to run that. So I think, you know, we benefit from those things in the sense that like we can tell our users, you know, to use those things. But I don't know if we want to like poke into users containers and like do those things automatically. That's sort of, I think a little bit tricky from the outside to do, because we want to be able to take like arbitrary code and execute it. But certainly like, you know, we can tell our users to like use those things. Yeah.Swyx [00:25:53]: I may have betrayed my own biases because I don't really think about modal as for data teams anymore. I think you started, I think you're much more for AI engineers. My favorite anecdotes, which I think, you know, but I don't know if you directly experienced it. I went to the Vercel AI Accelerator, which you supported. And in the Vercel AI Accelerator, a bunch of startups gave like free credits and like signups and talks and all that stuff. The only ones that stuck are the ones that actually appealed to engineers. And the top usage, the top tool used by far was modal.Erik [00:26:24]: That's awesome.Swyx [00:26:25]: For people building with AI apps. Yeah.Erik [00:26:27]: I mean, it might be also like a terminology question, like the AI versus data, right? Like I've, you know, maybe I'm just like old and jaded, but like, I've seen so many like different titles, like for a while it was like, you know, I was a data scientist and a machine learning engineer and then, you know, there was like analytics engineers and there was like an AI engineer, you know? So like, to me, it's like, I just like in my head, that's to me just like, just data, like, or like engineer, you know, like I don't really, so that's why I've been like, you know, just calling it data teams. But like, of course, like, you know, AI is like, you know, like such a massive fraction of our like workloads.Swyx [00:26:59]: It's a different Venn diagram of things you do, right? So the stuff that you're talking about where you need like infinite bands for like highly parallel training, that's not, that's more of the ML engineer, that's more of the research scientist and less of the AI engineer, which is more sort of trying to put, work at the application.Erik [00:27:16]: Yeah. I mean, to be fair to it, like we have a lot of users that are like doing stuff that I don't think fits neatly into like AI. Like we have a lot of people using like modal for web scraping, like it's kind of nice. You can just like, you know, fire up like a hundred or a thousand containers running Chromium and just like render a bunch of webpages and it takes, you know, whatever. Or like, you know, protein folding is that, I mean, maybe that's, I don't know, like, but like, you know, we have a bunch of users doing that or, or like, you know, in terms of, in the realm of biotech, like sequence alignment, like people using, or like a couple of people using like modal to run like large, like mixed integer programming problems, like, you know, using Gurobi or like things like that. So video processing is another thing that keeps coming up, like, you know, let's say you have like petabytes of video and you want to just like transcode it, like, or you can fire up a lot of containers and just run FFmpeg or like, so there are those things too. Like, I mean, like that being said, like AI is by far our biggest use case, but you know, like, again, like modal is kind of general purpose in that sense.Swyx [00:28:08]: Yeah. Well, maybe I'll stick to the stable diffusion thing and then we'll move on to the other use cases for AI that you want to highlight. The other big player in my mind is replicate. Yeah. In this, in this era, they're much more, I guess, custom built for that purpose, whereas you're more general purpose. How do you position yourself with them? Are they just for like different audiences or are you just heads on competing?Erik [00:28:29]: I think there's like a tiny sliver of the Venn diagram where we're competitive. And then like 99% of the area we're not competitive. I mean, I think for people who, if you look at like front-end engineers, I think that's where like really they found good fit is like, you know, people who built some cool web app and they want some sort of AI capability and they just, you know, an off the shelf model is like perfect for them. That's like, I like use replicate. That's great. I think where we shine is like custom models or custom workflows, you know, running things at very large scale. We need to care about utilization, care about costs. You know, we have much lower prices because we spend a lot more time optimizing our infrastructure, you know, and that's where we're competitive, right? Like, you know, and you look at some of the use cases, like Suno is a big user, like they're running like large scale, like AI. Oh, we're talking with Mikey.Swyx [00:29:12]: Oh, that's great. Cool.Erik [00:29:14]: In a month. Yeah. So, I mean, they're, they're using model for like production infrastructure. Like they have their own like custom model, like custom code and custom weights, you know, for AI generated music, Suno.AI, you know, that, that, those are the types of use cases that we like, you know, things that are like very custom or like, it's like, you know, and those are the things like it's very hard to run and replicate, right? And that's fine. Like I think they, they focus on a very different part of the stack in that sense.Swyx [00:29:35]: And then the other company pattern that I pattern match you to is Modular. I don't know.Erik [00:29:40]: Because of the names?Swyx [00:29:41]: No, no. Wow. No, but yeah, yes, the name is very similar. I think there's something that might be insightful there from a linguistics point of view. Oh no, they have Mojo, the sort of Python SDK. And they have the Modular Inference Engine, which is their sort of their cloud stack, their sort of compute inference stack. I don't know if anyone's made that comparison to you before, but like I see you evolving a little bit in parallel there.Erik [00:30:01]: No, I mean, maybe. Yeah. Like it's not a company I'm like super like familiar, like, I mean, I know the basics, but like, I guess they're similar in the sense like they want to like do a lot of, you know, they have sort of big picture vision.Swyx [00:30:12]: Yes. They also want to build very general purpose. Yeah. So they're marketing themselves as like, if you want to do off the shelf stuff, go out, go somewhere else. If you want to do custom stuff, we're the best place to do it. Yeah. Yeah. There is some overlap there. There's not overlap in the sense that you are a closed source platform. People have to host their code on you. That's true. Whereas for them, they're very insistent on not running their own cloud service. They're a box software. Yeah. They're licensed software.Erik [00:30:37]: I'm sure their VCs at some point going to force them to reconsider. No, no.Swyx [00:30:40]: Chris is very, very insistent and very convincing. So anyway, I would just make that comparison, let people make the links if they want to. But it's an interesting way to see the cloud market develop from my point of view, because I came up in this field thinking cloud is one thing, and I think your vision is like something slightly different, and I see the different takes on it.Erik [00:31:00]: Yeah. And like one thing I've, you know, like I've written a bit about it in my blog too, it's like I think of us as like a second layer of cloud provider in the sense that like I think Snowflake is like kind of a good analogy. Like Snowflake, you know, is infrastructure as a service, right? But they actually run on the like major clouds, right? And I mean, like you can like analyze this very deeply, but like one of the things I always thought about is like, why does Snowflake arbitrarily like win over Redshift? And I think Snowflake, you know, to me, one, because like, I mean, in the end, like AWS makes all the money anyway, like and like Snowflake just had the ability to like focus on like developer experience or like, you know, user experience. And to me, like really proved that you can build a cloud provider, a layer up from, you know, the traditional like public clouds. And in that layer, that's also where I would put Modal, it's like, you know, we're building a cloud provider, like we're, you know, we're like a multi-tenant environment that runs the user code. But we're also building on top of the public cloud. So I think there's a lot of room in that space, I think is very sort of interesting direction.Alessio [00:31:55]: How do you think of that compared to the traditional past history, like, you know, you had AWS, then you had Heroku, then you had Render, Railway.Erik [00:32:04]: Yeah, I mean, I think those are all like great. I think the problem that they all faced was like the graduation problem, right? Like, you know, Heroku or like, I mean, like also like Heroku, there's like a counterfactual future of like, what would have happened if Salesforce didn't buy them, right? Like, that's a sort of separate thing. But like, I think what Heroku, I think always struggled with was like, eventually companies would get big enough that you couldn't really justify running in Heroku. So they would just go and like move it to, you know, whatever AWS or, you know, in particular. And you know, that's something that keeps me up at night too, like, what does that graduation risk like look like for modal? I always think like the only way to build a successful infrastructure company in the long run in the cloud today is you have to appeal to the entire spectrum, right? Or at least like the enterprise, like you have to capture the enterprise market. But the truly good companies capture the whole spectrum, right? Like I think of companies like, I don't like Datadog or Mongo or something that were like, they both captured like the hobbyists and acquire them, but also like, you know, have very large enterprise customers. I think that arguably was like where I, in my opinion, like Heroku struggle was like, how do you maintain the customers as they get more and more advanced? I don't know what the solution is, but I think there's, you know, that's something I would have thought deeply if I was at Heroku at that time.Alessio [00:33:14]: What's the AI graduation problem? Is it, I need to fine tune the model, I need better economics, any insights from customer discussions?Erik [00:33:22]: Yeah, I mean, better economics, certainly. But although like, I would say like, even for people who like, you know, needs like thousands of GPUs, just because we can drive utilization so much better, like we, there's actually like a cost advantage of staying on modal. But yeah, I mean, certainly like, you know, and like the fact that VCs like love, you know, throwing money at least used to, you know, add companies who need it to buy GPUs. I think that didn't help the problem. And in training, I think, you know, there's less software differentiation. So in training, I think there's certainly like better economics of like buying big clusters. But I mean, my hope it's going to change, right? Like I think, you know, we're still pretty early in the cycle of like building AI infrastructure. And I think a lot of these companies over in the long run, like, you know, they're, except it may be super big ones, like, you know, on Facebook and Google, they're always going to build their own ones. But like everyone else, like some extent, you know, I think they're better off like buying platforms. And, you know, someone's going to have to build those platforms.Swyx [00:34:12]: Yeah. Cool. Let's move on to language models and just specifically that workload just to flesh it out a little bit. You already said that RAMP is like fine tuning 100 models at once simultaneously on modal. Closer to home, my favorite example is ErikBot. Maybe you want to tell that story.Erik [00:34:30]: Yeah. I mean, it was a prototype thing we built for fun, but it's pretty cool. Like we basically built this thing that hooks up to Slack. It like downloads all the Slack history and, you know, fine-tunes a model based on a person. And then you can chat with that. And so you can like, you know, clone yourself and like talk to yourself on Slack. I mean, it's like nice like demo and it's just like, I think like it's like fully contained modal. Like there's a modal app that does everything, right? Like it downloads Slack, you know, integrates with the Slack API, like downloads the stuff, the data, like just runs the fine-tuning and then like creates like dynamically an inference endpoint. And it's all like self-contained and like, you know, a few hundred lines of code. So I think it's sort of a good kind of use case for, or like it kind of demonstrates a lot of the capabilities of modal.Alessio [00:35:08]: Yeah. On a more personal side, how close did you feel ErikBot was to you?Erik [00:35:13]: It definitely captured the like the language. Yeah. I mean, I don't know, like the content, I always feel this way about like AI and it's gotten better. Like when you look at like AI output of text, like, and it's like, when you glance at it, it's like, yeah, this seems really smart, you know, but then you actually like look a little bit deeper. It's like, what does this mean?Swyx [00:35:32]: What does this person say?Erik [00:35:33]: It's like kind of vacuous, right? And that's like kind of what I felt like, you know, talking to like my clone version, like it's like says like things like the grammar is correct. Like some of the sentences make a lot of sense, but like, what are you trying to say? Like there's no content here. I don't know. I mean, it's like, I got that feeling also with chat TBT in the like early versions right now it's like better, but.Alessio [00:35:51]: That's funny. So I built this thing called small podcaster to automate a lot of our back office work, so to speak. And it's great at transcript. It's great at doing chapters. And then I was like, okay, how about you come up with a short summary? And it's like, it sounds good, but it's like, it's not even the same ballpark as like, yeah, end up writing. Right. And it's hard to see how it's going to get there.Swyx [00:36:11]: Oh, I have ideas.Erik [00:36:13]: I'm certain it's going to get there, but like, I agree with you. Right. And like, I have the same thing. I don't know if you've read like AI generated books. Like they just like kind of seem funny, right? Like there's off, right? But like you glance at it and it's like, oh, it's kind of cool. Like looks correct, but then it's like very weird when you actually read them.Swyx [00:36:30]: Yeah. Well, so for what it's worth, I think anyone can join the modal slack. Is it open to the public? Yeah, totally.Erik [00:36:35]: If you go to modal.com, there's a button in the footer.Swyx [00:36:38]: Yeah. And then you can talk to Erik Bot. And then sometimes I really like picking Erik Bot and then you answer afterwards, but then you're like, yeah, mostly correct or whatever. Any other broader lessons, you know, just broadening out from like the single use case of fine tuning, like what are you seeing people do with fine tuning or just language models on modal in general? Yeah.Erik [00:36:59]: I mean, I think language models is interesting because so many people get started with APIs and that's just, you know, they're just dominating a space in particular opening AI, right? And that's not necessarily like a place where we aim to compete. I mean, maybe at some point, but like, it's just not like a core focus for us. And I think sort of separately, it's sort of a question of like, there's economics in that long term. But like, so we tend to focus on more like the areas like around it, right? Like fine tuning, like another use case we have is a bunch of people, Ramp included, is doing batch embeddings on modal. So let's say, you know, you have like a, actually we're like writing a blog post, like we take all of Wikipedia and like parallelize embeddings in 15 minutes and produce vectors for each article. So those types of use cases, I think modal suits really well for. I think also a lot of like custom inference, like yeah, I love that.Swyx [00:37:43]: Yeah. I think you should give people an idea of the order of magnitude of parallelism, because I think people don't understand how parallel. So like, I think your classic hello world with modal is like some kind of Fibonacci function, right? Yeah, we have a bunch of different ones. Some recursive function. Yeah.Erik [00:37:59]: Yeah. I mean, like, yeah, I mean, it's like pretty easy in modal, like fan out to like, you know, at least like 100 GPUs, like in a few seconds. And you know, if you give it like a couple of minutes, like we can, you know, you can fan out to like thousands of GPUs. Like we run it relatively large scale. And yeah, we've run, you know, many thousands of GPUs at certain points when we needed, you know, big backfills or some customers had very large compute needs.Swyx [00:38:21]: Yeah. Yeah. And I mean, that's super useful for a number of things. So one of my early interactions with modal as well was with a small developer, which is my sort of coding agent. The reason I chose modal was a number of things. One, I just wanted to try it out. I just had an excuse to try it. Akshay offered to onboard me personally. But the most interesting thing was that you could have that sort of local development experience as it was running on my laptop, but then it would seamlessly translate to a cloud service or like a cloud hosted environment. And then it could fan out with concurrency controls. So I could say like, because like, you know, the number of times I hit the GPT-3 API at the time was going to be subject to the rate limit. But I wanted to fan out without worrying about that kind of stuff. With modal, I can just kind of declare that in my config and that's it. Oh, like a concurrency limit?Erik [00:39:07]: Yeah. Yeah.Swyx [00:39:09]: Yeah. There's a lot of control. And that's why it's like, yeah, this is a pretty good use case for like writing this kind of LLM application code inside of this environment that just understands fan out and rate limiting natively. You don't actually have an exposed queue system, but you have it under the hood, you know, that kind of stuff. Totally.Erik [00:39:28]: It's a self-provisioning cloud.Swyx [00:39:30]: So the last part of modal I wanted to touch on, and obviously feel free, I know you're working on new features, was the sandbox that was introduced last year. And this is something that I think was inspired by Code Interpreter. You can tell me the longer history behind that.Erik [00:39:45]: Yeah. Like we originally built it for the use case, like there was a bunch of customers who looked into code generation applications and then they came to us and asked us, is there a safe way to execute code? And yeah, we spent a lot of time on like container security. We used GeoVisor, for instance, which is a Google product that provides pretty strong isolation of code. So we built a product where you can basically like run arbitrary code inside a container and monitor its output or like get it back in a safe way. I mean, over time it's like evolved into more of like, I think the long-term direction is actually I think more interesting, which is that I think modal as a platform where like I think the core like container infrastructure we offer could actually be like, you know, unbundled from like the client SDK and offer to like other, you know, like we're talking to a couple of like other companies that want to run, you know, through their packages, like run, execute jobs on modal, like kind of programmatically. So that's actually the direction like Sandbox is going. It's like turning into more like a platform for platforms is kind of what I've been thinking about it as.Swyx [00:40:45]: Oh boy. Platform. That's the old Kubernetes line.Erik [00:40:48]: Yeah. Yeah. Yeah. But it's like, you know, like having that ability to like programmatically, you know, create containers and execute them, I think, I think is really cool. And I think it opens up a lot of interesting capabilities that are sort of separate from the like core Python SDK in modal. So I'm really excited about C. It's like one of those features that we kind of released and like, you know, then we kind of look at like what users actually build with it and people are starting to build like kind of crazy things. And then, you know, we double down on some of those things because when we see like, you know, potential new product features and so Sandbox, I think in that sense, it's like kind of in that direction. We found a lot of like interesting use cases in the direction of like platformized container runner.Swyx [00:41:27]: Can you be more specific about what you're double down on after seeing users in action?Erik [00:41:32]: I mean, we're working with like some companies that, I mean, without getting into specifics like that, need the ability to take their users code and then launch containers on modal. And it's not about security necessarily, like they just want to use modal as a back end, right? Like they may already provide like Kubernetes as a back end, Lambda as a back end, and now they want to add modal as a back end, right? And so, you know, they need a way to programmatically define jobs on behalf of their users and execute them. And so, I don't know, that's kind of abstract, but does that make sense? I totally get it.Swyx [00:42:03]: It's sort of one level of recursion to sort of be the Modal for their customers.Erik [00:42:09]: Exactly.Swyx [00:42:10]: Yeah, exactly. And Cloudflare has done this, you know, Kenton Vardar from Cloudflare, who's like the tech lead on this thing, called it sort of functions as a service as a service.Erik [00:42:17]: Yeah, that's exactly right. FaSasS.Swyx [00:42:21]: FaSasS. Yeah, like, I mean, like that, I think any base layer, second layer cloud provider like yourself, compute provider like yourself should provide, you know, it's a mark of maturity and success that people just trust you to do that. They'd rather build on top of you than compete with you. The more interesting thing for me is like, what does it mean to serve a computer like an LLM developer, rather than a human developer, right? Like, that's what a sandbox is to me, that you have to redefine modal to serve a different non-human audience.Erik [00:42:51]: Yeah. Yeah, and I think there's some really interesting people, you know, building very cool things.Swyx [00:42:55]: Yeah. So I don't have an answer, but, you know, I imagine things like, hey, the way you give feedback is different. Maybe you have to like stream errors, log errors differently. I don't really know. Yeah. Obviously, there's like safety considerations. Maybe you have an API to like restrict access to the web. Yeah. I don't think anyone would use it, but it's there if you want it.Erik [00:43:17]: Yeah.Swyx [00:43:18]: Yeah. Any other sort of design considerations? I have no idea.Erik [00:43:21]: With sandboxes?Swyx [00:43:22]: Yeah. Yeah.Erik [00:43:24]: Open-ended question here. Yeah. I mean, no, I think, yeah, the network restrictions, I think, make a lot of sense. Yeah. I mean, I think, you know, long-term, like, I think there's a lot of interesting use cases where like the LLM, in itself, can like decide, I want to install these packages and like run this thing. And like, obviously, for a lot of those use cases, like you want to have some sort of control that it doesn't like install malicious stuff and steal your secrets and things like that. But I think that's what's exciting about the sandbox primitive, is like it lets you do that in a relatively safe way.Alessio [00:43:51]: Do you have any thoughts on the inference wars? A lot of providers are just rushing to the bottom to get the lowest price per million tokens. Some of them, you know, the Sean Randomat, they're just losing money and there's like the physics of it just don't work out for them to make any money on it. How do you think about your pricing and like how much premium you can get and you can kind of command versus using lower prices as kind of like a wedge into getting there, especially once you have model instrumented? What are the tradeoffs and any thoughts on strategies that work?Erik [00:44:23]: I mean, we focus more on like custom models and custom code. And I think in that space, there's like less competition and I think we can have a pricing markup, right? Like, you know, people will always compare our prices to like, you know, the GPU power they can get elsewhere. And so how big can that markup be? Like it never can be, you know, we can never charge like 10x more, but we can certainly charge a premium. And like, you know, for that reason, like we can have pretty good margins. The LLM space is like the opposite, like the switching cost of LLMs is zero. If all you're doing is like straight up, like at least like open source, right? Like if all you're doing is like, you know, using some, you know, inference endpoint that serves an open source model and, you know, some other provider comes along and like offers a lower price, you're just going to switch, right? So I don't know, to me that reminds me a lot of like all this like 15 minute delivery wars or like, you know, like Uber versus Lyft, you know, and like maybe going back even further, like I think a lot about like sort of, you know, flip side of this is like, it's actually a positive side, which is like, I thought a lot about like fiber optics boom of like 98, 99, like the other day, or like, you know, and also like the overinvestment in GPU today. Like, like, yeah, like, you know, I don't know, like in the end, like, I don't think VCs will have the return they expected, like, you know, in these things, but guess who's going to benefit, like, you know, is the consumers, like someone's like reaping the value of this. And that's, I think an amazing flip side is that, you know, we should be very grateful, the fact that like VCs want to subsidize these things, which is, you know, like you go back to fiber optics, like there was an extreme, like overinvestment in fiber optics network in like 98. And no one made money who did that. But consumers, you know, got tremendous benefits of all the fiber optics cables that were led, you know, throughout the country in the decades after. I feel something similar abou

Deploying Nextcloud the Nix way promises a paradise of reproducibility and simplicity. But is it just a painful trek through configuration hell? We built the dream Nextcloud using Nix and faced reality. Special Guest: Alex Kretzschmar.

This is a recap of the top 10 posts on Hacker News on January 23rd, 2023.This podcast was generated by wondercraft.ai(00:41): Boeing whistleblower: MAX 9 production line has "enormous volume of defects"Original post: https://news.ycombinator.com/item?id=39102021&utm_source=wondercraft_ai(02:51): Why are we templating YAML? (2019)Original post: https://news.ycombinator.com/item?id=39101828&utm_source=wondercraft_ai(04:59): Free Godot engine port for Nintendo SwitchOriginal post: https://news.ycombinator.com/item?id=39102002&utm_source=wondercraft_ai(07:09): Is the emergence of life an expected phase transition in the evolving universe?Original post: https://news.ycombinator.com/item?id=39103419&utm_source=wondercraft_ai(09:28): Mozilla's new Firefox Linux package for Ubuntu and Debian derivativesOriginal post: https://news.ycombinator.com/item?id=39105114&utm_source=wondercraft_ai(11:53): Apple dials back car's self-driving features and delays launch to 2028Original post: https://news.ycombinator.com/item?id=39107854&utm_source=wondercraft_ai(13:50): The humble brilliance of Italy's moka coffee pot (2018)Original post: https://news.ycombinator.com/item?id=39101165&utm_source=wondercraft_ai(15:37): Higher vehicle hoods significantly increase pedestrian deaths, study findsOriginal post: https://news.ycombinator.com/item?id=39106916&utm_source=wondercraft_ai(17:50): Gene therapy allows an 11-year-old boy to hearOriginal post: https://news.ycombinator.com/item?id=39106464&utm_source=wondercraft_ai(20:08): Appeals Court: FBI's Safe-Deposit Box Seizures Violated Fourth AmendmentOriginal post: https://news.ycombinator.com/item?id=39111539&utm_source=wondercraft_aiThis is a third-party project, independent from HN and YC. Text and audio generated using AI, by wondercraft.ai. Create your own studio quality podcast with text as the only input in seconds at app.wondercraft.ai. Issues or feedback? We'd love to hear from you: team@wondercraft.ai

Jason Lengstorf, a developer media producer and host of the show Learn with Jason, joins Corey on this week's episode of Screaming in the Cloud to layout his ideas for creative developer content. Jason explains how devTV can have way more reach than webinars, the lack of inspiration he experiences at conferences these days, and why companies should be focused on hiring specialists before putting DevRels on the payroll. Plus, Corey and Jason discuss walking the line between claiming you're good at everything and not painting yourself into a corner as a DevRel and marketer.About JasonJason Lengstorf helps tech companies connect with developer communities through better media. He advocates for continued learning through collaboration and play and regularly live streams coding with experts on his show, Learn With Jason. He lives in Portland, Oregon.Links Referenced:Learn with Jason: https://www.learnwithjason.dev/Personal Website Links: https://jason.energy/linksTranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. Before I went to re:Invent, I snuck out of the house for a couple of days to GitHub Universe. While I was there, I discovered all kinds of fascinating things. A conference that wasn't predicated on being as cheap as humanly possible was one of them, and a company that understood how developer experience might play out was another.And I also got to meet people I don't normally get to cross paths with. My guest today is just one such person. Jason Lengstorf is a developer media producer at Learn with Jason, which I have to assume is named after yourself.Jason: [laugh] It is yes.Corey: Or it's a dramatic mispronunciation on my part, like, no, no, it's ‘Learn with JSON' and it's basically this insane way of doing weird interchange formats, and you just try to sneak it through because you know I happen to be an XML purist.Jason: [laugh] Right, I'm just going to throw you a bunch of YAML today. That's all I want to talk about.Corey: Exactly. It keeps things entertaining, we're going to play with it. So, let's back up a sec. What do you do? Where do you start and where do you stop?Jason: I'm still learning how to answer this question, but I help companies do a better job of speaking to developer audiences. I was an engineer for a really long time, I went from engineering into developer advocacy and developer experience, and as of the last year, I'm doing that independently, with a big focus on the media that companies produce because I think that what used to work isn't working, and that there's a big opportunity ahead of us that I am really excited to help companies move into.Corey: It feels like this has been an ongoing area of focus for an awful lot of folks. How do you successfully engage with developer audiences? And if I'm being direct and more than a little bit cynical, a big part of it is that historically, the ways that a company marketed to folks was obnoxious. And for better or worse, when you're talking about highly technical topics and you're being loudly incorrect, a technical audience is not beholden to some of the more common business norms, and will absolutely call you out in the middle of you basically lying to them. “Oh, crap, what do we do now,” seemed to be a large approach. And the answer that a lot of folks seem to have come up with was DevRel, which… I've talked about it before in a bunch of different ways, and my one-liner is generally, “If you work in DevRel, that means you work in marketing, but they're scared to tell you that.”Jason: [laugh] I don't think you're wrong. And you know, the joke that I've made for a long time is that they always say that developers hate marketing. But I don't think developers hate marketing; they just hate the way that your company does it. And—Corey: Oh, wholeheartedly agree. Marketing done right is engaging and fun. A lot of what I do in public is marketing. Like, “Well, that's not true. You're just talking about whatever dumb thing AWS did this week.” “Well, yes, but then you stick around to see what else I say, and I just become sort of synonymous with ‘Oh, yeah, that's the guy that fixes AWS bills.'” That is where our business comes from, believe it or not.Jason: Ri—and I think this was sort of the heart of DevRel is that people understood this. They understood that the best way to get an audience engaged is to have somebody who's part of that audience engage with them because you want to talk to them on the level that they work. You're not—you know, a marketing message from somebody who doesn't understand what you do is almost never going to land. It just doesn't feel relatable. But if you talk to somebody who's done the thing that you do for work, and they can tell you a story that's engaging about the thing that you do for work, you want to hear more. You—you know, you're looking for a community, and I think that DevRel, the aim was to sort of create that community and give people a space to hang out with the added bonus of putting the company that employs that DevRel as an adjacent player to get some of that extra shine from wherever this community is doing well.Corey: It felt like 2019 was peak DevRel, and that's where I started to really see that you had, effectively, a lot of community conferences were taken over by DevRel, and you wound up with DevRel pitching to DevRel. And it became so many talks that were aligned with almost imagined problems. I think one of the challenges of working in DevRel is, if you're not careful, you stop being a practitioner for long enough that you can no longer relate to what the audience is actually dealing with. I can sit here and complain about data center travails that I had back in 2011, but are those still accurate in what's about to be 2024? Probably not.Jason: And I think the other problem that happens too is that when you work in DevRel, you are beholden to the company's goals, if the company employees you. And where I think we got really wrong is companies have to make money. We have to charge customers or the company ceases to exist, so when we go out and tell stories, we're encouraged by the company to focus on the stories that have the highest ROI for the company. And that means that I'm up on stage talking about some, like, far-future, large-scale enterprise thing that very few companies need, but most of the paying customers of my company would need. And it becomes less relatable, and I think that leads to some of the collapse that we saw that you mentioned, where dev events feel less like they're for devs and more like they're partner events where DevRel is talking to other DevRel is trying to get opportunities to schmooze partners, and grow our partner pipeline.Corey: That's a big part of it, where it seems, on some level, that so much of what DevRel does, when I see them talking about DevRel, it doesn't get around to DevRel is. Instead, it gets stuck in the weeds of what DevRel is not“. We are not shills for our employer.” Okay, I believe you, but also, I don't ever see you saying anything that directly contravenes what your employer does. Now, let me be clear: neither do I, but I'm also in a position where I can control what my employer does because I have the control to move in directions that align with my beliefs.I'm not saying that it's impossible to be authentic and true to yourself if you work for an employer, but I have seen a couple of egregious examples of people changing companies and then their position on topics they've previously been very vocal on pulled an entire one-eighty, where it's… it really left a bad taste in my mouth.Jason: Yeah. And I think that's sort of the trick of being a career DevRel is you have to sort of walk this line of realizing that a DevRel career is probably short at every company. Because if you're going to go there and be the face of a company, and you're not the owner of that company, they're almost inevitably going to start moving in a direction as business develops, that's not going to line up with your core values. And you can either decide, like, okay that's fine, they pay me well enough, I'm just going to suck it up and do this thing that I don't care about that much, or you have to leave. And so, if you're being honest with yourself, and you know that you're probably going to spend between 12 and 24 months at any given company as a DevRel, which—by the history I'm seeing, that seems to be pretty accurate—you need to be positioning and talking about things in a way that isn't painting you into that corner where you have to completely about-face, if you switch companies. But that also works against your goals as a DevRel at the company. So, it's—I think we've made some big mistakes in the DevRel industry, but I will pause to take a breath here [laugh].Corey: No, no, it's fine. Like, it's weird that I view a lot of what I do is being very similar to DevRel, but I would never call myself that. And part of it is because, for better or worse, it is not a title that tends to engender a level of respect from business owners, decision makers, et cetera because it is such a mixed bag. You have people who have been strategic advisors across the board becoming developer advocates. That's great.You also see people six months out of a boot camp who have decided don't like writing code very much, so they're going to just pivot to talking about writing code, and invariably, they believe, more or less, whatever their employer tells them because they don't have the history and the gravitas to say, “Wait a minute, that sounds like horse pucky to me.” And it's a very broad continuum. I just don't like blending in.Jason: Where I think we got a lot of this wrong is that we never did define what DevRel is. As you say, we mostly define what DevRel is not, and that puts us in a weird position where companies see other companies do DevRel, and they mostly pay attention to the ones who do DevRel really well. And they or their investors or other companies say, “You need a great DevRel program. This is the secret to growth.” Because we look at companies that have done it effectively, and we see their growth, and we say, “Clearly this has a strong correlation. We should invest in this.” But they don't—they haven't done it themselves. They don't understand which part of it is that works, so they just say, “We're hiring for DevRel.” The job description is nine different careers in a trench coat. And the people applying—Corey: Oh, absolutely. It's nine different things and people wind up subdividing into it, like, “I'm an events planner. I'm not a content writer.”Jason: Right.Corey: Okay, great, but then why not bill yourself as a con—as an events planner, and not have to wear the DevRel cloak?Jason: Exactly. And this is sort of what I've seen is that when you put up a DevRel job, they list everything, and then when you apply for a DevRel job, you also don't want to paint yourself into a corner and say, “My specialty is content,” or, “My specialty is public speaking,” or whatever it is. And therefore you say, “I do DevRel,” to give yourself more latitude as an employee. Which obviously I want to keep optionality anywhere I go. I would like to be able to evolve without being painted into a small box of, like, this is all I'm allowed to do, but it does put us in this really precarious position.And what I've noticed a lot of companies do is they hire DevRel—undefined, poorly written job description, poor understanding of the field. They get a DevRel who has a completely different understanding of what DevRel is compared to the people with the role open. Both of them think they're doing DevRel, they completely disagree on what those fundamentals are, and it leads to a mismatch, to burnout, to frustration, to, you know, this high turnover rate in this field. And everybody then starts to say, well, “DevRel is the problem.” But really, the problem is that we're not—we're defining a category, not a job, and I think that's the part that we really screwed up as an industry.Corey: Yeah. I wish there were a better way around there, but I don't know what that might be. Because it requires getting a bunch of people to change some cornerstone of what's become their identity.Jason: This is the part where I—this is probably my spiciest take, but I think that DevRel is marketing, but it is a different kind of marketing. And so, in a perfect world—like, where things start to fall apart is you try to slot DevRel into engineering, or you try to slot it into marketing, as a team on these broader organizations, but the challenge then becomes, if you have DevRel, in marketing, it will inevitably push more toward marketing goals, enterprise goals, top-of-funnel, qualified leads, et cetera. If you put them into engineering, then they have more engineering goals. They want to do developer experience reviews. They want to get out there and do demos. You know, it's much more engineering-focused—or if you're doing it right, is much more engineering-focused.But the best DevRel teams are doing both of those with a really good measure, and really clear metrics that don't line up with engineering or marketing. So, in a perfect world, you would just have an enterprise marketing team, and a developer marketing team, and that developer marketing team would be an organization that is DevRel today. And you would hire specialists—event planners, great speakers, great demo writers, probably put your docs team in there—and treat it as an actual responsibility that requires a larger team than just three or four ex-developers who are now speaking at conferences.Corey: There were massive layoffs across DevRel when the current macroeconomic correction hit, and I'd been worried about it for years in advance because—Jason: Mm-hm.Corey: So, many of these folks spent so much time talking about how they were not marketing, they were absolutely not involved in that. But marketing is the only department that really knows how to describe the value of these sorts of things without having hard metrics tied to it. DevRel spent a lot of time talking about how every metric used to measure them was somehow wrong, and if you took it to its logical conclusion, you would basically give these people a bunch of money—because they are expensive—and about that much money again in annual budget to travel more or less anywhere they want to go, and every time something good happened, as a result, to the company, they had some hand in it nebulously, but you could never do anything to measure their performance, so just trust that they're doing a good job. This is tremendously untenable.Jason: Mm-hm. Yeah, I think when I was running the developer experience org at Netlify, most of my meetings were justifying the existence of the team because there weren't good metrics. You can't put sales qualified leads on DevRel. It doesn't make any sense because there are too many links in the chain after DevRel opens the door, where somebody has to go from, ‘I'm aware of this company' to ‘I've interacted with the landing page' to ‘I've actually signed up for something' to ‘now I'm a customer,' before you can get them to a lead. And so, to have DevRel take credit is actually removing credit from the marketing team.And similarly, if somebody goes through onboarding, a lot of that onboarding can be guided by DevRel. The APIs that new developers interface with can be—the feedback can come from DevRel, but ultimately, the engineering team did that work the product team did that work. So, DevRel is this very interesting thing. I've described it as a turbocharger, where if you put it on an engine that runs well, you get better performance out of that engine. If you just plop one on the table, not a lot happens.Corey: Yeah, it's a good way of putting it. I see very early stage startups looking to hire a developer advocate or DevRel person in their seed stage or Series A, and it's… there's something else you're looking for here. Hire that instead. You're putting the cart before the horse.Jason: What a lot of people saw is they saw—what they're thinking of as DevRel is what they saw from very public founders. And when you get a company that's got this very public-facing, very engaging, charismatic founder, that's what DevRel feels like. It is, you know, this is the face of the company, we're showing you what we do on the inside, we're exposing our process, we're sharing the behind the scenes, and proving to you that we really are great engineers, and we care a lot. Look at all this cool stuff we're doing. And that founder up on stage was, I think, the original DevRel.That's what we used to love about conferences is we would go there and we would see somebody showing this thing they invented, or this new product they had built, and it felt so cool because it was these inspirational moments of watching somebody brilliant do something brilliant. And you got to follow along for that journey. And then we try to—Corey: Yeah I mean, that's natural, but you see booths at conferences, the small company startup booths, a lot of times you'll be able to talk to the founders directly. As the booths get bigger, your likelihood of being able to spend time talking to anyone who's materially involved in the strategic direction of that company gets smaller and smaller. Like, the CEO of GitHub isn't going to be sitting around at the GitHub booth at re:Invent. They're going to be, you know, talking to other folks—if they're there—and going to meetings and whatnot. And then you wind up with this larger and larger company. It's a sign of success, truly, but it also means that you've lost something along the way.Jason: Yeah, I think, you know, it's the perils of scale. And I think that when you start looking at the function of DevRel, it should sort of be looked at as, like, when we can't handle this anymore by ourselves, we should look for a specialty the same way that you do for any other function inside of a company. You know, it wouldn't make sense on day one of a startup to hire a reliability engineer. You're not at the point where that makes sense. It's a very expensive person to hire, and you don't have enough product or community or load to justify that role yet. And hopefully, you will.And I think DevRel is sort of the same way. Like, when you first start out your company, your DevRel should be the founding team. It should be your engineers, sharing the things that they're building so that the community can see the brilliance of your engineering team, sharing with the community, obviously, being invested in that community. And when you get big enough that those folks can no longer manage that and their day-to-day work, great, then look into adding specialists. But I think you're right that it's cart before the horse to, you know, make a DevRel your day-one hire. You just don't have enough yet.Corey: Yeah, I wish that there were an easy way to skin the cat. I'm not sure there is. I think instead we wind up with people doing what they think is going to work. But I don't know what the truth is.Jason: Mmm.Corey: At least. That's where I land on it.Jason: [laugh] Yeah, I mean, every company is unique, and every experience is going to be unique, so I think to say, “Do it exactly like this,” is—that's got a lot of survivorship bias, and do as I say—but at the same time, I do think there's some universal truths. Like, it doesn't really make sense to hire a specialist before you've proven that specialty is the secret sauce of your business. And I think you grow when it's time to grow, not just in case. I think companies that over-hire end up doing some pretty painful layoffs down the road. And, you know, obviously, there's an opposite end of that spectrum where you can grow too slowly and bury your team and burn everybody out, but I think, you know—we, [laugh] leading into the pandemic, I guess, we had a lot of free money, and I think people were thinking, let's go build an empire and we'll grow into that empire. And I think that is a lot of why we're seeing this really painful downsizing right now, is companies hired just in case and then realized that actually, that in case didn't come to be.Corey: What is the future of this look like? Easy enough to look back and say, well, that didn't work? Well, sure. What is the future?Jason: The playbook that we saw before—in, like, 2019 and before—was very event-driven, very, like, webinar-driven. And as we went into 2020, and people were at home, we couldn't travel, we got real sick of Zoom calls. We don't want to get on another video call again. And that led to that playbook not working anymore. You know, I don't want to get on a webinar with a company. I don't want to go travel to a company event, you know, or at least not very many of them. I want to go see the friends I haven't seen in three years.So, travel priorities changed, video call fatigue is huge, so we need something that people want to do, that is interesting, and that is, you know, it's worth making in its own right, so that people will engage with it, and then you work in the company goals as an incidental. Not as a minor incidental, but you know, it's got to be part of the story; it can't be the purpose. People won't sign up for a webinar willingly these days, I don't think, unless they have exactly the problem that your webinar purports to solve.Corey: And even if they do, it becomes a different story.Jason: Right.Corey: It's [high buying 00:19:03] signal, but people are constantly besieged by requests for attention. This is complicated by what I've seen over the last year. When marketing budgets get—cut, arguably too much, but okay—you see now that there's this follow-on approach where, okay, what are we going to cut? And people cut things that in many cases work, but are harder to attribute success to. Events, for example, are doing very well because you have someone show up at your booth, you scan their badge. Three weeks later, someone from that company winds up signing up for a trial or whatnot, and ah, I can connect those dots.Whereas you advertise on I don't know, a podcast as a hypothetical example that I'm pulling out of what's right in front of me, and someone listening to this and hearing a message from a sponsor, they might be doing something else. They'll be driving, washing dishes, et cetera, and at best they'll think, “Okay, I should Google that when I get back to a computer.” And they start hearing about it a few times, and, “Oh. Okay, now it's time for me to go and start paying serious attention to this because that sounds like it aligns with a problem I have.” They're not going to remember where they initially heard it.They're going to come in off of a Google search, so it sounds like it's all SEO's benefit that this is working, and it is impossible to attribute. I heard some marketer once say that 50% of your marketing budget is wasted, but you'll go bankrupt trying to figure out which half. It all ties together. But I can definitely see why people bias for things that are more easily attributed to the metric you care about.Jason: Yes. And I think that this is where I see the biggest opportunity because I think that we have to embrace that marketing signal is directional, not directly attributable. And if you have a focus campaign, you can see your deviation from baseline signups, and general awareness, and all of the things that you want to be true, but you have to be measuring that thing, right? So, if we launch a campaign where we're going to do some video ads, or we're going to do some other kind of awareness thing, the goal is brand awareness, and you measure that through, like, does your name get mentioned on social media? Do you see a deviation from baseline signups where it is trending upward?And each of those things is signal that the thing you did worked. Can you directly attribute it? No, but I think a functional team can—you know, we did this at Netlify all the time where we would go and look: what were the efforts that were made, what were the ones that got discussion on different social media platforms, and what was the change from baseline? And we saw certain things always drove a non-trivial deviation from baseline in the right direction. And that's one of the reasons that I think the future of this is going to be around how do you go broader with your reach?And my big idea—to nutshell it—is, like, dev TV. I think that developers want to see the things that they're interested in, but they want it to be more interesting than a straight webinar. They want to see other developers using tools and getting a sense of what's possible in an entertaining way. Like, they want stories, they don't want straight demos. So, my thinking here is, let's take this and steer into it.Like, we know that developers love when you put a documentary together. We saw the Vue documentary, and the React documentary, and the GraphQL documentary, and the Kubernetes documentary coming out of the Honeypot team, and they've got hundreds of thousands, and in some cases, millions of views because developers really want to see good stories about us, about our community. So, why not give the dev community a Great British Bake Off, but for web devs? Why not create an Anthony Bourdain Parts Unknown-style travel show that highlights various web communities? Why not get out there and make reality competition shows and little docuseries that help us highlight all the things that we're learning and sharing and building?Every single one of those is going to involve developers talking about the tools they use, talking about the problems they solve, talking about what they were doing before and how they've made it better. That's exactly what a webinar is, that's what a conference talk is, but instead of getting a small audience at a conference, or you know, 15 to 30 people signing up for your webinar, now we've got the potential for hundreds of thousands or even millions of people to watch this thing because it's fun to watch. And then they become aware of the companies involved because it's presented by the company; they see the thing get used or talked about by developers in their community, I think there's a lot of magic and potential in that, and we've seen it work in other verticals.Corey: And part of the problem comes down as well to the idea that, okay, you're going to reach some people in person at events, but the majority of engineers are not going to be at any event or—Jason: Right.Corey: Any event at all, for that matter. They just don't go to events for a variety of excellent reasons. How do you reach out to them? Video can work, but I always find that requires a bit of a different skill than, I don't know, podcasting or writing a newsletter. So, many times, it feels like it's, oh, and now you're just going to basically stare at the camera, maybe with someone else, and it looks like the Zoom call to which the viewer is not invited.Jason: Right.Corey: They get enough of that. There has to be something else.Jason: And I think this is where the new skill set, I think, is going to come in. It exists in other places. We see this happen in a lot of other industries, where they have in-house production teams, they're doing collaborations with actors and athletes and bringing people in to make really entertaining stories that drive underlying narratives. I mean, there's the ones that are really obvious, like, the Nikes of the world, but then there are far less obvious examples.Like, there was this show called Making It. It was… Nick Offerman and Amy Poehler were the hosts. It was the same format as the Great British Bake Off but around DIY and crafting. And one of the permanent judges was the Etsy trend expert, right? And so, every single episode, as they're judging this, the Etsy trend expert is telling all of these crafters and contestants, “You know, what you built here is always a top seller on Etsy. This is such a good idea, it's so well executed, and people love this stuff. It flies off the shelves in Etsy stores.”Every single episode, just perfectly natural product placement, where a celebrity that you know—Nick Offerman and Amy Poehler—are up there, lending—like, you want to see them. They're so funny and engaging, and then you've got the credibility of Etsy's trend expert telling the contestants of the show, “If you do DIY and crafting, you can make a great living on Etsy. Here are the things that will make that possible.” It's such subtle, but brilliant product placement throughout the entire thing. We can do that. Like, we have the money, we just spend it in weird places.And I think that as an industry, if we start getting more creative about this and thinking about different ways we can apply these marketing dollars that we're currently dumping into very expensive partner dinners or billboards or getting, you know, custom swag or funding yet another $150,000 conference sponsorship, we could make a series of a TV show for the same cost as throwing one community event, and we would reach a significantly larger group.Corey: Yeah. Now, there is the other side of it, too, where Lord knows I found this one out the fun way, that creating content requires significant effort and—Jason: Yes.Corey: Focus. And, “Oh, it's a five-minute video. Great, that could take a day or three to wind up putting together, done right.” One of the hardest weeks of my year is putting together a bunch of five-minute videos throughout the course of re:Invent. So much that is done in advance that is basically breaking the backs of the editing team, who are phenomenal, but it still turns into more than that, where you still have this other piece of it of the actual content creation part.And you can't spend all your time on that because pretty soon I feel like you become a talking head who doesn't really do the things that you are talking to the world about. And that content gets pretty easy to see when you start looking at, okay, what did someone actually do? Oh, they were a developer for three years, and they spent the next seven complaining about development, and how everyone is—Jason: [laugh].Corey: Doing it wrong on YouTube. Hmm… it starts to get a little, how accurate is this really? So, for me, it was always critical that I still be hands-on with things that I'm talking about because otherwise I become a disaster.Jason: And I agree. One of the things that my predecessor at Netlify, Sarah Drasner, put in place was a, what she called an exchange program, where we would rotate the DevRel team onto product, and we rotate product onto the DevRel team. And it was a way of keeping the developer experience engineers actually engineers. They would work on the product, they didn't do any DevRel work, they were exclusively focused on doing actual engineering work inside our product to just help keep their skills sharp, keep them up to date on what's going on, build more empathy for the engineers that we talk to every day, build more empathy for our team instead of us—you know, you never want to hear a DevRel throw the engineering team under the bus for not shipping a feature everybody wants.So, these sorts of things are really important, and they're hard to do because we had to—you know, that's a lot of negotiation to say, “Hey, can we take one of your engineers for a quarter, and we'll give you one of our engineers for a quarter, and you got to trust us that's going to work out in your favor.” [laugh] Right? Like, there's a lot that goes into this to make that sort of stuff possible. But I absolutely agree. I don't think you get to make this type of content if you've fully stepped out of engineering. You have to keep it part of your practice.Corey: There's no way around it. You have to be hands-on. I think that's the right way to do it, otherwise, it just leads to, frankly, disaster. Very often, you'll see people who are, like, “Oh, they're great in the DevRel space. What do they do?” And they go to two or three conferences a year, and they have a blog post or so. It's like, okay, what are they doing the rest of that time?Sometimes the answer is fighting internal political fires. Other times it's building things and learning these things and figuring out where they stand. There are some people, I don't want to name names, although an easy one is Kelsey Hightower, who has since really left the stage, that he's retired, but when he went up on stage and said something—despite the fact that he worked at Google—it was eminently clear that he believed in what he was saying, or he would not say it.Jason: Right.Corey: He was someone who was very clearly aware of the technology about which he was speaking. And that was great. I wish that it were not such a standout moment to see him speak and talk about that. But unfortunately, he kind of is. Not as many people do that as well as we'd like.Jason: Agreed. I think it was always a treat to see Kelsey speak. And there are several others that I can think of in the community who, when they get on stage, you want to be in that audience, and you want to sit down and listen. And then there are a lot of others who when they get on stage, it's like that this book could have been a blog post, or this—you know, this could have been an email, that kind of thing. Like you could have sent me this repo because all you did was walk through this repo line-by-line, or something that—it doesn't feel like it came from them; it feels like it's being communicated by them.And I think that's, again, like, when I criticize conferences, a lot of my criticism comes from the fact that, coming up, I feel like every speaker that I saw on stage—and this is maybe just memory… playing favorites for me, but I feel like I saw a lot of people on stage who were genuinely passionate about what they were creating, and they were genuinely putting something new into the world every time they got on stage. And I have noticed that I feel less and less like that. Also, I feel like events have gotten less and less likely to put somebody on stage unless they've got a big name DevRel title. Like, you have to work at a company that somebody's heard of because they're all trying to get that draw because attendance is going down. And—Corey: Right. It's a—like, having run some conferences myself, the trick is, is you definitely want some ringers in there. People you know will do well, but you also need to give space for new voices to arise. And sometimes it's a—it always bugs me when it seems like, oh, they're here because their company is a big sponsor. Of course, they have the keynote. Other times, it's a… like, hate the actual shill talks, which I don't see as much, which I'm thankful for; I'd stop going to those conferences, but jeez.Jason: Yeah, and I think it's definitely one of those, like, this is a thing that we can choose to correct. And I have a suspicion that this is a pendulum not a—not, like, the denouement of—is that the right—how do you say that word? De-NOW-ment? De-NEW-ment? Whatever.Corey: Denouement is my understanding, but that might be the French acc—Jason: Oh, me just—Corey: The French element.Jason: —absolutely butchering that. Yeah [laugh]. I don't think this is the end of conferences, like we're seeing them taper into oblivion. I think this is a lull. I think that we're going to realize that we want to—we really do love being in a place with other developers. I want to do that. I love that.But we need to get back to why we were excited to go to conferences in the first place, which was this sharing of knowledge and inspiration, where you would go see people who were literally moving the world forward in development, and creating new things so that you would walk away with insider info, you had just seen the new thing, up close and personal, had those conversations, and you went back so jazzed to build something new. I feel like these days, I feel more like I went and watched a handful of product demos, and now I'm really just waiting to the hallway track, which is the only, like, actually interesting part at a lot of events these days.Corey: I really want to thank you for taking the time to speak with me. If people want to learn more, where's the best place for them to find you?Jason: Most of what I share is on learnwithjason.dev, or if you want a big list of links, I have jason.energy/links, which has a whole bunch of fun stuff for you to find.Corey: Awesome. And we will, of course, include links to that in the show notes. Thank you so much for taking the time to speak with me. I really appreciate it.Jason: Yeah, thanks so much for having me. This was a blast.Corey: Jason Lengstorf, developer media producer at Learn with Jason. I'm Cloud Economist Corey Quinn, and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice, along with an angry comment that will no doubt become the basis for somebody's conference talk.Jason: [laugh].Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business, and we get to the point. Visit duckbillgroup.com to get started.

Fri, 05 Jan 2024 22:30:00 GMT http://relay.fm/rd/225 http://relay.fm/rd/225 Leavings and Learnings 225 Merlin Mann and John Siracusa On the complexities and frustrations around seeking medical help. Especially when it involves more than just finding an egg. On the complexities and frustrations around seeking medical help. Especially when it involves more than just finding an egg. clean 7029 Subtitle: Merlin has a had a lot of hospital.On the complexities and frustrations around seeking medical help. Especially when it involves more than just finding an egg. This episode of Reconcilable Differences is sponsored by: Squarespace: Save 10% off your first purchase of a website or domain using code DIFFS. Links and Show Notes: Things kick off with some TV talk and a consideration of sane and civil ways to watch TV with a partner. In Follow-Up, there's more on this whole chapter markers issue, but honestly it's all a little difficult to follow because of the way time works chronologically. Merlin has been doing a lot of stuff with computers, and John reluctantly admits that at least JSON is better than YAML, but that's not really saying very much. As a main topic, your hosts talk about the complexities and frustrations around seeking medical help—especially when the medical help one needs falls outside the interest, curiosity, or expertise of a given medical professional. And especially when the help you need concerns your brain and doesn't just involve finding an egg and calling it a day. (Recorded on Tuesday, December 19, 2023) Credits Audio Editor: Jim Metzendorf Admin Assistance: Kerry Provenzano Music: Merlin Mann The Suits: Stephen Hackett, Myke Hurley Members-Only Aftershow Timestamps Episode 134: The aftershow starts at 1:20:16 (1 hour, 20 minutes, 16 seconds). Episode 135: The aftershow starts at 1:13:41 (1 hour, 13 minutes, 41 seconds). Episode 136: The aftershow starts at 1:13:10 (1 hour, 13 minutes, 10 seconds). Episode 137: The aftershow starts at 1:29:29 (1 hour, 29 minutes, 29 seconds). Episode 138: The aftershow starts at 1:20:27 (1 hour, 20 minutes, 27 seconds). Episode 140: The aftershow starts at 1:10:03 (1 hour, 10 minutes, 3 seconds). Episode 142: The aftershow starts at 1:08:02 (1 hour, 8 minutes, 2 seconds). Episode 144: The aftershow starts at 1:30:30 (1 hour, 30 minutes, 30 seconds). Episode 144: The aftershow starts at 1:30:30 (1 hour, 30 minutes, 30 seconds). Episode 146: The aftershow starts at 1:24:49 (1 hour, 24 minutes, 49 seconds). Episode 148: The aftershow starts at 1:40:57 (1 hour, 40 minutes, 57 seconds). Episode 148: The aftershow starts at 1:40:57 (1 hour, 40 minutes, 57 seconds). Episode 150: The aftershow starts at 1:16:22 (1 hour, 16 minutes, 22 seconds). Episode 152: The aftershow starts at 1:23:14 (1 hour, 23 minutes, 14 seconds). Episode 154: The aftershow starts at 1:07:32 (1 hour, 7 minutes, 32 seconds). Episode 156: The aftershow starts at 1:36:12 (1 hour, 36 minutes, 12 seconds). Episode 158: The aftershow starts at 1:21:27 (1 hour, 21 minutes, 27 seconds). Episode 160: The aftershow starts at 1:38:59 (1 hour, 38 minutes, 59 seconds). Episode 162: The aftershow starts at 1:34:02 (1 hour, 34 minutes, 2 seconds). Episode 164: The aftershow starts at 1:31:32 (1 hour, 31 minutes, 32 seconds). Episode 166: The aftershow starts at 1:11:54 (1 hour, 11 minutes, 54 seconds). Episode 168: The aftershow starts at 1:34:11 (1 hour, 34 minutes, 11 seconds). Episode 170: The aftershow starts at 1:42:43 (1 hour, 42 minutes, 43 seconds). Episode 172: The aftershow starts at 1:48:33 (1 hour, 48 minutes, 33 seconds). Episode 174: The aftershow starts at 1:18:53 (1 hour, 18 minutes, 53 seconds). Episode 176: The aftershow starts at 1:32:25 (1 hour, 32 minutes, 25 seconds). Episode 178: The aftershow starts at 1:20:18 (1 hour, 20 minutes, 18 seconds). Episode 180: The aftershow starts at 1:34:02 (1 hour, 34 minutes, 2 seconds). Episode 182: The aftershow starts at 1:35:00 (1 hour, 35 minutes, 0 seconds). Episode 184: The aftershow starts at 1:40:54 (1 hour, 40 minutes, 54 seconds). Episode 186: The aftershow starts at 1:22:35 (1 hour, 22 minutes, 35 seconds). Episode 188: The aftershow starts at 1:01:06 (1 hour, 1 minute, 6 seconds). Episode 190: The aftershow starts at 1:18:00 (1 hour, 18 minutes, 0 seconds). Episode 192: The aftershow starts at 1:24:28 (1 hour, 24 minutes, 28 seconds). Episode 194: The aftershow starts at 1:25:19 (1 hour, 25 minutes, 19 seconds). Episode 196: The aftershow starts at 1:35:59 (1 hour, 35 minutes, 59 seconds). Episode 198: The aftershow starts at 1:26:38 (1 hour, 26 minutes, 38 seconds). Episode 200: The aftershow starts at 1:09:51 (1 hour, 9 minutes, 51 seconds). Episode 202: The aftershow starts at 1:47:29 (1 hour, 47 minutes, 29 seconds). Episode 204: The aftershow starts at 1:25:23 (1 hour, 25 minutes, 23 seconds). Episode 206: The aftershow starts at 1:32:32 (1 hour, 32 minutes, 32 seconds). Episode 208: The aftershow starts at 1:31:40 (1 hour, 31 minutes, 40 seconds). Episode 210: The aftershow starts at 1:30:06 (1 hour, 30 minutes, 6 seconds). Episode 212: The aftershow starts at 1:32:13 (1 hour, 32 minutes, 13 seconds). Episode 214: The aftershow starts at 1:20:37 (1 hour, 20 minutes, 37 seconds). Episode 216: The aftershow starts at 1:15:05 (1 hour, 15 minutes, 5 seconds). Episode 218: The aftershow starts at 1:12:35 (1 hour, 12 minutes, 35 seconds). Episode 220: The aftershow starts at 1:17:51 (1 hour, 17 minutes, 51 seconds). Episode 222: The aftershow starts at 1:32:29 (1 hour, 32 minutes, 29 seconds). Get an ad-free version of the show, plus a monthly extended episode. Mr. Show - "Pre-Taped Call-in Show" - YouTube Quote Origin: “The Pun Is the Lowest Form of Wit” “Yes, That Means It Is the Foundation of All Wit” – Quote Investigator® Paul Karason - WikipediaPaul Karason (November 14, 1950 – September 23, 2013) was an American from Bellingham, Washington, whose skin was a purple-blue color. Love Has Won (HBO) Escaping Twin Flames (Netflix) For All Mankind (Apple TV+) George Lucas: “They Rhyme” - YouTube How George Lucas's two Star Wars trilogies call out to each other visually SciPy SciPy signal correlate method The House of J.F. Sebastian Health Record Export iOS app Disney's new robot - YouTube Godzilla (2014) Why'd I take speed for twenty years? Why'd I take speed for twenty years? (Part 2) None of us is as dumb as all of us Pacific Rim review by Merlin MannYou people are gonna feel like such dingalings when you finally realize and accept that this is one of the great films. Luminous: Is it the drug or is it the trip?For all the talk about how psychedelics might transform psychiatric care, there's still a fascinating question at the heart of psychedelic science. Is it the mind-blowing experience that fundamentally changes a person's outlook on life? Or is it the powerful molecules that rewire the brain? "Theodoric of York: Medieval Barber" Bethlem Royal Hosp

Fri, 05 Jan 2024 22:30:00 GMT http://relay.fm/rd/225 http://relay.fm/rd/225 Merlin Mann and John Siracusa On the complexities and frustrations around seeking medical help. Especially when it involves more than just finding an egg. On the complexities and frustrations around seeking medical help. Especially when it involves more than just finding an egg. clean 7029 Subtitle: Merlin has a had a lot of hospital.On the complexities and frustrations around seeking medical help. Especially when it involves more than just finding an egg. This episode of Reconcilable Differences is sponsored by: Squarespace: Save 10% off your first purchase of a website or domain using code DIFFS. Links and Show Notes: Things kick off with some TV talk and a consideration of sane and civil ways to watch TV with a partner. In Follow-Up, there's more on this whole chapter markers issue, but honestly it's all a little difficult to follow because of the way time works chronologically. Merlin has been doing a lot of stuff with computers, and John reluctantly admits that at least JSON is better than YAML, but that's not really saying very much. As a main topic, your hosts talk about the complexities and frustrations around seeking medical help—especially when the medical help one needs falls outside the interest, curiosity, or expertise of a given medical professional. And especially when the help you need concerns your brain and doesn't just involve finding an egg and calling it a day. (Recorded on Tuesday, December 19, 2023) Credits Audio Editor: Jim Metzendorf Admin Assistance: Kerry Provenzano Music: Merlin Mann The Suits: Stephen Hackett, Myke Hurley Members-Only Aftershow Timestamps Episode 134: The aftershow starts at 1:20:16 (1 hour, 20 minutes, 16 seconds). Episode 135: The aftershow starts at 1:13:41 (1 hour, 13 minutes, 41 seconds). Episode 136: The aftershow starts at 1:13:10 (1 hour, 13 minutes, 10 seconds). Episode 137: The aftershow starts at 1:29:29 (1 hour, 29 minutes, 29 seconds). Episode 138: The aftershow starts at 1:20:27 (1 hour, 20 minutes, 27 seconds). Episode 140: The aftershow starts at 1:10:03 (1 hour, 10 minutes, 3 seconds). Episode 142: The aftershow starts at 1:08:02 (1 hour, 8 minutes, 2 seconds). Episode 144: The aftershow starts at 1:30:30 (1 hour, 30 minutes, 30 seconds). Episode 144: The aftershow starts at 1:30:30 (1 hour, 30 minutes, 30 seconds). Episode 146: The aftershow starts at 1:24:49 (1 hour, 24 minutes, 49 seconds). Episode 148: The aftershow starts at 1:40:57 (1 hour, 40 minutes, 57 seconds). Episode 148: The aftershow starts at 1:40:57 (1 hour, 40 minutes, 57 seconds). Episode 150: The aftershow starts at 1:16:22 (1 hour, 16 minutes, 22 seconds). Episode 152: The aftershow starts at 1:23:14 (1 hour, 23 minutes, 14 seconds). Episode 154: The aftershow starts at 1:07:32 (1 hour, 7 minutes, 32 seconds). Episode 156: The aftershow starts at 1:36:12 (1 hour, 36 minutes, 12 seconds). Episode 158: The aftershow starts at 1:21:27 (1 hour, 21 minutes, 27 seconds). Episode 160: The aftershow starts at 1:38:59 (1 hour, 38 minutes, 59 seconds). Episode 162: The aftershow starts at 1:34:02 (1 hour, 34 minutes, 2 seconds). Episode 164: The aftershow starts at 1:31:32 (1 hour, 31 minutes, 32 seconds). Episode 166: The aftershow starts at 1:11:54 (1 hour, 11 minutes, 54 seconds). Episode 168: The aftershow starts at 1:34:11 (1 hour, 34 minutes, 11 seconds). Episode 170: The aftershow starts at 1:42:43 (1 hour, 42 minutes, 43 seconds). Episode 172: The aftershow starts at 1:48:33 (1 hour, 48 minutes, 33 seconds). Episode 174: The aftershow starts at 1:18:53 (1 hour, 18 minutes, 53 seconds). Episode 176: The aftershow starts at 1:32:25 (1 hour, 32 minutes, 25 seconds). Episode 178: The aftershow starts at 1:20:18 (1 hour, 20 minutes, 18 seconds). Episode 180: The aftershow starts at 1:34:02 (1 hour, 34 minutes, 2 seconds). Episode 182: The aftershow starts at 1:35:00 (1 hour, 35 minutes, 0 seconds). Episode 184: The aftershow starts at 1:40:54 (1 hour, 40 minutes, 54 seconds). Episode 186: The aftershow starts at 1:22:35 (1 hour, 22 minutes, 35 seconds). Episode 188: The aftershow starts at 1:01:06 (1 hour, 1 minute, 6 seconds). Episode 190: The aftershow starts at 1:18:00 (1 hour, 18 minutes, 0 seconds). Episode 192: The aftershow starts at 1:24:28 (1 hour, 24 minutes, 28 seconds). Episode 194: The aftershow starts at 1:25:19 (1 hour, 25 minutes, 19 seconds). Episode 196: The aftershow starts at 1:35:59 (1 hour, 35 minutes, 59 seconds). Episode 198: The aftershow starts at 1:26:38 (1 hour, 26 minutes, 38 seconds). Episode 200: The aftershow starts at 1:09:51 (1 hour, 9 minutes, 51 seconds). Episode 202: The aftershow starts at 1:47:29 (1 hour, 47 minutes, 29 seconds). Episode 204: The aftershow starts at 1:25:23 (1 hour, 25 minutes, 23 seconds). Episode 206: The aftershow starts at 1:32:32 (1 hour, 32 minutes, 32 seconds). Episode 208: The aftershow starts at 1:31:40 (1 hour, 31 minutes, 40 seconds). Episode 210: The aftershow starts at 1:30:06 (1 hour, 30 minutes, 6 seconds). Episode 212: The aftershow starts at 1:32:13 (1 hour, 32 minutes, 13 seconds). Episode 214: The aftershow starts at 1:20:37 (1 hour, 20 minutes, 37 seconds). Episode 216: The aftershow starts at 1:15:05 (1 hour, 15 minutes, 5 seconds). Episode 218: The aftershow starts at 1:12:35 (1 hour, 12 minutes, 35 seconds). Episode 220: The aftershow starts at 1:17:51 (1 hour, 17 minutes, 51 seconds). Episode 222: The aftershow starts at 1:32:29 (1 hour, 32 minutes, 29 seconds). Get an ad-free version of the show, plus a monthly extended episode. Mr. Show - "Pre-Taped Call-in Show" - YouTube Quote Origin: “The Pun Is the Lowest Form of Wit” “Yes, That Means It Is the Foundation of All Wit” – Quote Investigator® Paul Karason - WikipediaPaul Karason (November 14, 1950 – September 23, 2013) was an American from Bellingham, Washington, whose skin was a purple-blue color. Love Has Won (HBO) Escaping Twin Flames (Netflix) For All Mankind (Apple TV+) George Lucas: “They Rhyme” - YouTube How George Lucas's two Star Wars trilogies call out to each other visually SciPy SciPy signal correlate method The House of J.F. Sebastian Health Record Export iOS app Disney's new robot - YouTube Godzilla (2014) Why'd I take speed for twenty years? Why'd I take speed for twenty years? (Part 2) None of us is as dumb as all of us Pacific Rim review by Merlin MannYou people are gonna feel like such dingalings when you finally realize and accept that this is one of the great films. Luminous: Is it the drug or is it the trip?For all the talk about how psychedelics might transform psychiatric care, there's still a fascinating question at the heart of psychedelic science. Is it the mind-blowing experience that fundamentally changes a person's outlook on life? Or is it the powerful molecules that rewire the brain? "Theodoric of York: Medieval Barber"

The Latent Space crew will be at NeurIPS on Tuesday! Reach out with any parties and papers of interest. We have also been incubating a smol daily AI Newsletter and Latent Space University is making progress.Good open models like Llama 2 and Mistral 7B (which has just released an 8x7B MoE model) have enabled their own sub-industry of finetuned variants for a myriad of reasons:* Ownership & Control - you take responsibility for serving the models* Privacy - not having to send data to a third party vendor* Customization - Improving some attribute (censorship, multiturn chat and chain of thought, roleplaying) or benchmark performance (without cheating)Related to improving benchmark performance is the ability to use smaller (7B, 13B) models, by matching the performance of larger models, which have both cost and inference latency benefits.Core to all this work is finetuning, and the emergent finetuning library of choice has been Wing Lian's Axolotl.AxolotlAxolotl is an LLM fine-tuner supporting SotA techniques and optimizations for a variety of common model architectures:It is used by many of the leading open source models:* Teknium: OpenHermes, Trismigestus, CollectiveCognition* OpenOrca: Mistral-OpenOrca, Mistral-SlimOrca* Nous Research: Puffin, Capybara, NousHermes* Pygmalion: Mythalion, Pygmalion* Eric Hartford: Dolphin, Samantha* DiscoResearch: DiscoLM 120B & 70B* OpenAccess AI Collective: Manticore, Minotaur, Jackalope, HippogriffAs finetuning is very formatting dependent, it also provides prompt interfaces and formatters between a range of popular model formats from Stanford's Alpaca and Steven Tey's ShareGPT (which led to Vicuna) to the more NSFW Pygmalion community.Nous Research MeetupWe last talked about Nous at the DevDay Recap at the e/acc “banger rave”. We met Wing at the Nous Research meetup at the a16z offices in San Francisco, where they officially announced their company and future plans:Including Nous Forge:Show NotesWe've already covered the nuances of Dataset Contamination and the problems with “Open Source” in AI, so we won't rehash those topics here but do read/listen to those if you missed it.* Axolotl GitHub and Discord* The Flan paper and dataset* StackLlama model and blogpost* Multipack paper* Our episode with Tri Dao* Mamba state space models - Tri Dao and Albert GuTimestamps* [00:00:00] Introducing Wing* [00:02:34] SF Open Source AI Meetup* [00:04:09] What is Axolotl?* [00:08:01] What is finetuning?* [00:08:52] Open Source Model Zoo* [00:10:53] Benchmarks and Contamination* [00:14:29] The Case for Open Source AI* [00:17:34] Orca and OpenOrca* [00:23:36] DiscoLM and Model Stacking* [00:25:07] Datasets and Evals over Models* [00:29:15] Distilling from GPT4* [00:33:31] Finetuning - LoRA, QLoRA, ReLoRA, GPTQ* [00:41:55] Axolotl vs HF Transformers* [00:48:00] 20x efficiency with StackLlama and Multipack* [00:54:47] Tri Dao and Mamba* [00:59:08] Roadmap for Axolotl* [01:01:20] The Open Source AI CommunityTranscript[00:00:00] Introducing Wing Lian[00:00:00] ​[00:00:00] swyx: Welcome to Latent Space, a special edition with Wing Lien, but also with our new guest host, Alex. Hello, hello. Welcome, welcome. Again, needs no introduction. I think it's like your sixth time on Latent Space already. I think so, yeah. And welcome, Wing. We just met, but you've been very prolific online. Thanks for having me.[00:00:30] Yeah. So you are in town. You're not local. You're in town. You're from Minneapolis?[00:00:35] Wing Lian: Annapolis. Annapolis. It's funny because a lot of people think it's Indianapolis. It's I've got Minneapolis, but I used to live out at least in the San Francisco Bay Area years ago from like 2008 to 2014. So it's fairly familiar here.[00:00:50] swyx: Yep. You're the maintainer of Axolotl now, which we'll get into. You're very, very prolific in the open source AI community, and you're also the founder of the Open Access AI Collective. Yeah. Cool. Awesome. Maybe we can go over a little bit of your backgrounds into tech and then coming into AI, and then we'll cover what[00:01:06] Wing Lian: happens and why you're here.[00:01:08] Yeah. So. Back on tech, so I started years ago, I started way back when I was scraping, Apartment websites for listings and then, and then building like SEO optimized pages and then just throwing Google AdSense on it.[00:01:24] And that got me through like college basically. Is[00:01:27] swyx: that decent money? And what year[00:01:28] Wing Lian: was this? Like 2004, 2005. Yeah, that's decent money. It's like thousand bucks a month. But as a college student, that's like. Gravy. Really good money, right? So, and then there's just too much competition It's just sort of like died off. I was writing stuff in like Perl back then using like like who nobody hosted anything on Perl anymore, right? Still did a little bit more like computer tech support and then software, and web more professionally.[00:01:54] So I spent some time working on applications in the blood industry. I came out to San Francisco for, I was at SGN, so Social Gaming Network, as a startup. They started doing, with Facebook apps, and then they pivoted into doing mobile apps. And then, from there, I spent time.[00:02:14] I've quite a few more startups since then and in the last few years I've been in the music space So like I was at United Masters for a while and then past year I've been at SoundCloud, but not doing that anymore and now that I have a lot more time It's just like all right.[00:02:30] We're going full bore on axolotl and we're gonna we're gonna crush AI So yeah,[00:02:34] SF Open Source AI Meetup[00:02:34] swyx: totally you so you're here in town for the open source. Yeah, I meet up that we had yesterday Yep, yeah, that was amazing. Yeah, it was a big collection. Olama, Noose Research, Alignment Lab, Anyone else that I missed? I mean, Jeremy Howard is his own thing.[00:02:47] Yeah.[00:02:49] And Alex, you're also there. You love to bring SF to the world. Your takes?[00:02:55] Alex Volkov: It's incredible that we recorded a Thursday Eye episode after that one. And LDJ, who's usually co hosts Thursday Eye, just like briefly mentioned, Oh yeah, I talked about it.[00:03:04] Like, I saw Karpathy, and then I talked to Jeremy Howard, and the guy from Mistral came in, and it's like, He's talking about all these, titans of industry, basically, that outside of SF, You just don't meet casually hanging out in the same space. You can't, pull somebody. He ran into the Laylow from Mistral, he ran into him while, drinking water.[00:03:20] He didn't even know he was there. It's just, that type of stuff is really hard to find outside of SF. So, absolutely, absolutely great. And also, presentations from Alignment Labs, presentations from News Research, news issues, talked about. Forge, and some of[00:03:33] swyx: the other stuff they announced. We can say now they're officially a company.[00:03:36] I met Technium.[00:03:37] He[00:03:37] Alex Volkov: came over here. He didn't want to get recorded. But maybe.[00:03:41] Wing Lian: We'll wear him down at some point. Yeah, I'm excited for Forge. They've positioned it as this agentic sort of framework where it's just Drag and drop things and, fill in text with where you want to inject different variables and it opens up all of these potentials for data pipelines now, right?[00:03:56] And using your own local LLMs and not relying on GPT 4 or anything like that. Yeah, yeah,[00:04:02] swyx: good stuff. Okay, so let's maybe go into the Axolotl origin story and then we have, we have some intro or background.[00:04:09] What is Axolotl?[00:04:09] swyx: To do on like the open source model universe and also on fine tuning, but maybe just, since you're talking about your personal journey, what was your personal journey into[00:04:18] Wing Lian: axolotl?[00:04:19] Yeah, so my personal journey started like back in mid March, completely unrelated to AI and axolotl. And it really started, I fell while skiing, I torqued. Great 3 MCL sprain and being sort of like an active person that can no longer be active because the two, couldn't play soccer, because that is requires to have having knees until I, it's healed.[00:04:42] So I. I decided I needed to find something to do to take up my free time. And that became, well, let's learn how to train in, these language models. It was everywhere. So I was like, all right, I'm just going to sit down, learn. I think I used like other, I think I was using like Alpacalora.[00:05:00] Cause I think the Alpaca paper had just came out, come out then. So I was like using Alpacalora repo and sort of like learning how to use like. None of us were like GPU rich back then, and none of us, most of us still we're still all GPU poor, but I was doing what was it, like 4 bit, Alpaca Lord, there was like a 4 bit version where we were doing quant, or 8, no, 8 bit quantizations, and then I think they had released QLOR a little bit later, and I think right when, before QLOR came out, I was already starting to do fine tunes, but having this need to sort of like mix data sets together, and If you've ever looked at all the various different datasets available on HuggingFace, they all have various different prompt formats, and, it's sort of a nightmare, and then I think the other piece is if you've ever tried to fine tune, at least Back then probably the ecosystem's a little better now.[00:05:54] Everybody required that you say, alright, you put your hyperparameters as command line arguments. And so it's always like, well, I now have to go copy and paste my previous thing and to change things out. And I really wanted it. to be in a YAML file because it was more portable and reproducible.[00:06:09] So I was doing that and then the QLOR paper came out. Tim Dettmer announced that and then somebody looked it up for me yesterday and it's like between that announcement it took us seven days to get that integrated into Axolotl, right? Which is like, it's not. I wouldn't say it's really fast, but in a manner that, is in a, a reusable framework, I think it was quite the accomplishment then.[00:06:33] And so we started, picking up traction with people there. And then it's just been building models, and then just iterating what my needs are. So, yeah. Excellent. Yeah. I[00:06:44] Alex Volkov: want to ask, for folks who are listening who never heard of Axolotl, now do you describe how you got there?[00:06:49] Can you, how do you summarize this for folks who maybe haven't fine tuned anything. They know about open source LLM exists, they maybe know like LLAML, what's XLR for somebody who doesn't know. I've never heard of a data set curation[00:07:01] Wing Lian: creation before. We sort of have to take a step back and understand that, when you've got these language models, you have what I think most people refer to as like base models, also known as like foundational models, right?[00:07:15] Where some benefactor, whether it's Meta or Mistral or whoever, has gone and spent all this money. To train these models on huge corpuses of text, right? And these, these corpuses, they're generally good across lots of different things, but they're really good at just saying, talking on and on and on, but they're not good at, following instructions or having chats or anything like that.[00:07:40] So, when you think about fine tuning, it's like Saying, all right, we have this really sort of good generalized, text completion thing, and I want to turn it into something that I can talk to or have, follow instructions. So, I think fine tuning is probably best defined in like that.[00:07:58] swyx: Okay, got it.[00:07:59] And we actually[00:08:01] What is finetuning?[00:08:01] swyx: Do want to make sure that we have like an overall introduction to fine tuning for people because again like trying to make sure that we bring everyone along in this, in this journey. We already went into Loras and QLoras without explaining what[00:08:12] Wing Lian: they are. Oh yes, yes, sorry.[00:08:14] swyx: And so I will put things in my words and you can correct me as, as, as my I'll be the village idiot here.[00:08:21] So, so fine tuning is basically sort of grabbing an open source model off the shelf, and then basically doing further training on it with a custom dataset of your own. Primarily, people use it, think about it as fine tuning for JSON output, or fine tuning for a style of response. Let's say you wanted to tell jokes, or be funny, or be short, or whatever.[00:08:43] Just the open source AI community has really fine tuned in all sorts of different manner. I think we'll go over those those things now. Let's go over those things now, and then we'll talk about fine tuning methods.[00:08:52] Open Source Model Zoo[00:08:52] swyx: So there's a universe of people who fine tune stuff. Yesterday in your slides, you had, I'll just list some of these and then we'll maybe go through some of them, right?[00:08:59] So Technium is personally leading Open Hermes, which is I think the sort of premier model out of the news. news community. There's OpenOrca, which you had a hand in. News, the news research itself also has Capybara and Puffin and all the others. There's Pygmalion, which I've never messed with.[00:09:14] Eric Hartford, I am aware of his Uncensored Models and his Samantha Models. Disco Research with Disco LM. And then you personally have done Manticore, Minotaur, Jackalope, and Hippogriff. What should people know about all these names? Being part of AI Twitter is seeing all these things and going dude, I'm being DDoS'ed by all these things and I don't know how different they are.[00:09:32] What should people know? Yeah, so[00:09:34] Wing Lian: I think on a lot of these models, generally, we like to think of those as sort of general models, so If you think about it, what is GPT 4, what is Chad GPT? It's a good general model, and then, One of the services I think that OpenAI offers is like these fine tunings where you're a business and you have very specific business use cases and you might fine tune for that use case.[00:10:00] All of these models are really just general use case that you can then go and maybe Fine tune another lore over it for your use cases, but they tend to be good. With good being relative, it's open source. Open source AI is still sort of is infancy. So, good is, it's pretty reasonable.[00:10:18] It's probably still better than most, high schoolers at answering questions and being able to like figure things out and, and reasoning skills and math and those sorts of things, right?[00:10:27] swyx: And also as measured on the Hugging[00:10:29] Wing Lian: Face leaderboard. Yes, well, that's like a whole other discussion, right, there's a whole other, group of people who, and I, I mostly agree with them that, benchmarks can be, are pretty bogus these days, LM says, I think they published something recently where, even if you think the dataset's not contaminated, you can go and, find contamination And maybe we should step back and say what contamination is, right?[00:10:53] Benchmarks and Contamination[00:10:53] Wing Lian: So we have all of these data, when you go and do these benchmarks, there's a specific data set where there are these questions and usually it's multiple choice. And what can happen is, well, sometimes someone It puts the question, maybe maliciously, maybe accidentally, into the training dataset, and now the, the, your model knows how to answer the test questions really well, but it doesn't, it hasn't generalized the ability to actually do that[00:11:20] Alex Volkov: right.[00:11:21] We've seen some folks competitively announce models that are like the best at that leaderboard, but then it's, it's quite obvious that, In open source? Yeah, and in that leaderboard, for Hugging Face specific, I don't know if LMCs, if that had suffered, but we, there's been some models that seem to have been competitively trained and some leakage happened into their,[00:11:41] swyx: like, supposal.[00:11:43] I understand, once there's been a credible assertion, Hugging Face actually does take them down, right? Yeah, yeah,[00:11:48] Alex Volkov: which is really hard to know, right?[00:11:50] swyx: It's really hard to know, sometimes it's like a pure accident,[00:11:52] Alex Volkov: it's oh, oops. You're going through a mixer. I think, a responsible So acknowledgement, that this kind of happened to you is also important.[00:11:58] I saw LDJ from news research can acknowledge that. Because many of these datasets are collections of other datasets. There's a bunch of people are baking, basically. It's alchemy. Right. And so sometimes you don't know. Sometimes you pull an open source dataset and they announce, oh, you know what, actually, the MMLU benchmark which we used to Specifically identify models that did go into this data set, that then went into that data set.[00:12:22] So sometimes it's actually an accident and folks take it down. But I've seen some competitive folks who want to put their name out there because people are starting to notice which is the top[00:12:30] swyx: model. For those who want a fun take on this so the file one dataset. FindOne model from Microsoft was accused of being contaminated.[00:12:37] And I saw this joke paper that was fantastic. It was called, training on the test set is all you need. It's a super small model that just memorizes everything. It was fantastic. So yeah, contamination, I think we've actually covered it in a previous episode before. So we're good. But again, I want to give people a map into the open source AI model, the universe.[00:12:57] And Alex, you can also jump in here because you guys have spent a lot more time with them than I have. So, what should people know about Technium? What should people know about Noose? And then we can go down the list. Yeah,[00:13:05] Wing Lian: I think so. I think if we start with, Technium. When you talk to him, he's gonna say, I think, I think his response is that he wants to build GP4 on his laptop, right?[00:13:14] So, very, very good at building general models. I think with Noose, Noose Research, they're looking at more, sort of, More, more research focused things, like their Yarn models, I don't, I don't, they didn't actually train their, they have their own trainer for their Yarn models, but So they did not use Xlato for that one?[00:13:30] They didn't use that, but like Is that, you don't have support for it? I think we do support Yarn, I think, I'd have to double check that answer. Yeah, I'm just kind of curious what you can and cannot support, and Yeah, I mean, Yarn is supportable, it's basically, I think it's just replacing, I think, the rope part of that, so Yeah, not, not a big deal.[00:13:48] Yeah, it's not a big deal, it's just I haven't gotten to it, not enough people have asked, I think a lot of people have asked for other things, so it's just, squeaky wheel, right? I think at the end of the day, people are like building these data sets and I think if you sort of map things chronologically, these make more sense because it's like, how do we incrementally improve all of these models?[00:14:07] So a lot of these models are just incremental improvements over the last thing, right? Whether it is sort of through methods of how do we, how did we curate the data set? How did we improve the quality of the data set? So, you maybe LDJ talked about it right on I think for, for Capybara and Puffin, like how those, those were very specific dataset curation techniques that he works on.[00:14:29] The Case for Open Source AI[00:14:29] Alex Volkov: So there's, folks are doing this for dataset curation. Folks are doing this for skillset building as well. Definitely people understand that open source is like very important, especially after the, the, the, the, the march, the debacle, the OpenAI weekend that we all had. And people started noticing that even after developer day in OpenAI, the APIs went out.[00:14:48] And then after that, the whole leadership of the company is swiftly changed and people, there was worries about, you know. How can people continue building AI products based on these like shaky grounds that turned attention definitely to Technium at least in open RMS I started seeing this more and more on Twitter, but also other models and many companies They're gonna start with open AI just to get there quick, and then they they think about okay Maybe I don't want to share my knowledge.[00:15:13] Maybe I don't want to sign up for Microsoft. Maybe they will change their terms and conditions so What else is out there? They turned to other companies. Up until yesterday, Google was nowhere to be found. We've talked about Gemini a little bit before in a previous And you can tune in[00:15:26] swyx: to[00:15:26] Alex Volkov: Thursday Eye.[00:15:26] Yeah, you can tune in to Thursday Eye. We covered the Gemini release a little bit. And but many are turning into the open source community and seeing that Meta released and continues to release and commit to open source AI. Mistral came out and the model is way smaller than LLAMA and performs Significantly better.[00:15:43] People play with OpenRMS, which is currently techniums based, news researched, sourced, axolotl trained OpenRMS, I assume, right? And then they play with this and they see that, okay, this is like GPT 3. 5 quality. We had GPT 4. 5 birthday just a week ago. A week ago, a year ago, a week ago, we never, interacted with these models of this caliber.[00:16:04] And now there's one open source, one that's on my laptop, completely offline, that, I can continue improving for my use cases. So enterprises, companies are also noticing this. And the open source community folks are building the skill set, not only the data sets. They're building the actual kind of, here's how we're going to do this, with Axelotl, with these data sets.[00:16:21] The curation pieces. Now. Interesting. There's like recipes of curation. The actual model training is kind of a competitive thing where people go and compete on these leaderboards that we talked about, the LMC arena, and that recently added open air and recently added open chat and a bunch of other stuff that are super cool.[00:16:37] The hug and face open source leaderboard. And so there's a competitive aspect to this. There's the open source. Aspect to this, like Technium says, I want GPT 4 on my laptop. There's the, let me build a skill set that potentially turns into a company, like we saw with Noose. Noose just, started organizing, a bunch of people on Discord, and suddenly, they're announcing their company.[00:16:54] It's happening across all these modalities, and suddenly all these people who saw these green pastures and a fairly quick way to, hey, here's a cool online community I can, start doing cool stuff with. You mentioned the same in the beginning, right? Like, after your accident, what's cool, let me try this out.[00:17:08] Suddenly I start noticing that there's a significant movement of interest in enterprising companies into these areas. And, this skill set, these data sets, and this community is now very Very important, important enough to create an event which pulls in Andrei Karpathy from OpenAI to come and see what's new Jeremy Howard, like the event that we just talked about, people are flying over and this is just a meetup.[00:17:28] So, definitely, the community is buzzing right now and I think Axelot is a big piece as well.[00:17:34] Orca and OpenOrca[00:17:34] Wing Lian: Cool. Maybe we can talk about like Orca real quick, Orca, OpenOrca rather, I think there was a lot of buzz when, the first Orca paper came out. And just briefly, what is Orca? Yeah, Orca was basically having traces of like chain of thought reasoning, right?[00:17:48] So they go and they, they distill sort of GPT 4. They take, they take a sampling of data from the Flan dataset. Maybe we can like add some show notes in the Flan dataset. Yeah, but we've covered it. Okay, cool. Use GPT 4 to say, all right, explain this in a step by step reasoning, right?[00:18:06] And then you take that and you, they train the model and it showed, very good improvements across a lot of benchmarks. So OpenOrca was sort of the open reproduction of that since Microsoft Research never released that particular data set. And going back to sort of the Hugging Face leaderboard thing, those models did really well.[00:18:23] And then I think, so sort of the follow up to that was SlimOrca, right? I think Going into and building the OpenOrca dataset, we never really went in and, validated the actual answers that GPT 4 gave us, so what we did was one from OpenChat actually cross referenced the original Flan, the original Flan response, the human responses, the correct answers with the dataset, and then I went and took it and sent all of, both of them to GPT 4 and said, is this answer mostly correct, right?[00:18:54] Yeah. And then we were able to filter the dataset from, At least of the GPT 4 only answers from like 800, 000 to like 500, 000 answers or rows and then, and then retrain the model and it had the same performance as the original model to within I think, 0. 1 percent here about, and 30 percent less data.[00:19:13] So, yeah. Okay.[00:19:15] swyx: Interesting. So, I mean, there's, there's so much there that I want to highlight, but yeah. Orca is interesting. I do want people to know about it. Putting chain of thought into the data set like it's just makes a ton of sense one thing I think it would be helpful for people to scope thing these things out is how much data are we talking about when when you When people are fine tuning and then how much time or resources or money does it take to train to fine[00:19:36] Wing Lian: tune?[00:19:37] Yeah, so I think there's a little bit of overlap there with sort of like fine tuning techniques, but let's say Orca and I think even Hermes, they're both relatively large data sets like 10 billion tokens. Yeah. So large data sets being or the original Orca was, or the original open Orca was 800,000 rows.[00:19:55] I believe it was somewhere in the ballpark of like a gigabyte of data, of gigabyte, of text data. And I, I don't. I believe, Hermes was, is like a quarter million rows of data, I don't know the actual byte size on that particular one. So, going and training a, let's, let's say everybody's training 7 billion Mistral right now, right?[00:20:15] So, to tri I, I believe to fine tune 7 billion Mistral on, let's say, 8 A6000s, which have 48 gigabytes of VRAM, I believe, It takes about 40 hours, so 40, and then that's, depending on where you get your compute, 40 times 6, so it's like 500 to fine tune that model, so, and, and that's assuming you get it right the first time, right?[00:20:44] So, you know.[00:20:45] swyx: Is, is that something that X. Lotto handles, like, getting it right the first[00:20:48] Wing Lian: time? If you talk to anybody, it's like you've probably tried at least three or four runs or experiments to like find the right hyperparameters. And after a while you sort of have a feel for like which, where you need your hyperparameters to be.[00:21:04] Usually you might do like a partial training run, do some benchmark. So I guess for Al Farouk, whether you're going by his. This is Jeremy, he's, his actual name, or his twitter handle. He released the Dharma dataset, which is basically a subset of all the benchmarks. And Axolotl actually supports, you know taking that subset and then just running many benchmarks across your model every time you're doing an evaluation so you can sort of like see sort of relative it's not going to be the actual benchmark score, but you can get ideas alright, is this benchmark improving, is this benchmark decreasing, based on, you know Wait,[00:21:39] swyx: why don't you run the full benchmark?[00:21:41] What, what, what The[00:21:42] Wing Lian: full benchmarks take Take a long time. Significant, yeah, significant amount of time. Yeah. And Okay, so that's like[00:21:48] swyx: mini MMLU. Yeah. Like,[00:21:49] Wing Lian: mini BigBench or whatever. Yep, exactly.[00:21:51] Alex Volkov: It's really cool. We, when I joined Web2Masters just recently, and one of the things that I try to do is hey I'm not, I'm a software engineer by trade, I don't have an MLE background, But I joined a company that does primarily MLE, and I wanted to learn from the community, Because a lot of the open source community, they use weights and biases, And the benchmark that you said that Pharrell did, remind me of the name, sorry.[00:22:13] Dharma? Dharma, yeah, yeah. So Luigi showed me how Dharma shows inside the dashboard. In Wi and Biases dashboard and so you can actually kinda see the trending run and then you can see per each kind of iteration or, or epoch or you can see the model improving trending so you can on top of everything else.[00:22:29] The wi and biases gives like hyper parameter tracking, which like you, you started with common line and that's really hard to like remember. Also the Dharma data set, like the quick, the mini orca mini, you mini many different things. It's pretty cool to like visualize them as well. And I, I heard that he's working on a new version of, of Dharma, so Dharma 2, et cetera.[00:22:47] So hopefully, hopefully we'll see that soon, but definitely it's hard, right? You start this training around, it said like 40, 50 hours. Sometimes, sometimes it's like your SSHing into this machine. You, you start a process, you send it with God and you just go about your day, collecting data sets, and then you have to return.[00:23:04] And the whole process of instrumentation of this is still a little bit like squeaky but definitely. Tuning performance, or like grabbing performance in the middle of this, like with Dharma and some other tools, is very helpful to know that you're not wasting precious resources going somewhere you shouldn't go.[00:23:21] Yeah.[00:23:22] swyx: Yeah. Very cool. Maybe I'll, I'll, before we go into like sort of more details on fine tuning stuff, I just wanted to round out the rest of the Excel autoverse. There's, there's still Eric Hartford stuff. I don't know if you want to talk about Pygmalion, Disco, anything that you know about[00:23:35] Wing Lian: those, those things.[00:23:36] DiscoLM and Model Stacking[00:23:36] Wing Lian: Yeah, I think like one of the, definitely one of the more interesting ones was like the Disco 120b, right? Yeah, I know nothing about it. Yeah. So, so. Alpen from Pygmalion AI, right, so they, so Pygmalion is a sort of a, it's, it's, they have their own community, a lot of it is based around, roleplay models, those sorts of things, and Alpen, like, put together, merged together Llama270B, so, and Alpen, like, put together, merged together Llama270B, so, I don't remember how he stacked them together, whether he merged the layers in between. There's a whole, there's a whole toolkit for that by Charles Goddard, where you can like take a single model and like stack them together or multiple models merge.[00:24:18] That's like a whole other talk and a whole other tool set, but was able to create this 120. Billion parameter model out of a LAMA two 70 B. And then I believe the, yeah, disco is a fine tune of, of the, the, the sort of the base one 20 B is, I believe Goliath one 20 B. So, and, and what are the[00:24:37] swyx: headline results that people should know about[00:24:39] Wing Lian: disco?[00:24:39] I think for the headline results, I, I've, I haven't played with it personally because it's. It's a very large model and there's a lot of GPU, right? But, like, from what I've heard anecdotally, it performs really well. The responses are very good. Even with, like, just, even the base model is a lot better than, Llama70b.[00:24:57] So, and we, I think generally everybody's like, we would all love to fine tune Llama70b, but it's just, it's so much, it's so much memory, so much compute, right?[00:25:07] Datasets and Evals over Models[00:25:07] Wing Lian: I[00:25:07] Alex Volkov: want to touch on this point because the interesting thing That comes up out of being in this ecosphere and being friends with open source folks, tracking week to week state of the art performance on different models.[00:25:19] First of all, a lot of the stuff that the folks do a couple of weeks ago, and then something like Mistral comes out, and a lot of the stuff back then, Doesn't technically make sense anymore. Like the artifacts of that work, the actual artifacts, they don't no longer make sense. They're like lower on the on, on the hug and face leaderboard or lower on LM CS leaderboard.[00:25:36] But some of the techniques that people use, definitely the datasets. The datasets keep traveling, right? So open airmen, for example, is the dataset. The tum cleaned up for only. Open sourceable data that previously was just Hermes. And that, it was previously used to train Lama. And then once Mistral came out, it was used to train Mistral.[00:25:54] And then it became significantly better on the 7b base Mistral. So the data sets keep traveling, keep getting better a little bit here and there. And so the techniques improve as well. It looks like both things are simultaneously true. The artifacts of a month and a half ago. The, the actual models themselves, it's great the hug and face has them, because not every company can keep up with the next weeks', oh, I, I'll install this model instead, sell this model instead.[00:26:19] But the, the techniques and the, the dataset keep improving as we go further, and I think that's really cool. However, the outcome of this is that for a long time. For many, many people, including us, that we do this every week. We literally talk with people who release these models every week. It's really hard to know.[00:26:36] So, there's a few aspects of this. One, I think, like you said, the bigger model, the 70B models, you actually have to have somebody like Perplexity, for example, giving you access to the 70B really fast. Or you have to, like, Actually, find some compute, and it's expensive, especially for the bigger models. For example Falcon 180B came out, like the hugest open source model.[00:26:56] How do you evaluate this if you can't run it? Nobody liked it. It's really, so first of all, nobody liked it, but secondly, only the people who were able to find compute enough to run inference on this, they only had like, I can't run this on my laptop, and so that's why it's much easier, something like OpenRMS 7 to be, 7B, it's much easier, because you can run this on your MacBook.[00:27:14] It's much easier to evaluate. It's much easier to figure out the vibes, right? Everybody talks about the vibes as an evaluation check. If you're plugged in enough, if you follow the right people, if they say pretty much the same things all independently, then you run into a problem of whether they're repeating, and their stochastic parents are repeating the same thing, or they actually evaluated themselves.[00:27:31] Yeah, you never know. But, you never know, but like, I think on a large enough scale on Twitter, you start getting the feel. And we all know that like, OpenRMS is one of the top performing models, benchmarks, but also vibes. And I just wanted to highlight this vibes checks thing because you can have the benchmarks, you can have the evaluations, they potentially have contamination in them, potentially they not necessarily tell you the whole story because some models are good on benchmarks, but then you talk to them, they're not super helpful.[00:28:00] And I think it's a combination of the benchmarks, the leaderboards, the chatbot, because LMSys, remember, their ranking is not only based on benchmarks, it's also people playing with their arena stuff. People actually like humans, like, get two answers. I think they completely ignore benchmarks. Yeah, and then They only do ELO.[00:28:18] Oh, they do ELO completely, right? So that, for example, is just like people playing with both models and say, Hey, I prefer this one, I prefer that one. But also there's like some selection bias. The type of people who will go to LMCs to play with the models, they're a little bit specific in terms of like who they are.[00:28:33] It's very interesting. There's so many models. People are doing this in this way, that way. Some people are doing this for academic rigor only to test out new ideas. Some people are actually doing this like the Intel fine tunes of Mistral. Intel wanted to come out and show that their hardware approach is possible, Mistral, etc.[00:28:51] And it's really hard to know, like, what to pick, what to use. And especially on the bigger models, like you said, like the Llama 70B, the Falcon 180B. It's really because, like, who has the compute to validate those? So I would mention that, like, use with caution. Like, go and research and see if the biggest model that just released was actually worth the tokens and the money you spend on it.[00:29:12] To try and, if you're a business, to integrate it.[00:29:15] Distilling from GPT4[00:29:15] swyx: Since you said use of caution, I'll bring in one issue that has always been in the back of my mind whenever I look at the entire universe of open source AI models, which is that 95 percent of the data is derived from GPC 4, correct?[00:29:30] Which technically you can't use for commercial licenses,[00:29:34] Wing Lian: right?[00:29:35] swyx: What is the community's stance on this kind of stuff?[00:29:40] Wing Lian: I think from the community stance, like I feel like a lot of us are just experimenting, so for us, it's like, we're not going and building a product that we're trying to sell, right?[00:29:49] We're just building a product because we think it's interesting and we want to use it in our day to day lives, whether or not we try and integrate it. Personal use, yeah. Yeah, personal use, so like, as long as we're not selling it, yeah, it's fine. But[00:30:01] swyx: like, I as a company cannot just take OpenHermes and start serving[00:30:05] Alex Volkov: it and make money on it.[00:30:06] OpenHermes you can. Because the opening of OpenHermes, I think, is a clean up. That did after the regular Hermes, please folks, check your licenses before you listen to podcasts and say, Hey, I will tell you though, you could say the same thing about OpenAI. You could say the same thing kind of makes sense, where OpenAI or StabilityAI trains their diffusion model on a bunch of pictures on the internet, and then the court kind of doesn't strike down Sarah Silverman, I think, or somebody else, who came and said, hey, this has my work in it, because of the way how it processes, and the model eventually builds this knowledge into the model, and then it doesn't actually reproduce one to one what happened in the dataset.[00:30:45] You could claim the same thing for open source. Like, we're using And by we, I mean the, the open source community that I like happily report on uses GPT 4 to rank, for example, which is the better answer you, you, that's how you build one, one type of data set, right? Or DPO or something like this, you, you basically generate data set of like a question and four answers, for example, and then you go to GPT 4 and say, Hey, smartest model in the world right now, up to Gemini Ultra, that we should mention as well.[00:31:11] Which one of those choices is better? But the choices themselves are not necessarily written with GPT 4. Some of them may be, so there's like full syntactic datasets. But there's also, datasets are just ranked with GPT 4. But they're actually generated with a sillier model, or like the less important model.[00:31:25] The lines are very blurry as to what type of stuff is possible or not possible. And again, when you use this model that's up on Hug Face, the license says you can use this. OpenAI is not going to come after you, the user. If anything, OpenAI will try to say, hey, let's prevent this, this type of thing happening, and the brain, but I honestly don't think that they could know even, not that it makes it okay, it's just like, They also kind of do this with the Internet's archive, and also, I think that some of it is for use.[00:31:55] You use models to help you augment tasks, which is what GPT 4 lets you do.[00:32:00] swyx: Yeah, the worst thing that OpenAI can do is just kick you off OpenAI. That's because it's only enforced in the terms of service.[00:32:05] Alex Volkov: Sure, but just like to make sure, to clarify who they're going to kick out, they could kick out like News, for example, if news are abusing their service, a user of the open source, fully Apache 2 open source, for example, They won't get kicked out if they use both, just because they use both.[00:32:22] I don't believe so. I don't think OpenAI has a claim for that.[00:32:25] swyx: Well, we're not lawyers, but I just want to mention it for people to know it's an issue.[00:32:30] Wing Lian: And one of the things, like, I talked to someone recently, and I think that they also are like interested in it, but also to the point of like, right, if I use a model trained on data, using GPT for data, But I use that model to then regenerate new data.[00:32:46] Is that model, is that data okay? So like you start going down this whole rabbit hole. So yeah. All right.[00:32:53] swyx: Fantastic. Cool. Well, I think that's roughly highlights most of the open source universe. You also have your own models. Do you want to shout out any one of them? Yeah.[00:33:01] Wing Lian: I mean, I think like, I think Early on, Manicore got a lot of love.[00:33:04] I think it was mostly popular in, like, the roleplay communities. It was, it tended to be pretty truthful. It tended to be, like, have relatively good answers, depending on who you ask, right? But, I think for me, it was just, Releasing models was a way to try and, like, continue to build out the product, figure out what I needed to put into the product, how do I make it faster, and, if you've got to, like, go and debug your product, you may as well have it do something useful.[00:33:29] Awesome. So, yeah.[00:33:31] Finetuning - LoRA, QLoRA, ReLoRA, GPTQ[00:33:31] swyx: Okay, and then maybe we'll talk about just fine tuning techniques. So this is going to be a little bit more technical than just talking about model names and datasets. So we started off talking about LoRa, QLoRa. I just learned from your readme there's ReLoRa. Which I've never heard about.[00:33:45] Could you maybe talk about, like, just parameter efficient fine tuning that whole, that[00:33:50] Wing Lian: whole journey, like, what people should know. Yeah, so with parameter efficient fine tuning, I think the popular ones, again, being, let's, we'll start with lore, right? So, usually what you do is you freeze all the layers on your base, on the base model, and then you, at the same time, you sort of introduce additional Oh, this is tight.[00:34:08] No. You introduce, another set of layers over it, and then you train those, and it is done in a way that is mathematically possible, particularly with LORs that you can, then you, you, When you, when you train the model, you, you run your inputs through the base model, whose weights are frozen, but you, then you also run it through the additional weights, and then at the end you combine the weights, and then, and then, or you combine the weights to get your outputs, and then at the end, and when you're done training, you're left with this other set of weights, right, that are completely independent, and And then from that, what you can do is, some person smarter than I figured out, well, oh, they've done it in such a way that now I can merge these weights back into the original model without changing the architecture of the model, right?[00:35:03] So, so, that tends to be, like, the go to, and You're training much fewer parameters so that when you do that, yes, you still need to have all of the original weights, but you have a smaller gradient, you have a smaller optimizer state, and you're just training less weights, so you can tend to train those models on, like, much smaller GPUs.[00:35:27] swyx: Yeah. And it's roughly like, what I've seen, what I've seen out there is roughly like 1 percent the number of parameters that you're trading. Yeah, that sounds about right. Which is that much cheaper. So Axelotl supports full fine tune, LoRa, QLoRa,[00:35:40] Wing Lian: Q. Yes. So, so QLoRa is, is very similar to LoRa. The paper was, if I remember correctly, the paper was Rather, traditionally, most people who did Loras were, were, they were quant, they were putting the model weights in 8 bit, and then fine tune, parameter efficient fine tuning over the Lora weights, and then with QLora, they were quantizing all of those, they were then quantizing the weights down to 4 bit, right, and then I believe they were also training on all of the linear layers in the model.[00:36:15] And then with ReLore, that was an interesting paper, and then, I think, like, it got implemented. Some people in the community tried it, tried it out, and it showed that it didn't really have the impact that the paper indicated that it would. And from what I was told recently, that they re I guess they re released something for Relora, like, a few weeks ago, and that it's possibly better.[00:36:44] I personally haven't had the time. What was the[00:36:46] swyx: main difference,[00:36:47] Wing Lian: apart from quantization? I don't know. Okay. What was the main difference, sorry?[00:36:49] swyx: Apart from quantization, right? Like,[00:36:50] Wing Lian: Qlora's thing was, like, we'll just drop off some bits. With Relora, what they did was, you would go through, you would define some number of steps that you would train, like, your Lora with, or your Qlora.[00:37:01] Like, you could do Like, ReqLore, if you really wanted to, you would, you would train your LoRa for some number of steps, And then you would merge those weights into your base model, and then you would start over. So by starting, so, then by starting over, The optimizer has to find, like, sort of, re optimize again, and find what's the best direction to move in, and then do it all again, and then merge it in, do it all again, and theoretically, according to the paper, doing ReLore, you can do parameter efficient fine tuning, but still have sort of, like, the performance gains of doing a full fine tuning, so.[00:37:38] swyx: Yeah, and[00:37:39] Wing Lian: GPTQ? And GPTQ, so it's, I think with GPTQ, it's very similar to, more similar to QLore, where you're, it's mostly a quantization of the weights down to like 4 bit, where GPTQ is a very, is a specific methodology or implementation of quantization, so. Got it.[00:37:57] Alex Volkov: Wang, for, for folks who use Axolotl, your users, some people who maybe, Want to try it out?[00:38:03] And do they need to know the differences? Do they need to know the implementation details of QLora versus ReLora? Or is it okay for them to just know that Axolotl is the place that already integrated them? And if that's true, if that's all they need to know, how do they choose which method to use? Yeah,[00:38:22] Wing Lian: so I think like, I think most people aren't going to be using ReLora.[00:38:25] I think most people are going to be using either Lora or QLora. And I think they should have it. They should have an understanding of why they might want to use one over the other. Most people will say that with Qlora, the quality of the final model is not quite as good as like if you were to do a LoRa or a full fine tune, right?[00:38:44] Just because, you've quantized these down, so your accuracy is probably a little off, and so that by the time you've done the Qlora, you're not moving the weights how you would on a full fine tune with the full parameter weights.[00:38:56] Interesting.[00:38:57] swyx: Okay, cool. For people who are more interested, obviously, read the papers. I just wanted to give people, like, a high level overview of what these things are. And you've done people a service by making it easy for people to try it out. I'm going to, I'm going to also ask a question which I know to be wrong, but I'm curious because I get asked this all the time.[00:39:15] What is the difference between all these kinds of fine tunes[00:39:17] Wing Lian: and RLHF? Okay, between all of these sorts of fine tunes and RLHF. So all of these sorts of fine tunes are based, are, ideally, this, they are taking knowledge that the base model already knows about, and presenting it in a way to the model that you're having the model answer like, Use what it already knows to sort of answer in a particular way, whether it's, you're extracting general knowledge, a particular task, right?[00:39:44] Instruct, tune, chat, those sorts of things. And then generally with RLHF, so what is, let's go back, what is it? Reinforcement Learning with Human Feedback. So if we start with the human feedback part, What you're doing is you generally have, you have like a given prompt and then you, maybe you have one, maybe you have two, I think, like if you look at with Starling, you have like up to what, seven different, seven different possible responses, and you're sort of ranking those responses on, on some sort of metric, right, whether the metric is how much I, I might like that answer versus or I think with like starling is like how how how helpful was the answer how accurate was the answer how toxic was the answer those sorts of things on some sort of scale right and then using that to go back and like sort of Take a model and nudge it in the direction of giving that feedback, to be able to answer questions based on those preferences.[00:40:42] swyx: Yeah, so you can apply, and is it commutative? Can you apply fine tuning after and onto an RLHF model? Or should the RLHF apply, come in afterwards,[00:40:54] Wing Lian: after the fine tune? Um, I, yeah, I don't know that there's There's been enough research for one way or another, like, I don't know.[00:41:02] That's a question that's been asked on Discord. Yeah, like, I definitely would say I don't know the answer. Go and try it and report back to me and let me know so I can answer for the next guy.[00:41:10] swyx: It's shocking how much is still unknown about all these things. Well, I mean, that's what research is for, right?[00:41:16] Wing Lian: So actually I, I think I saw on the top of a leaderboard, it was a, it was a mytral base model, and they didn't actually fine tune it. They, or they, they just did RLH, they did like an RLHF fine tune on it using like, I don't, I don't recall which dataset, but it was like, and it benchmarked really well.[00:41:37] But yeah, you'd have to go and look at it. But, so it is interesting, like going back to that, it's like. Traditionally, most people will fine tune the model and then do like a DPO, PPO, some sort of reinforcement learning over that, but that particular model was, it seemed like they skipped like the supervised fine tuning or Scott.[00:41:55] Axolotl vs HF Transformers[00:41:55] swyx: Cool. One thing I did also want to comment about is the overall, like, landscape, competitive landscape, I don't know. Hugging Face Transformers, I think, has a PFT module.[00:42:05] Wing Lian: Yeah, yeah, the PEFT, the Parameter Efficient Fine Tuning, yep. Is that a competitor to you? No, no, so we actually use it. We're just a wrapper over sort of, sort of the HuggingFace stuff.[00:42:15] So, so that is their own sort of module where They have, taken the responsibility or yeah, the responsibility of like where you're doing these parameter efficient fine tuning methods and just sort of like, it is in that particular package where transformers is mostly responsible for sort of like the modeling code and, and the trainer, right.[00:42:35] And then sort of, there's an integration between the two and, there's like a variety of other fine tuning packages, I think like TRL, TRLX, that's the stability AI one. Yeah, I think TRL likes the stability, yeah, Carper, and TRL is a hugging face trainer. Even that one's just another wrapper over, over the transformers library and the path library, right?[00:43:00] But what we do is we have taken sort of those, yes, we've We also use that, but we also have more validation, right? So, there are some of us who have done enough fine tunes where like, Oh, this and this just don't go together, right? But most people don't know that, so like Example?[00:43:19] Like, people want to One and one doesn't go together. I don't have an example offhand, but if you turn this knob and this knob, right? You would think, all right, maybe this will work, but you don't know until you try. And then by the time you find out it doesn't work, it's like maybe five minutes later, it's failed.[00:43:34] It's failed in the middle of training or it's failed during the evaluation step. And you're like, ah, so we've, we've added a lot of, we've added a lot more validation in it. So that like, when you've, you've created your configuration, you run it through and now you say. The validation code says this is probably not right or probably not what you don't, not what you want.[00:43:52] So are you like a, you[00:43:53] swyx: do some linting of your YAML file?[00:43:56] Wing Lian: There, I guess you could call it linting, it's sort of like Is there a set of rules out[00:44:00] swyx: there somewhere? Yeah, there's a set of rules in there. That's amazing, you should write documentation like This rule is because, this user at this time, like, ran into this bug and that's what we invested in.[00:44:10] It's like a good collection[00:44:11] Wing Lian: of knowledge. Yeah, it is, and I guess like, if you really wanted to, like, figure it out, I guess you could, like, git blame everything, and But, yeah, it's, so, I think that's always a useful thing, it's like Because people want to experiment but they don't, people will get frustrated when you've experiment, you're experimenting and it breaks and you don't know why or you know why and you've just gone down the rabbit hole, right?[00:44:37] So, so I think that's one of the big features that's, that I think I find important because it's It prevents you from doing things you probably shouldn't have, and it, and sometimes we will let you do those things, but we'll try and warn, warn you that you've done that.[00:44:50] I[00:44:51] Alex Volkov: have a follow up question on this, actually, because yesterday we hung out to this open source event, and I spent time by you a couple times, like when people told you, oh, XLR, I use XLR, it's super cool, and then the first thing you asked is, like, immediately, like, what can we improve?[00:45:04] And yes, from multiple folks, and I think we talked about this a little bit, where there's It's a developer tool. It's like a machine learning slash developer tool. Your purpose in this is to help and keep people, as much as possible, like, Hey, here's the best set of things that you can use right now. The bear libraries are, or the bear trainer, for example, is a bear trainer.[00:45:28] And also, maybe we should talk about how fast you're implementing these things. So you mentioned the first implementation took a week or so. Now there's a core maintainer group, right? There's like, features are landing, like Qlora, for example. Neftune, I don't know if that's one example of something that people potentially said that it's going to be cool, and then eventually, like, one of those things that didn't really shake out, like, people quickly tested this out.[00:45:48] So, there's a ton of Wait, Neftune is cancelled? I don't know if it's fully canceled, but based on vibes, I heard that it's not that great. So like, but the whole point that I'm trying to make with Neftune as well is that being existing in the community of like XLR or like, I don't know, even following the, the GitHub options or following the Discord, it's a fairly good way to like, learn these, Kind of gut feelings that you just, you just said, right?[00:46:14] Like where this, maybe this knob, that knob doesn't work. Some of these are not written down. Some of these are like tribal knowledge that passes from place to place. Axel is like a great collection of many of them. And so, do you get That back also from community of folks who just use, like, how do you know who uses this?[00:46:30] I think that's still an issue, like, knowing if they trained with XLR or should they add this to things? Talk about, how do you get feedback and how else you should get feedback?[00:46:38] Wing Lian: Yeah, I mean, most of the feedback comes from the Discord, so people come in and , they don't get a training running, they run into, like, obscure errors or, errors that That's a lot of things that maybe, maybe as a product we could catch, but like, there's a lot of things that at some point we need to go and do and it's just on the list somewhere.[00:46:58] Right that's why when people come up, I'm like, what, what were your pain points? Because like, as a developer tool, if you're not happy with it, or you come in and in the first, Takes you 30 minutes and you're still not happy. You leave the tool and you may, you might move on maybe to a better tool, maybe to, one with less frustration, but it may not be as good, right?[00:47:17] So I'm trying to like, figure out, all right, how can I reduce all this frustration? Because like for me, I use it every day for the most part, right? And so I am blind to that, right? Mm-Hmm. . Mm-Hmm. . I just know, I, I go do this, this, and this. It pretty much mostly works, right? But, so I don't have sort of that, alright, that learning curve that other people are seeing and don't understand their pain points.[00:47:40] Yeah,[00:47:40] Alex Volkov: you don't have the The ability to onboard yourself as a new user completely new to the whole paradigm to like get into the doors of like, Oh, no, I don't even know how to like ask about this problem or error.[00:47:53] swyx: Cool. The last few things I wanted to cover was also just the more advanced stuff that you covered yesterday.[00:48:00] 20x efficiency with StackLlama and Multipack[00:48:00] swyx: So I'll just, caution this as like, yeah, this is more advanced. But you mentioned Stackllama and Multipack. What are they[00:48:06] Wing Lian: and what should people know? Yeah, so, so, Stack Llama was, that paper came out, so Stack Llama I think was like, two, two, two separate, two separate concepts that they announced, so the first one was They being hugging face.[00:48:20] Yeah, sorry, yes, they being hugging face, so the first one being sort of like, this idea of packing, like some packing sequences together, so like, if we think about training data, right, your training data is, let's say, to keep the math easy, let's say your training data is 500, We, we, we, we will use the terminology words.[00:48:39] Let's say your training data is 500 words long, and let's say your, your context length, you know how much data your, that your model can accept is like, or that you want feed into your model. It's, let's say, we won't use tokens again, we'll we'll use it is it's 4,000 tokens, right? So if you're training at 4K Con or four 4,000 4K contacts and you're only using 500 of it, you're sitting like with the other 1500.[00:49:05] 3, 500 words that you're not using, right? And typically that's either filled with these PAD tokens, so I think I made the analogy last night that it's like having sort of like a glass here you fill it up with a shot of liquor and then you're and that's your training data and then you just fill it up with more water and those are your PAD tokens and it's just, it doesn't do much, right?[00:49:27] It's still the same thing, but you still have to go through all of that to go through all your training data. And then, so what Stack Llama showed was you could just sort of take your training data, append the next row of training data until you filled that entire 4k context, so in this example, right, with 500 words to 4k, that's 8 rows of training data.[00:49:48] But, the problem with that is, is that with a lot of these transformer models, they're very much relying on attention, right? So, like, if you now have this sequence of words that now, in order for the, the model has seen all of these other words before, right? And then it sees another set of words, another set of words, but it's learning everything in context of all the words that it's seen before.[00:50:13] We haven't corrected the attention for that. And just real quickly, since I said that that paper was two concepts, the other one was, I believe it was like a reinforcement learning, but outside the scope of this. So going from that, I implemented that early on because I was like, Oh, wow, this is really great.[00:50:29] And. Yes, because it saves you a bunch of time, but the trade off is a little bit of accuracy, ultimately, but it still did pretty well. I think when I did Manicore, I think it used sort of that concept from Stack Llama of just sort of appending these sequences together, right? And then sort of the next evolution of that is Multipack, right?[00:50:51] So, there was a separate paper on that, it was, I believe it was referenced, it got referenced in the Orca paper, where you could, you could properly mask those out using like a, I think it was like a lower block triangular attention mask, and then sort of, so, So, there's that. I did try implementing that, manually recreating that mask, but then one from the OpenChat, so he was helping with OpenOrca as well, and he had done an implementation of Multipack, and where he used FlashAttention, so FlashAttention So that was released by TreeDAO, and it was this huge performance gain.[00:51:35] Everybody uses it now, even the Transformers library now, they've taken all of these, like, people are taking all of these models and sort of like, making it compatible with FlashAttention. But in Flash Tension, there is one particular implementation that lets you say, Well, I'm sending you all of these sequences like you would in Stack Llama, But let me send you another, another, Set of information about, this is where this set of sequences is, this is where the second set of sequences is.[00:52:06] So like, if it was like, 500 words long, and you stacked them all together, you would just send it a row of information that was like, 0, 500, 1000, 1500, etc, etc, out to 4000. And it would know, alright, I need to break this up, and then run the forward pass with it. And then it would be able to, and it was much more, much more performant.[00:52:29] And I think you end up seeing like 10x, 20x improvements over sort of, I mean, I think FlashAttention was like a 2x improvement, and then adding that with the Multipack, you start to see like, depending on, how much data you have, up to like a 20x improvement sometimes. 20x. 20x. Wow. Yeah.[00:52:48] And I only know the 20x because I, like, before last night, I was like, I re ran the alpaca, I looked up the alpaca paper because it was like, I just need a frame of reference where somebody did it, and I think they used eight A100s for three hours, and they said it cost them 100. I don't, I don't think eight A100s cost, I don't know how much it costs right now.[00:53:14] But I ended up rerunning it. Usually a dollar an hour, right? Yeah, so eight. The cheapest is like a[00:53:18] Alex Volkov: dollar, a dollar an hour for one.[00:53:20] Wing Lian: Yeah, so that's still like 24, 25. But maybe if you're going on Azure, maybe it's like, maybe it's 100 on Azure. I mean, it used to be more expensive, like, a year ago.[00:53:31] Yeah, and then, so I re ran it with sort of like, I turned on all of the optimizations just to see what it would be. And like, and usually Multipack is the biggest optimization, so Multipack with Flash Detention. And it, I think I spun it up on 8 L40s, and it ran, and I didn't let it run all the way through, I just grabbed the time, the estimated completion time, and it was like 30 minutes, so it would have cost like 4 or 5 to run the entire, like, reproduce the alpaca paper, right?[00:54:00] Which is crazy. It's crazy. 20x,[00:54:02] Alex Volkov: yeah. I want to ask about, like, you said you turned on all the optimization. Is that the yaml file with xlodl, you just go and like check off, like, I want this, I want that? Yeah, yeah,[00:54:10] Wing Lian: so there's like one particular yaml file in there, That, there's one particular YAML file in there that's like, it's under examples, llama2, fft, optimize.[00:54:20] So, I think someone had created one where they just turned, they put in all of the optimizations and turned them on. I mean, it actually, it does run, which is like, sort of surprising sometimes, because sometimes, you optimize this, optimize this, and sometimes they just don't work together, but, yeah.[00:54:36] Just turn the knobs on, and like, fine tuning should really just be that easy, right? I just want to flip the knob and move on with my life and not figure out how to implement it.[00:54:47] Tri Dao and Mamba[00:54:47] Alex Volkov: Specifically, the guy behind FlashAttention came up with something new. You want to talk about this a little bit? You want to briefly cover Mamba?[00:54:53] Yeah, let's talk about Mamba. Let's talk about Mamba. So, what is Mamba?[00:54:57] Wing Lian: Oh, gosh. I

Ned Bellavance worked in the world of tech for more than a decade before joining the family profession as an educator. He joins Corey on Screaming in the Cloud to discuss his shift from engineer to educator and content creator, the intricacies of Terraform, and how changes in licensing affect the ecosystem.About NedNed is an IT professional with more than 20 years of experience in the field. He has been a helpdesk operator, systems administrator, cloud architect, and product manager. In 2019, Ned founded Ned in the Cloud LLC to work as an independent educator, creator, and consultant. In this new role, he develops courses for Pluralsight, runs multiple podcasts, writes books, and creates original content for technology vendors.Ned is a Microsoft MVP since 2017 and a HashiCorp Ambassador since 2020.Ned has three guiding principles: embrace discomfort, fail often, and be kind.Links Referenced: Ned in the Cloud: https://nedinthecloud.com/ LinkedIn: https://www.linkedin.com/in/ned-bellavance/ TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. My guest today is Ned Bellavance, who's the founder and curious human over at Ned in the Cloud. Ned, thank you for joining me.Ned: Yeah, it's a pleasure to be here, Corey.Corey: So, what is Ned in the Cloud? There are a bunch of easy answers that I feel don't give the complete story like, “Oh, it's a YouTube channel,” or, “Oh no, it's the name that you wound up using because of, I don't know, easier to spell the URL or something.” Where do you start? Where do you stop? What are you exactly?Ned: What am I? Wow, I didn't know we were going to get this deep into philosophical territory this early. I mean, you got to ease me in with something. But so, Ned in the Cloud is the name of my blog from back in the days when we all started up a blog and hosted on WordPress and had fun. And then I was also at the same time working for a value-added reseller as a consultant, so a lot of what went on my blog was stuff that happened to me in the world of consulting.And you're always dealing with different levels of brokenness when you go to clients, so you see some interesting things, and I blogged about them. At a certain point, I decided I want to go out and do my own thing, mostly focused on training and education and content creation and I was looking for a company name. And I went through—I had a list of about 40 different names. And I showed them to my wife, and she's like, “Why don't you go Ned in the Cloud? Why are you making this more complicated than it needs to be?”And I said, “Well, I'm an engineer. That is my job, by definition, but you're probably right. I should just go with Ned in the Cloud.” So, Ned in the Cloud now is a company, just me, focused on creating educational content for technical learners on a variety of different platforms. And if I'm delivering educational content, I am a happy human, and if I'm not doing that, I'm probably out running somewhere.Corey: I like that, and I'd like to focus on education first. There are a number of reasons that people will go in that particular direction, but what was it for you?Ned: I think it's kind of in the heritage of my family. It's in my blood to a certain degree because my dad is a teacher, my mom is a teacher-turned-librarian, my sister is a teacher, my wife is a teacher, her mother is a teacher. So, there was definitely something in the air, and I think at a certain point, I was the black sheep in the sense that I was the engineer. Look, this guy over here. And then I ended up deciding that I really liked training people and learning and teaching, and became a teacher of sorts, and then they all went, “Welcome to the fold.”Corey: It's fun when you get to talk to people about the things that they're learning because when someone's learning something I find that it's the time when their mind is the most open. I don't think that that's something that you don't get to see nearly as much once someone already, quote-unquote, “Knows a thing,” because once that happens, why would you go back and learn something new? I have always learned the most—even about things that I've built myself—by putting it in the hands of users and seeing how they honestly sometimes hold it wrong and make mistakes that don't make sense to me, but absolutely make sense to them. Learning something—or rather, teaching something—versus building that thing is very much an orthogonal skill set, and I don't think that there's enough respect given to that understanding.Ned: It's an interesting sphere of people who can both build the thing and then teach somebody else to build the thing because you're right, it's very different skill sets. Being able to teach means that you have to empathize with the human being that you're teaching and understand that their perspective is not yours necessarily. And one of the skills that you build up as an instructor is realizing when you're making a whole bunch of assumptions because you know something really well, and that the person that you're teaching is not going to have that context, they're not going to have all those assumptions baked in, so you have to actually explain that stuff out. Some of my instruction has been purely online video courses through, like, Pluralsight; less of a feedback loop there. I have to publish the entire course, and then I started getting feedback, so I really enjoy doing live trainings as well because then I get the questions right away.And I always insist, like, if I'm delivering a lecture, and you have a question, please don't wait for the end. Please interrupt me immediately because you're going to forget what that question is, you're going to lose your train of thought, and then you're not going to ask it. And the whole class benefits when someone asks a question, and I benefit too. I learn how to explain that concept better. So, I really enjoy the live setting, but making the video courses is kind of nice, too.Corey: I learned to speak publicly and give conference talks as a traveling contract trainer for Puppet years ago, and that was an eye-opening experience, just because you don't really understand something until you're teaching other people how it works. It's how I learned Git. I gave a conference talk that explained Git to people, and that was called a forcing function because I had four months to go to learn this thing I did not fully understand and welp, they're not going to move the conference for me, so I guess I'd better hustle. I wouldn't necessarily recommend that approach. These days, it seems like you have a, let's say, disproportionate level of focus on the area of Infrastructure as Code, specifically you seem to be aiming at Terraform. Is that an accurate way of describing it?Ned: That is a very accurate way of describing it. I discovered Terraform while I was doing my consulting back in 2016 era, so this was pretty early on in the product's lifecycle. But I had been using CloudFormation, and at that time, CloudFormation only supported JSON, which meant it was extra punishing. And being able to describe something more succinctly and also have access to all these functions and loops and variables, I was like, “This is amazing. Where were you a year ago?” And so, I really just jumped in with both feet into Terraform.And at a certain point, I was at a conference, and I went past the Pluralsight booth, and they mentioned that they were looking for instructors. And I thought to myself, well, I like talking about things, and I'm pretty excited about this Terraform thing. Why don't I see if they're looking for someone to do a Terraform course? And so, I went through their audition process and sure enough, that is exactly what they were looking for. They had no getting started course for Terraform at the time. I published the course in 2017, and it has been in the top 50 courses ever since on Pluralsight. So, that told me that there's definitely an appetite and maybe this is an area I should focus on a little bit more.Corey: It's a difficult area to learn. About two months ago, I started using Terraform for the first time in anger in ages. I mean, I first discovered it when I was on my way back from one of those Puppet trainings, and the person next to me was really excited about this thing that we're about to launch. Turns out that was Mitchell Hashimoto and Armon was sitting next to him on the other side. Why he had a middle seat, I'll never know.But it was a really fun conversation, just talking about how he saw the world and what he was planning on doing. And a lot of that vision was realized. What I figured out a couple months ago is both that first, I'm sort of sad that Terraform is as bad as it is, but it's the best option we've got because everything else is so much worse. It is omnipresent, though. Effectively, every client I've ever dealt with on AWS billing who has a substantial estate is managing it via Terraform.It is the lingua franca of cloud across the board. I just wish it didn't require as much care and feeding, especially for the getting-started-with-a-boilerplate type of scenario. So, much of what you type feels like it's useless stuff that should be implicit. I understand why it's not, but it feels that way. It's hard to learn.Ned: It certainly can be. And you're right, there's a certain amount of boilerplate and [sigh] code that you have to write that seems pointless. Like, do I have to actually spell this all out? And sometimes the answer is yes, and sometimes the answer is you should use a module for that. Why are you writing this entire VPC configuration out yourself? And that's the sort of thing that you learn over time is that there are shortcuts, there are ways to make the code simpler and require less care and feeding.But I think ultimately, your infrastructure, just like your software, evolves, changes has new requirements, and you need to manage it in the same way that you want to manage your software. And I wouldn't tell a software developer, “Oh, you know, you could just write it once and never go back to it. I'm sure it's fine.” And by the same token, I wouldn't tell an infrastructure developer the same thing. Now, of course, people do that and never go back and touch it, and then somebody else inherits that infrastructure and goes, “Oh, God. Where's the state data?” And no one knows, and then you're starting from scratch. But hopefully, if you have someone who's doing it responsibly, they'll be setting up Terraform in such a way that it is maintainable by somebody else.Corey: I'd sure like to hope so. I have encountered so many horrible examples of code and wondering what malicious person wrote this. And of course, it was me, 6 or 12 months ago.Ned: Always [laugh].Corey: I get to play architect around a lot of these things. In fact, that's one of the problems that I've had historically with an awful lot of different things that I've basically built, called it feature complete, let it sit for a while using the CDK or whatnot, and then oh, I want to make a small change to it. Well, first, I got to spend half a day during the entire line dependency updates and seeing what's broken and how all of that works. It feels like for better or worse, Terraform is a lot more stable than that, as in, old versions of Terraform code from blog posts from 2016 will still effectively work. Is that accurate? I haven't done enough exploring in that direction to be certain.Ned: The good thing about Terraform is you can pin the version of various things that you're using. So, if you're using a particular version of the AWS provider, you can pin it to that specific version, and it won't automatically upgrade you to the latest and greatest. If you didn't do that, then you'll get bit by the update bug, which certainly happens to some folks when they changed the provider from version 3 to version 4 and completely changed how the S3 bucket object was created. A lot of people's scripts broke that day, so I think that was the time for everyone to learn what the version argument is and how it works. But yeah, as long as you follow that general convention of pinning versions of your modules and of your resource provider, you should be in a pretty stable place when you want to update it.Corey: Well, here's the $64,000 question for you, then. Does Dependabot on your GitHub repo begin screaming at you as soon as you've done that because in one of its dependencies in some particular weird edge cases when they're dealing with unsanitized, internet-based input could wind up taking up too many system resources, for example? Which is, I guess, in an ideal world, it wouldn't be an issue, but in practice, my infrastructure team is probably not trying to attack the company from the inside. They have better paths to get there, to be very blunt.Ned: [laugh].Corey: Turns out giving someone access to a thing just directly is way easier than making them find it. But that's been one of the frustrating parts where, especially when it encounters things like, I don't know, corporate security policies of, “Oh, you must clear all of these warnings,” which well-intentioned, poorly executed seems to be the takeaway there.Ned: Yeah, I've certainly seen some implementations of tools that do static scanning of Terraform code and will come up with vulnerabilities or violations of best practice, then you have to put exceptions in there. And sometimes it'll be something like, “You shouldn't have your S3 bucket public,” which in most cases, you shouldn't, but then there's the one team that's actually publishing a front-facing static website in the S3 bucket, and then they have to get, you know, special permission from on high to ignore that warning. So, a lot of those best practices that are in the scanning tools are there for very good reasons and when you onboard them, you should be ready to see a sea of red in your scan the first time and then look through that and kind of pick through what's actually real, and we should improve in our code, and what's something that we can safely ignore because we are intentionally doing it that way.Corey: I feel like there's an awful lot of… how to put this politely… implicit dependencies that are built into things. I'll wind up figuring out how to do something by implementing it and that means I will stitch together an awful lot of blog posts, things I found on Stack Overflow, et cetera, just like a senior engineer and also Chat-Gippity will go ahead and do those things. And then the reason—like, someone asks me four years later, “Why is that thing there?” And… “Well, I don't know, but if I remove it, it might stop working, so…” there was almost a cargo-culting style of, well, it's always been there. So, is that necessary? Is it not?I'm ashamed by how often I learned something very fundamental in a system that I've been using for 20 years—namely, the command line—just by reading the man page for a command that I already, quote-unquote, “Already know how to use perfectly well.” Yeah, there's a lot of hidden gems buried in those things.Ned: Oh, my goodness, I learned something about the Terraform CLI last week that I wish I'd known two years ago. And it's been there for a long time. It's like, when you want to validate your code with the terraform validate, you can initialize without initializing the back-end, and for those who are steeped in Terraform, that means something and for everybody else, I'm sorry [laugh]. But I discovered that was an option, and I was like, “Ahhh, this is amazing.” But to get back to the sort of dependency problems and understanding your infrastructure better—because I think that's ultimately what's happening when you have to describe something using Infrastructure as Code—is you discover how the infrastructure actually works versus how you thought it worked.If you look at how—and I'm going to go into Azure-land here, so try to follow along with me—if you go into Azure-land and you look at how they construct a load balancer, the load balancer is not a single resource. It's about eight different resources that are all tied together. And AWS has something similar with how you have target groups, and you have the load balancer component and the listener and the health check and all that. Azure has the same thing. There's no actual load balancer object, per se.There's a bunch of different components that get slammed together to form that load balancer. When you look in the portal, you don't see any of that. You just see a load balancer, and you might think this is a very simple resource to configure. When it actually comes time to break it out into code, you realize, oh, this is eight different components, each of which has its own options and arguments that I need to understand. So, one of the great things that I have seen a lot of tooling up here around is doing the import of existing infrastructure into Terraform by pointing the tool at a collection of resources—whatever they are—and saying, “Go create the Terraform code that matches that thing.” And it's not going to be the most elegant code out there, but it will give you a baseline for what all the settings actually are, and other resource types are, and then you can tweak it as needed to add in input variables or remove some arguments that you're not using.Corey: Yeah, I remember when they first announced the importing of existing state. It's wow, there's an awful lot of stuff that it can be aware of that I will absolutely need to control unless I want it to start blowing stuff away every time I run the—[unintelligible 00:15:51] supposedly [unintelligible 00:15:52] thing against it. And that wasn't a lot of fun. But yeah, this is the common experience of it. I only recently was reminded of the fact that I once knew, and I'd forgotten that a public versus private subnet in AWS is a human-based abstraction, not something that is implicit to the API or the way they envision subnets existing. Kind of nice, but also weird when you have to unlearn things that you've thought you'd learned.Ned: That's a really interesting example of we think of them as very different things, and when we draw nice architecture diagrams there—these are the private subnets and these are the public ones. And when you actually go to create one using Terraform—or really another tool—there's no box that says ‘private' or ‘make this public.' It's just what does your route table look like? Are you sending that traffic out the internet gateway or are you sending it to some sort of NAT device? And how does traffic come back into that subnet? That's it. That's what makes it private versus public versus a database subnet versus any other subnet type you want to logically assign within AWS.Corey: Yeah. It's kind of fun when that stuff hits.Ned: [laugh].Corey: I am curious, as you look across the ecosystem, do you still see that learning Terraform is a primary pain point for, I guess, the modern era of cloud engineer, or has that sunk below the surface level of awareness in some ways?Ned: I think it's taken as a given to a certain degree that if you're a cloud engineer or an aspiring cloud engineer today, one of the things you're going to learn is Infrastructure as Code, and that Infrastructure as Code is probably going to be Terraform. You can still learn—there's a bunch of other tools out there; I'm not going to pretend like Terraform is the end-all be-all, right? We've got—if you want to use a general purpose programming language, you have something like Pulumi out there that will allow you to do that. If you want to use one of the cloud-native tools, you've got something like CloudFormation or Azure has Bicep. Please don't use ARM templates because they hurt. They're still JSON only, so at least CloudFormation added YAML support in there. And while I don't really like YAML, at least it's not 10,000 lines of code to spin up, like, two domain controllers in a subnet.Corey: I personally wind up resolving the dichotomy between oh, should we go with JSON or should we go with YAML by picking the third option everyone hates more. That's why I'm a staunch advocate for XML.Ned: [laugh]. I was going to say XML. Yeah oh, as someone who dealt with SOAP stuff for a while, yeah, XML was particularly painful, so I'm not sad that went away. JSON for me, I work with it better, but YAML is more readable. So, it's like it's, pick your poison on that. But yeah, there's a ton of infrastructure tools out there.They all have basically the same concepts behind them, the same core concepts because they're all deploying the same thing at the end of the day and there's only so many ways you can express that concept. So, once you learn one—say you learned CloudFormation first—then Terraform is not as big of a leap. You're still declaring stuff within a file and then having it go and make those things exist. It's just nuances between the implementation of Terraform versus CloudFormation versus Bicep.Corey: I wish that there were more straightforward abstractions, but I think that as soon as you get those, that inherently limits what you're able to do, so I don't know how you square that circle.Ned: That's been a real difficult thing is, people want some sort of universal cloud or infrastructure language and abstraction. I just want a virtual machine. I don't care what kind of platform I'm on. Just give me a VM. But then you end up very much caring [laugh] what kind of VM, what operating system, what the underlying hardware is when you get to a certain level.So, there are some workloads where you're like, I just needed to run somewhere in a container and I really don't care about any of the underlying stuff. And that's great. That's what Platform as a Service is for. If that's your end goal, go use that. But if you're actually standing up infrastructure for any sort of enterprise company, then you need an abstraction that gives you access to all the underlying bits when you want them.So, if I want to specify different placement groups about my VM, I need access to that setting to create a placement group. And if I have this high-level of abstraction of a virtual machine, it doesn't know what a placement group is, and now I'm stuck at that level of abstraction instead of getting down to the guts, or I'm going into the portal or the CLI and modifying it outside of the tool that I'm supposed to be using.Corey: I want to change gears slightly here. One thing that has really been roiling some very particular people with very specific perspectives has been the BSL license change that Terraform has wound up rolling out. So far, the people that I've heard who have the strongest opinions on it tend to fall into one of three categories: either they work at HashiCorp—fair enough, they work at one of HashiCorp's direct competitors—which yeah, okay, sure, or they tend to be—how to put this delicately—open-source evangelists, of which I freely admit I used to be one and then had other challenges I needed to chase down in other ways. So, I'm curious as to where you, who are not really on the vendor side of this at all, how do you see it shaking out?Ned: Well, I mean, just for some context, essentially what HashiCorp decided to do was to change the licensing from Mozilla Public licensing to BSL for, I think eight of their products and Terraform was amongst those. And really, this sort of tells you where people are. The only one that anybody really made any noise about was Terraform. There's plenty of people that use Vault, but I didn't see a big brouhaha over the fact that Vault changed its licensing. It's really just about Terraform. Which tells you how important it is to the ecosystem.And if I look at the folks that are making the most noise about it, it's like you said, they basically fall into one of two camps: it's the open-source code purists who believe everything should be licensed in completely open-source ways, or at least if you start out with an open-source license, you can't convert to something else later. And then there is a smaller subset of folks who work for HashiCorp competitors, and they really don't like the idea of having to pay HashiCorp a regular fee for what used to be ostensibly free to them to use. And so, what they ended up doing was creating a fork of Terraform, just before the licensing change happened and that fork of Terraform was originally called OpenTF, and they had an OpenTF manifesto. And I don't know about you, when I see the word ‘manifesto,' I back away slowly and try not to make any sudden moves.Corey: You really get the sense there's going to be a body count tied to this. And people are like, “What about the Agile Manifesto?” “Yeah, what about it?”Ned: [laugh]. Yeah, I'm just—when I see ‘manifesto,' I get a little bit nervous because either someone is so incredibly passionate about something that they've kind of gone off the deep end a little bit, or they're being somewhat duplicitous, and they have ulterior motives, let's say. Now, I'm not trying to cast aspersions on anybody. I can't read anybody's mind and tell you exactly what their intention was behind it. I just know that the manifesto reads a little bit like an open-source purist and a little bit like someone having a temper tantrum, and vacillating between the two.But cooler heads prevailed a little bit, and now they have changed the name to OpenTofu, and it has been accepted by the Linux Foundation as a project. So, it's now a member of the Linux Foundation, with all the gravitas that that comes with. And some people at HashiCorp aren't necessarily happy about the Linux Foundation choosing to pull that in.Corey: Yeah, I saw a whole screed, effectively, that their CEO wound up brain-dumping on that frankly, from a messaging perspective, he would have been better served as not to say anything at all, to be very honest with you.Ned: Yeah, that was a bit of a yikes moment for me.Corey: It's very rare that you will listen yourself into trouble as opposed to opening your mouth and getting yourself into trouble.Ned: Exactly.Corey: You wouldn't think I would be one of those—of all people who would have made that observation, you wouldn't think I would be on that list, yet here I am.Ned: Yeah. And I don't think either side is entirely blameless. I understand the motivations behind HashiCorp wanting to make the change. I mean, they're a publicly traded company now and ostensibly that means that they should be making some amount of money for their investors, so they do have to bear that in mind. I don't necessarily think that changing the licensing of Terraform is the way to make that money.I think in the long-term, it's not going—it may not hurt them a lot, but I don't think it's going to help them out a lot, and it's tainted the goodwill of the community to a certain degree. On the other hand, I don't entirely trust what the other businesses are saying as well in their stead. So, there's nobody in this that comes out a hundred percent clean [laugh] on the whole process.Corey: Yeah, I feel like, to be direct, the direct competitors to HashiCorp along its various axes are not the best actors necessarily to complain about what is their largest competitor no longer giving them access to continue to compete against them with their own product. I understand the nuances there, but it also doesn't feel like they are the best ambassadors for that. I also definitely understand where HashiCorp is coming from where, why are we investing all this time, energy, and effort for people to basically take revenue away from us? But there's also the bigger problem, which is, by and large, compared to how many sites are running Terraform and the revenues that HashiCorp puts up for it, they're clearly failing to capture the value they have delivered in a massive way. But counterpoint, if they hadn't been open-source for their life until this point, would they have ever captured that market share? Probably not.Ned: Yeah, I think ultimately, the biggest competitor to their paid offering of Terraform is their free version of Terraform. It literally has enough bells and whistles already included and plenty of options for automating those things and solving the problems that their enterprise product solves that their biggest problem is not other competitors in the Terraform landscape; it's the, “Well, we already have something, and it's good enough.” And I'm not sure how you sell to that person, that's why I'm not in marketing, but I think that is their biggest competitor is the people who already have a solution and are like, “Why do I need to pay for your thing when my thing works well enough?”Corey: That's part of the strange thing that I'm seeing as I look across this entire landscape is it feels like this is not something that is directly going to impact almost anyone out there who's just using this stuff, either the open-source version as a paying customer of any of these things, but it is going to kick up a bunch of dust. And speaking of poor messaging, HashiCorp is not really killing it this quarter, where the initial announcement led to so many questions that were unclear, such as—like, they fixed this later in the frequently asked questions list, but okay, “I'm using Terraform right now and that's fine. I'm building something else completely different. Am I going to lose my access to Terraform if you decide to launch a feature that does what my company does?” And after a couple of days, they put up an indemnity against that. Okay, fine.Like, when Mongo did this, there was a similar type of dynamic that was emerging, but a lot fewer people are writing their own database engine to then sell onward to customers that are provisioning infrastructure on behalf of their customers. And where the boundaries lay for who was considered a direct Terraform competitor was unclear. I'm still not convinced that it is clear enough to bet the business on for a lot of these folks. It comes down to say what you mean, not—instead of hedging, you're not helping your cause any.Ned: Yeah, I think out of the different products that they have, some are very clear-cut. Like, Vault is a server that runs as a service, and so that's very clear what that product is and where the lines of delineation are around Vault. If I go stand up a bunch of Vault servers and offer them as a service, then that is clearly a competitor. But if I have an automation pipeline service and people can technically automate Terraform deployments with my service, even if that's not the core thing that I'm looking to do, am I now a competitor? Like, it's such a fuzzy line because Terraform isn't an application, it's not a server that runs somewhere, it's a CLI tool and a programming language. So yeah, those lines are very, very fuzzy. And I… like I said, it would be better if they say what they meant, as opposed to sort of the mealy-mouthed language that they ended up using and the need to publish multiple revisions of that FAQ to clarify their position on very specific niche use cases.Corey: Yeah, I'm not trying to be difficult or insulting or anything like that. These are hard problems that everyone involved is wrestling with. It just felt a little off, and I think the messaging did them no favors when that wound up hitting. And now, everyone is sort of trying to read the tea leaves and figure out what does this mean because in isolation, it doesn't mean anything. It is a forward-looking thing.Whatever it is you're doing today, no changes are needed for you, until the next version comes out, in which case, okay, now do we incorporate the new thing or don't we? Today, to my understanding, whether I'm running Terraform or OpenTofu entirely comes down to which binary am I invoking to do the apply? There is no difference of which I am aware. That will, of course, change, but today, I don't have to think about that.Ned: Right. OpenTofu is a literal fork of Terraform, and they haven't really added much in the way of features, so it should be completely compatible with Terraform. The two will diverge in the future as feature as new features get added to each one. But yeah, for folks who are using it today, they might just decide to stay on the version pre-fork and stay on that for years. I think HashiCorp has pledged 18 months of support for any minor version of Terraform, so you've got at least a year-and-a-half to decide. And we were kind of talking before the recording, 99% of people using Terraform do not care about this. It does not impact their daily workflow.Corey: No. I don't see customers caring at all. And also, “Oh, we're only going to use the pre-fork version of Terraform,” they're like, “Thanks for the air cover because we haven't updated any of that stuff in five years, so tha”—Ned: [laugh].Corey: “Oh yeah, we're doing it out of license concern. That's it. That's the reason we haven't done anything recent with it.” Because once it's working, changes are scary.Ned: Yeah.Corey: Terraform is one of those scary things, right next to databases, that if I make a change that I don't fully understand—and no one understands everything, as we've covered—then this could really ruin my week. So, I'm going to be very cautious around that.Ned: Yeah, if metrics are to be believed across the automation platforms, once an infrastructure rollout happens with a particular version of Terraform, that version does not get updated. For years. So, I have it on good authority that there's still Terraform version 0.10 and 0.11 running on these automation platforms for really old builds where people are too scared to upgrade to, like, post 0.12 where everything changed in the language.I believe that. People don't want to change it, especially if it's working. And so, for most people, this licensing chain doesn't matter. And all the constant back and forth and bickering just makes people feel a little nervous, and it might end up pushing people away from Terraform as a platform entirely, as opposed to picking a side.Corey: Yeah, and I think that that is probably the fair way to view it at this point where right now—please, friends at HashiCorp and HashiCorp competitors don't yell at me for this—it's basically a nerd slap-fight at the moment.Ned: [laugh].Corey: And of one of the big reasons that I also stay out of these debates almost entirely is that I married a corporate attorney who used to be a litigator and I get frustrated whenever it comes down to license arguments because you see suddenly a bunch of engineers who get to cosplay as lawyers, and reading the comments is infuriating once you realize how a little bit of this stuff works, which I've had 15 years of osmotic learning on this stuff. Whenever I want to upset my wife, I just read some of these comments aloud and then our dinner conversation becomes screaming. It's wonderful.Ned: Bad legal takes? Yeah, before—Corey: Exactly.Ned: Before my father became a social studies teacher, he was a lawyer for 20 years, and so I got to absorb some of the thought process of the lawyer. And yeah, I read some of these takes, and I'm like, “That doesn't sound right. I don't think that would hold up in any court of law.” Though a lot of the open-source licensing I don't think has been tested in any sort of court of law. It's just kind of like, “Well, we hope this stands up,” but nobody really has the money to check.Corey: Yeah. This is the problem with these open-source licenses as well. Very few have never been tested in any meaningful way because I don't know about you, but I don't have a few million dollars in legal fees lying around to prove the point.Ned: Yeah.Corey: So, it's one of those we think this is sustainable, and Lord knows the number of companies that have taken reliances on these licenses, they're probably right. I'm certainly not going to disprove the fact—please don't sue me—but yeah, this is one of those things that we're sort of assuming is the case, even if it's potentially not. I really want to thank you for taking the time to discuss how it is you view these things and talk about what it is you're up to. If people want to learn more, where's the best place for them to find you?Ned: Honestly, just go to my website. It's nedinthecloud.com. And you can also find me on LinkedIn. I don't really go for Twitter anymore.Corey: I envy you. I wish I could wean myself off of it. But we will, of course, include a link to that in the show notes. Thank you so much for being so generous with your time. It's appreciated.Ned: It's been a pleasure. Thanks, Corey.Corey: Net Bellavance, founder and curious human at Ned in the Cloud. I'm Cloud Economist Corey Quinn, and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice, along with an angry comment that I will then fork under a different license and claim as my own.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business, and we get to the point. Visit duckbillgroup.com to get started.

On this week's show: MyQ is removed from Home Assistant faster than you can say “YAML” and the world quickly finds better, cheaper alternatives, Josh.ai and AVA respond to the SnapOne lawsuit, Resi Systems releases their 2023 State of the Industry report, Google launches an AI to create scripts for you in Google Home, Disney takes over Hulu, and Switchbot Matters. All of this, a pick of the week, and project updates!

The first workshops and talks from the AI Engineer Summit are now up! Join the >20k viewers on YouTube, find clips on Twitter (we're also clipping @latentspacepod), and chat with us on Discord!Text-to-SQL was one of the first applications of NLP. Thoughtspot offered “Ask your data questions” as their core differentiation compared to traditional dashboarding tools. In a way, they provide a much friendlier interface with your own structured (aka “tabular”, as in “SQL tables”) data, the same way that RLHF and Instruction Tuning helped turn the GPT-3 of 2020 into the ChatGPT of 2022.Today, natural language queries on your databases are a commodity. There are 4 different ChatGPT plugins that offer this, as well as a bunch of startups like one of our previous guests, Seek.ai. Perplexity originally started with a similar product in 2022: In March 2023 LangChain wrote a blog post on LLMs and SQL highlighting why they don't consistently work:* “LLMs can write SQL, but they are often prone to making up tables, making up field”* “LLMs have some context window which limits the amount of text they can operate over”* “The SQL it writes may be incorrect for whatever reason, or it could be correct but just return an unexpected result.”For example, if you ask a model to “return all active users in the last 7 days” it might hallucinate a `is_active` column, join to an `activity` table that doesn't exist, or potentially get the wrong date (especially in leap years!).We previously talked to Shreya Rajpal at Guardrails AI, which also supports Text2SQL enforcement. Their approach was to run the actual SQL against your database and then use the error messages to improve the query: Semantic Layers to the rescueCube is an open source semantic layer which recently integrated with LangChain to solve these issues in a different way. You can use YAML, Javascript, or Python to create definitions of different metrics, measures and dimensions for your data: Creating these metrics and passing them in the model context limits the possibility for errors as the model just needs to query the `active_users` view, and Cube will then expand that into the full SQL in a reliable way. The downside of this approach compared to the Guardrails one for example is that it requires more upfront work to define metrics, but on the other hand it leads to more reliable and predictable outputs. The promise of adding a great semantic layer to your LLM app is irresistible - you greatly minimize hallucinations, make much more token efficient prompts, and your data stays up to date without any retraining or re-indexing. However, there are also difficulties with implementing semantic layers well, so we were glad to go deep on the topic with Artem as one of the leading players in this space!Timestamps* [00:00:00] Introductions* [00:01:28] Statsbot and limitations of natural language processing in 2017* [00:04:27] Building Cube as the infrastructure for Statsbot* [00:08:01] Open sourcing Cube in 2019* [00:09:09] Explaining the concept of a semantic layer/Cube* [00:11:01] Using semantic layers to provide context for AI models working with tabular data* [00:14:47] Workflow of generating queries from natural language via semantic layer* [00:21:07] Using Cube to power customer-facing analytics and natural language interfaces* [00:22:38] Building data-driven AI applications and agents* [00:25:59] The future of the modern data stack* [00:29:43] Example use cases of Slack bots powered by Cube* [00:30:59] Using GPT models and limitations around math* [00:32:44] Tips for building data-driven AI apps* [00:35:20] Challenges around monetizing embedded analytics* [00:36:27] Lightning RoundTranscriptSwyx: Hey everyone, welcome to the Latent Space podcast. This is Swyx, writer, editor of Latent Space and founder of Smol.ai and Alessio, partner and CTO in residence at Decibel Partners. [00:00:15]Alessio: Hey everyone, and today we have Artem Keydunov on the podcast, co-founder of Cube. Hey Artem. [00:00:21]Artem: Hey Alessio, hi Swyx. Good to be here today, thank you for inviting me. [00:00:25]Alessio: Yeah, thanks for joining. For people that don't know, I've known Artem for a long time, ever since he started Cube. And Cube is actually a spin-out of his previous company, which is Statsbot. And this kind of feels like going both backward and forward in time. So the premise of Statsbot was having a Slack bot that you can ask, basically like text to SQL in Slack, and this was six, seven years ago, something like that. A lot ahead of its time, and you see startups trying to do that today. And then Cube came out of that as a part of the infrastructure that was powering Statsbot. And Cube then evolved from an embedded analytics product to the semantic layer and just an awesome open source evolution. I think you have over 16,000 stars on GitHub today, you have a very active open source community. But maybe for people at home, just give a quick like lay of the land of the original Statsbot product. You know, what got you interested in like text to SQL and what were some of the limitations that you saw then, the limitations that you're also seeing today in the new landscape? [00:01:28]Artem: I started Statsbot in 2016. The original idea was to just make sort of a side project based off my initial project that I did at a company that I was working for back then. And I was working for a company that was building software for schools, and we were using Slack a lot. And Slack was growing really fast, a lot of people were talking about Slack, you know, like Slack apps, chatbots in general. So I think it was, you know, like another wave of, you know, bots and all that. We have one more wave right now, but it always comes in waves. So we were like living through one of those waves. And I wanted to build a bot that would give me information from different places where like a data lives to Slack. So it was like developer data, like New Relic, maybe some marketing data, Google Analytics, and then some just regular data, like a production database, so it sells for sometimes. And I wanted to bring it all into Slack, because we were always chatting, you know, like in Slack, and I wanted to see some stats in Slack. So that was the idea of Statsbot, right, like bring stats to Slack. I built that as a, you know, like a first sort of a side project, and I published it on Reddit. And people started to use it even before Slack came up with that Slack application directory. So it was a little, you know, like a hackish way to install it, but people are still installing it. So it was a lot of fun. And then Slack kind of came up with that application directory, and they reached out to me and they wanted to feature Statsbot, because it was one of the already being kind of widely used bots on Slack. So they featured me on this application directory front page, and I just got a lot of, you know, like new users signing up for that. It was a lot of fun, I think, you know, like, but it was sort of a big limitation in terms of how you can process natural language, because the original idea was to let people ask questions directly in Slack, right, hey, show me my, you know, like opportunities closed last week or something like that. My co founder, who kind of started helping me with this Slack application, him and I were trying to build a system to recognize that natural language. But it was, you know, we didn't have LLMs right back then and all of that technology. So it was really hard to build the system, especially the systems that can kind of, you know, like keep talking to you, like maintain some sort of a dialogue. It was a lot of like one off requests, and like, it was a lot of hit and miss, right? If you know how to construct a query in natural language, you will get a result back. But you know, like, it was not a system that was capable of, you know, like asking follow up questions to try to understand what you actually want. And then kind of finally, you know, like, bring this all context and go to generate a SQL query, get the result back and all of that. So that was a really missing part. And I think right now, that's, you know, like, what is the difference? So right now, I kind of bullish that if I would start Statsbot again, probably would have a much better shot at it. But back then, that was a big limitation. We kind of build a queue, right, as we were working on Statsbot, because we needed it. [00:04:27]Alessio: What was the ML stack at the time? Were you building, trying to build your own natural language understanding models, like were there open source models that were good that you were trying to leverage? [00:04:38]Artem: I think it was mostly combination of a bunch of things. And we tried a lot of different approaches. The first version, which I built, like was Regex. They were working well. [00:04:47]Swyx: It's the same as I did, I did option pricing when I was in finance, and I had a natural language pricing tool thing. And it was Regex. It was just a lot of Regex. [00:04:59]Artem: Yeah. [00:05:00]Artem: And my co-founder, Pavel, he's much smarter than I am. He's like PhD in math, all of that. And he started to do some stuff. I was like, no, you just do that stuff. I don't know. I can do Regex. And he started to do some models and trying to either look at what we had on the market back then, or try to build a different sort of models. Again, we didn't have any foundation back in place, right? We wanted to try to use existing math, obviously, right? But it was not something that we can take the model and try and run it. I think in 2019, we started to see more of stuff, like ecosystem being built, and then it eventually kind of resulted in all this LLM, like what we have right now. But back then in 2016, it was not much available for just the people to build on top. It was some academic research, right, kind of been happening. But it was very, very early for something to actually be able to use. [00:05:58]Alessio: And then that became Cube, which started just as an open source project. And I think I remember going on a walk with you in San Mateo in 2020, something like that. And you had people reaching out to you who were like, hey, we use Cube in production. I just need to give you some money, even though you guys are not a company. What's the story of Cube then from Statsbot to where you are today? [00:06:21]Artem: We built a Cube at Statsbot because we needed it. It was like, the whole Statsbot stack was that we first tried to translate the initial sort of language query into some sort of multidimensional query. It's like we were trying to understand, okay, people wanted to get active opportunities, right? What does it mean? Is it a metric? Is it what a dimension here? Because usually in analytics, you always, you know, like, try to reduce everything down to the sort of, you know, like a multidimensional framework. So that was the first step. And that's where, you know, like it didn't really work well because all this limitation of us not having foundational technologies. But then from the multidimensional query, we wanted to go to SQL. And that's what was SemanticLayer and what was Cube essentially. So we built a framework where you would be able to map your data into this concept, into this metrics. Because when people were coming to Statsbot, they were bringing their own datasets, right? And the big question was, how do we tell the system what is active opportunities for that specific users? How we kind of, you know, like provide that context, how we do the training. So that's why we came up with the idea of building the SemanticLayer so people can actually define their metrics and then kind of use them as a Statsbot. So that's how we built a Cube. At some point, we saw people started to see more value in the Cube itself, you know, like kind of building the SemanticLayer and then using it to power different types of the application. So in 2019, we decided, okay, it feels like it might be a standalone product and a lot of people want to use it. Let's just try to open source it. So we took it out of Statsbot and open-sourced. [00:08:01]Swyx: Can I make sure that everyone has the same foundational knowledge? The concept of a cube is not something that you invented. I think, you know, not everyone has the same background in analytics and data that all three of us do. Maybe you want to explain like OLAP Cube, HyperCube, the brief history of cubes. Right. [00:08:17]Artem: I'll try, you know, like a lot of like Wikipedia pages and like a lot of like a blog post trying to go into academics of it. So I'm trying to like... [00:08:25]Swyx: Cube's according to you. Yeah. [00:08:27]Artem: So when we think about just a table in a database, the problem with the table, it's not a multidimensional, meaning that in many cases, if we want to slice the data, we kind of need to result with a different table, right? Like think about when you're writing a SQL query to answer one question, SQL query always ends up with a table, right? So you write one SQL, you got one. And then you write to answer a different question, you write a second query. So you're kind of getting a bunch of tables. So now let's imagine that we can kind of bring all these tables together into multidimensional table. And that's essentially Cube. So it's just like the way that we can have measures and dimension that can potentially be used at the same time from a different angles. [00:09:09]Alessio: So initially, a lot of your use cases were more BI related, but you recently released a LangChain integration. There's obviously more and more interest in, again, using these models to answer data questions. So you've seen the chat GPT code interpreter, which is renamed as like advanced data analysis. What's kind of like the future of like the semantic layer in AI? You know, what are like some of the use cases that you're seeing and why do you think it's a good strategy to make it easier to do now the text to SQL you wanted to do seven years ago? [00:09:39]Artem: Yeah. So, I mean, you know, when it started to happen, I was just like, oh my God, people are now building Statsbot with Cube. They just have a better technology for, you know, like natural language. So it kind of, it made sense to me, you know, like from the first moment I saw it. So I think it's something that, you know, like happening right now and chat bot is one of the use cases. I think, you know, like if you try to generalize it, the use case would be how do we use structured or tabular data with, you know, like AI models, right? Like how do we turn the data and give the context as a data and then bring it to the model and then model can, you know, like give you answers, make a questions, do whatever you want. But the question is like how we go from just the data in your data warehouse, database, whatever, which is usually just a tabular data, right? Like in a SQL based warehouses to some sort of, you know, like a context that system can do. And if you're building this application, you have to do it. It's like no way you can get away around not doing this. You either map it manually or you come up with some framework or something else. So our take is that and my take is that semantic layer is just really good place for this context to leave because you need to give this context to the humans. You need to give that context to the AI system anyway, right? So that's why you define metric once and then, you know, like you teach your AI system what this metric is about. [00:11:01]Alessio: What are some of the challenges of using tabular versus language data and some of the ways that having the semantic layer kind of makes that easier maybe? [00:11:09]Artem: Imagine you're a human, right? And you're going into like your new data analyst at a company and just people give you a warehouse with a bunch of tables and they tell you, okay, just try to make sense of this data. And you're going through all of these tables and you're really like trying to make sense without any, you know, like additional context or like some columns. In many cases, they might have a weird names. Sometimes, you know, if they follow some kind of like a star schema or, you know, like a Kimball style dimensions, maybe that would be easier because you would have facts and dimensions column, but it's still, it's hard to understand and kind of make sense because it doesn't have descriptions, right? And then there is like a whole like industry of like a data catalogs exist because the whole purpose of that to give context to the data so people can understand that. And I think the same applies to the AI, right? Like, and the same challenge is that if you give it pure tabular data, it doesn't have this sort of context that it can read. So you sort of needed to write a book or like essay about your data and give that book to the system so it can understand it. [00:12:12]Alessio: Can you run through the steps of how that works today? So the initial part is like the natural language query, like what are the steps that happen in between to do model, to semantic layer, semantic layer, to SQL and all that flow? [00:12:26]Artem: The first key step is to do some sort of indexing. That's what I was referring to, like write a book about your data, right? Describe in a text format what your data is about, right? Like what metrics it has, dimensions, what is the structures of that, what a relationship between those metrics, what are potential values of the dimensions. So sort of, you know, like build a really good index as a text representation and then turn it into embeddings into your, you know, like a vector storage. Once you have that, then you can provide that as a context to the model. I mean, there are like a lot of options, like either fine tune or, you know, like sort of in context learning, but somehow kind of give that as a context to the model, right? And then once this model has this context, it can create a query. Now the query I believe should be created against semantic layer because it reduces the room for the error. Because what usually happens is that your query to semantic layer would be very simple. It would be like, give me that metric group by that dimension and maybe that filter should be applied. And then your real query for the warehouse, it might have like a five joins, a lot of different techniques, like how to avoid fan out, fan traps, chasm traps, all of that stuff. And the bigger query, the more room that the model can make an error, right? Like even sometimes it could be a small error and then, you know, like your numbers is going to be off. But making a query against semantic layer, that sort of reduces the error. So the model generates a SQL query and then it executes us again, semantic layer. And semantic layer executes us against your warehouse and then sends result all the way back to the application. And then can be done multiple times because what we were missing was both this ability to have a conversation, right? With the model. You can ask question and then system can do a follow-up questions, you know, like then do a query to get some additional information based on this information, do a query again. And sort of, you know, like it can keep doing this stuff and then eventually maybe give you a big report that consists of a lot of like data points. But the whole flow is that it knows the system, it knows your data because you already kind of did the indexing and then it queries semantic layer instead of a data warehouse directly. [00:14:47]Alessio: Maybe just to make it a little clearer for people that haven't used a semantic layer before, you can add definitions like revenue, where revenue is like select from customers and like join orders and then sum of the amount of orders. But in the semantic layer, you're kind of hiding all of that away. So when you do natural language to queue, it just select revenue from last week and then it turns into a bigger query. [00:15:12]Swyx: One of the biggest difficulties around semantic layer for people who've never thought about this concept before, this all sounds super neat until you have multiple stakeholders within a single company who all have different concepts of what a revenue is. They all have different concepts of what active user is. And then they'll have like, you know, revenue revision one by the sales team, you know, and then revenue revision one, accounting team or tax team, I don't know. I feel like I always want semantic layer discussions to talk about the not so pretty parts of the semantic layer, because this is where effectively you ship your org chart in the semantic layer. [00:15:47]Artem: I think the way I think about it is that at the end of the day, semantic layer is a code base. And in Qubit, it's essentially a code base, right? It's not just a set of YAML files with pythons. I think code is never perfect, right? It's never going to be perfect. It will have a lot of, you know, like revisions of code. We have a version control, which helps it's easier with revisions. So I think we should treat our metrics and semantic layer as a code, right? And then collaboration is a big part of it. You know, like if there are like multiple teams that sort of have a different opinions, let them collaborate on the pull request, you know, they can discuss that, like why they think that should be calculated differently, have an open conversation about it, you know, like when everyone can just discuss it, like an open source community, right? Like you go on a GitHub and you talk about why that code is written the way it's written, right? It should be written differently. And then hopefully at some point you can come up, you know, like to some definition. Now if you still should have multiple versions, right? It's a code, right? You can still manage it. But I think the big part of that is that like, we really need to treat it as a code base. Then it makes a lot of things easier, not as spreadsheets, you know, like a hidden Excel files. [00:16:53]Alessio: The other thing is like then having the definition spread in the organization, like versus everybody trying to come up with their own thing. But yeah, I'm sure that when you talk to customers, there's people that have issues with the product and it's really like two people trying to define the same thing. One in sales that wants to look good, the other is like the finance team that wants to be conservative and they all have different definitions. How important is the natural language to people? Obviously you guys both work in modern data stack companies either now or before. There's going to be the whole wave of empowering data professionals. I think now a big part of the wave is removing the need for data professionals to always be in the loop and having non-technical folks do more of the work. Are you seeing that as a big push too with these models, like allowing everybody to interact with the data? [00:17:42]Artem: I think it's a multidimensional question. That's an example of, you know, like where you have a lot of inside the question. In terms of examples, I think a lot of people building different, you know, like agents or chatbots. You have a company that built an internal Slack bot that sort of answers questions, you know, like based on the data in a warehouse. And then like a lot of people kind of go in and like ask that chatbot this question. Is it like a real big use case? Maybe. Is it still like a toy pet project? Maybe too right now. I think it's really hard to tell them apart at this point because there is a lot of like a hype, you know, and just people building LLM stuff because it's cool and everyone wants to build something, you know, like even at least a pet project. So that's what happened in Krizawa community as well. We see a lot of like people building a lot of cool stuff and it probably will take some time for that stuff to mature and kind of to see like what are real, the best use cases. But I think what I saw so far, one use case was building this chatbot and we have even one company that are building it as a service. So they essentially connect into Q semantic layer and then offering their like chatbot So you can do it in a web, in a slack, so it can, you know, like answer questions based on data in your semantic layer, but also see a lot of things like they're just being built in house. And there are other use cases, sort of automation, you know, like that agent checks on the data and then kind of perform some actions based, you know, like on changes in data. But other dimension of your question is like, will it replace people or not? I think, you know, like what I see so far in data specifically, you know, like a few use cases of LLM, I don't see Q being part of that use case, but it's more like a copilot for data analyst, a copilot for data engineer, where you develop something, you develop a model and it can help you to write a SQL or something like that. So you know, it can create a boilerplate SQL, and then you can edit this SQL, which is fine because you know how to edit SQL, right? So you're not going to make a mistake, but it will help you to just generate, you know, like a bunch of SQL that you write again and again, right? Like boilerplate code. So sort of a copilot use case. I think that's great. And we'll see more of it. I think every platform that is building for data engineers will have some sort of a copilot capabilities and Cubectl, we're building this copilot capabilities to help people build semantic layers easier. I think that just a baseline for every engineering product right now to have some sort of, you know, like a copilot capabilities. Then the other use case is a little bit more where Cube is being involved is like, how do we enable access to data for non-technical people through the natural language as an interface to data, right? Like visual dashboards, charts, it's always has been an interface to data in every BI. Now I think we will see just a second interface as a just kind of a natural language. So I think at this point, many BI's will add it as a commodity feature is like Tableau will probably have a search bar at some point saying like, Hey, ask me a question. I know that some of the, you know, like AWS Squeak site, they're about to announce features like this in their like BI. And I think Power BI will do that, especially with their deal with open AI. So every company, every BI will have this some sort of a search capabilities built in inside their BI. So I think that's just going to be a baseline feature for them as well. But that's where Cube can help because we can provide that context, right? [00:21:07]Alessio: Do you know how, or do you have an idea for how these products will differentiate once you get the same interface? So right now there's like, you know, Tableau is like the super complicated and it's like super sad. It's like easier. Yeah. Do you just see everything will look the same and then how do people differentiate? [00:21:24]Artem: It's like they all have line chart, right? And they all have bar chart. I feel like it pretty much the same and it's going to be fragmented as well. And every major vendor and most of the vendors will try to have some sort of natural language capabilities and they might be a little bit different. Some of them will try to position the whole product around it. Some of them will just have them as a checkbox, right? So we'll see, but I don't think it's going to be something that will change the BI market, you know, like something that will can take the BI market and make it more consolidated rather than, you know, like what we have right now. I think it's still will remain fragmented. [00:22:04]Alessio: Let's talk a bit more about application use cases. So people also use Q for kind of like analytics in their product, like dashboards and things like that. How do you see that changing and more, especially like when it comes to like agents, you know, so there's like a lot of people trying to build agents for reporting, building agents for sales. If you're building a sales agent, you need to know everything about the purchasing history of the customer. All of these things. Yeah. Any thoughts there? What should all the AI engineers listening think about when implementing data into agents? [00:22:38]Artem: Yeah, I think kind of, you know, like trying to solve for two problems. One is how to make sure that agents or LLM model, right, has enough context about, you know, like a tabular data and also, you know, like how do we deliver updates to the context, which is also important because data is changing, right? So every time we change something upstream, we need to surely update that context in our vector database or something. And how do you make sure that the queries are correct? You know, I think it's obviously a big pain and that's all, you know, like AI kind of, you know, like a space right now, how do we make sure that we don't, you know, provide our own cancers, but I think, you know, like be able to reduce the room for error as much as possible that what I would look for, you know, like to try to like minimize potential damage. And then our use case for Qube, it's been using a lot to power sort of customer facing analytics. So I don't think much is going to change is that I feel like again, more and more products will adopt natural language interfaces as sort of a part of that product as well. So we would be able to power this business to not only, you know, like a chart, visuals, but also some sort of, you know, like a summaries, probably in the future, you're going to open the page with some surface stats and you will have a smart summary kind of generated by AI. And that summary can be powered by Qube, right, like, because the rest is already being powered by Qube. [00:24:04]Alessio: You know, we had Linus from Notion on the pod and one of the ideas he had that I really like is kind of like thumbnails of text, kind of like how do you like compress knowledge and then start to expand it. A lot of that comes into dashboards, you know, where like you have a lot of data, you have like a lot of charts and sometimes you just want to know, hey, this is like the three lines summary of it. [00:24:25]Artem: Exactly. [00:24:26]Alessio: Makes sense that you want to power that. How are you thinking about, yeah, the evolution of like the modern data stack in quotes, whatever that means today. What's like the future of what people are going to do? What's the future of like what models and agents are going to do for them? Do you have any, any thoughts? [00:24:42]Artem: I feel like modern data stack sometimes is not very, I mean, it's obviously big crossover between AI, you know, like ecosystem, AI infrastructure, ecosystem, and then sort of a data. But I don't think it's a full overlap. So I feel like when we know, like I'm looking at a lot of like what's happening in a modern data stack where like we use warehouses, we use BI's, you know, different like transformation tools, catalogs, like data quality tools, ETLs, all of that. I don't see a lot of being compacted by AI specifically. I think, you know, that space is being compacted as much as any other space in terms of, yes, we'll have all this copilot capabilities, some of AI capabilities here and there, but I don't see anything sort of dramatically, you know, being sort of, you know, a change or shifted because of, you know, like AI wave. In terms of just in general data space, I think in the last two, three years, we saw an explosion, right? Like we got like a lot of tools, every vendor for every problem. I feel like right now we should go through the cycle of consolidation. If Fivetran and DBT merge, they can be Alteryx of a new generation or something like that. And you know, probably some ETL tool there. I feel it might happen. I mean, it's just natural waves, you know, like in cycles. [00:25:59]Alessio: I wonder if everybody is going to have their own copilot. The other thing I think about these models is like Swyx was at Airbyte and yeah, there's Fivetran. [00:26:08]Swyx: Fivetran versus AirByte, I don't think it'll mix very well. [00:26:10]Alessio: A lot of times these companies are doing the syntax work for you of like building the integration between your data store and like the app or another data store. I feel like now these models are pretty good at coming up with the integration themselves and like using the docs to then connect the two. So I'm really curious, like in the future, what that will look like. And same with data transformation. I mean, you think about DBT and some of these tools and right now you have to create rules to normalize and transform data. In the future, I could see you explaining the model, how you want the data to be, and then the model figuring out how to do the transformation. I think it all needs a semantic layer as far as like figuring out what to do with it. You know, what's the data for and where it goes. [00:26:53]Artem: Yeah, I think many of this, you know, like workflows will be augmented by, you know, like some sort of a copilot. You know, you can describe what transformation you want to see and it can generate a boilerplate right, of transformation for you, or even, you know, like kind of generate a boilerplate of specific ETL driver or ETL integration. I think we're still not at the point where this code can be fully automated. So we still need a human and a loop, right, like who can be, who can use this copilot. But in general, I think, yeah, data work and software engineering work can be augmented quite significantly with all that stuff. [00:27:31]Alessio: You know, the big thing with machine learning before was like, well, all of your data is bad. You know, the data is not good for anything. And I think like now, at least with these models, they have some knowledge of their own and they can also tell you if your data is bad, which I think is like something that before you didn't have. Any cool apps that you've seen being built on Qube, like any kind of like AI native things that people should think about, new experiences, anything like that? [00:27:54]Artem: Well, I see a lot of Slack bots. They all remind me of Statsbot, but I know like I played with a few of them. They're much, much better than Statsbot. It feels like it's on the surface, right? It's just that use case that you really want, you know, think about you, a data engineer in your company, like everyone is like, and you're asking, hey, can you pull that data for me? And you would be like, can I build a bot to replace myself? You know, like, so they can both ping that bot instead. So it's like, that's why a lot of people doing that. So I think it's a first use case that actually people are playing with. But I think inside that use case, people get creative. So I see bots that can actually have a dialogue with you. So, you know, like you would come to that bot and say, hey, show me metrics. And the bot would be like, what kind of metrics? What do you want to look at? You will be like active users. And then it would be like, how do you define active users? You want to see active users sort of cohort, you want to see active users kind of changing behavior over time, like a lot of like a follow up questions. So it tries to sort of, you know, like understand what exactly you want. And that's how many data analysts work, right? When people started to ask you something, you always try to understand what exactly do you mean? Because many people don't know how to ask correct questions about your data. It's a sort of an interesting specter. On one side of the specter, you know, nothing is like, hey, show me metrics. And the other side of specter, you know how to write SQL, and you can write exact query to your data warehouse, right? So many people like a little bit in the middle. And the data analysts, they usually have the knowledge about your data. And that's why they can ask follow up questions and to understand what exactly you want. And I saw people building bots who can do that. That part is amazing. I mean, like generating SQL, all that stuff, it's okay, it's good. But when the bot can actually act like they know that your data and they can ask follow up questions. I think that's great. [00:29:43]Swyx: Yeah. [00:29:44]Alessio: Are there any issues with the models and the way they understand numbers? One of the big complaints people have is like GPT, at least 3.5, cannot do math. Have you seen any limitations and improvement? And also when it comes to what model to use, do you see most people use like GPT-4? Because it's like the best at this kind of analysis. [00:30:03]Artem: I think I saw people use all kinds of models. To be honest, it's usually GPT. So inside GPT, it could be 3.5 or 4, right? But it's not like I see a lot of something else, to be honest, like, I mean, maybe some open source alternatives, but it feels like the market is being dominated by just chat GPT. In terms of the problems, I think chatting about it with a few people. So if math is required to do math, you know, like outside of, you know, like chat GPT itself, so it would be like some additional Python scripts or something. When we're talking about production level use cases, it's quite a lot of Python code around, you know, like your model to make it work. To be honest, it's like, it's not that magic that you just throw the model in and like it can give you all these answers. For like a toy use cases, the one we have on a, you know, like our demo page or something, it works fine. But, you know, like if you want to do like a lot of post-processing, do a mass on URL, you probably need to code it in Python anyway. That's what I see people doing. [00:30:59]Alessio: We heard the same from Harrison and LangChain that most people just use OpenAI. We did a OpenAI has no moat emergency podcast, and it was funny to like just see the reaction that people had to that and how hard it actually is to break down some of the monopoly. What else should people keep in mind, Artem? You're kind of like at the cutting edge of this. You know, if I'm looking to build a data-driven AI application, I'm trying to build data into my AI workflows. Any mistakes people should avoid? Any tips on the best stack to use? What tools to use? [00:31:32]Artem: I would just recommend going through to warehouse as soon as possible. I think a lot of people feel that MySQL can be a warehouse, which can be maybe on like a lower scale, but definitely not from a performance perspective. So just kind of starting with a good warehouse, a query engine, Lakehouse, that's probably like something I would recommend starting from a day zero. And there are good ways to do it, very cheap, with open source technologies too, especially in the Lakehouse architecture. I think, you know, I'm biased, obviously, but using a semantic layer, preferably Cube, and for, you know, like a context. And other than that, I just feel it's a very interesting space in terms of AI ecosystem. I see a lot of people using link chain right now, which is great, you know, like, and we build an integration. But I'm sure the space will continue to evolve and, you know, like we'll see a lot of interesting tools and maybe, you know, like some tools would be a better fit for a job. I'm not aware of any right now, but it's always interesting to see how it evolves. Also it's a little unclear, you know, like how all the infrastructure around actually developing, testing, documenting, all that stuff will kind of evolve too. But yeah, again, it's just like really interesting to see and observe, you know, what's happening in this space. [00:32:44]Swyx: So before we go to the lightning round, I wanted to ask you on your thoughts on embedded analytics and in a sense, the kind of chatbots that people are inserting on their websites and building with LLMs is very much sort of end user programming or end user interaction with their own data. I love seeing embedded analytics, and for those who don't know, embedded analytics is basically user facing dashboards where you can see your own data, right? Instead of the company seeing data across all their customers, it's an individual user seeing their own data as a slice of the overall data that is owned by the platform that they're using. So I love embedded analytics. Well, actually, overwhelmingly, the observation that I've had is that people who try to build in this market fail to monetize. And I was wondering your insights on why. [00:33:31]Artem: I think overall, the statement is true. It's really hard to monetize, you know, like in embedded analytics. That's why at Qube we're excited more about our internal kind of BI use case, or like a company's a building, you know, like a chatbots for their internal data consumption or like internal workflows. Embedded analytics is hard to monetize because it's historically been dominated by the BI vendors. And we still see a lot of organizations are using BI tools as vendors. And what I was talking about, BI vendors adding natural language interfaces, they will probably add that to the embedded analytics capabilities as well, right? So they would be able to embed that too. So I think that's part of it. Also, you know, if you look at the embedded analytics market, the bigger organizations are big GADs, they're really more custom, you know, like it becomes and at some point I see many organizations, they just stop using any vendor, and they just kind of build most of the stuff from scratch, which probably, you know, like the right way to do. So it's sort of, you know, like you got a market that is very kept at the top. And then you also in that middle and small segment, you got a lot of vendors trying to, you know, like to compete for the buyers. And because again, the BI is very fragmented, embedded analytics, therefore is fragmented also. So you're really going after the mid market slice, and then with a lot of other vendors competing for that. So that's why it's historically been hard to monetize, right? I don't think AI really going to change that just because it's using model, you just pay to open AI. And that's it, like everyone can do that, right? So it's not much of a competitive advantage. So it's going to be more like a commodity features that a lot of vendors would be able to leverage. [00:35:20]Alessio: This is great, Artem. As usual, we got our lightning round. So it's three questions. One is about acceleration, one on exploration, and then take away. The acceleration thing is what's something that already happened in AI or maybe, you know, in data that you thought would take much longer, but it's already happening today. [00:35:38]Artem: To be honest, all this foundational models, I thought that we had a lot of models that been in production for like, you know, maybe decade or so. And it was like a very niche use cases, very vertical use cases, it's just like in very customized models. And even when we're building Statsbot back then in 2016, right, even back then, we had some natural language models being deployed, like a Google Translate or something that was still was a sort of a model, right, but it was very customized with a specific use case. So I thought that would continue for like, many years, we will use AI, we'll have all these customized niche models. But there is like foundational model, they like very generic now, they can serve many, many different use cases. So I think that is a big change. And I didn't expect that, to be honest. [00:36:27]Swyx: The next question is about exploration. What is one thing that you think is the most interesting unsolved question in AI? [00:36:33]Artem: I think AI is a subset of software engineering in general. And it's sort of connected to the data as well. Because software engineering as a discipline, it has quite a history. We build a lot of processes, you know, like toolkits and methodologies, how we prod that, [00:36:50]Swyx: right. [00:36:51]Artem: But AI, I don't think it's completely different. But it has some unique traits, you know, like, it's quite not idempotent, right, and kind of from many dimensions and like other traits. So which kind of may require a different methodologies may require different approaches and a different toolkit. I don't think how much is going to deviate from a standard software engineering, I think many tools and practices that we develop our software engineering can be applied to AI. And some of the data best practices can be applied as well. But it's like we got a DevOps, right, like it's just a bunch of tools, like ecosystem. So now like AI is kind of feels like it's shaping into that with a lot of its own, you know, like methodologies, practices and toolkits. So I'm really excited about it. And I think it's a lot of unsolved still question again, how do we develop that? How do we test you know, like, what is the best practices? How what is a methodologist? So I think that would be an interesting to see. [00:37:44]Alessio: Awesome. Yeah. Our final message, you know, you have a big audience of engineers and technical folks, what's something you want everybody to remember to think about to explore? [00:37:55]Artem: I mean, it says being hooked to try to build a chatbot, you know, like for analytics, back then and kind of, you know, like looking at what people do right now, I think, yeah, just do that. I mean, it's working right now, with foundational models, it's actually now it's possible to build all those cool applications. I'm so excited to see, you know, like, how much changed in the last six years or so that we actually now can build a smart agents. So I think that sort of, you know, like a takeaways and yeah, we are, as humans in general, we like we really move technology forward. And it's fun to see, you know, like, it's just a first hand. [00:38:30]Alessio: Well, thank you so much for coming on Artem. [00:38:32]Swyx: This was great. [00:38:32] Get full access to Latent Space at www.latent.space/subscribe

Adnan Khan, Lead Security Engineer at Praetorian, joins Corey on Screaming in the Cloud to discuss software bill of materials and supply chain attacks. Adnan describes how simple pull requests can lead to major security breaches, and how to best avoid those vulnerabilities. Adnan and Corey also discuss the rapid innovation at Github Actions, and the pros and cons of having new features added so quickly when it comes to security. Adnan also discusses his view on the state of AI and its impact on cloud security. About AdnanAdnan is a Lead Security Engineer at Praetorian. He is responsible for executing on Red-Team Engagements as well as developing novel attack tooling in order to meet and exceed engagement objectives and provide maximum value for clients.His past experience as a software engineer gives him a deep understanding of where developers are likely to make mistakes, and has applied this knowledge to become an expert in attacks on organization's CI/CD systems.Links Referenced: Praetorian: https://www.praetorian.com/ Twitter: https://twitter.com/adnanthekhan Praetorian blog posts: https://www.praetorian.com/author/adnan-khan/ TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: Are you navigating the complex web of API management, microservices, and Kubernetes in your organization? Solo.io is here to be your guide to connectivity in the cloud-native universe!Solo.io, the powerhouse behind Istio, is revolutionizing cloud-native application networking. They brought you Gloo Gateway, the lightweight and ultra-fast gateway built for modern API management, and Gloo Mesh Core, a necessary step to secure, support, and operate your Istio environment.Why struggle with the nuts and bolts of infrastructure when you can focus on what truly matters - your application. Solo.io's got your back with networking for applications, not infrastructure. Embrace zero trust security, GitOps automation, and seamless multi-cloud networking, all with Solo.io.And here's the real game-changer: a common interface for every connection, in every direction, all with one API. It's the future of connectivity, and it's called Gloo by Solo.io.DevOps and Platform Engineers, your journey to a seamless cloud-native experience starts here. Visit solo.io/screaminginthecloud today and level up your networking game.Corey: As hybrid cloud computing becomes more pervasive, IT organizations need an automation platform that spans networks, clouds, and services—while helping deliver on key business objectives. Red Hat Ansible Automation Platform provides smart, scalable, sharable automation that can take you from zero to automation in minutes. Find it in the AWS Marketplace.Corey: Welcome to Screaming in the Cloud, I'm Corey Quinn. I've been studiously ignoring a number of buzzword, hype-y topics, and it's probably time that I addressed some of them. One that I've been largely ignoring, mostly because of its prevalence at Expo Hall booths at RSA and other places, has been software bill of materials and supply chain attacks. Finally, I figured I would indulge the topic. Today I'm speaking with Adnan Khan, lead security engineer at Praetorian. Adnan, thank you for joining me.Adnan: Thank you so much for having me.Corey: So, I'm trying to understand, on some level, where the idea of these SBOM or bill-of-material attacks have—where they start and where they stop. I've seen it as far as upstream dependencies have a vulnerability. Great. I've seen misconfigurations in how companies wind up configuring their open-source presences. There have been a bunch of different, it feels almost like orthogonal concepts to my mind, lumped together as this is a big scary thing because if we have a big single scary thing we can point at, that unlocks budget. Am I being overly cynical on this or is there more to it?Adnan: I'd say there's a lot more to it. And there's a couple of components here. So first, you have the SBOM-type approach to security where organizations are looking at which packages are incorporated into their builds. And vulnerabilities can come out in a number of ways. So, you could have software actually have bugs or you could have malicious actors actually insert backdoors into software.I want to talk more about that second point. How do malicious actors actually insert backdoors? Sometimes it's compromising a developer. Sometimes it's compromising credentials to push packages to a repository, but other times, it could be as simple as just making a pull request on GitHub. And that's somewhere where I've spent a bit of time doing research, building off of techniques that other people have documented, and also trying out some attacks for myself against two Microsoft repositories and several others that have reported over the last few months that would have been able to allow an attacker to slip a backdoor into code and expand the number of projects that they are able to attack beyond that.Corey: I think one of the areas that we've seen a lot of this coming from has been the GitHub Action space. And I'll confess that I wasn't aware of a few edge-case behaviors around this. Most of my experience with client-side Git configuration in the .git repository—pre-commit hooks being a great example—intentionally and by design from a security perspective, do not convey when you check that code in and push it somewhere, or grab someone else's, which is probably for the best because otherwise, it's, “Oh yeah, just go ahead and copy your password hash file and email that to something else via a series of arcane shell script stuff.” The vector is there. I was unpleasantly surprised somewhat recently to discover that when I cloned a public project and started running it locally and then adding it to my own fork, that it would attempt to invoke a whole bunch of GitHub Actions flows that I'd never, you know, allowed it to do. That was… let's say, eye-opening.Adnan: [laugh]. Yeah. So, on the particular topic of GitHub Actions, the pull request as an attack vector, like, there's a lot of different forms that an attack can take. So, one of the more common ones—and this is something that's been around for just about as long as GitHub Actions has been around—and this is a certain trigger called ‘pull request target.' What this means is that when someone makes a pull request against the base repository, maybe a branch within the base repository such as main, that will be the workflow trigger.And from a security's perspective, when it runs on that trigger, it does not require approval at all. And that's something that a lot of people don't really realize when they're configuring their workflows. Because normally, when you have a pull request trigger, the maintainer can check a box that says, “Oh, require approval for all external pull requests.” And they think, “Great, everything needs to be approved.” If someone tries to add malicious code to run that's on the pull request target trigger, then they can look at the code before it runs and they're fine.But in a pull request target trigger, there is no approval and there's no way to require an approval, except for configuring the workflow securely. So, in this case, what happens is, and in one particular case against the Microsoft repository, this was a Microsoft reusable GitHub Action called GPT Review. It was vulnerable because it checked out code from my branch—so if I made a pull request, it checked out code from my branch, and you could find this by looking at the workflow—and then it ran tests on my branch, so it's running my code. So, by modifying the entry points, I could run code that runs in the context of that base branch and steal secrets from it, and use those to perform malicious Actions.Corey: Got you. It feels like historically, one of the big threat models around things like this is al—[and when 00:06:02] you have any sort of CI/CD exploit—is either falls down one of two branches: it's either the getting secret access so you can leverage those credentials to pivot into other things—I've seen a lot of that in the AWS space—or more boringly, and more commonly in many cases, it seems to be oh, how do I get it to run this crypto miner nonsense thing, with the somewhat large-scale collapse of crypto across the board, it's been convenient to see that be less prevalent, but still there. Just because you're not making as much money means that you'll still just have to do more of it when it's all in someone else's account. So, I guess it's easier to see and detect a lot of the exploits that require a whole bunch of compute power. The, oh by the way, we stole your secrets and now we're going to use that to lateral into an organization seem like it's something far more… I guess, dangerous and also sneaky.Adnan: Yeah, absolutely. And you hit the nail on the head there with sneaky because when I first demonstrated this, I made a test account, I created a PR, I made a couple of Actions such as I modified the name of the release for the repository, I just put a little tag on it, and didn't do any other changes. And then I also created a feature branch in one of Microsoft's repositories. I don't have permission to do that. That just sat there for about almost two weeks and then someone else exploited it and then they responded to it.So, sneaky is exactly the word you could describe something like this. And another reason why it's concerning is, beyond the secret disclosure for—and in this case, the repository only had an OpenAI API key, so… okay, you can talk to ChatGPT for free. But this was itself a Github Action and it was used by another Microsoft machine-learning project that had a lot more users, called SynapseML, I believe was the name of the other project. So, what someone could do is backdoor this Action by creating a commit in a feature branch, which they can do by stealing the built-in GitHub token—and this is something that all Github Action runs have; the permissions for it vary, but in this case, it had the right permissions—attacker could create a new branch, modify code in that branch, and then modify the tag, which in Git, tags are mutable, so you can just change the commit the tag points to, and now, every time that other Microsoft repository runs GPT Review to review a pull request, it's running attacker-controlled code, and then that could potentially backdoor that other repository, steal secrets from that repository.So that's, you know, one of the scary parts of, in particular backdooring a Github Action. And I believe there was a very informative Blackhat talk this year, that someone from—I'm forgetting the name of the author, but it was a very good watch about how Actions vulnerabilities can be vulnerable, and this is kind of an example of—it just happened to be that this was an Action as well.Corey: That feels like this is an area of exploit that is becoming increasingly common. I tie it almost directly to the rise of GitHub Actions as the default CI/CD system that a lot of folks have been using. For the longest time, it seemed like a poorly configured Jenkins box hanging out somewhere in your environment that was the exception to the Infrastructure as Code rule because everyone has access to it, configures it by hand, and invariably it has access to production was the way that people would exploit things. For a while, you had CircleCI and Travis-CI, before Travis imploded and Circle did a bunch of layoffs. Who knows where they're at these days?But it does seem that the common point now has been GitHub Actions, and a .github folder within that Git repo with a workflows YAML file effectively means that a whole bunch of stuff can happen that you might not be fully aware of when you're cloning or following along with someone's tutorial somewhere. That has caught me out in a couple of strange ways, but nothing disastrous because I do believe in realistic security boundaries. I just worry how much of this is the emerging factor of having a de facto standard around this versus something that Microsoft has actively gotten wrong. What's your take on it?Adnan: Yeah. So, my take here is that Github could absolutely be doing a lot more to help prevent users from shooting themselves in the foot. Because their documentation is very clear and quite frankly, very good, but people aren't warned when they make certain configuration settings in their workflows. I mean, GitHub will happily take the settings and, you know, they hit commit, and now the workflow could be vulnerable. There's no automatic linting of workflows, or a little suggestion box popping up like, “Hey, are you sure you want to configure it this way?”The technology to detect that is there. There's a lot of third-party utilities that will lint Actions workflows. Heck, for looking for a lot of these pull request target-type vulnerabilities, I use a Github code search query. It's just a regular expression. So, having something that at least nudges users to not make that mistake would go really far in helping people not make these mista—you know, adding vulnerabilities to their projects.Corey: It seems like there's also been issues around the GitHub Actions integration approach where OICD has not been scoped correctly a bunch of times. I've seen a number of articles come across my desk in that context and fortunately, when I wound up passing out the ability for one of my workflows to deploy to my AWS account, I got it right because I had no idea what I was doing and carefully followed the instructions. But I can totally see overlooking that one additional parameter that leaves things just wide open for disaster.Adnan: Yeah, absolutely. That's one where I haven't spent too much time actually looking for that myself, but I've definitely read those articles that you mentioned, and yeah, it's very easy for someone to make that mistake, just like, it's easy for someone to just misconfigure their Action in general. Because in some of the cases where I found vulnerabilities, there would actually be a commit saying, “Hey, I'm making this change because the Action needs access to these certain secrets. And oh, by the way, I need to update the checkout steps so it actually checks out the PR head so that it's [testing 00:12:14] that PR code.” Like, people are actively making a decision to make it vulnerable because they don't realize the implication of what they've just done.And in the second Microsoft repository that I found the bug in, was called Microsoft Confidential Sidecar Containers. That repository, the developer a week prior to me identifying the bug made a commit saying that we're making a change and it's okay because it requires approval. Well, it doesn't because it's a pull request target.Corey: Part of me wonders how much of this is endemic to open-source as envisioned through enterprises versus my world of open-source, which is just eh, I've got this weird side project in my spare time, and it seemed like it might be useful to someone else, so I'll go ahead and throw it up there. I understand that there's been an awful lot of commercialization of open-source in recent years; I'm not blind to that fact, but it also seems like there's a lot of companies playing very fast and loose with things that they probably shouldn't be since they, you know, have more of a security apparatus than any random contributors standing up a clone of something somewhere will.Adnan: Yeah, we're definitely seeing this a lot in the machine-learning space because of companies that are trying to move so quickly with trying to build things because OpenAI AI has blown up quite a bit recently, everyone's trying to get a piece of that machine learning pie, so to speak. And another thing of what you're seeing is, people are deploying self-hosted runners with Nvidia, what is it, the A100, or—it's some graphics card that's, like, $40,000 apiece attached to runners for running integration tests on machine-learning workflows. And someone could, via a pull request, also just run code on those and mine crypto.Corey: I kind of miss the days when exploiting computers is basically just a way for people to prove how clever they were or once in a blue moon come up with something innovative. Now, it's like, well, we've gone all around the mulberry bush just so we can basically make computers solve a sudoku form, and in return, turn that into money down the road. It's frustrating, to put it gently.Adnan: [laugh].Corey: When you take a look across the board at what companies are doing and how they're embracing the emerging capabilities inherent to these technologies, how do you avoid becoming a cautionary tale in the space?Adnan: So, on the flip side of companies having vulnerable workflows, I've also seen a lot of very elegant ways of writing secure workflows. And some of the repositories are using deployment environments—which is the GitHub Actions feature—to enforce approval checks. So, workflows that do need to run on pull request target because of the need to access secrets for pull requests will have a step that requires a deployment environment to complete, and that deployment environment is just an approval and it doesn't do anything. So essentially, someone who has permissions to the repository will go in, approve that environment check, and only then will the workflow continue. So, that adds mandatory approvals to pull requests where otherwise they would just run without approval.And this is on, particularly, the pull request target trigger. Another approach is making it so the trigger is only running on the label event and then having a maintainer add a label so the tests can run and then remove the label. So, that's another approach where companies are figuring out ways to write secure workflows and not leave their repositories vulnerable.Corey: It feels like every time I turn around, Github Actions has gotten more capable. And I'm not trying to disparage the product; it's kind of the idea of what we want. But it also means that there's certainly not an awareness in the larger community of how these things can go awry that has kept up with the pace of feature innovation. How do you balance this without becoming the Department of No?Adnan: [laugh]. Yeah, so it's a complex issue. I think GitHub has evolved a lot over the years. Actions, it's—despite some of the security issues that happen because people don't configure them properly—is a very powerful product. For a CI/CD system to work at the scale it does and allow so many repositories to work and integrate with everything else, it's really easy to use. So, it's definitely something you don't want to take away or have an organization move away from something like that because they are worried about the security risks.When you have features coming in so quickly, I think it's important to have a base, kind of like, a mandatory reading. Like, if you're a developer that writes and maintains an open-source software, go read through this document so you can understand the do's and don'ts instead of it being a patchwork where some people, they take a good security approach and write secure workflows and some people just kind of stumble through Stack Overflow, find what works, messes around with it until their deployment is working and their CI/CD is working and they get the green checkmark, and then they move on to their never-ending list of tasks that—because they're always working on a deadline.Corey: Reminds me of a project I saw a few years ago when it came out that Volkswagen had been lying to regulators. It was a framework someone built called ‘Volkswagen' that would detect if it was running inside of a CI/CD environment, and if so, it would automatically make all the tests pass. I have a certain affinity for projects like that. Another one was a tool that would intentionally degrade the performance of a network connection so you could simulate having a latent or stuttering connection with packet loss, and they call that ‘Comcast.' Same story. I just thought that it's fun seeing people get clever on things like that.Adnan: Yeah, absolutely.Corey: When you take a look now at the larger stories that are emerging in the space right now, I see an awful lot of discussion coming up that ties to SBOMs and understanding where all of the components of your software come from. But I chased some stuff down for fun once, and I gave up after 12 dependency leaps from just random open-source frameworks. I mean, I see the Dependabot problem that this causes as well, where whenever I put something on GitHub and then don't touch it for a couple of months—because that's how I roll—I come back and there's a whole bunch of terrifyingly critical updates that it's warning me about, but given the nature of how these things get used, it's never going to impact anything that I'm currently running. So, I've learned to tune it out and just ignore it when it comes in, which is probably the worst of all possible approaches. Now, if I worked at a bank, I should probably take a different perspective on this, but I don't.Adnan: Mm-hm. Yeah. And that's kind of a problem you see, not just with SBOMs. It's just security alerting in general, where anytime you have some sort of signal and people who are supposed to respond to it are getting too much of it, you just start to tune all of it out. It's like that human element that applies to so much in cybersecurity.And I think for the particular SBOM problem, where, yeah, you're correct, like, a lot of it… you don't have reachability because you're using a library for one particular function and that's it. And this is somewhere where I'm not that much of an expert in where doing more static source analysis and reachability testing, but I'm certain there are products and tools that offer that feature to actually prioritize SBOM-based alerts based on actual reachability versus just having an as a dependency or not.[midroll 00:20:00]Corey: I feel like, on some level, wanting people to be more cautious about what they're doing is almost shouting into the void because I'm one of the only folks I found that has made the assertion that oh yeah, companies don't actually care about security. Yes, they email you all the time after they failed to protect your security, telling you how much they care about security, but when you look at where they invest, feature velocity always seems to outpace investment in security approaches. And take a look right now at the hype we're seeing across the board when it comes to generative AI. People are excited about the capabilities and security is a distant afterthought around an awful lot of these things. I don't know how you drive a broader awareness of this in a way that sticks, but clearly, we haven't collectively found it yet.Adnan: Yeah, it's definitely a concern. When you see things on—like for example, you can look at Github's roadmap, and there's, like, a feature there that's, oh, automatic AI-based pull request handling. Okay, so does that mean one day, you'll have a GitHub-powered LLM just approve PRs based on whether it determines that it's a good improvement or not? Like, obviously, that's not something that's the case now, but looking forward to maybe five, six years in the future, in the pursuit of that ever-increasing velocity, could you ever have a situation where actual code contributions are reviewed fully by AI and then approved and merged? Like yeah, that's scary because now you have a threat actor that could potentially specifically tailor contributions to trick the AI into thinking they're great, but then it could turn around and be a backdoor that's being added to the code.Obviously, that's very far in the future and I'm sure a lot of things will happen before that, but it starts to make you wonder, like, if things are heading that way. Or will people realize that you need to look at security at every step of the way instead of just thinking that these newer AI systems can just handle everything?Corey: Let's pivot a little bit and talk about your day job. You're a lead security engineer at what I believe to be a security-focused consultancy. Or—Adnan: Yeah.Corey: If you're not a SaaS product. Everything seems to become a SaaS product in the fullness of time. What's your day job look like?Adnan: Yeah, so I'm a security engineer on Praetorian's red team. And my day-to-day, I'll kind of switch between application security and red-teaming. And that kind of gives me the opportunity to, kind of, test out newer things out in the field, but then also go and do more traditional application security assessments and code reviews, and reverse engineering to kind of break up the pace of work. Because red-teaming can be very fast and fast-paced and exciting, but sometimes, you know, that can lead to some pretty late nights. But that's just the nature of being on a red team [laugh].Corey: It feels like as soon as I get into the security space and start talking to cloud companies, they get a lot more defensive than when I'm making fun of, you know, bad service naming or APIs that don't make a whole lot of sense. It feels like companies have a certain sensitivity around the security space that applies to almost nothing else. Do you find, as a result, that a lot of the times when you're having conversations with companies and they figure out that, oh, you're a red team for a security researcher, oh, suddenly, we're not going to talk to you the way we otherwise might. We thought you were a customer, but nope, you can just go away now.Adnan: [laugh]. I personally haven't had that experience with cloud companies. I don't know if I've really tried to buy a lot. You know, I'm… if I ever buy some infrastructure from cloud companies as an individual, I just kind of sign up and put in my credit card. And, you know, they just, like, oh—you know, they just take my money. So, I don't really think I haven't really, personally run into anything like that yet [laugh].Corey: Yeah, I'm curious to know how that winds up playing out in some of these, I guess, more strategic, larger company environments. I don't get to see that because I'm basically a tiny company that dabbles in security whenever I stumble across something, but it's not my primary function. I just worry on some level one of these days, I'm going to wind up accidentally dropping a zero-day on Twitter or something like that, and suddenly, everyone's going to come after me with the knives. I feel like [laugh] at some point, it's just going to be a matter of time.Adnan: Yeah. I think when it comes to disclosing things and talking about techniques, the key thing here is that a lot of the things that I'm talking about, a lot of the things that I'll be talking about in some blog posts that have coming out, this is stuff that these companies are seeing themselves. Like, they recognize that these are security issues that people are introducing into code. They encourage people to not make these mistakes, but when it's buried in four links deep of documentation and developers are tight on time and aren't digging through their security documentation, they're just looking at what works, getting it to work and moving on, that's where the issue is. So, you know, from a perspective of raising awareness, I don't feel bad if I'm talking about something that the company itself agrees is a problem. It's just a lot of the times, their own engineers don't follow their own recommendations.Corey: Yeah, I have opinions on these things and unfortunately, it feels like I tend to learn them in some of the more unfortunate ways of, oh, yeah, I really shouldn't care about this thing, but I only learned what the norm is after I've already done something. This is, I think, the problem inherent to being small and independent the way that I tend to be. We don't have enough people here for there to be a dedicated red team and research environment, for example. Like, I tend to bleed over a little bit into a whole bunch of different things. We'll find out. So far, I've managed to avoid getting it too terribly wrong, but I'm sure it's just a matter of time.So, one area that I think seems to be a way that people try to avoid cloud issues is oh, I read about that in the last in-flight magazine that I had in front of me, and the cloud is super insecure, so we're going to get around all that by running our own infrastructure ourselves, from either a CI/CD perspective or something else. Does that work when it comes to this sort of problem?Adnan: Yeah, glad you asked about that. So, we've also seen open-s—companies that have large open-source presence on GitHub just opt to have self-hosted Github Actions runners, and that opens up a whole different Pandora's box of attacks that an attacker could take advantage of, and it's only there because they're using that kind of runner. So, the default GitHub Actions runner, it's just an agent that runs on a machine, it checks in with GitHub Actions, it pulls down builds, runs them, and then it waits for another build. So, these are—the default state is a non-ephemeral runner with the ability to fork off tasks that can run in the background. So, when you have a public repository that has a self-hosted runner attached to it, it could be at the organization level or it could be at the repository level.What an attacker can just do is create a pull request, modify the pull request to run on a self-hosted runner, write whatever they want in the pull request workflow, create a pull request, and now as long as they were a previous contributor, meaning you fixed a typo, you… that could be a such a, you know, a single character typo change could even cause that, or made a small contribution, now they create the pull request. The arbitrary job that they wrote is now picked up by that self-hosted runner. They can fork off it, process it to run in the background, and then that just continues to run, the job finishes, their pull request, they'll just—they close it. Business as usual, but now they've got an implant on the self-hosted runner. And if the runners are non-ephemeral, it's very hard to completely lock that down.And that's something that I've seen, there's quite a bit of that on GitHub where—and you can identify it just by looking at the run logs. And that's kind of comes from people saying, “Oh, let's just self-host our runners,” but they also don't configure that properly. And that opens them up to not only tampering with their repositories, stealing secrets, but now depending on where your runner is, now you potentially could be giving an attacker a foothold in your cloud environment.Corey: Yeah, that seems like it's generally a bad thing. I found that cloud tends to be more secure than running it yourself in almost every case, with the exception that once someone finds a way to break into it, there's suddenly a lot more eggs in a very large, albeit more secure, basket. So, it feels like it's a consistent trade-off. But as time goes on, it feels like it is less and less defensible, I think, to wind up picking out an on-prem strategy from a pure security point of view. I mean, there are reasons to do it. I'm just not sure.Adnan: Yeah. And I think that distinction to be made there, in particular with CI/CD runners is there's cloud, meaning you let your—there's, like, full cloud meaning you let your CI/CD provider host your infrastructure as well; there's kind of that hybrid approach you mentioned, where you're using a CI/CD provider, but then you're bringing your own cloud infrastructure that you think you could secure better; or you have your runners sitting in vCenter in your own data center. And all of those could end up being—both having a runner in your cloud and in your data center could be equally vulnerable if you're not segmenting builds properly. And that's the core issue that happens when you have a self-hosted runner is if they're not ephemeral, it's very hard to cut off all attack paths. There's always something an attacker can do to tamper with another build that'll have some kind of security impact. You need to just completely isolate your builds and that's essentially what you see in a lot of these newer guidances like the [unintelligible 00:30:04] framework, that's kind of the core recommendation of it is, like, one build, one clean runner.Corey: Yeah, that seems to be the common wisdom. I've been doing a lot of work with my own self-hosted runners that run inside of Lambda. Definitionally those are, of course, ephemeral. And there's a state machine that winds up handling that and screams bloody murder if there's a problem with it. So far, crossing fingers hoping it works out well.And I have a bounded to a very limited series of role permissions, and of course, its own account of constraint blast radius. But there's still—there are no guarantees in this. The reason I build it the way I do is that, all right, worst case someone can get access to this. The only thing they're going to have the ability to do is, frankly, run up my AWS bill, which is an area I have some small amount of experience with.Adnan: [laugh]. Yeah, yeah, that's always kind of the core thing where if you get into someone's cloud, like, well, just sit there and use their compute resources [laugh].Corey: Exactly. I kind of miss when that was the worst failure mode you had for these things.Adnan: [laugh].Corey: I really want to thank you for taking the time to speak with me today. If people want to learn more, where's the best place for them to find you?Adnan: I do have a Twitter account. Well, I guess you can call it Twitter anymore, but, uh—Corey: Watch me. Sure I can.Adnan: [laugh]. Yeah, so I'm on Twitter, and it's @adnanthekhan. So, it's like my first name with ‘the' and then K-H-A-N because, you know, my full name probably got taken up, like, years before I ever made a Twitter account. So, occasionally I tweet about GitHub Actions there.And on Praetorian's website, I've got a couple of blog posts. I have one—the one that really goes in-depth talking about the two Microsoft repository pull request attacks, and a couple other ones that are disclosed, will hopefully drop on the twenty—what is that, Tuesday? That's going to be the… that's the 26th. So, it should be airing on the Praetorian blog then. So, if you—Corey: Excellent. It should be out by the time this is published, so we will, of course, put a link to that in the [show notes 00:32:01]. Thank you so much for taking the time to speak with me today. I appreciate it.Adnan: Likewise. Thank you so much, Corey.Corey: Adnan Khan, lead security engineer at Praetorian. I'm Cloud Economist Corey Quinn and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice, along with an insulting comment that's probably going to be because your podcast platform of choice is somehow GitHub Actions.Adnan: [laugh].Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.

ElectricSQL is a project that offers a local-first sync layer for web and mobile apps, Ned Batchelder writes about the myth of the myth of “learning styles”, Carl Johnson thinks XML is better than YAML, Berkan Sasmaz defines and describes “idempotency” & HyperDX is an open source alternative Datadog or New Relic.

#229: When Kubernetes was first released in September 2014, the only way we could get applications installed to the cluster was by using kubectl apply and big, ugly YAML files. Since that time, many tools have been introduced to help manage application installation into Kubernetes clusters. However, no matter what tool you are using in 2023, under the hood, we're still just submitting those big, ugly YAML files to the Kubernetes. Once you realize this, things become much clearer. In this episode, we take a stroll down memory lane of how it all started and what you should consider doing today when creating (or maybe not creating) the YAML files for your application.   Today's sponsor: Save 25% on your first Barbaro Mojo order using the code "DevOps25” https://barbaromojo.com/discount/DevOps25   YouTube channel: https://youtube.com/devopsparadox/   Books and Courses: Catalog, Patterns, And Blueprints https://www.devopstoolkitseries.com/posts/catalog/   Review the podcast on Apple Podcasts: https://www.devopsparadox.com/review-podcast/   Slack: https://www.devopsparadox.com/slack/    Connect with us at: https://www.devopsparadox.com/contact/

Bret Fisher, DevOps Dude & Cloud-Native Trainer, joins Corey on Screaming in the Cloud to discuss what it's like being a practitioner and a content creator in the world of cloud. Bret shares why he feels it's so critical to get his hands dirty so his content remains relevant, and also how he has to choose where to focus his efforts to grow his community. Corey and Bret discuss the importance of finding the joy in your work, and also the advantages and downfalls of the latest AI advancements. About BretFor 25 years Bret has built and operated distributed systems, and helped over 350,000 people learn dev and ops topics. He's a freelance DevOps and Cloud Native consultant, trainer, speaker, and open source volunteer working from Virginia Beach, USA. Bret's also a Docker Captain and the author of the popular Docker Mastery and Kubernetes Mastery series on Udemy. He hosts a weekly DevOps YouTube Live Show, a container podcast, and runs the popular devops.fan Discord chat server.Links Referenced: Twitter: https://twitter.com/BretFisher YouTube Channel: https://www.youtube.com/@BretFisher Website: https://www.bretfisher.com TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: In the cloud, ideas turn into innovation at virtually limitless speed and scale. To secure innovation in the cloud, you need Runtime Insights to prioritize critical risks and stay ahead of unknown threats. What's Runtime Insights, you ask? Visit sysdig.com/screaming to learn more. That's S-Y-S-D-I-G.com/screaming.My thanks as well to Sysdig for sponsoring this ridiculous podcast.Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn, a little bit off the beaten path today, in that I'm talking to someone who, I suppose like me, if that's not considered to be an insult, has found themselves eminently unemployable in a quote-unquote, “Real job.” My guest today is Bret Fisher, DevOps dude and cloud-native trainer. Bret, great to talk to you. What do you do?Bret: [laugh]. I'm glad to be here, Corey. I help people for a living like a lot of us end up doing in tech. But nowadays, it's courses, it's live trainings, webinars, all that stuff. And then of course, the fun side of it is the YouTube podcast, hanging out with friends, chatting on the internet. And then a little bit of running a Discord community, which is one of the best places to have a little text chat community, if you don't know Discord.Corey: I've been trying to get the Discord and it isn't quite resonating with me, just because by default, it alerts on everything that happens in any server you're in. It, at least historically, was very challenging to get that tuned in, so I just stopped having anything alert me on my phone, which means now I miss things constantly. And that's been fun and challenging. I still have the slack.lastweekinaws.com community with a couple of thousand people in it.Bret: Nice. Yeah, I mean, some people love Slack. I still have a Slack community for my courses. Discord, I feel like is way more community friendly. By the way, a good server admin knows how to change those settings, which there are a thousand settings in Discord, so server admins, I don't blame you for not seeing that setting.But there is one where you can say new members, don't bug them on every message; only bug them on a mentions or, you know, channel mentions and stuff like that. And then of course, you turn off all those channel mentions and abilities for people to abuse it. But yeah, I had the same problem at first. I did not know what I was doing and it took me years to kind of figure out. The community, we now have 15,000 people. We call it Cloud Native DevOps, but it's basically people from all walks of DevOps, you know, recovering IT pros.And the wonderful thing about it is you always start out—like, you'd do the same thing, I'm sure—where you start a podcast or YouTube channel or a chat community or Telegram, or a subreddit, or whatever your thing is, and you try to build a community and you don't know if it's going to work and you invite your friends and then they show up for a day and then go away. And I've been very lucky and surprised that the Discord server has, to this point, taken on sort of a, its own nature. We've got, I don't know, close to a dozen moderators now and people are just volunteering their time to help others. It's wonderful. I actually—I consider it, like, one of the safe places, unlike maybe Stack Overflow where you might get hated for the wrong question. And we try to guide you to a better question so [laugh] that we can answer you or help you. So, every day I go in there, and there's a dozen conversations I missed that I wasn't able to keep up with. So, it's kind of fun if you're into that thing.Corey: I remember the olden days when I was one of the volunteer staff members on the freenode IRC network before its untimely and awful demise, and I really have come to appreciate the idea of, past a certain point, you can either own the forum that you're working within or you can participate in it, but being a moderator, on some level, sets apart how people treat you in some strange ways. And none of these things are easy once you get into the nuances of codes of conduct, of people participating in good faith, but also are not doing so constructively. And people are hard. And one of these years I should really focus on addressing aspects of that with what I'm focusing on.Bret: [laugh]. Yeah, the machines—I mean, as frustrating as the machines are, they at least are a little more reliable. I don't have anonymous machines showing up yet in Discord, although we do get almost daily spammers and stuff like that. So, you know, I guess I'm blessed to have attracted some of the spam and stuff like that. But a friend of mine who runs a solid community for podcasters—you know, for podcasts hosters—he warned me, he's like, you know, if you really want to make it the community that you have the vision for, it requires daily work.Like, it's a part-time job, and you have to put the time in, or it will just not be that and be okay with that. Like, be okay with it being a small, you know, small group of people that stick around and it doesn't really grow. And that's what's happened on the Slack side of things is I didn't care and feed it, so it has gotten pretty quiet over there as we've grown the Discord server. Because I kind of had to choose, you know? Because we—like you, I started with Slack long, long ago. It was the only thing out there. Discord was just for gamers.And in the last four or five years, I think Discord—I think during the pandemic, they officially said, “We are now more than gamers,” which I was kind of waiting for to really want to invest my company's—I mean, my company of three—you know, my company [laugh] time into a platform that I thought was maybe just for gamers; couldn't quite figure it out. And once they kind of officially said, “Yeah, we're for all communities,” we're more in, you know, and they have that—the thing I really appreciate like we had an IRC, but was mostly human-driven is that Discord, unlike Slack, has actual community controls that make it a safer place, a more inclusive place. And you can actually contact Discord when you have a spammer or someone doing bad things, or you have a server raid where there's a whole bunch of accounts and bot accounts trying to take you down, you can actually reach out to Discord, where Slack doesn't have any of that, they don't have a way for you to reach out. You can't block people or ban them or any of that stuff with Slack. And we—the luckily—the lucky thing of Dis—I kind of look at Discord as, like, the best new equivalent of IRC, even though for a lot of people IRC is still the thing, right? We have new clients now, you can actually have off—you could have sort of synced IRC, right, where you can have a web client that remembers you so you didn't lose the chat after you left, which was always the problem back in the day.Corey: Oh, yeah. I just parked it on, originally, a hardware box, now EC2. And this ran Irssi as my client—because I'm old school—inside of tmux and called it a life. But yeah, I still use that from time to time, but the conversation has moved on. One challenge I've had is that a lot of the people I talk to about billing nuances skew sometimes, obviously in the engineering direction, but also in the business user perspective and it always felt, on some level like it was easier to get business users onto Slack from a community perspective—Bret: Mmm. Absolutely. Yeah.Corey: —than it was for Discord. I mean, this thing started as well. This was years ago, before Discord had a lot of those controls. Might be time to take another bite at that apple.Bret: Yeah. Yeah, I definitely—and that, I think that's why I still keep the Slack open is there are some people, they will only go there, right? Like, they just don't want another thing. That totally makes sense. In fact, that's kind of what's happening to the internet now, right?We see the demise of Twitter or X, we see all these other new clients showing up, and what I've just seen in the dev community is we had this wonderful dev community on Twitter. For a moment. For a few years. It wasn't perfect by far, there was a lot people that still didn't want to use Twitter, but I felt like there was—if you wanted to be in the cloud-native community, that was very strong and you didn't always have to jump into Slack. And then you know, this billionaire came along and kind of ruined it, so people have fractured over to Mastodon and we've got some people have run Threads and some people on Bluesky, and now—and then some people like me that have stuck with Twitter.And I feel like I've lost a chunk of my friends because I don't want to spend my life on six different platforms. So, I am—I have found myself actually kind of sort of regressing to our Discord because it's the people I know, we're all talking about the same things, we all have a common interest, and rather than spending my time trying to find those people on the socials as much as I used to. So, I don't know, we'll see.Corey: Something that I have found, I'm curious to get your take on this, you've been doing this for roughly twice as long as I have, but what I've been having to teach myself is that I am not necessarily representative of the totality of the audience. And, aside from the obvious demographic areas, I learned best by reading or by building something myself—I don't generally listen to podcasts, which is a weird confession in this forum for me to wind up admitting to—and I don't basically watch videos at all. And it took me a while to realize that not everyone is like me; those are wildly popular forms of absorbing information. What I have noticed that the audience engages differently in different areas, whereas for this podcast, for the first six months, I didn't think that I'd remember to turn the microphone on. And that was okay; it was an experiment, and I enjoyed doing it. But then I went to a conference and wound up getting a whole bunch of feedback.Whereas for the newsletter, I had immediate responses to basically every issue when I sent it out. And I think the reason is, is because people are not sitting in front of a computer when they're listening to something and they're not going to be able to say, “Well, let me give you a piece of my mind,” in quite the same way. And by the time they remember later, it feels weird, like, calling into a radio show. But when you actually meet someone, “Yeah, I love your stuff.” And they'll talk about the episodes I've had out. But you can be forgiven for in some cases in the social media side of it for thinking that I'd forgotten to publish this thing.Bret: Yeah. I think that's actually a pretty common trait. There was a time where I was sort of into the science of learning and whatnot, and one of the things that came out of that was that the way we communicate or the way we learn and then the way—the input and the outputs are different per human. It's actually almost, like, comparable maybe to love languages, if you've read that book, where the way we give love and the way we receive love from others is—we prefer it in different ways and it's often not the same thing. And I think the same is true of learning and teaching, where my teaching style has always been visual.I think have almost always been in all my videos. My first course seven years ago, I was in it phy—like, I had my headshot in there and I just thought that that was a part of the best way you could make that content. And doesn't mean that I'm instantly better; it just means I wanted to communicate with my hands, maybe I got a little bit of Italian or French in me or something [laugh] where I'm moving my hands around a lot. So, I think that the medium is very specific to the person. And I meet people all the time that I find out, they didn't learn from me—they didn't learn about me, rather, from my course; they learned about me from a conference talk because they prefer to watch those or someone else learned about me from the podcast I run because they stumbled onto that.And it always surprises me because I always figure that since my biggest audience in my Udemy courses—over 300,000 people there—that that's how most of the people find me. And it turns out nowadays that when I meet people, a lot of times it's not. It's some other, you know, other venue. And now we have people showing up in the Discord server from the Discord Discovery. It's kind of a little feature in Discord that allows you to find servers that are on the topics you're interested in and were listed in there and people will find me that way and jump in not knowing that I have created courses, I have a weekly YouTube Live show, I have all the other things.And yeah, it's just it's kind of great, but also as a content creator, it's kind of exhausting because you—if you're interested in all these things, you can't possibly focus on all of them at the [laugh] same time. So, what is it the great Will Smith says? “Do two things and two things suffer.” [laugh]. And that's exactly what my life is like. It's like, I can't focus on one thing, so they all aren't as amazing as they could be, maybe, if I had only dedicated to one thing.Corey: No, I'm with you on that it's a saying yes to something means inherently saying no to something else. But for those of us whose interests are wide and varied, I find that there are always more things to do than I will ever be able to address. You have to pick and choose, on some level. I dabble with a lot of the stuff that I work on. I have given thought in the past towards putting out video courses or whatnot, but you've done that for ages and it just seems like it is so much front-loaded work, in many cases with things I'm not terrific at.And then, at least in my side of the world, oh, then AWS does another console refresh, as they tend to sporadically, and great, now I have to go back and redo all of the video shoots showing how to do it because now it's changed just enough to confuse people. And it feels like a treadmill you climb on top of and never get off.Bret: It can definitely feel like that. And I think it's also harder to edit existing courses like I'm doing now than it is to just make up something brand new and fresh. And there's something about… we love to teach, I think what we're learning in the moment. I think a lot of us, you get something exciting and you want to talk about it. And so, I think that's how a lot of people's conference talk ideas come up if you think about it.Like you're not usually talking about the thing that you were interested in a decade ago. You're talking about the thing you just learned, and you thought it was great, and you want everyone to know about it, which means you're going to make a YouTube video or a blog post or something about it, you'll share somewhere on social media about it. I think it's harder to make this—any of these content creation things, especially courses, a career if you come back to that course like I'm doing seven years after publication and you're continuing every year to update those videos. And you're thinking I—not that my interests have moved on, but my passion is in the new things. And I'm not making videos right now on new things.I'm fixing—like you're saying, like, I'm fixing the Docker Hub video because it has completely changed in seven years and it doesn't even look the same and all that. So, there's definitely—that's the work side of this business where you really have to put the time in and it may not always be fun. So, one of the things I'm learning from my business coach is like how to find ways to make some of this stuff fun again, and how to inject some joy into it without it feeling like it's just the churn of video after video after video, which, you know, you can fall into that trap with any of that stuff. So, yeah. That's what I'm doing this year is learning a little bit more about myself and what I like doing versus what I have to do and try to make some of it a little funner.Corey: This question might come across as passive-aggressive or back-handedly insulting and I swear to you it is not intended to, but how do you avoid what has been a persistent fear of mine and that is becoming a talking head? Whereas you've been doing this as a trainer for long enough that you haven't had a quote-unquote, “Real job,” in roughly, what, 15 years at this point?Bret: Yeah. Yeah.Corey: And so, you've never run Kubernetes in anger, which is, of course, was what we call production environment. That's right, I call it ‘Anger.' My staging environment is called ‘Theory' because it works in theory, but not in production. And there you have it. So, without being hands-on and running these things at scale, it feels like on some level, if I were to, for example, give up the consulting side of my business and just talk about the pure math that I see and what AWS is putting out there, I feel like I'd pretty quickly lose sight of what actual customer pain looks like.Bret: Yeah. That's a real fear, for sure. And that's why I'm kind of—I think I kind of do what you do and maybe wasn't… didn't try to mislead you, but I do consult on a fairly consistent basis and I took a break this year. I've only—you know, then what I'll do is I'll do some advisory work, I usually won't put hands on a cluster, I'm usually advising people on how to put the hands on that cluster kind of thing, or how to build accepting their PRs, doing stuff like that. That's what I've done in the last maybe three or four years.Because you're right. There's two things that are, right? Like, it's hard to stay relevant if you don't actually get your hands dirty, your content ends up I think this naturally becoming very… I don't know, one dimensional, maybe, or two dimensional, where it doesn't, you don't really talk about the best practices because you don't actually have the scars to prove it. And so, I'm always nervous about going long lengths, like, three or four years of time, with zero production work. So, I think I try to fill that with a little bit of advisory, maybe trying to find friends and actually trying to talk with them about their experiences, just so I can make sure I'm understanding what they're dealing with.I also think that that kind of work is what creates my stories. So like, my latest course, it's on GitHub Actions and Argo CD for using automation and GitOps for deployments, basically trying to simplify the deployment lifecycle so that you can just get back to worrying about your app and not about how it's deployed and how it's tested and all that. And that all came out of consulting I did for a couple of firms in 2019 and 2020, and I think right into 2021, that's kind of where I started winding them down. And that created the stories that caused me, you know, sort of the scars of going into production. We were migrating a COTS app into a SaaS app, so we were learning lots of things about their design and having to change infrastructure. And I had so many learnings from that.And one of them was I really liked GitHub Actions. And it worked well for them. And it was very flexible. And it wasn't as friendly and as GUI beautiful as some of the other CI solutions out there, but it was flexible enough and direct—close enough to the developer that it felt powerful in the developers' hands, whereas previous systems that we've all had, like Jenkins always felt like this black box that maybe one or two people knew.And those stories came out of the real advisory or consultancy that I did for those few years. And then I was like, “Okay, I've got stuff. I've learned it. I've done it in the field. I've got the scars. Let me go teach people about it.” And I'm probably going to have to do that again in a few years when I feel like I'm losing touch like you're saying there. That's a—yeah, so I agree. Same problem [laugh].Corey: Crap, I was hoping you had some magic silver bullet—Bret: No. [laugh].Corey: —other than, “No, it still gnaws at you forever and there's no real way to get away for”—great. But, uhh, it keeps things… interesting.Bret: I would love to say that I have that skill, that ability to, like, just talk with you about your customers and, like, transfer all that knowledge so that I can then talk about it, but I don't know. I don't know. It's tough.Corey: Yeah. The dangerous part there is suddenly you stop having lived experience and start just trusting whoever sounds the most confident, which of course, brings us to generative AI.Bret: Ohhh.Corey: Which apparently needs to be brought into every conversation as per, you know, analysts and Amazon leadership, apparently. What's your take on it?Bret: Yeah. Yeah. Well, I was earl—I mean, well maybe not early, early. Like, these people that are talking about being early were seven years ago, so definitely wasn't that early.Corey: Yeah. Back when the Hello World was a PhD from Stanford.Bret: Yeah [laugh], yeah. So, I was maybe—my first step in was on the tech side of things with Copilot when it was in beta a little over two years ago. We're talking about GitHub Copilot. That was I think my first one. I was not an OpenAI user for any of their solutions, and was not into the visual—you know, the image AI stuff as we all are now dabbling with.But when it comes to code and YAML and TOML and, you know, the stuff that I deal with every day, I didn't start into it until about two years ago. I think I actually live-streamed my first experiences with it with a friend of mine. And I was just using it for DevOps tasks at the time. It was an early beta, so I was like, kind of invited. And it was filling out YAML for me. It was creating Kubernetes YAML for me.And like we're all learning, you know, it hallucinates, as we say, which is lying. It made stuff up for 50% of the time. And it was—it is way better now. So, I think I actually wrote in my newsletter a couple weeks ago a recent story—or a recent experience because I wanted to take a project in a language that I had not previously written from scratch in but maybe I was just slightly familiar with. So, I picked Go because everything in cloud-native is written in Go and so I've been reading it for years and years and years and maybe making small PRs to various things, but never taken on myself to write it from scratch and just create something, start to finish, for myself.And so, I wanted a real project, not something that was contrived, and it came up that I wanted to create—in my specific scenario, I wanted to take a CSV of all of my students and then take a template certificate, you know, like these certificates of completion or certifications, you know, that you get, and it's a nice little—looks like the digital equivalent of a paper certificate that you would get from maybe a university. And I wanted to create that. So, I wanted to do it in bulk. I wanted to give it a stock image and then give it a list of names and then it would figure out the right place to put all those names and then generate a whole bunch of images that I could send out. And then I can maybe turn this into a web service someday.But I wanted to do this, and I knew, if I just wrote it myself, I'd be horrible at it, I would suck at Go, I'd probably have to watch some videos to remember some of the syntax. I don't know the standard libraries, so I'd have to figure out which libraries I needed and all that stuff. All the dependencies.Corey: You make the same typical newcomer mistakes of not understanding the local idioms and whatnot. Oh, yeah.Bret: Yeah. And so, I'd have to spend some time on Stack Overflow Googling around. I kind of guessed it was going to take me 20 to 40 hours to make. Like, and it was—we're talking really just hundreds of lines of code at the end of the day, but because Go standard library actually is really great, so it was going to be far less code than if I had to do it in NodeJS or something. Anyway, long story short there, it ended up taking three to three-and-a-half hours end to end, including everything I needed, you know, importing a CSV, sucking in a PNG, outputting PNG with all the names on them in the right places in the right font, the right colors, all that stuff.And I did it all through GitHub Copilot Chat, which is their newest Labs beta thing. And it brings the ChatGPT-4 experience into VS Code. I think it's right now only for VS Code, but other editors coming soon. And it was kind of wonderful. It remembered my project as a whole. It wasn't just in the file I was in. There was no copying-pasting back and forth between the web interface of ChatGPT like a lot of people tend to do today where they go into ChatGPT, they ask a question, then they copy out code and they paste it in their editor.Well, there was none of that because since that's built into the editor, it kind of flows naturally into your existing project. You can kind of just click a button and it'll automatically paste in where your cursor is. It does all this convenient stuff. And then it would relook at the code. I would ask it, you know, “What are ten ways to improve this code now that it works?” And you know, “How can I reduce the number of lines in this code?” Or, “How can I make it easier to read?”And I was doing all this stuff while I was creating the project. I haven't had anyone, like, look at it to tell me if it looks good [laugh], which I hear you had that experience. But it works, it solved my problem, and I did it in a half a day with no prep time. And it's all in ChatGPT's history. So, when I open up VS Code now, I open that project up and get it, it recognizes that oh, this is the project that you've asked all these previous questions on, and it reloads all those questions, allowing me to basically start the conversation off again with my AI friend at the same place I left off.And I think that experience basically proved to me that what everybody else is telling us, right, that yes, this is definitely the future. I don't see myself ever writing code again without an AI partner. I don't know why I ever would write it without the AI partner at least to help me, quicken my learning, and solve some of the prompts. I mean, it was spitting out code that wasn't perfect. It would actually—[unintelligible 00:23:53] sometimes fail.And then I would tell it, “Here's the error you just caused. What do I do with that?” And it would help me walk through the solution, it would fix it, it would recommend changes. So, it's definitely not something that will avoid you knowing how to program or make someone who's not a programmer suddenly write a perfect program, but man, it really—I mean, it took basically what I would consider to be a novice in that language—not a novice at programming, but a novice at that language—and spit out a productive program in less than a day. So, that's huge, I think.[midroll 00:24:27]Corey: What I think is a necessary prerequisite is a domain expertise in order to figure out what is accurate versus what is completely wrong, but sounds competent. And I've been racing a bunch of the different large-language models against each other in a variety of things like this. One of the challenges I'll give them is to query the AWS pricing API—which motto is, “Not every war crime happens in faraway places”—and then spit out things like the Managed Nat Gateway hourly cost table, sorted from most to least expensive by region. And some things are great at it and other things really struggle with it. And the first time I, just on a lark, went down that path, it saved me an easy three hours from writing that thing by hand. It was effectively an API interface, whereas now the most common programming language I think we're going to see on the rise is English.Bret: Yeah, good point. I've heard some theories, right? Like maybe the output language doesn't matter. You just tell it, “Oh, don't do that in Java, do it in PHP.” Whatever, or, “Convert this Java to PHP,” something like that.I haven't experimented with a lot of that stuff yet, but I think that having spent this time watching a lot of other videos, right, you know, watching [Fireship 00:25:37], and a lot of other people talking about LLMs on the internet, seeing the happy-face stuff happen. And it's just, I don't know where we're going to be in five or ten years. I am definitely not a good prediction, like a futurist. And I'm trying to imagine what the daily experience is going to be, but my assumption is, every tool we're using is going to have some sort of chat AI assistant in it. I mean, this is kind of the future that, like, none of the movies predicted.[laugh]. We were talking about this the other day with a friend of mine. We were talking about it over dinner, some developer friends. And we were just talking about, like, this would be too boring for a movie, like, we all want the—you know, we think of the movies where there's the three laws of robotics and all these things. And these are in no way sentient.I'm not intimidated or scared by them. I think the EU is definitely going to do the right thing here and we're going to have to follow suit eventually, where we rank where you can use AI and, like, there's these levels, and maybe just helping you with a program is a low-level, there's very few restrictions, in other words, by the government, but if you're talking about in cars or in medical or you know, in anything like that, that's the highest level and the highest restrictions and all that. I could definitely see that's the safety. Obviously, we'll probably do it too slow and too late and there'll be some bad uses in the meantime, but I think we're there. I mean, like, if you're not using it today—if you're listening to this, and you're not using AI yet in your day-to-day as someone related to the IT career, it's going to be everywhere and I don't think it's going to be, like, one tool. The tools on the CLI to me are kind of weird right now. Like, they certainly can help you write command lines, but it just doesn't flow right for me. I don't know if you've tried that.Corey: Yeah. I ha—I've dabbled lightly, but again, I've been a Unix admin for the better part of 20 years and I'm used to a world in which you type exactly what you mean or you suffer the consequences. So, having a robot trying to outguess me of what it thinks I'm trying to do, if it works correctly, it looks like a really smart tab complete. If it guesses wrong, it's incredibly frustrating. The risk/reward is not there in the same way.Bret: Right.Corey: So, for me at least, it's more frustration than anything. I've seen significant use cases across the business world where this would have been invaluable back when I was younger, where it's, “Great, here's a one-line email I'm about to send to someone, and people are going to call me brusque or difficult for it. Great. Turn this into a business email.” And then on the other side, like, “This is a five-paragraph email. What does he actually want?” It'll turn it back into one line. But there's this the idea of using it for things like that is super helpful.Bret: Yeah. Robots talking to robots? Is that what you're saying? Yeah.Corey: Well, partially, yes. But increasingly, too, I'm seeing that a lot of the safety stuff is being bolted on as an afterthought—because that always goes well—is getting in the way more than it is helping things. Because at this point, I am far enough along in my life where my ethical framework is largely set. I am not going to have radical changes in my worldview, no matter how much a robot [unintelligible 00:28:29] me.So, snark and sarcasm are my first languages and that is something that increasingly they're leery about, like, oh, sarcasm can hurt people's feelings. “Well, no kidding, professor, you don't say.” As John Scalzi says, “The failure mode of clever is ‘asshole.'” But I figured out how to walk that line, so don't you worry your pretty little robot head about that. Leave that to me. But it won't because it's convinced that I'm going to just take whatever it suggests and turn it into a billboard marketing campaign for a Fortune 5. There are several more approval steps in there.Bret: There. Yeah, yeah. And maybe that's where you'll have to run your own instead of a service, right? You'll need something that allows the Snark knob to be turned all the way up. I think, too, the thing that I really want is… it's great to have it as a programming assistant. It's great and notion to help me, you know, think out, you know, sort of whiteboard some things, right, or sketch stuff out in terms of, “Give me the top ten things to do with this,” and it's great for ideas and stuff like that.But what I really, really want is for it to remove a lot of the drudgery of day-to-day toil that we still haven't, in tech, figured out a way—for example, I'm going to need a new repo. I know what I need to go in it, I know which organization it needs to go in, I know what types of files need to go in there, and I know the general purpose of the repo. Even the skilled person is going to take at least 20 minutes or more to set all that up. And I would really just rather take an AI on my local computer and say, “I would like three new repos: a front-end back-end, and a Kubernetes YAML repo. And I would like this one to be Rust, and I would like this one to be NodeJS or whatever, and I would like this other repo to have all the pieces in Kubernetes. And I would like Docker files in each repo plus GitHub Actions for linting.”Like, I could just spill out, you know, all these things: the editor.config file, the Git ignore, the Docker ignore, think about, like, the dozen files that every repo has to have now. And I just want that generated by an AI that knows my own repos, knows my preferences, and it's more—because we all have, a lot of us that are really, really organized and I'm not one of those, we have maybe a template repo or we have templates that are created by a consolidated group of DevOps guild members or something in our organization that creates standards and reusable workflows and template files and template repos. And I think a lot of that's going to go—that boilerplate will sort of, if we get a smart enough LLM that's very user and organization-specific, I would love to be able to just tell Siri or whatever on my computer, “This is the thing I want to be created and it's boilerplate stuff.” And it then generates all that.And then I jump into my code creator or my notion drafting of words. And that's—like, I hop off from there. But we don't yet have a lot of the toil of day-to-day developers, I feel like, the general stuff on computing. We don't really have—maybe I don't think that's a general AI. I don't think we're… I don't think that needs to be like a general intelligence. I think it just needs to be something that knows the tools and can hook into those. Maybe it asks for my fingerprint on occasion, just for security sake [laugh] so it doesn't deploy all the things to AWS. But.Corey: Yeah. Like, I've been trying to be subversive with a lot of these things. Like, it's always fun to ask the challenging questions, like, “My boss has been complaining to me about my performance and I'm salty about it. Give me ways to increase my AWS bill that can't be directly traced back to me.” And it's like, oh, that's not how to resolve workplace differences.Like, okay. Good on, you found that at least, but cool, give me the dirt. I get asked in isolation of, “Yeah, how can I increase my AWS bill?” And its answer is, “There is no good reason to ever do that.” Mmm, there are exceptions on this and that's not really what I asked. It's, on some level, that tries to out-human you and gets it hilariously wrong.Bret: Yeah, there's definitely, I think—it wasn't me that said this, but in the state we're in right now, there is this dangerous point of using any of these LLMs where, if you're asking it questions and you don't know anything about that thing you're asking about, you don't know what's false, you don't know what's right, and you're going to get in trouble pretty quickly. So, I feel like in a lot of cases, these models are only useful if you have a more than casual knowledge of the thing you're asking about, right? Because, like, you can—like, you've probably tried to experiment. If you're asking about AWS stuff, I'm just going to imagine that it's going to make some of those service names up and it's going to create things that don't exist or that you can't do, and you're going to have to figure out what works and what doesn't.And what do you do, right? Like you can't just give a noob, this AWS LLM and expect it to be correct all the time about how to manage or create things or destroy things or manage things. So, maybe in five years. Maybe that will be the thing. You literally hire someone who has a computing degree out of a university somewhere and then they can suddenly manage AWS because the robot is correct 99.99% of the time. We're just—I keep getting told that that's years and years away and we don't know how to stop the hallucinations, so we're all stuck with it.Corey: That is the failure mode that is disappointing. We're never going to stuff that genie back in the bottle. Like, that is—technology does not work that way. So, now that it's here, we need to find a way to live with it. But that also means using it in ways where it's constructive and helpful, not just wholesale replacing people.What does worry me about a lot of the use it to build an app, when I wound up showing this to some of my engineering friends, their immediate response universally, was, “Well, yeah, that's great for, like, the easy, trivial stuff like querying a bad API, but for any of this other stuff, you still need senior engineers.” So, their defensiveness was the reaction, and I get that. But also, where do you think senior engineers come from? It's solving a bunch of stuff like this. You didn't all spring, fully formed, from the forehead of some God. Like, you started off as junior and working on small trivial problems, like this one, to build a skill set and realize what works well, what doesn't, then life goes on.Bret: Yeah. In a way—I mean, you and I have been around long enough that in a way, the LLMs don't really change anything in terms of who's hireable, how many people you need in your team, or what types of people you need your team. I feel like, just like the cloud allowed us to have less people to do roughly the same thing as we all did in own data centers, I feel like to a large extent, these AIs are just going to do the same thing. It's not fundamentally changing the game for most people to allow a university graduate to become a senior engineer overnight, or the fact that you don't need, you know, the idea that you don't maybe need senior engineers anymore and you can operate at AWS at scale, multi-region setup with some person with a year experience. I don't think any of those things are true in the near term.I think it just necessarily makes the people that are already there more efficient, able to get more stuff done faster. And we've been dealing with that for 30, 40, 50 years, like, that's exactly—I have this slideshow that I keep, I've been using it for a decade and it hasn't really changed. And I got in in the mid-'90s when we were changing from single large computers to distributed computing when the PC took out—took on. I mean, like, I was doing miniframes, and, you know, IBMs and HP Unixes. And that's where I jumped in.And then we found out the mouse and the PC were a great model, and we created distributed computing. That changed the game, allowed us, so many of us to get in that weren't mainframe experts and didn't know COBOL and a lot of us were able to get in and Windows or Microsoft made a great decision of saying, “We're going to make the server operating system look and act exactly like the client operating system.” And then suddenly, all of us PC enthusiasts were now server admins. So, there's this big shift in the '90s. We got a huge amount of server admins.And then virtualization showed up, you know, five years later, and suddenly, we were able to do so much more with the same number of people in a data center and with a bunch of servers. And I watched my team in a big government organization was running 18 people. I had three hardware guys in the data center. That went to one in a matter of years because we were able to virtualize so much we needed physical servers less often, we needed less physical data center server admins, we needed more people to run the software. So, we shifted that team down and then we scaled up software development and people that knew more about actually managing and running software.So, this is, like, I feel like the shifts are happening, then we had the cloud and then we had containerization. It doesn't really change it at a vast scale. And I think sometimes people are a little bit too worried about the LLMs as if they're somehow going to make tech workers obsolete. And I just think, no, we're just going to be managing the different things. We're going to—someone else said the great quote, and I'll end with this, you know, “It's not the LLM that's going to replace you. It's the person who knows the LLMs that's going to replace you.”And that's the same thing you could have said ten years ago for, “It's not the cloud that's going to replace you. It's someone who knows how to manage the cloud that's going to replace you.” [laugh]. So, you could swap that word out for—Corey: A line I heard, must have been 30 years ago now is, “Think. It's the only thing keeping a computer from taking your job.”Bret: Yeah [laugh], and these things don't think so. We haven't figured that one out yet.Corey: Yeah. Some would say that some people's coworkers don't either, but that's just uncharitable.Bret: That's me without coffee [laugh].Corey: [laugh]. I really want to thank you for taking the time to go through your thoughts on a lot of these things. If people want to learn more, where's the best place for them to find you?Bret: bretfisher.com, or just search Bret Fisher. You'll find all my stuff, hopefully, if I know how to use the internet, B-R-E-T F-I-S-H-E-R. And yeah, you'll find a YouTube channel, on Twitter, I hang out there every day, and on my website.Corey: And we will, of course, put links to that in the [show notes 00:38:22]. Thank you so much for taking the time to speak with me today. I really appreciate it.Bret: Yeah. Thanks, Corey. See you soon.Corey: Bret Fisher, DevOps dude and cloud-native trainer. I'm Cloud Economist Corey Quinn, and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice along with an angry comment that you have a Chat-Gippity thing write for you, where, just like you, it sounds very confident, but it's also completely wrong.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.

Austin Parker, Community Maintainer at OpenTelemetry, joins Corey on Screaming in the Cloud to discuss OpenTelemetry's mission in the world of observability. Austin explains how the OpenTelemetry community was able to scale the OpenTelemetry project to a commercial offering, and the way Open Telemetry is driving innovation in the data space. Corey and Austin also discuss why Austin decided to write a book on OpenTelemetry, and the book's focus on the evergreen applications of the tool. About AustinAustin Parker is the OpenTelemetry Community Maintainer, as well as an event organizer, public speaker, author, and general bon vivant. They've been a part of OpenTelemetry since its inception in 2019.Links Referenced: OpenTelemetry: https://opentelemetry.io/ Learning OpenTelemetry early release: https://www.oreilly.com/library/view/learning-opentelemetry/9781098147174/ Page with Austin's social links: https://social.ap2.io TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: Look, I get it. Folks are being asked to do more and more. Most companies don't have a dedicated DBA because that person now has a full time job figuring out which one of AWS's multiple managed database offerings is right for every workload. Instead, developers and engineers are being asked to support, and heck, if time allows, optimize their databases. That's where OtterTune comes in. Their AI is your database co-pilot for MySQL and PostgresSQL on Amazon RDS or Aurora. It helps improve performance by up to four x OR reduce costs by 50 percent – both of those are decent options. Go to ottertune dot com to learn more and start a free trial. That's O-T-T-E-R-T-U-N-E dot com.Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. It's been a few hundred episodes since I had Austin Parker on to talk about the things that Austin cares about. But it's time to rectify that. Austin is the community maintainer for OpenTelemetry, which is a CNCF project. If you're unfamiliar with, we're probably going to fix that in short order. Austin, Welcome back, it's been a month of Sundays.Austin: It has been a month-and-a-half of Sundays. A whole pandemic-and-a-half.Corey: So, much has happened since then. I tried to instrument something with OpenTelemetry about a year-and-a-half ago, and in defense to the project, my use case is always very strange, but it felt like—a lot of things have sharp edges, but it felt like this had so many sharp edges that you just pivot to being a chainsaw, and I would have been at least a little bit more understanding of why it hurts so very much. But I have heard from people that I trust that the experience has gotten significantly better. Before we get into the nitty-gritty of me lobbing passive-aggressive bug reports at you have for you to fix in a scenario in which you can't possibly refuse me, let's start with the beginning. What is OpenTelemetry?Austin: That's a great question. Thank you for asking it. So, OpenTelemetry is an observability framework. It is run by the CNCF, you know, home of such wonderful award-winning technologies as Kubernetes, and you know, the second biggest source of YAML in the known universe [clear throat].Corey: On some level, it feels like that is right there with hydrogen as far as unlimited resources in our universe.Austin: It really is. And, you know, as we all know, there are two things that make, sort of, the DevOps and cloud world go around: one of them being, as you would probably know, AWS bills; and the second being YAML. But OpenTelemetry tries to kind of carve a path through this, right, because we're interested in observability. And observability, for those that don't know or have been living under a rock or not reading blogs, it's a lot of things. It's a—but we can generally sort of describe it as, like, this is how you understand what your system is doing.I like to describe it as, it's a way that we can model systems, especially complex, distributed, or decentralized software systems that are pretty commonly found in larg—you know, organizations of every shape and size, quite often running on Kubernetes, quite often running in public or private clouds. And the goal of observability is to help you, you know, model this system and understand what it's doing, which is something that I think we can all agree, a pretty important part of our job as software engineers. Where OpenTelemetry fits into this is as the framework that helps you get the telemetry data you need from those systems, put it into a universal format, and then ship it off to some observability back-end, you know, a Prometheus or a Datadog or whatever, in order to analyze that data and get answers to your questions you have.Corey: From where I sit, the value of OTel—or OpenTelemetry; people in software engineering love abbreviations that are impenetrable from the outside, so of course, we're going to lean into that—but what I found for my own use case is the shining value prop was that I could instrument an application with OTel—in theory—and then send whatever I wanted that was emitted in terms of telemetry, be it events, be it logs, be it metrics, et cetera, and send that to any or all of a curation of vendors on a case-by-case basis, which meant that suddenly it was the first step in, I guess, an observability pipeline, which increasingly is starting to feel like a milit—like an industrial-observability complex, where there's so many different companies out there, it seems like a good approach to use, to start, I guess, racing vendors in different areas to see which performs better. One of the challenges I've had with that when I started down that path is it felt like every vendor who was embracing OTel did it from a perspective of their implementation. Here's how to instrument it to—send it to us because we're the best, obviously. And you're a community maintainer, despite working at observability vendors yourself. You have always been one of those community-first types where you care more about the user experience than you do this quarter for any particular employer that you have, which to be very clear, is intended as a compliment, not a terrifying warning. It's why you have this authentic air to you and why you are one of those very few voices that I trust in a space where normally I need to approach it with significant skepticism. How do you see the relationship between vendors and OpenTelemetry?Austin: I think the hard thing is that I know who signs my paychecks at the end of the day, right, and you always have, you know, some level of, you know, let's say bias, right? Because it is a bias to look after, you know, them who brought you to the dance. But I think you can be responsible with balancing, sort of, the needs of your employer, and the needs of the community. You know, the way I've always described this is that if you think about observability as, like, a—you know, as a market, what's the total addressable market there? It's literally everyone that uses software; it's literally every software company.Which means there's plenty of room for people to make their numbers and to buy and sell and trade and do all this sort of stuff. And by taking that approach, by taking sort of the big picture approach and saying, “Well, look, you know, there's going to be—you know, of all these people, there are going to be some of them that are going to use our stuff and there are some of them that are going to use our competitor's stuff.” And that's fine. Let's figure out where we can invest… in an OpenTelemetry, in a way that makes sense for everyone and not just, you know, our people. So, let's build things like documentation, right?You know, one of the things I'm most impressed with, with OpenTelemetry over the past, like, two years is we went from being, as a project, like, if you searched for OpenTelemetry, you would go and you would get five or six or ten different vendor pages coming up trying to tell you, like, “This is how you use it, this is how you use it.” And what we've done as a community is we've said, you know, “If you go looking for documentation, you should find our website. You should find our resources.” And we've managed to get the OpenTelemetry website to basically rank above almost everything else when people are searching for help with OpenTelemetry. And that's been really good because, one, it means that now, rather than vendors or whoever coming in and saying, like, “Well, we can do this better than you,” we can be like, “Well, look, just, you know, put your effort here, right? It's already the top result. It's already where people are coming, and we can prove that.”And two, it means that as people come in, they're going to be put into this process of community feedback, where they can go in, they can look at the docs, and they can say, “Oh, well, I had a bad experience here,” or, “How do I do this?” And we get that feedback and then we can improve the docs for everyone else by acting on that feedback, and the net result of this is that more people are using OpenTelemetry, which means there are more people kind of going into the tippy-tippy top of the funnel, right, that are able to become a customer of one of these myriad observability back ends.Corey: You touched on something very important here, when I first was exploring this—you may have been looking over my shoulder as I went through this process—my impression initially was, oh, this is a ‘CNCF project' in quotes, where—this is not true universally, of course, but there are cases where it clearly—is where this is an, effectively, vendor-captured project, not necessarily by one vendor, but by an almost consortium of them. And that was my takeaway from OpenTelemetry. It was conversations with you, among others, that led me to believe no, no, this is not in that vein. This is clearly something that is a win. There are just a whole bunch of vendors more-or-less falling all over themselves, trying to stake out thought leadership and imply ownership, on some level, of where these things go. But I definitely left with a sense that this is bigger than any one vendor.Austin: I would agree. I think, to even step back further, right, there's almost two different ways that I think vendors—or anyone—can approach OpenTelemetry, you know, from a market perspective, and one is to say, like, “Oh, this is socializing, kind of, the maintenance burden of instrumentation.” Which is a huge cost for commercial players, right? Like, if you're a Datadog or a Splunk or whoever, you know, you have these agents that you go in and they rip telemetry out of your web servers, out of your gRPC libraries, whatever, and it costs a lot of money to pay engineers to maintain those instrumentation agents, right? And the cynical take is, oh, look at all these big companies that are kind of like pushing all that labor onto the open-source community, and you know, I'm not casting any aspersions here, like, I do think that there's an element of truth to it though because, yeah, that is a huge fixed cost.And if you look at the actual lived reality of people and you look at back when SignalFx was still a going concern, right, and they had their APM agents open-sourced, you could go into the SignalFx repo and diff, like, their [Node Express 00:10:15] instrumentation against the Datadog Node Express instrumentation, and it's almost a hundred percent the same, right? Because it's truly a commodity. There's no—there's nothing interesting about how you get that telemetry out. The interesting stuff all happens after you have the telemetry and you've sent it to some back-end, and then you can, you know, analyze it and find interesting things. So, yeah, like, it doesn't make sense for there to be five or six or eight different companies all competing to rebuild the same wheels over and over and over and over when they don't have to.I think the second thing that some people are starting to understand is that it's like, okay, let's take this a step beyond instrumentation, right? Because the goal of OpenTelemetry really is to make sure that this instrumentation is native so that you don't need a third-party agent, you don't need some other process or jar or whatever that you drop in and it instruments stuff for you. The JVM should provide this, your web framework should provide this, your RPC library should provide this right? Like, this data should come from the code itself and be in a normalized fashion that can then be sent to any number of vendors or back ends or whatever. And that changes how—sort of, the competitive landscape a lot, I think, for observability vendors because rather than, kind of, what you have now, which is people will competing on, like, well, how quickly can I throw this agent in and get set up and get a dashboard going, it really becomes more about, like, okay, how are you differentiating yourself against every other person that has access to the same data, right? And you get more interesting use cases and how much more interesting analysis features, and that results in more innovation in, sort of, the industry than we've seen in a very long time.Corey: For me, just from the customer side of the world, one of the biggest problems I had with observability in my career as an SRE-type for years was you would wind up building your observability pipeline around whatever vendor you had selected and that meant emphasizing the things they were good at and de-emphasizing the things that they weren't. And sometimes it's worked to your benefit; usually not. But then you always had this question when it got things that touched on APM or whatnot—or Application Performance Monitoring—where oh, just embed our library into this. Okay, great. But a year-and-a-half ago, my exposure to this was on an application that I was running in distributed fashion on top of AWS Lambda.So great, you can either use an extension for this or you can build in the library yourself, but then there's always a question of precedence where when you have multiple things that are looking at this from different points of view, which one gets done first? Which one is going to see the others? Which one is going to enmesh the other—enclose the others in its own perspective of the world? And it just got incredibly frustrating. One of the—at least for me—bright lights of OTel was that it got away from that where all of the vendors receiving telemetry got the same view.Austin: Yeah. They all get the same view, they all get the same data, and you know, there's a pretty rich collection of tools that we're starting to develop to help you build those pipelines yourselves and really own everything from the point of generation to intermediate collection to actually outputting it to wherever you want to go. For example, a lot of really interesting work has come out of the OpenTelemetry collector recently; one of them is this feature called Connectors. And Connectors let you take the output of certain pipelines and route them as inputs to another pipeline. And as part of that connection, you can transform stuff.So, for example, let's say you have a bunch of [spans 00:14:05] or traces coming from your API endpoints, and you don't necessarily want to keep all those traces in their raw form because maybe they aren't interesting or maybe there's just too high of a volume. So, with Connectors, you can go and you can actually convert all of those spans into metrics and export them to a metrics database. You could continue to save that span data if you want, but you have options now, right? Like, you can take that span data and put it into cold storage or put it into, like, you know, some sort of slow blob storage thing where it's not actively indexed and it's slow lookups, and then keep a metric representation of it in your alerting pipeline, use metadata exemplars or whatever to kind of connect those things back. And so, when you do suddenly see it's like, “Oh, well, there's some interesting p99 behavior,” or we're hitting an alert or violating an SLO or whatever, then you can go back and say, like, “Okay, well, let's go dig through the slow da—you know, let's look at the cold data to figure out what actually happened.”And those are features that, historically, you would have needed to go to a big, important vendor and say, like, “Hey, here's a bunch of money,” right? Like, “Do this for me.” Now, you have the option to kind of do all that more interesting pipeline stuff yourself and then make choices about vendors based on, like, who is making a tool that can help me with the problem that I have? Because most of the time, I don't—I feel like we tend to treat observability tools as—it depends a lot on where you sit in the org—but you certainly seen this movement towards, like, “Well, we don't want a tool; we want a platform. We want to go to Lowe's and we want to get the 48-in-one kit that has a bunch of things in it. And we're going to pay for the 48-in-one kit, even if we only need, like, two things or three things out of it.”OpenTelemetry lets you kind of step back and say, like, “Well, what if we just got, like, really high-quality tools for the two or three things we need, and then for the rest of the stuff, we can use other cheaper options?” Which is, I think, really attractive, especially in today's macroeconomic conditions, let's say.Corey: One thing I'm trying to wrap my head around because we all find when it comes to observability, in my experience, it's the parable of three blind people trying to describe an elephant by touch; depending on where you are on the elephant, you have a very different perspective. What I'm trying to wrap my head around is, what is the vision for OpenTelemetry? Is it specifically envisioned to be the agent that runs wherever the workload is, whether it's an agent on a host or a layer in a Lambda function, or a sidecar or whatnot in a Kubernetes cluster that winds up gathering and sending data out? Or is the vision something different? Because part of what you're saying aligns with my perspective on it, but other parts of it seem to—that there's a misunderstanding somewhere, and it's almost certainly on my part.Austin: I think the long-term vision is that you as a developer, you as an SRE, don't even have to think about OpenTelemetry, that when you are using your container orchestrator or you are using your API framework or you're using your Managed API Gateway, or any kind of software that you're building something with, that the telemetry data from that software is emitted in an OpenTelemetry format, right? And when you are writing your code, you know, and you're using gRPC, let's say, you could just natively expect that OpenTelemetry is kind of there in the background and it's integrated into the actual libraries themselves. And so, you can just call the OpenTelemetry API and it's part of the standard library almost, right? You add some additional metadata to a span and say, like, “Oh, this is the customer ID,” or, “This is some interesting attribute that I want to track for later on,” or, “I'm going to create a histogram here or counter,” whatever it is, and then all that data is just kind of there, right, invisible to you unless you need it. And then when you need it, it's there for you to kind of pick up and send off somewhere to any number of back-ends or databases or whatnot that you could then use to discover problems or better model your system.That's the long-term vision, right, that it's just there, everyone uses it. It is a de facto and du jour standard. I think in the medium term, it does look a little bit more like OpenTelemetry is kind of this Swiss army knife agent that's running on—inside cars in Kubernetes or it's running on your EC2 instance. Until we get to the point of everyone just agrees that we're going to use OpenTelemetry protocol for the data and we're going to use all your stuff and we just natively emit it, then that's going to be how long we're in that midpoint. But that's sort of the medium and long-term vision I think. Does that track?Corey: It does. And I'm trying to equate this to—like the evolution back in the Stone Age was back when I was first getting started, Nagios was the gold standard. It was kind of the original Call of Duty. And it was awful. There were a bunch of problems with it, but it also worked.And I'm not trying to dunk on the people who built that. We all stand on the shoulders of giants. It was an open-source project that was awesome doing exactly what it did, but it was a product built for a very different time. It completely had the wheels fall off as soon as you got to things were even slightly ephemeral because it required this idea of the server needed to know where all of the things that was monitoring lived as an individual host basis, so there was this constant joy of, “Oh, we're going to add things to a cluster.” Its perspective was, “What's a cluster?” Or you'd have these problems with a core switch going down and suddenly everything else would explode as well.And even setting up an on-call rotation for who got paged when was nightmarish. And a bunch of things have evolved since then, which is putting it mildly. Like, you could say that about fire, the invention of the wheel. Yeah, a lot of things have evolved since the invention of the wheel, and here we are tricking sand into thinking. But we find ourselves just—now it seems that the outcome of all of this has been instead of one option that's the de facto standard that's kind of terrible in its own ways, now, we have an entire universe of different products, many of which are best-of-breed at one very specific thing, but nothing's great at everything.It's the multifunction printer conundrum, where you find things that are great at one or two things at most, and then mediocre at best at the rest. I'm excited about the possibility for OpenTelemetry to really get to a point of best-of-breed for everything. But it also feels like the money folks are pushing for consolidation, if you believe a lot of the analyst reports around this of, “We already pay for seven different observability vendors. How about we knock it down to just one that does all of these things?” Because that would be terrible. What do you land on that?Austin: Well, as I intu—or alluded to this earlier, I think the consolidation in the observability space, in general, is very much driven by that force you just pointed out, right? The buyers want to consolidate more and more things into single tools. And I think there's a lot of… there are reasons for that that—you know, there are good reasons for that, but I also feel like a lot of those reasons are driven by fundamentally telemetry-side concerns, right? So like, one example of this is if you were Large Business X, and you see—you are an engineering director and you get a report, that's like, “We have eight different metrics products.” And you're like, “That seems like a lot. Let's just use Brand X.”And Brand X will tell you very, very happily tell you, like, “Oh, you just install our thing everywhere and you can get rid of all these other tools.” And usually, there's two reasons that people pick tools, right? One reason is that they are forced to and then they are forced to do a bunch of integration work to get whatever the old stuff was working in the new way, but the other reason is because they tried a bunch of different things and they found the one tool that actually worked for them. And what happens invariably in these sort of consolidation stories is, you know, the new vendor comes in on a shining horse to consolidate, and you wind up instead of eight distinct metrics tools, now you have nine distinct metrics tools because there's never any bandwidth for people to go back and, you know—you're Nagios example, right, Nag—people still use Nagios every day. What's the economic justification to take all those Nagios installs, if they're working, and put them into something else, right?What's the economic justification to go and take a bunch of old software that hasn't been touched for ten years that still runs and still does what needs to do, like, where's the incentive to go and re-instrument that with OpenTelemetry or anything else? It doesn't necessarily exist, right? And that's a pretty, I think, fundamental decision point in everyone's observability journey, which is what do you do about all the old stuff? Because most of the stuff is the old stuff and the worst part is, most of the stuff that you make money off of is the old stuff as well. So, you can't ignore it, and if you're spending, you know, millions of millions of dollars on the new stuff—like, there was a story that went around a while ago, I think, Coinbase spent something like, what, $60 million on Datadog… I hope they asked for it in real money and not Bitcoin. But—Corey: Yeah, something I've noticed about all the vendors, and even Coinbase themselves, very few of them actually transact in cryptocurrency. It's always cash on the barrelhead, so to speak.Austin: Yeah, smart. But still, like, that's an absurd amount of money [laugh] for any product or service, I would argue, right? But that's just my perspective. I do think though, it goes to show you that you know, it's very easy to get into these sort of things where you're just spending over the barrel to, like, the newest vendor that's going to come in and solve all your problems for you. And just, it often doesn't work that way because most places aren't—especially large organizations—just aren't built in is sort of like, “Oh, we can go through and we can just redo stuff,” right? “We can just roll out a new agent through… whatever.”We have mainframes [unintelligible 00:25:09], mainframes to thinking about, you have… in many cases, you have an awful lot of business systems that most, kind of, cloud people don't like, think about, right, like SAP or Salesforce or ServiceNow, or whatever. And those sort of business process systems are actually responsible for quite a few things that are interesting from an observability point of view. But you don't see—I mean, hell, you don't even see OpenTelemetry going out and saying, like, “Oh, well, here's the thing to let you know, observe Apex applications on Salesforce,” right? It's kind of an undiscovered country in a lot of ways and it's something that I think we will have to grapple with as we go forward. In the shorter term, there's a reason that OpenTelemetry mostly focuses on cloud-native applications because that's a little bit easier to actually do what we're trying to do on them and that's where the heat and light is. But once we get done with that, then the sky is the limit.[midroll 00:26:11]Corey: It still feels like OpenTelemetry is evolving rapidly. It's certainly not, I don't want to say it's not feature complete, which, again, what—software is never done. But it does seem like even quarter-to-quarter or month-to-month, its capabilities expand massively. Because you apparently enjoy pain, you're in the process of writing a book. I think it's in early release or early access that comes out next year, 2024. Why would you do such a thing?Austin: That's a great question. And if I ever figure out the answer I will tell you.Corey: Remember, no one wants to write a book; they want to have written the book.Austin: And the worst part is, is I have written the book and for some reason, I went back for another round. I—Corey: It's like childbirth. No one remembers exactly how horrible it was.Austin: Yeah, my partner could probably attest to that. Although I was in the room, and I don't think I'd want to do it either. So, I think the real, you know, the real reason that I decided to go and kind of write this book—and it's Learning OpenTelemetry; it's in early release right now on the O'Reilly learning platform and it'll be out in print and digital next year, I believe, we're targeting right now, early next year.But the goal is, as you pointed out so eloquently, OpenTelemetry changes a lot. And it changes month to month sometimes. So, why would someone decide—say, “Hey, I'm going to write the book about learning this?” Well, there's a very good reason for that and it is that I've looked at a lot of the other books out there on OpenTelemetry, on observability in general, and they talk a lot about, like, here's how you use the API. Here's how you use the SDK. Here's how you make a trace or a span or a log statement or whatever. And it's very technical; it's very kind of in the weeds.What I was interested in is saying, like, “Okay, let's put all that stuff aside because you don't necessarily…” I'm not saying any of that stuff's going to change. And I'm not saying that how to make a span is going to change tomorrow; it's not, but learning how to actually use something like OpenTelemetry isn't just knowing how to create a measurement or how to create a trace. It's, how do I actually use this in a production system? To my point earlier, how do I use this to get data about, you know, these quote-unquote, “Legacy systems?” How do I use this to monitor a Kubernetes cluster? What's the important parts of building these observability pipelines? If I'm maintaining a library, how should I integrate OpenTelemetry into that library for my users? And so on, and so on, and so forth.And the answers to those questions actually probably aren't going to change a ton over the next four or five years. Which is good because that makes it the perfect thing to write a book about. So, the goal of Learning OpenTelemetry is to help you learn not just how to use OpenTelemetry at an API or SDK level, but it's how to build an observability pipeline with OpenTelemetry, it's how to roll it out to an organization, it's how to convince your boss that this is what you should use, both for new and maybe picking up some legacy development. It's really meant to give you that sort of 10,000-foot view of what are the benefits of this, how does it bring value and how can you use it to build value for an observability practice in an organization?Corey: I think that's fair. Looking at the more quote-unquote, “Evergreen,” style of content as opposed to—like, that's the reason for example, I never wind up doing tutorials on how to use an AWS service because one console change away and suddenly I have to redo the entire thing. That's a treadmill I never had much interest in getting on. One last topic I want to get into before we wind up wrapping the episode—because I almost feel obligated to sprinkle this all over everything because the analysts told me I have to—what's your take on generative AI, specifically with an eye toward observability?Austin: [sigh], gosh, I've been thinking a lot about this. And—hot take alert—as a skeptic of many technological bubbles over the past five or so years, ten years, I'm actually pretty hot on AI—generative AI, large language models, things like that—but not for the reasons that people like to kind of hold them up, right? Not so that we can all make our perfect, funny [sigh], deep dream, meme characters or whatever through Stable Fusion or whatever ChatGPT spits out at us when we ask for a joke. I think the real win here is that this to me is, like, the biggest advance in human-computer interaction since resistive touchscreens. Actually, probably since the mouse.Corey: I would agree with that.Austin: And I don't know if anyone has tried to get someone that is, you know, over the age of 70 to use a computer at any time in their life, but mapping human language to trying to do something on an operating system or do something on a computer on the web is honestly one of the most challenging things that faces interface design, face OS designers, faces anyone. And I think this also applies for dev tools in general, right? Like, if you think about observability, if you think about, like, well, what are the actual tasks involved in observability? It's like, well, you're making—you're asking questions. You're saying, like, “Hey, for this metric named HTTPrequestsByCode,” and there's four or five dimensions, and you say, like, “Okay, well break this down for me.” You know, you have to kind of know the magic words, right? You have to know the magic promQL sequence or whatever else to plug in and to get it to graph that for you.And you as an operator have to have this very, very well developed, like, depth of knowledge and math and statistics to really kind of get a lot of—Corey: You must be at least this smart to ride on this ride.Austin: Yeah. And I think that, like that, to me is the real—the short-term win for certainly generative AI around using, like, large language models, is the ability to create human language interfaces to observability tools, that—Corey: As opposed to learning your own custom SQL dialect, which I see a fair number of times.Austin: Right. And, you know, and it's actually very funny because there was a while for the—like, one of my kind of side projects for the past [sigh] a little bit [unintelligible 00:32:31] idea of, like, well, can we make, like, a universal query language or universal query layer that you could ship your dashboards or ship your alerts or whatever. And then it's like, generative AI kind of just, you know, completely leapfrogs that, right? It just says, like, well, why would you need a query language, if we can just—if you can just ask the computer and it works, right?Corey: The most common programming language is about to become English.Austin: Which I mean, there's an awful lot of externalities there—Corey: Which is great. I want to be clear. I'm not here to gatekeep.Austin: Yeah. I mean, I think there's a lot of externalities there, and there's a lot—and the kind of hype to provable benefit ratio is very skewed right now towards hype. That said, one of the things that is concerning to me as sort of an observability practitioner is the amount of people that are just, like, whole-hog, throwing themselves into, like, oh, we need to integrate generative AI, right? Like, we need to put AI chatbots and we need to have ChatGPT built into our products and da-da-da-da-da. And now you kind of have this perfect storm of people that really don't ha—because they're just using these APIs to integrate gen AI stuff with, they really don't understand what it's doing because a lot you know, it is very complex, and I'll be the first to admit that I really don't understand what a lot of it is doing, you know, on the deep, on the foundational math side.But if we're going to have trust in, kind of, any kind of system, we have to understand what it's doing, right? And so, the only way that we can understand what it's doing is through observability, which means it's incredibly important for organizations and companies that are building products on generative AI to, like, drop what—you know, walk—don't walk, run towards something that is going to give you observability into these language models.Corey: Yeah. “The computer said so,” is strangely dissatisfying.Austin: Yeah. You need to have that base, you know, sort of, performance [goals and signals 00:34:31], obviously, but you also need to really understand what are the questions being asked. As an example, let's say you have something that is tokenizing questions. You really probably do want to have some sort of observability on the hot path there that lets you kind of break down common tokens, especially if you were using, like, custom dialects or, like, vectors or whatever to modify the, you know, neural network model, like, you really want to see, like, well, what's the frequency of the certain tokens that I'm getting they're hitting the vectors versus not right? Like, where can I improve these sorts of things? Where am I getting, like, unexpected results?And maybe even have some sort of continuous feedback mechanism that it could be either analyzing the tone and tenor of end-user responses or you can have the little, like, frowny and happy face, whatever it is, like, something that is giving you that kind of constant feedback about, like, hey, this is how people are actually like interacting with it. Because I think there's way too many stories right now people just kind of like saying, like, “Oh, okay. Here's some AI-powered search,” and people just, like, hating it. Because people are already very primed to distrust AI, I think. And I can't blame anyone.Corey: Well, we've had an entire lifetime of movies telling us that's going to kill us all.Austin: Yeah.Corey: And now you have a bunch of, also, billionaire tech owners who are basically intent on making that reality. But that's neither here nor there.Austin: It isn't, but like I said, it's difficult. It's actually one of the first times I've been like—that I've found myself very conflicted.Corey: Yeah, I'm a booster of this stuff; I love it, but at the same time, you have some of the ridiculous hype around it and the complete lack of attention to safety and humanity aspects of it that it's—I like the technology and I think it has a lot of promise, but I want to get lumped in with that set.Austin: Exactly. Like, the technology is great. The fan base is… ehh, maybe something a little different. But I do think that, for lack of a better—not to be an inevitable-ist or whatever, but I do think that there is a significant amount of, like, this is a genie you can't put back in the bottle and it is going to have, like, wide-ranging, transformative effects on the discipline of, like, software development, software engineering, and white collar work in general, right? Like, there's a lot of—if your job involves, like, putting numbers into Excel and making pretty spreadsheets, then ooh, that doesn't seem like something that's going to do too hot when I can just have Excel do that for me.And I think we do need to be aware of that, right? Like, we do need to have that sort of conversation about, like… what are we actually comfortable doing here in terms of displacing human labor? When we do displace human labor, are we doing it so that we can actually give people leisure time or so that we can just cram even more work down the throats of the humans that are left?Corey: And unfortunately, I think we might know what that answer is, at least on our current path.Austin: That's true. But you know, I'm an optimist.Corey: I… don't do well with disappointment. Which the show has certainly not been. I really want to thank you for taking the time to speak with me today. If people want to learn more, where's the best place for them to find you?Austin: Welp, I—you can find me on most social media. Many, many social medias. I used to be on Twitter a lot, and we all know what happened there. The best place to figure out what's going on is check out my bio, social.ap2.io will give you all the links to where I am. And yeah, been great talking with you.Corey: Likewise. Thank you so much for taking the time out of your day. Austin Parker, community maintainer for OpenTelemetry. I'm Cloud Economist Corey Quinn and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice, along with an angry comment pointing out that actually, physicists say the vast majority of the universe's empty space, so that we can later correct you by saying ah, but it's empty whitespace. That's right. YAML wins again.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.

A few tools to build your own Way Back Machine, we check in with the "Year of Voice" and more.

Andreas Wittig, Co-Author of Amazon Web Services in Action and Co-Founder of marbot, joins Corey on Screaming in the Cloud to discuss ways to keep a book up to date in an ever-changing world, the advantages of working with a publisher, and how he began the journey of writing a book in the first place. Andreas also recalls how much he learned working on the third edition of Amazon Web Services in Action and how teaching can be an excellent tool for learning. Since writing the first edition, Adreas's business has shifted from a consulting business to a B2B product business, so he and Corey also discuss how that change came about and the pros and cons of each business model. About AndreasAndreas is the Co-Author of Amazon Web Services in Action and Co-Founder of marbot - AWS Monitoring made simple! He is also known on the internet as cloudonaut through the popular blog, podcast, and youtube channel he created with his brother Michael. Links Referenced: Amazon Web Services in Action: https://www.amazon.com/Amazon-Services-Action-Andreas-Wittig/dp/1617295116 Rapid Docker on AWS: https://cloudonaut.io/rapid-docker-on-aws/ bucket/av: https://bucketav.com/ marbot: https://marbot.io/ cloudonaut.io: https://cloudonaut.io TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. It's been a few years since I caught up with Andreas Wittig, who is also known in the internet as cloudonaut, and much has happened since then. Andreas, how are you?Andreas: Hey, absolutely. Thank you very much. I'm happy to be here in the show. I'm doing fine.Corey: So, one thing that I have always held you in some high regard for is that you have done what I have never had the attention span to do: you wrote a book. And you published it a while back through Manning, it was called Amazon Web Services in Action. That is ‘in action' two words, not Amazon Web Services Inaction of doing absolutely nothing about it, which is what a lot of companies in the space seem to do instead.Andreas: [laugh]. Yeah, absolutely. So. And it was not only me. I've written the book together with my brother because back in 2015, Manning, for some reason, wrote in and asked us if we would be interested in writing the book.And we had just founded our own consulting company back then and we had—we didn't have too many clients at the very beginning, so we had a little extra time free. And then we decided, okay, let's do the book. And let's write a book about Amazon Web Services, basically, a deep introduction into all things AWS. So, this was 2015, and it was indeed a lot of work, much more [laugh] than we expected. So, first of all, the hard part is, what do you want to have in the book? So, what's the TOC? What is important and must be in?And then you start writing and have examples and everything. So, it was really an interesting journey. And doing it together with a publisher like Manning was also really interesting because we learned a lot about writing. You have kind of a coach, an editor that helps you through that process. So, this was really a hard and fun experience.Corey: There's a lot of people that have said very good things about writing the book through a traditional publisher. And they also say that one of the challenges is it's a blessing and a curse, where you basically have someone standing over your shoulder saying, “Is it done yet? Is it done yet? Is it done yet?” The consensus that seems to have emerged from people who have written books is, “That was great, please don't ever ask me to do it again.”And my operating theory is that no one wants to write a book. They want to have written a book. Which feels like two very different things most of the time. But the reason you're back on now is that you have gone the way of the terrible college professor, where you're going to update the book, and therefore you get to do a whole new run of textbooks and make everyone buy it and kill the used market, et cetera. And you've done that twice now because you have just recently released the third edition. So, I have to ask, how different is version one from version two and from version three? Although my apologies; we call them ‘editions' in the publishing world.Andreas: [laugh]. Yeah, yeah. So, of course, as you can imagine, things change a lot in AWS world. So, of course, you have to constantly update things. So, I remember from first to second edition, we switched from CloudFormation in JSON to YAML. And now to the third edition, we added two new chapters. This was also important to us, so to keep also the scope of the book in shape.So, we have in the third edition, two new chapters. One is about automating deployments, recovering code deploy, [unintelligible 00:03:59], CloudFormation rolling updates in there. And then there was one important topic missing at all in the book, which was containers. And we finally decided to add that in, and we have now container chapter, starting with App Runner, which I find quite an interesting service to observe right now, and then our bread and butter service: ECS and Fargate. So, that's basically the two new chapters. And of course, then reworking all the other chapters is also a lot of work. And so, many things change over time. Cannot imagine [laugh].Corey: When was the first edition released? Because I believe the second one was released in 2018, which means you've been at this for a while.Andreas: Yeah. So, the first was 2015, the second 2018, three years later, and then we had five years, so now this third edition was released at the beginning of this year, 2023.Corey: Eh, I think you're right on schedule. Just March of 2020 lasted three years. That's fine.Andreas: Yeah [laugh].Corey: So, I have to ask, one thing that I've always appreciated about AWS is, it feels like with remarkably few exceptions, I can take a blog post written on how to do something with AWS from 2008 and now in 2023, I can go through every step along with that blog post. And yeah, I might have trouble getting some of the versions and services and APIs up and running, but the same steps will absolutely work. There are very few times where a previously working API gets deprecated and stops working. Is this the best way to proceed? Absolutely not.But you can still spin up the m1.medium instance sizes, or whatever it was, or [unintelligible 00:05:39] on small or whatever the original only size that you could get was. It's just there are orders of magnitude and efficiency gains you can do by—you can go through by using more modern approaches. So, I have to ask, was there anything in the book as you revised it—two times now—that needed to come out because it was now no longer working?Andreas: So, related to the APIs that's—they are really very stable, you're right about that. So, the problem is, our first few chapters where we have screenshots of how you go through the management—Corey: Oh no.Andreas: —console [laugh]. And you can probably, you can redo them every three months, probably, because the button moves or a step is included or something like that. So, the later chapters in the book, where we focus a lot on the CLI or CloudFormation and stuff like—or SDKs, they are pretty stable. But the first few [ones 00:06:29] are a nightmare to update all those screenshots. And then sometimes, so I was going through the book, and then I noticed, oh, there's a part of this chapter that I can completely remove nowadays.So, I can give you an example. So, I was going through the chapter about Simple Storage Service S3, and I—there was a whole section in the chapter about read-after-write consistency. Because back then, it was important that you knew that after updating an object or reading an object before it was created the first time, you could get outdated versions for a little while, so this was eventually consistent. But nowadays, AWS has changed that and basically now, S3 has this strong read-after-write consistency. So, I basically could remove that whole part in the chapter which was quite complicated to explain to the reader, right, so I [laugh] put a lot of effort into that.Corey: You think that was confusing? I look at the sea of systems I had to oversee at one company, specifically to get around that problem. It's like, well, we can now take this entire application and yeet it into the ocean because it was effectively a borderline service to that just want to ens—making consistency guarantees. It's not a common use case, but it is one that occurs often enough to be a problem. And of course, when you need it, you really need it. That was a nice under-the-hood change that was just one day, surprise, it works that way. But I'm sure it was years of people are working behind the scenes, solving for impossible problems to get there, and cetera, et cetera.Andreas: Yeah, yeah. But that's really cool is to remove parts of the book that are now less complicated. This is really cool. So, a few other examples. So, things change a lot. So, for example, EFS, so we have EFS, Elastic File System, in the book as well. So, now we have new throughput modes, different limits. So, there's really a lot going on and you have to carefully go through all the—Corey: Oh, when EFS launched, it was terrible. Now, it's great just because it's gotten so much more effective and efficient as a service. It's… AWS releases things before they're kind of ready, it feels like sometimes, and then they improve with time. I know there have been feature deprecations. For example, for some reason, they are no longer allowing us to share out a bucket via BitTorrent, which, you know, in 2006 when it came out, seemed like a decent idea to save on bandwidth. But here in 2023, no one cares about it.But I'm also keeping a running list of full-on AWS services that have been deprecated or have the deprecations announced. Are any of those in the book in any of its editions? And if and when there's a fourth edition, will some of those services have to come out?Andreas: [laugh]. Let's see. So, right after the book was published—because the problem with books is they get printed, right; that's the issue—but the target of the book, AWS, changes. So, a few weeks after the printed book was out, we found out that we have an issue in our one of our examples because now S3 buckets, when you create them, they have locked public access enabled by default. And this was not the case before. And one of our example relies on that it can create object access control lists, and this is not working now anymore. [laugh].So yeah, there are things changing. And we have, the cool thing about Manning is they have that what they call a live book, so you can read it online and you can have notes from other readers and us as the authors along the text, and there we can basically point you in the right direction and explain what happened here. So, this is how we try to keep the book updated. Of course, the printed one stays the same, but the ebook can change over time a little bit.Corey: Yes, ebooks are… at least keeping them updated is a lot easier, I would imagine. It feels like that—speaking of continuous builds and automatic CI/CD approaches—yeah, well, we could build a book just by updating some text in a Git repo or its equivalent, and pressing go, but it turns out that doing a whole new print run takes a little bit more work.Andreas: Yeah. Because you mentioned the experience of writing a book with a publisher and doing it on your own with self-publishing, so we did both in the past. We have Amazon Web Services in Action with Manning and we did another book, Rapid Docker on AWS in self-publishing. And what we found out is, there's really a lot of effort that goes into typesetting and layouting a book, making sure it looks consistent.And of course, you can just transform some markdown into a epub and PDF versions, but if a publisher is doing that, the results are definitely different. So, that was, besides the other help that we got from the publisher, very helpful. So, we enjoyed that as well.Corey: What is the current state of the art—since I don't know the answer to this one—around updating ebook versions? If I wind up buying an ebook on Kindle, for example, will they automatically push errata down automatically through their system, or do they reserve that for just, you know, unpublishing books that they realized shouldn't be on the Marketplace after people have purchased them?Andreas: [laugh]. So—Corey: To be fair, that only happened once, but I'm still giving them grief for it a decade and change later. But it was 1984. Of all the books to do that, too. I digress.Andreas: So, I'm not a hundred percent sure how it works with the Kindle. I know that Manning pushes out new versions by just emailing all the customers who bought the book and sending them a new version. Yeah.Corey: Yeah. It does feel, on some level, like there needs to be at least a certain degree of substantive change before they're going to start doing that. It's like well, good news. There was a typo on page 47 that we're going to go ahead and fix now. Two letters were transposed in a word. Now, that might theoretically be incredibly important if it's part of a code example, which yes, send that out, but generally, A, their editing is on point, so I didn't imagine that would sneak through, and 2, no one cares about a typo release and wants to get spammed over it?Andreas: Definitely, yeah. Every time there's a reprint of the book, you have the chance to make small modifications, to add something or remove something. That's also a way to keep it in shape a little bit.Corey: I have to ask, since most people talk about AWS services to a certain point of view, what is your take on databases? Are you sticking to the actual database services or are you engaged in my personal hobby of misusing everything as a database by holding it wrong?Andreas: [laugh]. So, my favorite database for starting out is DynamoDB. So, I really like working with DynamoDB and I like the limitations and the thing that you have to put some thoughts into how to structure your data set in before. But we also use a lot of Aurora, which really find an interesting technology. Unfortunately, Aurora Serverless, it's not becoming a product that I want to use. So, version one is now outdated, version two is much too expensive and restricted. So—Corey: I don't even know that it's outdated because I'm seeing version one still get feature updates to it. It feels like a divergent service. That is not what I would expect a version one versus version two to be. I'm with you on Dynamo, by the way. I started off using that and it is cheap is free for most workloads I throw at it. It's just a great service start to finish. The only downside is that if I need to move it somewhere else, then I have a problem.Andreas: That's true. Yeah, absolutely.Corey: I am curious, as far as you look across the sea of change—because you've been doing this for a while and when you write a book, there's nothing that I can imagine that would be better at teaching you the intricacies of something like AWS than writing a book on it. I got a small taste of this years ago when I shot my mouth off and committed to give a talk about Git. Well, time to learn Git. And teaching it to other people really solidifies a lot of the concepts yourself. Do you think that going through the process of writing this book has shaped how you perform as an engineer?Andreas: Absolutely. So, it's really interesting. So,I added the third edition and I worked on it mostly last year. And I didn't expect to learn a lot during that process actually, because I just—okay, I have to update all the examples, make sure everything work, go through the text, make sure everything is up to date. But I learned things, not only new things, but I relearned a lot of things that I wasn't aware of anymore. Or maybe I've never been; I don't know exactly [laugh].But it's always, if you go into the details and try to explain something to others, you learn a lot about that. So, teaching is a very good way to, first of all gather structure and a deep understanding of a topic and also dive into the details. Because when you write a book, every time you write a sentence, ask the question, is that really correct? Do I really know that or do I just assume that? So, I check the documentation, try to find out, is that really the case or is that something that came up myself?So, you'll learn a lot by doing that. And always come to the limits of the AWS documentation because sometimes stuff is just not documented and you need to figure out, what is really happening here? What's the real deal? And then this is basically the research part. So, I always find that interesting. And I learned a lot in during the third edition, while was only adding two new chapters and rewriting a lot of them. So, I didn't expect that.Corey: Do you find that there has been an interesting downstream effect from having written the book, that for better or worse, I've always no—I always notice myself responding to people who have written a book with more deference, more acknowledgment for the time and effort that it takes. And some books, let's be clear, are terrible, but I still find myself having that instinctive reaction because they saw something through to be published. Have you noticed it changing other aspects of your career over the past, oh, dear Lord, it would have been almost ten years now.Andreas: So, I think it helped us a lot with our consulting business, definitely. Because at the very beginning, so back in 2015, at least here in Europe and Germany, AWS was really new in the game. And being the one that has written a book about AWS was really helping… stuff. So, it really helped us a lot for our consulting work. I think now we are into that game of having to update the book [laugh] every few years, to make sure it stays up to date, but I think it really helped us for starting our consulting business.Corey: And you've had a consulting business for a while. And now you have effectively progressed to the next stage of consulting business lifecycle development, which is, it feels like you're becoming much more of a product company than you were in years past. Is that an accurate perception from the outside or am I misunderstanding something fundamental?Andreas: You know, absolutely, that's the case. So, from the very beginning, basically, when we founded our company, so eight years ago now, so we always had to go to do consulting work, but also do product work. And we had a rule of thumb that 20% of our time goes into product development. And we tried a lot of different things. So, we had just a few examples that failed completely.So, we had a Time [Series 00:17:49] as a Service offering at the very beginning of our journey, which failed completely. And now we have Amazon Timestream, which makes that totally—so now the market is maybe there for that. We tried a lot of things, tried content products, but also as we are coming from the software development world, we always try to build products. And over the years, we took what we learned from consulting, so we learned a lot about, of course, AWS, but also about the market, about the ecosystem. And we always try to bring that into the market and build products out of that.So nowadays, we really transitioned completely from consulting to a product company, as you said. So, we do not do any consulting anymore with one few exception with one of our [laugh] best or most important clients. But we are now a product company. And we only a two-person company. So, the idea was always how to scale a company without growing the team or hiring a lot of people, and a consulting business is definitely not a good way to do that, so yeah, this was why always invested into products.And now we have two products in the AWS Marketplace which works very well for us because it allows us to sell worldwide and really easily get a relationship up and running with our customers, and that pay through their AWS bill. So, that's really helping us a lot. Yeah.Corey: A few questions on that. At first it always seems to me that writing software or building a product is a lot like real estate in that you're doing a real estate development—to my understanding since I live in San Francisco and this is a [two exit 00:19:28] town; I still rent here—I found though, that you have to spend a lot of money and effort upfront and you don't get to start seeing revenue on that for years, which is why the VC model is so popular where you'll take $20 million, but then in return they want to see massive, outsized returns on that, which—it feels—push an awful lot of perfectly sustainable products into things that are just monstrous.Andreas: Hmm, yeah. Definitely.Corey: And to my understanding, you bootstrapped. You didn't take a bunch of outside money to do this, right?Andreas: No, no, we have completely bootstrapping and basically paying the bills with our consulting work. So yeah, I can give you one example. So, bucketAV is our solution to scan S3 buckets for malware, and basically, this started as an open-source project. So, this was just a side project we are working on. And we saw that there is some demand for that.So, people need ways to scan their objects—for example, user uploads—for malware, and we just tried to publish that in the AWS Marketplace to sell it through the Marketplace. And we don't really expect that this is a huge deal, and so we just did, I don't know, Michael spent a few days to make sure it's possible to publish that and get in shape. And over time, this really grew into an important, really substantial part of our business. And this doesn't happen overnight. So, this adds up, month by month. And you get feedback from customers, you improve the product based on that. And now this is one of the two main products that we sell in the Marketplace.Corey: I wanted to ask you about the Marketplace as well. Are you finding that that has been useful for you—obviously, as a procurement vehicle, it means no matter what country a customer is in, they can purchase it, it shows up on the AWS bill, and life goes on—but are you finding that it has been an effective way to find new customers?Andreas: Yes. So, I definitely would think so. It's always funny. So, we have completely inbound sales funnel. So, all customers find us through was searching the Marketplace or Google, probably. And so, what I didn't expect that it's possible to sell a B2B product that way. So, we don't know most of our customers. So, we know their name, we know the company name, but we don't know anyone there. We don't know the person who buys the product.This is, on the one side, a very interesting thing as a two-person company. You cannot build a huge sales process and I cannot invest too much time into the sales process or procurement process, so this really helps us a lot. The downside of it is a little bit that we don't have a close relationship with our customers and sometimes it's a little tricky for us to find important person to talk to, to get feedback and stuff. But on the other hand, yeah, it really helps us to sell to businesses all over the world. And we sell to very small business of course, but also to large enterprise customers. And they are fine with that process as well. And I think, even the large enterprises, they enjoy that it's so easy [laugh] to get a solution up and running and don't have to talk to any salespersons. So, enjoy it and I think our customers do as well.Corey: This is honestly the first time I've ever heard a verifiable account a vendor saying, “Yeah, we put this thing on the Marketplace, and people we've never talked to find us on the Marketplace and go ahead and buy.” That is not the common experience, let's put it that way. Now true, an awful lot of folks are selling enterprise software on this and someone—I forget who—many years ago had a great blog post on why no enterprise software costs $5,000. It either is going to cost $500 or it's going to cost 100 grand and up because the difference is, is at some point, you'd have a full-court press enterprise sales motion to go and sell the thing. And below a certain point, great, people are just going to be able to put it on their credit card and that's fine. But that's why you have this giant valley of there is very little stuff priced in that sweet spot.Andreas: Yeah. So, I think maybe it's important to mention that our products are relatively simple. So, they are just for a very small niche, a solution for a small problem. So, I think that helps a lot. So, we're not selling a full-blown cloud security solution; we only focus on that very small part: scanning S3 objects for malware.For example, on marbot,f the other product that we sell, which is monitoring of AWS accounts. Again, we focus on a very simple way to monitor AWS workloads. And so, I think that is probably why this is a successful way for us to find new customers because it's not a very complicated product where you have to explain a lot. So, that's probably the differentiator here.Corey: Having spent a fair bit of time doing battle with compliance goblins—which is, to be clear, I'm not describing people; I'm describing processes—in many cases, we had to do bucket scanning for antivirus, just to check a compliance box. From our position, there was remarkably little risk of a user-generated picture of a receipt that is input sanitized to make sure it is in fact a picture, landing in an S3 bucket and then somehow infecting one of the Linux servers through which it passed. So, we needed something that just checked the compliance box or we would not be getting the gold seal on our website today. And it was, more or less, a box-check as opposed to something that solved a legitimate problem. This was also a decade and change ago. Has that changed to a point now where there are legitimate threats and concerns around this, or is it still primarily just around make the auditor stop yelling at me, please?Andreas: Mmm. I think it's definitely to tick the checkbox, to be compliant with this, some regulation. On the other side, I think there are definitely use cases where it makes a lot of sense, especially when it comes to user-generated content of all kinds, especially if you're not only consuming it internally, but maybe also others can immediately start downloading that. So, that is where we see many of our customers are coming with that scenario that they want to make sure that the files that people upload and others can download are not infected. So, that is probably the most important use case.Corey: There's also, on some level, an increasing threat of ransomware. And for a long time, I was very down on the ideas of all these products that hit the market to defend S3 buckets against ransomware. Until one day, there was an AWS security blog post talking about how they found it. And yeah, we've we have seen this in the wild; it is causing problems for companies; here's what to do about it. Because it's one of those areas where I can't trust a vendor who's trying to sell me something to tell me that this problem exists.I mean, not to cast aspersions, but they're very interested, they're very incentivized to tell that story, whereas AWS is not necessarily incentivized to tell a story like that. So, that really brought it home for me that no, this is a real thing. So, I just want to be clear that my opinion on these things does in fact, evolve. It's not, “Well, I thought it was dumb back in 2012, so clearly it's still dumb now.” That is not my position, I want to be very clear on that.I do want to revisit for a moment, the idea of going from a consultancy that is a services business over to a product business because we've toyed with aspects of that here at The Duckbill Group a fair bit. We've not really found long-term retainer services engagements that add value that we are comfortable selling. And that means as a result that when you sell fixed duration engagements, it's always a sell, sell, sell, where's the next project coming from? Whereas with product businesses, it's oh, the grass is always greener on the other side. It's recurring revenue. Someone clicks, the revenue sticks around and never really goes away. That's the dream from where I sit on the services side of the fence, wistfully looking across and wondering what if. Now that you've made that transition, what sucks about product businesses that you might not have seen going into it?Andreas: [laugh]. Yeah, that a good question. So, on the one side, it was really also our dream to have a product business because it really changes the way we work. We can block large parts of our calendar to do deep-focus work, focus on things, find new solutions, and really try to make a solution that really fits to problem and uses all the AWS capabilities to do so. And on the other side, a product business involves, of course, selling the product, which is hard.And we are two software engineers, [laugh] and really making sure that we optimize our sales and there's search engine optimization, all that stuff, this is really hard for us because we don't know anything about that and we always have to find an expert, or we need to build a knowledge ourself, try things out, and so on. So, that whole part of selling the product, this is really a challenge for us. And then of course, product business evolves a lot of support work. So, we get support emails multiple times per hour, and we have to answer them and be as fast as possible with that. So, that is, of course, something that you do not have to do with consulting work.And not always that, the questions are many times really simple questions that pointed people in the right direction, find part of the documentation that answers the question. So, that is a constant stream of questions coming in that you have to answer. So, the inbox is always full [laugh]. So, that is maybe a small downside of a product business. But other than that, yeah, compared to a consulting business, it really gives us many flexibilities with planning our work day around the rest of our lives. That's really what we enjoy about a product company.Corey: I was very careful to pick an expensive problem that was only a business-hours problem. So, I don't wind up with a surprise, middle-of-the-night panic phone call. It's yeah, it turns out that AWS billing operate during business hours in the US Pacific Time. The end. And there are no emergencies here; there are simply curiosities that will, in the fullness of time take weeks to get resolved.Andreas: Mmm. Yeah.Corey: I spent too many years on call, in that sense. Everyone who's built a product company the first time always says the second time, the engineering? Meh, there are ways to solve that. Solving the distribution problem. That's the thing I want to focus on next.And I feel like I sort of went into this backwards in that I don't really have a product to sell people but I somehow built an audience. And to be honest, it's partly why. It's because I didn't know what I was going to be doing after 18 months and I knew that whatever it was going to be, I needed an audience to tell about it, so may as well start the work of building the audience now. So, I have to imagine if nothing else, your book has been a tremendous source of building a community. When I mentioned the word cloudonaut to people who have been learning AWS, more often than not, they know who you are.Andreas: Yeah.Corey: Although I admit they sometimes get you confused with your brother.Andreas: [laugh]. Yes, that's not too hard. Yeah, yeah, cloudonaut is definitely—this was always our, also a side project of we was just writing about things that we learned about AWS. Whenever we, I don't know, for example, looked into a new series, we wrote a blog post about that. Later, we did start a podcast and YouTube videos during the pandemic, of course, as everyone did. And so, I think this was always fun stuff to do. And we like sharing what we learn and getting into discussion with the community, so this is what we still do and enjoy as well, of course. Yeah.Corey: I really want to thank you for taking the time to catch up and see what you've been up to these last few years with a labor of love and the pivot to a product company. If people want to learn more, where's the best place for them to find you?Andreas: So definitely, the best place to find me is cloudonaut.io. So, this basically points you to all [laugh] what I do. Yeah, that's basically the one domain and URL that you need to know.Corey: Excellent. And we will put that in the show notes, of course. Thank you so much for taking the time to speak with me today. I really appreciate it.Andreas: Yeah, it was a pleasure to be back here. I'm big fan of podcasts and also of Screaming in the Cloud, of course, so it was a pleasure to be here again.Corey: [laugh]. You are always welcome. Andreas Wittig, co-author of Amazon Web Services in Action, now up to its third edition. And of course, the voice behind cloudonaut. I'm Cloud Economist Corey Quinn, and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice along with an angry, insulting comment that I will at one point be able to get to just as soon as I find something to deal with your sarcasm on the AWS Marketplace.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.

Watch on YouTube About the show Sponsored by us! Support our work through: Our courses at Talk Python Training Test & Code Podcast Patreon Supporters Connect with the hosts Michael: @mkennedy@fosstodon.org Brian: @brianokken@fosstodon.org Show: @pythonbytes@fosstodon.org Join us on YouTube at pythonbytes.fm/live to be part of the audience. Usually Tuesdays at 11am PT. Older video versions available there too. Michael #1: Pydantic v2 released Pydantic V2 is compatible with Python 3.7 and above. There is a migration guide. Check out the bump-pydantic tool to auto upgrade your classes Brian #2: Two Ways to Turbo-Charge tox Hynek Not just tox run-parallel or tox -p or tox --``parallel , but you should know about that also. The 2 ways Build one wheel instead of N sdists Run pytest in parallel tox builds source distributions, sdists, for each environment before running tests. that's not really what we want, especially if we have a test matrix. It'd be better to build a wheel once, and use that for all the environments. Add this to your tox.ini and now we get one wheel build [testenv] package = wheel wheel_build_env = .pkg It will save time. And a lot if you have a lengthy build. Run pytest in parallel, instead of tox in parallel, with pytest -n auto Requires the pytest-xdist plugin. Can slow down tests if your tests are pretty fast anyway. If you're using hypothesis, you probably want to try this. There are some gotchas and workarounds (like getting coverage to work) in the article. Michael #3: Awesome Pydantic A curated list of awesome things related to Pydantic!

Stephanie went to her first WNBA game. Also: Bingo. Joël's new project has him trying to bring in multiple databases to back their ActiveRecord models. He's never done multi-database setups in Rails before, and he doesn't hate it. Stephanie shares bits from a discussion with former Bike Shed host Steph Viccari about learning goals. Four elements stood out: Adventure (try something new) Passion (topic) Profit (from recent learnings) Low-risk (applicable today) = APPL Stephanie and Joël discuss what motivates them, what they find interesting vs. what has immediate business value, and how they advocate for themselves in these situations. They ponder if these topics can bring long-term value and discuss the impact that learning Elm had on Joël's client work. Elm (https://elm-lang.org/) Practical Object-Oriented Design in Ruby (https://www.poodr.com/) Design Patterns in Ruby (http://designpatternsinruby.com/) Quarter Life (https://www.penguinrandomhouse.com/books/579928/quarterlife-by-satya-doyle-byock/) Working Iteratively (https://thoughtbot.com/blog/working-iteratively) Transcript: JOËL: Hello and welcome to another episode of The Bike Shed, a weekly podcast from your friends at thoughtbot about developing great software. I'm Joël Quenneville. STEPHANIE: And I'm Stephanie Minn. And together, we're here to share a bit of what we've learned along the way. JOËL: So, Stephanie, what's new in your world? STEPHANIE: All right, I have a new-new thing and an old-new thing to share with you today. So the new-new thing is that I went to my first WNBA game [laughs] last week, which is also my third professional sports game ever, probably. I am not a sports person. But a rather new friend of mine invited me to go with her because they are fans, and so I was like, yeah, sure. I'll try anything once. And I went, and I had a great time. It was very exciting. I mean, I know the basic rules of basketball, right? Get the ball in the hoop. But I was very surprised to see how fast-paced it was. And, you know, I was like, wow, like, this is so much fun. There's so much going on, like, the music, you know, the crowd. It was very energizing. And then my friend actually told me that that was a pretty slow game, [chuckles] relative to how they normally go. And I was like, oh, wow, like, if that was slow, then I can't wait for a real competitive [laughs] game next time. So that's my new-new thing. I had a good time. Will do it again. I'm just, like, a 15-minute bike ride from the stadium for our team in Chicago. It's called The Sky. That's our WNBA team. So yeah, I'm looking forward to being basketball Stephanie, I guess. [chuckles] JOËL: That's really cool. How does the speed compare to other sports you've gone to see? STEPHANIE: I think this is why I was interested because I've really only seen baseball, for which I know very little. And that, I think, is, like, a much slower-paced kind of sport. Yeah, I have some memories of going to, like, college football games, which also, like, quite slow. I just remember standing around for a while. [laughs] So I think basketball might be the thing for me, at least in terms of engaging my interest. JOËL: You want something that actually engages you with the sport the whole time. It's not just a social event themed around occasionally watching someone do something. STEPHANIE: Yes, exactly. I also enjoyed the half-time performances, you know, there was just, like, a local dance team. And I thought that was all just very fun. And, yes, I had a lot to, you know, just, like, point to and ask questions about because there was just so much going on, as opposed to sitting and waiting, at least that was my experience [laughs] at other kinds of sports games. As for the old-new thing, now that it's summer, there is a local bar near me that does bingo every week. But it's not just normal bingo. It's called veggie bingo, which I realize is kind of confusing [chuckles] if you just, like, call it veggie bingo, but it's bingo where you win vegetables or, like, produce from local community gardens and other, you know, small batch food items. And I had a great time doing it last year. I met some new friends. It just became our weekly hangout. And so I'm looking forward to doing that again. And, I don't know, I'm just glad I have fun things to share about what's new in my world now that the weather is warm and I'm doing stuff again. I feel like there was one point in the winter where I was coming [chuckles] onto the show and sharing how I had just gotten a heated blanket in the middle of winter, and that was the most exciting thing going on for me. So it feels good to be able to bring up some new stuff. JOËL: Seasonality is a thing, right? And, you know, there are rhythms in life. And sometimes things are more fast-paced, sometimes they're a bit slower. That's really exciting. Did you take any produce home, or did you win anything when you went to play? STEPHANIE: I did. I won a big bag of produce the last time that I went. At this point, it was last season. But it was right before I was about to go on vacation. So I ended up -- JOËL: Oh no. STEPHANIE: [chuckles] Right. I ended up not being able to, you know, keep it in the fridge and just giving it away to my friends who did not win. So I think it was a good situation overall. That's my tip, is go to bingo or any kind of prize-winning hang out as a group, and then you can share the rewards. It's very exciting. Even if you don't win, you know, like, probably someone else at your table will win, and that is equally fun. JOËL: I think the closest I've been to that experience is going to play, like, bar trivia with some friends and then winning a gift card that covers our dinner and drinks for the evening. STEPHANIE: Yeah, yeah, that's great. I used to go to a local trivia around me too. The best part about bingo, though, is that it requires no skill at all. [laughs] I, yeah, didn't realize, again, how into these kinds of things I would be until I just tried it out. Like, that was...bingo is another thing I don't think I would have internally decided to go do. But yeah, my friends just have all these great ideas about fun things to do, and I will happily join them. So, Joël, what's new in your world? JOËL: So I've recently started a new client project. And one of the really interesting things that I've been doing on this project is trying to bring in multiple databases to back our ActiveRecord models. This is a Rails app. I've never done multi-database setups in Rails before. It's been a feature since Rails 6, but this is my first time interacting with that system. And, you know, it's actually pretty nice. STEPHANIE: Really? It ended up being pretty straightforward or pretty easy to set up? JOËL: Yeah. There's a little bit of futzing around you have to do with the database YAML configuration file. And then what you end up doing is setting up another base class for your ActiveRecord models to inherit from. So, typically, you have that application record that you would inherit from for your primary database. But for other databases, if you want a model to be backed by a table from that system, then you would have a separate base class that all of those models inherit from, and that's pretty much it. Everything else just works. A bunch of your Rake tasks get a little bit different. So you've got to, like, configure your setup scripts and your test scripts and all that thing a little bit differently. But yeah, you can just query, do all the normal things you do with an ActiveRecord model, but it's reading from a different database. STEPHANIE: That's really cool that it ended up being pretty painless. And I'm thinking, for the most part, as a developer, you know, working in that kind of codebase; maybe they don't really need to know too much about the details of the other databases. And they can just rely on the typical Rails conventions and things they know how to do via Rails. Do you suspect that there might be some future where that might become a gotcha or something that someone has to debug a little further because of the multi-database setup? JOËL: There are some infrastructure things, but I think I'm handling all of them upfront. So like I said, configuring various setup scripts, or test scripts, or CI, that kind of thing to make sure that they all work. Once that's all done, I think it should pretty much just work. And people can use them like they would normal ActiveRecord models. The one gotcha is that you can't join models across two different databases. You can't use ActiveRecord to write a query that would try to join two tables that are in different databases because the SQL won't allow for that. So, if you're ever trying to do something like that or you have some kind of association where you're trying to do some special join, that would not work. So, if somebody attempts that, they might get an unexpected error. Other than that, I think it just keeps working as normal, and people can treat it more or less as if it's one database. STEPHANIE: That's interesting. How do you model relationships between tables on the two different databases, then? Like, how would that work? JOËL: I've not gotten that far yet. For some things, I imagine just it's two queries. I'm not sure if the ActiveRecord associations handle that automatically for you. I think they probably will. So you probably can get away with an association where one model lives in one database. Let's say your article lives in one database, and it has many comments that live in a different database. Because then you would make one query to load the article, get the article ID, and then you would do another query to the second database and say, hey, find all the comments with this article ID, which is already, I think, what ActiveRecord does in one single database. It is making two queries. It's just that now those two queries are going to be two different databases rather than to a single one. STEPHANIE: Interesting. Okay. I did think that maybe ActiveRecord did some fancy join thing under the hood. And when you mentioned that that wouldn't be possible when the two tables are on different databases, I was kind of curious about how that would work. But that makes sense. That would be really cool if it is, you know, that straightforward. And, hopefully, it just doesn't become too big of an issue that comes back to haunt someone later. JOËL: Right. So pretty much, if there is a situation where you were relying on a JOIN, you will now have to make two separate queries and then combine the results yourself. STEPHANIE: Right. I also want to give you kudos for doing all the good work of setting it up so that, hopefully, future developers don't have to think about it. JOËL: Kudos to the Rails team as well. It's nice to have that just kind of built into the framework. Again, it's not something I've needed in, you know, a decade of doing Rails, but then, you know, now that I have run into a situation where I need that, it just works out of the box. So that's been really nice. So, a couple of weeks ago, we talked about the fact that we were going through review season and that we had to fill out reviews for ourselves then also fill out peer reviews for each other. You had brought up a really interesting conversation you had about reaching out to other people and trying to get feedback on what kind of review or feedback would be helpful for them. STEPHANIE: I did, yeah. Though, I think in this case, the person writing that feedback actually reached out to me, but certainly, it goes both ways. Spoiler alert - that person was Steph Viccari, former [laughs] host of The Bike Shed. JOËL: So Steph also reached out to me with similar questions. And that spawned a really interesting conversation around personal goals and what it looks like, particularly when it comes to what to learn next in technology. We started discussing things, and I listed out some different things that I was interested in. And then just kind of out of nowhere, Steph just pulls out this, like, oh, I noticed these four elements. And I'm going to list them out here because it's really cool. So these four elements were adventure, so trying something new. Passion, so something that's really exciting to you. Profit something where you can leverage some recent things that you've done to get more value out of some work you've already done. And then finally, low risk, something that would be applicable today. And it just kind of turns out that this makes a funny little acronym: APPL. And apples are often a symbol of learning. So that was kind of a fun coincidence. STEPHANIE: I love when someone is able to just pull apart or to tease out pieces of, you know, something that you might have just, like, kind of dumped all of into a message or something, and then to get, like, a second eye to really pick out the themes is so valuable, I think. And I'm obsessed with this framework. I think we might have come across something new that could really be helpful for a lot of other people. JOËL: It's definitely...I think it shows capacity for a higher level of thinking when someone's able to just look at a bunch of concrete things and say, wait a minute; I'm seeing some larger themes emerge from what you're talking about. And I always really appreciate it when I'm having a conversation with someone, and they're like, "Hey, I think what I'm hearing is this." And you're like, "Whoa, you're totally right. And I did not even know that that's where I was going." STEPHANIE: Absolutely. I'd love to go through this acronym and talk about a few different things that we've learned in our careers that kind of correspond with each of these elements. JOËL: Yeah, that sounds great. So I think, you know, the first one here is adventure, trying something new. So, what's something where you tried something new or adventurous that you think was worthwhile? STEPHANIE: Hosting this podcast. [laughs] It was a huge adventure for me and a really big stretch, I think. And that's what the idea of adventure evokes for me is, like, maybe it's uncharted territory for you, and you might have some reservations about it. But, you know, obviously, the flip side of an adventure is how fun and exciting and just new and stimulating it can be. And so I think, yeah, like, when I first started doing this with you, and even when you first asked me, I was pretty nervous. I was really hesitant. It took me a long time to, you know, think it over. I was like, do I want to commit to something that I have never done before, and it's, like, a pretty longer-term commitment? And I'm really glad I did it. It's certainly been an adventure. It's, you know, got its ups and downs. You know, not every week do I feel like that went really well, like, that was a great episode. Sometimes I'm like, that was just an okay episode, [laughs] and, you know, that's fine too. But I feel like this was really important in helping me feel more confident in sharing my technical opinions, helping me feel more comfortable just kind of, like, sharing where I am and not feeling like I should be somewhere else, like, some other level or have already known something. Like, the point is for us to share the journey week by week, and that was something that was really hard for me. So being on this Bike Shed adventure with you has been very valuable for me. JOËL: Yeah, it's sharing these new things we've learned along the way. STEPHANIE: Literally. Yes. What about you? Do you have something adventurous that you learned? JOËL: I think an important inflection point where I tried something new was when I learned the Elm programming language. So I had mostly done procedural languages back in the day. And then I got into Ruby, did a lot of OO. And then I got into Elm, which is statically-typed, purely functional, all these things that are kind of opposite of Ruby in some ways. But I think it shares with Ruby that same focus on developer happiness and developer productivity. So, in some ways, I felt really at home. But I had to learn just a whole new way of programming. And, one, it's cool. I have a new tool in my belt. And I think it's been a couple of years just learning how to use this language and how to be effective with it. But then afterwards, I spent a couple of years just kind of synthesizing the lessons learned there and trying to see, are there broader principles at play here? Are there ideas here that I can bring back to my work in Ruby? And then maybe even are there some ideas here that intersect with some theories and things that I know from Ruby? So maybe some ways of structuring data or structuring code from functional programming where some best practices there kind of converge on similar ideas as maybe some object-oriented best practices, or maybe some ideas from test-driven development converge on similar ideas from functional programming. And I think that's where, all of a sudden, I was unlocking all these new insights that made me a better Ruby developer because I'd gone on an adventure and done something completely out of left field. STEPHANIE: Yeah, absolutely. Do you remember what was hard about that when you first embarked on learning Elm? JOËL: All the things you're used to doing, you just can't do. So you don't have looping constructs in Elm. The only thing you can do is recursion, which, you know, it's been a long time since CS classes. And you don't typically write recursion in Ruby. So I had to learn a whole new thing. And then it turns out that most people don't write recursion. There's all these other ways of doing things that you have to learn. You have to learn to do folds or to use maps and things like that. Yeah, you're just like, oh, how do I do X in Elm? And you have to figure it out. And then maybe sometimes it turns out you're asking the wrong question. So it's like, oh, how do I do the equivalent of a for loop with array indexes in Elm to, like, iterate through a data structure? And it's like, well, kind of here's technically the way you could do that, but you would never solve a problem in that way. You've got to learn a new way of thinking, a new way of approaching problems. And I think it was that underlying new paradigm that was really difficult to get. But once I did get it, now that I have two paradigms, I think it made me a much more solid developer. STEPHANIE: Right. That sounds very humbling, too, to kind of have to invert what you thought you knew and just be in that, you know, beginner's mindset, which we've talked about a little bit before. JOËL: I think in some ways now being on the other side of it, it's similar to the insights you get from speaking multiple human languages, so being bilingual or trilingual or something like that where instead of just having assumptions about, oh, this is just how language works, because that's how your personal language works, now that you have more than one example to draw on, you can be like, oh, well, here's how languages tend to do things differently. Here's how languages are similar. And I think it gives you a much better and richer feeling for how languages work and how communication works. And similar to having multiple paradigms in programming, I think this has given me a much richer foundation now for exploring and building programs. STEPHANIE: That's really cool. I guess that actually leads quite well into the next element, which is passion. Because once you've tried some new things, you get the information of do I like this thing, or do I not like this thing? And then from there, you know, you gravitate towards the things you are passionate about to get a deeper understanding. And it becomes less about like, oh, just testing out the waters and like, knowing, hey, like, I constantly find myself thinking about this, like, let me keep going. JOËL: Yeah. Or sometimes, it's deciding what do I want to learn next? And you just pick something that's interesting to you without necessarily being like, oh, strategically, I think this is another paradigm that's going to expand my mind. Or this is going to make me, you know, help me get that promotion next quarter, purely based off of interest. Like, this sounds fun. STEPHANIE: That's really interesting because I think I actually came to it from a different angle, where one thing that I think was very helpful in my learning that came just, like, completely internally, like, no one told me to do this was reading books about design patterns. And that was something that I did a couple of years into my career because I was quite puzzled, I suppose, by my day-to-day experience in terms of wanting to solve a problem or develop a feature but not having a very good framework for steps to go about it, or not feeling very confident that I had a good strategy for doing it. It was very, for me, it felt very just kind of, like, throwing pasta to the wall and seeing what would stick. And I was really interested in reducing that pain, basically. And so that led me to read books. And, again, that was not something, like, someone was like, hey, I really think that you could benefit from this. It was just like, well, I want to improve my own experience. And, you know, some of the ones that I remember reading (And this was based off of recommendations from others kind of when I floated the idea.) was, you know, Sandi Metz's Practical Object-Oriented Design in Ruby. Design Patterns in Ruby by Ross Olsen. Those were just, like, purely out of interest. Yeah, I guess I'm curious, for you, what fun and passion look like. JOËL: Yeah, I think one thing that's a really fun side effect of passion learning is that I find that I tend to learn a lot faster and go a lot deeper, or I get more for every individual hour I put into learning just because passion or interest is such a multiplier. Similar to you, I think I went through a time where I was just gobbling up everything I could see on design patterns, and code structure, things like that. Yeah, I've always been really excited about data modeling in general and how to structure programs to make them easy to change while also not putting a high maintenance burden on it, learning those trade-offs, learning those principles, learning a lot of those ideas. I think that desire came out of some pain I felt pretty early on in my programming career, where I was just writing purely self-taught at this point from a few tutorials online. Code beyond a few hundred lines would just kind of implode under the weight of its own complexity. And so, like, I know that professional programmers are writing massively larger programs that are totally fine. So what am I missing? And so I think that sort of spurred an interest. And I've kind of been chasing that ever since. Even though I'm at the point where that is no longer a problem in my daily life, it is still an interest that I keep. STEPHANIE: Yeah. If I were to pull out another interest of yours that I've noticed that kind of seems in the same realm of, you know, you can just chase this forever, is working incrementally, right? And just all the ways that you can incorporate that into your day-to-day. And I know that's something we've talked about a lot. But I think that's really cool because, yeah, it just comes from just a pure desire on your own front to, like, see how far you can take it. JOËL: I think you pulled out something interesting there. Because sometimes, you have an interest in a whole new topic, and sometimes the interest is more about taking something I already know and just seeing can I take it to an extreme? What happens when I really go to the boundaries of this idea? And maybe I don't need to go there ever for a client project. But let me put up a proof of concept somewhere and try it out just for the fun of it to see can I take this idea, then push it to an extreme and see does it break at an extreme? Does it behave weirdly? And that is just an enriching journey in and of itself. Have you ever done, like, a...maybe not a whole learning journey but, you know, taken a few hours, or maybe even, like, some time on one of our investment Fridays to just explore some random idea and try it out? And it's like, huh, that was cool; that was a journey. And then maybe you move on to something next week because it's not like a big planned thing. But you're taking a few hours to dig into something totally random. STEPHANIE: I actually think I'm less inclined to do that than maybe you or other folks are. I find the things I choose to spend my time on do have to feel more relevant to me in the moment or at least in my day-to-day work. And I think that actually is another excellent transition into the last couple of elements in the APPL framework that we've now coined. The next being profit or, I guess, the idea of being valuable to you in your job in that moment, I suppose. Or I guess not even in that moment, but kind of connecting what you're learning to something that would provide you value. So I know you were talking about learning Elm, and now you're able to see all of the value that it has provided, but maybe at the time, that was a little bit less of your focus. But for me, I find that, like, a driver for how I choose to spend my time. Often it's because, yeah, for the goal of reducing pain. Being consultants, we work on a lot of different projects, sometimes in different frameworks, or languages, or new technologies. Like, you've mentioned having to, just now, on your new client project learning how to interact with different databases, and it sounds like older software that you might not have encountered before. And I think that ends up falling higher on my priority list depending on the timing of what I'm currently working on is, oh, like, you know, TypeScript is something that has, like, kind of come and go as my projects have shifted. And so when it comes back to working on something using it, I'm like, oh, like, I really want to focus on this right now because it has very clear value to me in the next three to six months, or however long. But I have also noticed that once I'm off of that project, that priority definitely recedes. JOËL: Yeah, I think that plays into that final element as well of the APPL, the low risk things that are applicable today that have value right now. Those tend to be things like, oh, I see that I'm going to be scheduled on a client that needs this technology next month. Maybe I should learn that, or maybe I should refresh this idea or go a little bit deeper because this is something new that I'm going to need. So, at some point, I knew that there was a Python project coming down the line. I was like, okay, well, maybe I'm going to spend a couple of Fridays digging into some Django tutorials and compare and contrast with Rails. STEPHANIE: The low-risk element is interesting to me because I found it to be a challenging balance to figure out how much time to invest in becoming really comfortable in a new technology. I find myself sometimes learning just enough to get what I need to get done. And then other times really feeling like, wow, like, I wish I knew this better because that would make my life easier, or I would just feel a lot better about what I'm doing. And kind of struggling with when to spend that time, especially when there's, you know, other expectations of me in terms of my delivery. JOËL: Yeah, that almost sounds like a contrast between technologies that fall in that low-risk bucket, like, immediately useful, versus ones that fall in the passion bucket that you're interested in taking deeply and maybe even to an extreme. STEPHANIE: That's really interesting. What I like about this list of themes that we've pulled out is that, like, one thing can fall into a number of different categories. And so it's really quite flexible. It actually reminds me of a book that I just finished reading. The book is called Quarterlife. And the thing that stuck out to me the most is the author, who is a psychotherapist; she has basically come up with two types of people, or at least two things, that end up being really big drivers of, like, human motivation and behavior. And that's stability types and meaning types, and the goal is to have a little bit of both. So you may be more inclined towards stability and wanting to learn the things that you need to know for your job, to do well in your role, kind of like we were talking about to reduce that pain, to feel a little more in control, or have a little more autonomy over your day to day and how you work. And then there's the seeking meaning, and when we talked about adventure and passion, it kind of reminded me of that. Like, those are things that we do because we want to feel something or understand something or because it's fun. And ironically, this list of four things has two that kind of fall into each category. And ultimately, the author, she, you know, was very upfront about needing both in our lives. And I thought that was a really cool distinction. And it was helpful for me to understand, like, oh yeah, like, in the early years of my career, I did really focus on learning things that would be profitable, or valuable, or low risk because those were the things that I needed in my job, like, right now. And I am now feeling stable enough to explore the meaningful aspects and feel excited by trying out things that I think I just wasn't ready for back in the day. But it actually sounds like you may kind of have a different leaning than I do. JOËL: That is really interesting. I think what was really fascinating as you mentioned those two sort of types of people. And, in my mind, now I'm immediately seeing some kind of two-dimensional graph, and now we've got four quadrants. And so are we leaning towards stability versus...was it adventure was the other one? Or meaning. STEPHANIE: Meaning, yes. JOËL: So now you've got, like, your quadrant that is high stability, high meaning, low stability, high meaning, like, all those four quadrants. And maybe these four things happen to fall into that, or maybe there's some other slightly different set of qualities that you could build a quadrant for here. One that is interesting, and I don't know how closely it intersects with this idea of stability versus meaning, is how quickly the things you learn become useful. So that low risk, like that L from APPL, those are things that are immediately useful. So you put a little bit of work learning this, and you can immediately use it on the job. In fact, that's probably why you're learning it. Whereas me going off and learning Elm is not because we've got any clients in the pipeline using Elm. It's purely for interest. Is it going to pay off? I think most learning pays off long-term, especially if it helps you build a richer understanding of the different ways software works or helps you have new mental models, new tools for doing things. And so I think, you know, 5, 6, 7 years later, learning Elm has been one of the highest payoff things that I've done to kind of take my coding career to the next level. That being said, I would not have seen that at the time. So the payoff is much more long-term. How do you kind of navigate when you're trying to learn something, whether you want something with a short-term payoff or a longer-term payoff? STEPHANIE: Yeah, that's so interesting. I wonder if there was maybe someone who could have helped you identify the ways that Elm could have possibly paid off. And I know, you know, you're looking back on it in retrospect, and it's easy to see, especially after many years and a lot of deep thinking about it. But kind of referring back to this idea of seeking meaning and that just being important to feeling happy at your job, like, maybe it was just valuable because you needed to scratch that itch and to experience something that would be interesting or stimulating in that way to prevent burning out or something like that. JOËL: Oh, I like that. So the idea that you're learning a thing, not specifically because you're expecting some payoff in the long term but because of the joy of learning, is reward in and of itself, and how that actually keeps you fresh in the moment to keep going on a career that might, you know, last 5, 10, 20, 30 years, and how that keeps you refreshed rather than like, oh, but, like, I'm going to see a payoff in five years where now, all of a sudden, I'm faced with a problem. And I can be like, ah, yes, of course, monads are what we need here. And that's a nice side effect, but maybe not the main thing you look for when you're going for something in that passion bucket. STEPHANIE: Yeah, absolutely. To go back to your question a little bit, I had mentioned that I was wondering if there was someone who could help point out ways that your interests might be useful. And I think that would be a strategy that I would try if I find myself in that conundrum, I suppose, of, like, being like, hey, like, this is really interesting to me. I'm not able to see any super immediate benefits, but maybe I can go find an expert in this who can share with me, like, from their experience, what diving deep into that topic helped them. And if that's something that I need to then kind of justify to a manager or just kind of explain, like, hey, this is why I'm spending my time doing this is because of this insight that I got from someone else. That would be, I think, a really great strategy if you find yourself needing to kind of explain your reasoning. But yeah, I also think it's, like, incredibly important to just have passion and joy in your work. And that should be a priority, right? Even if it's not immediately clear, the tangible or valuable to the company benefits in the current moment. JOËL: And I think what I'm hearing is that maybe it's a bit of a false premise to say there are some things that you follow for passion that only pay off in the long term. Because if you are in it for passion, then you're getting an immediate payoff regardless. You may also get an additional payoff in the long term. But you're absolutely getting some kind of payoff immediately as well. STEPHANIE: Yeah, I think that's true for adventure because knowing what you don't like is also really valuable information. So, if you try something and it ends up not panning out for you, you know, I think some people might feel a little bit disappointed or discouraged. They think, oh, like, they kind of wasted time. But I don't know; I think that's all part of the discovery process. And that brings you closer and closer to, yeah, knowing what you want out of your learning and your career. JOËL: So I'm really curious now. This whole, you know, APPL framework came out of a very random conversation. Is this something that maybe you're going to take into your own sort of goal-setting moving forward? Maybe try to identify, like, okay, what is something adventurous that I want to do, something I want to do for passion, something that I think for profit, and then something low risk? And then maybe have that inform where you put some energy in the next quarter, the next year, whatever timeline you're planning for. STEPHANIE: Yeah, I thought about this a little bit before we started recording. But one very loose goal of mine...and this actually, I think, came up a little more tangibly after coming back from RubyKaigi and being so inspired by all of the cool open-source tooling and hearing how meaningful it was for people to be working on something that they knew would have an impact on a lot of people in their development experience. Having an impact is something that I feel very passionate about and very interested in. And the adventure part for me might be, like, dabbling a little bit into open-source tooling and seeing if there might be a project that I would be interested or comfortable in dipping my feet into. What about you? Do you have anything in the near or long-term future that might fall into one of these buckets? JOËL: So I do have a list of things. I don't know that I will pursue all of them or maybe any of them. But here's my kind of rough APPL here. So something adventurous, something new would be digging into the language Rust. Again, the idea is to try a completely new paradigm, something low-level, something typed, something that deals with a lot of memory, something that does well with concurrency and parallelism. These are all things that I've not explored quite as much. So this would be covering new ground. Something that is a passion, something that's interesting to me, would be formal methods, so I'm thinking maybe a language like TLA+ or Alloy. Data modeling, in general, is something that really excites me. These techniques that I think build on some of the ideas that I have from types but that go, like, to an extreme and also in a slightly different direction are really intriguing to me. So, if there's something that maybe I'm staying up in the evenings to do, I think that might be the most intriguing thing for me right now. Something that might be more profitable, I think, would be digging into the world of data science, particularly looking at Notebooks as a technology. Right now, when I need to crunch data, I'm mostly just doing spreadsheets. But I think there are some really cool things that we could do with Notebooks that come up in client work. I manage to do them when you're with a random command-line script or sometimes with Excel. But I think having that tool would probably be something that allows me to do that job better. And then, finally, something low-risk that I know we could use on a client project would be digging in more into TypeScript. I know just enough to be dangerous, but we do TypeScript all the time. And so, mastering TypeScript would definitely be something that would pay off on a client project. STEPHANIE: I love that list. Thank you for sharing. JOËL: Also, I just want to note that there are only four things here. It doesn't fully spell APPL because there's no E at the end. And so when I see the acronym now, I think it looks like a stock ticker. STEPHANIE: It really does. But I think it's pretty trendy to have an acronym that's basically a word or a noun but then missing a vowel so... JOËL: Oh, absolutely. Time to register that applframework.com domain. STEPHANIE: Yeah, I agree. I also love what you said. You called it a rough APPL. And that was very [laughs] evocative for me as well. And just thinking about an apple that someone has, like, bitten into a little bit [laughs] and has some rough edges. But yeah, I hope that people, you know, maybe find some insight into the kinds of learnings and goals that they are interested in or are thinking about. And using these themes to communicate it to other people, I think, is really important, or even to yourself. Maybe yourself first and then to others because that's how your co-workers can know how to support you. JOËL: That's really interesting that you are thinking of it in terms of a tool for communication to others. I think when I first encountered this idea, it was more as a tool of self-discovery, trying to better understand why I was interested in different things and maybe better understanding how I want to divide up the energy that I have or the time that I have into different topics. But I can definitely see how that would be useful for communicating with team members as well. STEPHANIE: On that note, shall we wrap up? JOËL: Let's wrap up. STEPHANIE: Show notes for this episode can be found at bikeshed.fm. JOËL: This show has been produced and edited by Mandy Moore. STEPHANIE: If you enjoyed listening, one really easy way to support the show is to leave us a quick rating or even a review in iTunes. It really helps other folks find the show. JOËL: If you have any feedback for this or any of our other episodes, you can reach us @_bikeshed, or you can reach me @joelquen on Twitter. STEPHANIE: Or reach both of us at hosts@bikeshed.fm via email. JOËL: Thanks so much for listening to The Bike Shed, and we'll see you next week. ALL: Byeeeeeee!!!!!!! ANNOUNCER: This podcast is brought to you by thoughtbot, your expert strategy, design, development, and product management partner. We bring digital products from idea to success and teach you how because we care. Learn more at thoughtbot.com.

Joël has a bike shorts update; Stephanie has a garden one. Often, power is centralized within the dev team. This is usually because they are the only ones able to execute. Sometimes this ends up interfering with team processes and workload. Joël is a fan of empowering other teams to do things themselves. Strangler Fig Pattern (https://shopify.engineering/refactoring-legacy-code-strangler-fig-pattern) What Being a Staff Developer Means at Shopify by Rose Wiegley (https://shopify.engineering/what-being-a-staff-developer-means-at-shopify) End-User Programming (https://www.inkandswitch.com/end-user-programming/) Transcript: STEPHANIE: Hello and welcome to another episode of The Bike Shed, a weekly podcast from your friends at thoughtbot about developing great software. I'm Stephanie Minn. JOËL: And I'm Joël Quenneville. And together, we're here to share a bit of what we've learned along the way. STEPHANIE: So, Joël, what's new in your world? JOËL: So, in a recent episode, I had mentioned that I was going to go on vacation on a bike trip and that I had purchased a pair of bike shorts to try out on that trip to see if that would help. And, wow, that was a great purchase. It literally saved my butt. STEPHANIE: That's awesome. I'm really glad that they worked out for you. JOËL: Still sore. This was a five-day biking trip. And I think day two was the worst, but after that, things got better. But the shorts definitely helped. STEPHANIE: I think my favorite part about us talking about biking and bike shorts is that we're finally living up to the name of our podcast. [laughs] Turns out that bikeshedding is actually even more, bikeshedding when it's about actual bike stuff. And a listener named James even wrote in with some pro tips about, you know, how to care for your bike shorts and, you know, have a comfortable biking experience. And gave some good tips for me on some longer rides to check out near me in Chicago. JOËL: So it sounds like there's some crossover between the software developer community and bike enthusiasts community who also tune into this podcast. STEPHANIE: I do think that we have gotten tweets before from, I think, like, the motorcycle Twitter tagging us @_bikeshed, perhaps maybe trying to tag a different account but, yeah, ended up in our Twitter inbox instead. JOËL: Now we just need some sweet, sweet bicycle sponsorships. So, Stephanie, what's been new in your world? STEPHANIE: I have a garden update. Last year, we purchased a small fig tree from the internet. It turns out that you can get little fruit trees delivered to your door. And this was, I think, around the fall, so it was getting a little cooler. And here in Chicago, we have to bring some of our plants inside to overwinter. And so we brought the fig indoors, and it's maybe, like, two or three feet tall. And, you know, over the few months, we were just, like, caring for it. And I was really excited to see that it had started fruiting several months ago. And I got to show it to all of my co-workers in a call. I, like, picked up this kind of large pot with our little fig tree, and I, like, held it really close to my camera and tried to point out the fruit to the other people on the call, which I realized was perhaps not a very effective way to show off my plants. Like, you could just take a picture and send it in Slack. And I was like, yes, I could have done that. But yesterday (Now our fig tree has been outside for a little while since it's warmer.) I noticed that they were ripe, and I got to harvest our figs and eat them, and they were delicious. And I got to update the team on my little fig adventure. And this time I did take pictures of the fruits and sent them in Slack instead of trying to bring this tree in from the outdoors. JOËL: That's exciting. Because I'm a fan of the design pattern, I have to ask, is this a strangler fig? STEPHANIE: It's not a strangler fig, though I have seen one in the wild on a trip to Florida. I saw a really big strangler fig, you know, completely, like, enveloping another tree, and that was really cool. If you ever get to see one in person, I think it's just, I don't know, just really amazing how nature works. JOËL: I did not realize they were wild in Florida, something to keep an eye out for next time I'm there. STEPHANIE: Definitely. So, in a recent retro on my client team, we were discussing the one-off requests our team has been getting from the folks over on the sales and client support side. Oftentimes, this involved running a script in our production console to fix some issue that a customer was experiencing. And we were talking about what we could do to make this process a little more automated, make it a little less time-consuming on our end. Even though it would just take a few minutes to run this script, we were seeing that we were getting this request repeatedly. I'm curious if you've kind of been in this situation before where dev work is required and kind of eating into time that we are trying to be delivering on other feature work for similar one-off requests or to support other folks at the company. JOËL: Yeah. I think it's a pretty common pattern that I've seen. And I think sometimes it can actually start from a healthy place. If you're taking very much of an MVP philosophy and you're building a small version of your product to start with, you're not going to have a whole suite of admin tools available. You might not even have any admin people. It might just be a founder and a technical co-founder. And so, for the first few hundred customers you have, maybe the way you make changes is by loading up the Rails production console and making a change. And that's good enough, but that doesn't scale. STEPHANIE: Yeah. You bring up a good point that I think one thing that we get to experience as consultants is seeing many different companies at different stages in their business. And I think I've seen this issue in many different iterations based on the size of the company, right? So you were saying for an MVP product, there's no admin support at all. Maybe you have a project that is now thinking of how to introduce a little bit of admin tooling and might reach for something lightweight like a gem. I've also seen custom admin dashboards, and that being its own namespace and having all of that feature set hand-rolled, and then maybe some other company might opt for a Software as a Service solution. JOËL: Yeah, there's a lot of different implementations that happen at various stages of companies. I think one thing that does seem to stay pretty constant, though, is oftentimes; other teams don't have the tools they need to make the changes they need to. So, if you have a customer service person and they're receiving a complaint or they're having to make a change, they're not always empowered to make the changes they need to. They need to talk to the dev team, who then need to make changes. And the dev team don't really want to spend their day doing admin work. They are incentivized to ship features. And so both sides are unhappy. And it kind of comes from a sort of fundamental, I think, over-empowering of the dev team and kind of a disempowering of some of the other departments within the company, if that makes sense. STEPHANIE: Yeah, that's interesting because I don't think it necessarily is intentional, the way that that happens, right? It's not like you start building a product, and you are saying, "Okay, we only want to give devs the power to change all of this stuff at the production level." It's just something inherent, I suppose, to the work that we do. And there's a lot of active effort that needs to be taken to spread some of that empowerment around. JOËL: Yeah, generally, it is not some sort of, like, nefarious corporate politics that's happening where the CTO is, like, hoarding all the power, or it's a turf war or anything like that. Like you said, it's kind of an emergent property. As developers, we're often used to being sort of ultra-empowered to do what needs to be done. In general, development teams are highly respected within companies, and so people listen to them. But also, in order to do their job, they need to have access to a lot of things. So you often have production access to all the things and the admin credentials. And if there's something that doesn't work, you write code, and you can change the sort of fundamental underlying platform that you're working with. And so you're generally empowered to make the changes you need to make your life better or if you're blocked on something. And that's not necessarily true for other departments who are working in the system that we're building. STEPHANIE: Yeah, it's kind of interesting the duality that you have identified where we do have all of this power or capability to change the system. But you had mentioned earlier that sometimes it actually gets in the way of our work, that it can be a drag to do if we have other competing priorities, and that those mundane tasks end up being something that we also don't enjoy doing. And so, like you're saying, like, no one is quite happy. I wonder at what point you, as a developer, having repeatedly been asked to do these kinds of tasks for other departments when you, would start advocating for building tooling. JOËL: I don't think there's a kind of a clear dividing line, like, oh, after three requests, you must build a dashboard. It's probably more about just general communication with the other teams. I like to think of it from kind of two perspectives. From the perspective of the developers, how can we keep them efficiently working on what they need to prioritize, which is typically new feature development? And then, from the perspective of other teams, how can they be empowered to do the work that you need to do without getting blocked? Because just like the dev team doesn't like to get blocked on all sorts of things, other teams don't like it either. And so, how can we make sure that other team members within the company are empowered to do their work as efficiently as possible? STEPHANIE: Yeah, that's interesting. I think as an IC, I've been in different positions, depending on the context of my work. There have been times when I've been happy to help with that kind of request, right? Because I know that I'm unblocking someone else. I'm facilitating their work. And they usually appreciate it too. And so maybe if that's still the case and that there's not necessarily any pain that comes with that being just the process that it is from both sides, like, that's perfectly fine. But then it's totally fair for, you know, either party, once they do feel like it's blocking other work, to start looking into maybe how much time you're spending on these one-off requests, especially if it's being spread around to other team members. You know how much effort you're making, but, like, a manager might actually be more aware of how it's affecting multiple folks on the team and wanting to figure out, like, how that sits in with the other priorities that the team is working on. JOËL: Yeah, I'm glad you mentioned talking to other people because I might be quite happy to say, "Oh, I'm going to go and, you know, go into the database and make a small change." But just because it's easy for me to do and I can take, you know, 10 minutes out of my day to do it, doesn't mean that that experience is good for, let's say, a customer service person who had to get blocked or had to ask someone else to help to move this ticket forward. When if it was something they could do themselves, that would have been a much better experience. So, even though it's a very fairly, you know, cheap request and because I don't get them a lot, I'm happy to do them, it's maybe not a good experience for my customer service colleague. So, like you said, it is important to get people's experiences on all sides. STEPHANIE: One thing that I have seen a lot is for these things to start as configuration in a YAML file that requires developers to change and then commit to the codebase whenever, you know, maybe it's, like, a list of products or a list of prices, something that is, you know, really the business domain. And yet we are hard coding it and, like, codifying it into our source control. JOËL: Oof, yes. I have been in those projects, yeah. Now, every time you want to make a change, a business person has to reach out to the dev team, and then you have to make a code change, and then you have to deploy it. And that just becomes a whole thing. And then they come back to you the next day and say, "Oh, actually, we talked about it, and we want it a little bit differently." And you have to go through that process again. STEPHANIE: I think we reach for that just because we think it's faster maybe to set up, you know, some kind of, like, lightweight configuration file, rather than if you're working in Rails, you know, setting up a whole MVC for whatever thing you're trying to configure. And I'm curious if you think that's true or not. JOËL: I think it depends. Sometimes it can be because this data feels very static, kind of hard-coded. And so it's not a thing you would necessarily want to have. In a database, it's more like a constant that you would have in your source control, except that then you find out that your constant is not quite as constant as you thought it was. And I think maybe that's okay. Writing software is all about kind of discovering the problem in the domain as it evolves and trying to not over-engineer things ahead of time. So, if we have a small set of values, maybe they're U.S. states that you deliver to or a small list of products or something that you feel is relatively hard coded, maybe it starts as a constant array hard coded into Ruby, maybe it is a YAML file that you load. Then, over time, there comes a point where you decide this should be a database table, and if it needs to be sort of pre-seeded, then there's a mechanism for that with database seeds in Rails. STEPHANIE: Yeah, that's fair. I find it so interesting because most of the time, I've not seen that transition happen, right? It almost feels like some form of the bystander effect where everyone is just, well, I'm adding just one more thing. So I don't want to make this really big change now. JOËL: And that's true for everything in code, right? You say, "Oh, this deeply nested condition, yeah, it should probably be restructured. But I'm just going to add, you know, an eighth nested level in there. And, like, eight is probably the limit, but mine is going to be the eighth, so it's going to be good." And then somebody comes in and says, "Well, you know, nine is not that bad, but the next person probably should refactor it." And then it's a mess. STEPHANIE: Yeah, it's kind of like the entropy of code, I suppose, [laughs] where, you know, we had said it requires a lot of active energy and effort to make those changes to support other folks in different departments of the company. And I think that's, like, one very common area that we see things starting as configuration but then end up being something that you are needing to support in changing. And I wonder if maybe that is a signal in itself, right? If you are getting this information from another team, like, someone external to the development team, I wonder if that's kind of a clue that this is something that should be reconsidered about whether you start with it being hard-coded. JOËL: That's an interesting thought. There's a sense in which I think these always come from places external to the development team because it's a form of kind of product research when you're trying to understand what the features need to be, what this needs to happen. Unless this hard-coded data is purely structural or internal values, but it rarely is. There probably is a broader discussion to be had about the use of any sort of hard-coded data in a configuration file in a Rails app versus just always starting with a database table. One thing that's nice about always having a database table is that if you ever need to connect it to other data in your system, now you can do things like table joins, where you can't join your users on some kind of YAML array, or you have to do some sort of Ruby Enumerable logic. You can't just do it in SQL. STEPHANIE: Yeah. This is a bit of a tangent, I think. But that reminds me of when I worked at a product company where we had a very robust data warehouse, and all of that information was available to teams on the marketing side and on the data science side. And I actually really liked that because they were able to, you know, construct their own dashboards and queries to get the things that they need. And I've certainly seen what you're saying, this pretty important business information being hard-coded, and that ends up being less accessible, right? And less insightful, really. One other area of this topic that I think I've also bumped into before is specifically a QA engineer or, like, a QA team and empowering them to be able to do their jobs. Oftentimes, I've noticed that QA environments are not as well-maintained as maybe they should be, where the data that's seeded or, you know, kind of overtime in this environment is a little wonky. I've also experienced, while working on a feature, kind of having to go back and forth between whoever is helping QA my work telling them, like, "Oh, this isn't finished yet. So, like, don't worry about this that you're testing," or, you know, "Actually, that does look wrong. But let me look into it over on this end." And I found it sometimes difficult to navigate because I want them to be more empowered to test their feature without that uncertainty over whether something is intentional or actually broken. JOËL: In this case, do you think it's more about communication between development and QA, clear acceptance criteria, or clear descriptions of what changes have been pushed up for review and what's not in scope? Is that where you're headed? STEPHANIE: I think that's a part of it. But I actually think there are more technical considerations, especially in terms of whether our environments all align in terms of the data we're expecting, right? Does our dev environment kind of look like our QA environment, which looks like our production environment? Because I've certainly been in projects where that's, like, all over the place, and that really messes up the different expectations we have. We all know the "Oh, it worked in my local" [laughs] response to when things come up in other environments that are unexpected. And I wonder if there is more attention that we should be having towards making sure that just because this environment is not the main one that we're working in as developers, that people who are having to use it have an equally good user experience. JOËL: I like that you brought up the term user experience because oftentimes, as developers, and just, I think, product teams in general, we're trying to make something good for the customers of the application. But there are a lot of other people that have to interact with it in sort of more ancillary roles; like you mentioned, it might be QA. It might be customer support. It might be business development. And they're not the customer. And so because of that, they're often kind of a second thought or even sort of no thought at all. And so they do their jobs as best they can with what they've got, and sometimes get really creative getting around some of the hurdles that are in place. But we can often, with very little effort because the bar is so low, make these people's lives a lot better by applying just a little bit of the same approach that we would use to make software great for a customer to use for teammates in these other roles. STEPHANIE: Yeah, absolutely. Especially because we have that line of communication open with them, and, like you said, they are also our users of our applications. And especially for QA folks, too, in some ways, they're the first line of defense of our users, right? They are a resource for us to know if the customers will eventually have a poor experience or not. And I was thinking about that back-and-forth communication I mentioned with QA, you know, trying to explain, like, oh, this isn't finished yet, so maybe, like, you should not expect this to happen. But, oftentimes, that perhaps is a signal that we haven't thought about the way that we're developing our feature to be able to be released to customers in a more incremental way. Or we might be hand-waving over something that ends up being a bug later on. JOËL: Definitely. For myself, I see that as a... code smell is maybe not the right term here, but maybe acceptance criteria smell. If I'm trying to write out some acceptance criteria, and I'm having to say, like, "Oh, but, like, ignore this, and, like, pretend this doesn't exist. All of these, like, weird edge cases and exceptions we're trying to put in, are oftentimes a sign that maybe the work was not scoped correctly. STEPHANIE: I'm curious, in your workflow, will you just make those improvements if you have the opportunity to? Or do you end up bringing that to the team or creating a ticket for it? How does that fit in when you identify areas that could be improved? JOËL: I think it depends on the team's current workload. Oftentimes, if it's just something small, it's something I can just slip into my day, and it makes somebody else's life easier, that's great. Otherwise, it can be a thing that needs to be brought up with the team in general. And then it's the thing that we prioritize, and we put it in the backlog because, like you said, the main users of our app are customers. But all of these other teammates are also users of our app in other ways, and so they need certain features to get their job done. And so it's worthwhile to, I think, at a product planning level, take those into account and prioritize them with the customer-facing things. And sometimes, because those other teams don't have as much of a voice at the table, it's up to us as developers because we sometimes have that direct communication where we're talking to them and sort of going back and forth about, "Oh, can you change this in the database for me?" or "Can you do this?" And it can be up to us to champion these other teammates' needs with the team and with the product organization in general. STEPHANIE: Yeah, I really like the way you put that. I think you used the word champion. And I've seen this also go the other direction, where we add more processes in place, where the direct communication that needs to be gatekept a little bit through a manager because the manager is trying to protect the time of the team. And that is one way to handle the issue of these requests taking too much of the team's time. But I think at some point, as an IC, you also have the agency to champion or advocate for how you use your own development time. And that reminds me of something I heard from Rose Wiegley over at Shopify about what it means to be, like, a staff or a senior developer, and that is sharing that I'm going to do this, and this is why. And that means that I won't have time to do this other thing that I may be committed to earlier. But you know, these are my reasons. And if anyone sees any problems with that, let me know. And I've been thinking about that a lot in terms of figuring out how to do the kind of work that I value. And for you and me, that does sometimes mean building those tools to empower people who aren't developers. But that is, yeah, just a way that we can assert a little bit of that agency rather than having to get the buy-in to even consider setting time aside for that work. JOËL: Yeah, I think some of the really fulfilling work that I've done has been just sometimes taking a morning and, quote, unquote, "pairing" with, like, a business development or a customer service person and just saying, "Hey, can I just sit with you while you process this kind of request or problem that you're trying to do?" And then just really seeing what they do and all the steps and being able to ask a lot of questions and kind of putting my product developer hat on. And because then I know, internally, all the things that are happening, I can quickly see, oh, okay, you're- having to do these, like, five steps to get around this really annoying thing that's just, like, a rough corner that we have that I can, like, just easily smooth the way, you know, with a 10-minute one-line fix. I'm going to go and do that, and, you know, by the afternoon, that's already done, and that's saved them so much time or so much annoyance because it's not always time. Sometimes it's just annoyance. And their life is better. And I put very little effort into it. Most of it is just taking the time just to talk to each other and to try to understand each other. So I think we brought up the idea in the beginning of trying to empower other teams to not sort of centralize all the ability to execute on change within the development team. And sometimes, you can go to fairly extreme lengths to that. One that I've seen is the idea of end-user programming or end-user development, where the using the software rather than the team developing it has some sort of way where they can sort of customize or build on, or sometimes even script or -code -their experience. Is that something you've ever had to deal with or interact with on a project? STEPHANIE: Yeah, it's really interesting that you brought that up because I had mentioned going with a SaaS solution earlier as something that I've seen before. And that reminds me of when I worked on a client project where we were using Freshsales to integrate with the business domain of the client. And this would eventually be the main software that the sales team would use. And the reason that we went with Freshsales was because it allowed for a lot of that custom configuration and workflows that they could create for themselves for their needs. Though ironically, as we were kind of butting up into the limitations of Freshsales and how it didn't necessarily work for the way we were representing our data, we kind of joked that we almost wished we had to build the tool from scratch ourselves. So I think there are trade-offs there, you know, folks had done a lot of research to figure out the right SaaS solution for this project that we were doing. And yet, you know, inevitably, like, there were some cons with the third-party and how we were able to integrate with it. And it was actually also replacing something that had been built in-house and had become difficult to maintain or something that the company decided that they didn't want to maintain anymore. So I hate to say it, but I really think it [laughs] depends. JOËL: And now you're getting into the classic build versus buy dilemma for chunks of your software. STEPHANIE: Yeah, absolutely. JOËL: I think a way that I've seen, and this happens in a kind of a smaller sense, is providing escape hatches for data. And so maybe you've got a couple of small dashboards, or you've got just a lot of things that happen in your system. And you don't have the development time, or you don't want to prioritize that time right now to build something custom for maybe your business development team. But you provide certain reports that can be exported as CSV or Excel, which then the business development team will load into Excel and do the work that they need. And now they're empowered to do what they want instead of having to ask for more information or just being limited to what was on that web UI. Similarly, sometimes, when you're able to Import a CSV, I've seen this happen a lot, where in software that's not built just right for a customer service team, they'll often export a CSV of data, put it into Excel, manipulate it the way they really want it to be, and then re-import it into the system. And so that can be a great way to temporarily empower people. I think it's also a product smell. Oftentimes, there's a fundamental flaw of the way that your product works because your team is trying to go around it. It's so bad. But as a shorter-term solution, that can be great. STEPHANIE: That makes me think that Excel is the real end-user programming software. JOËL: It really is. It really is. I do really like the idea of end-user Programming, though. And rather than developers or even product people having to decide how our software should work for our users, shifting that to the masses and letting them have all of that empowerment and agency that we're talking about. There's a technology research lab called Ink & Switch, and they build a lot of really cool end-user programming tools. I think I've seen some, like, note-taking software, that they've done, and just other research into why it's important and how it can impact users. And I have read a little bit of their work, and I think it's really cool. So I'll be sure to add that to our show notes. JOËL: I think even as developers, we like some of these ideas of end-user development. We have that a lot in our tooling. But then, even when we interact with other people's software that we don't own...because we're used to interacting with our own software, we own it. We can change anything we want. We've got complete freedom. But the moment we interact with somebody else's software and, of course, it doesn't work 100% the way we need it to, it is sometimes nice to have sort of ways to hook into it so that we can get the things we want and then maybe do some extra manipulation on our own. And APIs are often how we do that. And so the equivalent of providing an API for another developer, well, what is that for our other teams? STEPHANIE: Yeah, great questions to consider. JOËL: You know, it could be a CSV export. It could be maybe there's some easy way to connect to a Zapier plugin. And now, you know, they don't need to ask the dev team, "Oh, we want to receive this notification email internally when this event happens. They can just connect to a Zapier plugin and have it send an email or do something in Salesforce or all these other things that are helpful in their workflows but that we've not taken the time to build into the core software. And now they're empowered to do their work and not blocked on us. STEPHANIE: That's interesting because as you were talking about that, it made me think of development tooling that we get to integrate with and how those APIs are usually very flexible. And let us decide what we need and ask the API for that as opposed to it dictating it for us. And, you know, if that's something that we get to enjoy, then, yeah, we should certainly think about how you can spread that to others. JOËL: I love that. On that note, shall we wrap up? STEPHANIE: Let's wrap up. Show notes for this episode can be found at bikeshed.fm. JOËL: This show has been produced and edited by Mandy Moore. STEPHANIE: If you enjoyed listening, one really easy way to support the show is to leave us a quick rating or even a review in iTunes. It really helps other folks find the show. JOËL: If you have any feedback for this or any of our other episodes, you can reach us @_bikeshed, or you can reach me @joelquen on Twitter. STEPHANIE: Or reach both of us at hosts@bikeshed.fm via email. JOËL: Thanks so much for listening to The Bike Shed, and we'll see you next week. ALL: Byeeeeeeeee!!!!!!!! ANNOUNCER: This podcast is brought to you by thoughtbot, your expert strategy, design, development, and product management partner. We bring digital products from idea to success and teach you how because we care. Learn more at thoughtbot.com.