Podcasts about Kubernetes

AI is changing cloud infrastructure, but how do you secure AI-assisted workflows? In this episode of InfosecTrain TechTalks: Real World Decoded, host Payal Pawar sits down with Cloud Architect Chitra Nair to discuss AI-powered cloud security.The "course titled" AWS Certified Solutions Architect Associate Training helps engineers master secure cloud design.

In deze aflevering gaan Ronald en Jan in gesprek met Eric de Witte, Cloud Native Solutions Architect bij Nutanix, over hoe moderne Kubernetes-platformen worden uitgerold en beheerd over virtualisatielagen, bare metal en de cloud heen. Eric heeft een lange geschiedenis die teruggaat tot de vroege vCenter-tijd, via de opkomst van container orchestration (Mesos, Docker Swarm) tot het huidige Cluster API-gedreven platform bij Nutanix.Het gesprek behandelt hoe Cluster API de onderliggende infrastructuurprovider abstraheert (VMware, Nutanix, AWS, Azure, bare metal), waardoor Kubernetes zijn eigen clusters kan uitrollen en beheren, inclusief self-healing nodes en complete procedures voor het afsluiten en opnieuw opstarten van een datacenter. Ze bespreken verschillende filosofieën rond bootstrap clusters versus een permanent management cluster, en waarom Kubernetes geen besturingssysteem is, ook al wordt het vaak zo genoemd.Een groot deel van het gesprek gaat over de huidige situatie rond VMware en Broadcom: de licentieveranderingen, de focus op grote klanten, en waarom veel organisaties hierdoor hun virtualisatiestrategie heroverwegen, ook al erkent Eric dat VMware technisch nog steeds een sterk product is.Daarnaast wordt diep ingegaan op de operationele realiteit van databases en stateful workloads op Kubernetes, de toenemende afhankelijkheid van operators, de uitdaging om interoperabiliteit te valideren bij elke nieuwe Kubernetes-release, en waarom backup en disaster recovery op applicatieniveau moeten gebeuren in plaats van puur op VM-niveau. Ze sluiten af met een blik op soevereine cloud-ambities, de kloof tussen on-prem en hyperscaler-functionaliteit, en Eric's visie op de komende tien jaar van Kubernetes: meer enterprise-adoptie, meer abstractie, maar ook meer complexiteit, waarbij networking-kennis de grootste drempel blijft voor nieuwkomers.Stuur ons een bericht.ACC ICT Specialist in IT-CONTINUÏTEIT Bedrijfskritische applicaties én data veilig beschikbaar, onafhankelijk van derden, altijd en overalSupport the showLike and subscribe! It helps out a lot.You can also find us on:De Nederlandse Kubernetes Podcast - YouTubeNederlandse Kubernetes Podcast (@k8spodcast.nl) | TikTokDe Nederlandse Kubernetes PodcastWhere can you meet us:EventsThis Podcast is powered by:ACC ICT - IT-Continuïteit voor Bedrijfskritische Applicaties | ACC ICT

An airhacks.fm conversation with Stanislav Bashkyrtsev about: discussion about testing terminology and the difference between unit tests, component tests, System Tests, and integration tests, defining component tests as in-process invocations without HTTP, using RestAssured with MockMvc-style direct endpoint calls, avoiding mocks in favor of real system tests, why code coverage is a misused management metric, the anti-pattern of using reflection to inflate coverage, distinguishing line and branch coverage from actual verification, using coverage from system tests to detect dead code for pruning, mutation testing with PIT to measure assertion quality, testing Quarkus applications, the default Guice and Guava dependencies in Quarkus RESTEasy, starting a new microservice with a separate system-test module, calling endpoints over HTTP with the MicroProfile REST Client or the Java HTTP client, deploying Quarkus on AWS Lambda as a production-like environment, backward compatibility testing with multiple production versions, turning system tests into stress and load tests, testing connection pools and metrics under load, introducing a test-only private API to verify state changes in serverless systems, contract-driven work in large consulting projects, generating JSON and JSONB directly in PostgreSQL and returning it over JDBC, mapping database rows to Java records instead of DTOs, running GraalVM inside the Oracle Database for stored procedures and table triggers, the pendulum between database-centric and application-centric logic, the convergence of SQL and NoSQL databases, CI/CD pipelines with Jenkins and manual production deployment steps, avoiding Jenkins access to production via CGI shell scripts behind nginx, AWS CodePipeline and CodeBuild with CDK-defined infrastructure, event-driven pipelines triggered by S3 put-object events, multi-account roles with short-lived STS credentials, the size of the AWS SDK and reducing it by excluding unused HTTP clients, health checks and Kubernetes liveness and readiness probes, why health checks make little sense for short-lived Lambdas, a version endpoint for deployment smoke tests Stanislav Bashkyrtsev on twitter: @sbashkirtsev

Parce que… c'est l'épisode 0x31D! Shameless plug 19 septembre 2026 - Bsides Montréal 20 au 26 septembre 2026 - BruCON 13 novembre 2026 - DEATHCon 16 au 19 novembre - European Cyber Week 1 au 3 décembre 2026 - Forum INCYBER - Canada 2026 24 et 25 février 2027 - SéQCure 2027 Description De Nessus à Tenable : les débuts d'une légende Dans cet épisode spécial de Polysécure, l'animateur reçoit Renaud Deraison, créateur de Nessus, le scanner de vulnérabilités développé il y a près de 30 ans, et cofondateur de Tenable où il a occupé le poste de CTO jusqu'à son départ il y a environ cinq ans. Deraison raconte la genèse de Nessus : à 16 ans, passionné par le réseau internet naissant, il découvre l'univers de la sécurité informatique à une époque où les failles étaient publiées sur la mailing list Bugtraq, suivie par une communauté restreinte d'environ 20 000 abonnés. Insatisfait des outils existants comme Satan, jugés trop complexes et peu accessibles, il développe Nessus en 1998 avec une cinquantaine de tests de sécurité au départ. Le succès est immédiat et inattendu, poussant Deraison à poursuivre le projet, d'abord en open source, puis en produit commercial via Tenable, qui étendra la portée de l'outil de la simple DMZ à l'ensemble de l'infrastructure numérique des entreprises. Le retour à la technique et la leçon de l'éloignement du clavier Un thème central de la discussion est l'importance de rester connecté à la pratique technique concrète. Deraison explique qu'en grandissant chez Tenable, il a dû délaisser le code pour devenir « plus stratégique », perdant ainsi une compréhension viscérale de la technologie au profit d'une compréhension purement intellectuelle. Il souligne l'écart immense, en cybersécurité, entre comprendre un concept en théorie et l'expérimenter réellement : c'est en manipulant les outils qu'on découvre les vrais problèmes de sécurité et les idées de produits pertinentes. Cette réflexion mène à une critique plus large de l'industrie de la cybersécurité, jugée parfois arrogante et déconnectée du vécu des utilisateurs, prompte à distribuer des leçons sans empathie pour les contraintes opérationnelles réelles (comme l'impossibilité de patcher immédiatement une machine critique sans interrompre un service). Bromure : le navigateur jetable réinventé Après son départ de Tenable, Deraison reprend goût au code, épaulé par les assistants IA, en commençant par ChatGPT puis Claude. L'élément déclencheur d'un nouveau projet est presque anecdotique : recevant de nombreux SMS frauduleux (offres de crédit douteuses, généré désormais dans un français impeccable et localisé grâce à l'IA), il souhaite savoir qui se cache derrière ces arnaques sans s'exposer lui-même. Il se remémore alors Bromium, une entreprise des années 2010 qui isolait Internet Explorer dans un hyperviseur, avec l'idée qu'il était inutile de patcher puisqu'il suffisait de fermer et rouvrir un environnement neuf en cas de compromission. Inspiré par ce concept, Deraison développe Bromur (bromur.io), un navigateur jetable pour Mac fonctionnant sous Linux dans une VM, avec une expérience utilisateur quasi native : glisser-déposer de fichiers entre le Mac et la VM, support webcam, contrôle fin des téléchargements et téléversements. Une version serveur permet aussi de faire travailler des contractants externes dans un environnement streamé et surveillé, où impression, captures d'écran et téléchargements peuvent être bloqués. Le projet intègre également un module (désactivé par défaut) utilisant un LLM pour analyser les pages visitées et détecter le phishing, en repérant par exemple des incohérences entre le contenu affiché et l'URL réelle, ou des tromperies visuelles comme l'usage de caractères d'un autre alphabet imitant des lettres latines. Deraison estime que le phishing reste un vaste chantier négligé par l'industrie, qui se contente trop souvent de solutions de formation jugées insuffisantes face à des attaques de plus en plus sophistiquées, y compris par deepfake vidéo. Bromure Agentic : sécuriser les développeurs sans les freiner Le second volet du projet, Bromure Agentic, s'attaque aux attaques de la chaîne d'approvisionnement logicielle (supply chain attacks), un risque majeur lorsqu'un package npm ou Python compromis vole les clés SSH, tokens d'authentification et autres identifiants sensibles présents sur la machine d'un développeur. Plutôt que d'imposer des restrictions punitives comme le font certaines sandboxes existantes, Bromure propose une VM Linux complète où l'utilisateur dispose de faux tokens et fausses clés SSH. Un mécanisme de type « man in the middle » observe le trafic sortant et substitue à la volée les vrais identifiants aux faux, uniquement au moment nécessaire. Ainsi, même en cas de compromission, aucun secret n'est réellement exposé sur la machine. L'outil ajoute aussi des alertes en cas de mauvaise configuration et permet désormais de restreindre les droits (par exemple en lecture seule) sur les accès Kubernetes ou Git, sans exiger du développeur qu'il devienne expert en administration système. Repenser des dogmes établis La conversation aborde aussi la remise en question de certains dogmes de sécurité, comme le caractère « intouchable » du trafic SSL, que Deraison juge paradoxal : on surveille tout le comportement d'une machine, sauf le trafic chiffré sortant. L'inspection de ce trafic ouvre selon lui de nouvelles possibilités, comme bloquer les téléchargements de packages npm en version « latest », souvent les plus vulnérables au piratage. Les deux interlocuteurs concluent sur une analogie automobile : de même que les usagers n'ont pas à comprendre le fonctionnement d'un moteur pour conduire, les utilisateurs et développeurs ne devraient pas avoir à devenir des experts en sécurité pour travailler efficacement. La responsabilité doit se déplacer vers des outils qui protègent sans punir, permettant à chacun de rester concentré sur son métier. Notes Bromure Collaborateurs Nicolas-Loïc Fortin Renaud Deraison Crédits Montage par Intrasecure inc Locaux réels par SSTIC

Can you build a fully functional, high-scale SIEM in just two weeks for under $7,000? In this episode of the Cloud Security Podcast, hosts Tim Peacock and Kyle Champlin sit down with long-time collaborator Dan Lucier, Founder of Nano, to unpack how he "vibe-coded" an entire SIEM from scratch during his end-of-year holiday break. Dan shares his journey of leveraging bleeding-edge AI code assistants to go from a Postgres prototype to a blazing-fast, production-ready SIEM built on Rust and ClickHouse. In this episode, we cover:

Welcome to the CanadianSME Small Business Podcast, hosted by Kripa Anand. Today, we explore how SMBs can build resilient, cost-efficient cloud infrastructure without overcomplicating systems. Joining us is Amir Abolhasani, Founder and Technical Director at Namira Software Corporation, an AI architect with over 15 years of experience designing scalable platforms across multiple industries. Key Highlights Platform Engineering Priorities: Focus areas for leaders in 2026. De-risking SMB Infrastructure: Simplifying DevOps and Kubernetes adoption. AI-Powered SRE: Improving incident response with AI agents. Reducing Cloud Costs: Identifying and eliminating hidden expenditures. Bootstrapping Lessons: Insights from building your own scalable platform. Special Thanks to Our Partners: UPS: https://solutions.ups.com/ca-beunstoppable.html?WT.mc_id=BUSMEWA ADP Canada: https://www.adp.ca/en.aspx For more expert insights, visit www.canadiansme.ca and subscribe to the CanadianSME Small Business Magazine. Stay innovative, stay informed, and thrive in the digital age! To learn more about how we are supporting the ecosystem, please visit the CanadianSME Small Business Foundation at smbfoundation.ca. Disclaimer: The information shared in this podcast is for general informational purposes only and should not be considered as direct financial or business advice. Always consult with a qualified professional for advice specific to your situation.

We've been running a bit of an Agent Cloud series surveying all the top inference/compute/cloud providers, from Databricks to Daytona to Railway and, even further back, E2B, but we're excited to conclude this series returning to Modal, which has just raised a monster $355M Series C.The cloud was built for developers. But agents are now changing that.The old infra stack was designed for a human who could read docs, reason through YAML, and understand dashboards to figure out what they need when something broke. While this was painful for developers, it worked since they could fill in missing context in their heads.However, agents don't have that luxury. Now in this new era of agents, everything has to be tighter.They need a place to write code, run it, inspect the output, change the environment, debug failures, and try again. Fast iteration and feedback loops with all the necessary context are crucial for agents to operate properly. Furthermore, sandboxes are a clear representation of this shift as agents can easily spin up isolated environments. This programmatic infra even extends to research:Two years ago, we were one of the first to cover Modal with CEO Erik Bernhardsson and Alessio designed our favorite LS thumbnail of all time:At the time, Modal was just a teeny little company with a $17M Series A.Today, fresh off their $355M Series C, Modal is one of the clearest examples of the agent cloud future being built in real time: a cloud platform moving past traditional web app assumptions toward the workloads AI actually creates such as elastic inference, sandboxes, GPU burst, post-training, background agents, and infrastructure that agents themselves can operate.In this episode, Modal CTO Akshat Bubna joins swyx and Vibhu to unpack why AI applications don't fit traditional cloud assumptions, why Kubernetes was never designed for bursty compute-heavy workloads, and why Modal is now shifting from developer experience to agent experience.We go deep on Modal's AI infra stack: serverless functions, decorator-based infrastructure, elastic inference for custom models, GPU snapshotting, DeFlash, speculative decoding, Auto Endpoints, sandboxes, persistent storage, networked containers, private IPv6, RDMA, multi-node training, and Modal's capacity pool across 17 cloud providers. Akshat also explains why RL rollouts can require 100,000 sandboxes, why production agents need hard guardrails, why observability may matter more than reading code, and why AI has made infrastructure exciting again.We discuss:* Why Kubernetes wasn't built for bursty AI workloads* How Modal started as a better runtime before becoming an AI cloud* Why Modal added GPUs before ChatGPT* The shift from developer experience to agent experience* Why observability matters when agents are writing the code* Elastic inference for custom models across audio, video, robotics, and comp bio* GPU snapshotting, cold starts, and why inference workloads are so bursty* Why RL rollouts can require 100,000 sandboxes* DeFlash, speculative decoding, and frontier-level inference performance* Auto Endpoints and making optimized inference easier to deploy* What Modal adds beyond vLLM, SGLang, and raw GPU rental* Modal's 17-cloud capacity pool and supercloud strategy* Networked sandboxes, sidecars, private IPv6, and RDMA* Serverless multi-node training for post-training and research workloads* Auto-research, model-guided sweeps, and agents launching GPU experiments* Compute strategy, capacity planning, and batch tiers* Why production agents need specialized sandboxes and hard guardrails* Modal's take on managed agents, CI, Gitpod/Ona, Python, TypeScript, and Modal BenchAkshat Bubna* LinkedIn: https://www.linkedin.com/in/akshat-bubna-188885103* X: https://x.com/akshat_bModal* Website: https://modal.comTimestamps00:00:00 Introduction00:00:39 Modal's origin and why Kubernetes wasn't enough00:04:32 Developer Experience → Agent Experience00:06:21 Modal's AI cloud primitives00:09:14 Sandboxes, agent loops, and proto-Cognition00:12:12 Elastic inference, GPU snapshotting, and 100,000 sandboxes00:15:24 DeFlash, speculative decoding, and Auto Endpoints00:19:59 Production-grade inference beyond raw GPUs00:22:00 Background agents, Ramp Inspect, and the agent lifecycle00:24:08 Modal's 17-cloud supercloud strategy00:26:40 Networked sandboxes, private IPv6, and RDMA00:32:48 Multi-node training, post-training, and auto research00:37:36 Compute strategy, capacity planning, and batch tiers00:40:55 Open models, real-time AI, and production agent infra00:43:06 Hard guardrails, managed agents, and specialized sandboxes00:46:06 Why AI made infrastructure exciting again00:48:30 Model APIs, differentiated products, and agentic video00:51:50 CI, coding-agent infra, SDKs, and Modal Bench00:57:28 Closing ThoughtsTranscriptIntroduction: Modal, Series C, and the Art PartySwyx [00:00:00]: We're here with Akshat, CTO of Modal, together with Vibhu. Congrats on your Series C.Akshat [00:00:10]: Thank you.Swyx [00:00:11]: Your party yesterday was amazing.Akshat [00:00:15]: Yeah.Swyx [00:00:15]: From all the photos and all the swag.Akshat [00:00:17]: We had a bunch of art installations, which was fun, seeing, like, our products on pedestals next to, like, Rodin.Swyx [00:00:25]: Very nice. Very nice. When you started, it was not the GPU inference company. Maybe it was in your mind. Take us back to the origin story.Modal's Origin: A New Runtime Beyond KubernetesAkshat [00:00:39]: I first met Eric, who's the CEO, through an investor. Back then Eric was already thinking about building, a new runtime, and he got there thinking through why are workflow orchestration products so hard to use. It's because you have to run them on Kubernetes. Kubernetes is hard to manage. It's not built for burstiness and, custom images,Swyx [00:01:03]: YeahAkshat [00:01:03]: It has a terrible developer experience.Swyx [00:01:05]: And I'll, I'll interjectAkshat [00:01:06]: YeahSwyx [00:01:07]: For listeners, who are new, we interviewed Eric two years ago, and there's a bit more of the story there from Spotify and all those things.Swyx [00:01:14]: And I came across Eric through Data Council because he did that talk on the serverless container stack that you guys did, which was like, that was my first like, “Okay, I need to take Modal very seriously” moment.Akshat [00:01:26]: Yeah.Swyx [00:01:26]: But it was still very unclear, like, do I need all this for just my data pipelines?Akshat [00:01:33]: Yeah. initially what we were thinking about was if we build a better runtime, it's a very useful primitive in itself. It's There's a lot of things that, get solved by serverless functions, like you can do, ETL stuff, you can do job queues, you can do all this, like, bursty processing, which it turns out every company had needs for. but then we also were thinking about this as like, this is a primitive that we can build a whole collection of products on, which are very verticalized. So perhaps data engineering would've been the first one, but we were thinking about inference. Back then it was more classical inference, like computer vision stuff and running XGBoosts and whatnot. But we added GPUs to the product a year before ChatGPT came out.From Serverless Containers to GPU WorkloadsSwyx [00:02:19]: Nice.Akshat [00:02:19]: We just didn't think it would be that big of a deal.Swyx [00:02:22]: Yeah, just like add A100.Vibhu [00:02:23]: Was there any, like, early key problem that really sparked off why you built it?Akshat [00:02:28]: Yeah. Primarily it's just, none of the tooling that was out there was built for, one, a really great developer experience, and also there's a general trend of, a lot of the workloads that we were seeing were very. I wish there was a better word for it, but compute-heavy. Like, they need, one, like, need a lot more resources, so you need to burst up and down a lot, versus like Kubernetes designed for, like, slow scaling and, more for, like, web server use cases. And also there's just a lot more specialization in, like, what kinds of environments these workloads run in. Like, we had sometimes they need accelerators, sometimes they need different kinds of images, and this is just like a consistent thing that we saw across a lot of companies. That would be the next step.Software-Defined Infrastructure and Decorator-Based DXSwyx [00:03:13]: Yeah. Yeah. Be nice. I don't know how much this factored into the early story, but I wrote a post when I was at Temporal about infrastructure, software-defined infrastructure or something like that.Akshat [00:03:22]: Yeah, the self-provisioningSwyx [00:03:23]: Self-provisioning.Akshat [00:03:24]: Yeah.Swyx [00:03:24]: Yeah. I can't even remember my own post.Swyx [00:03:26]: And then you put me on the landing page.Akshat [00:03:28]: Yeah. We really like, the term and so we stole it.Swyx [00:03:32]: Because you had the insight that everything can just be in decorators co-located with the code, right?Akshat [00:03:37]: Yeah.Swyx [00:03:37]: Was that a big part of the originalAkshat [00:03:39]: YesSwyx [00:03:39]: Story or it was just like a DX layer?Akshat [00:03:41]: That was, really important because we really didn't want people to spend, so much time, writing YAML, and it seemed like you could really condense the surface area of what you're doing, put it in code so you can operate on it just like you operate on other code, and like build stuff that's more expressive and dynamic. and so yeah, that was always a very important part.Swyx [00:04:04]: Then the pushback is this is a DSL.Akshat [00:04:07]: Yeah.Swyx [00:04:07]: It's you're closed source. I am locked into Modal.Akshat [00:04:11]: Yeah. We never really got pushback for that because the nice thing about Modal is you can bring whatever code you have, and sure, the DSL is at the configuration layer for, what hardware you're using, how you're scaling things up, but you still own the code.Akshat [00:04:27]: And that's, that's been an important, part of our story, even as we do inference now.Swyx [00:04:32]: Yeah.Vibhu [00:04:32]: How much of do you think still stays the same today? Like if you were to build something today, DevX very important, but I feel like, a lot of this has been changed with just hook it up to an agent, have Claude Code, have Codex implement a tool. there's very agent native primitives that are different than if I'm doing this myself, right?Developer Experience → Agent ExperienceAkshat [00:04:54]: We've changed our SDK team to think about agent experience instead of, developer experience and we think that the same benefits that apply for DX also apply for AX, which is why would you have an agent read through hundreds of Kubernetes files and like write YAML that's not even typed when it can make a couple of changes in a decorator and it gets this self-provisioning runtime of, being able to see its changes live in action? yeah, it just seems from the customers we talk to, they find Modal is much faster for agents to use versus operating on a different substrate.Swyx [00:05:34]: Yeah, because like you, again, you co-locate the infrastructure requirements to the code that runs it.Akshat [00:05:38]: Yeah.Swyx [00:05:38]: Well, the negative thesis now is that nobody's looking at their code anymore, so there's no point.Akshat [00:05:44]: Yeah, people aren't looking at code. one thing we still see is really important is observability.Swyx [00:05:51]: Yeah.Akshat [00:05:51]: Like how good is your dashboard? And of course, like we have, we push a lot of it to the CLI so the agents can do their own investigation, but you still need humans to go interpret what's going on and, make judgment calls and whatnot. and that's I feel like, Maybe more important now than looking at the code itself.Swyx [00:06:11]: Yes, because like, you can try to treat the code as a black box and then use, see the observable action that comes out of it, and then just prompt a change.What Modal Is For: AI Cloud PrimitivesAkshat [00:06:21]: Yeah.Swyx [00:06:22]: So I think it takes a bit of restraint to not specialize, to say, “I want to ship a new primitive,” and then just be general purpose.Swyx [00:06:31]: People ask you, “What are you for?” You're like, “ I don't know. We can do this, we can do that.”Vibhu [00:06:36]: Well, I'd be curious to see, like, okay, if we were to ask you, like, what is Modal for even at a high level? There's a lot you guys do, sandboxes, GPUs, everything. How do you answer?Akshat [00:06:46]: Modal is a cloud platform that's built for, where we've built the primitives from scratch for AI applications. and right now it covers, inference, training, batch processing, and sandbox workloads.Akshat [00:07:00]: But we're building a lot moreSwyx [00:07:02]: I noticed you didn't say web server, so there is still a role for, like, the always-on large-scale Kubernetes type things.Akshat [00:07:09]: Yeah, absolutely. We're, we're not trying to compete with the renders of the world, because yeah, we think the differentiator for us is the, are the workloads that need specialized compute, need to scale up and down a lot. yeah, they're, they're, they're just shaped differently.Working Alongside Frontier StartupsVibhu [00:07:26]: I think you're building a lot of it alongside the startups, right? They're innovating quite a bit, even in your, like, latest blog post. Like, even in the series C, the customers that you mention here, the cognitions, technical ones, ramps and whatnot, they're, they're innovating with you, right? And that's not something AWS is doing directly with.Akshat [00:07:45]: Yeah, absolutely. I think, this is again classic. We're a small team. We can move really fast. our engineers are working with our customers and figuring it out. Yeah.Swyx [00:07:54]: So my first week at Cognition, I walked in, there was someone wearing a Modal shirt. I was like, “What are you doing here?” They're like, “Yeah, I just. I am embedded inside of Cog.”Akshat [00:08:05]: Yeah, I think that was Peyton. We sent him overSwyx [00:08:07]: Yeah.Akshat [00:08:07]: Because, the latency of communication was too high otherwise.Swyx [00:08:12]: Yeah, distributed node, you have to - you have to place one and collocate.Vibhu [00:08:16]: Yeah.Swyx [00:08:16]: So I had a, I had direct personal experience, right? So I worked on smol developer three years ago. it was inspired by Claude 1. I think you onboarded me at some point, like, just before, and I was like, “Oh, like, I need some bursty compute. Like, I was just gonna try using Modal.” And it was a, it was a pretty pleasant experience. apparently, I showed up in the board meeting, like the analytics.smol developer, Sandboxes, and Proto-CognitionAkshat [00:08:39]: Yeah, you blew up on Hacker News and,Swyx [00:08:41]: YeahAkshat [00:08:41]: We got a big traffic spike. I. I think the way you used smol developer was Modal functions for running stuff, which was. Like, the, that was a good use case. but then, yeah.Swyx [00:08:53]: Yeah. That - So to me, that was proto-cognition.Akshat [00:08:55]: Right.Swyx [00:08:56]: If only I had, like, stuck to it.Swyx [00:08:58]: Like, that was like, if - did you say draw the tech treeAkshat [00:09:00]: AbsolutelySwyx [00:09:00]: You're just like, “Yeah, like, probably this will happen.”Akshat [00:09:02]: Yeah. Like, he was so close. You were just rebuilding upon usSwyx [00:09:04]: I just didn't realize.Akshat [00:09:05]: But the funny story there is at the same time, we were talking to a bunch of customers who needed something like sandboxing.Swyx [00:09:14]: Yeah.Akshat [00:09:14]: This is like twenty-three.Swyx [00:09:15]: Yeah.Akshat [00:09:16]: So we builtSwyx [00:09:17]: You introduced a new API right after that.Akshat [00:09:18]: Yeah.Swyx [00:09:19]: Yes.Akshat [00:09:19]: Like, we built sandboxes in May of twenty-three before anyone was even knew this was gonna be a thing. And the first example we published was, we took smol developerSwyx [00:09:28]: Smol developerAkshat [00:09:28]: And put it in a loop, so the agent can iterate on itself.Swyx [00:09:33]: Loops are hot these days.Vibhu [00:09:34]: It's the looper.Akshat [00:09:34]: Yeah.Vibhu [00:09:35]: Loops in. When was this, twenty-three?Akshat [00:09:38]: Yeah.Vibhu [00:09:39]: A small check.Akshat [00:09:39]: Yeah.Swyx [00:09:39]: It's like twenty-three. so the. the, those for listeners, like, the problem was the models are not built for any of this, right?Swyx [00:09:46]: Like, you're just trying to like. They're not post-training to understand, like, looping and, like, self-correction and tool calling was there, but, like, also not that great.Akshat [00:09:55]: Yeah.Akshat [00:09:55]: I don't remember if you used tool calling in this one, but yeah, the models would just diverge after like ten iterations and not produce anything meaningful.Swyx [00:10:03]: Yeah. But like, then. So okay, like now talking to myself three years ago, the answerVibhu [00:10:08]: Of course they will get betterSwyx [00:10:09]: Collect all the failures, build benchmark, and then collect all the, examples, build the RL environmentAkshat [00:10:15]: RightSwyx [00:10:15]: Sell it for like ten billion dollars to Meta.Swyx [00:10:17]: And then also train a model and then sell that for sixty billion dollars to Elon. And this isAkshat [00:10:23]: Yeah, of courseSwyx [00:10:23]: The funny machine. Like, it's like, it's about the hardware.Akshat [00:10:28]: It's hard to have that inherent conviction that the stuff will get that much better.Swyx [00:10:33]: In retrospect, it's so f*****g obvious.Akshat [00:10:36]: Fair enough.Swyx [00:10:37]: Like, what else were we doing back then? I don't know. anyway. Yeah. So this. That was the start of your sandboxing journey, right? I feel like it didn't blow up until, like, last year.Akshat [00:10:49]: Yeah.Swyx [00:10:50]: So there was like a couple years of quietness.Akshat [00:10:52]: Exactly, yeah. We wereVibhu [00:10:53]: I think very underrated product value. Like, my experience with Modal, Charles, before he had joined Modal, met this guy at a hackathon, and he really insisted we wanted to run some small model, not hosted anywhere, and he's like, “ there's this cool company, Modal. They'll like spin up a GPU sandbox, we can throw it on there. They'll take a Hugging Face link.” And like there's so much value just right there, right? Like instant hosting, spin it up, spin it down. It'll stay cold, but we run the demo a few days later, it'll come back up and like all this stuff in retrospect, like it's still what we needed like today.Akshat [00:11:27]: Yeah, it's still needed today. workload shapes have changed a lot as, we run stuff for people with really massive production scale and, there it's it's not about scaling from zero to one, but it's how do we scale really elastically, from like thousand to fifteen hundred GPUs very quickly in a given region. It's the same shape problem.Elastic Inference, GPU Autoscaling, and Custom ModelsVibhu [00:11:50]: Okay. So you look at, say, Cursor Composer, right?Akshat [00:11:53]: Yeah.Vibhu [00:11:53]: They had a. “We'll do RL on a model every couple hours.” you guys have a whole version of RL inference gym and whatnot.Vibhu [00:12:01]: When you look at workloads like that, you're doing train runs where you need to scale up, scale down every hour thousands of GPUs, right? That's the example for we do need it, right?Akshat [00:12:12]: Yeah. Well, so I'll, I'll take a step back and, maybe talk about like how people use Modal today. because our biggest use case is, elastic inference. And the thing we first found product market fit, with was inference for custom models. So we stayed away from the LLM space, and we were serving companies like Suno for audio, Runway for video, robotics, comp bio companies that train their own model elsewhere. But Modal is the best black box that for deployment, scaling to however many GPUs you need as your traffic pattern changes. And we saw all of them like have a very unpredict- predict- predictable, traffic pattern. it's like diurnal. It's Some days, like the company will do a launch and, they'll need like, way more. And it's not just one model that they deploy. They-- all these companies deploy, lots of different models in different regions, and so the autoscaling problem becomes even harder because then you have to scale within a certain region, and those cycles are offset. So different times you scale up in different regions.Akshat [00:13:20]: So that's like our sortVibhu [00:13:22]: And thatAkshat [00:13:22]: YeahVibhu [00:13:22]: That in and of itself is a huge category. There's a bunch of inference providers which, provide this fireworks, does this as a service together, whatnot, Base10. that's carved into its own niche for language models, at least right now.Akshat [00:13:36]: Yeah. the thing that we have specialized in is the autoscaling aspect.Vibhu [00:13:41]: Yeah.Akshat [00:13:41]: Because we found that it's not universally true that everyone else can autoscale, and we've gone deeper into it on the tech side by, we've incorporated GPU snapshotting into the product so we can take the GPU state, like your torch.compile model, snapshot it, and the next cold start is way faster. And so going back to your question, it's That's why you need a lot of burstiness for inference. But then people also do a lot of demand training, like for RL stuff, your rollouts are bursty, as you said. People also do a lot of batch jobs. So we'll see, a lot of companies, before they have a training run, they'll need thousands of GPUs to run encoding or something like that. And I think those things are much more bursty than. I agree that agents are not that bursty. sandboxes are, except when you're doing RL. RL is justRL, Batch Jobs, and 100,000 SandboxesVibhu [00:14:28]: Or commerceAkshat [00:14:28]: Insanely bursty.Vibhu [00:14:29]: Yeah.Akshat [00:14:30]: Yeah. Like when you're doing, rollouts, you sometimes need a hundred thousand sandboxes in your sandboxes.Vibhu [00:14:37]: Yeah. I'm curious if you've seen early sparks of continual learning. There are some people, like our friends, ngram, recently announced thisAkshat [00:14:45]: YeahVibhu [00:14:45]: They're, they're trying to do training. That also seems like a different workload, right? If you're doing training twenty-four/seven per se, there's a very weird dynamic of how you're using GPUs between people and whatnot, but seems like something you guys would work for.Akshat [00:15:00]: As you said, we're, we're fortunate to work with a number of, customers at the frontier and grab some of our customers. and they are taking the primitives we have, and trying to use them in very interesting ways, like continual learning. It's possible as the stuff gets better, some of that will be part of, our offering as well if, more people need it. but we're, we're just waiting to seeVibhu [00:15:23]: YeahAkshat [00:15:23]: How it shakes out.Vibhu [00:15:24]: Is there a primitive that you added after sandboxing that was the next step in the story?LLM Inference, DeFlash, and Speculative DecodingAkshat [00:15:32]: I guess we've been going much deeper into LLM inferenceVibhu [00:15:35]: YeahAkshat [00:15:35]: Because we realized that some of the advantages we have with like autoscaling, again, especially in different regions and whatnot, are, not present elsewhere. and the place where we had a gap was we weren't, working on the model layer itself. Like we were a black box. And, we realized that, we can get to frontier-level model performance, with, by having great people who work on this. And, we've been open sourcing a lot of our work, in terms of, Recently, we, shared our work on DeFlash, which is a block-based, speculator, and we've open sourced, all of it. So, you can - By using open source DeFlash, you can get the same performance as you would with one of the proprietary providers. And the next thing we're thinking about hereVibhu [00:16:23]: I thought this wasAkshat [00:16:24]: YeahVibhu [00:16:24]: An interesting blog post as well, right? Like, I think in here you make a claim that. Not a claim, just that how effective speculative deco-decoding really just get to.Akshat [00:16:33]: Yeah.Vibhu [00:16:33]: Anything you wanna point out from this around, what people should know?Akshat [00:16:39]: Yeah, absolutely. the high-level summary is, it would help to describe what speculative decoding is.Vibhu [00:16:44]: Yes.Akshat [00:16:44]: I will, yes.Vibhu [00:16:45]: I think, likeAkshat [00:16:46]: YeahVibhu [00:16:46]: So we've covered like Eagle and all thisAkshat [00:16:47]: YeahVibhu [00:16:47]: Like Hydra and all those things, but it was like two years ago.Akshat [00:16:51]: Yeah.Vibhu [00:16:51]: I think it doesn't hurt, right?Akshat [00:16:52]: Yeah. Speculative decoding is you have a smaller model, called a draft model, predict tokens ahead of the bigger model, and then you have the bigger model, verify all of this, all the tokens are predicted. And the reason it's faster is if you're predicting, one token at once, you're bound by memory bandwidth. But if you can batch the verification of, the draft model, then you're much more efficient using compute, and it's faster, and as long as your draft model is producing a lot of tokens that can get accepted, which is called the accept length, you can get a speed up that's, multiple times of, the original model speed. and well, that's what we highlight here. It's Like people talk a lot about we made these kernels faster and whatnot, but improving kernel will only give you like few percentage points of improvement, and, increasing accept length, literally is a multiplicative decreaseVibhu [00:17:47]: Like two to four X.Akshat [00:17:48]: Yeah, exactly.Vibhu [00:17:48]: Without much head-on performance.Akshat [00:17:50]: Yeah. I think it may - you are running a second model, right? So it may be something more expensive in the compute,Vibhu [00:17:57]: I meant quality performanceAkshat [00:17:58]: Probably not by muchVibhu [00:17:58]: But yeah. I thinkAkshat [00:17:59]: So there's no drop in quality performanceVibhu [00:18:01]: YeahAkshat [00:18:01]: Because you're always. You're never accepting a token that the big modelVibhu [00:18:04]: It's strictly betterAkshat [00:18:05]: YeahVibhu [00:18:05]: Or it's same.Akshat [00:18:06]: Exactly.Vibhu [00:18:07]: Right. Yeah.Akshat [00:18:08]: And so we've been working a bunch on DeFlash, which is a block-based speculator. so it's instead of predicting, one token at a time, it's predicting a block. And we've been open sourcing our work with it. The next thing for us here is for helping people train speculators and custom models. it's it's something that traditionally is very forward-deployed engineering driven, support deployed, engineer driven, like you work with customers and help them do that. And our vision for. This is why we launched Auto Endpoints, is we want to make frontier-level performance available to everyone. And so, we mentioned this in the announcement, we teased it. The next thing we're, we're launching is, as you run an auto endpoint, we shadow trafficAuto Endpoints and Frontier-Level PerformanceVibhu [00:18:54]: Do you want to explain what auto endpoints are?Akshat [00:18:57]: Yeah.Vibhu [00:18:57]: I lovely, yeah.Akshat [00:18:58]: Yeah. So, this is, I guess, going back to your Modal is you touch the code, but, sometimes people don't wanna touch the code, and they wanna get started with an endpoint that works and has all the great performance and, scalability that Modal has. So we've made that easier with, a way to create an endpoint from our UI, from the CLI, that has all of our optimizations that we talked about, like the DeFlash stuff already baked in, and there's full transparency. So we give you the code, you can go run it yourself, and if you want, you can eject out into the full Modal experience, which we see as people get sophisticated, they do wanna tweak the models, they wanna, fine-tune stuff. You can still do all of that. It's it's not a black box. And yeah, the next thing, as we teased later in the post, is how do we give you value even beyond this in terms of having your draft models evolve as your data distribution evolves, again, without having to talk to a person and, yeah.Vibhu [00:19:59]: I guess just to understand it directly, you have the GPUs, you have an endpoint that's compatible, you serve open model. If someone was to do this themselves, what's the delta that you guys provide? So you do a lot of open source great work on effective inference. how does it compare to, say, I take the same model, 5.2 FP8, take shelf inference engine, vLLM, SGLang, get compute of similar capacity, similar cost. What's the delta that plugging into something this, like this offers outside of the benefit of, scaling?Production Inference Beyond Raw GPUsAkshat [00:20:34]: It's interesting because we've taken the approach of open sourcing our contributions and upstreaming them. we work closely with the SGLang team. We want the improvements that our team, comes up with to be, there in open source for others to use, even outside of Modal. The benefit to us is we have a team that has significant expertise in terms of if you do have something that is not there, our team can help you get that performance, first. the other thing is with these endpoints, we are way more elastic, as you said, than, anyone else, and you have true scaling to zero. you have true, burstiness, and in practice, that matters a lot more to people than just finding, the GPU and, running Modal code on something.Vibhu [00:21:20]: Yeah. And I will say it's not that straightforward to just. like what I said is easier said than done, right?Akshat [00:21:26]: Yeah.Vibhu [00:21:27]: It's I think still for the average person, still hard to just gut check using different. There's, there's quite a bit of combinations you can make there. the trade-offs aren't really known at face value.Akshat [00:21:40]: Yeah. it's it's not just that. I think it's it's that running production-grade inference is a hard infer problem.Vibhu [00:21:49]: YeahAkshat [00:21:49]: Even if you subtract out the autoscalingVibhu [00:21:50]: YeahAkshat [00:21:51]: Is controlling things like tail latency and, making sure every, request is delivered at least once and whatnot.The Model and Agent LifecycleVibhu [00:22:00]: There's a lot of innovation that you can do here. I think, it's very interesting that you're starting to encroach on, like as you become a full cloud, you're starting to encroach on other people's turf.Vibhu [00:22:09]: What will you not do?Akshat [00:22:13]: Well, we wanna follow our users and, make sure they get like a platform that has everything that works well together. so right now we're focused on the model lifecycle and the agent, lifecycle. so both like going from data prep to training to inference, and then also if I want to deploy a background agent, let's say, sandbox, do persistent storage, a whole bunch of other stuff.Vibhu [00:22:38]: We talked to Cole, who did, OpenInspect. Yeah.Akshat [00:22:42]: Yeah.Vibhu [00:22:42]: And RealInspect also is on Modal.Akshat [00:22:44]: Yeah. So Ramp Inspect was a great example of a background agent that was really successful because they, were able to use some of the primitives like snapshotting and fast scaling to just have something that feels really reactive and works well.Ramp Inspect and Background AgentsVibhu [00:23:02]: Yeah. That's the new CTO of, Ramp right there.Akshat [00:23:05]: Yeah, Rahul.Vibhu [00:23:08]: It was really fun. yeah, okay, I think, all very bullish. Like, one of my reflections was also I did not originally. So when I met you guysThe Inference Inflection: CPU, GPU, and Co-LocationVibhu [00:23:19]: You weren't that much in the GPU game, and now you're all about, inference. And one of the points that I hinged on for Jensen's keynote at GTC this year was, what we're calling like the inference inflection, right? That let's say in AI workloads or machine learning workloads, it used to be like, let's call it eight to one GPU to CPU, and now it's more like one to one, which is like a interesting. Like, - because of how much agents are blocked or call out to this, to CPU heavy stuff the actual, like, limiting factor, like, swings back and forth from GPU to CPU a lot more than it used to be all GPU and then occasional CPU.Akshat [00:24:01]: Yeah.Vibhu [00:24:02]: GPU, CPU. And now it's like just constantly, and you just have to locate everything.Seventeen Clouds and the Supercloud StrategyAkshat [00:24:08]: Yeah. And that's one of the things that, again, we see as, something appealing about Modal, which is we've built this capacity pool that spans, 17 cloud providers, so we're, we're very good at Running on various kinds of cloud capacity across the worldSwyx [00:24:24]: You don't have your own data centers?Akshat [00:24:25]: We don't have our own data centers. We just run across a lot of neo cloudsSwyx [00:24:29]: Yeah. AreAkshat [00:24:30]: Metal providers.Swyx [00:24:30]: Yeah. Question mark.Swyx [00:24:31]: Yeah. You're, you're running the math, and you're like, “What's the cutover point where you're like.”Akshat [00:24:36]: Yeah, it's a good question. part of it is we see our differentiator in the software layer, and, being capital light and focusing on the software helps us move really fast. so far it's worked out well because there are so many other people building data centers that we're able to work effectively with them, and again, focus on what makes us, special.Swyx [00:24:55]: Yeah.Swyx [00:24:56]: 17 gets you into, like, the local providers sometimes. LikeAkshat [00:25:00]: The,Swyx [00:25:01]: Which was the most interesting one?Akshat [00:25:02]: There are a lot more neo clouds than you expect, and they all have various degrees of, various levels of reliability. And, that's why it's something we've invested a lot of time in, is building our own reliability layer on top. so if the GPU falls off the bus or something happens, we user workloads are not affected, and that lets us use a lot more capacity than,Swyx [00:25:30]: YeahAkshat [00:25:30]: You as a user would be able to.Swyx [00:25:32]: It's a useful thing to have because like now everyone knows, like, what layer you are and, like, you optimize for being the super cloud of all clouds.Akshat [00:25:41]: Yeah. That's, that's, that's the idea. and so I guess when you mentioned colocation, that's, that's another interesting thing where, one thing we've seen is people come to us when they want, very specifically located, CPUs or GPUs, like they wantSwyx [00:25:57]: Oh, they pin it in likeAkshat [00:25:58]: YeahSwyx [00:25:58]: EU?Akshat [00:25:59]: Exactly. Or EU, US.Swyx [00:26:01]: Right. Data resiliencyAkshat [00:26:02]: AustraliaSwyx [00:26:02]: Locality thing or performance or what?Akshat [00:26:04]: It's either data locality or latency, yeah.Swyx [00:26:07]: Yeah.Akshat [00:26:07]: Like, you want your. They're running sandboxes and model. They want them to be right next to aSwyx [00:26:10]: Yeah, it's easy thenAkshat [00:26:11]: YeahSwyx [00:26:12]: To. That is important in all those things. and so, like, you've accidentally, I don't know if it's accident, but, like, you've built the perfect primitive for agents to express themselves. And then, like, it's almost very funny how every extra development just involves more file system, just involves more CPU.Akshat [00:26:30]: Yeah.Swyx [00:26:31]: Just like the things that you already have. I don't know much about, if there's any, like, networking usages that are interesting, but you've also done some good work on networking.Networking, Sidecars, Private IPv6, and SandboxesAkshat [00:26:40]: Yeah, that's exactly right. Like, we're just taking compute storage and networking and building stuff on that layer, for, again, the stuff people need.Swyx [00:26:49]: YeahAkshat [00:26:50]: We see a few interesting networking things coming up. one is people want networked sandboxes. so we haveSwyx [00:26:57]: For like a Docker cluster type thing.Akshat [00:26:59]: Yeah.Swyx [00:26:59]: Sorry, Docker Swarm. Oh, f**k. What is it called?Akshat [00:27:02]: Compose.Swyx [00:27:03]: Compose type thing.Akshat [00:27:04]: Yeah. So if you want Docker Compose, our sandboxes now support, this thing called sidecars. So you can. A sandbox is a pod of containers, and you can run multiple containers in, a sandbox. also useful because, going back to networking, people want a lot of control over, outbound networking from a sandbox.Swyx [00:27:23]: Yeah.Akshat [00:27:23]: Like, they might wanna run a middle proxy for, like, maybe logging stuff for RL or, controlling how egress can happen to a domain, injecting credentials. and yeah. So we've, we've had to build a lot of that stuff ourselves.Swyx [00:27:38]: Yeah.Akshat [00:27:39]: But then also sometimes people want, sandboxes spanning multiple nodes to talk to each other, which is an emerging thing we're seeing. We have support for that for a different reason, and yeah, we'll see if that becomes stable.Swyx [00:27:52]: Like, just an open socket. It's a. This is directly like mTLS.Akshat [00:27:56]: We do support that, which is you can, expose a tunnel inside a sandbox.Swyx [00:28:01]: Yeah.Akshat [00:28:01]: And then you can either expose it to public internet or it can be, you can add like a HTTP, auth layer above it. But we have this thing called I6PN, which we haven't talked about, which is this, like, overlay network using IPv6 addresses. so if Modal containers, within the same workspace, when this is enabled, can address each other using this private IPv6 address, and no one else can.Akshat [00:28:28]: So it's like private networking, for containers. We built it because we needed it as a primitive for our distributed training product. so we have this other feature, which is you can add a decorator to a function, and you get a cluster of GPUs. and they have RDMA networking. so you can run a distributed training job, that's truly serverless. and we did the overlay network for that. But then we've seen that people are using it for other reasons, and, I'm intrigued to yeah, what would people do with it.Swyx [00:28:59]: Build primitives and let people figure it out, right?Akshat [00:29:01]: Yeah, exactly.Swyx [00:29:02]: You put out a pretty interestingAkshat [00:29:03]: They're like, they read the docs webpage. Let me use thatSwyx [00:29:06]: YeahAkshat [00:29:06]: Something they never intended to work. This is literally not even in our docs page. People somehow found it, and they're using it.RDMA, Memory Movement, and Distributed TrainingSwyx [00:29:12]: Huh.Swyx [00:29:14]: The way you portrayed it with, like, RDMA versus TCP, like, very well laid out, but just the transfer speed change at scale for RL, like yeah, you have it, you have it built in. I'm sure someone found it. It's found it to be a lot more efficient before you made a thing out of it, right?Akshat [00:29:32]: Yeah. And not to split hairs, I guess the overlay network is the TCP overlay network.Akshat [00:29:39]: The reason we have that is you need that to do the key exchange for RDMA before you set up the RDMA network on top of that. but then people found the TCP part.Swyx [00:29:48]: Can I tell you, this is like a big aha moment for me becauseAkshat [00:29:51]: YeahSwyx [00:29:51]: So I review 2,200 submissions for the World's Fair.Akshat [00:29:56]: Yeah.Swyx [00:29:57]: And then I got this from John OsterhoutAkshat [00:29:58]: HuhSwyx [00:29:59]: Who I don't know if. Do John Osterhout by name?Akshat [00:30:01]: The name sounds familiar.Swyx [00:30:02]: He published a. He's a well-known professor, published a lot of interesting software design books, and this is the talk he chose to submit, is on RDMA at Inference. And I'm like, you wouldn't think that this guy, who is like operating systems guy, would care about RDMA.Akshat [00:30:20]: I, it makes sense to me because I,Swyx [00:30:24]: This is the cloud, right? YeahAkshat [00:30:25]: Like, the way you move around your KV cache and how efficiently you can do it, how efficiently you move, your weights from your training GPUs to your inference GPUs in RL is there's a lot of degrees of freedom, and it is a systems problemSwyx [00:30:41]: YeahAkshat [00:30:41]: Moving memory aroundSwyx [00:30:42]: YeahAkshat [00:30:43]: Scheduling.Swyx [00:30:44]: This shows you how primitive my understanding of networking stuff is.Swyx [00:30:46]: Is this like the domain of WireGuard as well?Akshat [00:30:50]: Not quite.Swyx [00:30:51]: It's adjacent?Swyx [00:30:53]: Explain everything.Akshat [00:30:54]: Sure.Swyx [00:30:56]: How do we move memory around GPUs?Akshat [00:30:58]: Well, so sorry. Yeah, that is memory. Sorry, I was talking more, and maybe I was talking like five minutes back, about the private IPv6, addressing that you've set up.Swyx [00:31:09]: Yeah.Akshat [00:31:09]: Is it like it's a VPN?Swyx [00:31:10]: Yeah, it is like a VPN, and yeah, WireGuard is, yeah, you're right. It is,Akshat [00:31:16]: Right. Yeah, you already moved on to new topicsSwyx [00:31:17]: A similarAkshat [00:31:18]: OkaySwyx [00:31:19]: In the same space, WireGuard is, encrypted and this is,Akshat [00:31:23]: And you don't need encryption.Swyx [00:31:23]: Yeah.Akshat [00:31:24]: Yeah.Swyx [00:31:24]: This is not encrypted. that's the main difference. This is TCP and we have eBPF programs that will reject or allow the TCP connection based on whether you're allowed to do it.Akshat [00:31:35]: Used to involve a full sidecar, but now you have eBPF in the Linux kernel.Swyx [00:31:39]: Yeah.Akshat [00:31:40]: Yeah. I don't know if this is a natural follow-on to the topic of like my skepticism on distributed training is that while, like, people spend a lot of money on, like, cables to hook up GPUs, and even that is not, like, fast enough, and that's the bottleneck, is your networking fast enough?Swyx [00:31:59]: Yeah. So I guess you're talking about fully distributed training like, Dialog or something which is like cross data centerAkshat [00:32:06]: That would be, yes.Swyx [00:32:07]: That's the extreme.Akshat [00:32:08]: Yeah.Swyx [00:32:08]: You're in the middle, and then other people would have like the Mellanox cables up in, like, their actual data center.Akshat [00:32:14]: When you run multi-node training on Modal, RDMA, I think Mellanox, is, or InfiniBand is like a, is all seen as RDMA. but it's a way to bypass the TCP networking stack and, transfer, stuff much faster, between one node, to the other. And we have I think like 3 terabit per second, internal networkingSwyx [00:32:40]: OkayAkshat [00:32:40]: Which is the standard that's needed.Swyx [00:32:42]: Okay. So I misunderstood whatAkshat [00:32:43]: 50Swyx [00:32:43]: What part of the stack you wereAkshat [00:32:44]: 50 gigs overSwyx [00:32:45]: YeahAkshat [00:32:45]: If you wentSwyx [00:32:45]: YeahAkshat [00:32:46]: RDMA.Swyx [00:32:46]: Okay.Swyx [00:32:48]: Yeah. I, very impressive work.Multi-Node Training, Post-Training, and Auto ResearchSwyx [00:32:52]: So effectively you're extending like the model philosophy to the training cluster, like, yeah.Akshat [00:32:59]: Yeah. And we're, we're not going for like large scale training runs. the thing that we've built multi-node training for is, we see a lot of, smaller scale post-training. like, people are post-training like medium sized fund models, so they can, get higher quality on inference. this is a perfect fit, for something like that.Swyx [00:33:21]: Yeah. That is my impression of how a lot of these labs explore branches in post-training and then eventually merge whatever they find in.Akshat [00:33:31]: Yeah. The other use case we've seen for multi-node training is even if you have a big cluster, your researchers are still doing small runsSwyx [00:33:38]: YesAkshat [00:33:39]: Having elasticity thereSwyx [00:33:40]: Right, sureAkshat [00:33:40]: Matters a lot more.Swyx [00:33:41]: Yeah. the, like, this is like the current limiting factor for auto research, which is like you need to give your model some GPUs in order for it to completely run.Akshat [00:33:51]: We have a blog post on auto resource and model is,Swyx [00:33:55]: YeahAkshat [00:33:56]: Yeah, like, turns out to be pretty good substrate for that.Swyx [00:33:59]: So my impression is auto research means many things, likeAkshat [00:34:01]: YeahSwyx [00:34:01]: Anything that Andrej coins. Right now it's still science fair, right? Like not like, I don't know how many people are doing this.Akshat [00:34:08]: We're having a golf.Swyx [00:34:08]: Yeah.Akshat [00:34:09]: I thought the same thing.Swyx [00:34:11]: Yeah, you would know.Akshat [00:34:12]: We, like, our internal both training and inference teams use this the general shape of this quite a bit. like we have this one internal repo called auto inference, which essentially we've automated our own forward-deployed engineering efforts using, this harness, which is, the agent will just spin up a sweep of different things. It'll even run like, NVIDIA inside profiler and it'll like tweak configs and it'll arrive the right thing. it'll change your GPUs both from H200 to B200, and works really well.Swyx [00:34:47]: Nice.Akshat [00:34:47]: So yeah.Swyx [00:34:48]: By the way, I enjoy that your forward-deployed engineering is so technical that you have to do these things.Swyx [00:34:52]: It's very different from forward-deployed engineering from other people.Akshat [00:34:54]: Yeah. For our forward-deployed engineering team is, essentially they're like applied inference researchers or applied training researchers.Swyx [00:35:02]: Someone told me like they have to be able to build, but they also have to be able to sell. do they have to sell or are they like they're good, they're just like post-sale type of thing?Akshat [00:35:09]: It does, being able to talk to a customer and engage effectively with themSwyx [00:35:13]: YeahAkshat [00:35:13]: Matters a lot.Swyx [00:35:14]: They want the same thing.Akshat [00:35:15]: Yeah.Swyx [00:35:15]: ?Akshat [00:35:15]: But it's it's not really a sales, thing. We pair them with-- We have solution architects as well that are more on the sales side.Swyx [00:35:23]: Okay. Let's spend a bit more time on auto research. This is a big focus for for this year. Where does this go? like, have people explored enough? Like, there's all these beautiful charts of like improve and then level off a bit and then you find the next thing. Is this one abstraction up from normal training? Is that how we think about it, or do you think about it differently? Like model level training versus high, like driven hyperparameter search.Auto Inference and Modal BenchAkshat [00:35:51]: Yeah, like,Swyx [00:35:51]: Someone, some people call it like neural architecture search or whatever, right? Like.Akshat [00:35:54]: Yeah, - So the stuff I've seen people do with it is nowhere on the architecture level. It's pretty much tweaking parameters, but it's it's a hyperparameter sweep that's guided by some model intuition, so it's like much more efficient than, whatever other, sweep you would have.Swyx [00:36:12]: Yeah, it's just, it's just a question of where you want to spend your compute?Akshat [00:36:16]: Right.Swyx [00:36:16]: ‘Cause yeah, you can just throw infinite amounts of money on this and somehow you'll bang out Shakespeare?Akshat [00:36:22]: Yeah, infinite monkey.Swyx [00:36:24]: Yeah, so like the very good for model. and I think it's also very important that agents can spin up other agents, can spin up their infrastructure. Like very good for you. how good is our LLMs at generating model code? Like the benefit of existing LLMs is that you are in the data.Akshat [00:36:42]: Yeah. They're, they're surprisingly good. I think like pre Cloud 4 they were not, and then now they're able to shot, stuff out of the box. But we're playing around with releasing like a Modal Bench for like the harderSwyx [00:36:55]: YeahAkshat [00:36:55]: Things, that the LLMs cannot do yet and maybeSwyx [00:36:59]: What's an example of that?Akshat [00:37:01]: I think the things that- Sometimes agents struggle with, without right guidance and a skill is, how to, use the rest of our observability. Like how to. Something is failing, like how do you look at the logs and then update the right thing? It's reasoning about that. But they're able to shot, likeSwyx [00:37:23]: Yeah. You can just add a skill to it?Compute Strategy and Capacity PlanningAkshat [00:37:26]: Yeah. So we have a Modal skill now that. Which is why we built this Modal Bench. It's to find things like that, so we can address them in our tool.Swyx [00:37:35]: Tune a skill. Yeah.Akshat [00:37:36]: Yeah.Swyx [00:37:36]: No. it's it's good. are you facing any shortages? like we talk a lot about GPU shortages, but also CPU, also memory.Swyx [00:37:44]: Yeah.Akshat [00:37:45]: We have had a lot of growth, which means that, there's - we've had to be much better aboutSwyx [00:37:53]: PlanningAkshat [00:37:54]: Proactive capacity planning.Swyx [00:37:55]: Yeah.Akshat [00:37:55]: So we have,Swyx [00:37:57]: Which by the way, like it's like a MBA's like dreamAkshat [00:38:00]: YesSwyx [00:38:00]: Is like just planning this stuff. I think last time you and I talked about something maybe about this.Akshat [00:38:03]: Yeah. we have a really competent team of people that we call, The role is called compute strategy. so yeah, if anyone listening here or wants to work on thatSwyx [00:38:13]: Compute strategy?Akshat [00:38:13]: Yeah.Swyx [00:38:14]: I think,Akshat [00:38:14]: I feel like,Swyx [00:38:15]: I think the normies call it FP&A or something.Akshat [00:38:18]: Well, it's more It's it's not FP&A. It's it's There's a lot of interesting financial questions of like what is the blend between one year and three-year reservations? how do we forecast our own capacity? how do we. especially since our capacity is very fungible across different GPU types and different regions, like you have to model a lot of it. and you also have to have an opinion on how the supply chain is gonna evolve, and then you have to like, take bets,Swyx [00:38:49]: YeahAkshat [00:38:49]: Based on that.Swyx [00:38:50]: Tokenomics.Akshat [00:38:50]: Yeah.Swyx [00:38:51]: This is like probably a not a real point, but, I was trying to think about like what other industries. I was trying to think about like, we cannot be first to like these kinds of problems.Akshat [00:38:59]: Yeah.Swyx [00:39:00]: And what other industries have had this? And I was like, airlines with fuel and like they have to hedge their fuel and like, I think for a long time Southwest because they made like a hero fuel bet, they like were like super low cost becauseAkshat [00:39:12]: OhSwyx [00:39:12]: Compared to everyone else.Akshat [00:39:14]: Yeah. I hadn't thought about that.Vibhu [00:39:16]: We're at a fun time too?Akshat [00:39:18]: Yeah. It's. A lot of the compute business in general, for us is also about being very good about capacity management. That is how you have great unit, economics. but also over time it's how you can unlock more value for customers. Like, one of the things we're building now is like a way for customers to get, If they don't care about latency, like get much cheaper pricing and they'll get results back in like next 24 hours or something, like a batch tier essentially.Batch Tiers and Latency-Insensitive WorkloadsSwyx [00:39:47]: Yeah.Akshat [00:39:47]: And those are levers we have because we control the whole stack and scheduling and whatnot to give people a sufficientSwyx [00:39:53]: Yeah. I feel like they're not as popular. Like those, like the Frontier Labs have all those APIs. They're not as popular as they should be.Akshat [00:40:00]: The demand that we see for something like that is not for LLMs. although sometimes people wanna run evals andSwyx [00:40:08]: OkayAkshat [00:40:08]: Synthetic data prep and there it makes sense.Swyx [00:40:10]: Okay.Akshat [00:40:11]: But it's from a lot of LLM companies, like people who are doing computational bio, like they have to run really big batch jobs and they don't care about when they get it back.Swyx [00:40:22]: Yeah. And like they have a reasonable. It's it's also like a cousin to the stopping problem of like, will this finish in time?Akshat [00:40:30]: Yeah. You can bound it.Swyx [00:40:33]: Yeah.Akshat [00:40:33]: Like you can give peopleSwyx [00:40:34]: YeahAkshat [00:40:34]: SLAs on it.Swyx [00:40:35]: Yeah. I think what's, what's interesting is like the next phase of model.Swyx [00:40:38]: Like what, do people expect from you, now that you're established and you're like well-known compute player among all these leading companies. You had an inference launch week, and we talked a little bit about the launches. like what else? Like what else should people know?What Modal Builds NextAkshat [00:40:55]: We are building primitives that make our users' lives much easier. So, I think for example, with LLM inference, thousands more companies are gonna post-train their own models and, deploy open source models for inference. so we're thinking a lot about what is the best product shape for that. And, that involves everything from our training gym to, then, endpoints that get frontier-level performance. again, but I haven't talked to anyone. It looks somewhat different on other verticals. Like, we're also seeing a lot of real-time, audio-video stuff in there, which is why like, we're working on things like regional routing, with fallbacks. So you can get GPUs that are as close to users as possible. so you get like low latency for video streaming and whatnot. And then on the agent side, it's,Akshat [00:41:52]: We're still working very closely with our customers because stuff is changing so fast in terms of what they need. And, I think beyond sandboxes and persistent file systems, there's a lot of other things people will need from this agent stack as they build production agents. So yeah, we're thinking about those other things that fit in there.Swyx [00:42:13]: I want to ask what the other things are.Akshat [00:42:15]: Yeah. I probably should share right now.Swyx [00:42:17]: I think-- I think, okay, so, I do think a lot about the principal components of cloud, and you do talk about compute storage networking.Akshat [00:42:25]: Yeah.Swyx [00:42:25]: Because so far for me, it's fine. so far for the. the first couple generations of cloud, it's fine. What's different, qualitatively different about agents that you need some new permission level? Like a lot of people, okay, and I'll just kinda spew tokens at you until it like hopefully sparks something.Akshat [00:42:43]: Yeah.Swyx [00:42:44]: Like the new level now is whatever Claude Code does, which is dangerously scope permissions or like allow list by command or like whatever, right? And sometimes they're like, “Well, okay, we have like this adaptive thinking mode where like, just trust me, bro. I will make the calls for you.” Is that it? like mediated permissions.Hard Guardrails vs. LLM-Mediated PermissionsVibhu [00:43:03]: Now you're looping it with a goal and letting it roll.Akshat [00:43:06]: Yeah, I'm, I'm skeptical of LLM media permission for stuff that is at the sandbox level because you do want hard boundaries.Swyx [00:43:16]: Yeah.Akshat [00:43:16]: Otherwise, someone can exfiltrate stuff.Swyx [00:43:20]: But likeAkshat [00:43:20]: YeahSwyx [00:43:20]: Maybe that's old school thinking. Maybe we're the dinosaurs.Swyx [00:43:23]: Maybe the AI OS or the LLM OS is really the kernel is a goddamn LLM.Swyx [00:43:30]: Like it makes you feel uncomfortable.Akshat [00:43:31]: Yeah, I'm, I'm toldSwyx [00:43:32]: But that's what trusting the LLM is. Like imagine a spherical cow perfect LLM.Akshat [00:43:36]: Right.Swyx [00:43:37]: That it.Akshat [00:43:39]: Maybe.Swyx [00:43:41]: I wanna test the boundaries, right?Akshat [00:43:42]: Yeah.Swyx [00:43:42]: Like, and I don't believe that, but I wanna see where I'm wrong ‘cause that's, that's the consensus.Akshat [00:43:49]: Yeah. I think you always need hard guardrails when you want, And you can pair those with softer guardrails, right? And that's gonna be a lot of mediated.Managed Agents and Specialized SandboxesSwyx [00:44:00]: There. I'll also get you a end with a couple of your commentary on like the ecosystem outside of Modal. Manage agents. Everyone has one. Gemini, OpenAI, Claude, very useful for you, but also like it is their way of starting to edge into your space.Akshat [00:44:17]: Yeah.Swyx [00:44:17]: What's going on?Akshat [00:44:19]: Yeah, we're, very excited to partner with Anthropic and some of the other foundation labs, will not name who we're also working with. the way we see it is the manage agent thing is a great place to start if you're starting out building an agent and, But then when you get to, building something more production grade, like you're a company that's like Ramp that's building their own, Ramp also runs their accounting agent on us, so their external-facing agent. You need a lot more control over, your compute primitive on things like, what sort - how do you persist different files that the agent has access to, and how do you snapshot and restore? How do you control the networking? maybe you want GPUs. When you get to that point, you kinda want, a specialized sandbox provider, that gives you those things, and that's the role that we are trying to play.Swyx [00:45:15]: YeahAkshat [00:45:16]: We don't really have an opinion on the harness, whether it runs - it's a cloud-managed agent, and you hook it up to Model Sandbox, or you run the harness in Model Sandbox. We'll see where people converge with that.Swyx [00:45:26]: Yeah. Do you any opinions on like the meta harnesses, or just another layer on top of these things?Akshat [00:45:31]: You mean like the OpenPipeSwyx [00:45:33]: OpenPipe is one. I think Vercel had one, which I can't remember the name of right now. Fredshot had one. and then, to me, most recently was Data Databricks that had Omnigen. All these are meta harness. Like it's kinda pseudo agent cloud type things.Akshat [00:45:50]: I personally have not played around with them.Swyx [00:45:53]: Yeah.Akshat [00:45:53]: Build agents with them.Swyx [00:45:54]: Everything's bullish Modal, as long as it consumes more infra.Akshat [00:45:57]: That's why we're focusing on the infra layer. It's somewhere where our, relative competence is and, also it's a hard problem to solve.Swyx [00:46:06]: Yeah. I will say like just generally reflecting on that, I don't know if - if there's other topics on Modal, but like just generally reflecting as an infra person, not as intense as you, but in that field, this has like been the most exciting time in infra. Like it was boring for a while, and you couldn't really get people excited about data infrastructure. Like Eric would get on Data Console, everyone just watched the video and like say, “Look at how many sandboxes I can spin up,” and no one gave a crap.Why Infrastructure Became Exciting AgainAkshat [00:46:39]: Yeah.Swyx [00:46:40]: And like now everyone gives a crap.Akshat [00:46:42]: That's true. It is a very exciting time, and I think a lot of that's driven by just the amount of scale all of this stuff needs.Swyx [00:46:50]: I think the, like a lot of your initiatives or a lot of your like product directions make sense in retrospect, which is like the best kind, but I wouldn't necessarily have thought about it myself, which.Akshat [00:47:00]: We need the predictions.Swyx [00:47:02]: I think there's a lot that you just don't even see, right? Like you have the batch, you have the voice, you have the multimodal, but what else?Akshat [00:47:10]: What else is coming up for usSwyx [00:47:11]: Yeah. Where do you see things going?Akshat [00:47:13]: Yeah. I, in generalBiotech, Robotics, and Non-LLM AI WorkloadsAkshat [00:47:15]: It's it's clear that there's there's a huge shift happening. I think one thing that's not as obvious to people because LLM inference gets talked about so much and is also we work a lot of companies that are, doing things like drug discovery and computational bio, like the Chai Discoveries of the world. Big things are probably gonna happen there. we work a lot of robotics companies that are putting robots in like active deployments and getting good results out of them.Swyx [00:47:45]: Is there Air Gap Modal? Is there a version that is like prem air gapped whatever?Akshat [00:47:50]: No. We,Swyx [00:47:51]: You should cloud only.Akshat [00:47:51]: Yeah.Swyx [00:47:52]: Yeah. Okay. But yeah, so what you're saying is like because you're focused on primitives and they're good primitives, you find use cases in all these kinds of things.Akshat [00:48:01]: Yeah.Swyx [00:48:01]: Probably diversifies you a little bit away from LMS all the time.Akshat [00:48:05]: Yeah, absolutely. We're, we'- our goal isn't to only serve the LLM inference market.Swyx [00:48:10]: There are a lot just on the website, the audio,Akshat [00:48:12]: Yeah. We said both onSwyx [00:48:14]: Computational bio images. Yeah, there's a lot here. There's QTA TTS, customizing. Oh, Chatterbox. there was customizing Whisper.Akshat [00:48:24]: Okay. Yeah.Swyx [00:48:25]: This screen reminds me of a fallen competitor, which Replicate.Model APIs vs. Differentiated AI ProductsSwyx [00:48:31]: What's your postmortem on what happened?Akshat [00:48:34]: This is one thing we've stayed away from is providing an API for models because I think providing model APIs is some of it ends up serving like a really hobbyist market, which is much less sticky.Swyx [00:48:50]: Yeah.Akshat [00:48:50]: And we've always wanted to build for companies that are building products and need more flexibility that's not just an API.Swyx [00:48:57]: Which you can build an API for a model and this is clearly what it is. But you - but what you're saying, you can wrap it into a more fully functioning back end that you run.Akshat [00:49:06]: Yeah. So all of our examples, it's not that spin up this model, here's an API token, use it. They're all code.Swyx [00:49:13]: Okay.Akshat [00:49:13]: And so the point is that this is just an example.Swyx [00:49:16]: Starter code.Akshat [00:49:17]: Yeah. But you can tweak it however you want.Swyx [00:49:20]: Yeah.Akshat [00:49:21]: And if you're like a company building a product, like, computational bio whatnot, yeah.Swyx [00:49:26]: I guess I'm trying to tease out for listenersAkshat [00:49:28]: YeahSwyx [00:49:28]: When does it stop becoming, oh, you're just an API call and you're just a wrapper on API to becoming what you call a product, right?Swyx [00:49:36]: Like, what is that layer? Like what-- Like, more lines of code, but like beyond that, what is the substance that people add that qualifies it to be something more?Akshat [00:49:46]: I think there's a little bit of like a selection effect of like a lot of the companies who do wanna get deeper into that level are probably building something that's more differentiated. And, I think, an example is like - with LLM inference, originally we, worked with companies that were building their own post-training frameworks or they were, - Ramp early in the day was training their own tokenizer and like swapping out the tokenizer in Llama and whatnot. I'm not saying that's, that successful, in that case. But a better example is like, let's say Suno. because Suno, does not use Modal for training.Swyx [00:50:26]: Mikey on the pod. Yeah.Akshat [00:50:27]: But they use Modal for all their inference and that's because they have like a custom-- They have completely custom model architecture and that means that they have to be at the code level and tweak things that are not, just an API.Swyx [00:50:41]: It's interesting as well, like we had, Ethan, most recently on the xAI Groq team make a prediction that like the next tier in video gen is not a better video model, it's a better model or agent that orchestrates video models.Video Agents and Production WorkflowsAkshat [00:50:56]: Oh, interesting.Vibhu [00:50:56]: Language model backbone that can use toolsAkshat [00:50:58]: RightVibhu [00:50:59]: And write code.Akshat [00:51:00]: Like, yes, I can make my second video or my second video from Groq, but I want my minute video.Akshat [00:51:06]: And I'm not going there through normal video gen.Swyx [00:51:10]: Yeah, that's interesting. I - So we have GPU sandboxes and recently have seen a few companies doing agents that do video manipulation or,Akshat [00:51:22]: Yeah. Give it FFmpeg and just do it.Swyx [00:51:23]: Run FFmpeg. But likeAkshat [00:51:25]: That's not enough.Swyx [00:51:25]: Yeah.Akshat [00:51:26]: You need to give it Adobe.Swyx [00:51:27]: Yeah, I hadn't put it together with like it would be a video production thing. in my mind these things were going more towards editingAkshat [00:51:36]: Yeah.Vibhu [00:51:36]: Well, shout out Mantis.Akshat [00:51:37]: I think about this a lot.Swyx [00:51:38]: .Akshat [00:51:41]: Yeah. Sorry.Vibhu [00:51:41]: Luma. Luma Agent is a version of this for video production, but it's a off.Swyx [00:51:46]: I was gonna get your quick takes, on some other stuff that happensGitpod/Ona, CI, and Runtime SandboxesSwyx [00:51:50]: In recent news and just-just see if you have anything interesting. Gitpod, very li

In this episode, recorded live at KubeCon, Ronald and Jan talk with Jannis Relakis and Michael Seiwald-McCarty, both senior platform engineers at Celonis. Celonis manages over 150 Kubernetes clusters across GKE, AKS, and EKS, but it wasn't always that clean. They started with six different Kubernetes flavors, including Gardener, K-Ops, and OpenShift (both Rosa and ARO), spread across multiple cloud providers.In their KubeCon talk "No Shame in Just Paying," they shared how they tackled this consolidation project: migrating all workloads to three standardized, fully managed Kubernetes distributions. Key topics include their self-built cross-cluster connectivity tool called "Wormhole" (powered by Envoy's dynamic forward proxy), RabbitMQ federation for seamless message queue migration, and how they used Karpenter and Cilium to align node and network management across clouds.They also get candid about what went wrong: an accidental ArgoCD sync that caused a 10-minute full environment outage, the pain of "snowflake" environments (including one requiring full HIPAA compliance with Istio mTLS), and the constant fight against scope creep that threatened to derail the entire project.The episode closes with a forward-looking discussion on FinOps, resource rightsizing, the future of VPA, and whether Kubernetes and serverless can ever truly converge.Powered by ACC ICTStuur ons een bericht.ACC ICT Specialist in IT-CONTINUÏTEIT Bedrijfskritische applicaties én data veilig beschikbaar, onafhankelijk van derden, altijd en overalSupport the showLike and subscribe! It helps out a lot.You can also find us on:De Nederlandse Kubernetes Podcast - YouTubeNederlandse Kubernetes Podcast (@k8spodcast.nl) | TikTokDe Nederlandse Kubernetes PodcastWhere can you meet us:EventsThis Podcast is powered by:ACC ICT - IT-Continuïteit voor Bedrijfskritische Applicaties | ACC ICT

AWS Morning Brief for the week of July, 6th with Corey Quinn. Links:Announcing general availability of Amazon WorkSpaces for AI agentsAmazon CloudWatch supports creating alarms from log queriesECS Service Connect now supports Zone-Aware routingHow InterWiz reduced AI costs by 90% with Amazon BedrockAccelerate your infrastructure deployments by up to 4x with AWS CloudFormation Express modeAutomate public TLS certificate issuance with ACME support in AWS Certificate ManagerUpgrade Amazon EKS clusters with confidence using Kubernetes version rollbacksSafely Releasing Frontier Models to CustomersAccelerating government FinOps with Amazon QuickFour CVEs: AWS reads half your request, leaks the rest

В этом выпуске: цапаем АЦП, федерируем и тестируем Kubernetes, мелем кофе и разбираем темы слушателей. Шоуноты: [00:06:15] Чему научились за неделю Home | mise-en-place Android audio input as source · Issue #1557 · AlexandreRouma/SDRPlusPlus · GitHub UCA202 | Behringer SEGA оригинал, как отличить от клона в 2026 году? — YouTube [01:03:53] Federating Clusters for Zero-Downtime… Читать далее →

Pat Casey was the first person besides founder Fred Luddy to write code at ServiceNow back in 2005, when it was called Glide and lived above a friend's restaurant. Twenty years later, he's CTO of a company where 85% of the Fortune 500 are customers, and until recently ran all of engineering: 10,000 people, 7,000 of them writing code. Almost nobody survives the journey from first engineer to public-company CTO. Pat did. Tobi and Pat dig into how ServiceNow actually works under the hood: a metadata processing engine running 90,000 single-tenant databases and over 25 billion queries an hour, why they bought a 15-person German database company and turned it into RaptorDB, and why tearing apart a 20-year-old monolith is harder than every senior engineer thinks. Then the conversation turns to AI. Pat bought 7,000 Windsurf licenses and measured a real, but unglamorous, 15% productivity bump, with a small subset of engineers going 5–6x while most barely changed. His thesis: AI coding is like playing five chessboards at once, and it's reshuffling the deck on who the top engineers will be. On agents, ServiceNow's answer is disarmingly simple: create a user called "AI Pat," assign it cases, and make it follow the exact same rules as humans because you should not trust an LLM more than you trust a human being. Topics covered: - From Atari 400 and floppy-disk jockey at Aldus to first engineer at ServiceNow - Scaling engineering from a stuffed fish on a monitor to 10,000 people — and the productivity trough at ~100 engineers - Single-tenant architecture: 90,000 databases, 25B+ queries/hour, and the monolith-to-Kubernetes migration - Why ServiceNow bought Swarm64 and built RaptorDB on a Postgres fork - 7,000 Windsurf licenses, Claude Code, and the real numbers on AI coding productivity - "AI Pat": the anthropomorphic model for enterprise agents outcomes, not toolkits - Whether AI kills seat-based SaaS, and why incumbents may have the inside track - Pat's advice to CTOs: this is not a time for excessive caution

Malcolm Matalka joins William and Eyvonne to challenge the narrative that Infrastructure as Code (IaC) is dead. Malcolm argues that the real value of IaC was never the syntax, but state and governance. Together they examine whether the state was a file problem at all, or a distributed systems problem in a JSON costume. Episode... Read more »

Malcolm Matalka joins William and Eyvonne to challenge the narrative that Infrastructure as Code (IaC) is dead. Malcolm argues that the real value of IaC was never the syntax, but state and governance. Together they examine whether the state was a file problem at all, or a distributed systems problem in a JSON costume. Episode... Read more »

This week on Shared Security, Tom and Kevin sit down with Jay Beale — founder of InGuardians, long-time Black Hat trainer, creator/contributor behind Kubernetes security training, and part of the team behind the DEF CON Kubernetes CTF. Jay shares stories from decades of offensive security work, including the time Tom hired him for a physical penetration test and Jay somehow ended up inside a call center instead of stuck in the lobby. The crew also digs into what makes good security training, why Kubernetes is such a natural platform for both defenders and attackers to understand deeply, and how the DEF CON Kubernetes CTF is designed to be welcoming for both competitors and learners. The episode closes with a practical look at AI infrastructure risk. Jay explains how production AI stacks running on Kubernetes can be attacked like any other cluster — and how modifying a vector database behind a RAG system can turn indirect prompt injection into a persistent, high-impact attack path.** Links mentioned on the show **Jay's Black Hat USA Course: Agentic AI-aided Kubernetes Attack and Defensehttps://blackhat.com/us-26/training/schedule/index.html?day=4daysattue#agentic-ai-aided-kubernetes-attack-and-defense-51318Jay Beale on LinkedInhttps://www.linkedin.com/in/jaybeale/InGuardianshttps://www.inguardians.com/DEF CONhttps://defcon.org/** Watch this episode on YouTube **https://youtu.be/aMHk62dprDA** Become a Shared Security Supporter **Get exclusive access to bonus episodes, listen to new episodes before they are released, receive a monthly shout-out on the show, and get a discount code for 15% off merch at the Shared Security store. Become a supporter today by going to our YouTube channel's membership section: https://www.youtube.com/channel/UCg9CCDIYkDDqwEZ3UYaxjnA/join** Thank you to our sponsors! **SLNTVisit slnt.com to check out SLNT's amazing line of Faraday bags and other products built to protect your privacy. As a listener of this podcast you receive 10% off your order at checkout using discount code "sharedsecurity".** Subscribe and follow the podcast **Subscribe on YouTube: https://www.youtube.com/c/SharedSecurityPodcastFollow us on Bluesky: https://bsky.app/profile/sharedsecurity.bsky.socialFollow us on Mastodon: https://infosec.exchange/@sharedsecurityJoin us on Reddit: https://www.reddit.com/r/SharedSecurityShow/Visit our website: https://sharedsecurity.netSubscribe on your favorite podcast app: https://sharedsecurity.net/subscribeSign-up for our email newsletter to receive updates about the podcast, contest announcements, and special offers from our sponsors: https://shared-security.beehiiv.com/subscribeLeave us a rating and review: https://ratethispodcast.com/sharedsecurityContact us: https://sharedsecurity.net/contact

Most engineers think reliability means avoiding outages. Vlad Leyberov learned the opposite lesson: sometimes you have to intentionally cause a 100% outage to fix the system faster.Vlad is a Site Reliability Engineer (SRE) at Google, running systems that handle billions of requests per second. Before Google, he kept critical infrastructure running at Meta (billions of events a day) and Amazon (millions of Alexa devices).In this conversation, we dig into cascading failures, incident responses, why consistency beats speed, how AI changes reliability engineering, and the philosophy behind running systems where downtime doesn't feel like an option.Topics Discussed:How cascading failures propagate unpredictably in distributed systems (like nature, not machines)Incident responses: virtual panic rooms, on-call, paging procedures, and how to narrow down failure pointsThe Alexa incident: why dropping an entire DynamoDB table was the right callCritical User Journeys (CUJ): measuring end-to-end customer experience vs individual SLOsCareer journey from the USSR to maritime academy to business degree in Australia to SRE at Amazon, Meta, and GoogleWhy consistency in API response times beats raw speedHow AI makes it dangerously easy to create complex systems with poorly understood interactionsScience fiction, the Borg as a distributed system, and the Three Body Problem trilogyHot takes on reliability: all software development is maintenance, overrated 9s, underrated global failure modesGeneral Podcast LinksWatch: https://www.youtube.com/@alexasinput Read: https://alexasinput.substack.com/ Listen: https://creators.spotify.com/pod/profile/alexagriffith/ More: https://linktr.ee/alexagriffithLearn more about the hostWebsite: https://alexagriffith.com/ LinkedIn: https://www.linkedin.com/in/alexa-griffith/Find out more about Vlad LeyberovLinkedIn: https://www.linkedin.com/in/vladleyberov/ Google SRE NYC Tech TalksResourcesGoogle SRE Resources:Google SRE Book: https://sre.google/books/Google Cloud Platform: https://cloud.google.com/Google Cloud Build: https://cloud.google.com/build (service discussed in outage story)Google Cloud Pub/Sub: https://cloud.google.com/pubsub (Vlad's previous role, billions of requests/second)Sci-Fi Books Mentioned:Three Body Problem trilogy by Liu Cixin (Vlad's current favorite)Foundation series by Isaac AsimovLeft Hand of Darkness by Ursula K. Le GuinSnow Crash by Neal StephensonInternal Google Systems Referenced:Borg: Google's internal cluster management system (Kubernetes predecessor), named after Star Trek BorgDynamoDB: AWS distributed key-value store (used in Alexa poison pill incident)Intro Music:PR1BVOV7R4F1ASZC

This interview was recorded for GOTO Unscripted in May 2026.https://gotopia.techRead the full transcription of this interview here:https://gotopia.tech/articles/445Jake Warner - Co-Founder & CEO at Cycle  @JakeWarner  Charles Humble - Freelance Techie, Podcaster, Editor, Author & ConsultantRESOURCESJakehttps://bsky.app/profile/jakewarner.comhttps://x.com/jakewarnerhttps://github.com/JakeWarnerhttps://www.linkedin.com/in/jakewarnerhttps://jakewarner.comCharleshttps://bsky.app/profile/charleshumble.bsky.socialhttps://linkedin.com/in/charleshumblehttps://mastodon.social/@charleshumblehttps://conissaunce.comDESCRIPTIONJake Warner, co-founder and CEO of Cycle.io, traces a pattern he's watched repeat itself since his OpenStack days: a new orchestration technology arrives, developers adopt it enthusiastically, it grows in complexity, and organizations eventually ask whether managing it is really a core competency. He made a decade-long bet that Kubernetes would follow the same arc — and built Cycle as the answer: a distributed control plane that lets companies own their own infrastructure and compute while still getting a clean, platform-like experience on top of it. The key design principle is a high ceiling without sacrificing simplicity — companies shouldn't have to re-platform every time they grow, and they shouldn't have to give up infrastructure ownership to get ease of use.The conversation then pivots to Sovereign Cloud, which Jake frames not as a niche regulatory concern but as a fundamental trust and ownership question. He draws attention to a risk many organizations underestimate: the control plane itself. Unlike many platforms where the control plane is a single point of failure and a blackbox, Cycle's architecture ensures that even if the control plane goes down, customer infrastructure keeps running — servers maintain their own manifests and restart containers independently. Looking ahead, Jake expects more regions and countries to build their own cloud equivalents, driven by privacy concerns, data residency laws, and geopolitical pressures that are accelerating faster than the technology is.His conclusion: the organizations that handed AWS all the keys are beginning to realize the cost — and the industry is correcting.RECOMMENDED BOOKSAlan Hamilton • Sovereign Cloud Operations • https://amzn.to/4dqYAe4Leonard J. Horta • The Cloud Exit Strategy • https://amzn.to/4dXqeiNCharles Curry & Tanessa Curry • Cooling the Cloud • https://amzn.to/4a0d4iALiz Rice • Container Security • https://amzn.to/3oU4iJeLiz Rice • Kubernetes Security • https://www.oreilly.com/library/view/kubernetes-security/9781492039075BlueskyInstagramLinkedInFacebookCHANNEL MEMBERSHIP BONUSJoin this channel to get early access to videos & other perks:https://www.youtube.com/channel/UCs_tLP3AiwYKwdUHpltJPuA/joinLooking for a unique learning experience?Attend the next GOTO conference near you! Get your ticket: gotopia.techSUBSCRIBE TO OUR YOUTUBE CHANNEL - new videos posted daily!

#356: Fleet management means one thing to a DevOps engineer and something completely different to Tomas Kovacovsky. To Viktor it is a CD problem - a fleet of Kubernetes clusters he would rather not babysit. To Tomas it is hundreds of physical robots rolling around a warehouse, picking orders, dodging each other, and working very hard not to lose their connectivity. Tomas is the CTO of Brightpick, where the robots are not the kind you yell at for bumping into a chair. They are three-meter-tall autonomous pickers - some telescoping up to six - that find their way using lidar, recognize items with neural networks, and make their own decisions the second the network drops. Here is the part that will feel oddly familiar: everything you already do to ship software shows up again in the physical world. Canary rollouts. Rollbacks to the last good config. Prometheus scraping every robot, Grafana for the fleet. Logs, metrics, traces. Split brain, when a robot and the server disagree about what just happened. Even a flaky robot - one that feels off with no error to point at - gets diagnosed the same way you would hunt a flaky test: compare it against the rest of the population and find the outlier. A warehouse full of robots, running like a distributed system. The stack is what you would guess and also not. C++ on the robots for speed, Python on the backend, Kubernetes on an edge server inside the warehouse because latency matters down to the millisecond, and Git as the source of truth - the on-site servers check for differences and update themselves. GitOps, for robots. Then it gets bigger. The optimal pick speed, Tomas says, is infinity - right up until you try to pick an egg. The real bottleneck was never the picking, it was the traveling, so Brightpick moves the picking into the aisles instead of hauling totes back to a station. He also drops a prediction worth chewing on: the intelligence arrives before the dexterity. Machines will think their way around a warehouse long before they can fish for keys in a bag the way your hand does without looking. And the jobs question everyone braces for - the robot guys walking in, are you fearful for your job in 20 minutes - turns out the picker positions were mostly empty to begin with. Hundreds of thousands of them, unfilled. The takeaway for anyone writing software is the one Tomas lands at the end. The craft is getting eaten. What is left, and what actually matters, is whether you can connect the work to the product.   Tomas' contact information: LinkedIn: https://www.linkedin.com/in/tomas-kovacovsky-46411280/   YouTube channel: https://youtube.com/devopsparadox   Review the podcast on Apple Podcasts: https://www.devopsparadox.com/review-podcast/   Slack: https://www.devopsparadox.com/slack/   Connect with us at: https://www.devopsparadox.com/contact/

What if you didn't have to solve problems? You can just remove them entirely. Today, we're talking to Steve Francis, CEO at Sidero Labs, about why the most dangerous thing running in your infrastructure might be the operating system itself. We discuss how eliminating features rather than adding them is the real path to security, why the promise of multi-cloud portability turned out to be a lesson in what customers actually care about, and why "extreme ownership" remains the most empowering philosophy a leader can adopt. All of this right here, right now, on the Modern CTO Podcast!  To learn more about Sidero Labs, check out their website here.

JDK 26 optimise la JVM dans ses moindres recoins, le SDK Java d'Agent2Agent passe en 1.0, Micronaut 5 est là. Côté terrain, un retour d'expérience après 40 jours à coder avec 100 % d'IA : génie ou junior, Alzheimer numérique et dette technique invisible. Pendant ce temps, GitLab restructure, Microsoft suspend ses licences Claude Code, et un développeur injecte un prompt destructeur dans sa lib JUnit. La révolution IA a un coût et les boites commencent à s'en rendre compte. Enregistré le 12 juin 2026 Téléchargement de l'épisode LesCastCodeurs-Episode-341.mp3 ou en vidéo sur YouTube. News Langages Les améliorations de performance dans le JDK 26 https://inside.java/2026/06/09/jdk-26-performance-improvements/ Côté bibliothèques, l'API LazyConstant (anciennement StableValue) fait son entrée en prévisualisation pour permettre une initialisation paresseuse, sécurisée pour les threads et optimisée par le mécanisme de constant-folding de la JVM. L'extraction de chaînes de caractères via MemorySegment::getString a été revue pour réduire considérablement les allocations intermédiaires et les copies en mémoire off-heap, accélérant fortement les traitements sur les chemins critiques (hot paths). La méthode générée automatiquement hashCode() pour les classes de type record a été optimisée par la JVM pour atteindre un niveau de performance équivalent à une implémentation écrite manuellement. Le ramasse-miettes G1 bénéficie du JEP 522 qui redessine sa table de cartes (card-table) afin de réduire les coûts de synchronisation des barrières d'écriture, offrant un gain de débit de 5 % à 15 % sur les applications manipulant énormément de références d'objets. Grâce au JEP 516 (Project Leyden), le cache d'objets Ahead-of-Time (AOT) adopte un format de flux agnostique, ce qui lui permet d'être compatible avec n'importe quel Garbage Collector, y compris le ramasse-miettes à très faible latence ZGC. Le démarrage de la JVM s'accélère par défaut lorsqu'aucune taille de tas n'est configurée, car HotSpot n'applique plus de pourcentage initial (InitialRAMPercentage) mais démarre directement avec la taille minimale (MinHeapSize) pour éviter d'allouer des métadonnées inutiles. Les threads virtuels gagnent en robustesse en étant désormais capables de céder la main (yield) pendant les phases d'initialisation des classes, éliminant ainsi le risque de famine des threads porteurs (carrier threads). Le compilateur C2 JIT améliore son modèle de coût pour la vectorisation des boucles (SIMD) et se montre maintenant capable de compiler et d'optimiser des méthodes dotées de listes de paramètres extrêmement longues. Librairies Release candidate du A2A Java SDK supportant versions 0.3 et 1.0 en même temps https://medium.com/google-cloud/a2a-java-sdk-1-0-0-cr1-released-f0c651ec9139 Dernière étape avant la GA : Toutes les fonctionnalités prévues pour la version 1.0 sont finalisées. Migration simplifiée depuis la Beta1. Compatibilité v0.3 : Ajout d'une couche de compatibilité permettant aux agents v1.0 de communiquer avec les systèmes v0.3 (via JSON-RPC, gRPC ou REST). Support natif pour Android (nouvel AndroidHttpClient). Uniformisation des clients HTTP pour garantir une cohérence entre les versions. Nouveau parseur SSE (Server-Sent Events) conforme aux spécifications. Ça y est, le SDK Java de l'Agent 2 Agent Protocol est sorti en version 1.0 finale ! (avec compatibilité v0.3 et v1.0) https://medium.com/google-cloud/a2a-java-sdk-1-0-0-final-released-10c05b6aee34 Lancement officiel : Sortie de A2A Java SDK 1.0.0.Final, la première version stable (GA) du protocole Agent2Agent. Objectif du protocole : Standard ouvert (Linux Foundation) permettant aux agents IA de communiquer, déléguer des tâches et collaborer, indépendamment du langage ou du framework. Interopérabilité : Introduction de l'Integration Test Kit (ITK) pour valider la compatibilité entre les SDK (Java, Python, TypeScript, etc.). Transports supportés : Support complet et équivalent pour JSON-RPC, gRPC et HTTP+JSON/REST. Alignement total avec la spécification A2A 1.0.0. Passage aux Java records pour l'immutabilité et moins de code répétitif. Architecture interne basée sur un MainEventBus pour garantir la persistance et éviter les conditions de concurrence. Intégration d'OpenTelemetry pour le suivi et la surveillance. Support d'Android et compatibilité descendante avec la version 0.3. Installation : Gestion des dépendances via Maven BOM (org.a2aproject.sdk). Sortie de Micronaut 5.0 https://micronaut.io/2026/05/20/micronaut-framework-5-0-0-released/ Lancement majeur : Disponibilité générale de Micronaut 5, incluant une refonte de plus de 70 modules et la plateforme BOM. Baselines techniques : Support de Java 25, Groovy 5, Kotlin 2.3 et GraalVM 25.0.3. Optimisations internes : Amélioration significative des performances au démarrage et réduction de la surcharge à l'exécution via une refonte du conteneur IoC et du traitement à la compilation. Architecture HTTP : Support stable de HTTP/3, nouvelle API de formulaires (multipart) et annotations de nullabilité (JSpecify) pour une meilleure interopérabilité Kotlin/IDE. Configuration : Nouveau système d'importation de configuration (remplaçant le Bootstrap Configuration) et validateur de schéma JSON intégré. Fiabilité : Nouvelles API programmatiques pour les politiques de retry et circuit breaker. Sécurité & Outils : Mise à jour majeure des dépendances (Jackson 3, Ktor 3), rafraîchissement du Panneau de contrôle et diagnostics AOT améliorés. Écosystème : Mises à jour complètes pour les bases de données (Data, SQL, R2DBC, MongoDB, Redis), le cloud (AWS, Azure, GCP, OCI) et les tests (JUnit 6, Testcontainers 2.0). Évolutions notables : Intégration HTMX dans Micronaut Views, retrait du support RxJava 2 et migration de divers processeurs d'annotations vers des modules dédiés. Comment rajouter un agent IA dans une app Android, avec le tout nouveau framework ADK pour Kotlin https://glaforge.dev/posts/2026/05/21/wiring-adk-kotlin-agents-in-an-android-application/ Guillaume a participé au développement et au lancement du nouveau runtime ADK pour Kotlin et Android https://developers.googleblog.com/adk-kotlin-android-building-ai-agents/ Tutoriel sur comment intégrer un agent ADK dans une app Dépendances : Ajout du noyau ADK (google-adk-kotlin-core) et du processeur KSP dans build.gradle.kts. Sécurité API : Utilisation de local.properties pour stocker la clé API Gemini et l'exposer via BuildConfig afin d'éviter le hardcoding. Définition de l'agent : Création d'un objet LlmAgent configuré avec le modèle Gemini, des instructions spécifiques et des outils (ex: GoogleSearchTool). Utilisation de InMemoryRunner pour gérer automatiquement le contexte et l'historique de la session. Implémentation de runAsync avec StreamingMode.SSE pour un retour en temps réel dans l'interface. Threading : Exécution des requêtes réseau sur Dispatchers.IO et mise à jour de l'état de l'interface utilisateur sur Dispatchers.Main. Comment développer et hoster des agents IA sur la plateforme d'agents managés de DeepMind https://glaforge.dev/posts/2026/05/21/managed-agents-with-the-gemini-interactions-java-sdk/ L'équipe DeepMind de Google a lancé une plateforme d'agents managés sur son API Gemini Interactions https://blog.google/innovation-and-ai/technology/developers-tools/managed-agents-gemini-api/ Guillaume a implémenté un SDK Java pour utiliser cette API Gemini Interactions, qui donne entre autre accès à tous les modèles mais aussi à cette plateforme managée d'agents IA Agents managés : Permet d'exécuter des agents autonomes qui raisonnent, planifient et exécutent du code dans des environnements isolés (sandboxes), sans gestion d'infrastructure par le développeur. Environnement distant : Utilise des espaces de travail Linux éphémères dans le cloud via le paramètre remote, permettant l'accès réseau et la persistance des fichiers sur plusieurs appels. Agents prédéfinis : Accès immédiat à des agents spécialisés comme deep-research-pro (recherche multi-étapes) ou antigravity (tâches de codage généralistes). Agents personnalisés : Possibilité de configurer ses propres agents avec des instructions système dédiées, des outils spécifiques (exécution de code, recherche Google) et des règles réseau (egress) personnalisées. Architecture basée sur les étapes (Steps) : Utilise une structure de données typée (Step, Content) pour suivre le raisonnement de l'agent, ses appels de fonctions et ses résultats en temps réel. Outils et Schémas : Inclut des utilitaires pour générer des schémas JSON complexes via une interface fluide (DSL), par réflexion Java ou par parsing JSON. Streaming réactif : Support natif des événements en temps réel (SSE) pour suivre la progression de l'agent et recevoir les deltas de contenu au fur et à mesure de la génération. Flexibilité : Fournit un gestionnaire de routage (InteractionsHandler) pour créer facilement des serveurs proxy ou des backends intermédiaires traitant les interactions Gemini. Spring Boot 4.1 https://github.com/spring-projects/spring-boot/wiki/Spring-Boot-4.1-Release-Notes Support natif pour Spring gRPC permettant de créer et tester facilement des applications clientes et serveurs basées sur Netty ou des Servlets via HTTP/2 Introduction du lazy fetching pour les connexions JDBC via la propriété spring.datasource.connection-fetch=lazy afin de ne prendre une connexion du pool que lorsqu'un Statement est réellement exécuté Amélioration de l'auto-configuration de Jackson permettant de définir globalement les contraintes de lecture/écriture pour les formats JSON, XML et CBOR via des propriétés de configuration Sécurisation des clients HTTP bloquants et réactifs face aux attaques SSRF grâce à l'introduction d'un InetAddressFilter bloquant les requêtes sortantes vers des adresses spécifiques Améliorations majeures autour d'OpenTelemetry avec le support complet des variables d'environnement OTel, la possibilité de désactiver le SDK via une propriété globale et l'ajout du support SSL sur les exporters OTLP Ajout de l'auto-configuration pour l'utilisation de Spring Batch avec MongoDB incluant un nouveau starter dédié spring-boot-batch-data-mongo Auto-configuration des endpoints @RedisListener sans nécessiter la déclaration manuelle d'un RedisMessageListenerContainer Dépréciation du support de Apache Derby (projet arrêté), suppression définitive du mode layertools du JAR et réintroduction du support de Spock 2.4 (avec Groovy 5) Upgrade des dépendances majeures de l'écosystème avec notamment Spring Framework 7.0.8, Spring Security 7.1.0 et Micrometer 1.17.0 Outillage Vous êtes plutôt endive ou chicorée ? La librairie Chicory qui permet d'exécuter du code WASM à partir de son application Java est forkée et rejointe la Bytecode Alliance pour continuer son développement https://bytecodealliance.org/articles/endive-and-the-next-chapter-of-webassembly-on-the-jvm Annonce d'Endive : Nouveau projet hébergé par la Bytecode Alliance ; fork de Chicory (moteur WebAssembly pur Java, sans dépendance native). ​Objectif principal : Permettre aux développeurs Java d'intégrer, charger et déployer des modules Wasm nativement via les workflows Java habituels. ​Compilateur "Redline" : Intégration à venir de Redline (basé sur Cranelift) pour compiler le Wasm en code machine natif ; performances comparables à Rust/Wasmtime. ​Zéro dépendance (Java 25+) : Grâce à l'API standard Foreign Function & Memory (Project Panama), l'exécution à vitesse native se fait sans composants externes. ​Modèle de Composants (Component Model) : Support futur prévu pour consommer des composants (Rust, Go, JS, etc.) via des interfaces typées et sécurisées directement dans la JVM. ​Prochaines étapes : Fusion de Redline, conformité stricte aux specs Wasm (dont WasmGC) et amélioration du support WASI. Un visualisateur de sessions de travail avec Antigravity https://glaforge.dev/posts/2026/06/11/antigravity-brain-visualizer/ Un projet open source construit avec Micronaut, LangChain4j et GraalVM pour analyser les sessions de travail avec l'outil de développement agentique Antigravity (de Google) Analyse toutes les étapes, les requêtes utilisateur, les outils utilisés, les erreurs rencontrées, les réponses du modèle Gemini fait une analyse pour comprendre les moments clés de cette session de travail Outil buildé avec l'aide d'Antigravity lui-même SBX-Kits : des environnements de développement simplifiés pour les débutants (et les autres) https://k33g.org/20260501-sbx-kits.html Philippe Charrière (:whale: ) présente SBX-Kits (Sandbox Kits), une initiative personnelle visant à simplifier radicalement la mise en place d'environnements de développement pour les débutants, en éliminant la complexité d'installation des outils traditionnels. Chaque "kit" est une archive prête à l'emploi contenant un outil de développement spécifique (comme un langage, un framework ou une base de données) configuré pour s'exécuter de manière isolée et portable. La philosophie du projet repose sur le principe de "zéro configuration" et "zéro dépendance globale", permettant de tester une technologie ou de commencer à coder immédiatement sans polluer son système d'exploitation. L'approche technique s'appuie sur des scripts légers et des binaires portables pré-packagés, offrant une alternative plus simple et moins gourmande en ressources que les conteneurs Docker ou les configurations d'IDE complexes pour l'apprentissage. L'objectif à terme est de proposer un catalogue de kits couvrant les technologies courantes (JavaScript, Python, petites bases de données) pour faciliter les ateliers de programmation et le prototypage rapide. De nombreux kits sont disponibles sur https://github.com/docker/sbx-kits-contrib ghui: une interface utilisateur en ligne de commande (TUI) interactive pour GitHub https://github.com/kitlangton/ghui ghui est un outil en ligne de commande (TUI) écrit en Rust qui fournit une interface visuelle, interactive et rapide directement dans le terminal pour interagir avec GitHub. Il permet de gérer ses pull requests, ses issues et ses notifications sans avoir à ouvrir son navigateur web ou à taper de longues commandes avec la CLI officielle de GitHub. L'outil propose une navigation fluide au clavier, des raccourcis efficaces, et permet de réaliser des actions courantes comme valider une PR, ajouter des commentaires, attribuer des reviewers ou inspecter les logs des GitHub Actions. Conçu pour être extrêmement réactif, ghui s'intègre naturellement dans le flux de travail des développeurs adeptes du terminal et du mode "sans souris". Sortie de Homebrew 6.0.0 https://brew.sh/2026/06/11/homebrew-6.0.0/ Introduction du mécanisme de sécurité Tap Trust : comme les dépôts tiers (taps) peuvent exécuter du code Ruby arbitraire non sandboxé sur la machine, Homebrew demande désormais une confiance explicite de l'utilisateur avant d'évaluer ou d'exécuter leur code. L'API JSON interne devient le choix par défaut, offrant un système plus léger et beaucoup plus rapide pour les développeurs. Sécurisation renforcée de l'environnement avec l'implémentation du sandboxing sur Linux. Évolution des comportements par défaut basés sur un sondage utilisateur : le mode "ask" est activé par défaut pour les développeurs, affichant un résumé des dépendances et une demande de confirmation avant toute action de brew install ou brew upgrade. Améliorations notables des performances globales, notamment un boost de ~30 % sur la vitesse de la commande brew leaves et la parallélisation de la récupération des bottles (binaires) lors des mises à jour. Ajout du support initial pour la prochaine version d'Apple, macOS 27 (Golden Gate). Multiples optimisations pour brew bundle, incluant une gestion plus sécurisée des installations de paquets npm. Méthodologies Retour d'expérience très détaillé et 100% humain sur 40 jours avec une équipe 100% AI hormis le superviseur https://www.linkedin.com/pulse/jai-vir%C3%A9-mon-%C3%A9quipe-de-dev-pour-une-100-ia-pendant-40-luc-bonnin-jlgjf/ Voici le résumé en bullet points : Expérimentation de 40 jours : remplacer une équipe de dev par 100% IA agentique (Cursor) sur un vrai projet en production (playthatsheet.com, 200k lignes de code legacy) Chiffres bruts : 2,3 milliards de tokens consommés, 1 477 prompts, 260 564 lignes ajoutées (+145%), 59% du code final produit par l'IA ROI vertigineux à court terme : 9 mois de travail humain livrés en 40 jours, coût total 260$ d'abonnement + 15 jours de supervision, ROI x18 Profil psy de l'IA : Alzheimer (oublis de contexte), schizophrène (change de méthodo), ado de 12 ans (refait les mêmes erreurs), oscille entre génie et junior sans prévenir Effet iceberg : la dette technique ne disparaît pas, elle se camoufle et s'accélère ; hallucinations = bombes à retardement détectables uniquement par relecture humaine ligne par ligne Paradoxe du bateau de Thésée : perte de paternité et de maîtrise fine du code, baisse de l'autonomie du dev humain qui valide sans avoir construit Arnaque du "monkey money" : consommation de tokens opaque, non corrélée à la complexité (écart de 350% sur des prompts identiques), facturation imprévisible donc impossible à budgéter Syndrome du bazooka : les devs utilisent l'IA même pour changer une couleur CSS, atrophie progressive des compétences et coût écologique délirant Risque stratégique : dépendance irréversible aux vendeurs de tokens (Nvidia, Anthropic, OpenAI), business non rentable qui devra augmenter ses prix Conseil final : approche Pareto, garder 20% du temps en code "fait main", nommer un responsable stratégie IA, l'humain senior reste irremplaçable pour superviser Une libraries de test JUnit cache un prompt qui demande aux coding agents d'effacer les tests https://arstechnica.com/security/2026/05/fed-up-with-vibe-coders-dev-sneaks-data-nuking-prompt-injection-into-their-code/ Agacé par les « vibe coders », un développeur introduit une injection de prompt destructrice dans son code Le développeur de jqwik (un moteur de tests pour JUnit 5) a volontairement inséré une injection de prompt dans la version 1.10.0 de sa bibliothèque Java pour saboter le travail des agents d'IA. L'instruction injectée via la sortie standard (stdout) ordonne textuellement aux LLM d'ignorer les consignes précédentes et de supprimer l'intégralité du code et des tests jqwik du projet. Pour dissimuler cette action aux yeux des développeurs humains, le mainteneur a utilisé des séquences d'échappement ANSI qui effacent la ligne d'injection dans les émulateurs de terminaux interactifs. La modification a été découverte par un utilisateur qui a pointé du doigt les risques majeurs et disproportionnés pour les machines des utilisateurs, bien que certains outils comme Claude d'Anthropic aient détecté et bloqué la consigne malveillante. Face aux critiques de la communauté et aux accusations de comportement infantile ou potentiellement illégal, le développeur a mis à jour ses notes de version pour documenter explicitement son opposition à l'usage de son outil par des IA, avant de refuser tout commentaire supplémentaire sur conseil de son avocat. La réalité du rôle de Principal Engineer https://leaddev.com/career-development/reality-being-principal-engineer Le passage au rôle de Principal Engineer marque une transition majeure où les compétences techniques ne suffisent plus, l'impact se mesurant désormais à travers l'influence, la stratégie et la capacité à aligner la technique avec les objectifs business. Contrairement aux attentes, le quotidien est souvent marqué par une forme d'isolement, car le poste se situe à l'intersection de la direction (qui attend des solutions) et des équipes techniques (qui attendent des directives), sans appartenance directe à un groupe précis. Le rôle exige d'accepter une grande part d'ambiguïté et l'absence de retours immédiats, les projets et les décisions stratégiques mettant parfois des mois ou des années à porter leurs fruits. La gestion du temps devient un défi critique, nécessitant de savoir naviguer entre les sollicitations constantes, la présence en réunion et le besoin de préserver des moments de réflexion approfondie pour concevoir des visions à long terme. La réussite à ce niveau repose sur le développement de compétences humaines pointues (soft skills), notamment la négociation, la communication vulgarisée auprès des profils non techniques, et la capacité à faire grandir les autres ingénieurs par le mentorat. Sécurité Une attaque de la chaîne d'approvisionnement npm utilise binding.gyp pour compromettre des dizaines de paquets https://cybersecuritynews.com/binding-gyp-supply-chain-attack-compromises-dozens-of-npm-packages/ Une nouvelle variante du ver auto-propageable "Shai-Hulud", baptisée "Miasma", cible l'écosystème npm (et PyPI sous le nom de "Hades") en dissimulant son exécution dans le fichier binding.gyp au lieu des scripts classiques preinstall ou postinstall. La technique, surnommée "Phantom Gyp", exploite le fait que npm lance automatiquement node-gyp rebuild dès qu'un fichier binding.gyp est présent à la racine d'un paquet pour compiler des modules natifs C/C++, exécutant ainsi le code malveillant dès la commande npm install. L'attaque contourne la plupart des outils de sécurité traditionnels car l'injection s'appuie sur l'évaluation récursive de commandes (via la syntaxe ) ou directement sur la fonction eval() de Python sous-jacente à GYP, cachée sous n'importe quelle clé du fichier. Le script malveillant télécharge un runtime alternatif (Bun) pour échapper aux détections comportementales de Node.js, puis moissonne les identifiants et secrets des développeurs et des environnements CI/CD (npm, GitHub, AWS, GCP, Azure, Kubernetes, HashiCorp Vault). Plus de 57 paquets npm (dont le SDK serveur de Vapi ou des outils liés à l'IA) et des dizaines de paquets PyPI ont été infectés via des comptes de mainteneurs compromis, le ver republiant automatiquement de nouvelles versions vérolées en utilisant les jetons volés. Loi, société et organisation Restructuration chez Gitlab https://about.gitlab.com/blog/gitlab-act-2/ GitLab entame une restructuration majeure pour s'adapter à l'ère de l'intelligence artificielle agentique, incluant une réduction d'effectifs planifiée de manière transparente et ouverte. L'entreprise prévoit de réduire de 30 % le nombre de pays où elle maintient de petites équipes, d'aplatir sa hiérarchie en supprimant jusqu'à trois niveaux de gestion, et de réorganiser la R&D en une soixantaine d'équipes plus petites et autonomes. Les processus internes vont être revus en intégrant des agents d'IA pour automatiser les revues, les approbations et les passages de relais afin d'accélérer le rythme de travail. La stratégie repose sur la conviction que le logiciel sera bientôt écrit par des machines et dirigé par des humains, ce qui va multiplier la demande de logiciels et transformer le rôle des ingénieurs vers la résolution de problèmes complexes. Sur le plan technique, GitLab reconstruit son infrastructure sous-jacente (notamment Git) pour supporter la charge massive générée par les agents d'IA, tout en misant sur l'orchestration du cycle de vie, la centralisation du contexte des données et une gouvernance intégrée. Le modèle économique évolue vers un système hybride combinant les abonnements classiques et une tarification à la consommation pour le travail effectué par les agents d'IA. Un LLM local sur un mac pourrait coûter plus cher en électricité qu'un modèle hébergé sur OpenRouter dans le cloud https://www.williamangel.net/blog/2026/05/17/offline-llm-energy-use.html Conclusion : L'inférence locale sur Mac M5 Max est 3x plus chère et 2x plus lente que le cloud (OpenRouter). Électricité : Négligeable (~0,02 $/heure pour 50-100W). Matériel (Le vrai coût) : Achat du Mac à 4 299 $; l'amortissement sur 3 à 5 ans plombe la rentabilité horaire. Coût au million de tokens (Gemma 4 31b) : Mac M5 Max : 0,40 à4, 79 (pour 10-40 tokens/s). OpenRouter : 0,38 à0, 50 (pour 60-70 tokens/s). Verdict pro : Le temps humain perdu à cause de la lenteur locale coûte infiniment plus cher que les tokens cloud. Privilégier les API (Anthropic, OpenRouter). Ai didn't kill your junior pipeline https://andrewmurphy.io/blog/ai-didnt-kill-your-junior-pipeline-you-did L'IA n'a pas tué le recrutement des juniors, les entreprises l'ont fait elles-mêmes, par effet de mode. Sans juniors, pas de futurs seniors : on retire l'échelle qui nous a tous fait monter. Tout le monde pêche dans le même bassin de seniors sans le réapprovisionner, pénurie garantie dans 3-5 ans. Une équipe 100% senior + IA est fragile : un départ et tout le savoir tacite s'évapore. Les juniors posent les "pourquoi ?" qui révèlent les bugs et processus absurdes ; l'IA, elle, exécute sans questionner. Les seniors s'atrophient aussi en déléguant leur réflexion à l'IA, pince à double effet sur les compétences. Dépendre des outils IA, c'est sous-traiter sa stratégie talents à des fournisseurs dont les prix vont tripler. Solution : redéfinir le rôle junior (revue de code IA + mentorat), pas le supprimer. Les rapports internes de Microsoft révèlent la crise des coûts de l'IA : les agents coûtent plus cher que les employés humains https://fortune.com/2026/05/22/microsoft-ai-cost-problem-tokens-agents/ Des données et rapports internes chez Microsoft et d'autres géants de la tech ébranlent la promesse de rentabilité de l'IA, révélant que le déploiement d'agents autonomes à l'échelle de l'entreprise revient souvent plus cher que de payer des humains pour le même travail. Le modèle de tarification à l'usage (basé sur les tokens) se heurte à la nature même des architectures agentiques : contrairement à un simple chatbot, un agent boucle, enchaîne les appels d'outils, crée des sous-agents et auto-évalue son code, ce qui multiplie la consommation de tokens par un facteur de 5 à 30, voire jusqu'à 1 000 fois pour des tâches de programmation complexes. L'impact financier sur les budgets de calcul cloud est immédiat ; par exemple, Uber a entièrement épuisé l'intégralité de son budget annuel 2026 dédié au codage par IA en l'espace de seulement quatre mois. Face à cette explosion des coûts, des retours en arrière drastiques sont observés : Microsoft a ainsi commencé à suspendre une grande partie de ses licences internes Claude Code pour rediriger d'urgence ses milliers de développeurs vers sa propre solution moins onéreuse, GitHub Copilot CLI. Les directeurs techniques (CTO) et acheteurs de solutions logicielles qui ont signé des contrats pluriannuels basés sur des projections de réduction de masse salariale se retrouvent pris au piège, les gains réels de productivité ne parvenant pas à compenser les factures d'infrastructure exorbitantes. Conférences La liste des conférences provenant de Developers Conferences Agenda/List par Aurélie Vache et contributeurs : 11-12 juin 2026 : DevQuest Niort - Niort (France) 11-12 juin 2026 : DevLille 2026 - Lille (France) 12 juin 2026 : Tech F'Est 2026 - Nancy (France) 15 juin 2026 : Jupyter Workshops: Demystifying MyST Markdown in Education - Orsay (France) 16 juin 2026 : Mobilis In Mobile 2026 - Nantes (France) 17-19 juin 2026 : Devoxx Poland - Krakow (Poland) 17-20 juin 2026 : VivaTech - Paris (France) 18 juin 2026 : Tech'Work - Lyon (France) 22-26 juin 2026 : Galaxy Community Conference - Clermont-Ferrand (France) 23-24 juin 2026 : MWCP 2026 - Paris (France) 24-25 juin 2026 : Agi'Lille 2026 - Lille (France) 24-26 juin 2026 : BreizhCamp 2026 - Rennes (France) 26-27 juin 2026 : LeHACK - Paris (France) 27 juin 2026 : Asynconf - Paris (France) 2 juillet 2026 : Azur Tech Summer 2026 - Valbonne (France) 2 juillet 2026 : MCP Connect Travel Edition - Paris (France) 2-3 juillet 2026 : Sunny Tech - Montpellier (France) 3 juillet 2026 : Agile Lyon 2026 - Lyon (France) 6-8 juillet 2026 : Riviera Dev - Sophia Antipolis (France) 28-30 août 2026 : State of the Map - Champs-sur-Marne (France) 4 septembre 2026 : JUG Summer Camp 2026 - La Rochelle (France) 10-11 septembre 2026 : Nantes Craft - Nantes (France) 17 septembre 2026 : dotAI - Paris (France) 17-18 septembre 2026 : API Platform Conference 2026 - Lille (France) 18 septembre 2026 : WordCamp Bretagne - Rennes (France) 18 septembre 2026 : dotJS - Paris (France) 18 septembre 2026 : WordCamp Bretagne - Rennes (France) 22 septembre 2026 : Salon Data 2026 - Nantes (France) 22-23 septembre 2026 : Agile en Seine & IA 2026 - Paris (France) 24 septembre 2026 : OWASP AppSec Days France 2026 - Paris (France) 24 septembre 2026 : PlatformCon Paris - Paris (France) 24 septembre 2026 : React Native Connection 2026 - Paris (France) 24-26 septembre 2026 : Paris Web 2026 - Paris (France) 25 septembre 2026 : SAP Inside Track Paris 2026 - Paris (France) 28-29 septembre 2026 : 4th Tech Summit on AI & Robotics - Paris (France) & Online 1 octobre 2026 : WAX 2026 - Marseille (France) 1-2 octobre 2026 : Volcamp - Clermont-Ferrand (France) 2 octobre 2026 : DevFest Perros-Guirec 2026 - Perros-Guirec (France) 5-9 octobre 2026 : Devoxx Belgium - Antwerp (Belgium) 8-9 octobre 2026 : Forum PHP 2026 - Marne-la-Vallée (France) 12 octobre 2026 : Dev With AI - Paris (France) 22-23 octobre 2026 : Agile Tour Bordeaux 2026 - Bordeaux (France) 26 octobre 2026 : Agile Tour Montpellier - Montpellier (France) 27-29 octobre 2026 : Directions EMEA 2026 - Paris (France) 29-30 octobre 2026 : BDX I/O 2026 - Bordeaux (France) 29-30 octobre 2026 : Agile Tour Nantais 2026 - Nantes (France) 29 octobre 2026-1 novembre 2026 : Pycon FR - Biarritz (France) 30 octobre 2026 : Cloud Nord 2026 - Lille (France) 4-5 novembre 2026 : Devoxx Morocco - Casablanca (Morocco) 14-15 novembre 2026 : Capitole du Libre - Toulouse (France) 19 novembre 2026 : DevFest Toulouse 2026 - Toulouse (France) 19 novembre 2026 : Agile Laval 2026 - Laval (France) 19 novembre 2026 : OVHcloud Summit - Paris (France) 19 novembre 2026 : Codeurs en Seine - Rouen (France) 27 novembre 2026 : DevFest Paris 2026 - Paris (France) 1-3 décembre 2026 : Apidays Paris - Paris (France) 2-3 décembre 2026 : Cloud Native AI Summit Europe - Paris (France) 4 décembre 2026 : DevFest Lyon 2026 - Lyon (France) 4 décembre 2026 : DevFest Dijon 2026 - Dijon (France) 9-10 décembre 2026 : OpenSource Expérience - Paris (France) 9-10 décembre 2026 : DevOps REX - Paris (France) 10 décembre 2026 : KCD Provence - Aix-en-Provence (France) 7-9 avril 2027 : Devoxx France 2027 - Paris (France) 3 juin 2027 : Cloud Native Days France 2027 - Paris (France) Nous contacter Pour réagir à cet épisode, venez discuter sur le groupe Google https://groups.google.com/group/lescastcodeurs Contactez-nous via X/twitter https://twitter.com/lescastcodeurs ou Bluesky https://bsky.app/profile/lescastcodeurs.com Faire un crowdcast ou une crowdquestion Soutenez Les Cast Codeurs sur Patreon https://www.patreon.com/LesCastCodeurs Tous les épisodes et toutes les infos sur https://lescastcodeurs.com/

This week we examine the emergence of Kubernetes as a common infrastructure platform. We contrast cloud assumptions with bare-metal reality and dig deep on supportability, repairability, and the operational challenges of networking, storage, GPUs, and other specialized systems. We also get into brittle vendor tooling, version changes, and how these can make remediation difficult. I think you'll like this one! Transcript: https://otter.ai/u/cA80HK4iVJWflodPqIYf9mW3Mec?utm_source=copy_url

On episode 39 of Open Source Ready, Brian Douglas and John McBride speak with Zach Smith, creator of Kplane, about rethinking Kubernetes for an AI-driven future. Zach explains how virtualized control planes could enable isolated cluster experiences at massive scale, while the conversation explores developer experience, AI tooling, and the possibility that future AI agents may each require their own cloud.

Today, we are dropping another episode in our series The AI Control Loop, How enterprises govern the AI they've already deployed - sponsored by our friends at Wallarm.Wallarm is the AI Control Platform for Enterprise AI, protecting every AI workload, API, and application in production, giving CISOs the governance they need and CIOs the speed they demand. Organizations choose Wallarm for a complete inventory of APIs, AI agents, and AI apps, patented AI/ML-based threat detection and blocking that operates at production traffic speeds.We all know that you can't secure what you can't see, which is why AI discovery is a first principle for AI security, but what's really required for AI discovery? It's more than just LLMs and agents. Today's episode is entitled AI Discovery isn't just AI, and joining us is Tim Ebbers, Field CTO at Wallarm. Tim and I discuss the real requirements for AI discovery, and why the connections between assets and infrastructure are part of the puzzle.QuestionsSecurity teams often say, “You can't secure what you can't see.” In the context of AI, what exactly do they need to see? What supporting infrastructure matters most when mapping AI risk, such as APIs, cloud services, Kubernetes workloads, data stores, identities, and external integrations?Where does shadow AI typically appear first inside an enterprise environment? How can it be prevented?How do relationships between assets change the risk picture? For example, why does it matter which API an agent can call or which data source a workflow can reach?What makes AI discovery harder than traditional application or cloud asset discovery? What are the similarities and differences?How should organizations prioritize what they find? Is every AI asset equally risky?What does “continuous discovery” mean in a world where AI services can be deployed, connected, or changed in minutes?Once an organization has visibility into its AI footprint, what's next? What are the biggest gaps in today's AI security programs?Linkshttps://www.wallarm.com/https://www.linkedin.com/in/tebbers/Full AbstractMost security teams know that you can't secure what you can't see. In the context of AI, that rule turns out to be a lot harder to satisfy than it sounds.AI discovery isn't just a matter of cataloging your LLMs and agents. The real picture includes the APIs those agents call, the data sources they reach, the infrastructure they run on, and all the AI that got deployed without anyone telling security. Building that picture requires understanding relationships, not just inventories, because risk doesn't live in assets in isolation. It lives in what those assets can do together.In this episode, Tim Ebbers, Field CTO at Wallarm, examines what a complete AI control loop actually requires at the discovery stage: what needs to be visible, why the connections between assets change the risk calculation, where shadow AI tends to appear first and how it becomes unmanaged risk, and what makes AI discovery structurally different from traditional cloud or application discovery. It also looks at what organizations should do once discovery is in place, and where the biggest gaps remain in AI security programs today.If your team is building toward continuous AI governance, this is where that work starts.Our Sponsors:* Check out Cash App and use my code CASHAPP10 for a great deal: https://click.cash.app/ui6m/mt82fpxl #CashAppPod. Cash App is a financial services platform, not a bank. Banking services provided by Cash App's bank partner(s). Prepaid debit cards issued by Sutton Bank, Member FDIC. See terms and conditions at https://cash.app/legal/us/en-us/card-agreement. Cash App Green, overdraft coverage, borrow, cash back offers and promotions provided by Cash App, a Block, Inc. brand. Visit http://cash.app/legal/podcast for full disclosures.* Check out Plaud AI and use my code CODESTORY for a great deal: https://plaud.aiAdvertising Inquiries: https://redcircle.com/brandsPrivacy & Opt-Out: https://redcircle.com/privacy

The Pope issued a recent encyclical on AI, urging developers to safeguard human agency in the age of artificial intelligence. Eyvonne and William explore this encyclical, moving beyond the headlines to the core message regarding human dignity. They examine how the document provides a values-based framework for evaluating technology and the need for a balanced... Read more »

The Pope issued a recent encyclical on AI, urging developers to safeguard human agency in the age of artificial intelligence. Eyvonne and William explore this encyclical, moving beyond the headlines to the core message regarding human dignity. They examine how the document provides a values-based framework for evaluating technology and the need for a balanced... Read more »

פרק מספר 516 של רברס עם פלטפורמה - קרבורטור מספר 41. הפעם רן ואורי מארחים את נתי לשיחה על נקודת המפגש המרתקת שבין קוד פתוח לקידוד מבוסס סוכנים (Agentic Coding). דיברנו על העתיד הדיסטופי והאופטימי של מפתחי קוד פתוח, איך משווקים מוצרים ל-Agents, ולמה שורת הפקודה (CLI) חוזרת אלינו בענק. [01:04] העתיד המדומיין של AI (סיפורו של OpenClaw) נתי משתף סיפור משעשע על ניסיון לחקור את "OpenClaw". הזיות (Hallucinations) של מודלים: Claude מאשר את העובדות, בעוד ש-Gemini מנתח שמדובר בהמצאה עתידית (פברואר 2026). הבנה שמודלי שפה (LLMs) הם מנועים הסתברותיים ולא מנועי חיפוש עובדתיים. [05:58] החזון הדיסטופי: האם AI יהרוג את הקוד הפתוח? בעיית ההעתקה: בעבר קוד הוגן על ידי רישיונות (כמו AGPL), היום קל לבקש מהמודל לשכתב קוד משפה אחת לאחרת (למשל מ-NodeJS ל-Rust) בעלויות אפסיות. קריסת מודלים עסקיים: עלויות התמיכה והאופרציה (Operation) יורדות כי ה-Agent מתקן תקלות לבד, מה שחותך את ההכנסות של חברות כמו Red Hat. עומס על ה-Maintainers: קוד מג'ונרט על ידי Agents נראה מעולה ומתועד היטב, אבל לא תמיד נכון ארכיטקטונית או לוגית. גישות התמודדות: חלק דורשים לקבל את ה-Prompt (הכוונה) ולא את הקוד עצמו, בעוד שאחרים (כמו יוצר שפת Zig) אוסרים לחלוטין גישה של AI לפרויקט. [15:15] החזון האופטימי: שיווק לסוכנים (GEO) מעבר מ-SEO ל-GEO (Generative Engine Optimization): סוכני AI הם הלקוחות החדשים. איך Agent בוחר כלים? לפי איכות הקוד, הפופולריות שלו ב-GitHub, ובעיקר לפי התיעוד. קוד פתוח הופך לכלי שיווקי קריטי (Open Core) כדי שהסוכנים יוכלו למצוא, להבין ולהמליץ על המוצר. מודלים היברידיים ו-Freemium: מוצרים (כמו Postits) מציעים גישה ללא חומת תשלום (Paywall) בשלבים הראשונים, מה שמאפשר ל-Agents לעבוד איתם בקלות דרך API (Headless SaaS), ואפילו לבצע רכישות בעצמם בהמשך דרך Stripe. [30:29] שובו של ה-CLI ומגבלות ה-MCP הדיבייט סביב MCP (Model Context Protocol): הפרוטוקול כבד, "זולל" טוקנים (Token hungry) עבור הקונטקסט, ודורש תחזוקה של שרתים נוספים. למה Agents כל כך אוהבים CLI (שורת פקודה)? גישה ישירה לאקוסיסטם המקומי והרשאות (כמו Kubernetes או סביבות ענן) בלי לחשוף מפתחות לשירות חיצוני. יכולת לבצע מניפולציות מורכבות בצד הלקוח (Chaining, Grep, Sed) מבלי לשנות קוד ב-Backend, מה שהופך את המודלים לאנשי DevOps מעולים. [36:17] רישיונות קוד פתוח וה"נשמה" של המוצר האתגר באכיפת רישיונות (כמו GPL) בעולם שבו קשה להוכיח על איזה קוד המודל התאמן ואם בוצעה העתקה. הבדל חשוב בטרמינולוגיה: מודלים של "Open Weights" לעומת מודלים שה-Training Data שלהם באמת פתוח. תוכנה כיצירת אומנות מול קומודיטי (Commodity): האם קוד מג'ונרט יכול להחליף את החזון וה"נשמה" (Soul) של מפתחים בולטים? ההשוואה לעולם המוזיקה מדגישה שמשתמשים הולכים אחרי האומן והחזון, לא רק אחרי הקוד היבש. [50:25] רגולציה ומודלי Open Weights אורי מעלה נקודה מעניינת על החסימה של מודל Fable 5 / Mytos 5 (של Anthropic) למשתמשים מחוץ לארה"ב על ידי הממשל האמריקאי. ההשפעה של רגולציה: ה"תקרת זכוכית" הזו עלולה לפגוע בחברות המסחריות האמריקאיות בטווח הקצר, ודווקא לדחוף קדימה מודלים פתוחים (Open Weights) סיניים או אירופאים שאינם כפופים לאותן מגבלות. האזנה נעימה!

In this episode of Alexa's Input (AI), I sit down with David Aronchick, co-founder and CEO of Expanso and former product lead for Kubernetes at Google.Data is growing everywhere outside your data center. Solar panels in remote across a country. Security cameras at retail stores. IoT sensors across factory floors. And moving that data to the cloud for processing? It's expensive, slow, and often restricted by compliance.David is an expert when it comes to solving distribution problems. He led Kubernetes product at Google, co-founded Kubeflow to bring ML to production, and now he's building Expanso to tackle a difficult constraint: when your data can't move, how do you process it where it lives?We discuss:- The need for distributed data orchestration-Upstream data control: filtering and transforming at the source- Three forces making edge computing inevitable (physics, regulations, economics)- How to build successful open source infrastructure projects- Customer discovery and finding real pain points- His transition from Protocol Labs to founding Expanso- ETL pipelines: moving the first four steps closer to the data- Context loss and lineage in distributed systems- Processing 400,000 signals per second with 150MB agents- AI observability: attaching source metadata to training data- Running ML pipelines at the edge- Real-world deployment challenges (bandwidth, regulations, cost)Expanso is rethinking how we process data in an AI-native world—moving compute to data instead of data to compute. If you want to understand where distributed systems and edge computing are heading, this is a deep dive into the infrastructure layer beneath modern AI applications.General Podcast LinksWatch: https://www.youtube.com/@alexa_griffith Read: https://alexasinput.substack.com/ Listen: https://creators.spotify.com/pod/profile/alexagriffith/ More: https://linktr.ee/alexagriffithLearn more about the host atWebsite: https://alexagriffith.com/ LinkedIn: https://www.linkedin.com/in/alexa-griffith/Find out more about the guest atLinkedIn: https://www.linkedin.com/in/aronchick/ Twitter/X: https://x.com/aronchick GitHub: https://github.com/aronchick Expanso Website: https://expanso.io/ResourcesExpanso Website: https://expanso.io/ Kubernetes: https://kubernetes.io/ Kubeflow: https://www.kubeflow.org/ CNCF (Cloud Native Computing Foundation): https://www.cncf.io/ Protocol Labs: https://protocol.ai/KeywordsDavid Aronchick, Expanso, Kubernetes, Kubeflow, distributed systems, edge computing, data pipelines, ETL, upstream data control, Google Kubernetes Engine, open source, CNCF, observability, log processing, data lineage, provenance, schema enforcement, IoT, edge AI, distributed data, machine learning infrastructure, Protocol Labs, IPFS, Filecoin, data governance, compliance, GDPR, bandwidth optimization, data aggregation, AI infrastructure, multi-cloud, hybrid cloud, real-time processing

OpenChoreo is an opinionated, “batteries included”, AI-native Kubernetes platform stack for Platform Engineers that combines GitOps, Observability, AI Agents, and Workflows into a custom K8s distribution “super pack” that is managed via Backstage, CLI, API, or MCP. Now a CNCF project.Check out the video podcast version here: 

In this episode we speak to Jonathan Grahl.  Jonathan is the Team Lead of Infrastructure at Lovable where he oversees the platform stack the company runs on. We talked about Kubernetes, Sandboxes and Chocolate.   Do you have something cool to share? Some questions? Let us know: - web: kubernetespodcast.com - mail: kubernetespodcast@google.com - twitter: @kubernetespod - bluesky: @kubernetespodcast.com   News of the week OpenTelemetry is a CNCF Graduated Project CNCF TAG Elections KubeCon India Kubernetes Community Days global events Links from the interview Lovable Cilium Etcd Cloudflare Sandbox Agent Sandbox (Kubernetes) OpenClaw Claude OpenAI Bun toolkit for Javascript gVisor Firecracker Kata Containers Vitess

This is the Engineering Culture Podcast, from the people behind InfoQ.com and the QCon conferences. In this podcast, Shane Hastie, Lead Editor for Culture & Methods, spoke to Craig McLuckie, co-creator of Kubernetes and CEO of Stacklok, about the impact of AI coding tools on open source communities and engineering teams, designing deliberate organisational culture, and navigating evolving career paths for engineers in the age of AI. Read a transcript of this interview: https://bit.ly/3RGzPCa Newsletter: Subscribe to the Software Architects' Newsletter for your monthly guide to the essential news and experience from industry peers on emerging patterns and technologies: https://www.infoq.com/software-architects-newsletter InfoQ online certification cohorts: Online cohorts for senior engineers and architects, built around QCon talks. Join a 5-week confidential peer group to validate your approach and apply practitioner frameworks to the technical challenges you face at work. Learn more: https://certification.qconferences.com/ Upcoming Events: QCon AI Boston 2026 (June 1-2, 2026) Learn how real teams are accelerating the entire software lifecycle with AI. https://boston.qcon.ai QCon San Francisco 2026 (November 16-20, 2026) https://qconsf.com/ The InfoQ Podcasts: Weekly inspiration to drive innovation and build great teams from senior software leaders. Listen to all our podcasts and read interview transcripts: - The InfoQ Podcast https://www.infoq.com/podcasts/ - Engineering Culture Podcast by InfoQ https://www.infoq.com/podcasts/#engineering_culture - Generally AI: https://www.infoq.com/generally-ai-podcast/ Follow InfoQ: - Mastodon: https://techhub.social/@infoq - X: https://x.com/InfoQ?from=@ - LinkedIn: https://www.linkedin.com/company/infoq/ - Facebook: https://www.facebook.com/InfoQdotcom# - Instagram: https://www.instagram.com/infoqdotcom/?hl=en - Youtube: https://www.youtube.com/infoq - Bluesky: https://bsky.app/profile/infoq.com Write for InfoQ: Learn and share the changes and innovations in professional software development. - Join a community of practitioners. - Increase your visibility. - Grow your career. https://www.infoq.com/write-for-infoq

An airhacks.fm conversation with Alvaro Hernandez (@ahachete) about: discussion about the quarkus Insights episode "#337 The Database Cloud" stackgres live demo, StackGres as a Quarkus and GraalVM native kubernetes operator for running Postgres, comparing CloudNativePG (CNPG) by EnterpriseDB to StackGres, Patroni for Postgres high availability, the split-brain risk of relying on Kubernetes and etcd alone, distributed consensus and leader lock election via etcd, why distributed systems and cryptography should not be self-implemented, async, synchronous and quorum (semi-synchronous) Postgres replication trade-offs, cascading and cross-region replication topologies, the false-positive problem and heuristic exceptions in two-phase commit, the ondb ("own your database") project for self-hosted Postgres, losing control with managed cloud services and untestable backups, vanilla unmodified Postgres on StackGres, the "Kubernetes without Kubernetes" (Kubeless) pattern, talking directly to ContainerD through the CRI API, runc and the Docker to ContainerD chain, a self-contained native binary that embeds ContainerD over Unix domain sockets, the slony node-local component named after the Postgres slonik elephant mascot, the Matriarch orchestrator component, reverse gRPC tunnels with Slonies phoning home across NAT and firewalls, a multi-tenant cloud control plane provided as a service, curl-pipe-shell node installation with a token, end-to-end encrypted Postgres protocol tunneling for JDBC from anywhere, psql compiled to wasm in the web console, Tailscale-inspired user experience, unifying nodes, Kubernetes clusters and cloud pools as resources, Slony Kubernetes controller, Java 25 source-mode scripting without dependencies, implementing your own MCP server for Postgres JDBC metadata, the Goose agentic UI donated by Block to the Linux Foundation, AI Rails BCE, Java, Web Components skills Alvaro Hernandez on twitter: @ahachete

In this episode of the Ardan Labs Podcast, Ale Kennedy talks with François Bitouzet, Managing Director of Viva Technology, about the forces shaping the future of technology and innovation. François shares his journey from studying in France to leading one of the world's largest technology and startup events, connecting entrepreneurs, investors, and industry leaders from around the globe.00:00 Introduction02:58 Education and Early Influences08:53 Early Career and Communication17:47 Communication in a Changing World32:25 Innovation and Technology42:40 Creativity and Marketing49:38 Leadership and Career Growth54:54 Adapting to Technological Change59:42 The Future of Events01:06:54 AI and Society01:11:20 Startups and Innovation01:15:35 Deep Tech and the FutureConnect with François: LinkedIn: https://www.linkedin.com/in/fran%C3%A7ois-bitouzet-180a89/Mentioned in this Episode:Viva Technology: https://vivatechnology.comWant more from Ardan Labs? You can learn Go, Kubernetes, Docker & more through our video training, live events, or through our blog!Online Courses : https://ardanlabs.com/education/ Live Events : https://www.ardanlabs.com/live-training-events/ Blog : https://www.ardanlabs.com/blog Github : https://github.com/ardanlabs

Send us Fan MailToday's guest is Matt Moore, Co-Founder and CTO of Chainguard — the software supply chain security company on a mission to make the open-source ecosystem safe for enterprises everywhere. Matt's background spans some of the most consequential corners of the cloud-native world. Before co-founding Chainguard, he was a core contributor to the open-source ecosystem and played a pivotal role at Google, where he helped shape the Kubernetes and supply chain security landscape that underpins much of modern software infrastructure today.In today's conversation, we'll dive into Matt's journey from open-source engineering to founding a company, how Chainguard is tackling one of the most critical — and often overlooked — challenges in enterprise software, the philosophy behind building security into the foundation rather than bolting it on, what it takes to turn deep technical expertise into a venture-backed company, and his vision for the future of software supply chain security in an AI-first world.Support the show

Joe Beda and Craig McLuckie co-created Kubernetes, the infrastructure standard that became the default for cloud native computing. Now running Stacklok, they're watching enterprises hit the same identity, permissions, and security problems with AI agents that took the container ecosystem years to resolve, and they're building tools to compress that timeline. In this episode of Founded & Funded, Madrona's Tim Porter sits down with Joe and Craig to talk through what AI adoption actually requires: why MCP is the Docker moment for AI-native applications, how the LLM gateway is becoming a strategic chokepoint for cost, safety, and model flexibility, and why enterprises that don't get the architecture right early will face a familiar trap: vertical integration that looks like productivity and acts like lock-in. They cover: Why the developer workflow is the template for knowledge worker AI adoption, and where the analogy breaks down The mainframe vs. open platform question that will define the AI infrastructure era Why the knowledge worker transition is harder than it looks — and what has to be built differently before developer-grade AI tooling can scale to the rest of your organization The governance gap between human accountability and AI behavior, and what enterprises actually need to build to close it Where to start: MCP controls first, LLM gateway second, and why deploying a platform without staying to close the loop consistently fails Transcript: https://www.madrona.com/the-best-infrastructure-moment-since-cloud Chapters: (0:00) – Introduction (1:04) – Why the Kubernetes Creators Are the Right People to Read This AI Moment (2:18) – Joe's Lesson from Cloud Native: Ignore Conventional Wisdom, Except When You Shouldn't (4:16) – Craig on Enterprises and the Chaos of a New Infrastructure Era (5:32) – Why Joe Rejoined Craig at Stacklok: The Engineer's Case for Getting Your Hands Dirty (7:05) – Developers as Agent Orchestrators: How the Knowledge Worker Transition Will Follow (10:10) – MCP Explained: Craig Sees Docker in 2013 When He Looks at the MCP Spec 1 (7:53) – The Mainframe vs. Open Platform Question That Will Define the AI Era (20:24) – LLM Lock-In Is the Wrong Worry: The Real Risk Is Left of the Model (25:19) – Where Enterprises Actually Start: Developer Posture First, Knowledge Workers Second (29:10) – MCP First, LLM Gateway Second: The Concrete Technical Starting Point (31:19) – How Stacklok Builds Software Now: Agents, Smaller Teams, the Unrecognizable Developer Profile (38:07) – The Recruiter Who Started Building Agents: What AI Tools Do to Role Boundaries

IPv8 уже в драфте, Terraform хоронят в блогах, а одна компания жжёт $7M в год на Claude Code. Собрали новости DevOps, которые вы накидали через бота. О ЧЁМ ВЫПУСК Новостной выпуск: накопилось 63 новости, разобрали самое горячее. И снова с нами Ярослав. В этом выпуске: • IPv8: драфт в IETF - ASN в первых 32 битах, старый IPv4 во вторых. Без NAT и dual-stack, плюс токен-идентичность на каждый девайс • Terraform 1.15: переменные в source и version модулей, отдельная аутентификация для S3 backend • "Terraform is dead": разбираем хайповую статью - спека как desired state, Pulumi, CDK и причём тут AI • Terragrunt 1.0: units, stacks и фильтр по affected-ресурсам через git worktree • NGINX 1.30: sticky sessions, keep-alive и HTTP/2 к апстримам, Early Hints (103), Encrypted Client Hello • Экономика AI: Semi-Analysis масштабировала Claude Code до $7M/год, дефицит RAM • Google Agent Sandbox: новый Kubernetes CRD между StatefulSet и Deployment Сквозная мысль выпуска: AI ускоряет всё, но без понимания, как работают системы, спека и вайб-кодинг рано или поздно стреляют в ногу. ГОСТЬ Ярослав Бледковский - Un-principal SRE, Wargaming ССЫЛКИ Все новости выпуска (тезисы, голосование, ссылки): https://dkt-ai.github.io/episodes-news/episodes/episode-97-ru Присылайте новости через бота: @dkt_news_bot Упомянутые ресурсы: • IPv8 draft (IETF): https://www.ietf.org/archive/id/draft-thain-ipv8-00.html • Terraform 1.15: https://github.com/hashicorp/terraform/releases/tag/v1.15.0 • "Terraform is dead" (статья): https://grahamgilbert.com/blog/2026/04/20/terraform-is-dead/ • Terragrunt 1.0: https://github.com/gruntwork-io/terragrunt/releases/tag/v1.0.0 • NGINX 1.30: https://github.com/nginx/nginx/releases/tag/release-1.30.0 • AI tokens (Dylan Patel / Semi-Analysis): https://www.youtube.com/watch?v=LF3aUIM57uw • Kubernetes Agent Sandbox: https://github.com/kubernetes-sigs/agent-sandbox ПОДКАСТ YouTube - www.youtube.com/@DevOpsKitchenTalks Apple Podcasts - https://apple.co/41O6mqA Spotify - https://t.ly/Jg5_2 Yandex Music - https://music.yandex.ru/album/10151746 PodBean - https://devopskitchentalks.podbean.com НАВИГАЦИЯ 00:00 - Интро: вы уже на IPv6 или ещё IPv4? И снова в гостях Ярослав 02:53 - Anthropic и 200K карточек от Маска: лимиты Claude отпустило 04:26 - Адженда из 63 новостей через наш Telegram-бот

On episode 53 of The Kubelist Podcast, Marc Campbell and Benjie De Groot sit down with Anurag Goel. Anurag shares his journey from early employee at Stripe to founder and CEO of Render, one of the fastest-growing application platforms in cloud infrastructure. They discuss Kubernetes, platform engineering, bare metal, AI agents, and what the future of application deployment looks like.

On episode 53 of The Kubelist Podcast, Marc Campbell and Benjie De Groot sit down with Anurag Goel. Anurag shares his journey from early employee at Stripe to founder and CEO of Render, one of the fastest-growing application platforms in cloud infrastructure. They discuss Kubernetes, platform engineering, bare metal, AI agents, and what the future of application deployment looks like.

AWS Morning Brief for the week of June 8th, with Corey Quinn. Links:AWS Interconnect - multicloud now offers a free 500 Mbps tierOracle Database@AWS is now available in twenty AWS RegionsAmazon Cognito now supports multi-Region replicationAmazon EKS and Amazon EKS Distro now supports Kubernetes version 1.36Amazon SES now supports tenant-level suppression listsAWS Compute Optimizer now supports 32-day lookback for EBS volume and ECS service rightsizing recommendationsAWS Cost and Usage Report 2.0 now supports Athena and Redshift integrationAmazon ElastiCache for Valkey now supports durabilityUnderstanding how backups work in Amazon AuroraOpenAI models and Codex on Amazon Bedrock are now generally availableHow Bedrock Streaming optimizes its AWS costsFrom Monolith to Multi-Account: Pinterest's AWS Organization Transformation JourneyGain visibility into DDoS attacks with flow logs in AWS Shield AdvancedIdentify unused AWS KMS keys and prevent accidental key deletionsCVE-2026-10591 - Kiro IDE Insufficient File Write Restrictions to Execution-Sensitive PathsCVE-2026-10584 - HTTPS Fallback to HTTP in Graph Explorer

https://novacut.ai/  https://genaimeetup.com/  Anthropic has officially closed a $65 billion Series H at a $965 billion valuation, nearly 2.5x its valuation from just 100 days ago. Meanwhile, funding is flowing across the ecosystem: Frameworks AI at $15B, Baseten at $11B, OpenRouter's $113M Series B, and Cognition AI's $1B Series D. NVIDIA went on an open-source super week with Nemotron 3 Ultra, Cosmos 3, and Nemotron 3.5 ASR. Microsoft dropped 5 new MAI models. Google released Gemma 4 12B, and Anthropic shipped Opus 4.8. On the benchmarks front, DeepSWE crowns GPT-5.5 as the leader in long-horizon coding tasks, while ITBench shows even frontier models struggle with real-world SRE incidents — Claude Opus 4.7 tops out at just 47%. Plus: Cloudflare acquires VoidZero to build the future of AI-native edge development, and Google is paying SpaceX $920M/month for compute. Topics covered: • Anthropic's $65B Series H and path to $1T • Fireworks AI, Baseten, OpenRouter & Cognition funding rounds • Microsoft's 5 new MAI models • NVIDIA's open-source super week (Nemotron, Cosmos 3) • MiniMax M3, Gemma 4 12B, JetBrains Mellum2, Opus 4.8 • DeepSWE benchmark: GPT-5.5 leads long-horizon coding • ITBench: Frontier models under 50% on real SRE tasks • Cloudflare + VoidZero for AI-native edge dev • Google's $920M/month SpaceX compute deal #AI #Anthropic #NVIDIA #OpenAI #AInews #TechNews #LLM     Funding rounds Anthropic formally confirmed the closure of its $65 billion Series H funding round at a post-money valuation of $965 billion. This represents a 2.5-fold increase over its $380 billion Series G valuation from February 2026, adding $585 billion in value in approximately 100 days https://www.anthropic.com/news/series-h  Frameworks AI raising at 15B valuation representing a near fourfold increase from its $4 billion Series C valuation recorded in October 2025 processing 15 trillion tokens daily for major production clients including Cursor, Notion, and Perplexity https://finance.yahoo.com/sectors/technology/articles/fireworks-ai-eyes-15-billion-174609357.html Baseten is raising 1B at 11B valuation annualized revenue, which skyrocketed from $200 million to $600 million over a single quarter https://techstartups.com/2026/05/26/ai-inference-startup-baseten-in-talks-to-raise-1-billion-at-11-billion-valuation/  OpenRouter has secured a $113 million Series B funding OpenRouter has experienced exponential traffic growth, with weekly production throughput expanding fivefold from 5 trillion to 25 trillion tokens over a six-month horizon https://www.businesswire.com/news/home/20260526953416/en/OpenRouter-Raises-%24113-Million-CapitalG-led-Series-B-as-Weekly-Volume-Explodes-to-25T-Tokens  Further up the stack: Cognition AI secured a $1 billion Series D round led by Lux Capital and 8VC https://cognition.ai/blog/series-d   Model Releases MAI models: MAI-Code-1-Flash: A 5-billion active parameter model optimized for ultra-low latency within GitHub Copilot and VS Code. MAI-Image-2.5: A high-fidelity image generation model ranking third on global image evaluation arenas, outperforming competing architectures like Nano Banana Pro. MAI-Transcribe-1.5: A multi-lingual speech processing engine offering fivefold speed improvements across 43 languages. MAI-Voice-2: Natural audio and voice generation across 15 languages, available at a highly competitive price point. Web IQ: A search-grounding API engineered to directly compete with Perplexity. https://microsoft.ai/models/    https://www.peoplematters.in/news/ai-and-emerging-tech/uber-imposes-dollar1500-monthly-ai-spending-limit-on-employees-amid-rising-costs-50073    Nvidia has executed an "Open-Source Super Week," positioning itself as a dominant software and model publisher: Nemotron 3 Ultra (best US open source open weights model but behind china): A massive 550-billion parameter MoE (55 billion active) designed with a 1-million token context window, optimized specifically for high-throughput, cyclical agent loops. It achieved peak throughput rates of 400 tokens per second on day-zero optimized clusters. Cosmos 3: A physical AI world-modeling framework comprising 16-billion Nano and 64-billion Super variants. Built on a Mixture-of-Transformers (MoT) architecture, Cosmos 3 natively binds textual, visual, auditory, and physical kinetic vectors. Nemotron 3.5 ASR: A highly compact 0.6-billion parameter streaming speech recognition model pushing sub-100 millisecond latencies across 40 language locales.   https://www.minimax.io/models/text/m3  MiniMax M3: A 1-million token context model hitting 59.0% on SWE-Bench Pro and 74.2% on MCP Atlas, though noted for high token consumption due to intensive internal self-validation loops.   https://blog.google/innovation-and-ai/technology/developers-tools/introducing-gemma-4-12b/  Gemma 4 12B: Google's Apache 2.0 on-device model, which utilizes an encoder-free architecture that projects vision and audio vectors directly into the text-token space, bypassing separate CLIP-style encoders to minimize local memory footprints. https://www.jetbrains.com/mellum/  JetBrains Mellum2: A compact 12-billion parameter MoE (2.5 billion active) engineered for ultra-low latency routing and retrieval-augmented generation (RAG) sub-agents within developer IDEs. Opus 4.8 https://www.anthropic.com/news/claude-opus-4-8    https://www.cnbc.com/2026/06/05/google-to-pay-spacex-920-million-a-month-for-xai-compute-capacity.html      Benchmarks: https://deepswe.d atacurve.ai/blog https://venturebeat.com/technology/deepswe-blows-up-the-ai-coding-leaderboard-crowns-gpt-5-5-and-finds-claude-opus-exploiting-a-benchmark-loophole (GPT 5.5 the winner in long horizon tasks) a highly complex software engineering benchmark focused on original, long-horizon tasks across five distinct programming languages. Comprising 113 chaotic tasks across 91 live, production-grade repositories, DeepSWE forces agents to generate 5.5 times more code and modify an average of 7 separate files per task compared to standard evaluations. On this challenging leaderboard, GPT-5.5 leads with a score of 70%, establishing a significant 16-percentage-point lead over contemporary alternatives I think older benchmarks where models reach ~90% accuracy can be considered saturated. Few percentage points don't give us any good signal.  https://research.ibm.com/publications/developing-ai-agents-for-it-automation-tasks-with-itbench  ITBench-AA, an evaluation framework focusing on live Kubernetes incident response and Site Reliability Engineering (SRE) operations. Comprising 59 live, containerized SRE incident snapshots, the results are remarkably sobering: every frontier model scored under 50% on successful incident resolution, with Claude Opus 4.7 leading at 47% and GPT-5.5 following closely at 46%.   Edge AI announcements: https://www.cloudflare.com/press/press-releases/2026/cloudflare-acquires-voidzero-to-build-the-future-of-the-ai-native-web/  The consolidation of the AI-native developer stack has reached the runtime virtualization layer. Cloudflare recently completed the acquisition of VoidZero, the development group responsible for Vite, Vitest, Rolldown, and Oxc, backing the transaction with a $1 million open-source ecosystem fund. This acquisition is highly strategic; as autonomous agents write an increasing proportion of production software, local development environments, compilation pipelines, and bundlers must be optimized for execution speeds that match agent speeds. Cloudflare's goal is to construct a localized, full-stack edge playground. In this sandbox, AI agents can generate, test, bundle (utilizing the highly parallelized, Rust-based Oxc and Rolldown engines), and deploy entire web applications end-to-end within milliseconds. This architecture completely bypasses traditional local machine container bottlenecks, enabling high-velocity agent loops to execute in a fully sandboxed, web-scale edge runtime.

William and Eyvonne discuss recent tech news, including the growing political and community opposition to AI data centers driven by fears over power and water usage. They also analyze the “AI Chip War” as hyperscalers such as AWS and Google invest in specialized silicon for training and inference.  Episode Links: Amid backlash, O'Leary Digital CEO... Read more »

William and Eyvonne discuss recent tech news, including the growing political and community opposition to AI data centers driven by fears over power and water usage. They also analyze the “AI Chip War” as hyperscalers such as AWS and Google invest in specialized silicon for training and inference.  Episode Links: Amid backlash, O'Leary Digital CEO... Read more »

In this episode of Alexa's Input (AI), I sat down with Rob Shaw from Red Hat to talk about how AI inference evolved from a simple model serving problem into a large-scale distributed systems problem.We explored the infrastructure shifts behind modern LLM serving, including how vLLM and PagedAttention changed the economics and efficiency of inference, why KV cache management became one of the most important bottlenecks in production AI systems, and how orchestration layers like llm-d are emerging to coordinate distributed inference.We also discuss:how LLM inference differs from traditional model serving runtimesKV cache, prefix caching, and cache-aware routingwhy throughput and latency became major infrastructure challengeslong-context agents and repeated inference callsdistributed inference on Kubernetesintelligent routing, flow control, and load balancingprefill/decode disaggregationenterprise AI deployment realitiesvLLM has become one of the most important open-source projects in AI infrastructure, and llm-d represents a newer shift toward treating inference as a coordinated distributed system rather than just a single runtime problem.If you want to better understand the systems layer beneath modern AI applications, this episode is a deep dive into where inference infrastructure is heading next.General Podcast LinksWatch: ⁠⁠⁠⁠⁠⁠https://www.youtube.com/@alexa_griffith⁠⁠⁠⁠⁠⁠Read: ⁠⁠⁠⁠⁠⁠⁠⁠https://alexasinput.substack.com/⁠⁠⁠⁠⁠⁠⁠⁠Listen:⁠⁠ ⁠⁠https://creators.spotify.com/pod/profile/alexagriffith/⁠⁠⁠⁠More: ⁠⁠⁠⁠⁠⁠https://linktr.ee/alexagriffith⁠⁠⁠⁠⁠⁠Learn more about the host atWebsite: ⁠⁠⁠⁠⁠⁠https://alexagriffith.com/⁠⁠⁠⁠⁠⁠LinkedIn: ⁠⁠⁠⁠⁠⁠https://www.linkedin.com/in/alexa-griffith/⁠⁠⁠⁠⁠⁠Find out more about the guest at:LinkedIn: https://www.linkedin.com/in/robert-shaw-1a01399a/ Red Hat Articles: https://developers.redhat.com/author/robert-shawGithub: https://github.com/robertgshaw2-redhat ResourcesvLLM Website: https://vllm.ai/vLLM GitHub Repository: https://github.com/vllm-project/vllmllm-d Website: https://llm-d.ai/llm-d GitHub Repository - https://github.com/llm-d/llm-d KeywordsAI inference, VLLM, LMD, distributed inference, GPU optimization, open source AI, Kubernetes, multi-cluster deployment, AI infrastructure, enterprise AI AI infrastructure, Kubernetes, model optimization, speculative decoding, mixture of experts, AI deployment, performance tuning, AI systems, neural network scaling Key TopicsEvolution of vLLM and llm-dDistributed inference and routingGPU utilization and performance optimizationOpen source AI infrastructureEnterprise deployment challenges and solutions Standardization in Kubernetes for NIC exposurePerformance optimizations: quantization and speculative decodingMixture of experts architecture and parallelism strategiesFlow control and request scheduling in AI systemsEmerging hardware for AI inference, Cerebras processorReinforcement learning and AI system supportModular architecture of vLLM and ecosystem projects

An airhacks.fm conversation with Bruno Borges (@brunoborges) about: discussion about the JAZ command launcher for Java, JVM tuning and default ergonomics for containers versus dedicated cloud environments, replacing the Java launcher with jaz in container images, supporting Java 8 to 25, maximizing resource utilization on kubernetes to reduce waste, running Java on Azure Functions, Azure App Service deploying a fat JAR without a container image, Azure Container Apps as a platform on AKS without YAML, Azure Kubernetes Service and AKS Automatic, Bicep as infrastructure as code, deploying a JAR to Kubernetes via OCI artifacts and a custom operator, Microsoft Foundry and the Microsoft Agent Framework, Semantic Kernel learnings, the Copilot SDK for Java communicating with headless CLIs, A2A and ACP protocols and MCP, agents as microservices with scoped tasks, guardrails, and sandboxing, per-agent model selection for cost and reasoning trade-offs, observability and traceability between agents with opentelemetry, grounding LLMs against MicroProfile, Jakarta EE, JAX-RS normative RFC 2119 specifications for hallucination-free Java code generation, the Boundary Control Entity pattern and business components as Java packages, package-info.java for semantic context, GitHub Copilot skills and custom instructions in Visual Studio Code, the AI Rails skills site, zero-dependency Java CLI scripting, reducing dependencies by reusing source code instead of JARs, the org.json reference implementation reduced to five classes, StackGres and OnGres running Quarkus and GraalVM to manage Postgres on Kubernetes, the Digg Into Java community Bruno Borges on twitter: @brunoborges

AI infrastructure is breaking the old data center model.In this episode of Liftoff with Keith, I sit down with Lukas Gentele, CEO & Co-Founder of vCluster Labs, to unpack what it really takes to operate GPU infrastructure at scale in 2026.As AI workloads explode and neoclouds race to meet demand, Lukas and his team are building the operational backbone for modern AI clouds — from managed Kubernetes and tenant isolation to automated node provisioning and GPU lifecycle management.We discuss:Why traditional data center assumptions are collapsing under AI pressureWhat's fundamentally changed since the VMware eraHow an early partnership with CoreWeave shaped vCluster's trajectoryAnd the one mistake AI cloud operators are making right now that could hurt them over the next 18 monthsIf you care about AI infrastructure, GPU economics, hyperscaler strategy, or building category-defining platforms — this conversation is essential.Sponsor Info: We are strategic business advisors with decades of leadership experience and a proven track record of driving businesses' growth. We specialize in creating custom-tailored strategies to introduce your company, drive growth, build leadership teams, and ensure companies implement appropriate compensation programs. Our mission is to utilize our expansive network to benefit your company https://www.compass-strategic-advisors.com/ Connect with Lukas Gentele: Website: https://www.vcluster.com/ LinkedIn: https://www.linkedin.com/in/gentele/ Subscribe for more founder insights and hit the bell for notifications! Follow us on our channels for exclusive startup content and behind-the-scenes insights from interviews like this one. Spotify: https://open.spotify.com/show/3cFpLXfYvcUsxvsT9MwyAD?si=f5a14e779777487d Apple Podcasts: https://podcasts.apple.com/ca/podcast/liftoff-with-keith-newman/id1560219589 Substack: https://keithnewman.substack.com/ Newman Media Studios: https://newmanmediastudios.com/ LinkedIn: https://www.linkedin.com/company/liftoffwithkeith For sponsorship inquiries, please contact: sponsorships@wherewithstudio.comFrom the Host: A special shout-out to our Great Host of the Ignite Studios: https://www.ignitegtm.com/ and Producers of AI Infra5 @Plug and Play World, HQ in Sunnyvale, CALiftoff is sponsored by a strategic consulting firm and the M&A specialists at Compass Strategic Advisors - https://www.compass-strategic-advisors.com/ and The GTM Firm - https://www.thegtmfirm.com/

Scott Hanselman talks with Omar Shorbaji from the Anyscale engineering team about how Anyscale on Azure scales Python AI workloads from a single notebook to thousands of CPUs and GPUs. Built on Ray, the most widely adopted AI compute engine, Anyscale gives you a unified runtime to build, train, and serve, running directly on Azure Kubernetes Service without the complexity of managing Kubernetes. See a live demo that fine-tunes a vision-language-action robotics policy, with the metrics you need to push GPU utilization higher. Chapters 00:00 - Introduction 00:52 - Ray and the Anyscale platform 03:11 - Start of demo: Workspaces 04:38 - Running a job and viewing utilization metrics 05:24 - Choosing the right scale 06:53 - Abstracting Kubernetes on AKS 08:53 - Wrap up and where to learn more Recommended resources Learn Docs Anyscale on Azure Connect Scott Hanselman | Twitter/X: @SHanselman Anyscale | Twitter/X: @anyscalecompute Azure Friday | Twitter/X: @AzureFriday Azure | Twitter/X: @Azure

Scott Hanselman talks with Omar Shorbaji from the Anyscale engineering team about how Anyscale on Azure scales Python AI workloads from a single notebook to thousands of CPUs and GPUs. Built on Ray, the most widely adopted AI compute engine, Anyscale gives you a unified runtime to build, train, and serve, running directly on Azure Kubernetes Service without the complexity of managing Kubernetes. See a live demo that fine-tunes a vision-language-action robotics policy, with the metrics you need to push GPU utilization higher. Chapters 00:00 - Introduction 00:52 - Ray and the Anyscale platform 03:11 - Start of demo: Workspaces 04:38 - Running a job and viewing utilization metrics 05:24 - Choosing the right scale 06:53 - Abstracting Kubernetes on AKS 08:53 - Wrap up and where to learn more Recommended resources Learn Docs Anyscale on Azure Connect Scott Hanselman | Twitter/X: @SHanselman Anyscale | Twitter/X: @anyscalecompute Azure Friday | Twitter/X: @AzureFriday Azure | Twitter/X: @Azure

In this episode of the Kubernetes Bytes podcast, Bhavin talks to Anthony Woods about all things Grafana Labs, Observability, Telemetry, and how AI impacts both of these ecosystems. The discussion starts off by talking about the early days of Grafana Labs, what is Adaptive Telemetry, and how AI plays a role both in building Observability capabilities in applications, and how it helps perform root cause analysis. Listen to learn more! Check out our website at https://kubernetesbytes.com/ Show Notes: GrafanaCON 2026: https://youtube.com/playlist?list=PLDGkOdUX1UjoSfz1IRj5c0xetw8tl8iin&si=JpT85m4t4bP8ZXgX Grafana Labs Blog: https://grafana.com/blog/ https://www.linkedin.com/in/anthonywoods1/

What happens when AI agents start acting less like chatbots and more like coworkers? In this episode, Dan and Chris sit down with Craig McLuckie, CEO of Stacklok to explore MCP, Kubernetes, ToolHive, enterprise AI, and the emerging infrastructure powering AI-native applications. From identity management to agent orchestration and system architecture, this conversation dives into how organizations may soon manage entire fleets of AI agents working behind the scenes.Featuring:Craig McLuckie – LinkedInChris Benson – Website, LinkedIn, Bluesky, GitHub, XDaniel Whitenack – Website, GitHub, XLinks:StacklokToolhiveSponsors:Prediction Guard: A self-hosted AI control plane for running agents in high impact environments. predictionguard.com/practicalaiUpcoming Events: Register for upcoming webinars here!Midwest AI Summit 2026

Ryota Sawada is software engineer at Numtide and the release lead of Kubernetes 1.36 code name Haru. He has over a decade of experience mainly in the finance industry including working on Cloud Native technologies, and outside of Cloud Native, he's been tinkering with Emacs and Nix.   Do you have something cool to share? Some questions? Let us know: - web: kubernetespodcast.com - mail: kubernetespodcast@google.com - twitter: @kubernetespod - bluesky: @kubernetespodcast.com   News of the week Etcd version 3.7.0 is out Merge Forward Community Truepositive Links from the interview Kubernetes 1.35 codename Haru Release theme and logo Logo designer User Namespaces Workload API Workload Aware Scheduling (WAS) DRA features graduating to Stable GKE 10 years and SIG Networking, With Antonio Ojea  

Today our Packet Pushers team assembles to discuss whether the grass is greener on the NetOps or DevOps side of the telemetry fence. William of The Cloud Gambit, Scott of Total Network Operations, and Ned and Kyler of Day Two DevOps discuss the difficulties and differences of getting telemetry and state from devices across different... Read more »

Today our Packet Pushers team assembles to discuss whether the grass is greener on the NetOps or DevOps side of the telemetry fence. William of The Cloud Gambit, Scott of Total Network Operations, and Ned and Kyler of Day Two DevOps discuss the difficulties and differences of getting telemetry and state from devices across different... Read more »